OpenCores
URL https://opencores.org/ocsvn/openrisc/openrisc/trunk

Subversion Repositories openrisc

[/] [openrisc/] [trunk/] [gnu-stable/] [doc/] [testing.txt] - Blame information for rev 862

Go to most recent revision | Details | Compare with Previous | View Log

Line No. Rev Author Line
1 548 jeremybenn
                            OpenRISC Test Scripts
2
                            =====================
3
 
4
Rationale
5
=========
6
 
7
The standard GNU tool chain regression is quite capable of running on multiple
8
remote targets.
9
 
10
However it does not include a facility to restart individual threads of
11
testing if a target fails. It is thus unsuitable for remote targets which are
12
in some respect unreliable.
13
 
14
The script run-all-tests.sh fixes this. It uses the standard GNU tool chain
15
DejaGnu test framework. However it does its own partitioning of threads. As
16
each thread completes, it then tests that the remote target is still alive,
17
and if not it marks that target as unavailable and repeats the test.
18
 
19
Under simulation it is also appropriate to use finer grain parallelism than is
20
available under the standard GNU toolchain test framework.
21
 
22
 
23
Prerequisites
24
=============
25
 
26
The tests require DejaGnu to be installed, and assume that the standard
27
OpenRISC build hierarchy (as used by bld-all.sh) is adopted.
28
 
29
 
30
DejaGnu Test Framework
31
======================
32
 
33
Board specifications are provided in the boards directory.
34
 
35
Testing of the or32-linux tool chain uses multiple remote targets with telnet
36
and FTP. The target may be physical hardware or software simulations
37
implementing Ethernet and running FTP and telnet daemons. The standard DejaGnu
38
scripts are incomplete for telnet and have errors for FTP, so the board
39
definition includes missing and fixed expect/TCL code to fix this.
40
 
41
The test script expects the DEJAGNU environment variable to point to the
42 549 jeremybenn
site.exp file in the main or32 directory (the one containing the boards
43
directory). It uses the get-ip.sh script in that directory to obtain the IP
44
address of the next available remote target for testing.
45 548 jeremybenn
 
46
The list of target IP addresses is placed in the file ${DEJAGNU}/ip-avail.txt.
47
 
48
 
49
Linux Configuration
50
===================
51
 
52
The standard Linux configuration for OpenRISC does not start an FTP
53
daemon. Modify arch/openrisc/support/initramfs/etc/inetd.conf as follows:
54
 
55
  telnet stream tcp nowait root /usr/sbin/telnetd telnetd -i
56
  ftp stream tcp nowait root ftpd ftpd -w /
57
 
58
The test scripts expect a /tmp directory to be present, so create this as
59
arch/openrisc/support/initramfs/tmp.
60
 
61
The test scripts FTP programs into the root directory. However telnet logs in
62
as root, with a home directory in /root. Fix this by editing
63
arch/openrisc/support/initramfs/etc/passwd to read
64
 
65
  root::0:0::/:/bin/sh
66
 
67
Remember to rebuild Linux after these changes.
68
 
69
 
70
Board Configuration
71
===================
72
 
73
The or32-linux-sim.exp script in the boards directory sets various
74
parameters, which may need adjusting. In particular it specifies the absolute
75
location of the C and C++ compilers and gives timeouts for command execution
76
(1200 seconds) and FTP upload/download (both 120 seconds).
77
 
78 550 jeremybenn
 
79 548 jeremybenn
run-all-tests.sh
80
================
81
 
82
Check the function "set_globals" to ensure that the various files and
83
directories are correct.
84
 
85
The main function is "run_tool_tests". The first argument is the tool test
86
directory, the second the tool source directory, the third the tool name and
87
the remaining arguments a list of test thread specifications.
88
 
89
Each test thread specification takes the form:
90
 
91
  #
92
 
93
A runtest target is typically an expect script, qualified by the names of
94
specific source files and/or directories to be tested, for example
95
 
96
  executed.exp=execute/2001*%execute/2002*
97
 
98
The default script runs all the tests for gcc, g++ and libstdc++-v3 (using
99
three calls to "run-tool_tests". Edit the file to adjust this.
100
 
101
Having set up ip-avail.txt, the script is then run as:
102
 
103
  ./run-all-tests.sh
104
 
105
 
106 550 jeremybenn
Diagnostics
107
===========
108
 
109
To increase the diagnostic output, find the "runtest" command in
110
run-all-tests.sh and add "-v" to its options (for even more output add "-v -v"
111
or even "-v -v -v").
112
 
113
Note this is useful for debugging the test scripts rather than actual tests
114
themselves.
115
 
116
 
117 548 jeremybenn
Supplementary Scripts
118
=====================
119
 
120
run-all-tests.sh leaves each tests results in its own log and summary
121
files. The script check-results.sh collates all the results. For example
122
 
123
  ./check-results.sh bd-linux/or32-linux/libstdc++-v3/testsuite/libstdc++/*.log
124
 
125
It is sometimes useful to see how long each test thread took. This may lead to
126
adjustment in partitioning of the tests. This can be achieved with the
127
check-times.sh script. For example:
128
 
129
  ./check-times.sh bd-linux/or32-linux/libstdc++-v3/testsuite/libstdc++/*.log
130
 
131
 
132
Feedback
133
========
134
 
135
Suggestions for improvements always welcome.
136
 
137
 
138
Jeremy Bennett 
139
3 June 2011

powered by: WebSVN 2.1.0

© copyright 1999-2024 OpenCores.org, equivalent to Oliscience, all rights reserved. OpenCores®, registered trademark.