?rev1line? |
?rev2line? |
|
OpenRISC Test Scripts
|
|
=====================
|
|
|
|
Rationale
|
|
=========
|
|
|
|
The standard GNU tool chain regression is quite capable of running on multiple
|
|
remote targets.
|
|
|
|
However it does not include a facility to restart individual threads of
|
|
testing if a target fails. It is thus unsuitable for remote targets which are
|
|
in some respect unreliable.
|
|
|
|
The script run-all-tests.sh fixes this. It uses the standard GNU tool chain
|
|
DejaGnu test framework. However it does its own partitioning of threads. As
|
|
each thread completes, it then tests that the remote target is still alive,
|
|
and if not it marks that target as unavailable and repeats the test.
|
|
|
|
Under simulation it is also appropriate to use finer grain parallelism than is
|
|
available under the standard GNU toolchain test framework.
|
|
|
|
|
|
Prerequisites
|
|
=============
|
|
|
|
The tests require DejaGnu to be installed, and assume that the standard
|
|
OpenRISC build hierarchy (as used by bld-all.sh) is adopted.
|
|
|
|
|
|
DejaGnu Test Framework
|
|
======================
|
|
|
|
Board specifications are provided in the boards directory.
|
|
|
|
Testing of the or32-linux tool chain uses multiple remote targets with telnet
|
|
and FTP. The target may be physical hardware or software simulations
|
|
implementing Ethernet and running FTP and telnet daemons. The standard DejaGnu
|
|
scripts are incomplete for telnet and have errors for FTP, so the board
|
|
definition includes missing and fixed expect/TCL code to fix this.
|
|
|
|
The test script expects the DEJAGNU environment variable to point to the
|
|
site.exp file in the main or32 directory (the one containing the boards
|
|
directory). It uses the get-ip.sh script in that directory to obtain the IP
|
|
address of the next available remote target for testing.
|
|
|
|
The list of target IP addresses is placed in the file ${DEJAGNU}/ip-avail.txt.
|
|
|
|
|
|
Linux Configuration
|
|
===================
|
|
|
|
The standard Linux configuration for OpenRISC does not start an FTP
|
|
daemon. Modify arch/openrisc/support/initramfs/etc/inetd.conf as follows:
|
|
|
|
telnet stream tcp nowait root /usr/sbin/telnetd telnetd -i
|
|
ftp stream tcp nowait root ftpd ftpd -w /
|
|
|
|
The test scripts expect a /tmp directory to be present, so create this as
|
|
arch/openrisc/support/initramfs/tmp.
|
|
|
|
The test scripts FTP programs into the root directory. However telnet logs in
|
|
as root, with a home directory in /root. Fix this by editing
|
|
arch/openrisc/support/initramfs/etc/passwd to read
|
|
|
|
root::0:0::/:/bin/sh
|
|
|
|
Remember to rebuild Linux after these changes.
|
|
|
|
|
|
Board Configuration
|
|
===================
|
|
|
|
The or32-linux-sim.exp script in the boards directory sets various
|
|
parameters, which may need adjusting. In particular it specifies the absolute
|
|
location of the C and C++ compilers and gives timeouts for command execution
|
|
(1200 seconds) and FTP upload/download (both 120 seconds).
|
|
|
|
|
|
run-all-tests.sh
|
|
================
|
|
|
|
Check the function "set_globals" to ensure that the various files and
|
|
directories are correct.
|
|
|
|
The main function is "run_tool_tests". The first argument is the tool test
|
|
directory, the second the tool source directory, the third the tool name and
|
|
the remaining arguments a list of test thread specifications.
|
|
|
|
Each test thread specification takes the form:
|
|
|
|
#
|
|
|
|
A runtest target is typically an expect script, qualified by the names of
|
|
specific source files and/or directories to be tested, for example
|
|
|
|
executed.exp=execute/2001*%execute/2002*
|
|
|
|
The default script runs all the tests for gcc, g++ and libstdc++-v3 (using
|
|
three calls to "run-tool_tests". Edit the file to adjust this.
|
|
|
|
Having set up ip-avail.txt, the script is then run as:
|
|
|
|
./run-all-tests.sh
|
|
|
|
|
|
Diagnostics
|
|
===========
|
|
|
|
To increase the diagnostic output, find the "runtest" command in
|
|
run-all-tests.sh and add "-v" to its options (for even more output add "-v -v"
|
|
or even "-v -v -v").
|
|
|
|
Note this is useful for debugging the test scripts rather than actual tests
|
|
themselves.
|
|
|
|
|
|
Supplementary Scripts
|
|
=====================
|
|
|
|
run-all-tests.sh leaves each tests results in its own log and summary
|
|
files. The script check-results.sh collates all the results. For example
|
|
|
|
./check-results.sh bd-linux/or32-linux/libstdc++-v3/testsuite/libstdc++/*.log
|
|
|
|
It is sometimes useful to see how long each test thread took. This may lead to
|
|
adjustment in partitioning of the tests. This can be achieved with the
|
|
check-times.sh script. For example:
|
|
|
|
./check-times.sh bd-linux/or32-linux/libstdc++-v3/testsuite/libstdc++/*.log
|
|
|
|
|
|
Feedback
|
|
========
|
|
|
|
Suggestions for improvements always welcome.
|
|
|
|
|
|
Jeremy Bennett
|
|
3 June 2011
|
