OpenCores
URL https://opencores.org/ocsvn/openrisc/openrisc/trunk

Subversion Repositories openrisc

[/] [openrisc/] [trunk/] [gnu-stable/] [doc/] [testing.txt] - Diff between revs 816 and 824

Go to most recent revision | Show entire file | Details | Blame | View Log

Rev 816 Rev 824
?rev1line?
?rev2line?
 
                            OpenRISC Test Scripts
 
                            =====================
 
 
 
Rationale
 
=========
 
 
 
The standard GNU tool chain regression is quite capable of running on multiple
 
remote targets.
 
 
 
However it does not include a facility to restart individual threads of
 
testing if a target fails. It is thus unsuitable for remote targets which are
 
in some respect unreliable.
 
 
 
The script run-all-tests.sh fixes this. It uses the standard GNU tool chain
 
DejaGnu test framework. However it does its own partitioning of threads. As
 
each thread completes, it then tests that the remote target is still alive,
 
and if not it marks that target as unavailable and repeats the test.
 
 
 
Under simulation it is also appropriate to use finer grain parallelism than is
 
available under the standard GNU toolchain test framework.
 
 
 
 
 
Prerequisites
 
=============
 
 
 
The tests require DejaGnu to be installed, and assume that the standard
 
OpenRISC build hierarchy (as used by bld-all.sh) is adopted.
 
 
 
 
 
DejaGnu Test Framework
 
======================
 
 
 
Board specifications are provided in the boards directory.
 
 
 
Testing of the or32-linux tool chain uses multiple remote targets with telnet
 
and FTP. The target may be physical hardware or software simulations
 
implementing Ethernet and running FTP and telnet daemons. The standard DejaGnu
 
scripts are incomplete for telnet and have errors for FTP, so the board
 
definition includes missing and fixed expect/TCL code to fix this.
 
 
 
The test script expects the DEJAGNU environment variable to point to the
 
site.exp file in the main or32 directory (the one containing the boards
 
directory). It uses the get-ip.sh script in that directory to obtain the IP
 
address of the next available remote target for testing.
 
 
 
The list of target IP addresses is placed in the file ${DEJAGNU}/ip-avail.txt.
 
 
 
 
 
Linux Configuration
 
===================
 
 
 
The standard Linux configuration for OpenRISC does not start an FTP
 
daemon. Modify arch/openrisc/support/initramfs/etc/inetd.conf as follows:
 
 
 
  telnet stream tcp nowait root /usr/sbin/telnetd telnetd -i
 
  ftp stream tcp nowait root ftpd ftpd -w /
 
 
 
The test scripts expect a /tmp directory to be present, so create this as
 
arch/openrisc/support/initramfs/tmp.
 
 
 
The test scripts FTP programs into the root directory. However telnet logs in
 
as root, with a home directory in /root. Fix this by editing
 
arch/openrisc/support/initramfs/etc/passwd to read
 
 
 
  root::0:0::/:/bin/sh
 
 
 
Remember to rebuild Linux after these changes.
 
 
 
 
 
Board Configuration
 
===================
 
 
 
The or32-linux-sim.exp script in the boards directory sets various
 
parameters, which may need adjusting. In particular it specifies the absolute
 
location of the C and C++ compilers and gives timeouts for command execution
 
(1200 seconds) and FTP upload/download (both 120 seconds).
 
 
 
 
 
run-all-tests.sh
 
================
 
 
 
Check the function "set_globals" to ensure that the various files and
 
directories are correct.
 
 
 
The main function is "run_tool_tests". The first argument is the tool test
 
directory, the second the tool source directory, the third the tool name and
 
the remaining arguments a list of test thread specifications.
 
 
 
Each test thread specification takes the form:
 
 
 
  #
 
 
 
A runtest target is typically an expect script, qualified by the names of
 
specific source files and/or directories to be tested, for example
 
 
 
  executed.exp=execute/2001*%execute/2002*
 
 
 
The default script runs all the tests for gcc, g++ and libstdc++-v3 (using
 
three calls to "run-tool_tests". Edit the file to adjust this.
 
 
 
Having set up ip-avail.txt, the script is then run as:
 
 
 
  ./run-all-tests.sh
 
 
 
 
 
Diagnostics
 
===========
 
 
 
To increase the diagnostic output, find the "runtest" command in
 
run-all-tests.sh and add "-v" to its options (for even more output add "-v -v"
 
or even "-v -v -v").
 
 
 
Note this is useful for debugging the test scripts rather than actual tests
 
themselves.
 
 
 
 
 
Supplementary Scripts
 
=====================
 
 
 
run-all-tests.sh leaves each tests results in its own log and summary
 
files. The script check-results.sh collates all the results. For example
 
 
 
  ./check-results.sh bd-linux/or32-linux/libstdc++-v3/testsuite/libstdc++/*.log
 
 
 
It is sometimes useful to see how long each test thread took. This may lead to
 
adjustment in partitioning of the tests. This can be achieved with the
 
check-times.sh script. For example:
 
 
 
  ./check-times.sh bd-linux/or32-linux/libstdc++-v3/testsuite/libstdc++/*.log
 
 
 
 
 
Feedback
 
========
 
 
 
Suggestions for improvements always welcome.
 
 
 
 
 
Jeremy Bennett 
 
3 June 2011

powered by: WebSVN 2.1.0

© copyright 1999-2024 OpenCores.org, equivalent to Oliscience, all rights reserved. OpenCores®, registered trademark.