| 1 |
149 |
jeremybenn |
This file contains a list of notes for those writing testcases and those
|
| 2 |
|
|
writing expect scripts. At present, they're in random order.
|
| 3 |
|
|
|
| 4 |
|
|
Verbosity Levels
|
| 5 |
|
|
|
| 6 |
|
|
- each level adds verbosity so level 2 prints all level 1 and level 2 stuff.
|
| 7 |
|
|
|
| 8 |
|
|
1) Print a one-liner indicating the testcase being run (and maybe special
|
| 9 |
|
|
compiler options).
|
| 10 |
|
|
|
| 11 |
|
|
2) Print compiler and program invocation including arguments and their output.
|
| 12 |
|
|
Proc's gcc_load and gcc_start handle the latter two.
|
| 13 |
|
|
|
| 14 |
|
|
3) Print detailed testcase analysis like "Looking for pattern ...", etc.
|
| 15 |
|
|
|
| 16 |
|
|
4) Maximum verbosity. Print anything else of interest.
|
| 17 |
|
|
|
| 18 |
|
|
send_log conventions
|
| 19 |
|
|
|
| 20 |
|
|
Various messages are stored in gcc.log by the testing framework and we
|
| 21 |
|
|
try to augment them with some of our own. The format of the framework
|
| 22 |
|
|
messages are:
|
| 23 |
|
|
|
| 24 |
|
|
PASS: blah blah ...
|
| 25 |
|
|
FAIL: blah blah ...
|
| 26 |
|
|
|
| 27 |
|
|
so we use
|
| 28 |
|
|
|
| 29 |
|
|
XXXX: blah blah ...
|
| 30 |
|
|
|
| 31 |
|
|
Current messages are:
|
| 32 |
|
|
|
| 33 |
|
|
EXEC: program being executed (so compiler path and args are recorded)
|
| 34 |
|
|
STAT: intermediate pass/fail statistics
|
| 35 |
|
|
|
| 36 |
|
|
Test scripts must ignore the compiler messages "path prefix never used"
|
| 37 |
|
|
and "linker input file unused". Don't let their appearance cause a testcase
|
| 38 |
|
|
to fail. See lib/dg.exp for the exact regsub to use.
|
| 39 |
|
|
|
| 40 |
|
|
If you're unclear about which directory a testcase should be installed in,
|
| 41 |
|
|
ask gcc-local.
|
| 42 |
|
|
|
| 43 |
|
|
Have the text of a fail message be the same as that for pass.
|
| 44 |
|
|
IE: have
|
| 45 |
|
|
|
| 46 |
|
|
if ...success...
|
| 47 |
|
|
pass "pr 1234"
|
| 48 |
|
|
else
|
| 49 |
|
|
fail "pr 1234"
|
| 50 |
|
|
|
| 51 |
|
|
not
|
| 52 |
|
|
|
| 53 |
|
|
if ...success...
|
| 54 |
|
|
pass "pr 1234 passed"
|
| 55 |
|
|
else
|
| 56 |
|
|
fail "pr 1234 failed"
|
| 57 |
|
|
|
| 58 |
|
|
|
| 59 |
|
|
This lets test-tool (which drives the nightly tests) do a better job
|
| 60 |
|
|
at tracking which tests have digressed or been fixed.
|
| 61 |
|
|
|
| 62 |
|
|
DO NOT PUT NON-PORTABLE TESTCASES IN gcc.c-torture.
|
| 63 |
|
|
|
| 64 |
|
|
ANY TARGET SPECIFIC TESTCASE MUST HAVE APPROPRIATE CODE TO PREVENT IT FROM
|
| 65 |
|
|
CAUSING A `FAILURE' ON UNSUPPORTED PLATFORMS.
|
| 66 |
|
|
|
| 67 |
|
|
The "torture" tests are meant to be generic tests that can run on any
|
| 68 |
|
|
target. So you have to be careful about endianness, assumptions about
|
| 69 |
|
|
sizes of datatypes, etc etc.
|
| 70 |
|
|
|
| 71 |
|
|
For tests that merely need to compile, put them in the "compile" directory.
|
| 72 |
|
|
|
| 73 |
|
|
For tests which should give an error, put them in the "noncompile" directory
|
| 74 |
|
|
and update noncompile.exp appropriately (see examples in noncompile.exp).
|
| 75 |
|
|
|
| 76 |
|
|
For IEEE FP specific tests, put them in execute/ieee.
|
| 77 |
|
|
|
| 78 |
|
|
For execution tests, put them in execute.
|
| 79 |
|
|
|
| 80 |
|
|
If a test does not fit into the torture framework, use the dg framework.
|
| 81 |
|
|
|
| 82 |
|
|
|
| 83 |
|
|
|
