This file contains a list of notes for those writing testcases and those
|
This file contains a list of notes for those writing testcases and those
|
writing expect scripts. At present, they're in random order.
|
writing expect scripts. At present, they're in random order.
|
|
|
Verbosity Levels
|
Verbosity Levels
|
|
|
- each level adds verbosity so level 2 prints all level 1 and level 2 stuff.
|
- each level adds verbosity so level 2 prints all level 1 and level 2 stuff.
|
|
|
1) Print a one-liner indicating the testcase being run (and maybe special
|
1) Print a one-liner indicating the testcase being run (and maybe special
|
compiler options).
|
compiler options).
|
|
|
2) Print compiler and program invocation including arguments and their output.
|
2) Print compiler and program invocation including arguments and their output.
|
Proc's gcc_load and gcc_start handle the latter two.
|
Proc's gcc_load and gcc_start handle the latter two.
|
|
|
3) Print detailed testcase analysis like "Looking for pattern ...", etc.
|
3) Print detailed testcase analysis like "Looking for pattern ...", etc.
|
|
|
4) Maximum verbosity. Print anything else of interest.
|
4) Maximum verbosity. Print anything else of interest.
|
|
|
send_log conventions
|
send_log conventions
|
|
|
Various messages are stored in gcc.log by the testing framework and we
|
Various messages are stored in gcc.log by the testing framework and we
|
try to augment them with some of our own. The format of the framework
|
try to augment them with some of our own. The format of the framework
|
messages are:
|
messages are:
|
|
|
PASS: blah blah ...
|
PASS: blah blah ...
|
FAIL: blah blah ...
|
FAIL: blah blah ...
|
|
|
so we use
|
so we use
|
|
|
XXXX: blah blah ...
|
XXXX: blah blah ...
|
|
|
Current messages are:
|
Current messages are:
|
|
|
EXEC: program being executed (so compiler path and args are recorded)
|
EXEC: program being executed (so compiler path and args are recorded)
|
STAT: intermediate pass/fail statistics
|
STAT: intermediate pass/fail statistics
|
|
|
Test scripts must ignore the compiler messages "path prefix never used"
|
Test scripts must ignore the compiler messages "path prefix never used"
|
and "linker input file unused". Don't let their appearance cause a testcase
|
and "linker input file unused". Don't let their appearance cause a testcase
|
to fail. See lib/dg.exp for the exact regsub to use.
|
to fail. See lib/dg.exp for the exact regsub to use.
|
|
|
If you're unclear about which directory a testcase should be installed in,
|
If you're unclear about which directory a testcase should be installed in,
|
ask gcc-local.
|
ask gcc-local.
|
|
|
Have the text of a fail message be the same as that for pass.
|
Have the text of a fail message be the same as that for pass.
|
IE: have
|
IE: have
|
|
|
if ...success...
|
if ...success...
|
pass "pr 1234"
|
pass "pr 1234"
|
else
|
else
|
fail "pr 1234"
|
fail "pr 1234"
|
|
|
not
|
not
|
|
|
if ...success...
|
if ...success...
|
pass "pr 1234 passed"
|
pass "pr 1234 passed"
|
else
|
else
|
fail "pr 1234 failed"
|
fail "pr 1234 failed"
|
|
|
|
|
This lets test-tool (which drives the nightly tests) do a better job
|
This lets test-tool (which drives the nightly tests) do a better job
|
at tracking which tests have digressed or been fixed.
|
at tracking which tests have digressed or been fixed.
|
|
|
DO NOT PUT NON-PORTABLE TESTCASES IN gcc.c-torture.
|
DO NOT PUT NON-PORTABLE TESTCASES IN gcc.c-torture.
|
|
|
ANY TARGET SPECIFIC TESTCASE MUST HAVE APPROPRIATE CODE TO PREVENT IT FROM
|
ANY TARGET SPECIFIC TESTCASE MUST HAVE APPROPRIATE CODE TO PREVENT IT FROM
|
CAUSING A `FAILURE' ON UNSUPPORTED PLATFORMS.
|
CAUSING A `FAILURE' ON UNSUPPORTED PLATFORMS.
|
|
|
The "torture" tests are meant to be generic tests that can run on any
|
The "torture" tests are meant to be generic tests that can run on any
|
target. So you have to be careful about endianness, assumptions about
|
target. So you have to be careful about endianness, assumptions about
|
sizes of datatypes, etc etc.
|
sizes of datatypes, etc etc.
|
|
|
For tests that merely need to compile, put them in the "compile" directory.
|
For tests that merely need to compile, put them in the "compile" directory.
|
|
|
For tests which should give an error, put them in the "noncompile" directory
|
For tests which should give an error, put them in the "noncompile" directory
|
and update noncompile.exp appropriately (see examples in noncompile.exp).
|
and update noncompile.exp appropriately (see examples in noncompile.exp).
|
|
|
For IEEE FP specific tests, put them in execute/ieee.
|
For IEEE FP specific tests, put them in execute/ieee.
|
|
|
For execution tests, put them in execute.
|
For execution tests, put them in execute.
|
|
|
If a test does not fit into the torture framework, use the dg framework.
|
If a test does not fit into the torture framework, use the dg framework.
|
|
|
|
|
|
|
|
|
