OpenCores
URL https://opencores.org/ocsvn/openrisc/openrisc/trunk

Subversion Repositories openrisc

[/] [openrisc/] [trunk/] [gnu-old/] [gcc-4.2.2/] [gcc/] [testsuite/] [README.QMTEST] - Blame information for rev 816

Details | Compare with Previous | View Log

Line No. Rev Author Line
1 149 jeremybenn
Testing with QMTest
2
===================
3
 
4
You can use QMTest to test G++.  (In the future, it may be possible to
5
test other parts of GCC with QMTest as well, but it is not possible
6
yet.)
7
 
8
The use of QMTest to run the G++ tests has not been approved as an
9
officially supported testing procedure.  Therefore, you must run the
10
tests using DejaGNU (with "make check-g++") before committing changes
11
that affect G++.
12
 
13
QMTest emulates DejaGNU behavior very closely when running the tests.
14
 
15
QMTest has two output modes: a DejaGNU emulation mode and a native
16
QMTest mode.
17
 
18
In the DejaGNU mode, you should receive output that is almost exactly
19
the same as the DejaGNU output; in particular, you should see the same
20
number of passes, failures, etc.  When using the DejaGNU-style output,
21
QMTest uses the "xfail" indications in the test cases to determine
22
which tests are expected to pass and which are expected to fail, and
23
presents that information in the same way as DejaGNU.
24
 
25
In the QMTest mode, the number of passes and failures will be
26
different from that obtained when using DejaGNU.  The reason is that a
27
single source file may contain multiple DejaGNU tests.  In DejaGNU,
28
each line where a diagnostic is expected is considered a separate
29
test.  Testing for successful compilation and testing for successful
30
execution of the generated program are considered separate tests.  So,
31
a single source file "test.C" could contain, say, seven tests; some of
32
which might pass and some of which might fail.
33
 
34
In the QMTest mode, each source file is considered a single test.  If
35
any of the seven sub-tests fail, the entire test is considered to
36
fail.  However, QMTest does present information about *why* the test
37
failed, so the same information is effectively available.
38
 
39
In the QMTest mode, whether or not a test is expected to fail is
40
determined not by an indication in the test, but rather by comparing
41
the new results to the results of a previous run.  Testing for whether
42
a change caused a regression is very simple: run the tests before
43
making the change, run them again after making the change, and let
44
QMTest compare the results.
45
 
46
The mode chosen only affects the output from QMTest, not how it runs
47
the tests or how it stores the data.  Therefore, if you choose to run
48
in the QMTest mode and later want to get the DejaGNU style output, or
49
vice versa, you can do that as described below.
50
 
51
Setting Up
52
==========
53
 
54
You must download and install the following software:
55
 
56
- Python 2.2 (or greater)
57
 
58
  See http://www.python.org.
59
 
60
  You may already have Python on your system; in particular, many
61
  GNU/Linux systems ship with Python installed.
62
 
63
  Installation instructions are available on the web-site.
64
 
65
- A current version of QMTest.  No released version provides all of
66
  the functionality required, so you must obtain QMTest from CVS.
67
 
68
  To do that, follow the instructions at:
69
 
70
    http://www.codesourcery.com/qmtest
71
 
72
  Installation instructions are available in the file called README
73
  after you check out QMTest.
74
 
75
- The "qmtc" and "qmtest_gcc" QMTest support packages.  These are
76
  available from the same CVS repository as QMTest.  For example, to
77
  check out "qmtc", do:
78
 
79
    cvs -d :pserver:anoncvs@cvs.codesourcery.com:/home/qm/Repository \
80
      co qmtc
81
 
82
  You do not have to install these packages; you need only check them
83
  out.
84
 
85
Running the Tests
86
=================
87
 
88
First, you must set QMTEST_CLASS_PATH so that it can find the qmtc and
89
qmtest_gcc support packages:
90
 
91
  export QMTEST_CLASS_PATH=/path/to/qmtc:/path/to/qmtest_gcc
92
 
93
The, run "make qmtest-g++" in the gcc directory of your build tree.
94
 
95
Here are some more advanced usage instructions:
96
 
97
1. To run a particular set of tests (rather than all of the tests),
98
   use the make variable "QMTEST_GPP_TESTS".  For example,
99
 
100
     make QMTEST_GPP_TESTS="g++.dg" qmtest-g++
101
 
102
   will run only the tests in the g++.dg subdirectory, and:
103
 
104
     make QMTEST_GPP_TESTS="g++.dg/special/conpr-1.C \
105
                            g++.old-deja/g++.other/access2.C"
106
          qmtest-g++
107
 
108
   will run only the two tests indicated.
109
 
110
2. To run qmtest with particular flags, use the make variables
111
   "QMTESTFLAGS" and "QMTESTRUNFLAGS".  For example:
112
 
113
      make QMTESTFLAGS="-v" QMTESTRUNFLAGS="-f full" qmtest-g++
114
 
115
   will run qmtest like this:
116
 
117
      qmtest -v run -f full ...
118
 
119
   (The "-f full" mode will provide detailed information about each
120
   test as it runs.)
121
 
122
3. To run the compiler with particular flags, use QMTESTRUNFLAGS to
123
   set the QMTest context variable "CompilerTable.cplusplus_options",
124
   like this:
125
 
126
      make \
127
        QMTESTRUNFLAGS='-c CompilerTable.cplusplus_options="-funroll-loops"' \
128
        qmtest-g++
129
 
130
   The compiler will then use the "-funroll-loops" switch when
131
   compiling.
132
 
133
4. If qmtest is not in your path, you can indicate the full path to
134
   QMTest by using the make variable "QMTEST_PATH", like this:
135
 
136
      make QMTEST_PATH=/path/to/qmtest qmtest-g++
137
 
138
5. To start the QMTest GUI, use:
139
 
140
      make qmtest-gui
141
 
142
   (Note that this will run the program called "mozilla" in your path.
143
   If you want to use another browser, you must configure qmtest as
144
   described in its manual.)
145
 
146
   Bear in mind that the QMTest GUI is insecure; malicious users with
147
   access to your machine may be able to run commands as if they were
148
   you.  The QMTest GUI only binds to the loopback IP addresss, which
149
   provides a measure of security, but not enough for use in untrusted
150
   environments.
151
 
152
6. If you have a multiprocessor, you can run the tests in parallel by
153
   passing the "-j" option to qmtest:
154
 
155
      make QMTESTRUNFLAGS="-j 4" qmtest-g++
156
 
157
   will run tests in four threads.  (It is also possible to run tests
158
   across multiple machines; for more information see the QMTest
159
   manual.)
160
 
161
7. If a test (say "g++.dg/abi/bitfield1.C") fails, and you want to get
162
   more detailed information, you can do:
163
 
164
      cd qmtestsuite
165
      qmtest summarize g++.qmr g++.dg/abi/bitfield1.C
166
 
167
   to get more information about the commands that were run and the
168
   output produced.

powered by: WebSVN 2.1.0

© copyright 1999-2024 OpenCores.org, equivalent to Oliscience, all rights reserved. OpenCores®, registered trademark.