| 1 |
268 |
jeremybenn |
<html lang="en">
|
| 2 |
|
|
<head>
|
| 3 |
|
|
<title>Installing GCC: Testing</title>
|
| 4 |
|
|
<meta http-equiv="Content-Type" content="text/html">
|
| 5 |
|
|
<meta name="description" content="Installing GCC: Testing">
|
| 6 |
|
|
<meta name="generator" content="makeinfo 4.12">
|
| 7 |
|
|
<link title="Top" rel="top" href="#Top">
|
| 8 |
|
|
<link href="http://www.gnu.org/software/texinfo/" rel="generator-home" title="Texinfo Homepage">
|
| 9 |
|
|
<!--
|
| 10 |
|
|
Copyright (C) 1988, 1989, 1992, 1993, 1994, 1995, 1996, 1997,
|
| 11 |
|
|
1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007,
|
| 12 |
|
|
2008 Free Software Foundation, Inc.
|
| 13 |
|
|
|
| 14 |
|
|
Permission is granted to copy, distribute and/or modify this document
|
| 15 |
|
|
under the terms of the GNU Free Documentation License, Version 1.2 or
|
| 16 |
|
|
any later version published by the Free Software Foundation; with no
|
| 17 |
|
|
Invariant Sections, the Front-Cover texts being (a) (see below), and
|
| 18 |
|
|
with the Back-Cover Texts being (b) (see below). A copy of the
|
| 19 |
|
|
license is included in the section entitled "GNU Free Documentation License".
|
| 20 |
|
|
|
| 21 |
|
|
(a) The FSF's Front-Cover Text is:
|
| 22 |
|
|
|
| 23 |
|
|
A GNU Manual
|
| 24 |
|
|
|
| 25 |
|
|
(b) The FSF's Back-Cover Text is:
|
| 26 |
|
|
|
| 27 |
|
|
You have freedom to copy and modify this GNU Manual, like GNU
|
| 28 |
|
|
software. Copies published by the Free Software Foundation raise
|
| 29 |
|
|
funds for GNU development.-->
|
| 30 |
|
|
<meta http-equiv="Content-Style-Type" content="text/css">
|
| 31 |
|
|
<style type="text/css"><!--
|
| 32 |
|
|
pre.display { font-family:inherit }
|
| 33 |
|
|
pre.format { font-family:inherit }
|
| 34 |
|
|
pre.smalldisplay { font-family:inherit; font-size:smaller }
|
| 35 |
|
|
pre.smallformat { font-family:inherit; font-size:smaller }
|
| 36 |
|
|
pre.smallexample { font-size:smaller }
|
| 37 |
|
|
pre.smalllisp { font-size:smaller }
|
| 38 |
|
|
span.sc { font-variant:small-caps }
|
| 39 |
|
|
span.roman { font-family:serif; font-weight:normal; }
|
| 40 |
|
|
span.sansserif { font-family:sans-serif; font-weight:normal; }
|
| 41 |
|
|
--></style>
|
| 42 |
|
|
</head>
|
| 43 |
|
|
<body>
|
| 44 |
|
|
<h1 class="settitle">Installing GCC: Testing</h1>
|
| 45 |
|
|
<a name="index-Testing-1"></a><a name="index-Installing-GCC_003a-Testing-2"></a><a name="index-Testsuite-3"></a>
|
| 46 |
|
|
Before you install GCC, we encourage you to run the testsuites and to
|
| 47 |
|
|
compare your results with results from a similar configuration that have
|
| 48 |
|
|
been submitted to the
|
| 49 |
|
|
<a href="http://gcc.gnu.org/ml/gcc-testresults/">gcc-testresults mailing list</a>.
|
| 50 |
|
|
Some of these archived results are linked from the build status lists
|
| 51 |
|
|
at <a href="http://gcc.gnu.org/buildstat.html">http://gcc.gnu.org/buildstat.html</a>, although not everyone who
|
| 52 |
|
|
reports a successful build runs the testsuites and submits the results.
|
| 53 |
|
|
This step is optional and may require you to download additional software,
|
| 54 |
|
|
but it can give you confidence in your new GCC installation or point out
|
| 55 |
|
|
problems before you install and start using your new GCC.
|
| 56 |
|
|
|
| 57 |
|
|
<p>First, you must have <a href="download.html">downloaded the testsuites</a>.
|
| 58 |
|
|
These are part of the full distribution, but if you downloaded the
|
| 59 |
|
|
“core” compiler plus any front ends, you must download the testsuites
|
| 60 |
|
|
separately.
|
| 61 |
|
|
|
| 62 |
|
|
<p>Second, you must have the testing tools installed. This includes
|
| 63 |
|
|
<a href="http://www.gnu.org/software/dejagnu/">DejaGnu</a>, Tcl, and Expect;
|
| 64 |
|
|
the DejaGnu site has links to these.
|
| 65 |
|
|
|
| 66 |
|
|
<p>If the directories where <samp><span class="command">runtest</span></samp> and <samp><span class="command">expect</span></samp> were
|
| 67 |
|
|
installed are not in the <samp><span class="env">PATH</span></samp>, you may need to set the following
|
| 68 |
|
|
environment variables appropriately, as in the following example (which
|
| 69 |
|
|
assumes that DejaGnu has been installed under <samp><span class="file">/usr/local</span></samp>):
|
| 70 |
|
|
|
| 71 |
|
|
<pre class="smallexample"> TCL_LIBRARY = /usr/local/share/tcl8.0
|
| 72 |
|
|
DEJAGNULIBS = /usr/local/share/dejagnu
|
| 73 |
|
|
</pre>
|
| 74 |
|
|
<p>(On systems such as Cygwin, these paths are required to be actual
|
| 75 |
|
|
paths, not mounts or links; presumably this is due to some lack of
|
| 76 |
|
|
portability in the DejaGnu code.)
|
| 77 |
|
|
|
| 78 |
|
|
<p>Finally, you can run the testsuite (which may take a long time):
|
| 79 |
|
|
<pre class="smallexample"> cd <var>objdir</var>; make -k check
|
| 80 |
|
|
</pre>
|
| 81 |
|
|
<p>This will test various components of GCC, such as compiler
|
| 82 |
|
|
front ends and runtime libraries. While running the testsuite, DejaGnu
|
| 83 |
|
|
might emit some harmless messages resembling
|
| 84 |
|
|
‘<samp><span class="samp">WARNING: Couldn't find the global config file.</span></samp>’ or
|
| 85 |
|
|
‘<samp><span class="samp">WARNING: Couldn't find tool init file</span></samp>’ that can be ignored.
|
| 86 |
|
|
|
| 87 |
|
|
<p>If you are testing a cross-compiler, you may want to run the testsuite
|
| 88 |
|
|
on a simulator as described at <a href="http://gcc.gnu.org/simtest-howto.html">http://gcc.gnu.org/simtest-howto.html</a>.
|
| 89 |
|
|
|
| 90 |
|
|
<h3 class="section"><a name="TOC0"></a>How can you run the testsuite on selected tests?</h3>
|
| 91 |
|
|
|
| 92 |
|
|
<p>In order to run sets of tests selectively, there are targets
|
| 93 |
|
|
‘<samp><span class="samp">make check-gcc</span></samp>’ and ‘<samp><span class="samp">make check-g++</span></samp>’
|
| 94 |
|
|
in the <samp><span class="file">gcc</span></samp> subdirectory of the object directory. You can also
|
| 95 |
|
|
just run ‘<samp><span class="samp">make check</span></samp>’ in a subdirectory of the object directory.
|
| 96 |
|
|
|
| 97 |
|
|
<p>A more selective way to just run all <samp><span class="command">gcc</span></samp> execute tests in the
|
| 98 |
|
|
testsuite is to use
|
| 99 |
|
|
|
| 100 |
|
|
<pre class="smallexample"> make check-gcc RUNTESTFLAGS="execute.exp <var>other-options</var>"
|
| 101 |
|
|
</pre>
|
| 102 |
|
|
<p>Likewise, in order to run only the <samp><span class="command">g++</span></samp> “old-deja” tests in
|
| 103 |
|
|
the testsuite with filenames matching ‘<samp><span class="samp">9805*</span></samp>’, you would use
|
| 104 |
|
|
|
| 105 |
|
|
<pre class="smallexample"> make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <var>other-options</var>"
|
| 106 |
|
|
</pre>
|
| 107 |
|
|
<p>The <samp><span class="file">*.exp</span></samp> files are located in the testsuite directories of the GCC
|
| 108 |
|
|
source, the most important ones being <samp><span class="file">compile.exp</span></samp>,
|
| 109 |
|
|
<samp><span class="file">execute.exp</span></samp>, <samp><span class="file">dg.exp</span></samp> and <samp><span class="file">old-deja.exp</span></samp>.
|
| 110 |
|
|
To get a list of the possible <samp><span class="file">*.exp</span></samp> files, pipe the
|
| 111 |
|
|
output of ‘<samp><span class="samp">make check</span></samp>’ into a file and look at the
|
| 112 |
|
|
‘<samp><span class="samp">Running ... .exp</span></samp>’ lines.
|
| 113 |
|
|
|
| 114 |
|
|
<h3 class="section"><a name="TOC1"></a>Passing options and running multiple testsuites</h3>
|
| 115 |
|
|
|
| 116 |
|
|
<p>You can pass multiple options to the testsuite using the
|
| 117 |
|
|
‘<samp><span class="samp">--target_board</span></samp>’ option of DejaGNU, either passed as part of
|
| 118 |
|
|
‘<samp><span class="samp">RUNTESTFLAGS</span></samp>’, or directly to <samp><span class="command">runtest</span></samp> if you prefer to
|
| 119 |
|
|
work outside the makefiles. For example,
|
| 120 |
|
|
|
| 121 |
|
|
<pre class="smallexample"> make check-g++ RUNTESTFLAGS="--target_board=unix/-O3/-fmerge-constants"
|
| 122 |
|
|
</pre>
|
| 123 |
|
|
<p>will run the standard <samp><span class="command">g++</span></samp> testsuites (“unix” is the target name
|
| 124 |
|
|
for a standard native testsuite situation), passing
|
| 125 |
|
|
‘<samp><span class="samp">-O3 -fmerge-constants</span></samp>’ to the compiler on every test, i.e.,
|
| 126 |
|
|
slashes separate options.
|
| 127 |
|
|
|
| 128 |
|
|
<p>You can run the testsuites multiple times using combinations of options
|
| 129 |
|
|
with a syntax similar to the brace expansion of popular shells:
|
| 130 |
|
|
|
| 131 |
|
|
<pre class="smallexample"> ..."--target_board=arm-sim\{-mhard-float,-msoft-float\}\{-O1,-O2,-O3,\}"
|
| 132 |
|
|
</pre>
|
| 133 |
|
|
<p>(Note the empty option caused by the trailing comma in the final group.)
|
| 134 |
|
|
The following will run each testsuite eight times using the ‘<samp><span class="samp">arm-sim</span></samp>’
|
| 135 |
|
|
target, as if you had specified all possible combinations yourself:
|
| 136 |
|
|
|
| 137 |
|
|
<pre class="smallexample"> --target_board=arm-sim/-mhard-float/-O1
|
| 138 |
|
|
--target_board=arm-sim/-mhard-float/-O2
|
| 139 |
|
|
--target_board=arm-sim/-mhard-float/-O3
|
| 140 |
|
|
--target_board=arm-sim/-mhard-float
|
| 141 |
|
|
--target_board=arm-sim/-msoft-float/-O1
|
| 142 |
|
|
--target_board=arm-sim/-msoft-float/-O2
|
| 143 |
|
|
--target_board=arm-sim/-msoft-float/-O3
|
| 144 |
|
|
--target_board=arm-sim/-msoft-float
|
| 145 |
|
|
</pre>
|
| 146 |
|
|
<p>They can be combined as many times as you wish, in arbitrary ways. This
|
| 147 |
|
|
list:
|
| 148 |
|
|
|
| 149 |
|
|
<pre class="smallexample"> ..."--target_board=unix/-Wextra\{-O3,-fno-strength\}\{-fomit-frame,\}"
|
| 150 |
|
|
</pre>
|
| 151 |
|
|
<p>will generate four combinations, all involving ‘<samp><span class="samp">-Wextra</span></samp>’.
|
| 152 |
|
|
|
| 153 |
|
|
<p>The disadvantage to this method is that the testsuites are run in serial,
|
| 154 |
|
|
which is a waste on multiprocessor systems. For users with GNU Make and
|
| 155 |
|
|
a shell which performs brace expansion, you can run the testsuites in
|
| 156 |
|
|
parallel by having the shell perform the combinations and <samp><span class="command">make</span></samp>
|
| 157 |
|
|
do the parallel runs. Instead of using ‘<samp><span class="samp">--target_board</span></samp>’, use a
|
| 158 |
|
|
special makefile target:
|
| 159 |
|
|
|
| 160 |
|
|
<pre class="smallexample"> make -j<var>N</var> check-<var>testsuite</var>//<var>test-target</var>/<var>option1</var>/<var>option2</var>/...
|
| 161 |
|
|
</pre>
|
| 162 |
|
|
<p>For example,
|
| 163 |
|
|
|
| 164 |
|
|
<pre class="smallexample"> make -j3 check-gcc//sh-hms-sim/{-m1,-m2,-m3,-m3e,-m4}/{,-nofpu}
|
| 165 |
|
|
</pre>
|
| 166 |
|
|
<p>will run three concurrent “make-gcc” testsuites, eventually testing all
|
| 167 |
|
|
ten combinations as described above. Note that this is currently only
|
| 168 |
|
|
supported in the <samp><span class="file">gcc</span></samp> subdirectory. (To see how this works, try
|
| 169 |
|
|
typing <samp><span class="command">echo</span></samp> before the example given here.)
|
| 170 |
|
|
|
| 171 |
|
|
<h3 class="section"><a name="TOC2"></a>Additional testing for Java Class Libraries</h3>
|
| 172 |
|
|
|
| 173 |
|
|
<p>The Java runtime tests can be executed via ‘<samp><span class="samp">make check</span></samp>’
|
| 174 |
|
|
in the <samp><var>target</var><span class="file">/libjava/testsuite</span></samp> directory in
|
| 175 |
|
|
the build tree.
|
| 176 |
|
|
|
| 177 |
|
|
<p>The <a href="http://sourceware.org/mauve/">Mauve Project</a> provides
|
| 178 |
|
|
a suite of tests for the Java Class Libraries. This suite can be run
|
| 179 |
|
|
as part of libgcj testing by placing the Mauve tree within the libjava
|
| 180 |
|
|
testsuite at <samp><span class="file">libjava/testsuite/libjava.mauve/mauve</span></samp>, or by
|
| 181 |
|
|
specifying the location of that tree when invoking ‘<samp><span class="samp">make</span></samp>’, as in
|
| 182 |
|
|
‘<samp><span class="samp">make MAUVEDIR=~/mauve check</span></samp>’.
|
| 183 |
|
|
|
| 184 |
|
|
<h3 class="section"><a name="TOC3"></a>How to interpret test results</h3>
|
| 185 |
|
|
|
| 186 |
|
|
<p>The result of running the testsuite are various <samp><span class="file">*.sum</span></samp> and <samp><span class="file">*.log</span></samp>
|
| 187 |
|
|
files in the testsuite subdirectories. The <samp><span class="file">*.log</span></samp> files contain a
|
| 188 |
|
|
detailed log of the compiler invocations and the corresponding
|
| 189 |
|
|
results, the <samp><span class="file">*.sum</span></samp> files summarize the results. These summaries
|
| 190 |
|
|
contain status codes for all tests:
|
| 191 |
|
|
|
| 192 |
|
|
<ul>
|
| 193 |
|
|
<li>PASS: the test passed as expected
|
| 194 |
|
|
<li>XPASS: the test unexpectedly passed
|
| 195 |
|
|
<li>FAIL: the test unexpectedly failed
|
| 196 |
|
|
<li>XFAIL: the test failed as expected
|
| 197 |
|
|
<li>UNSUPPORTED: the test is not supported on this platform
|
| 198 |
|
|
<li>ERROR: the testsuite detected an error
|
| 199 |
|
|
<li>WARNING: the testsuite detected a possible problem
|
| 200 |
|
|
</ul>
|
| 201 |
|
|
|
| 202 |
|
|
<p>It is normal for some tests to report unexpected failures. At the
|
| 203 |
|
|
current time the testing harness does not allow fine grained control
|
| 204 |
|
|
over whether or not a test is expected to fail. This problem should
|
| 205 |
|
|
be fixed in future releases.
|
| 206 |
|
|
|
| 207 |
|
|
<h3 class="section"><a name="TOC4"></a>Submitting test results</h3>
|
| 208 |
|
|
|
| 209 |
|
|
<p>If you want to report the results to the GCC project, use the
|
| 210 |
|
|
<samp><span class="file">contrib/test_summary</span></samp> shell script. Start it in the <var>objdir</var> with
|
| 211 |
|
|
|
| 212 |
|
|
<pre class="smallexample"> <var>srcdir</var>/contrib/test_summary -p your_commentary.txt \
|
| 213 |
|
|
-m gcc-testresults@gcc.gnu.org |sh
|
| 214 |
|
|
</pre>
|
| 215 |
|
|
<p>This script uses the <samp><span class="command">Mail</span></samp> program to send the results, so
|
| 216 |
|
|
make sure it is in your <samp><span class="env">PATH</span></samp>. The file <samp><span class="file">your_commentary.txt</span></samp> is
|
| 217 |
|
|
prepended to the testsuite summary and should contain any special
|
| 218 |
|
|
remarks you have on your results or your build environment. Please
|
| 219 |
|
|
do not edit the testsuite result block or the subject line, as these
|
| 220 |
|
|
messages may be automatically processed.
|
| 221 |
|
|
|
| 222 |
|
|
<p><hr />
|
| 223 |
|
|
<p><a href="./index.html">Return to the GCC Installation page</a>
|
| 224 |
|
|
|
| 225 |
|
|
<!-- ***Final install*********************************************************** -->
|
| 226 |
|
|
<!-- ***Binaries**************************************************************** -->
|
| 227 |
|
|
<!-- ***Specific**************************************************************** -->
|
| 228 |
|
|
<!-- ***Old documentation****************************************************** -->
|
| 229 |
|
|
<!-- ***GFDL******************************************************************** -->
|
| 230 |
|
|
<!-- *************************************************************************** -->
|
| 231 |
|
|
<!-- Part 6 The End of the Document -->
|
| 232 |
|
|
</body></html>
|
| 233 |
|
|
|
