OpenCores
URL https://opencores.org/ocsvn/scarts/scarts/trunk

Subversion Repositories scarts

[/] [scarts/] [trunk/] [toolchain/] [scarts-gcc/] [gcc-4.1.1/] [INSTALL/] [test.html] - Blame information for rev 13

Details | Compare with Previous | View Log

Line No. Rev Author Line
1 13 jlechner 
<html lang="en">
2 
<head>
3 
<title>Installing GCC: Testing</title>
4 
<meta http-equiv="Content-Type" content="text/html">
5 
<meta name="description" content="Installing GCC: Testing">
6 
<meta name="generator" content="makeinfo 4.8">
7 
<link title="Top" rel="top" href="#Top">
8 
<link href="http://www.gnu.org/software/texinfo/" rel="generator-home" title="Texinfo Homepage">
9 
<!--
10 
Copyright (C) 1988, 1989, 1992, 1993, 1994, 1995, 1996, 1997, 1998,
11 
1999, 2000, 2001, 2002, 2003, 2004, 2005 Free Software Foundation, Inc.
12 
 
13 
   Permission is granted to copy, distribute and/or modify this document
14 
under the terms of the GNU Free Documentation License, Version 1.2 or
15 
any later version published by the Free Software Foundation; with no
16 
Invariant Sections, the Front-Cover texts being (a) (see below), and
17 
with the Back-Cover Texts being (b) (see below).  A copy of the
18 
license is included in the section entitled "GNU Free Documentation License".
19 
 
20 
(a) The FSF's Front-Cover Text is:
21 
 
22 
     A GNU Manual
23 
 
24 
(b) The FSF's Back-Cover Text is:
25 
 
26 
     You have freedom to copy and modify this GNU Manual, like GNU
27 
     software.  Copies published by the Free Software Foundation raise
28 
     funds for GNU development.-->
29 
<meta http-equiv="Content-Style-Type" content="text/css">
30 
<style type="text/css"><!--
31 
  pre.display { font-family:inherit }
32 
  pre.format  { font-family:inherit }
33 
  pre.smalldisplay { font-family:inherit; font-size:smaller }
34 
  pre.smallformat  { font-family:inherit; font-size:smaller }
35 
  pre.smallexample { font-size:smaller }
36 
  pre.smalllisp    { font-size:smaller }
37 
  span.sc    { font-variant:small-caps }
38 
  span.roman { font-family:serif; font-weight:normal; }
39 
  span.sansserif { font-family:sans-serif; font-weight:normal; }
40 
--></style>
41 
</head>
42 
<body>
43 
<h1 class="settitle">Installing GCC: Testing</h1>
44 
<a name="index-Testing-1"></a><a name="index-Installing-GCC_003a-Testing-2"></a><a name="index-Testsuite-3"></a>
45 
Before you install GCC, we encourage you to run the testsuites and to
46 
compare your results with results from a similar configuration that have
47 
been submitted to the
48 
<a href="http://gcc.gnu.org/ml/gcc-testresults/">gcc-testresults mailing list</a>.
49 
Some of these archived results are linked from the build status lists
50 
at <a href="http://gcc.gnu.org/buildstat.html">http://gcc.gnu.org/buildstat.html</a>, although not everyone who
51 
reports a successful build runs the testsuites and submits the results.
52 
This step is optional and may require you to download additional software,
53 
but it can give you confidence in your new GCC installation or point out
54 
problems before you install and start using your new GCC.
55 
 
56 
   <p>First, you must have <a href="download.html">downloaded the testsuites</a>.
57 
These are part of the full distribution, but if you downloaded the
58 
&ldquo;core&rdquo; compiler plus any front ends, you must download the testsuites
59 
separately.
60 
 
61 
   <p>Second, you must have the testing tools installed.  This includes
62 
<a href="http://www.gnu.org/software/dejagnu/">DejaGnu</a>, Tcl, and Expect;
63 
the DejaGnu site has links to these.
64 
 
65 
   <p>If the directories where <samp><span class="command">runtest</span></samp> and <samp><span class="command">expect</span></samp> were
66 
installed are not in the <samp><span class="env">PATH</span></samp>, you may need to set the following
67 
environment variables appropriately, as in the following example (which
68 
assumes that DejaGnu has been installed under <samp><span class="file">/usr/local</span></samp>):
69 
 
70 
<pre class="smallexample">          TCL_LIBRARY = /usr/local/share/tcl8.0
71 
          DEJAGNULIBS = /usr/local/share/dejagnu
72 
</pre>
73 
   <p>(On systems such as Cygwin, these paths are required to be actual
74 
paths, not mounts or links; presumably this is due to some lack of
75 
portability in the DejaGnu code.)
76 
 
77 
   <p>Finally, you can run the testsuite (which may take a long time):
78 
<pre class="smallexample">          cd <var>objdir</var>; make -k check
79 
</pre>
80 
   <p>This will test various components of GCC, such as compiler
81 
front ends and runtime libraries.  While running the testsuite, DejaGnu
82 
might emit some harmless messages resembling
83 
`<samp><span class="samp">WARNING: Couldn't find the global config file.</span></samp>' or
84 
`<samp><span class="samp">WARNING: Couldn't find tool init file</span></samp>' that can be ignored.
85 
 
86 
<h3 class="section"><a name="TOC0"></a>0.1 How can you run the testsuite on selected tests?</h3>
87 
 
88 
<p>In order to run sets of tests selectively, there are targets
89 
`<samp><span class="samp">make check-gcc</span></samp>' and `<samp><span class="samp">make check-g++</span></samp>'
90 
in the <samp><span class="file">gcc</span></samp> subdirectory of the object directory.  You can also
91 
just run `<samp><span class="samp">make check</span></samp>' in a subdirectory of the object directory.
92 
 
93 
   <p>A more selective way to just run all <samp><span class="command">gcc</span></samp> execute tests in the
94 
testsuite is to use
95 
 
96 
<pre class="smallexample">         make check-gcc RUNTESTFLAGS="execute.exp <var>other-options</var>"
97 
</pre>
98 
   <p>Likewise, in order to run only the <samp><span class="command">g++</span></samp> &ldquo;old-deja&rdquo; tests in
99 
the testsuite with filenames matching `<samp><span class="samp">9805*</span></samp>', you would use
100 
 
101 
<pre class="smallexample">         make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <var>other-options</var>"
102 
</pre>
103 
   <p>The <samp><span class="file">*.exp</span></samp> files are located in the testsuite directories of the GCC
104 
source, the most important ones being <samp><span class="file">compile.exp</span></samp>,
105 
<samp><span class="file">execute.exp</span></samp>, <samp><span class="file">dg.exp</span></samp> and <samp><span class="file">old-deja.exp</span></samp>.
106 
To get a list of the possible <samp><span class="file">*.exp</span></samp> files, pipe the
107 
output of `<samp><span class="samp">make check</span></samp>' into a file and look at the
108 
`<samp><span class="samp">Running ...  .exp</span></samp>' lines.
109 
 
110 
<h3 class="section"><a name="TOC1"></a>0.2 Passing options and running multiple testsuites</h3>
111 
 
112 
<p>You can pass multiple options to the testsuite using the
113 
`<samp><span class="samp">--target_board</span></samp>' option of DejaGNU, either passed as part of
114 
`<samp><span class="samp">RUNTESTFLAGS</span></samp>', or directly to <samp><span class="command">runtest</span></samp> if you prefer to
115 
work outside the makefiles.  For example,
116 
 
117 
<pre class="smallexample">         make check-g++ RUNTESTFLAGS="--target_board=unix/-O3/-fno-strength-reduce"
118 
</pre>
119 
   <p>will run the standard <samp><span class="command">g++</span></samp> testsuites (&ldquo;unix&rdquo; is the target name
120 
for a standard native testsuite situation), passing
121 
`<samp><span class="samp">-O3 -fno-strength-reduce</span></samp>' to the compiler on every test, i.e.,
122 
slashes separate options.
123 
 
124 
   <p>You can run the testsuites multiple times using combinations of options
125 
with a syntax similar to the brace expansion of popular shells:
126 
 
127 
<pre class="smallexample">         ..."--target_board=arm-sim/{-mhard-float,-msoft-float}{-O1,-O2,-O3,}"
128 
</pre>
129 
   <p>(Note the empty option caused by the trailing comma in the final group.)
130 
The following will run each testsuite eight times using the `<samp><span class="samp">arm-sim</span></samp>'
131 
target, as if you had specified all possible combinations yourself:
132 
 
133 
<pre class="smallexample">         --target_board=arm-sim/-mhard-float/-O1
134 
         --target_board=arm-sim/-mhard-float/-O2
135 
         --target_board=arm-sim/-mhard-float/-O3
136 
         --target_board=arm-sim/-mhard-float
137 
         --target_board=arm-sim/-msoft-float/-O1
138 
         --target_board=arm-sim/-msoft-float/-O2
139 
         --target_board=arm-sim/-msoft-float/-O3
140 
         --target_board=arm-sim/-msoft-float
141 
</pre>
142 
   <p>They can be combined as many times as you wish, in arbitrary ways.  This
143 
list:
144 
 
145 
<pre class="smallexample">         ..."--target_board=unix/-Wextra{-O3,-fno-strength-reduce}{-fomit-frame-pointer,}"
146 
</pre>
147 
   <p>will generate four combinations, all involving `<samp><span class="samp">-Wextra</span></samp>'.
148 
 
149 
   <p>The disadvantage to this method is that the testsuites are run in serial,
150 
which is a waste on multiprocessor systems.  For users with GNU Make and
151 
a shell which performs brace expansion, you can run the testsuites in
152 
parallel by having the shell perform the combinations and <samp><span class="command">make</span></samp>
153 
do the parallel runs.  Instead of using `<samp><span class="samp">--target_board</span></samp>', use a
154 
special makefile target:
155 
 
156 
<pre class="smallexample">         make -j<var>N</var> check-<var>testsuite</var>//<var>test-target</var>/<var>option1</var>/<var>option2</var>/...
157 
</pre>
158 
   <p>For example,
159 
 
160 
<pre class="smallexample">         make -j3 check-gcc//sh-hms-sim/{-m1,-m2,-m3,-m3e,-m4}/{,-nofpu}
161 
</pre>
162 
   <p>will run three concurrent &ldquo;make-gcc&rdquo; testsuites, eventually testing all
163 
ten combinations as described above.  Note that this is currently only
164 
supported in the <samp><span class="file">gcc</span></samp> subdirectory.  (To see how this works, try
165 
typing <samp><span class="command">echo</span></samp> before the example given here.)
166 
 
167 
<h3 class="section"><a name="TOC2"></a>0.3 Additional testing for Java Class Libraries</h3>
168 
 
169 
<p>The Java runtime tests can be executed via `<samp><span class="samp">make check</span></samp>'
170 
in the <samp><var>target</var><span class="file">/libjava/testsuite</span></samp> directory in
171 
the build tree.
172 
 
173 
   <p>The <a href="http://sources.redhat.com/mauve/">Mauve Project</a> provides
174 
a suite of tests for the Java Class Libraries.  This suite can be run
175 
as part of libgcj testing by placing the Mauve tree within the libjava
176 
testsuite at <samp><span class="file">libjava/testsuite/libjava.mauve/mauve</span></samp>, or by
177 
specifying the location of that tree when invoking `<samp><span class="samp">make</span></samp>', as in
178 
`<samp><span class="samp">make MAUVEDIR=~/mauve check</span></samp>'.
179 
 
180 
   <p><a href="http://sources.redhat.com/mauve/jacks.html">Jacks</a>
181 
is a free testsuite that tests Java compiler front ends.  This suite
182 
can be run as part of libgcj testing by placing the Jacks tree within
183 
the libjava testsuite at <samp><span class="file">libjava/testsuite/libjava.jacks/jacks</span></samp>.
184 
 
185 
<h3 class="section"><a name="TOC3"></a>0.4 How to interpret test results</h3>
186 
 
187 
<p>The result of running the testsuite are various <samp><span class="file">*.sum</span></samp> and <samp><span class="file">*.log</span></samp>
188 
files in the testsuite subdirectories.  The <samp><span class="file">*.log</span></samp> files contain a
189 
detailed log of the compiler invocations and the corresponding
190 
results, the <samp><span class="file">*.sum</span></samp> files summarize the results.  These summaries
191 
contain status codes for all tests:
192 
 
193 
     <ul>
194 
<li>PASS: the test passed as expected
195 
<li>XPASS: the test unexpectedly passed
196 
<li>FAIL: the test unexpectedly failed
197 
<li>XFAIL: the test failed as expected
198 
<li>UNSUPPORTED: the test is not supported on this platform
199 
<li>ERROR: the testsuite detected an error
200 
<li>WARNING: the testsuite detected a possible problem
201 
</ul>
202 
 
203 
   <p>It is normal for some tests to report unexpected failures.  At the
204 
current time the testing harness does not allow fine grained control
205 
over whether or not a test is expected to fail.  This problem should
206 
be fixed in future releases.
207 
 
208 
<h3 class="section"><a name="TOC4"></a>0.5 Submitting test results</h3>
209 
 
210 
<p>If you want to report the results to the GCC project, use the
211 
<samp><span class="file">contrib/test_summary</span></samp> shell script.  Start it in the <var>objdir</var> with
212 
 
213 
<pre class="smallexample">         <var>srcdir</var>/contrib/test_summary -p your_commentary.txt \
214 
             -m gcc-testresults@gcc.gnu.org |sh
215 
</pre>
216 
   <p>This script uses the <samp><span class="command">Mail</span></samp> program to send the results, so
217 
make sure it is in your <samp><span class="env">PATH</span></samp>.  The file <samp><span class="file">your_commentary.txt</span></samp> is
218 
prepended to the testsuite summary and should contain any special
219 
remarks you have on your results or your build environment.  Please
220 
do not edit the testsuite result block or the subject line, as these
221 
messages may be automatically processed.
222 
 
223 
   <p><hr />
224 
<p><a href="./index.html">Return to the GCC Installation page</a>
225 
 
226 
<!-- ***Final install*********************************************************** -->
227 
<!-- ***Binaries**************************************************************** -->
228 
<!-- ***Specific**************************************************************** -->
229 
<!-- ***Old documentation****************************************************** -->
230 
<!-- ***GFDL******************************************************************** -->
231 
<!-- *************************************************************************** -->
232 
<!-- Part 6 The End of the Document -->
233 
</body></html>
234 
 

powered by: WebSVN 2.1.0

© copyright 1999-2024 OpenCores.org, equivalent to Oliscience, all rights reserved. OpenCores®, registered trademark.