mirrors/gcc
0
0
Fork 0
mirror of https://gcc.gnu.org/git/gcc.git synced 2024-11-30 07:14:09 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

README: Move to...

Browse Source 
2003-05-20  Benjamin Kosnik  <bkoz@redhat.com>

	* testsuite/README: Move to...
	* docs/html/test.html: ...here. Add documentation.
	* docs/html/install.html: Move testing bits out..
	* docs/html/documentation.html: Add separate testing link.
        * testsuite/performance: Add.
        * testsuite/performance/allocator.cc: New.
        * testsuite/performance/complex_norm.cc: New.
        * testsuite/performance/cout_insert_int.cc: New.
        * testsuite/performance/fstream_seek_write.cc: New.
        * testsuite/performance/ifstream_getline.cc: New.
        * testsuite/performance/map_create_fill.cc: New.
        * testsuite/performance/ofstream_insert_float.cc: New.
        * testsuite/performance/ofstream_insert_int.cc: New.
        * testsuite/performance/string_append.cc: New.
	* testsuite/lib/libstdc++-v3-dg.exp (v3-compute-tests): Filter
	performance tests.

From-SVN: r67040
This commit is contained in:
Benjamin Kosnik 2003-05-20 22:25:09 +00:00 committed by Benjamin Kosnik
parent f737a52d03
commit ca5e1be83d
15 changed files with 1040 additions and 239 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
libstdc++-v3/ChangeLog
View File

@ -1,3 +1,22 @@
2003-05-20 Benjamin Kosnik <bkoz@redhat.com>
* testsuite/README: Move to...
* docs/html/test.html: ...here. Add documentation.
* docs/html/install.html: Move testing bits out..
* docs/html/documentation.html: Add separate testing link.
* testsuite/performance: Add.
* testsuite/performance/allocator.cc: New.
* testsuite/performance/complex_norm.cc: New.
* testsuite/performance/cout_insert_int.cc: New.
* testsuite/performance/fstream_seek_write.cc: New.
* testsuite/performance/ifstream_getline.cc: New.
* testsuite/performance/map_create_fill.cc: New.
* testsuite/performance/ofstream_insert_float.cc: New.
* testsuite/performance/ofstream_insert_int.cc: New.
* testsuite/performance/string_append.cc: New.
* testsuite/lib/libstdc++-v3-dg.exp (v3-compute-tests): Filter
performance tests.
2003-05-20 Gabriel Dos Reis <gdr@integrable-solutions.net>
PR libstdc++/10689

3
libstdc++-v3/docs/html/documentation.html
View File

@ -45,11 +45,12 @@
<hr />
<br />
<h2><a name="2">Configuring, Building, Installing</a></h2>
<h2><a name="2">Configuring, Building, Testing, Installing</a></h2>
<ul>
<li><a href="configopts.html">Configure options</a></li>
<li><a href="install.html">Getting started: configure, build, install</a>
</li>
<li><a href="test.html">Testing details</a>
<li><a href="debug.html">Debugging schemes and strategies</a>
</li>
</ul>

32
libstdc++-v3/docs/html/install.html
View File

@ -14,7 +14,7 @@
</head>
<body>
<h1 class="centered"><a name="top">libstdc++-v3 INSTALL</a></h1>
<h1 class="centered"><a name="top">Getting started: configure, build, install</a></h1>
<p class="fineprint"><em>
The latest version of this document is always available at
@ -312,36 +312,6 @@ se_NO.UTF-8 UTF-8
the headers and library files will be moved under
<code>lib/gcc-lib/</code> instead.
</p>
<p>You can check the status of the build without installing it using</p>
<pre>
make check</pre>
<p>or you can check the status of the installed library using</p>
<pre>
make check-install</pre>
<p>in the <em>libbuilddir</em> directory.
These commands will create a 'testsuite' directory underneath
<em>libbuilddir</em> containing the results of the tests. We are
interested in any strange failures of the testsuite; please see
<a href="faq/index.html#2_4">FAQ 2.4</a> for which files to examine.
</p>
<p> In addition, there are some testing options that are mostly of
interest to library maintainers and system integrators. As such,
these tests may not work on all cpu and host combinations. These
options include, but are not necessarily limited to, the following:
</p>
<p>The library ABI can be tested using</p>
<pre>
make check-abi</pre>
<p>The library can also be tested using a bash script, instead of
the default dejagnu test harness</p>
<pre>
make check-script</pre>
<p>or</p>
<pre>
make check-script-install</pre>
<hr />
<h2><a name="usage">Using the library</a></h2>

564
libstdc++-v3/docs/html/test.html Normal file
View File

@ -0,0 +1,564 @@
<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE html
PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta name="AUTHOR" content="bkoz@gcc.gnu.org (Benjamin Kosnik)" />
<meta name="KEYWORDS" content="c++, libstdc++, test, regression, g++" />
<meta name="DESCRIPTION" content="README for the GNU libstdc++ effort." />
<meta name="GENERATOR" content="vi and eight fingers" />
<title>libstdc++-v3 Testing Instructions</title>
<link rel="StyleSheet" href="lib3styles.css" />
</head>
<body>
<h1 class="centered"><a name="top">Testing Details</a></h1>
<p class="fineprint"><em>
The latest version of this document is always available at
<a href="http://gcc.gnu.org/onlinedocs/libstdc++/test.html">
http://gcc.gnu.org/onlinedocs/libstdc++/test.html</a>.
</em></p>
<p><em>
To the <a href="http://gcc.gnu.org/libstdc++/">libstdc++-v3 homepage</a>.
</em></p>
<!-- ####################################################### -->
<hr />
<h2>Contents</h2>
<ul>
<li><a href="#org">Testsuite organization and naming conventions</a></li>
<li><a href="#util">Utilities: abicheck and libv3test</a></li>
<li><a href="#new">How to write a new test case</a></li>
<li><a href="#check">Options for running the tests</a></li>
<li><a href="#future">Future</a></li>
</ul>
<hr />
<!-- ####################################################### -->
<h2><a name="org">Testsuite organization and naming conventions</a></h2>
<p>
The directory <em>libsrcdir/testsuite</em> contains the test
files, test harness, and utility information for verifying the
correctness of C++ library on a given host. It includes the
following directories, each named after a specific chapter of
the C++ standard, and each containing test files or
subdirectories of test files that test for that particular part
of the standard.
<p>
<pre>
17_intro
18_support
19_diagnostics
20_util
21_strings
22_locale
23_containers
25_algorithms
26_numerics
27_io
</pre>
<p>
In addition, the following directories include test files:
</p>
<pre>
backward Tests for backwards compatibility and deprecated features.
demangle Tests for __cxa_demangle, the IA 64 C++ ABI demangler
ext Tests for extensions.
performance Tests for performance analysis, and performance regressions.
thread Tests for threads.
</pre>
<p>
Some directories don't have test files, but instead contain
auxiliary information:
</p>
<pre>
config Files for the dejagnu test harness.
lib Files for the dejagnu test harness.
libstdc++-v3.dg Files for the dejagnu test harness.
data Sample text files for testing input and output.
</pre>
<p>
Within a directory that includes test files, there may be
additional subdirectories, or files: this particular point is in
flux. Originally, test cases were appended to one file that
represented a particular section of the chapter under test, and
was named accordingly. For instance, to test items related to
<code> 21.3.6.1 - basic_string::find [lib.string::find]</code>
in the standard, the following was used:
<p>
<pre>
21_strings/find.cc
</pre>
<p>
However, that practice soon became a liability as the test cases
became huge and unwieldy, and testing new or extended
functionality (like wide characters or named locales) became
frustrating, leading to aggressive pruning of test cases on some
platforms that covered up implementation errors. Now, the test
suite is converging on a policy of one file, one test case,
which solves the above issues and gives finer grained results
and more manageable error debugging. As an example, the test case
quoted above becomes:
</p>
<pre>
21_strings/basic_string/find/char/1.cc
21_strings/basic_string/find/char/2.cc
21_strings/basic_string/find/char/3.cc
21_strings/basic_string/find/wchar_t/1.cc
21_strings/basic_string/find/wchar_t/2.cc
21_strings/basic_string/find/wchar_t/3.cc
</pre>
<p>
All new tests should be written with the policy of one test
case, one file in mind. At some point the entire testsuite will
be converted: the current status is that the 21_string,
22_locale, 27_io, and demangle directories have all been
transitioned.
</p>
<p>
In addition, there are some special names and suffixes that are
used within the testsuite to designate particular kinds of
tests.
</p>
<ul>
<li>
<em>_xin.cc</em>
<p>
This test case expects some kind of interactive input in order
to finish or pass. At the moment, the interactive tests are not
run by default. Instead, they are run by hand, like:
<pre>
g++ 27_io/objects/char/3_xin.cc
cat 27_io/objects/char/3_xin.in | a.out
</pre>
</p>
</li>
<li>
<em>.in</em>
<p>
This file contains the expected input for the corresponding <em>
_xin.cc</em> test case.
</p>
</li>
<li>
<em>_neg.cc</em>
<p>
This test case is expected to fail: it's a negative test. At the
moment, these are almost always compile time errors.
</p>
</li>
<li>
<em>char</em>
<p>
This can either be a directory name or part of a longer file
name, and indicates that this file, or the files within this
directory are testing the <code>char</code> instantiation of a
template.
</p>
</li>
<li>
<em>wchar_t</em>
<p>
This can either be a directory name or part of a longer file
name, and indicates that this file, or the files within this
directory are testing the <code>wchar_t</code> instantiation of
a template. Some hosts do not support <code>wchar_t</code>
functionality, so for these targets, all of these tests will not
be run.
</p>
</li>
<li>
<em>performance</em>
<p>
This can either be an enclosing directory name or part of a
specific file name. This indicates a test that is used to
analyze runtime performance, for performance regression testing,
or for other optimization related analysis. At the moment, these
test cases are not run by default, and instead assumed to be run
manually.
</p>
</li>
</ul>
<hr />
<h2><a name="util">Utilities: abicheck and libv3test</a></h2>
<p>
The testsuite directory also contains some files that implement
functionality that is intended to make writing test cases easier,
or to avoid duplication, or to provide error checking in a way that
is consistent across platforms and test harnesses. A stand-alone
executable, called <em>abi_check</em>, and a static library called
<em>libv3test</em> are constructed during the build. Both of these
items are not installed, and only used during testing.
</p>
<p>
These files include the following functionality:
</p>
<ul>
<li>
<em>abi_check.cc</em>
<p>
Creates the executable <em>abi_check</em>.
Used to check correctness of symbol versioning, visibility of
exported symbols, and compatibility on symbols in the shared
library, for hosts that support this feature. More information
can be found in the ABI documentation <a href="abi.txt"> here</a>
</p>
</li>
<li>
<em>testsuite_allocator.h and </em>
<em>testsuite_allocator.cc</em>
<p>
Specialized allocators that keep track of construction and destruction
</p>
</li>
<li>
<em>testsuite_hooks.h and </em>
<em>testsuite_hooks.cc</em>
<p>
A large number of utilities, including:
</p>
<ul>
<li>VERIFY</li>
<li>set_memory_limits</li>
<li>verify_demangle</li>
<li>run_tests_wrapped_locale</li>
<li>run_tests_wrapped_env</li>
<li>try_named_locale</li>
<li>counter</li>
<li>copy_constructor</li>
<li>assignment_operator</li>
<li>destructor</li>
<li>copy_tracker</li>
<li>pod_char, pod_int and associated char_traits specializations</li>
</ul>
<p></p>
</li>
<li>
<em>printnow.c</em>
<p>
A cross-platform timer for use in one of the older harnesses
to determine compilation and link time.
</p>
</li>
</ul>
<hr />
<h2><a name="new">How to write a new test case</a></h2>
<p>
The first step in making a new test case is to choose the correct
directory and file name, given the organization as previously
described.
</p>
<p>
All files are copyright the FSF, and GPL'd: this is very
important. The first copyright year should correspond to the date
the file was checked in to CVS.
</p>
<p>
As per the dejagnu instructions, always return 0 from main to
indicate success.
</p>
<p>
A bunch of utility functions and classes have already been
abstracted out into the testsuite utility library, <code>
libv3test</code>. To use this functionality, just include the
appropriate header file: the library will automatically be linked
in as part of the testsuite run.
</p>
<p>
For a test that needs to take advantage of the dejagnu test
harness, what follows below is a list of special keyword that
harness uses. Basically, a test case contains dg-keywords (see
dg.exp) indicating what to do and what kinds of behavior are to be
expected. New test cases should be written with the new style
DejaGnu framework in mind.
</p>
<p>
To ease transition, here is the list of dg-keyword documentation
lifted from dg.exp.
</p>
<pre>
# The currently supported options are:
#
# dg-prms-id N
# set prms_id to N
#
# dg-options "options ..." [{ target selector }]
# specify special options to pass to the tool (eg: compiler)
#
# dg-do do-what-keyword [{ target/xfail selector }]
# `do-what-keyword' is tool specific and is passed unchanged to
# ${tool}-dg-test. An example is gcc where `keyword' can be any of:
# preprocess|compile|assemble|link|run
# and will do one of: produce a .i, produce a .s, produce a .o,
# produce an a.out, or produce an a.out and run it (the default is
# compile).
#
# dg-error regexp comment [{ target/xfail selector } [{.|0|linenum}]]
# indicate an error message <regexp> is expected on this line
# (the test fails if it doesn't occur)
# Linenum=0 for general tool messages (eg: -V arg missing).
# "." means the current line.
#
# dg-warning regexp comment [{ target/xfail selector } [{.|0|linenum}]]
# indicate a warning message <regexp> is expected on this line
# (the test fails if it doesn't occur)
#
# dg-bogus regexp comment [{ target/xfail selector } [{.|0|linenum}]]
# indicate a bogus error message <regexp> use to occur here
# (the test fails if it does occur)
#
# dg-build regexp comment [{ target/xfail selector }]
# indicate the build use to fail for some reason
# (errors covered here include bad assembler generated, tool crashes,
# and link failures)
# (the test fails if it does occur)
#
# dg-excess-errors comment [{ target/xfail selector }]
# indicate excess errors are expected (any line)
# (this should only be used sparingly and temporarily)
#
# dg-output regexp [{ target selector }]
# indicate the expected output of the program is <regexp>
# (there may be multiple occurrences of this, they are concatenated)
#
# dg-final { tcl code }
# add some tcl code to be run at the end
# (there may be multiple occurrences of this, they are concatenated)
# (unbalanced braces must be \-escaped)
#
# "{ target selector }" is a list of expressions that determine whether the
# test succeeds or fails for a particular target, or in some cases whether the
# option applies for a particular target. If the case of `dg-do' it specifies
# whether the test case is even attempted on the specified target.
#
# The target selector is always optional. The format is one of:
#
# { xfail *-*-* ... } - the test is expected to fail for the given targets
# { target *-*-* ... } - the option only applies to the given targets
#
# At least one target must be specified, use *-*-* for "all targets".
# At present it is not possible to specify both `xfail' and `target'.
# "native" may be used in place of "*-*-*".
Example 1: Testing compilation only
// { dg-do compile }
Example 2: Testing for expected warnings on line 36, which all targets fail
// { dg-warning "string literals" "" { xfail *-*-* } 36
Example 3: Testing for expected warnings on line 36
// { dg-warning "string literals" "" { target *-*-* } 36
Example 4: Testing for compilation errors on line 41
// { dg-do compile }
// { dg-error "no match for" "" { target *-*-* } 41 }
</pre>
<p>
More examples can be found in the libstdc++-v3/testsuite/*/*.cc files.
</p>
<hr />
<h2><a name="check">Options for running the tests</a></h2>
<p> There are several ways to run the testsuite. There are two
harnesses, one using dejagnu and one using bash. In addition, there
is a special rule for checking the ABI of the shared library.
</p>
<p>You can check the status of the build without installing it
using the dejagnu harness, much like the rest of the gcc tools.</p>
<p>
<pre> make check</pre> in the <em>libbuilddir</em> directory.</p>
<p>or</p>
<p><pre> make check-target-libstdc++-v3</pre> in the
<em>gccbuilddir</em> directory.</p>
<p>
These commands are equivalent and will create a 'testsuite'
directory underneath <em>libbuilddir</em> containing the results
of the tests. Two results files will be generated: <em>
libstdc++-v3.sum</em>, which is a PASS/FAIL summary for each
test, and <em>libstdc++.log</em> which is a log of the exact
command line passed to the compiler, the compiler output, and
the executable output (if any). In addition, four files are
generated that determine what test files are run. These files
are:
</p>
<ul>
<li>
<em>testsuite_files </em>
<p> This is a list of all the test cases that will be run. Each
test case is on a separate line, given with an absolute path
from the <em>libsrcdir/testsuite</em> directory.
</p>
</li>
<li>
<em>testsuite_files_interactive </em>
<p> This is a list of all the interactive test cases, using the
same format as the file list above. These tests are not run by default.
</p>
</li>
<li>
<em>testsuite_files_performance</em>
<p> This is a list of all the performance test cases, using the
same format as the file list above. These tests are not run by default.
</p>
</li>
<li>
<em>testsuite_wchar_t </em>
<p> This file indicates that the host system can run the wchar_t
tests, and corresponds to the macro definition <code>
_GLIBCPP_USE_WCHAR_T</code> in the file c++config.h.
</p>
</li>
</ul>
<p>
To debug the dejagnu test harness during runs, try invoking with a
specific argument to the variable RUNTESTFLAGS, as below.
</p>
<pre>
make check-target-libstdc++-v3 RUNTESTFLAGS="-v"
</pre>
or
<pre>
make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v"
</pre>
There are two ways to run on a simulator: set up DEJAGNU to point to a
specially crafted site.exp, or pass down --target_board flags.
Example flags to pass down for various embedded builds are as follows:
<pre>
--target=powerpc-eabism (libgloss/sim)
make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim"
--target=calmrisc32 (libgloss/sid)
make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid"
--target=xscale-elf (newlib/sim)
make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim"
</pre>
<p> In addition, there are some testing options that are mostly of
interest to library maintainers and system integrators. As such,
these tests may not work on all cpu and host combinations, and must
be executed in the <em>libbuilddir</em> directory.</p> These options
include, but are not necessarily limited to, the following:
</p>
<p>
The library can also be tested using a bash script, instead of
the default dejagnu test harness.
</p>
<pre>
make check-script</pre>
<p>
These commands use the generated test_file lists as above, but
run all the tests using both shared and static linking, and in
addition provide some additional diffing of expected output
files for the input/output tests. (This added diff may or may
not be useful or necessary at the moment.) In addition, these
tests provide size information for all the generated test cases,
so that size data for new compiler or linker features can be
collected. At one time timing information was attempted, so that
compile speeds, link speeds, etc. could be measured, however at
the moment all timing information is currently disabled.
</p>
<pre>
make check-script-install</pre>
<p> As directly above, but tests an installed library, not the
library and compiler in the build tree.
</p>
<pre>
make check-abi</pre>
<p>The library ABI can be tested. This involves testing the shared
library against an ABI-defining previous version.</p>
<p>
We are interested in any strange failures of the
testsuite; please see <a href="faq/index.html#2_4">FAQ 2.4</a>
for which files to examine.
</p>
<hr />
<h2><a name="future">Future</a></h2>
<p>
Shared runs need to be implemented, for targets that support shared libraries.
</p>
<p>
Diffing of expected output to standard streams needs to be finished off.
</p>
<p>
The V3 testing framework supports, or will eventually support,
additional keywords for the purpose of easing the job of writing
test cases. All V3-keywords are of the form @xxx@. Currently plans
for supported keywords include:
</p>
@require@ <files>
The existence of <files> is essential for the test to complete
successfully. For example, a test case foo.C using bar.baz as
input file could say
// @require@ bar.baz
The special variable % stands for the rootname, e.g. the
file-name without its `.C' extension. Example of use (taken
verbatim from 27_io/filebuf.cc)
// @require@ %-*.tst %-*.txt
@diff@ <first-list> <second-list>
After the test case compiles and ran successfully, diff
<first-list> against <second-list>, these lists should have the
same length. The test fails if diff returns non-zero a pair of
files.
<!-- ####################################################### -->
<hr />
<p class="fineprint"><em>
See <a href="17_intro/license.html">license.html</a> for copying conditions.
Comments and suggestions are welcome, and may be sent to
<a href="mailto:libstdc++@gcc.gnu.org">the libstdc++ mailing list</a>.
</em></p>
</body>
</html>

207
libstdc++-v3/testsuite/README
View File

@ -1,207 +0,0 @@
We're in the process of converting the existing testsuite machinery to
use the new style DejaGnu framework. Eventually, we'll abandon
../mkcheck.in in favor of this new testsuite framework.
// 1: Thoughts on naming test cases, and structuring them.
The testsuite directory has been divided into 11 directories, directly
correlated to the relevant chapters in the standard. For example, the
directory testsuite/21_strings contains tests related to "Chapter 21,
Strings library" in the C++ standard.
So, the first step in making a new test case is to choose the correct
directory. The second item is seeing if a test file exists that tests
the item in question. Generally, within chapters test files are named
after the section headings in ISO 14882, the C++ standard. For instance,
21.3.7.9 Inserters and Extractors
Has a related test case:
21_strings/inserters_extractors.cc
Not so hard. Some time, the words "ctor" and "dtor" are used instead
of "construct", "constructor", "cons", "destructor", etc. Other than
that, the naming seems mostly consistent. If the file exists, add a
test to it. If it does not, then create a new file. All files are
copyright the FSF, and GPL'd: this is very important.
In addition, some of the locale and io code tests different
instantiating types: thus, 'char' or 'wchar_t' is appended to the name
as constructed above.
Also, some test files are negative tests. That is, they are supposed
to fail (usually this involves making sure some kind of construct gets
an error when it's compiled.) These test files have 'neg' appended to
the name as constructed above.
Inside a test file, the plan is to test the relevant parts of the
standard, and then add specific regressions as additional test
functions, ie test04() can represent a specific regression noted in
GNATS. Once test files get unwieldy or too big, then they should be
broken up into multiple sub-categories, hopefully intelligently named
after the relevant (and more specific) part of the standard.
// 2: How to write a test case, from a dejagnu perspective
As per the dejagnu instructions, always return 0 from main to indicate
success.
Basically, a test case contains dg-keywords (see dg.exp) indicating
what to do and what kinds of behaviour are to be expected. New
testcases should be written with the new style DejaGnu framework in
mind.
To ease transition, here is the list of dg-keyword documentation
lifted from dg.exp -- eventually we should improve DejaGnu
documentation, but getting checkin account currently demands Pyrrhic
effort.
# The currently supported options are:
#
# dg-prms-id N
# set prms_id to N
#
# dg-options "options ..." [{ target selector }]
# specify special options to pass to the tool (eg: compiler)
#
# dg-do do-what-keyword [{ target/xfail selector }]
# `do-what-keyword' is tool specific and is passed unchanged to
# ${tool}-dg-test. An example is gcc where `keyword' can be any of:
# preprocess|compile|assemble|link|run
# and will do one of: produce a .i, produce a .s, produce a .o,
# produce an a.out, or produce an a.out and run it (the default is
# compile).
#
# dg-error regexp comment [{ target/xfail selector } [{.|0|linenum}]]
# indicate an error message <regexp> is expected on this line
# (the test fails if it doesn't occur)
# Linenum=0 for general tool messages (eg: -V arg missing).
# "." means the current line.
#
# dg-warning regexp comment [{ target/xfail selector } [{.|0|linenum}]]
# indicate a warning message <regexp> is expected on this line
# (the test fails if it doesn't occur)
#
# dg-bogus regexp comment [{ target/xfail selector } [{.|0|linenum}]]
# indicate a bogus error message <regexp> use to occur here
# (the test fails if it does occur)
#
# dg-build regexp comment [{ target/xfail selector }]
# indicate the build use to fail for some reason
# (errors covered here include bad assembler generated, tool crashes,
# and link failures)
# (the test fails if it does occur)
#
# dg-excess-errors comment [{ target/xfail selector }]
# indicate excess errors are expected (any line)
# (this should only be used sparingly and temporarily)
#
# dg-output regexp [{ target selector }]
# indicate the expected output of the program is <regexp>
# (there may be multiple occurrences of this, they are concatenated)
#
# dg-final { tcl code }
# add some tcl code to be run at the end
# (there may be multiple occurrences of this, they are concatenated)
# (unbalanced braces must be \-escaped)
#
# "{ target selector }" is a list of expressions that determine whether the
# test succeeds or fails for a particular target, or in some cases whether the
# option applies for a particular target. If the case of `dg-do' it specifies
# whether the testcase is even attempted on the specified target.
#
# The target selector is always optional. The format is one of:
#
# { xfail *-*-* ... } - the test is expected to fail for the given targets
# { target *-*-* ... } - the option only applies to the given targets
#
# At least one target must be specified, use *-*-* for "all targets".
# At present it is not possible to specify both `xfail' and `target'.
# "native" may be used in place of "*-*-*".
Example 1: Testing compilation only
(to just have a testcase do compile testing, without linking and executing)
// { dg-do compile }
Example 2: Testing for expected warings on line 36
// { dg-warning "string literals" "" { xfail *-*-* } 36
Example 3: Testing for compilation errors on line 41
// { dg-do compile }
// { dg-error "no match for" "" { xfail *-*-* } 41 }
More examples can be found in the libstdc++-v3/testsuite/*/*.cc files.
// 3: Test harness notes, invocation, and debugging.
Configuring the dejagnu harness to work with libstdc++-v3 in a cross
compilation environment has been maddening. However, it does work now,
and on a variety of platforms. Including solaris, linux, and cygwin.
To debug the test harness during runs, try invoking with
make check-target-libstdc++-v3 RUNTESTFLAGS="-v"
or
make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v"
There are two ways to run on a simulator: set up DEJAGNU to point to a
specially crafted site.exp, or pass down --target_board flags.
Example flags to pass down for various embedded builds are as follows:
--target=powerpc-eabism (libgloss/sim)
make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim"
--target=calmrisc32 (libgloss/sid)
make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid"
--target=xscale-elf (newlib/sim)
make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim"
// 4: Future plans, to be done
Shared runs need to be implemented, for targets that support shared libraries.
Diffing of expected output to standard streams needs to be finished off.
The V3 testing framework supports, or will eventually support,
additional keywords for the purpose of easing the job of writing
testcases. All V3-keywords are of the form @xxx@. Currently plans
for supported keywords include:
@require@ <files>
The existence of <files> is essential for the test to complete
successfully. For example, a testcase foo.C using bar.baz as
input file could say
// @require@ bar.baz
The special variable % stands for the rootname, e.g. the
file-name without its `.C' extension. Example of use (taken
verbatim from 27_io/filebuf.cc)
// @require@ %-*.tst %-*.txt
@diff@ <first-list> <second-list>
After the testcase compiles and ran successfully, diff
<first-list> against <second-list>, these lists should have the
same length. The test fails if diff returns non-zero a pair of
files.
Current testing problems with cygwin-hosted tools:
There are two known problems which I have not addressed. The first is
that when testing cygwin hosted tools from the unix build dir, it does
the wrong thing building the wrapper program (testglue.c) because host
and target are the same in site.exp (host and target are the same from
the perspective of the target libraries)
Problem number two is a little more annoying. In order for me to make
v3 testing work on Windows, I had to tell dejagnu to copy over the
debug_assert.h file to the remote host and then set the includes to
-I./. This is how all the other tests like this are done so I didn't
think much of it. However, this had some unfortunate results due to
gcc having a testcase called "limits" and C++ having an include file
called "limits". The gcc "limits" binary was in the temporary dir
when the v3 tests were being built. As a result, the gcc "limits"
binary was being #included rather than the intended one. The only way
to fix this is to go through the testsuites and make sure binaries are
deleted on the remote host when testing is done with them. That is a
lot more work than I want to do so I worked around it by cleaning out
D:\kermit on compsognathus and rerunning tests.

23
libstdc++-v3/testsuite/lib/libstdc++-v3-dg.exp
View File

@ -233,6 +233,7 @@ proc v3-compute-tests { } {
set tests_file "${outdir}/testsuite_files"
set tests_file_inter "${outdir}/testsuite_files_interactive"
set tests_file_perf "${outdir}/testsuite_files_performance"
set sfiles ""
# If there is a testsuite_file, use it.
@ -281,6 +282,21 @@ proc v3-compute-tests { } {
}
set sfiles ${restwo}
# Filter performance analysis tests.
# Use special rules to run these tests.
set resthree {}
set sfiles_perf {}
foreach i $sfiles {
if [regexp "performance" $i] {
verbose "element performance list is $i"
lappend sfiles_perf $i
} else {
verbose "element non-performancet list is $i"
lappend resthree $i
}
}
set sfiles ${resthree}
# Write out testsuite_files.
set f [open $tests_file "w"]
foreach t $sfiles {
@ -294,6 +310,13 @@ proc v3-compute-tests { } {
puts $f $t
}
close $f
# Write out testsuite_files_performance.
set f [open $tests_file_perf "w"]
foreach t $sfiles_perf {
puts $f $t
}
close $f
}
proc v3-list-tests { filename } {

39
libstdc++-v3/testsuite/performance/allocator.cc Normal file
View File

@ -0,0 +1,39 @@
// Copyright (C) 2003 Free Software Foundation, Inc.
//
// This file is part of the GNU ISO C++ Library. This library is free
// software; you can redistribute it and/or modify it under the
// terms of the GNU General Public License as published by the
// Free Software Foundation; either version 2, or (at your option)
// any later version.
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License along
// with this library; see the file COPYING. If not, write to the Free
// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
// USA.
// As a special exception, you may use this file as part of a free software
// library without restriction. Specifically, if other files instantiate
// templates or use macros or inline functions from this file, or you compile
// this file and link it with other files to produce an executable, this
// file does not by itself cause the resulting executable to be covered by
// the GNU General Public License. This exception does not however
// invalidate any other reasons why the executable file might be covered by
// the GNU General Public License.
#include <list>
// Primarily a test of the default allocator.
// http://gcc.gnu.org/ml/libstdc++/2001-05/msg00105.html
int main ()
{
std::list<int> List;
for (int i = 0; i < 10000000; ++i )
List.push_back( int() );
return 0;
}

59
libstdc++-v3/testsuite/performance/complex_norm.cc Normal file
View File

@ -0,0 +1,59 @@
// Copyright (C) 2003 Free Software Foundation, Inc.
//
// This file is part of the GNU ISO C++ Library. This library is free
// software; you can redistribute it and/or modify it under the
// terms of the GNU General Public License as published by the
// Free Software Foundation; either version 2, or (at your option)
// any later version.
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License along
// with this library; see the file COPYING. If not, write to the Free
// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
// USA.
// As a special exception, you may use this file as part of a free software
// library without restriction. Specifically, if other files instantiate
// templates or use macros or inline functions from this file, or you compile
// this file and link it with other files to produce an executable, this
// file does not by itself cause the resulting executable to be covered by
// the GNU General Public License. This exception does not however
// invalidate any other reasons why the executable file might be covered by
// the GNU General Public License.
#include <complex>
// based on libstdc++/5730, use --fast-math
int main()
{
typedef std::complex<double> complex_type;
complex_type u[2048];
for (int i = 0; i < 2048; ++i)
u[i] = 1.0;
for (int i = 0; i < 2000; ++i)
{
complex_type * p = u;
for (int j = 0; j < 2048; ++j)
{
#if 1
double u2 = norm(*p);
#else
// Shouldn't be slower than the above.
double ur = real(*p);
double ui = imag(*p);
double u2 = ur * ur + ui * ui;
#endif
double t = u2 * 0.1;
*p *= complex_type(cos(t), sin(t));
++p;
}
}
return 0;
}

36
libstdc++-v3/testsuite/performance/cout_insert_int.cc Normal file
View File

@ -0,0 +1,36 @@
// Copyright (C) 2003 Free Software Foundation, Inc.
//
// This file is part of the GNU ISO C++ Library. This library is free
// software; you can redistribute it and/or modify it under the
// terms of the GNU General Public License as published by the
// Free Software Foundation; either version 2, or (at your option)
// any later version.
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License along
// with this library; see the file COPYING. If not, write to the Free
// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
// USA.
// As a special exception, you may use this file as part of a free software
// library without restriction. Specifically, if other files instantiate
// templates or use macros or inline functions from this file, or you compile
// this file and link it with other files to produce an executable, this
// file does not by itself cause the resulting executable to be covered by
// the GNU General Public License. This exception does not however
// invalidate any other reasons why the executable file might be covered by
// the GNU General Public License.
#include <iostream>
// libstdc++/7076
int main()
{
for (int i = 0; i < 150000; i++)
std::cout << i << '\n';
return 0;
}

46
libstdc++-v3/testsuite/performance/fstream_seek_write.cc Normal file
View File

@ -0,0 +1,46 @@
// Copyright (C) 2003 Free Software Foundation, Inc.
//
// This file is part of the GNU ISO C++ Library. This library is free
// software; you can redistribute it and/or modify it under the
// terms of the GNU General Public License as published by the
// Free Software Foundation; either version 2, or (at your option)
// any later version.
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License along
// with this library; see the file COPYING. If not, write to the Free
// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
// USA.
// As a special exception, you may use this file as part of a free software
// library without restriction. Specifically, if other files instantiate
// templates or use macros or inline functions from this file, or you compile
// this file and link it with other files to produce an executable, this
// file does not by itself cause the resulting executable to be covered by
// the GNU General Public License. This exception does not however
// invalidate any other reasons why the executable file might be covered by
// the GNU General Public License.
#include <fstream>
// libstdc++/10672
int main()
{
using namespace std;
fstream s("tmp_perf_seek", ios::binary | ios::in | ios::out | ios::trunc);
if (s.good())
{
for (int i = 0; i < 300000; i++)
{
s.seekp(0);
s.write((char *) & i, sizeof(int));
s.seekp(sizeof(int));
s.write((char *) & i, sizeof(int));
}
}
return 0;
}

42
libstdc++-v3/testsuite/performance/ifstream_getline.cc Normal file
View File

@ -0,0 +1,42 @@
// Copyright (C) 2003 Free Software Foundation, Inc.
//
// This file is part of the GNU ISO C++ Library. This library is free
// software; you can redistribute it and/or modify it under the
// terms of the GNU General Public License as published by the
// Free Software Foundation; either version 2, or (at your option)
// any later version.
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License along
// with this library; see the file COPYING. If not, write to the Free
// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
// USA.
// As a special exception, you may use this file as part of a free software
// library without restriction. Specifically, if other files instantiate
// templates or use macros or inline functions from this file, or you compile
// this file and link it with other files to produce an executable, this
// file does not by itself cause the resulting executable to be covered by
// the GNU General Public License. This exception does not however
// invalidate any other reasons why the executable file might be covered by
// the GNU General Public License.
#include <fstream>
// libstdc++/5001 (100,000 line input file)
int main ()
{
using namespace std;
const char* name = "/usr/share/dict/linux.words";
ifstream in(name);
char buffer[BUFSIZ];
while(!in.eof())
{
in.getline(buffer, BUFSIZ);
}
}

49
libstdc++-v3/testsuite/performance/map_create_fill.cc Normal file
View File

@ -0,0 +1,49 @@
// 2003-03-01 gp dot bolton at computer dot org
// Copyright (C) 2003 Free Software Foundation, Inc.
//
// This file is part of the GNU ISO C++ Library. This library is free
// software; you can redistribute it and/or modify it under the
// terms of the GNU General Public License as published by the
// Free Software Foundation; either version 2, or (at your option)
// any later version.
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License along
// with this library; see the file COPYING. If not, write to the Free
// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
// USA.
#include <map>
#include <testsuite_hooks.h>
static bool test = true;
static void create_and_fill(const unsigned int n)
{
typedef std::map<int, int> Map;
Map m;
for (unsigned int i = 0; i < n; ++i)
m[i] = i;
VERIFY ( m.size() == n );
}
void test01()
{
const unsigned n = 10000000;
for (unsigned int i = 0; i < n; ++i)
create_and_fill( 0 );
}
// http://gcc.gnu.org/ml/libstdc++/2003-03/msg00000.html
int main()
{
test01();
return 0;
}

40
libstdc++-v3/testsuite/performance/ofstream_insert_float.cc Normal file
View File

@ -0,0 +1,40 @@
// Copyright (C) 2003 Free Software Foundation, Inc.
//
// This file is part of the GNU ISO C++ Library. This library is free
// software; you can redistribute it and/or modify it under the
// terms of the GNU General Public License as published by the
// Free Software Foundation; either version 2, or (at your option)
// any later version.
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License along
// with this library; see the file COPYING. If not, write to the Free
// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
// USA.
// As a special exception, you may use this file as part of a free software
// library without restriction. Specifically, if other files instantiate
// templates or use macros or inline functions from this file, or you compile
// this file and link it with other files to produce an executable, this
// file does not by itself cause the resulting executable to be covered by
// the GNU General Public License. This exception does not however
// invalidate any other reasons why the executable file might be covered by
// the GNU General Public License.
#include <fstream>
// based on libstdc++/8761 poor fstream performance (converted to float)
int main()
{
std::ofstream out("tmp_perf_float.txt");
for (int i = 0; i < 10000000; ++i)
{
float f = static_cast<float>(i);
out << f << "\n";
}
return 0;
};

37
libstdc++-v3/testsuite/performance/ofstream_insert_int.cc Normal file
View File

@ -0,0 +1,37 @@
// Copyright (C) 2003 Free Software Foundation, Inc.
//
// This file is part of the GNU ISO C++ Library. This library is free
// software; you can redistribute it and/or modify it under the
// terms of the GNU General Public License as published by the
// Free Software Foundation; either version 2, or (at your option)
// any later version.
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License along
// with this library; see the file COPYING. If not, write to the Free
// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
// USA.
// As a special exception, you may use this file as part of a free software
// library without restriction. Specifically, if other files instantiate
// templates or use macros or inline functions from this file, or you compile
// this file and link it with other files to produce an executable, this
// file does not by itself cause the resulting executable to be covered by
// the GNU General Public License. This exception does not however
// invalidate any other reasons why the executable file might be covered by
// the GNU General Public License.
#include <fstream>
// libstdc++/8761 poor fstream performance
int main()
{
std::ofstream out("tmp_perf_int.txt");
for (int i = 0; i < 10000000; ++i)
out << i << "\n";
return 0;
};

83
libstdc++-v3/testsuite/performance/string_append.cc Normal file
View File

@ -0,0 +1,83 @@
// Copyright (C) 2003 Free Software Foundation, Inc.
//
// This file is part of the GNU ISO C++ Library. This library is free
// software; you can redistribute it and/or modify it under the
// terms of the GNU General Public License as published by the
// Free Software Foundation; either version 2, or (at your option)
// any later version.
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License along
// with this library; see the file COPYING. If not, write to the Free
// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
// USA.
// As a special exception, you may use this file as part of a free software
// library without restriction. Specifically, if other files instantiate
// templates or use macros or inline functions from this file, or you compile
// this file and link it with other files to produce an executable, this
// file does not by itself cause the resulting executable to be covered by
// the GNU General Public License. This exception does not however
// invalidate any other reasons why the executable file might be covered by
// the GNU General Public License.
#include <ctime>
#include <iostream>
#include <string>
using namespace std;
void
test_append_char(int how_much)
{
string buf; // no preallocation
for (int i = 0; i < how_much; ++i)
buf.append(static_cast<string::size_type>(1) , 'x');
}
void
test_append_string(int how_much)
{
string s(static_cast<string::size_type>(1) , 'x');
string buf; // no preallocation
for (int i = 0; i < how_much; ++i)
buf.append(s);
}
void
run_benchmark1(int how_much)
{
clock_t t0 = clock();
test_append_char(how_much);
clock_t t1 = clock();
cout << "Execution time of " << how_much
<< " string::append(char) calls: "
<< (static_cast<float>(t1 - t0)/CLOCKS_PER_SEC) << " sec."<< endl;
}
void
run_benchmark2(int how_much)
{
clock_t t0 = clock();
test_append_string(how_much);
clock_t t1 = clock();
cout << "Execution time of " << how_much
<< " string::append(const string&) calls: "
<< (static_cast<float>(t1 - t0)/CLOCKS_PER_SEC) << " sec." << endl;
}
// libstdc++/5380
// libstdc++/4960
int main()
{
run_benchmark1(10000);
run_benchmark2(10000);
run_benchmark1(100000);
run_benchmark2(100000);
run_benchmark1(1000000);
run_benchmark2(1000000);
}