Subversion Repositories or1k

  DejaGnu is a framework for testing other programs. Its purpose is to

provide a single front end for all tests.  Beyond this, DejaGnu offers

several advantages for testing:

        - The flexibility and consistency of the DejaGnu framework

          make it easy to write tests for any program.

        - DejaGnu provides a layer of abstraction which makes all

          tests (if correctly written) portable to any host or target

          where a program must be tested.  For instance, a test for

          GDB can run (from any Unix based host) on any target

          architecture supported by DejaGnu. Currently DejaGnu runs

          tests on several single board computers, whose operating

          software ranges from just a boot monitor to a full-fledged,

          Unix-like realtime OS.

        - DejaGnu is written in expect, which in turn uses Tcl

          (Tool command language).  The framework comprises two parts:

          the testing framework and the testsuites themselves. Tests

          are usually written in expect using Tcl.

  Bugs can be reported to bug-dejagnu@prep.ai.mit.edu.

                      How To Configure and Build

 To build DejaGnu, run the ``configure'' script here, e.g.:

        ./configure MYHOSTTYPE

followed by running ``make''.  (MYHOSTTYPE is a name for your host computer,

for instance "sun4".  You can use the script ``config.sub'' to test whether

a name is recognized; if it is, config.sub translates it to a triplet

specifying CPU, vendor, and OS.) This is used when you plan to

configure and build in the source tree.

  If you use a separate tree for object files, (the recommended way),

then the --srcdir option must also be specified. This would also

require that the configure script be run from the top level directory.

        PATH/configure MYHOSTYPE --srcdir PATH/dejagnu

where PATH is is the directory that the contains the sources.

  To configure it so it gets installed somewhere other than the

default of /usr/local, use the --prefix option.

        configure MYHOSTYPE --prefix [PATH]

where PATH is the prefix used to install the programs.

  The configure testing and building will use the native compiler "cc"

on your host machine. To change which compiler gets used (like gcc)

set a the variable "CC" in your environment to point to it.

        For csh users:          "setenv CC gcc"

        For bourne shell users: "CC=gcc;export CC"

  Then when you compile, use "make CC=$CC".

See etc/cfg-paper.texi, etc/configure.texi, and/or the README files in

various subdirectories, for more details.

 As DejaGnu is a Tcl program, there is nothing to build. However, the

documentation is not built by default. Use these targets:

"make info"     - Convert the texinfo document to something that can

                  be used the GNU info program or info mode in emacs.

"make dvi"      - Convert the texinfo document to something that can

                  be printed. This produces dvi output.

"make doc"      - This builds both.

"make ps"       - This converts the dvi file into postscript. This

                  requires a copy of dvips.

"make install"  - This installs DejaGnu based on the --prefix option

                  when configuring. Otherwise it defaults to

                  /usr/local. See the DejaGnu manual for more

                  information on installation.

                          Changes from 1.1.1

        1. Works with (included in release) Tcl 7.3 and Expect 5.6.

        2. Much better error trapping and handling, including the

           execution of sub scripts.

        3. Re-worked configuration subsystem.

        4. Default handling for testing unknown targets.

        5. New testsuite for expect and runtest.

        6. More debugging procedures.

                          Changes from 1.0

        1. DejaGnu now conforms to POSIX 1003.3, a standard for

           testing frameworks.

        2. A Tcl debugger written by Don Libes has been added.

        3. Lots of bug fixes.

                          Changes from 0.9

        1. DejaGnu now installs itself like other utilities.

        2. 700 G++ tests are included.

        3. The bugs in the GCC tests have been fixed.

        4. Testsuites are released separately.

        5. Testsuite sources now reside with the within each tool's

           source tree.

-------------------------------------------------------------------

Here's is a posting on the Free OS testing project. (FROST)

-------------------------------------------------------------------

If you have built up your own tests for system calls, library

routines, networking protocols, or common utilities, this posting

offers you an opportunity to put them to good use.  You can do a good

deed for the Linux community, the Berkeley UNIX community, and the

wider world of free software users.

Also, if you would like to learn some good test tools and strategies,

and devote some time on a volunteer basis to writing tests -- perhaps

a couple of weeks full-time, or a couple hours per week over a period

of several months -- then here is a project you should get involved

in.

I am helping to coordinate a test effort for Linux.  In the following

four sections of this message I will describe the goals, the process,

the people trying to do it, and what you can do to help.

Goals

  Linux and BSD share a number of libraries and utilities, both

  because BSD software was ported to Linux, and because several free

  software (GNU) utilities have been ported to both operating

  systems.

  Linux shows startling differences and failures as you move from one

  set of hardware to another.  People who hope to base their own

  commercial products on Linux would like to see proof that it is

  robust, portable, and standard-conforming.  The problem is that

  cross-platform testing is very hard to do, and few decent test

  suites exist either for free software or for UNIX.  (Look at all the

  differences you find among systems that are SVID-conforming,

  particularly at higher levels such as the utilities.)

  Therefore, a number of us who are interested n Linux have decided to

  enter the modern age of formal, automated software testing -- but in

  a manner that is proper for the free software community.

  Using DejaGnu, a test platform developed by Cygnus Support and

  placed under the GPL, we want to collect the best tests we can from

  people everywhere, and integrate them into suites that any user can

  download, run, and interpret with a few commands.  DejaGnu already

  runs under BSD, and Cygnus is porting it to Linux.

  Our goal is to test as many parts of the system as possible,

  including system calls (particularly for POSIX 1003.1 compliance),

  libraries in the official distribution, networking protocols,

  and utilities.  We need help with

    1) integrating good tests donated by the community (and probably

       written in a variety of programming languages and command

       shells) into the DejaGnu framework, and

    2) writing new tests for major functional areas for which no

       adequate tests exist.

  The tests we use will be placed under the GPL and distributed by

  Cygnus Support along with DejaGnu.  H. J. Lu, who maintains the C

  library for Linux, has offered to run the POSIX 1003.1 tests and as

  many others as he can.

Process

  First let me summarize the advantages of using DejaGnu, a free

  software product that will be the umbrella for all the tests, and

  then explain how we plan to conduct this project.

  Cygnus Support released DejaGnu to the public on January 3, along

  with several test suites for GNU language tools.  The product is

  designed for portability and easy cross-platform development and

  execution.  It works more uniformly than typical tests using the

  UNIX shell, and also supports interactive testing better than most

  test platforms -- for instance, Cygnus has written over 1300 unit

  tests for the gdb debugger using it.

  The implementation of DejaGnu is based on tcl and expect, two simple

  existing languages for writing commands.  You can develop new tests

  in two major ways: by placing keyword-based comments in C source

  code, or by writing tests in tcl.

  While Cygnus is still increasing their body of tests for language

  tools, they are also turning their resources toward the libraries

  and GNU utilities.  While most of the tests are written at Cygnus,

  they have reached out to communities of testers and are now porting

  substantial sets of donated tests.

  We are hoping to broaden this successful use of collaboration across

  user communities.  We hope that suites of useful tests are sitting

  in desk drawers out there.  We also hope to gather and mobilize

  people who appreciate the value of formal tests in legitimizing free

  software, and would like to help write them.

  I am not an employee of Cygnus, but have volunteered to talk to

  interested people and do an initial classification of tests, just to

  offload some of this routine work from them.  I will discuss all

  offers with Cygnus staff to find out what is most needed and decide

  which tests to incorporate into DejaGnu.  There are several criteria

  for choosing tests, including the degree to which an area is

  covered, and its importance in the overall stability of the

  operating system.  Some tests may have to be rejected just because

  they are hard to fit into the DejaGnu model.

People

  Now you can find out why each of us got involved with this project.

  Cygnus writes and distributes free software, selling support as a

  means to make money.  The company is not directly in the business of

  supporting operating systems or common UNIX utilities, so this kind

  of testing is tangential to their main goals.  But they may want to

  support those things in the future.  In any case, they would like to

  see Linux do well, and this big test project will be a good

  promotion for DejaGnu.

  H. K. Lu, as a volunteer for Linux, has been using his own system to

  implement, port, maintain, and ensure ANSI and POSIX compliance for

  the C library (mostly the GNU C library along with the iostream

  class from the g++ library).  He is looking for ways to validate the

  work that he and many other contributors have put in.

  As an editor at a publisher of computer books, my relationship to

  Linux and BSD is even more distant than that of Cygnus.  But we are

  thinking about putting out books about Linux, from either the Linux

  Documentation Project or independent authors, and would like to make

  sure Linux is stable enough to be documented.  The testing of

  utilities is particularly important to me, because it can provide

  quality assurance for our books, including the BSD version of UNIX

  in a Nutshell.

What you can do

  We ask people who have written tests in the areas I have described

  to donate them to this project and see them benefit the public.

  Don't just send me stuff -- write or call to discuss what you've got

  in general.  You also have to assign the tests to the Free Software

  Foundation (a simple matter, so long as you are the owner of the

  tests) so that they can be distributed under the GPL.

  Also, please don't starting flaming about the GPL.  Either join our

  project and donate your tests, or don't.  Personally, I do not take

  a side in this war.

  People who are interested in testing as a discipline, and have some

  time to donate, are invited to contact me to help write tests.  Tell

  me:

        Your knowledge of computer languages and tools

        Previous software test efforts you have engaged in, if any

        How much time you can devote

        What functional areas you think are important and have a

        particular interest in

  Cygnus staff will tell volunteers which areas need testing, and

  offer guidance while you learn and apply DejaGnu.  The most

  important trait for the project is rigorous thinking along the lines

  of software quality (so the best preparation is to have done some

  formal testing before).  You do not need to know any particular

  language or tool; you can learn tcl or other aspects of DejaGnu

  fairly quickly.

  Contact information for me is in my signature.  I am at the phone

  number during traditional U.S. business hours.  So send mail any

  time, or try calling from Monday through Friday, 9:00 AM to 5:00 PM,

  Eastern Daylight Saving Time (four hours later than Greenwich Mean

  Time).

  Feel free to distribute and repost this message elsewhere, in its

  entirety.

----------------------------------------------------------------------

Andy Oram    O'Reilly & Associates, Inc.                 andyo@ora.com

             90 Sherman Street, Cambridge, MA 02140     (617) 354-5800

                                                    fax (617) 661-1116

----------------------------------------------------------------------

There is a new listserv based mailing list for the FROST project. To

subscribe, send email to "listserv@cygnus.com" and the body should

contain "SUBSCRIBE FROST ". Messages sent to

frost@cygnus.com go to the mailing list.