Subversion Repositories w11

# $Id: w11a_tb_guide.txt 352 2011-01-02 13:01:37Z mueller $

Guide to running w11a test benches

  Table of content:
  
  1. Unit tests benches
  2. Available unit tests benches
  3. System tests benches
  4. Available system tests benches


1. Unit tests benches -----------------------------------------------------

   All unit test benches have the same simple structure:

   - a stimulus process reads test patterns as well as the expected
     responses from a stimulus file

   - the responses are checked in very simple cases by the stimulus process,
     in general by a monitoring process

   - the test bench produces a comprehensive log file. For each checked
     response the line contains the word "CHECK" and either an "OK" or a
     "FAIL", in the later case in general with an indication of whats wrong.
     Other unexpected behaviour, like timeouts, will also result in a line
     containing the word "FAIL".

   - at the end a line with the word "DONE" is printed.

   - the test bench is run like

       tbw <testbenchname> [stimfile] | tee <logfile> | egrep "(FAIL|DONE)"

     where 
       - 'tbw' is a small perl script setting up a symbolic link to the
         stimulus file, the default extracted from the file tbw.dat, if
         an optional file name is give this one will be used instead.
       - 'tee' ensured that the full log is saved
       - 'egrep' filters FAIL and DONE lines, a successful run will
         produce a single DONE line

   - Most tests can be run against 
       - the functional model
       - gate level models at three stages
         - the post-xst model   (produced by netgen from ngc xst output)
         - the post-map model   (produced by netgen from ncd ngdbuild output)
         - the post-par model   (produced by netgen from ncd par output)
     This is simply done using
        make <testbench>_ssim       for post-xst
        make <testbench>_fsim       for post-map
        make <testbench>_tsim       for post-par
     all the rest is handled by the build environment.
     An example of a post-synthesis model is given for the w11a core test.

2. Available unit tests benches -------------------------------------------

   In the following the available tests are listed with 
     - the 'make' command to build them
     - the pipe setup to run them
     - the expected output (the run time measured on a 3 GHz system)

   - serport receiver test
     cd $RETROBASE/rtl/vlib/serport/tb
     make tb_serport_uart_rx
     time tbw tb_serport_uart_rx |\
       tee tb_serport_uart_rx_dsim.log | egrep "(FAIL|DONE)"
     -> 1269955.0 ns  63488: DONE 
     -> real    0m1.178s   user    0m1.172s   sys     0m0.020s


   - serport receiver/transmitter test
     make tb_serport_uart_rxtx
     time tbw tb_serport_uart_rxtx |\
       tee tb_serport_uart_rxtx_dsim.log | egrep "(FAIL|DONE)"
     ->  52335.0 ns   2607: DONE 
     -> real    0m0.094s   user    0m0.092s   sys     0m0.008s

   - serport autobauder test
     make tb_serport_autobaud
     time tbw tb_serport_autobaud |\
       tee tb_serport_autobaud_dsim.log | egrep "(FAIL|DONE)"
     -> 367475.0 ns  18364: DONE 
     -> real    0m0.610s   user    0m0.612s   sys     0m0.004s

   - rlink core test

     cd $RETROBASE/rtl/vlib/rlink/tb
     make tb_rlink_direct
     time tbw tb_rlink_direct |\
       tee tb_rlink_direct_dsim.log | egrep "(FAIL|DONE)"
     ->  142355.0 ns   7108: DONE 
     -> real    0m0.317s   user    0m0.324s   sys     0m0.028s

   - rlink core test via serial port interface

     make tb_rlink_serport
     time tbw tb_rlink_serport tb_rlink_serport_stim.dat |\
       tee tb_rlink_serport_stim2_dsim.log | egrep "(FAIL|DONE)"
     ->   72735.0 ns   3627: DONE 
     -> real    0m0.266s   user    0m0.264s   sys     0m0.008s

     time tbw tb_rlink_serport tb_rlink_stim.dat |\
       tee tb_rlink_serport_dsim.log | egrep "(FAIL|DONE)"
     -> 536155.0 ns  26798: DONE 
     -> real    0m1.714s   user    0m1.704s   sys     0m0.044s

   - w11a core test (using behavioural model)

     cd $RETROBASE/rtl/w11a/tb
     make tb_pdp11core
     time tbw tb_pdp11core |\
       tee tb_pdp11core_dsim.log | egrep "(FAIL|DONE)"
     -> 1220255.0 ns  61003: DONE 
     -> real    0m10.736s   user    0m10.713s   sys     0m0.060s

   - w11a core test (using post-synthesis model)

     make ghdl_tmp_clean tb_pdp11core_ssim
     time tbw tb_pdp11core_ssim |\
       tee tb_pdp11core_ssim.log | egrep "(FAIL|DONE)"
     ->  1220255.0 ns  61003: DONE 
     -> real    1m9.738s   user    1m9.588s   sys     0m0.096s

3. System tests benches ---------------------------------------------------

   The system tests allow to verify to verify the full 11/70 SoC design.
   In this case vhdl test bench code contains
     - (simple) models of the memories used on the FPGA boards
     - drivers for the rlink connection (currently just serialport)
     - code to interface the rlink data stream to a UNIX 'named pipe',
       implemented with a C routine which is called via VHPI from VHDL.
   This way the whole ghdl simulation can be controlled via a di-directional
   byte stream. 

   The rlink backend process, currently a perl script named pi_rri, can connect
   either via a named pipe to a ghdl simulation, or via a serial port to a 
   FPGA board. This way the same tests can be executed in simulation and
   on real hardware.

4. Available system tests benches -----------------------------------------

   The stimulus file used in the w11a core test can be executed in the
   full system context (both s3board and nexys2 versions) with the 
   following commands. Note that the cycle number printed in the DONE
   line can now vary slightly because the response time of the rlink
   backend process and thus scheduling of backend vs. ghdl process
   can affect the result.

   - sys_w11a_s3 system test

     cd $RETROBASE/rtl/sys_gen/w11a/s3board/tb
     make tb_w11a_s3
     time pi_rri --fifo --timeout=40. --cmax=3 \
        --run="tbw tb_w11a_s3" -- \
         @../../../../w11a/tb/tb_pdp11core_stim.dat |\
       tee tb_w11a_s3_stim2_dsim.log | egrep "(-[EW]:|FAIL|PEND|DONE)"
     -> 7757655.0 ns 387873: DONE
     -> real    0m49.835s   user    0m50.203s   sys     0m0.696s

   - sys_w11a_n2 system test

     cd $RETROBASE/rtl/sys_gen/w11a/nexys2/tb
     make tb_w11a_n2
     time pi_rri --fifo --timeout=40. --cmax=3 \
        --run="tbw tb_w11a_n2" -- \
         @../../../../w11a/tb/tb_pdp11core_stim.dat |\
       tee tb_w11a_n2_stim2_dsim.log | egrep "(-[EW]:|FAIL|PEND|DONE)"
     -> 6673237.2 ns 387035: DONE
     -> real    0m56.173s   user    0m56.612s   sys     0m0.604s