Subversion Repositories w11

# $Id: w11a_tb_guide.txt 442 2011-12-23 10:03:28Z mueller $

Guide to running w11a test benches

  Table of content:

  1. Unit tests benches

  2. Available unit tests benches

  3. System tests benches

  4. Available system tests benches

1. Unit tests benches -----------------------------------------------------

   All unit test benches have the same simple structure:

   - a stimulus process reads test patterns as well as the expected

     responses from a stimulus file

   - the responses are checked in very simple cases by the stimulus process,

     in general by a monitoring process

   - the test bench produces a comprehensive log file. For each checked

     response the line contains the word "CHECK" and either an "OK" or a

     "FAIL", in the later case in general with an indication of whats wrong.

     Other unexpected behaviour, like timeouts, will also result in a line

     containing the word "FAIL".

   - at the end a line with the word "DONE" is printed.

   - the test bench is run like

       tbw  [stimfile] | tee  | egrep "(FAIL|DONE)"

     where

       - 'tbw' is a small perl script setting up a symbolic link to the

         stimulus file, the default extracted from the file tbw.dat, if

         an optional file name is give this one will be used instead.

       - 'tee' ensured that the full log is saved

       - 'egrep' filters FAIL and DONE lines, a successful run will

         produce a single DONE line

   - Most tests can be run against

       - the functional model

       - gate level models at three stages

         - the post-xst model   (produced by netgen from ngc xst output)

         - the post-map model   (produced by netgen from ncd ngdbuild output)

         - the post-par model   (produced by netgen from ncd par output)

     This is simply done using

        make _ssim       for post-xst

        make _fsim       for post-map

        make _tsim       for post-par

     all the rest is handled by the build environment.

     An example of a post-synthesis model is given for the w11a core test.

2. Available unit tests benches -------------------------------------------

   In the following the available tests are listed with

     - the 'make' command to build them

     - the pipe setup to run them

     - the expected output (the run time measured on a 3 GHz system)

   - serport receiver test

     cd $RETROBASE/rtl/vlib/serport/tb

     make tb_serport_uart_rx

     time tbw tb_serport_uart_rx |\

       tee tb_serport_uart_rx_dsim.log | egrep "(FAIL|DONE)"

     -> 1269955.0 ns  63488: DONE

     -> real 0m01.178s   user 0m01.172s   sys 0m00.020s

   - serport receiver/transmitter test

     make tb_serport_uart_rxtx

     time tbw tb_serport_uart_rxtx |\

       tee tb_serport_uart_rxtx_dsim.log | egrep "(FAIL|DONE)"

     ->  52335.0 ns   2607: DONE

     -> real 0m00.094s   user 0m00.092s   sys 0m00.008s

   - serport autobauder test

     make tb_serport_autobaud

     time tbw tb_serport_autobaud |\

       tee tb_serport_autobaud_dsim.log | egrep "(FAIL|DONE)"

     -> 367475.0 ns  18364: DONE

     -> real 0m00.610s   user 0m00.612s   sys 0m00.004s

   - rlink core test

     cd $RETROBASE/rtl/vlib/rlink/tb

     make tb_rlink_direct

     time tbw tb_rlink_direct |\

       tee tb_rlink_direct_dsim.log | egrep "(FAIL|DONE)"

     ->  142355.0 ns   7108: DONE

     -> real 0m00.317s   user 0m00.324s   sys 0m00.028s

   - rlink core test via serial port interface

     make tb_rlink_serport

     time tbw tb_rlink_serport tb_rlink_serport_stim.dat |\

       tee tb_rlink_serport_stim2_dsim.log | egrep "(FAIL|DONE)"

     ->   72735.0 ns   3627: DONE

     -> real 0m00.266s   user 0m00.264s   sys 0m00.008s

     time tbw tb_rlink_serport tb_rlink_stim.dat |\

       tee tb_rlink_serport_dsim.log | egrep "(FAIL|DONE)"

     -> 536155.0 ns  26798: DONE

     -> real  0m01.714s   user  0m01.704s   sys  0m00.044s

   - w11a core test (using behavioural model)

     cd $RETROBASE/rtl/w11a/tb

     make tb_pdp11core

     time tbw tb_pdp11core |\

       tee tb_pdp11core_dsim.log | egrep "(FAIL|DONE)"

     -> 1220255.0 ns  61003: DONE

     -> real 0m10.736s   user 0m10.713s   sys 0m00.060s

   - w11a core test (using post-synthesis model)

     make ghdl_tmp_clean tb_pdp11core_ssim

     time tbw tb_pdp11core_ssim |\

       tee tb_pdp11core_ssim.log | egrep "(FAIL|DONE)"

     ->  1220255.0 ns  61003: DONE

     -> real 1m09.738s   user 1m09.588s   sys 0m00.096s

   - s3board sram controller test

     cd $RETROBASE/rtl/bplib/s3board/tb

     make tb_s3_sram_memctl

     time tbw tb_s3_sram_memctl |\

       tee tb_s3_sram_memctl_dsim.log | egrep "(FAIL|DONE)"

     -> 5015.0 ns    241: DONE

     -> real 0m00.113s   user 0m00.068s   sys 0m00.016s

   - nexys2/nexys3 cram controller test

     cd $RETROBASE/rtl/bplib/nxcramlib/tb

     make tb_nx_cram_memctl_as

     time tbw tb_nx_cram_memctl_as |\

       tee tb_nx_cram_memctl_as_dsim.log | egrep "(FAIL|DONE)"

     -> 24272.5 ns   1204: DONE

     -> real 0m00.343s   user 0m00.248s   sys 0m00.100s

3. System tests benches ---------------------------------------------------

   The system tests allow to verify to verify a full system design.

   In this case vhdl test bench code contains

     - (simple) models of the memories used on the FPGA boards

     - drivers for the rlink connection (currently just serialport)

     - code to interface the rlink data stream to a UNIX 'named pipe',

       implemented with a C routine which is called via VHPI from VHDL.

   This way the whole ghdl simulation can be controlled via a di-directional

   byte stream.

   The rlink backend process can connect either via a named pipe to a ghdl

   simulation, or via a serial port to a FPGA board. This way the same tests

   can be executed in simulation and on real hardware.

   Currently two backend implementations are available:

   - pi_rri: written in perl (old, slow, but full functionality)

   - ti_tti: written in C++ and Tcl (new, fast, but as of V0.55 only with

               limited functionality; will replace pi_rri).

4. Available system tests benches -----------------------------------------

4a. serport tester -- --------------------------------------------

   The sys_tst_serloop design is a test target for validating the serial

   link UART stack. Send and receive throughput as well as loop-back tests

   are supported

   - sys_tst_serloop_s3 test bench

     cd $RETROBASE/rtl/sys_gen/tst_serloop/s3board/tb

     make tb_tst_serloop_s3

     time tbw tb_tst_serloop_s3 |\

       tee tb_tst_serloop_s3_dsim.log | egrep "(FAIL|DONE)"

     -> 301353.3 ns  18068: DONE

     -> real 0m1.422s   user 0m1.372s   sys 0m0.024s

   - sys_tst_serloop_n2 test bench

     cd $RETROBASE/rtl/sys_gen/tst_serloop/nexys2/tb

     make tb_tst_serloop1_n2

     time tbw tb_tst_serloop1_n2 |\

       tee tb_tst_serloop1_n2_dsim.log | egrep "(FAIL|DONE)"

     -> 361560.0 ns  18068: DONE

     -> real 0m1.341s   user 0m1.340s   sys 0m0.016s

     make tb_tst_serloop2_n2

     time tbw tb_tst_serloop2_n2 |\

       tee tb_tst_serloop2_n2_dsim.log | egrep "(FAIL|DONE)"

     -> 304353.3 ns  18248: DONE

     -> real 0m1.933s   user 0m1.924s   sys 0m0.024s

   - sys_tst_serloop_n3 test bench

     cd $RETROBASE/rtl/sys_gen/tst_serloop/nexys3/tb

     make tb_tst_serloop1_n3

     time tbw tb_tst_serloop1_n3 |\

       tee tb_tst_serloop1_n3_dsim.log | egrep "(FAIL|DONE)"

     -> 361560.0 ns  18068: DONE

     -> real 0m1.371s   user 0m1.372s   sys 0m0.016s

4b. rlink tester -----------------------------------------------------

   The sys_tst_rlink design is a test target for validating the rlink

   and rbus functionality at all levels.

   - sys_tst_rlink_s3 test bench

     cd $RETROBASE/rtl/sys_gen/tst_rlink/s3board/tb

     make tb_tst_rlink_s3

     time ti_rri --run="tbw tb_tst_rlink_s3" --fifo --logl=3 -- \

       "package require tst_rlink" "tst_rlink::setup" "tst_rlink::test_all" |\

       tee tb_tst_rlink_s3_dsim.log | egrep "(-[EFW]:|FAIL|PEND|DONE)"

     -> 1822195.0 ns  91100: DONE

     -> real 0m13.281s

   - sys_tst_rlink_n2 test bench

     cd $RETROBASE/rtl/sys_gen/tst_rlink/nexys2/tb

     make tb_tst_rlink_n2

     time ti_rri --run="tbw tb_tst_rlink_n2" --fifo --logl=3 -- \

       "package require tst_rlink" "tst_rlink::setup" "tst_rlink::test_all" |\

       tee tb_tst_rlink_n2_dsim.log | egrep "(-[EFW]:|FAIL|PEND|DONE)"

     -> 1769140.0 ns  88446: DONE

     -> real 0m15.289s

   - sys_tst_rlink_n3 test bench

     cd $RETROBASE/rtl/sys_gen/tst_rlink/nexys3/tb

     make tb_tst_rlink_n3

     time ti_rri --run="tbw tb_tst_rlink_n3" --fifo --logl=3 -- \

       "package require tst_rlink" "tst_rlink::setup" "tst_rlink::test_all" |\

       tee tb_tst_rlink_n3_dsim.log | egrep "(-[EFW]:|FAIL|PEND|DONE)"

     -> 893590.0 ns  89338: DONE

     -> real 0m9.510s

4c. w11a systems -----------------------------------------------------

   The stimulus file used in the w11a core test can be executed in the

   full system context (both s3board and nexys2 versions) with the

   following commands. Note that the cycle number printed in the DONE

   line can now vary slightly because the response time of the rlink

   backend process and thus scheduling of backend vs. ghdl process

   can affect the result.

   - sys_w11a_s3 test bench

     cd $RETROBASE/rtl/sys_gen/w11a/s3board/tb

     make tb_w11a_s3

     time pi_rri --fifo --timeout=40. --cmax=3 \

        --run="tbw tb_w11a_s3" -- \

         @../../../../w11a/tb/tb_pdp11core_stim.dat |\

       tee tb_w11a_s3_stim2_dsim.log | egrep "(-[EFW]:|FAIL|PEND|DONE)"

     -> 7852095.0 ns 392595: DONE

     -> real 0m49.835s   user 0m50.203s   sys 0m00.696s

   - sys_w11a_n2 test bench

     cd $RETROBASE/rtl/sys_gen/w11a/nexys2/tb

     make tb_w11a_n2

     time pi_rri --fifo --timeout=40. --cmax=3 \

        --run="tbw tb_w11a_n2" -- \

         @../../../../w11a/tb/tb_pdp11core_stim.dat |\

       tee tb_w11a_n2_stim2_dsim.log | egrep "(-[EFW]:|FAIL|PEND|DONE)"

     -> 7836580.0 ns 391818: DONE

     -> real 1m0.854s   user 1m1.332s   sys 0m0.800s

   - sys_w11a_n3 test bench

     cd $RETROBASE/rtl/sys_gen/w11a/nexys3/tb

     make tb_w11a_n3

     time pi_rri --fifo --timeout=40. --cmax=3 \

        --run="tbw tb_w11a_n3" -- \

         @../../../../w11a/tb/tb_pdp11core_stim.dat |\

       tee tb_w11a_n3_stim2_dsim.log | egrep "(-[EFW]:|FAIL|PEND|DONE)"

     -> 3956540.0 ns 395633: DONE

     -> real 1m13.811s   user 1m14.389s   sys 0m0.948s