Subversion Repositories w11

# $Id: w11a_tb_guide.txt 779 2016-06-26 15:37:16Z mueller $

Note: - Ghdl is used for all behavioral simulations
      - Optionally Vivado xsim can be used
      - For post synthesis or post implementation functionnal simulations 
        either Ghdl or Vivado xsim can be used.
      - For timing simulations only Vivado xsim can be used.
      - ISE isim is also available, but considered legacy support

Guide to running w11a test benches

  Table of content:
  
  1. Unit tests benches
  2. Available unit tests benches
  3. System tests benches
  4. Available system tests benches


1. Unit tests benches -----------------------------------------------------

   All unit test benches have the same simple structure:

   - a stimulus process reads test patterns as well as the expected
     responses from a stimulus file

   - the responses are checked in very simple cases by the stimulus process,
     in general by a monitoring process

   - the test bench produces a comprehensive log file. For each checked
     response the line contains the word "CHECK" and either an "OK" or a
     "FAIL", in the later case in general with an indication of whats wrong.
     Other unexpected behaviour, like timeouts, will also result in a line
     containing the word "FAIL".

   - at the end a line with the word "DONE" is printed.

   - the test bench is run like

       tbw <testbenchname> [stimfile] | tee <logfile> | egrep "(FAIL|DONE)"

     where 
       - 'tbw' is a small perl script setting up a symbolic link to the
         stimulus file, the default extracted from the file tbw.dat, if
         an optional file name is give this one will be used instead.
       - 'tee' ensures that the full log is saved
       - 'egrep' filters FAIL and DONE lines, a successful run will
         produce a single DONE line

   - Most tests can be run against 
       - the behavioral model
       - post-synthesis    functional
       - post-optimization functional
       - post-routing      functional
       - post-synthesis    timing
       - post-optimization timing
       - post-routing      timing

     Building the simulation models is handled by the build environment. See 
     README_buildsystem_Vivado.txt for details of the vivado flow and 
     README_buildsystem_ISE.txt for the ISE flow.

     An example of a post-synthesis model is given for the w11a core test.

   - for convenience a wrapper script 'tbrun_tbw' is used to generate the 
     tbw|tee|egrep pipe. This script also checks with 'make' whether the
     test bench is up-to-date or must be (re)-compiled.

2. Available unit tests benches -------------------------------------------

   In the following the available tests are listed with their tbrun_tbw which
     - will call 'make' to build them
     - and create the pipe setup to run them
   and the expected output (the run time measured on a 3 GHz system)

   - serport receiver test
     cd $RETROBASE/rtl/vlib/serport/tb
     tbrun_tbw tb_serport_uart_rx 
     -> 1269955.0 ns  63488: DONE 
     -> real 0m0.531s   user 0m0.392s   sys 0m0.014s

   - serport receiver/transmitter test
     tbrun_tbw tb_serport_uart_rxtx
     -> 52335.0 ns   2607: DONE 
     -> real 0m0.120s   user 0m0.065s   sys 0m0.013s

   - serport autobauder test
     tbrun_tbw tb_serport_autobaud
     -> 367475.0 ns  18364: DONE 
     -> real 0m0.343s   user 0m0.316s   sys 0m0.003s

 
   - 9 bit comma,data to Byte stream converter test
     cd $RETROBASE/rtl/vlib/comlib/tb
     tbrun_tbw tb_cdata2byte
     -> 7261.0 ns    354: DONE 
     -> real 0m0.088s   user 0m0.057s   sys 0m0.013s

   - rlink core test

     cd $RETROBASE/rtl/vlib/rlink/tb
     tbrun_tbw tb_rlink_direct
     -> 78975.0 ns   3939: DONE 
     -> real 0m0.270s   user 0m0.222s   sys 0m0.026s

   - rlink core test via serial port interface

     cd $RETROBASE/rtl/vlib/rlink/tb
     tbrun_tbw --lsuf stim2_bsim   tb_rlink_sp1c tb_rlink_sp1c_stim.dat
     -> 27595.0 ns   1370: DONE 
     -> real 0m0.184s   user 0m0.145s   sys 0m0.011s

     tbrun_tbw --lsuf stim1_bsim   tb_rlink_sp1c tb_rlink_stim.dat
     -> 420295.0 ns  21005: DONE 
     -> real 0m0.939s   user 0m0.945s   sys 0m0.026s

   - w11a core test 
     - using behavioral model

       cd $RETROBASE/rtl/w11a/tb
       tbrun_tbw tb_pdp11core
       -> 225355.0 ns  61258: DONE 
       -> real 0m6.446s   user 0m6.387s   sys 0m0.024s

     - using Vivado post-synthesis vhdl model and ghdl

       tbrun_tbw tb_pdp11core_ssim
       ->  1225355.0 ns  61258: DONE 
       -> real 1m40.446s   user 1m40.344s   sys 0m0.075s

     - using Vivado post-synthesis verilog model and xsim

       tbrun_tbw tb_pdp11core_XSim_ssim
       -> 1225355.0 ns  61258: DONE 
       -> real 1m14.835s   user 1m13.997s   sys 0m1.011s

   - s3board sram controller test 

     cd $RETROBASE/rtl/bplib/s3board/tb
     tbrun_tbw tb_s3_sram_memctl
     -> 5015.0 ns    241: DONE 
     -> real 0m0.075s   user 0m0.045s   sys 0m0.022s

   - nexys2/nexys3 cram controller test 

     cd $RETROBASE/rtl/bplib/nxcramlib/tb
     tbrun_tbw tb_nx_cram_memctl_as
     -> 24272.5 ns   1204: DONE 
     -> real 0m0.337s   user 0m0.147s   sys 0m0.146s

3. System tests benches ---------------------------------------------------

   The system tests allow to verify to verify a full system design.
   In this case vhdl test bench code contains
     - (simple) models of the memories used on the FPGA boards
     - drivers for the rlink connection (currently just serialport)
     - code to interface the rlink data stream to a UNIX 'named pipe',
       implemented with a C routine which is called via VHPI from VHDL.
   This way the whole ghdl simulation can be controlled via a di-directional
   byte stream. 

   The rlink backend process can connect either via a named pipe to a ghdl 
   simulation, or via a serial port to a FPGA board. This way the same tests 
   can be executed in simulation and on real hardware.

4. Available system tests benches -----------------------------------------

4a. serport tester ---------------------------------------------------

   The sys_tst_serloop design is a test target for validating the serial
   link UART stack. Send and receive throughput as well as loop-back tests
   are supported

   - sys_tst_serloop_s3 test bench

     cd $RETROBASE/rtl/sys_gen/tst_serloop/s3board/tb
     tbrun_tbw tb_tst_serloop_s3
     -> 301353.3 ns  18068: DONE 
     -> real 0m0.832s   user 0m0.765s   sys 0m0.036s

   - sys_tst_serloop_n2 test bench

     cd $RETROBASE/rtl/sys_gen/tst_serloop/nexys2/tb
     tbrun_tbw tb_tst_serloop1_n2
     -> 361560.0 ns  18068: DONE
     -> real 0m0.799s   user 0m0.758s   sys 0m0.021s

     tbrun_tbw tb_tst_serloop2_n2
     -> 304353.3 ns  18248: DONE
     -> real 0m1.274s   user 0m1.236s   sys 0m0.017s

   - sys_tst_serloop_n3 test bench

     cd $RETROBASE/rtl/sys_gen/tst_serloop/nexys3/tb
     tbrun_tbw tb_tst_serloop1_n3
     -> 361560.0 ns  18068: DONE
     -> real 0m0.841s   user 0m0.820s   sys 0m0.014s

4b. rlink tester -----------------------------------------------------

   The sys_tst_rlink design is a test target for validating the rlink and 
   rbus functionality at all levels.

   - Artix based systems

     - sys_tst_rlink_arty test bench
  
       cd $RETROBASE/rtl/sys_gen/tst_rlink/arty/tb
       tbrun_tbwrri --hxon --pack tst_rlink   tb_tst_rlink_arty \
           "tst_rlink::setup"  "tst_rlink::test_all"
       -> 1028590.0 ns 102838: DONE 
       -> real 0m14.163s   user 0m12.637s   sys 0m0.152s

     - sys_tst_rlink_b3 test bench
  
       cd $RETROBASE/rtl/sys_gen/tst_rlink/basys3/tb
       tbrun_tbwrri --hxon --pack tst_rlink   tb_tst_rlink_b3 \
           "tst_rlink::setup"  "tst_rlink::test_all"
       -> 1028820.0 ns 102861: DONE 
       -> real 0m9.275s   user 0m9.041s   sys 0m0.094s

     - sys_tst_rlink_n4 test bench
  
       cd $RETROBASE/rtl/sys_gen/tst_rlink/nexys4/tb
       tbrun_tbwrri --pack tst_rlink   tb_tst_rlink_n4 \
           "tst_rlink::setup"  "tst_rlink::test_all"
       -> 1020240.0 ns 102003: DONE 
       -> real 0m9.751s   user 0m9.544s   sys 0m0.081s


   - Spartan based systems

     - sys_tst_rlink_n3 test bench
  
       cd $RETROBASE/rtl/sys_gen/tst_rlink/nexys3/tb
       tbrun_tbwrri --fusp --pack tst_rlink   tb_tst_rlink_n3 \
           "tst_rlink::setup"  "tst_rlink::test_all"
       -> 1024980.0 ns 102477: DONE 
       -> real 0m8.081s   user 0m7.904s   sys 0m0.106s

     - sys_tst_rlink_n2 test bench
  
       cd $RETROBASE/rtl/sys_gen/tst_rlink/nexys2/tb
       tbrun_tbwrri --fusp --pack tst_rlink   tb_tst_rlink_n2 \
           "tst_rlink::setup"  "tst_rlink::test_all"
       -> 2049320.0 ns 102455: DONE 
       -> real 0m7.934s   user 0m7.748s   sys 0m0.114s

     - sys_tst_rlink_s3 test bench
  
       cd $RETROBASE/rtl/sys_gen/tst_rlink/s3board/tb
       tbrun_tbwrri --fusp --pack tst_rlink   tb_tst_rlink_s3 \
           "tst_rlink::setup"  "tst_rlink::test_all" 
       -> 2049720.0 ns 102476: DONE 
       -> real 0m7.612s   user 0m7.437s   sys 0m0.075s

4c. rlink tester, Cypress FX2 based version --------------------------

   The sys_tst_rlink_cuff design is a test target for validating the rlink and 
   rbus functionality at all levels over the Cypress FX2 USB interface which
   is provided by the Nexys2 abd Nexys3 boards.

   - sys_tst_rlink_cuff_ic_n3 test bench

     cd $RETROBASE/rtl/sys_gen/tst_rlink_cuff/nexys3/ic/tb
     tbrun_tbwrri --cuff --pack tst_rlink   tb_tst_rlink_cuff_ic_n3 \
         "tst_rlink::setup"  "tst_rlink::test_all"
     -> 558770.0 ns  55856: DONE 
     -> real 0m7.679s   user 0m7.433s   sys 0m0.185s

   - sys_tst_rlink_cuff_ic_n2 test bench

     cd $RETROBASE/rtl/sys_gen/tst_rlink_cuff/nexys2/ic/tb
     tbrun_tbwrri --cuff --pack tst_rlink   tb_tst_rlink_cuff_ic_n2 \
         "tst_rlink::setup"  "tst_rlink::test_all"
     -> 596300.0 ns  29804: DONE 
     -> real 0m3.741s   user 0m3.542s   sys 0m0.127s

4d. w11a systems -----------------------------------------------------

   The stimulus file used in the w11a core test can be executed in the full
   system context with the following commands. Note that the cycle number 
   printed in the DONE line can now vary slightly because the response time of
   the rlink backend process and thus scheduling of backend vs. ghdl process
   can affect the result.

   For convenience a wrapper script 'tbrun_tbwrri' is used to generate the 
   required quite long ti_rri command. Like for 'tbrun_tbw' the script also
   checks with 'make' whether the test bench is up-to-date or must be 
   (re)-compiled.

   - Artix based systems

       cd $RETROBASE/rtl/sys_gen/w11a/nexys4/tb
       tbrun_tbwrri --pack rw11  tb_w11a_n4 \
           "rw11::setup_cpu" \
           "rw11::run_pdpcp ../../../../w11a/tb/tb_pdp11core_stim.dat"
       -> 4812818.3 ns 577513: DONE 
       -> real 1m11.139s   user 1m10.726s   sys 0m0.545s

   - Spartan based systems
     - sys_w11a_n3 test bench
  
       cd $RETROBASE/rtl/sys_gen/w11a/nexys3/tb
       tbrun_tbwrri --cuff --pack rw11  tb_w11a_n3 \
           "rw11::setup_cpu" \
           "rw11::run_pdpcp ../../../../w11a/tb/tb_pdp11core_stim.dat"
       -> 3612428.7 ns 231182: DONE 
       -> real 0m47.454s   user 0m47.241s   sys 0m0.456s

     - sys_w11a_n2 test bench
  
       cd $RETROBASE/rtl/sys_gen/w11a/nexys2/tb
       tbrun_tbwrri --cuff --pack rw11  tb_w11a_n2 \
           "rw11::setup_cpu" \
           "rw11::run_pdpcp ../../../../w11a/tb/tb_pdp11core_stim.dat"
       -> 4009900.0 ns 200484: DONE 
       -> real 0m45.429s   user 0m45.215s   sys 0m0.480s

     - sys_w11a_s3 test bench
  
       cd $RETROBASE/rtl/sys_gen/w11a/s3board/tb
       tbrun_tbwrri --fusp --pack rw11  tb_w11a_s3 \
           "rw11::setup_cpu" \
           "rw11::run_pdpcp ../../../../w11a/tb/tb_pdp11core_stim.dat"
       -> 10528880.0 ns 526434: DONE 
       -> real 1m13.706s   user 1m13.483s   sys 0m0.470s


   A new, modular w11a test bench is under construction. So far it is very
   incomplete. This very preliminary version can be executed with

   - sys_w11a_n2 test bench

     cd $RETROBASE/rtl/sys_gen/w11a/nexys2/tb
     tbrun_tbwrri  --cuff --lsuf tbench_bsim --pack rw11  tb_w11a_n2 \
        "rw11::setup_cpu" "rw11::tbench @cpu_all.dat"
     -> 3268940.0 ns 163436: DONE 
     -> real 0m30.761s   user 0m31.576s   sys 0m0.502s

     tbrun_tbwrri --cuff --lsuf tbench_bsim --pack rw11  tb_w11a_n2 \
        "rw11::setup_cpu" "rw11::tbench @dev_all.dat"
     -> 1376360.0 ns  68807: DONE 
     -> real 0m16.991s   user 0m17.049s   sys 0m0.235s