Subversion Repositories w11

# $Id: w11a_tb_guide.txt 810 2016-10-02 16:51:12Z mueller $

Note: - Ghdl is used for all behavioral simulations
      - Optionally Vivado xsim can be used
      - For post synthesis or post implementation functionnal simulations 
        either Ghdl or Vivado xsim can be used.
      - For timing simulations only Vivado xsim can be used.
      - ISE isim is also available, but considered legacy support

Guide to running test benches

  Table of content:
  
  1. Tests bench environment
  2. Unit test benches
  3. System test benches
  4. Test bench driver
  5. Execute all available tests
  6. Available unit tests benches
  7. Available system tests benches


1. Tests bench environment ------------------------------------------------

   All test benches have the same simple structure:

   - the test benches are 'self-checking'. For unit tests a stimulus process 
     reads test patterns as well as the expected responses from a stimulus file

   - the responses are checked in very simple cases by the stimulus process,
     in general by a monitoring process

   - the test bench produces a comprehensive log file. For each checked
     response the line contains the word "CHECK" and either an "OK" or a
     "FAIL", in the later case in general with an indication of whats wrong.
     Other unexpected behaviour, like timeouts, will also result in a line
     containing the word "FAIL".

   - at the end a line with the word "DONE" is printed.

   - Most tests can be run as 
       - bsim: the behavioral model
       - ssim: post-synthesis    functional
       - osim: post-optimization functional
       - rsim: post-routing      functional
       - esim: post-synthesis    timing
       - psim: post-optimization timing
       - tsim: post-routing      timing

     Building the simulation models is handled by the build environment. See 
     README_buildsystem_Vivado.txt for details of the vivado flow and 
     README_buildsystem_ISE.txt for the ISE flow.

2. Unit test benches ------------------------------------------------------

   All unit test are executed via 'tbw' (test bench warpper) script.

   - the test bench is run like

       tbw <testbenchname> [stimfile] | tbfilt --tee <logfile>

     where 
       - tbw sets up the environment of the test bench and starts it.
         It generates required symbolic links, e.g. to the stimulus file,
         the defaults extracted from the file tbw.dat, if an optional file
         name is give this one will be used instead.
       - tbfilt saves the full test bench output to a logfile and filters
         the output for PASS/FAIL criteria

   - for convenience a wrapper script 'tbrun_tbw' is used to generate the 
     tbw|tbfilt pipe. This script also checks with 'make' whether the
     test bench is up-to-date or must be (re)-compiled.

3. System test benches ----------------------------------------------------

   The system tests allow to verify to verify a full system design.
   In this case vhdl test bench code contains
     - (simple) models of the memories used on the FPGA boards
     - drivers for the rlink connection (currently just serialport)
     - code to interface the rlink data stream to a UNIX 'named pipe',
       implemented with a C routine which is called via VHPI from VHDL.
   This way the whole ghdl simulation can be controlled via a di-directional
   byte stream. 

   The rlink backend process can connect either via a named pipe to a ghdl 
   simulation, or via a serial port to a FPGA board. This way the same tests 
   can be executed in simulation and on real hardware.

   In general the script 'tbrun_tbwrri' is used to generate the quite lengthy 
   ommand to properly setup the tbw|tbfilt pipe.  This script also checks 
   with 'make' whether the test bench is up-to-date or must be (re)-compiled.

4. Test bench driver ------------------------------------------------------

   All available tests (unit and system test benches) are described in a
   set of descriptor files, usually called 'tbrun.yml'. The top level file
   in $RETROBASE includes other descriptor files located in the source 
   directories of the tests.

   The script 'tbrun' reads these descriptor files, selects tests based
   on --tag and --exclude options, and executes the tests with the
   simulation engine and simulation type given by the --mode option.
   For full description of see 'man tbrun'.

   The low level drivers 'tbrun_tbw' and 'tbrun_tbwrri' will automatically 
   build the model if it is not available or outdated. This is very convenient
   when working with a single test bench during development.

   When executing a large number of them it's in general better to separate
   the model building (make phase) made model execution (run phase). Both
   the low level drivers as well as 'tbrun' support this via the options
   --nomake and --norun.

   The individial test benches are simplest started via tbrun and a proper
   selection via --tag. Very helpful is

     cd $RETROBASE
     tbrun --dry --tag=.*

   which gives a listing of all available test. The tag list as well as
   the shell commands to execute the test are shown.

5. Execute all available tests --------------------------------------------

   As stated above it is in general better to to separate the model building 
   (make phase) made model execution (run phase). The currently recommended
   way to execute all test benches is given below.
   The run time is measured on a 3 GHz dual core system.

     cd $RETROBASE
     # build all behavioral models
     #   first all with ISE work flow
     time nice tbrun -j 2 -norun -tag=ise -tee=tbrun_make_ise_bsim.log
       # --> real 3m41.732s   user 6m3.381s   sys 0m24.224s

     #   than all with vivado work flow
     time nice tbrun -j 2 -norun -tag=viv -tee=tbrun_make_viv_bsim.log
       # --> real 3m36.532s   user 5m58.319s   sys 0m25.235s
     
     # than execute all behavioral models
     time nice tbrun -j 2 -nomake -tag=ise -tee=tbrun_run_ise_bsim.log
       # --> real 3m19.799s   user 5m45.060s   sys 0m6.625s
     time nice tbrun -j 2 -nomake -tag=viv -tee=tbrun_run_viv_bsim.log
       #--> real 3m49.193s   user 5m44.063s   sys 0m5.332s

   All test create an individual logfile. 'tbfilt' can be used to scan
   these logfiles and create a summary with

     tbfilt -all -sum -comp
   
   It should look like
     76m   0m00.034s c    0.92u   0 PASS tb_is61lv25616al_bsim.log
     76m   0m00.153s c    4.00u   0 PASS tb_mt45w8mw16b_bsim.log
     76m   0m00.168s c     1146   0 PASS tb_nx_cram_memctl_as_bsim.log
     ...
     ...
     76m   0m03.729s c    61258   0 PASS tb_pdp11core_bsim_base.log
     76m   0m00.083s c     1121   0 PASS tb_pdp11core_bsim_ubmap.log
     76m   0m00.068s c     1031   0 PASS tb_rlink_tba_pdp11core_bsim_ibdr.log

6. Available unit tests benches -------------------------------------------

     tbrun --tag=comlib                    # comlib unit tests
     tbrun --tag=serport                   # serport unit tests
     tbrun --tag=rlink                     # rlink unit tests
     tbrun --tag=issi                      # SRAM model unit tests
     tbrun --tag=micron                    # CRAM model unit tests
     tbrun --tag=sram_memctl               # SRAM controller unit tests
     tbrun --tag=cram_memctl               # CRAM controller unit tests
     tbrun --tag=w11a                      # w11a unit tests

7. Available system tests benches -----------------------------------------

     tbrun --tag=sys_tst_serloop.*         # all sys_tst_serloop designs
     tbrun --tag=sys_tst_rlink             # all sys_tst_rlink designs
     tbrun --tag=sys_tst_rlink_cuff        # all sys_tst_rlink_cuff designs
     tbrun --tag=sys_tst_sram              # all sys_tst_sram designs
     tbrun --tag=sys_w11a                  # all w11a designs