# $Id: w11a_tb_guide.txt 810 2016-10-02 16:51:12Z mueller $
|
# $Id: w11a_tb_guide.txt 810 2016-10-02 16:51:12Z mueller $
|
|
|
Note: - Ghdl is used for all behavioral simulations
|
Note: - Ghdl is used for all behavioral simulations
|
- Optionally Vivado xsim can be used
|
- Optionally Vivado xsim can be used
|
- For post synthesis or post implementation functionnal simulations
|
- For post synthesis or post implementation functionnal simulations
|
either Ghdl or Vivado xsim can be used.
|
either Ghdl or Vivado xsim can be used.
|
- For timing simulations only Vivado xsim can be used.
|
- For timing simulations only Vivado xsim can be used.
|
- ISE isim is also available, but considered legacy support
|
- ISE isim is also available, but considered legacy support
|
|
|
Guide to running test benches
|
Guide to running test benches
|
|
|
Table of content:
|
Table of content:
|
|
|
1. Tests bench environment
|
1. Tests bench environment
|
2. Unit test benches
|
2. Unit test benches
|
3. System test benches
|
3. System test benches
|
4. Test bench driver
|
4. Test bench driver
|
5. Execute all available tests
|
5. Execute all available tests
|
6. Available unit tests benches
|
6. Available unit tests benches
|
7. Available system tests benches
|
7. Available system tests benches
|
|
|
|
|
1. Tests bench environment ------------------------------------------------
|
1. Tests bench environment ------------------------------------------------
|
|
|
All test benches have the same simple structure:
|
All test benches have the same simple structure:
|
|
|
- the test benches are 'self-checking'. For unit tests a stimulus process
|
- the test benches are 'self-checking'. For unit tests a stimulus process
|
reads test patterns as well as the expected responses from a stimulus file
|
reads test patterns as well as the expected responses from a stimulus file
|
|
|
- the responses are checked in very simple cases by the stimulus process,
|
- the responses are checked in very simple cases by the stimulus process,
|
in general by a monitoring process
|
in general by a monitoring process
|
|
|
- the test bench produces a comprehensive log file. For each checked
|
- the test bench produces a comprehensive log file. For each checked
|
response the line contains the word "CHECK" and either an "OK" or a
|
response the line contains the word "CHECK" and either an "OK" or a
|
"FAIL", in the later case in general with an indication of whats wrong.
|
"FAIL", in the later case in general with an indication of whats wrong.
|
Other unexpected behaviour, like timeouts, will also result in a line
|
Other unexpected behaviour, like timeouts, will also result in a line
|
containing the word "FAIL".
|
containing the word "FAIL".
|
|
|
- at the end a line with the word "DONE" is printed.
|
- at the end a line with the word "DONE" is printed.
|
|
|
- Most tests can be run as
|
- Most tests can be run as
|
- bsim: the behavioral model
|
- bsim: the behavioral model
|
- ssim: post-synthesis functional
|
- ssim: post-synthesis functional
|
- osim: post-optimization functional
|
- osim: post-optimization functional
|
- rsim: post-routing functional
|
- rsim: post-routing functional
|
- esim: post-synthesis timing
|
- esim: post-synthesis timing
|
- psim: post-optimization timing
|
- psim: post-optimization timing
|
- tsim: post-routing timing
|
- tsim: post-routing timing
|
|
|
Building the simulation models is handled by the build environment. See
|
Building the simulation models is handled by the build environment. See
|
README_buildsystem_Vivado.txt for details of the vivado flow and
|
README_buildsystem_Vivado.txt for details of the vivado flow and
|
README_buildsystem_ISE.txt for the ISE flow.
|
README_buildsystem_ISE.txt for the ISE flow.
|
|
|
2. Unit test benches ------------------------------------------------------
|
2. Unit test benches ------------------------------------------------------
|
|
|
All unit test are executed via 'tbw' (test bench warpper) script.
|
All unit test are executed via 'tbw' (test bench warpper) script.
|
|
|
- the test bench is run like
|
- the test bench is run like
|
|
|
tbw [stimfile] | tbfilt --tee
|
tbw [stimfile] | tbfilt --tee
|
|
|
where
|
where
|
- tbw sets up the environment of the test bench and starts it.
|
- tbw sets up the environment of the test bench and starts it.
|
It generates required symbolic links, e.g. to the stimulus file,
|
It generates required symbolic links, e.g. to the stimulus file,
|
the defaults extracted from the file tbw.dat, if an optional file
|
the defaults extracted from the file tbw.dat, if an optional file
|
name is give this one will be used instead.
|
name is give this one will be used instead.
|
- tbfilt saves the full test bench output to a logfile and filters
|
- tbfilt saves the full test bench output to a logfile and filters
|
the output for PASS/FAIL criteria
|
the output for PASS/FAIL criteria
|
|
|
- for convenience a wrapper script 'tbrun_tbw' is used to generate the
|
- for convenience a wrapper script 'tbrun_tbw' is used to generate the
|
tbw|tbfilt pipe. This script also checks with 'make' whether the
|
tbw|tbfilt pipe. This script also checks with 'make' whether the
|
test bench is up-to-date or must be (re)-compiled.
|
test bench is up-to-date or must be (re)-compiled.
|
|
|
3. System test benches ----------------------------------------------------
|
3. System test benches ----------------------------------------------------
|
|
|
The system tests allow to verify to verify a full system design.
|
The system tests allow to verify to verify a full system design.
|
In this case vhdl test bench code contains
|
In this case vhdl test bench code contains
|
- (simple) models of the memories used on the FPGA boards
|
- (simple) models of the memories used on the FPGA boards
|
- drivers for the rlink connection (currently just serialport)
|
- drivers for the rlink connection (currently just serialport)
|
- code to interface the rlink data stream to a UNIX 'named pipe',
|
- code to interface the rlink data stream to a UNIX 'named pipe',
|
implemented with a C routine which is called via VHPI from VHDL.
|
implemented with a C routine which is called via VHPI from VHDL.
|
This way the whole ghdl simulation can be controlled via a di-directional
|
This way the whole ghdl simulation can be controlled via a di-directional
|
byte stream.
|
byte stream.
|
|
|
The rlink backend process can connect either via a named pipe to a ghdl
|
The rlink backend process can connect either via a named pipe to a ghdl
|
simulation, or via a serial port to a FPGA board. This way the same tests
|
simulation, or via a serial port to a FPGA board. This way the same tests
|
can be executed in simulation and on real hardware.
|
can be executed in simulation and on real hardware.
|
|
|
In general the script 'tbrun_tbwrri' is used to generate the quite lengthy
|
In general the script 'tbrun_tbwrri' is used to generate the quite lengthy
|
ommand to properly setup the tbw|tbfilt pipe. This script also checks
|
ommand to properly setup the tbw|tbfilt pipe. This script also checks
|
with 'make' whether the test bench is up-to-date or must be (re)-compiled.
|
with 'make' whether the test bench is up-to-date or must be (re)-compiled.
|
|
|
4. Test bench driver ------------------------------------------------------
|
4. Test bench driver ------------------------------------------------------
|
|
|
All available tests (unit and system test benches) are described in a
|
All available tests (unit and system test benches) are described in a
|
set of descriptor files, usually called 'tbrun.yml'. The top level file
|
set of descriptor files, usually called 'tbrun.yml'. The top level file
|
in $RETROBASE includes other descriptor files located in the source
|
in $RETROBASE includes other descriptor files located in the source
|
directories of the tests.
|
directories of the tests.
|
|
|
The script 'tbrun' reads these descriptor files, selects tests based
|
The script 'tbrun' reads these descriptor files, selects tests based
|
on --tag and --exclude options, and executes the tests with the
|
on --tag and --exclude options, and executes the tests with the
|
simulation engine and simulation type given by the --mode option.
|
simulation engine and simulation type given by the --mode option.
|
For full description of see 'man tbrun'.
|
For full description of see 'man tbrun'.
|
|
|
The low level drivers 'tbrun_tbw' and 'tbrun_tbwrri' will automatically
|
The low level drivers 'tbrun_tbw' and 'tbrun_tbwrri' will automatically
|
build the model if it is not available or outdated. This is very convenient
|
build the model if it is not available or outdated. This is very convenient
|
when working with a single test bench during development.
|
when working with a single test bench during development.
|
|
|
When executing a large number of them it's in general better to separate
|
When executing a large number of them it's in general better to separate
|
the model building (make phase) made model execution (run phase). Both
|
the model building (make phase) made model execution (run phase). Both
|
the low level drivers as well as 'tbrun' support this via the options
|
the low level drivers as well as 'tbrun' support this via the options
|
--nomake and --norun.
|
--nomake and --norun.
|
|
|
The individial test benches are simplest started via tbrun and a proper
|
The individial test benches are simplest started via tbrun and a proper
|
selection via --tag. Very helpful is
|
selection via --tag. Very helpful is
|
|
|
cd $RETROBASE
|
cd $RETROBASE
|
tbrun --dry --tag=.*
|
tbrun --dry --tag=.*
|
|
|
which gives a listing of all available test. The tag list as well as
|
which gives a listing of all available test. The tag list as well as
|
the shell commands to execute the test are shown.
|
the shell commands to execute the test are shown.
|
|
|
5. Execute all available tests --------------------------------------------
|
5. Execute all available tests --------------------------------------------
|
|
|
As stated above it is in general better to to separate the model building
|
As stated above it is in general better to to separate the model building
|
(make phase) made model execution (run phase). The currently recommended
|
(make phase) made model execution (run phase). The currently recommended
|
way to execute all test benches is given below.
|
way to execute all test benches is given below.
|
The run time is measured on a 3 GHz dual core system.
|
The run time is measured on a 3 GHz dual core system.
|
|
|
cd $RETROBASE
|
cd $RETROBASE
|
# build all behavioral models
|
# build all behavioral models
|
# first all with ISE work flow
|
# first all with ISE work flow
|
time nice tbrun -j 2 -norun -tag=ise -tee=tbrun_make_ise_bsim.log
|
time nice tbrun -j 2 -norun -tag=ise -tee=tbrun_make_ise_bsim.log
|
# --> real 3m41.732s user 6m3.381s sys 0m24.224s
|
# --> real 3m41.732s user 6m3.381s sys 0m24.224s
|
|
|
# than all with vivado work flow
|
# than all with vivado work flow
|
time nice tbrun -j 2 -norun -tag=viv -tee=tbrun_make_viv_bsim.log
|
time nice tbrun -j 2 -norun -tag=viv -tee=tbrun_make_viv_bsim.log
|
# --> real 3m36.532s user 5m58.319s sys 0m25.235s
|
# --> real 3m36.532s user 5m58.319s sys 0m25.235s
|
|
|
# than execute all behavioral models
|
# than execute all behavioral models
|
time nice tbrun -j 2 -nomake -tag=ise -tee=tbrun_run_ise_bsim.log
|
time nice tbrun -j 2 -nomake -tag=ise -tee=tbrun_run_ise_bsim.log
|
# --> real 3m19.799s user 5m45.060s sys 0m6.625s
|
# --> real 3m19.799s user 5m45.060s sys 0m6.625s
|
time nice tbrun -j 2 -nomake -tag=viv -tee=tbrun_run_viv_bsim.log
|
time nice tbrun -j 2 -nomake -tag=viv -tee=tbrun_run_viv_bsim.log
|
#--> real 3m49.193s user 5m44.063s sys 0m5.332s
|
#--> real 3m49.193s user 5m44.063s sys 0m5.332s
|
|
|
All test create an individual logfile. 'tbfilt' can be used to scan
|
All test create an individual logfile. 'tbfilt' can be used to scan
|
these logfiles and create a summary with
|
these logfiles and create a summary with
|
|
|
tbfilt -all -sum -comp
|
tbfilt -all -sum -comp
|
|
|
It should look like
|
It should look like
|
76m 0m00.034s c 0.92u 0 PASS tb_is61lv25616al_bsim.log
|
76m 0m00.034s c 0.92u 0 PASS tb_is61lv25616al_bsim.log
|
76m 0m00.153s c 4.00u 0 PASS tb_mt45w8mw16b_bsim.log
|
76m 0m00.153s c 4.00u 0 PASS tb_mt45w8mw16b_bsim.log
|
76m 0m00.168s c 1146 0 PASS tb_nx_cram_memctl_as_bsim.log
|
76m 0m00.168s c 1146 0 PASS tb_nx_cram_memctl_as_bsim.log
|
...
|
...
|
...
|
...
|
76m 0m03.729s c 61258 0 PASS tb_pdp11core_bsim_base.log
|
76m 0m03.729s c 61258 0 PASS tb_pdp11core_bsim_base.log
|
76m 0m00.083s c 1121 0 PASS tb_pdp11core_bsim_ubmap.log
|
76m 0m00.083s c 1121 0 PASS tb_pdp11core_bsim_ubmap.log
|
76m 0m00.068s c 1031 0 PASS tb_rlink_tba_pdp11core_bsim_ibdr.log
|
76m 0m00.068s c 1031 0 PASS tb_rlink_tba_pdp11core_bsim_ibdr.log
|
|
|
6. Available unit tests benches -------------------------------------------
|
6. Available unit tests benches -------------------------------------------
|
|
|
tbrun --tag=comlib # comlib unit tests
|
tbrun --tag=comlib # comlib unit tests
|
tbrun --tag=serport # serport unit tests
|
tbrun --tag=serport # serport unit tests
|
tbrun --tag=rlink # rlink unit tests
|
tbrun --tag=rlink # rlink unit tests
|
tbrun --tag=issi # SRAM model unit tests
|
tbrun --tag=issi # SRAM model unit tests
|
tbrun --tag=micron # CRAM model unit tests
|
tbrun --tag=micron # CRAM model unit tests
|
tbrun --tag=sram_memctl # SRAM controller unit tests
|
tbrun --tag=sram_memctl # SRAM controller unit tests
|
tbrun --tag=cram_memctl # CRAM controller unit tests
|
tbrun --tag=cram_memctl # CRAM controller unit tests
|
tbrun --tag=w11a # w11a unit tests
|
tbrun --tag=w11a # w11a unit tests
|
|
|
7. Available system tests benches -----------------------------------------
|
7. Available system tests benches -----------------------------------------
|
|
|
tbrun --tag=sys_tst_serloop.* # all sys_tst_serloop designs
|
tbrun --tag=sys_tst_serloop.* # all sys_tst_serloop designs
|
tbrun --tag=sys_tst_rlink # all sys_tst_rlink designs
|
tbrun --tag=sys_tst_rlink # all sys_tst_rlink designs
|
tbrun --tag=sys_tst_rlink_cuff # all sys_tst_rlink_cuff designs
|
tbrun --tag=sys_tst_rlink_cuff # all sys_tst_rlink_cuff designs
|
tbrun --tag=sys_tst_sram # all sys_tst_sram designs
|
tbrun --tag=sys_tst_sram # all sys_tst_sram designs
|
tbrun --tag=sys_w11a # all w11a designs
|
tbrun --tag=sys_w11a # all w11a designs
|
|
|
