Subversion Repositories w11

.\"  -*- nroff -*-
.\"  $Id: tbfilt.1 803 2016-08-28 12:39:00Z mueller $
.\"
.\" Copyright 2016- by Walter F.J. Mueller <W.F.J.Mueller@gsi.de>
.\" 
.\" ------------------------------------------------------------------
.
.TH TBFILT 1 2016-08-27 "Retro Project" "Retro Project Manual"
.\" ------------------------------------------------------------------
.SH NAME
tbfilt \- filter for and analysis of test bench log files
.\" ------------------------------------------------------------------
.SH SYNOPSIS
.
.SY tbfilt
.B \-\-tee
.I  OFILE
.OP OPTIONS
.
.SY tbfilt
.OP OPTIONS
.RI [ FILES ]...
.YS
.
.\" ------------------------------------------------------------------
.SH DESCRIPTION
.SS Principle of Operation
\fBtbfilt\fP is the central tool to analyze the log files created by the test
benches. It scans the test bench output for messages which indicate a test
failure and based on this marks a test as \fBPASS\fPed or \fBFAIL\fPed.
It can be used in two modes:

.RS 2
.IP "-" 2
as filter during test bench execution, typically in a setup like
.EX
    tbw <test_bench> 2>&1 | tbfilt --tee=<log_file>
.EE
tbfilt reads the output from the test bench via stdin and a pipe, filters
out the messages indicating a failure and shows them on stdout, and saves
the full test bench output to the file given in the \fB\-\-tee\fP option.
In this mode tbfilt works similar to a
.EX
    tee ... | egrep ...
.EE
pipeline with a very involved egrep selection expression.
The exit status of tbfilt is 1 in case the test is considered as \fBFAIL\fPed.
.
.IP "-" 2
as log file analysis tool. In this case the test bench log files are either
specified explicitly as arguments or determined via the \fB\-\-find\fP or
\fB\-\-all\fP options.
If the \fB\-\-summary\fP option is specified a one line summary for each
test log file is displayed. The format of this summary is configurable via
the \fB\-\-format\fP, \fB\-\-wide\fP, and \fB\-\-compact\fP options and via
the \fB\TBFILT_FORMAT\fP environment variable.
The exit status of tbfilt is 1 in case any of the tests is considered as
\fBFAIL\fPed.
.
.RE
.PP
.
.SS Filter Criteria
A line which contains any of the following strings is considered as an
indication of a \fBFAIL\fPed test:
.RS 2
.PD 0
.IP "\fB-E:\fR"
.IP "\fB-F:\fR"
.IP "\fBERROR\fR"
.IP "\fBFAIL\fR"
.IP "\fB:(assertion warning):\fR"
.IP "\fB:(assertion error):\fR"
.IP "\fB:(assertion failure):\fR"
.PD
.RE

As excption to the general rules above the following assertion messages
are accepted:
Assertion warnings from IEEE libraries at startup (t=0) are ignored. They are
hard to avoid in complex models and in general don't indicate a real issue.
.RS 2
.PD 0
.IP  "-" 2
assertion warnings from IEEE libraries at startup (t=0). They are hard to
avoid in complex models and in general don't indicate a real issue. Best
is to suppress them in \fBghdl\fP(1) with the
option '--ieee-asserts=disable-at-0'.
.IP  "-" 2
assertion failure with the text 'Simulation Finished'. It is used to end
simulations in \fBghdl\fP(1) in some test benches.
.PD
.RE

tbfilt also expects a line in one of the formats
.EX
      xxx ns: DONE               -- tb'swithout clock
    xxx.x ns    xxx: DONE        -- single clock tb's
    xxx.x ns    xxx: DONE-xxx    -- multiclock tb's  (max taken)
.EE
and considers a test \fBFAIL\fPed if it is missing.

In addition lines containing
.RS 4
.PD 0
.IP "\fB-W:\fR"
.IP "\fBPASS\fR"
.PD
.RE
will be displayed. If the \fB\-\-pcom\fP option is specified also all lines
starting with 'C'.

Finally, tbfilt checks for a line of the format
.EX
    real xmx.xxxs   user xmx.xxxs   sys xmx.xxxs
.EE
and extracts the test bench execution times from this. It can be generated
by a \fBbash\fP(1) 'time' command when
.EX
    export TIMEFORMAT=$'real %3lR   user %3lU   sys %3lS'
.EE
is set. The wrapper scripts \fB\tbrun_tbw\fP(1) or \fBtbrun_tbwrri\fP(1)
are in general used to set this up correctly. 
.
.\" ------------------------------------------------------------------
.SH OPTIONS
.
.\" --------------------------------------------------------
.SS Filter Options 
.
.\" -- --tee -------------------------------------
.IP "\fB\-\-tee=\fIofile\fR"
All log file input is written to \fIofile\fP. Typically used to save the
test bench output in a file when tbfilt is used in filter mode in a pipeline
and reads from stdin.
.
.\" -- --pcom ------------------------------------
.IP \fB\-\-pcom\fP
Enables that lines starting with "C" are also displayed.
.
.\" --------------------------------------------------------
.SS File Selection Options
.
.\" -- --find ------------------------------------
.IP "\fB\-\-find=\fIpatt\fR"
When given the input files are determined with a \fBfind\fP(1) command.
The selection pattern \fIpatt\fR is used with a find -regex in egrep mode.
This is functionally similar to a
.EX
    find -regextype egrep -regex '\fIpatt\fR' | sort | tbfilt ....
.EE
pipeline.

When no '*' wildcard is found in \fIpatt\fR it is assumed to be a mode 
specification and the pattern is prefixed by
.EX
    .*/tb_.*_
.EE
and suffixed by
.EX
    .*\\.log
.EE
to select all log files of a given mode (e.g. 'bsim').
.
.\" -- --all -------------------------------------
.IP \fB\-\-all\fP
When given uses as input files all test bench files which conform the
naming convention. Is equivalent to the option
.EX
    --find '.*/tb_.*_[bfsorept]sim(_.*)?\\.log'
.EE
.
.\" --------------------------------------------------------
.SS Summary Options
.
.\" -- --summary----------------------------------
.IP \fB\-\-summary\fP
Selects summary mode. Only a single summary line per input file is written.
The format is configurable via the \fB\-\-format\fP, \fB\-\-wide\fP, and
\fB\-\-compact\fP options and via the \fBTBFILT_FORMAT\fP environment variable.
The precedence is (in increasing priority):
.RS 
.PD 0
.IP "  -" 4
build default ('%ec %pf %nf')
.IP "  -" 
\fBTBFILT_FORMAT\fP option
.IP "  -" 
\fB\-\-wide\fP option
.IP "  -" 
\fB\-\-compact\fP option
.IP "  -" 
\fB\-\-format\fP option
.PD
.RE
.
.\" -- --wide ------------------------------------
.IP \fB\-\-wide\fP
Selects a wide format for summary outputs, designed to give the most pertinent
information. Uses a format of "%fd %fs %tr %tc %sc %ec %pf %nf".
.
.\" -- --compact ---------------------------------
.IP \fB\-\-compact\fP
Selects a compact format for summary outputs, designed to give the key info
on a 80 character wide line. Uses a format of "%fa %tg %sg %ec %pf %ns".
.
.\" -- --nohead ----------------------------------
.IP \fB\-\-nohead\fP
Suppresses the head line of summary outputs. Useful of summary output is
piped into sort or other tools.
.
.\" -- --format ----------------------------------
.IP "\fB\-\-format=\fIfmt\fR"
Defined the format of the summary lines.
The format specification \fIfmt\fR string is a sequence of conversion
specifications of the form '%xx', which will be replaces by the respective
values and other characters which are simply copied (usually a blank as
delimiter).
The supported conversion specifications are:
.RS
.PD 0
.IP \fB%fd\fP 5
modification date of input file (as yyyy-mm-dd)
.IP \fB%ft\fP 
modification time of input file (as hh:mm:ss)
.IP \fB%fs\fP 
modification time of input file short format (as hh:mm)
.IP \fB%fa\fP 
age of input file in seconds, minutes, hours or days
.IP \fB%tr\fP
real (wall clock) time of test bench run
.IP \fB%tu\fP
user time of test bench run
.IP \fB%ts\fP
system time of test bench run
.IP \fB%tc\fP
total cpu (user+system) time of test bench run
.IP \fB%tg\fP
show '%tc c' if cpu time significant, otherwise '%tr r'
.IP \fB%st\fP
simulation time in ns
.IP \fB%ss\fP
simulation time short format (in usec, msec, or sec)
.IP \fB%sc\fP
main system clock cycles till DONE
.IP \fB%sg\fP
use %sc, if available, otherwise %ss
.IP \fB%sp\fP
cpu time per simulation clock cycle (in usec or msec)
.IP \fB%sm\fP
estimate of system clock rate (in MHz)
.IP \fB%ec\fP
error count
.IP \fB%pf\fP
PASS or FAIL, derived from error count
.IP \fB%nf\fP
full file name (with path)
.IP \fB%ns\fP
short file name (without path)
.PD
.RE
.
.\" ------------------------------------------------------------------
.SH EXIT STATUS
In case the test bench is considered FAILed an exit status 1 is returned.
In case of an error at startup, e.g. no input files or invalid format
specification, an error message to stderr or printed and an exit status
of 2 is returned.
.
.\" ------------------------------------------------------------------
.SH ENVIRONMENT
.IP "\fBTBFILT_FORMAT\fR" 4
Defines the default summary format and overwrites the build-in default of
"%ec %pf %nf".
.
.\" ------------------------------------------------------------------
.SH EXAMPLES
.SS As Filter
Usually used together with \fBtbw\fP(1) in pipelines like
.EX
    tbw <test_bench> 2>&1 | tbfilt --tee=<log_file>
.EE
Since tbfilt expects also the output of a \fBbash\fP(1) 'time' command
in the input stream the setup of the pipeline is more involved.
In general the wrapper scripts \fB\tbrun_tbw\fP(1) or \fBtbrun_tbwrri\fP(1)
are used.
.
.SS As Analysis Tool
To generate a compact overview of all test bench outputs use
.EX
    cd $RETROBASE
    tbfilt -all -summary -compact
.EE
To generate a report indicating all \fBFAIL\fPed test use
.EX
    cd $RETROBASE
    tbfilt -all -summary -nohead | grep FAIL
.EE
.
.\" ------------------------------------------------------------------
.SH "SEE ALSO"
.BR tbw (1),
.BR tbrun_tbw (1),
.BR tbrun_tbwrri (1)
.\" ------------------------------------------------------------------
.SH AUTHOR
Walter F.J. Mueller <W.F.J.Mueller@gsi.de>