Subversion Repositories funbase_ip_library

\documentclass[a4paper,10pt,oneside,final]{article}
 
 
\usepackage[english]{babel}
\usepackage[T1]{fontenc}
\usepackage{tabularx}
\usepackage[usenames,dvipsnames]{color}
\usepackage[table]{xcolor}
\usepackage[left=3.0cm, right=2.5cm, top=2.5cm, bottom=2.5cm]{geometry}
\usepackage{graphicx}
\usepackage{float}
\usepackage{caption}
\usepackage{listings}
\usepackage{subfigure}
\usepackage{here}
\usepackage{multirow}
 
\lstdefinestyle{ccc} 
{ 
numbers=none, 
basicstyle=\small\ttfamily, 
keywordstyle=\bf\color[rgb]{0,0,0}, 
%commentstyle=\color[rgb]{0.133,0.545,0.133}, 
stringstyle=\color[rgb]{0.627,0.126,0.941}, 
backgroundcolor=\color{white}, 
frame=tb, %frame= lrtb, 
framerule=0.5pt, 
linewidth=\textwidth,
%aboveskip=-4.0pt,
%belowskip=-4.0pt,
lineskip=-5.0pt,
}
 
%
% Define author(s) and  component's name
%
\def\defauthor{Antti Alhonen}
\def\deftitle{FPGA---PC Ethernet Communication IPs\\Reference Manual}
 
 
\author{\defauthor}
\title{\deftitle}
 
\usepackage{fancyhdr} 
\pagestyle{fancy} 
\lhead{\bfseries Department of Computer Systems\\
  Faculty of Computing and Electrical Engineering}
\chead{} 
\rhead{\bfseries \deftitle} 
\lfoot{\thepage} 
\cfoot{}
\rfoot{\includegraphics[height=1.0cm]{fig/tty_logo.png}}
\renewcommand{\headrulewidth}{0.4pt}
\renewcommand{\footrulewidth}{0.4pt}
 
 
\def\deftablecolora{blue!10!white}
\def\deftablecolorb{white}
 
\setlength{\parskip}{2ex}
 
\begin{document}
 
 
%\maketitle
%\thispagestyle{empty}
 
\begin{titlepage}
\begin{center}
 
\vspace{6.0cm}
\textsc{\LARGE Tampere University of Technology}\\[1.0cm]
\textsc{\Large Faculty of Computing and Electrical Engineering}\\[1.0cm]
\textsc{\Large Department of Computer Systems}\\[1.0cm]
\vspace{6.0cm}
\hrule
\vspace{0.4cm}
{ \huge \bfseries FPGA---PC Ethernet Communication IPs\\[0.5cm]Reference Manual}
\vspace{0.4cm}
\hrule
 
%\vspace{2.0cm}
 
\vfill
 
\begin{minipage}{0.4\textwidth}
\begin{flushleft} \large
\emph{Author:}\\
Antti Alhonen
\end{flushleft}
\end{minipage}
\begin{minipage}{0.4\textwidth}
\begin{flushright} \large
\emph{Updated:} \\
\today
\end{flushright}
\end{minipage}
 
\end{center}
\end{titlepage}
 
\newpage
\tableofcontents
 
 
 
\newpage
\section{REVISION HISTORY}
\setcounter{page}{1}
 
\begin{center}
  \rowcolors{3}{\deftablecolora}{\deftablecolorb}
 
  \captionof{table}{}
  \begin{tabularx}{\textwidth}{|lllX|}
    \hline
    Revision & Author          & Date       & Description\\
    \hline
    1.0  & Antti Alhonen  & 28.10.2011 & First published version\\
    & & & \\
    & & & \\
    & & & \\
    \hline
  \end{tabularx}
\end{center}
 
 
 
\newpage
\section{DOCUMENT OVERVIEW}
 
\subsection{SCOPE}
 
This documentation describes the usage of DCS/TUT made IPs to 
allow simple communication between a computer and various FPGA
development boards. We give an overview to all parts needed for
bi-directional communication over UDP/IP protocol at 100 Mbit/s
with one computer.
 
The main scope is on the simplicity of usage; therefore, some
features such as reliable operation in a network consisting of
numerous devices and computers are not guaranteed to work.
 
\subsection{AUDIENCE}
 
\begin{itemize}
\item SoC developers
\item Kactus 2 design tool users
\item Users of development boards that include an FPGA and Davicom DM9000A (or possibly SMSC LAN91C111) Ethernet MAC/PHY
\end{itemize}
 
\subsection{RELATED DOCUMENTATION}
 
\begin{center}
  \rowcolors[]{2}{\deftablecolora}{\deftablecolorb}
 
  \captionof{table}{}
  \begin{tabularx}{\textwidth}{ | p{8cm} | X | }    %{|lX|}
    \hline
    Document & Description\\
    \hline    
    UDP/IP with VHDL & Implementation specification of the UDP/IP packetizer used in Trace Monitor \\
    DM9kA controller & Implementation specification of the DM9000A ethernet MAC/PHY interface \\
    \hline
  \end{tabularx}
\end{center}
 
%\subsection{DOCUMENT CONVENTIONS}
 
 
 
\newpage
\section{INTRODUCTION}
 
\subsection{CONTACT INFORMATION}
 
If you face any problems using the IPs offered, please
do not hesitate to ask for help or to give suggestions.
You can contact Antti Alhonen (\textit{antti.alhonen@tut.fi}).
 
\subsection{INTRODUCTION}
 
It is very usual to use FPGA's for non-self-contained systems that
need to communicate with computers to transfer large amounts of data.
Standard UART is very simple for transferring small commands and
debug information, but not fast enough to transfer, for example, video
or audio data.
 
Multiple solutions exist, such as USB, PCI, PCI Express, Firewire and
Ethernet connection. Ethernet connection has some benefits such as
simplicity and existing communication routines in all major operating
systems without a need to write device drivers. The biggest drawback
is the lack of guarantee for transfers. However, this problem can be
managed.
 
Usually, a soft-core processor is synthesized on FPGA; then, a
\textit{software} controller and drivers are provided for the external
ethernet chip. In this case, it is possible to implement complex
protocols such as TCP/IP. The drawback is the area overhead of the
processor, extra latency because of the software, possible bandwidth
decrease depending on the processor and the need of embedded processor
and software. Sometimes, a more simple HW approach is needed or
preferred.
 
\subsection{BRIEF DESCRIPTION}
 
This solution implements UDP/IP protocol; a simple, packet-switched, 
low-latency, low-overhead communication between
two devices: an FPGA board and a PC.
 
The HW interface consists of two FIFO interfaces (send and receive) and a 
few intuitive control signals and busses such as ``start new transfer'', 
``destination IP address'', ``new packet received'' etc. These separate
control signals allow very simple usage from HW (VHDL, Verilog etc.) 
applications.
 
The implementation includes UDP/IP packetizer and controllers for
external Ethernet MAC/PHY chips -- currently, two different brands are
supported with some restrictions. Our implementation exploits the FIFO
buffers provided by these external chips, thus the on-chip FPGA memory
requirement is zero.
 
Ethernet controllers currently supported by us include:
\begin{itemize}
\item Davicom DM9000A
\item SMSC LAN91C111 (partially)
\end{itemize}
 
A GMII output will be offered in the future to enable Gigabit Ethernet
operation.
 
%\begin{figure}[H]
%\begin{centering}
%  \includegraphics[width=0.8\textwidth]{fig/router_connections.pdf}
%  \captionof{figure}{Links and FIFOs in one network router in a mesh
%    NoC. The FIFO handshake signals are actually monitored; for
%    Trace Monitor HW and SW.}
%  \label{fig:router_connections}
%\end{centering}
%\end{figure}
 
%Fig.~\ref{fig:mesh3x3} shows a mesh NoC. All links shown as arrows are
 
It is \textbf{very strongly recommended} that you connect these
controllers to a separate network interface card in your PC directly,
i.e.  without any switch, let alone a router. Both of the network
chips mentioned support autonegotiation that should be capable of
working with normal cable in this situation, but if the link does not
go up, you can try a cross-over cable (cable with RX and TX swapped to
allow direct connection).
 
\subsection{Designs provided}
 
\begin{center}
%  \rowcolors[]{2}{\deftablecolora}{\deftablecolorb}
 
  \captionof{table}{}
  \begin{tabular}{ | p{3cm} | p{5cm} | p{6cm} | }    %{|lX|}
    \hline
    Design & Files & Description\\
    \hline    
    \multirow{9}{3cm}{UDP/IP Packetizer} & udp\_ip.vhd (toplevel) & \multirow{9}{6cm}{Offers a simple interface for user application logic 
      to perform TX and RX operations over Ethernet with UDP/IP protocol. The MAC/IP addresses for the FPGA board are set in udp\_ip\_pkg.vhd.
      This block is connected to either one of the Ethernet Chip Controllers provided. Combined toplevels to provide this connection are provided, too.} \\
    & udp\_ip\_pkg.vhd (config package) & \\
    & udp.vhd & \\
    & udp\_arp\_data\_mux.vhd & \\
    & ip\_checksum.vhd & \\
    & arp3.vhd (optional) & \\
    & arpsnd.vhd (optional) & \\
    & & \\
    & & \\
    \hline
    \multirow{9}{3cm}{DM9000A or LAN91C111 Controller} & DM9kA\_controller.vhd (toplevel) & \multirow{9}{6cm}{Connected between UDP/IP block and IO pins of
    the FPGA, these control blocks first initialize the external Ethernet MAC/PHY in question and then control the writes and reads to/from the TX/RX
    FIFOs, commands etc. The entity-level construction is identical for the two supported controllers but the implementations differ due to very different
    interfaces of these chips. You can set the FPGA board MAC address in \_ctrl\_pkg.vhd.} \\
    & DM9kA\_ctrl\_pkg.vhd (config package) & \\
    & DM9kA\_comm\_module.vhd & \\
    & DM9kA\_init\_module.vhd & \\
    & DM9kA\_interrupt\_handler.vhd & \\
    & DM9kA\_read\_module.vhd & \\
    & DM9kA\_send\_module.vhd & \\
    & (substitute lan91c111 for DM9kA for LAN91C111 Controller) & \\
    & & \\
    \hline
    Simple UDP\newline Flood Example & simple\_udp\_\newline flood\_example.vhd & 
    A minimum example to show how TX operations are performed. Creates continuous traffic with
    adjustable packet size to an IP/MAC address defined as generic parameters. First four bytes include an increasing count to detect packet loss.
    Connects to the UDP/IP block. A Kactus 2 example file is provided. Also includes the interface for RX operations; it reads out and ignores all incoming
    packets. You can leave it disconnected if you want to connect the Receiver Example at the same time. \\
    \hline
    Simple UDP\newline Receiver Example & simple\_udp\_\newline receiver\_example.vhd & 
    A minimum example to show how incoming packets can be handled. It reads out the
    incoming packet from the RX FIFO and changes a LED status every time a packet is received.    
    Connects to the UDP/IP block. A Kactus 2 example file is provided. Includes also the interface for TX operations; it never sends anything. You can
    leave it disconnected if you want to connect the Flood Example at the same time. \\
    \hline
  \end{tabular}
\end{center}
 
 
\subsection{Clock and reset}
 
Ethernet controller block and UDP/IP block both run on a synchronous
25 MHz clock. You may need to instantiate an FPGA vendor specific PLL
to generate this clock. Naturally, you can use an integer-multiple
synchronous higher clock for your own application as long as you
make sure the control signals connected to UDP/IP are synchronous and
stable to the 25 MHz clock.
 
Our DM9000A controller outputs the incoming 25 MHz clock directly to an
output IO pin that is connected to the clock input of DM9000A, hence,
the communication with the chip is synchronous. DM9000A is connected
like this in Altera DE2 development board.
 
Our LAN91C111 controller, on the other hand, communicates asynchrously
with the chip. Hence, the chip needs its own clock source. LAN91C111 is
connected like this in Altera Stratix II S180 development board.
 
An asynchronous active low reset signal is used throughout these
designs. After the reset is released (to the high level), the external
ethernet chip is reset and the autonegotiation process is
automatically started. This will take around five to ten seconds, after
which the link\_up signal is provided high, new tx operations accepted
and incoming packets handled. Note that LAN91C111 uses a ``soft'' reset
in the initialization process and therefore the link does not go down
when activating reset but when it is released from the reset.
 
\subsection{DM9000A}
 
Our implementation of DM9000A controller is complete for basic TX and RX
operations and comparably well-tested.
 
\subsection{LAN91C111}
\label{sec:LAN91C111}
 
Due to the very high complexity of usage of LAN91C111 controller and a
number of critical HW bugs in the chip, we have not been able to
demonstrate a reliably working connection with this chip. We are not
sure if we want to work with this chip anymore, so we cannot guarantee
any updates. Hence, we would encourage not to use this very peculiar
and obsolete chip unless absolutely necessary.
 
Currently, simple TX-only and RX-only operations are working in test
environments, but using ARP causing simultaneous TX and RX operations
causes a freezing of the chip hard to analyze. The chip simply stops
giving receive interrupts.
 
Furthermore, due to a critical HW bug in LAN91C111, sending smaller than
66-byte long packets does not work as intended. As a workaround, our controller
pads the packets with zeros but cannot provide correct packet length field.
 
\subsection{IP-XACT files provided}
 
We have created a set of IP-XACT descriptions in Kactus 2 design software.
It is possible to construct a complete working example using these. Two
examples are provided; a TX (send) example and an RX (receive) example.
 
You need to instantiate one (or both) of the two examples and one of the
UDP/IP/Eth CTRL combinations, depending on your external MAC/PHY chip brand.
 
Ready-to-use examples are provided. Please note that as described in the
previous subsection, the LAN91C111 examples may or may not work depending
on the network setup.
 
\newpage
\section{Usage}
\subsection{Sending a packet (TX)}
 
First, make sure the link is up, i.e., link\_up is high. Usually, it
takes a few seconds from a reset to link go up, but this can depend on
the PC, too. Status LED at the RJ-45 connector lits a few seconds
before the link\_up signal goes high.
 
When you have the first word (16 bits) of your payload data ready, do
the following:
 
\begin{enumerate}
\item Output the amount of your actual payload data \textbf{in bytes} in tx\_len
\item Output the destination IP address (the PC) in target\_addr, most
  significant byte first, e.g. x"0A000001" in VHDL for 10.0.0.1.
\item \textbf{If} the ARP functionality is not enabled, output the destination
  MAC address (the PC) in no\_arp\_target\_mac.
\item Output the IP port number where you want the packet to be sent
  in target\_port. (Remember that you need to open this port
  when designing the PC software.)
\item Output the desired IP port number of the FPGA board in
  source\_port. Usually this does not matter but you can check it
  in your PC software.
\item Output the first data word in tx\_data. This will be in
  \textbf{MSbyte last} endianness; thus, the first byte on the line is
  located in tx\_data(7~downto~0).
\item Output tx\_data\_valid = '1'. If you are using a FIFO buffer,
  you can use not empty signal for this one.
\item Raise the new\_tx signal---this, alongwith tx\_data\_valid,
  causes the UDP/IP packetizer to start reading the
  data. \footnotemark
\item tx\_re works as an acknowledgement signal; supply the next data
  word immediately, or, if not possible with the next clock cycle,
  lower the data\_valid for the next cycle.
\item After the amount of data indicated by tx\_len (i.e., ceiling of
  tx\_len/2) has been read, the chip starts sending
  \textbf{automatically}.  Just make sure you feed the amount of data
  you promised in step 1.
\item You can now start the next packet almost immediately. The
  controllers have FIFO buffer memory area for the next packet while
  it is sending the previous one.
\end{enumerate}
 
\footnotetext{If the ARP functionality is enabled and the FPGA does
  not yet have the PC MAC address (i.e., this is the first TX
  operation after power-up, programming or reset), the ARP query will
  be sent before any payload data is read. It may take a variable time
  for the PC to answer. Therefore, for time-critical applications, or
  just to keep things simple, we recommend to disable the ARP if it's
  not needed. The downside to this is that you need to supply the PC
  MAC address to the FPGA and if you don't have a separate input
  system, you need to resynthesize every time if you use multiple
  PCs.\newline~}
 
You can set all of these signals simultaneously. However, if you don't,
then set the new\_tx last.
 
If you communicate with only one PC and want to hard-code the
addresses (necessiating a re-synthesis if the addresses or ports have
to be changed), it's easiest to hard-wire these ports when
instantiating the component.
 
Please see the provided example design, simple\_udp\_flood\_example.vhd.
 
\newpage
\subsection{Receiving a packet (RX)}
 
After the chip receives a packet from the network with a matching MAC
address and a matching IP address (set your FPGA's IP address in
udp\_ip\_pkg.vhd), having correct UDP protocol headers, your
application will be notified by rising new\_rx. If the IP address or
protocol is wrong, the packet will be ignored without a notification.
 
When the new\_rx is up, you can read out the packet as follows:
\begin{enumerate}
\item If you need the information, you can read source\_addr (source
  IP address, i.e. the PC), source\_port (i.e. the output IP port on
  the PC, usually not interesting), dest\_port (the ``FPGA port''
  where it was ``sent to'', may be interesting to check to ignore
  (read out) accidental interactions from PC.)
\item Copy rx\_len to a register; you will need to count the bytes you
  are going to read.
\item If you decide to ignore the packet, still read it out like any
  data.
\item When rx\_data\_valid is high and you did not read on the
  previous clock cycle, you can read out the next data word from
  rx\_data and acknowledge it by setting rx\_re = '1' for one cycle.
\item After you have asserted rx\_re high for ceil(rx\_len/2) times
  correctly, you have read the whole packet. You don't need to do
  anything more.
\end{enumerate}
 
Please note that if you do not read out the whole packet, no new
packets are received after a few packets (RX FIFO gets full).
 
Please see the provided example design, simple\_udp\_receiver\_example.vhd.
 
\newpage
\section{Application development tips}
 
\subsection{MANAGING PACKET LOSS IN HIGH-DATA RATE APPLICATIONS}
 
UDP/IP is a ``send-and-forget'' protocol. The HW reliability is,
however, very high if not practically perfect in a direct connection.
 
However, due to the nature of PC's and especially their operating
systems, a small percentage of packets may be dropped in high-data
rate applications; it depends on your needs whether this is a problem
or not.
 
First, to identify the possible problem, it is recommended that your
application inserts a packet number to the start of every packet
payload, increasing by one after every packet. This way, dropped
packets can be counted.  You can design your application so that the
PC software part can ask to resend a missed packet, however, if the
data cannot be regenerated in-situ, you will need a relatively large
buffer memory near the FPGA to do this.
 
Luckily, in practice, it is possible to reduce the number of dropped
packets to zero even in long runs with data rates close to maximum. Naturally,
this cannot be formally guaranteed but it is a very attractive option due
to the simplicity.
 
There are practically two reasons for dropping a packet:
 
\begin{itemize}
\item (1) Your PC software does not have enough time to read out packets
  from the the RX buffer of the network socket in the OS network
  implementation; after the buffer is full, packets are thrown away.
\item (2) The operating system's network kernel or device driver does
  not have enough time to read out the packets from the RX FIFO of the
  Ethernet card and place them to the buffer mentioned in (1).
\end{itemize}
 
To solve problem 1, increase the RX buffer size. You will need OS-specific
API calls. If you don't want to do it in your software, you can just increase
the default RX buffer of your OS for all programs that do not use their
own setting. For example, in MS Windows XP, open regedit, locate\\
HKEY$\backslash$LOCAL\_MACHINE$\backslash$SYSTEM$\backslash$CurrentControlSet$\backslash$Services$\backslash$AFD$\backslash$Parameters
and add a new DWORD value named DefaultReceiveWindow with a desired RX
buffer size in bytes. I have used values in range of about 300~000 with great 
results (zero drops at 80 Mbps of payload). The default is very small,
only 8192 bytes. Note that this will be used for all programs as a
default unless the programs define a different buffer size and thus
the memory usage may increase. For Linux and other systems, you have to Google around.
 
To solve problem 2, buy a network adapter with longer RX
buffer. Simply put, the older 100 Mbps cards have very short
buffers, regardless of the manufacturer. On the other hand, Gigabit
cards, even the cheapest ones, have longer buffers. I have about
10-100 ppm of packet loss with a 3Com 100 Mbps card (with 8 kbyte RX
FIFO) but zero packet loss with a cheap Realtek Gigabit card, at 80
Mbps, on Windows XP, with increased OS RX buffer size.
 
\newpage
\section{KNOWN ISSUES}
 
\begin{itemize}
\item Lack of workarounds for some LAN91C111 problems; see Section \ref{sec:LAN91C111}.
\end{itemize}
 
\end{document}