DESCRIPTION
|
DESCRIPTION
|
|
|
This archive implements the simple and light protocol for transmission
|
This archive implements the simple and light protocol for transmission
|
of data from low resources FPGA connected to the Ethernet MAC
|
of data from low resources FPGA connected to the Ethernet MAC
|
and an embedded system running Linux OS.
|
and an embedded system running Linux OS.
|
The main goal was to assure the reliable transmission over unreliable
|
The main goal was to assure the reliable transmission over unreliable
|
Ethernet link without need to buffer significant amount of data
|
Ethernet link without need to buffer significant amount of data
|
in the FPGA. This created a need to obtain possibly early
|
in the FPGA. This created a need to obtain possibly early
|
acknowledgment of received packets from the embedded system,
|
acknowledgment of received packets from the embedded system,
|
and therefore the protocol had to be implemented in layer 3.
|
and therefore the protocol had to be implemented in layer 3.
|
|
|
The Ethernet type 0xfade was used (unregistered, but as this
|
The Ethernet type 0xfade was used (unregistered, but as this
|
protocol should be used only in a small private networks,
|
protocol should be used only in a small private networks,
|
without routers, with switches only, it should not be a problem).
|
without routers, with switches only, it should not be a problem).
|
|
|
We assume, that the FPGA is capable to store one "set" of packets
|
We assume, that the FPGA is capable to store one "set" of packets
|
(in the example design length of this set is equal to 32).
|
(in the example design length of this set is equal to 32).
|
To start the transmission, receiver sends the "start transmission"
|
To start the transmission, receiver sends the "start transmission"
|
packet:
|
packet:
|
TGT,SRC,0xfade,0x0001,pad to 64 bytes
|
TGT,SRC,0xfade,0x0001,pad to 64 bytes
|
|
|
After reception of the "start transmission" packet, the transmitter
|
After reception of the "start transmission" packet, the transmitter
|
(FPGA) starts to send the data packets:
|
(FPGA) starts to send the data packets:
|
TGT,SRC,0xfade,0xa5a5,set & packet number, delay, 1024 bytes of data
|
TGT,SRC,0xfade,0xa5a5,set & packet number, delay, 1024 bytes of data
|
|
|
|
|
After reception of the correct data packet, the receiver sends the
|
After reception of the correct data packet, the receiver sends the
|
"acknowledge" packet:
|
"acknowledge" packet:
|
TGT,SRC,0xfade,0x0003,set & packet number, pad to 64 bytes
|
TGT,SRC,0xfade,0x0003,set & packet number, pad to 64 bytes
|
|
|
Another packet may be used to request immediate stop of transmission:
|
Another packet may be used to request immediate stop of transmission:
|
TGT,SRC,0xfade,0x0005, pad to 64 bytes
|
TGT,SRC,0xfade,0x0005, pad to 64 bytes
|
|
|
When first packets from the current set buffered in FPGA are
|
When first packets from the current set buffered in FPGA are
|
transmitted and acknowledged, they may be replaced with the packets
|
transmitted and acknowledged, they may be replaced with the packets
|
from the next set - the current state of transmission is stored
|
from the next set - the current state of transmission is stored
|
in desc_memory in the desc_manager entity.
|
in desc_memory in the desc_manager entity.
|
|
|
When particular packet is not acknowledged, it is transmitted once
|
When particular packet is not acknowledged, it is transmitted once
|
again. In current example design each packet has simple attributes:
|
again. In current example design each packet has simple attributes:
|
1. set number
|
1. set number
|
2. valid (ready to be sent)
|
2. valid (ready to be sent)
|
3. sent (has been sent at least once - used for delay adaptation)
|
3. sent (has been sent at least once - used for delay adaptation)
|
4. confirmed (reception has been confirmed, packet may be replaced
|
4. confirmed (reception has been confirmed, packet may be replaced
|
with the same packet from the next set)
|
with the same packet from the next set)
|
|
|
List of packets is cyclically browsed to move the "head" and "tail"
|
List of packets is cyclically browsed to move the "head" and "tail"
|
pointers.
|
pointers.
|
I've also tried another approach with more sophisticated packet
|
I've also tried another approach with more sophisticated packet
|
manager based on linked lists, but it is not fully debugged and not
|
manager based on linked lists, but it is not fully debugged and not
|
ready for release yet. However the approach with cyclic browsing is
|
ready for release yet. However the approach with cyclic browsing is
|
sufficient, as anyway an additional delay between packets had to be
|
sufficient, as anyway an additional delay between packets had to be
|
introduced to achieve optimal transmission.
|
introduced to achieve optimal transmission.
|
|
|
If the data packets are sent too quickly, the acknowledge
|
If the data packets are sent too quickly, the acknowledge
|
packets from the embedded system are received too late,
|
packets from the embedded system are received too late,
|
and the packet is retransmitted before acknowledge arrives.
|
and the packet is retransmitted before acknowledge arrives.
|
The same may occur if the embedded system is overloaded
|
The same may occur if the embedded system is overloaded
|
with packets from different slaves and drops some packets.
|
with packets from different slaves and drops some packets.
|
|
|
Therefore paradoxically resending of packets as soon as possible
|
Therefore paradoxically resending of packets as soon as possible
|
does not provide the maximal throughput, and a delay between
|
does not provide the maximal throughput, and a delay between
|
packets must be introduced.
|
packets must be introduced.
|
Of course if this delay is too big, the transmission also slows down.
|
Of course if this delay is too big, the transmission also slows down.
|
|
|
To find the optimal delay, I have implemented a simple adaptive
|
To find the optimal delay, I have implemented a simple adaptive
|
algorithm based on analysis of the ratio between number of all sent
|
algorithm based on analysis of the ratio between number of all sent
|
packets and of retransmitted packets: Nretr/Nall
|
packets and of retransmitted packets: Nretr/Nall
|
If the data packets are sent too quickly, the ratio of Nretr/Nall
|
If the data packets are sent too quickly, the ratio of Nretr/Nall
|
increases indicating, that the delay should be higher.
|
increases indicating, that the delay should be higher.
|
If the ratio Nretr/Nall is near to 0, we may reduce the delay.
|
If the ratio Nretr/Nall is near to 0, we may reduce the delay.
|
Such a simple algorithm works quite satisfactory.
|
Such a simple algorithm works quite satisfactory.
|
|
|
In the embedded system, the fpga_l3_fade.ko driver allows you
|
In the embedded system, the fpga_l3_fade.ko driver allows you
|
to service multiple FPGA slaves connected to different network
|
to service multiple FPGA slaves connected to different network
|
interfaces.
|
interfaces.
|
The "max_slaves" parameter lets you to set the maximum number of
|
The "max_slaves" parameter lets you to set the maximum number of
|
slaves, when module is loaded.
|
slaves, when module is loaded.
|
|
|
After that, you can open /dev/l3_fpga0, /dev/l3_fpga1 ...
|
After that, you can open /dev/l3_fpga0, /dev/l3_fpga1 ...
|
devices, to connect different slaves.
|
devices, to connect different slaves.
|
To connect one of those devices to particular FPGA slave,
|
To connect one of those devices to particular FPGA slave,
|
you need to use the ioctl command L3_V1_IOC_STARTMAC
|
you need to use the ioctl command L3_V1_IOC_STARTMAC
|
(please see the attached receiver2.c application for
|
(please see the attached receiver2.c application for
|
an example).
|
an example).
|
The data received from the FPGA are placed in a kernel
|
The data received from the FPGA are placed in a kernel
|
buffer (each subdevice has its own buffer) which may be mmapped
|
buffer (each subdevice has its own buffer) which may be mmapped
|
to the user space application, providing very quick access
|
to the user space application, providing very quick access
|
to the data. Another ioctl commands: L3_V1_IOC_READPTRS
|
to the data. Another ioctl commands: L3_V1_IOC_READPTRS
|
and L3_V1_IOC_WRITEPTRS allow you to read the head and tail
|
and L3_V1_IOC_WRITEPTRS allow you to read the head and tail
|
pointers in this buffer and to confirm reception of data.
|
pointers in this buffer and to confirm reception of data.
|
The attached receiver2.c application uses the described
|
The attached receiver2.c application uses the described
|
mechanisms and simply tests, if the connected FPGA slave
|
mechanisms and simply tests, if the connected FPGA slave
|
sends consecutive 32-bit integers.
|
sends consecutive 32-bit integers.
|
|
|
DISCLAIMER:
|
DISCLAIMER:
|
The published sources are "the first iteration". They work for me,
|
The published sources are "the first iteration". They work for me,
|
but I do not provide any warranty. You can use it only on your
|
but I do not provide any warranty. You can use it only on your
|
own risk!
|
own risk!
|
|
|
I hope to prepare the new, more mature version, which will be
|
I hope to prepare the new, more mature version, which will be
|
described in a "official" publication (I'll send the reference,
|
described in a "official" publication (I'll send the reference,
|
when it is ready).
|
when it is ready).
|
|
|
I'll also publish further versions of sources on my website:
|
I'll also publish further versions of sources on my website:
|
http://www.ise.pw.edu.pl/~wzab/fpga_l3_fade
|
http://www.ise.pw.edu.pl/~wzab/fpga_l3_fade
|
|
|
|
|
LICENSING:
|
LICENSING:
|
1. My kernel driver is released under the GPL license
|
1. My kernel driver is released under the GPL license
|
2. My user space application is public domain
|
2. My user space application is public domain
|
3. My FPGA code is published with BSD license
|
3. My FPGA code is published with BSD license
|
4. I include also very slightly modified Ethernet MAC
|
4. I include also very slightly modified Ethernet MAC
|
http://opencores.org/project,ethernet_tri_mode
|
http://opencores.org/project,ethernet_tri_mode
|
which is published under LGPL.
|
which is published under LGPL.
|
5. Due to licensing issues I can include only xco files for blocks
|
5. Due to licensing issues I can include only xco files for blocks
|
generated by Xilinx tools (in case of sources for
|
generated by Xilinx tools (in case of sources for
|
Spartan 3E Starter Kit instead of binary dcm1.xaw file
|
Spartan 3E Starter Kit instead of binary dcm1.xaw file
|
I had to include the generated dcm1.vhd file to avoid binary
|
I had to include the generated dcm1.vhd file to avoid binary
|
attachment in shar archive).
|
attachment in shar archive).
|
I hope that you'll be able to rebuild my design with them
|
I hope that you'll be able to rebuild my design with them
|
|
|
REBUILDING of FPGA CORES
|
REBUILDING of FPGA CORES
|
|
The sources are split into two sections:
|
|
FPGA_with_MAC - this is the older version with Ethernet MAC taken from
|
|
http://opencores.org/project,ethernet_tri_mode
|
|
FPGA_no_MAC - this is the newer version with renoved Ethernet MAC
|
|
instead two small state machines are implemented in
|
|
ethernet_sender_X and ethernet_receiver_X (X=4 or 8)
|
|
controlling the PHY directly.
|
|
|
My sources have been tested with three boards: SP601, Atlys and
|
My sources have been tested with three boards: SP601, Atlys and
|
Spartan-3E Starter Kit. In the FPGA subdirectory there are
|
Spartan-3E Starter Kit. In the FPGA subdirectory there are
|
three subdirectories: sp601, atlys and sk3e. In each of those
|
three subdirectories: sp601, atlys and sk3e. In each of those
|
subdirectories you there is the "build.sh" script, which
|
subdirectories you there is the "build.sh" script, which
|
should recreate the .bit file needed to configure particular
|
should recreate the .bit file needed to configure particular
|
board.
|
board.
|
|
|
If you create something basing on this my work, I'll be glad if you
|
If you create something basing on this my work, I'll be glad if you
|
provide information about my project (especially if you cite my
|
provide information about my project (especially if you cite my
|
article, after it is ready and published)
|
article, after it is ready and published)
|
|
|
|
|
