xmatchpro lossless data compressor

Benefits of data compression

If you have problems downloading the latest version can also be obtained from:
http://seis.bris.ac.uk/~eejlny/downloadsxmatchpro.tar.gz

The use of lossless data compression can bring about a number of increasingly important benefits to an electronic system. The term ‘lossless’ means that the original data can be exactly recreated after a decompression operation, and should not be confused with audio and video compression systems (such as JPEG and MPEG) which are lossy and hence only recreate an approximation of the original data.
The most obvious benefit of data compression is a reduction in the volume of data which must be stored. This is important where the storage media itself is costly (such as memory) or other parameters, such as power consumption, weight or physical volume, are critical to product feasibility. Using data compression
reduces the total storage requirement, thus effecting a cost saving.

There are also two other positive effects that data compression brings. The first of these is a reduction in the bandwidth required to transmit a given amount of data –less data must be transmitted when in compressed form, and hence less bandwidth is required. This can effect a cost saving in cabling operations, where a lower bandwidth link will be sufficient to meet demand. The second effect is that given a fixed bandwidth, the total time required to transmit compressed data is less than for uncompressed data. This can lead to a performance benefit, as the bandwidth of a link appears greater when transmitting compressed data and hence more data can be transmitted in a given amount of time.

X-MatchPROvw design architecture

The X-MatchPROvw compressor/decompressor processor is a fully contained unit having a simple architecture and uncomplicated interface.

The X-MatchPROvw design is a dictionary style compressor based around a dictionary implemented in the form of a content addressable memory (CAM). The length of the physical CAM varies with values ranging from 16 to 1024 tuples (4-byte locations) trading complexity for compression. Typically, the device complexity increases by a factor of 1.5 each time the dictionary doubles. The physical dictionary size is, then, variable to be able to adapt algorithm complexity to the resources available in the selected FPGA. The logical length of the dictionary always starts at zero (empty) and grows as new data needs to be accomodated. The logical width of the dictionary also adapts to the data input ranging from 2-bytes up to 4-bytes to improve compression.

The dictionary adaptively stores the most recent phrases that have occurred in the data stream. Compression is achieved by replacing repeated phrases with references to the dictionary (these are codewords witch are sorter than the phrase itself). A number ot techniques such as partial matching and internal run length coding are used to improve compression.

The coding section is active during compression. This generates the required codewords and forms successive codewords into fixed 32-bit width words for writing to external medium. The decoding section is responsible for the reverse process –data is read from the external medium and generates the required dictionary references to allow the decompressed data to be recreated. The process is fully lossless and the compression process is automatically verified using CRC codes.

Features and Applications

Features
• New version targets Xilinx V4/V5 devices at 100 MHz and 140 MHz respectevely. Throughput of 400 Mbytes/second and 560 Mbytes/second in these devices.
• High-speed lossless data compressor supports compression and decompression in a single FPGA.
• Altera APEX20KE prototype implementation available on PCI board.
• Throughput up to 200 Mbytes/second compression/decompression with low latency clocking at 50 MHz on a APEX FPGA. Higher on Stratix or Virtex-4 devices.
• Full-duplex operation enables simultaneous compression/decompression for a combined performance of 400 Mbytes/s.
• Full-duplex architecture enables self-checking test mode using CRC (Cyclic Redundancy Check) codes.
• 32-bit high-performance coprocessor-style interface.
• Fully contained 32-bit architecture does not require any external components and supports operation on blocked data.
• Easy migration to ASIC technology enables 3-5 times increase in performance.
• Compression ratio comparable to HiFn LZS and IBM ALDC using comparable dictionary sizes.

Applications
• Computer systems.
• Networking products.
• High performance storage devices.
• Data logging equipment.
• Remote sensing applications.

To learn more contact us at j.l.nunez-yanez@bris.ac.uk.

The opencores version is generic and can target different FPGA and ASIC technologies. If you are having problems downloading the latest version can also be obtained from
http://seis.bris.ac.uk/~eejlny/downloadsxmatchpro.tar.gz