LZRW3 Data Compression CoreLZRW3 Data Compression Core

Dual semester project April 2013

Project part A final presentation

Shahar Zuta Netanel Yamin

Advisor: Moshe porian

Contents Project Goals Top Architecture Micro Architecture Problems & Solutions Work method Test Plan Live simulation Synthesis Results Part B Schedule

Project GolasImplementation of LZRW3 data compression

algorithm High performance- data transfer of 1GbpsAdapted to data templates of 2Kbyte to 32KbyteInternal memory on FPGA only ( Virtex-5 ), no

interface to external memory

Implementing strong debugging capabilities

via GUI

algorithm overview

INPUT FILE

-------------------------------------------------------

A copy item consists of two bytes that represent from 3 to 18 bytes. literal item consist of one byte which represents himself

LZRW3 COMPRESSO

R

OUTPUT FILE

-]-----[]----[]-------[]-----------[]----[

GROUPS OF ITEMS(literal/Copy)

LITERALS ONLY

Top Architecture

Rx PATH

Tx PATH

INPUT BLOCK memory LZRW3

COMPRESSOR

CORE

COMPRESSED FILE memory

GUI

XILINX VIRTEX 5 ON XUVP505 BOARD

UART

UART

Top Architecture

Rx PATH

Tx PATH

INPUT BLOCKmemory LZRW3

COMPRESSOR

CORE

COMPRESSED FILE memory

GUI

XILINX VIRTEX 5 ON XUVP505 BOARD

UART

UART

Problems & solutions

Problem # 1Hash table (stage III) should not update if we

build a copy item.The decision if a copy item will be built must

consider results of the comparison(stage IV)In the original design we have four clock

cycles untill stage IV will have the comparison resultbut, we have to deal with some cases like

lzrw3ABC(index,length)ABCABC

Original design

Hash

Table

STAGE III

Address Alignmen

t

Address Alignmen

t

Tentative

Next Address

RAM BANKS

COMPARATOR

UPDATE ENABLE

STAGE IV

clk

clk

clk

clk

clk

ME

M R

EG

.

4

12

3

2

DATA

O

1

Heart of the problemDealing with cases like ABCABC mean that we

need toupdate the hash table entry with ABC offset very

fast.

ABC BCA CAB ABC

Two clocks only

:Stage III samples

2nd ABC need to read 1st

ABC offset!

1st ABC at Stage III

2nd ABC at Stage III

Solution

Hash

Table

STAGE III

Address Alignmen

t

Address Alignmen

t

Tentative

Next Address

RAM BANKS

COMPARATOR

UPDATE ENABLE

STAGE IV

clk

clk

clk

clk

clk

ME

M R

EG

.

ABC offset

1

Problem # 2The GOLDEN MODEL does not start building

a new copy item if it’s in the last 18 bytes of the file, if it’s in the middle of building a copy item it will finish it.

Solution # 1 Maintain a FIFO buffer that will hold the two

last groups, then when end of file is reached we will send the buffered bytes and not compressed bytes.

This solution would demand hardware addition and would demand thorough synchronization which could make the system unstable.

Solution # 2Taking advantage of the pipeline design:

Until the data reaches stage IV (the comparator) it goes through 9 buffers.

Buffer another 9 will complete to 18 clocks buffer before a received data will be compared.

FORWARD the EOF core input signal to stage IV and make it cause automatic comparison failure in the last 18 bytes which will result that no copy items will be created.

OUTPUT GROUP

HashFunc

Hash table

INPUT FILE

memory

COMPARATOR

4 5

HAS

H

REG

TABL

E R

EG

GRO

UP

REG

3 By

tes

buff

er

INPU

T RE

G

1234589 76

INPUT BYTE

End Of File

When EOF FARWARD UP,NEW COPY ITEMS will not

be created

Implementation

9 NEW BUFFERS

9 clocks buffer in the original design

Work methodGenerating DIRECT/RANDOM input file.Compress the input file using the GOLDEN

MODEL.Compress the same input file using our CORE.Perform a comparison between the two files using

DIFF software.

Work method (cont.)

ExamDiff

DIRECT input

RANDOM input

CCODE

GM CORE

simulation

chars

ASCII value

C

ASCII value

ExamDiff

Test Plan

Test PlanBasic set: include large amount of tests

Second set: Basic tests + Random client-ready toggles

Third set: Second set + Random inputs valid toggles

Basic set examplesRandom input (Length, Num Of Vars)

INPUTVAR.LENGTHOUTPUTLENGTH

CASE

RANDOM1032K23KReasonable compression

RANDOM1032K7407Each variable repeated 18 times

RANDOM132K3886Very high compression ratio ~90 % compression

RANDOM25632K32KOutput = input

Basic set examplesDirect input

PERFORM A COPY ITEM

COPY ITEMSHOULD

NOTPERFORM

Basic set examplesFiles concatenation

DATA PROVIDER

DATA CLIENT

BUSYFINIS

H

C L A E R

READY

LZRW3 CORE

Live simulation

Synthesis Results

Synthesis ResultsADD PLAN AHEAD/SYNPLIFY REPORT

MAX clock rateBlocks usage

Part B

Part BAssemble lzrw3 core with peripheral units Logic Test End To EndBurning to FPGA Hardware test & optimizationVerification Environment - GUI (Visual

Studio)All system Test & Debug

Schedule

Schedule

DateGoals

11/04/2013-9/05/2013

Assemble units together

12/05/2013-09/06/2013

Test logic End To End

16/06/2013 -10/06/2013

Synthesis

24/06/2013 -17/06/2013

Implementation

08/07/2013 -25/06/2013

Verification Environment – GUI

23/07/2013 -09/07/2013

Lab Tests

08/08/2013Part B Final presentation