Tape Storage and CRC Protection

Logical Block Protection Previous results Multithreading Benchmarks Conclusion

Tape Storage and CRC ProtectionSummer Student Project 2014

Karel Ha

IT-DSS-TABCERN

1 / 17


Outline

1 Logical Block Protection

2 Previous results

3 Multithreading

4 Benchmarks

5 Conclusion

2 / 17


Basic picture

3 / 17


Used CRC algorithms

CRC algorithm supported on tapes:

Reed-Solomon

CRC32-C

4 / 17


File verification of protection information

5 / 17


Previous benchmarks

tape-write (no CRC protection)

cca. 250 MB/s

tape-write with Reed-Solomon CRC

cca. 130 MB/s

Reed-Solomon CRC with output to /dev/null

cca. 200 MB/s

6 / 17


CRC concurrently

CRC computations for every block are independent of one another.

Speed can be increased using parallel computing.This technique is applicable for any CRC algorithm!

0

5

10

15

20

25

30

50 100 150 200 250 300 350 400 450 500

# t

hre

ad

s n

ee

de

d

speed of the CRC algorithm (MB/s)

# threads needed to reach 250MB/s drive speed

7 / 17


CRC concurrently

CRC computations for every block are independent of one another.Speed can be increased using parallel computing.

This technique is applicable for any CRC algorithm!

0

5

10

15

20

25

30

50 100 150 200 250 300 350 400 450 500

# t

hre

ad

s n

ee

de

d



7 / 17


CRC concurrently

CRC computations for every block are independent of one another.Speed can be increased using parallel computing.This technique is applicable for any CRC algorithm!

0

5

10

15

20

25

30

50 100 150 200 250 300 350 400 450 500

# t

hre

ad

s n

ee

de

d



7 / 17


CRC concurrentlyOverview

CRCs

Compute

Threads

Input

Ouput

8 / 17


Architecturetpsrv212

Host: tpsrv212.cern.ch

Indexes: physical

Date: mer. 17 sept. 2014 11:30:15 CEST

Machine (16GB)

Socket P#0

L3 (8192KB)

L2 (256KB) L2 (256KB) L2 (256KB) L2 (256KB)

L1d (32KB) L1d (32KB) L1d (32KB) L1d (32KB)

L1i (32KB) L1i (32KB) L1i (32KB) L1i (32KB)

Core P#0 Core P#1 Core P#2 Core P#3

PU P#0

PU P#4

PU P#1

PU P#5

PU P#2

PU P#6

PU P#3

PU P#7

9 / 17


Architecturetpsrv100

Host: tpsrv100.cern.ch

Indexes: physical

Date: mer. 17 sept. 2014 11:31:05 CEST

Machine (63GB)

Socket P#0

L3 (15MB)

L2 (256KB) L2 (256KB) L2 (256KB) L2 (256KB) L2 (256KB) L2 (256KB)

L1d (32KB) L1d (32KB) L1d (32KB) L1d (32KB) L1d (32KB) L1d (32KB)

L1i (32KB) L1i (32KB) L1i (32KB) L1i (32KB) L1i (32KB) L1i (32KB)

Core P#0 Core P#1 Core P#2 Core P#3 Core P#4 Core P#5

PU P#0

PU P#6

PU P#1

PU P#7

PU P#2

PU P#8

PU P#3

PU P#9

PU P#4

PU P#10

PU P#5

PU P#11

10 / 17


0

100

200

300

400

500

600

700

800

900

0 2 4 6 8 10 12 14 16

MB

/s

n threads

Reed-Solomon on tpsrv212, output to /dev/null

11 / 17


0

200

400

600

800

1000

1200

1400

0 5 10 15 20

MB

/s

n threads

Reed-Solomon on tpsrv100, output to /dev/null

12 / 17


0

500

1000

1500

2000

2500

0 2 4 6 8 10 12 14 16

MB

/s

n threads

CRC32-C on tpsrv212, output to /dev/null

13 / 17


0

500

1000

1500

2000

2500

3000

3500

0 5 10 15 20 25 30

MB

/s

n threads

CRC32-C on tpsrv100, output to /dev/null

14 / 17


0

50

100

150

200

250

300

0 2 4 6 8 10 12 14 16

MB

/s

n threads

Reed-Solomon on tpsrv212, output to tape

15 / 17


Conclusion

Using only 1-2 additional threads, we are already able to reachmaximum tape speed.

And CRC32-C algorithm promises even greater speeds!

16 / 17


Conclusion

Using only 1-2 additional threads, we are already able to reachmaximum tape speed.And CRC32-C algorithm promises even greater speeds!

16 / 17


Thank you!Merci!

17 / 17

Tape Storage and CRC Protection

Technology

Transcript of Tape Storage and CRC Protection