Coding is more than error correction A bit of magic A.J. Han Vinck A.J. Han Vinck, Joburg September...

A.J. Han Vinck, Joburg September 2012

Coding is more than error correction

A bit of magic

A.J. Han Vinck

Inspired by Claude Shannon

http://www.uni-due.de/


http://www.ieee-isplc.org/index.htm


Coding is needed in high tech devices


fundamental limits set by Shannon (1948)

Bell Labs, 1955.

Fundamental problem: reproducing at the receiver a message selected at the transmitter

Shannon‘s contribution: - bounds on the efficiency (capacity) - how to achieve the bounds

The Bell System Technical Journal, Vol. 27, pp. 379–423, 623–656, October, 1948.


message: Alpha

codeword: (0 1 0 1 0 1 0 1)

transmission error (0 0 0 1 0 1 0 1)

decode (0 1 0 1 0 1 0 1)

translate: Alpha

How can we do that?

Introduction to the basic concepts


It works as follows

• Use the difference in transmitted vectors to recover the transmitted original

Example:

4 messages encoded as A 00 000B 01 011C 10 101D 11 110the words differ in at least 3 positions

Hence, one difference (error) can be detected and corrected:

receive: 10110

the transmitted message is? 11110


1-Error correction started by Hamming (1950)

(31,26)(7,4)

(3,1)

2 1 3

123

000111

3567 124

0000 0001000 1100100 101•••1111 111 minimum distance dmin = 3

Every circle has even parity

info parities

http://upload.wikimedia.org/wikipedia/commons/0/00/Hamming(7,4)_as_bits.svg


What are the classical research problems

Codebook: maximize - # of codewords of length n - minimum distance dmin

Channel characterization:- Types of errors; memory in the noise behavior ? - Channel Capacity?

Decoder:- Design of a decoding/decision algorithm- Minimize complexity


Lecture content

Goal: give examples, different from just error correction

- improve storage efficiency in flash memory

- give biometrics privacy

- code design for variable efficiency in networking

- permutation codes for non-traditional error models


Ibm punch card „a flash memory“

On the Capacity of Write-Unidirectional Memories with Nonperiodic Codes”, IEEE Transactions on Information theory, April 2004, pp. 649-656, (Fang-Wei Fu, A.J. Han Vinck, Victor Wei, Raymond Yeung)

Write, read and throw away, but •••

Write, read and refresh

Write, read and erase, but •••


A modern ROM: the Flash Memory

Cell array in a flash memory A flash memory cell

(Floating gate)

- A flash memory cell stores charge / data- We can only increase the cell charge- If charge top level is reached, erase memory block


Writing in a q-ary flash memory cell

cell charge level value

1, 0, 1, 0, 0, 0, 1, 1, 0, 1 1, 2, 3, 4, 4, 4, 5, 5, 6, 7 Cell Level 0

Input

Strategy for writing: increase charge content or stay at present content

Start at 0

7 16 05 14 03 12 01 10 0

bit

Use the memory T=4 times: how many sequences we can write into the

memory?

q = 3: 5 +T(T-1)/2 = 11 words

write 0000 0011 0111 1111 0001 0012 0112 1112

0122 1122 1222

q = 2: T+1 = 5 words

write 0000 0011 0111 1111 0001

Capacity: log2|#sequences | ≈ (q-1) log2 (T+1)

Note: writer and reader must know the content!



cell level

average number of writes to reach top level

Performance of the 1 step up strategy

probabilityP(1) = 1 – P(0) = p

p

1-p

Probability (level change) = 2p(1-p)

p)2p(11q

T

!capacity theof 50% iswhich

1)(Tlog2

1q

pT

Tlog :is stored bits of# the 22

10101010

pp1-p

1-p0 1

Example of Codes: 2 bits in 3 cellsbefore erasing


2 bits

2 bits

00 01 10 11

00

01

10

11


An artist‘s view of fingerprints


Coding for Noisy Biometrics used as a key/password

- verify claimed identity with stored information

problems:

• Errors at authentication: False Rejection (FRR)/ False Acceptance (FAR)

• Leakage: attacker finds information about bio from Data Base (DB)

I am Han, can I come in?

http://images.google.com/imgres?imgurl=http://ita.ucsd.edu/workshop/06/participants/images/regphotos/167_thumb.jpg&imgrefurl=http://ita.ucsd.edu/workshop/06/participants/&usg=__L0JESSR4Aq4g12hVZivs8LqyFpA=&h=150&w=149&sz=7&hl=en&start=4&um=1&itbs=1&tbnid=JcFh4NQV-0hqfM:&tbnh=96&tbnw=95&prev=/images?q=han+vinck&hl=en&sa=N&um=1


the Juels-Wattenberg scheme (binary)

b(io)

c bRandomCodeword c

b‘(io)

DataBase

c b b‘ = c e

Hash(c)

Enrollment Authentication c b

Hash(c)

Hash(c)

c b

= ?Y/N

Our contributions: use knowledge of bio at enrollment to encodeuse c b from database to improve decision on c

Yanling Chen and Han Vinck, A fresh look into the biometric authentication: Perspective from Shannon’s secrecy system and a special wiretap channel, Secrypt, 2011V. B. Balakirsky, A. R. G. and A. J. Han Vinck, Permutation block coding from biometrical authentication, Proc. 7th Int. Conf. on Computer Science and Info. Technologies, 2009


Juels-Wattenberg for linear codes

Secret s { 1,2,•••,2k } → RS codeword c = c1,c2,c3,••• ,cn

c* b(io) changes t positions b => --- --- -----

Enrollment: store c*

SECURITY: 2t +1 > dmin Observer „cannot decode“

Authentication: b‘ points t positions in c*

b‘ => --- --- -----

due to „dust“ b‘ points at (t – e) incorrect and e correct positions (e errors in b‘)

DECODABILITY: dmin – t > 2e

WHY? Intuitive: Increasing dmin improves decodability, but decreases security and v.v.


• Translation from Bio to Template– Iris, fingerprint, etc

• Investigation of error behavior– Burst, impulsive, non-linear

• Performance estimation for real dataPractical performance FAR, FRR, etc.

• Using other techniques, such as data compression– Fault tolerant hashing

Research topics we work on

011001…

000001…


Escher Found this one somewhere


Communication over powerlines

• Disturbances: – noise, attenuation, EMC, cable properties

• Noise: – Narrowband noise – Impulsive noise (broadband)

• Catastrophic effect on OFDM (multi-carrier)


A simple solution: Spread-FSK

Send: f0 or f1 :

Detector: uses FSK or On-Off keying mode

Advantages: robust against narrowband noise

alternative

Send: (f0,f1 ) or (f1,f0 ) i.e. R = 1/2

Detector: select max. # of agreements

Advantages: robust

against: narrowband + impulsive noise + random

f

f0 f1

2-FSK mode

f

f1

On-off mode

f0

f1

t

f0

f1

t


effect of correctable noise (simplified)here 2 x 2

f0

f1

t

f0

f1

t

f0

f1

t

f0

f1

t

f0

f1

t

Red always has the majority!


General idea: represent code words as 2-dimensionalTime and frequency picture

t

f

t

f

t

f

f f f

t t t

(123) (132) (231)

(213) (321) (312)

Minimum number of differences is TWO !


effect of noise on permutation codes (simplified)here 3 x 3

Transmitted Background-insertion Background-deletion

Impulsive-broadband narrowband-jammer frequency selective fading

f

t

f

Research questions:

• Design of code books: bounds on cardinality

• Design of decoding algorithm: low complexity

• Performance analysis: practical channels


1)!(DM!

|C|p

# of permutations

price for distance

„performance“

Hendrik C. Ferreira, A. J. Han Vinck, Theo G. Swart, Ian de Beer. Permutation trellis codes. IEEE Transactions on Communications, 2005: 1782~1789

Constructing Coset Codes With Optimal Same-Symbol Weight

for Detecting Narrowband Interference in M-FSK SystemsIEEE Transactions on Information theory, December 2010, pp. 6347-6353Versfeld, D. J. J. Vinck, A. J. H. Ridley, J. N. Ferreira, H. C.


Our (Olaf hooijen and me) modem design based on permutation codes,

1997, Cebit Hannover


Piet Mondriaan

Simple ideas need creativity

(De Zuid-Afrikaanse gruwelprenten van Piet Mondriaan, wie kent ze nog?)

• Coding has a mathematical and a systems aspect

• Applications are widespread: from networking to chip level

• Implementations lead to low complexity algorithms– Viterbi (Convolutional Codes)– Berlekamp Massey (RS-codes)

• Background knowledge in Information and Communication theory and combinatorics necessary

• Cooperation with different disciplines important


Some lessons I learned

http://www.uni-due.de/


Journal Papers with UJ-Wits-PCS scientists

Constructing Coset Codes With Optimal Same-Symbol Weight for Detecting Narrowband Interference in M-FSK Systems IEEE Transactions on Information theory, December 2010, pp. 6347-6353Versfeld, D. J. J. Vinck, A. J. H. Ridley, J. N. Ferreira, H. C.

-„Combined Spectral Shaping Codes and OFDM Modulation for Narrowband Interference Channels,” K. Ouahada, H.C. Ferreira, A.J. Han Vinck, A.J. Snijders and T.G. Swart SAIEE Africa Research Journal, pp. 11-17, March 2008, ISSN 1991-169

- “Permutation Trellis Codes,” IEEE Tr. On Communications, pp. 1782-1789, November 2005, H.C. Ferreira, A.J. Han Vinck, T.G. Swart and I. de Beer.

-“Power Line Communication”, Encyclopedia of Electrical and Electronics Engineering, Wiley, New York, Edited by JohnWebster, pp. 706-716, 199H. Ferreira, H. Grove, O. Hooijen and A.J. Han Vinck,

-“Insertion/Deletion Correction with Spectral Nulls”IEEE Trans. on Information Theory, March 1997, pp. 722-731.(H.C. Ferreira, W.A. Clarke, A. Helberg, K.A.S. Abdel-Ghaffarand A.J. Han Vinck)

-"On the Undetected Error Probability of Linear Block Codes on Channels with Memory"IEEE Trans. on Information theory, Vol. 42, No. 1, January 1996, pp. 303-309.(A. Kuznetsov, F. Swarts, A.J. Han Vinck and H.C. Ferreira)

-"A Class of DC-Free, Synchronization Error Correcting Codes"IEEE Trans. on Magnetics, Vol. MAG-29, No. 5, pp. 4048-4049, September 1993.(A.S.J. Helberg, W.A. Clarke, H.C. Ferreira and A.J. Han Vinck)

my scientific Genealogy (coincidence?)http://genealogy.math.ndsu.nodak.edu/

• Friedrich Leibnitz (1597, Leipzig)

• Carl Friedrich Gauß (Göttingen)

• John M. Wozencraft (MIT) (1925-2009)

• Me (University Duisburg-Essen, Germany)– Digital Communications and - Coding techniques



baie dankie!

ngiyabonga ka khulu


EARLIEST DEPICTION OF JUGGLING known shows skillful Egyptian women on the 15th Beni Hassan

tomb of an unknown prince from the Middle Kingdom period of about 1994 to 1781 B.C

Shannon juggling machine and juggling theorem

http://www.juggling.org/jw/86/2/Pics/egypt-close.gif

http://www.juggling.org/jw/86/2/Pics/egypt-close.gif

Coding is more than error correction A bit of magic A.J. Han Vinck A.J. Han Vinck, Joburg September...

Documents

Transcript of Coding is more than error correction A bit of magic A.J. Han Vinck A.J. Han Vinck, Joburg September...