Organizing Files for Performance

Organizing Files for Organizing Files for PerformancePerformance6

• Data Compression

Data CompressionData Compression6.1

CIS 256 (File Structures)

ContentContent

►Introduction to Compression►Methods in Data Compression

– Run-Length Coding– Huffman Coding

Data compression Data compression ► Data compression methods are used to make files smaller by

re/encoding data that goes into a file .► There are many reasons for making file smaller

– Use less storage , resulting in cost saving– Can be transmitted faster , decreasing access time or,

alternatively ,allowing the same access time with a lower And cheaper bandwidth

– Can be processed faster sequentially

Techniques of compressions Techniques of compressions

►Using different notation►Suppressing Repeating Sequences►Assigning Variables Length Codes► Irreversible Compression Techniques (Lossy)

► Fixed-Length fields are good candidates► Decrease the number of bits by finding a more compact notation► Cons.

– unreadable by human– cost in encoding time – decoding modules increase the complexity of s/w used for particular application.

►Example: The state fields in the person records. 6 bits (for 50 states) instead of 16.

►It’s classified as redundancy reduction technique. ►With so many costs, is this kind of compression worth it?

Using different NotationUsing different Notation

Using different NotationUsing different Notation► The notation used for representing information can often be

made more compact.► EX if we are going to write a file that contains information about

students such as name, marks , and major, we can declare the mark as byte instead of integer, in this way we can save a space.

ST_REC = Name_Stu : string[50]; Mark_Stu : int ; byte Major_Stu : string[30]; string[3];

// using lookup table.

Suppressing Repeating SequencesSuppressing Repeating Sequences

► Run-length encoding (RLE): encode sequences of repeating values rather than writing all the values in the file.

► EX: Suppose we wish to compress an image using run –length encoding, and we find that can be omit the byte 0xff

from the representation of image . - How would we encode the following sequence of hexadecimal

byte values ? 22 23 24 24 24 24 24 24 24 25 26 26 26 26 26 26 25 24

- The way: the first three pixels are to be copied in sequence. The runs of 24 and 26 are both run length encoded .the remaining pixels are copied in sequence ,the resulting sequence is:

22 23 ff 24 07 25 ff 26 06 25 24

► Run-length encoding (cont’d)

– example of redundancy reduction

– cons.

• not guarantee any particular amount of space savings

• under some circumstances, compressed image is larger than original image

– Why? Can you prevent this?

Suppressing Repeating SequencesSuppressing Repeating Sequences

RLERLE

Here we have a series of blue x 6, magenta x 7, red x 3, yellow x 3 and green x 4, that is:

► Morse code: oldest & most common scheme of variable-length code

► Some values occur more frequently than others– that value should take the least amount of space

► Huffman coding– base on probability of occurrence

• determine probabilities of each value occurring

• build binary tree with search path for each value

• more frequently occurring values are given shorter search paths in tree

Assigning Variable-Length CodesAssigning Variable-Length Codes

Variable-length encoding Variable-length encoding

► Any encoding scheme in which the codes are of different lengths. More frequently occurring codes are given shorter lengths than frequently occurring codes. Huffman encoding is an example of variable-length encoding.

► Huffman code which determines the probabilities of each value occurring in the data set and then builds a binary tree in which the search path for each value represent the code for that value.

Huffman EncodingHuffman Encoding

►Compression– Typically, in files and messages,

• Each character requires 1 byte or 8 bits• Already wasting 1 bit for most purposes!

►Question– What’s the smallest number of bits that can be used to

store an arbitrary piece of text?

► Idea– Find the frequency of occurrence of each character– Encode Frequent characters short bit strings– Rarer characters longer bit strings

►Encoding– Use a tree– Encode by following

tree to leaf– eg

• E is 00• S is 011

– Frequent charactersE, T 2 bit encodings

– Others A, S, N, O 3 bit encodings

► Encoding– Use a tree

• Inefficient in practice– Use a direct-addressed lookup

? Finding the optimal encoding– Smallest number of bits to

represent arbitrary text

►Divide and conquer– Decide on a root - n choices– Decide on roots for sub-trees - n choices– Repeat n times

►Greedy Approach– Sort characters by frequency– Form two lowest weight nodes into a sub-tree

• Sub-tree weight = sum of weights of nodes– Move new tree to correct place

Huffman Encoding - OperationHuffman Encoding - Operation

Initial sequenceSorted by frequency

Combine lowest twointo sub-tree

Move it to correctplace

After shifting sub-treeto its correct place ...

Combine next lowestpair

Move sub-tree to correct place

Move the new tree to the correct place ...

Now the lowest two are the“14” sub-tree and D

Combine and move to correct place

Move the new tree to the correct place ...

Now the lowest two are thethe “25” and “30” trees

Combine and move to correct place

Combine last two trees

Huffman Encoding - DecodingHuffman Encoding - Decoding

Huffman Encoding - Time ComplexityHuffman Encoding - Time Complexity

►Sort keys O(n log n)►Repeat n times

– Form new sub-tree O(1)– Move sub-treeO(logn)

(binary search)

– Total O(n log n) ►Overall O(n log n)

Irreversible Compression TechniquesIrreversible Compression Techniques

► Compression in which information is lost.– EX : Shrinking a raster image from 400 by 400 pixels to 100 by 100

pixels .

► There is no way to determine what the original pixels were from the one new pixel.

► Irreversible Compression is less common in data files than reversible compression but there are times when the info. That is lost of little or no value. – EX: Speech Compression.

►Some information can be sacrificed►Less common in data files►Shrinking raster image

– 400-by-400 pixels to 100-by-100 pixels

– 1 pixel for every 16 pixels

►Speech compression – voice coding (the lost information is of no little or no value)

Lossy Compression TechniquesLossy Compression Techniques

Organizing Files for Performance

Documents

Transcript of Organizing Files for Performance

Community-based organizing, how to assess its performance?

ORGANIZING PUBLIC DIPLOMACY: A LAYERED SYSTEMuscpublicdiplomacy.org/sites/uscpublicdiplomacy.org/files... · Organizing Public Diplomacy: ... Any company selling a product or ...

2017 Organizing Team Recruiting Overview - NIBCnibc.ca/sites/default/files/NIBC 2017 - Recruiting Presentation.pdf · 2017 Organizing Team Recruiting Overview ... Excel Shortcuts/Charts

Ch. 2 Organizing and Summarizing Datasite.iugaza.edu.ps/mriffi/files/2018/02/Statistics... · 2018. 2. 6. · Ch. 2 Organizing and Summarizing Data 2.1 Organizing Qualitative Data

CPSC 231 Organizing Files for Performance (D.H.) 1 LEARNING OBJECTIVES Data compression. Reclaiming space in files. Compaction. Searching. Sorting, Keysorting.

Organizing the US Health Care Delivery System for High Performance

Organizing for Performance

Organizing for Higher Performance: Case Studies of Organized

United Way Business Performance Overview United Way’s Organizing Framework for Performance.

Organizing footage - Center for Science and Technology Policy …sciencepolicy.colorado.edu/students/envs3173/... · 2018-10-23 · Organizing footage Consider renaming files from

Chap6. Organizing Files for Performance. Chapter Objectives(1) Look at several approaches to data compression Look at storage compaction as a simple.

Organizing data using frequency distributions. Illustrating data ...fmalam.kau.edu.sa/Files/0007085/Files/157188_STAT_110_CH...Overview of Chapter 2 Sec. # Title Page(s) 2 - 1 Organizing

Computer Science Worksheet—Organizing Files · Computer Science Worksheet—Organizing Files An example of a bad organizational system would be organizing them by initial letter,

Performance Appraisal Systems - Dattner Consultingdattnerconsulting.com/presentations-files/performanceappraisal.pdf · Performance appraisal ... Organizing logistics for the report

FILE MANAGEMENT FOR THE GENEALOGISTS Creating/organizing files and folders.

Chapter 5 Organizing Files and Folders. Organizing Files and Folders FAQs: – How do I get a list of my files? – How do I navigate to different folders.

Organizing for Content Summary Contextual Campaignsrebeccalieb.com/sites/default/files/downloads... · REBECCA LIEB Organizing for Content 1 Organizing for Content ... Example: The

High Performance Self Organizing Dynamic Dictionariesweb2py.iiit.ac.in/research_centres/publications/download/masters... · High Performance Self Organizing Dynamic Dictionaries ...

Journal of Management Excellence, Organizing for ... of Management Excellence Organizing for Management Excellence An Oracle Business Intelligence and Performance Management Publication

Developing and Organizing Ideas: Webbing, Mapping and …oame.on.ca/main/files/thinklit/WebbingMapping.pdf · Outlining – organizing main ideas, information, and supporting details