Hp Man DP7.00 EncodingCompress Wp PDF

7/28/2019 Hp Man DP7.00 EncodingCompress Wp PDF

1/28

Data Protector Encoding/Compression plug-in

Whitepaper

HP OpenView Storage

July 2006


2/28

CONTENTS:

INTRODUCTION........................................................................................................................................ 3

OVERVIEW................................................................................................................................................. 4

DATA PROTECTORENCODING/COMPRESSION PLUG-IN FEATURES ............................................................. 4

DATA PROTECTORENCODING/COMPRESSION PLUG-IN LIMITATIONS ......................................................... 4

DATA PROTECTOR ENCODING/COMPRESSION PLUG-IN API ................................................... 6

ENCODING API............................................................................................................................................ 6

COMPRESSION API...................................................................................................................................... 9

ERRORHANDLING..................................................................................................................................... 14

MESSAGES................................................................................................................................................. 14

INTEGRATING USERLIBRARIES WITH DATA PROTECTOR.......................................................................... 16

WINDOWS DISASTER RECOVERY..................................................................................................... 25

QUESTIONS AND ANSWERS ................................................................................................................ 27

APPENDIX 1 TERMS AND DEFINITIONS ....................................................................................... 28

2


3/28

Introduction

With A.06.00 Release, HP OpenView Storage Data Protector includes an enhanced version ofencoding/compression plug-in, which supports implementation of user-specific compression andencoding algorithms instead of the default Data Protector compression/encoding algorithms.

The purpose of this document is to explain Data Protectors encoding/compression plug-infunctionality and implications of its use to system and backup administrators. It also givesguidance to developers of applications and libraries for encoding/compression plug-in, on how totake a full advantage of this plug-in.Check HP OpenView Storage Data Protector Concepts Guide and HP OpenView Storage DataProtector Administrators Guide for general information on Data Protector. Refer to man pages onUNIX or HP OpenView Storage Data Protector Command Line Interface Reference forinformation on Data Protector commands.

Figure 1: The Data Protector Encoding/Compression plug-in

Cell Manager

Client System

Disk Disk Agent

Client System

Media AgentData Flow

DE plug-in

DC plug-in

Control ControlDeInit()

DeEncode()

DeDecode()DeExit()

DcInit()DcCompress()

DcDecompress()DcExit()

3


4/28

Overview

This chapter gives an overview of the Data Protector encoding/compression plug-in functionalityand covers several key points for application developers as well as system/backup administratorsusing the encoding/compression plug-in.

Data Protector Encoding/Compression Plug-in Features

Data Protector currently supports encoding and compression features employing its ownimplementation. The default encoding algorithm is XOR while the default compression algorithm is

Lempel-Ziv 4.3 (UNIX compress). It is possible to use these features using the omni b commandor the GUI.

The Data Protector data compression and encoding logic is implemented as shared libraries1

separated from the Disk Agent (DA). The Data Protector modules using theencoding/compression plug-in are:

Volume Backup/Restore Disk Agent

Data Protector Internal Database (IDB) Backup Disk Agent

Raw Disk Backup/Restore Disk Agent

Integration with HP OpenView OmniStorage

Online Integrations

Data Protector Encoding/Compression Plug-in Limitations

Features described in this document are limited to Data Protector A.06.00 and newer.See the HP OpenView Storage Data Protector Software Release Notes for informationon the platforms on which the DA is supported. Encoding/compression plug-in notsupported on HP NAS, OpenVMS and Windows Alpha platforms. Compression is notsupported on Novell NetWare systems. However, it is still possible to decompress NovellNetWare files that were compressed during backup using Data Protector A.06.00 or olderversions.

The current version of Data Protector encoding/compression plug-in API is not fullycompatible with previous versions.

The libraries used with Data Protector A.05.00 cannot be used as is with Data Protector

A.06.00 and vice versa. You have to modify and recompile your libraries using the APIprovided with Data Protector A.06.00 (described in this document).Backups, which were created with Data Protector A.05.00, can be restored using theData Protector A.06.00 library, as long as the algorithm you used is the same as before.However, the default encoding/compression algorithm (using the libraries shipped withthe product) is fully compatible with older versions of Data Protector and old backups.

4

1This includes Windows dlls.


5/28

Data Protector encoding/compression plug-in does not provide the support forinitialization/cleanup of user libraries. In case of an error in initialization functions, theDisk Agent will proceed without encoding/compression.

It is not possible to dynamically change encoding/compression algorithms during a run-time depending on file system object type.

User libraries must be redesigned to fit into Data Protector encoding/compression plug-in

API. All errors in user libraries are treated as fatal, except for errors in initialization functions.

In case of a fatal error, the Disk Agent will be immediately aborted, whereas in case of anerror in initialization functions, the DA will proceed without encoding/compression.

Sparse files are not supported.

Software data compression for online database backups, such as Oracle, Sybase, SAPR/3, Informix, and Microsoft SQL, is not supported.

5


6/28

Data Protector Encoding/Compression Plug-in

API

This chapter contains explanation of Data Protector encoding/compression plug-in API.

Encoding API

Initialization - DeInit()

Name:DeI ni t ( ) - Initializes Data Encoding Library

Synopsis:

UNIXunsi gned l ong DeI ni t ( char *) ;

WINDOWS__decl spec( dl l export ) unsi gned l ong DeI ni t ( TCHAR *) ;

NOVELL NETWAREunsi gned l ong HPDeI ni t ( char * ) ;

Parameters:

char * - a pointer to a 125 character long buffer.

Description:The DeI ni t function is called by the DA module during the initialization procedure at startup. Itis expected that a library, which uses this call, will perform all necessary initialization steps(obtaining key, internal initialization, etc.) and return the indication of what encoding will be used.

The user custom library can put any identification string into this buffer. Length of theidentification string is limited to 125 characters, including the terminating \ 0 character. If thecustom user library puts any string into this buffer, this string will be displayed with theappropriate Data Protector message, otherwise only Data Protector message indicating usage ofthe custom user library will be displayed.

Return Values:0 - Initialization failed, encoding disabled1 - XOR encoding[ 50- 200] - User-defined encoding types

Range 50-200 is reserved for user defined encoding types. Usage of values outside of thespecified range can result in undefined behavior.

6


7/28

During the restore, return values will be used to check for the encoding type.

Error Handling:Initialization failure during the backup will be reported by an error message on the console. DAwill proceed without encoding the data.

Encoding - DeEncode()

Name:DeEncode( ) - Performs data encoding.

Synopsis:

UNIXi nt DeEncode( unsi gned char *dat a, unsi gned l ong si ze) ;

WINDOWS__decl spec( dl l export )

i nt DeEncode ( unsi gned char *dat a, unsi gned l ong si ze) ;

NOVEL NETWAREi nt HPDeEncode(unsi gned char *dat a, unsi gned l ong si ze) ;

Parameters:data - A pointer to a raw data buffer;si ze - Data buffer size

Description:DeEncode( ) performs encoding of the data buffer. It is expected that encoding will be done in-place and that the data buffer size will be maintained while its contents will be overwritten by theencoded data.

Return Values:0 - Error in encoding process1 - Data encoded successfully.

Error Handling:All errors will be treated as fatal and DA will be aborted.

Decoding - DeDecode()

Name:

DeDecode( ) - Decode data;.

Synopsis:

UNIXi nt DeDecode(unsi gned char *data, unsi gned l ong si ze) ;

WINDOWS__decl spec( dl l export )

7


8/28

i nt DeDecode ( unsi gned char * data, unsi gned l ong si ze) ;

NOVEL NETWAREi nt HPDeDecode(unsi gned char *dat a, unsi gned l ong si ze) ;

Parameters:data - A pointer to a raw data buffer;si ze - Size of the data buffer

Description:DeDecode( ) performs decoding of the raw data buffer. It is expected that decoding will be donein-place and that the data buffer size will be maintained while its contents will be overwritten bythe decoded text.

Return Values:0 - Error in decoding process1 - Data decoded successfully.


Cleanup - DeExit()

Name:DeExi t ( ) - cleanup process

Synopsis:

UNIXi nt DeExi t ( voi d) ;

WINDOWS__decl spec( dl l export ) i nt DeExi t ( voi d) ;

NOVEL NETWAREi nt HPDeExi t ( voi d) ;

Parameters:

Description:

DeExi t ( ) is called to allow post-process cleanup.

Return Values:0 - Error

1 - OK

Error Handling:Return values of this function are ignored.

8


9/28

Compression API

Callback functions

Data Protector provides two callback functions to read and write data. Pointers to these callback

functions are passed as arguments to DcCompr ess ( ) and DcDecompress( ) functions. Thesecallback functions should be used for all read/write requests from the library.

dcCallback();

Name:dcCal l back() - reads from the file system or writes to the file system the object beingbacked up.

Synopsis:t ypedef i nt ( *dcCal l back) ( unsi gned char *, unsi gned l ong ) ;

Parameters:

Description:Callback functions are provided in order to enable read/write from user libraries.Data Protector will pass pointers to callback functions toDcCompress( ) and DcDecompress( ) functions.

Return Values:Since dcCal l back function is used as a prototype for read/write, its return values depend on thecontext.r eadCal l back:

>0 - Number of bytes read

0 - Error or EOFwr i t eCal l back:

>0 - number of bytes written0 - Error

Notes:The r eadCal l back function returns the number of bytes read. This number can, under certaincircumstances, be less than the number of bytes actually requested. The compression engineshould not treat this as an error but simply issue a second read (this is similar to BSD sockets,reads may succeed partially). The r eadCal l back function returns zero in case of error or atEOF.

The wr i t eCal l back function always succeeds completely. If an error condition is encountered,wr i t eCal l back never returns.

9


10/28

Initialization - DcInit()

Name:DcI ni t ( ) - Initializes the Compression Library.

Synopsis:

UNIXunsi gned l ong DcI ni t ( char *) ;

WINDOWS__decl spec( dl l export ) unsi gned l ong DcI ni t ( TCHAR *) ;

NOVEL NETWAREunsi gned l ong HPDcI ni t ( char * ) ;

Parameters:char * - a pointer to a 125 character long buffer.

Description:This function is called by the DA initialization procedure at startup. It is expected that the librarywill perform all initialization steps through this call and return an indication of what compressiontype will be used. User library can put any identification string into the passed buffer. Length ofthe identification string is limited to 125 characters including the terminating \ 0 character. If theuser library puts any string into this buffer, the string will be displayed with an appropriate DataProtector message, otherwise only Data Protector message indicating the usage of custom userlibrary will be displayed.During the restore, return value will be used to check for the type of compression.

Return Values:0 - Initialization failed, compression disabled.

0x0b0b0b0b Lempel-Ziv 4.3[ 50- 200] - User-defined encoding types

Range 50-200 is reserved for user defined encoding types. Usage of values outside of thespecified range can result in undefined behavior.

Error Handling:Initialization failure during the backup will be reported by an error message on the console. DAwill proceed without encoding the data.

Compression - DcCompress()

Name:DcCompress( ) - Compress Data.

Synopsis:

UNIXi nt DcCompr ess( dcCal l back r eadCal l back, dcCal l back wr i t eCal l back) ;

10


11/28

WINDOWS__decl spec( dl l export )i nt DcCompr ess ( dcCal l back r eadCal l back, dcCal l back wr i t eCal l back) ;

Parameters:r eadCal l back - A pointer to the data input callbackwr i t eCal l back - A pointer to the data output callback

Description:This is the actual stream compression function. It receives input data using the r eadCal l backdata input method and writes output using the wr i t eCal l back output functions. These callbackfunctions are provided by Data Protector.

The nature of the input and output media is unknown to the compression module. Both input andoutput should be treated as read-only and write-only streams respectively.

The DcCompr ess function is called once for each file to be compressed. It should read its inputuntil EOF (the input handler returns zero) and make sure all intermediate output buffers areflushed before it returns.Each call to the input method is sooner or later directly translated into a r ead( ) system call.

Thus, for better performance, the compression engine should try to read its input in large chunksand store them in an internal input data buffer.

Return Values:0 - Error during compression1 - Compression successfulAll errors will be treated as fatal and DA will be aborted.

Notes:Never call the readCallback to read zero bytes, as you will be returned zero, which means 'Erroror EOF'. Also for the same reason, never attempt to write zero bytes.

Decompression - DcDecompress ()

Name:DcDecompr ess ( ) Decompress data

Synopsis:

UNIXi nt DcDecompr ess( dcCal l back readCal l back, dcCal l back wr i t eCal l back) ;

WINDOWS

__decl spec( dl l export )i nt DcDecompr ess ( dcCal l back r eadCal l back, dcCal l back wr i t eCal l back) ;

NOVEL NETWAREi nt HPDcDecompr ess( dcCal l back readCal l back, dcCal l back wr i t eCal l back) ;

Parameters:r eadCal l back - A pointer to the data input callbackwr i t eCal l back - A pointer to the data output callback

11


12/28

Description:

This is the actual stream decompression function. It receives its input data using ther eadCal l back data input method and writes its output using the wr i t eCal l back outputfunctions.Here are a few guidelines the compression module developer should keep in mind:

The nature of the input and output media is unknown to the decompression module. Bothinput and output should be treated as read-only and write-only streams, respectively.

The DcDecompr ess ( ) function is called once for each file to be decompressed. Itshould read its input (until EOF the input handler returns zero) and make sure allintermediate output buffers are flushed before it returns.

Both input and output may be done in byte chunks of arbitrary length. Ther eadCal l back typically returns data from an internal Data Protector data buffer. Toincrease performance, output larger amounts of data at once, because calls to thewr i t eCal l back may be directly translated into the wr i t e( ) system calls.

The r eadCal l back may succeed partially. A return value of zero denotes an error orthe end of the current file to be decompressed. The wr i t eCal l back always succeedscompletely and never reports any errors.

r eadCal l back returns:>0 - Number of bytes read0 - Error or EOF

wr i t eCal l back r et ur ns:>0 - Number of bytes written0 - Error

Return Values:0 - Error1 - OK


Notes:Never call the r eadCal l back to read zero bytes, as you will be returned zero, which means'Error or EOF'. Also for the same reason, never attempt to write zero bytes.

Cleanup - DcExit ()

Name:DcExi t ( ) Post-process cleanup

Synopsis:

UNIXi nt DcExi t ( voi d) ;

WINDOWS__decl spec( dl l export ) i nt DcExi t ( voi d) ;

NOVEL NETWARE

12


13/28

i nt HPDcExi t ( voi d) ;

Parameters:

Description:This function allows DC module to perform post-process cleanup.

Return Values:0 - Error1 - OK

13


14/28

Error Handling

Data Protector does not provide support for any kind of recovery in case of errors in user library,so all errors in user libraries will be treated as fatal and DA will be aborted immediately. In case of

initialization failure, DA will proceed without encoding/compression.

Messages

In the backup session reports, user (encoding/compression) library ID will be shown only ifencode orcompr ess is selected (in GUI or CLI).During the restore process, messages indicating library initialization failure will be shown even ifdata was not encoded or compressed.

Encoding Related

[ Normal ] Fr om: VBDA@f a. her mes. si "msg_t est " Ti me: 09/ 03/ 02 10: 15: 46 l i br ary used f or encodi ng! 2

This message will be shown only if library initialization passes, otherwise the following messagewill be reported on the console:

[ Warni ng] Fr om: VBDA@col orado. her mes. si "DEDC" Ti me: 03/ 07/ 03 16: 13: 06Encodi ng l i brary i ni t i al i zat i on f ai l ed!

If the encode or decode process fails, the following message will be displayed:

[ Cr i t i cal ] From: VRDA@f a. her mes. si "msg_t est " Ti me: 09/ 03/ 02 10: 15: 46I nt er nal er r or i n encodi ng l i br ar y. Abor t i ng sessi on!

If encoding type is not recognized during restore, the following error will be displayed:

[ Warni ng] Fr om: VRDA@f a. her mes. si "msg_t est " Ti me: 09/ 03/ 02 10: 49: 10[ 84: 119] Unknown t ype of encodi ng !

Compression Related

[ Normal ] Fr om: VBDA@f a. her mes. si "msg_t est " Ti me: 09/ 03/ 02 10: 15: 46

l i br ary used f or compr essi on!

[ Warni ng] Fr om: VRDA@col orado. her mes. si "DEDC" Ti me: 03/ 07/ 03 16: 49: 53Compr essi on l i br ar y i ni t i al i zat i on f ai l ed!

If compress or decompress process fails, the following message will be displayed:

14

2If the user library doesnt return the identification string, Data Protector will display the same message replacing

part with the string Unknown. The same goes for compression.


15/28

[ Cr i t i cal ] From: VRDA@f a. her mes. si "msg_t est " Ti me: 09/ 03/ 02 10: 15: 46I nt er nal er r or i n compr essi on l i br ar y. Abor t i ng sessi on!

If compression type is not recognized during restore, the following error will be displayed:

[ Warni ng] Fr om: VRDA@f a. her mes. si "msg_t est " Ti me: 09/ 03/ 02 10: 15: 46[ 84: 114] Unknown t ype of compressi on: 185273099.

[ Warni ng] Fr om: VRDA@f a. her mes. si "msg_t est " Ti me: 09/ 03/ 02 10: 32: 17I nval i d compr ess data !

15


16/28

Integrating User Libraries with Data Protector

Rationale

Data Protector encoding/compression plug-in enables users to embed their ownencoding/compression libraries into the Data Protector modules. We assume that a user alreadyhas his own encoding/compression libraries3 and would like to embed them into Data Protector,therefore significant changes in user libraries are not necessary.

Creating Libraries

Steps to create user libraries are explained. Process of creating shared libraries is not covered.

Encoding:1. Declare encoding functions conforming DE encoding/compression plug-in API;2. Implement those functions so they call the appropriate users library functions in the

following way:

All initialization tasks should be done in DeI ni t ( ) ; Use DeEncode( ) and DeDecode( ) to perform encoding and decoding;Use DeExi t ( ) to perform post-process cleanup;

3. Build new binaries linking with already existing library.4

Note that Windows libraries mustsupport UNICODE.

4. Create new shared library. It must me named l i bde. s l ;5

Compression:1. Declare compression functions conforming DE encoding/compression plug-in API;2. Implement those functions so they call appropriate users library functions in the following

way:All initialization tasks should be done in DcI ni t ( ) ; Use DcCompress( ) and DcDecompress( ) to perform compression and

decompression;Use DcExi t ( ) to perform post-process cleanup;

3. Build new binaries linking with already existing library. Note that Windows libraries mustsupport UNICODE.

4. Create a new shared library. It must be named l i bdc. s l ;6

Configuration

Replace the Data Protector native libraryl i bde. s l (l i bdc. s l ) with the newly built user library.It is recommended to keep the original libraries in a safe place.

3This assumption also covers writing the library from scratch, because a developer can easily design it to be compatible

with Data P rotector encoding/compression plug-in API.4Data Protector does not provide separate binaries for 64 bit platforms, therefore binaries must be portable. On most

platforms this can be done specifying a compiler option.5

This is HP-UX specific naming convention and it will be used in the rest of the document. Shared library extensions ondifferent platforms are: AIX l i bde. a, l i bde. o; IRIX, Solaris, Linux, SCO, SCO-5 - l i bde. so; Windows -l i bde. dl l , MPE l i bde. s l

16

6This is HP-UX specific naming convention and it will be used in the rest of the document. Shared library extensions on

different platforms are: AIX l i bdc. a, IRIX, Solaris, Linux, SCO, SCO-5- l i bdc. so; Windows - l i bdc. dl l ., MPE l i bdc.s l


17/28

Novell Netware Specific

In order to support Encoding/Compression plug-in on Novell NetWare platforms, Library NLMs(named HPLI BDE. NLMand HPLI BDC. NLM) are provided.Library NLMs are automatically loaded by Data Protector DA main modules (HPVBDA. NLMorHPVRDA. NLM) when DA is first loaded. Once the Library NLMs are loaded, all symbols areexported and any client NLM (Data Protector NLM) can use them. Once the library NLMs are

loaded, they should not be loaded any more. Library NLMs will be unloaded only when the servergoes down.If you would like to uninstall Data Protector, manually unload library NLMs before uninstallingData Protector. To manually unload library NLMs, unload HPI NET. NLMfirst, thenHPLI BDC. NLM/ HPLI BDE. NLMlibraries and finally theHPBRAND. NLMlibrary.HPLI BDC. NLMand HPLI BDE. NLMlibraries are relatively small and should not take up too muchserver resources or affect server performance in any way.

Note that library NLMs can cause some unique problems. For example, if a library NLM allocatesresources (such as memory) on behalf of another NLM, the library NLM needs to know when thatanother NLM is unloaded or otherwise terminated so that it can free the allocated resources.Refer to Novell NetWare development documentation for more information on library NLMsproblems.

Encoding/Compression Library Templates

Source code templates for encoding/compression libraries are provided here. These examplesprovide instructions on how to build and create encoding/compression libraries on HP-UXsystems. On Windows systems, data encoding/compression libraries are ordinary DLL files.

These libraries must support Unicode; in other words, your project settings should contain_UNI CODE preprocessor definition. For more information about Unicode features,TCHAR typeand_T( ) macro, check MSDN documentation. For any other platform, check compiler/linkerdocumentation.

Encoding

/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -| encode. h|| I nt er f ace decl ar at i on- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /

#i f _WI N32__decl spec( dl l expor t )unsi gned i nt DeI ni t ( TCHAR *) ;#el seunsi gned i nt DeI ni t ( char *) ;

#endi f

#i f _WI N32__decl spec( dl l expor t )#endi fi nt DeEncode( unsi gned char *dat a, unsi gned l ong si ze) ;

#i f _WI N32

17

__decl spec( dl l expor t )


18/28

#endi fi nt DeDecode(unsi gned char *data, unsi gned l ong si ze) ;

#i f _WI N32__decl spec( dl l expor t )#endi fi nt DeExi t ( voi d) ;

/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -| encode. c|| DP Encode/ Compress Pl ug- I n Sampl e Code| The sampl e code can be used onl y as an exampl e on how t o wr i t e Dat a| Pr ot ect or Encodi ng/ Compr essi on l i br ari es. You shoul d cust omi ze i t| based on your r equi r ement s and exi st i ng code base.- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /

/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -| Compi l e Your code:

| cc +DApor t abl e - c +z encode. c|| Cr eat e l i br ar y| l d - b - o l i bdemo. sl encode. o- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /

#i ncl ude #i ncl ude #i ncl ude

#i f def _WI N32#i ncl ude #i ncl ude #el se

#i ncl ude #endi f

unsi gned i nt EncodeType=114;

#i ncl ude "encode. h"

/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -| Cur r ent i mpl ement at i on of Data Pr otect or Encodi ng/ Compr essi on pl ug- i n| al l ows st r i ng i dent i f i cat i on of encodi ng/ compr essi on l i br ar i es as a

| message di spl ayed i n t he Data Pr otect or consol e.| Not e t hat t hi s i dent i f i cat i on i s j ust an addi t i on t o t he encode t ype,| whi ch must be r et ur ned by t he DeI ni t ( ) cal l and t her ef ore cannot be| used i nst ead of encodi ng type.- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /

/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -| Let ' s use f eat ur e ment i oned above to pr ovi de user f r i endl y| associ at i on of l i br ar i es we use.

18

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /


19/28

#i f def _WI N32const TCHAR *e_l i bI d =_T( "Encode Demo - Ver 1. 2") ;

#el seconst char *e_l i bI d="Encode Demo - Ver 1. 2";

#endi f

/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - LI BRARY I MPLEMENTATI ON - - - - - - - - - - -- - - - - - - - - - - - - - - - * /

unsi gned i nt DeI ni t (#i f def _WI N32

TCHAR *l i bI d#el se

char * l i bI d#endi f

)

{/ * Per f or m any ki nd of i ni t i al i zat i on her e*/

/ *Use pr ovi ded buf f er t o put ext r a i dent i f i cat i on i nf or mat i on.These i nf or mat i on wi l l be di spl ayed on DP consol e*/

#i f def _WI N32wcscpy( l i bI d, e_l i bI d) ;

#el ses t rcpy( l i bI d, e_ l i bI d) ;

#endi f/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -| I t i s assumed t hat l i br ar y wi l l r et ur n uni que i dent i f i er of t he used| encodi ng al gor i t hm upon successf ul i ni t i al i zat i on. So, do not f or get| to do i t ! !- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /

r etur n( EncodeType) ;}

i nt DeEncode( unsi gned char *dat a, unsi gned l ong si ze) {/ *Ther e i s pl ace wher e you shoul d perf orm data encodi ng.

Note t hat data si ze must not be changed ( Check API Sect i on) */

return(1);}

i nt DeDecode(unsi gned char *data, unsi gned l ong si ze) {/ *Ther e i s pl ace wher e you shoul d perf orm data decodi ng.

Note that data si ze must not be changed ( Check API Sect i on) . */

return(1);

19

}


20/28

i nt DeExi t ( voi d) {/ *Thi s i s t he pl ace wher e you shoul d per f orm any cl eanup*/

return(0);}

Compression

/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -| compr ess. h|| I nt er f ace decl ar at i on- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /

t ypedef i nt ( *dcCal l back) ( unsi gned char *, unsi gned l ong ) ;

#i f def _WI N32__decl spec( dl l expor t )unsi gned i nt DcI ni t ( TCHAR *) ;#el seunsi gned i nt DcI ni t ( char *) ;#endi f

#i f def _WI N32__decl spec( dl l expor t )#endi fi nt DcCompr ess( dcCal l back r eadCal l back, dcCal l back wr i t eCal l back) ;

#i f def _WI N32__decl spec( dl l expor t )#endi fi nt DcDecompr ess( dcCal l back readCal l back, dcCal l back wr i t eCal l back) ;

#i f def _WI N32__decl spec( dl l expor t )#endi fi nt DcExi t ( voi d) ;

/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -| compr ess. c|| DP Encode/ Compress Pl ug- I n Sampl e Code| The sampl e code can be used onl y as an exampl e on how t o wr i t e Dat a| Pr ot ect or Encodi ng/ Compr essi on l i br ari es. You shoul d cust omi ze i t| based on your r equi r ement s and exi st i ng code base.- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /

20


21/28

/ *- - - - - - - - - - - - - HPUX Bui l d I nst r uct i on- - - - - - - - - - - - - - - - - - - - - - -| Compi l e Your code:| cc +DApor t abl e - c +z compress. c|| Cr eat e l i br ar y| l d - b - o l i bdemo. sl compr ess. o- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /

#i ncl ude #i ncl ude #i ncl ude

#i f def _WI N32#i ncl ude #i ncl ude

#el se#i ncl ude

#endi f

#i ncl ude "compress. h"

/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -| Cur r ent i mpl ement at i on of Data Pr otect or Encodi ng/ Compr essi on pl ug- i nal l ows st r i ng i dent i f i cat i on of| encodi ng/ compr essi on l i br ar i es as a message di spl ayed i n the DataPr ot ect or consol e.| Not e t hat t hi s i dent i f i cat i on i s j ust an addi t i on t o the compr essi ont ype, whi ch must be| r et ur ned by DcI ni t ( ) cal l and t her ef or e cannot be used i nst ead ofcompressi on t ype.- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /

/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -| Let ' s use t he f eature ment i oned above t o pr ovi de user f r i endl y| associ at i on of l i br ar i es we use- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /

#i f def _WI N32const TCHAR *_l i bI d = _T( "Compressi on Demo - Ver 0. 5") ;

#el seconst char *_ l i bI d = "Compr essi on Demo - Ver 0. 5";

#endi f

/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -| Every compressi on al gor i t hm shoul d be r epr esent ed by a uni que number

i n t he al l owed r ange. Check| API sect i on of Encodi ng/ compr essi on Whi t epaper f or i nf ormat i on aboutal l owed val ues.- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /

unsi gned i nt const CompressType=75;

/ * - - - - - - - - - - - - - - - - - - - - - - - LI BRARY I MPLEMENTATI ON - - - - - - - - - * /

21


22/28

unsi gned i nt DcI ni t (#i f def _WI N32

TCHAR *l i bI d#el se

char * l i bI d#endi f

){/ * Per f or m any ki nd of i ni t i al i zat i on necessar y t o Your l i br ar y*/

/ *Use pr ovi ded buf f er t o put ext r a i dent i f i cat i on i nf or mat i on.These i nf or mat i on wi l l be di spl ayed i n t he Dat a Pr ot ect or

consol e*/#i f def _WI N32

wcscpy( l i bI d, _ l i bI d) ;#el se

s t rcpy( l i bI d, _ l i bI d) ;#endi f

/ * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -| I t i s assumed t hat l i br ar y wi l l r et ur n uni que i dent i f i er of t he used| compr essi on al gor i t hm upon successf ul i ni t i al i zat i on. So, do not| f orget t o do i t ! !- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * /

r et urn(CompressType) ;}

i nt DcCompr ess( dcCal l back r eadCal l back, dcCal l back wr i t eCal l back) {unsi gned char *Dat aBuf f er=NULL;unsi gned l ong ReadSi ze=1024;unsi gned l ong Read=0;unsi gned l ong Wr i t eSi ze=0;unsi gned l ong Wr i t t en=0;

i f ( NULL == ( DataBuf f er=( unsi gned char *) mal l oc( ReadSi ze *si zeof ( unsi gned char ) ) ) ) {

return(0);}

do {Read=r eadCal l back(Dat aBuf f er, ReadSi ze) ;i f ( 0 == Read) {

/ *Handl e possi bl e r ead er r or s*/break;

}/ *I t i s t i me t o compr ess dat a. Thi s i s t he pl ace wher e you

shoul d t o i t */

22

/ *Don' t f or get er r or handl i ng*/


23/28

/ *. . . Some compr essi on code or cal l t o f unct i on per f or mi ngcompressi on*/

/ *Wr i t e data back t o Data Pr otect or st r eam*/Wr i t eSi ze=Read;i f ( 0 ! = Wr i t eSi ze) {

Wr i t t en=wr i t eCal l back(DataBuf f er , Wr i t eSi ze) ;i f ( 0 == Wr i t t en) {

/ *Handl e possi bl e wr i t e er r or s*/

return(0);}

} / *end wr i t e data*/}whi l e(0 ! = Read) ;

i f ( NULL ! = Dat aBuf f er ) { / * Rel ease buf f er * /f r ee( Dat aBuf f er ) ;

}

return(1);}

i nt DcDecompr ess( dcCal l back readCal l back, dcCal l back wr i t eCal l back) {unsi gned char *Dat aBuf f er=NULL;unsi gned l ong ReadSi ze=1024;unsi gned l ong Read=0;unsi gned l ong Wr i t eSi ze=0;unsi gned l ong Wr i t t en=0;

i f ( NULL == ( DataBuf f er=( unsi gned char *) mal l oc( ReadSi ze *si zeof ( unsi gned char ) ) ) ) {

return(0);}

do {Read=r eadCal l back(Dat aBuf f er, ReadSi ze) ;/ *I t i s t i me t o decompr ess dat a. Thi s i s t he pl ace wher e

you shoul d t o i t *// *. . . Some compr essi on code or cal l t o f unct i on per f or mi ng

compressi on*// *Don' t f or get er r or handl i ng*/

/ *Wr i t e data back t o Data Pr otecot r st r eam*/Wr i t eSi ze=Read;i f ( 0 ! = Wr i t eSi ze) {

Wr i t t en=wr i t eCal l back(DataBuf f er , Wr i t eSi ze) ;i f ( 0 == Wr i t t en) {

return(0);}

} / *end wr i t e data*/}whi l e(0 ! = Read) ;

23


24/28

i f ( NULL ! = Dat aBuf f er ) { / * Rel ease buf f er */

f r ee( Dat aBuf f er ) ;}return(1);

}

i nt DcExi t ( voi d) {/ *Thi s i s t he pl ace wher e you shoul d per f orm any cl eanup*/

r et ur n 0;}

Useful tips to test your library

After you have created the library, it is recommended to test it:1. Configure the newly created library as described in the previous chapter.2. Start few backup/restore sessions on different files, for example regular and null-content

files.3. Change the size of these files and try to perform backup and restore.

Your libraries should be capable of handling any buffer size passed to functions performingencoding/decoding; including 1 byte.

24


25/28

Windows Disaster Recovery

Data Protector uses a collection of files packed in two (32-bit systems) or three (64-bit systems)disaster recovery cab files to perform a disaster recovery of Windows systems. The defaultencrypting and compressing libraries are also included in the disaster recovery cab file. Thismeans that if a different encrypting/compressing library was used for backup, all files restoredduring the disaster recovery will not be decrypted/decompressed (because the defaultencrypting/compressing library will be used to decrypt/decompress the data) and thereforeuseless. In order to be able to successfully perform disaster recovery on Windows, you will haveto replace the default Data Protector encrypting (l i bde. dl l ) or compressing (l i bdc. dl l )library in the disaster recovery cab file with your own custom library with the same filename as theoriginal encrypting/compressing library.

You can use the Data Protectoromni cab. exe utility (stored in\ bi n\ ut i l ns on a Windows Cell Manager) to unpack and pack thedisaster recovery cab file. The Omni cab. exe utility is not installed on 64-bit systems by default;but you can copy it along with the omni cab_ns. i ni file from a 32-bit system or Data Protectorinstallation medium (it is compressed in the \ i 386\ aut odr . cab file).

When you are preparing a disaster recovery image (DR diskettes for Assisted Manual DisasterRecovery, ISO CD image for Enhanced Disaster Recovery, OBDR tape for One Button DisasterRecovery or ASR set for Automated System Recovery), Data Protector uses the disasterrecovery cab files stored on the local machine to prepare the disaster recovery image. Disasterrecovery cab files are stored in:

On 32- bi t syst ems :\ Depot \ Dr set up\ di sk1 and

\ Depot \ Dr set up\ di sk2 On 64- bi t syst ems:

\ Depot \ Dr set up64\ di sk1,\ Depot \ Dr set up64\ di sk2 and\ Depot \ Dr set up64\ di sk3.

You have to replace the encrypting/compressing library only on those systems where the disasterrecovery images are prepared, because Data Protector will always copy the disaster recoverycab files from the local system when preparing an image. You have to replace theencrypting/compressing library only once, because Data Protector always uses the same disasterrecovery cab file from the local system to prepare a disaster recovery image.

Perform the following steps to replace the encrypting/compressing library in the disaster recoverycab file:

1. In the Windows command prompt, type:set DPRoot = . Example: set DPRoot =c: \ pr ogr am f i l es\ omni back.

2. Type set I nst Pat h= where determines where the contents of the disaster recovery cab files are to be extracted.Example: set I nst Pat h=c: \ t emp.

3. Type set srcPath=. The argumentdetermines the location of the files that are to be compressed in the disaster recovery cabfile. This should be the same location as for decompressing files.

25


26/28

4. Type omni cab. exe - pat h - decompress to decompress thecontent of the disaster recovery cab file to the specified instep 2. The argument is the location of the omni cab_ns. i ni file.Example: omni cab. exe - path "c: \ pr ogr amf i l es\ omni back\ bi n\ ut i l ns\ omni cab_ns. i ni " decompr ess.

5. Replace the libraries in .

6. Type omni cab. exe - pat h


27/28

Questions and Answers

Q: How to perform version management in order to use different encoding/compressionalgorithms?

A: Values returned by the DeI ni t ( ) / DcI ni t ( ) calls are used to uniquely identifyencoding/compression algorithms. At the same time, user libraries can use buffer passed tothe DeI ni t ( ) / DcI ni t ( ) to put extra information on used encoding/compression algorithms.Note that this information is shown only in the Data Protector console.Here is an example of using 3 different encoding algorithms:1. Define 3 unique values in the allowed range, which will be returned by the DeI ni t ( ) .

Each of these unique values should represent a different encoding algorithm.2. For each encoding algorithm, create a corresponding library as described in chapter

Creating Libraries.

3. Depending on which encoding is being used, configure the appropriate library asdescribed in chapter Configuration.4. For each additional algorithm you are planning to use, configure the appropriate library as

described in chapter Configuration.

The same procedure also applies to compression algorithms.

Q: Are sparse files supported?A: No.

Q: Does Data Protector encoding/compression plug-in handle null-content files properly?A: Yes.

Q: How does encoding/compression plug-in handle multistream files?

A: Windows NTFS file system supports files with alternate data streams. Data Protector A.06.00also supports files with alternate data streams, which means that a multistream file is backedup and restored together with its alternate data streams. Although multistream files aresupported, alternate data streams are not compressed or encoded since they are treated asfile attributes. During a backup session, when compression and encoding options areselected, only the main stream, which contains files data, is compressed and encoded.

Q: Can both encoding and compression be performed together?A: Yes. Encoding and compression libraries are independent. If both are used at the same time,

encoding and compression will be performed.

Q: How does OB2ENCODE omni r c variable affect custom encoding/compression?A: IfOB2ENCODE is specified, it will override the Encode option in GUI (- encode option in CLI).

The encoding/decoding algorithm used is determined by the library, which is installed on theclient being backed up or restored. You must set the OB2ENCODE omni r c variable on theCell Manager system, in order to control the behavior in this way.

27


28/28

Appendix 1 Terms and Definitions

DA Disk AgentDE Data EncodingDC Data Compression

28

Hp Man DP7.00 EncodingCompress Wp PDF

Documents

Transcript of Hp Man DP7.00 EncodingCompress Wp PDF