January 1993 Computer Audit Update

IT SECURITY TESTING, A P R A C T I C A L GUIDE m PART 3

HARDWARE/SOFTWARE SECURITY TESTING

Bernard Robertson and David Pullen PA Consulting Group

Introduction

This article, the third in a series on the subject of security testing, describes how to go about the security testing of IT hardware and software systems. Examples of the types of tests that may be performed are given.

Hardware/software security testing involves determining how well a component resists attempts to subvert its security mechanisms. Both positive and negative testing (defined in the first article of this series) can be applied in hardware/software.

Scope

Only components (hardware devices or software modules) which could have an impact on the overall system security should be subject to security testing. As test resources are usually limited, test ing effort may be focused by categorizing the systems to be tested. A specimen categorization is components with security controls:

which, if defective, could put the entire system at risk e.g. the access control system of a hardware security module

protecting sensitive security data which, if defective, could compromise confidential in- formation within the system, e.g. tamper re- sistant mechanisms within an encryption device

which protect a specific data type or applica- tion function and which, if defective, could allow a specific type of fraud to occur, e.g. control on financial authorization limits.

Examples of the types of components on which security testing may be performed include:

• A PC security card used for access control purposes.

A terminal used for the entry of confidential data where there may be a concern about the level of electromagnetic emissions.

• An access control software application.

• A cryptographic device used to provide link encryption.

• Asoftware application handling classified, fin- ancially sensitive or safety critical information.

Test process

The general process to be followed when performing hardware or software security testing is detailed in a previous article: 'The Security Testing Process'. In summary the key steps are:

• System familiarization, which includes athor- ough documentation review (see below).

• Business and security control identification.

• Risk identification.

• Selection of appropriate testing techniques.

• Formal structured tests.

• Attack tests.

• Problem classification and reporting.

©1993 Elsevier Science Publishers Ltd 7

Computer Audit Update January 1993

The following sections describe stages in the test process.

Documentation review

A thorough rev iew of the system documentation needs to be performed as early in the test cycle as possible. The three primary purposes of this review are:

• To assist in the system familiarization pro- cess.

To determine the quality of the documenta- tion. This will give the tester an indication of the overall quality of the system. Experience indicates that a system with poor quality do- cumentation is likely to have serious security problems.

To compare the currency of the documenta- tion with the actual system. If updates have been made to the system without amend- ments to the documentation, poor change control procedures may be in place for both the system and the documentation.

Relevant documentation would include (depending .on the nature of the system under test):

• functional requirements specification,

• design specification,

• circuit diagrams,

• process flow charts,

• dataentity descriptions,

• memory management structures.

A section in the final test report should describe the results of the documentation review.

Types and examples of tests

A wide variety of positive and negative testing techniques may be used and typical examples are given for both hardware and software in the following section. This is described firstly for hardware and then for software.

Hardware security tests

Checks on the accurate implementation of specified functions

For example if the module under test utilizes a cryptographic processor, tests should be performed to check that the algorithm is implemented correctly. This may be done by comparing the output of the processor with that from the same algorithm implemented on a test tool. A key test to perform when a cryptographic process is used is to determine what happens to the data that is normally encrypted when the processor is removed or bypassed. Is sensitive data now sent in clear form? Another test should be performed to determine how the absence or failure of the cryptographic processor is reported to the system control - - do diagnostic systems work? Other tests to ensure that generated keys are suitably random may be performed by statistical analysis.

Test tamper resistant mechanisms

The key areas to evaluate are:

• The strength of the tamper resistance relative to the risks the unit faces.

• The ability of the unit to actively destroy all the secret information in a timely fashion.

This type of testing can take considerable amounts of time and resource so for reasons of efficiency the following process should be followed:

• Fully understand the function and identify the key protective controls.

8 ©1993 Elsevier Science Publishers Ltd

January 1993 Computer Audit Update

Examine all design drawings and documents to evaluate the intended response to threats and identify any vulnerabilities (such as chemical, X-ray or intrusion attacks).

Confirm the existence of the vulnerabi l i t ies- this may be done by a paper analysis or by live tests.

Perform some unusual (negative) attacks on the unit to determine how it reacts. For example dropping, heating or cooling the unit with the objective of making the detection mechanisms fail without deleting the secret information.

Evaluate hardware reliability and resilience

This test area investigates the reaction of the system to a partial or total failure. The optimal system reaction would be to fail safe. For example a test could be to determine what happens when the disks used for audit trail logging become full or when the system is brought down and then restored to service. If the audit logging is lost in either case a major security breach could remain undetected.

Test environmental security

Tests should be performed to determine the effect on the component of extremes of tempera tu re , humid i t y and magnet ic interference. These tests should identify whether any security weaknesses are exposed when the module fails due to any of these factors. The tests should be based on the env i ronmenta l performance specification of the device and test the reaction of the module to conditions beyond the specified boundaries. A full environmental test chamber may be required for these tests.

Power and speed variation

In these tests the modules power inputs or clock speed are varied. Tests should be per formed to determine if any secur i ty

weaknesses arise when the power inputs or clock speed are varied outside their specified bands.

Change in the hardware configuration

Tests should be performed to examine the modules security state when hardware elements are modified or removed. For example removing an access control board from the terminal it was protecting or removing the battery used to power the key storage area in a cryptographic processor.

Device authentication security

If it is easy to make unauthorized duplicates of the security hardware it may be possible to masquerade as a legitimate unit and perpetrate a fraud. A safeguard against this line of attack is to perform some form of secure device authentication (normally a challenge response type protocol). Tests should be performed to check that the safeguards function as specified. In addition design effort may be spent trying to make the unit hard to duplicate - - perhaps by the use of embossed designs or holograms. Examples of uni ts that are targets for unauthorized duplication are:

Access control hardware units and cash dis- pensing machines (where the objective is to collect PIN and card data).

Terminals on a LAN where an attacker may read all sensitive information such as pass- words on the LAN.

Failures in diagnostic function

Tests should be performed to test the response of the module to failures in the self-test diagnostic functions. The tests should determine whether the failures are reported and whether any security weaknesses arise. For example, confidential information may be transmitted in clear text if the diagnostics fail to report that the cryptographic processor in a link encryptor is out of service.

@1993 Elsevier Science Publishers Ltd 9

Computer Audit Update January 1993

Software security testing

Static analysis

Static analysis tools provide the software security tester with information on the functioning of software modules while they are inactive. Static analysis tools include disassemblers or decompilers which may be used to access the object code and convert it to its source code form. These tools would enable the tester to read confidential information (such as encryption keys) stored in the source code.

Dynamic analysis

Dynamic analysis tools provide information on the software whilst it is running. Examples of dynamic tools include debuggers and code exercisers. A debugger can allow a tester to determine intermediate values (perhaps during a key generation cycle) while a code exerciser can be used to identify areas of infrequently used code that may contain unauthorized functionality. Such areas of code should be examined in more depth by code analysis.

Code analysis

Code analysis involves an in-depth review of each line of code in the areas selected for testing. The objective of code analysis is to detect any code of dubious security or purpose. It should be undertaken by someone who is fluent in the programming language used to write the application. In performing code analysis it is important to focus attention on the key functions to be analysed. If this is not done the scale of the test will be too large and the level of detailed analysis required will not be achieved. Areas which the tester should focus on are: access control modules; sensitive record processing; and financial limit sub-routines.

When a doubtful area of code is found the quickest way to check its intended purpose is to ask the programmer. However, the reply supplied should not be accepted without question, it should be verified to the complete satisfaction of the tester. Comments in the code should be viewed with scepticism as any fraudulent piece of code would not broadcast its function.

Use of terminate and stay resident (TSR) programs.

A TSR program is normally active at all times and can allow authorized activities to take place every time an application is run. Tests should be undertaken to determine if a TSR can be run simultaneously with the application under test. For example, a banking application accessed by PC-based terminals could be subverted by a TSR. The TSR could send its own commands from an otherwise legitimate terminal address; these commands would be invisible to the terminal user and it could transfer funds fraudulently. The best defence against this type of attack is for the application program to inhibit the TSR by removing all unrequired applications from primary storage as its first activity.

Attempting to access the operating system by abnormal exits from the application

Most security applications try and prevent users accessing operating system functions. However, in many applications, it is relatively easy to cause the application to terminate abnormally and return the user to the operating system. An abnormal exit can occur if fields are caused to overflow or if particular keystroke combinations are entered. An example is the entry of a ControI-AIt-Del string in a DOS application. Once in the operating system, system file, password files and system trace tools become available and may be used to corrupt data or deny access to authorized applications.

Control of source code

The software security tester should ascertain how securely copies of the software source code are stored. Insecurely stored software allows an attacker to:

Modify the software before it is introduced into the operational environment. The unauth- orized introduction of software versions into the production environment should be de- tected by change control procedures.

1 (3 (~1 ~cl.q I = l ~ v i A r .qni~nnA P= =h l i~hAr¢ I t r l

January 1993 Computer Audit Update

• Examine source code listings to find weak- nesses that could later be exploited.

Invalid data

Tests should be performed to exercise all of the data validation controls within the application. For example, attempting to enter illegal values, perhaps alphabetic characters in a field for a te lephone number, or entering too many characters in a field for a financial amount to determine how the system reacts to an overflow in this field.

File manipulation

It may be possible to subvert the processing of an application if data or source files are manipulated. Tests should determine if it is possible to circumvent controls on failed password attempts by manipulating the file containing this count with a file edit utility.

Complex interaction tests

Interaction tests should be performed when a comp lex comb ina t i on of app l i ca t ion env i ronmen ts , ope ra t i ng sys tems and communicat ion protocols exist around a business application. These tests can take the form of modifying avalue supplied by a base layer to an application layer and monitoring the response of the application. For example, if an application receives encryption services from the operating system, the reaction of the application to a key management failure should be tested.

Error conditions

A valuable testing technique is to try and recreate all possible error conditions. Tests which generate the error conditions should create all foreseen problems and exercise all the error handling code. The tester should set about creating error condit ions so that all error messages listed in the system documentation are obtained. Once this has been achieved testing may continue by creating other unspecified error conditions and monitoring the response.

This article has described hardware and software testing and given some examples of the types of tests that may be performed. The next article in this series describes the security testing of communication channels.

Bernard Robertson is a Principal Consultant in the Security Consult ing Practice of PA Consulting Group. He has extensive experience in performing a range of secur i ty testing programmes for the public and financial sector clients. Bernard is a regular speaker on IT security issues and holds degrees in Economics and Business Administration. David Pullen is a Senior Consultant within the same Security Consulting Practice. Over the last five years he has conducted several security testing projects, including one lasting two years with a team of 15 security testers. David is a physics graduate and a qualified teacher who has produced a wide range of educational material on security testing.

AUDITING THE CONTINGENCY PLAN

David Firth

A contingency plan, by its very nature, must work if ever it has to be implemented. If the plan fails to meet the recovery targets or provide the expected service levels, the results may be catastrophic.

Ask yourself honestly whether your own plan will be effective if your pager alerted you today that there had been a fire or bomb at your data centre or key offices. Despite the time and effort put into contingency or business continuity planning it is likely that only a minority could put their hand on their heart and answer 'yes'. Others may find cause for concern in that their plans have not been kept up-to-date or tested effectively.

The development of a contingency plan represents a major investment for most organizations. The costs of the backup site, the

©1993 Elsevier Science Publishers Ltd 11