- i -

2006-09-25 DRAFT 8.9

=WORK IN PROCESS – FOR INTERNAL DISCUSSION ONLY=

DESIGNING THE HOUSE OF SECURITY: STAKEHOLDER PERCEPTIONS OF SECURITY ASSESSMENT AND IMPORTANCE

EXECUTIVE SUMMARY ...............................................................................................IV

1. INTRODUCTION......................................................................................................... 1

2. LITERATURE REVIEW ............................................................................................. 2

2.1 Factors Affecting Security Perceptions................................................................................................................2

2.2 Stakeholders of Information Security..................................................................................................................2

2.3 Development of Constructs of Information Security..........................................................................................3

2.4 Literature Sources for Information Security Constructs...................................................................................4 2.4.1 Information Security Evolution and Information Flow Perspective ................................................................5 2.4.2 Industry Standards ...........................................................................................................................................6 2.4.3 The McCumber Cube.......................................................................................................................................7

3. SECURITY CONSTRUCTS FORMING THE HOUSE OF SECURITY........................ 8

3.1 House of Security ...................................................................................................................................................8

3.2 Security Constructs ...............................................................................................................................................9 3.2.1 Vulnerability ....................................................................................................................................................9 3.2.2 Accessibility...................................................................................................................................................10 3.2.3 Confidentiality ...............................................................................................................................................10 3.2.4 Technology Resources for Security ...............................................................................................................10 3.2.5 Financial Resources for Security ...................................................................................................................10 3.2.6 Business Strategy for Security .......................................................................................................................10 3.2.7 Security Policy and Procedures......................................................................................................................10 3.2.8 Security Culture .............................................................................................................................................10

3.3 Extended Enterprise Security.............................................................................................................................11

4. METHODOLOGY..................................................................................................... 11

4.1 Survey Construction and Implementation ........................................................................................................11 4.1.1 Demographic Section.....................................................................................................................................11 4.1.2 Assessment and Importance Section..............................................................................................................11

4.2 Problems Encountered and Addressed in Survey.............................................................................................13 4.2.1 Parallelism .....................................................................................................................................................13 4.2.2 Multiple Aspects of the Same Construct........................................................................................................13

- ii -

4.2.3 All Components Worded in Positive Form....................................................................................................13 4.2.4 Inherent Skewing of Results ..........................................................................................................................13

4.3 Survey Reliability and Validity ..........................................................................................................................13 4.3.1 Construct Reliability - Cronbach’s Alpha Test ..............................................................................................14 4.3.2 Construct Validity..........................................................................................................................................16

5. PRELIMINARY FINDINGS ....................................................................................... 16

5.1 Secured Web-Based Survey Instrument............................................................................................................16

5.2 Analysis Tools ......................................................................................................................................................17

5.3 Gap Analysis ........................................................................................................................................................17 5.3.1 Statistical Significance of Gaps .....................................................................................................................18 5.3.2 Some Examples of Gap Results .....................................................................................................................19 5.3.3 Significance of Gaps for Comprehensive Data ..............................................................................................20 5.3.4 Methodology for Regrouping Comprehensive Data .....................................................................................22

5.4 Survey Reliability and Validity Analysis ...........................................................................................................23 5.4.1 Reliability Analysis........................................................................................................................................23 5.4.2 Validity Analysis – Convergent & Discriminant Validity .............................................................................23 5.4.3 Survey Reliability and Validity Concluding Comments................................................................................24

6. PRELIMINARY RESULTS........................................................................................ 24

6.1 Individual Questions............................................................................................................................................24

6.2 Constructs ............................................................................................................................................................25 6.2.1 Construct gap .................................................................................................................................................26 6.2.2 Standard Deviation.........................................................................................................................................29

6.3 Perceptions by Industry ......................................................................................................................................30 6.3.1 Construct gap .................................................................................................................................................32

6.4 Perceptions by Role .............................................................................................................................................34 6.4.1 Construct gap .................................................................................................................................................36

6.5 Perceptions by Area.............................................................................................................................................38

7. CONCLUSIONS........................................................................................................ 39

7. BIBLIOGRAPHY....................................................................................................... 41

APPENDIX I – SURVEY INSTRUMENT....................................................................... 43

APPENDIX II – LIST OF COMPONENTS FOR EACH CONSTRUCT AND MA VS. MI GAP ANALYSIS ........................................................................................................... 48

APPENDIX III - SAS CODE FOR THE CALCULATIONS ............................................ 50

- iii -

APPENDIX IV - CRONBACH’S ALPHA ANALYSIS.................................................... 53

APPENDIX V – METHODOLOGY FOR REGROUPING MERGED DATA................... 54

APPENDIX VI – MISCELLANIOUS.............................................................................. 58

- iv -

Technology Resources

For Security

Financial Resources

For Security

Business Strategy

For Security

Security Policy &

Procedures

Security Culture

Accessibility Confidentiality

Vulnerability

The Eight Constructs Organized as the House of Security

DESIGNING THE HOUSE OF SECURITY: STAKEHOLDER PERCEPTIONS OF

SECURITY ASSESSMENT AND IMPORTANCE

Executive Summary Introduction Security is crucial for the success of any organization. Unauthorized users frequently steal information while hackers constantly disrupt flows of information. In response, organizations have adopted new security policies. It is clear that many of these security policies are valuable; however an organization may be limited in how much of its resources it can devote to protecting its flows of information. Security costs can be incurred monetarily (e.g., the price of a new firewall) or non-monetarily (e.g., requiring employees to use convoluted passwords or confusing software-protection programs). An organization’s goal should be to develop the most appropriate approach to security (i.e., a balance between cost and effectiveness). This is further complicated by the fact that there are likely to be different priorities for the various stakeholders in the organization. Furthermore, as organizations evolve towards becoming extended enterprises, including close ties with suppliers, customers, and other partners, there will be a significant increase in the number of stakeholders and thus a wider range of security requirements. Purpose of Study Many scholars have approached the study of security by focusing specifically on the detailed elements of the security systems themselves, such as effectiveness of different cryptographic codes or firewall technologies, or have measured specific events, such as mean-time-to-failure. However, these works do not look at security holistically and commonly neglect to consider the members of the organization themselves. They especially neglect to consider the perceived needs and security views of an organization’s members. In this project, we take a different approach. We seek to identify the commonalities and differences both within and between different organizations with respect to perceptions of security held by different members of the organization. In order to accomplish this, there are three major objectives: • To identify how perceptions both shape and should shape decisions in investments in security systems, with a particular focus on identifying the most important constructs of security, as perceived by the individuals in the organization. • To identify differences between the importance and assessment of the various security constructs among different organizational systems (e.g., comparing two different companies). • To identify differences between the importance and assessment of the various security constructs among organizational systems (e.g., comparing the views of mid-level managers to that of the senior management). The House of Security: Analysis Methodology Through a comprehensive literature review, web searches, and several surveys, researchers at MIT have identified about 300 security issues. These security issues were found to be grouped primarily into eight meta-groupings, or constructs, as follows: Good Security provides Accessibility to data and networks to appropriate users while simultaneously protecting Confidentiality of data and minimizing Vulnerabilities to attacks and threats. Good Security Practice goes beyond technical IT solutions. It is driven by a Business Strategy with associated Security Policies

- v -

and Procedures implemented in a Culture of Security. These are practices are supported by IT Resources and Financial Resources dedicated to Security. These eight constructs form our House of Security. The best tool for identifying variations in perceptions of security is a survey, broadly distributed to an array of members of an organization (from employees to top-level managers across all functional areas), that addresses both their organization and the extended enterprise. In our survey, respondents are asked to rate a series of statements about their perception of security, specifically:

(1) the current state of that security issue within their organization; (2) the important of that security issue for their organization; (3) the current state of that security issue for a partner organization; and (4) the importance of that security issue for the same partner organization.

A key part of this study involves performing gap analyses (e.g., how much does the perception of the current state of that security issue in the organization differ from the perception of the importance of that security issue.) Such gaps represent opportunities for improvement and better understanding within the enterprise and across the extended enterprise. When current status is below the ideal, these represent areas for possible improvement. When there are differences in status or gaps perceived among different stakeholders, these represent areas for investigating sources of the differences: the gaps may represent misunderstandings or the gaps may represent differences in local knowledge and needs. While a key goal of this survey is to measure perceptions of the different constructs of security, we also want to understand the causes of these perception variations. For this reason, this survey also asks a series of “demographic” questions, such as the size of the organization and its industry. Finally, we evaluate the quality of the survey instrument by measuring the statistically significance of the questions and the constructs, the reliability of the constructs (by computing Cronbach Alphas) and the content, convergent and discriminant validity of the constructs. Preliminary Results In our pilot survey, interesting results have arisen in several categories: (1) the individual questions, (2) the constructs, and (3) the construct gaps. Some examples are presented here. (1) Individual Questions: Respondents were asked to assess whether “people in the organization were aware of good security practices.” They were then asked the importance of that issue in the organization. This was to be answered on a 7-point scale (where 1 means “true to a small extent” and 7 means “true to a large extent..”) The overall results are shown on the top line of the graph to the right. The current assessment (marked MA) is the left part of that line (in yellow) while the importance (marked MI) represents the entire line. The right part of the line (in blue in the top line) represents the gap. In this example, there was a large gap, statistically significant at the 99.99% level. This suggests that awareness of good security practices falls far short of what is perceived to be needed among the respondents. When comparing individual organizations, such as Company X and Company I, we also observed major differences in assessment, importance, and gap size. One of the goals of this research is to understand these differences.

4 5 6 7

Comp I

Comp W

Comp X

Misc.

Overall

MA Gap

MI

4 5 6 7

Comp I

Comp W

Comp X

Misc.

Overall

MA Gap

MI

- vi -

(2) Constructs: The questions in the survey are aggregated to form the eight constructs that constitute our House of Security. An example of the assessment (current situation) of all 8 constructs can be seen in the diagram to the right. We can see that for a given company, the assessment values are likely to differ for the eight constructs. Comparing companies, we can see both significant similarities and differences between Company X and Company I again. For example, these companies are very similar in their perceptions of “Accessibility” but very different in their perceptions of the state of “Security Policy.” (3) Construct gaps: Although viewing the actual values of each of the constructs provides some insights, it is often more interesting to examine the “gaps.” For example, one organization might have an assessment of “5”, but if it views that construct as only having an importance value of “5”, the gap would be zero and it might be content. Whereas, if another organization had an assessment of “6”, but viewed that construct importance as being “7”, that is a gap of 1 and might indicate an area for improvement. Some examples of these construct gaps are seen in the diagram to the right. Comparing companies X and I again, we observed some differences in gaps (which might be considered measures of discontent) in “Accessibility” but much bigger differences in gaps in “Security Culture.” Conclusion The security of information systems is vital to any organization. In order to identify security strategies and to identify cross-organizational trends, we analyze perceptions of importance and assessment along eight security constructs. In addition to being a unique way of considering the “security dilemma,” such an analysis will demonstrate the importance of considering perceptions and may shed some light on how perceptions shape decision-making in an organization. We believe the results from this work will have tremendous implications in a number of areas including assessing an organization’s security needs, marketing of security products, and the development of an organization’s security technology and policy. This paper focuses on the development and validation of the methodology used.

0.000

0.400

0.800

1.200

1.600Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

Company XCompany WCompany IOverall

0.000

0.400

0.800

1.200

1.600Accessibility

Vulnerability

Confidentiality0.000

0.400

0.800

1.200

1.600Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

Company XCompany WCompany IOverall

4.5

5

5.5

6

6.5 Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

Company X Company W Company I Overall

- 1 -

Designing the House of Security: Stakeholder Perceptions of

Security Assessment and Importance

Wee Horng ANG, Yang LEE, Stuart MADNICK, Dinsha MISTREE, Michael SIEGEL, Diane STRONG, Richard WANG

Abstract:

In this paper we introduce a methodology for analyzing differences regarding security perceptions within and between stakeholders, and the elements which affect these perceptions. Following a comprehensive discussion of theories pertaining to security, we design the “House of Security”, a security assessment model that provides the basic framework for considering eight different constructs of security: Vulnerability, Accessibility, Confidentiality, Technology Resources for Security, Financial Resources for Security, Business Strategy for Security, Security Policy and Procedures, and Security Culture. We designed and performed a survey to perform a gap analysis to uncover differences (1) between the different constructs and aspects of security, (2) between different enterprise stakeholder roles, and (3) between different organizations. This paper describes the development of the security constructs, the testing of the survey instruments for construct reliability and validity, and details some of the preliminary findings. Specific focus is on the evaluation of the quality of the survey instrument by measuring the statistically significance of the questions and the constructs, the reliability of the constructs (by computing Cronbach Alphas) and the content, convergent and discriminant validity of the constructs.

1. Introduction Providing effective information security is crucial for any organization’s success. There can

be significant costs and benefits involved. The costs can be direct, like installing a new firewall system, or can be more intrinsic, such as placing additional burdens on the organization. These burdens include requiring the use of convoluted passwords or other complicated procedures, impacting an individual’s level of productivity and morale. Gordon and Loeb showed that the optimal amount to spend on information security is an increasing function of an organization’s level of information vulnerability [1]. Assessing this level of vulnerability is an important step in developing an economically optimal approach for information security.

The goal of this project is to understand the nature of and perceptions regarding security within an organization as well as in extended enterprises, that is, multiple organizations working together. There are two tacit assumptions: (1) that there are different notions of security and (2) there can be different elements that affect these notions. The rest of the paper is organized to explain how we identify and assess these notions, which we will refer to as constructs of security.

We start by examining the relevant literature. Strikingly, we find that while there has been much theorizing and speculation about the nature of security in organizations, there has been a paucity of actual scientific research. Building on the existing literature and studies that we have conducted, we establish a House of Security—based on eight constructs of security. Through a survey instrument, gaps between security assessment and security importance among different individuals and across different organizations are identified, analyzed and compared.

- 2 -

2. Literature Review

2.1 Factors Affecting Security Perceptions Although we are addressing security in general, there is an emphasis on information security – we will use the terms information security and security interchangeably. For our purposes, we will consider information security to be the processes and policies which aid in the protection and rightful dissemination of information. We have found that most security research has either been very myopic or very broad. For instance, many studies focus on specific information security tools such as firewalls [2-4], encryption [5-7], or antivirus technology [8, 9]. On the other hand, there is another line of literature typified by sweeping claims about the critical importance of security with no real recourse for action. In “A Conceptual Foundation for Organizational Information Security Awareness,” Mikko T. Siponen typifies this corpus of literature by calling for increased awareness of security measures among users without tying this call for awareness to any specific recommendations or policy suggestions [10]. Between the overly myopic and the overly broad, little work has been conducted that bridges these two disparate viewpoints. In the domain of security there is a paucity of quantitative and empirical research, especially with respect to analyzing perceptions about security within and between organizations. Indeed, Sinclaire reviewed a range of current MIS journals and other research literature for the purpose of identifying the nature of recent study in information security and privacy and concluded that there is a dearth of research in the area of information security and privacy, particularly at the organizational level [11]. Kotulic indicated that a majority of firms are unwilling to divulge information security data without strong assurances that the information provided do not harm them. Also, the perception that such studies lacked insight for organization improvement is another reason cited for the lack of cooperation.[12].

One of the few empirical organizational studies conducted was by Straub and Welke [13] which focused on organization-level security planning models, rather than security perceptions. Interestingly, Straub and Welke concluded by noticing a disturbing trend in security analysis as a whole:

“The interview and action research findings offer empirical support for the propositions that measurable improvements can be made in these critical activities [of security]. While there is no doubt that many security consultants have excellent instincts with regard to what works and what does not, these empirical approaches help to raise the discussion above the level of folklore and into the realm of science. Additional work along these lines is very much needed, however” (Straub and Welke 1998; pg 461).

A online search confirms that even today, there has been little empirical work conducted on security analysis, affirming the continued dearth of empirical work in organizational information security. Thus, one of the objectives of our study is to fill this gap in empirical information security research.

2.2 Stakeholders of Information Security Every organization, especially large organizations, has various stakeholders – each with differing needs and perceptions. It is important to identify the various stakeholders related to information security. A broad view can be seen in Figure 1 (a), where information security stakeholders can be classified into three different subsets: Within the Enterprise, the Extended Enterprise and the General Public.

- 3 -

(a) Stakeholders of Extended Enterprise Security

EnterpriseEnterprise

General Public

Extended Enterprise

Security personnel (e.g., guard)

IT non-security personnel -------------------------IT security personnel

Business personnel

Workers

Security managers

IT non-security managers -------------------------IT security manager

Business unit manager

Line/middle manager

Top Security Mgt / CSO

Top IT Mgt/CIOCEO, CFO, …Top exec

Partners (Extended Enterprise)

General security/ physical security

IT OrganizationGeneral business

Level/Rank

Domain/Role

Security personnel (e.g., guard)

IT non-security personnel -------------------------IT security personnel

Business personnel

Workers

Security managers

IT non-security managers -------------------------IT security manager

Business unit manager

Line/middle manager

Top Security Mgt / CSO

Top IT Mgt/CIOCEO, CFO, …Top exec

Partners (Extended Enterprise)

General security/ physical security

IT OrganizationGeneral business

Level/Rank

Domain/Role

(b) Stakeholders & Roles

Figure 1: Information Security Stakeholders

The first subset of stakeholders, within the enterprise, can be further stratified based on the role and rank of the individual within the organization, as illustrated in Figure 1 (b). The stratification is classified into two dimensions. The first dimension is the domain or role: General Employee, IT-related Personnel, and General Security (i.e., non-IT) related, such as security guards. The second dimension is the level or rank: ranging from top executives, to line or middle managers, to professionals or other general workers in the organization.

The extended enterprise encompasses partner organizations involved in the organization’s daily functioning. These organizations can be directly supporting daily business operations, such as materials or services suppliers, or can have indirect organizational links, such as banks or financial companies. Most organizations have vested interests in the security status of their partners for many reasons, such as to ensure that their organization’s information security is not compromised.

The general public category includes customers of the company as well as government policy makers and regulators. The perceptions of this group also can impact how that organization functions and responds to potential and actual security threats.

Though all three subsets are important to ensure the strength of security, for the purposes of limiting the scope of our study, we shall only be primarily concerned with security perceptions within the organization and in the extended enterprise.

2.3 Development of Constructs of Information Security To assess security, it was necessary to determine the important constructs (sometimes called notions, concepts, factors, metrics, components etc.1) of information security. We wanted to understand how people conceptualized security, as well as their views and concepts associated with the term “holistic security” through the use of three successive surveys. The first survey was an open-ended set of questions, asking respondents for qualitative descriptions of “holistic security”. Recognizing that many of the concepts garnered from the first survey indicated the presence of

1 We actually use the terms aspects and components for specific purposes, as explained later in this paper.

- 4 -

general security ideas, the questionnaire was modified to incorporate input from the first survey. The respondents were asked to comment on those suggestions as well as provide additional ones. These next two surveys were much more structured and delved deeper into notions of “extended enterprise” and its relationship to security. In addition to these surveys, a search of the web was conducted to ascertain other security-related concepts. The combination of the surveys plus the web search revealed an extensive list of over 300 topics. Drawing upon this list and our own experiences, we identified eight important security constructs as shown in Figure 2 below.

Vulnerability

Accessibility

Confidentiality

Technology Resources for Security

Financial Resources for Security

Business Strategy for Security

Security Policy and Procedures

Security Culture Figure 2: Eight Security Constructs

Although these eight constructs arose from our studies, we will show how they also relate to prior research. We explain the literature sources of these constructs in the following sections.

2.4 Literature Sources for Information Security Constructs We examined the available literature for major concepts and ideas which also helped shape

our eight security constructs. A 1999 Information Security Magazine survey identified the key obstacles to a company achieving adequate information security, the principle obstacle cited was budgetary constraints. This was the perception of 29% of the survey respondents (See Figure 3). This shows that Financial Resources for Security is a key factor impacting information security within the organization.

Other major factors identified in that study included a lack of senior management support, which indicates the lack of a role of security in Business Strategies, lack of training and internal policies (Security Policies and Procedures) and technical complexity (Technology Resources for Security). These factors highlight several of the important areas with which information security should be assessed.

- 5 -

Figure 3 : From Information Security Magazine 1999 - "Top Obstacle is Budget: What is the SINGLE greatest obstacle to achieving adequate information security at your organization?"

In another survey by the University of Washington [14], respondents defined a secure Web in descending order of importance: Transit, Encryption and Remote Site. Transit refers to protecting the Confidentiality of information while it moves between machines on the Web. Encryption refers to specific mechanism of encoding and decoding information. Remote Site refers to protecting information once it has arrived at its destination. This survey once again portray the emphasis placed on the Confidentiality of information, as well as the importance of securitizing data as it flows between organizations. Issues such as Confidentiality, Security Policies, Business Strategies, Financial Resources for Security as well as Security in Information Flows are repeatedly mentioned in security perception surveys. This lends further support for the relevance and importance of our security constructs.

2.4.1 Information Security Evolution and Information Flow Perspective Within the information security literature there are two main developments that are pertinent to our study of perceptions. First is the concept of Information Security Evolution, how the field of Information Security has developed over the previous 30 years. Next is the concept of examining security from an Information Flow perspective. This states that since data is in a continuous state of flux, all data transits points must be equally secure. Within these two developments, we will attempt to highlight the key security constructs that are pertinent to our discussion of security perception. Evolution of Information Security

The development of information security can be described in three successive “waves:” Technical, Management, and Institutionalization Wave [15].

- 6 -

Technical Wave - Security Access Control Traditional computer security concepts deal with determining accessibility to data in a

database system, specifically which users have the authorization to read, write, or delete files and how such control is shared or cascaded down to other users. Access Control is any mechanism by which a system grants or revokes the right to access some data, or perform some action. This indicates the importance of Accessibility and Confidentiality of secured data, two key appraisal points of information security. Management Wave - Security Policy and Procedures

Along with the development of distributed computing, and in parallel with increasing growth in the business-to-business information sharing sphere, management recognized that pure technical approach to security consistently can lead to information security deficiencies. As a result, the Data Security Policies and Business Strategies for Security are increasingly important. Institutionalization Wave - Security Culture

To ensure a holistic approach towards data security, the human dimension of information security is a key precept that cannot be ignored. The main characteristic of the institutionalization wave is to nurture an Information Security Culture in such a way that information security becomes a natural aspect of the daily activities of all employees of the organization. Increasing organization information security led some companies to promoting their organization’s information security integrity as a competitive advantage over their rivals, and thus leading to a rise in Business Strategies for Security. The evolution of Information Security shows us once again the same recurring security themes, its evaluation and its penetration to mainstream acceptability. These themes are Confidentiality, Accessibility, Vulnerability, Security Culture, Business Strategies and Security Policies. By looking at information flow analysis towards information security, we can identify another angle by which information security is perceived by the research community. Information Flow Analysis

Information Flow Analysis considers the coupling of information dependency between variables. Sabelfeld and Myers surveyed the use of information flow analysis in software [16]. The ability to track information flow in computing systems serves as a proxy for maintaining security. They concluded that standard security practices do not and cannot enforce the end-to-end confidentiality required by common security models.

Any structured security management methodology or model must take into account the fluid nature of information. For large organizations, information does not survive in vacuum. There is a constant need to operationally exchange information on a secure basis with associated upstream or downstream businesses. In this paper we will label them as partners. Deriving security measures utilizing information flow analysis dictates conceptualizing security holistically, both within the company and with the company’s partners. Accessing a company’s security perceptions of itself is therefore insufficient. A holistic approach to security must therefore examine an organization’s perceptions of their partners’ security. Thus in our study, apart from examining an organization’s perceptions and assessments of its own security, we will also consider how that organization regards its partner’s security.

2.4.2 Industry Standards As the Information Security industry has been established for over 30 years, there has been growing Information Security standardizations and certifications efforts. We examined several

- 7 -

industry information security standards, such as BS 7799, for ideas of constructs that are applicable to our study. BS 7799: Specification for Information Security Management

The BS 7799 standard setting for Information Security Management System, created by the British Standards Institute, provides guidance and recommendations on best practices for information security management. It assesses businesses’ information security in 10 areas:

• Security policy - This provides management direction and support for information security • Organization of assets and resources - To help manage information security within the

organization • Asset classification and control - To help identify organizational assets and appropriately

protect them • Personnel security - To reduce the risks of human error, theft, fraud or misuse of facilities • Physical and environmental security - To prevent unauthorized access, damage and

interference to business premises and information • Communications and operations management - To ensure the correct and secure operation of

information processing facilities • Access control - To control access to information • Systems development and maintenance - To ensure that security is built into information

systems • Business continuity management - To counteract interruptions to business activities and to

protect critical business processes from the effects of major failures or disasters • Compliance - To avoid breaches of any criminal and civil law, statutory, regulatory or

contractual obligations, and any security requirement2

Many areas of this security standardization pertain to our eight security constructs, such as Security Policy, and the importance of Resources (Technology and Financial) allocated for the purposes of security. As we are attempting to assess general security perceptions, we will not be looking into specific areas such as Personnel, Physical and Environmental Security, as they are already taken into consideration under the categories of Accessibility, Vulnerability and Confidentiality.

Though the BS 7799 control areas for determining information security management serves well as a practical information security assessor tool, it is insufficient for our purposes of identifying the full range of key constructs in information security perceptions.

2.4.3 The McCumber Cube Another model for measuring security perception is the McCumber Cube [17] depicted in

Figure 4. The model was developed as a response to the attempts in the late 1980s and early 1990s to define the relationship between the communications and computer security disciplines. As a measure of its value and effectiveness, this model was utilized by the National Security Telecommunications and Information Systems Security Committee (NSTISSC) and published in National Security Telecommunications and Information Systems Security Instruction’s (NSTISSI) National Information Systems Security (INFOSEC) Glossary.

2 Available from the BSI Management System Information Security webpage http://www.bsiamericas.com/InformationSecurity/Overview/WhatisBS7799.xalter (Accessed December 2, 2005)

- 8 -

McCumber defines the three main constructs of Information Security to be: Confidentiality,

Integrity and Availability. The three concepts, respectively, correspond to our three primary indicators of Information Security: Confidentiality, Vulnerability and Accessibility [17]. Under the Security Measures of the Cube, the three subgroups of Technology, Policy and Practices and Education, Training, Awareness pertain to the importance of Security Policy and implicitly the Security Culture, precepts covered by our security model.

The McCumber cube contains several security ideals or constructs pertinent to our study. However, the over-emphasis on data storage processing procedures, underlined by the transmission, storage and processing aspects of the cube, obscures the importance of more relevant security areas, such as a corporation’s utilization of financial and technological tools to ensure its data integrity. Also, the McCumber Cube does not assess the level of technological and financial resources available for an organization’s information security. Financial concerns and the technical complexity of security contribute significantly to a lack of overall security in a corporation, and should therefore not be overlooked in our security perceptions study. Thus, when constructing the security model used in our study, we augmented and improved on the McCumber Cube’s security assessors to ensure that all major security components are considered.

3. Security Constructs Forming the House of Security

3.1 House of Security Security is a system problem [18] in that “several different elements working together usually

compose a security system to protect something.” Any judgment regarding the degree of protection or security afforded by a particular security system involves a fairly complex set of interrelated factors. We recognize that each of the eight constructs identified share inherent affiliations with one another. These qualities are highly correlated, many supporting and reinforcing each other. As a result of the recognition of these relationships, we constructed the “House of Security” model, as depicted in Figure 5.

Figure 4: McCumber Cube

- 9 -

Figure 5: House of Security Structure In the “House of Security” the three major principles of information security -- Vulnerability,

Accessibility and Confidentiality -- serve as the ‘roof’ or apex of Information Security Architecture, while the five other constructs -- Technology Resources, Financial Resources, Business Strategy, Security Policy and Security Culture -- form the structural foundations and pillars that support the three main principles.

As the analogy goes, without the pillars of support, the main roofs or tenets of security are unsustainable. Conversely, without the main tenets, the resultant pillar would serve no centralized theme or purpose. Thus these eight security constructs or building bricks share an intricate relationship while addressing disparate areas of security. To possess a strong information security infrastructure, all factors must therefore be considered.

In the next section, we will describe and define these eight constructs in more detail.

3.2 Security Constructs We do not attempt to argue that the eight constructs that we identified are the only way to describe or decompose security issues, but rather that they are a useful framework that is supported by our surveys, web search, and review of related literature. In this section we restate the eight major security constructs and the reasons for their inclusion in our security model.

3.2.1 Vulnerability Vulnerability is the level at which information integrity is maintained. Integrity is the ability

of an organization to ensure that their secured information, through the prevention of the data from being adulterated or manipulated by unintended users, remains accurate and secure. Vulnerability is cited by the McCumber Cube, and is synonymous with the concept of integrity of a security system. The integrity of a system is a latent quality of all major security-related research, and hence the inclusion as a major factor in assessing security.

Technology Resources

For Security

Financial Resources

For Security

Business Strategy

For Security

Security Policy &

Procedures

Security Culture

Accessibility Confidentiality

Vulnerability

- 10 -

3.2.2 Accessibility Another inherent quality of all major security research is the accessibility of a company’s

secured data. Accessibility is the ability of stored, transmitted, or processed information to be used for its intended purposes when required. McCumber states that the most significant imperative for confidentiality is not the element of secrecy, but the capability to ensure the appropriate subjects (both people and other processes or systems) have the requisite access when needed.

3.2.3 Confidentiality Confidentiality is defined by the International Standards Organization (ISO) as "ensuring

that information is accessible only to those authorized to have access." Confidentiality is one of the main goals of information security [19]. It is also referred to as one of three critical information characteristics in the McCumber Cube. Confidentiality is a chief trait in the “first wave” of security evolution. Thus the inclusion of confidentiality as a security measure is deemed necessary for the completeness of our model.

3.2.4 Technology Resources for Security From the 1999 Information Security Management survey (Figure 1), it has been determined that Technical Complexity and Lack of End-User Awareness account for approximately 15% of the reasons cited as biggest obstacles to good organizational information security. Even if technological tools were available, which was another obstacle cited (3%), the lack of sufficient internal technical assistance or security software knowledge leads to a security lapse.

3.2.5 Financial Resources for Security Information Security is not a free asset, and is severely draining on resources with often no

visible effects of remuneration. From Figure 3, it can be seen that budgetary constraints form the primary obstacle behind poor information security infrastructure, accounting for 29% of the reasons cited for greatest obstacles to good information security. Thus a key indicator of data security would be the financial resources made available for security.

3.2.6 Business Strategy for Security A successful corporation’s organizational structure often encapsulates information security as

one of its key fundamental goals for success. As shown under the “second wave” of the von Solms model of information security evolution, management ownership of security leads to the integration of security together with business decisions, and hence its assessment determines the level of maturity of information security in the corporation.

3.2.7 Security Policy and Procedures Security policies are derived from organizational objectives [20]. Organizational objectives

refer primarily to organizational goals, decisions and the enabling resources to achieve these goals. The McCumber model, the international standards, and various surveys all specify the importance of having security policies and procedures that dictate the implementation of overall security.

3.2.8 Security Culture Security culture is a leading indicator of how much proper security practices are accepted and absorbed within a company, such that it is encompassed in its corporate culture. Hallmarks of security culture include: setting adequate levels of information security as key corporate performance indicators, emphasizing overall security awareness, and developing security policy that involves and

- 11 -

includes all levels of employees. Many researchers concur that the security culture in an organization is an important aspect of information security [15, 21].

3.3 Extended Enterprise Security From the information flow analysis view on security, as well as from the various surveys and

literature review conducted on information security, it is evident that people are just as concerned about the confidentiality of inter-organization data flows as they are intra-organization data flows. In an increasingly networked business world, one company’s information security certainly affects their business partners. For this reason it is often important that business partners demand an acceptable level of information security from one another [22].

It is therefore inadequate to solely address information security perceptions within the organization. As confidential information flows within and between organizations, security perceptions must therefore also reflect that flow. Thus, to ensure the completeness of our security perception survey, it is imperative to create a test that assesses a partner organization’s security. We will then make relevant conclusions from the deviations between the partner’s security assessments and the organization’s own internal security assessments.

4. Methodology

4.1 Survey Construction and Implementation We designed a survey instrument to assess perceptions of various stakeholders regarding the constructs of the House of Security while also testing the factors that affect these constructs. Since we are also interested in a holistic approach to analyzing security, we wanted to determine whether individuals have differences in perceptions about the security of their own organization versus security in a partner organization.

4.1.1 Demographic Section The survey has two main sections. The first section is composed of a set of demographic

questions related to the nature of their organization, respondent’s role and position within the organization, etc. This demographic information can be used to identify potential independent variables that shape and determine security perceptions (e.g., whether top executives have different perceptions from middle-level managers or whether size or industry of organizations leads to different perceptions regarding security.).

4.1.2 Assessment and Importance Section Following the demographic section, the respondents are asked to rate a series of security-

related components. For each component respondents are asked to provide their “Assessment,” which is intended to capture their view of how well their organization is doing on that issue and “Importance,” which is intended to capture their view of how important this issue is to them. Figure 6 provides five examples of these components.

Your Partner Components Organization Organization Assessment Importance Assessment Importance

The organization’s data and networks are rarely tampered with by unauthorized access.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

- 12 -

In the organization, security is adequately funded.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

Customers trust the organization not to disclose data about them.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

The organization’s security strategy sets direction for its security practices.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

Business managers in the organization are involved with IT security policies.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

Figure 6: Five Example Components from Survey Instrument

Figure 6 also shows how we endeavor to take a holistic approach to security analysis, whereby respondents are asked about their perceptions related to the extended enterprise. In particular, the respondents are asked to state, for each component, their perceptions of a partners’ current performance and importance. As a result, each component is evaluated from four different aspects3: (1) Assessment of the security construct within an individual’s organization, MA; (2) Importance placed on that security component within an individual’s organization, MI;

(3) Assessment of the same security construct for a partner organization, PA; and (4) Importance placed on that security component in that partner organization, PI.

Since our components and our survey target any individual in any organization, we have a large respondent base (or population). The survey is web-based to maximize the number of respondents, ease data collation and facilitate promptness of response. This also offers a greater sense of anonymity, which is vital when garnering responses on the information security of organizational partners.4 Overall, the survey takes about 20 minutes to complete. In Appendix I we have included a copy of the survey instrument. Each component utilizes a forced-choice Likert scale that ranges from 1 to 7, with a 1 corresponding to a low (or unsatisfactory) grade, while a 7 corresponds to a high (or satisfactory) grade.5 In a typical Likert Scale, the respondent can only choose whole numbers and must answer a critical number of components in each subject in order for the survey to be validated and utilized. However, in a normal Likert Scale, the middle answer (usually 3) represents an “uncertain” or a “don’t know” response. In a forced-choice scale, our middle answer (4) is instead construed as a “medium” or “neutral.” This gives answers on a continuous range and makes statistical assessment easier. Typically forced-choice Likert scales also vary from 1 to 4; in our case we increased the range to 7 not just in order to better capture the attitude of the respondent, but also to aid in our gap analysis (discussed later). The security components correspond to the eight constructs in the House of Security, with typically five components devoted to each construct, thus producing a total of 40 components. Rather than have components relating to the same aspect in the same section of the survey, components are instead randomly distributed so that respondents are not motivated into providing the same exact response for components arising from the same construct. The following sections present a discussion

3 We use the term “aspect” to refer to these four different views of each component. 4 Many respondents are hesitant to discuss the qualities of their partners lest they be seen as critical of an organization which they are trying to build ties with. 5 For more information about Likert Scales, please visit the Social Research Methods discussion of Likert Scaling at http://www.socialresearchmethods.net/kb/scallik.htm (Accessed October, 2005).

- 13 -

about the process of mapping, constructing, and verifying the relationship of components to constructs, as well as a process of determining the number of components for each security construct.

4.2 Problems Encountered and Addressed in Survey

4.2.1 Parallelism Rather than directly assessing respondents’ attitudes on the eight different aspects of security,

which may be too abstract a concept to be applicable to the respondents, we asked about five different components about the organization’s environment. These questions designed to be indicators of the various security constructs. In the case of the security construct Vulnerability, respondents are tasked to assess the construct through directly relatable questions such as: “The organization’s data and networks are rarely tampered with by unauthorized access”, rather than “How vulnerable is your organization?” This is done to prevent the components posed from being too abstract for a valid response, thereby improving the clarity and quality of the components.

4.2.2 Multiple Aspects of the Same Construct For the purpose of gathering data on perspective differences or gaps, respondents are asked to answer each component from four different aspects. There are four different values for each security construct, one from MA, MI, PA, and PI. As a result, when performing statistical analysis on our data, such as Cronbach’s alpha test of reliability [23], each different aspect will produce a set of data that assesses the same question from a different perspective. This will produce four different alpha values for each security construct, introducing complexities when we try to assess the constructs reliability, as explained later.

4.2.3 All Components Worded in Positive Form Often a survey will be designed to prompt reverse values for different related components

(e.g., components of job satisfaction might include “How much do you love your job?” and “How much do you hate your job?”). This is intended to avoid the habit of always giving the same answer and helps to recognize whether the respondent is consistent (e.g., if “love job” is high, we would expect “hate job” to be low.)

However, we chose not to reverse the wording of the components. This decision is based on the unique characteristics inherent within our survey. The purpose of the survey is to gauge the discrepancy between the present state of each security construct and the importance placed upon that construct by respondent. In order to make fair and valid gap assessments, we need to link and pair these aspects together. Linking these aspects together qualitatively involves phrasing components that can be assessed on two different scales: the current level of security and the importance placed on that security element. As it is not always feasible to reverse the wording of all components in a meaningful way in our survey instrument, all components are worded in a positive form.

4.2.4 Inherent Skewing of Results We also recognize that the nature of the survey questions prompted an overall inclination for

higher values (most people prefer more security rather than less security.). As a result, although the respondents were given a 7-point scale, most only used the upper 4 values. This is one of the important reasons why we used a 7-point scale instead of a 4-point or 5-point scale.

4.3 Survey Reliability and Validity To verify the correctness of the survey instrument, there is a need to determine its reliability and validity. A reliable survey means that it produces consistent results. For each of our security

- 14 -

constructs, we asked about five components6 pertaining to that construct. We will refer to each question as a component of the construct. In a reliable survey, these five components of a construct should yield consistent responses since they should all be addressing very closely related components of the construct.

In addition to reliability, the validity of a survey must also be tested. A component is considered valid when it is more closely correlated with the other components of its construct than it does with another construct. In particular, we need to establish that components used to assess a particular security construct directly pertain to that same construct rather than another construct. This will prevent the situation where a component that is intended to measure Confidentiality is actually measuring Vulnerability instead.

Thus it is necessary to conduct both Reliability and Validity testing to establish the correctness of our survey and to allow for correct conclusions to be made from our survey results.

4.3.1 Construct Reliability - Cronbach’s Alpha Test Cronbach’s alpha is a common measure of reliability cited in the psychological and

sociological literature [24-26]. Cronbach’s alpha measures how well a set of items (or variables) measures a single one-dimensional construct. This coefficient varies from 0 to 1.0 and represents a lower bound of the reliability estimate. The alpha value usually increases using a larger measurement scale such as our 7-point Likert scale [27]. Cronbach’s alpha is calculated thus:

1)/1( 22

−Σ−

=n

n TI σσα , where 2Iσ : Individual component variance 2Tσ : Composite construct variance

n : Number of components within that construct Minimum Threshold for Cronbach’s Alpha Value and Deletion Method to Improve Value A Cronbach’s alpha value below a threshold value indicates that the components of a security construct (as in the components themselves) have inherent inconsistencies. This means that the components that form a construct cannot be reliably used as an indicator for that particular construct. However, by deleting the question shown to be the least correlated within a particular construct, the Cronbach’s alpha value can often be improved.

There is no established standard on the Cronbach’s alpha threshold value that will determine construct reliability. An acceptable reliability coefficient is 0.7 [28, 29], though a lower threshold of 0.60 is also acceptable [23, 30], especially given a small sample size [31] In our studies, due to having multiple aspects of the same construct, we will employ the lower-bound Cronbach’s alpha value of 0.60 to determine that the construct under analysis is acceptably valid. We shall now examine the algorithm used for choosing the question that will be removed from a particular security construct to improve its alpha value. Algorithm Choice for Resolving Cronbach’s Alpha Deletion Selection Since in our study there are four different aspect measures of each construct validity (i.e. measures for each MA, MI, PA, and PI), there is a possibility that the component removed by the

6 Although most constructs had 5 components, one construct had 4 components and one had 6 components.

- 15 -

Cronbach’s alpha deletion selection process will impact various aspects dissimilarly. A problem arises when a component chosen for deletion to improve the alpha value of a single aspect (e.g., MA), significantly reduces the Cronbach’s alpha value for a different aspect of the same construct (e.g., MI.).

When assessing multiple components and aspects for the same data, we will encounter misleading reliability estimates if a factor that varies in our assessment is not included in the estimated error of measurement [32]. Swanson concluded that “Regardless of the assessment method used, performance in one context does not predict performance in other contexts very well”[33]. With that consideration, it is evident that the reliability values for each of the four aspects will likely give varying results and merely aggregating the alpha values of the differing aspects (e.g., by taking the average) will be defective. A typical solution is by standardizing the task direction, disaggregating the various aspects of a component and scoring them individually, and just accepting the best results from the four aspects [32]. In relation to standardizing the task direction, our survey achieves this by keeping the same scale and direction for all four security aspects. However, disaggregating the four separate aspects would produce a total of 160 separate questions, which is far too many to allocate for a single survey. We feel that taking the best correlation data is unacceptable, as it completely buries all relevant data, especially if three out of the four aspects failed the Cronbach’s alpha test. Thus, given our situation, there is a need to conceive and utilize an approach for removing a component from a construct that best improve the overall Cronbach’s alpha value of the four aspects. A first approach for a given component removed from the construct would be to take the average Cronbach’s alpha value and compare it with our threshold of acceptability of 0.60. A major failing of this approach is that this method aggregates information, and in doing so losing considerable valuable information about the respective Cronbach’s alpha values. For example, a significantly high enough Cronbach’s alpha value in one aspects (such as > 0.90) can appreciably improve poor Cronbach’s alpha results in the other three aspects (values below 0.60), and result in a poor choice selection for removal. A better approach would be to assess the Cronbach’s alpha value of each aspect of a construct individually, and determine if all the Cronbach’s alpha are above the threshold value before deletion. If so, no action is taken, and the construct is accepted without revision. If this is not the case, choosing the component whose removal would allow for the Cronbach’s alpha values of all aspects to be higher than our stated threshold of validity acceptance.

Using such a method, there will still be situations where the best choice for component removal will adjust the Cronbach’s alpha above and below the threshold value for different aspects. In such considerations there is a need to revise the component sets entirely, since it inherently fails the Cronbach’s alpha construct validity test. This will be our approach for acceptability and improvement of any survey component deemed unsuitable by Cronbach’s alpha analysis. Mean Replacement7

As with most surveys, respondents are generally unwilling or unable to answer all the components posed – due to lack of time, inappropriateness of certain components, a fear of retribution due to responding or other reasons. A commonly accepted approach is to attribute a value of “zero” for missing responses of a survey.

But this approach to handling missing information is inappropriate as inserting zeroes in our 1-7 Likert scale will skew the survey results downwards unnecessarily. We will use the Mean

7 Just to be clear, this method is only being used for the purposes of Cronbach’s alpha assessment and not for establishing the significance of the findings themselves. As we shall explain later, in order to understand a difference between two concepts or constructs or components, one needs an observable gap for each respondent. Thus, the Mean Replacement Method does not serve our purposes for statistical analysis.

- 16 -

Replacement Method [34] to handle missing data in our survey. For a component that has no data, either due to neglect or unwillingness to answer, the method will take as substitution for the missing data, the average of the answers given by other respondents for the same component. However, if the respondent chooses not to answer more than one component out of the same construct, we would remove that respondent’s answers for the entire construct, so as not to overly dilute our results.

4.3.2 Construct Validity Construct Validity consists of the correlation between two or more separate ideas of the same

construct. This measure is based on a set of theoretically based predictions as to the direction of the relationships between several separate instruments [27]. In our study, we will be assessing construct validity quantitatively using techniques developed by Hair (1998) and Fornell (1981). To fulfill Construct Validity, for each latent variable (security constructs), we must establish both Convergent and Discriminant Validity. Convergent Validity Convergent Validity indicates if the components of a construct under consideration converge together to form a single construct. A latent variable (security construct) is deemed to have convergent validity if the average variance extracted calculated exceeded the threshold value of 0.50 [35]. In these calculations Iλ is the standardized loading factor of a latent variable. Loading Factor is defined as the correlations between the factors (security constructs in our case) and their underlying variables. Average Variance Extracted (AVE) is calculated thus:

Discriminant Validity Discriminant validity indicates that the construct being measured is not similarly measured

under a different construct name. Fornell and Larcker [36] stated that to determine discriminant validity, the squared multiple correlation between two constructs must be less than the average variance extracted (AVE) of each construct.

Thus a matrix is constructed, the diagonals set with the values of the AVE of the respective security constructs, and the lower triangular filled with the squared multiple correlation values between all the constructs. To fulfill the discriminant validity assessment, all the squared multiple correlation (R-squared) values, on the lower triangular of the matrix, of a particular security construct and the other constructs must be shown to be lower than the AVE value of that particular security construct, located on the diagonal of the matrix.

5. Preliminary Findings

5.1 Secured Web-Based Survey Instrument A Web-based survey instrument was developed that corresponds to the survey instrument shown in Appendix I. In ensure confidentiality, the interaction with the web site uses secure communications. Data gathered from survey respondents is gathered in a secure data base and can be downloaded into Excel spreadsheets via a password protected interface.

))1(()(

22

2

II

IAVEλλ

λ−Σ+Σ

Σ=

- 17 -

5.2 Analysis Tools In our analysis, we used the SAS statistical software, as well as Microsoft Excel to transfer the data gathered from the survey respondents to the SAS program. By minimizing human interaction there is less possibility of erroneous data-manipulations, and increasing scalability of the analysis when final survey is completed. Cronbach’s alpha and Factor Analysis are both fundamental function within SAS, and SAS was used to analyze the data from the four distinct aspects. Factor Analysis states the loading factor for a particular construct, and is used to calculate the AVE value for measuring construct validity. Since a single-dimensional scale was used (1 to 7 for all components), the raw value was utilized rather than the standardized value of Cronbach’s alpha. The SAS code for the calculations can be found in Appendix III.

5.3 Gap Analysis Gap analysis is often used in the social sciences as well as in management consulting. It was first developed as a tool for measuring customer satisfaction [37], and has subsequently been used to identify differences between consumer expectations and management expectations in the construction industry [38], for the benchmarking of Korean luxury hotels [39], and for ventures in e-business[40]. Gaining industry acceptance, gap analysis has even become standard practice for Six Sigma assessment for large companies.8

The purpose of using gap analysis in this project is to identify differences within security perceptions, both within the organization and across the Extended Enterprise. Also, when there are obvious response gaps between different stakeholders, the cause of these discrepancies represents areas for further study. These discrepancies can be attributed to security knowledge or contextual gaps between stakeholders. In our case, we are concerned with several gaps including:

(1) Performance Gaps These gaps represent the differences between the “assessment” and the “importance” aspects within security components. It is the discrepancy between their desired state of security versus their perception of the current state of security in their organization. Gaps are calculated for both their own organization and a partner organization. The partner organization is not matched in anyway and is simply the respondents’ assessment of status and importance for an organization with whom they interact with.

(2) Role Gaps The goal here is to identify if there are any significant gaps between the various enterprise roles, such as C-level executives v. professionals or managers v. staff workers.

(3) Industry Gaps Here, we want to determine if there are potential differences in security perceptions between stakeholders in different industries, such as between manufacturing organizations and financial services organizations.

(4) Area Gaps We want to determine if there are differences in security perceptions between stakeholders in different areas of an organization. We are interested in whether people in IT or security had dramatically different views from people not in those areas.

8 To learn more about Six Sigma accreditation, see http://www.pqa.net/ProdServices/sixsigma/W06002006.html

- 18 -

5.3.1 Statistical Significance of Gaps Unfortunately, gap analysis is often not conducted in manner that uses rigorous statistical

analysis. Looking at the previous examples, in every case identified but one [39] important gaps were determined by qualitative means. In these cases, the important findings were nothing more than noting the largest findings and no statistical cutoff was ever established.

We utilize statistical analysis to assess the presence of gaps that are not attributable to random variations in response. By using paired t-tests, we can determine the statistical significance of a gap. Statistical significance is a measure of how likely the result happens non-randomly. A paired t-test offers a way of showing whether the differences between two given distributions are statistically significant. The formula for a paired t-test is:

where n = Number of Respondents X = Average of Distribution X Y = Average of Distribution Y

iX∧

= )( XX i −

iY∧

= )( YYi − In order for the paired t-test to be valid, one must assume that the paired differences are independent (i.e. nobody is answering the same component with the same responses multiple times) and that these differences are identically normally distributed. We then compare the size of the gap to the null hypothesis: in this case our null hypothesis is that no gap exists and that the difference should therefore be zero. Since gaps can be both positive and negative, we use a two-tailed t-test to assess their statistical significance.9 In Figure 9, the center line equals the null hypothesis and the observation has to be so extreme as to fall in one of the Critical Value Regions. This is the same style of applying statistics to gap analysis as used by Min and Min (1996).

Figure 9: A Normal Distribution and Two-Tails (With Shaded Areas)

In order to conduct analysis on Role Gaps and Industry Gaps, we will also conduct paired t-

tests, but we will have to control our variables accordingly. For example, rather than comparing the

9 For a quick and dirty summary of paired t-tests, visit: http://mathworld.wolfram.com/Pairedt-Test.html

- 19 -

distribution of all answers of MA vs. MI, we would compare the distribution of answers of executive level personnel versus professional staff. We also plan to utilize traditional regression analysis in order to further understand the nature of these gaps, and in the case of Role Gaps and Industry Gaps, to determine which are the biggest influencers of Performance Gaps. In this case, the Performance Gaps are considered as dependent variables, while the demographic measures collected at the beginning of the survey are considered as independent variables.

5.3.2 Some Examples of Gap Results Using our current sample of responses, we examined the performance gaps that formed between each aspect of each component.10 Figure 10 shows the number of significant responses for the M-gaps (MA vs. MI) for the forty questions.

Significant at 99.99% level 28

Significant at 99% level 11

Significant at 95% level 0

Significant at 90% level 1

Less than 90% 0

Total 40

Figure 10: Number of Significant Results for M-gaps

Partially due to the reduced number of responses regarding Partner Enterprises, the M-gaps were the most significant (with all but one being significant at either the 99% or 99.99% level). We expect that the significance levels for the other gaps: A-gap (MA vs. PA), I-gap (MI vs. PI), the M-gap (MA vs. MI), and the P-gap (PA vs. PI) to increase as we gather more survey data. As some examples, let us consider the M-gaps in Accessibility as compared to the M-gaps in Security Culture, shown in Figure 11.

Q # “Accessibility” Construct Average Assessment

Average Importance

Average Gap

7 The organization checks the identity of users before allowing access to data and networks.

6.123 (1.107)

6.520 (0.805) 0.397**

14 The organization’s data and networks are only available to approved users.

6.110 (1.162)

6.548 (0.729) 0.438**

22 The organization has adequate policies about user identifications, passwords, and access privileges.

6.041 (1.054)

6.611 (0.544) 0.569***

36 The organization provides access to data and networks to legitimate users.

6.197 (0.906)

6.633 (0.569) 0.437***

10 Since this analysis was performed on the surveys before the Cronbach’s alpha analysis and the Construct analysis, all forty components have been analyzed. In future surveys, some of these components will be reworded or deleted.

- 20 -

40 The organization’s data and networks are usually available when needed.

6.319 (0.827)

6.722 (0.585) 0.403**

“Security Culture” Construct Average Assessment

Average Importance

Average Gap

9 The burden of security policies on people in the organization is minimal.

5.082 (1.466)

5.671 (1.256)

0.589**

11 People in the organization are knowledgeable about IT security tools and practices.

4.671 (1.477)

5.863 (1.165)

1.192***

18 People in the organization carefully follow good security practices.

5.000 (1.311)

6.250 (0.792)

1.236***

26 People in the organization can be trusted not to tamper with data and networks.

5.366 (1.184)

6.380 (0.730)

1.014***

32 People in the organization can be trusted to engage in ethical practices with data and networks.

5.347 (1.151)

6.319 (0.827)

1.972***

39 In the organization, people are aware of good security practices.

5.042 (1.329)

6.319 (0.861)

1.278***

Notes: Numbers in parentheses represent standard deviations; ***Gap is significant at the 99.99% level; **Significant at the 99% level; *Significant at the 95% level; ~Significant at the 90% level.

Figure 11: M-gaps Between Accessibility Construct and Financial Resources for Security Construct

Looking at the number of statistically significant results as well as the average gap size (with each gap calculated MI minus MA), it is clear that there is a larger gap between the importance and the current amount of financial resources for security than accessibility. There are big gaps between the importance and assessment of the number of security personnel and whether or not the security for the organization is adequately funded. In other words, many respondents claim that financing of security is falling short of its importance.

5.3.3 Significance of Gaps for Comprehensive Data Statistical Significance from p-values

The p-value is the probability of how unlikely the observed sample results would be, given the null hypothesis (µ0=0) to be true. In other words, the smaller the p-value, the more likely you can reject the null hypothesis (H0). Traditionally, a p-value of less than 0.05 indicates the alternate hypothesis (µa≠0) is highly likely.

Using the t-test, the p-value was calculated for both one-tailed and two-tailed. There is a case to be argued for both of these cases. If we are only concerned with if importance is better than assessment than we would use the numbers from the one-tailed approach. If we only care whether there is a difference in perception and reality, then the two-tailed test would appropriate. As it turns out, the results were such that importance rating is on average higher than the assessment rating. This can be attributed to the fact that most people probably would not want less security.

- 21 -

Results from t-test MA v. MI observed t-value (t) -4.64998 p-value one-tail 0.00019 t-critical one-tail (tc) 1.76131 p-value two-tail 0.00038 t-critical two-tail (tc) 2.14479

PA v. PI observed t-value (t) -4.48341 p-value one-tail 0.00026 t-critical one-tail (tc) 1.76131 p-value two-tail 0.00026 t-critical two-tail (tc) 2.14479

Interpretation of the Results

Sample evidence is statistically significant at the alpha level as long as the p-value is less than α=.05. The p-value was calculated from the average of each individual construct – not each individual question. The results from this t-test confirmed that gaps were statistically sigficant (p<.05). Here, the p-values are << α whether it is one-tailed or two-tailed. There was not that much of a difference in the one or two tailed tests. Our null hypothesis (µ0=0) can be rejected since observed t-value is greater than t-critical. If observed t-value is greater than or equal to the t-critical value (tc), then H0 can be rejected. This is true in all the above cases. For example, assuming a two-tailed test for MA v. MI, t=4.65>tc2.14, µ0=0 can be said to be false. Confidence Level Confidence level is the region where H0 can be rejected, and is by definition 1- α. Confidence Level MA v. MI one-tailed 99.98%MA v. MI two-tailed 99.96%PA v. PI one-tailed 99.97%PA v. PI two-tailed 99.97%

So in the case of MA v. MI, it says that we can reject the H0 with 99.98% confidence level. Results from t-test (from average of each question) ma v. mi observed t-value -8.30426 p-value one-tail 0.00000 t-critical one-tail 1.66901 p-value two-tail 0.00000 t-critical two-tail 1.99773 pa v. pi observed t-value -7.88080 p-value one-tail 0.00000 t-critical one-tail 1.66901 p-value two-tail 0.00000 t-critical two-tail 1.99773

- 22 -

Results from t-test (from each individual question) MA v. MI (Qs. 1) observed t-value -10.82702 p-value one-tail 0.00000 t-critical one-tail 1.64537 p-value two-tail 0.00000 t-critical two-tail 1.96077 MA v. MI (Qs. 2) observed t-value -6.10889 p-value one-tail 0.00000 t-critical one-tail 1.64538 p-value two-tail 0.00000 t-critical two-tail 1.96078

For sample of 30 0.144 (p-value) > 0.05 (alpha) Not significant 2.00 (t-critical) > 1.48 (observed t-value) For entire sample of 1440 p-value for all purposes is zero t-critial of 1.96 is << than t-stat of 11.97 MA v. MI (Qs. 3) with 30 samples observed t-value -1.47884 p-value one-tail 0.07230 t-critical one-tail 1.67155 p-value two-tail 0.14459 t-critical two-tail 2.00172 MA v. MI (Qs. 3) with 1400 samples observed t-value -11.97414 p-value one-tail 0.00000 t-critical one-tail 1.64538 p-value two-tail 0.00000 t-critical two-tail 1.96078

5.3.4 Methodology for Regrouping Comprehensive Data

The same method was used to find the gaps of the comprehensive data. Initial analysis was performed on the raw data. However, the results were quite different from the pilot data. For one, the gaps in general became smaller. But this could be explained by the fact that, in a small sample, just a few extreme results could make a large difference – whereas in the large sample, the extreme values became insignificant or canceled out with values that were of equally opposite extremes. Second, the gaps analysis broken down by roles in the pilot data revealed that executives had a very different perception than those in the rest of the organization. The gap results from the merged data revealed

- 23 -

similar gaps for all the roles. This led us to investigate why this might be happening. Specifically, the Harris data set was given a closer look. Given the knowledge that people answering these online surveys might not be fully attentive when filling out these survey questions we wanted to check if there were chunks of data that were possibly skewing the results.

All data had been manually checked for “if it made sense” in regards to the industries they entered on the pull-down menu versus other info they provided. The only exception is the ~200 data points from the company data. Since there was no pull-down menu, all of them had to be manually classified based on what they entered for industries- unless it was inconsistent with other responses they provided. The three “checks” used were specifically questions where the respondent had to manually fill out a response (as opposed to just clicking a button):

1. company name 2. role 3. area / department

Sections affected were the industry, role, and area classification. Only when two of the three “checks” proved to be inconsistent was the data was manually regrouped. While this was in no way an automated process, we tried to make our process of regrouping the data as consistent as possible. There will be charts summarizing these changes in the parts of the data affected. Specific examples of where and how the responses were regrouped will be in Appendix V.

5.4 Survey Reliability and Validity Analysis

5.4.1 Reliability Analysis By several iterations of question deletion or manipulation, we were able to produce a

statistically robust set of security constructs with high reliability as well as fair construct validity. Table 1 shows the results of our final Cronbach’s alpha analysis. For the full Cronbach’s alpha analysis, please refer to Appendix IV.

MA MI PA PI

Accessibility 0.695488 0.659241 0.828488 0.777826 Vulnerability 0.592442 0.610124 0.837599 0.763809

Confidentiality 0.660975 0.719417 0.8123 0.776087 Financial

Resources 0.740053 0.748803 0.881737 0.821156 IT Resources 0.640141 0.701367 0.793913 0.782604

Business Strategy 0.844291 0.790248 0.866102 0.711468

Security Policy 0.745929 0.673521 0.888389 0.892586 Security Culture 0.682589 0.706474 0.78854 0.793887

Table 1: Final Cronbach's values

5.4.2 Validity Analysis – Convergent & Discriminant Validity Table 2 shows the results of the Construct Analysis. The diagonal represent the Average

Variance Extracted (AVE) values for each security construct. All the values exceed 0.50, fulfilling the requisites set out by Hair in the Convergent Validity test. Also, the lower triangular values of the

- 24 -

matrix are the squared multiple correlations (SMC) values between construct pairs. For each construct, the SMC values of all possible pairings are below the AVE values, thus the survey possesses discriminant validity.

Accessibility Vulnerability Confidentiality Financial

Resources IT

Resources Business Strategy

Security Policy

Security Culture

Accessibility 0.647724793 0.47097 0.52719 0.35937 0.47631 0.18955 0.25447 0.24363 Vulnerability 0.47097 0.538408803 0.45185 0.50057 0.62317 0.57171 0.58709 0.46202

Confidentiality 0.52719 0.45185 0.616511103 0.49552 0.57952 0.42095 0.53149 0.34463 Financial

Resources 0.35937 0.50057 0.49552 0.769241644 0.69041 0.66325 0.58769 0.58476 IT Resources 0.47631 0.62317 0.57952 0.69041 0.621012913 0.53096 0.54148 0.48422

Business Strategy 0.18955 0.57171 0.42095 0.66325 0.53096 0.863862672 0.72785 0.66837 Security Policy 0.25447 0.58709 0.53149 0.58769 0.54148 0.72785 0.77396066 0.5285

Security Culture 0.24363 0.46202 0.34463 0.58476 0.48422 0.66837 0.5285 0.583653002

Table 2: Results of Construct Analysis

5.4.3 Survey Reliability and Validity Concluding Comments As shown in Table 2, for constructs such as Vulnerability, IT Resources and Security

Culture, although they fared well for convergent validity, there are areas where its discriminant validity can be improved. For construct reliability, Table 1 shows the final overall Cronbach’s alpha values for all four aspects. Only 1 out of 32 constructs had a Cronbach’s alpha value below 0.60, and for that particular construct, the difference is less than 0.008 from the acceptability cutoff. Overall, we believe that the revised and improved survey is acceptable and will be employed on a large scale. In the future, with additional time and resources, one could reword and retest the survey for better construct acceptability.

6. Preliminary Results In our pilot survey, interesting results have arisen in several categories: (1) the individual questions, (2) the constructs, and (3) the construct gaps. Some examples are presented here.

6.1 Individual Questions Respondents were asked to assess whether “people in the organization were aware of good

security practices.” They were then asked the importance of that issue in the organization. This was to be answered on a 7-point scale (where 1 means “true to a small extent” and 7 means “true to a large extent..”) The overall results are shown on the top line of the graph in Figure 6. The current assessment (marked MA) is the left part of that line (in yellow) while the importance (marked MI) represents the entire line. The right part of the line (in blue in the top line) represents the gap. In this example, there was a large gap, statistically significant at the 99.99% level. This suggests that awareness of good security practices falls far short of what is perceived to be needed among the respondents. When comparing individual organizations, such as Company X and Company I, we also observed major differences in assessment, importance, and gap size. One of the goals of this research is to understand these differences.

- 25 -

4 5 6 7

Comp I

Comp W

Comp X

Misc.

Overall

MA Gap

MI

4 5 6 7

Comp I

Comp W

Comp X

Misc.

Overall

MA Gap

MI

Figure 6: Individual Question Breakdown by Company

6.2 Constructs

4.0

4.5

5.0

5.5

6.0

6.5Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

MAMIPAPI

Figure 7: Security Perceptions by Construct

- 26 -

The distribution of security perceptions by the 4 aspects are illustrated in

4.0

4.5

5.0

5.5

6.0

6.5Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

MAMIPAPI

Figure 7. Assessment of security status for my and partner organization are about the same, while importance ratings tend to be naturally higher. Importance ratings do not vary too much within the 8 constructs. However, assessment ratings are not all the same when we look at the individual constructs. Just by inspection, the graph shows confidentiality and accessibility to be quite a bit higher than security culture. The aspects of security that seem to be lagging behind are security culture, financial resources, and vulnerability. Highest assessments in accessibility indicate that businesses are still primarily concerned with information access and use. Low assessment in security culture suggets that security management have yet to mature to the same level of security awareness and depth.

The ratings for My Importance consistently ranks highest for all constructs, which is an indication that people believe that their own company generally aspires to achieve higher security levels than partner organizations.

6.2.1 Construct gap

- 27 -

0.0

0.2

0.4

0.6

0.8

1.0Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

|MI-MA||PA-MA||PI-MI||PI-PA|

Figure 8: Security Construct Gaps

Although viewing the actual values of each of the constructs provides some insights, it is

often more interesting to examine the “gaps.” For example, one organization might have an assessment of “5”, but if it views that construct as only having an importance value of “5”, the gap would be zero and it might be content. Whereas, if another organization had an assessment of “6”, but viewed that construct importance as being “7”, that is a gap of 1 and might indicate an area for improvement. The categorization of perception gaps by constructs is shown in .

Low Gaps in overall Accessibility levels states that accessibility is very well-established, perhaps to the point of saturation or maturity. This can be attributed to the fact that for most companies, security is perceived as the ability to achieve secured data access, and is usually the focal point of most of the resources spent on security.

In our analysis with a larger and more varied data set, accessibility is still the area with the lowest gap. The gap had a value of .33 for |MI-MA| and .25 for |PI-PA|., compared with gaps in other constructs where the gaps ranged from .50 to .82.

The large MI-MA gap, and PI -PA gap in security culture indicates that companies are beginning to understand the need to achieve further improvement, highlighting an important area of potential growth. Security culture may be the weakest link in the house of security since human factors are involved. In our survey instrument, we had questions that asked if people are aware of good security practices and if they actually follow them. The high gaps in perception indicate that people are not generally aware of good security practices, and it would be even more unlikely that they actually follow through. In terms of incentives, there are often no direct benefits for people to follow through in practice. Thus, organizations need to have some standard way to assess their own security culture in order to determine what is holding back its member from following good security practices. Perhaps there needs to be more education to bridge the gap or better incentives. These gaps gives a good starting point to work with.

Partners assessment lower than self assessment indicates the aura of "invincibility" is present that companies believe they are safer than their partners. Further evidence is shown by the findings on the gap between partner’s importance (PI) and the own organization’s importance (MI) of security.

- 28 -

shows clearly that PI ranks consistently lower than MI, reiterating the point that in general, people believe their own company rates these qualities higher on their agenda than would their organizational partners.

- 29 -

6.2.2 Standard Deviation

0

0.2

0.4

0.6

0.8

1

1.2

1.4

Acces

sibilit

y

Vulnera

bility

Confid

entia

lity

Financia

l Res

ources

IT Res

ource

s

Busines

s Stra

tegy

Secur

ity Polic

y

Secur

ity C

ulture

MAMI

PAPI

Figure 9: Security Construct Standard Deviation

Standard deviation indicates the variation within the construct of people’s answers. A lower

standard deviation indicates that respondents generally feel the same way for that particular construct, while a high standard deviation indicates that there might be several levels of responses, and that the respondents can be further stratified into smaller and more homogenous groups.

Standard deviation of My Importance (MI) consistently ranks lowest in all constructs and all aspects. It is an indication of the level of importance that one person has of his own company remains pretty consistent. Another finding is that Partner’s Assessments and Importance consistently more variable than one’s own self assessment. In our survey, we asked respondents to answer PA and PI aspects with their own chose of partner organization. This will naturally lead to a higher variation (standard deviation) of answers given.

- 30 -

6.3 Perceptions by Industry

4.5

5

5.5

6

6.5Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

Company XCompany WCompany IOveral

Figure 10: Security Assessment by Industry by Companies

As there are several corporations that worked in alliance with us on this project, we managed to aggregate data based on industry. Company X represents a firm that specializes in data security, Company W represents a commercial education institute, and Company I represents an overseas firm which participated in our study. Unfortunately, with the lack of sufficient corporate data collected and analyzed, it is erroneous to generalize findings from a particular company, and translate it into implications for its industry field.

Company X has the best overall security perception assessment, scoring particular well in Security Policy and Business Strategy. This could be attributed to the fact that Company X’s business is in security, and therefore security is a significant factor in its corporate strategy. This is in contrast to Company W, where openness for knowledge distribution could be a key component to an educational institute’s success.

However, all companies fare remarkably similar in terms of security culture, IT resources and accessibility, even though the nature of business for all 3 companies are exceptionally dissimilar. These illustrate that across different fields and industries, there are several security areas that remain consistent.

Though the explanation for these cross-industry similarity and differences remain the topic of study for further research, one might postulate that there are several security areas like Accessibility that have been well addressed in the entire security space, leading to consistently high security perception levels. The disparity widens for constructs like Business Strategy and Security Policy, where organizational goals are often in contrast against security goals and policies. Regarding the consistently low assessments for security culture, it could be that these organizations have not yet incorporated security into its own corporate culture, which is often considered the highest level of security development.

- 31 -

The specific industries that were examined are summarized in Figure 11.

•10% Banking & Finance (162) –Ex. Bank of America •11% Tele/Communications (182) –Ex. Cisco, Verizon, Nortel •33% Health & Social Assistance (539) –Ex. Hospitals, Clinics, Medtronic •21% Manufacturing (347) –Ex. GE, Embraer, Lockheed Martin •15% Retail (259) –Ex. Wal-Mart, Target, Best Buy •10 % Technology Services (165) –Ex. HP, MITRE, IBM, Intel •2% Education (28) –Ex. Universities

Figure 11: Industry Breakdown

4.4

4.6

4.8

5.0

5.2

5.4

5.6

5.8

6.0

6.2

6.4

Accessibility Vulnerability Confidentiality FinancialResources

IT Resources BusinessStrategy

Security Policy Security Culture

Security Constructs

Ass

essm

ent R

atin

g

Banking & Finance Tele/Communication Health & Social Assistance Manufacturing Retail Technology Services

Figure 12: Security Assessment by Industry Upon further analysis from the larger data set, we wanted to confirm whether the results from the specific companies held true for the industries in general. We found the trends in the specific industries to be similar to the overall trend, but certain industries definitely had distinct levels of assessed security. In other words, looking at figure 12, the banking, telecomm, and technology

- 32 -

services have the highest assessment ratings overall. Even for the other constructs, those industries tend to have one of the highest assessments. With the nature of the industry and the kinds of information exchange involved, it makes sense that they are more accustomed to good security practices simply because there is more at stake. However, does this mean industries such as manufacturing and retail do not need good security? To a certain extent, this depends how important the various aspects of security are to their organization. But even then, is it possible that industries like manufacturing and retail have much room for improvement relative to other industries? While a manufacturing company may not need the same level of security practices as banking, they are not immune to security threats.

There is a need for further in-depth research into assessing the difference between the various industries in terms of security perceptions, to further understand and aid in corporate security development.

6.3.1 Construct gap

0.000

0.400

0.800

1.200

1.600Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

Company XCompany WCompany IOverall

0.000

0.400

0.800

1.200

1.600Accessibility

Vulnerability

Confidentiality0.000

0.400

0.800

1.200

1.600Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

Company XCompany WCompany IOverall

Figure 13: Construct Gaps by Industry by Company

As shown in Figure 13, Company W has the highest overall perception gaps. This is an

indicator for further improvement in these various fields such as Security Culture, Policy and Business Strategy. The largest gaps for all companies are also in these three areas, showing that different fields have a similar concept, though to a different degree, of the security areas they need to improve on.

- 33 -

0.0

0.2

0.4

0.6

0.8

1.0Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture Banking & FinanceTele/CommunicationHealth & Social AssistanceManufacturingRetailTechnology Services

Figure 14: Construct Gaps by Industry

Contrary to our previous conjectures, the gaps in the different industries turned out to be more or less the same throughout. In Figure 14, the only industry that is slightly more distinct is the manufacturing industry with security culture, security policy and business strategy having higher gaps than the other industries. There are differences among other industries too, but within each industry, security culture still has the highest gaps while accessibility has the lowest gaps. Despite the inconclusive findings from the industry analysis, it still remains a fact that there are significant gaps in security perception that needs to be addressed within each of the industries.

- 34 -

6.4 Perceptions by Role

4

5

6

7AccessibilityMA

VulnerabilityMA

ConfidentialityMA

FinancialResourcesMA

ITResourcesMA

BusinessStrategyMA

SecurityPolicyMA

SecurityCultureMA

Executive (CEO, CFO, VP)

Functional/Line Manager

Professional (Consultant,Engineer)Other Organizational Member

Overall

Figure 15: Security Construct Assessment by Role for Select Companies

Different people in the organization have different levels of awareness and perceptions

regarding their own company’s security. Figure 15 shows the distribution of security perceptions based on the role of the respondent.

Our preliminary findings show that top-level management, such as CEOs and CFOs, tend to have a lower perception regarding their own organization’s security than middle and lower level personnel. This variation is particularly notable in the areas of business strategy, policy, culture and financial resources.

In general, there is a trend of decreasing confidence in one’s own security as organizational seniority increases. This could be attributed to the lack of actual knowledge and personal experience with respect to security as a person approaches executive level work. In our analysis of the larger data set, we had more representation from companies of different industries. Many respondents commented on the threat of intrusions from hackers or viruses. In some cases, they revealed that they essentially have no (effective) security systems in place. One commented that the security responsibilities within the organization were segregated. The theory is that different members in different roles may have different attitudes towards security practices as it pertains to their own responsibilities. Since executives are generally responsible for seeing the bigger picture, they may be able to see the overall process better than those functioning within their own silos.

- 35 -

•8% Executives (125) –Includes owners, CEOs, CFO, VP, Directors •25% Managers (372) –Includes managers, supervisors •48% Professionals (725) –Includes consultants, accounts, engineers, etc. •5% Customer Service (69) –Probably should be grouped with others since it made no difference •14% Others (203) –General Laborer’s, office workers, office assistants

Figure 16: Role Breakdown for Merged Data Set

4.5

5.0

5.5

6.0Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

ExecutivesManagersProfessionalsCustomer ServiceOthers

Figure 17: Security Construct Assessment by Role

The results from the merged data set confirmed that C-level executives have a lower assessment on average than other roles in an organization. The notable variations are still in the areas of business strategy, policy, culture and financial resources. While the select companies might not have been an accurate representation of the different industries, there is a clear distinction the in different roles. It was surprising to find that, regardless of roles, respondents viewed the importance of the various aspects of security to be about the same (as can be seen in Figure 18), given that executives assess security to be significantly worse off. There is also the possibility that executives do not share the same understanding as others in the organization. It is a possibility that they follow different sets of security practices than the rest of the organization. For example, executives might have received different security practices training than the rest of the organization.

- 36 -

4.5

5.0

5.5

6.0

6.5Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security CultureExecutivesManagersProfessionalsCustomer ServiceOthers

Figure 18: Security Construct Importance by Role

6.4.1 Construct gap In our gap analysis of the different roles, we found that gaps, on average, were 60% greater than those in other roles. All gaps share similar trends and take on the same shapes with the exception of executives. The disparity in perception reiterates the idea that executives are more dissatisfied with the level of security within their own organizations. Perhaps, executives think situations are worse off than they really are because they do not understand how and if security measures are being correctly implemented. The reverse can also be true – that executives see problems that people in other roles do not see and as a result, their perception of security gap is higher. Follow-up studies or case studies would be needed to understand the actual cause in the differences in perception.

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security CultureExecutivesManagersProfessionalsCustomer ServiceOthers

Figure 19: Security Construct Gaps by Role

- 37 -

Within the scope of this research study, we were able to gain further insight by examining the data on a secondary level. For example, we analyzed whether executives in certain industries had differences in perception.

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60

1.80

Accessibility Vulnerability Confidentiality FinancialResources

IT Resources BusinessStrategy

SecurityPolicy

SecurityCulture

Exec Banking Exec Health Exec Manufacturing Exec Retail Exec Tech Services Exec Telecomm

Figure 20: Security Construct Gaps of Executives by Industry

Previously, we found that the banking was one of the industries with the highest assessment.

In the analysis of executives within different industries, we also found banking executives to have the highest gaps. Even though manufacturing had the highest industry gaps, there was not a clear trend that manufacturing executives had higher gaps than others in the industries. Executives in healthcare and manufacturing seems to share similar trends as does technology services and retail, just at different magnitudes. The telecomm industry seems to be most unpredictable and have the most variability.

- 38 -

6.5 Perceptions by Area

4.5

5.0

5.5

6.0

6.5Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

Business Security PolicyIT SecurityIT, Not SecurityGeneral / Physical SecurityNot Security

Figure21: Security Construct Assessment by Area

This is also a differentiation of security perceptions based on the area of work of the respondents. Figure21 shows the breakdown of security assessment based on the respondents’ area of work.

Generally, we find that there is less security perception variation between the work areas of respondents, although it should be noted that respondents in General/Physical Security line of work or those not in Security or IT tends to have a lower assessment of their own company’s security. Respondents from other areas of work generally provide similar assessments on their own security.

4.5

5.0

5.5

6.0

6.5Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

Business Security PolicyIT SecurityIT, Not SecurityGeneral / Physical SecurityNot Security

Figure22: Security Construct Importance by Area

- 39 -

The security assessment construct by area did not reveal a clear trend. However, those not working in areas of security perceived the importance of security to be less than those working in security or IT related areas. Our gap analysis confirmed that people not in security or IT perceived security to be, on average, 8% less important than those in security. Even though 8% may not seem like a large gap, we have to keep in mind that their assessment and gaps were among the smallest. Put another way, this means that those not working in areas of security or IT perceive security to be bad, but do not seem to think there is need for much improvement either. While this seems contradictory, it would not be unusual for those not working in security to place a lesser importance on security.

0.2

0.4

0.6

0.8

1.0Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

Security / ITNot Security

Figure23: Security Construct Gap by Area

7. Conclusions Understanding the perceptions of security are important to the success of any organization and there many drivers which could be shaping these perceptions. This project seeks to both theoretically and empirically demonstrate the important determinants of these notions. There were three main steps that we have undertaken here. Firstly, we observed that literature pertaining to security is either too myopic or it is too broad. This led us to eight key constructs, which we organized into the House of Security. However, in order to make our project more rigorous, we have designed a method by which to test the relative importance of different security constructs and a method to determine what may affect these perceptions. The second half of the paper is concerned with the methodology and especially with the design and validity of a survey that can shed some light on notions of security and the influencers of these notions.

- 40 -

The next steps for this project include the revision and the administration of the survey. Following this, we will use the already-agreed-upon methods of gap analysis presented in this paper to identify gaps in perceptions along three lines – between security notions, between different individuals, and between different organizations. Combined, we hope that this methodology will help us understand not just security, but also organizational behavior at large.

- 41 -

7. Bibliography 1. Gordon, L.A. and M.P. Loeb, The economics of information security investment. ACM

Transactions on Information and System Security, 2002. 5(4): p. 438 - 457. 2. Cheswick, W.R., S.M. Bellovin, and A.D. Rubin, Firewalls and Internet Security:

Repelling the Wily Hacker. 2003: Addison-Wesley. 3. Oppliger, R., Internet Security: Firewalls and Beyond. Association for Computing

Machinery 1977. 40(5): p. 92-103. 4. Zwicky, E., et al., Building Internet Firewalls. 2000: O'Reilly & Associates. 5. Boneh, D. and M. Franklin, Identity Based Encryption From the Weil Pairing. Siam

Journal of Computing 2003. 32(3): p. 586-615. 6. Dolev, D. and A. Yao, On the Security of Public Key Protocols. IEEE Transactions on

Information Theory 1983. 29(2): p. 198-208. 7. Needham, R.M. and M.D. Schroeder, Using Encryption for Authentication in Large

Networks for Computers. Communications 1978. 21(12): p. 993-999. 8. Furnell, S. Cyber Threats: What Are the Issues and Who Sets the Agenda? in SGIR

Conference. 2004. 9. Kephart, J., G. Sorkin, and D.W. Chess, S., Fighting Computer Viruses. Scientific

American, 1997. November. 10. Siponen, M., A Conceptual Foundation for Organizational Information Security

Awareness. Information Management and Computer Security 2000. 8(1): p. 31-41. 11. Sinclaire, J.K. Current Research in Information Security and Privacy. in Southern

Association of Information Systems Conference. 2005: University of Memphis. 12. Kotulic, A.G. and J.G. Clark, Why there aren’t more information security research

studies. Information & Management, 2004 41: p. 597-607. 13. Straub, D.W. and R.J. Welke, Coping with Systems Risk: Security Planning Models for

Management Decision Making. MIS Quarterly, 1998. 22(4): p. 441-464. 14. Friedman, et al., Users’ Conceptions of Web Security: A Comparative Study. 2002,

University of Washington. 15. Von Solms, B., Information Security – The Third Wave. Computers & Security, 2000. 19:

p. 615-620. 16. Sabelfeld, A. and A. Myers, Language-Based Information-Flow Security. IEEE Journal

on Selected Areas in Communications, 2003. 21(1): p. 5-19. 17. McCumber, J., Assessing and Managing Security Risk in IT Systems. 2005: Auerbach

Publications. 18. Gaines, R.S. and N.Z. Shapiro, Some Security Principles and Their Application to

Computer Security. ACM SIGOPS Operating Systems Review, 1978. 12(3): p. 19-28. 19. Klein, S.A., Information Security Considerations in Open Systems Architectures. IEEE

Transactions on Power System, 1993. 8(1). 20. Dobson, J. New Security Paradigms: What Other Concepts Do We Need as Well? in

1992-1993 Workshop of New Security Paradigms. 1993: ACM Press. 21. Sizer, R. and J. Clark, Computer Security - A Pragmatic Approach For Managers.

Information Age, 1989. 11(2): p. 88-98. 22. Solms, R.V., Information Security Management: why standards are important.

Information Management & Computer Security, 1999. 7(1): p. 5-5(1).

- 42 -

23. Cronbach, L.J., Coefficient Alpha and the internal structure of tests. Psychometricka, 1951. 16(3): p. 297-334.

24. Foa, E.B., et al., Reliability and validity of a brief instrument for assessing post-traumatic stress disorder. Journal of Traumatic Stress (Historical Archive), 1993. 6(4): p. 459 - 473.

25. Kandel, D.B., V.H. Raveis, and M. Davies, Suicidal ideation in adolescence: Depression, substance use, and other risk factors. Journal of Youth and Adolescence (Historical Archive), 1991. 20(2): p. 289 - 309.

26. Little, P., et al., Observational study of effect of patient centredness and positive approach on outcomes of general practice consultations. BMJ, 2001. 323(7318): p. 908-911.

27. Maxim, P.S., Quantitative Research Methods in the Social Sciences. 1999: Oxford University Press, Inc.

28. Nunnaly, J., Psychometric theory. 1978, New York: McGraw-Hill. 29. Wallen, N. and J. Fraenkel, Educational Research: A Guide to the Process. 2001,

Mahwah, N.J.: Lawrence Erlbaum Associates. 30. Hinkle, D., S. Jurs, and W. Wiersma, Applied statistics for the behavioral sciences. 2nd

ed. 1988, Boston: Houghton Mifflin. 31. DeVellis, R.F., Scale Development: Theory and Applications. 1991, Newbury Park, CA:

SAGE Publications. 32. Moss, P.A., et al., Interrogating the generalizability of portfolio assessments of beginning

teachers: A qualitative study. Education Policy Analysis Archives, 2004. 12(32). 33. Swanson, D., G.R. Norman, and R.L. Linn, Performance-based assessment: Lessons

from the health professions. Educational Researcher, 1995. 24(5): p. 5-11,35. 34. Afifi, A.A. and R.M. Elashoff, Missing Observations in Multivariate Statistics –I. Review

of The Literature. JASA, 1966. 61: p. 595-604. 35. Hair, J., Joseph F., et al., Multivariate Data Analysis. 1998: Prentice Hall. 36. Fornell, C. and D.F. Larcker, Evaluating Structural Equation Models with Unobservable

Varibles and Measurement Error. Journal of Marketing Research, 1981. 18. 37. Parasuraman, A., et al., A Conceptual Model of Service Quality and its Implications for

Future Research. Journal of Marketing Research, 1985. 49: p. 41-50. 38. Winch, G., A. Usmani, and A. Edkins, Towards Total Project Quality: A Gap Analysis

Approach. Construction Management and Economics 1998. 16(2 ): p. 193-207. 39. Min, H. and H. Min, Competitive Benchmarking of Korean Luxury Hotels Using the

Analytic Hierarchy Process and Competitive Gap Analysis. The Journal of Services Marketing, 1996. 10(3): p. 58-72.

40. Davis, S., K. Siau, and K. Dhenuvakonda, A Fit-Gap Analysis of E-Business Curricula vs. Industry Needs. Communications of the ACM 2003. 46(12): p. 167-177.

- 43 -

Appendix I – Survey Instrument Form # 01-23

Organization Code* ________________

Your assigned Code* _______________

* If assigned by your survey coordinator

Towards Total Security Quality Management (TSQM) MIT’s Extended Enterprise Security Survey

Introduction

The following survey is part of a research project at MIT to develop a holistic framework to study enterprise security within and between organizations. Your responses to the following survey will provide us valuable insight about extended enterprise security. The extended enterprise includes an organization and its suppliers, customers, partners, and competitors. Extended enterprise security is concerned with security both within and between these organizations. The survey should take you about 20 minutes to fill out. Note about confidentiality: Your responses to questionnaire items will not be revealed to your organization or to any other organization. Only aggregate results will be used in our analyses. Your participation in this survey is completely voluntary and you are free to decline to answer any or all questions. If you would like to receive a copy of our research results, please provide your email address at the bottom of the survey. General Instructions 1. What does it mean by “assessment” and “importance”?

The survey asks you to give your impression of the “assessment” and “importance” of various security issues.

• “Assessment,” means your view of how well your organization is doing on these issues.

• “Importance” means your view of how important this issue is to you.

2. There is no right or wrong answer to any question. We are asking for your view.

You may not know exact details about your company’s security. We are not asking for these details, but asking for your views. Please give your best estimate.

3. What is “Partner Organization”? The survey also asks you to give your impressions of “assessment” and “importance” for ONE partner organization. This partner organization should be one of your suppliers, if feasible. Alternatively, please select a customer or a collaborator organization.

4. There is no right or wrong answer about a partner’s security.

We are asking your views of the partner organization’s security, you do not need to know exact details. Please give your best estimate. If you have no knowledge at all of an aspect of your partner security, you may leave that question blank.

- 44 -

Extended Enterprise Security Survey Section 1: Your Organization

Your Organization/Company

Organization Name__________________________________________________________

Industry____________________________________________________________________

Approximate total number of employees in your entire organization: ________________

Your Job Title and Work Role ________________________________________________

___________________________________________________________________________

Department/Division/Group___________________________________________________

In my organization, I am a: _____(1) Executive (CEO,CFO, VP etc.) _____(2) Functional or Line Manager _____(3) Professional (Consultant, Engineer, In-house Expert, etc.) _____(4) Other Organizational Member

In my organization, I work in the area of: _____(1) Business Security Policy and Management _____(2) IT Security _____(3) IT but not in Security, _____(4) General/Physical Security, _____(5) Not in Security or in IT.

Section 2: Your Partner Organization

Pick one partner organization for answering these questions. The survey administrator may give you additional instructions about picking a partner origination. All answers about your partner organization should be about ONE specific organization.

Your Partner Organization/Company

Partner Organization’s Name (optional)__________________________________________

Partner’s Industry_____________________________________________________________

Approximate total number of employees in your partner organization: ________________

Your Partner Organization is your organization’s: _____(1) Supplier ____(2) Customer ____(3) Collaborator ____(4) Competitor

Major Group/Division/Department you usually work with:

_______________________________________________________________________

- 45 -

Section 3: Security Questions

Assessment Scale: 1= In my view, this security statement is true to a very SMALL extent in my (partner) organization. 7= In my view, this security statement is true to a very LARGE extent in my (partner) organization. Importance Scale: 1= In my view, it is NOT at all Important to me that my (partner) organization address this security statement. 7= In my view, it is VERY Important to me that my (partner) organization address this security statement. Procedurally, you can answer all four columns at once, or, if you prefer, you could answer the first two columns about your organization first and then come back and answer the right two columns about your partner.

Your Organization

Partner Organization

Questions

Assessment Importance Assessment Importance 1 The organization’s data and networks

are rarely tampered with by unauthorized access.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

2 In the organization, security is adequately funded.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

3 Customers trust the organization not to disclose data about them.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

4 The organization’s security strategy sets direction for its security practices.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

5 Business managers in the organization are involved with IT security policies.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

6 The organization has enough IT security specialists to cover its security needs.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

7 The organization checks the identity of users before allowing access to data and networks.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

8 The organization has adequate safe guards against internal and external threats to its data and networks.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

9 The burden of security policies on people in the organization is minimal.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

10 The organization has policies for regularly-scheduled security audits.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

11 People in the organization are knowledgeable about IT security tools and practices.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

12 In the organization, security funds are appropriately distributed based on needs.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

13 In the organization, the IT group takes security seriously.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

14 The organization’s data and networks are only available to approved users.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

- 46 -

Your

Organization Partner

Organization

Questions Assessment Importance Assessment Importance

15 The organization’s network is rarely unavailable due to attacks (for example, denial of service, hacker break-ins, viruses and worms).

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

16 The organization has adequate policies for when and how data can be shared.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

17 The organization has adequate technology for supporting security

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

18 People in the organization carefully follow good security practices.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

19 The organization has a well-defined and communicated security strategy.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

20 Security is a funding priority in the organization.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

21 The organization uses its IT security resources effectively to improve security.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

22 The organization has adequate policies about user identifications, passwords, and access privileges.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

23 The organization improves its security by learning from previous attacks on its data and networks.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

24 The organization protects privacy of personal data (for example, customer data, data about employees).

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

25 The organization has adequate procedures for ensuring the physical security of buildings and equipment.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

26 People in the organization can be trusted not to tamper with data and networks.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

27 Security is a business agenda item for top executives in the organization.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

28 The organization has enough security personnel to cover its security needs.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

29 The organization has a well-defined policies and procedures for data and network security..

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

30 The organization has procedures for detecting and punishing security violations.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

- 47 -

Your

Organization Partner

Organization

Questions Assessment Importance Assessment Importance

31 This organization has someone who manages the use, storage, and sharing of confidential data.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

32 People in the organization can be trusted to engage in ethical practices with data and networks.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

33 In the organization, business managers help set the security strategy.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

34 The organization makes good use of available funds for security.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

35 The organization’s security strategy is well publicized in the organization.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

36 The organization provides access to data and networks to legitimate users.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

37 The organization has a rapid response team ready for action when attacks occur.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

38 The organization provides good protection of confidential corporate data.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

39 In the organization, people are aware of good security practices.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

40 The organization’s data and networks are usually available when needed.

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

Final question: What is the biggest concern that you have about security (need not be included in the

questions above): _____________________________________________________________________

____________________________________________________________________________________

____________________________________________________________________________________

Any other comments or suggestions?: Thank you very much, The MIT TSQM Team

- 48 -

Appendix II – List of Components for Each Construct and MA vs. MI Gap Analysis

Qs # Qs #

MIT Harris MA MI Gap MA MI Gap MA STD MI STD

Company Data (200 data points)

Merged Data (1400 data points)

Results of Gaps Within an Individual’s Organization: MA vs. MI (1-7 Scale)

Accessibility

7 4The organization checks the identity of users before allowing access to data and networks.

6.123 (1.107)

6.520 (0.805) 0.397** 5.820 6.080 0.259 1.706 1.568

14 11The organization’s data and networks are only available to approved users.

6.110 (1.162)

6.548 (0.729) 0.438** 5.803 6.102 0.299 1.680 1.536

22 xThe organization has adequate policies about user identifications, passwords, and access privileges.

6.041 (1.054)

6.611 (0.544) 0.569*** x x x x x

36 30The organization provides access to data and networks to legitimate users.

6.197 (0.906)

6.633 (0.569) 0.437*** 5.676 5.972 0.296 1.652 1.558

40 34The organization’s data and networks are usually available when needed.

6.319 (0.827)

6.722 (0.585) 0.403** 5.623 6.066 0.443 1.641 1.535

Vulnerability

1 1The organization’s data and networks are rarely tampered with by unauthorized access.

5.375 (1.437)

6.603 (0.612) 1.222*** 5.447 6.120 0.673 1.808 1.555

8 5The organization has adequate safe guards against internal and external threats to its data and networks.

5.904 (0.909)

6.630 (0.612) 0.726*** 5.402 6.032 0.630 1.688 1.532

15 xThe organization’s network is rarely unavailable due to attacks (for example, denial of service, hacker break-ins, viruses and worms).

6.306 (1.020)

6.597 (0.765) 0.292** x x x x x

23 19The organization improves its security by learning from previous attacks on its data and networks.

6.333 (0.987)

6.606 (0.644) 0.282** 5.278 5.799 0.521 1.822 1.659

37 31The organization has a rapid response team ready for action when attacks occur.

5.847 (1.271)

6.403 (0.857) 0.556*** 4.875 5.690 0.814 2.045 1.767

Confidentiality

3 xCustomers trust the organization not to disclose data about them.

5.946 (1.130)

6.466 (0.903) 0.534** x x x x x

16 12The organization has adequate policies for when and how data can be shared.

5.527 (1.226)

6.384 (0.640) 0.849*** 5.381 5.893 0.512 1.759 1.566

24 20The organization protects privacy of personal data (for example, customer data, data about employees).

6.268 (1.024)

6.671 (0.602) 0.414** 5.549 6.128 0.579 1.776 1.497

31 xThis organization has someone who manages the use, storage, and sharing of confidential data.

5.549 (1.377)

6.254 (0.867) 0.704*** x x x x x

38 32The organization provides good protection of confidential corporate data.

5.916 (1.006)

6.493 (0.697) 0.578*** 5.475 6.006 0.531 1.716 1.551

x 18 (22)

The organization has adequate policies about user identifications, passwords, and access privileges.

x x x 5.576 5.942 0.366 1.738 1.576

Qs # Qs #

MIT Harris MA MI Gap MA MI Gap MA STD MI STD

Company Data (200 data points)

Merged Data (1400 data points)

- 49 -

Financial Resources For Security

2 xIn the organization, security is adequately funded.

5.608 (1.115)

6.392 (0.741) 0.784*** x x x x x

12 9In the organization, security funds are appropriately distributed based on needs.

5.123 (1.229)

6.014 (0.934) 0.890*** 4.712 5.313 0.601 1.838 1.736

20 16 Security is a funding priority in the organization. 5.137 (1.167)

6.085 (0.956) 0.958*** 4.603 5.407 0.804 1.923 1.730

28 23The organization has enough security personnel to cover its security needs.

5.282 (1.292)

6.254 (0.833) 0.972*** 4.858 5.690 0.833 1.894 1.641

34 28The organization makes good use of available funds for security.

5.648 (1.057)

6.141 (0.884) 0.493*** 4.922 5.540 0.619 1.824 1.669

IT Resources For Security

5 xBusiness managers in the organization are involved with IT security policies.

4.878 (1.631)

5.959 (1.093) 1.055*** x x x x x

6 3The organization has enough IT security specialists to cover its security needs.

5.137 (1.289)

6.194 (0.904) 1.014*** 4.965 5.753 0.788 1.887 1.644

13 10In the organization, the IT group takes security seriously.

6.411 (0.931)

6.589 (0.620) 0.178~ 5.667 5.978 0.312 1.764 1.607

17 13The organization has adequate technology for supporting security.

5.861 (1.022)

6.366 (0.663) 0.507** 5.261 5.914 0.653 1.743 1.550

21 17The organization uses its IT security resources effectively to improve security.

5.764 (0.925)

6.282 (0.729) 0.529*** 5.010 5.636 0.625 1.815 1.648

Business Strategy for Security

4 2The organization’s security strategy sets direction for its security practices.

5.689 (1.068)

6.324 (0.765) 0.635*** 5.450 5.824 0.374 1.705 1.598

19 xThe organization has a well-defined and communicated security strategy.

5.055 (1.258)

6.139 (0.915) 1.069*** x x x x x

27 22Security is a business agenda item for top executives in the organization.

5.592 (1.253)

6.085 (0.925) 0.493** 5.010 5.615 0.605 1.911 1.682

33 27In the organization, business managers help set the security strategy.

4.514 (1.619)

5.500 (1.443) 0.986*** 4.683 5.284 0.601 1.899 1.747

35 29The organization’s security strategy is well publicized in the organization.

5.000 (1.439)

6.056 (0.938) 1.056*** 4.721 5.440 0.720 2.049 1.799

Policy and Procedures for Security

10 7The organization has policies for regularly-scheduled security audits.

5.603 (1.189)

6.055 (1.018) 0.452** 5.051 5.600 0.550 1.939 1.688

25 xThe organization has adequate procedures for ensuring the physical security of buildings and equipment.

5.375 (1.410)

6.423 (0.736) 1.070*** x x x x x

29 24The organization has a well-defined policies and procedures for data and network security.

5.514 (1.156)

6.319 (0.736) 0.806*** 5.249 5.807 0.558 1.814 1.606

30 25The organization has procedures for detecting and punishing security violations.

5.306 (1.381)

6.250 (0.844) 0.944*** 4.956 5.673 0.717 1.929 1.669

x 15(19)

The organization has a well-defined and communicated security strategy.

x x x 4.931 5.736 0.805 1.892 1.621

Qs # Qs #

MIT Harris MA MI Gap MA MI Gap MA STD MI STD

Company Data (200 data points)

Merged Data (1400 data points)

- 50 -

Security Culture

9 xThe burden of security policies on people in the organization is minimal.

5.082 (1.466)

5.671 (1.256) 0.589** x x x x x

11 8People in the organization are knowledgeable about IT security tools and practices.

4.671 (1.477)

5.863 (1.165) 1.192*** 4.743 5.561 0.817 1.865 1.660

18 12People in the organization carefully follow good security practices.

5.000 (1.311)

6.250 (0.792) 1.236*** 4.760 5.845 1.085 1.789 1.564

26 21People in the organization can be trusted not to tamper with data and networks.

5.366 (1.184)

6.380 (0.730) 1.014*** 5.153 5.936 0.783 1.786 1.576

32 26People in the organization can be trusted to engage in ethical practices with data and networks.

5.347 (1.151)

6.319 (0.827) 1.972*** 5.222 5.914 0.692 1.755 1.554

33 33In the organization, people are aware of good security practices.

5.042 (1.329)

6.319 (0.861) 1.278*** 5.066 5.810 0.743 1.813 1.575

APPENDIX III - SAS CODE FOR THE CALCULATIONS Sample of SAS code used in analysis of data. Code snippets include everything needed to calculate each distinct item:

• Cronbach's Alpha • R-Squared of Constructs (Non-Diagonals) • R-Squared of Constructs (Diagonals) • My Assessment, My Importance, Overall • 2nd Level Segmentation

For example, it will show how to calculate all the Cronbach's Alpha for just the MA or just the R-Squared of Constructs value for just one particular construct. Also note, the same code was used for before and after data had been merged or regrouped since the method to find the gaps were still the same. -------------------------- START OF SAS CODE -------------------------- /* ************************************ CRONBACH'S ALPHA ********************************** */ DATA WORK.Harris; SET WORK.Harris; TITLE "Cronbach's Alpha MA"; proc corr alpha nomiss; var q701a4 q701a11 q701a30 q701a34; proc corr alpha nomiss; var q701a1 q701a5 q701a19 q701a31; proc corr alpha nomiss;

Notes: Numbers in parentheses represent standard deviations; ***Gap is significant at the 99.99% level; **Significant at the 99% level; *Significant at the 95% level; ~Significant at the 90% level. Data from comprehensive results are all significant at the 99.99% level.

- 51 -

var q701a12 q701a18 q701a20 q701a32; proc corr alpha nomiss; var q701a9 q701a16 q701a23 q701a28; proc corr alpha nomiss; var q701a3 q701a10 q701a13 q701a17; proc corr alpha nomiss; var q701a2 q701a22 q701a27 q701a29; proc corr alpha nomiss; var q701a7 q701a15 q701a24 q701a25; proc corr alpha nomiss; var q701a8 q701a14 q701a21 q701a26 q701a33; /* ************************************ R-SQUARED of CONSTRUCTS FOR USE IN CONSTRUCT VALIDITY TESTING SPECIFICALLY FOR DISCRIMINANT ANALYSIS (THE NON-DIAGONALS) ********************************** */ data WORK.Harris; SET WORK.Harris; TITLE "Setting the Constructs Value Through Using Mean Function"; AccessibilityMA = mean(of q701a4, q701a11, q701a30, q701a34); VulnerabilityMA = mean(of q701a1, q701a5, q701a19, q701a31); ConfidentialityMA = mean(of q701a12, q701a18, q701a20, q701a32); FinancialResourcesMA = mean(of q701a9, q701a16, q701a23, q701a28); ITResourcesMA = mean(of q701a3, q701a10, q701a13, q701a17); BusinessStrategyMA = mean(of q701a2, q701a22, q701a27, q701a29); SecurityPolicyMA = mean(of q701a7, q701a15, q701a24, q701a25); SecurityCultureMA = mean(of q701a8, q701a14, q701a21, q701a26, q701a33); proc corr data= WORK.Harris; TITLE "R-Squared for MA Constructs"; var AccessibilityMA VulnerabilityMA ConfidentialityMA FinancialResourcesMA ITResourcesMA BusinessStrategyMA SecurityPolicyMA SecurityCultureMA; /* ************************************ START OF CONFIRMATORY FACTOR ANALYSIS FOR USE IN CONSTRUCT VALIDITY TESTING SPECIFICALLY FOR CONVERGENT VALIDITY ANALYSIS (THE DIAGONALS) ********************************** */ proc calis data=WORK.Harris cov; TITLE "Diagonals"; lineqs q701a4 = ps04f1 F1 + e04, q701a11 = ps11f1 F1 + e11, q701a30 = ps30f1 F1 + e30, q701a34 = ps34f1 F1 + e34; std e04 = vare04, e11 = vare11, e30 = vare30, e34 = vare34, F1 = 1; var q701a4 q701a11 q701a30 q701a34; run;

- 52 -

/* ************************************ MY ASSESSMENT ********************************** */ proc means n mean std; TITLE "MA Constructs"; var AccessibilityMA q701a4 q701a11 q701a30 q701a34 VulnerabilityMA q701a1 q701a5 q701a19 q701a31 ConfidentialityMA q701a12 q701a18 q701a20 q701a32 FinancialResourcesMA q701a9 q701a16 q701a23 q701a28 ITResourcesMA q701a3 q701a10 q701a13 q701a17 BusinessStrategyMA q701a2 q701a22 q701a27 q701a29 SecurityPolicyMA q701a7 q701a15 q701a24 q701a25 SecurityCultureMA q701a8 q701a14 q701a21 q701a26 q701a33; proc means n mean std; Title "Overall MA Constructs"; var AccessibilityMA VulnerabilityMA ConfidentialityMA FinancialResourcesMA ITResourcesMA BusinessStrategyMA SecurityPolicyMA SecurityCultureMA; /* ************************************ MY IMPORTANCE ********************************** */ proc means n mean std; TITLE "MI Constructs"; var AccessibilityMI q702a4 q702a11 q702a30 q702a34 VulnerabilityMI q702a1 q702a5 q702a19 q702a31 ConfidentialityMI q702a12 q702a18 q702a20 q702a32 FinancialResourcesMI q702a9 q702a16 q702a23 q702a28 ITResourcesMI q702a3 q702a10 q702a13 q702a17 BusinessStrategyMI q702a2 q702a22 q702a27 q702a29 SecurityPolicyMI q702a7 q702a15 q702a24 q702a25 SecurityCultureMI q702a8 q702a14 q702a21 q702a26 q702a33; proc means n mean std; Title "Overall MI Constructs"; var AccessibilityMI VulnerabilityMI ConfidentialityMI FinancialResourcesMI ITResourcesMI BusinessStrategyMI SecurityPolicyMI SecurityCultureMI; /* ****************************** OVERALL ********************************* */ proc means n mean std; Title "Overall MA Constructs"; var AccessibilityMA VulnerabilityMA ConfidentialityMA FinancialResourcesMA ITResourcesMA BusinessStrategyMA SecurityPolicyMA SecurityCultureMA; /* ************************************ SECOND LEVEL ANALYSIS CLASS SEGMENTS DIFFERENT AREAS ********************************** */ proc means n mean std; Title "Overall MA Constructs"; var AccessibilityMA VulnerabilityMA ConfidentialityMA FinancialResourcesMA ITResourcesMA BusinessStrategyMA SecurityPolicyMA SecurityCultureMA;

- 53 -

class area cosize; -------------------------- END OF SAS CODE --------------------------

APPENDIX IV - CRONBACH’S ALPHA ANALYSIS Cronbach’s Alpha analysis using preliminary data to determine how well each question fit within a specific construct.

Cronbach’s Alpha analysis using comprehensive data.

Reliability - Cronbach's Alpha ValuesMA MI Qn Removed Qn Added

Accessibility 0.695488 0.659241 22Vulnerability 0.592442 0.610124 15Confidentiality 0.660975 0.70541 3, 31 22FinancialResources 0.740053 0.748803 2ITResources 0.640141 0.701367 5BusinessStrategy 0.807072 0.711837 19SecurityPolicy 0.745929 0.671639 25 19SecurityCulture 0.682589 0.706474 9

Construct Validity - Convergent and Discriminant ValidityAccessibility Vulnerability Confidentiality FinancialResITResources BusinessStr SecurityPolicy SecurityCulture

Accessibility 0.64772 0.47097 0.52719 0.35937 0.47631 0.18955 0.25447 0.24363Vulnerability 0.47097 0.53841 0.45185 0.50057 0.62317 0.57171 0.58709 0.46202Confidentiality 0.52719 0.45185 0.61651 0.49552 0.57952 0.42095 0.53149 0.34463FinancialResources 0.35937 0.50057 0.49552 0.76924 0.69041 0.66325 0.58769 0.58476ITResources 0.47631 0.62317 0.57952 0.69041 0.62101 0.53096 0.54148 0.48422BusinessStrategy 0.18955 0.57171 0.42095 0.66325 0.53096 0.86386 0.72785 0.66837SecurityPolicy 0.25447 0.58709 0.53149 0.58769 0.54148 0.72785 0.77396 0.5285SecurityCulture 0.24363 0.46202 0.34463 0.58476 0.48422 0.66837 0.5285 0.58365

In the Construct Validity table, diagonals >0.50 indicates good convergent validity, and having the values of the columns of each construct lower than the diagonals indicates good discriminant validity.

- 54 -

APPENDIX V – Methodology for Regrouping Merged Data 1. Changes in Industry Records going out of these industries Banking 35Healthcare & Social Assistance 140Manufacturing 43Retail Trade 24Technology Services 30Communication & Telecomm 15Education 0Total 287 Records going into these industries Banking 58Healthcare & Social Assistance 74Manufacturing 38Retail Trade 57Technology Services 43Communication & Telecomm 12Education 0

Reliability - Cronbach's Alpha ValuesMA MI

Accessibility 0.90758 0.93701Vulnerability 0.83714 0.91012Confidentiality 0.91808 0.94026FinancialResources 0.91878 0.92768ITResources 0.91023 0.93680BusinessStrategy 0.86877 0.89343SecurityPolicy 0.92184 0.93834SecurityCulture 0.92188 0.94296

For good reliability, want Cronbach's Alpha Values to be >0.6, better if >0.7

Construct Validity - Convergent and Discriminant Validity

Accessibility Vulnerability Confidentiality Financial Resources ITResources Business

StrategySecurity Policy

Security Culture

Accessibility 0.96606 0.82730 0.86289 0.72385 0.81193 0.75817 0.75993 0.77299Vulnerability 0.82730 0.89537 0.85986 0.83791 0.88582 0.83439 0.85439 0.83308Confidentiality 0.86289 0.85986 0.97320 0.79234 0.86494 0.83070 0.85867 0.85271FinancialResources 0.72385 0.83791 0.79234 0.97366 0.88814 0.86196 0.86675 0.84406ITResources 0.81193 0.88582 0.86494 0.88814 0.96623 0.84474 0.87556 0.85137BusinessStrategy 0.75817 0.83439 0.83070 0.86196 0.84474 0.93056 0.88216 0.85515SecurityPolicy 0.75993 0.85439 0.85867 0.86675 0.87556 0.88216 0.97341 0.84505SecurityCulture 0.77299 0.83308 0.85271 0.84406 0.85137 0.85515 0.84505 0.96241

For good Convergent Validity, want diagonals >0.50For good Discriminant Validity, want all values in columns of each construct to be lower than the diagonals.

- 55 -

Total 282* Examples and cases where data was regrouped into different industries: Case 1: At least 2 out of 3 “checks” are consistent, but industry entered was obviously wrongly classified. Ex. 1: Banking Healthcare & Social Assistance

Role: nurse, Area: nursing department Ex. 2: Health & Social Assistance Retail Role: assistant manager of retail store, Area: traditional store Ex. 3: Health & Social Assistance Manufacturing Role: safety manager at refinery, Area: safety, health & environmental, Company: Exxon Mobil Case 2: Industry entered is obviously inconsistent. Based on company name, industry was regrouped (used Hoover’s as a reference to decide which industry to place them in).. Ex. 1: Banking Retail Role: business professional, Area: computers, Company: Best Buy Ex. 2: Health & Social Assistance Banking Role: data entry clerk, Area: operations, Company: USBank Ex. 3: Banking Telecomm Role: telecomm specialist, Area: network services, Company: AT&T / SBC Ex. 4: Manufacturing Banking Role: tech manager, Area: credit card marketing operations, Company: JP Morgan Case 3: Misclassification of industry due to industry overlaps or confusion. Again, company name was used as the determining factor. Ex. 1: Tech Services Telecomm Role: consultant, Area: Training, Company: Verizon Ex. 2: Manufacturing Retail Role: Department Manager, Area: Grocery, Company: Wal-Mart Case 4: The same companies from the company data set had many different industries. They were all regrouped into the same industry for consistency. Ex. 1: Retail, Manufacturing, or Tech Services Telecomm

- 56 -

Company: Cisco, Nortel Ex. 2: [no entry] Manufacturing Company: Embraer, Lockheed Martin 2. Changes in Roles Records going out of these roles Executives 69 Managers 152 Professionals 241 Others 554 Total 1016

Records going into these roles Executives 109 Managers 288 Professionals 439 Others 136 Total 972

Note that most of the changes came from the category “Others” so it was as if we had 554 extra data points to analyze more specifically. Sometimes “Professionals” and “Others” were not differentiable so whatever the respondent had entered was left as is. Examples and cases where data was regrouped into different roles: Case 1: Misclassification of role due obvious error Ex. 1: Exec Manager Role: Department Manager, Area: Grocery, Company: Wal-Mart Ex. 2: Exec Professional Role: Registered Nurse, Area: Resource Team, Company: Carondelet Health Ex. 3: Exec Others Role: x-ray secretary, Area: x-ray Ex. 4: Manager Exec Role: President, Area: Administration Ex. 5: Manager Professional Role: programmer, Area: IT

- 57 -

Case 2: Selected “others” in pull down menu, so they were manually classified if applicable; Most of the changes came from this category Ex. 1: Others Exec Role: Director, Area: Executive Ex. 2: Others Exec Role: CFO, Area: Finance Ex. 3: Others Manager Role: Marketing Manager, Area: Consumer Marketing Ex. 4: Others Professionals Role: Registered Nurse, Area: Mental health Ex.5: Others Professionals Role: Quality Assurance analyst, Area: IT Case 3: Anything with “manager” or “supervisor” was classified as manager role. Ex. 1: Professional Manager Role: product manager, Area: treasury management Ex. 2: Professional Manager 3. Changes in Area Records going out of these areas Business Policy 14 IT Security 34 IT, Not Security 64 General security 27 Not Security 60 Total 199*

Records going into these areas Business Policy 11 IT Security 1 IT, Not Security 33 General security 3 Not Security 73 Total 121*

- 58 -

* Note discrepancies in the total count were result of the fact that some people had entered more than one area in the pull down menu. Items regrouped in “Area” were much more straightforward than the other two categories Examples and cases where data was regrouped into different areas: Ex. 1: General Security IT Security Role: information security, Area: IT Ex. 2: General Security Not Security Role: Physician in group practice, Area: medical staff

APPENDIX VI – Miscellanious

0

0.2

0.4

0.6

0.8

1

1.2Accessibility

Vulnerability

Confidentiality

Financial Resources

IT Resources

Business Strategy

Security Policy

Security Culture

Construct Variation MI-MAConstruct Variation |PA-MA|Construct Variation |PI-MI|Construct Variation PI-PA

Security Gap Overall for Pilot Data.

- 59 -

4

5

6

7

AccessibilityMA

VulnerabilityMA

ConfidentialityMA

FinancialResourcesMA

ITResourcesMA

BusinessStrategyMA

SecurityPolicyMA

SecurityCultureMA

Business Security Policy andManagementIT Security

IT but not in Security

General/Physical Security

Not in Security or in IT

Security Assessment by Area for Pilot Data.