Understanding the Risk Management Framework & (ISC)2 CAP Module 9: Assess Controls
108
-
Upload
donald-e-hester -
Category
Government & Nonprofit
-
view
138 -
download
1
Transcript of Understanding the Risk Management Framework & (ISC)2 CAP Module 9: Assess Controls
Presenter
Presentation Notes
© 2016 Maze & Associates Revision 10 (April 2016) Images from Microsoft Clipart unless otherwise noted, Other Sources: NIST and Donald E. Hester Picture: Muir Beach, North of San Francisco, CA, Photo by Donald E. Hester all rights reserved
Categorize
Select
Implement
Assess
Authorize
Monitor
Presenter
Presentation Notes
The Six Steps in the RMF
Presenter
Presentation Notes
RMF Step 4 Assess Security Controls Assessment Preparation Develop, review, and approve a plan to assess the security controls Security Control Assessment Assess the security controls in accordance with the assessment procedures defined in the security assessment plan Security Assessment Report Prepare the security assessment report documenting the issues, findings, and recommendations from the security control assessment Remediation Actions Conduct initial remediation actions on security controls based on the findings and recommendations of the security assessment report and reassess remediated control(s), as appropriate
Presenter
Presentation Notes
Assessing Security Controls Picture: Flowers: Crown Imperial (fritillaria imperialis), Mt. Vernon, VA; Photo by Donald E. Hester all rights reserved NIST SP 800-115 Technical Guide to Information Security Testing and Assessment NIST SP 800-53 A Rev 1 (Draft May 2010) Guide for Assessing the Security Controls in Federal Information Systems and Organizations Read: Official (ISC)2 Guide to CAP CBK Second Edition Chapter 5
Categorize
Select
Implement
Assess
Authorize
Monitor
Presenter
Presentation Notes
Risk Management Framework (RMF) NIST SP 800-37 Rev 1, § 2.1
Presenter
Presentation Notes
Situation Organizations are becoming increasingly dependent on technology and the Internet The loss of technology or the Internet would bring operations to a halt The need for security increases as our dependence on technology increases Management wants to have assurance that technology has the attention it deserves Can you live without technology? What would happen to operations if the Internet was down for an extended period of time?
Presenter
Presentation Notes
Management Questions Does our current security posture address what we are trying to protect? Do we know what we need to protect? Where can we improve? Where do we start? Are we compliant with laws, rules, contracts and organizational policies? What are your risks?
Presenter
Presentation Notes
Reasons for Assessments Provide Assurance Demonstrate due diligence Make risk based decisions Required by FISMA FISMA Section 3544 requires the “periodic testing and evaluation of the effectiveness of information security policies, procedures, and practices, to be performed with a frequency depending on risk, but no less than annually.” FISMA is available at http://csrc.nist.gov/drivers/documents/FISMA-final.pdf.
Presenter
Presentation Notes
Terms Certification Assessment Audit Review ST&E = Security Test & Evaluation Testing Evaluation Interviewing
Management Operational Technical
Implemented correctly Operating as intended Producing the desired outcome
factual basis for an authorizing official to render a security accreditation decision
Presenter
Presentation Notes
Certification (now Assessment) Detailed security review of an information system Comprehensive assessment of Management security controls Operational security controls Technical security controls To determine the extent to which the controls are Implemented correctly Operating as intended Producing the desired outcome Providing the factual basis for an authorizing official to render a security accreditation decision
An information security assessment is the process of determining how effectively an entity being assessed meets specific security objectives. Three types of assessment methods can be used to accomplish this—testing, examination, and interviewing.Assessment results are used to support the determination of security control effectiveness over time.- NIST SP 800-115
Presenter
Presentation Notes
Assessment An information security assessment is the process of determining how effectively an entity being assessed meets specific security objectives. Three types of assessment methods can be used to accomplish this—testing, examination, and interviewing. Assessment results are used to support the determination of security control effectiveness over time. - NIST SP 800-115 An information security assessment is the process of determining how effectively an entity being assessed (e.g., host, system, network, procedure, person—known as the assessment object) meets specific security objectives. Three types of assessment methods can be used to accomplish this—testing, examination, and interviewing. NIST SP 800-115 Security Control Assessment The testing and/or evaluation of the management, operational, and technical security controls in an information system to determine the extent to which the controls are implemented correctly, operating as intended, and producing the desired outcome with respect to meeting the security requirements for the system. NIST SP 800-37 Rev 1
“Independent review and examination of records and activities to assess the adequacy of system controls and ensure compliance with established policies and operational procedures.”- CNSS Instruction No. 4009
Presenter
Presentation Notes
Audit “Independent review and examination of records and activities to assess the adequacy of system controls and ensure compliance with established policies and operational procedures.” - CNSS Instruction No. 4009
“Examination and analysis of the safeguards required to protect an information system, as they have been applied in an operational environment, to determine the security posture of that system.”- CNSSI No. 4009
Presenter
Presentation Notes
Security Testing and Evaluation (ST&E) “Examination and analysis of the safeguards required to protect an information system, as they have been applied in an operational environment, to determine the security posture of that system.” - CNSSI No. 4009
Testing is the process of exercising one or more assessment objects under specified conditions to compare actual and expected behaviors.
Examination is the process of checking, inspecting, reviewing, observing, studying, or analyzing one or more assessment objects to facilitate understanding, achieve clarification, or obtain evidence.
Interviewing is the process of conducting discussions with individuals or groups within an organization to facilitate understanding, achieve clarification, or identify the location of evidence.
Source NIST SP 800-115
Presenter
Presentation Notes
Testing, Examination and Interviewing Testing is the process of exercising one or more assessment objects under specified conditions to compare actual and expected behaviors. Examination is the process of checking, inspecting, reviewing, observing, studying, or analyzing one or more assessment objects to facilitate understanding, achieve clarification, or obtain evidence. Interviewing is the process of conducting discussions with individuals or groups within an organization to facilitate understanding, achieve clarification, or identify the location of evidence. Source NIST SP 800-115
“The security certification and accreditation process is designed to ensure that an information system will operate with the appropriate management review, that there is ongoing monitoring of security controls, and that reaccreditation occurs periodically.” NIST SP 800-100
Presenter
Presentation Notes
Risk Management Framework “The security certification and accreditation process is designed to ensure that an information system will operate with the appropriate management review, that there is ongoing monitoring of security controls, and that reaccreditation occurs periodically.” NIST SP 800-100
“Security certification is a comprehensive assessment of the management, operational, and technical security controls in an information system, made in support of security accreditation, to determine the extent to which the controls are implemented correctly, operating as intended, and producing the desired outcome with respect to meeting the security requirements for the system. The results of a security certification are used to reassess the risks and update the system security plan, thus providing the factual basis for an authorizing official to render a security accreditation decision.” NIST SP 800-100
Presenter
Presentation Notes
Scope Scope of the assessment (certification) should include the controls of the entire system being certified Using the SSP the assessor will start by reviewing the listed controls May identify additional areas of weakness “Security certification is a comprehensive assessment of the management, operational, and technical security controls in an information system, made in support of security accreditation, to determine the extent to which the controls are implemented correctly, operating as intended, and producing the desired outcome with respect to meeting the security requirements for the system. The results of a security certification are used to reassess the risks and update the system security plan, thus providing the factual basis for an authorizing official to render a security accreditation decision.” NIST SP 800-100
Presenter
Presentation Notes
Multiple levels of assessment NIST SP 800-37 Rev 1, § 2.1
Presenter
Presentation Notes
Program Level The Office of Inspector General (OIG) will conduct an audit of the agency’s FISMA compliance. (Program Level) The OIG will select a number of individual systems to audit. These systems will already have an ATO. OIG will conduct the assessment with internal staff or external contractors A memorandum for the OIG will be given to the head of the agency and the Office of Management and Budget (OMB) As of 2010 OIGs will use CyberScope for FISMA reporting CyberScope is the platform for the FY 2010 FISMA submission process.
FromOMB M-10-15 - FY 2010
Presenter
Presentation Notes
OIG Inspector Generals (IGs) are required to assess agency performance in the following programs: Certification and Accreditation Configuration Management Security Incident Management Security Training Remediation/Plans of Actions and Milestones Remote Access Identity Management Continuous Monitoring Contractor Oversight Contingency Planning From OMB M-10-15 - FY 2010 Reporting Instructions for the Federal Information Security Management Act and Agency Privacy Management
Presenter
Presentation Notes
System Level The testing and/or evaluation of the management, Operational, and technical security controls in an information system to: Determine the extent to which the controls are implemented correctly, Operating as intended, and Producing the desired outcome Security Control Assessor (Certification Agent) - Pre-Authorization The individual, group, or organization responsible for conducting a security control assessment Conducts an assessment at the system level System Owner - Maintenance Phase Perform various assessments during the lifecycle of the system
Categorize
Select
Implement
Assess
Authorize
Monitor
Presenter
Presentation Notes
Risk Management Framework (RMF) NIST SP 800-37 Rev 1, § 2.1
Planning
Information Gathering
Business Process Assessment
Technology Assessment
Risk Analysis & Reporting
Presenter
Presentation Notes
Assessment Lifecycle Organizations should develop an information security assessment policy to provide direction and guidance for their security assessments. This policy should identify security assessment requirements, and hold accountable those individuals responsible for ensuring that assessments comply with the requirements. It should address: Organizational requirements with which assessments must comply Appropriate roles and responsibilities (at a minimum, for those individuals approving and executing assessments) Adherence to established methodology Assessment frequency Documentation requirements, such as assessment plans and assessment results. Planning Information Gathering Business Process Assessment Technology Assessment Risk Analysis & Reporting
Presenter
Presentation Notes
Common Types of Assessments Vulnerability Assessment Penetration Test Application Assessment Code Review Standard Audit/Review Compliance Assessment/Audit Configuration Audit Wireless Assessment Physical/Environmental Assessment Policy Assessment
use independent assessorself-assessment
Presenter
Presentation Notes
Determine Your Scope What will be the scope of the assessment? Network (Pen Test, Vul Scan, wireless) Application (Code or Vul scan) Process (business or automated) How critical is the system you are assessing? High, medium – use independent assessor Low – self-assessment Audit Risk What is the risk that an auditor will miss a material error (Important or significant) What level of risk is the auditor willing to accept Assessment Objective There are many factors to consider when determining which technical testing and examination techniques should be used for a particular assessment. An organization should first determine its assessment objectives, such as focusing on verifying compliance with a particular mandate, verifying a system’s security as part of certification and accreditation (C&A) activities, identifying exploitable vulnerabilities in a group of systems, or evaluating intrusion detection system and incident handling procedure performance. Next, the organization should select the classes of techniques (e.g., review, target identification and analysis, target vulnerability validation) to be used to obtain information that supports those objectives, and specific techniques within each selected class. For some testing techniques, the organization must also determine the assessors’ viewpoint (e.g., internal versus external, covert versus overt) and select corresponding techniques.
Presenter
Presentation Notes
Material Material in the context of risk refers to an error that should be considered significant to any part of the area in question or to anyone one with vested interest. The failure or absence of a control (or combination of controls) leaves the organization highly susceptible to the occurrence of a threat Materiality is often left to professional judgment. Take into consideration: Effect on the organization as a whole What types of errors or irregularities can be expected Any illegal acts that may arise Materiality is more difficult for IS auditors than it is for financial auditors
Presenter
Presentation Notes
Level of Effort Testing levels will be dictated by the sensitivity of the data and criticality of the system. Audit risk and system risk are not the same but are tied closely together Lower risk systems May allow for a self-assessment (less independence) Checklist based Higher risk systems Will require independent review (more independence) Testing Sample sizes increase with risk System risk is based upon the sensitivity and criticality of the system (based on the data in the system). Audit risk is the risk the auditor (assessor) will not catch or find a material (significant or important) error or control failure. When the assessor (auditor) has higher risk they will do more testing then normal.
Presenter
Presentation Notes
Risk & Audit Inherent Risk – risk that an error exists assuming there are not compensating controls (This will tie directly to the system criticality and sensitivity) Control Risk – the risk that a control failure or error will not be prevented or detected in a timely manner during the normal course of business. Detection Risk – the risk the auditor/assessor will use inadequate testing procedure and conclude that material errors do not exist when they actual do exist. Overall Audit Risk – Combination of risk related to different categories of controls.
Presenter
Presentation Notes
Independence The level of the system will dictate the level of independence required Must be independent of the entire process, especially the system owner Can use internal or external auditors Often use independent contractors There should be a level of review with the auditors May use automated tools for testing such as CAATs (Computer Aided Audit Tools)
Plan
ExecutePost
Execution
FISCAM NIST SP 800-115
Plan
PerformReport
Presenter
Presentation Notes
Assessment Life Cycle 3 phased information security assessment methodology Plan Execute Post Execution NIST SP 800-115 has a 3 step process This process mirrors FISCAM’s 3 phases: Plan, Perform and Report (Federal Information System Control Audit Manual) available from the US Government Accountability Office (GAO) Planning. Critical to a successful security assessment, the planning phase is used to gather information needed for assessment execution—such as the assets to be assessed, the threats of interest against the assets, and the security controls to be used to mitigate those threats—and to develop the assessment approach. A security assessment should be treated as any other project, with a project management plan to address goals and objectives, scope, requirements, team roles and responsibilities, limitations, success factors, assumptions, resources, timeline, and deliverables. Execution. Primary goals for the execution phase are to identify vulnerabilities and validate them when appropriate. This phase should address activities associated with the intended assessment method and technique. Although specific activities for this phase differ by assessment type, upon completion of this phase assessors will have identified system, network, and organizational process vulnerabilities. Post-Execution. The post-execution phase focuses on analyzing identified vulnerabilities to determine root causes, establish mitigation recommendations, and develop a final report. Source NIST SP 800-115 Plan, Perform, Report - Titles for the three phases used by FISCAM
Presenter
Presentation Notes
Plan (FISCAM) Understand the Overall Audit Objectives and Related Scope of the Information System Controls Audit Understand the Entity’s Operations and Key Business Processes Obtain a General Understanding of the Structure of the Entity’s Networks Identify Key Areas of Audit Interest Assess Information System Risk on a Preliminary Basis Identify Critical Control Points Obtain a Preliminary Understanding of Information System Controls Perform Other Audit Planning Procedures
Presenter
Presentation Notes
Perform (FISCAM) Understand Information Systems Relevant to the Audit Objectives Determine which IS Control Techniques are Relevant to the Audit Objectives For each Relevant IS Control Technique Determine Whether it is Suitably Designed to Achieve the Critical Activity and has been Implemented Perform Tests to Determine Whether such Control Techniques are Operating Effectively Identify Potential Weaknesses in IS Controls and Consider Compensating Controls
Presenter
Presentation Notes
Report (FISCAM) Evaluate the Effects of Identified IS Control Weaknesses Financial Audits, Attestation Engagements, and Performance Audits Consider Other Audit Reporting Requirements and Related Reporting Responsibilities
Presenter
Presentation Notes
Certification Process (NIST SP 800-37) Security Control Assessment Prepare for Assessment Conduct Assessment Document Results Security Certification Documentation Provide the certification findings & recommendations Update the system security plan (SSP) Prepare the plan of actions and milestones (POA&M) Assemble accreditation package
Task 1“Assemble any documentation and supporting materials necessary for the assessment of the security controls in the information system; if these documents include previous assessments of security controls, review the findings, results, and evidence.”Task 2 “Select, or develop when needed, appropriate methods and procedures to assess the management, operational, and technical security controls in the information system.” Task 3 “Assess the management, operational, and technical security controls in the information system using methods and procedures selected or developed.” Task 4 “Prepare the final security assessment report.”
NIST SP 800-37
Presenter
Presentation Notes
Security Control Assessment Tasks Task 1“Assemble any documentation and supporting materials necessary for the assessment of the security controls in the information system; if these documents include previous assessments of security controls, review the findings, results, and evidence.” Task 2 “Select, or develop when needed, appropriate methods and procedures to assess the management, operational, and technical security controls in the information system.” Task 3 “Assess the management, operational, and technical security controls in the information system using methods and procedures selected or developed.” Task 4 “Prepare the final security assessment report.” NIST SP 800-37
Task 1“Provide the information system owner with the security assessment report.”Task 2 “Update the system security plan (and risk assessment) based on the results of the security assessment and any modifications to the security controls in the information system.”Task 3 “Prepare the plan of action and milestones based on the results of the security assessment.”Task 4 “Assemble the final security accreditation package and submit to authorizing official.”
NIST SP 800-37
Presenter
Presentation Notes
Security Documentation Tasks Task 1“Provide the information system owner with the security assessment report.” Task 2 “Update the system security plan (and risk assessment) based on the results of the security assessment and any modifications to the security controls in the information system.” Task 3 “Prepare the plan of action and milestones based on the results of the security assessment.” Task 4 “Assemble the final security accreditation package and submit to authorizing official.” NIST SP 800-37
Presenter
Presentation Notes
Changes with NIST SP 800-37 Rev 1
Presenter
Presentation Notes
Assessment Tasks (NIST SP 800-37 Rev 1) Task 1: Identify and select the security control assessor(s) and determine if the selected assessor(s) possess the required degree of independence for the assessment. Task 2: Develop a plan to assess the security controls. Task 3: Review and approve the plan to assess the security controls. Task 4: Obtain appropriate documentation, records, artifacts, test results, and other materials needed to assess the security controls. Task 5: Assess the security controls in accordance with the assessment procedures defined in the security assessment plan. Task 6: Prepare the preliminary security assessment report documenting the issues, findings, and recommendations from the security control assessment. Note the changes from NIST SP 800-37 to NIST SP 800-37 rev 1 Notice there is now interaction between the assessor and the system owner. The system owner now gets a draft report and has the option to correct problems or dispute the findings. This is new for NIST but not for other types of Audits. These new steps are more in line with traditional financial audit procedures.
Presenter
Presentation Notes
Assessment Tasks (NIST SP 800-37 Rev 1) Task 7: Review the preliminary security assessment report. Task 8: If necessary, conduct remediation actions based on the preliminary security assessment report. Task 9: Assess the remediated security controls. Task 10: Update the security assessment report and prepare the executive summary. Task 11: If necessary, prepare an addendum to the security assessment report that reflects the initial results of the remediation actions taken and provides the information system owner or common control provider perspective on the assessment findings and recommendations. Task 12: Update the security plan based on the findings and recommendations of the security assessment report and any remediation actions taken. Task 13: Prepare the plan of action and milestones based on the findings and recommendations of the security assessment report.
Presenter
Presentation Notes
Assessor Selection The assessor should have the level of independence required for the system being evaluated AO or designate determines the level of independence needed (risk based) Capable of conducting an impartial assessment Impartial: free from any perceived or actual conflicts Can be from the public or private sector, can also be from within the organization The assessor should have the technical expertise to conduct the assessment Knowledge of software, hardware and firmware being assessed Any special circumstances that impede independence of the assessor should be discussed with the CIO, SISO and Office of the Inspector General
Knowledge
SkillAbility
Presenter
Presentation Notes
Assessor KSAs Knowledge Skill Ability
*GAO recommends 65% of audit staff to be CISA
Presenter
Presentation Notes
Assessor Competence Priority Certifications Certified Information Systems Auditor (CISA)* GIAC Systems and Network Auditor (GSNA) Secondary Certifications Vendor Neutral: CISSP, Security+, GIAC, CISM, etc… Vendor Specific: Microsoft, Cisco, etc… For example if staff is assessing Windows servers, The staff should have Priority certifications (e.g. CISA) In addition the vendor specific certifications (e.g. MCITP) would be helpful *GAO recommends 65% of audit staff to be CISA Competence and Retention of Qualified Staff Benchmarks in this area might include that a specific percentage of IS security staff possess professional certifications and/or have graduate degrees in specified disciplines. For example, a benchmark could be that 65 percent of IS security staff be a CISA or have a graduate degree in computer science or management information systems. In addition, there should be benchmarks with respect to the amount and nature of continuing professional education that each IS auditor must obtain annually. For example, a benchmark could be that each IS auditor is required to obtain at least 20 hours of continuing professional education in IS-related subjects or managerial subjects each year and 120 hours over a three-year period. Source: Management Planning Guide for Information Systems Security Auditing, National State Auditors Association (NSAA) and the U. S. General Accounting Office (GAO) A Joint Initiative
Presenter
Presentation Notes
Legal Considerations At the discretion of the organization Legal Review Reviewing the assessment plan Providing indemnity or limitation of liability clauses (Insurance) Particularly for tests that are intrusive Nondisclosure agreements Privacy concerns Rules of Engagement (ROE) Detailed guidelines and constraints regarding the execution of information security testing. The ROE is established before the start of a security test, and gives the test team authority to conduct defined activities without the need for additional permissions. [NIST SP 800-115]
Presenter
Presentation Notes
Developing the Test Plan Assessment Procedures Aka: Audit approach or ST&E (Security Testing & Evaluation) Procedures Detailed description of the testing methodology that will be used Will include the following Scope Testing requirements Testing approach Tests to be used Timeline Responsibilities Test team Remediation plan, recommendations Assessment Procedure A set of activities or actions employed by an assessor to determine the extent to which a security control is implemented correctly, operating as intended, and producing the desired outcome with respect to meeting the security requirements for the system. SOURCE: SP 800-53
Presenter
Presentation Notes
Assessment Methodology It is important to have repeatable and documented security assessment methodologies Benefits: Lower detection and audit risk Provide consistency and structure to security testing, which can minimize testing risks Expedite the transition of new assessment staff Address resource constraints associated with security assessments Assessment Method: A focused activity or action employed by an assessor for evaluating a particular attribute of a security control. SOURCE: SP 800-53
Presenter
Presentation Notes
Assessors Will want to see prior assessments Helps with scope Determine is progress has been made Determine audit risk What has changed since last assessment Time required What was the level of independence
Presenter
Presentation Notes
Key Definitions Passive Security Testing: Security testing that does not involve any direct interaction with the targets, such as sending packets to a target. Active Security Testing: Security testing that involves direct interaction with a target, such as sending packets to a target. Covert Testing: Testing performed using covert methods and without the knowledge of the organization’s IT staff, but with full knowledge and permission of upper management. Overt Testing: Security testing performed with the knowledge and consent of the organization’s IT staff. Source: NIST SP 800-115
Presenter
Presentation Notes
Key Definitions (cont.) Internal Security Testing: Security testing conducted from inside the organization’s security perimeter. External Security Testing: Security testing conducted from outside the organization’s security perimeter. Target Identification and Analysis Techniques: Information security testing techniques, mostly active and generally conducted using automated tools, that are used to identify systems, ports, services, and potential vulnerabilities. Target identification and analysis techniques include network discovery, network port and service identification, vulnerability scanning, wireless scanning, and application security testing. Source: NIST SP 800-115
“Risk assessments should be used to guide the rigor and intensity of all security control assessment related activities associated with the information system to enable cost effective, risk-based implementation of key elements in the organization’s information security program”- NIST SP 800-37 rev 1
Presenter
Presentation Notes
How much testing? “Risk assessments should be used to guide the rigor and intensity of all security control assessment related activities associated with the information system to enable cost effective, risk-based implementation of key elements in the organization’s information security program” - NIST SP 800-37 rev 1
Presenter
Presentation Notes
Sample Size Testing a sample Test a subset of the population Not testing the entire population It may be much more efficient and nearly as effective (statistically) Feasibility of using a sample for assessment Selection of sample size should be based on the risk of not finding a material weakness or control deficiency If the thousands of hosts are managed and similarly configured You can select a smaller percentage to test If weaknesses are found you increase your sample size to determine if the weaknesses found are pervasive or isolated incidents
Presenter
Presentation Notes
Sampling Statistical Sampling Selecting an appropriate portion of an entire population based on mathematical calculations and probability Purpose is to make scientifically and mathematically sound inferences about the entire population Should group items by characteristics (auditors judgment) Each item in the populations should have an equal chance to be selected for testing Need to ensure the sample is unbiased and that the ones selected are representative of the whole population For example you have 50 computers 25 are Mac and 25 are Windows. If your sample size was 5 you would want to ensure you select 3 Mac and 3 PC to test. Generally…
Populations over 250Control Testing Sample Size Table
Significance of Control Inherent Risk Minimum Sample Size1
High High 60High Low 40
Moderate High 40Moderate Low 25
Compliance Testing Sample Size TableDesired Level of
Assurance Minimum Sample Size1
High 60Moderate 40
Low 25
1: No exceptions expected
Presenter
Presentation Notes
Typical Sampling and Evaluation Criteria With populations under 250, the minimum sample size is 10%, no exceptions expected.
Presenter
Presentation Notes
Assessment Methods Objective Determination of security control existence, functionality, correctness, completeness, and potential for improvement over time Examine Checking, inspecting, reviewing, observing, studying, or analyzing Interview Conducting discussions with individuals or groups Test Compare actual with expected behavior NIST SP 800-53 A Rev 1 Draft May 2010
Presenter
Presentation Notes
Assessment Objectives and Guidance NIST SP 800-53A Rev 1 (June 2010) NIST SP 800-53A Rev 1 Final June 2010 and correlates to NIST SP 800-53 Rev 3
Presenter
Presentation Notes
NIST SP 800-53A Rev 1 Example
Presenter
Presentation Notes
Identify and Select Automated Tools Computer Assisted Audit Techniques or Computer Aided Audit Tools (CAATS) Computer Assisted Audit Tools and Techniques (CAATTs) SQL queries Scanners Excel programs Live CDs Checklists
Presenter
Presentation Notes
Checklists AuditNet www.auditnet.org ISACA & IIA Member Resources DoD Checklists iase.disa.mil/stigs/checklist/ NIST Special Publications csrc.nist.gov/publications/PubsSPs.html http://www.isaca.org/Knowledge-Center/ITAF-IT-Assurance-Audit-/Audit-Programs/Pages/ICQs-and-Audit-Programs.aspx
Presenter
Presentation Notes
Live CD Distributions for Security Testing For example: BackTrack Knoppix Security Tool Distribution F.I.R.E. Helix http://www.backtrack-linux.org/ http://s-t-d.org/index.html http://biatchux.dmzs.com/ http://www.e-fense.com/products.php
Presenter
Presentation Notes
Review Techniques Documentation Review Log Review Rule set Review System Configuration Review Network Sniffing File Integrity Checking Documentation review can discover gaps and weaknesses that could lead to missing or improperly implemented security controls. Log review determines if security controls are logging the proper information, and if the organization is adhering to its log management policies. Rulesets to review include network- and host-based firewall and IDS/IPS rulesets, and router access control lists. System configuration review is the process of identifying weaknesses in security configuration controls, such as systems not being hardened or configured according to security policies. The Security Content Automation Protocol (SCAP) is a method for using specific standards to enable automated vulnerability management, measurement, and policy compliance evaluation. Network sniffing is a passive technique that monitors network communication, decodes protocols, and examines headers and payloads to flag information of interest. File integrity checkers provide a way to identify that system files have been changed computing and storing a checksum for every guarded file, and establishing a file checksum database. NIST SP 800-115 Technical Guide to Information Security Testing and Assessment
Presenter
Presentation Notes
Target Identification and Analysis Techniques Network Discovery Network Port and Service Identification OS fingerprinting Vulnerability Scanning Wireless Scanning Passive Wireless Scanning Active Wireless Scanning Wireless Device Location Tracking (Site Survey) Bluetooth Scanning Infrared Scanning Network discovery uses a number of methods to discover active and responding hosts on a network, identify weaknesses, and learn how the network operates. Both passive (examination) and active (testing) techniques exist for discovering devices on a network. Network port and service identification involves using a port scanner to identify network ports and services operating on active hosts—such as FTP and HTTP—and the application that is running each identified service, such as Microsoft Internet Information Server (IIS) or Apache for the HTTP service. Organizations should conduct network port and service identification to identify hosts if this has not already been done by other means (e.g., network discovery), and flag potentially vulnerable services. This information can be used to determine targets for penetration testing. Like network port and service identification, vulnerability scanning identifies hosts and host attributes (e.g., operating systems, applications, open ports), but it also attempts to identify vulnerabilities rather than relying on human interpretation of the scanning results. Many vulnerability scanners are equipped to accept results from network discovery and network port and service identification, which reduces the amount of work needed for vulnerability scanning. Also, some scanners can perform their own network discovery and network port and service identification. Vulnerability scanning can help identify outdated software versions, missing patches, and misconfigurations, and validate compliance with or deviations from an organization’s security policy. This is done by identifying the operating systems and major software applications running on the hosts and matching them with information on known vulnerabilities stored in the scanners’ vulnerability databases. Wireless technologies, in their simplest sense, enable one or more devices to communicate without the need for physical connections such as network or peripheral cables. Wireless scanning tools used to conduct completely passive scans transmit no data, nor do the tools in any way affect the operation of deployed wireless devices.
Presenter
Presentation Notes
Target Vulnerability Validation Techniques Password Cracking Transmission / Storage Penetration Testing Automated / Manual Social Engineering Phishing Password cracking is the process of recovering passwords from password hashes stored in a computer system or transmitted over networks. Penetration testing is security testing in which assessors mimic real-world attacks to identify methods for circumventing the security features of an application, system, or network.
Presenter
Presentation Notes
Checklists / MSAT Microsoft Security Assessment Tool (MSAT) http://technet.microsoft.com/en-us/security/cc185712.aspx
Governance
RiskCompliance
DashboardsMetricsChecklistsReportingTrend AnalysisRemediation
Presenter
Presentation Notes
GRC Tools Governance Risk Compliance Dashboards Metrics Checklists Reporting Trend Analysis Remediation
Presenter
Presentation Notes
Test Types Black Box Testing Assessor starts with no knowledge White Box Testing Assessor starts with knowledge of the system, i.e. the code Grey Box Testing Assessor has some knowledge, not completely blind
Presenter
Presentation Notes
Testing Basic Testing (black box) No knowledge of the internal structure and implementation detail of the assessment object Focused Testing (grey box) Some knowledge of the internal structure and implementation detail of the assessment object. Comprehensive Testing (white box) Explicit and substantial knowledge of the internal structure and implementation detail of the assessment object NIST SP 800-53 A Rev 1 Draft May 2010
Presenter
Presentation Notes
Incremental Testing Testing before the complete implementation of a system Not testing all the controls in the system security plan just those ready This may be more cost effective and efficient Common Controls are a good example
Input • Data Entry
Data Collection
• Database Storage
Output • Reports
Presenter
Presentation Notes
Verification Testing Examples Audit logs Financial Applications
Presenter
Presentation Notes
Application Testing Code Review Automated/Manual Vulnerability scanning Configuration review Verification testing Authentication Information leakage Input/output Manipulation
Presenter
Presentation Notes
Database Auditing Native Audit (Provided by DB) SIEM & Log Management Database Activity Monitoring Database Audit Platforms Remote journaling & analytics Compliance testing Performance NATIVE AUDIT: Refers to the use of native database auditing for data capture, but use of the database system itself to store, sort, filter and reporting of events. IBM, Microsoft, Oracle and Sybase all offer slightly different variations, but capture essentially the same information. SIEM AND LOG MANAGEMENT: Security information and event management and similar log management tools are capable of collecting audit files, but offer many more features that the native database tools. DAM: Database Activity Monitoring platforms are designed to monitor database activity for threats and enforce compliance controls DATABASE AUDIT PLATFORMS: Some of the database vendors offer specialized databases that resemble log management servers. http://viewer.media.bitpipe.com/1152629439_931/1286461369_610/1010_ISM_eMag.pdf
Presenter
Presentation Notes
Intrusion Detection/Prevention Configuration Verification testing Log and Alert review
Presenter
Presentation Notes
TEMPEST Image source: http://apod.nasa.gov/apod/image/0211/field_glatz_big.gif
Presenter
Presentation Notes
EMR Testing Electromagnetic Radiation Emissions Security (EMSEC) Van Eck phreaking Tempest Tempest surveillance prevention Faraday Cage Picture Sources: http://demo.physics.uiuc.edu/lectdemo/scripts/demo_descript.idc?DemoID=169 http://www.set.nl/WG/emr200.html Remember the movie Enemy of the State with Will Smith
Presenter
Presentation Notes
Green Computing Assessment on the use of resources Power Management Virtualization Assessment The Green and Virtual Data Center Greg Schulz Hardcover: 396 pages Publisher: CRC/Auerbach Publications; 1 edition (January 26, 2009) Language: English ISBN-10: 1420086669 ISBN-13: 978-1420086669
Presenter
Presentation Notes
Business Continuity Plan Testing, Training, and Exercises (TT&E) Tabletop Exercises Checklist Assessment Walk Through Functional Exercises Remote Recovery Full Interruption Test Testing is a critical element of a viable contingency capability. Testing enables plan deficiencies to be identified and addressed by validating one or more of the system components and the operability of the plan. Testing can take on several forms and accomplish several objectives but should be conducted in as close to an operating environment as possible. Each information system component should be tested to confirm the accuracy of individual recovery procedures. The following areas should be addressed in a contingency plan test, as applicable: Notification procedures; System recovery on an alternate platform from backup media; Internal and external connectivity; System performance using alternate equipment; Restoration of normal operations; and Other plan testing (where coordination is identified, i.e., COOP, BCP). NIST Special Publication 800-34 Rev. 1 Contingency Planning Guide for Federal Information Systems
Presenter
Presentation Notes
Vulnerability Scanning Vulnerability: Weakness in an information system, or in system security procedures, internal controls, or implementation, that could be exploited or triggered by a threat source. Vulnerability Scanning: A technique used to identify hosts/host attributes and associated vulnerabilities. (Technical) Target Vulnerability Validation Techniques: Active information security testing techniques that corroborate the existence of vulnerabilities. They include password cracking, remote access testing, penetration testing, social engineering, and physical security testing. Source: NIST SP 800-115 Vulnerability: Weakness in an information system, or in system security procedures, internal controls, or implementation, that could be exploited or triggered by a threat source. Vulnerability Scanning: A technique used to identify hosts/host attributes and associated vulnerabilities. Target Vulnerability Validation Techniques: Active information security testing techniques that corroborate the existence of vulnerabilities. They include password cracking, remote access testing, penetration testing, social engineering, and physical security testing. Source: NIST SP 800-115
Presenter
Presentation Notes
MBSA Microsoft Baseline Security Analyzer 2.2 http://technet.microsoft.com/en-us/security/cc184923.aspx
Presenter
Presentation Notes
Vulnerability Reports Sample from Qualys
Where is the best place to scan from?What strategy would you use to scan systems?
External scan found 2 critical vulnerabilities
Internal scan found 15 critical vulnerabilities
Authenticated internal scan found 35 critical
vulnerabilities
Presenter
Presentation Notes
External and Internal Where is the best place to scan from? Authenticated Internal scan found 35 critical vulnerabilities Internal scan found 15 critical vulnerabilities External scan found 2 critical vulnerabilities
Presenter
Presentation Notes
Vulnerability Scanners Source: http://www.gartner.com/technology/media-products/reprints/rapid7/173772.html http://sectools.org/vuln-scanners.html
Observers & Referees
Mimic real-world attacksUnannounced
Presenter
Presentation Notes
Red, White and Blue Teams Mimic real-world attacks unannounced Red: Penetration Testers Blue: Incident Responders White: Observers and Referees Penetration Testing: Security testing in which evaluators mimic real-world attacks in an attempt to identify ways to circumvent the security features of an application, system, or network. Penetration testing often involves issuing real attacks on real systems and data, using the same tools and techniques used by actual attackers. Most penetration tests involve looking for combinations of vulnerabilities on a single system or multiple systems that can be used to gain more access than could be achieved through a single vulnerability. Source: NIST SP 800-115
Announced
Presenter
Presentation Notes
Red, White and Blue Teams Mimic real-world attacks unannounced Red: Penetration Testers Blue: Incident Responders White: Observers and Referees Penetration Testing: Security testing in which evaluators mimic real-world attacks in an attempt to identify ways to circumvent the security features of an application, system, or network. Penetration testing often involves issuing real attacks on real systems and data, using the same tools and techniques used by actual attackers. Most penetration tests involve looking for combinations of vulnerabilities on a single system or multiple systems that can be used to gain more access than could be achieved through a single vulnerability. Source: NIST SP 800-115
Red Team
“A test methodology in which assessors, typically working under specific constraints, attempt to circumvent or defeat the security features of an information system. “- CNSSI No. 4009
Presenter
Presentation Notes
Penetration Testing Aka “Red Team” Assessors attempt to circumvent security controls and features of a given system Should have an agreed upon rules of engagement Should have a clear and documented scope It is dramatic proof of the existence of vulnerabilities and how much effort is required to circumvent them “A test methodology in which assessors, typically working under specific constraints, attempt to circumvent or defeat the security features of an information system. “ - CNSSI No. 4009 Red Team “A group of people authorized and organized to emulate a potential adversary’s attack or exploitation capabilities against an enterprise’s security posture. The Red Team’s objective is to improve enterprise Information Assurance by demonstrating the impacts of successful attacks and by demonstrating what works for the defenders (i.e., the Blue Team) in an operational environment.” Blue Team “1. The group responsible for defending an enterprise’s use of information systems by maintaining its security posture against a group of mock attackers (i.e., the Red Team). Typically the Blue Team and its supporters must defend against real or simulated attacks 1) over a significant period of time, 2) in a representative operational context (e.g., as part of an operational exercise), and 3) according to rules established and monitored with the help of a neutral group refereeing the simulation or exercise (i.e., the White Team). 2. The term Blue Team is also used for defining a group of individuals that conduct operational network vulnerability evaluations and provide mitigation techniques to customers who have a need for an independent technical review of their network security posture. The Blue Team identifies security threats and risks in the operating environment, and in cooperation with the customer, analyzes the network environment and its current state of security readiness. Based on the Blue Team findings and expertise, they provide recommendations that integrate into an overall community security solution to increase the customer's cyber security readiness posture. Often times a Blue Team is employed by itself or prior to a Red Team employment to ensure that the customer's networks are as secure as possible before having the Red Team test the systems.” White Team “1. The group responsible for refereeing an engagement between a Red Team of mock attackers and a Blue Team of actual defenders of their enterprise’s use of information systems. In an exercise, the White Team acts as the judges, enforces the rules of the exercise, observes the exercise, scores teams, resolves any problems that may arise, handles all requests for information or questions, and ensures that the competition runs fairly and does not cause operational problems for the defender's mission. The White Team helps to establish the rules of engagement, the metrics for assessing results and the procedures for providing operational security for the engagement. The White Team normally has responsibility for deriving lessons-learned, conducting the post engagement assessment, and promulgating results. 2. Can also refer to a small group of people who have prior knowledge of unannounced Red Team activities. The White Team acts as observers during the Red Team activity and ensures the scope of testing does not exceed a pre-defined threshold.” CNSSI No. 4009
Presenter
Presentation Notes
Penetration Test Phases Penetration Testing: Security testing in which evaluators mimic real-world attacks in an attempt to identify ways to circumvent the security features of an application, system, or network. Penetration testing often involves issuing real attacks on real systems and data, using the same tools and techniques used by actual attackers. Most penetration tests involve looking for combinations of vulnerabilities on a single system or multiple systems that can be used to gain more access than could be achieved through a single vulnerability.
Presenter
Presentation Notes
Attack Phases Source: NIST SP 800-115
Authenticated internal scan found 35 critical
vulnerabilities
Discovery
Gain Access
Escalate Privilege
System Browsing
Install Tools
External scan found 2 critical vulnerabilities
Discovery
Gain Access
Escalate Privilege
System Browsing
Install Tools
Presenter
Presentation Notes
External and Internal Where is the best place to scan from? Authenticated Internal scan found 35 critical vulnerabilities Internal scan found 15 critical vulnerabilities External scan found 2 critical vulnerabilities
Presenter
Presentation Notes
Penetration Assessment Reports Sample from CoreImpact
Presenter
Presentation Notes
Vulnerability Information Open Source Vulnerability DB http://osvdb.org/ National Vulnerability Database http://nvd.nist.gov/ Common Vulnerabilities and Exposures http://cve.mitre.org/ Exploit Database http://www.exploit-db.com/
Presenter
Presentation Notes
Physical Assessments Posture Review Access Control Testing Perimeter review Monitoring review Alarm Response review Location review (Business Continuity) Environmental review (AC / UPS)
Presenter
Presentation Notes
The Role of the Host In order for the auditor to finish on time he/she will need to have access to documents, systems and people Delays in providing the auditor with needed items will slow the process The host organization should ensure the auditor has what he/she needs in a timely fashion, preferably before they ask for it
Presenter
Presentation Notes
Test Execution Document the testing procedures Who conducted the test What was the results of the test Signed off by tester Reviewed by supervisor Rank findings by severity (not required but useful) Provide interim results before final report No need to test controls that are not in place Documented so that another auditor with no knowledge of the system can follow the findings
Presenter
Presentation Notes
Post-Testing Activities Assessment Findings (Assessor) Security Assessment Report (SAR) Certification Mitigation Recommendations (Assessor) Technical, Managerial or Operational Reporting Draft and Final Reports (Assessor) Comments and System Owner response (System Owner) Remediation / Mitigation (System Owner) Not enough to finds problems need to have a process to fix them
Presenter
Presentation Notes
Documenting the Results Security Assessment Report (SAR) Convey the results of the security assessment to appropriate organizational officials Reliable indication of the overall security state of the information system Provide authorizing officials with the information necessary to make credible, risk-based decisions NIST SP 800-53 A Rev 1 Draft May 2010
Presenter
Presentation Notes
SAR Executive Summary report Summary Geared for managers without the technical background Synopsis of key findings Detail report Each control covered with either pass or fail Results of the test Reference Recommendations for remediation Level of detail determined by risk to system
Presenter
Presentation Notes
Included in the SAR Information system name; Security categorization; Site(s) assessed and assessment date(s); Assessor’s name/identification; Previous assessment results (if reused); Security control or control enhancement designator; Selected assessment methods and objects; Depth and coverage attributes values; Assessment finding summary (indicating satisfied or other than satisfied); Assessor comments (weaknesses or deficiencies noted); Assessor recommendations (priorities, remediation, corrective actions, or improvements) NIST SP 800-53 A Rev 1 Draft May 2010
*Definition form ISACA
Presenter
Presentation Notes
Audit / Assessment Documentation Audit Trail – “A visible trail of evidence enabling one to trace information contained in statements or reports back to the original input source.”* Evidence – supports the audit conclusion Includes: Notes taken from interviews Internal documentation Correspondence Results of tests Remember some evidence is more reliable than others Definition form ISACA
Presenter
Presentation Notes
DIACAP Scorecard A summary report that succinctly conveys information on the IA posture of a DoD IS in a format that can be exchanged electronically. DoDI 8510.01, November 28, 2007 It shows the implementation status of a DoD IS’s assigned IA controls (i.e., compliant (C), non compliant (NC), or not applicable (NA)) as well as the C&A status. DoDI 8510.01, November 28, 2007
DoDI 8510.01, November 28, 2007
Presenter
Presentation Notes
DIACAP Severity Category The category a CA assigns to a system security weakness or shortcoming as part of a certification analysis to indicate the risk level associated with the security weakness and the urgency with which the corrective action must be completed. Severity categories are expressed as “Category (CAT) I, CAT II, or CAT III,” with CAT I indicating the greatest risk and urgency. Severity categories are assigned after consideration of all possible mitigation measures that have been taken within system design/architecture limitations for the DoD IS in question.
DoDI 8510.01, November 28, 2007
Presenter
Presentation Notes
DIACAP Severity Categories CAT 1 (High) Assigned to findings that allow primary security protections to be bypassed, allowing immediate access by unauthorized personnel or unauthorized assumption of super-user privileges. An ATO will not be granted while CAT I weaknesses are present. CAT II (Moderate) Assigned to findings that have a potential to lead to unauthorized system access or activity. CAT II findings that have been satisfactorily mitigated will not prevent an ATO from being granted. CAT III (Low) Assigned findings that may impact IA posture but are not required to be mitigated or corrected in order for an ATO to be granted. Note: NIST would use Low, Moderate, and High for assessed control weaknesses. For NIST the risk level of Low, Moderate, and High may corresponded with Priority levels of a particular control.
Presenter
Presentation Notes
Audit Papers The auditor / assessor should document the procedures used and evidence such that another auditor reviewing that documentation will come to the same conclusion. If another auditor /assessor cannot follow the documentation your documentation is inadequate.
Presenter
Presentation Notes
Concurrent Remediation Remediation taken place during the assessment period Assessor will communicate results as they perform their assessments System Owner many take corrective actions during that period of time Assessor will have to re-test to see if the control deficiency has truly been corrected Items that cannot be corrected will be entered into the Plan of Action and Milestones (POA&M) for future remediation
Presenter
Presentation Notes
Disagreements with Findings Optional addendum to the Security Assessment Report This addendum gives the Information system owner and common control providers an opportunity to respond to the assessor's findings Sometimes called management’s response This does not change the assessors report Is it an substantive issue? Each organization or agency may wish to employ a issue resolution process
Presenter
Presentation Notes
Organizations That can Help Other than DoD and NIST U.S. Government Accountability Office (GAO) Information Systems Audit and Control Association (ISACA) American Institute of Certified Public Accountants (AICPA) Institute of Internal Auditors (IIA) SANS National State Auditors Association (NSAA)
Presenter
Presentation Notes
Resources NIST SP 800-115 NIST SP 800-53a Rev 1 Federal Information System Controls Audit Manual (FISCAM) Open Source Security Testing Methodology Manual (OSSTMM) MarketScope for Vulnerability Assessment, 17 February 2010, Kelly M. Kavanagh, Mark Nicolett, John Pescatore Gartner RAS Core Research Note G00173772 Http://www.gartner.com/technology/media-products/reprints/rapid7/173772.html Managing A Network Vulnerability Assessment [Paperback] Thomas R. Peltier (Author), Justin Peltier (Author), John A. Blackley (Author) Publisher: Auerbach Publications; 1 edition (May 28, 2003) ISBN-13: 978-0849312700 A Practical Guide to Security Assessments [Hardcover] Sudhanshu Kairab Publisher: Auerbach Publications; 1 edition (September 29, 2004) ISBN-13: 978-0849317064 NIST SP 800-115 Technical Guide to Information Security Testing and Assessment Management Planning Guide for Information Systems Security Auditing by NSAA & GAO Twenty Critical Controls for Effective Cyber Defense: Consensus Audit Guidelines COBIT: Control Objectives for Information and Related Technology Federal Information System Controls Audit Manual (FISCAM) OSSTMM - Open Source Security Testing Methodology Manual
implemented correctly, operating as intended, and producing the desired outcome
Presenter
Presentation Notes
Summary Assessment (Certification) process needs To be performed by professionals Level of independence depends on the level of the system Should be well-planned Should be well-documented Should be a basis of remediation Assessment (Certification) Provides assurance that the implemented controls are functioning as expected “…to determine the extent to which the controls are implemented correctly, operating as intended, and producing the desired outcome…” NIST SP 800-100
Presenter
Presentation Notes
Class Discussion: Assessment What are some reasons why an assessor might increase the amount of testing? What can you do to ease the assessment pains? Is an assessment (audit) a adversarial process? What are some things you would consider in selecting an assessor? Why should auditors document what tests they conducted and the results? What do Red Teams do? What do you do if you disagree with an assessors findings?
