KVM_security

Defense in Depth 2014 | Frank Caviggia 1

Top Secret KVM, Lessons Learned

from an ICD 503 Deployment

Frank Caviggia

July 30, 2014

Defense in Depth 2014


Overview

System Configuration

- Hardware

- Software

Security Controls

- Security Concepts

- Government Standards

- Hardening Scripts (STIG FIX, SSG)

- DISA STIG Kickstart DVD

SELinux Concepts

- What is SELinux?

- DAC and MAC

- Polyinstantiation and Multitenancy

KVM Security Features

- sVirt – SELinux Labels on VMs

- Multiple Firewall Levels

- cgroups – Control Groups (Limiters)

More Information


System Configuration


“Commodity Hardware”

Rack Servers

- Dell R200s (IdM)

- Dell R710s (RHN)

Blade Server

- HP C7000 with ProCurve 6120 Switch

- 4x HP BL460C G7 Servers

- HP D2200sb Storage Blade

System Configuration: Hardware


Red Hat Software

- Red Hat Enterprise Linux 6.5 Server x86_64

- Authentication (IdM) [389-DS(LDAP)/Kerberos]

- Red Hat supported version of FreeIPA

- Kernel Virtual Machine (KVM)

- Red Hat Enterprise Virtualization (RHEV) 3.3

- RHEV-M (Management Console for KVM Hypervisors)

- Red Hat Network (RHN) Satellite 5.6

System Configuration: Software


Kernel Cryptographic API "2.0" #1901 Certified, Level 1

Disk Volume Cryptographic API "2.0" #1933 Certified, Level 1

libgcrypt "2.0" #1757 Certified, Level 1

OpenSSH Client "2.0" #1791 Certified, Level 1

OpenSSH Server "2.0" #1792 Certified, Level 1

OpenSSL "2.0" #1758 Certified, Level 1

Openswan "2.0" #1859 Certified, Level 1

NSS (Freebl) 3.12.9.1 #1710 Certified, Level 1

NSS 3.12.9.1 #1837 Certified, Level 1

Red Hat Enterprise Linux 6 EAL4+

OSPP, including Labeled Security,

Advanced Audit, Advanced

Management, and Virtualization

Extended Modules

Dell

HP

IBM

SGI (report, target)

Red Hat Enterprise Linux 6:

- RHEL is Common Criteria Evaluated (Certified EAL 4+)

- FIPS 140-2 (Level 1 – Certified, Level 2 – In Evaluation)

- Linux Unified Key Setup (LUKS) Encryption (Data at Rest) – 8 Keys

- SSH uses AES256 (Counter mode) Encryption (Data in Motion)

- Web portals (Apache HTTPD) will require PKI authentication to view login page

System Configuration: RHEL, RHEV, KVM

http://csrc.nist.gov/groups/STM/cmvp/documents/140-1/1401val2013.htm









https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Zertifizierung/Reporte07/0754a_pdf.pdf?__blob=publicationFile

https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Zertifizierung/Reporte07/0754b_pdf.pdf?__blob=publicationFile


KVM SPECvirt 2010 Results – KVM vs VMware

Kernel Virtual Machine (KVM):

- Type 1 Hypervisor (Bare-Metal) – runs as Kernel Module

- Utilizes native virtualization in Intel (VT-i,VT-d) and AMD hardware

- RHEL/KVM is Common Criteria Evaluated (Certified EAL 4+)

- Inherits Security (SELinux, Auditing) and Performance

- VM Limits (Leading Performance – SPECvirt)

- 160 CPUs

- 4TB RAM*

* As of RHEL 6.5/ RHEV 3.4 and newer

System Configuration: RHEL, RHEV, KVM (continued)


Red Hat Enterprise Virtualization Manager (RHEV-M):

- Web-based portal access to Virtual Machines (VMs)

- Role Based Access Control (RBAC) to VMs

- SPICE protocol (plug-ins for IE or Firefox) or Standalone Client

- Support for up to 4 independent monitors per VM

- Akin to vCenter in VMware



Red Hat Enterprise Linux Lifecycle:

- 10 years of support (up to 13 years with just security fixes)

- Common Vulnerabilities and Exposures (CVE) are fixed through

Red Hat Security Advisory (RHSA) process:

- Ensures IAVAs are patched

- Ensures system stability and support (backporting)

- There are no licenses, only subscriptions (stable budgeting)

- Upgrades are included with subscription

Support Lifecycle of Red Hat Enterprise Linux 6



RHN Satellite: SCAP Compliance Reporting

Red Hat Network Satellite 5.6

- IAVA patching and validation (patch management)

- File provisioning to connected hosts (configuration management)

- SCAP compliance scans (continuous monitoring) more on this later…

System Configuration: RHN Satellite


Security Controls


Security is like an onion – the more layers you peel the more you cry…

Goal: Create a secure virtualization environment using a standard set of packaged

scripts, configurations, and policies to be deployed across systems.

Controls are implemented through the following mechanisms:

- Hardening Scripts, Kickstart Installation

- Discretionary Access Controls (DAC)

- SELinux Policies

- Mandatory Access Controls (MAC)

- Network Controls (TCP_WRAPPERS, iptables, ebtables)

- Process and Memory Controls (cgroups)

- Administrative Controls (physical, policies, etc.)

- Continuous Monitoring (SCAP, RHN Satellite)

Security Overview


There are multiple government standards and regulations – some of which overlap:

Cross Domain Controls

NSA

SNAC DISA STIGS

Common Criteria EAL 4+ System Security Controls

CAPP 1

RBACPP 2

LSPP 3

1 Controlled Access Protection Profile (CAPP)

2 Role-Based Access Control Protection Profile (RBACPP)

3 Labeled Security Protection Profile (LSPP)

NIST 800-53

(USGCB)

FIPS 140-2

Security Overview: Government Regulation…

http://www.niap-ccevs.org/cc-scheme/pp/PP_OS_CA_V1.d.pdf

http://www.commoncriteriaportal.org/files/ppfiles/lspp.pdf


Hardening will be applied by shell scripts, configurations, and policies based upon

several government standards and open-source projects to standardize configuration:

Apply Security Best Practices to Base Operating System

1 SCAP Security Guide - https://fedorahosted.org/scap-security-guide/

2 USGB Content -http://usgcb.nist.gov/usgcb/rhel/download_rhel5.html

3 Aqueduct Project - https://fedorahosted.org/aqueduct/

SCAP Security Guide1

- RHEL 6 SCAP, Security Configuration

NIST 800-53 - United States Government Configuration Baseline (USGCB)2

DISA Unix STIGs - Aqueduct Project3

- Tresys’ Certifiable Linux Integration Platform (CLIP)4

NSA Security Configuration Guide5

- USB blocking, configurations, and other lockdowns (Gnome)

4 Tresys CLIP Project - http://oss.tresys.com/projects/clip

5 NSA SNAC Guide -

http://www.nsa.gov/ia/mitigation_guidance/security_configuration_guides/operating_systems.shtml#linux2

Hardening Scripts

https://fedorahosted.org/scap-security-guide/





http://usgcb.nist.gov/usgcb/rhel/download_rhel5.html





https://fedorahosted.org/aqueduct/

http://oss.tresys.com/projects/clip

http://www.nsa.gov/ia/mitigation_guidance/security_configuration_guides/operating_systems.shtml


The hardening shell script serves several purposes in hardening the system:

- Distributes ‘baseline’ system configurations and policies for authentication,

auditing, accounts, and services

- Modular code in folders and separate scripts allows for adaptation to meet

changing system and security needs of customer code

- Verifies application of hardening with logging, hardening can be re-applied in case

of modification of baseline, fits in with continuous monitoring

apply.sh

gen1000.sh

gen2000.sh

…

gen9999.sh gen1000.sh

gen2000.sh

…

gen9999.sh

gen1000.sh

gen2000.sh

…

gen9999.sh gen1000.sh

gen2000.sh

…

gen9999.sh

CAT I

CAT II

CAT III

CAT IV

nist1000.sh

nist2000.sh

…

nist9999.sh

NIST 800-53

NSA SNAC nsa1000.sh

nsa2000.sh

…

nsa9999.sh Hardening Script Function (apply.sh)

Hardening Script: Implementation


Hardening scripts were packaged in RPM for the following reasons:

- Integrity verification

- Integrated version control and configuration management

- Distribute scripts via RHN Satellite Server

Check out the open source project here:

https://github.com/redhatgov/stig-fix-el6/

# rpm –V stig-fix-1.0.el6

Verify the integrity of an RPM

Hardening Script: Packaging







SCAP is implemented on Red Hat Enterprise Linux by OpenSCAP (oscap) and the

SCAP Security Guide (SSG) developed with collaboration with the NSA, NIST, and

DISA.

RHN Satellite can run SCAP Scans against a defined security baseline to check for

configuration compliance on a schedule. This helps to maintain continuous monitoring:

# oscap xccdf eval --profile stig-rhel6-server-upstream --results results.xml --report report.html --cpe /usr/share/xml/scap/ssg/content/ssg-rhel6-cpe-dictonary.xml /usr/share/xml/scap/ssg/content/ssg-rhel6-xccdf.xml

OpenSCAP XCCDF System Compliance Check

# wget http://www.redhat.com/security/data/oval/com.redhat.rhsa-all.xml # wget http://www.redhat.com/security/data/metrics/com.redhat.rhsa-all.xccdf.xml # oscap xccdf eval --results results.xml --report report.html com.redhat.rhsa-all.xccdf.xml

OpenSCAP XCCDF Patch (CVE) Compliance Check

- XCCDF (eXtensible Configuration Checklist Description Format)

- Creates checklist for security configuration on a target system

- OVAL (Open Vulnerability and Assessment Language)

- Standardized security information content

- CPE (Common Platform Enumeration) Dictionary

- Names and Metadata for security Evaluation

- CCE (Common Configuration Enumeration)

- Identifies mappings between SCAP security checks and STIG/NIST 800-53 settings

SCAP Terms & Definitions

Security Configuration Automation Protocol (SCAP)


The hardening script RPM was combined with a customized Kickstart to produce a

standardized installation DVD to help meet security requirements right from installation.

Screenshots of DISA STIG Kickstart DVD

Security Configuration: DISA STIG Kickstart DVD

More Information: https://people.redhat.com/fcaviggi/stig-fix/

https://people.redhat.com/fcaviggi/stig-fix/





SELinux Concepts


SELinux Policy Example Watertight Compartments in Ship Design1

1 Picture Source: Wikipedia – Bulkhead (Partition)

Security-Enhanced Linux (SELinux) was a research project sponsored by the NSA to

provide Mandatory Access Controls (MAC) to the Linux kernel

SELinux mainlined into the Linux kernel in August 2003 (2.6.0-test3), it was first

enabled for general use in Red Hat Enterprise Linux 4

Kernel enforcement based on “security context” provided by policies rather than standard

permissions. Think of it like a chroot jail on steroids or “watertight” compartments in ship

design

SELinux: Overview


Traditional Unix Permissions

- User, Group, Others (ugo)

- Read, Write, Execute (rwx)

Access Control Lists (ACLs)

- POSIX1 compliant ACLs standard in Linux filesystems (ext3, ext4, XFS, etc.)

- Extends DAC controls to specific user(s) and group(s)

1 Portable Operating System Interface EXchange

Discretionary Access Controls (DAC)

Concepts: Discretionary Access Control (DAC)


SELinux has 3 defined policy modes - Targeted (Default), Strict, and MLS.

Security Context implemented through extended attributes (xattr) in filesystem and

enforced by the Linux Kernel according to SELinux Policy

Unix concept of “everything is a file” (devices, processes, files, directories, etc.) Thus,

everything is labeled with a Security Context

SELinux policy defines the “watertight” compartments – the SELinux policy control how

users, services, files, and binaries interact

Policy is generally developed with software vendor when possible. Otherwise,

developing policy can be achieved through testing and evaluation giving “least privilege”

to allow completion of a job function

Security Context in SELinux

Concepts: Mandatory Access Control (MAC)


Type Enforcement (TE) used by Targeted policy (Default)

in SELinux

- The Linux Kernel enforces transactions between

processes and objects via domain transitions

- Further control can be specified using different policy

SELinux Domain Transitions

Compromised Apache process cannot access /etc/shadow

Concepts: Type Enforcement (TE)


1 IBM Developer Works Article “Improve Security with Polyinstantiation”

2 https://securityblog.redhat.com/2014/04/09/new-red-hat-enterprise-linux-7-security-feature-privatetmp/

Polyinstantiation1 is the process used on MLS systems to ensure data being processed by

users at separate security levels do so in isolated spaces to use to prevent unauthorized

access to data.

Data written to these directories will be stored in an independent directory at the security

level that they were written, particularly important for shared temporary directories (/tmp,

/var/tmp, /dev/shm/)

User will not see the redirection to a secure folder, SELinux handles the transition

transparently. See the “Private Tmp” feature in RHEL 72

Multitenancy extends the concept of polyinstantiation with cgroups and Linux Containers

(LXC) to ensure that applications are securely separated from each other through Type

Enforcement (TE) and MCS (the c0.c1023 attributes of the security level)

Multitenancy in OpenShift

Concepts: Polyinstantiation and Multitenancy

https://securityblog.redhat.com/2014/04/09/new-red-hat-enterprise-linux-7-security-feature-privatetmp/


















Kernel Virtual Machine Security


Each VM has their own “container” via SELinux Type Enforcement (TE) and

Multi-Category Security (MCS) which uses random compartments to keep the

VMs separate…

KVM Security: sVirt (SELinux for KVM)


Compromised VM uses hypervisor exploit to compromise other VMs

VS.

Compromised VM containment with KVM and sVirt (SELinux Labels)

KVM Security: sVirt (SELinux for KVM) (continued)


KVM Security: Multiple Firewall Levels


The ebtables firewall that enables basic ethernet frame filtering on a Linux network bridge,

logging, MAC (network address) NAT, and brouting.

The firewalls (iptables and ebtables) will be used to complement each other.

What can ebtables do?

# ebtables –A FORWARD –i vnet+ –-among-src ! 54:52:00:5b:1a:cd=192.168.0.100 –j DROP

Prevent IP-MAC Address Spoofing from VMs

# ebtables –A OUTPUT –i vnet+ –p IPv6 –j DROP

Drop All Outbound IPv6 Packets

• Ethernet protocol filtering

• MAC address filtering

• Simple IP header filtering

• ARP header filtering

• 802.1Q VLAN filtering

• In/Out interface filtering (logical

and physical device).

• MAC address NAT

• Logging

• Frame counters

• Ability to add, delete and insert rules; flush chains; zero counters

• Brouter facility

• Ability to atomically load a complete table, containing the rules you made,

into the kernel

• Support for user defined chains

• Support for marking frames and matching marked frames

KVM Security: Multiple Firewall Levels (continued)


cgroups (control groups) is a Linux kernel feature to limit, account and isolate

resource usage (CPU, memory, disk I/O, etc.) of process groups.

• Resource limiting: groups can be set to not exceed a set memory limit —

this also includes file system cache.

• Prioritization: some groups may get a larger share of CPU or disk I/O

throughput.

• Accounting: to measure how much resources certain systems use (e.g.

billing purposes)

• Control: freezing groups or checkpointing and restarting.

Dynamic Changes in Workload and Priority (e.g. Number Crunching Overnight, Web Servers during Work Hours)

KVM Security: Control Groups


KVM Security: Secured Development Litterbox!


Questions?


More Information

DISA STIG Kickstart DVD:

http://people.redhat.com/fcaviggi/stig-fix/

Hardening Scripts:

https://github.com/RedHatGov/stig-fix-el6

SCAP Security Guide:


VDSM Hooks for RHEVM:

http://www.ovirt.org/VDSM-Hooks_Catalogue

Classification Banner:

https://github.com/RedHatGov/classification-banner


























KVM_security

Documents

Transcript of KVM_security