Integrate 3rd party security solution into CloudStack
-
Upload
micexia -
Category
Technology
-
view
1.518 -
download
1
description
Transcript of Integrate 3rd party security solution into CloudStack
![Page 2: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/2.jpg)
About me
• About me– Project manager from TCloud Computing Ltd, Beijing, China– Apache CloudStack Committer– About TCloud
• Provides cloud platform product/service to Chinese customers• Based on CloudStack
• About this topic– Based on the practice and on-going work– Goal: bring some insights into integration between CS and security
product
![Page 3: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/3.jpg)
Outline
• Background• Three categories of solutions• Integration goal and problems• Architecture• Workflow• Future work
![Page 4: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/4.jpg)
Background
• CS does an excellent job for building clouds• Growing needs for solutions to secure the Cloud
• Seek security solutions to work with CS• Works for both private and public cloud• Multi-tenant ,self-service, • Easy to provision• Resource effective• Easy to integrate
![Page 5: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/5.jpg)
Agent-based
Guest Network
Guest VM 1
Guest VM
Guest VM
Guest VM
CSVirtual Router
Agent
Agent
Agent
Agent
• Widely implemented in physical environment
• Relies on the agent installed on guest OS
• Network connection to pattern/mgmt server
• Not much IAAS level integration • Cons• Instant-on gap• Resource contention
Pattern/MgmtServer
![Page 6: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/6.jpg)
Gateway-based
• Agentless, widely adopted in physical environment
• Detect and prevent by capturing network traffics
• Works fine with simple requirements
• Cons• Network traffic based• Uneasy to provide Inter-user and
inter-VM protection
Router
L3 Core Switch
…… …
Pod 1 Pod N
Offline mode
Available Zone
Inline mode
![Page 7: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/7.jpg)
Hypervisor
Hypervisor-based
• Dedicated for virtual environment• Agentless for user VM, need an
‘agent’ on hypervisor• Capture network traffics• Scan VM memory
• Can work with any complex network environment
User VM
User VM
Module
![Page 8: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/8.jpg)
Brief intro to ElasterShield
• Hypervisor based security product• Features
• Firewall (L2, L3, L4)• IDS/IPS• Application Control• Web server protection• Anti-malware (roadmap)
• For IT admin, no multi-tenant• Rules: defines what triggers event• Profiles: a set of enabled rules
ElasterShield
<VM, Profile/Rules>
Events/Alerts
![Page 9: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/9.jpg)
Brief intro to ElasterShield (cont.)
• ElasterShield Manager (ESM)• Storing rules/profiles• Provide UI/ web service API• Send command to ESVA
• Security Virtual Appliance (ESVA)• One for each hypervisor, stateless• Hypervisor redirects VM’s traffic to
ESVA• Includes an engine to do the
detection/prevention
• Bi-directional communicate between ESM and ESVA
• No impact to user VM if ESVA stops
ESM
Hypervisor
User VM
Hypervisor
User VMESVA
ESVA
![Page 10: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/10.jpg)
Integration goal
• CS Integrates ES• Security as a service, ES as a service provider
• Multi-tenant, defined by admins, subscribed by users, provisioned to VMs
• Security offering == Security Profiles• Export new CS APIs
• Generic to hypervisor-based solution (hopefully)• No change to ES code, keep it as it is• Minimum change to CS 4.0 existing code
![Page 11: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/11.jpg)
Problems : Networking
• Bi-directional communication is required between ESM and ESVA
• Deploy ESVA as a user VM– Connects to guest network– Basic Zone
• Ingress rules– Advanced zone
• Virtual router NAT problem• Another dedicated shared network?
• Solution: Via management network– Works for both basic and adv zone– ESVA as a special VM managed by CS– Connect it to management network
programmatically
ESM
ESVA
User VM
User VM
VRouter
Shared Guest Network
Isolated Guest Network
Public Network
?Management Network
![Page 12: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/12.jpg)
Problems: ESVA management
• Make sure ESVA is running on each hypervisor– When a new host is added, ESVA should be deployed
automatically on the host– When hypervisor is rebooted, ESVA should be started
automatically– Can be destroyed manually and re-deployed
• Similar to SystemVM/VRouter– Managed by CS, transparent to end user– Per hypervisor basis
![Page 13: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/13.jpg)
Problems: VM life cycle conflicts
• User VM State mismatch– ES can detect VM state changes by polling events directly from
Hypervisor• Enable VM’s profile when it gets started• Disable VM’s profile when it gets stopped• Un-assign profile when it gets destroyed
– It does not work. CS removes VM from hypervisor when it gets stopped
– Solution: CS controls profile (un)assignment to VM , based on VM state in the CS
Stopped Destroyed
Destroyed (Missing)
CS VM state
Hypervisor VM state
Removed
![Page 14: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/14.jpg)
Architecture Overview
CloudStack Manager
ES Plug-in ESM
Hypervisor
ESVA
User VM
Hypervisor
ESVA
User VM
ESM APINew CS API
Mgmt network
Mgmt network
Managed by CS
![Page 15: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/15.jpg)
Integrate as a CS Plug-in
• Provide security related APIs• Treat ESM as a resource and send commands to it• ESVA monitor and management• Tailor User VM lifecycle with existing framework• Tailor Hypervisor management
![Page 16: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/16.jpg)
Plug-in
ESM
Reso
urce
ESM
Reso
urce
Rest API
Plug
in A
PI
Data Access Layer
ESVAManagerESVAManager
ESMManagerESMManager
ESMUserVMManagerImplESMUserVMManagerImpl
…
• New manager/service for ESVA and ESM
• Extends several CS manager to tailor process
• A new resource for calling ESM APIs
• DAOs for security related database operation
![Page 17: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/17.jpg)
Workflow: Add ESM
CloudStack Manager
ElasterShield Plug-inElasterShield
Manager
2) Connect to ES manager1) Call plug-in API
Database 3) Persist into DB
1) Admin user or external UI calls Plug-in API (addElasterShield)• Parameters include ESM url/account/password• One ESM per zone or per installation?
2) Plug-in calls ESM’s APIs to:• test connection with specified authentication• Get ESM info such like version, license, etc.
3) Plug-in persists the ESM info into CS database
![Page 18: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/18.jpg)
Workflow: Enable Security protection
CloudStack Manager
ES Plug-in ESM
3) Activate ESVA
1 ) Call plug-in API: enableSecurityProtection1) Admin user or external UI calls plug-in
API (enableSecurityProtection)• Cluster wide enable
2) ES Plug-in deploys ESVA• Deploy ESVA on each host of the
cluster• With specified service offering
and template• Start ESVA
3) Activate ESVA• Plug-in calls ESM API to activate
each ESVA
Hypervisor
2) Start ESVA on each hypervisor of the cluster
Hypervisor
ESVA ESVA
![Page 19: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/19.jpg)
Workflow: List Security Offering
CloudStack Manager
ElasterShield Plug-in
ElasterShield Manager
2) Call ESM API, listSecurityProfiles
1) Call plug-in APIlistSecurityOffering
1) User or external UI calls Plug-in API (listSecurityOffering)• Fetch all security offerings with query parameters• Id, name, description
2) Plug-in calls ESM’s APIs to:• List security profiles from ESM
![Page 20: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/20.jpg)
Workflow: Define Security Profiles
CloudStack Manager
ElasterShield Plug-in
ElasterShield Manager
2) Call ESM API listSecurityProfiles
1) Call plug-in APIlistSecurityOffering
• For this Stage, this will be not implemented in plug-in, because it’s product-specific
• Admins use ESM’s UI to define profiles• Users use Plug-in API to list and apply profiles
Admin defines Rules/profiles
![Page 21: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/21.jpg)
Workflow: Apply security offering
CloudStack Manager
ElasterShield Plug-inElasterShield
Manager
3) Call ESM API, assignSecurityProfile
1) Call plug-in APIapplySecurityOffering
1) User or external UI calls plug-in API (applySecurityOffering)• Specify VM id and security offering id
2) Plug-in persists <VM, offering> mapping into DB• Address multi-tenant problem
3) Plug-in calls ESM API to assign corresponding profile if VM is running
Database 2) Persist into DB
![Page 22: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/22.jpg)
Workflow: Start Virtual Machine
CloudStack Manager
ElasterShield Plug-in ElasterShield Manager
4) Call ESM API, assignSecurityProfile
1) Call CS APIstartVirtualMachine
1) User or UI calls CS API (startVirtualMachine)2) CS checks ESVA status and starts user VM as usual3) After user VM is started, plug-in gets this VM’s profile
(VirtualMachineGuru.finalizeStart)4) Plug-in call ESM API to assign profile to the VM5) ESM notifies ESVA in the same hypervisor with VM to take effect profile
Database
3) Get profile
HypervisorESVA
User VM
2) Start User VM
5) ESM notifies ESVA to take effect
![Page 23: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/23.jpg)
Workflow: Stop Virtual Machine
CloudStack Manager
ElasterShield Plug-in ElasterShield Manager
4) Call ESM API, unassignSecurityProfile
1) Call CS APIstopVirtualMachine
1) User or UI calls CS API (stopVirtualMachine)2) CS powers off user VM as usual3) After user VM is shut down, plug-in gets this VM’s profile
(VirtualMachineGuru.finalizeStop)4) Plug-in calls ESM API to unassign profile to the VM5) ESM notifies ESVA in the same hypervisor with VM to take profile effect
Database
3) Get profile
HypervisorESVA
User VM
2) Stop user VM
5) ESM notifies ESVA to take effect
![Page 24: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/24.jpg)
Workflow: Destroy Virtual Machine
CloudStack Manager
ElasterShield Plug-in
1) Call CS APIdestroyVirtualMachine
1) User or UI calls CS API (destroyVirtualMachine)• VM is destroyed as usual by CS
2) When VM gets expunged, plug-in removes <VM, profile> mapping from DB• VirtualMachineGuru.finalizeExpunge
Database2) Remove from DB
![Page 25: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/25.jpg)
Workflow: Add a new host
CloudStack Manager
ElasterShield Plug-inElasterShield
Manager
5) Call ESM API to activate
1) Call CS APIaddHost
1) User or UI calls CS API (addHost)2) CS add a new hypervisor host as usual3) Deploy ESVA 4) Plug-in persist ESVA info
Hypervisor
ESVA
2) Add host 3) Deploy ESVA
![Page 26: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/26.jpg)
Workflow: Maintain a host
CloudStack Manager
ElasterShield Plug-in ElasterShield Manager
3) Call ESM API to de-activate ESVA
1) Call CS APIprepareHostForMaintenance
1) User or UI calls CS API (prepareHostForMaintenance)2) Stop ESVA on it
• update ESVA status into DB3) Plug-in call ESM API to deactivate ESVA
• unassign security profiles to VMs that cannot be migrated
Hypervisor
ESVA
2) Power off ESVA
![Page 27: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/27.jpg)
Workflow: Query Events/Alerts
CloudStack Manager
ElasterShield Plug-in
1) Call plug-in APIquerySecurityEvents
1) User or external UI calls plug-in API (querySecurityEvents)• With query parameters such as time, VM id, user id, etc
2) Plug-in calls ESM’s API to fetch events/alerts
2) Call ESM API
ElasterShield Manager
![Page 28: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/28.jpg)
Workflow: ESVA upgrading
CloudStack Manager
ElasterShield Plug-in ElasterShield Manager
2) Call ESM API to upgrade
1) Call plug-in APIupgradeESVA
1) Call plug-in API to upgrade ESVA , with a URL parameter2) Plug-in forward the API to ESM3) ESM notifies all ESVA to upgrade4) ESVA downloads upgrading package from the URL specified5) ESVA upgrade itself, it still functions during the upgrading.
Hypervisor
ESVA
3) Notify ESVA
Web server
4) Download upgrade package
![Page 29: Integrate 3rd party security solution into CloudStack](https://reader035.fdocuments.us/reader035/viewer/2022062511/54b8b6e84a7959df298b457b/html5/thumbnails/29.jpg)
Summary & Future work
• Summary– Develop plug-in with existing CS framework– Leverage CS to manage security virtual appliance– Tailor process
• Future work– Make the security offering more generic– Make it easier for CS to introduce a new system
VM