2013 Trend Micro

25th Anniversary

CLOUDSTACK PLUGIN STRUCTURE AND IMPLEMENTATION

TCloud Computing, Inc.

2013 Trend Micro

25th Anniversary

Outline

• Management Internal

• Anatomy of Plugin

• Practical Example – Third Party Security Solution

2013 Trend Micro

25th Anniversary

Management Server Internal

2013 Trend Micro

25th Anniversary

Architecture Overview

• Includes all the implementation of CloudStack's HTTP Query API

• Includes all supported Cloudstack Java Interfaces that can be used to bridge against external systems.

Services

•Defines all the processes of CloudStack Management Server.

• Interacts with the Adapter Framework to provide customizations to this process.

Management

•Contains the implementation that map CloudStack commands to the various physical elements including network, storage, and hypervisor resources

Resources

•Contains the implementation to the Database Access layer. Currently it is MySQL but does not have to be limited to a database.

Data Access

2013 Trend Micro

25th Anniversary

Service Layer

• Synchronous command

– Scope of mgmt server or to the database

– Short-live

• Asynchronous command

– Require a call to an external system

– Take long time

• Command configuration

– commands.properties

– <command name>=<fully qualified java classname>;<ACL>

2013 Trend Micro

25th Anniversary

Management Layer

• Kernel of CloudStack

• Manager

– Singleton to control a process

• Adapter

– Different ways to implement the same functionality.

– Often used when there can be multiple ways to implement that step.

• Configuration

– components.xml

2013 Trend Micro

25th Anniversary

Resource Layer

• Map commands to the physical elements

– Hypervisor resource

– Network resoure

– Storage resource

2013 Trend Micro

25th Anniversary

Inside a Management Server

API Servlet

Async Job

Queue Mgr

CS API Services A

PI

Cmds

Responses

cmd.execute()

Kernel

Agent Manager

Resources

Agent API (Commands)

Hypervisor Native APIs

Local Or Remote

Network Device API

MySQL

2013 Trend Micro

25th Anniversary

Accounts

Security Manager

Events Manager

Usage Manager

Domain Manager

Account Manager

Limits Manager

CloudStack Orchestration

Vir

tual

Mac

hin

e M

anag

er

Sto

rage

Man

ager

Net

wo

rk M

anag

er

Tem

pla

te

Man

ager

Snap

sho

t M

anag

er

CloudStack WebServices API

OAM&P API End User

API AWS API Pluggable Service API Engine

CloudStack Plugins

Net

Scal

er N

etw

rok

Serv

ice

Pro

vid

er

Vir

tual

Ro

ute

r N

etw

rok

Serv

ice

Pro

vid

er

Business Logic R

eso

urc

e M

anag

er

Ru

les

Man

ager

Up

dat

e M

anag

er

HA

M

anag

er

Cap

acit

y M

anag

er

Framework

Agent Manager Cluster Manager Data Access Layer

Adapters

Network Guru

Network Element

Deployment Planner

Hypervisor Guru

2013 Trend Micro

25th Anniversary

Plugins

• Various ways to add more capability to CloudStack

• Implements clearly defined interfaces

• All operations must be idempotent

• All calls are at transaction boundaries

• Compiles only against the Plugin API module

• Deployed on management server

2013 Trend Micro

25th Anniversary

Anatomy of a plugin

2013 Trend Micro

25th Anniversary

Anatomy of a Plugin

• Server Component: – Can implement multiple

Plugin APIs to affect its feature

– Can expose its own API through Pluggable Service so administrators can configure the plugin

• ServerResource: – Deployed co-located with

the physical resource

ServerResource - Optional. Required if Plugin needs to be co-

located with the resource - Implements translation layer to talk to resource - Communicates with server component via JSON

Rest API

Plu

gin

AP

I

Data Access Layer

Implementation

2013 Trend Micro

25th Anniversary

Components of an plug-in

• PluggableService that defines APIs to configure the plug-in

• ServerResource extension that is meant to be run co-located with the resource

• Manager code that runs within the management server with access to management database

• Implement Adaptor interfaces to implement functionalities required by CloudStack

• Change components.xml to make the plug-in and adaptors active

2013 Trend Micro

25th Anniversary

Adding a Plugin to CloudStack

• CloudStack assembles the components according to the xml at startup time

• CloudStack assembles the list of APIs it supports according to the PlugableService specified in the properties file

2013 Trend Micro

25th Anniversary

Components.xml Example <components.xml>

......

<management-server class="com.cloud.server.ManagementServerExtImpl"

library="com.cloud.configuration.PremiumComponentLibrary">

.......

<pluggableservice name="VirtualRouterElementService”

key="com.cloud.network.element.VirtualRouterElementService"

class="com.cloud.network.element.VirtualRouterElement"/>

</management-server>

......

</components.xml>

2013 Trend Micro

25th Anniversary

Practical Example

2013 Trend Micro

25th Anniversary

• CS does an excellent job for building clouds • Growing needs for solutions to secure the Cloud

• Seek security solutions to work with CS • Works for both private and public cloud

• Multi-tenant ,self-service,

• Easy to provision

• Resource effective

• Easy to integrate

• ElasterShield to combine the existing security solution with CloudStack

Third party security solution integration

2013 Trend Micro

25th Anniversary

Brief intro to ElasterShield

• Bridge between DeepSecurity and CloudStack

• Hypervisor-based, agentless security protection

• Features • Firewall (L2, L3, L4)

• IDS/IPS

• Application Control

• Web server protection

• Anti-malware

• XenServer

ElasterShield

CloudStack API

DeepSecurity API

2013 Trend Micro

25th Anniversary

Hypervisor

Hypervisor-based

• Agentless for user VM, need an ‘appliance’ on hypervisor

• Capture network traffics

• Scan VM memory

• Can work with any complex network environment

User VM

User VM

Appliance

2013 Trend Micro

25th Anniversary

Hypervisor-Based Solution

• Security Manager (DSM)

• Storing rules/profiles

• Provide UI/ web service API

• Send command to DSVA

DSM

Hypervisor

User VM

Hypervisor

User VM DSVA

DSVA • Security Virtual Appliance (DSVA) • One for each hypervisor

• Hypervisor redirects VM’s traffic to DSVA

• Includes an engine to do the detection/prevention

• Bi-directional communicate between DSM and DSVA

• No impact to user VM if DSVA stops

2013 Trend Micro

25th Anniversary

Integrate as a CS Plug-in

• Security as a service, DeepSecurity as a service provider

– Multi-tenant, defined by admins, subscribed by users, provisioned to VMs

• No modification on CloudStack kernel, keep it as it is

• Provide new security related APIs

• Treat DSM as a external device and send commands to it

• DSVA monitor and security management

• Tailor User VM lifecycle and Hypervisor management with existing CloudStack framework

2013 Trend Micro

25th Anniversary

Architecture Overview

CloudStack Kernel

ElasterShield DSM

Hypervisor

DSVA

User VM Hypervisor

DSVA

User VM

DS API New CS API

Managed by CS

2013 Trend Micro

25th Anniversary

Workflow: Add DSM

CloudStack Manager

ElasterShield Plug-in

DSM

2) Connect to DSM 1) Call plug-in API

Database 3) Persist into DB

1) Admin user or external UI calls Plug-in API • Parameters include DSM url/account/password • One DSM per data center

2) Plug-in calls DSM’s APIs to:

• test connection with specified authentication • Get DSM info such like version, license, etc.

3) Plug-in persists the DSM info into CS database

2013 Trend Micro

25th Anniversary

Workflow: Enable Security protection

CloudStack Manager

ES Plug-in DSM

3) Activate DSVA

1 ) Call plug-in API: 1) Admin user or external UI calls plug-in

API

2) ES Plug-in deploys DSVA • Deploy DSVA on each host of the

cluster • With specified service offering

and template • Start DSVA

3) Activate DSVA • Plug-in calls DSM API to activate

each DSVA

Hypervisor

2) Start DSVA on each hypervisor of the cluster

Hypervisor

DSVA DSVA

2013 Trend Micro

25th Anniversary

Workflow: Define Security Profiles

CloudStack Manager

ElasterShield Plug-in

DSM

2) Call DSM API

1) Call plug-in API

• For this Stage, this will be not implemented in plug-in, because it’s product-specific

• Admins use DSM’s UI to define profiles • Users use Plug-in API to list and apply profiles

Admin defines Rules/profiles

2013 Trend Micro

25th Anniversary

Workflow: Apply security offering

CloudStack Manager

ElasterShield Plug-in

DSM

3) Call DSM API, 1) Call plug-in API

1) User or external UI calls plug-in API • Specify VM id and security offering id

2) Plug-in persists <VM, offering> mapping into DB

• Address multi-tenant problem

3) Plug-in calls DSM API to assign corresponding profile if VM is running

Database 2) Persist into DB

2013 Trend Micro

25th Anniversary

Workflow: Add a new host

CloudStack Manager

ElasterShield Plug-in DSM

5) Call DSM API to activate

1) Call CS API addHost

1) User or UI calls CS API 2) CS add a new hypervisor host as usual 3) Deploy DSVA 4) Plug-in persist DSVA info

Hypervisor

DSVA

2) Add host 3) Deploy DSVA

2013 Trend Micro

25th Anniversary

Workflow: Maintain a host

CloudStack Manager

ElasterShield Plug-in DSM

3) Call DSM API to de-activate DSVA

1) Call CS API prepareHostForMaintenance

1) User or UI calls CS API 2) Stop DSVA on it

• update DSVA status into DB 3) Plug-in call DSM API to deactivate DSVA

• unassign security profiles to VMs that cannot be migrated

Hypervisor

DSVA

2) Power off DSVA

2013 Trend Micro

25th Anniversary

Workflow: Start Virtual Machine

CloudStack Manager

ElasterShield Plug-in DSM

3) Fetch VM info

1) Call CS API startVirtualMachine

1) User or UI calls CS API 2) CS checks DSVA status and starts user VM as usual 3) DSM calls CS API to fetch VM ‘s information 4) DSM notifies DSVA in the same hypervisor with VM

Hypervisor

DSVA

User VM

2) Start User VM

4) DSM notifies ESVA

2013 Trend Micro

25th Anniversary

Workflow: Stop Virtual Machine

CloudStack Manager

ElasterShield Plug-in DSM

3) Fetch VM info

1) Call CS API stopVirtualMachine

1) User or UI calls CS API 2) CS powers off user VM as usual 3) DSM calls CS API to fetch VM’s information 4) DSM notifies DSVA in the same hypervisor with VM

Hypervisor

DSVA

User VM

2) Stop user VM

4) DSM notifies DSVA

2013 Trend Micro

25th Anniversary

Workflow: Destroy Virtual Machine

CloudStack Manager

ElasterShield Plug-in

1) Call CS API destroyVirtualMachine

1) User or UI calls CS API • VM is destroyed as usual by CS

2) When VM gets expunged, plug-in removes <VM, profile> mapping from

DB • VirtualMachineGuru.finalizeExpunge

Database 2) Remove from DB

2013 Trend Micro

25th Anniversary

Workflow: Query Events/Alerts

CloudStack Manager

ElasterShield Plug-in

1) Call plug-in API

1) User or external UI calls plug-in APIx • With query parameters such as time, VM id, user id, etc

2) Plug-in calls DSM’s API to fetch events/alerts

2) Call DSM API DSM

2013 Trend Micro

25th Anniversary

Workflow: DSVA upgrading

CloudStack Manager

ElasterShield Plug-in DSM

2) Call DSM API to upgrade

1) Call plug-in API

1) Call plug-in API to upgrade DSVA , with a URL parameter 2) Plug-in forward the API to DSM 3) DSM notifies all DSVA to upgrade 4) DSVA downloads upgrading package from the URL specified 5) DSVA upgrade itself, it still functions during the upgrading.

Hypervisor

DSVA

3) Notify DSVA

Web server

4) Download upgrade package

2013 Trend Micro

25th Anniversary

ElasterShield Installation

• Target: CloudStack 4.0 and later

• Installation package

– Plug-in binary - ElasterShield

– DSM installation package/binary

– DSVA template

2013 Trend Micro

25th Anniversary

Q & A

2013 Trend Micro

25th Anniversary

Thank You