1. Future of Apache CloudStack: modular, distributed, and hackable Darren Shepherd CitrixCCCEU 2013

2. About Me Apache CloudStack CommitterPreviously - Cloud Architect @ GoDaddy.com Built three generations of clouds 2nd Custom CloudStack Compatible1st CloudStack 2.x, custom storage framework 3rd 100% from scratchBeen developing IaaS orchestration systems full time for the last 4 yearsCurrently Do something @ Citrix Focus on open sourceI care about architecture and design 3. What's This About My opinionsWhat I think we should focus on architecturallyWhat I'm trying to accomplish 4. General Comments Do more with less Identify patterns in code and create frameworks For example, make Abstract base classesCopy and paste is badNeed to leverage more open source frameworks jOOQ, Spring TX, Netflix Archaius/Servo, Jackson, Apache jclouds, Flyway/Liquibase, Apache CommonsPast/Current ACS adds cloudiness to legacy technologiesCurrent/Future ACS orchestrates cloudy technologiesAlways provide a way for Apache CloudStack to do nothing and delegate to something else 5. Modular Why? Enforce design and architectural boundariesBetter pluggabilityQuality/TestingLays the foundation for distributed architecture 6. Modular What's been done 4.3 - Spring Modularization META-INF/cloudstack/mycomponent/module.propertiesMETA-INF/cloudstack/mycomponent/*context.xmlOver 50 Modules 7. Stats ~400k lines of Java (not 1.2m) 32% - engine, framework, services, usage, util 42% - agent, api, core, server 26% - pluginsWe still have big blobs of code 74% Core CodeCore Code is not really core. Includes VR, CP, SSVM, XenServer, others 8. What needs to be done Package plug-ins in separate RPM/DEBsBreak out more code from /server/srcBreak apart *ManagerImpl classes Implement too many interfaces that belong in different architectural layersModularize VR and VPC code 9. Distributed Architecture Maybe you don't care Many don't Monolithic deployment is simple and scalable enough for most use caseSupport multiple deployment modelsScalability Scale tiers according to load characteristicsReliability Put certain components closer to resources 10. Monolithic Single JVMRedundant JVM 11. Fully Distributed System Services Event Bus Lock ManagerServers APIBack end JobsHost Manager 12. How Event Bus Hazelcast or RedisDo not use reliable messaging Lock Manager Works more like UDP multicastHazelcast or ZooKeeperSame binary on all servers Profiles used to determine which Spring modules are activated on start 13. Hackability Should be relatively simple to add one off functionality Every cloud deployment is slightly differentNeed to stay relevant with upcoming technologies Want people to integrate first with ACS because It worksIt's easy 14. Totally scientific observation Two types of sys admins Those who hate Java for no good reasonThose who hate Java for a good reasonThose that will tolerate developing in JavaThose who love systems (admins/engineers) 15. Dynamic languages can be goodThose that Those who will tolerate developing love systems in a dynamic (admins/engineers) language 16. Hackability Move systems related code to pythonNo Java on the hypervisor!Need a new ServerResource architecture that delegates to agents in any language Work for Hyper-V is a step in the right directionDecouple VR from HypervisorResourceCommunity help needed Good work, GCE API and hopefully new AWS API 17. Questions?