1  

Apache  Stratos    Building  a  PaaS  using  OSGi  and  Equinox  

 Paul  Fremantle  

CTO  and  Co-­‐Founder,  WSO2  CommiCer,  Apache  Stratos  

 @pzfreo  #wso2  #apache  

paul@wso2.com    pzf@apache.org    

About  me  

o  CTO  and  Co-­‐Founder  WSO2  

o Working  in  Apache  for  14  years  

o Working  with  Cloud,  SOA,  APIs,  MQTT,  IoT  

o  Also  presenQng  tomorrow  about  IoT  security  (M2M  day  14:15)  

2  

The  Small  Print  

3  

Disclaimer Apache Stratos is an effort undergoing incubation at The Apache Software Foundation (ASF). Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.

4  

Today’s  Buzzword  Bingo!  

PaaS Cloud Elastic Policy

Auto-Scale DevOps jClouds Puppet

Platform IaaS Chef Cassandra

Multi-tenant Load Balancer Governance Monitoring

5  

What  is  a  PaaS  anyway?  

6  

Gartner  Reference  Model  for  Pla0orm  as  a  Service  

Hardware

Integrated PaaS Platform Management, Monitoring, Governance, Provisioning

System Infrastructure or System Infrastructure Services (IaaS)

PaaS  Service  Offerings  

PaaS Technology Core

Cloud Performance Foundation

Cloud Value Foundation (Shared Resources, Multitenancy, Self-Service, Elasticity, Real-Time Versioning, Metadata Management, Subscription/Use Billing)

(In-Memory Computing, Grid/Massive Scale, Auto-Scaling, SLA Enforcement, Use Tracking, High Availability, Security, Data Integrity, Parallel Processing)

Integrated PaaS Application Modeling, Design, Development, Maintenance, Life Cycle Management

Applications

PaaS  

IaaS  

SaaS  

ApplicaQon  Pla]orm  

IntegraQon  Pla]orm  

Business  Process  Management  Pla]orm  

Cloud  Database  Pla]orm  

Other  User  Experience  Pla]orm  

What  is  Cloud?  

o  Depends  who  you  are  o My  daughter:  iCloud  (her  music  in  the  cloud)  o My  mum:  gmail  (her  email  in  the  cloud)  o My  VP  sales:  Salesforce  (his  prospects  in  the  cloud)  

o  Sysadmin:  Amazon/Rackspace/etc  (his  infrastructure  in  the  cloud)  

o  *:  what  you  care  about,  self-­‐provisioned,  managed,  metered  and  paid  per  use,  in  the  cloud  

8

PaaS  is  a  cloud  for  developers  

o  Provisioning  stuff  developers  care  about:  o  Not  VMs,  networks  and  disk  (though  that  might  sQll  be  useful)  

o  ApplicaQon  environments,  databases,  queues  o  Dev  /  Staging  /  ProducQon  o  Different  runQmes:  

o  Java  Servlet/Tomcat,  Node.js,  PHP,  Python  o MySQL,  MongoDB,  Cassandra  o  Etc  

o Working  with  Git,  SVN,  Jenkins,  etc  

9  

Who  needs  a  PaaS  anyway?  

o  Anyone  who  wants  to  provision  applicaQon  environments  on-­‐the-­‐fly  

o  A  basis  for  mulQ-­‐tenant  SaaS  applicaQons  o  ElasQc  scaling  and  management  of  the  infrastructure  

o  ApplicaQon  based  monitoring  of  the  environment  

o More  intelligent  use  of  resources  

10  

What  is  Apache  Stratos?  

●  A  Pla]orm-­‐as-­‐a-­‐Service  (PaaS)  Framework  ●  Built  on  OSGi  and  Equinox  ●  Donated  to  Apache  Solware  FoundaQon  

●  CommiCers  from  Cisco,  Citrix,  SunGard,  Indiana  U,  NASA  JPL,  etc  

●  Deploys  onto  an  Infrastructure-­‐as-­‐a-­‐Service  (IaaS)  Including  Amazon  EC2,  VMWare  vCloud,  OpenStack…etc  

●  Creates  a  secure,  mulQ-­‐tenant,  elasQc,  metered,  billed  PaaS  Supports  private,  public  or  hybrid  PaaS  

 

Quick  Demo    

o Warning:  this  is  alpha  solware  o  Running  on  Amazon  o  Accessed  via  Conference  wifi  o  Hmmm?  Ok  who’s  idea  was  that    

12  

13  

14  

Why  OSGi?  

o  A  set  of  servers  that  need  to  be  deployed  in  different  architectures:  o  Single  JVM  for  developer  tesQng  /  small  footprint  o  Scale-­‐out  for  large  scale  deployment  

o  Re-­‐usable  components  from  a  wider  set  of  middleware  o  Message  Broker  o  HTTP  Load  Balancer  o  IdenQty  Services  o  Logging  and  Metering  o  etc  

15  

© WSO2 2011

Carbon  Framework  

© WSO2 2011

Carbon  Component  Manager  

More  on  Carbon  

o  Based  on  Equinox  o  Using  p2  feature  model  

o  Allows  components  to  be  managed  and  deployed  from  either  local  or  remote  repo  

o  Handles  versions  and  dependencies  o  Supports  rollback    

o  Command  line  p2  management  and  OSGi  Console  o  Consistent  model  for  scaling    

o  Master/Slave  +  Deployment  Synchronizer  

o  Consistent  Registry/Repository  o  Security  model  

18  

Lessons  learnt  from  Carbon  

o  OSGi  is  a  bit  of  a  pain  someQmes  o  We  hide  as  much  as  possible  from  end-­‐users  

o  Used  properly  it  enables  massive  producQvity  gains  o  Small  company  with  >  15  products  and  world  class  deployments  

o  Small  teams  can  be  producQve  o  End  users  can  enhance  the  products  with  their  own  components  

o  MinificaQon  o  Scaling  out  

19  

What  does  a  PaaS  look  like  

>  stratos  subscribe-­‐cartridge  php  myphp    -­‐-­‐repo-­‐url  hCps://github.com/pzfreo/php-­‐s4    -­‐-­‐deployment-­‐policy  economy          Subscribes  to  a  runQme  (e.g.  php)  >  git  push  php-­‐s4        Auto  deploys  the  code  into  the  cartridge  

20  

Stratos  Layered  Architecture  

Stratos  Cartridges  ●  A  component  which  can  be  plugged  into  Stratos,  so  that  it  

can  use  Stratos  core  services  of  the  framework  layer  

●  Cloud-­‐aware  pla]orm  environment  extending  legacy  technologies  into  the  cloud  and  delivering  cloud  benefits  

●  Users/DevOps  may  create  custom  cartridges  and  host  any  applicaQon,  container,  or  framework  in  a  Stratos  Cloud.    (Eg:  node.js,  Ruby,  MongoDB,  Wordpress,  .net,  Oracle  WebLogic  or  JBoss  SOA  Pla]orm..etc  

●  Single  tenant  or  mulQ-­‐tenanted  

Stratos  Cartridges  

Puppet  

o  Puppet    o  Powerful  devops  and  provisioning  tool  o  Used  by  the  cartridges  to  auto-­‐provision  o  Today  to  create  a  cartridge  you  basically  build  a  puppet  script  

o  Next  step  is  that  will  be  a  generic  Puppet  cartridge  

24  

MulG-­‐factored  Auto  Scaling  

●  Integrate  both  real-­‐Qme  and  rule-­‐based  decision  making  

●  Scaling  algorithm  can  use  mulQple-­‐factors  ●  Capable  of  predicQng  future  load  

Scalable  and  Dynamic  Load  Balancing  

●  Comes  with  an  HTTP  Load  Balancer  as  a  cartridge  ●  Service  level  load  balancing  ●  Auto  Scaling  load  balancers  ●  External  load  balancer  integraQon  support  

●  Supports  HAProxy  for  non-­‐HTTP  loads  

Cloud  BursGng  

Controlling  IaaS  Resources  ParGGons  ●  Logically  group  IaaS  resource  locaQons  ●  ParQQons  are  important  to  make  applicaQon  high  availability  ●  Cartridge  instances  are  spawned  inside  these  parQQons  ●  ParQQons  are  defined  by  DevOps  

Network  ParGGons  ●  Logical  groups  of  mulQple  parQQons,  that  are  in  the  same  

network.  ●  Stratos  will  spawn  Load  Balancers  per  network  parQQon.  ●  Since  LB  instances  and  cartridge  instances  reside  in  same  

network,  they  can  communicate  using  private  IP  addresses.  ●  Used  in  deployment  policies.  

Smart  Policies  Auto  Scaling  Policy  ●  Defines  threshold  values  pertaining  to  scale  up/down  

decision  ●  Auto  scaler  will  refer  this  policies  ●  Defined  by  DevOps  ●  During  cartridge  subscripQon  user  specifies  the  auto  scaling  

policy  to  be  used  

Deployment  Policy  ●  Defines  how  and  where  to  spawn  cartridge  instances  ●  Defines  minimum  and  maximum  instances  in  a  selected  

service  cluster  ●  Defined  by  DevOps  based  on  deployment  paCerns  ●  Guarantee  SLA  of  cloud  applicaQons      

More  highlights…  ●  Unified  communicaQon  ●  Centralized  monitoring  and  metering  ●  Web  UI  and  CLI  ●  REST  API  for  integraQon  with  external  PaaS  

management  interface  for  both  DevOps  and  user  interacQon  

●  ArQfact  distribuQon  coordinator  ●  Persistence  volume  support  for  cartridges  ●  Gracefully  shutdown  instance  when  scale  down  

App  Factory  Apache  License  Open  Source  

31  

Why  Stratos?  

o  Inherently  MulQ-­‐tenant  o  Highly  Extensible    

o  OSGi  components,  AMQP-­‐based  architecture,  Policies,  IaaS,  etc  

o  Built  in  Single  Sign  On/IdenQty  management  model  o  Policy  based  o  Supports  a  simple  model  of  parQQons    o  Smart  Load  Balancing  o  Non-­‐HTTP  Load  balancing  o  LXC  support  o  Apache  FoundaQon  o  App  Factory  takes  it  to  the  next  level  

32  

Want  to  help?  

o  Feedback  and  tesQng  o  CreaQng  Cartridges  

o  Virgo?  Birt?  MosquiCo?  Etc  o  ExisQng  apps  too  (e.g.  Drupal,  Wordpress,  etc)  

o  PorQng  to  Google  Compute  o  Autoscaling  policies  and  rules  o  Ease  of  use  –  creaQng  a  Vagrant/LXC  distro  o  Help  with  App  Factory      

33  

Stratos  @  Google  Summer  of  Code  

6  proposals  o  Python  Agent  o  AWS  LB  support  o  GCE  LB  support  o  New  scaling  algorithms  o  New  LB  algorithms  o  GCE  support    

34  

Resources  

o  hCp://stratos.incubator.apache.org/  o  Hangout  archives  (and  future  hangouts)  

o  hCp://stratos.incubator.apache.org/events/hangouts.html    

o  Carbon:  o  hCp://www.slideshare.net/wso2.org/introducQon-­‐to-­‐the-­‐wso2-­‐carbon-­‐pla]orm-­‐webinar    

o  Stratos  o  hCp://www.slideshare.net/lakwarus/overview-­‐of-­‐apache-­‐stratos-­‐incubaQon-­‐40-­‐architecture    

35  

QuesQons?