OpenCloud At PayPal

Anand Palanisamy 15 Dec 2012

VISION

2 Confidential and Proprietary

Provide a platform that enables agility, availability and innovation.

IAAS  

Risk   Payments   Wallet   Mobile   Ads  

Infrastructure  Services  (Compute,  Storage,  Switches,  FWs,  LBs,  DNS  )  

DEV  Product  Mgmt  

Centralized  Monitoring  

QA   PROD   M&A’s  

Unified  PAAS  

Pre-­‐PROD  

GUIDING PRINCIPLES

Technology !  Adopt Open Source Solutions where ever possible !  No Vendor Lock-in !  Industry Best Practices !  Leverage Industry Investments Functionality !  Self-Service !  Developers First Methodology !  Run Operations as an Engineering Organization !  Agile Development Model and Small 3 to 4 team members !  Delivery Every 2 weeks

VISION

4 Confidential and Proprietary

Provide a platform that enables agility, availability and innovation.

IAAS  

Risk   Payments   Wallet   Mobile   Ads  

Infrastructure  Services  (Compute,  Storage,  Switches,  FWs,  LBs,  DNS  )  

DEV  Product  Mgmt  

Centralized  Monitoring  

QA   PROD   M&A’s  

Unified  PAAS  

Pre-­‐PROD  

•  Closely known Success Stories

•  Lots of customization capability

•  Wider support by all major vendors

•  Very Active Open Source Community

•  We are using Linux, Java, Hadoop

•  Opportunity to hire top talents

•  Foundation

WHY OPENSTACK?

TECHNOLOGY STACK

6 Confidential and Proprietary

Operations Portal Horizon, Ceilometer

PD Deployment Portal

Orchestration Engine Cloud Formation (Heat)

Compute Nova, Swift, Glance, Key Stone

LB Management LBaaS

DNS Management DNSaaS

Cobbler ISC DHCP

Salt Bind RHEL 6.3 KVM Nagios

Compute HP SL230. Gen 8

Local Storage HP 4X600 GB(Mirror)

Network Cisco 4948 & Arista 7050

Load Balancer F5 10.2.2

User Interface

Orchestration

Foundational Services

Software Infrastructure

Hardware Infrastructure

OpEx

Traffic Mgmt Monitoring Metering Stages Workflow Monitoring

•  1 OpenStack deployment per Data Center

•  96 Compute Nodes in a Rack

•  4 Top of Rack Switches (2 PROD, 2 Management)

•  /23 subnet X 2 (PROD), /23 subnet X 2 (Management)

•  Supporting 2 Flavors (12 to 16 VMs per host) = ~ 1300 VMs

•  Utilized existing physical L3, Firewall

•  Nova, Glance, Swift, Keystone, Dashboard (Essex Stable 3)

•  Integrated with LBs, DNS, Change Management, naggios monitoring

CURRENT DEPLOYMENT

•  How to fit into existing infrastructure?

•  Availability Zone Customization

•  Plug-in Options for DNS and LB •  Lab Infrastructure

•  OpenStack Dev. Life Cycle

LESSONS LEARNED

•  Sophisticated networking

•  Bare Metal Provisioning

•  Compute Cells

•  Open Hardware Spec

•  Extend to Dev., QA, Staging, L&P

•  Self-Service, Security

•  Migration to Folsom and beyond

•  Design, Develop, Fix and Contribute to community

•  Inter-Operability Lab and ebay inc distribution

WHAT’S NEXT

COMMUNITY TARGET

CLOUD

YES. WE ARE HIRING

apalanisamy@paypal.com http://ebaycareers.com