InterSystems Caché Database: Spinning up hundreds of nodes in seconds
-
Upload
intersystems-corporation -
Category
Software
-
view
276 -
download
4
Transcript of InterSystems Caché Database: Spinning up hundreds of nodes in seconds
![Page 1: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/1.jpg)
© 2016 InterSystems Corporation. All rights reserved.© 2016 InterSystems Corporation. All rights reserved.
Spinning up 100s of Nodes in SecondsDefine, Provision and Deploy a Cloud Infrastructure
![Page 2: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/2.jpg)
© 2016 InterSystems Corporation. All rights reserved.
The Context
2
![Page 3: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/3.jpg)
© 2016 InterSystems Corporation. All rights reserved.
The issue
3
![Page 4: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/4.jpg)
© 2016 InterSystems Corporation. All rights reserved.
Cloud Architecture & Scaling
4
![Page 5: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/5.jpg)
© 2016 InterSystems Corporation. All rights reserved.
Abc.
The Answer
5
![Page 6: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/6.jpg)
© 2016 InterSystems Corporation. All rights reserved.
Cloud-Native Architecture: Pet or Cattle?
6
![Page 7: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/7.jpg)
© 2016 InterSystems Corporation. All rights reserved.
• Cloud-Native Architecture• “Infinite” Horizontal Scalability
• Microservices• DevOps• Continuous Delivery (CI/CD)• Open Source software
Big-Data Architecture Landscape
7
0
5
10
15
20
2007 2008 2009 2010 2011 2012 2013 2014
Billions ReqSource:2015 State of the Software Supply Chain Report – from Sonatype
![Page 8: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/8.jpg)
© 2016 InterSystems Corporation. All rights reserved.
82% of components used by automotive manufacturers are now sourced from suppliers
The Supply Chain
8
![Page 9: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/9.jpg)
© 2016 InterSystems Corporation. All rights reserved.
Automation
9
![Page 10: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/10.jpg)
© 2016 InterSystems Corporation. All rights reserved.
• System Administrators knows• 6 Servers
• 2 x Intel Xeon E5-2699 v4
• 128GB RAM
• 800GB SSD auto-tiering
• 10TB HDDs for backup, snapshots
• Storage SAN discussion• Data heat, auto-tiering
• SSD for production
• HDD for snapshots, backup etc.
Not So Cloud-Ready Architecture
10
![Page 11: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/11.jpg)
© 2016 InterSystems Corporation. All rights reserved.
5MB HDD being delivered (circa 1956)
11
![Page 12: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/12.jpg)
© 2016 InterSystems Corporation. All rights reserved.
• Cloud Service Models• SaaS• PaaS• IaaS
• We’ll need a design and awareness• Networking needs to be understood & configured• Security configuration verified• Dependencies checked• All infrastructure concerns brought forward
Cloud-Native Infrastructure
12
![Page 13: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/13.jpg)
© 2016 InterSystems Corporation. All rights reserved.
• Time for Interdisciplinary Teams• SysAdmin + Network + DBA + Developer:
• Developers & Operation +
• Microservices =
• Functional App Service Team
Cloud-Native Infrastructure How-To
13
![Page 14: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/14.jpg)
© 2016 InterSystems Corporation. All rights reserved.
DevOps
14
![Page 15: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/15.jpg)
© 2016 InterSystems Corporation. All rights reserved.
"Organizations which design systemsare constrained to produce systems
which are copies of the communication structures of these organizations.”
Q: What is the communication structure like in our organisations?
Conway’s Law
15
![Page 16: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/16.jpg)
© 2016 InterSystems Corporation. All rights reserved.
16
• Grassroots in 2008• Mainstream on CIO & CTO agenda now• Agile Process: from Dev to Ops• It’s all about ALM (release management) or
• Application Lifecycle Management• CD tools, the most influential• DevOps, for rapid changes and CD, drive
the adoption of microservices architecture (MSA)
DevOps
![Page 17: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/17.jpg)
© 2016 InterSystems Corporation. All rights reserved.
• Let’s Admit it, we haven’t learnt the lesson yet:• A live production environment is never finished• Are we building a software architecture that is designed for changes?• Do we adhere and have we signed up to the Reactive Manifesto?• What about the 12 factor app?
Cloud-Native Architecture: MSA & DevOps
17
![Page 18: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/18.jpg)
© 2016 InterSystems Corporation. All rights reserved.
• Image Artifact Provisioning• VMware, KVM, Xen, Vagrant• Docker, CoreOS, LXD• Packer• Veewee• CloudFoundry
• Configuration Management tool• Ansible• Puppet• Chef• Salt• etc.
DevOps & Tooling
18
• Infrastructure Provisioning• Amazon AWS CloudFormation• Google GCP • Microsoft Azure Automation and Azure Resource
Manager (ARM)• Cobbler• Apache Provisionr• Apache Brooklyn• Canonical Juju• Dell Crowbar• HPE Stackato• OpenStack Heat• Capistrano• Boto• Fog• Terraform• custom solutions…
![Page 19: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/19.jpg)
© 2016 InterSystems Corporation. All rights reserved.
Provisioning Artifacts & Infrastructures
19
Caché ActorsActors
A
VMware, KVM, Xen, VagrantDocker, CoreOS, LXDPacker Veewee CF
AnsiblePuppet ChefSalt
CloudFormation GCP ARM Cobbler ProvisionrBrooklyn Juju CrowbarStackato HeatCapistrano BotoFog Terraform
![Page 20: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/20.jpg)
© 2016 InterSystems Corporation. All rights reserved.
The Strategy
20
InterSystems
Actors
A
![Page 21: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/21.jpg)
© 2016 InterSystems Corporation. All rights reserved.
• Terraform• Infrastructure Provisioning & Configuration• Simple Declarative Configuration Files
• You configure resources (security groups, TCP rules, NAT, compute nodes, storage, etc.)
• Cloud agnostic• Idempotent• Adaptive• Supports Multiple Cloud Providers• Supports Variables and• Variable Interpolation• Simple Math operators• Functions• Templates• Infrastructure State awareness
Infrastructure as Code
21
![Page 22: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/22.jpg)
© 2016 InterSystems Corporation. All rights reserved.
Terraform Declaration
22
![Page 23: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/23.jpg)
© 2016 InterSystems Corporation. All rights reserved.
• Terraform and other tools can perform initial Deployment • Can they all handle more subtle and custom needs?• What happens when we need to run an Update and an Upgrade?• How do you perform the above in the Cloud?
• Issues• Network configuration• 1000s of instances• Security Settings• Service Discovery• What is your SLA?• Safeguarding the Data
Deployment
23
solid Lego bricks
![Page 24: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/24.jpg)
© 2016 InterSystems Corporation. All rights reserved.
What if…• I was able to package my app in such a way that compilers, interpreters, OS lib dependencies and other
libs, npm, .so, jvm, jars, RESTful APIs etc. are never going to be a problem?• I was able to have full portability• I was able to secure it• I was able to version control the full “service” image I’m spinning up and it has a:
• Lightweight footprint• Minimal overhead• Fast boot time• Simplified maintenance and• Packaging is supported by all Cloud providers?
Containers
24
![Page 25: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/25.jpg)
© 2016 InterSystems Corporation. All rights reserved.
• Why do tools like Ansible, Puppet, Chef and Salt exist?• to take care of consistency across the infrastructure for updates…
• What if our infrastructure was un-touchable so that you would not need CM tools and the risk?• How do you update 1000s of nodes?• How do you guarantee consistency?• Can you offer zero-downtime?• How long does it take you to perform that update?• What about DB schema migration?• etc.
Immutable Infrastructure
25
![Page 26: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/26.jpg)
© 2016 InterSystems Corporation. All rights reserved.
• Egenera• RightScale• Scalr• CliQr• Flexiant• Cloudbolt• Cloudfoundry• VMware vCloud Air• Kubernetes
• Google sponsored• Mesosphere
• Apache Mesos• Docker UCP (Universal Control Pane)
• Panamax• Rancher• Shipyard• Vamp• Kitematic
Cloud Management Platforms (CMP)
26
![Page 27: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/27.jpg)
© 2016 InterSystems Corporation. All rights reserved.
A Long List…
27
![Page 28: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/28.jpg)
© 2016 InterSystems Corporation. All rights reserved.
• Mesos & Mesospshere• The DCOS • Init for the DC
• Kubernetes• OSS for automating
• Deployment
• Operations and
• Scaling of containerized apps
• Docker Swarm• Host clustering and• Container Scheduling
• Rancher• Enterprise-level access control• Container orchestration & scheduling
Cloud Cluster Resource Orchestrators
28
![Page 29: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/29.jpg)
© 2016 InterSystems Corporation. All rights reserved.
1. One single Trend makes you trendy A Confluence of trends determines a cultural shift for major changes
2. Cloud-Native architecture is here The truth is in the code Infrastructure-as-code Immutable infrastructure
3. The future will feature Automation Autonomic Computing & Promise Theory
Summary
29
![Page 30: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/30.jpg)
© 2016 InterSystems Corporation. All rights reserved.
30
![Page 31: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/31.jpg)
© 2016 InterSystems Corporation. All rights reserved.
Questions
31
https://community.intersystems.com/
![Page 32: InterSystems Caché Database: Spinning up hundreds of nodes in seconds](https://reader031.fdocuments.us/reader031/viewer/2022021919/587a76211a28abf0468b5173/html5/thumbnails/32.jpg)
© 2016 InterSystems Corporation. All rights reserved.
• http://www.appian.com/blog/bpm/avoiding-the-two-biggest-mistakes-in-software-tco-analysi
• http://kief.com/configuration-drift.html
• http://martinfowler.com/bliki/ImmutableServer.html
• http://martinfowler.com/articles/microservices.html
• https://www.ibm.com/developerworks/community/blogs/BizTechAlign/entry/devops-ard-mgr5?lang=en
• https://www-01.ibm.com/marketing/iwm/iwm/web/signup.do?source=swg-rtl-sd-calc&S_PKG=ov26141&S_TACT=C25600PW&CT=ISM0056
• http://www.infoq.com/articles/microservices-intro
• http://highscalability.com/amazon-architecture
• http://techblog.netflix.com/
• http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf
• http://pivotal.io/platform/migrating-to-cloud-native-application-architectures-ebook
• “The Phoenix Project” by Jim Kim (DevOps), IT Revolution Press, ISBN: 978-0988262591
• “Release it!” by Mike Nygard, Pragmatic Bookshelf, ISBN: 978-0978739218
• “Deliver better, safer software even faster with sonatype software supply chain solutions”, PDF by Sonatype
• https://www.youtube.com/watch?v=GaHzdqFithc (microservices & Docker @GILT)
• https://www.youtube.com/watch?v=HYiLzji7MuY (Stefan Tilkov on architecture)
• http://www.reactivemanifesto.org/
• http://12factor.net/
References
32