DevOps on Steroids Featuring Red Hat & Alantiss - Pop-up Loft Tel Aviv
EC2 Container Service - Distributed Applications at Scale - Pop-up Loft Tel Aviv
-
Upload
amazon-web-services -
Category
Technology
-
view
687 -
download
1
Transcript of EC2 Container Service - Distributed Applications at Scale - Pop-up Loft Tel Aviv
EC2 Container Service -Distributed Applications at Scale
Yaniv Donenfeld, Solutions Architect
Amazon Web Services
APIsec2-run-instances
Order UI User UI Shipping UI
Order Service User Service Shipping
Service
Data Access
Microservices:Taming the monolith
Order UI User UI Shipping UI
Order Service User Service Shipping
Service
Order UI User UI UI
Order Service Service Shipping Service
Order UIOrder UI
User UI UIShipping UI
Order ServiceOrder Service
ServiceService
ServiceService
User Service
Shipping Service
Can we take this paradigm to the
next level?
Image: Izabella Reimers
Image: Izabella Reimers
Resource management
Resourceisolation
Speed
Package Ship Run
docker pushdocker pull
docker runDockerfile
How should we think about building and running distributed
applications?
A different primitive
• {• "environment": [],• "name": "simple-demo",• "image": "my-demo",• "cpu": 10,• "memory": 500,• "portMappings": [• {• "containerPort": 80,• "hostPort": 80• }• ],• "mountPoints": [• {• "sourceVolume": "my-vol",• "containerPath":
"/var/www/my-vol"• }• ],• "entryPoint": [• "/usr/sbin/apache2",• "-D",• "FOREGROUND"• ],• "essential": true• },
Managing the cluster
Image: NATS Press Office
http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41684.pdf
Image: highersights
Listening to our customers
https://github.com/aws/amazon-ecs-agent
100 Node Cluster (End-to-End) 1000 Node Cluster (End-to-End)
https://aws.amazon.com/blogs/compute/how-to-create-a-custom-scheduler-for-amazon-ecs/
# Describe all instances in the ECS cluster containerInstancesArns = getInstanceArns(clusterName) response = ecs.describe_container_instances( cluster=clusterName, containerInstances=containerInstancesArns ) containerInstances = response['containerInstances']
# Sort instances by number of running tasks sortedContainerInstances = sorted( containerInstances, key=lambda containerInstances: containerInstances['runningTasksCount'] )
# Get the instance with the least number of tasks startOn.append(sortedContainerInstances[0]['containerInstanceArn']) logging.info('Starting task on instance %s...', startOn)
# Start a new task response = ecs.start_task( cluster=clusterName, taskDefinition=taskDefinition, containerInstances=startOn, startedBy='LeastTasksScheduler' )
But you don’t have to
The ECS Service Scheduler
Create services
Services launch tasks
Desired state
Desired state
Update service
Update service
Update service
Events
Using metricsto scale your fleet
Deployments
Deployments
Service Discovery
Image: NATS Press Office
ELB
Consul or etcd
https://aws.amazon.com/blogs/compute/service-discovery-via-consul-with-amazon-ecs/
Weave
http://weave.works/guides/service-discovery-with-weave-aws-ecs.html
Image: Steve Davis
“Moving to Amazon ECS significantly improved our service performance. We reduced service response times in the 99th percentile by 50%”
Jason FischlVP of Engineering
“I have managed the orchestration service at Heroku, and experimented with configuring and running numerous open-source orchestration systems, and I am relieved that Amazon's world-class engineering is tackling this problem and offers it as a service.”
Noah ZoschkeFounder
“Out of the box ECS lets us run thousands of containers across multiple availability zones. It's let our development team focus on building the Meteor-specific services needed for our customers to build amazing Connected Client apps. Because ECS pairs well with other technologies like CloudFormation and auto scaling, it dramatically simplified our own devops compared to other options. It's made it possible to run multiple Galaxies and to bring up faithful development environments for each person on the core team in a fraction of the time previously possible.”
Matt DeBergalisCo-founder and VP Product
Image: Robert Couse-Baker
Amazon EC2 Container Registry
• Fully - managed Docker container registry
• Integrated with Amazon ECS• Encrypted in transit and at rest• IAM users and roles• Highly available and scalable• Available in multiple regions• $0.10/GB/Month + AWS data transfer
costs
aws.amazon.com/ecr
ECS CLI
• First version• Simplify local development• Easily set up ECS clusters• Supports Docker Compose• Open source
github.com/aws/amazon-ecs-cli
$ ecs-cli configure -i$ ecs-cli up$ ecs-cli compose up$ ecs-cli compose ps
New: Improved Docker Container Configuration Options
• More Docker options supported in ECS task definitions
• Ideal for advanced Docker users• New additions
– Hostname– Docker labels– Working directory– Privileged execution– Log configuration– …and more (see Amazon ECS
docs
aws.amazon.com/ecr
A system that’s designed to run container-enabled applications in Production
Without worrying about scalability, performance, IAM