Ensuring a secure foundation for your AWS Containers - Chris Swan's AWS Loft talk in London
TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft
-
Upload
amazon-web-services -
Category
Technology
-
view
400 -
download
0
Transcript of TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft
![Page 1: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/1.jpg)
TurboCharge Your Continuous Delivery Pipeline with ContainersYaniv Donenfeld, Solutions ArchitectAmazon Web Services
![Page 2: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/2.jpg)
What to expect from the session
• Best practices for containers in continuous delivery solutions
• Toolset to implement such solutions• Demos
![Page 3: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/3.jpg)
Why use containers?
• Process isolation• Portable• Fast• Efficient
![Page 4: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/4.jpg)
Why use containers for continuous delivery?
• Roll out features as quickly as possible• Predictable and reproducible environment• Fast feedback
![Page 5: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/5.jpg)
Demo application architecture
Nginx Proxy Ruby on Rails web app
PostgreSQL on RDS
![Page 6: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/6.jpg)
Amazon EC2 Container Service
• Highly scalable container management service• Easily manage clusters for any scale• Flexible container placement• Integrated with other AWS services• Extensible
• Amazon ECS concepts• Cluster and container instances• Task definition and task
![Page 7: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/7.jpg)
Development and deployment workflow
Orchestration layer
Code repository Build environment
Test environment
Deployment environment
Source
![Page 8: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/8.jpg)
Stage 1 - Source
![Page 9: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/9.jpg)
Development environment
Code repository
Source
![Page 10: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/10.jpg)
AWS CodeCommit
• Private Git repository• Fully managed• Secure• Highly available and scalable
• Alternatives• GitHub• Bitbucket
![Page 11: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/11.jpg)
Docker and Docker Toolbox
• Docker (Linux > 3.10) or Docker Toolbox (OS X, Windows)
• Define app environment with Dockerfile
![Page 12: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/12.jpg)
Dockerfile
FROM ruby:2.2.2RUN apt-get update -qq && apt-get install -y build-essential libpq-devRUN mkdir -p /opt/webWORKDIR /tmpADD Gemfile /tmp/ADD Gemfile.lock /tmp/RUN bundle installADD . /opt/webWORKDIR /opt/web
![Page 13: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/13.jpg)
Docker Compose
Define and run multi-container applications:1. Define app environment with Dockerfile2. Define services that make up your app in
docker-compose.yml3. Run docker-compose up to start and run
entire app
![Page 14: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/14.jpg)
docker-compose.yml
proxy: build: ./proxy ports: - "80:80" links: - webweb: build: ./web command: bundle exec rails server -b 0.0.0.0 environment: - SECRET_KEY_BASE=secretkey expose: - "3000"
![Page 15: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/15.jpg)
Amazon ECS CLI
• Easily create Amazon ECS clusters & supporting resources such as EC2 instances
• Run Docker Compose configuration files on Amazon ECS
New
![Page 16: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/16.jpg)
Amazon ECS CLI
> ecs-cli configure> ecs-cli compose build> ecs-cli compose up --local
New
![Page 17: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/17.jpg)
It’s Dem-o-clock!
![Page 18: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/18.jpg)
Stage 2 - Build
![Page 19: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/19.jpg)
Build environment
Build environment
![Page 20: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/20.jpg)
Partners
![Page 21: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/21.jpg)
Jenkins
• Extensible• Flexible builds
• Ant or Maven based projects• Docker images
• Optionally runs in Docker container
![Page 22: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/22.jpg)
CloudBees Docker Build and Publish plugin
![Page 23: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/23.jpg)
Amazon EC2 Container Registry
• Private Docker Repository• v2 Docker Registry• AWS Identity and Access Management (IAM)
and AWS Auth integration• Low latency push, pulls, and inspection
• Alternatives: • DockerHub• Docker Trusted Registry
New
![Page 24: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/24.jpg)
Stage 3 - Test
![Page 25: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/25.jpg)
Test environment
Test environment
![Page 26: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/26.jpg)
rspec and capybara-webkit
require 'rails_helper.rb'
feature 'Signing in' do scenario 'can sign in' do visit '/users/sign_in' within("#new_user") do fill_in 'Email', :with => '[email protected]' fill_in 'Password', :with => 'password' end click_button 'Log in' expect(page).to have_content('Signed in successfully.') endend
![Page 27: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/27.jpg)
Jenkins
• Run tests directly via Docker run• Run tests in a Docker slave on Amazon ECS
![Page 28: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/28.jpg)
CloudBees Jenkins ECS plugin
![Page 29: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/29.jpg)
Jenkins slave Dockerfile
FROM jenkinsci/jnlp-slaveUSER rootRUN apt-get update -qq && \ apt-get install -y -qq git curl wget build-essential […]RUN apt-get install -y qt5-default libqt5webkit5-devRUN apt-get install -y xvfb x11-xkb-utils xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic x11-appsENV RUBY_VERSION 2.2.2RUN echo 'gem: --no-document' >> /usr/local/etc/gemrc &&\ mkdir /src && cd /src && git clone https://github.com/sstephenson/ruby-build.git &&\ cd /src/ruby-build && ./install.sh &&\ cd / && rm -rf /src/ruby-build && ruby-build $RUBY_VERSION /usr/local
![Page 30: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/30.jpg)
Jenkins slave Dockerfile
RUN gem update --system && gem install bundler
# Install GemsWORKDIR /tmpADD Gemfile /tmp/ADD Gemfile.lock /tmp/RUN bundle install
USER jenkins
![Page 31: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/31.jpg)
It’s Dem-o-clock!
![Page 32: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/32.jpg)
Stage 4 - Deploy
![Page 33: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/33.jpg)
Deployment environment
Deployment environment
![Page 34: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/34.jpg)
Amazon ECS CLI
> ecs-cli up > ecs-cli compose up> ecs-cli ps
![Page 35: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/35.jpg)
AWS Elastic Beanstalk
• Deploy and manage applications without worrying about the infrastructure
• AWS Elastic Beanstalk manages your database, Elastic Load Balancing (ELB), Amazon ECS cluster, monitoring, and logging
• Docker support• Single container (on Amazon EC2)• Multi container (on Amazon ECS)
![Page 36: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/36.jpg)
Putting it all together
![Page 37: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/37.jpg)
Putting it all together
Orchestration layer
![Page 38: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/38.jpg)
AWS CodePipeline
Model and automate your software release processes• Rapid delivery• Configurable workflow• Customizable• Highly integrated
![Page 39: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/39.jpg)
It’s Dem-o-clock!
![Page 40: TurboCharge Your Continuous Delivery Pipeline with Containers - Pop-up Loft](https://reader035.fdocuments.us/reader035/viewer/2022081517/58850cfc1a28abd05e8b4919/html5/thumbnails/40.jpg)
Takeaways
• Use Amazon ECS CLI to run application• Run Jenkins jobs in containers• Let AWS CodePipeline orchestrate your pipeline