Terraform in deployment pipelineby Anton Babenko

Hi!I am Anton Babenko, and I enjoy:

● AWS & DevOps

● AWS User Group Norway and DevOpsDays Oslo organizer

● Solve problems

“Getting Started with Terraform”,

terraform-community-modules, Terraform modules generator

(Terrapin), and more…

https://github.com/antonbabenko https://www.linkedin.com/in/antonbabenko

1. Become more familiar with managing infrastructure using CD pipeline

2. See scenarios of integrating Terraform and Packer

3. How to structure infrastructure code?

4. How to version your infrastructure between environments and make it DRY?

Goals of this talk

Do you know?What is:

● Infrastructure as code?

● Deployment pipeline?

● Pipeline as code?

Featuring...

Write, plan, and create infrastructure as code Build automated machine images

Typical CI/CD pipeline

source: https://dzone.com/articles/what-is-continuous-delivery-pipeline

Where are infrastructure changes here?

CI/CD pipeline (CircleCI 2.0)

Structure - all-in-one vs split~/all-in-one-repo/├── packer # Packer configs│ └── app.json├── terraform # Terraform configs│ ├── main.tf│ └── terraform.tfvars└── web # Application code └── index.html

~/infra-repo/├── packer # Packer configs│ └── app.json└── terraform # Terraform configs ├── main.tf └── terraform.tfvars

~/app-repo/└── web # Application code └── index.html

Structure - evolving infrastructure repository~/infra-repo/├── packer # Packer configs│ └── app.json└── terraform # Terraform configs ├── modules # Terraform modules │ ├── network │ │ └── main.tf │ └── service1 │ └── main.tf ├── main.tf └── terraform.tfvars

~/infra-repo/├── packer # Packer configs│ └── app.json└── terraform # Terraform configs ├── modules # Terraform modules │ ├── network │ │ └── main.tf │ └── service1 │ └── main.tf └── environments ├── non-prod │ └── us-east-1 │ ├── main.tf │ └── terraform.tfvars └── prod ├── eu-west-1 │ ├── main.tf │ └── terraform.tfvars └── us-east-1 ├── main.tf └── terraform.tfvars

example1/main.tfresource "random_pet" "bucket" {}

resource "aws_s3_bucket" "app" { bucket = "fullstackfest-${ random_pet .bucket.id}" acl = "public-read"

website { index_document = "index.html" }}

data "template_file" "index" { template = "${file("../../web/index.html")}"

vars { BUILD_DETAILS = "${aws_s3_bucket .app.website_endpoint }" }}

resource "aws_s3_bucket_object" "object" { bucket = "${aws_s3_bucket .app.id}" key = "index.html" content = "${data.template_file .index.rendered}" etag = "${md5(data. template_file .index.rendered)}" content_type = "text/html" acl = "public-read"}

output "app_website_endpoint" { value = "${aws_s3_bucket .app.website_endpoint }"}

FullStackFest!

${BUILD_DETAILS}

$ terraform init...$ terraform plan...$ terraform apply...

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Outputs:

app_website_endpoint = fullstackfest-feasible-basilisk.s3-website-eu-west-1.amazonaws.com

example2/main.tfvariable "subnet_id" { description = "ID of subnet where resources will be created"}

variable "security_groups" { description = "ID of security group EC2 instance will use"}

variable "instance_type" { description = "Type of EC2 instance to launch"}

data "aws_ami" "app" { most_recent = true

filter { name = "name" values = ["fullstackfest-demo-*" ] }}

resource "aws_instance" "app" { ami = "${data.aws_ami.app.id}" instance_type = "${var.instance_type }" subnet_id = "${var.subnet_id}" vpc_security_group_ids = ["${var.security_groups }"]}

output "app_public_ip" { description = "Public IP of EC2 instance running an application" value = "${aws_instance .app.public_ip}"}

packer/app.json{ "builders" : [ { "ami_name" : "fullstackfest-demo-{{uuid | clean_ami_name}}" , "ami_description" : "FullStackFest demo AMI based on Amazon Linux", "instance_type" : "t2.micro" , "region": "eu-west-1" , "type": "amazon-ebs" , "ssh_username" : "ec2-user" , "source_ami_filter" : { "filters": { "virtualization-type" : "hvm", "name": "amzn-ami-hvm-*-x86_64-gp2" , "root-device-type" : "ebs" }, "owners": [ "137112412989" ], "most_recent" : true } } ], "provisioners" : [ { "type": "shell", "inline": [ "# Install nginx, copy index.html into web-root" ] } ]}

# Avoid hard-coded values in *.tf files, use data sources or *.tfvarsdata "aws_ami" "app" { most_recent = true

filter { name = "name" values = ["fullstackfest-demo-*" ] }}

# Tag and name resources consistentlyresource "aws_instance" "app" { ami = "${data.aws_ami.app.id}" instance_type = "${var.instance_type }" subnet_id = "${var.subnet_id}" vpc_security_group_ids = ["${var.security_groups }"]

tags { Name = "fullstackfest-demo-${var. environment }" }}

variable "environment" { description = "Name of environment to create infrastructure (eg, staging, production)"}

# terraform.tfvarsenvironment = "non-prod"

FTP (Frequent Terraform Problems)● Avoid hard-coded values => use data

sources

● Tag and name resources consistently

Next: Terraform modules = reusability

module "sg_web" { source = "git@github.com:terraform-community-modules/tf_aws_sg.git//sg_web?ref=v0.2.3" security_group_name = "fullstackfest-demo-web" vpc_id = "vpc-12345678" source_cidr_block = ["0.0.0.0/0" ]}

resource "aws_instance" "app" { # ...

vpc_security_group_ids = ["${module. sg_web.security_group_id_web }"]

# ...}

Terraform modules● Versioning

● Public/private access

● Local dir or hosted

● Allows:

○ code reuse

○ encapsulate groups of resources

○ testing

Demo - infrastructure as code and deployment pipeline

https://github.com/antonbabenko/terraform-deployment-pipeline-talk

Further thoughts…

● Use linters (tflint), coding styles (fmt), pre-commit hooks

● Automate (no excuses)

● Terraform workspaces, Terragrunt, Atlantis by Hootsuite (super!)

● Version & release infrastructure as the app code

● Using pipelines to manage environments with infrastructure as code by Kief

Morris

Thank you!