Building infrastructure as code using Terraform (Q&P)

DevOps Krakow meetup17.1.2018

Agenda

1. Terraform basics2. Frequent Terraform Questions3. Frequent Terraform Problems4. Your turn

Any questions?I read all emails - anton@antonbabenko.com

Follow me on twitter and github - @antonbabenko

Anton BabenkoI enjoy:

● DevOps, AWS, Terraform (since 2015)● Open-source:

○ https://github.com/terraform-aws-modules○ https://modules.tf (work in progress)○ https://github.com/antonbabenko - more projects

● Organise events (AWS User Group Norway, HashiCorp User Group Oslo, DevOpsDays Oslo)

● Solving problems

PS: I am looking for Terraform companions to join me!

Some facts about terraform-aws-modules● Terraform AWS modules have 450K+ downloads per month (Dec 2017) from the

Terraform Registry● Terraform AWS security group module was mostly written offline

Featuring...

Write, Plan, and Create Infrastructure as Code

Terraform 101 (main.tf)provider "aws" { region = "eu-west-1" }

resource "random_pet" "bucket" {}

resource "aws_s3_bucket" "app" { bucket = "hi-${random_pet .bucket.id}"

website { index_document = "index.html" }}

data "template_file" "index" { template = "${file("index.html")}"

vars { BUCKET = "${aws_s3_bucket .app.website_endpoint }" }}

resource "aws_s3_bucket_object" "object" { bucket = "${aws_s3_bucket .app.id}" key = "index.html" content = "${data.template_file .index.rendered}" etag = "${md5(data. template_file .index.rendered)}" content_type = "text/html" acl = "public-read"}

output "app_website_endpoint" { value = "${aws_s3_bucket .app.website_endpoint }"}

index.html can access:${BUCKET}

$ terraform init$ terraform plan$ terraform apply

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Outputs:

app_website_endpoint = hi-feasible-basilisk.s3-website-eu-west-1.amazonaws.com

Terraform training materialhttps://www.slideshare.net/AntonBabenko/continuously-delivering-infrastructure-using-terraform-and-packer-training-material

Frequent Terraform Questions (FTQ)

So, how to get started with Terraform?1. https://www.terraform.io/intro/getting-started/install.html 2. Get infrastructure modules from Terraform Registry. For example, AWS modules -

https://registry.terraform.io/modules/terraform-aws-modules 3. Follow instructions in README.md, check examples, open issues and pull requests4. Read a book (Getting Started with Terraform or Terraform Up & Running)

Why Terraform and not AWS CloudFormation/Azure ARM templates/Google Cloud Deployment Manager?

Terraform manages 70+ providers, has easier syntax (HCL), has native support for modules and remote states, has teamwork related features. Terraform is an open-source project (670 stars on AWS provider, 10K stars on Terraform core).

https://medium.com/@piotrgospodarek/cloudformation-vs-terraform-990318d6a7de

https://cloudonaut.io/cloudformation-vs-terraform/

https://www.slideshare.net/AntonBabenko/continuously-delivering-infrastructure-using-terraform-and-packer-training-material

What is the point of using Terraform if you’re running AWS only? Isn’t Terraform just an unnecessary abstraction, why not stick to CloudFormation?

● Terraform has easier syntax (HCL)● Native support for modules and remote states● Teamwork related features (eg, lock, plan to file)● Abstractions (primitives and modules) are necessary for anything good● Terraform Registry (check verified modules)● Terraform is an open-source project!

What are the tools/solutions out there?● Terraform Registry (https://registry.terraform.io/) - collection of public Terraform modules for

common infrastructure configurations for any provider. I maintain verified AWS modules there.● Thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules -

https://github.com/gruntwork-io/terragrunt ● Terraform linter to detect errors that can not be detected by `terraform plan` -

https://github.com/wata727/tflint ● Terraform version manager - https://github.com/kamatama41/tfenv ● A web dashboard to inspect Terraform States - https://github.com/camptocamp/terraboard ● Jsonnet - The data templating language - http://jsonnet.org● A unified workflow for collaborating on Terraform through GitHub and GitLab - https://atlantis.run/

This list is much longer, really…

How to handle secrets in Terraform?1. Can you accept secrets to be saved in state file in plaintext? Probably not.

a. AWS IAM password & access secret keys - use PGP as keybase.iob. AWS RDS - set dummy password and change after DB is createdc. AWS RDS - use iam_database_authentication_enabled = trued. EC2 instance user-data + AWS KMSe. EC2 instance user-data + AWS System Manager’s Parameter Store

2. Other options:a. Secure remote state location (S3 bucket policy, KMS key)

How to integrate Terraform with ...?● Use outputs (human-friendly)● Use null_resource + local-provisioner for WAF associations

resource "null_resource" "auto_instructions" { triggers = { waf_acl_id = "${aws_waf_web_acl .this.id}" }

provisioner "local-exec" { command = "aws waf-regional associate-web-acl --web-acl-id ${ aws_waf_web_acl .this.id} --resource-arn ${data. terraform_remote_state .alb_public.this_alb_arn}" }}

Frequent Terraform Problems (FTP)

Upgraded Terraform version, and there is a breaking bug, so I want to rollback, but I can’t because state file has been upgraded already.

● State file should be versioned (!), download previous version of state file, run “terraform state push old_version.tfstate”

● Lock terraform version, lock module and providers version (available in Terraform 0.11)● Read upgrade guides and CHANGELOG.md files:

○ https://www.terraform.io/upgrade-guides/0-11.html○ https://github.com/hashicorp/terraform/blob/master/CHANGELOG.md○ https://github.com/terraform-providers/terraform-provider-aws/blob/master/CHANG

ELOG.md

What is your Terraform question or problem?

Hints: Testing? Versioning? Code structure? Working as a team? CI/CD? Automation? Integration with other tools? modules.tf ? Code generation? Missing

tools/features? Syntax sugar (features and types of variables)? How to contribute?

Thanks!

Any questions?anton@antonbabenko.com

twitter.com/antonbabenko

linkedin.com/in/antonbabenko