Post on 21-May-2020
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Infrastructure-as-Code with Oracle Cloud Platform
Umesh TannaPrincipal Technology Sales ConsultantSales Consulting Centers (SCC)Apr 10, 2018
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
2
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Terraform
Chef
Stack Manager
PSM
CLI
1
2
3
3
4
5
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Infrastructure as Code (IaC)
Infrastructure as code (IaC) refers to the process of provisioning and managing (provisioning, updating and destroying) data centers through machine-readable definition files, as opposed to interactive configuration tools, or even physical hardware configuration
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Infrastructure as Code (IaC)
•Agile
•Consistent
•Repeatable
•Extensible
•Standardization
•Scale
•Version control
•Peer review
•Automated testing
•Release management
•Documentation
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
TerraformOCI and OCI(Classic), Example is OCI
6
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform – built by HashiCorp
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform – What it is?
• A tool for building, changing, and versioning infrastructure
• Manage major cloud service providers.
• Configuration files are used to describe resources to Terraform.
• Terraform generates an execution plan describing what it will do to reach the desired state, and then executes it to build it
• As the configuration changes, Terraform is able to determine what changed and create incremental execution plans
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform for Oracle Cloud Platform and Infrastructure
DATABASE | JAVA | APP CONTAINER+ | MYSQL+ | …
COMPUTE | NETWORK | STORAGE | CLOUD @ CUSTOMER
Oracle Cloud Platform ProviderOracle Cloud Infrastructure Classic | Oracle Cloud Infrastructure*
ORACLE PROVIDERS
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform for Oracle Cloud Platform and Infrastructure
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform For Oracle Cloud Infrastructure Classic – Built-in
The Identity Domain name (for Traditional accounts)
Service Instance ID (for IDCS accounts) of the env
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform For Oracle Cloud Platform(PaaS) – Built-in
The Identity Domain name (for Traditional accounts)
Identity Service ID (for IDCS accounts) of the env
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform For Oracle Cloud Infrastructure – Plug-in
https://github.com/oracle/terraform-provider-oci/
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform For Oracle Public Cloud – Also available as RPM
http://yum.oracle.com/repo/OracleLinux/OL7/developer/x86_64/index.html
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Getting Started with Terraform (OCI)• Download
– binary, apt, yum, choco, brew
• Create a .tf file in a workspace
• hw.tf
• output "hw" {
• value = "test” }
• $ terraform apply
• Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
• Outputs:
• hw = test
• Providers… ->
./├── terraform├── terraform-provider-atlas├── terraform-provider-aws├── terraform-provider-azure├── terraform-provider-azurerm├── terraform-provider-chef├── terraform-provider-cloudflare├── terraform-provider-cloudstack├── terraform-provider-consul├── terraform-provider-digitalocean
├── terraform-provider-OCI
alicloud archive arukas atlas aws azure azurerm bitbucket chef circonus clc cloudflare cloudstack cobbler consul datadog digitalocean dme dns dnsimple docker dyn external fastly github gitlab google grafana heroku http icinga2 ignition influxdb kubernetes librato local logentries mailgun mysql newrelic nomad ns1 oneandone opc openstackopsgenie packet pagerduty postgresql powerdns profitbricks rabbitmq rancher random rundeck scaleway softlayerspotinst statuscake template terraform tls triton ultradns vault vcd vsphere
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
HCL – Basic Terraform .tf Format.
Terraform configuration is written into files named .tf files.
It is based on the HashiCorp Configuration Language (HCL) https://github.com/hashicorp/hcl
JSON is supported for code generation purposes.
Most of the configuration takes the form:
keyword1 "some_name" {key = "value"nested {
key = "value' }
}
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform – Providers.
First thing to do is to use a provider
Providers abstract the APIs from any given third party in order to create infrastructure. Example:
The OCI provider enables Terraform to create, manage and destroy resources in your tenancy on OCI.
Tenancy is the OCID of the tenant. User OCID is the users identifier. Fingerprint is the md5 fingerprint of the private key being used to access the API, and private key path is where the API PEM private key is stored.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform – ResourcesResourcesOnce a provider is configured we can start using that providers resources.
With the OCI provider, we can start creating instances, block and object storage, networks, etc.
The following example starts an instance:
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform – Planning Phase
Terraform Init
Terraform plan
Terraform apply
Terraform plan
destroy
Terraform destroy
For Initial Setup Only
Initialize a working directory- For ex. plugin search/install
• On Windows, in the sub-path terraform.d/plugins beneath your user's "Application Data" directory.• On all other systems, in the sub-path .terraform.d/plugins in your user's home directory
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform – Planning Phase
Once we have put together a configuration to try we can dry-run test this with the planning phase.
"terraform plan" will take the configuration and give a detailed report on which resources will be created, deleted or modified plus identify what dependent resources are effected by these changes.
terraform plan -out=plan1
Saving the plan is useful to ensure that all the steps in the plan were actually applied.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform – Apply
Once the plan looks good we can go and apply the configuration.
$ terraform apply
There is also an option to use saved plans for an apply operation.
$ terraform apply plan1
Plan and apply can also target particular resource(s) using the -target flag.
Plans that are too old will be detected, they are created against a given version of the terraform.tfstate file.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform - Destroy When infrastructure needs to be retired, destroying it and all of its dependencies is straightforward with
$ terraform destroy
Terraform destroy will ask for permission , requiring an explicit “yes” as input.
$ terraform plan -destroyShows what will be destroyed without actually doing it.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform – Resource Graph - Visualization• Terraform builds dependency graphs for
planning state management and more.
• $ terraform graph | dot -Tpng > tgraph1.png
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Terraform Kubernetes InstallerOCI
24
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Open Source Terraform Template For K8S In OCI
https://github.com/oracle/terraform-kubernetes-installer/
• Customizable• Highly Available
Deployment• OCI LB integration
(CCM)• OCI BV integration
(Flex Volume Driver)
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Chef for Oracle Cloud InfrastructureOCI and OCI(Classic)
26
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
What is Chef
https://docs.chef.io/platform_overview.html
Chef is a powerful automation platform that transforms infrastructure into
code. Whether you’re operating in the cloud, on-premises, or in a hybrid
environment, Chef automates how infrastructure is configured,
deployed, and managed across your network, no matter its size.
This diagram shows how you develop, test, and deploy your Chef code.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Chef Plugin for Oracle Public Cloud
• Plugin is available for OCI from Oracle–Documentation Reference at https://docs.us-phoenix-
1.oraclecloud.com/Content/API/SDKDocs/knifeplugin.htm
– https://github.com/oracle/knife-oci/releases
• Plugin is available for OCI Classic from Chef–https://blog.chef.io/2015/10/27/new-chef-integrations-for-
oracle-cloud/
–https://github.com/chef-partners/knife-oraclecloud
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Knife-oci plugin for Oracle Cloud Infrastructure (OCI)
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Knife-oci plugin configuration
https://docs.us-phoenix-1.oraclecloud.com/Content/API/Concepts/sdkconfig.htm
knife[:oci_config_file] = 'D:\oci.txt'knife[:compartment_id] = 'ocid1.compartment.oc1..aaaaaaaaldctk5h5nnvemgl6vudavygi276bq55wh7zbzldaewxqdcfw45fq'user=ocid1.user.oc1..aaaaaaaafnfudi6p2twvkqbonbuvuuzzzvnyly73rhodbln6ax6k5bvccikq
[DEFAULT]user=ocid1.user.oc1..aaaaaaaafnfudi6p2twvkqbonbuvuuzzzvnyly73rhodbln6ax6k5bvccikqfingerprint=98:93:64:1d:b8:47:df:88:c9:57:82:04:83:a4:05:ackey_file=C:\Users\utanna\.oci\oci_api_key.pemtenancy=ocid1.tenancy.oc1..aaaaaaaawz52pbyqeud4ryne7ojegn2bzavhie4bgki5j6k7fwp5asbs5dqaregion=us-phoenix-1
Knife.rb content
Oci.txt content
Complete blog available at -> https://medium.com/oracledevs/using-oracles-chef-plugin-to-provision-resource-in-oracle-cloud-infrastructure-5891100e20ab
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Knife-oci plugin for OCI Classic
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Oracle-knife plugin for OCI Classic
• knife oraclecloud image list
• knife oraclecloud orchestration delete ORCHESTRATION_ID [ORCHESTRATION_ID] (options)
• knife oraclecloud orchestration list
• knife oraclecloud orchestration show ORCHESTRATION_ID (options)
• knife oraclecloud server create (options)
• knife oraclecloud server delete INSTANCE_ID [INSTANCE_ID] (options)
• knife oraclecloud server list
• knife oraclecloud server show INSTANCE_ID (options)
• knife oraclecloud shape list
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Stack ManagerOracle Cloud Infrastructure (Classic)
33
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 34
Cloud Stack Manager for Multi-Service Cloud Environments
REST
CLI
WEB UI* DBCS
JCS
Cloud Stack
• Cloud Stack defined with a template
• Multiple services provisioned as single unit by Stack Manager
• In-built dependency management
• Fault recovery and error handling
• Maps inter-service association
Cloud Stack
Template
Stack Manager
create-stack
import-template
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
What is a Template?
• Defines cloud services that make up a stack
• Template consists of:
– Resources that represent cloud services
– Parameters to customize and reuse
– Dependency to associate services
– Conditions to dynamically change behavior based on parameters, or runtime attributes
– Attributes to pass results to next step in DevOpsprocess
• Imported into a repository with version control for sharing and reuse
35
template:
parameters:
resources:
<conditions>
<depends_on>
attributes:
import template
Repository
PaaS & IaaS
Create stack
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
What is a Template? (Cont’d)
• Stack templates defined using YAML
• Developed using any text editor of your choice – Visual Editor Coming
• Imported into Cloud Stack Manager using UI, REST API or CLI
• Supports any source-code-control, workflows, including Developer Cloud Service, for collaboration, version control and history
36
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Where To Access Stack Manager GUI
Stack Manager GUI can be accessed by clicking the icon on the left hand side of
“Oracle Cloud My Services” on the home page of for ex. JCS or DBCS service
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Stack Manager GUI
Stack Manager GUI before stack creation started. Click Templates to see the templates
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Stack Manager GUI - Templates
Available templates can be browsed here in Templates tab. It(YAML file) can be downloaded as well
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Cloud Stack Templates – Import Templates
• Templates are imported into Cloud Stack Manager using CLI or REST
• Specify template file to upload – validation is automatically performed
40
$ psm stack import-template -f myTemplate.yaml
$ curl -i -X POST -u joe@example.com:Mypassword1! \
-H "Content-Type:multipart/form-data" \
-H "X-ID-TENANT-NAME:MyIdentityDomain” \
-F "template=@myTemplate.yaml" \
https://psm.us.oraclecloud.com/paas/api/v1.1/instance
mgmt/MyIdentityDomain/templates/cst/instances
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
PSMCLI – PaaS Service Manager Command Line Interface Oracle Cloud Infrastructure (Classic)
Confidential – Oracle Internal/Restricted/Highly
41
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
PSMCLI – PaaS Service Manager Command Line Interface
• Oracle PaaS Service Manager provides a command line interface (CLI) with which you can manage the lifecycle of various services in Oracle Public Cloud
• Prerequisites for CLI installation and configuration are:– cURL command-line tool.
– Python 3.3 or later.
• You can download the CLI zip file directly from the UI for your PaaS service– Downloading the CLI from the Oracle Cloud User Interface.
– Download the CLI by using a REST API.
• Install the PaaS CLI as a Python package.
http://docs.oracle.com/en/cloud/paas/java-cloud/pscli/abouit-paas-service-manager-command-line-interface.html
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
PSM - Available services
43
ANALYTICS : Oracle Analytics CloudAPICS : Oracle API Platform Cloud ServiceBDCSCE : Oracle Big Data Cloud Service - Compute EditionBigDataAppliance : Oracle Big Data Cloud ServiceCONTAINER : Oracle Container Cloud ServiceIDCS : Oracle Identity Cloud ServiceIDCSControlPlane : Oracle Identity Cloud ServiceIOTAssetMon : Oracle IoT Asset Monitoring Cloud ServiceIOTConnectedWrker : Oracle IoT Connected Worker IOTEnterpriseApps : Oracle Internet of Things Cloud -EnterpriseIOTFleetMon : Oracle IoT Fleet Monitoring Cloud ServiceIOTProdMonitoring : Oracle IoT Prod Monitoring Cloud IOTSvcAsset : Oracle IoT Asset Monitoring CX Cloud Service
MySQLCS : Oracle MySQL Cloud ServiceOEHCS : Oracle Event Hub Cloud Service - TopicsOEHPCS : Oracle Event Hub Cloud Service -PlatformSOA : Oracle SOA Cloud Serviceaccs : Oracle Application Container Cloud Servicecaching : Oracle Application Cachedbcs : Oracle Database Cloud Serviceggcs : Oracle GoldenGate Cloud Servicejcs : Oracle Java Cloud Servicestack : Oracle Cloud Stack Managerstackvm : Oracle Stack VM
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
PSM Setup
44
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
PSM DBCS Commands
45
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
PSM DBCS Commands - Continue
46
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
PSM DBCS Create-Service
47
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Calling PSMcli in Developer Cloud Service(DevCS)
48
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Invoking PSMcli In DevCS to Provision PaaS service
Local GIT repository
Provisioned Pass Service
Local Workstation
Cloud A/c - DevCSCloud A/c in which provisioning
Project
Repository Build Job
Source Control
Step 1 – Invoke PSMcli
Step 2 – Shell Execute
* DevCS provides built-in build executor to invoke PSMcli. Use this to provision most of PaaS service.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Invoking Stack Manager In DevCS to Provision Cloud Stacks
Local GIT repository
Provisioned Pass Service
Local Workstation
Cloud A/c - DevCS Cloud A/c in which provisioning
Project
Repository Build Job
Source Control
Step 1 – Invoke PSMcli
Step 2 – Shell Execute
Stack Manager uses PSMcli only so this as same as
invoking PSMcli
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Running Execute Shell In DevCS to Run Program In Remote VM
Local GIT repository
Program Installation and Configurations Files
Local Workstation
Cloud A/c - DevCS
VM in which Program is installed
Project
Repository Build Job
Source Control
Step 1 – Shell Execute(Run remote execution)
Provisioned ResourcesCloud A/c
* This can be used for OPCCLI and other external tool as well
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Calling PSMcli in Developer Cloud Service(DevCS)
52
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Calling PSMcli in Developer Cloud Service(DevCS)
53
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
CLICommand Line Interface
54
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
OPCCLI – OCI(Classic) Command Line Interface• CLI commands to provision and manage compute, storage, and network resources
• Download http://www.oracle.com/technetwork/topics/cloud/downloads/index.html#opccli.
• Download installer
• Actual executable name is “opc”
• Create a profile file to store user name, pwd file location, and REST API endpoint
• Store your password in a plain-text file of your choice
• Ensure that the profile file and password file isn't world-readable, by changing the permission to 600
• Store the name of the profile file in the OPC_PROFILE_FILE environment variable.
• Store the name of the folder in the OPC_PROFILE_DIRECTORY environment variable
https://docs.oracle.com/en/cloud/iaas/compute-iaas-cloud/stopc/getting-started-oracle-compute-cloud-service-cli.html
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
OCI – OCI Command Line Interface• CLI commands to provision and manage compute, storage, and network resources
• Setup with either automatic Installer(Installs python and other dependencies) or manual installer
• Actual executable name is “oci”
• Setup CLI using config file providing PEM format API keys details, OCID details of oftenancy and compartment etc. Use this URL for detail instruction.
• Detail help available here URL
• Getting started tutorial available here URL
• oci <service> <type> <action> <options>– compute is the <service>
– instance is the resource <type>
– launch is the <action>, and
– the rest of the command string consists of <options>.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Summary - Oracle Cloud Platform - Automation
57
OCI Classic OCI
REST API Yes Yes
Orchestration Yes
Command Line Utility CLI (Executable name opc) CLI (Executable name oci)
PaaS Service Manager PSM
Stack Manager Yes PSM/UI/REST API
Instance Initialization opc-init Third-party OS provides cloud-init. Check this blog
SDK Java, Python, Ruby, Go etc.
Terraform Available out of the box in terraform Plug-in available from http://www.github.com/oracle
Chef Chef provides knife-oraclecloud plugin Plug-in knife-oci from http://www.github.com/oracle
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Github Resources
58
https://github.com/oracle/terraform-provider-ocihttps://github.com/oracle/terraform-exampleshttps://github.com/oracle/terraform-kubernetes-installerhttps://github.com/oracle/terraform-ceph-installerhttps://github.com/oracle/terraform-oci-cf-installhttps://github.com/oracle/fmw-chef-cookbookhttps://github.com/oracle/knife-ocihttps://github.com/oracle/compute-cloud-service-demos