@rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account...
Transcript of @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account...
![Page 1: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/1.jpg)
Alex StephenSoftware Engineer, Google
@rambleraptor
Google's Journey to Collections
![Page 2: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/2.jpg)
+
![Page 3: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/3.jpg)
Collections are a distribution format for Ansible content. They
can be used to package and distribute playbooks, roles,
modules, and plugins.
![Page 4: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/4.jpg)
Current World
![Page 5: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/5.jpg)
Collections World
Source: 2019 GitHub State of the Octoverse
![Page 6: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/6.jpg)
Machine Learning
ML Engine ModelML Engine VersionTPU Node
Https Health Check
Health Check
Instance Template
Image
Instance
Instance Group
Instance Group Manager
Interconnect Attachment
Network
Region Disk
Route
Router
Snapshot
Ssl Certificate
Ssl Policy
Subnetwork
Target Http Proxy
KubernetesKubernetes Engine ClusterKubernetes Engine Node Pool
Big DataBigQuery TableBigQuery Dataset
StorageCloud Storage BucketCloud Storage ObjectCloud Storage Bucket Access ControlCloud Source Repositories
ComputeAddress
Autoscaler
Backend Bucket Backend Service
Disk
Firewall
Forwarding Rule
Global Address
Global Forwarding Rule
Http Health Check
Target Https Proxy
Target Pool
Target Ssl Proxy
Target Tcp Proxy
Target Vpn Gateway
UrlMap
Vpn Tunnel
Databases
SQL InstanceSQL UserSQL DatabaseSpanner InstanceSpanner DatabaseMemorystore InstanceFilestore Instance
Identity & Security
KMS Key RingKMS Crypto KeyResource Manager ProjectIAM RoleIAM Service AccountIAM Service Account Key
Serverless
App Engine Firewall RuleCloud FunctionsCloud Scheduler JobsCloud Task Queues
DNS
Managed ZonesResource Record Sets
Pub/SubTopicsSubscriptions
![Page 7: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/7.jpg)
Modules
Utils
Tests Documentation
Plugins
Issues PRs
Deprecated/Renamed Modules
![Page 8: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/8.jpg)
Ansible 2.7
10/2017
![Page 9: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/9.jpg)
Ansible 2.8
5/2019
![Page 10: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/10.jpg)
Collections World
![Page 11: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/11.jpg)
Modules
Utils
Tests Documentation
Plugins
Issues PRs
Deprecated/Renamed Modules
![Page 12: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/12.jpg)
01The Users
![Page 13: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/13.jpg)
User Migration Story
![Page 14: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/14.jpg)
02The Code
![Page 15: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/15.jpg)
Repo● As an Ansible partner, we've
got a repo in the ansible org
● Basic folder structure + a galaxy.yml metadata file
![Page 16: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/16.jpg)
Repo● As an Ansible partner, we've
got a repo in the ansible org
● Basic folder structure + a galaxy.yml metadata file
![Page 17: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/17.jpg)
Modules/Plugins● Now live in collection
● Copy and paste the modules into their new home
![Page 18: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/18.jpg)
Module Utils● module_utils now live inside of the
collection● You have to change imports on every
module/plugin● Very well-documented change● Challenge: very easy to forget
![Page 19: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/19.jpg)
Documentation● It's still part of the module
● Collection has a README now
● Modules still have YAML docs
● Challenge: documentation home is coming soon
![Page 20: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/20.jpg)
Integration Tests● Ansible-test now works with
collections
● Could be a cleaner experience
● Challenge: how do you know what modules it's running?
![Page 21: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/21.jpg)
Migration Process Automation● ansible-galaxy collection init
● Ansible built out ansible/content_collector
● The migration process isn't automated at the moment.
● Challenge: May be challenging to those without knowledge of full ecosystem.
![Page 22: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/22.jpg)
03The Process
![Page 23: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/23.jpg)
Issues● ansibot tries to label
issues
● My @-mentions are plentiful
● Challenge: How to move these to collection repo?
![Page 24: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/24.jpg)
Issues● Every collection
can have its own set of issues now
● Challenge: What does the community want?
![Page 25: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/25.jpg)
Making a PR (now)● Write the PR (preferably using Magic Modules)
● Make a PR against ansible/ansible
● ansibot hopefully labels + mentions the proper reviewers
● Tests run in Shippable
● Wait for reviewer (core or otherwise)
● shipit (or go back to beginning)
● ansibot ships the PR
![Page 26: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/26.jpg)
Making a PR (collections)● Write the PR (preferably using Magic Modules)
● Make a PR against ansible/ansible_collections_google
● Tests run
● Get approved and merged.
![Page 27: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/27.jpg)
Code Review● GitHub > ansibot
● My GitHub mentions are saved
● My favorite part of the migration
![Page 28: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/28.jpg)
Code Review● Quicker turn-around
● Challenge: how do we keep our standards in line with the community?
![Page 29: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/29.jpg)
Sanity Tests (Shippable CI)● Ansible-test now works with collections
● Exact same set of tests as ansible/ansible
● Challenge: some bugs. Can't use for CI just yet.
● Challenge: should these tests remain the same for collections?
![Page 30: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/30.jpg)
Galaxy● Home for roles
● Now also a home for collections
● Challenge: When to upload to Galaxy?
● Challenge: Discoverability galaxy.ansible.com/google/cloud
![Page 31: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/31.jpg)
Galaxy - Upload Process● Two CLI commands
(build + publish)
● Super easy.
● Super automatable.
![Page 32: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/32.jpg)
Deprecation● Unclear how current deprecation
schedule moves over
![Page 33: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/33.jpg)
Automation Hub
![Page 34: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/34.jpg)
04The Magic
![Page 35: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/35.jpg)
Magic Modulesgithub.com/googlecloudplatform/magic-modules
![Page 36: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/36.jpg)
05The Future
![Page 37: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/37.jpg)
What can we do now?● More frequent releases
● More granular changelogs
● Pick your releases
![Page 38: @rambleraptor to Collections Software Engineer, … ATL Slide Decks...IAM Role IAM Service Account IAM Service Account Key Serverless App Engine Firewall Rule Cloud Functions Cloud](https://reader033.fdocuments.us/reader033/viewer/2022043006/5f90a43f02884573383742bc/html5/thumbnails/38.jpg)
Thank You!
cloud.google.comgalaxy.ansible.com/google/cloud
@rambleraptor