9/26/2019

1

9/26/2019

2

Ansiblefest 2019

Agenda

What we inherited and what we created

A closer look at my environment

How we achieved the automation

1

2

3

Part 1 – What we inherited and what we created

9/26/2019

3

Ansiblefest 2019

Manual Firewall Change Request Form

Ansiblefest 2019

The Firewall process – the manual way

4) Network team configures firewall

3) Infosec and Change Advisory Board

2) Network team review daily

1) Firewall rule request submittedFree-form text fields

Fixes all the errors in the form; still free-form text

Approvals from both required.

Manual configuration of firewall policy

9/26/2019

4

Ansiblefest 2019

Common defects in the manual process

Not all protocol/ports in requestCommon examples for my use cases include Salt, MS SQL and Splunk.

Mistake in policy implementationNetwork engineers would miss or add extra ports, source and/or destinations. Wrong firewall configured, etc.

Mistakes in manual reviews

Network team have to manually review each request and “clean up” the freeform input into something actionable.

Naming Convention

Engineers making objects manually in the firewall do not always follow the naming convention, causing duplication of objects, incorrectly scoped objects and more.

Ansiblefest 2019

Automated Firewall Change Request Form

9/26/2019

5

Ansiblefest 2019

Quantitative benefits of the automated “next level”

In CY2018

• Old, manual process

• 700 firewall rule requests.

• 1,470 hours of effort spent

• 37,100 hours of wait

In CY2019*

• Automated process

• We are on track for 650 firewall rule requests for the year

• 975 hours of effort will be required

• Only 1514 hours of “wait” time*projection

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Y/Y reduction

# of FW requests Effort hours Wait hours

Year/year productivity gains

Ansiblefest 2019

• Can only request elements that exist in our network

• No mistakes on deployment

• Network team does not expend effort at all

• Standardized firewall object naming convention

• Transparently add all ports to policy when only one port requested

* Wait times might be exaggerated due to the nature our of Change process. (CAB meets M-Th 4PM only).

Qualitative benefits of the automated way

1 8 64

Manual Typical

Automated Typical

Firewall Rule ProcessTime to Deployment (hours)

Effort Wait *

9/26/2019

6

Part 2 – A closer look at my environment

Ansiblefest 2019

System Hierarchy from a networking perspective• Palo security rule

o Inter-tenant

o User-ID to anything

o Internet traffic

• ACI contract

o Intra-tenant

9/26/2019

7

Part 3 –How we achieved the automation

Ansiblefest 2019

The steps to an automated process

Describe your environments

Hierarchy

Naming conventions

Establish a process

Create a UI Code!

9/26/2019

8

Ansiblefest 2019

Design principles

Simplify administration

Don’t change the process, automate what is in place

Rule requests can only be made for existing network elements

Let the network describe itself

My guardrails

Ansiblefest 2019

Ansible role diagram

9/26/2019

9

Ansiblefest 2019

Ansible role diagram

Ansiblefest 2019

ACI_BD_FACTS: (main.yml)- name: Pull list of all BD

aci_bd:host: "{{ ansible_host }}"username: "{{ ansible_user }}"password: "{{ password }}"validate_certs: nostate: query

register: aci_bd_conf

- name: Pull list of all aci subnetsaci_bd_subnet:

host: "{{ ansible_host }}"username: "{{ ansible_user }}"password: "{{ password }}"validate_certs: nostate: query

register: aci_subnet_conf# SECTION END - One time pull configs from ACI

- name: Loop through each fvSubnet retrieved via the aci_bd_subnet call. This includes EPG and BD svi.include: loop.ymlwith_items: "{{ aci_subnet_conf.current }}"

9/26/2019

10

Ansiblefest 2019

ACI_BD_FACTS (loop.yml) *truncated- name: If this is an BD SVI "{{ item.fvSubnet.attributes.dn }}"set_fact:significant_name_regex: "^.*\\/BD-BD-“

when: item.fvSubnet.attributes.dn is search('/BD-')

- name: Parse out the tenant nameset_fact:tenant_name: "{{ item.fvSubnet.attributes.dn | regex_replace('uni/tn-', '') | regex_replace(tenant_name_regex, '') }}"

- name: Parse out the subnetset_fact:ip_subnet: "{{ item.fvSubnet.attributes.ip | ipaddr('network/prefix') }}“

- name: Add this BD to output file {{ tenant_name }} / {{ vrf_field }} / {{ significant_name }} / {{ ip_subnet }}"lineinfile:path: "{{ aci_bd_facts_file }}"line: "{{ lookup('template', '{{ aci_bd_facts_line_template }}') }}"insertafter: EOF

when: tenant_name not in excluded_tenants

aci_bd_facts_line_template (j2 template)#jinja2:lstrip_blocks: True

- bd_name: '{{ significant_name }}'tenant: '{{ tenant_name }}'vrf: '{{ vrf_field }}'subnet: '{{ ip_subnet }}'

Ansiblefest 2019

output of aci_bd role---aci_subnet:

- bd_name: 'APP'tenant: 'LAB-DEV'vrf: 'LABPROD'subnet: '192.168.0.128/26'

- bd_name: 'WEB'tenant: 'LAB-DEV'vrf: 'LABPROD'subnet: '192.168.0.64/26'

- bd_name: 'DB'tenant: 'LAB-DEV'vrf: 'LABPROD'subnet: '192.168.0.0/26'

- bd_name: 'DB'tenant: 'LAB-PROD'vrf: 'LABPROD'subnet: '10.16.0.0/26'

- bd_name: 'WEB'tenant: 'LAB-PROD'vrf: 'LABPROD'subnet: '10.16.0.64/26'

- bd_name: 'APP'tenant: 'LAB-PROD'vrf: 'LABPROD'subnet: '10.16.0.128/26'

9/26/2019

11

Ansiblefest 2019

Ansible role diagram

Ansiblefest 2019

Ansible role diagram

9/26/2019

12

Ansiblefest 2019

Palo_epg_import role (main.yml)

- name: Run each BD through loop.ymlinclude: loop.ymlloop: "{{ aci_subnet }}"loop_control:

loop_var: outer_bd

requirements.yml- src: paloaltonetworks.paloaltonetworks

Ansiblefest 2019

Palo_epg_import role (loop.yml) *truncated

- name: Create address object for "{{ address_object_name }}" if it does not existpanos_address_object:

ip_address: "{{ ip_address }}"api_key: "{{ api_string }}"device_group: "{{ palo_device_group }}"state: presentname: "{{ address_object_name }}"value: "{{ outer_bd.subnet }}"address_type: "{{ addrformat }}"description: "Imported from ACI to PANOS via Ansible"tag: ['{{ tenant_name_var }}', '{{ datacenter }}', '{{ global_group_object_name }}', '{{ local_group_object_name

}}']commit: false

when: not(address_object_name in address_object_already_created)

9/26/2019

13

Ansiblefest 2019

Palo_epg_import role (loop.yml) continued

- name: create group "{{ global_group_object_name }}" if it does not existpanos_address_group:

ip_address: "{{ ip_address }}"api_key: "{{ api_string }}"device_group: "{{ palo_device_group }}"name: "{{ global_group_object_name }}"dynamic_value: "{{ global_group_object_name }}"tag: 'cherwell_do_export'state: presentcommit: False

when: not(global_group_object_name in addr_groups_already_created)

- name: Add "{{ global_group_object_name }}" to list addr_groups_already_created if it's not there alreadyset_fact:

addr_groups_already_created: "{{ addr_groups_already_created }} + [ '{{ global_group_object_name }}' ]"when: not(global_group_object_name in addr_groups_already_created)

Ansiblefest 2019

Panorama Config after import (Address Groups)

9/26/2019

14

Ansiblefest 2019

Panorama Config after import (Address Objects)

Ansiblefest 2019

Ansible role diagram

9/26/2019

15

Ansiblefest 2019

Ansible role diagram

Ansiblefest 2019

Palo_address_object_export_role (main.yml)- name: Query for address object list from PANOS

get_url:url: "{{ panos_rest_url_address_object_config }}"

dest: "{{ address_name_xml_file }}"

- name: Query for address group list from PANOSget_url:

url: "{{ panos_rest_url_address_group_config }}"dest: "{{ address_group_xml_file }}"

- name: Parse PANOS address group object namexml:

path: "{{ address_group_xml_file }}"xpath: //tag[member="cherwell_do_export"]/parent::entrycontent: attributeattribute: name

register: address_group_name_output

- name: Run through playbooks for parsing for each address group and adding to CSV upload file.include: address_group_loop.ymlwith_sequence: start=0 end={{ address_group_name_output.count - 1 }}

9/26/2019

16

Ansiblefest 2019

Palo_address_object_export_role (loop.yml)

- name: Parse address group name from XML query pulled in main.ymlset_fact:

address_group_name_var: "{{ address_group_name_output.matches[ item|int ].entry.name }}“

- name: playbook for handling with normal objects, like EPG, offices, Internet objectsinclude: normal_address_groups_outer_loop.yml

Ansiblefest 2019

Palo_address_object_export_role (outer_loop.yml)- name: get the dynamic filter name for "{{ address_group_name_var }}"

xml:path: "{{ address_group_xml_file }}"xpath: /response/result/address-group/entry[@name='{{ address_group_name_var }}']/dynamic/filtercontent: text

register: address_group_dynamic_filter_var

- name: Find the address object names tagged with "{{ address_group_dynamic_filter_var }}"xml:path: "{{ address_name_xml_file }}"xpath: //tag[member='{{ address_group_dynamic_filter_var }}']/parent::entrycontent: attributeattribute: name

register: address_objectsignore_errors: yes

- name: Loop through each address object associated with tag "{{ address_group_dynamic_filter_var }}" and to find IP addresses that are children of group "{{ address_group_name_var }}"

include: normal_address_groups_inner_loop.ymlwith_sequence: start=0 end={{ address_objects.count - 1 }}loop_control:loop_var: address_object_item

ignore_errors: yes# Ignore errors to handle address-groups that have no members.

9/26/2019

17

Ansiblefest 2019

Palo_address_object_export_role (inner_loop.yml)

- name: Initialize vars for normal address groups inner loop {{ address_objects.matches[ address_object_item|int ].entry.name }}set_fact:address_object_name_inner_loop_var: "{{ address_objects.matches[ address_object_item|int ].entry.name }}"address_object_type: "ip-netmask"ip_value: ''

- name: Get IP.subnet associated with "{{ address_object_name_inner_loop_var }}" if it's a range or ip-netmastobject.xml:path: "{{ address_name_xml_file }}"xpath: /response/result/address/entry[@name='{{ address_object_name_inner_loop_var }}']/{{

address_object_type }}content: text

register: ip_outputignore_errors: yes

# The real path is ip_output.matches[0].ip-netmask. The dash is not an addressable var in Ansible and escaping doesnt work.- name: Parse IP values from the xml query above if it's an ip-netmask or ip-range valueset_fact:ip_value: "{{ ip_output.matches[0] | regex_replace('.u.ip.*:.u.','') }}"

when: not ip_output.failed

Ansiblefest 2019

Palo_address_object_export_role (inner_loop.yml)

- name: Add this object to CSV filelineinfile:path: "{{ address_group_csv_file }}"line: "{{ lookup('template', 'csv_template_address_groups.j2') }}"insertafter: EOF

And the output file looks like:

LAB-DEV,LAB-DEV,ALL

LAB-PROD,LAB-PROD,ALL

LAB-DEV,APP,192.168.0.128/26

LAB-DEV,WEB,192.168.0.64/26

LAB-DEV,DB,192.168.0.0/26

LAB-PROD,DB,10.16.0.0/26

LAB-PROD,WEB,10.16.0.64/26

LAB-PROD,APP,10.16.0.128/26

9/26/2019

18

Ansiblefest 2019

Ansible role diagram

Ansiblefest 2019

Ansible role diagram

9/26/2019

19

Ansiblefest 2019

Example Firewall Change Request

Ansiblefest 2019

Firewall_Security_rules_master (playbook)

- hosts: localhostgather_facts: noroles:- paloaltonetworks.paloaltonetworks

- hosts: localhostgather_facts: noroles:- ansible-role_firewall_security_rules_master

vars:run_mode: "retrieve"

- hosts: "{{ selected_panos_hosts }}"connection: localgather_facts: noroles:- ansible-role_palo_security_rules

- hosts: "{{ selected_aci_hosts }}"connection: localgather_facts: noroles:- ansible-role_aci_contracts

- hosts: localhostgather_facts: yesroles:- ansible-role_firewall_security_rules_master

vars:run_mode: "close"

9/26/2019

20

Ansiblefest 2019

Firewall_Security_rules_master (retrieve/truncated*)- name: Retrieve the individual CR with cherwell record id "{{ item }}"

uri:url: "{{ cherwell_url_a }}{{ cherwell_url_get_specific_cr_a }}{{ change_request_business_object_id }}{{

cherwell_url_get_specific_cr_b }}{{ item }}"method: GETheaders:

authorization: "bearer {{ cherwell_api_token }}"body_format: json

register: fw_cr_data_results

- name: Parse out the Dest_Address_1 field.set_fact:

dest_addr_1: "{{ fw_cr_data_results | json_query(\"json.fields[?name=='FW2DestAddress1'].value\") | first }}"

- name: Parse out the Dest_Address_2 field.set_fact:

dest_addr_2: "{{ fw_cr_data_results | json_query(\"json.fields[?name=='FW2DestAddress2'].value\") | first }}“

Ansiblefest 2019

Output of Change Request retrieval- cr_num: '40003'

cr_tag_name: 'CR40003'cherwell_business_object_record_id: '9'desc: "It does web type stuff"app_ci_name: 'Web-app'src_tenant:- 'LAB-DEV'- 'LAB-DEV'- 'LAB-DEV'- 'LAB-DEV'- 'LAB-DEV'- 'LAB-DEV'src_addr:- 'LAB-DEV_WEB'- 'LAB-DEV_WEB'- 'LAB-DEV_WEB'- 'LAB-DEV_WEB'- 'LAB-DEV_WEB'- 'LAB-DEV_WEB' dst_tenant:- 'LAB-DEV'- 'LAB-PROD'- 'LAB-DEV'- 'LAB-DEV'- 'LAB-DEV'- 'LAB-DEV'

dst_addr:- 'LAB-DEV_DB'- 'LAB-PROD_APP'- 'LAB-DEV_DB'- 'LAB-DEV_DB'- 'LAB-DEV_DB'- 'LAB-DEV_DB‘apps:- 'TCP_00080_HTTP'- ''- ''- ''- ''- ''- ''- ''- ''- ''service_protocol:- 'TCP'- 'TCP'- ''- ''- ''- ''- ''- ''- ''- ''

start_port:- '80'- '22' - '' - '' - '' - '' - '' - '‘- ''- '' end_port:- ''- ''- ''- ''- ''- ''- ''- ''- ''- ''

palo_service_name:- 'application-default'- 'TCP_00022'- ''- ''- ''- ''- ''- ''- ''- ''aci_filter_name:- 'TCP-00080'- 'TCP-00022'- ''- ''- ''- ''- ''- ''- ''- ''

9/26/2019

21

Ansiblefest 2019

Ansible role diagram

Ansiblefest 2019

Ansible role diagram

9/26/2019

22

Ansiblefest 2019

Role-palo_security_rules (create the service)- name: Create services if applicable.include: create_service.ymlwith_together:- "{{ cr_line.palo_service_name }}"- "{{ cr_line.service_protocol }}"- "{{ cr_line.start_port }}"- "{{ cr_line.end_port }}“

------- name: Create port range var if applicable. palo_security_rules/create_service.ymlset_fact:palo_service_port: "{{ '-'.join(('{{ item.2 }}', '{{ item.3 }}')) }}"

when: not(item.3 == '' and item.0 == '')

- name: Create port norange var if applicable. palo_security_rules/create_service.ymlset_fact:palo_service_port: "{{ item.2 }}"

when: item.3 == '' and not(item.0 == '')

- name: Create service "{{ item.0 }}"panos_service_object:ip_address: "{{ ip_address }}"api_key: "{{ api_string }}"device_group: "{{ palo_device_group_for_objects }}"state: 'present'name: "{{ item.0 }}"protocol: "{{ item.1 | lower() }}"destination_port: "{{ palo_service_port }}"commit: false

ignore_errors: yes# Ignore errors cause this service might already exist; faster then checking before trying to add

Ansiblefest 2019

Role-palo_security_rules (set vars for WAN policy)- name: Set vars to intertenant user id rules

set_fact:device_group_to_conf: "{{ palo_device_group_wan }}"source_address_list_to_conf: "any"source_user_list_to_conf: "{{ intertenant_source_user_list }}"dest_address_list_to_conf: "{{ intertenant_dest_address_list }}"rule_name_standard_app: "asau{{ cr_line.cr_num }}"rule_name_standard_service: "assu{{ cr_line.cr_num }}"rule_name_nonstandard_app_service: "ansu{{ cr_line.cr_num }}"security_profile_to_conf: "{{ security_profile_wan }}"

- name: include panos_security_rule_create when intertenant source/dest pairs existinclude: panos_security_rule_create.ymlwhen: intertenant_source_user_list | length > 0 and intertenant_dest_address_list | length > 0

9/26/2019

23

Ansiblefest 2019

Role-palo_security_rules- name: Create Palo APP-ID standard rule for "{{ cr_line.cr_tag_name }}".

panos_security_rule:ip_address: "{{ ip_address }}"api_key: "{{ api_string }}"devicegroup: "{{ device_group_to_conf }}"state: presentrule_name: "{{ rule_name_standard_app }}"description: "{{ cr_line.desc }}"source_ip: "{{ source_address_list_to_conf }}"source_user: "{{ source_user_list_to_conf }}"destination_ip: "{{ dest_address_list_to_conf }}"application: "{{ applications_appid_list }}"tag_name: ["{{ cr_line.cr_tag_name }}"]rulebase: "{{ palo_rulebase }}"log_setting: "{{ palo_logging_profile }}"group_profile: "{{ security_profile_to_conf }}"action: 'allow'

when: ((applications_appid_list | length) > 0)

Ansiblefest 2019

Palo Screenshot of policy

9/26/2019

24

Ansiblefest 2019

Ansible role diagram

Ansiblefest 2019

Ansible role diagram

9/26/2019

25

Ansiblefest 2019

Role-aci_contracts – create ACI filters- name: include aci_filter_create.yml

include: aci_filter_create.ymlwith_together:

- "{{ cr_line.aci_filter_name }}"- "{{ cr_line.service_protocol }}"- "{{ cr_line.start_port }}"- "{{ cr_line.end_port }}“

- name: create filter "{{ item.0 }}"aci_filter:

host: "{{ ansible_host }}"username: "{{ ansible_user }}"password: "{{ password }}"state: presentfilter: "{{ item.0 }}"tenant: "{{ aci_filter_tenant_name }}"description: 'Created by ansible'

- name: Add filter entry "{{ item.0 }}" to filter "{{ item.0 }}"aci_filter_entry:

host: "{{ ansible_host }}"username: "{{ ansible_user }}"password: "{{ password }}"state: presenttenant: "{{ aci_filter_tenant_name }}"filter: "{{ item.0 }}"entry: "{{ item.0 }}"ether_type: ipip_protocol: "{{ item.1|lower() }}"dst_port_start: "{{ item.2 }}"dst_port_end: "{{ end_port_for_filter }}"

Ansiblefest 2019

Role-aci_contracts – create ACI contract- name: Create contract "{{ contract_name }}"

aci_contract:host: "{{ ansible_host }}"state: presenttenant: "{{ source_item.split('_')[0] | regex_replace('(common).*','\\1') }}"contract: "{{ contract_name }}"description: "{{ cr_line.cr_tag_name}} {{ cr_line.desc | truncate(116) }}"scope: context

- name: Create contract subj for "{{ contract_name }}"aci_contract_subject:host: "{{ ansible_host }}"state: presenttenant: "{{ source_item.split('_')[0] | regex_replace('(common).*','\\1') }}"contract: "{{ contract_name }}"description: "created by ansible"subject: "Contract-Sub"

- name: loop through filters to attach on contract "{{ contract_name }}"aci_contract_subject_to_filter:host: "{{ ansible_host }}"state: presenttenant: "{{ source_item.split('_')[0] | regex_replace('(common).*','\\1') }}"contract: "{{ contract_name }}"subject: "Contract-Sub"filter: "{{ item }}"

with_items:- "{{ cr_line.aci_filter_name }}"

9/26/2019

26

Ansiblefest 2019

Role-aci_contracts – Attach contract to EPG- name: Add consumed contract "{{ contract_name }}" to source EPG "{{ source_epg }}"

aci_epg_to_contract:host: "{{ ansible_host }}"state: presenttenant: "{{ source_item.split('_')[0] | regex_replace('(common).*','\\1') }}"ap: "{{ source_ap_aci_string }}"epg: "{{ source_epg_aci_string }}"contract: "{{ contract_name }}"contract_type: consumer

- name: Add provided contract "{{ contract_name }}" to dest EPG "{{ return_val_epg }}"aci_epg_to_contract:

host: "{{ ansible_host }}"state: presenttenant: "{{ destination_item.split('_')[0] | regex_replace('(common).*','\\1') }}"ap: "{{ return_val_ap }}"epg: "{{ return_val_epg }}"contract: "{{ contract_name }}"contract_type: provider

Ansiblefest 2019

ACI contract screenshots

9/26/2019

27

Ansiblefest 2019

Ansible role diagram

Ansiblefest 2019

Ansible role diagram

9/26/2019

28

Ansiblefest 2019

Role-firewall_master – close mode- name: Close change requestsuri:

url: "{{ cherwell_url_a }}{{ cherwell_save_bus_ob_url }}"method: POSTheaders:authorization: "bearer {{ cherwell_api_token }}"

body_format: jsonbody: '{"busObId": "{{ change_request_business_object_id }}","busObRecId": "{{ item.cherwell_business_object_record_id }}","fields": [

{"dirty": true,"fieldId": "{{ change_request_fieldid_disposition }}","value": "Successful"

},{

………………………………………………..}]

}'with_items:

- "{{ cr }}"

- name: Create list of all CR closedset_fact:

list_of_cr_closed: "{{ list_of_cr_closed }} + [ '{{ item.cr_tag_name }}' ]"with_items:

- "{{ cr }}"

- name: Courtesy message, all CR completed and closed by this job rundebug:

msg: "{{ list_of_cr_closed }}"

Ansiblefest 2019

Tower log screenshot

9/26/2019

29

Ansiblefest 2019

Tower scheduling and UI

Ansiblefest 2019

Tower Surveys

9/26/2019

30

Ansiblefest 2019

• Collaborate / cast a wide net

• Overcommunicate

• Keep a paper trail

• Add code to optimize performance

• Document failure/troubleshooting scenarios

• TENACITY!

Lessons Learned

Ansiblefest 2019

• Service Now to replace Cherwell

• Role/process for Palo NAT policy

• IPAM

o Dynamically pull/document IP for VIPs, NAT, etc.

• F5 administration processes refined and automated

• Firewall process

o Continual code optimization and minor feature enhancements

Anticipated highlights for 2020

60September 26, 2019 | Proprietary and Confidential

9/26/2019

31

Ansiblefest 2019 61September 26, 2019 | Proprietary and Confidential

About Us

We are the business behind business.®

We are a business, legal, and financial services company the provides knowledge-based solutions to clients worldwide.

We provide solutions for every phase of the business life cycle, helping to form entities, maintain compliance, execute secured transaction work, and support real estate, M&A, and other corporate transactions. We help effectively manage, promote, and secure our clients’ valuable brand assets against the threats of the online world. We offer a single tax and risk management platform that helps clients better manage risk, achieve greater automation and date transparency, and stay in compliance.

Ansiblefest 2019 62September 26, 2019 | Proprietary and Confidential62September 26, 2019 | Proprietary and Confidential

9/26/2019

32

Ansiblefest 2019 63September 26, 2019 | Proprietary and Confidential

Ansiblefest 2019

Thank you.