Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

45
1 Positioning Agile and Continuous Delivery for Auditors and Examiners

description

Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

Transcript of Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

Page 1: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

1

Positioning Agile and

Continuous Delivery for

Auditors and Examiners

Page 2: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

2

Where to Start

The single most important step in preparing for an audit or

examination is to put yourself in the auditors shoes and

understand their goals:

• Does this entity have a sound development practice?

• Do they have repeatable processes that ensure

consistent results?

• Do they have the appropriate controls in place?

• Does the management team understand the risk they

are exposed to?

Page 3: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

3

Taking a Step Back…Let’s Start with the Bible

During an examination, the examiner explained that he

wanted to see our “Bible”, aka our SDLC. He wanted every

step to be documented and auditable so he could be sure

that every project followed the exact process, every time.

Credit: http://www.stpatselkhorn.org/AdultFormation/BibleStudy.aspx

Page 4: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

4

How We Responded

1. The Mammoth Waterfall SDLC

2. The Mammoth SDLC & SDLC Lite

3. Agile SDLC

4. Agile & Continuous Delivery

Page 5: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

5

Enough about us…

We have turned the corner and are now reaping the

rewards of properly implementing Agile and Continuous

Delivery.

We now find that WE HAVE TIME to automate and

strengthen our processes.

Let’s get to the 25 things you can do to better prepare for

your next audit or exam!

Page 6: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

6

Tips and Tricks for Audits and Exams

1 - 6 : Agile Education

7 - 12 : Continuous Delivery Education

13 - 18 : Demonstrating Maturity

19 - 21 : Orchestrate for Improved Quality

22 - 24 : Source Code Control is KEY

25 : Getting Ahead

Page 7: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

7

Agile Education

Credit: http://flickfacts.com/movie/4925/back-to-school

Page 8: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

8

#1 – Socialize Your Plans

Don’t surprise your auditor with a major change to your

process.

Provide Useful Information:

• Agile Overview:

https://www.youtube.com/watch?v=502ILHjX9EE

• Continuous Delivery Overview:

Continuous Delivery: Reliable Software Releases Through Build, Test

and Deployment Automation by Jez Humble and David Farley

• Continuous Delivery Adoption:

http://www.thoughtworks.com/insights/blog/case-continuous-delivery

http://www.perforce.com/continuous-delivery-report

Page 9: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

9

#2 – Don’t Risk the Crown Jewels

If possible, demonstrate the new technologies and

procedures on a lower risk application.

You will thank me later….because there will be bumps

If you do start with a major application, find a way to

segment the implementation to minimize the up front risk

Page 10: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

10

#3 – Demonstrate Your Expertise

While many of these technologies and procedures are not

new, they may be new to you or your organization. Make

sure you can demonstrate your expertise:

Certifications - Scrum Alliance, etc.

Training Programs – Learning Tree, Scrum Alliance, etc.

Meetups & User Groups – Continuous Delivery, Agile, etc.

Social Media – LinkedIn Continuous Delivery Group, etc.

Page 11: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

11

#4 - Map Agile SDLC to Waterfall SDLC

Design Waterfall Agile

Design The entire application is designed at one

time

The design evolves as the application is

developed

The design is created by technical resources

working from the requirements

The design is created by the developers

working with the key stakeholders

The design is based on the best estimate of

how the application is used

The design is based on customer behavior

Design Review The design is reviewed by technical

resources to ensure completeness and

accuracy

The design is shown as a working solution to

the Product Owner and other stakeholders

Changes to the design may have a major

ripple effect to the rest of the application

The design is continually revisited and

adjusts to customer need

Design Sign Off Specific step where designated parties agree

that the design is complete and accurate

Implicit to the process when everyone

agrees that the work is acceptable to go to

production (Sprint Review)

Page 12: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

12

#5 – Explain Benefits of Shorter Cycle Time

When a vulnerability is found, how quickly

can you address it?

When a new OS patch is released, how long

until it is on all of your servers?

Page 13: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

13

#6 – Explain How Small Batches Reduces Risk

• Schedule risk

– Feature creep

– Gold plating

• Quality risk

– New bugs

– Instability

• Business risk

– Wrong functionality

– Missed opportunity

Page 14: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

14

Continuous Delivery Education

Page 15: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

15

#7 – A More Auditable Process

The key takeaway….

An automated process is far more auditable!

Page 16: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

16

#8 – Correct Version of the Application

Everyone needs environments and now there are great tools

that make it even easier to enable environment sprawl.

Every developer has a local environment

3 Development environments

4 QA environments

4 Staging environments

4 Production environments

Page 17: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

17

#9 – Infrastructure as Code

1. Baseline Image

– The latest patched base server OS, ssh, etc

2. Apply common applications (that require configuration)

– TripWire, Splunk, PostFix, etc

3. Application critical applications

– Java, App server, etc

4. Deploy your software

** Even with configuration management, you still need a tool like TripWire

Page 18: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

18

Infrastructure as Code – Benefits

• Environments stay in sync

– Changes are made in development and migrated

– Administrators should not make changes directly to environments

– Changes made manually to an environment are undone with the

next migration through the pipeline

• Environments can be built on demand

– Becomes faster to rebuild an environment than to troubleshoot

– A process to build an environment that took weeks can now be

completed in under an hour

– Environments will no longer be a bottleneck to new functionality

• Environments are documented and version controlled

– Each setting change is a line of code that can be read

– All configurations reside in GIT so that the team can recover or

revert to a prior configuration

Page 19: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

19

#10 – Static Code Analysis

Page 20: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

20

Sonar – Security Tests

Page 21: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

21

Sonar – Test Changelog

Page 22: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

22

Sonar – Additional Tracking

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

Number of Issues

Issues

Issues - Blocker

Issues - Critical

Issues - Major

Issues - Minor

Issues - Info

Page 23: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

23

#11 – Automated Testing

Automated tests are the answer to MANY questions about

reducing risk….but they open the door to a whole new

world of questions

• Who validated that the automated test worked

correctly?

• How do you know that the test meets the desired

result?

• How can you be sure you have sufficient coverage?

• Where are the tests for specific user stories?

Page 24: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

24

User Acceptance Test

Page 25: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

25

#12 – Repository Management

Single source for software, binaries & libraries

demonstrates:

• Consistency across environments

• Single, auditable repository of external resources

• Control access to external sites

Page 26: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

26

Demonstrating Maturity

Credit: http://ihkstories.com/maturity-is-not-when-we-start-speaking-big-thingsit-is-when-we-start-understanding-small-things/

Page 27: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

27

#13 – Go Digital

Online Agile Boards

An Auditor once pulled a sticky off our physical board that

was in the Ready for Test queue. He asked “if I don’t put

this back, how do you know this was tested?”

Page 28: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

28

#14 – Automating Sign-Offs

Credit: http://www.polscheit.de/plugins/jira/group-sign-off/images/GroupSignOff-Banner.png

Page 29: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

29

#15 – Automating Documentation

Credit: http://jiraxporter.xpand-it.com/download/attachments/327684/Banner.png?version=1&modificationDate=1364461203281&api=v2

Page 30: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

30

Bank Assetpoint Agile Implementation

Retrievedfrom Jira

Retrievedfrom Jira

Page 31: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

31

#16 – Logging Pipeline Activity

Page 32: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

32

#17 – Capturing Meaningful Metrics

0

10

20

30

40

50

60

70

80

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Positive Sprint Quality Trend

0

2

4

6

8

10

12

14

16

18

1 2 3 4 5 6 7 8 9 10

Sprint 2014-1

Done QA In Progress Backlog

Page 33: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

33

#18 – Add one more meeting

Sprint Planning Review Meeting

• Additional demonstration of oversight

• Shows that we are willing to adapt to meet company

goals

• Great catch-all for interested stakeholders

Page 34: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

34

Orchestrate for Improved Quality

Credit: http://accupackmidwest.com/quality-control/

Page 35: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

35

#19 – Keep QA Firmly in the Process

• When new code comes into Test Environment

• When new code can be moved to a higher environment

• Perform the deployment to the Staging Environment

• Perform the deployment to Production Environment

Page 36: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

36

#20 – Don’t Forget Operations

The System Engineering

Team to controls when

code can enter the

Staging Environment

Application Engineering

Team controls when

code can enter the

Production Environment

Page 37: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

37

#21 – When All Else Fails – Email!

Email notifications keep parties informed

Security

Compliance

Management

Operations

Product Owner

Page 38: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

38

Source Code Control is KEY

Page 39: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

39

#22 – Demonstrate Permissions

Making sure that the appropriate controls are in place in

GIT are critical. You will need to use a management tool on

top of GIT like Stash.

Page 40: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

40

#23 –Code Reviews with Pull Requests

Page 41: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

41

#24 – Secure Your Pull Requests

Custom GIT Hook

Page 42: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

42

Administrator approved pull request alert

Page 43: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

43

Getting Ahead

Credit: https://dzihxiql01vk4.cloudfront.net/wp-content/uploads/2013/06/Get-Ahead-with-Repricing.jpg

Page 44: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

44

#25 - Be Aware of Outstanding Audit Risks

• Get Ahead of Permission Questions

– Jenkins, Puppet, Nexus, Stash, etc.

• Continuous Improvement means that you are not

following the same process over and over

– Allowing Agile Teams to change their development process to

make themselves more efficient is scary to auditors

• Management (e.g. upgrades) of Pipeline software

• Separation of duties

• Management aware (and approving) work

• Continuous Deployment may be a step too far

– There is a lot of value in ensuring that humans are involved in

the process

Page 45: Agile and Continuous Delivery for Audits and Exams - DC Continuous Delivery Meetup

45

Questions