Acceptance Testing for Continuous Farley Acceptance Testing... Acceptance Testing for Continuous...

download Acceptance Testing for Continuous Farley Acceptance Testing... Acceptance Testing for Continuous Delivery

If you can't read please download the document

  • date post

    22-Jun-2020
  • Category

    Documents

  • view

    11
  • download

    1

Embed Size (px)

Transcript of Acceptance Testing for Continuous Farley Acceptance Testing... Acceptance Testing for Continuous...

  • Dave Farley http://www.davefarley.net @davefarley77

    http://www.continuous-delivery.co.uk

    Acceptance Testing for Continuous Delivery

    http://www.continuous-delivery.co.uk

    http://www.davefarley.net

  • The Role of Acceptance Testing

    Local Dev. Env.

    Source Repository

  • The Role of Acceptance Testing

    Artifact Repository

    Local Dev. Env.

    Deployment Pipeline Commit

    Production Env.

    Deployment App.

    Commit Acceptance

    Manual

    Perf1 Perf2

    Staged

    Production

    Source Repository

    Acceptance

    Component Performance

    System Performance

    Staging Env.

    Deployment App.

    Manual Test Env.

    Deployment App.

  • The Role of Acceptance Testing

    Artifact Repository

    Local Dev. Env.

    Deployment Pipeline Commit

    Production Env.

    Deployment App.

    Commit Acceptance

    Manual

    Perf1 Perf2

    Staged

    Production

    Source Repository

    Acceptance

    Component Performance

    System Performance

    Staging Env.

    Deployment App.

    Manual Test Env.

    Deployment App.

    Staging Env.

    Deployment App.

    Manual Test Env.

    Deployment App.

    Component Performance

    System Performance

    Acceptance

  • The Role of Acceptance Testing

    Artifact Repository

    Local Dev. Env.

    Deployment Pipeline Commit

    Production Env.

    Deployment App.

    Commit Acceptance

    Manual

    Perf1 Perf2

    Staged

    Production

    Source Repository

    Acceptance

    Component Performance

    System Performance

    Staging Env.

    Deployment App.

    Manual Test Env.

    Deployment App.

    Staging Env.

    Deployment App.

    Manual Test Env.

    Deployment App.

    Component Performance

    System Performance

    Acceptance

  • What is Acceptance Testing? • Asserts that the code does what the users want.

    • An automated “definition of done”

    • Asserts that the code works in a “production-like” test environment.

    • A test of the deployment and configuration of a whole system.

    • Provides timely feedback on stories - closes a feedback loop.

    • Acceptance Testing, ATDD, BDD, Specification by Example, Executable Specifications.

  • What is Acceptance Testing?

    A Good Acceptance Test is:

    An Executable Specification of the Behaviour of the System

  • What is Acceptance Testing?

    Unit Test CodeIdea Executable spec. Build Release

  • What is Acceptance Testing?

    Unit Test CodeIdea Executable spec. Build Release

  • What is Acceptance Testing?

    Unit Test CodeIdea Executable spec. Build Release

  • So What’s So Hard? • Tests break when the SUT changes (Particularly UI)

    • Tests are complex to develop

    • This is a problem of design, the tests are too tightly- coupled to the SUT!

    • The history is littered with poor implementations:

    • UI Record-and-playback Systems

    • Record-and-playback of production data

    • Dumps of production data to test systems

    • Nasty automated testing products.

  • So What’s So Hard? • Tests break when the SUT changes (Particularly UI)

    • Tests are complex to develop

    • This is a problem of design, the tests are too tightly- coupled to the SUT!

    • The history is littered with poor implementations:

    • UI Record-and-playback Systems

    • Record-and-playback of production data

    • Dumps of production data to test systems

    • Nasty automated testing products.

    Anti-Pattern! Anti-Pattern! Anti-Pattern! Anti-Pattern!

  • Who Owns the Tests?

    • Anyone can write a test

    • Developers are the people that will break tests

    • Therefore Developers own the responsibility to keep them working

    • Separate Testing/QA team owning automated tests

  • Who Owns the Tests?

    • Anyone can write a test

    • Developers are the people that will break tests

    • Therefore Developers own the responsibility to keep them working

    • Separate Testing/QA team owning automated tests Anti-Pattern!

  • Who Owns the Tests?

    Developers Own Acceptance Tests!

  • Properties of Good Acceptance Tests • “What” not “How”

    • Isolated from other tests

    • Repeatable

    • Uses the language of the problem domain

    • Tests ANY change

    • Efficient

  • Properties of Good Acceptance Tests • “What” not “How”

    • Isolated from other tests

    • Repeatable

    • Uses the language of the problem domain

    • Tests ANY change

    • Efficient

  • Public API FIX API Trade

    Reporting Gateway

    “What” not “How”

    API Traders Clearing Destination Other

    external end-points

    Market Makers

    UI Traders

  • Public API FIX API Trade

    Reporting Gateway

    “What” not “How”

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

  • Public API FIX API Trade

    Reporting Gateway

    …FIX API

    “What” not “How”

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

  • Public API FIX API Trade

    Reporting Gateway

    “What” not “How”

    API Traders Clearing Destination Other

    external end-points

    Market Makers

    UI Traders

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

  • Public API FIX API Trade

    Reporting Gateway

    “What” not “How”

    FIX API

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    API External StubsFIX-APIUI FIX-APIFIX-API

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

  • Public API FIX API Trade

    Reporting Gateway

    “What” not “How”

    FIX API

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    API External StubsFIX-APIUI FIX-API

  • Public API FIX API Trade

    Reporting Gateway

    “What” not “How” Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    Test Case

    API External StubsFIX-APIUI FIX-API

  • Public API FIX API Trade