Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using...
Transcript of Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using...
T11 Virtualization Testing Thursday, May 3rd, 2018 11:15 AM
Integration and Functional Testing Using Dockerized Dependencies
Presented by:
Irene Dhong
Bazaarvoice
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073 888-- -268- - -8770 ·· 904- --278-- -0524 - [email protected] - http://www.stareast.techwell.com/
Irene Dhong Bazaarvoice Irene Dhong has been a QA Engineer for over 6 years. She has experience as a build engineer and a system administrator. She's passionate about building frameworks and tools that maintain a high level of quality in lean, rapid development environments. She enjoys bringing quality to new challenges like machine learning, natural language processing, and complex microservice ecosystems. Irene loves to encourage community among women in technology and runs a women's leadership guest speaker series at Bazaarvoice.
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 0
Integration and Functional Testing Using Dockerized Dependencies
Irene Dhong
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 1
Ice Cream Cone
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 2
•High quality low defects
•Expensive
•Long running tests
•Extremely hard to add a new test
•Debugging difficult or near impossible
The Struggle
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 3
•Amount of mocking out needed
•Interactions hard to simulate
•Manual Custom ops code
•Dozens of dependencies
•Fast pace of changing code
How did we get here?
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 4
•Be able to write and run tests quickly
•Have all dependencies mocked out
•Try to catch quality issues before it even reaches ci
•Test on a very low level
•Write hermetic tests
•Have access to service/application/logs while developing
•Reuse code
My wish list
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 5
•Docker is a computer program that performs operating-system-level virtualization also known as containerization. *wikipedia
•Docker is a platform for developers and sysadmins to develop, deploy, and run applications with containers. The use of Linux containers to deploy applications is called containerization. Containers are not new, but their use for easily deploying applications is. *Docker.com
Docker
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 6
DockerizedWorld
•Elements of our system are run in containers
•Links used to expose elements to each other
•Use official images for infrastructure
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 7
•Unit-like
•Local Integration
•Can be run locally or in ci
•Deployed integration tests in production or production-like environment
Supports different kinds of testing
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 8 The image part with relationship ID rId22 was not found in the file.
Unit like testing environment
@BeforeSuite
• Start dep
@Test
• Test function
• Verify data
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 9 The image part with relationship ID rId22 was not found in the file.
A Hermetic functional test environment
@BeforeSuite
• Start dep and local app
@Test
• Setup, Start operation, Verify data
@AfterTest
• Clean up data
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 10 The image part with relationship ID rId22 was not found in the file.
Local Integration Test Environment
@BeforeSuite
• Start dep and local app
@Test
• Setup data, Trigger app, Verify data
@AfterTest
• Clean up
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 11 The image part with relationship ID rId22 was not found in the file.
Local Integration Test Env or CI• Start dockerized
dependencies
• Start dockerizedapplication
• Run tests
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 12 The image part with relationship ID rId22 was not found in the file.
Integration tests run in deployed env
• Reuse test framework using config.yaml
• Omit startup, setup, and cleanup steps
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 13
•Simulate real life situation
•Reuse existing data loading capabilities
•Explore transitional states
•Insight to data
•Large datasets
•Clean databases
•Hermitic data
Issue: Hard to add data
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 14
•One time cost
•Maintained through dockerfile
•Stubbed dockerized dependency only tests interactions that tests require
•As dependency changes, maintenance is little as can use POJO’s
Issue: Huge amounts of mocking out
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 15 The image part with relationship ID rId22 was not found in the file.
Issue: Manual custom ops code• Database table
creation in dockerfile
• Devops/dev updates table files in one file
• Same image used for dev, devops, and testing
The image part with relationship ID rId2 was not found in the file.
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 16
•Redeploy application pointing to dockerized dependency
•Hybridization
Issue: Disaster Recovery/Perf Test
16
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 17
•Advantages over everything running locally
•Advantages over deployed environment
Issue: Debugging in Deployed Environment
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 18 The image part with relationship ID rId22 was not found in the file.
Challenge: No official docker image
• S3, sqs
• FakeS3
• Redis
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 19 The image part with relationship ID rId22 was not found in the file.
Challenge: Impossible to dockerize
• Stub it and dockerize it!
• Quick flask server
• Don’t hardcode, make stub interactive
• Risk analysis
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 20
•Keys
•Mimic
Challenge: Security
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 21
•Update shared pojos, classes, and tests
•Build latest image with latest table scripts or changes
•PR build which runs all local tests
Challenge: Development changes code
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 22
•Mvn, gradle, sbt
•Python, Java, Javascript, Scala
Challenge: Multitude of technologies
The image part with relationship ID rId2 was not found in the file.
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 23
Challenge: Multitude of technologies• Use common scripts
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 24
•Investment of time
•Mocked/Stubbed components
•Isolated from the bigger picture/world
•Too close to the code
Testing specific cons
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 25
•Offline development
•Containers are isolated so don’t have to pollute machine with various packages and system services
•Dev use same infrastructure for unit like dependency testing
•Devops use same image for deployment with different config
•Reuse of code in all aspects
Pros generally
Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 26
Testing specific pros