Continuous Testing - What QA means for DevOps
Transcript of Continuous Testing - What QA means for DevOps
![Page 1: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/1.jpg)
Continuous Testing What QA means for DevOps
![Page 2: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/2.jpg)
Eran SherCo-founder & CEO of SeaLights
Nolio (Acquired by CA Technologies)Mercury Interactive (Acquired by HP)
Conduct (Acquired by Mercury)
![Page 3: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/3.jpg)
The State of QA & DevOps
“managing and implementing quality and test measures represent a challenge for teams trying to increase productivity (or release velocity) with acceptable quality”WORLD QUALITY REPORT 2016-2017
IncreasingRelease speed
Decreasing Release quality
Continuous Delivery is becoming the de-facto standard
![Page 4: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/4.jpg)
The Industry Reaction
![Page 5: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/5.jpg)
The Industry Reaction
Organizational People Skills Tools Practices Communication
Now that the hysteria has passed
![Page 6: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/6.jpg)
What QA Means for DevOps
Continuous IntegrationHow we build software
Continuous DeploymentHow we deploy software
Continuous TestingHow we test software
Continuous Delivery
![Page 7: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/7.jpg)
Continuous Testing
Increase quality while increasing speed
![Page 8: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/8.jpg)
Step 1
Understand what is causing quality to decrease when speed is increased
![Page 9: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/9.jpg)
Understanding Test Quality
How do you measure test quality?
![Page 10: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/10.jpg)
Quality Holes
How do you stop untested code changes from reaching production?
![Page 11: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/11.jpg)
Centralized Quality Dashboard
How do you collect quality metrics across environments and tools?
![Page 12: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/12.jpg)
Step 2
Find/ develop a solution for the identified root cause problems
![Page 13: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/13.jpg)
Uncover Code Coverage
Unit Integration Functional UI API Exploratory
? ? ? ? ?
![Page 14: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/14.jpg)
Identify Quality Holes
Scan every build and analyze code changes
Track every test and analyze footprints
![Page 15: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/15.jpg)
Build Quality Dashboard
Quality report per build Collect statuses and coverage across tools, environments and silos
Track executed tests
![Page 16: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/16.jpg)
Step 3
Integrate solutions into CI/CD
![Page 17: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/17.jpg)
Case Study
Our Technical Stack:1. Node.js, Java, and Python
2. Microservices architecture, Mongo DB
3. CI: Jenkins
4. SCM: Github
5. Test automation frameworks: Mocha, TestNG, JUnit, and Py.test
6. Infrastructure: AWS
7. CD: homegrown
![Page 18: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/18.jpg)
Case Study
Our CI/CD Workflow1. Pull request
2. Build + unit test
3. Verify that all tests passed, no decrease in coverage, and no quality holes
4. Integration build
5. Setup and deploy build to integration lab (AWS)
6. Run integration tests
7. Verify that all tests passed, no decrease in coverage, and no quality holes
8. Ready for production » Deploy
![Page 19: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/19.jpg)
Copyright © 2016 Sealights. All rights reserved.
Q&A
![Page 20: Continuous Testing - What QA means for DevOps](https://reader036.fdocuments.us/reader036/viewer/2022062316/589de25c1a28ab773b8b460f/html5/thumbnails/20.jpg)
Start testing with the lights on
Copyright © 2016 Sealights. All rights reserved.