Continuous Integration, Deploy, Test From Beginning To End 2014
-
Upload
appvance-inc -
Category
Software
-
view
168 -
download
2
description
Transcript of Continuous Integration, Deploy, Test From Beginning To End 2014
![Page 1: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/1.jpg)
Continuous Integration, Deployment, Test From Beginning To End October 22, 2014
www.appvance.com
Frank Cohen, CTO/Founder at Appvance, teaches developers and testers how to become more closely aligned with easily deployable and configurable tools such as Jenkins CI and performance testing platforms. Learn how to: * Merge Agile SDLC and Performance Testing * Speed-up test operations by configuring Jenkins for automatic build, deploy, and tests * Instantly identify functional and performance issues using Jenkins as a performance dashboard * Use effective choices in Jenkins deployment – in the cloud, hosted, or in your datacenter [email protected]
![Page 2: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/2.jpg)
Continuous Integration, Deployment, Test From Beginning To End
October 22, 2014 Frank Cohen, [email protected], (408) 364-5508
![Page 3: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/3.jpg)
What We Will Talk About
•Agile Methodology Expects Build, Deploy, Test Automation
•A Continuous Integration Strategy
•Ajax and Mobile Testing Methodology
•Repurposing for Functional and Performance Testing
•Becoming A CI Architect
![Page 4: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/4.jpg)
![Page 5: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/5.jpg)
Agile Methodology in a Process
7-day Sprint
Highest Priority
Backlog of Changes,Improvements
• App does not run on iOS • Rendering problems• UX fails at 45 users • Overnight reports needed
Funding and Feedback
Sprint Review
Acceptance
✓ Unit Tests Pass✓ Functional Tests Pass✓ Usability Tests Pass✓ Perf Under Load Pass✓ Stress Tests Completed✓ Production Monitor In Place
Daily Work
Release
![Page 6: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/6.jpg)
Agile Without CI?
![Page 7: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/7.jpg)
Why Do Continuous Integration?
•It’s free and easy
•Your Test Suites Take Longer Than “Overnight” To Run
•High Risk App Launch
•Software build/deploy/test plan of record (How do I?)
•Easy to go from functional to performance testing
•Keeps you productive, while long projects build/deploy/test
![Page 8: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/8.jpg)
Why Avoid Continuous Integration
•You Work In A Small Shop
•Stakeholders Unable To Participate
•No One To Manage Projects on Jenkins
•One-shot project (build, launch, forget)
•Deploy/Test Requires Human Intervention
•Low Risk App Launches
![Page 9: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/9.jpg)
Continuous DevOps
•Continuous Integration
• Dashboard To Testing
•Continuous Deploy
• Automatic app deployment
•Continuous Testing
• Regression on Commit
• Repurpose for Load Tests
Test CI Runtime
MonitorTicketsRepository
![Page 10: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/10.jpg)
How Do I Get Jenkins?
•Run on Cloudbees (Fully Managed)
•Run on AWS, IBM Softlayer (Self Managed)
•Download and Run Internally (Self Managed)
•Then, Plug-in Test As A Service Platform
![Page 11: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/11.jpg)
![Page 12: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/12.jpg)
What Is The Measurement of Success
•Functional Testing Defines End User Experience Success
• Browser Compatibility, UX Latency, Configuration/Integration Problems
•Performance Testing Identifies Business Risk To Launch
• How Many People Are We Ready To Serve At Any Given Time
•Stress Testing Identifies The Experience As The App Fails
• International Bank App Showing Other User Details
Testing Measures Business Risk
Sprint Review
Acceptance
✓ Unit Tests Pass✓ Functional Tests Pass✓ Usability Tests Pass✓ Perf Under Load Pass✓ Stress Tests Completed✓ Production Monitor In Place
![Page 13: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/13.jpg)
A Testing Strategy for You
•Working With Today’s Apps, Pages, Tools • Serverless Architecture means pages that have state, workflow, communication
• Real time analytics and immediate fix - at the same time
• Extending tools to support new technology
•Beginning To End Testing •Scaling Up Your Team’s Efforts
• Higher Productivity
• Test Management
•Cost Effectiveness and Immediate Support
![Page 14: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/14.jpg)
How Do I Get App Testing?
•Ajax App Testing
• Sahi, Selenium, WebDriver
• Unit Testing more difficult, easier to maintain
• JMeter does not work in Ajax
•Mobile App Testing
• Appium for iOS, Android
• Unit Testing of REST and SOAP services using soapUI, HAR, JMeter
• Others, very limited support and no repurposing to performance tests
![Page 15: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/15.jpg)
Continuous DevOps In Practice
Data Center Switches Testing
svn, GIT
TestNode4
TestNode5
OracleAPC Controller 1
Test Scenario
1
DPL,Logging
![Page 16: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/16.jpg)
Scale Up To Multiple Teams
Data Center Switches Testing
svn, GIT
TestNode4
TestNode5
OracleAPC Controller 1
Test Scenario
1
DPL,Logging
TestNode2
TestNode3
APC Controller 2
Test Scenario
2
TestNode1
Router Team Testing
![Page 17: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/17.jpg)
Connect To Continuous Integration, Deployment
Data Center Switches Testing
svn, GIT
TestNode4
TestNode5
OracleAPC Controller 1
Test Scenario
1
DPL,Logging
TestNode2
TestNode3
APC Controller 2
Test Scenario
2
TestNode1
![Page 18: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/18.jpg)
Executing Appium Tests from Jenkins
#Start appium server appium_log=appium.log appium_cmd="node <REPLACE_WITH_APPIUM_PATH>/appium/lib/server/main.js" echo "[INFO] Appium command: $appium_cmd" $appium_cmd &> $appium_log& appium_pid=$! #check if appium was successfully started sleep 1 appium_pid_start=`ps -p $appium_pid | grep node | awk '{ print $1 }'` if [ -z $appium_pid_start ] ; then echo "[ERROR] Appium failed to start! Port $appiumport may be already in use. " exit 1 fi echo "[INFO] Appium started with pid: $appium_pid . Logging to: ${appium_log}" # be sure to kill appium on script stop trap "kill $appium_pid" SIGINT SIGTERM EXIT
A
B
C
![Page 19: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/19.jpg)
![Page 20: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/20.jpg)
![Page 21: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/21.jpg)
![Page 22: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/22.jpg)
Why Treat Functional/Performance Tests Differently?
•Repurposing Benefits
• Single Test Scripts for All Testing
• Pivot Back and Forth Rapidly
• Natural for Continuous Integration and Automation
•Performance Test
• Run at Low, Medium, High Virtual User Levels
• Scalability Index Report Answers Risk Questions
![Page 23: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/23.jpg)
TestScenario Repurposing
![Page 24: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/24.jpg)
Correlation To Back End Load
![Page 25: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/25.jpg)
Backend Correlation
![Page 26: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/26.jpg)
Strategy for Mobile Test Runtime
•Run Test in Emulators (Mac and Android) 85%
•Run Test in Real Devices 10%
•Run Test with Real People 5%
We’re going to run the test between 1-2 pm, get as many real people on it then
![Page 27: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/27.jpg)
Become A CI Architect
•Draw A Line From Business Risk To Your Integration Plan
![Page 28: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/28.jpg)
How To Get Started
•Attend Scrum Meetings, Ask A Few Questions
•Ask The Business Manager for an App About Risks
•Ask the Developers Where The Test Coverage Is Thin
•Choose An Appropriate Test Script Authoring Tool
•Write or Record A Test
•Commit The Test to svn, CVS, other repository
•Plug-in To CI and Maintain
![Page 29: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/29.jpg)
Contents of a CI Plan
•Identify the Parts
• Beginning-To-End of User Experience, including Data
• Units of App Function
•Success/Fail Criteria
• 50,000 User Transactions per 10 Minutes with less than 1% Errors
• Functional Test on IE 8, Firefox 16, iOS 5
•User Archetypes and Workflow
•Performance Under Load and Stress
![Page 31: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/31.jpg)
What We Covered Today
•Agile Methodology Expects Build, Deploy, Test Automation
•A Continuous Integration Strategy
•Ajax and Mobile Testing Methodology
•Repurposing for Functional and Performance Testing
•Becoming A CI Architect
![Page 32: Continuous Integration, Deploy, Test From Beginning To End 2014](https://reader033.fdocuments.us/reader033/viewer/2022042700/5563a843d8b42a2b6a8b54ed/html5/thumbnails/32.jpg)
Are You Ready?
•When You Build Modern Apps And Wonder “How Risky Is That App Launch Going To Be?”
•Call Us At (855) 254-1164 Or Email Us [email protected] We Will Show You