Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data...
Transcript of Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data...
![Page 1: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/1.jpg)
BIO PRESENTATION
Better Software Conference June 26 – 29, 2006 Las Vegas, NV USA
T17
6/29/2006 1:30 PM
TESTING SOA MIDDLEWARE: AUTOMATING WHAT YOU CAN'T
SEE
Jon Howarth and Robert Ryan Wells Fargo
![Page 2: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/2.jpg)
Jon Howarth Jon Howarth is the current manager of the Enterprise Business Services & Information Management (EBS&IM) Regression, Tools, Data Warehousing, and Data Services QA groups at Wells Fargo Bank. Jon Howarth has over 7 years of QA experience with a focus on automation testing. He has experience in developing automation frameworks for a variety of technologies including the web, middleware (web services, MQ), and the mainframe. He is always looking for new ways to innovate with tools for efficiency.
Robert Ryan
Robert Ryan leads special projects for Enterprise Business Services QA (EBS QA) at Wells Fargo Bank. He has 11 years of experience building and implementing IT systems that support financial transactions. He has been developing HTTP-based messaging systems since 1998 and has worked with Innoventry, Pacific Gas & Electric, and Wells Fargo to define and implement web services. His current focus is implementing test tools and processes for service-oriented architectures.
![Page 3: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/3.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 1
![Page 4: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/4.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 2
![Page 5: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/5.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 3
Presentation Goal
• Describe the complexities of testing a web services/SOA (Service Oriented Architecture) based middleware in an enterprise environment and our solution to this problem
![Page 6: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/6.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 4
Frame the Problem
• The application• SOA one of the complexities• Rapidly changing• Contract compliance
• The backend dependencies• Not the owner of the environment• Volatile data
• The tools• Application does not have a GUI• Immature tool set
• The QA process• Not traditional QA scope• Not black box, post condition verification• Non intuitive failure analysis
![Page 7: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/7.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 5
Frame the Problem
![Page 8: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/8.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 6
Background & Format
• Presenter background• Enterprise middleware QA group• Mission critical bank SOA software
• Presentation format• Frame the problem• The search for the solution• Tools• Skill set• Framework• Metrics
![Page 9: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/9.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 7
The Search for the Solution
• The requirements• Data management requirements• Tool(s) requirements• Vendor requirements
• The identification of tools• Research• Leverage vendors
• Proof of concept• Acceptance criteria• Tool integration• Tool acceptance
![Page 10: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/10.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 8
Solution - Architecture
• Manual functional testing• Test case management and workflow• Automated test execution• Data management
![Page 11: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/11.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 9
Solution - Tools
• Manual functional testing • Soap Scope, Mindreef• An industry standard web service testing tool
• Test case management and workflow• Test Director for Quality Center, Mercury Interactive• Integrated with Integra Enterprise
• Automated test execution• Integra Enterprise, Solstice Software• Supports key protocols for SOA and integration testing
• Data management• Data Repository, developed in-house• Tracks data conditioning requirements
![Page 12: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/12.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 10
Tool Architecture
![Page 13: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/13.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 11
Our Skill Sets
• Implementation team• Broader than testing• Diverse skill sets
• End users• Established QA disciplines
– Subject matter experts – Junior testers
![Page 14: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/14.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 12
Project Methodology
• Analysis of requirements• Used analysis best practices• Understand the needs of the QA group• Included end-users• Workflow and reporting
• Solution architecture• Map requirements to tools• Define interfaces between tools
![Page 15: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/15.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 13
Project Implementation
• Goals and deliverables• Defined milestones• Dedicated team members• Iterative approach• Commitment and support
![Page 16: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/16.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 14
Framework
• Data driven model• Traceability from test cases to test scripts• Flexibility of test script suites• Council
• Medium for improving workflow, tool, and process issues
• QA discipline representation
• Infrastructure in place to automate future web service applications
![Page 17: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/17.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 15
Lessons Learned
• Management support• Full buy-in• Facilitate change
• Variety of reactions from the end-users• We could have used an ambassador• Migration of legacy scripts is complex• Keep homebuilt tools simple
![Page 18: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/18.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 16
Project Metrics
![Page 19: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/19.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 17
Framework Metrics
• Project execution status• Release readiness status• State of data• Adherence to best practices
![Page 20: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/20.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 18
Summary
• A solution is possible• Take the time to architect the solution• The investment pays off• Vendor support
• Treat it like a project• Have a methodology• Dedicated resources• Management support
![Page 21: Merged Covers - StickyMinds · PDF fileManagement (EBS&IM) Regression, Tools, Data Warehousing, ... • Traceability from test cases to test scripts • Flexibility of test script](https://reader031.fdocuments.us/reader031/viewer/2022011723/5a8dcf447f8b9abb068cc1ee/html5/thumbnails/21.jpg)
June 29, 2006 Automating what you can’t see: Testing middleware for the enterprise 19
Questions