High level QA strategy for SQL Server enforcer
description
Transcript of High level QA strategy for SQL Server enforcer
High level QA strategy for SQL Server enforcer
Presentation for Nextlabs by Alex Todortsev
Agenda for Discussion
• Understanding Customer’s Environment– Challenges, Stages, Requirements
• QA Strategy• Different aspects of the product
– Functional testing– Authentication/access control handling– QA coverage for different customer
topologies• QA Process• Automation part of Test Plan
Customer Environment
• Understanding the environment– Topology for each client, 3rd party or
proprietary software, legacy system support.– Number of users and authentication system– Access rights and rules enforcement on all
levels– Average number of concurrent users, picks,
any known bottlenecks
• QA task – design universal scale down environment that will allow to test different customer topologies without recreating every single one.
Architectures are Complex
Databases
Web ServerInternetHTMLJava
JavaScript
ExtranetHTMLJava
IntranetJavaC++
ActiveX,Flash, etc.
Application ServersWebLogic
WebSphereSilverStream, etc.
CoreApplications
3rd Party Softwareand Middleware
LegacySystems
Firewall
At what stage QA should be involved?
Requirements
FunctionalSpecifications
Design
Code
FunctionTesting
SystemTesting
SpecializedTesting
Review
Review
Review
Review Unit Testing
Function Test Design (test cases)
System Test Design
(Specialized Test Scenarios)
QA Approach• Phase 1 – Define Needs
– Understand client’s QA requirements
• Phase 2 – Define Testing Plan– Determine test strategy– Form team – Create QA project plan
• Phase 3 – Design Test– Create test cases– Set up tools and environment
• Phase 4 – Implement Test– Execute test cases– Report bugs– Fix, verify, regression test loop
• Phase 5 – Analysis and Report– Analyze process, defects, and application– Incorporate data from analysis into test process– Knowledge transfer
QA strategy
• What is necessary for successful testing:– Test environment that will be universal by
allowing us to recreate specific customer topology
– Highly skilled and dedicated staff focused on QA– Use of flexible and dynamic QA process
• Testing areas:– Support multiple configurations and platforms– Authentication and access control – Functional testing– System, stress and load test
QA strategy (contd.)
• Aspects that need to be tested:– Verify correct enforcement of policies and
access control based on/for:• user/group• objects (Tables, Indexes, Triggers, Columns,
etc.) and actions (Create, Delete, Insert, Update, etc.)
• different aspects of the Query (Joins, new indexes, etc.)
• data size (Insert/Delete/Update, query size, etc)
– Verify that full and correct report is provided to policy officers and user is informed when access was denied due to policies and access control.
QA process• Components and parts:
– Build system (automated build + scripted acceptance test)
– Bug/defects lifecycle – Unit test library and code review– Test case design based on user experience
• Potential addition and changes to functionality/support should be taken into account (incorporate customers support feedback)
– Test metrics and test subsets for specific test cycle– Internal use of product– “Sandbox” as a testing ground for pilots and new
functionality– “Client” mentality through development/QA
process
QA process (contd.)
• “White box” test approach (resources, test cases, development/QA cooperation)
• Test tools and areas targeted for automation
• Automation and regression library• Analyze bug/defect ratio, test case
coverage, usability feedback, identify weak areas of the product and incorporate results in test process
Testing Details(some aspects of
automation area in test plan)
QA Test plan (automation overview)
Each section of a test plan offers a detailed view on how testing will use its many weapons and tools to attack the product. The audience for this is primarily Testing, since the plan specifies what will be done to test the product. Development may also find it useful so they know how we intend to test their product.
• Strategy SummaryFrom a testing perspective, we identify the main issues / issues
that will be involved in testing of the product. – Goals
The main goal is to implement as much automation for UI and functional test as possible. Specific attention should be paid to the process of authentication and access control.
- Non Goals (for example)In the future we should support Mac OS
QA Test plan (automation overview) (contd.)
• ApproachThe whole application should be divided into small parts and tested
accordingly to test matrix. At a glance those parts are:– UI/functional test cases – Installation (including different configurations and
platforms)– Synchronization of access control based on user/group
(includes some boundary cases like disconnecting laptop during synchronization and multiple users updating/downloading access information for the same group)
– Server side testing (includes queries, server response time, points of failure, back-up plan)
– System test– Stress/load/volume test
QA Test plan (automation overview) (contd.)
• Automation– UI automation tools and areas of UI that appropriate for
automation (choosing tool will depend on UI specifics such as platform, elements and areas of automation)
– SQL Server test tool (based on SQL client that most users will use I’ll have to pick a tool that will fit in our authentication/access control schema) For example SQL Server Query Analyzer might be useful to see statistics on query performance and table/action execution.
– Stress, load, volume tests – (Do we need to do it for this project? Making sure we test our product and not SQL server)
– Review of existing (in house) tools, what could be used, how much additional effort required to adapt tool for our needs
– Any custom tool that could be created in the house?– List of high level test scenarios for automation that will be
expended with test cases later in the process.
Thank you for your time!