Outfittery Software QA Where we started

Where we areWhere we are headed

How we are going to get there

Where we startedNo handoff documentationNo general test planNo automationNo scheduleNo maintenance of existing test case

management systemFocus on IAS queue

Where we areNo automation frameworkNew test case management system (Zephyr)Return to scheduled releasesRemoved focus on IAS queueTicket validation not testingNo metrics for QA effort

Where we are headedAutomation frameworkTeam adoption of ZephyrTest orchestrationAutomated BVTTicket validation and testingTransparency via metrics

ChallengesVelocity of features clearing the Product

boardComplexity of architectureLack of available QA resourcesRandomization/Context switchingLack of separate QA/Staging environment

Velocity of features to be deliveredFeatures are planned, groomed and

scheduledUnplanned features request arise from the

business, must remain responsive.Bottleneck will be QA unless QA tasks are

moved upstream and distributed Leverage the team to increase QA coverage

when surge capacity is needed

Complexity of ArchitectureMultiple interface points for testing causes

context switching issuesComplete tickets that combine FE and BE to

maintain contextDefining what does not need to be testedIdentifying the components in the

Architecture that will benefit most from testing

Lack of available QA resources1.5 QA currently, 1 additional head count Test automation is part of the solutionLeverage available resources via test

distributionIntroduce Bug Bash

Randomization/Context switchingQA must be present to react to new issues as

they ariseMultiple, disparate tools required to interface

with the complex architectureLeverage the Atlassian toolset to reduce

context switching. Zephyr is a prime exampleIntegrate other tools with Atlassian tool set

Test AutomationMust integrate with our architectureMust be easy to maintainMust integrate with JenkinsMust support cross browserMust support mobile

Test AutomationTest framework selectionMust be robust, scalable and supportedThis excludes many tools found on GIT HUBSuggestuions include

The InternRobotCucumberPhantom.JS

Automation toolsCode profiling tools for JavaJaCoCoCloverRequires developer assistance for

selection/implementationIntended to evaluate effectiveness of test

passes

Automation ToolsJavascript code coverageBlanket.jsnode-coverageBrowser basedFirebugScript-cover

Automation toolsSQL ProfilingOpen SCGOtherWiresharkOWASP ZAPjMETER/Blazemeter

Automation ToolsSikuli, facilitates automation exclusive of

DOM changesUsed to generate real world user activity for

Synthetic MonitoringLeverage with Kibana/Logstash

StrategyImplement automated real user activity

against production, Synthetic MonitoringMaximize code exercised during manual

passesAutomate the repeated test passesExecute tests outside of ticket validation

Tasks Create full user persona to drive synthetic

monitoring across all locals Outfittery servesCreate fake entities e.g. fake stylists that the

test user persona will interact withDevelop means to identify test persona

activity to automatically route to fake stylists

TasksDefine customer flows for:CallNo CallFollow on orderReturns

TasksIntegrate financial informationTest addresses from Arvato as part of the test

user personaPayment information from our selected

provider

TasksCreate test branch in GITHUB for ALL

testing

TasksAutomate order page that runs exclusivley of

funnell to run automated via jenkins at build time.