Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2
-
Upload
ca-technologies -
Category
Technology
-
view
68 -
download
2
Transcript of Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2
![Page 1: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/1.jpg)
World®’16
HighPerformanceMaskingandTestDataGenerationforIMSandDB2PrashantPandeyCATechnologies
MFX59E
MAINFRAMEANDWORKLOADAUTOMATION
![Page 2: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/2.jpg)
2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.
Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.
ForInformationalPurposesOnlyTermsofthisPresentation
![Page 3: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/3.jpg)
3 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Abstract
Withfinesaveragingmillionsofdollars,andprohibitivelyhighinfrastructurecosts,maskingandsubsetting themainframeisamust.ThissessionwillcoverhowtheCATestDataManagerarchitectureavoidsthediresecurityandperformanceimplicationsofmovingdatatoandfromz/OS.
PrashantPandeyCATechnologiesSr PrincipalConsultant,Presales
![Page 4: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/4.jpg)
4 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
TestDataonz/OS(theBadNews)
§ z/OSsystemofrecord(70%datastillresideshere)§ ManyorganisationsrelyoncopiesofProductionfortesting§ Secureandreliableinproduction– whatabouttest?§ WhatandWhereisPIIdata§ Legislation– GDPR,demonstratecompliance§ BreachNotification– reputation&finesrisk§ Costofmovingtestdatato/fromplatform
(theGoodNews)BestPracticeTestDataManagementalreadyaddressesthisimpact
![Page 5: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/5.jpg)
5 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
WhatisTestDataManagement(TDM)?
‘…isthepracticeofapplyingastructuredandcentralizedapproachtothemanagementoftestdataatanenterpriselevel,inordertoreducecostwhilstincreasingefficiencyandqualityindevelopmentandtesting’
![Page 6: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/6.jpg)
6 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
CA-TestDataManagerDeployment&Process
1. Centralize2. Discover3. Subset4. Mask5. Synthesize6. Find&Reserve7. Automate
Synthetic/CloneData
Masked
Subset
Created
ProductionDevelopm
ent
Web- SOATestHarness
Virtualization
DataMasking
Subsetting/DataSlicing
DataProfilingCoverageMetrics
Agile-Designer
TestDesign
DataDesign
TestDataonDemand
UI- LoadTestHarness
TestMart
FindandReserve
REQUEST/RESPONSEPOOLS
VERSIONCONTROLEXPECTEDRESULTS
REGRESSIONPACKS
DATADRIVENTESTPACKS
TEMPLATES
Agile-DataHPALM/QC
Automation
DataVisualization
Javelin
–Da
taOrche
stratio
n
![Page 7: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/7.jpg)
7 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
CA-TestDataManagerDeployment&Process
Adabas
z/Series
IMSDB2
VSAM
Unisys
DMSII
LUW
Informix
MSSQLOracle
Teradata
DB2/400 FlatFiles(CSV,Excel,XMLetc.)
i/Series
WindowsUI
ODBC/JDBCCOBOL/JCL
JAVASQL
LUW
CA– TestDataManager
Register&Profile
DefineTestDataEngineeringOperation
ProcessRun-time
EnableasOn-DemandRequest
TestDataOn-Demand
IMGR&ServiceLayerOracleorSQL
ServerRepository
![Page 8: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/8.jpg)
8 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
SecureMaskedSubsetsof
Data
MainframeEnvironment
DB2z/OSVSAMIMSnon-indexedflatfilesOtherDBs
ValidationProfilingPIIDiscoveryDataSubsetDefinitionBuilt-InMaskingFunctionsSyntheticDataRules
RulesandDefinitions
MetaData
SecureandeffectivetestdataengineeringonMainframe
CATDMWindowsWorkbench
DataneverhastoleavetheMainframe
MainframeBatchOperation
Rules
Repository(RulesandMetaStore)
PDSJCL&CTL)
FTP
ODBC/Copybooks
![Page 9: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/9.jpg)
9 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
End-EndIntegrity
§ Acrossmultiplesource(andplatforms!)– z/Series:DB2,IMS,VSAM(KSDS,ESDS,etc.)orflatfiles(PS)etc.– i/Series– AS400:DB2/400– Distributed:Oracle,SQLServer,DB2UDBetc.– Filestructures:CSV,Excel,XML,SOAP,REST,JSONetc.
Maintainingdatabase,businessandlogicalintegrity
CustID Item# OrderDate
07221 80-25442 2015-01-25
07221 80-25442 2015-03-14
CustID Item# OrderDate
10002 80-25442 2015-02-15
10002 80-25442 2015-05-19
Dataretainsstructure,contextandreferentialintegritybutdoesnotcontain“real”information.
CustID Name SSN Street
08052 AndrewChapman 181-23-7171 87Featherbed Lane
09112 Brian James 112-32-4435 12ParkLane
08052 AndrewChapman 181-23-7171 87Featherbed Lane
07221 TomAngle 771-11-8972 8Boardwalk
CustID Name SSN Street
10000 Hikaru Sulu 545-95-1245 87Featherbed Lane
10001 Nyota Uhura 802-41-3411 12ParkLane
10000 Hikaru Sulu 545-95-1245 87Featherbed Lane
10002 PavelChekov 325-74-6517 8Boardwalk
![Page 10: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/10.jpg)
10 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
ProjectandVersionCentralizedControl
§ Re-usableassetsinaTestDataWarehouse
§ Providesacentrallibraryoftestdataengineeringoperations
§ Assetscanbeupgradedacrossversions(releases)
§ Regressionpackscanbestored
CATestDataManagerTestDataWarehouse(Repository)
![Page 11: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/11.jpg)
11 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Automaticallydiscoversensitivedatastoredenterprise-wide
§ Powerful,mathematically-basedalgorithmsdiscoverandtagallsensitivedata,enterprise-wide
§ Simple, easytousefilteringbytagwhenselectingcolumnsortablestomask
§ Supportforeverymajordatabasetype,mainframeplatforms andflatfiles
CATestDataManagerDataProfiler
![Page 12: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/12.jpg)
12 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Subsetting:datasetsareextractedbasedonexactcriteriaCATestDataManagerDataSubset
Referentiallyintactsubsetsofdata
ProductionAttributes
ScenarioSubset
CoveredSubsetProduction&Enhanced
Attributes
![Page 13: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/13.jpg)
13 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
De-sensitizemillionsofrowsofreferentiallyintactdatainminutesCATestDataManagerFastDataMasker
Theseincludesubstitution,randomization,hashingandseeddata.Oncecreated,maskingrulescanbestoredandre-usedfromacentralrepository.
GTFastDataMaskerwillonlydisplaysuitablemaskingroutines,basedontheselectedcolumnandtables
Deterministicmaskingfunctionsandbuilt-incross-referencingensureconsistency.Thereferentiallyintact,realisticdata,canbeinjectedintomultiplesystemsatonce
NumberofRows Timeto Mask
201,722,392 9minutes, 42seconds
453,877,152 11minutes,22seconds
768,088,071 7minutes,57seconds
17,422,541 1minute
46,579,485 1minute, 25seconds
1,759,612 13seconds
47,895 5seconds
GTFastDataMaskerusesnativedatabasescriptstoproducethehighestpossibleperformancewhenmaskingOracle,MSServer,Teradataandmainframeplatforms
§ Maskmillionsofrowsofdata inminutesusingnativemaskingengines
§ Over 80built-indatamaskingfunctions
§ Replacementswithrealistic,referentiallyintact, appliedtomultiplesystems
§ DemonstrateCompliancewiththeEUGDPR,GLBA,HIPAA,PCIDSS,PIPEDA,andmore
![Page 14: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/14.jpg)
14 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
GenerateRequiredDataScenariosCATestDataManagerSyntheticDataGeneration
Thismeansthatnewdatacanefficientlyandquicklyinjectedintoexistingdatabasesforexactlytherequirementsyouneed
§ ExtensivealgorithmsforSyntheticdataappliedinaDataPainterstyle
§ UseSymbolic values,combine functions,include conditional valuesandmine datafromothersourcesintothenewdata
§ Publish(inject/load)singledataobjectasmultiple/variednewrowsinexistingdatabase
§ Storedataobjectscentrally,asre-usableassetsinaTestDataWarehouse
§ Datacanbere-usedacrossversionsandreleases,withversionedregressionpacks
![Page 15: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/15.jpg)
15 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Find&ReserveExistingDataCATestDataManagerTestMatching
SAP-1
DataAttributeMining
DataTestConditions
DatamakerTestMatching
DB-2
XML
MatchedTest
QTPScripts
![Page 16: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/16.jpg)
16 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
CombineTDMOperationstoProvideComprehensiveSetofData
Referential Integrity
Profiling
PII Discovery
Native DB Engines
Seed Replacement
Audit
CATestDataManager
1 2 3SecureDataSubsets
Test/DevEnvironments
Gold Copy
Subset
Seed Tables
Built-in Masking Functions
Cross Referencing
Masked
Generated Scenarios
Find & Reserve Data
SyntheticContent
SyntheticContent
Productiondatasourcesandfiles
XMLFilesExcelFilesSQLFiles
CSVFiles FixedDefinitionFiles
HTMLFiles
VSAM/ISAMSwift
TXTFiles
TestData
Warehouse
![Page 17: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/17.jpg)
17 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Avoidcostlymistakeswhenmanagingz/OSTestData
Avoiduseofrawproductiondataisamandatoryoperationformodernorganizations.Theaveragecostperincidentofdatabreachworldwideis$3.5m.IntheUS,theriskwasashighas$5.85m1.
Inasmanyas59%oftheseincidents,theriskcouldhavebeenmitigatedbyimplementingTestDataManagementbestpractices
1 Ponemon Institute,2014
CATDMallowsuserstoapplyTestDataManagementpracticesdirectlyonthemainframe reducingtherelianceonslow,expensiveExtract,Transformandreloadtechnologiestoensure complianceandtakeadvantageoffit-for-purposetestdata.
Processingdatadirectlyonthemainframe canprovideteamswithmillionsofrowsofhighquality,meaningful,referentiallyintactdata,inashortperiodoftimeandatlessrisktoquality!
![Page 18: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/18.jpg)
18 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
RecommendedSessions
SESSION# TITLE DATE/TIME
MFX58ECCATestingToolsEducationalSession(Part3of3):CAMainframeApplicationTunerDemoandWhat'sNew 11/15/2016at2:00pm
MFX112S MainframeApplicationDevelopmentPortfolioReview 11/16/2016at12:45pm
MFT148SAchieveDevOpsontheMainframeforFasterTimetoMarket 11/16/2016at4:30pm
![Page 19: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/19.jpg)
19 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
MustSeeDemos
ContinuousDevelopmentandTestingMainframeTheater
ContinuousDelivery
MainframeTheater
![Page 20: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/20.jpg)
20 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Questions?
![Page 21: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/21.jpg)
21 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Thankyou.
Stayconnectedatcommunities.ca.com
![Page 22: Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2](https://reader030.fdocuments.us/reader030/viewer/2022020314/587f03521a28abc26f8b45d9/html5/thumbnails/22.jpg)
@CAWORLD#CAWORLD ©2016CA.AllRIGHTSRESERVED.22 @CAWORLD#CAWORLD
MainframeandWorkloadAutomation
Formoreinformationpleasevisitca.com/appdev
PrashantPandeySr PrincipalConsultant,[email protected]