Veridata for Data Consistency in WLCG
description
Transcript of Veridata for Data Consistency in WLCG
![Page 1: Veridata for Data Consistency in WLCG](https://reader035.fdocuments.us/reader035/viewer/2022062315/56814fc8550346895dbd84a9/html5/thumbnails/1.jpg)
Veridata for DataConsistency in
WLCG
OpenLab lightning talks 2014
Anti AskoSupervisor: Lorena Lobato Pardavila
![Page 2: Veridata for Data Consistency in WLCG](https://reader035.fdocuments.us/reader035/viewer/2022062315/56814fc8550346895dbd84a9/html5/thumbnails/2.jpg)
2
The Project Challenge
› IT-DB provides replication services between CERN and specific Tier-1 data centers world wide as part of the WLCG project
› Checking data consistency is a challenging task
› Oracle GoldenGate Veridata seems a promising solution
19/08/2014 Anti Asko
![Page 3: Veridata for Data Consistency in WLCG](https://reader035.fdocuments.us/reader035/viewer/2022062315/56814fc8550346895dbd84a9/html5/thumbnails/3.jpg)
3
GoldenGate Veridata
› Data comparison tool
› High performance
› Cross platform
› Checking between divergent databases
› Repair out-of-sync data
19/08/2014 Anti Asko
![Page 4: Veridata for Data Consistency in WLCG](https://reader035.fdocuments.us/reader035/viewer/2022062315/56814fc8550346895dbd84a9/html5/thumbnails/4.jpg)
4
GoldenGate Veridata
19/08/2014 Anti Asko
![Page 5: Veridata for Data Consistency in WLCG](https://reader035.fdocuments.us/reader035/viewer/2022062315/56814fc8550346895dbd84a9/html5/thumbnails/5.jpg)
5
Evaluation Process
› Test Process Mock data generated
Replicated from Cern to Tier-1s
Check data consistency with
Oracle GoldenGate Veridata
19/08/2014 Anti Asko
![Page 6: Veridata for Data Consistency in WLCG](https://reader035.fdocuments.us/reader035/viewer/2022062315/56814fc8550346895dbd84a9/html5/thumbnails/6.jpg)
6
Evaluation Process
› Real Production Data Atlas data through Tier-1s Check the consistency with
Oracle GoldenGate Veridata
› More than 1.23 TB of data checked
› More than 21.000 tables were processed
19/08/2014 Anti Asko
![Page 7: Veridata for Data Consistency in WLCG](https://reader035.fdocuments.us/reader035/viewer/2022062315/56814fc8550346895dbd84a9/html5/thumbnails/7.jpg)
7
Conclusions
› Powerful tool for data comparison and repair
› Trustworthy for the WLCG’s rapidly adjusted environment
› A tool IT-DB needs
19/08/2014 Anti Asko
![Page 8: Veridata for Data Consistency in WLCG](https://reader035.fdocuments.us/reader035/viewer/2022062315/56814fc8550346895dbd84a9/html5/thumbnails/8.jpg)
Anti Asko 8
THANK YOU!
19/08/2014