Unlocking the full potential of data storage technologies

11
Data Storage Technologies: A Wealth of Potential! 1 Brent Maxwell Head of Data and Intelligence The Iconic

Transcript of Unlocking the full potential of data storage technologies

Page 1: Unlocking the full potential of data storage technologies

1

Data Storage Technologies:

A Wealth of Potential!

Brent MaxwellHead of Data and Intelligence

The Iconic

Page 2: Unlocking the full potential of data storage technologies

2

Key topics

• Examining when and how to apply new data warehousing technologies

• Creating fast and convenient access to all of your data for advanced analytics

• Taking advantage of high performance data warehousing technologies to personalise the customer experience

Page 3: Unlocking the full potential of data storage technologies

3

THE ICONIC

• Founded in 2011• Leading the Australian online fashion market• Humongous growth in sales and the hearts of

Australians everywhere• 70% YOY data growth• Gone through 2 data warehouses• Gone through 3 BI tools

Page 4: Unlocking the full potential of data storage technologies

4

Data warehousing: context

What is notable about our architecture?Hint: The Operational Data Store (ODS) is in the

wrong place!!!…or is it?

Page 5: Unlocking the full potential of data storage technologies

5

What we’re doing that’s awesome

• Near-real-time data• Ultra-low maintenance• Self-documenting data warehouse

Page 6: Unlocking the full potential of data storage technologies

6

What does it mean for the business?

• Customer Service have rich data• Web/App Catalogue Sorting adapts to

events• Further Personalisation

Page 7: Unlocking the full potential of data storage technologies

7

The cloud is the way forward

Until recently it hasn’t been practical. What’s changed?• Source systems are in the cloud• Feature sets have been refined• The technologies have stabilized

It’s a good time to move your DWH to the cloud!

Page 8: Unlocking the full potential of data storage technologies

8

How did we do it?

Page 9: Unlocking the full potential of data storage technologies

9

How did we do it?

• Near-real-time processing where desired

• Test-driven database development• Dynamic schema creation• Hybrid Slowly Changing Dimensions• Re-runnable data loads• Cloud-based logging

Page 10: Unlocking the full potential of data storage technologies

10

What next?!?

• Continuous Integration• Just need to build the artefact!

• Improved Job Orchestration• Control-M• Rundeck• Any recommendations?

Page 11: Unlocking the full potential of data storage technologies

11

Thank you!

Brent MaxwellHead of Data and Intelligence

THE ICONIC

Contact me at [email protected]