PyCon 2012 - Data Driven Design

22
Data Driven Design Build app design around data flow rather than program flow Max Klymyshyn CTO at GVMachines tweet tweet @maxmaxmaxmax

description

 

Transcript of PyCon 2012 - Data Driven Design

Page 1: PyCon 2012 -  Data Driven Design

Data Driven DesignBuild app design around dataflow rather than program flow

Max KlymyshynCTO at GVMachines

tweet tweet @maxmaxmaxmax

Page 2: PyCon 2012 -  Data Driven Design

What's the problem?

Handling many of different data sources and keep context

Page 3: PyCon 2012 -  Data Driven Design

Typical task for this approach

Random picture from Google

Page 4: PyCon 2012 -  Data Driven Design

Yup, honestly it's depend on the task.

There's a lot of examples of data-driven programming around.

The most known example isDjango Middleware

Really?

Page 5: PyCon 2012 -  Data Driven Design

So, let's go deeper

Page 6: PyCon 2012 -  Data Driven Design

Data Drivenprogramming

definition

Page 7: PyCon 2012 -  Data Driven Design

Data driven programming is a programming model where

the data itself controls the flow of the program and not the program logic

What is Data Driven programming

Page 8: PyCon 2012 -  Data Driven Design

Data Driven definition is quite strict.

Typically it's mixed with other approaches

Not so strict

Page 9: PyCon 2012 -  Data Driven Design

Real world tasksample usage

Page 10: PyCon 2012 -  Data Driven Design

I want to grab weather from different sources and display only sunny days of the month

Show only sunny days in the month

Page 11: PyCon 2012 -  Data Driven Design

To grab data we should define pipeline - the way our data going to go

Pipeline

Page 12: PyCon 2012 -  Data Driven Design

Pipeline

Grab Parse Validate Display

Page 13: PyCon 2012 -  Data Driven Design

In detailsGrab

Parse

Fetch data from weather.com

Fetch data from pogoda.yandex.ua

Fetch data from weather.yahooapis.com

Parse weather.com

Parse pogoda.yandex.ua

Parase api.aerisapi.com

Validate Pass sunny days only

Page 14: PyCon 2012 -  Data Driven Design

At this moment we have only sunny days to display

Pipeline

Page 15: PyCon 2012 -  Data Driven Design

Approacheswhen this may be be effective?

Page 16: PyCon 2012 -  Data Driven Design

ifttt.comifttt is a web service platform that connects various web services together to automate common tasks on the web

If this then that

Page 17: PyCon 2012 -  Data Driven Design

When you have a lot of mostly similar data from different sources

You need to keep only data which matters for you

Various sources

Page 18: PyCon 2012 -  Data Driven Design

In case you need to generate a lot of similar data sets filtered by many of params

Data slices generation

Page 19: PyCon 2012 -  Data Driven Design

In general, approach is the same as usage of UNIX pipes:

cat file.dat | grep something | sed 's/xxx/yyy/g'

Pipes

Page 20: PyCon 2012 -  Data Driven Design

Summary

Grab Parse Validate Display

Page 21: PyCon 2012 -  Data Driven Design

https://github.com/joymax/data-driven-design

Example

Page 22: PyCon 2012 -  Data Driven Design

That's all, thank you.

Questions?

tweet tweet @maxmaxmaxmax

Github: joymax