Genomes On Rails
-
Upload
matt-wood -
Category
Technology
-
view
3.714 -
download
0
description
Transcript of Genomes On Rails
![Page 1: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/1.jpg)
Genomes on Railshas_many :sequences
![Page 2: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/2.jpg)
Hello
![Page 3: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/3.jpg)
➊ Previously
➋ Production
➌ Process
![Page 4: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/4.jpg)
➊ Previously
![Page 5: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/5.jpg)
The human genome
15 years to decode
3 billion letters
![Page 6: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/6.jpg)
$3 billion
![Page 7: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/7.jpg)
$3 billion ++
![Page 8: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/8.jpg)
Race for the prize
![Page 9: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/9.jpg)
![Page 10: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/10.jpg)
![Page 11: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/11.jpg)
Open data
![Page 12: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/12.jpg)
Open source
![Page 13: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/13.jpg)
Perl
![Page 14: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/14.jpg)
Lots of Perl
![Page 15: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/15.jpg)
Lots of Perl~4500 modules
![Page 16: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/16.jpg)
Onwards!
![Page 17: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/17.jpg)
40 species
![Page 18: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/18.jpg)
![Page 19: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/19.jpg)
![Page 20: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/20.jpg)
![Page 21: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/21.jpg)
Map evolutionaryspace
![Page 22: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/22.jpg)
Compare genomes
![Page 23: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/23.jpg)
Compare genomes
compare species
![Page 24: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/24.jpg)
Compare genomes
compare species
compare individuals
![Page 25: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/25.jpg)
More Perl~1500 modules
![Page 26: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/26.jpg)
![Page 27: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/27.jpg)
![Page 28: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/28.jpg)
![Page 29: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/29.jpg)
Quantum leap!
![Page 30: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/30.jpg)
1000 personal genomes
![Page 31: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/31.jpg)
1000 personal genomes
beyond 23andme
![Page 32: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/32.jpg)
Hypertension
![Page 33: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/33.jpg)
Diabetes
![Page 34: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/34.jpg)
Coronary heart disease
![Page 35: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/35.jpg)
Bipolar disorder
![Page 36: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/36.jpg)
Malaria
![Page 37: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/37.jpg)
➋ Production
![Page 38: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/38.jpg)
Register projects
Register samples
Sample prep
Sequencing
Analysis
![Page 39: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/39.jpg)
![Page 40: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/40.jpg)
![Page 41: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/41.jpg)
![Page 42: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/42.jpg)
Change!
![Page 43: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/43.jpg)
Flexible data capture
![Page 44: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/44.jpg)
Virtual fields
![Page 45: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/45.jpg)
Sample
Name
Organism
Concentration
![Page 46: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/46.jpg)
class Sample < ActiveRecord::Base has_many :descriptors has_many :descriptor_valuesend
![Page 47: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/47.jpg)
Key value pairs
![Page 48: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/48.jpg)
Faster than you’d think
![Page 49: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/49.jpg)
![Page 50: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/50.jpg)
![Page 51: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/51.jpg)
Change!
![Page 52: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/52.jpg)
Sample
Name
Organism
Concentration
Sample
Name
Organism
Concentration
Origin
Quality metric
V1 V2
![Page 53: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/53.jpg)
![Page 54: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/54.jpg)
![Page 55: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/55.jpg)
Rationalize!
![Page 56: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/56.jpg)
Sample
Name
Organism
Concentration
Sample
Name
Organism
Concentration
Origin
Quality metric
V1 V2
![Page 57: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/57.jpg)
Mapping!
![Page 58: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/58.jpg)
Sample
Name
Organism
Concentration
Sample
Name
Species
Concentration
Origin
Quality metric
V1 V3
Origin
![Page 59: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/59.jpg)
Pipeline management
![Page 60: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/60.jpg)
Task 1 Task 2 Task 3
Workflow
Name
Operator
Instrument
Name
Serial number
Kit
Name
Passed
![Page 61: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/61.jpg)
![Page 62: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/62.jpg)
![Page 63: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/63.jpg)
![Page 64: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/64.jpg)
Throughput!
![Page 65: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/65.jpg)
![Page 66: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/66.jpg)
320Tb 450 CPU
![Page 67: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/67.jpg)
320Tb 450 CPU Archive
![Page 68: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/68.jpg)
75Tb
![Page 69: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/69.jpg)
![Page 70: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/70.jpg)
![Page 71: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/71.jpg)
![Page 72: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/72.jpg)
![Page 73: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/73.jpg)
pilot study!
![Page 74: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/74.jpg)
Multiple apps
![Page 75: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/75.jpg)
Multiple instances
![Page 76: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/76.jpg)
Loosely coupled
![Page 77: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/77.jpg)
Loose coupling is hard
![Page 78: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/78.jpg)
Deployment
![Page 79: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/79.jpg)
Maintenance
![Page 80: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/80.jpg)
Monitoring
![Page 81: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/81.jpg)
Hard to maintain separation
![Page 82: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/82.jpg)
Support novel science
![Page 83: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/83.jpg)
Single code base
![Page 84: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/84.jpg)
nginx reverse proxy
![Page 85: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/85.jpg)
fairnginx
![Page 86: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/86.jpg)
Mongrel
![Page 87: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/87.jpg)
Fast deployment
![Page 88: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/88.jpg)
Automate everything
![Page 89: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/89.jpg)
![Page 90: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/90.jpg)
Interoperability!
Play well with others!
![Page 91: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/91.jpg)
Legacy databases
![Page 92: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/92.jpg)
RESTful services
![Page 93: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/93.jpg)
Generate API stubs
![Page 94: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/94.jpg)
![Page 95: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/95.jpg)
SCALE!
![Page 96: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/96.jpg)
Trillionics
![Page 97: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/97.jpg)
2X
![Page 98: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/98.jpg)
150Tb per week
![Page 99: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/99.jpg)
Over 6 months
![Page 100: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/100.jpg)
More hardware
![Page 101: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/101.jpg)
400 additional nodes
![Page 102: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/102.jpg)
additional 360 Tb
![Page 103: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/103.jpg)
Towards a Virtual Institute
![Page 104: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/104.jpg)
Lots of data
![Page 105: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/105.jpg)
Lots of data, lots of people
![Page 106: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/106.jpg)
Lots of data, lots of people, lots of compute
![Page 107: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/107.jpg)
Lots of data, lots of people, lots of compute,
lots of uses
![Page 108: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/108.jpg)
Lots of data, lots of people, lots of compute, lots of uses, lots and lots
and lots and lots...
![Page 109: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/109.jpg)
➌ Process
![Page 110: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/110.jpg)
Concept Requirements Development Product
![Page 111: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/111.jpg)
Concept Requirements Development Product
takes too long
![Page 112: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/112.jpg)
RequirementsConcept Development Product
these change
takes too long
![Page 113: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/113.jpg)
Concept
What we need Get ready
DevelopmentPlan
REVIEW
![Page 114: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/114.jpg)
Focused
![Page 115: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/115.jpg)
Project owner is key
![Page 116: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/116.jpg)
Weekly releases
![Page 117: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/117.jpg)
More flexible
![Page 118: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/118.jpg)
Less time
![Page 119: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/119.jpg)
Better transparency
![Page 120: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/120.jpg)
Less software
![Page 121: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/121.jpg)
Sequencing informatics
![Page 122: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/122.jpg)
Thank you
![Page 123: Genomes On Rails](https://reader036.fdocuments.us/reader036/viewer/2022062704/556132f9d8b42a92358b470f/html5/thumbnails/123.jpg)
GREENISGOOD.CO.UK