D Baker - Galaxy Update
-
Upload
jan-aerts -
Category
Technology
-
view
435 -
download
0
description
Transcript of D Baker - Galaxy Update
![Page 1: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/1.jpg)
Galaxy Update
Dannon BakerGalaxy Team
Emory University
![Page 2: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/2.jpg)
• API
• Automatic Parallelization
• Tool Shed
• ?
Topics
![Page 3: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/3.jpg)
API Overview
• RESTful API supporting CRUD operations• Uses generated keys for per-user
authentication
• No username/password
• No credential caching (via keys)
• Request parameters and responses are in JSON (JavaScript Object Notation)
![Page 4: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/4.jpg)
Quickstart
![Page 5: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/5.jpg)
Raw GET Example
» GET /api/histories?key=966354fc14c9e427cee380ef50a72a21
« [
« {
« 'url': '/api/histories/d0bfe935d0f5258d',
« 'id': 'd0bfe935d0f5258d',
« 'name': 'Demo History 1'
« }
« ]
![Page 6: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/6.jpg)
Wrapper methods exist (in /scripts/api/) to make calls easier:
python scripts/api/{action}.py <api key> http://<ip>/api/{module}/[id] [args] action: create | display | update | delete
api_key: obtained from the UI
module: datasets | forms | histories | libraries | permissions | quotas | requests | roles | samples | tools | users | visualizations | workflows unit: dataset_id / history_id / library_id / …
args: name / key-value pair / …
Making the Calls
![Page 7: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/7.jpg)
Create a history./create.py <api_key> https://localhost:8080/api/histories name="from API”
Display all histories./display.py <api_key> https://localhost:8080/api/histories
Display information about a history./display.py <api_key> https://localhost:8080/api/histories/6b5cf7e7ef797b21
View datasets in a given history./display.py <api_key> https://localhost:8080/api/histories/6b5cf7e7ef797b21/contents
Delete a history./delete.py <api_key> https://localhost:8080/api/histories/976a9ce09b49502a
Sample Invocations
![Page 8: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/8.jpg)
End-to-end pipelines
• Push data from instruments to Galaxy
• Execute a workflow
• Associate with a user or add to a Data Library
• Export outputs
• Or a totally custom interface to Galaxy
![Page 9: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/9.jpg)
Automatic Parallelism
BLAST/strip?input output
![Page 10: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/10.jpg)
Parallelism
• Take maximum advantage of available resources
• Less costly fault recovery (spot instances)
• Overhead in splitting time
• Increased temporary storage requirement
![Page 11: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/11.jpg)
Use it now
use_tasked_jobs = True
Tools supported:
BLAST, BWA, Bowtie
Yours?
![Page 12: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/12.jpg)
Try it for your tool<parallelism
method = "multi”
split_inputs = "query"
split_mode = "number_of_parts”
split_size = "4"
shared_inputs = "subject"
merge_outputs = "output1”
/>
![Page 13: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/13.jpg)
Advanced Splitting • FUSE
• No disk write required
• But it is slightly slower to read
• More splitters:
• Chromosome based?
![Page 14: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/14.jpg)
1 2 3 ∞
http://usegalaxy.org
http://usegalaxy.org/community
Galaxy Tool Galaxy Tool ShedShed
...
Galaxies on private clouds
Galaxies on public clouds
...
private Galaxy installations
priv
ate
Tool S
hed
s
![Page 15: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/15.jpg)
Tool Shed - Developer
![Page 16: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/16.jpg)
Tool Shed - Developer
![Page 17: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/17.jpg)
Tool Shed - User
![Page 18: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/18.jpg)
Tool Shed
![Page 19: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/19.jpg)
Tool Shed
![Page 20: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/20.jpg)
Tool Shed - User
![Page 21: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/21.jpg)
Tool Shed
• Simple installation to galaxy of tools, workflows
• Dependencies
• Automatic update notifications
• Multiple Tool Versions installed
![Page 22: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/22.jpg)
Tool Shed References
• Main Tool Shed: usegalaxy.org/toolshed
• ISMB Tech Track w/ Greg – Sunday, TT10
![Page 23: D Baker - Galaxy Update](https://reader031.fdocuments.us/reader031/viewer/2022013101/554e8b03b4c905fc368b48d5/html5/thumbnails/23.jpg)
if more_time:• S3ObjectStore
• Documentation
• <galaxy_dir>/doc – `make html`
• Join us in #galaxyproject on irc.freenode.net
• Acknowledgements:
• Galaxy Team, Community, people who send pull requests