Post on 01-Aug-2020
GPU Computing with Amazon EC2
Deepak Singh@mndoci
P r i n c i p a l P r o d u c t M a n a g e r
Amazon Web Services
4
2
1. Infrastructure
ec2-run-instances
secure
global
on demand
programmable
elastic
instance types
standard (m1)high memory (m2)
high CPU (c1)
t1.micro
high performance
“Our 40-instance (m2.2xlarge) cluster can scan, filter, and aggregate 1 billion rows in 950 milliseconds.”
Mike Driscoll - Metamarkets
cluster computing
MPI
bandwidth intensive
Cluster Compute Instance
2*Intel Xeon 55708 cores w/HT23 GB RAM1.7 TB disk
HVMcc1.4xlarge
10 gig E
PlacementGroup
Placementgroup
full-
bisection
linpack
Cores 7040
Rmax 41.82
Rpeak 82.51
231
November 2010
451
June 2011
42
November 2011
HPC is evolving
2*Intel Xeon 557022 GB RAM1.7 TB disk
HVM2*NVidia M2050
AMBER
JAC NVE: 46.92 ns/day
Factor IX: 13.49 ns/day
Cellulose NVE: 3.07 ns/day
optimizing costs
on-demand
reserved
spot
30,472 cores
$1279/hr
2. Orchestration
AWS CloudFormation
bootstrap
Cloud Init
#cloud-configpackages:! - httpd!runcmd:! - /etc/init.d http start! - echo "<h1>hello, world"</h1> \! ! > /var/www/html/index.html
#!/bin/shec2-run-instances ami-8c1fece5 \! -n 1 \! -t m1.small \! -g deesinghdemo-SG \! -k deesinghdemo-keypair \! --user-data-file \.\cloudconfig.txt
chef/puppet
familiar tools
LSF
Grid Engine
Bright Cluster Manager
combining worlds
MIT Starcluster
$ starcluster start mycluster$ starcluster listclusters
http://www.bioteam.net/2011/03/dude-you-got-some-chef-in-my-starcluster/
Provisions ClusterShared Storage
MonitoringBootstraps StarCluster
Includes 200 GB Public Dataset
Provisioned Stack = Submit jobs to Grid Engine