Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205
-
Upload
martin-hamilton -
Category
Technology
-
view
630 -
download
1
Transcript of Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205
15/04/2023
Cloud for Research and Innovation - UK USA HPC Workshop July 2015 1
Cloud for Research and InnovationMartin Hamilton
Photo CC BY Flickr user atosorigin
Cloud for research and innovation
1. Cloud adoption in enterprise IT– Trends in UK higher education– Bimodal IT versus bipolar institutions
2. Public clouds are starting to look more like HPC clusters– Orchestration, Infiniband, GPU and FPGA availability (at a price)
3. Code re-use and app portability– Will Docker and OpenStack be our Rosetta Stone(s)?– What does a unified computing model look like?
15/04/2023
Cloud for Research and Innovation - UK USA HPC Workshop July 2015 3
1. Cloud adoptionin enterprise IT
Cloud in UK higher education
Collaboration suites: Google Apps for Education» The University of Westminster estimates that it saved £1m by moving to
Google Apps for Education for email and other online collaboration facilities. They also state that “an additional benefit has been the reduced time spent in systems and user support with a minimal number of calls for support for such a significant system”, and that moving to the cloud has “liberated staff and students from the smaller storage limits of the previous in-house solution.”
http://services.google.com/fh/files/misc/univofwestminster.pdf
» Ealing, Hammersmith and West London College have also migrated to Google Apps. They observe that “each year, 20,000 new accounts are created providing 600 terabytes (around 150,000 DVDs worth) of storage in less than five minutes.” Subsequently to this article being written, Google moved to offering free unlimited storage.
http://www.wlc.ac.uk/college/news/article.asp?newsID=4228
Cloud in UK higher educationCollaboration suites: Microsoft Office365» The University of the West of London moved its 14,000 students to Office365, and
observes that once a student graduates “they can maintain that Office365 account through the University for life. We can continue to supply them with information about the University, help them find their second job, their third job and so on. This isn’t just about collaboration while studying, it’s about creating a life-long connection between the students and the University.”
http://www.slideshare.net/Microsofteduk/university-of-west-london-case-study-office-365
» The University of Dundee migrated all of its 22,000 student accounts to Microsoft’s Office365 for Education service over a week. They note that this was achieved with zero down time, and estimate that Office365 “will save us at least £500,000 in infrastructure and staffing over five years.”
http://www.slideshare.net/Microsofteduk/university-of-dundee-case-study-office-365
Cloud in UK higher education
Jisc Amazon cloud portal:› Monthly invoicing - credit cards are no
longer required for payment
› Itemized billing - consolidated across users/departments
› Billing in GBP, not dollars
› Setting of budget limits for individual user accounts or departments
› The retrieval of service usage information within own areas of responsibility
› Volume-discounts through aggregation across multiple educational institutions
Cloud in UK higher education
Leveraging the Janet network:
Cloud in UK higher education
Private peerings:
Cloud in UK higher education
Private peerings:
Cloud in UK higher education
Dogfooding:› We moved
our own core infrastructure to AWS
› Estimate that this saved £100K p.a.
› Cloud first / mobile first for new stuff
› NB we are not an HPC provider
15/04/2023
Cloud for Research and Innovation - UK USA HPC Workshop July 2015 12
2. Public clouds arestarting to look like HPC clusters
Case study: Azure HPC
Case study: Azure HPC
Case study: Azure HPC
Case study: Azure HPC
http://azure.microsoft.com/blog/2015/07/09/azure-linux-rdma-hpc-available/
Case study: Azure HPC
Case study: Azure HPC
Case study: Azure HPC
Case study: Azure HPC
Case study: Google Compute Engine
Case study: AWS
Case study: AWS
Caveat Emptor: Magellan
http://science.energy.gov/~/media/ascr/pdf/program-documents/docs/Magellan_Final_Report.pdf
Caveat Emptor: Magellan
http://science.energy.gov/~/media/ascr/pdf/program-documents/docs/Magellan_Final_Report.pdf
15/04/2023
Cloud for Research and Innovation - UK USA HPC Workshop July 2015 26
3. Code re-use &application portability
Re-use & portability: OpenStack
http://www.datacenterknowledge.com/archives/2014/11/07/cerns-openstack-cloud-to-reach-150000-cores-by-2015/
Re-use & portability: OpenStack
http://www.datacenterknowledge.com/archives/2014/11/07/cerns-openstack-cloud-to-reach-150000-cores-by-2015/
http://www.datanami.com/2013/09/23/cern_turns_to_google_for_datacenter_direction/
Re-use & portability: OpenStack
Re-use & portability: OpenStack
Re-use & portability: Containers
Re-use & portability: Containers
https://wiki.openstack.org/wiki/Docker
Re-use & portability: Performance
http://en.community.dell.com/techcenter/high-performance-computing/b/general_hpc/archive/2014/11/04/containers-docker-virtual-machines-and-hpc
Re-use & portability: Open Container Project
Re-use & app portability: Kubernetes & Heat
http://kubernetes.io http://techs.enovance.com/7104/multi-tenant-docker-with-openstack-heat
Re-use & app portability: A unified model?
This is one model. Other models are available › Run OpenStack on your in-house facility [Q: Still need one?]
– This creates your own private/community cloud– Machine images can be programmatically converted to/from QCOW2
› Package your app with Docker and you can run it anywhere– Embraced by public cloud providers, can run in-house too as open source– Could be bare metal or on top of e.g. OpenStack [Q: Performance?]
› Lots of options for orchestration from general purpose uses– Kubernetes scheduling particularly interesting from an HPC perspective
› Open questions from an e-Infrastructure perspective:– Integrating identity management, authentication, authorization, accounting– User driven (instant gratification) versus controlled environment– Funding model for use of public and community clouds, potential for national deals– Lots of new stuff to learn about! (for developer, sysadmin, …)– That’s all fine for the compute, but what about the data?
Photo CC-BY Flickruser codep8
15/04/2023
Cloud for Research and Innovation - UK USA HPC Workshop July 2015 37
That’s all folks…
Except where otherwise noted, this work is licensed under CC-BY
Martin HamiltonFuturist, Jisc, London