National Aeronautics and Space Administration

Cloud Launch

1

The NASA Nebula Project

Chris C. KempChief Technology Officer for IT

National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 2

National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 3

National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 4

• As a result of NASA’s diverse mission and distributed workforce (across centers, labs, and universities), we have a uniquely balkanized – and expensive - IT environment.

• Typical Enterprise IT infrastructure is only utilized to 5-20% of its capacity1

• NASA’s Supercomputers are also only available to the largest projects, with small projects often waiting many weeks in queue… many jobs do not require HPC.

• Thousands of new computers are inefficiently procured and operated each year, continuing the cycle. This process often takes many months and costs more in overhead than the actual value of the computers.

1 Source: http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf

NASA’s IT Challenges

National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 5

National Aeronautics and Space Administration 6

National Aeronautics and Space Administration 7

National Aeronautics and Space Administration 8

National Aeronautics and Space Administration 9

REST, CLI, Web Interfaces

National Aeronautics and Space Administration 10

WISE in the Cloud

124

Nebula Users by Location

7

12

2

5

4

9109

6

11

National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 11

Wide-Field Infrared Survey Explorer (WISE)

Science 8X sky survey• Find the most luminous galaxies in the

universe• Find the closest stars to the sun

Primary Data Products• WISE Image Atlas

10,464 calibrated FITS image sets (4 bands/set), 4kx4k pix @1.375”/pixFormed by combining all single exposures covering Atlas Tile footprintPixel depth-of-coverage, uncertainty maps and metadata for each image

• WISE Source CatalogJ2000 positions, calibrated 4-band photometry, quality and value-added flags and parameters for over 200 million objects in the release area

National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 12

Angular resolution processing.

Processing to resolve and measure source sizes.

#1 1 1 100

1 1 1 2000

1 1 1 100,000

1 80 100 10

1 1 1 100

1 1 1 41,253

Processing to differentiate point-like emission from distributed emission.

Surveying for distributed star formations in our galaxy.

A few large regions can be processed to tremendous depth.

High resolution processing of the entire sky.

WISE in the Cloud

National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 13

Air Traffic Control Algorithm

• An Ames researcher developed an algorithm to produce a useful result from standard ATC data.

• Took 20 hrs on Mac laptop for each airport-day (or 83% of real time)

• Nebula user asked to re-create researcher’s prototype and run a year’s worth of data.

• Initial results: 10241 site-hours (about 1/7 year, first of six phases) in 17 compute hours• or ~85 times faster than the Mac

Laptop• Using 4 tiny instances

National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 14

SERVIR and SPoRT

Science• Numerical Weather Prediction• High-Res Satellite Imagery Processing

Primary Data Products• Rapid Weather Modeling

Shared resources for high resolution processing or large forecast domain.Rapid responses to new events or research opportunities without impacting other resources.

• Satellite ImagesProcess and distribute high resolution images on demand using web-driven, scalable architecture capable of processing large datasets.

National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 15

1 Source: http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf

User Notes

“After having access to Nebula for only a few days, I find that I am already able to accomplish more data-intensive calculations than I can do on any of the local servers we have here, and with no difficulties at all!”

“The system is easy to access and use, and offers a capability that I absolutely need occasionally but for which I could never justify the expense if it were for my needs alone.”

“With the recent addition of a large-RAM instance, I am now able to conduct calculations that could not be done on our project's large server farm.  Nebula has provided me with a tool for science data analysis that far surpasses anything that I could envision in a single-user context.  NASA Cloud computing may be the way forward for our data-intensive projects in the future, since only a NASA system could provide the necessary reliability and proprietary controls on our data.”

National Aeronautics and Space Administration

Nebula’s Computing Engine, NOVA, is the foundation of OpenStack Compute.

OpenStack is to the datacenter what Linux is to the personal computer.

An open source cloud stack that enables any organization to build large scale private cloud services on commodity hardware.

Standard Apache 2.0 license

National Aeronautics and Space Administration 16

National Aeronautics and Space Administration

600 developers,100 companies,12 countries, 6 months

National Aeronautics and Space Administration 18

Chris C. KempChief Technology Officer for IT

chris.c.kemp@nasa.gov

@kemp