Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04...
Transcript of Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04...
![Page 1: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/1.jpg)
NASA EOSDIS Cloud Prototype Systems
![Page 2: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/2.jpg)
2
Annual distribution is on the same order of magnitude as the total archive volume
![Page 3: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/3.jpg)
3
And our archive is scheduled to grow dramatically
You are here
![Page 4: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/4.jpg)
4
If these high volumes are a challenge for robust, experienced data centers to manage...
...what about the science end users?
![Page 5: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/5.jpg)
5
Conceptual “Data Close to Compute”
Applications using cloud compute, storage, and services
EOSDIS Applications
Science community brings algorithms to the data.
Community Applications
Archive
Catalog
Search
Ingest Access
Analytics
Processing Application
Data
Datasets stored in cloud object storage
Large Volume Data Storage
Scalable ComputeProvision, use, and terminate as needed
![Page 6: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/6.jpg)
6
Conceptual “Data Close to Compute”
Applications using cloud compute,
storage, and services
EOSDIS Applications
Science community brings algorithms to the data.
Community Applications
Cumulus
Common Metadata
RepositoryEarthdata
Search
Global Imagery Browse Service
Web Object
Storage
NEXUSGiovanniCAaaS*
Getting Ready for NISAR
AppEEARS
Data
Datasets stored in cloud object storage
Large Volume Data Storage
Scalable ComputeProvision, use, and terminate as needed
*CAaaS = Climate Analytics as a Service
NASA-compliant General Application Platform
![Page 7: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/7.jpg)
7
Specific Benefits
Cumulus
Common Metadata
RepositoryEarthdata
Search
Global Imagery Browse Service
Web Object
Storage
NEXUSGiovanni
CAaaS
Getting Ready for NISAR
AppEEARS
Data
On-demand regeneration of virtual products
Acceleration of end-user analysis
Acceleration of OPeNDAP access to data
High-bandwidth edge server for radar data
Serverless architecture demonstration
Improvement of ingest performance
Avoidance of expensive hardware purchase
First public EOSDIS application in the cloud
NASA-compliant General Application Platform
Approval to operate publicly
![Page 8: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/8.jpg)
8
NASA-compliant General Application Platform: “Compliance-as-a-Service”
8
Infrastructure-as-a-Service
Platform-as-a-Service
Compliance-as-a-Servicesecurity controls, Authorization to Operate
governanceprocurement
reliability and availability
Software-as-a-Service
![Page 9: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/9.jpg)
9
Key Cloud Benefits for Science Users
Immediate direct access to dataImmediate availability of compute
Dynamic scaling for bursting Variety of CPU and GPU
Purchase compute as neededSpot market for cost control No need for local copy of data
Decrease in startup time for projectsAnalysis at scale for anyoneRapid iteration enabled
Lower Barrier to Entry
Scalable Performance
Cost
Repeatability & Sharing
Promoting Science Discovery
Experiment replication enabledSharing of complex analysis workflows
![Page 10: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/10.jpg)
10
Key Cloud Benefits for Data Systems (EOSDIS)
Costs only for the compute and storage used Separate accounting for project/function
Auto scaling, elastic load balancingAdjustable duty cycle
Storage type, CPU sizeOperating system, languages
Data next to computeAll data in the same virtual “place”
Cost-Effective
Scalable Performance
Flexibility
Deployment Speed
Accessibility
Simplified procurement + provisioningDevOps automation
![Page 11: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/11.jpg)
11
1
Decision Considerations
High level decision consideration for individual project prototypes and capabilities to operationalize into AWS (commercial cloud)
01
02
03
04
Cost
IT Security
Performance
Operational
Is AWS (commercial cloud) affordable?.
Is NASA IT Security compliance and tactical operations achievable in AWS (commercial cloud)?
Is performance equal to or better than current on-premises solutions?
Can we operate “Operationally” in AWS (commercial cloud), technical and business?
![Page 12: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/12.jpg)
12
•
Most Obvious-yet-surprising Lesson:Savings from Test Environments
![Page 13: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/13.jpg)
13
••
Most Obvious-yet-surprising Lesson:Savings from Test Environments
![Page 14: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/14.jpg)
14
•••
Most Obvious-yet-surprising Lesson:Savings from Test Environments
35% Savings from just the Test Environment!
![Page 15: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/15.jpg)
15
•–
–
–
•
–
Most Important Lesson: Cloud-Native Architecture
![Page 16: Cloud Prototype Systems NASA EOSDISceos.org/document_management/Working_Groups/WGISS...2017/04/04 · 10 Key Cloud Benefits for Data Systems (EOSDIS) Costs only for the compute and](https://reader035.fdocuments.us/reader035/viewer/2022071018/5fd2312ea2dc2f1f7b1b1baa/html5/thumbnails/16.jpg)
16
•–
•––
•––
Cloud Architecture Game Changers