Post on 26-Jun-2020
OpenStack Implementation: A case study of Chula eScience and Computer Engineering Krerk Piromsopa, Ph.D. Computer Engineering, Chulalongkorn Bangkok, Thailand
Outline
OpenStack
eScience
Requirements
Design Decisions
Implementation
Openstack
OpenStack Installation
Many Components
A little error, it failed.
Take days (if not months) to manage and to install.
eScience
eScience is computationally intensive science that is carried out in highly distributed network environments, or science that uses immense data sets that require grid computing; the term sometimes includes technologies that enable distributed collaboration, such as the Access Grid [wikipedia]
eScience @ Chula / ThailandeScience @ Chulalongkorn Univeristy/Thailand
High-Energy (Particle) Physics (Chula + European Organization for Nuclear Research / CERN)
Climate Changes
Water Resources
etc.
Requirements (physics + Computer Engineering)
eScience - dedicate resources + on-demand projects
Students - Quota per student(300-400 active students.)
Accounts
Students - Use username and password from existing university accounts.
Others - Users for each project
Our HardwareDisk RAM CPUs(TB) (GB) (GHz)
DELL PowerEdge R630 U20
1.8 64 40x3LENOVO 3550 M5 1.1 32 40x3.4DELL PowerEdge R430 3.6 64 32x3.4LENOVO SR850 0.13 318.4 174x2.1IBM 3755 M3 1 29.3 30x3DELL PowerEdge R630 U21
1.8 64 40x3
IBM iDataPlex DX360M4 0.5 48 169.93 619.7 372
More hardware are coming.
Other constraints
Our eScience storage system is based on IBM Spectrum Scale (aka. GPFS). This also is used for storing data from CMS/LHC experiment (running at CERN)
The physics analysis should not be stopped.
Few (if any) administrator… (we aim for zero administration)
Security - (No direct/public access to students’ VMs)
Design DecisionsTry to automate/streamline process as much as possible.
Implement OpenStack (compute nodes and storage nodes) on top of existing hardware and existing filesystem (GPFS).
Allow additional hardware to be added later
Use LDAP with additional mechanism to map user from university’s LDAP.
Automatically create a project for each user (student) on sign up (account mapping).
Design Decisions
The VM can only be accessed though our SSH gateway.
Students should use standard tools and native OpenStack tools (eg. Horizon) for managing and accessing their VM. - part of the learning experiences.
OpenStack Installation - Fuel
https://www.slideshare.net/justiceform/case-study-utilizing-mirantis-fuel-to-install-openstack-ansible
Fuel
Cinder
Architecture eScience
Node
GPFS
Compute Node (KVM)
eScience Node
Compute Node (KVM)
Compute Node
Keystone
LDAPLdapMap
Univ. LDAP
Controller
Project DB
Project Creator
LdapMap
Univ. LDAP
local LDAP
Java App
Project Creator
local LDAP
Controller
Project Creator Use local database to automatically create project for each user.
Python Script
Horizon
Management
OpenStack.cp.eng.
chula.ac.th
SSH Gateway
Horizon
LdapMAP
SSH, HTTPS
reverse proxy, port map
Security
Use gateway with LDAP accounts
Gateway is also a VM scaling on demand.
Security
Use cloud-init images for security
only accessible via initial private key.
No public IP for general VMs.
Only ip (port/host) forwarding using iptables.
Our tricksUse wiki for knowledge management/user training
Automate SSH tunneling through ProxyCommand
~/.ssh/config
Students feel like having direct access to VM.
ssh -L80:localhost:80 myHost.eScience
~/.ssh/config
Lessons Learned
Wiki allows users to help themselves.
A project per user makes it easy to control.
Use KVM over KVM as compute node for resource isolation.
Next Step
Allow containers/Hadoop/Spark cluster to be run on the same infrastructures.
Implement efficient monitoring systems.
Dynamic scale
Will deprecate Fuel
Thank you Q&A
Join us in AINTEC2018