Docker talk at Cloud Austin
-
Upload
flux7 -
Category
Technology
-
view
105 -
download
1
description
Transcript of Docker talk at Cloud Austin
![Page 1: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/1.jpg)
DOCKER @ FLUX7 LABS
![Page 2: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/2.jpg)
DOCKER AUSTIN
FOR MORE DISCUSSIONS ON DOCKER!
![Page 3: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/3.jpg)
Who am I?
● Aater Suleman○ Part-time UT Professor ○ Geek, Architect, Developer, Ops, DevOps …
● Co-founder & CEO Flux7 Labs ○ DevOps Solutions
■ Deployments■ Cost/performance optimized large scale
website (Ruby on rails, node.js, Django) and Hadoop deployments
![Page 4: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/4.jpg)
VyScale Dev Flow
Docker's impact on performance (whitepaper WIP)
Multi-tenancy
Live process migration using CRIU (criu.org)
Four projects:
![Page 5: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/5.jpg)
Application: Single Service Provider
Receive Sensor Data
Report Generation
based on data
Report sent to End User
Internet of Things -- Solar Panel Monitoring
XML Data over TCP
Big Data Analytics
![Page 6: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/6.jpg)
Single Provider System
Provider Span
Location1 Location2Location3
Gateway1 Gateway2
Gateway3sensors
A provider has Mifi routers installed at multiple Locations which collect data from sensors and sends it to a remote TCP server via the internet.
TCP serverPort 6000
Cassandra port 9160
Flask AppPort 80
BrowserUses the Flask app at port 80Internet
![Page 7: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/7.jpg)
COMPONENTS
1. Cassandra for data persistence which we later use for generating reports for each gateway.
2. A Twisted TCP server listening at PORT 6000, for data ingestion from multiple gateways owned by the provider.
3. A Flask app serving at PORT 80 as the admin panel for setting customizations and viewing reports.
![Page 8: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/8.jpg)
Customer NCustomer 2 …Customer 1 Each customer can have multiple gateways
commissioned to them.
Remote Twister TCP Server (Non–Blocking I/O)
Cassandra NoSQL data store(High Volume High Velocity Write which scales Linearly across the
cluster )
Power consumption status on website and
mails
*G - Gateway
Web App
Mailer
![Page 9: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/9.jpg)
SINGLE PROVIDER LAUNCH
For launching the single provider version, the following was done:
1. nohup python tcp_server.py & # For firing up the TCP server.
2. nohup python flask_app.py & # For firing up the admin panel
Both these code bases houses hard-coded Cassandra KEYSPACE
Success!
![Page 10: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/10.jpg)
Application: Multiple Service Providers
…
![Page 11: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/11.jpg)
Provider 1 sends data to port 6001
and accesses flask app at port 8081
Provider 2 sends data to port 6002 and accesses flask app at
port 8082
Flask container-runs flask app at port 80. Exposes port 80 and
published it to port 8081 for provider 1
Flask container-runs flask app at port 80. Exposes
port 80 and published it to port 8082 for provider 2
TCP server container-runs at port 6000.
Exposes port 6000 and published it to port 6001
for provider 1
TCP server container-runs at port 6000. Exposes
port 6000 and published it to port 6002 for provider 2
Cassandra
Internet
![Page 12: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/12.jpg)
KNEE-JERK APPROACH
Sprinkle Tenant ID everywhere in the code and DB
Time consuming
Expensive
Poor isolation
Security
Maintenance
Rigidity
![Page 13: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/13.jpg)
An alternate solution is to use Virtual Machine (VM)
Hosts are expensive ($)VMs are expensive (high overhead)
MULTIPLE HOST/VMS
![Page 14: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/14.jpg)
AND THE SOLUTION
![Page 15: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/15.jpg)
How: Isolated environments for running multiple instances of the app
WHY DOCKER?
Docker containers provide isolation that is
Fast
Inexpensive
![Page 16: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/16.jpg)
Create a docker container for the new version of the app
PLAN
Setup environments/dependencies correctly
Start a Cassandra container.
![Page 17: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/17.jpg)
# start a docker container for consuming gateway data at gateway_portstart_command = 'python software/remote_server.py ' + provider_idremote_server = docker_client.create_container('flux7/labs', # docker imagecommand=start_command, # start command contains the keyspace parameter, keyspace is the provider_idname='remote_server_' + provider_id, # name the container, name is provider_idports=[(6000, 'tcp'),]) # open port for binding, remote_server.py listens at 6000docker_client.start(remote_server, port_bindings={6000: ('0.0.0.0', gateway_port)}, links={'db': 'cassandra'})
AUTOMATION
# start a docker container for serving admin panel at admin_portstart_command = 'python software/flask_app.py ' + provider_idremote_server = docker_client.create_container('flux7/labs', # docker imagecommand=start_command, # start command contains the keyspace parameter, keyspace is the provider_idname='admin_panel_' + provider_id, # name the container, name is provider_idports=[(80, 'tcp'),]) # open port for binding, remote_server.py listens at 6000docker_client.start(remote_server, port_bindings={80: ('0.0.0.0',admin_port)}, links={'db': 'cassandra'})
An automation was the next foreseeable step, and for that we found Docker-py extremely useful. We used something like:
# Yes. We love Python!def start_provider(provider_id, gateway_port, admin_port ):docker_client = docker.Client(base_url='unix://var/run/docker.sock', version='1.6', timeout=100)
![Page 18: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/18.jpg)
For now, a locally running container serving at PORT 9160 using the command
similar to this:
docker run -d -p 9160:9160 -name db flux7/cassandra
OUR SOLUTION- EXPLAINED
![Page 19: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/19.jpg)
▪ Create a keyspace ‘provider1’ using pycassaShell.
We fired up our two code bases on two separate containers like this:
OUR SOLUTION- EXPLAINED
docker run -name remote_server_1 -link db:cassandra -
p 6000:6000 flux7/labs python software/remote_server.
py provider1
docker run -name flask_app_1 -link db:cassandra -p
6000:6000 flux7/labs python software/flask_app.py
provider1
![Page 20: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/20.jpg)
DOCKER ISSUES DISCOVERED
Docker does not support multiple instances of Cassandra running on the
same machine.
Hosting multiple database instances on a single machine can quickly
cause resource shortages
![Page 21: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/21.jpg)
❑ Followed the traditional solution to make an application multi-tenant
OUR SOLUTION
Code Changes •• To data ingestion server and web server by adding the keyspace parameter to the DB accesses.
Cassandra KEYSPACE /
provider ID •• Passed to each instance of the app on the
command line.
❑ Each provider in the data store gets a separate namespace without making
any changes to the column family schema.
Use of KEYSPACE as the namespace for each provider in the data store
![Page 22: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/22.jpg)
LESSONS WE LEARNED
1. Docker is an extremely fast and elegant isolation framework: easy to port, cheap to run, easy to orchestrate
2. Multi-tenancy != changing the app to support multiple tenants
3. Docker orchestration frameworks are not at par with Docker today. What we have written is yet another one but for multi-tenancy.
4. Dockerfiles still need work -- we used shell scripts in some places
5. We can run multiple commands/container
![Page 23: Docker talk at Cloud Austin](https://reader033.fdocuments.us/reader033/viewer/2022052820/54c6c8e14a795938448b45a8/html5/thumbnails/23.jpg)
DOCKER AUSTIN
FOR MORE DISCUSSIONS ON DOCKER!