Single tenant software to multi-tenant SaaS using K8S

By: Igor Seletskiyi@cloudlinux.com

Back Story

§ Successful Linux distro for service providers

o That sell shared hosting

§ Started to work on platform to sell Docker

o Added multi-tenancy

o Network isolation

o Resource metering

o billing

Back Story

§ Realized smaller hosts cannot

out-compete AWS/Google

Application hosting?§ Decided to pivot to

Application Hosting

o Wordpress

o Joomla

o ElasticSearch

§ Each app would be a pod

o But we need persistence

• K8s Volumes

Our KD hat with apps going out of it?

Where do we fit in?

§ We would simplify packaging

o integrate billing

• Prepare signup

o Do user management

o Use k8s to orchestrate

Can we have more fun?

§ Realized there is more interesting case

o Single tenant to SaaS

§ Or “new” way to do SaaS

New way? Huh?

Where are my microservices?

§ More like really old way

o Wrong way

• Against all the best practices

way

So, what is the new way

§Single tenant

o Single instance of application

• Single or multiple containers (pods)

§K8S to orchestrate

SaaS burden

§ SaaS companies have an idea that they love to implement

o Yet, they also have to implement…

• registration

• customer management

• billing

• upgrade procedures

SaaS burden

o They also have to deal with

• scalability

• reliability

• QoS

• backups

Multi-tenancy

Subscription & Billing

Customization & Configurability

Scalability & Availability

Integration

Security

§ High availability

§ Upgrades

o Rollbacks

o Staged upgrades

§ A/B testing

§ Custom deployments

§ on demand scalability

§ caching proxy

§ CDN

SaaS: Nice To Have

§ Single entity -> small amount of data (10mb?

10gb?)

o easy to deal with

o easy to process

o fast to access

§ Multiply by 10,000x customers

o 100GB to 100T

o Requires advanced database clustering

o costly to access

o waste of resources scanning other’s people data

§ 100,000 customers - petabytes...

Scalability / Data

Sharding at it best

§ Perfect sharding

o single shard / single customer

Shard A Shard B Shard C Shard DCollection I

256GB 256GB 256GB 256GB

Collection I

1 TB

Large Scale Database HA

mm m m

Apps§ Large clusters are hardo requires advance knowledge

o requires advance tuning• support contracts

• high end db admins

Small Scale Database HA

§ Trivial to setupo easy to maintain

MySQL Master

Clients/Web apps

MySQL Replication

MySQL Slave

Reads and writes Only reads

Requests/Second

§ If one client generates 1 request per second at peak time

o 10,000 clients could easily generate 2,000 requests per second at peak time

It is a HARD problem to scale a single application to 2,000 requests per second.

APP

The end of ‘Old’ SaaS

§ Hard to deploy

§ Difficult to update

§ Huge overhead

§ Impossible to manage

The end of ‘Old’ SaaS

Nah, Just Kidding

But we do solve problems

§Easy to make existing single tenants apps into SaaS

oJust package as a pod (group of pods)

§Easy to write new ones

oAll the plumbing done

oNo need to deal with lots of data

APP

Solve§ scalability, reliability & high availability

§ customer signup & managemento billing integration

§ QoSo on demand scalability

o customer separation

§ rolling upgrades

§ a/b testing

KuberDock for ISV

But what about density?

§OverlayFS is great

oSaves plenty of memory & IO

§Container check pointing (future)

oNo tenant activity – checkpoint all the pods

• Bring the back up on first packet for the tenant

o In the right order

Why it wasn’t done before

§Wrong technologies

o50k VMs are much scarier then 50k containers

oPuppet is great

• no atomic updates

§OverlayFS is important – saves tons of

memory

§Some hybrids were done before

But how much would it scale

§Don’t know yet…

§Not recomended design for your b2c venture

software with 100M free users

But how much would it scale

§But it would work for:

oB2B software with deployment of 50k users

• Especially if each tenant needs a lot of data

o 100MB+

Did we ever…

§Work in progress..oLaunch in a month

oBilling software (PHP/MySQL) – 5+ years old• Used by 10K+ companies

• Single tenant application, runs on client’s servers

oHad demand for hosted version• No resources to re-design

• No resources to automate

What does it take?

§TimelineoMinor software redesign:

• putting all customizable files into single folder

• Changing setup wizard

• Adjusting licensing process

• Packaging as pod

Total dev time so far < 40h

The End

Thank You

Igor Seletskiyi@cloudlinux.com

More info:https://kuberdock.com

EASY SCALABILITY

§ Scale to more customers just by adding serverso Limited only k8s scalability

• support for multiple k8s clusters

§ Using native k8s replication & load balancing functionality to

scale for larger customers

Implementable QoS

§ Tenants isolation

o make sure that one tenant never affects other tenants,

no matter what they do

§ Native high availability

§ Issues affects subset of the customers - if any

Upgrades

§ Docker ‘image’ based upgradeso Roll updates to X% of customer base

• multiple versions in production

• A/B testing

o Roll back updates

o Custom releases for those who need it

All managed using centralized UI

Easy Packaging Provisioning

§ Applications described as YAML

o multi-server, high availability, custom settings…

§ Instantiated using native KuberDock/k8s YAML provisioning

Complex Apps

BILLING MADE EASY

§ Integrate with common billing platforms

§ Let ISV bill per resources, customers, or other metrics

DevOps & Programmers friendly

§ Fully automated

§ micro-services or monolithic

§ any programming language

§ any DBMS

SIGNIFICANTLY SIMPLIFIED DEVELOPMENT

Resource Management

§ Dashboard on server utilization, peak usage

o allows for planing & scaling

§ Vertical scale out using public clouds (AWS) when additional computing needs

required

More to come...

§ Provide platform for in-app user management

o dual auth

o facebook/google auth

§ Built in help desk integration

§ build it app monitoring

§ cross customer jobs