Database Considerations for SaaS Products

66
Database Architecture for Software as a Service ConFoo Vancouver 2016 Blog - shawnhooper.ca Twitter - @shawnhooper

Transcript of Database Considerations for SaaS Products

Database Architecture for

Software as a Service

ConFoo Vancouver 2016

Blog - shawnhooper.ca Twitter - @shawnhooper

Blog - shawnhooper.ca Twitter - @shawnhooper

Director of IT at Actionable.co

Web Developer for the past 20 years

Happy to be visiting here from Ottawa

Hi, I’m Shawn

Blog - shawnhooper.ca Twitter - @shawnhooper

Centrally Hosted

Subscription Basis

Usually Accessed Through a Web Browser

What is SaaS ?

Blog - shawnhooper.ca Twitter - @shawnhooper

Learning & Development Platform (SaaS)

Team conversations, individually-relevant actions, peer accountability and real-time, applicable business insights.

Sold and supported through a network of professional consultants.

Actionable Conversations

Blog - shawnhooper.ca Twitter - @shawnhooper

* Team has an in-person conversation around a pain point * Each participant creates a commitment

* Participants “check-in” (1-10) on how they’re feeling about that commitment

* Assign an accountability buddy to encourage / keep them on track * After a month the user, buddy, and manager “close the loop”.

* User, Manager and Consultant all get high-level insights.

Actionable Conversations

Blog - shawnhooper.ca Twitter - @shawnhooper

Database Considerations

Blog - shawnhooper.ca Twitter - @shawnhooper

Users need to be able to sign up for your service without the need for your manual involvement.

If you had to manually configure each client, this wouldn’t exactly be “on demand”.

Self-Service

Blog - shawnhooper.ca Twitter - @shawnhooper

How does your database design handle tens, hundreds,

thousands, or millions (hey, I can dream!) of users?

Scalability

Blog - shawnhooper.ca Twitter - @shawnhooper

What impact does your architecture selection have on the complexity of your code?

(Application code, Database Queries, etc.)

Code Complexity

Blog - shawnhooper.ca Twitter - @shawnhooper

“With Great Power Comes Great Responsibility.”

How does your database design impact data security ?

Security

Blog - shawnhooper.ca Twitter - @shawnhooper

What impact does your architecture decision have on

your ability to make changes your application?

Extendability / Customization

Blog - shawnhooper.ca Twitter - @shawnhooper

* Multi Tenancy * Single Tenancy * Micro Service

Models We’ll Look At

Blog - shawnhooper.ca Twitter - @shawnhooper

Our tenants are companies.

Each tenant has many users.

Tenant

Blog - shawnhooper.ca Twitter - @shawnhooper

Multi-Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

All Users/Clients Share One Database

Multi-Tenancy Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Multi-Tenancy Model

Blog - shawnhooper.ca Twitter - @shawnhooper

All Users/Clients Share One Database

Self-Service: Easy - Just Add Database Records

Multi-Tenancy Model

Blog - shawnhooper.ca Twitter - @shawnhooper

All Users/Clients Share One Database

Self-Service: Easy - Just Add Database Records

Scalability: Limited by storage & system resources

Multi-Tenancy Model

Blog - shawnhooper.ca Twitter - @shawnhooper

All Users/Clients Share One Database

Self-Service: Easy - Just Add Database Records

Scalability: Limited by storage & system resources

Code Complexity: Queries need to include client ID

Multi-Tenancy Model

Blog - shawnhooper.ca Twitter - @shawnhooper

All Users/Clients Share One Database

Self-Service: Easy - Just Add Database Records

Scalability: Limited by storage & system resources

Code Complexity: Queries need to include client ID

Security Concerns: Poorly written queries, single point of data breach.

Multi-Tenancy Model

Blog - shawnhooper.ca Twitter - @shawnhooper

All Users/Clients Share One Database

Upgrades:

Easy! One copy of the database to perform migrations on.

Also means everyone gets upgraded at the same time.

Multi-Tenancy Model

Blog - shawnhooper.ca Twitter - @shawnhooper

All Users/Clients Share One Database

Customization:

Extensions or custom features available to all clients.

Potential for more bugs to reach all users.

Multi-Tenancy Model

Blog - shawnhooper.ca Twitter - @shawnhooper

All Users/Clients Share One Database

Data Storage:

All data lives in the same place. This needs to be a consideration for where, geographically, you’ll

store your data.

Multi-Tenancy Model

Blog - shawnhooper.ca Twitter - @shawnhooper

All Users/Clients Share One Database

“Oversight Accounts”:

For our resellers (consultants) and support staff, this model made it easy to create dashboards that could look at the activity of many

tenants at the same time.

Multi-Tenancy Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Separate Database or Schema For Each Tenant

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Tenant 1 Tenant 2 Tenant 3

Blog - shawnhooper.ca Twitter - @shawnhooper

Separate Database or Schema For Each Tenant

Self-Service:

More difficult. Requires the system to be able to create a new database schema or database when a user signs up for your service.

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Separate Database or Schema For Each Tenant

Scalability:

Imbalance of server resources Some users taking up more than their “share”

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Separate Database or Schema For Each Tenant

Scalability:

This can be an advantage too. It gives you the ability to move databases between servers to balance load.

You can even distribute based on geography for data storage.

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Separate Database or Schema For Each Tenant

Security:

Data Separation! Excellent!

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Separate Database or Schema For Each Tenant

Code Complexity:

No need to include Tenant ID in every query.

Instead, you need to manage connection strings.

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Separate Database or Schema For Each Tenant

Upgrades:

In this model, you could upgrade some tenants and not others to do a controlled release.

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Separate Database or Schema For Each Tenant

Customizations:

You could fork and/or provide add-ons only available to some tenants without an impact on other’s databases.

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Separate Database or Schema For Each Tenant

Data Migrations:

A benefit of this model is that you could easy backup/move one tenant.

Example : Refresh Staging / Test Environment

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Separate Database or Schema For Each Tenant

Oversight Accounts:

Not a great model for the “oversight accounts”. Hard to build dashboards, aggregating data from

multiple databases.

Single Tenant Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Micro Services Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Databases Separated by Function

Micro Services Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Authentication Products Transactions

Blog - shawnhooper.ca Twitter - @shawnhooper

Databases Separated by Function

This is much closer to the multi-tenant model we first explored.

Micro Services Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Databases Separated by Function

Self-Service:

You’re still just adding rows to tables, just in multiple databases.

Micro Services Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Databases Separated by Function

Scalability:

Assign more/less resources to the services that need them the most.

Micro Services Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Databases Separated by Function

Security:

You’ll be including the Tenant ID again.

However, less data is stored in each service, limiting potential for loss in the event of a breach.

Micro Services Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Databases Separated by Function

Code Complexity:

Multiple Database Connections to Maintain

(Multiple APIs too)

Micro Services Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Databases Separated by Function

Code Complexity:

Lack of Referential Integrity between services.

Micro Services Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Databases Separated by Function

Code Complexity:

Lots of traffic between services!

Micro Services Model

Blog - shawnhooper.ca Twitter - @shawnhooper

Databases Separated by Function

Customization:

Bolting new features onto your application may be adding a new micro-service.

Micro Services Model

Blog - shawnhooper.ca Twitter - @shawnhooper

We looked at:

* Multi Tenancy * Single Tenancy * Micro Service

Models We’ll Look At

Blog - shawnhooper.ca Twitter - @shawnhooper

Blog - shawnhooper.ca Twitter - @shawnhooper

Blog - shawnhooper.ca Twitter - @shawnhooper

Blog - shawnhooper.ca Twitter - @shawnhooper

So…. what’s the answer?

Blog - shawnhooper.ca Twitter - @shawnhooper

What did we do ?

Actionable Conversations

Blog - shawnhooper.ca Twitter - @shawnhooper

Multi-Tenant Model

* Easy Scalability * Support for Oversight Accounts (Consultants / Resellers )

* Allowed users to be part of multiple tenants with one account

Downside:

We had to pick one geography in which to store data.

Actionable Conversations

Blog - shawnhooper.ca Twitter - @shawnhooper

Micro-Services Model

One service to handle the “Common” parts of the system:

User Accounts Partner Profiles (Consultants) Wholesale Pricing and Sales

Lists - ex: countries, industries, etc.

Actionable Conversations

Blog - shawnhooper.ca Twitter - @shawnhooper

Micro-Services Model

One service to handle the “Tenants” parts of the system:

Memberships Conversations Run

Feedback & Check-Ins Credit Usage / Purchase History

Actionable Conversations

Blog - shawnhooper.ca Twitter - @shawnhooper

Micro-Services Model

One service to handle the “Notifications” parts of the system:

E-Mail & SMS Message Schedules, Queues and Logs

Actionable Conversations

Blog - shawnhooper.ca Twitter - @shawnhooper

Micro-Services Model

One service to handle the “Content” parts of the system:

Why re-invent the wheel?

We used WordPress, connected with a REST API.

Actionable Conversations

Blog - shawnhooper.ca Twitter - @shawnhooper

Hosting

RDS Instance with Amazon Web Services.

Allowed for scaling as the platform grows.

Actionable Conversations

Blog - shawnhooper.ca Twitter - @shawnhooper

API

We built the application using API First methodology

using the Loopback Framework & Strongloop.

Actionable Conversations

Blog - shawnhooper.ca Twitter - @shawnhooper

Actionable Conversations

Blog - shawnhooper.ca Twitter - @shawnhooper

A new platform that allowed clients to use the “Actionable” method to deliver

their own onboarding programs.

Duplicates the application & database using the same Multi-Tenant Model

with only one tenant in it.

Actionable Training

Blog - shawnhooper.ca Twitter - @shawnhooper

Blog - shawnhooper.ca Twitter - @shawnhooper

[email protected]

Slides will be available shortly at

shawnhooper.ca

Questions? Tweet @shawnhooper

Thank You!