7 Tips to design web-centric, high performance applications

A DB Architect’s Perspective

19 March 2016

• New Age Applications

• Challenges of Web 2.0

• Catering to Web 2.0: Challenges of Database Design

• My learning from Developers

Agenda

2

• A random Oracle-DB2 DBA who started working on an Open Source Database – PostgreSQL

• And found it absolutely great

• Works for Ashnik Pte Ltd – Open Source Solutions and consulting

• Who also works on noSQL databases

• Hobbies – Cycling and Photography

3

About Me

• Web or Mobile facing

• Futuristic Scale out Requirement – growing user base

• Rapidly changing business requirements

• Beyond user interface – User Experience

New age applications

4

• Mobile Rendering/Mobile App

• Multiple touch-points for end-users

Challenges of Web 2.0

• Social Media Interaction

• Interoperability and Usability

5

• Modularity

• Response Time

Catering to Web 2.0: Challenges of Database Design

• Scalability

• Concurrent

6

What I have learnt while working with Developers!

My 7 tips!

• Avoid one monolithic application design

• Create smaller modules or Microservices which can function with other Microservices/modules

• Use modular design with interface and contracts

• Keep a separate data access API which is responsible for database CRUD

• Use standard format instead of using proprietary or specific format e.g. JSON, XML for exchanging data between API

e.g. Data Access API always returns data in JSON format

• Postgres Supports JSON datatype and function for converting relational data to JSON

Tip #1: Break-down the application

8

• Avoid sending all requests to the database/application at once

• Use message/request queue

• Use connection pooling and connection queueing for database access

• Postgres can work with various message queues e.g. Rabbit MQ

• Use the database side tools available to setup connection pool, like:

• pgpool

• pgBouncer

Tip #2: Use message/request queues

9

System ages over a period of time, User base increases, Data will gradually increase & Large volume may slow you down !!

Use effective partitioning with:

• Right Partitioning scheme

• Right Partition Key

• Distributing your data across servers using

Third-party APIs

Some natural key for distributing data

Partitioning with Postgres

• Postgres supports Range and List partitioning

• Postgres v9.5 added basic support for sharding

Tip #3: Use effective partitioning

10

• Use data-type which best describes your data

• Avoid using strings for storing everything

• Choose a data-type considering how the data will grow

e.g. choosing between int and bigint for storing Primary Key

How? - Using different data-types supported by Postgres:

• Numeric, Fixed Width String

• Variable Length String

• Numeric, floating numbers, integers, money

• Time, data, timestamp, interval

• Ranges

• GeoSpatial, Unstructured- JSON, XML

• Many more

Tip #4: Strings can store all but not represent all

11

• Business scenarios are changing quite rapidly

• Normalization – flexibility to accommodate changes

• Normalization – Too many entities and relationships to handle

• De-normalize!

• Lot of modern day entities are best represented as de-normalized un-structured records

• For example Contact Details• They evolve as your relationship with customer increases• As your touchpoint increases

• Postgres allows you to store data in JSON and index and search it• It can store multi-value attribute e.g. one customer with phone number• Allows storing nested data – Address with details like street, city etc

Tip #5: De-normalize if needed

12

• Avoid using SELECT FOR…UPDATE to keep up your concurrency

• Avoid READER-WRITER blocking

• Use MVCC model which ensures read-consistency

• Postgres with MVCC model ensures that,

• Readers don’t block writers

• Writers don’t block readers

• Use Hibernate support for Postgres which has inbuilt mechanism for implementing Optimistic Locking

Tip #6: Boost Concurrency with Optimistic Locking

13

• A lot of static/configuration data can be cached by application frequently

• Some counter/summary data which can be cached by application

• Updated frequently

• Derived/summarized from other sources e.g. log

• Stored separately to avoid runtime calculation

• Caching these data can save read-write overheads at DB

• Upon crash, these data can be rebuilt

e.g. ATM withdrawal limit

• Caching per user counter for limit current day, month and quarter

• Allows rebuilding the counters using transactions in case application server crashes

Tip #7: Cache frequently accessed configuration

14

Twitter - @sameerkasi200x | @ashnikbiz

Email - sameer.kumar@ashnik.com | success@ashnik.com

LinkedIn - https://www.linkedin.com/in/samkumar150288

We are hiring!