© 2014 EnterpriseDB Corporation. All rights reserved. 1

What is Your High Availability Plan? Devrim Gündüz

Principal Systems Engineer EnterpriseDB

© 2014 EnterpriseDB Corporation. All rights reserved. 2

•  Introduction to EnterpriseDB •  Postgres clustering options

•  What is Red Hat Cluster Suite? •  RHCS solves important High Availability problems •  RHCS implementation basics

•  Tips for selecting hardware •  PostgreSQL installation tips for RHCS •  Q & A

Agenda

© 2014 EnterpriseDB Corporation. All rights reserved. 3

•  PostgreSQL contributor, and advocate

for more than 12 years

•  Responsible for PostgreSQL official YUM repository

•  Fedora contributor since 2006

•  Principal Systems Engineer at EDB since 2011

Your Presenter - Devrim Gündüz

© 2013 EDB All rights reserved 8.1. 4

Introduction to EDB

© 2014 EnterpriseDB Corporation. All rights reserved. 5

POSTGRES innovation

ENTERPRISE reliability

24/7 support

Services & training

Enterprise-class features & tools

Indemnification

Product road-map

Control

Thousands of developers

Fast development

cycles

Low cost

No vendor lock-in

Advanced features

Enabling commercial adoption of Postgres

© 2014 EnterpriseDB Corporation. All rights reserved. 6

Postgres Plus Advanced Server Postgres Plus

Cloud Database

High Availability Performance Management

REMOTE DBA 24x7

SUPPORT PROFESSIONAL

SERVICES

TRAINING

EDB Serves All Your Postgres Needs

PostgreSQL

Security

© 2014 EnterpriseDB Corporation. All rights reserved. 7

Key Strategic Partners

© 2014 EnterpriseDB Corporation. All rights reserved. 8

EDB Global 2000 Customers

© 2013 EnterpriseDB. Privileged and Confidential. All rights reserved.

Accenture Advanced Auto

Aetna Airgas AOL AT&T

Atos Worldwide BAE Systems

Banco do Brasil Barclays

Bayer Healthcare Boeing

Bouygues Telecom CGI Federal

Check Point Software Cisco Systems

Citigroup Citrix

CME-Commodities CMS

Cognizant Technology Community Health

CSC Deere & Company

Dell Deutsche Börse AG

Direccion General del Dongfeng Honda

eBay

Shinsei Financial Siemens Softbank

Sony St Jude Healthcare

State Farm Swisscom

Syngenta Crop Protection Tata Consultancy Services

TD Ameritrade Telefonica

Telstra Teradata The GAP

Tokio Marine Toyota

Union Pacific Railroad Vattenfall AB

Vivendi Mobile VMWare

Volvo Walt Disney

Wipro Xerox Yahoo

Yamaha Zeejiand Bafang Zions Bancorp

InTouch Technology Intuit

JPMorgan Chase KDDI KT

Kubota Kyocera

LG Electronics Lockheed Martin

8

Malaysia Telecom MasterCard McKesson Michelin

Mitsubishi Moodys

Mosaic ATM Motorola

NEC

Eisai EMC Corporation Emerson Electric

ENSCO Ericsson Expedia Fujitsu

G4S Deposita General Electric (GE)

Google HCL Infosystems

Hitachi HP

HTC Global Services Huwaei Technologies

IBM ICICI Lombard

Infosys

NetApp Systems NTT

Nokia Northrop Grumman

Nucor ONGEI

Panasonic PDVSA

Phillips Electronics

Phoenix Sage QUALCOMM

Raytheon Rite Aid

RSA Schneider Electric

Science Applications Serco-NA SGS SA

© 2013 EDB All rights reserved 8.1. 9

Postgres Clustering Options

© 2014 EnterpriseDB Corporation. All rights reserved. 10

Postgres: Several Clustering Options •  No single definition for word “cluster”

•  Solutions for different problems

•  Popular clustering alternatives

•  Where does RHCS stand in here? −  Actually “Red Hat High Availability Add-on”

Bucardo Multimaster replication PGpool Load balancing, connection pooling PL/Proxy Database partitioning Postgres-XC Write-scalable synchronous multimaster

Skytools WAL shipping, queuing, and replication Slony Active-passive (+cascading) replication xDB FM EDB Failover Manager (details on next slide)

© 2014 EnterpriseDB Corporation. All rights reserved. 11

EDB EDB Failover Manager creates fault tolerant database clusters to minimize downtime when a master database fails by keeping data online in High Availability configurations

FAILOVER MANAGER Master

Streaming Replica

Clients

Witness

Network / Internet

The MASTER is down!

I agree! Take control now!

© 2013 EDB All rights reserved 8.1. 12

Red Hat High Availability Add-on

© 2014 EnterpriseDB Corporation. All rights reserved. 13

•  Open Source Clustering Solution

•  Supports a wide range of hardware

•  Developed by Red Hat, with the community

•  Relatively low cost

•  No data loss

•  Requires a Red Hat Enterprise Linux base subscription

Red Hat High Availability Add-on

© 2013 EDB All rights reserved 8.1. 14

Red Hat Cluster Suite Solves Important HA Problems

© 2014 EnterpriseDB Corporation. All rights reserved. 15

•  Minimizes downtime

•  Automatic failover

•  Transparent to application

•  Active/passive clustering

•  Redundancy −  Data −  Network −  Server & Power

However, the Red Hat HA Add-on is not −  A replication solution −  A multimaster solution

Red Hat HA Add-on with Postgres

© 2013 EDB All rights reserved 8.1. 16

Implementation Basics

© 2014 EnterpriseDB Corporation. All rights reserved. 17

•  Needs team work −  DBA, Network, Sysadmin teams

•  No need to use Global File System (GFS)

•  Manual editing of cluster.conf −  Avoid, or sync them all the time

•  No auto-starting of Postgres

•  Postgres' init script compatibility

Implementation Basics – Part 1

© 2014 EnterpriseDB Corporation. All rights reserved. 18

•  Quorum disk

•  Firewall considerations

•  Benefiting from clusterssh on Linux

•  Yum groupinstall “High Availability”

•  Yum groupinstall “High Availability Management”

Implementation Basics – Part 2

© 2014 EnterpriseDB Corporation. All rights reserved. 19

•  The “High Availability Management” group −  luci, ricci

•  One web interface for managing cluster and storage

•  Automated deployment of HA packages

•  Easy integration with existing cluster

•  Managing multiple clusters with a single interface

•  Integration of cluster status and logs

Using Conga

© 2014 EnterpriseDB Corporation. All rights reserved. 20

•  Everything is handled by the add-on

•  Detection of failed node −  Split brain scenarios

•  Failing back −  Disabling it to avoid extra downtime

•  Total downtime: 30-60 seconds

Failover

© 2014 EnterpriseDB Corporation. All rights reserved. 21

Implementation basics

Active PostgreSQL Server

Passive PostgreSQL Server

Private Network

Shared Storage

© 2014 EnterpriseDB Corporation. All rights reserved. 22

Implementation basics

Sharded nodes, each backing the others up

Shared Storage

Private Network Switch

Node1 Node2 Node3 Node4

© 2014 EnterpriseDB Corporation. All rights reserved. 23

Implementation basics

Data files and archived xlogs of db1 and db2

(/pg01, /pg02, /pga01, /pga02)

Data files and resync'd xlogs of db1-RO and db2-RO

(/pg03, /pg04, /pga03, /pga04)

Passive: DB1

Active: DB2 Active: DB1 Passive: DB2

Streaming replication

Streaming replication

Shared Storage

Shared Storage Active HS: DB1

Passive HS: DB2

Passive HS: DB1

Active HS: DB2

© 2013 EDB All rights reserved 8.1. 24

Choosing the Right Hardware

© 2014 EnterpriseDB Corporation. All rights reserved. 25

•  This is probably the most important part

•  You must consult the Red Hat Hardware Compatibility Guide

https://access.redhat.com/kb/docs/DOC-30004

•  You have to consider firmware

•  You must have a fencing device −  What is fencing? −  Importance of fencing −  Well-known fencing devices

− HP ILO − DELL DRAC −  IBM BladeCenter − Cisco UCS − VMware

Choosing the Right Hardware

© 2014 EnterpriseDB Corporation. All rights reserved. 26

•  Multicast support

•  Internal network setup

•  Enable ICMP echo echo "net.ipv4.icmp_echo_ignore_broadcasts = 0" \

>> /etc/sysctl.conf

•  Fiber Channel Switch for storage

•  SAN −  RAID arrays −  What about NFS? DAS? NAS?

•  At least 2 NICs −  Why is 4 preferred?

Choosing the Right Hardware

© 2013 EDB All rights reserved 8.1. 27

Postgres Installation tips

© 2014 EnterpriseDB Corporation. All rights reserved. 28

•  Postgres version

•  Changing Postgres parameters: −  listen_addresses −  unix_domain_socket −  external_pid_file

•  What about using the passive server as a replica?

Postgres Installation Tips

© 2014 EnterpriseDB Corporation. All rights reserved. 29

•  High Availability is important for most enterprise databases

•  Red Hat Cluster Suite solves important HA problems

•  Best practices for RHCS implementation −  Assemble the right team −  Choose appropriate hardware

− Fencing is a critical consideration −  Configure your network −  Configure Postgres for your RHCS infrastructure

•  Walk through each step carefully to save time and aggravation

•  Get Help! EDB can help you implement an enterprise-grade RHCS solution

Summary

© 2013 EDB All rights reserved 8.1. 30

Q&A For further information, contact sales@enterprisedb.com