WordPress Multisite at Large Scale - · PDF file3 WordPress Multisite at Large Scale For all...

WordPress Multisite at Large Scale

2


WordPress Multisite Networks for Very Large ScaleA How to Guide

So yoursquove decided to go with WordPress for your large scale website There

are three basic options for setting up the site

Option 1

If itrsquos a single brand with design consistent across the entire site and you have one team with up to 50-60 people working on it and they all work together you should go for a regular WordPress install (no need for something like Multisite)

Option 2

If there are multiple departments or organizations and each of them will have its own subdirectory or subdomain each with a different design but quite a few common denominators between the different sites a good choice will be WordPress Multisite WordPress Multisite will allow you to have one network administrator who will create one or more sites and then user accounts who can manage each site without necessarily being able to manage the entire network There can be one central thememdashmanagers of each site can select different options in the sitersquos admin interface to achieve a different look and feel Yoursquoll need to build these options into the code of your theme To allow more customization at the code level you can use a base theme and child themes

Option 3

If there are multiple different sites or brands and the codepluginsgoalsfunctionality of each site is very different andor there are different teams working on each site and each of them need to develop different functionality at the code levelmdashMultisite is probably not suitable Each site should be defined as a separate WordPress site and yoursquoll have to manage your multiple sites using automation Automatic deployment and updates will be needed to handle the complexity and large volume of changes across the multiple sites as well as issues like robustness resilience and scale

2

3


For all these use cases there is also a secret option you could work with Pantheon Our enterprise-grade WordPress hosting supports both Multisite and multiple WordPress instances with an automated devtestdeployment workflow and is proven to run large scale WordPress sites

Donrsquot worrymdashwe didnrsquot write this whole thing as a shameless plug The rest of this resource will explain in detail how to go about doing options 1 2 or 3 to implement a large scale WordPressmdashwithout using Pantheonmdashbased on our experience running thousands of demanding WordPress sites In addition we show how to do the same thing on Pantheon and get a scalable infrastructure out of the box

How Do We Define Scale

ldquoLarge scalerdquo in our definition includes three dimensions

High Load Upward of 1 million site sessions per month potentially with high concurrency

Large Team More than 50 organizational users actively working on the WordPress sites

Many Sites More than 10 different departmentsorganizationsbrands who need to be accommodated in the WordPress installation probably with continuous growth

Business Requirements at Large ScaleStability and Resilience The system will typically be critical and used by large numbers of customers or internal users Low tolerance for downtime errors and data loss

Performance At the current scale and as it scales up the system needs to perform reasonably in the eyes of end users both on the frontend and backend

Security Larger organizations and websites represent a bigger target and need to be well protected from attacks

Ease of Management In a large scale WordPress setup work needs to be managed in an organized manner using workflows and admin interfaces baked into the WordPress site(s) The larger the team the more this needs to be thought out in advance

Option 2

Option 3

PANTHEON

Option 1 Single WordPress Installation

WordPress Multisite Network

Multiple WordPress Installations Managed by Automation

Single InstallationMultisiteMultiple WordPress with Wings

4

WordPress Multisite at Large ScaleOption 1

Technical Requirements for Large-Scale WordPress

A large-scale WordPress instance requires a horizontally scalable architecture reverse-proxy page caching a persistent object cache to offload queries as much as possible from the database database replication an optimized search index and automated devtestdeployment workflows In the following sections we explain a bit more about each requirement and how it is typically implemented in WordPress

Elastic ArchitecturemdashHorizontal Scalability

An elastic architecture means that when traffic increases you can provision more machines to run your website When traffic calms down you can save resources and turn off your extra capacity This also provides high availability

SINGLE WORDPRESS

INSTALL AT LARGE SCALE

SINGLE INSTALL

If you only need one WordPress site or even if you plan to do

Multisite (Option 2) or several WordPress sites managed by

automation (Option 3) you will need to make sure that each

of your WordPress instances is built for large scale This section

discusses basic considerations and implementation strategies

for WordPress at large scale

Single brand

Consistent design across site

One team of up to 50-60 peopleSERVER

5


Technical Requirements

Load Balancing Across the available PHP App servers Can be done with software tools (eg haproxy) dedicated hardware or a cloud service (eg Amazon ELB)

Shared Media You must find a way for uploads to be available to all PHP App servers Can be done with open source tools like GlusterFS NFS and Ceph or Amazon EFS on the cloud

Consistency Servers and databases need to have the same configuration and changes to WordPress itself must be deployed consistently to avoid complex issues as you scale Requires an automated workflow using tools like Puppet Chef or Ansible

Page Caching The Key to Serving Internet-Scale Traffic

Caching is a key part of a WordPress performance and scalability strategy At large scale the solution cannot be driven by the app itselfmdashas shown in these charts from Joe Hoyle comparing Batcache (driven by WordPress) to Varnish (which runs independently)

Option 1 SINGLE INSTALL

MySQLMASTER

MySQLREPLICA

MySQL n

PHP APP 1 PHP APP 2 PHP APP 3 PHP APP n

MySQLMASTER

MySQLREPLICA

PHP APP 1 PHP APP 2

7

525

35

175

020 50 100 150 200 250

LOAD v CONCURRENCY

VARNISH BATCACHE

70

55

40

25

1020 50 100 150 200 250

VARNISH BATCACHE

Source Joe Hoyle httpshmnmd20121217testing-batcache-versus-varnish

RESPONSE TIME v CONCURRENCY

6


A well-established pattern for large scale WordPress is a reverse-proxy requests and responses tofrom WordPress flow through an intermediary service which provides a cached copy of a response for a specified time A reverse-proxy can be up to 1000x more efficient than a PHP web server at delivering cached responses

Popular reverse-proxy tools are Varnish and NGINXrsquos built-in caching Another option is a CDN like Akamai Fastly or CloudFlare


Cache TTL and Expiration You will have to devise a mechanism for clearing the cache without restarting the proxy

Cookies Most proxies rely heavily on cookies to decide whether to provide a cached response or pass the request to WordPress Both proxy and WordPress need to be properly configured to avoid too many requests going to WordPress

Learning Curve For many developers it is more complex to maintain and debug a system in which the browser does not talk ldquodirectlyrdquo to WordPress Take the learning curve into account

Object Caching Speed Up Dynamic Pageviews

WordPress offers an internal object cachemdasha way of automatically storing data from the database (not just objects) in PHP memory to prevent unnecessary queries Out of the box this is inefficient but WordPress easily integrates with persistentexternal storage backends like Redis or Memcached (or on the cloud AWS ElastiCache or Azure Managed Cache) These backends persist objects between requests speeding up execution while reducing database load


MySQL

OBJECT CACHE

PHP APP PHP APP

Fewer request forfully-built objectsNo post-processing

Many queriesplus post-queryprocessing

7



Complexity This adds a layer to the stack and you need to make sure you have the same Object Caching solution present for development and testing environments

Invalidation WordPress data might be updated frequently but you need to intelligently decide when to purge your cache data Frequent purging on every change will reduce performance

Eviction Cache backends use a Last Resource Used (LRU) strategy for ldquoevictingrdquo items from the cache when more room is needed You need to account for unexpected cache expirations that might occur as a result of the cache running out of space

Optimization Yoursquoll need to smartly cache data based on an equation of how expensive it is to generate how frequently itrsquos requested (aka likelihood of actually being served) and how much capacity you have in your persistent storage backend

Query Performance The Database Is the Bottleneck

Your websitersquos database is the ultimate bottleneck when scaling An elastic architecture allows you to increase your capacity for read queries through replicas The HyperDB plugin lets you scale out the Database while managing the use of master and replicasmdashfor example administrative functionality happens only on the master to provide guaranteed access and performance to site editors


MySQLMASTER

MySQLREPLICA

PHP APP 1 PHP APP 2

Master handles both reads and writes

Replicas take on more read queries

Ideally replication lag is sub-second

8


Avoiding ldquoQueries of Deathrdquo

Scaling via database replication still assumes that your queries are generally performant If your use-case means you have a content footprint (100s of thousands or millions of posts) the WordPress default query builder (aka WordPress_Query) may generate ldquoqueries of deathrdquo requests to the database that can take several seconds to compute

These are called ldquoqueries of deathrdquo for a reason They can suddenly and drastically affect site performance even to the point of causing downtime Long-running queries are intensive often involving the creation of a whole temporary table to compute the result They bog down database performance for all queries and tie up PHP application capacity a lose-lose combination

Slow queries block the PHP application threads that kick them off If theyrsquore happening in high volume they can overwhelm even a horizontally scalable ldquoelasticrdquo infrastructure Eventually all your PHP threads are waiting for slow queries to respond at which point the site is effectively offline

Even with best practice architecture an important part of scalability hygiene is reviewing query performance MySQL has a built-in capability called the slow query log which allows you to build and analyze data on your query times You may also find value in using application performance monitoring tools such as New Relic


Avoiding ldquoQueries of Deathrdquo default WordPress queries (eg those generated by WP_Query) can generate database requests that take seconds to computemdashif this happens often it can bring even an elastic architecture to its knees

Query Routing Keep it simple for starters in most cases itrsquos best not to customize HyperDB and have less moving parts

Replication Lag Replication should be instantaneous but sometimes it isnrsquot so ensure that you can handle multi-second lag and monitor for unacceptable levels

Debuggability You will need a slow query log and a way to reliably replicate the situation that caused a slow query This requires a debugging environment with all the data needed to trigger the slow query

Regressions For sites with large datasets it is important to examine new queries and test them before using in production


9


Scaling Site Search with an Index

The WordPress built-in content search runs slowly if you have a large number of posts and cannot produce results based on relevance A dedicated search index improves performance significantly and allows a richer user experience It does this by circumventing the WordPress database for search-related queries and sending these queries to a high-performance subsystem This allows more and more complex queries to be performed allowing better search functionality (autocomplete searching by meta data)

Common options are ElasticSearch Apache Solr search index and AWSrsquos CloudSearch


Overriding WordPress_Query Most implementations of a search index backed involve overriding the built-in WP_Query() object Doing this for specific queries will require care and attention from a developer

Index Rebuilds While not common you may come across a situation that requires you to rebuild your content index This means being able to ldquofall backrdquo to the database at least temporarily

Complexity As with other dedicated subsystems a search index is yet another piece of infrastructure to set up monitor and manage While the payoffs are clearly worth it this does become another ongoing responsibility to maintain


MySQL MySQL

PHP APP PHP APP

Fast query toget posts detailsby post id

Fast query to fetch a listof posts

Content is indexed afterbeing posted

Slow query to fetch a list ofposts from a big dataset

Fast query to get post detailsby post id

SEARCHINDEX

10


A Real-World Scalable Architecture

Putting all these components together the stack will include

Linux

Varnish

Apache or Nginx

PHP

Memcached or Redis

ElasticSearch or Apache Solr

MySQL

Donrsquot panic This is why the discipline of DevOps exists There are many professionals who are experienced with setting up this style of implementation and many of the component pieces are now available as cloud services

Also you donrsquot have to run it yourself This style of architecture is also available from many managed hosting and platform providers

Important Considerations When Outsourcing Your Infrastructure

If you are looking to outsource your website infrastructuremdashan increasingly common choicemdash you should now be armed with sufficient knowledge to evaluate various providers

Do they provide load-balancing and reverse-proxy caching

Is their infrastructure truly elastic What is the turnaround time for scaling horizontally

How do they handle the need for a network filesystem for uploads

Can they provide MySQL replication Does it support HyperDB

What options do they offer for a Search Index


11


At the code level WordPress Multisite is one codebase with one common set of plugins and one or more themes This means that launching a new site on a Multisite Network can be done through the WordPress admin and doesnrsquot require managing an entire second (or third or fourth) copy of WordPress

WordPress Multisite allows you to have one network administrator who creates user accounts and each user account can have its own separate site There is one central theme and each of the user accounts can select different options in its admin interface to achieve a different look and feel Yoursquoll need to build these options into the code of your theme Alternatively you can have a base theme and child themes

Before reading on make sure yoursquore familiar with the requirements for a single WordPress instance at scale (detailed in Option 1) A Multisite instance will have the same requirements and more

WORDPRESS MULTISITE NETWORK AT LARGE SCALE

WORDPRESS MULTISITE

If there are multiple departments or organizations using WordPress and each of them has its own subdirectory or subdomain each with a different design but quite a few common denominators between the different sites a good choice will be WordPress Multisite

Multiple departmentsorganizations

Different design in different parts of site

Large common denominator between designsSERVER

12


Basic Considerations for Multisite (At Any Scale)

The Decision is Permanent

The choice between running classic single-site WordPress or WordPress in Multisite mode is permanent Once you perform the setup (which is relatively straightforward to do) itrsquos technically challenging to switch back to single-site

Choose Between Subdirectories and Subdomains

When configuring WordPress Multisite yoursquoll be need to choose between using subdirectories or subdomains Yoursquore choosing between mydomaincomfirst-sitemydomaincomsecond-site or first-sitemydomaincomsecond-sitemydomaincom The key difference is that custom domains can be mapped to sites on subdomains or in subdirectories but subdirectories canrsquot be mapped to sites on subdomains and serving subdomains over HTTPS requires a wildcard HTTPS certificate or individual HTTPS certificates for each subdomain

Users Are Shared

User data is shared among all sites on a WordPress Site Network If you were to create a user with a username of ldquojanedoerdquo she will have the only ldquojanedoerdquo username across all of the sites If you were to change her display name from ldquoJane Doerdquo to ldquoJ Doerdquo the change would apply everywhere her name is displayed regardless of the site However the same user can have different user roles between sites (eg an Editor on one site and an Administrator on another site)

Sites Share Themes and Plugins

Code for themes and plugins is shared among all sites on a Multisite Themes can be activated on a per-site basis or network-enabled for activation on any site (and individually activated at the site level) Plugins can be activated individually on each site or a plugin can be automatically activated across multiple sites When plugins are installed but not network-activated the site administrator has the ability to activate the plugin

Option 2 WORDPRESS MULTISITE

13


AdministrativeWorkflow Considerations for WordPress Multisite at Large

Scale

Theming Strategy

There are three main theming strategies and at the large scale only the first two are a viable option

Admin interface You can create an option in the admin interface of each of the sites to change a design elementmdashfor example change the logo Itrsquos the same theme just different options selected by the operators on each of the sites

Child themes There is a main theme and a child theme in each child the design can be different For example each child theme can contain a different logo image In this case making changes requires development by the central development team Can be suitable if there are only slight variations between the different sites

Different themes each with their own codebase This is very difficult to manage at a large scale because of possible conflicts between the themes There will be exponentially-increasing complexity as the Multisite grows and as each site adds functionality and revises its design This complexity will lead to security issues instability and difficulty to debug issues that arise from conflicts between numerous different codebases


14


Development Strategy

Do you have your own in-house developers

If there are in-house developers child themes are preferred and this will provide much bigger flexibility in the design of each site within the Multisite

If not better to prepare a central theme with numerous options controlled in the admin interface This will allow the teams working on the individual sites to make changes to the design without requiring development

Content Update Strategy and User Management

Who will be making changes How many site editors and contributors will there be

If mainly developers will be working on the sites a child theme approach is preferred

If itrsquos webmasters or non-technical staff a central theme with admin interface is preferred

Account creation process In a larger organization yoursquoll need a process for creating new user accounts (naming policy determining who gets what user role on what site sending link to documentation) Take into account if you need to connect existing user accounts with Single Sign On

Regular account cleanup Removing or downgrading users who have left the organization

Securityaccount monitoring Keeping tabs that users are doing what they should be doing


15


Automatic deployment and updates will be needed to handle the complexity and large volume of changes across the multiple sites as well as issues like robustness resilience and scale

Why Automation

Even for one mission-critical WordPress site automation is essential for several reasons

Ongoing Updates can become a major task and human error can pose serious risk

Testing is something that if done only manually can be spotty at best

Dev Test and Production Environments are going to deviate over time (what is known as ldquoconfiguration driftrdquo) because theyrsquore not being replicated automatically This will create changes in the transition between environments (ldquoit worked on my machinerdquo)

Opportunity Cost Most companies will assign tasks such as version control deployment backup monitoring user managementmdashto their most experienced developers If it can be done automatically that could free time for those star developers to improve your websites

MULTIPLE WORDPRESS SITES MANAGED BY AUTOMATION

MULTIPLE WORDPRESS SITES

If you have multiple sites or brands and the goals and functionality and accordingly the code and plugins used on each site are very different andor there are different teams working on each site and each of them need to develop different functionality at the code levelmdashthis is not a Multisite You should build it out as multiple WordPress sites and manage them using automation and modern DevOps practices

Multiple brandsdifferent sites

Different goalsdesignfunctionalityplugins

Different teams each developing functionality and adding codeSERVER SERVER

16


Assurance When you automate you know every task gets done every single time That means less risk less need to oversee tasksmdashcomputers donrsquot forget

Perfecting the Architecture By automating you have an opportunity to fine-tune the infrastructure Find out over time what is the perfect server config to run your sites on deploy that consistently across all sites and environments and continually improve it keeping all environments consistent using ldquoinfrastructure as coderdquo

Automation Gives You Peace of Mind You donrsquot have to waste energy by continually answering questions about the performance or functionality of the website Any improvement you makemdashfor example a performance optimization is put into code and automatically propagates across all your sites in every deploy on every applicable environment from that point onwards

To take another example say you define in your test suite that a broken link fails the test and we set up the automatic deployment to never deploy when that test fails In this case you know that yoursquoll never deploy a broken link And then yoursquoll never get a ticket to your support team saying ldquoyour link is brokenrdquo which is a waste of time for the team for your customer and provides a bad user experience Over time you can get these issues out of the way automatically allowing you to focus on building the functionality of the website

Importance of Automation for Multiple Sites

Across multiple large scale WordPress sites the problems detailed above are compounded by an order of magnitude

Multiplied opportunity cost Multiple sites multiply the opportunity cost across multiple development teams and we multiply the risk of manual updates and changes The challenge of hosting a matrix of devtestprod environments for numerous sites and having to install update and maintain each of those environments can become a nightmare not to mention the detrimental effect to your software quality and visibility over the process

Dangers of a Single DevTest Environment In reality most shops resort to setting up a single development or testing server for all the teams This creates a single point of failure so that when that server goes down development is stalled across all your sites

Standardization and On-Boarding Another important benefit of automation is standardizationmdash on-boarding new people to the team becomes easy as they donrsquot need to figure out whatrsquos different about each different website in your organization Itrsquos much easier to move developers between teams websites and projects compared to a bespoke project-specific architecture

Option 3 MULTIPLE WORDPRESS SITES

17


The DevTestDeployment Workflow

Here are the main building blocks required to have a fully automated development lifecycle for your WordPress sites which is crucial to operate multiple sites at large scale

First of all you need to ensure each WordPress instance is scalable and performant as we laid out in Option 1

Beyond that there must be version control for the codebase and yoursquoll also need an automated deployment workflow and monitoring tools Optionally by adding Continuous Integration you will gain a more streamlined dev cycle with faster feedatabaseack for developers Below we explain each one of these requirements in detail with a quick guide to get you started in your real WordPress environment

Version Control for the Codebase

An easy way to get started with version control in WordPress is the SourceTree plugin which works with BitBucket There are many other source control systems you could use and you can also do this the geeky way with git

Using the SourceTree plugin moving your WordPress theme code into a BitBucket repository is as easy as

Setting up a BitBucket repository

Creating local repository in SourceTree

In repository settings add the remote BitBucket repo

Select ldquounstagedrdquo files type a commit message Commit then Push

And your code is now in version control For example if you commit something that breaks your site you can easily roll back to a previous committed version Right-click the version in the left nav and select Reverse Commit And yoursquore back to the previous version

The preceding workflow is based on the excellent post by Lucy Beer mdashread it to get all the details + screenshots


18


Deployment Automation

This will require rolling up your sleeves a bit A method we recommend for automating deployment on WordPress sites is to create a local dev environment using Vagrant and use Ansible to automatically provision the server for development and production with the appropriate differences between them

A tried and tested tool you can use is Trellis it provides a ready made stack for a WordPress server (based on Ubuntu NGINX MariaDatabase and Memcached) and provides ready-made Ansible playbooks that allow you to automatically set up this server environment Check out the Trellis Documentation for more information

If thatrsquos not cool enough for you check out this Sitepoint tutorial explaining how to deploy WordPress with Docker but be warned that we have not had experience with this in production

Backup and RestoremdashWithout Plugins

Itrsquos crucial to have an automated method for backing up and restoring your WordPress sites At large scale you donrsquot want to rely on a WordPress backup plugin because typically such plugins will use the PHP environment to perform your backups PHP is running the site and delivering pages to your users and running a backup which is a costly operation within PHP will slow down the site

So running site backups in a way that consumes PHP resources will threaten the performance and availability of your sitemdashor it will limit your ability to do regular backups to times when there is low traffic or enough free resources With multiple sites and large traffic you donrsquot want to have this constraintmdashyou should be able to back up on a regular schedule with no risk

Therefore our recommendation is to do backup and restore outside the PHPWordPress environment Generate a SQL dump of your database using a CLI tool (you can write a script which automates the required commands) and save the result of the SQL dump on a third-party storage service such as Amazon S3 In addition save an archive of your code and the WordPress-contentuploads folder which is frequently updated with multiple versions of user-uploaded files

To restore you would write a script that pulls these three things from third-party storage and uses them to reconstruct the site You would need to drop the existing database import the database archive into MySQL remove existing uploads and code and replace them with the archives you had backed up It will take time to get this working but it will result in clean backups and restores every time

To learn more about WordPress backups and ways of achieving them without relying on plugins see this detailed post by Laurence Bradford of SkillCrush


19


Monitoring

For a large scale WordPress deployment with multiple sites yoursquoll probably need several layers of monitoring

Basic Uptime Monitoring Being notified when a site goes down so you can do something about it There are numerous WordPress plugins that do this but a more industrial-strength option is a dedicated service like Pingdom

Application Performance Monitoring Monitoring the health of your WordPress application with application-specific metrics that can help you identify and diagnose problems A popular option is New Relic which provides data on WordPress MySQL performance a metric indicating how WordPress is performing for end users and visibility into external service calls such as plugins and APIs to understand what might be slowing down pages on the site

Network Monitoring If yoursquore a bit more advanced and want much more granular visibility as to whatrsquos happening on the network as WordPress serves your websites you can add an enterprise-grade monitoring tool like Nagios This post by Nick Bettison (somewhat dated) explains how to setup Nagios with a WordPress installation


20


Continuous Integration

Continuous integration is a widespread development practice in which developers frequently commit their changes to a central code repository and an automated process is run to build test and possibly deploy the software This provides an important feedatabaseack loop to developers and helps them find issues early preventing ldquointegration hellrdquo that happens after numerous developers commit complex changes to a program or website

In the context of WordPress we highly recommend setting up a CI server that sets up the WordPress environment with your latest code changes runs at least some basic automated tests and provides a quick response saying if your current version is working or not (ie if the ldquobuild is brokenrdquo)

Here is how to achieve this relatively simply with CircleCI a lightweight Continuous integration server If your WordPress installation is not super complex this could also make do as a deployment automation strategy

Include the circleyml file in your projectmdashthese are instructions for the CI environment

In circleyml define your ldquomachinerdquo and specify your database under ldquodependenciesrdquo

Use cURL to fetch the WP-CLI binary

Still in the ldquodependenciesrdquo section use WP-CLI to download WordPress

Use WP-CLI to generate wp-configphp including your database credentials

Clone your Git repository locally and use WP-CLI to install and activate your themes amp plugins

Prepare at least some basic ldquosmoke testsrdquo that automatically verify if your WordPress installation is running and functional CircleCI can then accept the result of these tests and give developers instant feedatabaseack on whether your latest build is working or not


21

WordPress Multisite at Large ScalePANTHEON

Beyond typical WordPress hosting we provide a central dashboard for managing your websites built-in CI automated deployment automatic dev environments smooth scaling to numerous WordPress servers with load balancing role-based access and much more

So what about Multisite Well on the Pantheon platform you can run a website in either one of the 3 deployment models we described Single WordPress Instance Multisite Network or Multiple WordPress Instances with Automation

You can save the trouble of building enterprise-grade infrastructure for your large-scale WordPress sites by deploying and managing your sites as a service on Pantheon

21

SINGLE WORDPRESSMULTISITE NETWORKMULTIPLE SITES WITH WINGS

Pantheon is an enterprise-grade WordPress hosting service that provides high performance and advanced DevOps features using an automated container-based platform

MULTISITE OR INDEPENDENT

Any of these scenarios

Built-in automation performance scalability and reliability

Managed serviceSERVER SERVER

SERVER

SERVER

SERVER

22

WordPress Multisite at Large ScaleMULTISITE OR INDEPENDENT

Horizontally-Scalable Architecture Pantheonrsquos engine is a big grid of powerful dedicated machines each of which runs modern containers that are an order of magnitude faster and more reliable than virtual machines Depending on your service level your website can be smoothly scaled up and replicated on as many containers as necessary with no need for manual configuration and with automated load balancing and failover making it transparent for your users Read more about the Pantheon engine

Database Replication Database scalability is the achilles heel of the traditional hosting architecture Pantheon has broken through several technical barriers to create the Content Basemdashan N+1 redundant scalable completely automated service that leverages our containers technology The Content Base acts as a database distributed file system and version control system The database service uses MariaDatabase running on optimized containers and provides redundancy and horizontal scalability using a self-healing replication topology Read more about Pantheon Content Base

Web-Scale Reverse-Proxy Cache Pantheon uses Varnish a robust high-speed battle-tested reverse-proxy cache This provides a consistent managed layer which is architecturally separate from any specific site It can soak up hundreds of millions of pageviews without missing a beat removing this load from your WordPress server Varnish is pre-integrated on Pantheon and requires no configuration Read more about Pantheon Varnish support

Persistent Object Cache Pantheon uses Redis to deliver data objects faster without loading your database When generating a dynamic request WordPress reads data objects from the database in a resource-intensive manner Redis is an in-memory key-value store with support for structured data which can cache these objects and deliver them to users without requiring access to the database or any activity on the WordPress server This can result in dramatic performance improvements

22

PANTHEON

One Large Scale WordPress Installationmdash

on Pantheon

Here are the technical requirements for a large scale WordPress instance and how Pantheon provides them

23


Optimized Search Index Pantheon has made Apache Solrmdasha robust enterprise grade search enginemdashavailable to WordPress sites running on our platform This resolves the two main issues users experience with plain WordPress searchmdashslow search once your content goes above a few thousand posts and weak search capabilities with only a chronological list of post matches Read more about Pantheonrsquos Solr integration

Development and Deployment Workflows Pantheon lets you use both version control (like git push) and direct development Developers familiar with local development using version control can push their changes to the platform Those who prefer working online can work directly in a development environment New branches are created for a feature or sprint and then integrated into the main codeline for deployment Pantheon allows you to ldquobranch work merge deleterdquo on a full-stack WordPress instance where any change to code content or configuration is fully version-controlled Read more about Pantheon version control and workflow

Continuous Integration Every site on Pantheon comes with three identical environments out of the boxmdashdev test and live This allows you to run the industry-standard continuous integration workflow You can run automated testing or conduct manual QA and regression testing This shortens the feedatabaseack loop for developers allowing development to happen in parallel with QA

Licensing Note The devdeployment workflow features are only available as part of Pantheon Enterprise Also above 500000 monthly page views andor 30GB total storage for all sites you will need to upgrade to the Elite Plan Other features such as horizontal scalability caching and search are partially supported in our regular plans starting from $25 per month For specifics please see our pricing page

23

PANTHEON

24


Managed Horizontal Scaling for Multisite A Multisite has several dimensions of growth because each of Multisite network might grow at a different pace in terms of user load number of editors plugins used and changes made to the site To accommodate this Pantheon assigns an enterprise onboarding manager with scalability expertise who will deploy your site on a number of containers appropriate for your load and in coordination with you add or remove containers when load changes Scale events happen transparently with no migration of your Multisite and no downtime Read more about scalability in Pantheon

Cluster-based Architecture for Resilience Your Multisite network and its data are always replicated across at least 2 cluster nodes If a machine or container fails the Multisite self-heals seamlesslymdashyou or your users wonrsquot even know about it Read more about high availability on Pantheon

Performance Features Performance is particularly important for a Multisite because of the potentially massive load of users and editors Pantheon offers a built-in reverse-proxy cache (Varnish) persistent object cache (Redis) and enterprise grade search performance with Solr We are the fastest WordPress hosting on the planet with page response time less than half that of hosting services like WordPressEngine or GoDaddymdashsee our performance benchmark

Replica Database and HyperDB In a Multisite there can be a large number of tables in the WordPress database and database readwrite operations need to be managed A Multisite on Pantheon comes with a dedicated database replica and you can use the WordPress HyperDB plugin to shift writes and reads to the replica The drop-in plugin supports failover load balancing and database partitioning Pantheon is pre-integrated with HyperDB making it easier to set up (but still requiring technical knowledge) Read more about HyperDB on Pantheon

24

PANTHEON

WordPress Multisite Networkmdashon Pantheon

Pantheon lets you run a WordPress site on our platform and specify that it should be a Multisite Network Please note that to enable multisite yoursquoll have to upgrade your site to our Elite Plan at additional cost

Your Multisite Network on Pantheon will behave exactly the same as a locally hosted WordPress Multisite But there are a few important differences on the infrastructure and maintenance side

25


Object Caching with Redis for Scalability in Number of Editors while user read operations can easily be replicated to additional WordPress servers (using Pantheonrsquos container architecture) write operations by editors are more difficult to scale HyperDB will help but yoursquoll also need an object cache to prevent reading from the database each time an editor accesses the same form or opens a page for editing

By caching these operations and storing them on another servermdashthe Redis cache servermdashyou offload these operations from your Multisite which dramatically boosts performance Redis is built-into Pantheon and can be activated at the push of a button with no configuration Read more about Redis on Pantheon

One-click Updates of WordPress Core Easily push updates to the Multisite instance with no downtime

25

PANTHEON

26


All Pantheon WordPress instances come with these automation capabilities built-inmdashto learn more see the section ldquoOne Large Scale WordPress Installationmdashon Pantheonrdquo

Beyond ldquoordinaryrdquo automation and DevOps practices a unique challenge of managing multiple sites is managing a central code base for shared elements like server configuration utility plugins etc while having a separate code base for specific site features Pantheon lets each team in your organization easily create and update sites from a common codebase while allowing individual sites to branch out with their own specific functionality Herersquos how it works

Create a Git repository which will be the source for your common code (use our pre-tested distributions or create your own)

Provide a link and access to your repository and your Pantheon account will get access to the codebase

Create sites based on that codebasemdashdevelopers can start using the latest and greatest code from your repo

Install and develop your sites as usual Pantheon gives each site its own Git repository to store site-specific changes But each site retains a connection to the original distribution enabling updates

Commit to one placemdashyour common repository Once that commit has been made you are alerted on each site that an update is available

With a single click you can apply that update to any site

If there are conflicts we have an option to auto-resolve in favor of the upstream repository (or manual resolution using the command line)

26

PANTHEON

Multiple WordPress Sites with Automationmdash

on Pantheon

As we mentioned in our page on Managing Multiple WordPress Sites with Automation (Option 3) devtestdeployment workflow automation is key to managing multiple mission-critical WordPress sites

27


Learn more about how to build sites from a common codebase in Patheon

Pantheon Dashboard As a bonus our dashboard provides the ability to filter for sites that are running off your custom common codebase Once filtered you can also see the ldquoCode Statusrdquo for each site to quickly drill down to sites that are up to date or sites that still have updates to apply This makes it extremely easy to manage the evolution of your sites even as the number of sites grows to dozens or hundreds

Licensing Note The custom upstream feature that powers the functionality we describe above is only available as part of Pantheonrsquos Enterprise Plan Agency Partner or Pantheon for EDU+ program

MULTISITE OR INDEPENDENTPANTHEON

SCALE WORDPRESS ON PANTHEON

Get started with Pantheon for your large scale WordPress deployment

LEARN MOREpantheoniowordpress-hosting

27

2


WordPress Multisite Networks for Very Large ScaleA How to Guide

So yoursquove decided to go with WordPress for your large scale website There

are three basic options for setting up the site

Option 1

If itrsquos a single brand with design consistent across the entire site and you have one team with up to 50-60 people working on it and they all work together you should go for a regular WordPress install (no need for something like Multisite)

Option 2

If there are multiple departments or organizations and each of them will have its own subdirectory or subdomain each with a different design but quite a few common denominators between the different sites a good choice will be WordPress Multisite WordPress Multisite will allow you to have one network administrator who will create one or more sites and then user accounts who can manage each site without necessarily being able to manage the entire network There can be one central thememdashmanagers of each site can select different options in the sitersquos admin interface to achieve a different look and feel Yoursquoll need to build these options into the code of your theme To allow more customization at the code level you can use a base theme and child themes

Option 3

If there are multiple different sites or brands and the codepluginsgoalsfunctionality of each site is very different andor there are different teams working on each site and each of them need to develop different functionality at the code levelmdashMultisite is probably not suitable Each site should be defined as a separate WordPress site and yoursquoll have to manage your multiple sites using automation Automatic deployment and updates will be needed to handle the complexity and large volume of changes across the multiple sites as well as issues like robustness resilience and scale

2

3













Option 2

Option 3

PANTHEON





4






SINGLE WORDPRESS


SINGLE INSTALL







Single brand



5









MySQLMASTER

MySQLREPLICA

MySQL n


MySQLMASTER

MySQLREPLICA

PHP APP 1 PHP APP 2

7

525

35

175

020 50 100 150 200 250

LOAD v CONCURRENCY

VARNISH BATCACHE

70

55

40

25

1020 50 100 150 200 250

VARNISH BATCACHE



6











MySQL

OBJECT CACHE

PHP APP PHP APP



7










MySQLMASTER

MySQLREPLICA

PHP APP 1 PHP APP 2




8














9










MySQL MySQL

PHP APP PHP APP






SEARCHINDEX

10




Linux

Varnish

Apache or Nginx

PHP

Memcached or Redis


MySQL











11






WORDPRESS MULTISITE





12







Users Are Shared





13



Scale

Theming Strategy






14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

3













Option 2

Option 3

PANTHEON





4






SINGLE WORDPRESS


SINGLE INSTALL







Single brand



5









MySQLMASTER

MySQLREPLICA

MySQL n


MySQLMASTER

MySQLREPLICA

PHP APP 1 PHP APP 2

7

525

35

175

020 50 100 150 200 250

LOAD v CONCURRENCY

VARNISH BATCACHE

70

55

40

25

1020 50 100 150 200 250

VARNISH BATCACHE



6











MySQL

OBJECT CACHE

PHP APP PHP APP



7










MySQLMASTER

MySQLREPLICA

PHP APP 1 PHP APP 2




8














9










MySQL MySQL

PHP APP PHP APP






SEARCHINDEX

10




Linux

Varnish

Apache or Nginx

PHP

Memcached or Redis


MySQL











11






WORDPRESS MULTISITE





12







Users Are Shared





13



Scale

Theming Strategy






14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

4






SINGLE WORDPRESS


SINGLE INSTALL







Single brand



5









MySQLMASTER

MySQLREPLICA

MySQL n


MySQLMASTER

MySQLREPLICA

PHP APP 1 PHP APP 2

7

525

35

175

020 50 100 150 200 250

LOAD v CONCURRENCY

VARNISH BATCACHE

70

55

40

25

1020 50 100 150 200 250

VARNISH BATCACHE



6











MySQL

OBJECT CACHE

PHP APP PHP APP



7










MySQLMASTER

MySQLREPLICA

PHP APP 1 PHP APP 2




8














9










MySQL MySQL

PHP APP PHP APP






SEARCHINDEX

10




Linux

Varnish

Apache or Nginx

PHP

Memcached or Redis


MySQL











11






WORDPRESS MULTISITE





12







Users Are Shared





13



Scale

Theming Strategy






14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

5









MySQLMASTER

MySQLREPLICA

MySQL n


MySQLMASTER

MySQLREPLICA

PHP APP 1 PHP APP 2

7

525

35

175

020 50 100 150 200 250

LOAD v CONCURRENCY

VARNISH BATCACHE

70

55

40

25

1020 50 100 150 200 250

VARNISH BATCACHE



6











MySQL

OBJECT CACHE

PHP APP PHP APP



7










MySQLMASTER

MySQLREPLICA

PHP APP 1 PHP APP 2




8














9










MySQL MySQL

PHP APP PHP APP






SEARCHINDEX

10




Linux

Varnish

Apache or Nginx

PHP

Memcached or Redis


MySQL











11






WORDPRESS MULTISITE





12







Users Are Shared





13



Scale

Theming Strategy






14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

6











MySQL

OBJECT CACHE

PHP APP PHP APP



7










MySQLMASTER

MySQLREPLICA

PHP APP 1 PHP APP 2




8














9










MySQL MySQL

PHP APP PHP APP






SEARCHINDEX

10




Linux

Varnish

Apache or Nginx

PHP

Memcached or Redis


MySQL











11






WORDPRESS MULTISITE





12







Users Are Shared





13



Scale

Theming Strategy






14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

7










MySQLMASTER

MySQLREPLICA

PHP APP 1 PHP APP 2




8














9










MySQL MySQL

PHP APP PHP APP






SEARCHINDEX

10




Linux

Varnish

Apache or Nginx

PHP

Memcached or Redis


MySQL











11






WORDPRESS MULTISITE





12







Users Are Shared





13



Scale

Theming Strategy






14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

8














9










MySQL MySQL

PHP APP PHP APP






SEARCHINDEX

10




Linux

Varnish

Apache or Nginx

PHP

Memcached or Redis


MySQL











11






WORDPRESS MULTISITE





12







Users Are Shared





13



Scale

Theming Strategy






14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

9










MySQL MySQL

PHP APP PHP APP






SEARCHINDEX

10




Linux

Varnish

Apache or Nginx

PHP

Memcached or Redis


MySQL











11






WORDPRESS MULTISITE





12







Users Are Shared





13



Scale

Theming Strategy






14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

10




Linux

Varnish

Apache or Nginx

PHP

Memcached or Redis


MySQL











11






WORDPRESS MULTISITE





12







Users Are Shared





13



Scale

Theming Strategy






14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

11






WORDPRESS MULTISITE





12







Users Are Shared





13



Scale

Theming Strategy






14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

12







Users Are Shared





13



Scale

Theming Strategy






14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

13



Scale

Theming Strategy






14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

14














15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

15



Why Automation












16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

16












17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

17
















18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

18













19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

19


Monitoring






20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

20














21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

21





21







SERVER

SERVER

SERVER

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

22






22

PANTHEON


on Pantheon


23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

23






23

PANTHEON

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

24






24

PANTHEON




25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

25





25

PANTHEON

26











26

PANTHEON


on Pantheon


27









27

26











26

PANTHEON


on Pantheon


27









27

27









27

WordPress Multisite at Large Scale - · PDF file3 WordPress Multisite at Large Scale For all...

Documents

Transcript of WordPress Multisite at Large Scale - · PDF file3 WordPress Multisite at Large Scale For all...