Post on 22-Apr-2015
description
How we maintain 200+ Drupal sites in
Georgetown University
Vadym Myrgorodvm386@georgetown.edu
@dealancer
About me
Doing Drupal for more then 6 years
Originally from Ukraine
Currently work in Georgetown University
About Georgetown University
Was founded in 1789 a year earlier then the Washington D.C. was founded
Attended by Bill Clinton, Jose Manuel Barroso
Approx. 100 employees in IT department
There is a big migration into Drupal going on right now
258 sites that looks the same
Session plan
What infrastructure we have
What development workflow we use
How we perform daily tasks
How we create new sites
Infrastructure
Unit install profile School install profile
Drupal CMS
FeatureA
FeatureB
FeatureC
FeatureD
FeatureE
Site 1 Site 2 Site 3 Site 4 Site 5 Site 6 Site 7
Unit install profile School install profile
Drupal CMS
FeatureA
FeatureB
FeatureC
FeatureD
FeatureE
Multisiting
sites.php
<?php
#biology $sites['biology.georgetown.edu'] = 'biology'; $sites['biology.gudrupalstg.georgetown.edu'] = 'biology'; $sites['biology.gudrupaldev.georgetown.edu'] = ‘biology';
Multisiting
Site 1 Site 2 Site 3 Site 4 Site 5 Site 6 Site 7
Well known Drupal cloud
Dev Test Prod
Local development
Setup any subsite locallywithin a few minutes
Sitesync script
Creates files directory
Exports DB from dev/test/prod and imports it locally
Enables dev modules
Re-saves theme settings
Clears cache
Logs you in
Script highlights
# Getting DB from the remote server ssh $HOST "drush @gudrupal.$ENV -l $SUBSITE sql-dump --skip-tables-key=common --gzip" > $DUMPFILE_GZ
# Resave theme settings drush @gudrupal.local -l $SUBSITE php-eval "# module_load_include('inc', 'system', 'system.admin'); foreach (array('at_georgetown') as \$theme_name) { \$form_state = form_state_defaults(); \$form_state['build_info']['args'][0] = \$theme_name; \$form_state['values'] = array(); drupal_form_submit('system_theme_settings', \$form_state); } "
local.settings.inc
Stores local credentials and local settings
Is included into common.settings.phpif (file_exists('sites/local.settings.inc')) { require_once('sites/local.settings.inc');}
local.settings.inc
Stage File Proxy$remote_url = 'http://' . $sitedir . '.georgetown.edu'; $conf[‘stage_file_proxy_origin'] = $remote_url; $files_dir = ‘sites/' . $sdir . '/files'; $conf[‘stage_file_proxy_origin_dir'] = $files_dir;
Reroute Email$conf['reroute_email_enable'] = 1; $conf['reroute_email_address'] = 'email@example.com';$conf['reroute_email_enable_message'] = 1;
Development workflow
Deploy new sites,don’t deploy untested code
http://nvie.com/posts/a-successful-git-branching-model/
A successful Git branching model
Environments
Dev Test Prod
develophotfixes
personal branchesfeatures branches
master branchtag tag
Change Control
Script deployment and backup plan
Schedule deployment in the outage window
Perform deployment
Test changes
Roll back if necessary
All of above is done on Test and Prod environments
Be careful with Featuresdo not overuse it
Features are not a panacea
Avoid storing any objects that contains numeric IDs, because same objects can have different IDs across multiple sites
AUTO_INCREMENT delta could be different
Our vocabulary permission and IMCE profiles were lost cause of AUTO_INCREMENT delta change
Use hook_install and hook_update_N as a workaround
DevOps
Run certain amount of tasks almost each day across 258 sites
How I did it 6 years ago
Hit update.php url for every site
Go to the Features page and revert a feature one by one
Go to the modules page and enable new modules
Advice from my PM: do as described above if it is urgent :)
What I’ve discovered
Use Drush:$ drush updb -y
Use Drush aliases:$ drush @gudrupal.prod -l biology fra -y
Squeeze every penny out of Drush aliases:$ drush @sites dis devel
What I am doing now
$ ./gudrupal-bulk-drush.sh subsites.txt prod "updb -y"
"fra -y" "cc all" > ~/deployment-2014-07-21.txt
What script can do that Drush can’t
Perform multiple Drush operations in a batch
Perform operations on multiple sites
Control sites order
Set timeout between iterations
What else can be done
Run PHP code fore every site
Run bash commands for every running server
Creating a new site
We needed to create 80 new websites for less then a week
Before we launched at least 5 sites a week
It took me more then a day to create my first new site
4 scripts to automate site creation
Site creation
Site installation
Sign SAML file
Sync site with other environments
Site creation script
Checks if new site name is valid and site does not exist
Creates subdirectory in sites directory
Copies settings.php file
Adds domains into sites.php
Commits changes
Adds dev, test and prod domain names
Cretes dev, test and prod databases
Site installation script
Runs a specific install profile via drush
Resaves theme settings
Updates regional settings
Script that adds new SAML record
Creates backup of old SAML metadata file
Launches editor so user can make changes in metadata
Signs metadata file
Makes user to confirm changes
Commits changes
Sync script
Copies files from one environment to another
Copies database from one environment to another
Finally, bulk scripts that do all previous tasks in a batch
Then and now
It took ~3 hours of work to create a website manually
Now we can create as many websites as we need with running just a single script
I saved 1.5 month of work to create 80 new websites just running a script overnight
Why do producers of the hardware make it slower than it
actually could be?
Add a delay into your bulk scripts to prevent server from the high load
Introducing Druml
github.com/georgetown-university/druml