SUPERSIZE MEMaking Drupal go large

INTRODUCTION

•Who are we?

• Building BIG Drupal

• Three pillars of scale

• Care in the community

•Q&A

WHO ARE WE?

COMMON MISCONCEPTIONS

• “Drupal is slow”

• “Drupal doesn’t scale”

• “It’s only good for a hobby site”

WRONG!

DEVELOPING BIG WEBSITES WITH DRUPAL IS NO DIFFERENT TO

DEVELOPING THEM WITH ANY OTHER SOFTWARE

DEVELOPING BIG WEBSITES WITH DRUPAL IS NO DIFFERENT TO

DEVELOPING THEM WITH ANY OTHER SOFTWARE

DRUPAL IS NO DIFFERENT TO

ANY OTHER SOFTWARE

SCALING PERFORMANCEThe first pillar-box of scale

SCALING COMPLEXITY

• Caching and Reverse proxy caches

•Queues

• Alternative database backends

• Throttling

MONITORING

• All the common open source tools...

•Munin, Nagios, etc

• Plus some new ones

•New Relic, Boundary

• Social media

• saved our site...

SCALING PEOPLEWe’re going to need a bigger boat

COMMUNICATION

PRIORITISATION

DISTRIBUTED WORKING

REDUNDANCY

PROJECTS CONSIST OF CODE

PROJECTS CONSIST OF CODE

CODE IS WRITTEN BY PEOPLE

PEOPLE DELIVER PROJECTS

PROJECTS CONSIST OF CODE

CODE IS WRITTEN BY PEOPLE

SCALING PROCESSES

YOU STILL NEED TO DO ALL THE THINGS YOU WOULD DO

ON ANY BIG SOFTWARE PROJECT

WHAT THINGS?

• Provide clarity of purpose

• Automated and manual testing

• Continuous integration

• Performance testing and benchmarking

• Code review

• Source control

• Release management

• Address technical debt

• Be agile (with a little ‘a’)

CLARITY OF PURPOSE

• Your team must know how to “do”

• The definition of “done”

•What “good” looks like

TESTING

• Know how you’re going to test, before you write a single line of code

•Do it regularly

•Make it repeatable and consistent

• Automate as much as possible

• Test Driven Development (TDD) or Behaviour Driven Development (BDD) is even better

CONTINUOUS INTEGRATION

• Fail fast, Fail early

• Start your project the way you mean to go on

• Provide an environment where developers are allowed to make mistakes (not just dev)

• Automate your environment setup

• Regularly build your environments - including deploying to production

“IT DOESN’T MATTER HOW YOUR BUILD SCRIPT WORKS, IT ONLY MATTERS THAT IT

WORKS”

PERFORMANCE TESTING

• Know what you’re aiming for

• Test early, test regularly

• Benchmark common journeys and monitor changes over time

•Memory usage, Page execution time etc

• Acknowledge that Production is your only real test

KNOW YOUR PEAKS

CODE REVIEW

TECHNICAL DEBT

CARE IN THE COMMUNITYThe good news

YOU DON’T NEED TO REINVENT THE WHEEL

YOU’RE NOT THE FIRST

• Pressflow

• Published approaches to solving performance problems

• Catch’s Github page

•Drupal.org documented performance improvements

•Drupal 7 Fields optimisation (ported from Drupal 8)

• Porting performance improvements from newer versions

EXPERTS ARE OUT THERE

•Drupal issue queues

•Drupal High Performance Group

• IRC

•Meet-ups

•DrupalCamps and DrupalCons

• Large Scale Drupal (LSD) programme

DRUPAL PLAYS WELL WITH OTHERS

CONTRIBUTION

• Contribution encourages good developer practice

•Testing: core changes always accompanied by a test

•Review: minimum of 2 people will review any change to core

•Technical debt: critical issue thresholds block new features

SHARE AND SHARE ALIKE

• Route

• STOMP

• SQL No Revisions

• Drupal 8 Configuration Management Initiative (CMI)

• Test and Target

• Panels Filter Cache

• Various core and contrib patches

CONTRIBUTION CYCLE

WRAPPING UP

DRUPAL IS NO DIFFERENT TO

ANY OTHER SOFTWARE

THE COMMUNITY IS

WRAPPING UP

• If you’re building big, a lot of problems exist regardless of the technology

•Drupal provides ways to solve these problems

• The Drupal community is one of the best communities to have a problem in

•Drupal does scale, in performance, in people, in process

QUESTIONS?

Tweet: @tsphetheanDrupal.org: tsphethean

IRC: tsphethean

Tweet: @alexpottDrupal.org: alexpott

IRC: alexpott

WE’RE HERE ALL WEEKENDCome visit us on our stand

and see our other presentations

• Drupal 8 Configuration system for coders - Alex Pott (Sunday 10:30am)

• Please hold: your call is in a queue - Tom Phethean (Sunday 10:30am)

• zip-BDD-do-dah zip-BDD-ay! - Graham Taylor and Andrew Larcombe (Sunday 11:30am)

IMAGE ATTRIBUTIONS

• The Elephants are coming - http://www.fromthewilderness.com/free/ww3/011905_university_washington.shtml

• Pillar Box - http://www.oxfordhistory.org.uk/streets/post_boxes/edward_vii/index.html

• Jaws - http://edge.ascd.org/_Youre-Gonna-Need-a-Bigger-Boat/blog/6287739/127586.html

• Communication - http://familybugs.wordpress.com/2012/04/30/communication-part-1/

• Prioritisation - http://neilkillick.com/2011/12/17/tips-on-speedy-product-backlog-prioritisationordering/

• Remote working - http://www.wolfden.co.uk/RemoteWorking.html

• Redundancy - http://blog.800hosting.com/2011/07/workstation-backup-solutions-pt-3-redundancy/

• Scaling rock face - http://www.flickr.com/photos/alexindigo/2571638877

• Christmas - http://www.tunisia-live.net/2011/12/23/christmas-celebrations-in-tunisia-tis-the-season/

• Volcanic ash cloud - http://www.guardian.co.uk/world/blog/2010/apr/15/volcano-airport-disruption-iceland

• Technical debt - http://agileandbeyond.blogspot.co.uk/2011/05/velocity-handle-with-care.html

• Reinventing the wheel - http://www.sbi-conferences.com/dont-reinvent-the-wheel.html