Transforming legacy applications with Symfony2 and Varnish

JULY 2012

10 CONCURRENT USERS

~20 SECONDS AVERAGE

HOMEPAGE -> LOGIN -> PERSONAL DETAILS

Why so slow?

PHP4

SQL Server and Windows Server

Sub-optimal caching strategy

“Rewriting the code from scratch ... (is) the single worst strategic mistake a software company can make.”

- Joel Spolsky

“... the quick fix is like quicksand: The clarity of code goes down and confusion is harvested in its place.”

- Venkat Subramaniam and Andy Hunt

The Plan

Reduce cost, improve performance

Develop new application framework

Gradual deployment to careerswales.com

API to access shared data

Single sign on and shared sessions

Legacy CMS New Applications

CAREERSWALES.COM

... ENTER SYMFONY2

Why Symfony2?

Used successfully in other projects

Excellent 3rd party libraries

Flexible and easy to extend

Facilitates best practice

PART 1: THE CMS

Catalyst CMS

Cutting edge stack - PHP 5.4, Nginx, FPM

Reusable code through Symfony2 bundles

Inspired by Symfony2 CMF and Sonata

Fraction of the code to maintain

PART 2: THE API

Take it to the bridge

Bridges MSSQL & MySQL databases

Acts as a facade

Linux + MSSQL = PAIN

Caching with Symfony2 and Doctrine

SQL Server

API Application (Symfony2)

Third parties

MySQLWebsite

Application(Symfony2)

PART 3: SINGLE SIGN ON

Single sign on

Single authentication system for all applications

Uses Central Authentication Service (CAS) protocol

BeSimpleSsoAuthBundle does heavy lifting

Custom modifications via DI container

PART 4: VARNISH

Varnish

Frontend proxy or HTTP accelerator

Sits between client and server

Shares cached responses between clients

Can perform load balancing and routing

Without Varnish

Client

Client

Client

HTTP://WWW.CAREERSWALES.COM/EN/

Without Varnish

Client

Client

Client

TTL = 1 DAY

TTL = 1 DAY

TTL = 1 DAY

HTTP://WWW.CAREERSWALES.COM/EN/

With Varnish

Client

Client

Client

Varnish

HTTP://WWW.CAREERSWALES.COM/EN/

With Varnish

Client

Client

Client

Varnish

TTL = 1 DAYTTL = 1 DAY

TTL = 1 DAY

TTL = 1 DAY

HTTP://WWW.CAREERSWALES.COM/EN/

With VarnishClient

Client

Client

Varnish

Client Client Client Client

HTTP://WWW.CAREERSWALES.COM/EN/

With VarnishClient

Client

Client

Varnish

Client Client Client Client

HTTP://WWW.CAREERSWALES.COM/EN/

ESI CachingEdge Side Includes

Identifies blocks of HTML which are dynamic

Blocks are independent of the main response

Blocks can be cached and expired individually

Blocks can be identified as public (cacheable) or private (not cacheable)

TTL = 5 MINS

TTL = 5 DAYS

INVALIDATED

INVALIDATED

INVALIDATED

Carousel Block Caching

Straightforward caching

No personalisation, same for all users

Carousel Block Cachingpublic function indexAction(){ $response = new Response(); // cache for 1 minute $response->setMaxAge(60); $response->setPublic(); return $this->render( 'BoxUK:Default:index.html.twig', array(), $response );}

Login Button Caching

More complex: two different states

Can use varying to store two different caches

Use cookies to provide information to Varnish

BUT don’t want to vary on cookie

Login Button Cachingpublic function indexAction(){ $response = new Response(); // cache for 1 minute $response->setMaxAge(60); $response->setPublic(); $response->setVary('Logged-In'); return $this->render( 'BoxUK:Default:index.html.twig', array(), $response );}

Login Button Cachingpublic function onKernelResponse( FilterResponseEvent $event){ $response = $event->getResponse();

$loggedIn = $this->context->isGranted( 'IS_AUTHENTICATED_FULLY' ) ? 'true' : 'false';

$response ->headers ->set('Logged-In', $loggedIn);}

Login Button Cachingpublic function onKernelResponse( FilterResponseEvent $event){ $response = $event->getResponse();

$cookie = new Cookie('Logged-In',$loggedIn, ...);

$response ->headers ->setCookie($cookie);}

Login Button Caching

if (req.http.Cookie ~ 'Logged-In=true') { set req.http.Logged-In = 'true';}

Varnish cache keys are hashed on URL (host, path, etc) and vary data

Symfony takes care of the way out

On the way in, we have to fake the header to reproduce the hash

Routing

Varnish directs requests to given ‘backends’

Load balancing, e.g. round robin

More complex logic via VCL (Varnish Configuration Language)

No EC2 load balancers!

RoutingOld

applications

New applications

10.1.2.1

10.1.2.2

Client Varnish

Examine request

Defining backendsbackend legacy_applications { .host = "10.1.2.1"; .port = "80";}

backend new_applications { .host = "10.1.2.2"; .port = "80";}

Routing by regexsub vcl_recv {! if (req.url ~ "^(.*)www.careerswales.com/old-request-format") {! set req.backend = legacy_applications;! return(pipe);! }! if (req.url ~ "^(.*)www.careerswales.com/new-request-format") {! set req.backend = new_applications;! return(pipe);! }}

OLD APPLICATIONS

Client

Varnish

NEW APPLICATIONSSQL

Server MySQL

SSO App

API APP

www.careerswales.com/old-request-format

www.careerswales.com/new-request-format

JULY 2012

10 CONCURRENT USERS

~20 SECONDS AVERAGE

HOMEPAGE -> LOGIN -> PERSONAL DETAILS

JULY 2013

10 CONCURRENT USERS

~4 SECONDS AVERAGE

HOMEPAGE -> LOGIN -> PERSONAL DETAILS

Conclusions

Observable speed increase of 560%No loss of functionality or downtime

No loss of functionality or downtime

Substantially less ‘hardware’

More complex environment

Questions?

@craigmarvelley