A Pure Digital News Service for Ireland

14 Adorable Pictures of Cats Coding

How We Grew TheJournal.ie

John Needham – Web Developer

Who Are We?•Irish and International Content•News, Sports, Entertainment and discussion•Launched Sept 2010•iOS and Android apps•Desktop and touch

websites

Who Are We?

•25 Journalists•2 Web Developers•2 App Developers•1 Cat

Site Launch

Basic Wordpress site:

• Quick to build• Large feature set• Large and experienced WP talent pool• Lots of online help• Good CMS

Launched our iOS app within first monthAndroid app Feburary 2011

Site Growth

1 million page views a month after 3 months:

• WP does not scale easily• Lots of features = Complex code base• Debugging a nightmare (hooks..)• No native caching• ( °□°╯ ）╯︵ ┻━┻

Site Growth - Maintenance

Developing in WordPress:

• Tried to use the WP framework (plugins)• Slow and messy (hooks)

Gave Up:

• Edit the core code• Faster development• Upgrading issues

Site Growth - Maintenance

$query->get_posts();1 Function – 875 lines

Site Growth - Caching

Caching in WordPress is a nightmare:

• All plugin based• Used w3-cache - cached to file• Dynamic Elements – not maintainable• Want to increase amount of user specific content

Site Growth – Reliability

Dara O'Briain – crashes the site with only 10k views

Solutions?

Buy – options are too customised

More Cache (varnish?) – wont fix dynamic content problem

Build New System – Resources and time?

Our Solution

Goal – Build a simple API to power site and apps

• Use WordPress database, keep the CMS• Cherry-pick from WordPress for formatting and styling

• Lightweight and simple PHP front end• All in EC2

Site Structure

Challenges - Caching

110,000 posts 100,000 tag rivers 1,063,000 comments

Solution:• Cache all objects forever• Cache all rivers by IDs• Immediate cache invalidation

Challenges - Content

Making Content Work Everywhere

• Platforms: browser, iOS and Android• Rich media support? (flash, silverlight, gifs)

Journalists. Awkward.

Challenges - Content

Solution:

• Make our server do as much heavy lifting as possible

• Keep app html parsing to zero• Handle unknown embeds as a link• A lot easier to update a server than an app

Challenges - Content

- Faster render time for apps

- Unknown content still accessible

Mistakes along the way

Overdependent on external plugins• Liveblogs, JSON API, Social

Could have made better use of (small) frameworks• Avoid learning lessons the hard way.• Slim Framework

Doing Friday Deploys

Mistakes along the way

EC2 does quirky things:

• Random IP changes causing an accidental DDoS

• Unpredictable Auto-Scaling – killing a healthy Varnish instance

Mistakes along the way

Keeping Clients in Sync

• Could stop comment on apps, not on desktop...

Some Lessons Learned Lots of good start-ups to use:

• Pusher – realtime websocket api, used on liveblogs and scores

• HostedGraphite – hassle free stats, essential for our API

Some Lessons Learned

Profile EVERYTHING

• Xhprof• 3x load time improvement

Some Lessons Learned

Know when Good Enough is OK:

• Too tempting to try and perfect everything• Ship it, Iterate

Hat tip: @dermdaly

Results

Desktop Growth in 2012:

Results

3x mobile growth in 2012:

Results

Traffic?• Handling 50 million page views a month

Spikes?• Reddit test 2 weeks ago• 477,735 page views• No crash :)

Results

Rolling out new features on new site

• Easier to release new features

• Live scores rolled out over last few weeks

Going Forward

Continue our growth – 3x in 3 years Build our commenter communityNumber 1 news service in Ireland by next year

We're Hiring!

www.thejournal.ie/jobs/

Can find me @JohnANeedham

Thanks to:Miguel MolinaBrian Fallon (@brianfallon)Paul Conroy (@conroyp)Steven McMemeBride (@steemcb)