APIs: The Problems with Eating your Own Dog Food

Post on 27-Jan-2015

110 views 1 download

Tags:

description

SoundCloud's web and mobile properties are all built on top of our Public API. While building the user-interface atop a RESTful layer has proven itself a sound decision, the one-size-fits-all nature of a Public API is not ideal. When it comes to data transfer and HTTP resource modelling, each client has their own needs, and in the end hacks and workarounds have to be implemented in both clients and servers. Feature development also becomes complicated, with coordination between multiple teams required for every single little feature. SoundCloud is now moving to a different model, where clients have their own façade APIs, modelled after their core characteristics and needs. We are also using the architecture to break away from Conway’s Law and building more cross-functional, end-to-end teams. From: http://gotocon.com/berlin-2013/presentation/APIs:%20The%20Problems%20with%20Eating%20your%20Own%20Dog%20food To implement this, a lot of change in our architecture, tech stack and development processes were required. In this talk we will explore the challenges we had, the options we investigated and how ultimately SoundCloud decided to move forward.

Transcript of APIs: The Problems with Eating your Own Dog Food

The problems with

dog food

eating your own

Phil CalçadoSoundCloud

Thursday, October 17, 13

Thursday, October 17, 13

>11 hours ofaudio uploaded every minute

Thursday, October 17, 13

~200million usersevery month

Thursday, October 17, 13

Thursday, October 17, 13

Thursday, October 17, 13

http://bit ly/tech-behind-new-twittercomThursday, October 17, 13

Thursday, October 17, 13

Just before

releasethe beta

Thursday, October 17, 13

http://bit ly/improving-performance-twittercomThursday, October 17, 13

Thursday, October 17, 13

Ship it?

Thursday, October 17, 13

Ship it!(just overprovision the api)

Thursday, October 17, 13

~3 back-end requests

Thursday, October 17, 13

~159 back-end requests

Thursday, October 17, 13

haproxywere the

First to break

Thursday, October 17, 13

memcachedThen

Thursday, October 17, 13

Rails/MySQLThen

Thursday, October 17, 13

need faster api

Railswon’t go away

Thursday, October 17, 13

SoundCloud.com

Thursday, October 17, 13

Sounds & Sets

Social Graph

Premium Features Search

Activity Stream

API

Thursday, October 17, 13

Sounds & Sets

Social Graph

Premium Features Search

Activity Stream

API

Thursday, October 17, 13

0%

25%

50%

75%

100%

Original Services Extracted

Database Other HTTP

Thursday, October 17, 13

0%

25%

50%

75%

100%

Original Services Extracted

Database Other HTTP

Thursday, October 17, 13

0%

25%

50%

75%

100%

Original Services Extracted

Database Other HTTP

about the same, tbh

Thursday, October 17, 13

need faster Rails

Railswon’t go away

Thursday, October 17, 13

Thursday, October 17, 13

lots of I/Oconcurrency friendly

Thursday, October 17, 13

parallelismdoesn’t like

Rails

Thursday, October 17, 13

timeThursday, October 17, 13

concurrency?

Can wemake it like

Thursday, October 17, 13

timeThursday, October 17, 13

timeThursday, October 17, 13

Thursday, October 17, 13

Thursday, October 17, 13

is there?

Whatelse

Thursday, October 17, 13

Thursday, October 17, 13

Thursday, October 17, 13

Thursday, October 17, 13

Thursday, October 17, 13

Thursday, October 17, 13

timeThursday, October 17, 13

timeThursday, October 17, 13

Thursday, October 17, 13

about

the network?

What

Thursday, October 17, 13

GET /tracks/trending/music

Thursday, October 17, 13

GET /tracks/genres

GET /tracks/trending/music

Thursday, October 17, 13

GET /tracks/trending/music

GET /tracks/genres

GET /tracks/genres/jazz

Thursday, October 17, 13

GET /tracks/trending/music

GET /tracks/genres

GET /tracks/genres/jazz

GET /tracks/123Thursday, October 17, 13

a custom API?about

What

Thursday, October 17, 13

GET /tracks/suggested

Thursday, October 17, 13

GET /tracks/suggestedGET /tracks/123

Thursday, October 17, 13

api.soundcloud.com

iOS Android Desktop Widget 3rd PartyThursday, October 17, 13

api-mobile

iOS Android Desktop Widget

api web api-partn

3rd Party

Thursday, October 17, 13

comes

next?

What

Thursday, October 17, 13

babysitting

Rails

Stop

Thursday, October 17, 13

Enable &

app devsempower

Thursday, October 17, 13

“Experience

api?based”

Thursday, October 17, 13

phil calçado

http://philcalcado.com@pcalcado

www.soundcloud.com

Thursday, October 17, 13