Finagle @ SoundCloud
Phil CalçadoSoundCloud
> 11 hours of audio uploaded every minute
~ 300 million people every month
some more micro than others
~100 services ~130 engineers
each service has a cost we need to reduce
fixed cost
1) complexity cost
edge
value-added
foundation
edge
value-added
foundation
api-web
user-profile
user-metadata playlists
api-android
• Building in Scala should be a no-brainer
• Building in other JVM languages should be easy
• Building in something else should be possible
2) setup cost
+
3) RPC cost
the only thing these things had in common
was HTTP+JSON
typical flow
1. Get the IDs of all tracks by an artist
2. For each ID:2.1.Get track metadata from X2.2.Get usage permissions from Y2.3.Get comments from Z
3. Return list to client app
typical flow
1. Get the IDs of all tracks by an artist
2. For each ID:2.1.Get track metadata from X2.2.Get usage permissions from Y2.3.Get comments from Z
3. Return list to client appparallel concurrent
what to use?
lots of resistance
> That said, my overall critique of > [Finagle] is that I feel a lot of > their published implementations are > exceptionally complicated for the > problem domain or entail a bunch of > onerous transitive dependencies.
ultimately…
…the only three things you need to understand
basically
ActualFeature
Request Validator
User
Authentication Geo IP Rate Limiting AvailableFeatures
ActualFeature
User
ActualFeature
User
Filters
Service
}
we use filters a lot
10 services + 42 spans + 3 levels deep: 92ms
REST is always good enough.
or is it?
migrating to thriftmux
where we don’t follow the "defaults"
DNS for service discovery with SRV records
Prometheus.io for metrics
twitter-server jvmkit
verify new versions of Finagle
Q&A