Scaling the real-time web with ASP.NET SignalRDamian Edwards @damianedwardsSenior Program ManagerScaling the real-time web with ASP.NET SignalR3-502

This is not an introduction to SignalR talk…

See www.asp.net/signalr for getting started information

Scaling traditional vs. real-time web

The same… but differentScaling real-time web traffic shares many considerations with traditional web traffic, e.g. CPU, bandwidth, memoryApplication scenarios have huge impact on scaling patternsBig difference is in concurrency, supporting many long running idle and active connections vs. short requestsDifferent SignalR transports have different overheads

General things to watch out forBlocking calls, e.g. block I/ONever ever block a Hub method, it jams up the pipesUse 4.5 and async where availableSending large messagesMemory leaks caused by misunderstanding SignalR object lifetime, e.g. Hub instancesSession – don’t use it from SignalR, at all, ever (no support)Use Hub state, cookies, browser storage, database, etc. instead

Remember the secret of scale:

“Have your app do as little as possible. If you do nothing, you can scale infinitely.” – Scott Hanselman

Understanding SignalR load patterns

message bus

SignalR core architecture: pub/sub

message cache

publisher

client client client client

1. Message serialized, saved to cache

associated with signal, topic is marked for

delivery, publish call returns

client

worker worker worker

2. Worker is scheduled for a signal, selects a waiting subscriber, retrieves message

from cache

3. Worker sends message to client as bytes over a

transport

Pattern 1. Server broadcastLow rate broadcast of the same payload to all clientsOne message bus send maps to many users (fan out)More clients don’t increase message bus traffic

e.g. application wide alerts, simple stock ticker

Pattern 2. Server pushLow rate broadcast of the unique payload to each clientOne message bus send maps to one user (no fan out)More clients means more message bus traffic

e.g. unique stock watch list, job monitoring

Pattern 3. User event drivenBroadcast on client actionsOne message bus send maps to many users (fan out)More clients means more message bus traffic

e.g. chat application, document collaboration

Pattern 4. High frequency real-timeFixed high rate broadcast from servers and clientsWe don’t recommend >25HzOne message bus send maps to one user (no fan out)More clients means more message bus traffic

e.g. real-time gaming, shootr.signalr.net

Load patterns summary1. Server broadcast – low rate, message to all

clients2. Server push – low rate, message to unique

clients3. User event driven – broadcast on client action4. High frequency – fixed high rate, unique

messages

Demo: Tools for scale testing

Tools for scale testing summaryPerformance countersMicrosoft.AspNet.SignalR.Utils, signalr.exe -ipccrank.exe for generating SignalR loadLoad Test Harness for basic endpoint hostingstress.exe for running isolated scenarios in-procIIS settings for adjusting concurrency limits

Scaling to multiple servers

Scale-out issues: message delivery

?How do messages from one server get to the other

servers in my web farm?

Scale-out issues: client transience

server 1

client

server 2

When is a client disconnected from my app?

How do I avoid duplicate &

missed messages as I move from

server to server?

Scale-out issues: client distribution

server 1

server 2

client “foo”

client “foo”

client “foo,ba

r”client “foo”

client “bar”

client “bar”

client “foo”

client “foo”

What happens if “foo” clients get

many more messages than

others?

Plug-in scale-out providersSQL Server, Redis & Windows Azure Service BusIncredibly simple setup (NuGet package + 1 line of code)Great for the server broadcast load patternLimited for other scenariosEvery message goes to every server, so as traffic increases you’re limited by how fast any one web server can pull messages off of the backplaneBackplanes are *much* slower than single-server performance; throughput != scale

Scale-out provider architecture

web nodes

clients

backplaneMessages sharded

over multiple backplane streams

Custom scale-out: Common serverGood for document collaboration, white-boardingClients working on same subject go to the same serverApplication knows which subject is on each serverSimilar to the idea of a real-time gaming lobby and then connecting to a game serverUse groups to segregate traffic, e.g. a group per documentPersist all state changes to your app stateScale up to a dedicated server per collaboration subject

I want to work on document “foo”Sure, connect to http://server2

Custom scale-out: Server affinity

server 1

server 2

client

Connect for “foo”!

client

foo

Custom scale-out: Specific serverGood for the server push scenario; unique payloadsConnect client to a specific server & save that mappingPublish messages for client to mapped server via endpoint

Sure, connect to http://server2

I want to connect

Custom scale-out: Specific server

server 1

server 2

client #1

Connect!

Client #1 on server 2

Message for client #1

Which server is client #1 on?Client #1 is on

server 2

Custom scale-out: Filtered message busGood for the server push scenario; unique payloadsWeb servers connected to a bus with a filtered subscriptionUpdate the filter on each connection connectWhen message arrives do a local broadcast

Custom scale-out: Filtered message bus

“foo”

“foo”

web nodes

clients

backplane

Custom scale-out: Server transitionGood for real-time gaming when coupled with an in-game experience, e.g. hyperspace to another quadrant (move to another server)Single-box equiv. performance (very high)Can’t send messages between clients on different servers without more custom work

Custom scale-out: HybridMix and match the various patternse.g. Combine the plug-in scale-out providers with server affinity approach for fault-tolerant document collaboration, where user first arrives at lobby, chooses document/room, then is connected to the cluster currently serving that document/room, which itself is configured with one of the plug-in scale-our providerse.g. Use server transition for the high-frequency aspect of the app and custom server-to-server delivery for in-app messaging or alerts

We’re working on providing better guidance & samples in this space

2.0 scale & performance improvementsFirst of all, 1.1 included many performance improvements so use thatSupport for pre-serialization of messages for when you want to send data that is already JSON formatted Support for single-serialization when sending to multiple signals

Resourceswww.asp.net/signalrwww.campusmvp.net/signalr-ebookgithub.com/signalr/signalrtwitter.com/damianedwards

Evaluate this session

Scan this QR code to evaluate this session and be automatically entered in a drawing to win a prize!

© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.