Real time recommendations using Web Sockets and Redis

Real time recommendations using Web Sockets and RedisNinad Divadkar

Hello everybody, thank you for coming and welcome to my presentation. My name is Ninad Divadkar, and Im going to talk about Real time recommendations using Web Sockets and Redis.1

Ninad DivadkarQuickBooks, Intuit Inc.

Full stack developer, Spring contributor

A little bit about my self. I work for Intuit Inc in the QuickBooks Self Employed division. Im a full stack developer, and although I have used quite a few different programming languages over the years, Id say my backend expertise is in JAVA and .NET, and front end in AngularJS.

Im also a Spring contributor. My latest contribution is to the spring-data-redis project. Redis is introduced Geo commands in the latest version, and I have added support for those in spring-data-redis. In addition, I am active on stackoverflow, mainly in JavaScript, Java and Spring topics.2

QuickBooks Self Employed

Before we dive into the presentation, let me tell you about QuickBooks Self Employed, because it will help us understand our use case better. QuickBooks Self Employed helps self employed people like artists, dancers, uber drivers, etc find tax deductions and be complaint with IRS rules and regulations.We do that by connecting to their bank and importing the bank transactions. Users then categorize these transactions as business or personal. Anything transaction categorized as a business expense can be a tax deduction.We have an interface so that users can pay the IRS their estimated taxes directly through our software. And we integrate with TurboTax for hassle free filing.

In addition to our web presence, our iOS and Android apps help self employed people do mileage tracking.

We launched a year and a half ago in the US, and have also launched the product in the UK, with more countries to follow shortly.3

QuickBooks Self Employed

Let me demo you our iOS app to show what the product does.

Users categorize these transactions as business or personal.

Users track their mileage, so, for example if they are working for Uber they can deduct the miles they drove their car for work.

We give the users an overview of their finances, because many users do not have a great way of know whether they have a profit or not.

And we integrate with TurboTax (which is also an Intuit product), so that users can easily file their taxes.4

Bank Connection Flow

Now, Im going to demo two workflows in QuickBooks Self Employed that demonstrate how we use web sockets and Redis Pub/Sub in tandem.

Im just going to demo the UI, and as go further into the presentation, Ill show you the architecture and deeper layers of our product. The first workflow is the Bank Connection Flow.

After launching the app, the user is led to the connect a bank account screen. The user selects a bank and enters his credentials. We support 10,000 plus banks and credit unions in the US.

Connecting to a bank takes anywhere from 30 seconds to 5 minutes. During this time were communicating with a backend service that actually logs in to the bank and downloads transactions.

This is a critical process in the user flow, almost a make or break point. We need to WOW the user, and make sure the user understands the value of our product. At this moment, we want the UI to be updated with what stage the back end is at in the process of connecting a bank.

5

Recommend rules based on user behavior

Backend analyzes your behavior, and recommends rules.

After the user has started categorizing transactions, the product learns from the categorization, and recommends rules to help the user save time. So if, for example, the user categorizes 3 Wal-mart transactions as business, we understand that he or she is going to categorize the rest as business as well. So the product pops up a recommendation to the user.

6

Increased the wow factor of our product.People who accept the recommended rules are more likely to subscribe.> 60% acceptance rateRecommending rules

Recommending rules to user has made our users very happy. When they start using the product, they learn the value right away. We have an over 60% rate of acceptance with our rule recommendation engine, and it is going up all the time.

Best of all, it is a unified experience across web and mobile. The web as well as mobile apps use web-sockets to get the recommendations.7

This is enabled byCommunication between:Browser and app servers using Web socketsBack end processes using Redis Pub/Sub

These two workflows are enabled by..Web Sockets for communicating between the front end (browser) and the back end.Redis Pub/Sub for communicating between back end processes.

8

WebSocketsWebSockets is an application protocol.Enables two way communication between Server and Client.Supported by the latest browsers and web servers.Spring supports websockets.

Some of you may be familiar with what websockets are, but let me spend a couple of minutes explaining:

WebSockets is a application protocol, that works over TCP. The idea behind the WebSocket protocol consists of reusing the established TCP connection between a Client and Server. After the HTTP handshake, the Client and Server start speaking WebSocket protocol by exchanging WebSocket envelopes.

What this means for us is that it allows two way communication in the browser. This allows us to communicate with the browser in message bus type fashion.Before web socket support, we used to have long polling and other stop gap measures, but they are not required anymore.9

Redis Pub/SubPublish to a channelSubscribe to a pattern or a channelLinear complexity for most operations O(N+M)

Now, lets move to Redis Pub/Sub. Most of us use Redis for caching, but Redis also has a very good pub/sub messaging framework built in. The terminology that Redis uses is a channel instead of a queue or a topic.

A publisher publishes to a channel. A Subscriber subscribes to a channel or a pattern.These commands are super fast, and all of the pub/sub commands take linear time, O(n+m), where N is the number of clients subscribed directly to the channel and M is the number subscribed patterns.10

Architecture

The architecture of our site.We have web app processes that serve requests coming in through our web app. We have API processes that serve API requests coming in through our mobile apps.

We run embedded Tomcat 8 in both servers.

We have worker processes who do the heavy lifting, allowing us to scale without having to increase our app and api processes.

Intra process communication is done using Redis pub/sub, i.e. Web app and API processes communicate with the worker process using Redis Pub/Sub11

Spring on the backend..Spring-sessionSpring-mvcSpring-websocketsSpring-data-redisSpring-*

Spring is used extensively on the backend. We use a lot more Spring projects than shown above.12

And AngularJS on the frontAngularJSSockJS JS web socket clientStompJS Text protocol over web socket

SockJS is a browser JavaScript library that provides a WebSocket like object. Under the hood SockJS tries to use native WebSockets first. If that fails it can use a variety of browser-specific transport protocols and presents them through WebSocket-like abstractions.

STOMP is a simple text messaging protocol, and stompJS implements that protocol. Spring supports it, and so we use it. 13

The sequenceBrowserWebApp serversWorker processes

Create web socket connectionSubscribe to queue

Create Redis Pub Sub channel for user

Connect a Bank

Relay bank connection request to workerListen for bank connection publish event

Connect to bank, Import transactions, etc.

Publish to web socket

Inform user of successful connection

Publish connection successful event

14

Why Redis for Pub/Sub?Our use case Create and close channels dynamically, 100s of times a minute.1000s of channels (users) live at any given time.For our use case, RabbitMQ did not perform.

The big question, why redis for Pub Sub.

Our use case is a little different from regular use cases for RabbitMQ. RabbitMQ can handle thousands of messages a second, but it is not built to handle hundreds of queues being created dynamically in a minute.

We went to production with RabbitMQ. We have multiple app servers, but a web socket is created to only one app server. That server needs to listen to a message sent by the worker process. So we need a unique queue per user.RabbitMQ performance dropped drastically after around 200 queues. CPU spiked and we had to drop the idea of using RabbitMQ. Instead of being able to create ~50 queues / sec, throughput dropped to ~2 queues / sec.

15

Now, the workflow

Now, well go through the entire workflow, of a user launching the app and seeing a recommendation.16

When a user launches the app

Web socket connection created.Subscribe to the users topic.

Creating a web socket connection is not enough. Web-sockets connection opens a pipe to the server. However you want to publish to a topic. That way different parts of the code can use different topics, just like on the back end.17

The front end_connect() Connects the web socket, subscribes to a queue called/user/{username}/queue/user_recommendation.

fireCallbacks() fire callbacks to UI subscribers, informing them about events.

18

Some spring web socket configAssign /queue endpoint to web sockets.Assign /ws to annotated methods.Stomp endpoint is /ws/ws-init

EnableWebSocketMessageBroker is the Spring annotation to enable broker backed mesasaging over webSocket.

The MessageBrokerRegistry configuration reserves certain endpoints for websocket brokerStomp endpoint is ws-init, for regular cases19

When a web socket subscription occurs we create a Redis PUBSUB channel.Every user has a unique PUBSUB channel.Only the App process that listens to the websocket subscribes to the Redis channel.

Were implementing an ApplicationListener where we are listening to the SessionSubscribeEvent. When a SessionSubscribeEvent occurs, we create a Redis Pub Sub channel for the user.

Every user has a unique PUBSUB channel.And only the App process that listens to the websocket subscribes to the Redis channel.For example, if there are 5 app server taking requests, the web socket is connected to only 1 app server, and only that app server will subscribe to the users PUBSUB channel.

20

And this is what you see in Redis

We do end up 1000s of channels, but Redis doesnt blink! RabbitMQ did not scale above 200 queues

During tax peak we had over 4k simultaneous users on our site.21

User starts categorizing transactions

22

When it detects a possible rule, it publishes a message to the users topic.In the worker process

When the worker process detects a possible rule, it publishes a message to the users topic.23

and publishes the same message on the Web SocketApp Server receives Message

24

A rule recommendation.And the user sees

25

PUBSUB CHANNELS gives an exact count of how many users are using the system at a given time.Why DevOps loves this

An unexpected benefit of creating Redis channels whenever a user launches our app is that we can count the exact number of users who are on our site.

Our DevOps has written a script that runs every minute and captures the number of live channels in Redis. The picture above is of a Splunk search where we have charted the number of users using our app over a 7 day period.

Mind you this is after tax season is over. During tax season we had record traffic.

Things like these help us understand how many users are using our web app or mobile apps at any given time, and how long they are staying on the site as well.

26

What about mobile?Web socket supported in iOS and Android.iOS StarScreamAndroid - android-websocketsStomp support not so great but its there:iOS WebSocketStompKitAndroid GozirraYou can use Web Sockets without stomp.

Websockets are supported in both iOS and Android. There are quite a few open source libraries that work great.

We are using https://github.com/daltoniam/Starscream in iOS and https://github.com/koush/android-websockets for android.

There isnt good support for Stomp.

But that is not an issue. Spring has support for just Websockets without stomp. Youll just have to make your own protocol, and for our use case it is not a problem.http://stackoverflow.com/questions/27158106/websocket-with-sockjs-spring-4-but-without-stomp

27

iOS Demo (Bank)

Heavy lifting done by the back end.Same User Experience across all devices.

28

iOS Demo (Rules)Heavy lifting done by the back end.Same User Experience across all devices.

29

Q and A

Redis channel limits - https://groups.google.com/forum/#!topic/redis-db/G-rFG0k7NiY. At least up to 10k. We have tested until 5k simultaneous channels, our peak traffic has reached 4k simultaneous connections.

Websockets for iOS and android WebsocketStompKit (iOS), gozirra (android) while there is support, it is not well maintained.

XMPP vs websocket - http://stackoverflow.com/questions/31108780/stomp-or-xmpp-over-websocket . Stomp is easier to implement in Spring

30

ContactGithub - http://github.com/nsdivEmail ninad.divadkar @ gmail.com

31