Post on 05-Apr-2017
Letgo chat From polling to real timeScala, Akka, and WebSockets from scratch
@SergiGP @GVico46
@JavierCane#scbcn16 - Software Craftsmanship Barcelona 2016
@JavierCane@SergiGP
Welcome!
@GVico46
Contents
Context(not Bounded)
Legacy
Getting started
Pain Points
From PHP to Scala
1. Context (not Bounded)
App downloads
Messages sent monthly growth
Messages sent every day
30M
20 - 40%
~4M
Context (not Bounded) Where do we come
● Mobile first
◕ Internal REST API ● Startup with less than 2 years
◕ Externalize services (Parse, Kahuna…) ● Funding: $200M
◕ Ads in TV in USA and Turkey
2. Legacy
Legacy
● PHP ● No test ● New Features
Legacy REST API in PHP
Do I have new messages? No
And now?
And now?
And now?
And now?
No
No!!
NO!!
😑 🔫 💣
Legacy No test
● Rebuild a system without tests => 🦄💩💣💀
● Coupled system => Acceptance tests
◕ Learning what the system does
◕ Find existing weird behaviors
Background: Given there are test users: | user_object_id | user_name | user_token | | 19fd3160-8643-11e6-ae22-56b6b6499611 | seller | sellerToken | | 120291b2-8643-11e6-ae22-56b6b6499611 | buyer | buyerToken | And user "seller" has a product with: | id | objectId | | 120291b2-8643-11e6-ae22-56b6b6499611 | SuperProductId | Scenario: A user can get messages from another user associated to product Given user "seller" has a conversation related to product "SuperProductId" with user "buyer" When user "seller" asks for messages related to product "SuperProductId" from user "buyer" Then the response status code should be 200 And the response should be in JSON And the JSON should be valid according to the schema "messages.schema"
Acceptance test with Behat
Legacy Taking advantage of backwards compatibility
Leaving The Monolith thanks to #EventSourcing @ #scpna
Legacy New Features
● Product always want more features ● Negotiation:
◕ Archive conversations
◕ Mute interlocutor
◕ Stickers
3. Getting started
Getting started
● Why and how to switch to Scala ● Scala and Akka crash course ● Takeaways
Why and how to switch to Scala
We want a WhatsApp inside
Letgo
LOL
I’ve payed $22 Billion for WhatsApp
Getting started Why and how to switch to Scala
● Realtime (WebSockets) ● Akka ● Scale!
Why How
● Learning a lot ● External consultancy ● Akka :) ● Backwards Compatible
Scala quick start
Getting started Scala quick start
● Case classes ● Functional ● Optionals ● Futures ● OOP
class User { private $id; private $name; public function __construct(Uuid $id, string $name) { $this!→id = $id; $this!→name = $name; } public function id() : Uuid { return $this!→id; } public function name() : string
Case Class
{ return $this!→id; } public function name() : string { return $this!→name; } public function setId(Uuid $id) : self { return new static($id, $this!→name); } public function setName(string $name) : self { return new static($this!→id, $name); }}
Case Class
rafa.name = "Santi"val santi = rafa.copy(name = "Santi")println(santi.name) #$ Santi
val rafa = User(UUID.randomUUID(), "Rafa")println(rafa.name) #$ Rafa
case class User(id: UUID, name: String)
Case classes
Does not compile
Usage
Immutability
val users = List( User(UUID.randomUUID(), "Rafa"), User(UUID.randomUUID(), "Santi"), User(UUID.randomUUID(), "Jaime"), User(UUID.randomUUID(), "Diyan"))
Functional
Mutable state
val names = users.map(user %& user.name)val names = users.map(_.name)
List[String] names = new ArrayList();for (User user: users) { names.add(user.name)}
Procedural
Option
Option[A]
Some(x: A) None
def searchUser(id: UUID): Option[User] = { #$ …search user in database (blocking) Some(rafa)}
Option
searchUser(userId) match { case Some(user) %& #$ do stuff case None %& #$ user not found}
Usage (pattern matching)
Option usage (functional)
searchUser(userId) match { case Some(user) %& #$ do stuff case None %& #$ user not found}
searchUser(userId).map { userFound %& #$ do stuff}
Futures
def searchUser(id: UUID): Future[Option[User]] = { Future { Thread.sleep(1000) Some(rafa) } }
Futures usage
searchUser(userId).onComplete { case Success(Some(user)) %& #$ do stuff case Success(None) %& #$ user not found case Failure(exception) %& #$ future has crashed}searchUser(userId).map { case Some(user) %& #$ do stuff case None %& #$ user not found}
searchUser(userId).map(_.map(_.name))
trait UserRepository { def search(id: UUID): Future[Option[User]]}trait ConsoleLogger { def warning(message: String) = { println(message) }}
OOP - DIP
OOP - DIP
class MysqlUserRepository extends UserRepository with ConsoleLogger { def search(id: UUID): Future[Option[User]] = { #$ implementation warning("user not found") Future(Some(rafa)) }}
OOP - Companion object
object UserId { def random: UserId = { UserId(UUID.randomUUID()) }}case class UserId(id: UUID)
val userId = UserId.randomprintln(userId.id) case class User(id: UserId, name: String)
Usage
Akka (actor model)
Scala quick start Akka (actor model)
● Concept ● Introductory examples ● Chat actors architecture
Scala quick start Akka (actor model) - Concept
● Mailbox (1 each time) ● receive to handle incoming messages ● ActorRef ● Tell or ask methods to interact with the ActorRef ● Location transparency
final class ConnectionActor extends Actor { }
object ConnectionActor { def props: Props = Props(new ConnectionActor)}
Building our first actor
Instantiationval connection: ActorRef = context.actorOf(ConnectionActor.props)
object ConnectionActor { def props: Props = Props(new ConnectionActor)}
final class ConnectionActor extends Actor {
override def receive: Receive = { case PingQuery %& } }
Building our first actor
Instantiationval connection: ActorRef = context.actorOf(ConnectionActor.props)
final class ConnectionActor(webSocket: ActorRef) extends Actor { override def receive: Receive = { case PingQuery %& webSocket ! PongResponse } }
Tell (Fire & forget)
object ConnectionActor { def props(webSocket: ActorRef): Props = Props(new ConnectionActor(webSocket)) }
Building our first actor
Instantiationval connection: ActorRef = context.actorOf(ConnectionActor.props(webSocket))
case class ConnectionActorState( lastRequestSentAt: Option[DateTime]) { def requestSent: ConnectionActorState = copy(lastRequestSentAt = Some(DateTime.now))}
Dealing with state
case class ConnectionActorState( lastRequestSentAt: Option[DateTime]) { def requestSent: ConnectionActorState = copy(lastRequestSentAt = Some(DateTime.now))}
final class ConnectionActor(webSocket: ActorRef) extends Actor { var state = ConnectionActorState(lastRequestSentAt = None) override def receive: Receive = { case PingQuery(requestId) %& state = state.requestSent webSocket.actorRef ! PongResponse }
Dealing with state
State model
Akka: 1 message at a time (no race conditions)
final class ConnectionActor(webSocket: ActorRef) extends Actor { var state = ConnectionActorState(lastRequestSentAt = None) override def preStart(): Unit = { context.system.scheduler.schedule( initialDelay = 1.minute, interval = 1.minute, receiver = self, message = CheckWebSocketTimeout ) } override def receive: Receive = { case PingQuery(requestId) %& state = state.requestSent
Lifecycle
override def preStart(): Unit = { context.system.scheduler.schedule( initialDelay = 1.minute, interval = 1.minute, receiver = self, message = CheckWebSocketTimeout ) } override def receive: Receive = { case PingQuery(requestId) %& state = state.requestSent webSocket ! PongResponse() case CheckWebSocketTimeout %& if (state.hasBeenIdleFor(5.minutes)) { self ! PoisonPill } }}
Lifecycle
override def receive: Receive = { case PingQuery %& Future { Thread.sleep(1000) sender() ! PongResponse }}
Akka and Futures - SHIT HAPPENS
sender() could have changed
Be careful dealing with futures - sender()
override def receive: Receive = { case PingQuery %& Future { Thread.sleep(1000) PongResponse }.pipeTo(sender())}
sender() outside Future
Same happens with self
Chat actors architecture
TalkerS
ConversationSJ
TalkerJ
ConnectionS1 ConnectionJ1
Maintains consistency between 2 talkers : 1 conversation
Kill connections if shit happens
Chat actors architecture
TalkerS
ConversationSJ
ConnectionS1CS2CS3
Connection Supervisor
Talker Provider
Conversation Provider
Maintains consistency between N connections : 1 talker
“Singleton” actor
Non “singleton” actor
Takeaways
Scala quick start Takeaways
● Neophyte guide for scala ● Raúl Raja ● Akka Concurrency book ● Scala for the impatient ● Lightbend webinars ● Lightbend activator
4. Pain Points
Pain Points
● MaxScale ● Slick ● Deploy ● Dependency Injection ● Sync between chats
Chat protocol
SERVER TO CLIENT
Events
Commands
Queries
ACK
RESPONSE
Events
CLIENT TO SERVER
SERVER TO CLIENTCommands
typing_started
typing_stopped
interlocutor_typing_started
interlocutor_typing_stopped interlocutor_message_sent
interlocutor_reception_confirmed
interlocutor_read_confirmed
Events
Queries
Events
fetch_conversations
fetch_conversation_details
fetch_messages fetch_messages_newer_than_id
fetch_messages_older_than_id
confirm_reception
confirm_read
archive_conversations
unarchive_conversations
CLIENT TO SERVER
Chat protocol
authenticate
create_conversation
send_message
DB initial import
DB initial import
Legacy events
workerN
Legacy events
worker…
Legacy events
worker2
Scaling domain events workers
Legacy events
worker1
Auto scaling supervisor actor
SQS
Scaling domain events workers
Legacy events
worker1
Legacy events
worker2
Legacy events
worker…
Legacy events
workerN
Auto scaling supervisor actor
SQS
5. From PHP to Scala
From PHP to Scala
● Language community ● Composer vs SBT
◕ Semantic Versioning (scalaz, play…) ● Developer eXperience
◕ Not descriptive errors
◕ Scala and IntelliJ ● Learning Curve ● Loving and hating the compiler ● Another set of problems
Questions?
Thanks!Contact
@JavierCane@SergiGP@GVico46
Credits
● Presentation base template by SlidesCarnival ● Graphics generated using draw.io