Scala at Netflix

Scala at NetflixScala Bay Meetup

Netflix, Sept 9th 2013

Manish [email protected]

@lobster1234

Agenda

Background

Netflix OSS Components

Development

Deployment/Delivery

Open Floor

Partner Product Engineering

Smart devices

+

Certification

=

Lots of Device Metadata!

Model

Firmware

Screen Resolution

Subtitle Support

3D

DRM

Remote Control

Netflix SDK

…

Architecture

Cassandra

HTTP Layer, and Manager Layer EVCache

Crowd/SSO

RDS

Astyanax

Netflix OSS Cloud Components

Netflix OSS Components

https://github.com/netflix

http://techblog.netflix.com

https://github.com/netflix

http://techblog.netflix.com/

Simian army

Asgard

Eureka

Karyon

Astyanax

https://github.com/Netflix/astyanax

Astyanax is a Java Client for Cassandra



EVCache

Development

We to code.

Test first

We write a ton of tests.

Test coverage is a measure of confidence.

ScalaTest

simple

intuitive

english-like

rich

promotes BDD

Test first

Manager Layer

No HTTP requests, simple method invocation

/**Lets try to build a login endpoint. It should support a method calledlogin(user:String,pass:String) that returns an Option[String].

*/

class LoginManagerSpec extends FlatSpec with ShouldMatchers {

}


*/


it should " Be able to login a valid user and get a token " in {fail()

}

}


*/


it should " Be able to login a valid user and get a token " in { val token = LoginManager.login("someuser", "somepassword") token should not be None }}


*/


it should " Be able to login a valid user and get a token " in { val token = LoginManager.login("someuser", "somepassword") token should not be None }

it should " Fail to login an invalid user " in {fail

}

}


*/


it should " Be able to login a valid user and get a token " in { val token = LoginManager.login("someuser", "somepassword") token should not be None }

it should " Fail to login an invalid user " in { val token = LoginManager.login("fail", "fail") token should be (None) }

}

HTTP, or the Scalatra Layer

Scalatra

very simple, Sinatra-inspired framework

routes defined with code

json4s

Swagger

ScalatraSpec

A Simple API in Scalatra

class LoginService extends ScalatraServlet


class LoginService extends ScalatraServlet {

post("/") { }}


class LoginService extends ScalatraServlet {

before() { contentType = formats("json") }

post("/") { val token = LoginManager.login(params("user"), params("password"))

token match{case None => Unauthorized(Map("message"->"Login failed"))case Some(x) => Ok(Map("token"->x))

} }}

Putting it all together..

class ScalatraBootstrap extends LifeCycle {

override def init(context: ServletContext) {context.mount(new LoginService, "/login/*")

}}

<listener> <listener-class>org.scalatra.servlet.ScalatraListener</listener-class> </listener>

$ curl http://localhost:8080/login --form "user=foo&password=bar"

{"token":"02055794-1928-11e3-9a3b-f23c91aec05e"}

ScalatraSpec

traits to take ScalaTest to the next level

support for all HTTP methods

helpers for JSON parsing

plenty of wrappers (body, headers..)

class LoginServiceSpec extends ScalatraFlatSpec {

}


addServlet(classOf[LoginService], "/*")

}



it should "log in valid users" in { post("/", body = """user=gooduser&password=goodpassword""") { status should equal(200) body should include "token" } }}



it should "log in valid users" in { post("/", body = """user=gooduser&password=goodpassword""") { status should equal(200) body should include "token" } }

it should "not allow invalid users to log in" in { post("/", body = """user=baduser&password=badpassword""") { status should equal(401) body should include "message" } }}

APIs Best Practices

Use Proper HTTP Response Codes

Set Proper HTTP Headers

Break up your data into groups

Pop Quiz!

Lets do some HTTP response codes..

Pop Quiz!

What is the response code for an async operation?

Pop Quiz!

…forbidden?

Pop Quiz!

…a delete?

Git Workflow

work on the dev branch

write tests

leave the rest to Jenkins

Git Workflow

$ git status

# On branch dev# Changes not staged for commit:# (use "git add <file>..." to update what will be committed)# (use "git checkout -- <file>..." to discard changes in working directory)## modified: src/main/scala/com/netflix/nrdportal/http/DpiService.scala# modified: src/test/scala/com/netflix/nrdportal/http/DpiServiceSpec.scala

Automated Code Pushes

Push to dev

Jenkins runs dev build,

tests, merges to

master

Jenkins runs master build,

makes an RPM

Aminator bakes an

AMI from the RPM

asgard deploys the

AMI in staging cloud

Scala Best Practices

Using Options

Using Try[A] vs. Exceptions

Wrappers

Control Abstractions

The dreaded null

public String willReturnNullForOdds(int x){ if(x%2==0) return "Even"; else return null;}

Using Options

def willReturnNullForOdds(x: Int): Option[String] = { if (x % 2 == 0) Some("Even") else None }

Using Options as Wrappers

…where you have to call code that can return null

def returnNull(x:Int) = if(x%2 == 0) "Even" else null

scala> Option(returnNull(3)) res01: Option[String] = None

scala> Option(returnNull(2)) res02: Option[String] = Some(Even)

Exceptions?

public String willThrowExceptionForOdds(int x){if(x%2==0) return "Even";else throw new IllegalArgumentException("Odd Number!");

}

Using Try[A]

def someFunction(x: Int): Try[String] = { if (x % 2 == 0) Success("Even") else Failure(new IllegalArgumentException("Odd number!")) }

Control Abstractions

def withAuthenticatedUser(f: (String) => ActionResult) = { getUserNameFromCookie match { case Some(userName) => f(userName) case None => Unauthorized("You are not logged in!") } }

def printCurrentUserName = { withAuthenticatedUser { userName => Ok(s"Your username is ${userName}") } }

Finally…

Avoid writing complex code at all costs – there are better ways to prove your awesomeness!

Manish Pandit

@lobster1234

[email protected]

linkedin.com/in/mpandit

slideshare.net/lobster1234

Scala at Netflix

Technology

Transcript of Scala at Netflix