Node.js Web Apps @ ebay scale
-
Upload
dmytro-semenov -
Category
Internet
-
view
894 -
download
0
Transcript of Node.js Web Apps @ ebay scale
By Dmytro SemenovMember of Technical Staff @ NodeJS, Cloud and Platform Services, eBay Inc.
1995 2004 2013 present2006
Perl
C++
XSL/XML
Java
Node
Scala
Java UI, JSP Extended
Python Go ???
future
Marko/Lasso(Tessa)
Polyglot initiative
OSGi
Programming Language History
● Perl/Apache (no layers)
○ Scalability limit of 50K active listings
● C/C++/IIS/XML/XSL (monolithic client/server architecture)
○ Compiler limit of methods per class, very long build times (2+ hours)
● Java/J2EE (3 tier architecture, split build into domain specific)
○ XML/XSL (MSXSL) as presentation layer hit native memory limit
○ Java UI components, proprietary technology, steep learning curve
○ Startup time up to 40 minutes
● Java Stack (embrace open-source, JSP, modular approach)
○ OSGi conflicts at startup, very hard to get everything working at the start
○ Slow startup time (2+ minutes)
● Polyglot initiative to support popular languages/technologies
● Started in 2013
○ CubeJS, first attempt was not so good,
○ Unified NodeJS, eBay and PayPal platform merge
○ Big projects decided to move to NodeJS
● Embedding into application teams
○ Accumulation of issue knowledge base
○ Build confidence and form local knowledge centers
○ Fast feedback
○ Fast project startup
○ Pick new ideas from application teams
● Node 4.x
● 200 applications and growing
● 80 million requests / day now 1 billion
● Platform team of 6 developers
● Very vibrant internal community
○ Local pool of knowledge among teams
○ Self sustaining support by internal community
○ PRs are welcome
○ 45 contributors to eBay NodeJS platform
● 80 platform modules
● 330 total modules
● More agile
○ Startup time <2s
○ Test coverage close to 90%-100%
○ Faster releases (every day) vs 2 weeks cycle
○ Automatic upgrades with semver
○ Modular UI architecture based on UI components
● Better tools
○ flame graphs, waterfall charts, pre-production testing
● Learnings Applied to Java Platform
○ Incorporate best practices from NodeJS
○ Startup time < 1 minute
○ Embrace modular approach and semver
○ Lighter Java stacks
NPM: Private
KappaCouchDB
Primary
CouchDB
Backup
replicate
registry.npmjs.org
npmjs.ebay.com
fallback
LB
Kappa
Kappa
● Experience Service
○ Provides view models for UI modules
○ View model is specific to the device type (native, size, desktop, tablet)
● NodeJS Web App
○ Talk to experience service
○ Handles UI functionality
○ Renders desktop and mobile web pages
● Native Apps
○ Talk to experience service via API gateway
Module View
pm2
node.js
express (3 workers)
platform-ebay
analytics
logging
config
service client
lasso plugin
app
servicespages components
kraken-js lasso/tessamarko
src/
pages/
component
s/src/components/login-form/
index.js
style.css
template.marko
src/pages/login/
index.js
style.css
browser.json
● Async + streaming
● Custom tags
● Browser + Node.js runtime
● Lightweight
● Extremely fast
● Compiles to CommonJS
markojs.comgithub.com/marko-js/marko
github.com/marko-js/marko-widgets
Templating
UI components
● DOM diffing/patching
● Batched updates
● Stateful widgets
● Declarative event binding
● Efficient event delegation
● Lightweight runtime
UI components built using Marko Widgets
Uses Marko templates as the view. Plus:
github.com/marko-js/marko-widgets
● DOM diffing/patching
● Batched updates
● Stateful widgets
● Declarative event binding
● Efficient event delegation
● Lightweight runtime
Node is built on a streaming interface
HTTP is a streaming protocol
So why then do most templating and UI libraries render
as a single chunk?
● Faster perceived page load
● Better time, resource and traffic utilization
● Progressive rendering between browser, frontend and backend
○ Frontend<->Backend: Stream of server side events (SSE)
○ Browser<->Frontend: Module render and flush on arrival
● Marko helps:
○ Async fragments support by marko-async
○ Re-order on client side
logging
error handling
analytics
circuit breaker
retry
security/oauth
http/https/ssese
rvic
e in
vo
ca
tio
n h
an
dle
rs
service client
middlewares
service
web app
request
response
co
okie
s
body-p
ars
er
an
aly
tics
● Generic Service Client with pluggable handlers
● Handler performs a single function
● The handler pipeline is configurable per client
Configuration based Code based
"my-service-client": {
"protocol": "http:",
"hostname": "myservice.com",
“path”: “/path/to/resource”,
"port": 8080,
"socketTimeout": 1000,
"pipeline": [
"logging/handler",
"circuit-breaker/handler",
"error/handler" ]
}
var serviceClient = require(‘service-client’);
serviceClient.use(require(‘logging/handler’));
serviceClient.use(require(‘circuit-breaker/handler’));
serviceClient.use(require(‘error/handler’));
serviceClient
.get(‘http://myservice.com/path/to/resource’)
.end((err, response) => {
console.log(response.body);
})
Resource
Server
App
Server
1. GET
http://www.ebay.com
2. GET
http://rs/check/hp-24512.js
3. PUT
http://rs/upload/hp-24512.js4. <script src=”hp-
24512.js”>
5. GET http://hp-
24512.js
6. GET http://hp-
24512.js
Akamai
browser
● Supported resources
○ Defined by lasso plugins
○ CSS/JS/Images
○ Templates
○ I18n content
○ Bundle definition
● Lasso plugin
○ Adaptor between lasso and resource server
● Benefits
○ Single build and deployment
○ No synchronization problems between app and resource server deployments
○ Externalization at startup and during runtime on-the-fly
app/config.json
/node_modules
/moduleA/config.json
/moduleB/config.json
CMS
app/1.0.1/config: {}
moduleA/1.1.1/config: {}
moduleB/1.0.0/config: {}
pull every minute
Config
Deployment
App
Deployment
App
Server
App
Server
App
Server
● Module configuration
○ Local, packaged with module
○ Remote, hot deployed
● Application configuration
○ Local
○ Remote, hot deployed
● Application can override any module configuration
● Configuration can be injected via Admin Console
● Future - Code and Config “separation”, but
○ Keep app and config together in git repo and separate at deployment.
○ Easier to manage
● Use krakenjs/spud module
○ Property file format
○ Marko tags/helpers
● Externalizable as resources
● 17 main languages
● Support multiple languages per country
● Application and modules can have localizable content
app/locales/
US/
en/
ProjectName/
xxx.properties
yyy.properties
ru/
ProjectName/
xxx.properties
yyy.properties
DE/
de/
ProjectName/
xxx.properties
yyy.properties
Folder Structure Example
● Nsp tool
○ On-demand scan for every application project
○ Security badge for every platform module
● ScanJS
○ Source code scan
● CSRF tokens
● Redirect validation
● XSS
● Rate limiter
● Logging every transaction/subtransaction
○ Central Logging Repository (CAL) provides log aggregation per pool/box/datacenter
○ Use Domains to maintain context per request and avoid passing it around
● Explicit code instrumentation
○ Support DEBUG, INFO, WARN, ERROR, FATAL
○ Time span to record transaction (start and end)
○ Nested spans
● Health checks/stats monitoring and alerts
● Crash/OOM emails to the owner of the pool
● Early problem detection using traffic mirror
● Proactive testing in pre-production
○ Traffic mirroring of read-only traffic to the box with new build
○ Easy upgrades
● Handling uncaught errors
○ Domains used to capture context
○ Send email to the owner with stack trace, group and box name, request info
○ Graceful restart
● Handling memory leaks
○ Email sent to the owner with group and box name, request info
○ Graceful restart when memory threshold is reached
● Too busy load shedding
○ 503 or connection reset to trigger browser DNS fallback
○ Filter bots traffic under heavy load
● Planning for failure
● Hystrix like service calls
○ Fail fast
○ Circuit breaker
● Fast startup/re-start
○ Cold cache to avoid service invocation
○ Pre-compiling template @ deployment
○ Pre-externalizing resources @ deployment
● Progressive rendering/streaming @ browser side
● Progressive chunking/streaming @ service side
● Performance tuning
○ Flame Graphs
○ Waterfall charts @ server side
● How to
○ Use v8-profiler to generate json data file
○ Aggregate stack frames into json
○ Render using d3-flame-graph
● No special environment
● No special steps
● Single button generation
● Can be used in production/dev/qa
● Exposes only javascript side of the code
● CPU profiling
● Troubleshooting in production the problem at hand
● Memory leak investigation
● Regular sampling
● Requires code instrumentation
● Easy and quick to assess what is going on
● Easy to spot synchronous events
● Analyze for possible task parallelization
● Latency, TCP_NODELAY=true
● Handling request close, finish, error events is important
● No dns cache out of the box
○ Use OS level caching to allow restarts
● Avoid modules with a state
● Embedding within App teams to bootstrap works great
● Use cold cache to keep restarts fast
● Version control
○ npm shrinkwrap does not guarantee versions
○ switched to Uber shrinkwrap
● App and platform coupling in one build
○ It is still monolithic, platform coupled to app
● Upgrading to major versions
○ Need to keep backwards compatible
○ Teams go at their own pace
● Memory leak analysis
● Debugging
○ Not stable, gets broken frequently
● 1 billion requests / day
● Decoupling platform from application
○ Moving platform components into separate processes
○ Independent platform deployments
○ More resilient apps
○ Problem isolation (easier memory leak detection)
● Platform microservices
● Docker
● Kubernetes
● SenecaJS
● NodeJS services
References
• Progressive rendering: http://www.ebaytechblog.com/2014/12/08/async-fragments-
rediscovering-progressive-html-rendering-with-marko
• AMP: http://www.ebaytechblog.com/2016/06/30/browse-ebay-with-style-and-speed/ - sse
between frontend and backend - streaming
• http://www.ebaytechblog.com/2016/06/15/igniting-node-js-flames/
• http://www.ebaytechblog.com/2016/07/14/mastering-the-fire/
• http://www.slideshare.net/tcng3716/ebay-architecture
• Cloud http://www.computerweekly.com/news/2240222899/Case-study-How-eBay-uses-its-
own-OpenStack-private-cloud
• http://www.ebaytechblog.com/2014/10/02/dont-build-pages-build-modules/
• history: http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf
• lasso: https://www.npmjs.com/package/lasso
• marko: http://markojs.com/
• https://github.com/spiermar/d3-flame-graph