Concurrent Ruby Application Servers
-
Upload
lin-jen-shin -
Category
Technology
-
view
5.232 -
download
0
Transcript of Concurrent Ruby Application Servers
![Page 1: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/1.jpg)
Lin Jen-Shin (godfat) @ Rubyconf.tw/2012
Concurrent
RubyApplication Servers
![Page 2: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/2.jpg)
Lin Jen-Shin (godfat) @ Rubyconf.tw/2012
http://godfat.org/slide/ 2012-12-07-concurrent.pdf
![Page 3: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/3.jpg)
Concurrency?
What We Have?
App Servers?
Me?
Q?
![Page 4: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/4.jpg)
Concurrency?
What We Have?
App Servers?
Me?
Q?
![Page 5: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/5.jpg)
Lin Jen-Shin (godfat) @ Rubyconf.tw/2012
• Programmer at Cardinal Blue
![Page 6: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/6.jpg)
Lin Jen-Shin (godfat) @ Rubyconf.tw/2012
• Programmer at Cardinal Blue
• Use Ruby from 2006
![Page 7: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/7.jpg)
Lin Jen-Shin (godfat) @ Rubyconf.tw/2012
• Programmer at Cardinal Blue
• Use Ruby from 2006
• Interested in PL and FP (e.g. Haskell)
![Page 8: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/8.jpg)
Lin Jen-Shin (godfat) @ Rubyconf.tw/2012
• Programmer at Cardinal Blue
• Use Ruby from 2006
• Interested in PL and FP (e.g. Haskell)Also concurrency recently
![Page 9: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/9.jpg)
Lin Jen-Shin (godfat) @ Rubyconf.tw/2012
• Programmer at Cardinal Blue
• Use Ruby from 2006
• Interested in PL and FP (e.g. Haskell)
• https://github.com/godfat
Also concurrency recently
![Page 10: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/10.jpg)
Lin Jen-Shin (godfat) @ Rubyconf.tw/2012
• Programmer at Cardinal Blue
• Use Ruby from 2006
• Interested in PL and FP (e.g. Haskell)
• https://github.com/godfat
• https://twitter.com/godfat
Also concurrency recently
![Page 11: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/11.jpg)
Lin Jen-Shin (godfat) @ Rubyconf.tw/2012
PicCollage
![Page 12: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/12.jpg)
Lin Jen-Shin (godfat) @ Rubyconf.tw/2012
in 7 days
• ~3.1k requests per minute
• Average response time: ~135 ms
• Average concurrent requests per process: ~7
• Total processes: 18
• Above data observed from NewRelic
• App server: Zbatery with EventMachine and thread pool on Heroku Cedar stack
PicCollage
![Page 13: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/13.jpg)
Lin Jen-Shin (godfat) @ Rubyconf.tw/2012
Recent Gems
• jellyfish - Pico web framework for building API-centric web applications
• rib - Ruby-Interactive-ruBy -- Yet another interactive Ruby shell
• rest-core - Modular Ruby clients interface for REST APIs
• rest-more - Various REST clients such as Facebook and Twitter built with rest-core
![Page 14: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/14.jpg)
Lin Jen-Shin (godfat) @ Rubyconf.tw/2012
Special Thanksihower, ET Blue and Cardinal Blue
![Page 15: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/15.jpg)
Concurrency?
What We Have?
App Servers?
Me?
Q?
![Page 16: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/16.jpg)
Caveats No disk I/O
No pipelined requests
No chunked encoding
No web sockets
No …
To make things simpler for now.
![Page 17: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/17.jpg)
It's not faster for a userCaution
![Page 18: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/18.jpg)
10 moms can't produce
1 child in 1 month
![Page 19: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/19.jpg)
10 moms can produce
10 children in 10 months
![Page 20: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/20.jpg)
Resource matters
![Page 21: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/21.jpg)
We might not always have enough processors
Resource matters
![Page 22: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/22.jpg)
We might not always have enough processors
Resource matters
We need multitasking
![Page 23: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/23.jpg)
Imagine 15 children have to be context switched amongst 10 moms
![Page 24: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/24.jpg)
Multitasking is not free
![Page 25: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/25.jpg)
If your program context switches, then actually it runs slower
Multitasking is not free
![Page 26: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/26.jpg)
If your program context switches, then actually it runs slower
Multitasking is not free
But it's more fair for users
![Page 28: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/28.jpg)
It's more fair if they are all served equally in terms of waiting time
![Page 30: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/30.jpg)
I just want a drink!
![Page 31: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/31.jpg)
#04
#03
#02
#01
#00
#05
WaitingProcessingContext Switching
Sequential
Time
User ID
To illustrate the overall running time...
![Page 32: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/32.jpg)
#04
#03
#02
#01
#00
#05
WaitingProcessingContext Switching
Concurrent
Time
User ID
#04
#03
#02
#01
#00
#05
WaitingProcessingContext Switching
Sequential
Time
User ID
![Page 33: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/33.jpg)
#04
#03
#02
#01
#00
#05
WaitingProcessingContext Switching
ConcurrentTotal Waiting and ProcessingTime
Time
User ID
#04
#03
#02
#01
#00
#05
WaitingProcessingContext Switching
SequentialTotal Waiting and ProcessingTime
Time
User ID
![Page 34: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/34.jpg)
Scalable != FastScalable == Less complaining
![Page 35: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/35.jpg)
Rails is not fastRails might be scalable
Scalable != FastScalable == Less complaining
![Page 36: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/36.jpg)
So, When do we want concurrency?
![Page 37: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/37.jpg)
So, When do we want concurrency?
When context switching cost is much cheaper than a single task
![Page 38: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/38.jpg)
So, When do we want concurrency?
When context switching cost is much cheaper than a single taskOr if you have much more cores than your clients (= no need for context switching)
![Page 39: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/39.jpg)
If context switching cost is at about 1 second
![Page 40: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/40.jpg)
If context switching cost is at about 1 second
It's much cheaper than 10 months, so it might be worth it
![Page 41: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/41.jpg)
#04
#03
#02
#01
#00
#05
#04
#03
#02
#01
#00
#05
Time
User ID
Time
User ID
But if context switching cost is at about 5 months...
![Page 42: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/42.jpg)
Do kernel threads context switch fast?
![Page 43: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/43.jpg)
Do kernel threads context switch fast?
Do user threads context switch fast?
![Page 44: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/44.jpg)
Do kernel threads context switch fast?
Do user threads context switch fast?
Do fibers context switch fast?
![Page 45: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/45.jpg)
A ton of different concurrency models then invented
![Page 46: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/46.jpg)
A ton of different concurrency models then invented
Each of them has different strengths to deal with different tasks
![Page 47: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/47.jpg)
A ton of different concurrency models then invented
Each of them has different strengths to deal with different tasks
Also trade off, trading with the ease of programming and efficiency
![Page 48: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/48.jpg)
So, How many different types of tasks are there?
![Page 49: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/49.jpg)
Two patternsLinear data dependency
of composite tasks
go to resturant
order food order drink
eat drink
order_food -> eat order_tea -> drink
![Page 50: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/50.jpg)
go to resturant
order food order drink
eat drink
go to resturant
order food order spices
mix
eat
Two patternsLinear data dependency Mixed data dependency
of composite tasks
order_food -> eat order_tea -> drink
[order_food, order_spices] -> add_spices -> eat
![Page 51: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/51.jpg)
prepare to share a photo
to facebook to flickr
save save
prepare to merge photos
from facebook from flickr
merge
save
prepare to share a photo
to facebook to flickr
save save
prepare to merge photos
from facebook from flickr
merge
save
Linear data dependency Mixed data dependency
![Page 52: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/52.jpg)
prepare to share a photo
to facebook to flickr
save save
prepare to share a photo
to facebook to flickr
save save
CPU
CPU CPU
I/O I/O
CPU IOOne single task could be or bound
Linear data dependency Mixed data dependency
![Page 53: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/53.jpg)
prepare to share a photo
to facebook to flickr
save save
prepare to merge photos
from facebook from flickr
merge
save
prepare to share a photo
to facebook to flickr
save save
prepare to merge photos
from facebook from flickr
merge
save
CPU
CPU CPU
I/O I/O
CPU
CPU
CPU
I/O I/O
CPU IOOne single task could be or bound
Linear data dependency Mixed data dependency
![Page 54: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/54.jpg)
Concurrency?
What We Have?
App Servers?
Me?
Q?
![Page 55: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/55.jpg)
• Performance
Separation of Concerns
![Page 56: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/56.jpg)
• Performance
• Ease of programming
Separation of Concerns
![Page 57: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/57.jpg)
• Performance (implementation)
• Ease of programming (interface)
Separation of Concerns
![Page 58: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/58.jpg)
Advantages if interface is orthogonal to its implementation
• Change implementation with ease
![Page 59: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/59.jpg)
• Change implementation with ease
• Don't need to know impl detail in order to use this interface
Advantages if interface is orthogonal to its implementation
![Page 60: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/60.jpg)
interface implementation
![Page 61: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/61.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
interface implementation
![Page 62: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/62.jpg)
Interface forLinear data dependency
![Page 63: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/63.jpg)
blocking threads
callback reactor fibers
CPU
I/O
order_food{ |food| eat(food) } order_tea{ |tea| drink(tea) }
Callback
callback reactor
blocking threads
fibers
CPU
I/O
![Page 64: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/64.jpg)
blocking threads
callback reactor fibers
CPU
I/O
eat(order_food) drink(order_tea)
If we could control side-effect and do some static analysis...
callback reactor
blocking threads
fibers
CPU
I/O
![Page 65: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/65.jpg)
blocking threads
callback reactor fibers
CPU
I/O
Not going to happen on Ruby though
callback reactor
blocking threads
fibers
CPU
I/O
eat(order_food) drink(order_tea)
If we could control side-effect and do some static analysis...
![Page 66: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/66.jpg)
Interface forMixed data dependency
![Page 67: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/67.jpg)
If callback is the only thing we have...
![Page 68: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/68.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
# order_food is blocking order_spices order_food{ |food| order_spices{ |spices| eat(add_spices(spices, food)) } }
We don't want to do this
![Page 69: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/69.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
food, spices = nil order_food{ |arrived_food| food = arrived_food start_eating(food, spices) if food && spices } order_spices{ |arrived_spices| spices = arrived_spices start_eating(food, spices) if food && spices } ## def start_eating food, spices superfood = add_spices(spices, food) eat(superfood) end
Tedious, but performs better
![Page 70: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/70.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
food = order_food spices = order_spices superfood = add_spices(spices, food) eat(superfood) # or one liner eat(add_spices(order_spices, order_food))
Ideally, we could do this with futures
![Page 71: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/71.jpg)
but how?
![Page 72: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/72.jpg)
Implementation
![Page 73: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/73.jpg)
Forget about data dependency for now
Implementation
![Page 74: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/74.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
Implementation: 2 main choices
![Page 75: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/75.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
def order_food Thread.new{ food = order_food_blocking yield(food) }end
with a thread
If order_food is I/O bound
![Page 76: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/76.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
def order_food make_request('order_food'){ |food| yield(food) }end
with a reactor
If order_food is I/O bound
![Page 77: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/77.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
def order_food buf = [] reactor = Thread.current[:reactor] sock = TCPSocket.new('example.com', 80) request = "GET / HTTP/1.0\r\n\r\n" reactor.write sock, request do reactor.read sock do |response| if response buf << response else yield(buf.join)ennnnd
with a very simple reactor
If order_food is I/O bound
https://github.com/godfat/ruby-server-exp/blob/master/sample/reactor.rb
![Page 78: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/78.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
If order_food is CPU bound
def order_food Thread.new{ food = order_food_blocking yield(food) }end
with a thread
![Page 79: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/79.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
def order_food Thread.new{ food = order_food_blocking yield(food) }end
with a thread
If order_food is I/O bound
![Page 80: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/80.jpg)
You don't have to care whether it's CPU bound or I/O bound with a thread
![Page 81: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/81.jpg)
And you won't want to process a CPU bound task inside a reactor blocking other clients.
![Page 82: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/82.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
Summary
• Threads for CPU bound task
• Reactor for I/O bound task
![Page 83: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/83.jpg)
Back to mixed data dependency
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
![Page 84: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/84.jpg)
If we could have some other interface than callbacks
![Page 85: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/85.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
Threads
food, spices = nilt0 = Thread.new{ food = order_food }t1 = Thread.new{ spices = order_spices }t0.joint1.joinsuperfood = add_spices(spices, food)eat(superfood)
We can do it with threads easily
![Page 86: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/86.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
what if we still want callbacks, since then we can pick either threads or reactors as the implementation detail?
![Page 87: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/87.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
food, spices = nilorder_food{ |arrived_food| food = arrived_food start_eating(food, spices) if food && spices}order_spices{ |arrived_spices| spices = arrived_spices start_eating(food, spices) if food && spices}##def start_eating food, spices superfood = add_spices(spices, food) eat(superfood)end
we could use threads or fibers to remove the need for defining another callback (i.e. start_eating)
instead of writing this...
![Page 88: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/88.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
food, spices = nilorder_food{ |arrived_food| food = arrived_food start_eating(food, spices) if food && spices}order_spices{ |arrived_spices| spices = arrived_spices start_eating(food, spices) if food && spices}##def start_eating food, spices superfood = add_spices(spices, food) eat(superfood)end
![Page 89: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/89.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
condv = ConditionVariable.newmutex = Mutex.newfood, spices = nilorder_food{ |arrived_food| food = arrived_food condv.signal if food && spices}order_spices{ |arrived_spices| spices = arrived_spices condv.signal if food && spices}##mutex.synchronize{ condv.wait(mutex) } superfood = add_spices(spices, food) eat(superfood)
Turn threads callback back to synchronized likeThreads
![Page 90: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/90.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
Turn reactor callback to synchronized styleFibersfiber = Fiber.current food, spices = nil order_food{ |arrived_food| food = arrived_food fiber.resume if food && spices } order_spices{ |arrived_spices| spices = arrived_spices fiber.resume if food && spices } ## Fiber.yield superfood = add_spices(spices, food) eat(superfood)
![Page 91: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/91.jpg)
Threads vs Fibers
![Page 92: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/92.jpg)
threads if your request is wrapped inside a thread (e.g. thread pool strategy)
fibers if your request is wrapped inside a fiber (e.g. reactor + fibers)
![Page 93: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/93.jpg)
we're using eventmachine + thread pool with thread synchronization
we used to run fibers, but it didn't work well with other libraries
e.g. activerecord's connection pool didn't respect fibers, only threads
![Page 94: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/94.jpg)
also, using fibers we're running a risk where we might block the event loop somehow we don't know
so using threads is easier if you consider thread-safety is easier than fiber-safety + potential risk of blocking the reactor
![Page 95: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/95.jpg)
and we can even go one step further...
![Page 96: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/96.jpg)
...into the futures!
![Page 97: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/97.jpg)
this is also a demonstration that some
interfaces are only available to some
implementations
food = order_foodspices = order_spicessuperfood = add_spices(spices, food)eat(superfood)
# or one linereat(add_spices(order_spices, order_food))
![Page 98: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/98.jpg)
Who got futures?
• rest-core for HTTP futures
• celluloid for general futures
• also check celluloid-io for replacing eventmachine
![Page 99: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/99.jpg)
a more complex (real world) example• update friend list from facebook
• get photo list from facebook
• download 3 photos from the list
• detect the dimension of the 3 photos
• merge above photos
• upload to facebook
this example shows a mix model of
linear and mixed data dependency
![Page 100: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/100.jpg)
![Page 101: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/101.jpg)
Concurrency?
What We Have?
App Servers?
Me?
Q?
![Page 102: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/102.jpg)
Again:
we don't talk about chunked encoding and web sockets or so for now; simply plain old HTTP 1.0
![Page 103: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/103.jpg)
Network concurrency
Application concurrency
![Page 104: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/104.jpg)
sockets I/O bound tasks would be ideal for an event loop to process them efficiently
however, CPU bound tasks should be handled in real hardware core
nginx, eventmachine,
libev, nodejs, etc.
e.g. kernel process/thread
![Page 105: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/105.jpg)
we can abstract the http server (reverse proxy) easily, since it only needs to do one thing and do it well (unix philosophy)
that is, using an event loop
to buffer the requests
![Page 106: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/106.jpg)
however, different application does different things, one strategy might work well for one application but not the other
![Page 107: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/107.jpg)
we could have an universal concurrency model which could do averagely good, but not perfect for say, *your* application
![Page 108: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/108.jpg)
that is why Rainbows provides all possible concurrency models for you to choose from
![Page 109: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/109.jpg)
what if we want to make external requests to outside world?
it's I/O bound, and could be the most significant bottleneck, much slower than your favorite database
e.g. facebook
![Page 110: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/110.jpg)
before we jump into the detail...
let's see some concurrent popular ruby application servers
![Page 111: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/111.jpg)
Thin, Puma, Unicorn family
![Page 112: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/112.jpg)
Default Thin
eventmachine (event loop) for buffering requests
no application concurrency
you can run thin cluster for
application concurrency
![Page 113: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/113.jpg)
Threaded Thin
eventmachine (event loop) for buffering requests
thread pool to serve requests
you can of course run
cluster for this
![Page 114: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/114.jpg)
Puma
zbatery + ThreadPool = puma
![Page 115: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/115.jpg)
Unicorn
no network concurrency
worker process application concurrency
![Page 116: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/116.jpg)
Rainbows
another concurrency model + unicorn
![Page 117: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/117.jpg)
Zbatery
rainbows with single unicorn (no fork)
saving memories
![Page 118: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/118.jpg)
Zbatery + EventMachine = default Thin
![Page 119: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/119.jpg)
Rainbows + EventMachine = cluster default Thin
![Page 120: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/120.jpg)
Zbatery + EventMachine + TryDefer (thread pool) = threaded Thin
![Page 121: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/121.jpg)
Each model has its strength to deal with different task
![Page 122: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/122.jpg)
blocking threads
callback reactor fibers
CPU
I/O callback reactor
blocking threads
fibers
CPU
I/O
Remember? threads for cpu operations, reactor for I/O operations
![Page 123: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/123.jpg)
What if we want to resize images, encode videos?
it's of course CPU bound, and should be
handled in a real core/CPU
![Page 124: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/124.jpg)
What if we want to do both? what if we first request facebook, and then encode video, or vice versa?
or we need to request facebook and encode
videos and request facebook again?
![Page 125: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/125.jpg)
The reactor could be used for http concurrency and also making external requests
![Page 126: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/126.jpg)
Ultimate solutionfor what i can think of right now
![Page 127: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/127.jpg)
USE EVERYTHING
![Page 128: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/128.jpg)
Rainbows + EventMachine + Thread pool + Futures!
![Page 129: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/129.jpg)
And how do we do that in a web application? We'll need to do the above example in a concurrent way. i.e.
![Page 130: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/130.jpg)
To compare all the application servers above
![Page 131: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/131.jpg)
Thin (default)
Thin (threaded)
Puma
Unicorn
Rainbows
Zbatery
Passenger
Goliath
EventMachine
EventMachine
Thread pool
Worker processes
Worker processes + depends on configurations
Depends on configurations
I/O threadslibev
EventMachine
N/A
Thread pool
Thread pool
Worker processes
Process poolProcess/thread pool
N/A
Rack
Rack
Rack
Rack
Rack
Rack
Rack
Wrapped Rack
Network Interface Application
![Page 132: Concurrent Ruby Application Servers](https://reader031.fdocuments.us/reader031/viewer/2022013013/58f316151a28abad468b4569/html5/thumbnails/132.jpg)
Concurrency?
What We Have?
App Servers?
Me?
Q?