Highly concurrent yet natural programming
Transcript of Highly concurrent yet natural programming
![Page 1: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/1.jpg)
Highly concurrent yet naturalprogramming
Version 1.2
![Page 2: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/2.jpg)
Infinit & me
Me
• Quentin "mefyl" Hocquet• Epita CSI (LRDE) 2008.• Ex Gostai• Into language theory• Joined Infinit early two years ago.
![Page 3: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/3.jpg)
Infinit & me
Me
• Quentin "mefyl" Hocquet• Epita CSI (LRDE) 2008.• Ex Gostai• Into language theory• Joined Infinit early two years ago.
Infinit
• Founded my Julien "mycure" Quintard, Epita SRS 2007• Based on his thesis at Cambridge• Decentralized filesystem in byzantine environment• Frontend: file transfer application based on the technology.• Strong technical culture
![Page 4: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/4.jpg)
Concurrent and parallelprogramming
![Page 5: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/5.jpg)
Know the difference
Parallel programmingAims at running two tasks simultaneously. It is a matter of performances.
Concurrent programmingAims at running two tasks without inter-blocking. It is a matter of behavior.
![Page 6: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/6.jpg)
Task 1 Task 2
Know the difference
![Page 7: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/7.jpg)
Task 1 Task 2
Know the difference
Sequential
![Page 8: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/8.jpg)
Task 1 Task 2
Know the difference
Parallel
![Page 9: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/9.jpg)
Task 1 Task 2
Know the difference
Concurrent
![Page 10: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/10.jpg)
Sequential Concurrent
Know the difference
Parallel
![Page 11: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/11.jpg)
Sequential Concurrent
Know the difference
Parallel
Sequential Concurrent Parallel
CPU usage N N N
Execution time Long Short Shorter
![Page 12: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/12.jpg)
Sequential Concurrent
Know the difference
Parallel
Sequential Concurrent Parallel
CPU usage N N N
Execution time Long Short Shorter
Need to run in parallel No No Yes
![Page 13: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/13.jpg)
TV
Commercials
TV
Peeling
Some real life examples
You are the CPU. You want to:
• Watch a film on TV.• Peel potatoes.
![Page 14: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/14.jpg)
SequentialTV
Commercials
TV
Peeling
ConcurrentTV
Peeling
TV
Peeling
Some real life examples
ParallelTV Peeling
Commercials
TV
![Page 15: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/15.jpg)
Load
Unload
Load
Unload
Some real life examples
You are the CPU. You want to:
• Do the laundry.• Do the dishes.
![Page 16: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/16.jpg)
SequentialLoad
UnloadLoad
Unload
ConcurrentLoadLoad
UnloadUnload
Some real life examples
ParallelLoad Load
Unload Unload
![Page 17: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/17.jpg)
Some programming examples
Video encoding: encode a raw 2GB raw file to mp4.
• CPU bound.• File chunks can be encoded separately and then merged later.
![Page 18: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/18.jpg)
ParallelEncodefirst half
Encodesecond half
SequentialConcurrentEncode first
half
Encodesecond half
Some programming examples
Video encoding: encode a raw 2GB raw file to mp4.
• CPU bound.• File chunks can be encoded separately and then merged later.
Parallelism is a plus, concurrency doesn't apply.
![Page 19: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/19.jpg)
Some programming examples
An IRC server: handle up to 50k IRC users chatting.
• IO bound.• A huge number of clients that must be handled concurrently and mostly
waiting.
![Page 20: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/20.jpg)
Concurrent Parallel
Some programming examples
An IRC server: handle up to 50k IRC users chatting.
• IO bound.• A huge number of clients that must be handled concurrently and mostly
waiting.
Concurrency is needed, parallelism is superfluous.
![Page 21: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/21.jpg)
Know the difference
Parallelism
• Is never needed for correctness.• Is about performances, not correct behavior.• Is about exploiting multi-core and multi-CPU architectures.
Concurrent programming
• Can be needed for correctness.• Is about correct behavior, sometimes about performances too.• Is about multiple threads being responsive in concurrent.
![Page 22: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/22.jpg)
Know the difference
Parallelism
• Is never needed for correctness.• Is about performances, not correct behavior.• Is about exploiting multi-core and multi-CPU architectures.
Concurrent programming
• Can be needed for correctness.• Is about correct behavior, sometimes about performances too.• Is about multiple threads being responsive in concurrent.
A good video encoding app:
• Encodes 4 times faster on a 4-core CPU. That's parallelism.
![Page 23: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/23.jpg)
Know the difference
Parallelism
• Is never needed for correctness.• Is about performances, not correct behavior.• Is about exploiting multi-core and multi-CPU architectures.
Concurrent programming
• Can be needed for correctness.• Is about correct behavior, sometimes about performances too.• Is about multiple threads being responsive in concurrent.
A good video encoding app:
• Encodes 4 times faster on a 4-core CPU. That's parallelism.• Has a responsive GUI while encoding. That's concurrency.
![Page 24: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/24.jpg)
Who's best ?
If you are parallel, you are concurrent. So why bother ?
![Page 25: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/25.jpg)
Who's best ?
If you are parallel, you are concurrent. So why bother ?
• Being parallel is much, much more difficult. That's time, money andprogrammer misery.
![Page 26: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/26.jpg)
Who's best ?
If you are parallel, you are concurrent. So why bother ?
• Being parallel is much, much more difficult. That's time, money andprogrammer misery.
• You can't be efficiently parallel past your hardware limit. Those are systemcalls, captain.
![Page 27: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/27.jpg)
Threads, callbacks
So, how do you write an echo server ?
![Page 28: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/28.jpg)
The sequential echo server
TCPServer server;server.listen(4242);while (true){
TCPSocket client = server.accept();
}
![Page 29: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/29.jpg)
The sequential echo server
TCPServer server;server.listen(4242);while (true){
TCPSocket client = server.accept();
while (true){std::string line = client.read_until("\n");client.send(line);
}
}
![Page 30: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/30.jpg)
The sequential echo server
TCPServer server;server.listen(4242);while (true){
TCPSocket client = server.accept();try{while (true){std::string line = client.read_until("\n");client.send(line);
}}catch (ConnectionClosed const&){}
}
![Page 31: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/31.jpg)
The sequential echo server
TCPServer server;server.listen(4242);while (true){
TCPSocket client = server.accept();serve_client(client);
}
![Page 32: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/32.jpg)
The sequential echo server
TCPServer server;server.listen(4242);while (true){
TCPSocket client = server.accept();serve_client(client);
}
• Dead simple: you got it instantly. It's natural.• But wrong: we handle only one client at a time.• We need ...
![Page 33: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/33.jpg)
The sequential echo server
TCPServer server;server.listen(4242);while (true){
TCPSocket client = server.accept();serve_client(client);
}
• Dead simple: you got it instantly. It's natural.• But wrong: we handle only one client at a time.• We need ... concurrency !
![Page 34: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/34.jpg)
The parallel echo server
TCPServer server;server.listen(4242);
while (true){
TCPSocket client = server.accept();
serve_client(client);
}
![Page 35: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/35.jpg)
The parallel echo server
TCPServer server;server.listen(4242);std::vector<std::thread> threads;while (true){
TCPSocket client = server.accept();std::thread client_thread([&]{serve_client(client);
});client_thread.run();vectors.push_back(std::move(client_thread));
}
![Page 36: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/36.jpg)
The parallel echo server
TCPServer server;server.listen(4242);std::vector<std::thread> threads;while (true){
TCPSocket client = server.accept();std::thread client_thread([&]{serve_client(client);
});client_thread.run();vectors.push_back(std::move(client_thread));
}
• Almost as simple and still natural,• To add the concurrency property, we just added a concurrency construct
to the existing.
![Page 37: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/37.jpg)
But parallelism is too much
• Not scalable: you can't run 50k threads.
![Page 38: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/38.jpg)
But parallelism is too much
• Not scalable: you can't run 50k threads.• Induces unwanted complexity: race conditions.
![Page 39: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/39.jpg)
But parallelism is too much
• Not scalable: you can't run 50k threads.• Induces unwanted complexity: race conditions.
int line_count = 0;while (true){
TCPSocket client = server.accept();
while (true){
std::string line = client.read_until("\n");client.send(line);++line_count;
}
}
![Page 40: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/40.jpg)
But parallelism is too much
• Not scalable: you can't run 50k threads.• Induces unwanted complexity: race conditions.
int line_count = 0;while (true){
TCPSocket client = server.accept();std::thread client_thread([&]{while (true){
std::string line = client.read_until("\n");client.send(line);++line_count;
}});
}
![Page 41: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/41.jpg)
We need concurrency without threads
We need to accept, read and write to socket without threads so withoutblocking.
![Page 42: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/42.jpg)
We need concurrency without threads
We need to accept, read and write to socket without threads so withoutblocking.
• Use select to monitor all sockets at once.• Register actions to be done when something is ready.• Wake up only when something needs to be performed.
![Page 43: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/43.jpg)
We need concurrency without threads
We need to accept, read and write to socket without threads so withoutblocking.
• Use select to monitor all sockets at once.• Register actions to be done when something is ready.• Wake up only when something needs to be performed.
This is abstracted with the reactor design pattern:
• libevent• Boost ASIO• Python Twisted• ...
![Page 44: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/44.jpg)
The callback-based echo server
Reactor reactor;TCPServer server(reactor);
server.accept(&handle_connection);reactor.run();
![Page 45: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/45.jpg)
The callback-based echo server
Reactor reactor;TCPServer server(reactor);
server.accept(&handle_connection);reactor.run();
voidhandle_connection(TCPSocket& client){
client.read_until("\n", &handle_read);}
![Page 46: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/46.jpg)
The callback-based echo server
Reactor reactor;TCPServer server(reactor);
server.accept(&handle_connection);reactor.run();
voidhandle_connection(TCPSocket& client);
voidhandle_read(TCPSocket& c, std::string const& l, Error e){
if (!e)c.send(l, &handle_sent);
}
![Page 47: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/47.jpg)
The callback-based echo server
Reactor reactor;TCPServer server(reactor);
server.accept(&handle_connection);reactor.run();
voidhandle_connection(TCPSocket& client);
voidhandle_read(TCPSocket& c, std::string const& l, Error e);
voidhandle_sent(TCPSocket& client, Error error){
if (!e)client.read_until("\n", &handle_read);
}
![Page 48: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/48.jpg)
How do we feel now ?
• This one scales to thousands of client.
![Page 49: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/49.jpg)
How do we feel now ?
• This one scales to thousands of client.• Yet to add the concurrency property, we had to completely change the way
we think.
![Page 50: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/50.jpg)
How do we feel now ?
• This one scales to thousands of client.• Yet to add the concurrency property, we had to completely change the way
we think.• A bit more verbose and complex, but nothing too bad ... right ?
![Page 51: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/51.jpg)
Counting lines with threads
try{
while (true){std::string line = client.read_until("\n");
client.send(line);}
}catch (ConnectionClosed const&){
}
![Page 52: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/52.jpg)
Counting lines with threads
int lines_count = 0;try{
while (true){std::string line = client.read_until("\n");++lines_count;client.send(line);
}}catch (ConnectionClosed const&){
std::cerr << "Client sent " << lines_count << "lines\n";}
![Page 53: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/53.jpg)
Counting lines with callbacks
voidhandle_connection(TCPSocket& client){
int* count = new int(0);client.read_until("\n", std::bind(&handle_read, count));
}
![Page 54: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/54.jpg)
Counting lines with callbacks
voidhandle_connection(TCPSocket& client);
voidhandle_read(TCPSocket& c, std::string const& l,
Error e, int* count){
if (e)std::cerr << *count << std::endl;
elsec.send(l, std::bind(&handle_sent, count));
}
![Page 55: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/55.jpg)
Counting lines with callbacks
voidhandle_connection(TCPSocket& client);
voidhandle_read(TCPSocket& c, std::string const& l,
Error e, int* count);
voidhandle_sent(TCPSocket& client, Error error, int* count){
if (e)std::cerr << *count << std::endl;
elseclient.read_until("\n", std::bind(&handle_read, count));
}
![Page 56: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/56.jpg)
Callback-based programming considered harmful
• Code is structured with callbacks.
![Page 57: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/57.jpg)
Callback-based programming considered harmful
• Code is structured with callbacks.• Asynchronous operation break the flow arbitrarily.
![Page 58: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/58.jpg)
Callback-based programming considered harmful
• Code is structured with callbacks.• Asynchronous operation break the flow arbitrarily.• You lose all syntactic scoping expression (local variables, closure,
exceptions, ...).
![Page 59: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/59.jpg)
Callback-based programming considered harmful
• Code is structured with callbacks.• Asynchronous operation break the flow arbitrarily.• You lose all syntactic scoping expression (local variables, closure,
exceptions, ...).• This is not natural. Damn, this is pretty much as bad as GOTO.
![Page 60: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/60.jpg)
Are we screwed ?
Threads
• Respect your beloved semantic and expressiveness.• Don't scale and introduce race conditions.
![Page 61: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/61.jpg)
Are we screwed ?
Threads
• Respect your beloved semantic and expressiveness.• Don't scale and introduce race conditions.
Callbacks
• Scale.• Ruins your semantic. Painful to write, close to impossible to maintain.
![Page 62: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/62.jpg)
Are we screwed ?
Threads
• Respect your beloved semantic and expressiveness.• Don't scale and introduce race conditions.
Callbacks
• Scale.• Ruins your semantic. Painful to write, close to impossible to maintain.
I lied when I said: we need concurrency without threads.
![Page 63: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/63.jpg)
Are we screwed ?
Threads
• Respect your beloved semantic and expressiveness.• Don't scale and introduce race conditions.
Callbacks
• Scale.• Ruins your semantic. Painful to write, close to impossible to maintain.
I lied when I said: we need concurrency without threads.
We need concurrency without system threads.
![Page 64: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/64.jpg)
Coroutines
Also known as:
• green threads• userland threads• fibers• contexts• ...
![Page 65: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/65.jpg)
Coroutines
• Separate execution contexts like system threads.• Userland: no need to ask the kernel.• Non-parallel.• Cooperative instead of preemptive: they yield to each other.
![Page 66: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/66.jpg)
Coroutines
• Separate execution contexts like system threads.• Userland: no need to ask the kernel.• Non-parallel.• Cooperative instead of preemptive: they yield to each other.
By building on top of that, we have:
• Scalability: no system thread involved.• No arbitrary race-conditions: no parallelism.• A stack, a context: the code is natural.
![Page 67: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/67.jpg)
Coroutines-based scheduler
• Make a scheduler that holds coroutines .• Embed a reactor in there.• Write a neat Socket class.
![Page 68: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/68.jpg)
Coroutines-based scheduler
• Make a scheduler that holds coroutines .• Embed a reactor in there.• Write a neat Socket class. When read, it:
◦ Unschedules itself.◦ Asks the reactor to read◦ Pass a callback to reschedule itself◦ Yield control back.
![Page 69: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/69.jpg)
Coroutines-based echo server
TCPServer server; server.listen(4242);std::vector<Thread> threads;int lines_count = 0;while (true){
TCPSocket client = server.accept();Thread t([client = std::move(client)] {try{while (true){
++lines_count;client.send(client.read_until("\n"));
}}catch (ConnectionClosed const&) {}
});threads.push_back(std::move(t));
}
![Page 70: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/70.jpg)
What we built at Infinit: the reactor.
![Page 71: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/71.jpg)
What we built at Infinit: the reactor.
• Coroutine scheduler: simple round robin• Sleeping, waiting• Timers• Synchronization• Mutexes, semaphores• TCP networking• SSL• UPnP• HTTP client (Curl based)
![Page 72: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/72.jpg)
Coroutine scheduling
reactor::Scheduler sched;reactor::Thread t1(sched,
[&]{print("Hello 1");reactor::yield();print("Bye 1");
});reactor::Thread t2(sched,
[&]{print("Hello 2");reactor::yield();print("Bye 2");
}););sched.run();
![Page 73: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/73.jpg)
Coroutine scheduling
reactor::Scheduler sched;reactor::Thread t1(sched,
[&]{print("Hello 1");reactor::yield();print("Bye 1");
});reactor::Thread t2(sched,
[&]{print("Hello 2");reactor::yield();print("Bye 2");
}););sched.run();
Hello 1Hello 2Bye 1Bye 2
![Page 74: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/74.jpg)
Sleeping and waiting
reactor::Thread t1(sched,[&]{print("Hello 1");reactor::sleep(500_ms);print("Bye 1");
});reactor::Thread t2(sched,
[&]{print("Hello 2");reactor::yield();print("World 2");reactor::yield();print("Bye 2");
}););
![Page 75: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/75.jpg)
Sleeping and waiting
reactor::Thread t1(sched,[&]{print("Hello 1");reactor::sleep(500_ms);print("Bye 1");
});reactor::Thread t2(sched,
[&]{print("Hello 2");reactor::yield();print("World 2");reactor::yield();print("Bye 2");
}););
Hello 1Hello 2World 2Bye 2
![Page 76: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/76.jpg)
Sleeping and waiting
reactor::Thread t1(sched,[&]{print("Hello 1");reactor::sleep(500_ms);print("Bye 1");
});reactor::Thread t2(sched,
[&]{print("Hello 2");reactor::yield();print("World 2");reactor::yield();print("Bye 2");
}););
Hello 1Hello 2World 2Bye 2Bye 1
![Page 77: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/77.jpg)
Sleeping and waiting
reactor::Thread t1(sched,[&]{print("Hello 1");reactor::sleep(500_ms);print("Bye 1");
});reactor::Thread t2(sched,
[&]{print("Hello 2");reactor::yield();print("World 2");reactor::wait(t1); // Waitprint("Bye 2");
}););
![Page 78: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/78.jpg)
Sleeping and waiting
reactor::Thread t1(sched,[&]{print("Hello 1");reactor::sleep(500_ms);print("Bye 1");
});reactor::Thread t2(sched,
[&]{print("Hello 2");reactor::yield();print("World 2");reactor::wait(t1); // Waitprint("Bye 2");
}););
Hello 1Hello 2World 2
![Page 79: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/79.jpg)
Sleeping and waiting
reactor::Thread t1(sched,[&]{print("Hello 1");reactor::sleep(500_ms);print("Bye 1");
});reactor::Thread t2(sched,
[&]{print("Hello 2");reactor::yield();print("World 2");reactor::wait(t1); // Waitprint("Bye 2");
}););
Hello 1Hello 2World 2Bye 1Bye 2
![Page 80: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/80.jpg)
Synchronization: signals
reactor::Signal task_available;std::vector<Task> tasks;
reactor::Thread handler([&] {while (true){if (!tasks.empty()){std::vector mytasks = std::move(tasks);for (auto& task: tasks)
; // Handle task}elsereactor::wait(task_available);
}});
![Page 81: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/81.jpg)
Synchronization: signals
reactor::Signal task_available;std::vector<Task> tasks;
reactor::Thread handler([&] {while (true){if (!tasks.empty()){std::vector mytasks = std::move(tasks);for (auto& task: tasks)
; // Handle task}elsereactor::wait(task_available);
}});
tasks.push_back(...);task_available.signal();
![Page 82: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/82.jpg)
Synchronization: signals
reactor::Signal task_available;std::vector<Task> tasks;
reactor::Thread handler([&] {while (true){if (!tasks.empty()) // 1{std::vector mytasks = std::move(tasks);for (auto& task: tasks)
; // Handle task}elsereactor::wait(task_available); // 4
}});
tasks.push_back(...); // 2task_available.signal(); // 3
![Page 83: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/83.jpg)
Synchronization: channels
reactor::Channel<Task> tasks;
reactor::Thread handler([&] {while (true){Task t = tasks.get();// Handle task
}});
tasks.put(...);
![Page 84: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/84.jpg)
Mutexes
But you said no race conditions! You lied again!
![Page 85: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/85.jpg)
Mutexes
But you said no race conditions! You lied again!
reactor::Thread t([&] {while (true){
for (auto& socket: sockets)socket.send("YO");
}});
{
socket.push_back(...);
}
![Page 86: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/86.jpg)
Mutexes
But you said no race conditions! You lied again!
reactor::Mutex mutex;reactor::Thread t([&] {
while (true){reactor::wait(mutex);for (auto& socket: sockets)socket.send("YO");
mutex.unlock();}
});
{reactor::wait(mutex);socket.push_back(...);mutex.unlock();
}
![Page 87: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/87.jpg)
Mutexes
But you said no race conditions! You lied again!
reactor::Mutex mutex;reactor::Thread t([&] {
while (true){reactor::Lock lock(mutex);for (auto& socket: sockets)socket.send("YO");
}});
{reactor::Lock lock(mutex);socket.push_back(...);
}
![Page 88: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/88.jpg)
Networking: TCP
We saw a good deal of TCP networking:
try{
reactor::TCPSocket socket("battle.net", 4242, 10_sec);// ...
}catch (reactor::network::ResolutionFailure const&){
// ...}catch (reactor::network::Timeout const&){
// ...}
![Page 89: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/89.jpg)
Networking: TCP
We saw a good deal of TCP networking:
voidserve(TCPSocket& client){
try{std::string auth = server.read_until("\n", 10_sec);if (!check_auth(auth))// Impossible with callbacksthrow InvalidCredentials();
while (true) { ... }}catch (reactor::network::Timeout const&){}
}
![Page 90: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/90.jpg)
Networking: SSL
Transparent client handshaking:
reactor::network::SSLSocket socket("localhost", 4242);socket.write(...);
![Page 91: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/91.jpg)
Networking: SSL
Transparent server handshaking:
reactor::network::SSLServer server(certificate, key);server.listen(4242);while (true){
auto socket = server.accept();reactor::Thread([&] { ... });
}
![Page 92: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/92.jpg)
Networking: SSL
Transparent server handshaking:
SSLSocket SSLServer::accept(){
auto socket = this->_tcp_server.accept();// SSL handshakereturn socket
}
![Page 93: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/93.jpg)
Networking: SSL
Transparent server handshaking:
reactor::Channel<SSLSocket> _sockets;
void SSLServer::_handshake_thread(){
while (true){auto socket = this->_tcp_server.accept();// SSL handshakethis->_sockets.put(socket);
}}
SSLSocket SSLServer::accept(){
return this->_accepted.get;}
![Page 94: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/94.jpg)
Networking: SSL
Transparent server handshaking:
void SSLServer::_handshake_thread(){
while (true){auto socket = this->_tcp_server.accept();reactor::Thread t([&]{
// SSL handshakethis->_sockets.put(socket);
});}
}
![Page 95: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/95.jpg)
HTTP
std::string google = reactor::http::get("google.com");
![Page 96: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/96.jpg)
HTTP
std::string google = reactor::http::get("google.com");
reactor::http::Request r("kissmetrics.com/api",reactor::http::Method::PUT,"application/json",5_sec);
r.write("{ event: \"login\"}");reactor::wait(r);
![Page 97: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/97.jpg)
HTTP
std::string google = reactor::http::get("google.com");
reactor::http::Request r("kissmetrics.com/api",reactor::http::Method::PUT,"application/json",5_sec);
r.write("{ event: \"login\"}");reactor::wait(r);
• Chunking• Cookies• Custom headers• Upload/download progress• ... pretty much anything Curl supports (i.e., everything)
![Page 98: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/98.jpg)
HTTP streaming
std::string content = reactor::http::get("my-api.infinit.io/transactions");
auto json = json::parse(content);
![Page 99: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/99.jpg)
HTTP streaming
std::string content = reactor::http::get("my-api.infinit.io/transactions");
auto json = json::parse(content);
reactor::http::Request r("my-api.production.infinit.io/transactions");
assert(r.status() == reactor::http::Status::OK);// JSON is parsed on the fly;auto json = json::parse(r);
![Page 100: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/100.jpg)
HTTP streaming
std::string content = reactor::http::get("my-api.infinit.io/transactions");
auto json = json::parse(content);
reactor::http::Request r("my-api.production.infinit.io/transactions");
assert(r.status() == reactor::http::Status::OK);// JSON is parsed on the fly;auto json = json::parse(r);
reactor::http::Request r("youtube.com/upload", http::reactor::Method::PUT);
std::ifstream input("~/A new hope - BrRIP.mp4");std::copy(input, r);
![Page 101: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/101.jpg)
Better concurrency: futures, ...
std::string transaction_id = reactor::http::put("my-api.production.infinit.io/transactions");
// Ask the user files to share.reactor::http::post("my-api.infinit.io/transaction/", file_list);std::string s3_token = reactor::http::get("s3.aws.amazon.com/get_token?key=...");
// Upload files to S3
![Page 102: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/102.jpg)
Better concurrency: futures, ...
std::string transaction_id = reactor::http::put("my-api.production.infinit.io/transactions");
// Ask the user files to share.reactor::http::post("my-api.infinit.io/transaction/", file_list);std::string s3_token = reactor::http::get("s3.aws.amazon.com/get_token?key=...");
// Upload files to S3
reactor::http::Request transaction("my-api.production.infinit.io/transactions");
reactor::http::Request s3("s3.aws.amazon.com/get_token?key=...");
// Ask the user files to share.auto transaction_id = transaction.content();reactor::http::Request list("my-api.infinit.io/transaction/", file_list);
auto s3_token = transaction.content();// Upload files to S3
![Page 103: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/103.jpg)
Version 1Wait meta
Ask files
Wait meta
Wait AWS
Version 2Ask files
Better concurrency: futures, ...
Version 2Ask files
![Page 104: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/104.jpg)
How does it perform for us ?
• Notification server does perform:
◦ 10k clients per instance◦ 0.01 load average◦ 1G resident memory◦ Cheap monocore 2.5 Ghz (EC2)
![Page 105: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/105.jpg)
How does it perform for us ?
• Notification server does perform:
◦ 10k clients per instance◦ 0.01 load average◦ 1G resident memory◦ Cheap monocore 2.5 Ghz (EC2)
• Life is so much better:
◦ Code is easy and pleasant to write and read◦ Everything is maintainable◦ Send metrics on login without slowdown? No biggie.◦ Try connecting to several interfaces and keep the first to respond? No
biggie.
![Page 106: Highly concurrent yet natural programming](https://reader031.fdocuments.us/reader031/viewer/2022030305/5871ae311a28abda6a8b6133/html5/thumbnails/106.jpg)
Questions ?