Accelerating Snabb Programs€¦ · Accelerating Snabb Programs Fabian Bonk May 1, 2018 Chair of...
Transcript of Accelerating Snabb Programs€¦ · Accelerating Snabb Programs Fabian Bonk May 1, 2018 Chair of...
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Accelerating Snabb Programs
Fabian Bonk
May 1, 2018
Chair of Network Architectures and ServicesDepartment of Informatics
Technical University of Munich
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Snabb
Snabb is an extensible, virtualized, Ethernet networkingtoolkit.
Programmers write Lua modules that handle packets and connect themtogether.
Fabian Bonk – Accelerating Snabb Programs 2
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Snabb
Snabb is an extensible, virtualized, Ethernet networkingtoolkit.
Programmers write Lua modules that handle packets and connect themtogether.
Fabian Bonk – Accelerating Snabb Programs 2
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Naming conventions
• Apps (= Modules)• Programs (= Configs)
Fabian Bonk – Accelerating Snabb Programs 3
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Naming conventions
• Apps (= Modules)
• Programs (= Configs)
Fabian Bonk – Accelerating Snabb Programs 3
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Naming conventions
• Apps (= Modules)• Programs (= Configs)
Fabian Bonk – Accelerating Snabb Programs 3
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Fabian Bonk – Accelerating Snabb Programs 4
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Fabian Bonk – Accelerating Snabb Programs 5
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Snabb Core
Snabb Core = Snabb runtime
• initialize apps• set up links between apps• memory management• main loop
Fabian Bonk – Accelerating Snabb Programs 6
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Snabb Core
Snabb Core = Snabb runtime
• initialize apps• set up links between apps• memory management• main loop
Fabian Bonk – Accelerating Snabb Programs 6
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Snabb Core
Snabb Core = Snabb runtime
• initialize apps
• set up links between apps• memory management• main loop
Fabian Bonk – Accelerating Snabb Programs 6
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Snabb Core
Snabb Core = Snabb runtime
• initialize apps• set up links between apps
• memory management• main loop
Fabian Bonk – Accelerating Snabb Programs 6
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Snabb Core
Snabb Core = Snabb runtime
• initialize apps• set up links between apps• memory management
• main loop
Fabian Bonk – Accelerating Snabb Programs 6
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Snabb Core
Snabb Core = Snabb runtime
• initialize apps• set up links between apps• memory management• main loop
Fabian Bonk – Accelerating Snabb Programs 6
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Snabb API
Appsmyapp:new(arg)myapp:push()myapp:pull()
Programslocal c = config.new()config.app(c, "app1", myapp, "params for app 1")config.app(c, "app2", myapp, "params for app 2")config.link(c, "app1.output -> app2.input")
engine.configure(c)engine.main()
Running
$ snabb run myprogram
Fabian Bonk – Accelerating Snabb Programs 7
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Snabb APIAppsmyapp:new(arg)myapp:push()myapp:pull()
Programslocal c = config.new()config.app(c, "app1", myapp, "params for app 1")config.app(c, "app2", myapp, "params for app 2")config.link(c, "app1.output -> app2.input")
engine.configure(c)engine.main()
Running
$ snabb run myprogram
Fabian Bonk – Accelerating Snabb Programs 7
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Snabb APIAppsmyapp:new(arg)myapp:push()myapp:pull()
Programslocal c = config.new()config.app(c, "app1", myapp, "params for app 1")config.app(c, "app2", myapp, "params for app 2")config.link(c, "app1.output -> app2.input")
engine.configure(c)engine.main()
Running
$ snabb run myprogram
Fabian Bonk – Accelerating Snabb Programs 7
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Snabb APIAppsmyapp:new(arg)myapp:push()myapp:pull()
Programslocal c = config.new()config.app(c, "app1", myapp, "params for app 1")config.app(c, "app2", myapp, "params for app 2")config.link(c, "app1.output -> app2.input")
engine.configure(c)engine.main()
Running
$ snabb run myprogramFabian Bonk – Accelerating Snabb Programs 7
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
DPDK
DPDK = Data Plane Development KitDPDK is a set of libraries and drivers forfast packet processing.
Fabian Bonk – Accelerating Snabb Programs 8
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
DPDK
DPDK = Data Plane Development Kit
DPDK is a set of libraries and drivers forfast packet processing.
Fabian Bonk – Accelerating Snabb Programs 8
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
DPDK
DPDK = Data Plane Development KitDPDK is a set of libraries and drivers forfast packet processing.
Fabian Bonk – Accelerating Snabb Programs 8
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
libmoon
LuaJIT + DPDK = fast and flexible packet processing atspeeds above 100 Gbit/s.
Lua wrapper around DPDK
Fabian Bonk – Accelerating Snabb Programs 9
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
libmoon
LuaJIT + DPDK = fast and flexible packet processing atspeeds above 100 Gbit/s.
Lua wrapper around DPDK
Fabian Bonk – Accelerating Snabb Programs 9
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
What we have
Fabian Bonk – Accelerating Snabb Programs 10
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
What we want
Fabian Bonk – Accelerating Snabb Programs 11
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Solution
Build snabb-libmoon-compat
• Reimplement Snabb Core to call into libmoon• Expose DPDK devices as Snabb Apps
Fabian Bonk – Accelerating Snabb Programs 12
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Solution
Build snabb-libmoon-compat
• Reimplement Snabb Core to call into libmoon• Expose DPDK devices as Snabb Apps
Fabian Bonk – Accelerating Snabb Programs 12
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Solution
Build snabb-libmoon-compat
• Reimplement Snabb Core to call into libmoon
• Expose DPDK devices as Snabb Apps
Fabian Bonk – Accelerating Snabb Programs 12
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Solution
Build snabb-libmoon-compat
• Reimplement Snabb Core to call into libmoon• Expose DPDK devices as Snabb Apps
Fabian Bonk – Accelerating Snabb Programs 12
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Caveats
• no multiprocessing/multithreading (yet?)• only partial API compatibility
Fabian Bonk – Accelerating Snabb Programs 13
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Caveats
• no multiprocessing/multithreading
(yet?)• only partial API compatibility
Fabian Bonk – Accelerating Snabb Programs 13
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Caveats
• no multiprocessing/multithreading (yet?)
• only partial API compatibility
Fabian Bonk – Accelerating Snabb Programs 13
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Caveats
• no multiprocessing/multithreading (yet?)• only partial API compatibility
Fabian Bonk – Accelerating Snabb Programs 13
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
snabb-libmoon-compat
• Apache 2.0 License• Lua only (no C)• https://github.com/Reperator/snabb-libmoon-compat
• https://fabianbonk.de/snabb-libmoon-compat/
Fabian Bonk – Accelerating Snabb Programs 14
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
snabb-libmoon-compat
• Apache 2.0 License
• Lua only (no C)• https://github.com/Reperator/snabb-libmoon-compat
• https://fabianbonk.de/snabb-libmoon-compat/
Fabian Bonk – Accelerating Snabb Programs 14
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
snabb-libmoon-compat
• Apache 2.0 License• Lua only (no C)
• https://github.com/Reperator/snabb-libmoon-compat
• https://fabianbonk.de/snabb-libmoon-compat/
Fabian Bonk – Accelerating Snabb Programs 14
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
snabb-libmoon-compat
• Apache 2.0 License• Lua only (no C)• https://github.com/Reperator/snabb-libmoon-compat
• https://fabianbonk.de/snabb-libmoon-compat/
Fabian Bonk – Accelerating Snabb Programs 14
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
snabb-libmoon-compat
• Apache 2.0 License• Lua only (no C)• https://github.com/Reperator/snabb-libmoon-compat
• https://fabianbonk.de/snabb-libmoon-compat/
Fabian Bonk – Accelerating Snabb Programs 14
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Performance
Performance target was achieved
Snabb’s maximum packet rate: 14.88 Mpps (no support for faster NICs)
snabb-libmoon-compat: up to 18.65 Mpps (using 40GbE NIC)
Fabian Bonk – Accelerating Snabb Programs 15
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Performance
Performance target was achieved
Snabb’s maximum packet rate: 14.88 Mpps (no support for faster NICs)
snabb-libmoon-compat: up to 18.65 Mpps (using 40GbE NIC)
Fabian Bonk – Accelerating Snabb Programs 15
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Performance
Performance target was achieved
Snabb’s maximum packet rate: 14.88 Mpps (no support for faster NICs)
snabb-libmoon-compat: up to 18.65 Mpps (using 40GbE NIC)
Fabian Bonk – Accelerating Snabb Programs 15
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Performance
Performance target was achieved
Snabb’s maximum packet rate: 14.88 Mpps (no support for faster NICs)
snabb-libmoon-compat: up to 18.65 Mpps (using 40GbE NIC)
Fabian Bonk – Accelerating Snabb Programs 15
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Test setup
Fabian Bonk – Accelerating Snabb Programs 16
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Test systems
CPU CPU freq. L3 cache NICIntel Xeon E5-2620 v3 3.2 GHz 15 MiB Intel XL710
Intel 82599ES2 Intel Xeon E5-2630 v4 3.1 GHz 25 MiB Intel XL710
Intel 82599ESIntel i7-3770K 4.5 GHz 8 MiB Intel 82599ESAMD Threadripper 1950X 4.2 GHz 32 MiB Intel 82599ES
Fabian Bonk – Accelerating Snabb Programs 17
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Intel Xeon E5-2620 v3 + Intel XL710/Intel 82599ES
2 4 6 8 10 12 14 16 18 200
10
20
30
10GbE maximum
~XL710 maximum
Number of forward apps
Pac
ketr
ate
[Mpp
s]
Snabbsnabb-libmoon-compat(10GbE)snabb-libmoon-compat(40GbE)
Fabian Bonk – Accelerating Snabb Programs 18
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Intel Xeon E5-2620 v3 + Intel XL710/Intel 82599ES
10 20 30 40 50 60 70 80 90 1000
10
20
30
10GbE maximum
~XL710 maximum
Number of forward apps
Pac
ketr
ate
[Mpp
s]
Snabbsnabb-libmoon-compat(10GbE)snabb-libmoon-compat(40GbE)
Fabian Bonk – Accelerating Snabb Programs 19
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Intel Xeon E5-2630 v4 + Intel XL710/Intel 82599ES
2 4 6 8 10 12 14 16 18 200
10
20
30
10GbE maximum
~XL710 maximum
Number of forward apps
Pac
ketr
ate
[Mpp
s]
Snabbsnabb-libmoon-compat(10GbE)snabb-libmoon-compat(40GbE)
Fabian Bonk – Accelerating Snabb Programs 20
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Intel Xeon E5-2630 v4 + Intel XL710/Intel 82599ES
10 20 30 40 50 60 70 80 90 1000
10
20
30
10GbE maximum
~XL710 maximum
Number of forward apps
Pac
ketr
ate
[Mpp
s]
Snabbsnabb-libmoon-compat(10GbE)snabb-libmoon-compat(40GbE)
Fabian Bonk – Accelerating Snabb Programs 21
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Intel i7-3770K + Intel 82599ES
2 4 6 8 10 12 14 16 18 200
5
10
15
20
10GbE maximum
Number of forward apps
Pac
ketr
ate
[Mpp
s]
Snabbsnabb-libmoon-compat(10GbE)
Fabian Bonk – Accelerating Snabb Programs 22
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Intel i7-3770K + Intel 82599ES
10 20 30 40 50 60 70 80 90 1000
5
10
15
20
10GbE maximum
Number of forward apps
Pac
ketr
ate
[Mpp
s]
Snabbsnabb-libmoon-compat(10GbE)
Fabian Bonk – Accelerating Snabb Programs 23
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
AMD Threadripper 1950X + Intel 82599ES
2 4 6 8 10 12 14 16 18 200
5
10
15
20
10GbE maximum
Number of forward apps
Pac
ketr
ate
[Mpp
s]
Snabbsnabb-libmoon-compat(10GbE)
Fabian Bonk – Accelerating Snabb Programs 24
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
AMD Threadripper 1950X + Intel 82599ES
10 20 30 40 50 60 70 80 90 1000
5
10
15
20
10GbE maximum
Number of forward apps
Pac
ketr
ate
[Mpp
s]
Snabbsnabb-libmoon-compat(10GbE)
Fabian Bonk – Accelerating Snabb Programs 25
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Future Development
• automatic/manual multithreading• expanded API compatibility• UI improvements
Fabian Bonk – Accelerating Snabb Programs 26
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Future Development
• automatic/manual multithreading
• expanded API compatibility• UI improvements
Fabian Bonk – Accelerating Snabb Programs 26
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Future Development
• automatic/manual multithreading• expanded API compatibility
• UI improvements
Fabian Bonk – Accelerating Snabb Programs 26
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Future Development
• automatic/manual multithreading• expanded API compatibility• UI improvements
Fabian Bonk – Accelerating Snabb Programs 26
Chair of Network Architectures and ServicesDepartment of InformaticsTechnical University of Munich
Links
Questions:
• [email protected](PGP 0x04124B4CFA189C26)
• [email protected](PGP 0x578B2D00E529403B)
Code:
• https://github.com/Reperator/snabb-libmoon-compat
Info & Thesis:
• https://fabianbonk.de/snabb-libmoon-compat/
Fabian Bonk – Accelerating Snabb Programs 27