Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified...
Transcript of Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified...
![Page 1: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/1.jpg)
NoahHypervisor-Based Darwin Subsystem for Linux
Takaya Saeki, Yuichi Nishiwaki
![Page 2: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/2.jpg)
Self IntroductionNoah Developer Team
Takaya Saeki Yuichi Nishiwaki
They both are graduate students at the University of Tokyo. They are developing Noah in their free time.Noah was selected as one of MITOH projects, which is a financial assistance program by the government of Japan for outstanding young programmers
![Page 3: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/3.jpg)
Noah• A middleware that runs
unmodified Linux ELF applications on macOS
• Reduce cost of waiting / creating macOS port of Linux apps
• Accomplish it by special hypervisor. Load ELF binary into VM and let it run instead of kernel, trap system calls from it by hypervisor, and translate them to corresponding system calls on macOS.
![Page 4: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/4.jpg)
Noah• A middleware that runs
unmodified Linux ELF applications on macOS
• Reduce cost of waiting / creating macOS port of Linux apps
• Accomplish it by special hypervisor. Load ELF binary into VM and let it run instead of kernel, trap system calls from it by hypervisor, and translate them to corresponding system calls on macOS.
We discuss the architecture in detail later!
![Page 5: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/5.jpg)
Short Demo; What it looks like
![Page 6: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/6.jpg)
Agenda• What is Noah (Done!)
• Background
• Noah in Detail
• Architecture Overview
• Advantages of Noah Architecture
• Subsystem Implementation
• Memory management, VFS, and the other
• Current Implementation Status and Performance
• Related Technologies and Comparison (Windows Subsystem for Linux, Linuxulator, and so on)
• Their Possible Impact on Cross Platform Development
![Page 7: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/7.jpg)
Agenda• What is Noah (Done!)
• Background
• Noah in Detail
• Architecture Overview
• Advantages of Noah Architecture
• Subsystem Implementation
• Memory management, VFS, and the other
• Current Implementation Status and Performance
• Related Technologies and Comparison (Windows Subsystem for Linux, Linuxulator, and so on)
• Their Possible Impact on Cross Platform Development
![Page 8: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/8.jpg)
Background
![Page 9: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/9.jpg)
Linux
• One of the most important operating systems today
• Most popular OS for WEB servers
• Many apps and middleware come from the Linux ecosystem, and they are ported to other operating systems for developers
![Page 10: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/10.jpg)
Problem: Porting costLater..
![Page 11: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/11.jpg)
Porting cost• Many Linux applications are ported to macOS,
FreeBSD, and Windows…
• But it takes time and effort!
• Windows decided to have Linux compatibility layer called “Windows Subsystem for Linux” in 2016 to benefit from the Linux ecosystem directly
• FreeBSD also already has Linux compatibility layer
![Page 12: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/12.jpg)
• macOS does not have Linux compatibility layer yet despite its large number of developers
• Noah fills the missing piece!
![Page 13: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/13.jpg)
If major operating systems have Linux compatibility, developers don’t have to port Linux
applications nor wait for them to be ported
![Page 14: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/14.jpg)
Architecture Overview of Noah
![Page 15: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/15.jpg)
Architecture OverviewNoah’s architecture consists of three components
1. VT-x Virtual Machines
They have NO kernel inside them, but directly boot an ELF binary and let it run instead.
2. Host Noah Processes
Processes that run on the host OS, which actually work as Linux compatibility layer
3. Virtual Machine Monitor module (VMM).
Actually, not a part of Noah itself, but a kernel API of the host OS for virtualization. Apple Hypervisor Framework in macOS, KVM in Linux, for example.
![Page 16: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/16.jpg)
Architecture Overview1. Host Noah process
launches a new VM and loads ELF inside it by ELF loader implemented in the host Noah process
2. The VM runs ELF in its virtualized userland
3. The ELF application calls Linux system calls when running
4. VMM module traps the system call and passes it to the host Noah process
5. Host Noah process emulates the behavior of Linux system call by host OS’s system calls
![Page 17: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/17.jpg)
Architecture Overview
A pair of a host Noah process and a VM corresponds to a Linux application. So, when there are multiple Linux applications, there are also multiple pairs of a host Noah process and a VM.
![Page 18: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/18.jpg)
$ noah /bin/hello
macOS
Example1: How “Hello, world” works
![Page 19: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/19.jpg)
$ noah /bin/hello
Noah
macOS
Example1: How “Hello, world” works
![Page 20: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/20.jpg)
Noah
macOS
Example1: How “Hello, world” works
$ noah /bin/hello
![Page 21: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/21.jpg)
Noah
macOS
ELF loader
Example1: How “Hello, world” works
$ noah /bin/hello
![Page 22: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/22.jpg)
Noah
macOS
hello
ELF loader
Example1: How “Hello, world” works
$ noah /bin/hello
![Page 23: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/23.jpg)
$ noah /bin/hello
Noah
macOS
hello glibc
ELF loader
Example1: How “Hello, world” works
![Page 24: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/24.jpg)
$ noah /bin/hello
Noah
macOS
hello glibc stack area
ELF loader
Example1: How “Hello, world” works
![Page 25: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/25.jpg)
$ noah /bin/hello
Noah
macOS
hello glibc
Run!
Example1: How “Hello, world” works
stack area
![Page 26: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/26.jpg)
$ noah /bin/hello
Noah
macOS
hello glibc
Example1: How “Hello, world” works
stack area
![Page 27: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/27.jpg)
$ noah /bin/hello
Noah
macOS
hello glibc
syscall!
Example1: How “Hello, world” works
stack area
![Page 28: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/28.jpg)
$ noah /bin/hello
Noah
macOS
hello glibc
write(1, “hello”, 6)
Example1: How “Hello, world” works
stack area
![Page 29: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/29.jpg)
$ noah /bin/hello
Noah
macOS
hello glibc
write(1, “hello”, 6)write(1, “hello”, 6)
Example1: How “Hello, world” works
stack area
![Page 30: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/30.jpg)
$ noah /bin/hello
Noah
macOS
hello glibc
6write(1, “hello”, 6)
Example1: How “Hello, world” works
stack area
![Page 31: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/31.jpg)
$ noah /bin/hello
Noah
macOS
hello glibc
6
6
write(1, “hello”, 6)
Example1: How “Hello, world” works
stack area
![Page 32: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/32.jpg)
$ cat file | grep 2017
Noah
macOS
bash
Example2: Interaction between processes
![Page 33: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/33.jpg)
$ cat file | grep 2017
Noah
macOS
bash
fork!
Example2: Interaction between processes
![Page 34: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/34.jpg)
$ cat file | grep 2017
Noah
macOS
bash
Example2: Interaction between processes
![Page 35: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/35.jpg)
$ cat file | grep 2017
Noah
macOS
bash
Noah
Noah forks!
Example2: Interaction between processes
![Page 36: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/36.jpg)
$ cat file | grep 2017
Noah
macOS
bash
Noah
Example2: Interaction between processes
![Page 37: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/37.jpg)
Example2: Interaction between processes
$ cat file | grep 2017
Noah
macOS
bash
Noah
bash
Clone the VM state
![Page 38: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/38.jpg)
$ cat file | grep 2017
Noah
macOS
bash
Noah
bash
Example2: Interaction between processes
![Page 39: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/39.jpg)
$ cat file | grep 2017
Noah
macOS
bash
Noah
bash exec!
Example2: Interaction between processes
![Page 40: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/40.jpg)
$ cat file | grep 2017
Noah
macOS
bash
Noah
cat exec!
Example2: Interaction between processes
![Page 41: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/41.jpg)
$ cat file | grep 2017
Noah
macOS
bash
Noah
cat
Example2: Interaction between processes
![Page 42: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/42.jpg)
$ cat file | grep 2017
Noah
macOS
bash
Noah
cat
Noah
grep
Example2: Interaction between processes
![Page 43: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/43.jpg)
$ cat file | grep 2017
Noah
macOS
bash
Noah
cat
Noah
grep
Example2: Interaction between processes
![Page 44: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/44.jpg)
$ cat file | grep 2017
Noah
macOS
bash
Noah
cat
Noah
grep
Example2: Interaction between processes
![Page 45: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/45.jpg)
$ cat file | grep 2017
Noah
macOS
bash
Noah
cat
Mac App
Example2: Interaction between processes
![Page 46: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/46.jpg)
Example2: Interaction between processes
$ cat file | grep 2017
Noah
macOS
bash
Noah
cat
Mac App
Linux and macOS applications can
communicate naturally
![Page 47: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/47.jpg)
Advantages of Noah Architecture
![Page 48: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/48.jpg)
Unique Characteristics1. All syscall translation is done in user land instead of kernel land
• Still, any sensitive events are trappable with VT-x
2. Launch as many VMs as virtual Linux processes
• No kernel running inside VMs
3. Virtualization is per syscall, not per device I/O
• No need to worry about hardware device emulation
![Page 49: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/49.jpg)
Advantages of Noah Architecture
1. Robustness
Bugs in Noah never cause kernel panic because Noah is just an ordinary userland program (let’s google “WSL bluescreen” now).
2. Portability
The architecture is independent from host OS’s architecture. Syscall calling convention, memory layout, page fault handling rules, …etc are all configurable.
3. Smooth interaction with host OS
Linux process runs as if it is the host OS’s process. Resources such as memory, network, and so on are managed by host OS. No need to worry about the amount of virtual memory allocation like full virtual machines.
![Page 50: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/50.jpg)
Agenda• What is Noah
• Background
• Noah in Detail
• Architecture Overview
• Advantages of Noah Architecture
• Subsystem Implementation
• Memory management, VFS, and the other
• Current Implementation Status and Performance
• Related Technologies and Comparison (Windows Subsystem for Linux, Linuxulator, and so on)
• Their Possible Impact on Cross Platform Development
![Page 51: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/51.jpg)
Subsystem Implementation
![Page 52: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/52.jpg)
Noah Subsystems
Noah consists of subsystems such as memory management, IPC, or file system just like a real kernel.Needs ingenuity to implement some of them because of the nature of Noah’s architecture.
![Page 53: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/53.jpg)
Noah Subsystems
Today we explain two subsystems in detail.
1. Memory management
2. Virtual file system
![Page 54: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/54.jpg)
Memory Management
![Page 55: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/55.jpg)
Memory Management
• Since a Linux ELF binary runs inside a VM, Noah must manage address translation between the VM memory space and the host memory space
• It gives us copy on write ability, efficient exec implementation, but also some difficulty
![Page 56: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/56.jpg)
Guest Virtual Memory
Guest Physical Memory
Host Physical Memory
Guest
Host
48bit
39bit
≦39bit
LINUX APPLICATION
Memory Translation
![Page 57: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/57.jpg)
Guest Virtual Memory
Guest Physical Memory
Host Physical Memory
Guest
Host
48bit
39bit
≦39bit
Memory Translation
LINUX APPLICATION
![Page 58: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/58.jpg)
Guest Virtual Memory
Guest Physical Memory
Host Physical Memory
Guest
Host
48bit
39bit
≦39bit
Memory Translation
LINUX APPLICATION
![Page 59: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/59.jpg)
Guest Virtual Memory
Guest Physical Memory
Host Physical Memory
Guest
Host
48bit
39bit
≦39bit
Memory Translation
LINUX APPLICATION
![Page 60: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/60.jpg)
Guest Virtual Memory
Guest Physical Memory
Host Physical Memory
Guest
Host
48bit
39bit
≦39bit
Memory Translation
LINUX APPLICATION
Double Address Translation!
![Page 61: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/61.jpg)
LINUX APPLICATION
Guest Virtual Memory
Guest Physical Memory
Host Physical Memory
Guest
Host
48bit
39bit
≦39bit
=
Straight Mapping
Memory Translation
![Page 62: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/62.jpg)
LINUX APPLICATION
Guest Virtual Memory
Guest Physical Memory
Host Physical Memory
Guest
Host
48bit
39bit
≦39bit
=
Disable
Memory Translation
Straight Mapping
![Page 63: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/63.jpg)
Guest Virtual Memory
Guest Physical Memory
Host Physical Memory
Guest
Host
48bit
39bit
≦39bit
=
Memory Translation
LINUX APPLICATION
![Page 64: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/64.jpg)
Guest Virtual Memory
Guest Physical Memory
Host Physical Memory
Guest
Host
48bit
39bit
≦39bit
=
Memory Translation
LINUX APPLICATION
![Page 65: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/65.jpg)
Guest Virtual Memory
Guest Physical Memory
Host Physical Memory
Guest
Host
48bit
39bit
≦39bit
=
Memory Translation
LINUX APPLICATION
![Page 66: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/66.jpg)
Guest Virtual Memory
Guest Physical Memory
Host Physical Memory
Guest
Host
48bit
39bit
≦39bit
=
Memory Translation
LINUX APPLICATION
Single Address Translation
![Page 67: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/67.jpg)
Virtual File System
![Page 68: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/68.jpg)
from user
resolve_path
flag_conv strncpy_from_user
fs.ops.open
Symlinks & Mountpoints
to userOOP
Virtual File System
• open system call
![Page 69: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/69.jpg)
/
usretc
Usersdev
tmp
Virtual File System
![Page 70: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/70.jpg)
/
usretc
Usersdev
tmp
/Users
Virtual File System
![Page 71: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/71.jpg)
/
usretc
Usersdev
tmp
/Users/dev
Virtual File System
![Page 72: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/72.jpg)
/
usretc
Usersdev
tmp
/Users/dev~/.noah/tree/usr
Virtual File System
![Page 73: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/73.jpg)
/
usretc
Usersdev
tmp
/Users/dev~/.noah/tree/usr~/.noah/tree/etc
Virtual File System
![Page 74: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/74.jpg)
/
usretc
Usersdev
tmp
/Users/dev~/.noah/tree/usr~/.noah/tree/etc/tmp
Virtual File System
![Page 75: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/75.jpg)
Other System CallsJust call macOS’s one
getuid getpid
alarm
semget
getpgidgetgid
time
Need conversion
futex emulate with conditional value
socket integrate with VFS
sigaction create signal frame inside VM
gettid generate from thread-id
![Page 76: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/76.jpg)
Agenda• What is Noah
• Background
• Noah in Detail
• Architecture Overview
• Advantages of Noah Architecture
• Subsystem Implementation
• Memory management, VFS, and the other
• Current Implementation Status and Performance
• Related Technologies and Comparison (Windows Subsystem for Linux, Linuxulator, and so on)
• Their Possible Impact on Cross Platform Development
![Page 77: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/77.jpg)
Current Implementation Status of Noah
![Page 78: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/78.jpg)
Current Implementation Status of Noah
• Still in development
• Currently capable of running
• apt-get, pacman (Not all subcommands are supported yet)
• vim, gcc, make
• Ruby
• Binutils, ls, cat, …
• X applications; xeyes, xclock, xfwrite, doom 3, …
• sudo, curl, nc, man, …
• The most easiest way to build Linux kernel on macOS is to use Noah!
![Page 79: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/79.jpg)
Performance
• Performance data will be made public in the presentation since it contains unpublished materials
![Page 80: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/80.jpg)
Related Technologies
![Page 81: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/81.jpg)
Linux Compatibility Layers
• OS Built-in
• Windows Subsystem for Linux
• FreeBSD’s Linuxulator
• Third Party Middleware
• Foreign LINUX
![Page 82: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/82.jpg)
Windows Subsystem for Linux (WSL)
• Built-in Linux compatibility layer for Windows 10 by Microsoft
• Picoprocess contains a Linux ELF binary. Kernel drivers (LXCore and LXSS) handle system calls from it in kernel mode
Quoted from https://blogs.msdn.microsoft.com/wsl/2016/04/22/windows-subsystem-for-linux-overview/
![Page 83: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/83.jpg)
Linuxulator
• Built-in Linux compatibility layer for FreeBSD
• FreeBSD has a loader for Linux ELF and implementations of Linux system calls
• Similar approach to WSL’s LXSS / LXCore (Linuxulator is older, though)
![Page 84: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/84.jpg)
Foreign Linux
• https://github.com/wishstudio/flinux
• Run unmodified Linux applications on Windows by dynamic binary translation and system call emulation
• It seems that the overhead of dynamic binary translation is a bit heavy, however…
![Page 85: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/85.jpg)
ComparisonOS Binary
Compatibility Portable No Kernel Modification
Smooth Interaction
Reasonable Overhead
Noah macOS ✔ ✔ ✔ ✔ ✔
WSL Windows ✔ ✗ ✗ ✔ ✔
Linuxulator FreeBSD ✔ ✗ ✗ ✔ ✔
Foreign LINUX Windows ✔ ✔ ✔ ✔ ✗
Full VM Any ✔ ✔ ✔ ✗ ✔*with processor hardware
virtualization
![Page 86: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/86.jpg)
Future Cross Platform Development
![Page 87: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/87.jpg)
Now major four operating systems have Linux compatibility
![Page 88: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/88.jpg)
• Linux could be regarded as “standard” like POSIX
• In the future, once you write a Linux application, it simply could run on any platforms
• Noah could help it!
![Page 89: Noah Hypervisor-Based Darwin Subsystem for Linux-4-3...Noah • A middleware that runs unmodified Linux ELF applications on macOS • Reduce cost of waiting / creating macOS port](https://reader034.fdocuments.us/reader034/viewer/2022051804/5fee659f89d8dc67d06d6a13/html5/thumbnails/89.jpg)
Summary• We introduced Noah, which is a middleware that
runs unmodified Linux ELF applications on macOS
• Noah adopts a new hypervisor based approach with many merits
• Now four major operating systems have Linux compatibility. In the future, your Linux applications could run anywhere!