OSb: OSv on BitVisor
75
OSb: OSv on BitVisor @ysmoo Saturday, October 18, 14
-
Upload
university-of-tsukuba -
Category
Engineering
-
view
392 -
download
2
description
An attempt to run OSv on BitVisor.
Transcript of OSb: OSv on BitVisor
OSb: OSv on BitVisor@ysmoo
Saturday, October 18, 14
BitVisor
BitVisor
Hardware
OS
App App App
http://www.justis.as-1.co.jp
Saturday, October 18, 14
OSv
VMM
Hardware
OSvApp
http://medical-care.feed.jp
Saturday, October 18, 14
BitVisor
Saturday, October 18, 14
BitVisor OSv
Saturday, October 18, 14
BitVisor OSv
http://www.root.ne.jp/nishide/shs/
Saturday, October 18, 14
OSv on BitVisor
BitVisor
Hardware
OSvApp
Saturday, October 18, 14
OSv on BitVisor
BitVisor
Hardware
OSvApp
OSbSaturday, October 18, 14
OSv on BitVisor
BitVisor
Hardware
OSvApp
OSb
What the fuckno Virtio !
Physical interfacesucks !
What the hell isPRO/1000 !
Saturday, October 18, 14
OSv on BitVisor
BitVisor
Hardware
OSvApp
OSb
What the fuckno Virtio !
Physical interfacesucks !
What the hell isPRO/1000 !
You have no choice.
Virtio
Saturday, October 18, 14
~ The road to OSb ~
Saturday, October 18, 14
• Boot process
• Some drivers
OSv code reading
Saturday, October 18, 14
• Boot process
• Some drivers
OSv code reading- MBR- Read local disk with INT13/42.- Load command line, boot loader, OSv kernel.- Get memory map with INT15/E820.- Setup segment descriptors/page tables.- Switch to 64-bit mode.- premain()- main()...
Saturday, October 18, 14
• Boot process
• Some drivers
OSv code reading- MBR- Read local disk with INT13/42.- Load command line, boot loader, OSv kernel.- Get memory map with INT15/E820.- Setup segment descriptors/page tables.- Switch to 64-bit mode.- premain()- main()...
- Serial / VGA output.- SATA.- Virtio NIC/BLK/RNG/SCSI.- ACPI.- APIC....
Saturday, October 18, 14
• Boot process
• Some drivers
OSv code reading- MBR- Read local disk with INT13/42.- Load command line, boot loader, OSv kernel.- Get memory map with INT15/E820.- Setup segment descriptors/page tables.- Switch to 64-bit mode.- premain()- main()...
- Serial / VGA output.- SATA.- Virtio NIC/BLK/RNG/SCSI.- ACPI.- APIC....
Straightforward...
Saturday, October 18, 14
Booting OSb
./scripts/run.py
Saturday, October 18, 14
Booting OSb
./scripts/run.py
VMM
OSv
In the case of normal OSv...
Saturday, October 18, 14
Booting OSb
Hardware
./scripts/run.py
Saturday, October 18, 14
Booting OSb
Hardware
./scripts/run.py
ImageUpdate
Saturday, October 18, 14
Booting OSb
Hardware
./scripts/run.py
ImageUpdate
IPMI
Saturday, October 18, 14
Booting OSb
Hardware
./scripts/run.py
ImageUpdate
IPMIPXE Boot
Saturday, October 18, 14
Booting OSb
Hardware
./scripts/run.py
ImageUpdate
IPMIPXE Boot
BitVisor
Saturday, October 18, 14
Booting OSb
Hardware
./scripts/run.py
ImageUpdate
IPMIPXE Boot
BitVisor
OSvNetwork Boot
Saturday, October 18, 14
Booting OSb
Hardware
./scripts/run.py
ImageUpdate
IPMIPXE Boot
BitVisor
OSvNetwork Boot
App
Saturday, October 18, 14
Booting OSb
BitVisor
Hardware
OSvApp
Network Boot
IPMI
./scripts/run.py
TTY
ImageUpdate
PXE Boot
Saturday, October 18, 14
ACPI workaround
drivers/acpi.cc:
// Copy the root table list to dynamic memory if (!is_bitvisor()) { status = AcpiReallocateRootTable(); if (ACPI_FAILURE(status)) { acpi_e("AcpiReallocateRootTable failed: %s\n", AcpiFormatException(status)); return; } }
Saturday, October 18, 14
ACPI workaround
drivers/acpi.cc:
// Copy the root table list to dynamic memory if (!is_bitvisor()) { status = AcpiReallocateRootTable(); if (ACPI_FAILURE(status)) { acpi_e("AcpiReallocateRootTable failed: %s\n", AcpiFormatException(status)); return; } }
Triple fault
Saturday, October 18, 14
ACPI workaround
drivers/acpi.cc:
// Copy the root table list to dynamic memory if (!is_bitvisor()) { status = AcpiReallocateRootTable(); if (ACPI_FAILURE(status)) { acpi_e("AcpiReallocateRootTable failed: %s\n", AcpiFormatException(status)); return; } }
Triple faultSkip in case of OSb
Saturday, October 18, 14
ACPI workaround
drivers/acpi.cc:
// Copy the root table list to dynamic memory if (!is_bitvisor()) { status = AcpiReallocateRootTable(); if (ACPI_FAILURE(status)) { acpi_e("AcpiReallocateRootTable failed: %s\n", AcpiFormatException(status)); return; } }
Triple faultSkip in case of OSb
Machine-specific...?
Saturday, October 18, 14
Hello world !
Saturday, October 18, 14
Performance?
20.75& 21.20&
0.00&
5.00&
10.00&
15.00&
20.00&
25.00&
OSb& OSv&
Elap
sed(Time((sec)�
for((12(billion(6mes)(val++;�
Saturday, October 18, 14
Virtio NIC
BitVisor
Hardware
OSvApp Actually PRO/1000...
Saturday, October 18, 14
Virtio NIC
BitVisor
Hardware
OSvApp
Virtio NIC
Actually PRO/1000...
Fake Virtio NIC...
Saturday, October 18, 14
Virtio NIC
BitVisor
Hardware
OSvApp
Virtio NIC
Actually PRO/1000...
Fake Virtio NIC...
Hook PCI config access, MMIO, PIO and fake!Saturday, October 18, 14
FakingPCI configuration space
• Faking IDs
• Faking BARs
• Faking capabilities
Saturday, October 18, 14
Faking IDs 0x1AF4(Virtio Device)
Saturday, October 18, 14
Faking IDs 0x1AF4(Virtio Device)
0x1000(Virtio NIC)
Saturday, October 18, 14
Faking IDs 0x1AF4(Virtio Device)
0x1000(Virtio NIC)
0 (Legacy Virtio)
Saturday, October 18, 14
Faking IDs 0x1AF4(Virtio Device)
0x1000(Virtio NIC)
0 (Legacy Virtio)
1 (Virtio NIC)
Saturday, October 18, 14
Faking BARsBAR0: IO spacefor Virtio
BAR1: MMIO spacefor MSI-X
Saturday, October 18, 14
Faking BARsBAR0: IO spacefor Virtio
BAR1: MMIO spacefor MSI-X
= PRO/1000 IO space
Saturday, October 18, 14
Faking BARsBAR0: IO spacefor Virtio
BAR1: MMIO spacefor MSI-X
= PRO/1000 IO space
= PRO/1000 MMIO space
Saturday, October 18, 14
Faking capabilities
Add MSI-Xcapability
Saturday, October 18, 14
Faking capabilities
Add MSI-Xcapability Support
3 vectors
Saturday, October 18, 14
Faking capabilities
Add MSI-Xcapability Support
3 vectors
Vector 0:Reception
Saturday, October 18, 14
Faking capabilities
Add MSI-Xcapability Support
3 vectors
Vector 0:Reception
Vector 1:Transmission
Saturday, October 18, 14
Faking capabilities
Add MSI-Xcapability Support
3 vectors
Vector 0:Reception
Vector 1:Transmission
Vector 2:Control
Saturday, October 18, 14
Virtio NIC Operations
• Virtio ring
• Packet transmission
• Packet reception
Saturday, October 18, 14
Virtio ringControlled bythese registers
BAR0(IO)
Saturday, October 18, 14
Virtio ringControlled bythese registers
3 ringsavailable
BAR0(IO)
Saturday, October 18, 14
Virtio ringControlled bythese registers
3 ringsavailable
Ring 0: ReceptionBAR0(IO)
Saturday, October 18, 14
Virtio ringControlled bythese registers
3 ringsavailable
Ring 0: Reception
Ring 1: Transmission
BAR0(IO)
Saturday, October 18, 14
Virtio ringControlled bythese registers
3 ringsavailable
Ring 0: Reception
Ring 1: Transmission
Ring 2: Control
BAR0(IO)
Saturday, October 18, 14
Virtio ringControlled bythese registers
3 ringsavailable
Ring 0: Reception
Ring 1: Transmission
Ring 2: Control
BAR0(IO)
(Memory)
Saturday, October 18, 14
Virtio ringControlled bythese registers
3 ringsavailable
Ring 0: Reception
Ring 1: Transmission
Ring 2: Control
BAR0(IO)
(Memory)Driver 2 Device
Device 2 Driver
Saturday, October 18, 14
Packet transmission
OSv(Driver)
BitVisor(Device)
PRO/1000
Saturday, October 18, 14
Packet transmission
OSv(Driver)
BitVisor(Device)
Req.
PRO/1000
Req.Req.
Saturday, October 18, 14
Packet transmission
OSv(Driver)
BitVisor(Device)
Req.
PRO/1000
Req.Req.
Req.Req.Req.
Saturday, October 18, 14
Packet transmission
OSv(Driver)
BitVisor(Device)
Req.
PRO/1000
Req.
TX Ring
Req.
Req. Req. Req.
Req.Req.Req.
Saturday, October 18, 14
Packet transmission
OSv(Driver)
BitVisor(Device)
Req.
PRO/1000
Req.
TX Ring
Req.
Req. Req. Req.
Req.Req.Req.
Req.
Saturday, October 18, 14
Packet transmission
OSv(Driver)
BitVisor(Device)
Req.
PRO/1000
Req.
TX Ring
Req.
Req. Req. Req.
Req.Req.Req.
Req.
Notify!(PIO)
Saturday, October 18, 14
Packet transmission
OSv(Driver)
BitVisor(Device)
Req.
PRO/1000
Req.
TX Ring
Req.
Req. Req. Req.
Req.Req.Req.
Req.
Notify!(PIO)
Req.
Req.
Saturday, October 18, 14
Packet transmission
OSv(Driver)
BitVisor(Device)
Req.
PRO/1000
Req.
TX Ring
Req.
Req. Req. Req.
Req.Req.Req.
Req.
Notify!(PIO)
Req.
Req.
Saturday, October 18, 14
Packet transmission
OSv(Driver)
BitVisor(Device)
Req.
PRO/1000
Req.
TX Ring
Req.
Req. Req. Req.
Req.Req.Req.
Req.
Notify!(PIO)
Req.
Req.
Reuse
Saturday, October 18, 14
Packet reception
OSv(Driver)
BitVisor(Device)
Buf.
PRO/1000
Buf.
RX Ring
Buf.
Buf. Buf. Buf.
Buf.Buf.Buf.
Saturday, October 18, 14
Packet reception
OSv(Driver)
BitVisor(Device)
Buf.
PRO/1000
Buf.
RX Ring
Buf.
Buf. Buf. Buf.
Buf.Buf.Buf.
Buf.
Saturday, October 18, 14
Packet reception
OSv(Driver)
BitVisor(Device)
Buf.
PRO/1000
Buf.
RX Ring
Buf.
Buf. Buf. Buf.
Buf.Buf.Buf.
Buf.
Buf.
Saturday, October 18, 14
Packet reception
OSv(Driver)
BitVisor(Device)
Buf.
PRO/1000
Buf.
RX Ring
Buf.
Buf. Buf. Buf.
Buf.Buf.Buf.
Buf.
Buf.
Buf.
Saturday, October 18, 14
Packet reception
OSv(Driver)
BitVisor(Device)
Buf.
PRO/1000
Buf.
RX Ring
Buf.
Buf. Buf. Buf.
Buf.Buf.Buf.
Buf.
Buf.
Buf.
Saturday, October 18, 14
Packet reception
OSv(Driver)
BitVisor(Device)
Buf.
PRO/1000
Buf.
RX Ring
Buf.
Buf. Buf. Buf.
Buf.Buf.Buf.
Buf.
Buf.
Buf.
Int!
Saturday, October 18, 14
Packet reception
OSv(Driver)
BitVisor(Device)
Buf.
PRO/1000
Buf.
RX Ring
Buf.
Buf. Buf. Buf.
Buf.Buf.Buf.
Buf.
Int!
Buf.
Buf.
Receive
Saturday, October 18, 14
InterruptBAR1
(MMIO)MSI-X Message Data:Int. vector number (e.g. 0x26)
VMX event injectionfor virtual interrupts
Saturday, October 18, 14
DHCP succeeded !
Saturday, October 18, 14
Summary &future work
• Summary
• Booting OSv on BitVisor
• Virtio NIC
• Future work
• Further testing
• Virtio BLK/RNG/...
Saturday, October 18, 14
Thank you !
http://www.root.ne.jp/nishide/shs/
Saturday, October 18, 14
