Stefano Stabellini @stabellinist
Xen on ARM,
and the Art of Embedded Virtualization
Security, Isolation, Partitioning
Why Xen?
Why an hypervisor?
Galois
SMACCMPPilot
Demo
Xen Summit 2014
Why Xen?
• Efficiency and Consolidation
• Isolation and Partitioning
• Componentization
• Resilience
• Scaling
• Portability
Xen: a type-1 hypervisor
Hardware
Xen
Dom0 DomU
HW drivers
PV backends
PV
Frontends
DomU
PV
Frontends
DomU
PV
Frontends
Xen: the gears of the cloud
• Large user base (> 10M individual users)
• Powers the largest clouds in production
• Not just servers
Xen: Open Source
Xen: Open Source
partial
Embedded != Cloud
Different requirements:
• short boot times
• small footprint
• small codebase (certifications)
• non-PCI device assignment
• driver domains
• low, deterministic irq latency
• real time schedulers
• co-processor virtualization
Xen on ARM
• A lean and simple architecture
– No cruft
– No emulation, No QEMU
– Small attack surface
– One type of guest
• Exploit the hardware as much as possible
• A very good match for the hardware
• Clean architecture = a very small code base
– Xen, ARM and ARM64 =~ 30K LOC
Xen on ARM: a perfect match for the HW
Xen on ARM: unique features
• Device Passthrough (even Non-Discoverable Devices)
– iomem and irqs VM config parameters
• No guest firmware by default - fast VM boot
• Certifications efforts ongoing
• Low, Deterministic IRQ latency (WARM_MAX < 2000ns)
Low IRQ latency: no maintenance interrupts
DomU
Xen
irq 109
virq 109
DomU
Xen
EOI
DomU
Xen
Maintenance
interrupt
GICH_LR
Write
GICH_LR
Clear
Low IRQ latency: physical follow virtual
vcpu0 vcpu1
pcpu0 pcpu1
irq 109
virq 109
Low IRQ latency: physical follow virtual
vcpu0 vcpu1
pcpu0 pcpu1
irq 109
virq 109
Low IRQ latency: physical follow virtual
vcpu0 vcpu1
pcpu0 pcpu1
irq 109
virq 109
Xen Schedulers
CPU CPU CPU CPU
CPU CPU CPU CPU
Xen Schedulers
CPU CPU CPU CPU
CPU CPU CPU CPU
Real Time Scheduler
ARINC 653
Regular VM Scheduler
Credit
Dedicated
to 1 VCPU
Dedicated
to 1 VCPU
Memory Introspection
PV Protocols
Existing: net, block, console, keyboard, mouse, framebuffer,
XenGT
New: 9pfs, PVCalls, Multi Touch, Sound, Display
Driver Domains
Hardware
Xen
Dom0 DomU
NetFront
Disk Driver
Domain
Toolstack Disk Driver
BlockBack
Network
Driver
Domain
Network
Driver
NetBack BlockFront
Automotive
Hardware
Xen
Dom0
Linux Control Domain
UI Domain
Automotive Grade Android
HW Drivers
GPU
Driver
PV Block & Net
frontends
PV Block & Net
Backends
Audio
Driver
GlobalLogic
EPAMEPAM
EPAM: DEMO
https://www.youtube.com/watch?v=jMmz1odBZb8
Xilinx Zynq MPSoC
Xen
Dom0
Linux
Baremetal
App
Toolstack FPGA Driver
Baremetal
App
FPGA Driver
Baremetal
App
FPGA Driver
Baremetal
App
FPGA Driver
FPGA
Dedicated CPU Dedicated CPU Dedicated CPU Dedicated CPU
Xen: best security process in the industry
• A very transparent process
• Responsible disclosure
• Few security issues for Xen on ARM
• Xen stable trees maintained for security for 3 years
Release process
• 6 month release
– December
– June
• Xen 4.8 released on the 5th of December 2016
• Xen 4.9 planned for the 2nd of June 2017
Xen on ARM: what’s next
● Guest creation directly from Xen at boot via Device Tree
● Dynamic Memory Map
● Setup VM-to-VM communication channels from VM
config
More resources
• Port Xen to a new SOC: https://goo.gl/384aD8
• Add Xen support Xen to your OS: https://goo.gl/3qgqcM
• Xen on ARM whitepaper: https://goo.gl/TcuqXd
• Xen on ARM wiki: https://goo.gl/9qsfMf
• Device Passthrough presentation: https://goo.gl/KM0f8c
• OE meta-virtualization Xen recipe:
https://goo.gl/m7GuXR
• OpenXT (Xen + OpenEmbedded): http://openxt.org
• Biweekly ARM Community Call: https://goo.gl/8ULYRn
Please engage!
• Xen devel ML: xen-devel@lists.xenproject.org
• Xen user ML: xen-users@lists.xenproject.org
• IRC on freenode: #xenarm or #xen-devel
Fin

Xen and the art of embedded virtualization (ELC 2017)

  • 1.
    Stefano Stabellini @stabellinist Xenon ARM, and the Art of Embedded Virtualization Security, Isolation, Partitioning
  • 2.
    Why Xen? Why anhypervisor?
  • 4.
  • 5.
    Why Xen? • Efficiencyand Consolidation • Isolation and Partitioning • Componentization • Resilience • Scaling • Portability
  • 6.
    Xen: a type-1hypervisor Hardware Xen Dom0 DomU HW drivers PV backends PV Frontends DomU PV Frontends DomU PV Frontends
  • 7.
    Xen: the gearsof the cloud • Large user base (> 10M individual users) • Powers the largest clouds in production • Not just servers
  • 8.
  • 9.
  • 10.
    Embedded != Cloud Differentrequirements: • short boot times • small footprint • small codebase (certifications) • non-PCI device assignment • driver domains • low, deterministic irq latency • real time schedulers • co-processor virtualization
  • 11.
    Xen on ARM •A lean and simple architecture – No cruft – No emulation, No QEMU – Small attack surface – One type of guest • Exploit the hardware as much as possible • A very good match for the hardware • Clean architecture = a very small code base – Xen, ARM and ARM64 =~ 30K LOC
  • 12.
    Xen on ARM:a perfect match for the HW
  • 13.
    Xen on ARM:unique features • Device Passthrough (even Non-Discoverable Devices) – iomem and irqs VM config parameters • No guest firmware by default - fast VM boot • Certifications efforts ongoing • Low, Deterministic IRQ latency (WARM_MAX < 2000ns)
  • 14.
    Low IRQ latency:no maintenance interrupts DomU Xen irq 109 virq 109 DomU Xen EOI DomU Xen Maintenance interrupt GICH_LR Write GICH_LR Clear
  • 15.
    Low IRQ latency:physical follow virtual vcpu0 vcpu1 pcpu0 pcpu1 irq 109 virq 109
  • 16.
    Low IRQ latency:physical follow virtual vcpu0 vcpu1 pcpu0 pcpu1 irq 109 virq 109
  • 17.
    Low IRQ latency:physical follow virtual vcpu0 vcpu1 pcpu0 pcpu1 irq 109 virq 109
  • 18.
    Xen Schedulers CPU CPUCPU CPU CPU CPU CPU CPU
  • 19.
    Xen Schedulers CPU CPUCPU CPU CPU CPU CPU CPU Real Time Scheduler ARINC 653 Regular VM Scheduler Credit Dedicated to 1 VCPU Dedicated to 1 VCPU
  • 20.
  • 21.
    PV Protocols Existing: net,block, console, keyboard, mouse, framebuffer, XenGT New: 9pfs, PVCalls, Multi Touch, Sound, Display
  • 22.
    Driver Domains Hardware Xen Dom0 DomU NetFront DiskDriver Domain Toolstack Disk Driver BlockBack Network Driver Domain Network Driver NetBack BlockFront
  • 23.
    Automotive Hardware Xen Dom0 Linux Control Domain UIDomain Automotive Grade Android HW Drivers GPU Driver PV Block & Net frontends PV Block & Net Backends Audio Driver
  • 24.
  • 25.
  • 26.
  • 27.
    Xilinx Zynq MPSoC Xen Dom0 Linux Baremetal App ToolstackFPGA Driver Baremetal App FPGA Driver Baremetal App FPGA Driver Baremetal App FPGA Driver FPGA Dedicated CPU Dedicated CPU Dedicated CPU Dedicated CPU
  • 28.
    Xen: best securityprocess in the industry • A very transparent process • Responsible disclosure • Few security issues for Xen on ARM • Xen stable trees maintained for security for 3 years
  • 29.
    Release process • 6month release – December – June • Xen 4.8 released on the 5th of December 2016 • Xen 4.9 planned for the 2nd of June 2017
  • 30.
    Xen on ARM:what’s next ● Guest creation directly from Xen at boot via Device Tree ● Dynamic Memory Map ● Setup VM-to-VM communication channels from VM config
  • 31.
    More resources • PortXen to a new SOC: https://goo.gl/384aD8 • Add Xen support Xen to your OS: https://goo.gl/3qgqcM • Xen on ARM whitepaper: https://goo.gl/TcuqXd • Xen on ARM wiki: https://goo.gl/9qsfMf • Device Passthrough presentation: https://goo.gl/KM0f8c • OE meta-virtualization Xen recipe: https://goo.gl/m7GuXR • OpenXT (Xen + OpenEmbedded): http://openxt.org • Biweekly ARM Community Call: https://goo.gl/8ULYRn
  • 32.
    Please engage! • Xendevel ML: xen-devel@lists.xenproject.org • Xen user ML: xen-users@lists.xenproject.org • IRC on freenode: #xenarm or #xen-devel
  • 33.