Following on from last week’s introduction to the Zynq UltraScale+ MPSoC, this tutorial takes a look at how you can get started with using Xen Hypervisor on Zynq UltraScale+ MPSoCs.
This tutorial will show you how to use the Xen Hypervisor (HV) on Xilinx’s Zynq UltraScale+ MPSoC.
Since the Zynq UltraScale+ MPSoC is not yet widely available, this tutorial leverages the emulation capabilities of QEMU, which is shipped with Xilinx PetaLinux tools. We will be showing you how to run the Xen Hypervisor on the ZCU102 development platform using a PetaLinux-built HV and a Linux Dom0.
Background Information
Xilinx PetaLinux Tools
The Xilinx PetaLinux tools simplify build, configuration, and deployment steps enabling designers to concentrate efforts on application and platform development instead of building and deployment. Xilinx and partners provide board support packages (BSP) for select boards with build configurations and pre-built images.
PetaLinux is our full Linux distribution which includes the Linux OS as well as a complete configuration, build and deploy environment for Xilinx silicon. Because PetaLinux is fully featured, integrated, tested, and documented, it is the recommended and supported Linux deployment mechanism for our customers. As a Xilinx product, PetaLinux enjoys the same level of product management, support, bug tracking and emphasis that other Xilinx tools receive.
Source: Xilinx Wiki
Xen Hypervisor (HV)
The Zynq UltraScale+ MPSoC contains an ARM Cortex-A53 core. The ARM Cortex-A53 comes with built-in hardware hypervisor support; i.e., EL2 and hypervisor instructions. Xen HV is a type 1 HV, running directly on the hardware. Xen HV runs independent domains on top of it, referred to as Dom0 for the host domain and DomUs for guest domains. Dom0 controls and launches guest domains. Dom0 contains applications that speak directly to the HV to launch, destroy, check for status, etc.
The Xen Project hypervisor is an open-source type-1 or baremetal hypervisor, which makes it possible to run many instances of an operating system or indeed different operating systems in parallel on a single machine (or host).
Source: Xen Project Wiki
Instructions
Requirements
- Xilinx PetaLinux tools 2015.4 running on Linux
- Latest revision of the ZCU102 PetaLinux 2015.4 BSP
Create the Project
Use the BSP to create a new project.
- Create the project using the following command:
petalinux-create -t project -s Xilinx-ZCU102-v2015.4-final-dec.bsp.
NB : The project is already set up to run on Xen, to build the hypervisor, and build the hypervisor control applications in Dom0. For specific settings, such as the kernel configuration, consult the Xilinx wiki on Xen.
Modify the DTS
Replace the default device tree file (.dts) with a template file for running Xen and then change it to downgrade the driver used for the UART.
- Delete (or rename ‘system-top.dts’)
subsystems/linux/configs/device-tree/system-top.dts.
- Rename
subsystems/linux/configs/device-tree/xen_qemu.dts
, tosystem-top.dts.
- In this new
system-top.dts
file, add the following to the end of the file:NB : Xen doesn’t support Cadence’s UART IP version r1p12, only version r1p8 as observed in the Xen source code.
Observe: In subsystems/linux/configs/device-tree/xen-overlay.dtsi
there is a block labelled chosen
with two lines containing the keyword bootargs
.
The first line specifies boot arguments for the Xen HV : use serial0
(an alias for uart0
, see system-conf.dtsi
) as console, allocates 512MB for dom0, etc. For additional arguments, consult the complete list of Xen command line options.
The second line specifies Linux boot arguments.
Build the Project
Use the PetaLinux tools to build the new image.
- Run the following command to build the system:
petalinux-build
- Once the build is over, a number of files are created in the
images/linux/
directory. These files include:- The ARM Trusted Firmware (ATF): bootloader firmware which initializes the power-on exception level (EL3)
- U-Boot, the Second Stage Boot Loader (SSBL): loads the operating system into memory
xen.ub
: a U-Boot executable version of the Xen HypervisorImage
: the Linux binary (kernel and RootFS)system.dtb
: the device blob used by Xen- A number of other files exist including the Linux kernel and RootFS in other formats
Boot the Image
- Start U-Boot with Xilinx PetaLinux.
The PetaLinux command requires some extra QEMU arguments due to the Ethernet configuration of the ZCU102 (Ethernet is needed as you are booting from tftp). The extra -net arguments allow QEMU’s DHCP server to assign an address to the U-Boot instance.
petalinux-boot --qemu --u-boot --qemu-args="-tftp images/linux -net nic -net nic -net nic -net nic -net user"
Once the U-Boot has started, press any key to stop auto-boot and drop down to a command prompt. - Load the Xen HV image, Dom0 image, and device tree blob into memory using tftp.
- Configure tftp’s server address with the command
setenv serverip 10.0.2.2.
- Load images via tftp
tftpb 0x4000000 system.dtb && tftpb 0x80000 Image && tftpb 0x6000000 xen.ub.
- This loads the device-tree blob (DTB) at address 0x4000000, the Xen image at address 0x6000000 , and the Linux image at address 0x80000.
- Configure tftp’s server address with the command
- Boot the Xen HV using the command
bootm 0x6000000 - 0x4000000.
This tells U-Boot to boot the application at memory address 0x6000000 (xen.ub
) with no initial ramdisk (denoted by the -
), the third argument is the flattened device tree (system.dtb
) at address 0x4000000.
The Xen HV will bring up Dom0 on its own. Once Linux has loaded there will be the standard login prompt to log into Linux. Log in as ‘root’ (password: root). You can use the Xen pre-installed tools to check the status of the HV system (xl).
xl list
will list the running domains, their IDs, available memory, number of virtual CPUs available, and their state.
Load a Guest OS using Xen
Now that the host domain (Dom0) is running, it can load guest domains which will run on their own, isolated thanks to the Xen HV. Many different guests can run from full-fledged Linux to Xen’s MiniOS.
This example will load the same Dom0 Linux image as a guest OS. A Xen configuration file must be created for it.
Create the DomU Configuration File
This step will create a configuration file on the host computer. It will be transferred via tftp to the running QEMU session.
- Create a new file called
<PetaLinux_project_root>/images/linux/DomU_Linux.cfg.
- Open the file in a text editor.
- Place the following in the file (modified from the Xen boilerplate configuration file):
# ===================================================================== # Example PV Linux guest configuration # ===================================================================== # # This is a fairly minimal example of what is required for a # Paravirtualised Linux guest. For a more complete guide see xl.cfg(5) # Guest name name = "DomU_Linux" # Kernel image to boot kernel = "/home/root/Image" # Initial memory allocation (MB) memory = 512 # Number of VCPUS vcpus = 1 extra="console=hvc0"
- Save the file.
NB : The extra="console=hvc0"
enables the user to pass input to the guest OS through the console.
Load the Guest OS
Return to the QEMU terminal, now logged into Linux. First, transfer the Linux image and the configuration files to the embedded Linux system (running on QEMU) via tftp. Then use the Xen tools to create and run the guest OS.
- Go to the root user’s home directory:
cd /home/root
- Transfer the Linux image over:
tftp -g -r Image 10.0.2.2
- Transfer the configuration file over:
tftp -g -r DomU_Linux.cfg 10.0.2.2
- Create the new domain:
xl create -c DomU_Linux.cfg
The guest domain should now be outputting messages to the console and a login prompt should appear. Log in as root again.
Both the Host domain and Guest domain are right now indistinguishable (they have the same bash prompt). However, running xl list
in the guest domain will return an error, whereas running it in the host domain will now display two running domains; i.e., the host and the newly created guest domain.
To test this, return to the host domain’s console by pressing CTRL+]. To return to the guest OS’s console (from the host OS), run xl console <domainName or domainID>
.
To remove a domain you can run xl shutdown <domainName or domainID>
to forward a shutdown command or xl destroy <domainName or domainID>
to forcefully remove a domain. Xilinx PetaLinux builds the required output products while the BSP supplies Xen specific configurations and utilities.
Summary
The Xilinx-provided BSP and PetaLinux tools can be used to generate everything needed to run and manage Xen on the Zynq UltraScale+ MPSoC.
Further Information
- Zynq UltraScale+ MPSoC for the Software Developer Training
- Embedded Design with Xilinx PetaLinux Tools Training
- Zynq UltraScale+ MPSoC Technical Reference Manual
- Zynq UltraScale+ MPSoC Product Brief
- Zynq UltraScale+ MPSoC White Paper
- UltraScale Architecture and Product Overview
Just getting started with using Zynq UltraScale+ MPSoCs and need additional support? Contact us to find out how Zynq UltraScale+ MPSoC training can help get you up and running!