Back to
Blog

How to Use Xilinx PetaLinux Tools to Deploy Xen Hypervisor & Linux on the Zynq UltraScale+ MPSoC

Following on from last week’s introduction to the Zynq UltraScale+ MPSoC, this tutorial takes a look at how you can get started with using Xen Hypervisor on Zynq UltraScale+ MPSoCs.

This tutorial will show you how to use the Xen Hypervisor (HV) on Xilinx’s  Zynq UltraScale+ MPSoC.
Since the Zynq UltraScale+ MPSoC is not yet widely available, this tutorial leverages the emulation capabilities of QEMU, which is shipped with Xilinx PetaLinux tools. We will be showing you how to run the Xen Hypervisor on the ZCU102 development platform using a PetaLinux-built HV and a Linux Dom0.

Background Information

Xilinx PetaLinux Tools

The Xilinx PetaLinux tools simplify build, configuration, and deployment steps enabling designers to concentrate efforts on application and platform development instead of building and deployment. Xilinx and partners provide board support packages (BSP) for select boards with build configurations and pre-built images.

PetaLinux is our full Linux distribution which includes the Linux OS as well as a complete configuration, build and deploy environment for Xilinx silicon. Because PetaLinux is fully featured, integrated, tested, and documented, it is the recommended and supported Linux deployment mechanism for our customers. As a Xilinx product, PetaLinux enjoys the same level of product management, support, bug tracking and emphasis that other Xilinx tools receive.
Source: Xilinx Wiki

Xen Hypervisor (HV)

The Zynq UltraScale+ MPSoC contains an ARM Cortex-A53 core. The ARM Cortex-A53 comes with built-in hardware hypervisor support; i.e., EL2 and hypervisor instructions. Xen HV is a type 1 HV, running directly on the hardware. Xen HV runs independent domains on top of it, referred to as Dom0 for the host domain and DomUs for guest domains. Dom0 controls and launches guest domains. Dom0 contains applications that speak directly to the HV to launch, destroy, check for status, etc.

The Xen Project hypervisor is an open-source type-1 or baremetal hypervisor, which makes it possible to run many instances of an operating system or indeed different operating systems in parallel on a single machine (or host).
Source: Xen Project Wiki

Instructions

Requirements

  • Xilinx PetaLinux tools 2015.4 running on Linux
  • Latest revision of the ZCU102 PetaLinux 2015.4 BSP

Create the Project

Use the BSP to create a new project.

  1. Create the project using the following command:
    petalinux-create -t project -s Xilinx-ZCU102-v2015.4-final-dec.bsp.
    NB : The project is already set up to run on Xen, to build the hypervisor, and build the hypervisor control applications in Dom0. For specific settings, such as the kernel configuration, consult the Xilinx wiki on Xen.

Modify the DTS

Replace the default device tree  file (.dts) with a template file for running Xen and then change it to downgrade the driver used for the UART.

  1. Delete (or rename ‘system-top.dts’) subsystems/linux/configs/device-tree/system-top.dts.
  2. Rename subsystems/linux/configs/device-tree/xen_qemu.dts, to system-top.dts.
  3. In this new system-top.dts file, add the following to the end of the file: 
    &uart0 {
    compatible = "cdns,uart-r1p8";
    };
    &uart1 {
    compatible = "cdns,uart-r1p8";
    };

    NB : Xen doesn’t support Cadence’s UART IP version r1p12, only version r1p8 as observed in the Xen source code.

Observe: In subsystems/linux/configs/device-tree/xen-overlay.dtsi there is a block labelled chosen with two lines containing the keyword bootargs.

xen,xen-bootargs = "console=dtuart dtuart=serial0 dom0_mem=512M bootscrub=0 maxcpus=1 timer_slop=0";

xen,dom0-bootargs = "console=hvc0 earlycon=xen earlyprintk=xen maxcpus=1";

The first line specifies boot arguments for the Xen HV : use serial0 (an alias for uart0, see system-conf.dtsi) as console, allocates 512MB for dom0, etc. For additional arguments, consult the complete list of Xen command line options.

The second line specifies Linux boot arguments.

Build the Project

Use the PetaLinux tools to build the new image.

  1. Run the following command to build the system:petalinux-build
  2. Once the build is over, a number of files are created in the images/linux/ directory. These files include:
    • The ARM Trusted Firmware (ATF): bootloader firmware which initializes the power-on exception level (EL3)
    • U-Boot, the Second Stage Boot Loader (SSBL): loads the operating system into memory
    • xen.ub: a U-Boot executable version of the Xen Hypervisor
    • Image: the Linux binary (kernel and RootFS)
    • system.dtb: the device blob used by Xen
    • A number of other files exist including the Linux kernel and RootFS in other formats

Boot the Image

  1. Start U-Boot with Xilinx PetaLinux.
    The PetaLinux command requires some extra QEMU arguments due to the Ethernet configuration of the ZCU102 (Ethernet is needed as you are booting from tftp). The extra -net arguments allow QEMU’s DHCP server to assign an address to the U-Boot instance.
    petalinux-boot --qemu --u-boot --qemu-args="-tftp images/linux -net nic -net nic -net nic -net nic -net user"
    Once the U-Boot has started, press any key to stop auto-boot and drop down to a command prompt.
  2. Load the Xen HV image, Dom0 image, and device tree blob into memory using tftp.
    • Configure tftp’s server address with the command  setenv serverip 10.0.2.2.
    • Load images via tftp tftpb 0x4000000 system.dtb && tftpb 0x80000 Image && tftpb 0x6000000 xen.ub.
    • This loads the device-tree blob (DTB) at address 0x4000000, the Xen image at address 0x6000000 , and the Linux image at address 0x80000.
  3. Boot the Xen HV using the command bootm 0x6000000 - 0x4000000.

This tells U-Boot to boot the application at memory address 0x6000000 (xen.ub) with no initial ramdisk (denoted by the -), the third argument is the flattened device tree (system.dtb) at address 0x4000000.

The Xen HV will bring up Dom0 on its own. Once Linux has loaded there will be the standard login prompt to log into Linux. Log in as ‘root’ (password: root). You can use the Xen pre-installed tools to check the status of the HV system (xl).

xl list will list the running domains, their IDs, available memory, number of virtual CPUs available, and their state.

Load a Guest OS using Xen

Now that the host domain (Dom0) is running, it can load guest domains which will run on their own, isolated thanks to the Xen HV. Many different guests can run from full-fledged Linux to Xen’s MiniOS.

This example will load the same Dom0 Linux image as a guest OS. A Xen configuration file must be created for it.

Create the DomU Configuration File

This step will create a configuration file on the host computer. It will be transferred via tftp to the running QEMU session.

  1. Create a new file called <PetaLinux_project_root>/images/linux/DomU_Linux.cfg.
  2. Open the file in a text editor.
  3. Place the following in the file (modified from the Xen boilerplate configuration file): 
    # =====================================================================
# Example PV Linux guest configuration
# =====================================================================
#
# This is a fairly minimal example of what is required for a
# Paravirtualised Linux guest. For a more complete guide see xl.cfg(5)
# Guest name
name = "DomU_Linux"
# Kernel image to boot
kernel = "/home/root/Image"
# Initial memory allocation (MB)
memory = 512
# Number of VCPUS
vcpus = 1
extra="console=hvc0"
  4. Save the file.

NB : The extra="console=hvc0" enables the user to pass input to the guest OS through the console.

Load the Guest OS

Return to the QEMU terminal, now logged into Linux. First, transfer the Linux image and the configuration files to the embedded Linux system (running on QEMU) via tftp. Then use the Xen tools to create and run the guest OS.

  1. Go to the root user’s home directory:cd /home/root
  2. Transfer the Linux image over:tftp -g -r Image 10.0.2.2
  3. Transfer the configuration file over:tftp -g -r DomU_Linux.cfg 10.0.2.2
  4. Create the new domain:xl create -c DomU_Linux.cfg

The guest domain should now be outputting messages to the console and a login prompt should appear. Log in as root again.

Both the Host domain and Guest domain are right now indistinguishable (they have the same bash prompt). However, running xl list in the guest domain will return an error, whereas running it in the host domain will now display two running domains; i.e., the host and the newly created guest domain.

To test this, return to the host domain’s console by pressing CTRL+]. To return to the guest OS’s console (from the host OS), run xl console <domainName or domainID>.

To remove a domain you can run xl shutdown <domainName or domainID> to forward a shutdown command or xl destroy <domainName or domainID> to forcefully remove a domain. Xilinx PetaLinux builds the required output products while the BSP supplies Xen specific configurations and utilities.

Summary

The Xilinx-provided BSP and PetaLinux tools can be used to generate everything needed to run and manage Xen on the Zynq UltraScale+ MPSoC.

Further Information

Just getting started with using Zynq UltraScale+ MPSoCs and need additional support? Contact us to find out how Zynq UltraScale+ MPSoC training can help get you up and running!