QEMU Virtual Machines

Pre-requisites

IMPORTANT: QEMU is installed during the REVEN server installation process.

The next sections introduce useful QEMU notions to work with REVEN:

  • KVM vs Emulated QEMU VMs.
  • Disk and live snapshots in QEMU.

KVM vs Emulation

There are two ways to run VMs with QEMU: either using KVM (virtualized mode) or emulated. The former is much faster, but the latter is required when recording. Saved VM states cannot be shared between the two modes, but disk snapshots can.

In the Project Manager's VM view you can select either modes, but when recording a scenario only the emulation mode is available.

KVM mode is convenient when installing software or OSes. See typical workflow below.

For a complete documentation on QEMU tools, please refer to QEMU's online documentation.

How do snapshots work in QEMU

IMPORTANT This section is crucial: REVEN uses the native disk / live snapshot mechanism from QEMU which might differ from what users expect.

There are two types of snapshots available in the qcow file format:

  • Disk snapshots represent the state of a disk, and may be organized in a tree structure to save disk space.
  • Live snapshots represent the full state of a VM, including memory, cpu registers and disk. They are stored inside a disk snapshot, and are what most users expect.

The two are used in conjunction to provide various functionalities:

  • Live snapshots allow the user to store the full state of a VM:
    • Loading a live snapshot will allow restoring a VM that is booted.
    • These are tightly coupled to the options the VM has been started with: not selecting the right options will prevent snapshots from loading. These options include "kvm", "network", or any custom option.
    • They are immutable by design
  • Disk snapshots contains the disk only:
    • They are mutable: they live with the VM
    • Loading a live snapshot will alter the disk snapshot, by restoring it to the saved state. Any modification is lost.
    • Starting a VM from a disk snapshot will, by design, require a full boot.
    • They can be linked to parent disk snapshots, to limit disk usage on the host. Note that altering a parent snapshot may render children unusable!

Disk and Live snapshots

Adding QEMU VMs for scenario recording

Adding an existing VM with the preparation Wizard

REVEN offers a VM preparation Wizard that will guide you in the process of registering and preparing a new QEMU VM in REVEN.

Using this Wizard, you will be able to register:

  • VMs that are in supported formats and live in the directory identified as QUASAR_QEMU_SCAN_PATH in your settings (by default, ~/VMs). The supported formats are listed by the QUASAR_QEMU_SCAN_FORMATS variable in your settings, see the documentation of this variable in settings.py for more information.
  • Other VMs from your local disk or from a remote URL.

After registering a VM, the Wizard will guide you through the initial steps of snapshot creation and lightening of the VM.

Example 1: adding a Windows 10 VM provided by Microsoft

Microsoft provides some free Windows 10 virtual machines that you can download and use locally, such as the MSEdge VM. Please review the license terms of this VM before using it.

To use this VM with REVEN, please follow the steps below:

  1. Download the HyperV version from your browser. The HyperV version is preferred because it is known to be compatible, and is lighter than the other versions.
  2. Unzip the MSEdge.Win10.HyperV.zip file you just downloaded.
  3. Use the VM preparation Wizard to upload the Virtual Hard Disks/MSEdge - Win10.vhdx of the archive from your disk to the server.
  4. Let the VM Wizard guide you through the following steps.

Example 2: adding a VM in OVA format

To use a VM in Open Virtual Appliance (OVA) format with REVEN, please follow the steps below:

  1. Unzip the OVA archive.
  2. Use the VM preparation Wizard to upload the .vmdk or .vdi (depending on the OVA) file contained in the archive from your disk to the server.
  3. Let the VM Wizard guide you through the following steps.

Creating a new QEMU VM from an ISO

You can create a QEMU .qcow2 VM by installing an operating system from an ISO.

To do so, you can follow the steps described below:

  1. Run qemu-img create -f qcow2 /path/to/VMs/myimage.qcow2 80G, where /path/to/VMs/ is the directory identifed as QUASAR_QEMU_SCAN_PATH in your settings (by defaults, ~/VMs), to create a disk file of 80GB. If the qemu-img program is not found, retry after running source /path/to/REVEN/install/sourceme.

  2. Manually start the VM, inserting the iso, by running the qemu-system-x86_64 -m "2048M" -hda "/path/to/VMs/myimage.qcow2" -usbdevice tablet -enable-kvm -cdrom "/path/to/ISO" command, where /path/to/ISO is the path to your ISO.

  3. In the running VM, install the operating system from your ISO. Now would also be a good time to disable unnecessary services and make sure autorun is enabled.

  4. Shutdown the VM properly.

  5. Add the VM to REVEN using the VM preparation Wizard.

Working with QEMU snapshots in REVEN

IMPORTANT NOTE: QEMU replayers do NOT support VMs with strictly more than 3072MB of RAM. DO NOT try to record with more than 3072MB of RAM or the replay will fail. The Project Manager web interface will prevent you from doing that.

In the Project Manager:

  • You register a QEMU VM and corresponding disk snapshots are automatically linked. Besides, both disk and live snapshots can be created.
    • Disk snapshots (usually generated with QEMU qemu-img) can be taken through the Take snapshot button on the VM list page. NOTE: Disks snapshots imply booting the VM, which can be quite long with QEMU without KVM (several minutes for a Windows 10 VM).
    • Live snapshots (usually generated with QEMU savevm) can be taken through the Manage button on the VM list page, then in the Running the VM section. You can also access them on the record page of a scenario.

QEMU snapshots options

By default, when you start a snapshot, it is launched with the VM options (RAM size, network, custom QEMU options) that were provided during the Register VM step. You can override these options for this specific snapshot in the Running the VM section. Overridden options for a snapshot will be applied when starting the VM on this snapshot. You can restore an option to its VM value by unchecking the checkbox associated to this option.

Typical QEMU workflow example

To illustrate the previous explanations, here is how users can typically work with QEMU to configure and prepare a VM before recording a trace:

In the Project Manager VM Manager

  1. Register an existing QEMU VM.
  2. Create a disk snapshot for a new project from a clean parent.
  3. Boot this snapshot with KVM enabled.
  4. Install required software (using the CD-Rom mounting feature to upload files to the VM).
  5. Properly shutdown the VM (on Windows, using Shift+Click on the Shutdown option is required, otherwise the VM is only hibernated!).
  6. You can now Prepare the snapshot: all required binaries are present on the disk.
  7. Boot the VM again in emulated mode, i.e. with KVM disabled, with the required options for recording.
  8. Run operations required before the scenario recording but that need not be recorded.
  9. Take a live snapshot.

WARNING: live snapshots taken with KVM enabled can not be used for recording in REVEN Project Manager with QEMU.

Then, in the Scenario Manager

  1. Create a new scenario, selecting the previously created disk snapshot.
  2. Load the previously created live snapshot.
  3. Record your trace.
  4. Force shutdown the VM.

NOTE: At this point, the disk snapshot contains an OS that didn't properly shutdown: it is usually not an issue because restoring the live snapshot will overwrite this state, but booting the VM from the disk snapshot itself will likely trigger any disk verification process the guest OS may have.

NOTE: You can save live snapshot during scenario creation as well, if necessary.

NOTE: For simpler situations, you might have a few live snapshots in emulation mode for various use cases: one with network, one without, etc.

Help! My snapshot doesn't load!

There are a few situations that will prevent a snapshot from loading. In all cases, you can go to the list of Sessions in the Project Manager to get the log of what went wrong. Several checks can be done, depending on the type of snapshot concerned.

Live snapshots

  • Make sure the selected options match that of the live snapshot, including kvm mode and custom options. As a convenience, the snapshots's name contains a summary of common ones.

Disk snapshots

  • Has the VM been properly shutdown? (Shift + Click on Shutdown in windows)
  • Have the parent disk snapshot been modified? If so, children snapshots are unusable

Note that in some cases your disk snapshot may become corrupted leading to the error Image is corrupt; cannot be opened read/write when launching QEMU. It can sometimes occur when having heavy disk I/O or killing QEMU.

To assert the level of corruption of your snapshot you can use the command qemu-img check /path/to/your/snapshot.qcow2.
A possible fix is to ask qemu-img to fix the corruption qemu-img check -r all /path/to/your/snapshot.qcow2.

©2019-2020 Tetrane