REVEN v2 installation guidelines

This section will guide you through the process of installing REVEN. You will find in Annex I an overview of REVEN deployment.

Pre-requisites

Server hardware

Here are some configuration hints for running an instance of REVEN:

Minimum configuration:

  • Intel processor with VT-x such as Intel Core i7 or Intel Xeon
  • 16GB RAM
  • 1TB SSD

Recommended configuration:

  • Intel processor with VT-x such as Intel Core i7 or Intel Xeon, running at 4GHz, with at least 8 hardware threads
  • 64GB RAM
  • 2TB SSD connected with NVMe + 4TB HDD
  • Internet access for symbol information downloads

Any type of RAID-0 configuration may help reduce the disk I/O bottleneck.

Debian operating system

The REVEN v2 package must be installed on a Debian 9 Stretch amd64 system.

Storage

The list below shows how REVEN organizes its data, together with the corresponding configuration variables that can be set during the installation process.

  • Installation path: QUASAR_REVEN_ROOT_PATH
    Automatically set by start.sh with its own directory.

  • VM: QUASAR_QEMU_SCAN_PATH
    The VM repository containing the QEMU images, should be fast for snapshot save/load operations.

  • REVEN scenarios: QUASAR_ROOT
    VM-specific files such as their filesystems, the REVEN recordings, the replay files, which may be quite large (hundreds of GB). Storage requires a high I/O throughput, to get the best performance out of REVEN (e.g. SSD).
    Since this directory will contain SQlite databases, be careful not to have it being in an NFS mount, or you may experience some difficulties and bugs.

  • PDBs: QUASAR_SYMBOL_STORE
    Can be shared between users and/or machines.

  • Archives: QUASAR_ARCHIVES_PATH
    The scenarios exports. Can be used for backups. Storage can be slow, should be safe (RAID, ZFS, ...).

  • Temporary directory: QUASAR_TMP
    A work directory for REVEN. The faster the better. Putting that directory in a RAMFS mount point will even help reduce latency during scenario recording.

Networking

The list below shows the network connections that must be open between the REVEN server and other machines. Make sure any filtering device is configured to allow these connections.

  • Main project manager interface: QUASAR_UWSGI_PORT
    By default, the project manager listens on port 8880.

  • VMs and Axion Web usage: QUASAR_USE_VNC=True
    By default, VMs and Axion displays are served through a random port. The settings variable QUASAR_WEBSOCKIFY_PORT allows to set a fixed value for this port.

  • VMs and Axion X server usage: QUASAR_USE_VNC=False
    You must be able to run X server applications in the terminal where REVEN is started: remote X or native X server will both work.
    A common situation is to use SSH X forwarding, in which case the SSH port must be open.

  • REVEN server ports: REVEN server listens on any port of the ephemeral port range, which defaults to [32768, 60999] on Debian. You may want to access these ports when using the REVEN Python API remotely, in which case a VPN may prove useful.

NOTE: implementing a reverse-proxy in front of REVEN may simplify the requirements on network filtering. Please refer to the specific reverse-proxy section.

  • Connections to the symbol servers: QUASAR_SYMBOL_SERVERS
    Any symbol server listed in the symbol server list must be accessible to the project manager and the REVEN server.

  • Connection to ret-sync (IDA):
    IDA synchronization with a REVEN trace requires Axion to connect to the machine running IDA (port 9100 by default). Here again, a VPN may prove useful.

Installing REVEN v2

Follow the steps below to install REVEN v2:

  1. Unwrap the REVEN v2 package. You can choose to install it anywhere you want on your file system. Change your current directory to the root of the unwrapped package.

  2. Install the required system dependencies:
    ./install.sh
    This will ask for your sudo password in order to install the system dependencies.

    NOTE: If you have a separate privileged account, then it's not a problem to run install.sh with this account, since the install process does not depend on any particular user-specific variable.

  3. Add your user to the group kvm:
    sudo adduser your_user kvm

  4. Install the REVEN environment:
    ./start.sh

    NOTE: You may encounter an exception from the /usr/lib/python3.5/weakref.py file. It does not affect the successful completion of the installation, see the associated FAQ entry for more information.

  5. Adjust the installation settings:

  • Edit the settings.py file to fit your needs.
    • You'll probably want to touch the QUASAR_SYMBOL_SERVERS variable.
    • By default, the QUASAR_ROOT will point in your home directory (~/Reven2). See the storage requirements above for more information.

NOTE: user-wide parameters can also be set in a ~/.config/tetrane/quasar.py file. It's the recommended place for QUASAR_SYMBOL_SERVERS and custom QUASAR_ROOT, QUASAR_QEMU_SCAN_PATH, QUASAR_ARCHIVES_PATH, QUASAR_SYMBOL_STORE, and QUASAR_USE_VNC.

Using REVEN v2 for the first time

REVEN v2 comes with new services accessible through a new web interface named REVEN Project Manager to manage Virtual Machines, scenarios and associated data, and to launch Axion on those scenarios.

Local use from the REVEN host

When locally logged in the REVEN host, use REVEN v2 as follows:

  1. Launch the Project Manager services, with the following command at the root of the installation directory:
  • ./start.sh
  1. Then point your favorite web browser to the Project Manager's homepage: http://your-reven-host:8880/

NOTE: The Project Manager will run X programs for you to record a scenario, or to access the Axion GUI.

Remote use

Through web browser

The simplest way to use REVEN v2 on a remote server, is to set the QUASAR_USE_VNC setting to True. This will make everything, from the virtual machines to Axion, to be accessible directly from the browser.
Don't forget to restart the Project Manager after changing this setting.

This feature is known to have a few limitations by design:

  • Copy-pasting will work with Axion sessions, but not with VM sessions, since this would require QEMU or VirtualBox to have hooks in the OS they virtualize. This is the same for the "Remote resizing" feature.
  • Some keyboard shortcuts may be catched by your desktop environment or web browser before being sent to VNC. This can usually be worked around using the "Extra keys" in the NoVNC UI or reconfiguring your Axion shortcuts.

With X forwarding

When not locally logged in the REVEN host, but working from a remote client, use REVEN v2 as follows:

  1. Connect to the REVEN v2 host through SSH with X forwarding enabled:
  • ssh -Y your-login@your-reven-host Then use the Project Manager as you would locally.
  1. Launch the Project Manager services, with the following command at the root of the installation directory:
  • ./start.sh
  1. Point your favorite web browser to the Project Manager's homepage: https://your-reven-host:8880/

NOTE: in this configuration, other users should not use your REVEN v2 instance as X applications would be redirected to your display.

They will need to launch their own instance of the Project Manager (See next section).

Stopping REVEN v2

Simply run ./stop.sh at the root of the installation directory, with the same user that launched start.sh.

Troubleshooting

If you have any trouble somewhere, don't hesitate to take a look at the logs located in ~/Reven2/<version>/Logs and contact support@tetrane.com for any help.

Moving a REVEN v2 installation directory

If you move the installation directory after running the first start.sh, you'll have to manually reset the QUASAR_REVEN_ROOT_PATH variable in the settings.py file.

Installing the REVEN v2 Python API

Please refer to the dedicated installation document.

FAQ

What is Quasar?

Quasar is the component implementing the project manager.
The Project Manager is REVEN v2 Web user interface to manage Virtual Machines, scenarios and associated data, and to launch Axion on those scenarios.

Where do I change the settings?

Per user settings

Some Project Manager settings can be changed at the user level, for all the versions installed for this user, in the file ~/.config/tetrane/quasar.py. This file can store settings common to all the user's versions of the Project Manager, such as the list of symbol servers or some storage paths. It is created the first time a Project Manager is started for the user.

IMPORTANT: Remember to run stop.sh then start.sh for every running version of the Project Manager in order to take new settings into account.

Per instance settings

Some Project Manager settings can be changed at the instance level, at the root of the installation directory, in the file settings.py. It is created the first time an instance is started.

IMPORTANT: Remember to run stop.sh then start.sh in order to take new settings into account.

What settings may I change?

Here are some important settings you may want to tune:

  • QUASAR_ROOT: this is where most of the files will be stored. It is by default in your home directory (~/Reven2), but you may change it to use a custom path.
  • QUASAR_QEMU_SCAN_PATH: this defines where the QEMU Virtual Machine QCOW files will be scanned for registration in the Project Manager. Since this will be scanned recursively, be careful not to put too large a file tree in there, or some Project Manager pages will become very slow.
  • QUASAR_SYMBOL_STORE: this is the symbol cache that REVEN will use for storing PDB files it downloads. If this path is attached to a slow mount point, such as a centralized SSHFS mount point, you may experience latency in the Axion GUI. However, it may be a good idea to share this path with others, since everyone will take advantage of the cache.
  • QUASAR_UWSGI_PORT: port the Project Manager web server will listen to.

What if I want to run multiple instances on the same machine?

You can run multiple instances of Project Manager on the same machine, as long as you set the multiple web interfaces to listen on different ports. You have two solutions to do that:

  • Give the port number to start.sh as its first argument:
    ./start.sh 4000 will make the web interface accessible on port 4000.
  • Set the port number permanently in one of your setting files, with the variable QUASAR_UWSGI_PORT, depending on your deployment configuration.

How to use REVEN behind a reverse-proxy?

It's no problem running a REVEN instance behind a reverse-proxy, as long as you follow those recommendations:

  • (optional) Change the QUASAR_UWSGI_PORT to a custom value (e.g. 8888)
  • Set the QUASAR_USE_VNC value to True
  • Set the QUASAR_WEBSOCKIFY_PORT to a fixed value (e.g. 6080)
  • Set the QUASAR_WEBSOCKIFY_PUBLIC_PORT to the proxied value (e.g. 80)

Here is an example of a working nginx configuration using the above example values:

server {
    listen 80;

    location / {
        proxy_pass http://127.0.0.1:8888/;
        proxy_http_version 1.1;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $server_name;
        proxy_set_header Authorization ""; # Ensure we clear the Authorization header for DRF.
    }

    location /websockify {
        proxy_pass http://127.0.0.1:6080/websockify;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $http_connection;
        proxy_read_timeout 1800s; # Default is 60s, which is really low in this use-case.
    }
}

I am getting exceptions from /usr/lib/python3.5/weakref.py, is it an issue?

Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7f2bc24c40d0>
Traceback (most recent call last):
 File "/usr/lib/python3.5/weakref.py", line 117, in remove
TypeError: 'NoneType' object is not callable

This behavior is not an issue. It is a well known exception (Python issue 29519) that affects the version of Python used in Debian 9 Stretch. When encountered, it may leak some memory but won't impact the successful completion of the operations.

A patch is available, but since it should be applied to /usr/lib/python3.5/weakref.py, which is a system file, we don't apply it ourselves.

diff --git a/usr/lib/python3.5/weakref.py b/usr/lib/python3.5/weakref.py
index c66943f02e24..3e1fb8158069 100644
--- a/usr/lib/python3.5/weakref.py
+++ b/usr/lib/python3.5/weakref.py
@@ -53,7 +53,7 @@ def __init__(*args, **kw):
         args = args[1:]
         if len(args) > 1:
             raise TypeError('expected at most 1 arguments, got %d' % len(args))
-        def remove(wr, selfref=ref(self)):
+        def remove(wr, selfref=ref(self), _atomic_removal=_remove_dead_weakref):
             self = selfref()
             if self is not None:
                 if self._iterating:
@@ -61,7 +61,7 @@ def remove(wr, selfref=ref(self)):
                 else:
                     # Atomic removal is necessary since this function
                     # can be called asynchronously by the GC
-                    _remove_dead_weakref(self.data, wr.key)
+                    _atomic_removal(self.data, wr.key)
         self._remove = remove
         # A list of keys to be removed
         self._pending_removals = []

Annex I: deployment overview

Deployment overview

©2019 Tetrane