Here you may find answers to various questions about the behavior of REVEN-Axion. If you think you hit a bug, please read the list of known issues for your version in the Change Log

FAQ

Why can't I pause an execution instantly?

An execution produces a lot of data, most of which is written on disk. When you ask for the execution to pause, REVEN must flush its buffer first: that may take up to 30 seconds depending on the trace and on your disk bandwidth.

Where are the log and configuration files?

You can find comprehensive information about the server configuration file in the setup page. The client configuration files are located in ~/.config/tetrane/.

The server log files are (assuming the default configuration):

The per-project log file: /home/reven/reven_data/<project>/output/reven.log
The global log file: /var/log/reven_launcher.log

The client outputs its logs on the stderr file, so you can find them in your common X session errors log file, usually located at ~/.xsession-errors. You can also start the Axion client directly in a console, and see its logs printed there.

Can I auto load a plugin on Axion startup?

Yes you can! See the init script setting where you can specify the load init script. You can copy this script somewhere, edit it to your liking, and configure Axion to load it at startup.

Oddities

Why aren't runs in the Current run combo box sorted?

They actually are sorted chronologically, as expected. But their names include the parent run's sequence number where this run has started; in the case of nested runs, you could have something like this:

Execution run
    #3186 PageFault
        #21 Nested PageFault
    #5445 Another PageFault
    ...

Displaying this indentation would be impractical, so it may look as if runs are unsorted. We intend to rework on how we display this information in the future.

Why do I get errors like "sgdt, at 0x1b:0x77d94eb8"?

There are a few other CPU instructions that REVEN does not handle yet. Those instructions are ignored but raise warnings, and depending on the program this may cause a out of sync error later. For example, you will often see this error message in Windows crash handler code (during a BSOD)

If one of these errors prevents you from working, please contact the support team. We will certainly implement it in a future revision.

Why are there symbols for which I can see the arguments but others for which I can't?

Sometimes symbols are deduced from the pdb-like files, which also contains information about the arguments. In this case, you will see the arguments and their content in the trace view.

Although when symbols are o deduced from the binary, REVEN doens't have enough information to help you further.

I'm selecting the pointer operand of an LEA instruction. Is it normal that the selected operand is a register and not the pointer ?

The LEA instruction does not dereference the given pointer. In fact if the address was to be completely invalid, the instruction would still work.

Let's take the following example:

lea   eax, [ebp+edx*4+0x7]

Selecting the second operand will select ebp. This is an arbitrary decision here. This LEA is a shortcut for something like:

mov eax, edx
shl eax, 2
add eax, ebp
add eax, 0x7

If we were selecting the pointer instead of the register that gives the pointer, a use of the "previous" or "next" feature would lead to the previous or next use of the pointer, which is not necessarily related to this context. Maybe the pointer in question will never be accessed in the whole program. In this case, the shown LEA would just be a more compact way to compute a value.

Why do I keep having symbols like `func_a3143`?

The symbols of the form func_<rva> are generated by REVEN during the execution (see Symbol documentation). This happens when REVEN can not find the symbol information at the rva location neither in the binary it self nor in the associated PDB file (if one exists).

If you think you should have more symbols, perhaps you should check the configured pdb_path contains the PDB files you need (you can retrieve them by following the PDB documentation).

Why does the tainter seem to hang sometimes?

The tainter widget gives limited information when it is waiting for an anwser from the server. This may be confusing, especially if other projects are being executed on the same server, slowing it down a lot. We are working on that.

Troubleshooting

If you have a problem, you should always first check that:

There is enough remaining disk on the server and on the client
There is enough remaining disk on the server's tmpfs (/tmp)
There is enough remaining RAM on the server and on the client.

Still, my execution seems much slower than it should

REVEN's execution is heavily dependent on the server's specifications. Using various unix tools on the server (htop, iotop, etc.) you can look for the bottleneck:

If your RAM is full, there is a fair chance the OS is swapping a lot of data on the disk, in which case performance will be abysmal.
If the CPU seems idle, certainly the bottleneck is the disk I/O throughput. Try and get faster disks (RAID, SSD, or simply disks with a higher RPM). Especially in the case of multiple executions happening on the same server, the disks can easily be overwhelmed.
If one of your CPU's threads is constantly at 100%, then there is no bottleneck other than pure CPU performance. For the moment, REVEN does not take advantage of multi-core setups.

See also scenario tips for tips on how to make your scenario smaller, if possible.

My execution stopped and the error log says "out of sync"

First of all, make sure the VM you produced your scenario with does not have Virtual Box's Guest Additions enabled. These implement custom emulated hardware that we do not support for now, and are bound to desync quickly. If your VM has them enabled, you will have to deinstall them, restart the VM and re-record your scenario from scratch.

If not, then REVEN's CPU has diverged from the reference execution recorded in the scenario, and stopped because it can no longer produce relevant results. This is current limitation of REVEN, and will certainly happen from time to time, especially if you let REVEN work for more than a few days.

You still have a chance at producing a usable output. The point where the desync happened is not necessarily where the problem appeared, so you should always try taking the following mitigations steps:

First of all, it is possible that the execution went past your point of interest. You should check the execution trace to see if it is enough.
If not, then your best chance at getting a workable execution trace is reducing your scenario:
- Make sure you followed the optimisation rules
- See scenario tips as well.
If nothing works, then you hit an important bug. Please contact the support team to let us know. We are constantly improving REVEN's symbolic CPU, as well as working on limiting the impact of such problems.

My client froze

You have hit a bug. Make sure you follow the basic checks. If it's not enough and if you know how to reproduce this behavior, please contact the support team to let us know. The bug may have appeared on the client's side or on the server's side:

Try killing your client (killall -9 axion) first, and then reconnect to your project. If this works, no work has been lost.
If the client cannot reconnect to the opened project, then this project has frozen. You have no choice but to force close it, and any unsaved work will be lost.

There is an opened project I can't force close

Your project is frozen, you have hit a bug. Make sure you follow the basic checks. If it's not enough and if you know how to reproduce this behavior, please contact the support team to let us know.

For the moment you have no other choice but to manually kill the frozen process:

In the "Opened projects" tab, note the port of your project
Connect to the server via SSH:
- using for instance ps aux | grep reven, list the reven processes and note the pid corresponding to your project's port.
- kill this process.

If it's not yet enough, you have to restart the REVEN daemon, but understand that this will force close all opened projects on the server. Make sure you take the appropriate steps before going this far.

I can't seem to record a scenario

There is a whole page dedicated to scenario creation, with a specific troubleshooting section.

When I create a new project, I get the error "boost::filesystem::create_directory: Permission denied"

This is a server configuration problem. If you recently created a new user, make sure its directory and all its children are owned by the reven user.

Numpad digits and some other keys don't seem to work

This is a known bug, probably related to the way Qt5 handles Apple keyboards. Unfortunately, we don't have a workaround for now except using the upper-row digit keys. If you see this problem on a non-Apple hardware, please contact the support.

Certain shortcuts just won't work

We have seen configurations where shortcuts don't all work: some are caught by the Window Manager and never make it to the Axion app, some are caught before entering the Axion VM (if one is used). If you're not on Apple hardware (which, as seen above, can lead to another kind of issues), please try to make sure your Window Manager doesn't silently catch the shortcuts in question.

While controlling a VM through VNC, the keymap is all wrong

This is a known problem if your system's keyboard layout isn't US qwerty. A quick workaround for this issue is to edit your VNC program and surround the current value with calls to setxkbmap: you will set the keyboard layout to us first, call the vnc program, then to restore it. This could be something like setxkbmap us; echo p | xtightvncviewer -autopass h::n; setxkbmap fr

I get an error on `VMPTRLD` or `VMPTRST` instruction

This is a known bug. It is actually either RDRAND or RDSEED, which are not disassembled correctly. These instructions are declared as unavailable but programs can still make blind use of them, which will likely cause a desynchronisation.

Linux in particular is a known offender. You can still forbid it from using it:

in the guest VM, add nordrand in the option field GRUB_CMDLINE_LINUX of the file /etc/default/grub
type the command sudo update-grub to rebuild the grub configuration
reboot the VM and create a new snapshot that you can use to record new scenarios.

I get an error with scenarios from VM running with linux >=4.9

This is known. Recent Linux kernels make use of newer, unsupported CPU features. We try to disable these features, but sometimes it is not possible.

Note that when we disable a CPU feature, your existing VM snapshots will not know about it and may still try to use it. Hence, you can try to recreate your VM snapshots to work around your issue:

Restore your VM snapshot.
Shut the VM down (do not simply reboot it).
Turn the VM back on, and save a new snapshot.
You can now try to record new scenarios.

My /tmp is frequently filled even when mounted in tmpfs

Two kinds of information get written to the /tmp at two different times:

When starting the recording of a scenario, the core dump is written to /tmp/core_gen_temp/<random_name>. This core dump is as big as the RAM used by the VM, so make sure you don't have any VM that runs with more than half the total RAM of your reven server. Once the recording ends, the core dump is moved to the project directory.
When a REVEN project is opened, part of your project's data is uncompressed in /tmp/reven. The size of this data depends on how long your scenario is.

In the specific case where you intend to have many projects opened or recording simultaneously, and space on your /tmp directory becomes an issue, please consider the following solutions:

Use a non-tmpfs:

This solution has the drawback that core dumps will also be written to a non-tmpfs system (so not in RAM), which might slow down the time between the start of a recording and the recording itself. This might make the process of recording more intrusive (as guest programs can detect that there is a long pause when starting the recording), hence it is not officially supported.

Modify the default directory where the decompressed archive of an opened project is stored:

In this solution, the /tmp folder remains a tmpfs, which allows you to get fast core dumps. To modify the default directory where the archive is stored, do the following:

Locate the following line in /usr/share/reven/launch_reven_server.sh:

export PYTHONPATH="${BASE_DIR}/python:$PYTHONPATH"

Then, directly below it, add the following line:

export REVEN_TEMP_LOCATION=<path of your choice>

where <path of your choice> is set to where you want the archive to be decompressed.

Then restart the reven service:

service reven-daemon restart

Note that this will kill all processes started by the reven user, so:

Don't perform the restart from a ssh started as the reven user (not even after a su)
All opened servers (running executions, etc.) will be interrupted and will need to be restarted (you will still be able to use your existing save states).

Note that this solution isn't officially supported either, because the modified sh file will be overwritten at the next update. Do this only if strictly necessary.

Table of Contents

FAQ