2.11.0

Highlights

REVEN version 2.11 is packed with new features, with the following highlights:

  • A fuzzing and triage platform to analyze crashes found by a fuzzer using REVEN's powerful features such as semantic tainting.
  • A new WinDbg-assisted workflow to perform fine-grained recording of Windows scenarios, whether applicative or full-system, at the breakpoint level.
  • The most "featureful" release of the Analysis Python API since REVEN 2.2 (which introduced the API)! This release introduces:
    • Essential vocabulary types such as MemoryRange, RegisterSlice.
    • Helpers for the Taint API that facilitate the implementation of semantic tainting.
    • A new API to inspect handles inside the trace of a Windows scenario.
    • An extension of the Type API to read data structures in the trace of a Windows scenario, directly after their description in the PDBs.

In more details:

  • Fuzzing & Triage Platform: this release includes a first iteration of the vision we shared in the dedicated blog article and on our stand at Offensive Con 2022. The platform monitors a directory for crash reproduction files, produced by the fuzzer of your choice. Then for each such file, it records a scenario by passing the file as input for the target binary, then it replays, analyzes and performs some initial triage of the recorded crashes. Note that the analysis is provided for Windows scenarios only so far: you will need to either fuzz directly on Windows, or to cross-compile your target binary to fuzz it on Linux and analyze crashes on Windows. Find more in the user documentation. The platform's crash analyzer is available in all REVEN Editions and can be manually run against any existing replayed scenario, while the platform's workflow itself (monitor, record, replay, analyze) is exclusive to the Enterprise Edition.

  • Fine-grained recording with WinDbg: in all editions, you can now use WinDbg to finely control a Windows scenario's VM for start and stop recording points. We recently demonstrated the capability with GDB, allowing you to make records shorter and to the point, using GDB's familiar breakpoint interface. Additionally in the WinDbg integration, you can apply breakpoints directly on function names at record time. This technology combines the QEMU Virtual Machine Introspection (VMI) technology with REVEN's OSSI. A dedicated blog article on using WinDbg and VMI to make fine-grained recordings will follow, so stay tuned! Meanwhile, you can already refer to the user documentation for the feature.

  • New "vocabulary" types for the Analysis API: up to now, most of the API would model a range of memory by accepting two parameters (start address and size) and returning tuples in the functions manipulating such objects. The MemoryRange concept is often needed to keep track of the raw information (address, size) and add additional functionality on top (such as range intersection, union, concatenation, ...). This led to multiple, custom, similar implementations of the concept in users' scripts, which is indicative that MemoryRange is a "vocabulary" type: a type that is always implemented in a similar fashion and that is frequently useful. Providing a "canonical" implementation in the API spares users from the multiple reimplementations and makes it easier to communicate between scripts, that would previously need to convert between the various implementations of the type. This release also implements a RegisterSlice type that allows to express that you only want a subslice of the bits of a register (for example eax is rax[0:32]), and MemoryRangeMap/RegisterSliceMap types that store non-overlapping ranges associated with arbitrary data, very useful for some applications such as the new Taint helpers.

  • More powerful Taint API: speaking of the new Taint helpers, we now provide easier ways to query what is tainted in a state, and to add or remove tainted data from a taint state. You can then restart a new Taint from the modified taint state. This simplifies the implementation of semantic tainting in the fuzzing platform, that often requires checking that some registers or memory areas are tainted, and then modify the state before restarting the Taint.

  • Windows Handle inspection: we provide a new reven2.preview.windows module that contains some Windows-specific API entries, the main one for this release being the reven2.preview.windows.Context.handle and reven2.preview.windows.Context.handles methods. They allow to fetch the object corresponding to each OS handle in a REVEN scenario, so that you can easily get e.g. the filename corresponding to the handle parameter in a NtReadFile call.

  • Revamped types API package, now with Windows structure information from PDBs: the API always provided a way to read from a source (register, memory) at any context, and interpret the result as some type such as signed or unsigned integers, floats, or even strings. In addition to these primitive types, this API has now been extended to support structure and enumeration types, such as the types defined by the programmers of the recorded programs. For Windows scenarios, these types are fetched from the PDBs, so that you can ask any binary executed in a REVEN scenario for any named type defined by this binary. Here is a quick example of using the new types API package to find all files created by NtCreateFile in a scenario:

    >>> import reven2
>>> from reven2 import types

>>> # Connecting to a reven server
>>> hostname = "localhost"
>>> port = 13370
>>> server = reven2.RevenServer(hostname, port)

>>> # Getting the first binary named "ntoskrnl.exe" in the list of executed binaries in the trace
>>> ntoskrnl = next(server.ossi.executed_binaries("ntoskrnl.exe"))
>>> # Getting the exact symbol "NtCreateFile" in "ntoskrnl.exe"
>>> nt_create_file = next(ntoskrnl.symbols("^NtCreateFile$"))

>>> # Finding all files that are created in a call to NtCreateFile
>>> def read_filename(ctx) -> str:
...     # The third argument to the call is an _OBJECT_ATTRIBUTES structure, the filename is stored in:
...     # _OBJECT_ATTRIBUTES->ObjectName.Buffer
...     ty : types.Struct = ntoskrnl.exact_type("_OBJECT_ATTRIBUTES")
...     object_attribute: StructInstance = ctx.deref(reven2.arch.x64.r8, types.Pointer(ty))
...     unicode_string = object_attribute.field("ObjectName").deref_struct()
...     return unicode_string.field("Buffer").deref_str(
...         types.CString(encoding=types.Encoding.Utf16, max_size=unicode_string.field("Length").read_int())
...     )
...
>>> for (index, ctx) in enumerate(server.trace.search.symbol(nt_create_file)):
...     if index > 5:
...         break
...     print("{}: {}".format(ctx, read_filename(ctx)))
...
Context before #14771105: \??\C:\Windows\SystemApps\ShellExperienceHost_cw5n1h2txyewy\resources.pri
Context before #14816618: \??\PhysicalDrive0
Context before #16353064: \??\C:\Users\reven\AppData\Local\...\AC\Microsoft
Context before #16446049: \??\C:\Users\reven\AppData\Local\...\AC\Microsoft\Windows
Context before #16698900: \??\C:\Windows\rescache\_merged\2428212390\2218571205.pri
Context before #26715236: \??\C:\Windows\system32\dps.dll

Improvements

REVEN

  • It is now possible to record interactions with some USB devices configured as USB passthrough.
  • Linux kernel debug symbols are now correctly retrieved if the Linux kernel debug binary is present in the prepared file system. As a result of this change, the name of the executed binary corresponding to the kernel in a trace is now its actual name on disk (e.g. /boot/vmlinuz-<kernel version>) rather than kernel like it used to be.
  • Searching for executed binaries in the trace is now faster when using the "Not Match" operator.
  • This version improves performance of replaying the Memory History resource by about 10%.

Project Manager

  • The VM's full path is now displayed when registering a new VM, to disambiguate VMs with the same name in different directories.

Axion

  • When hovering over a transition in a widget, its location is displayed in timeline.

Analysis Python API

types package

  • Add reven2.types.Struct, reven2.types.Enumeration, reven2.types.UnresolvedStruct, reven2.types.UnresolvedEnumeration to model user-defined types.
  • Add reven2.types.ErrorType and reven2.types.VoidType that respectively represent types that couldn't be parsed from a debug object and the void type.
  • Add reven2.types.StructInstance, reven2.types.EnumerationInstance, reven2.types.ArrayInstance, reven2.types.PointerInstance to represent values read from these types.
  • Add reven2.ossi.Binary.exact_type to get a named type from a debug object for Windows scenarios.
  • Add reven2.types.F16.
  • reven2.types.Pointer now accepts an explicit size arguments. Pointers created this way don't have a context-dependent size.
  • Add reven2.trace.Context.deref_all to explicitly ask dereferencing nested pointer types.
  • Add reven2.types.Pointer.cast_inner to cast the inner type of a pointer.

Taint helpers

  • You can now modify a reven2.preview.taint.TaintState to add or remove elements of the state, and restart a new taint from this modified state using reven2.preview.taint.Tainter.taint_from_state.

Register slices

  • Building a reven2.RegisterSlice by indexing a register, e.g. reven2.arch.x64.rax[0:-3] is now possible.
  • The following items have been added:
    • reven2.arch.helpers.is_flag_register and reven2.arch.helpers.is_condensed_flags_register.
    • reven2.RegisterSlice to build bitwise slices of a reven2.Register.
    • reven2.register_slice.RegisterSliceMap and reven2.register_slice.RegisterSliceSet to store multiple reven2.RegisterSlices.
    • reven2.Register.root and reven2.Register.children to retrieve the base register/children registers of a register.
  • reven2.preview.taint.TaintedRegisterSlice now accepts a reven2.RegisterSlice as tainted data.
  • reven2.arch.register.Register and reven2.arch.register.Category are now exposed as reven2.arch.Register and reven2.arch.Category.

Memory ranges

  • Add reven2.MemoryRange to model ranges of memory address.
  • Add reven2.memory_range.MemoryRangeMap and reven2.memory_range.MemoryRangeSet to store multiple reven2.MemoryRange
  • Add memory range support to various types of the API.

OSSI

  • Add support of data symbol:
    • reven2.ossi.Symbol is now an abstract class
    • Add reven2.ossi.FunctionSymbol and reven2.ossi.DataSymbol to ossi
    • reven2.ossi.Ossi.symbols and reven2.ossi.Binary.symbols return an iterator of reven2.ossi.FunctionSymbol and reven2.ossi.DataSymbol
    • Add new methods reven2.ossi.Binary.function_symbols and reven2.ossi.Binary.data_symbols
  • Add reven2.ossi.BinaryMapping and reven2.ossi.OssiContext.kernel_mappings to access to the kernel mapping at a context.
  • Add the reven2.ossi.Thread.owner_process method to find the process owning a the current thread, when available. On Windows, threads can be detached from their owner process to execute in the context of different processes.

Python API type annotations

  • Add marker file indicating that the API is typed. This results in type checkers no longer ignoring types of the API when typechecking scripts that use the API.
  • Typing: Expose reven2.address.AddressType to represent any address type to the type checker and reven2.address.VirtualAddress to represent virtual addresses to the type checker.

Miscellaneous

  • Add preview.windows package containing Windows 10 utilities (e.g to parse handles and objects)
  • Add string representations for classes in the reven2.types package.
  • Classes in the reven2.types package are now comparable for equality.
  • Made the following classes hashable:
    • reven2.arch.Register
    • reven2.trace.Context
    • reven2.trace.Transition
    • classes in the reven2.types package
    • classes in the reven2.address module

Analysis Python API script library

  • Added examples/analyze/memory/symbols_access_memory_range.ipynb and memory_ranges_accessed_by_a_symbol.ipynb scripts that demonstrate how to search the trace for the symbols that access a given memory range and for the memory accesses made during the execution of a symbol, respectively.

Fixed issues

REVEN

  • Forward tainting rep-prefixed instructions no longer causes overtaint in some situations.
  • Fixed display of operands that only consist of an absolute address in the API and in Axion.
  • When running a VM, the I/O thread spun for 1000 iterations condition is no longer encountered.
  • Filters would not return a range when the last context was included in the query.

Project Manager

  • Scenario deletion no longer fails with an error in some cases.
  • When attempting to replay a kernel description for a Linux kernel version that is outside of the supported range, the supported range is now correctly displayed.
  • In some cases, failing to download a PDB would preclude from downloading the others from the PDB server.
  • VM download from a URL no longer fails in some cases
  • The Light Filesystem resource is no longer marked as "not compatible" on imported scenarios (where it cannot be regenerated).
  • Sometimes artifacts from older automatic records would interfere with newer automatic records during the Light Filesystem generation, leading to wrong or missing symbols.
  • Replayers would sometimes output terminate called recursively multiple times in the logs when failing.

Axion

  • The Memory Watcher View no longer triggers the save confirmation dialog when the memory range wasn't edited/modified.
  • Calltree view improvements:
    • Better display of binary name
    • Enable horizontal scrolling in the view
    • Fix vertical scrolling when adding new items
  • The Search Symbol completion no longer freezes for a long time on binaries that contain a lot of symbols.
  • The Calltree view no longer requires multiple clicks to respond when clicking on sibling and level up buttons.

Analysis Python API

  • reven2.trace.Trace.filter would raise an exception if to_context == from_context + 1.
  • Equality comparisons of objects reven2.ossi.Binary, reven2.ossi.BinaryMapping, reven2.ossi.Symbol, reven2.trace.Context, reven2.trace.Transition and reven2.memhist.MemoryAccess now all return False when comparing against an object of a different type. Previously the behavior was unspecified.
  • Fixed an issue where reading a reven2.types.Array of e.g. a reven2.types.CString, a reven2.types.Pointer or a nested reven2.types.Array would return values of incorrect types. As a result of this fix, reading a value as a reven2.types.Array will now return a reven2.types.ArrayInstance.

Analysis Python API Compatibility Notes

  • The Python packages installed system-wide are no longer available in the REVEN virtual environment. This is to prevent possible conflicts between REVEN requirements and the system packages. You can install any needed Python package in the REVEN virtual environment using pip.

  • reven2.preview.taint.Tainter.simple_taint no longer attempts to convert the passed tag argument to int when it doesn't match any of the supported types.

  • Comparing an address with another type for equality no longer raise a ValueError, but returns False instead.

  • Calling Trace.memory_accesses with a size of 0 now raises a ValueError.

  • reven2.ossi.FunctionSymbol.prototype is deprecated. Use reven2.ossi.FunctionSymbol.demangled_name instead

  • Since reven2.ossi.Ossi.symbols and reven2.ossi.Binary.symbols now return reven2.ossi.DataSymbol in addition to reven2.ossi.FunctionSymbol, indiscriminately calling reven2.ossi.FunctionSymbol.name_only or reven2.ossi.FunctionSymbol.prototype on its returned values may now raise an exception. If you have code like the following:

    >>> for symbol in binary.symbols():
>>>     symbol.prototype

    Modify it to use the reven2.ossi.Binary.function_symbols function instead:

    >>> for symbol in binary.function_symbols():
>>>     symbol.prototype

  • Constructing a reven2.preview.taint.TaintedRegisterSlice with a non-positive size now raises a ValueError.

  • reven2.trace.Context.deref is now deprecated when the target is a type containing nested pointers. Either use reven2.trace.Context.deref_all to explicitly ask for nested-pointer dereference, or replace the inner Pointer types with USize. In the future reven2.trace.Context.deref will only dereference the outermost pointer.

  • reven2.trace.Context.read is now deprecated when the target is a pointer type. To compute an address, add the base address of the pointer type to the offset computed by reading as a USize. In the future, reven2.trace.Context.read will return a reven2.types.PointerInstance when reading a pointer.

  • Reading a value as a reven2.types.Array will now return a reven2.types.ArrayInstance rather than a list. The former can be converted into the latter by passing it to the list constructor.

©2019-2021 Tetrane