To make it as easy as possible to write new static analysis algorithms based on REIL, BinNavi comes with a static code analysis framework called MonoREIL. This framework is exposed to the plugin API and can therefore be used from scripts and plugins that need to do their own static code analysis.
The monotone solver is the class which executes code analysis algorithms. To execute an analysis algorithm, the algorithm needs to be aware of a few concepts and implement a few classes. These are described below.
All MonoREIL-based algorithms work on so called lattice graphs, directed graphs which are generated from the analyzed REIL code. Right now only one type of lattice graph, the instruction graph, is supported out of the box. Instruction graphs are directed graphs whose nodes contain exactly one REIL instruction and whose directed edges specify the potential control flow between the instructions in the nodes.
It is possible for plugins to define new lattice graphs if instruction graphs are not suitable for a given code analysis algorithm. Another option could be to create operand graphs where nodes contain REIL operands and the edges between the nodes indicate how the operands depend on each other.
MonoREIL works by walking through the lattice graph and propagating analysis results from node to node along the edges of the graph. When the code analysis algorithm starts it is often useful to have an initial analysis state that already contains some analysis results. For example if an algorithm tracks the potential values of registers and it is known beforehand that the register r0 always has the value 0x78110 at instruction 0x1002552 the initial state vector should reflect this fact.
The state vector is basically a mapping between nodes of the lattice graph and arbitrary analysis results that depend on the implemented analysis algorithm. While MonoREIL is executing the code analysis algorithm, analysis results for all nodes are collected and when execution is complete, a final state vector that contains all analysis results is returned.
The values of the mapping defined in the state vector must be so called lattice elements. Lattice elements can be arbitrary objects. The only requirement for lattice elements is that MonoREIL must be able to compare different lattice elements to determine whether two lattice elements are equal or not.
Different analysis algorithms require MonoREIL to take different paths through the lattice graph. Some analysis algorithms require that only the children of a lattice graph node need to be updated when the analysis result of the node changed while others require all parents of the node to be updated. Other algorithms might require completely different ways to navigate through the lattice graph.
Using a graph walker it is possible for code analysis algorithms to define how MonoREIL should walk through the graph. By default there is an UpWalker class and a DownWalker class which implement graph traversal towards the parents or children of a node. Plugins that require other ways to traverse lattice graphs can easily implement their own custom graph walkers.
While walking through the lattice graph, new information about the analysis state of the visited nodes must be discovered. This is the role of transformation providers which define how lattice graph nodes influence the analysis results of the nodes in their neighborhood.
The transformation provider is the core part of each code analysis algorithm. Depending on the type of the lattice graph and the complexity of the code analysis algorithm a transformation provider can be between ten lines and several thousands lines long.
When using the default instruction graph, the transformation provider of an analysis algorithm defines how each instruction influences the tracked information. Assume the implementation of a code analysis algorithm that tracks register values and encounters the REIL instruction add r0, 8, r0. Knowing that before the execution of this instruction the value of register r0 is n the transformation provider would update the state of the lattice graph node that represents that instruction to contain the information that the value of r0 is n+8 after the instruction is executed.
In nearly all cases lattice graphs do not take the form of linked lists. Branches in the input REIL code cause branches in the lattice graph. Whenever two paths of the lattice graph end in the same node, information from both paths is propagated into the node. Since there is no general way of knowing how to combine information from two analysis paths, it is necessary that all code analysis algorithms implement a so called lattice object that defines how to merge information from converging lattice graph paths.
Once all components described above are defined for a static code analysis algorithm, the solve function of the MonotoneSolver class can be called. This function takes the defined start vector and begins to walk through the lattice graph according to the traversal order defined by the graph walker. On visiting a node, the transformation provider discovers analysis results for the node. This information is passed to the nodes in the neighborhood of the node which might then use the information to update their own analysis state. This process continues until MonoREIL notices that the analysis algorithm can not find any more information. The final analysis state of all nodes is returned as the result of the analysis algorithm.
Since information is propagated between nodes until no more information can be generated by the analysis algorithm it is vitally important that information is never lost. If a transformation provider or a lattice ever removes information from the current analysis state, the termination of the Monotone Solver can not be guaranteed and the code analysis algorithm might get stuck in an endless loop.
You can find an example implementation of a MonoREIL-based static code analysis algorithm in the scripts/mono/sample subdirectory of your BinNavi installation.