Building Zyrox: A Custom LLVM Obfuscator (Part II — Control-Flow Flattening)

In Part I, I introduced the foundational passes in Zyrox: MBA substitution, SIBR, and Basic Block Splitting. These techniques make static analysis harder, but they all share a limitation:

They rely on the control flow that already exists in the function.

If a function is mostly linear or contains very few branches, then SIBR can’t do much. MBA obfuscates arithmetic, but the overall structure remains the same. Even with BBS, decompilers and static analysis tools can often recover the high-level CFG, returning to a more understandable form.

To truly break the structure of a function, we need something stronger.

That something is Control-Flow Flattening (CFF).

Understanding Control-Flow Flattening

Control-Flow Flattening rewrites a function into a state machine. Instead of basic blocks branching directly to each other, they all route through a central block, the dispatcher. This destroys the original CFG shape and makes the function appear as a giant, opaque loop.

Before:

Normal control flow looks like this:

entry:
    if (x == 0) goto A;
    goto B;
A:
    ...
    goto end;
B:
    ...
end:expand code

After (conceptual, our obfuscator does more :^):

state = S0;
dispatch:
    if (state == S0) goto entry;
    if (state == S1) goto A;
    if (state == S2) goto B;
    goto end;
entry:
    state = S_next;
    goto dispatch;
A:
    state = S_next;
    goto dispatch;
B:
    state = S_next;
    goto dispatch;
end:expand code

Every block transitions into the dispatcher, which then decides what the next block should be.

While most open source obfuscators use a simple switch-case structure for the dispatcher, Zyrox employs multiple layers of complexity to thwart decompilers, and uses if-statements to allow using SBF and other passes more easily:

State values are large random 32/64-bit integers (depends on target architecture, native pointer size)
The dispatcher chain is built as a linked list of condition blocks, not a switch
States may be hashed with SipHash (more on this below)
Comparisons may go through a resolver function wrapper
States may be transformed through opaque arithmetic sequences (XOR, ADD, ROL, etc.)
Target state values may be stored in global variables

These layers stack to significantly increase complexity.

Implementation Overview

Here’s the high-level flow of Zyrox’s flattening pass:

Gather all basic blocks except the entry block
Assign each block a unique random state
Create a dispatcher_state variable
Generate a chain of "condition blocks", one for each original block
Each condition block:
- Loads dispatcher_state
- Optionally transforms it:
  - SipHash
  - Opaque arithmetic
- Compares it with the target state
- Branches to either:
  - The original block
  - Or the next condition block
At the end of each original block:
- Set dispatcher_state to the next block’s state
- Jump back to the dispatcher

Below is the core initialization from Zyrox:

AllocaInst *dispatcher_state =
    builder.CreateAlloca(int_ty, nullptr, "state");
builder.CreateStore(ConstantInt::get(int_ty, 0), dispatcher_state, true);expand code

Every block then receives:

BasicBlockUtils::AddMetaData(bb, "cff.dispatcher_state", dispatcher_state);
BasicBlockUtils::AddMetaData(bb, "cff.block_state", block_state_map[bb]);expand code

This metadata can be used in future passes that might be applied on top of state dispatching.

Building the Dispatcher

Zyrox constructs a “dispatcher block” followed by N condition blocks:

BasicBlock *dispatch_bb = BasicBlock::Create(ctx, "dispatch", &f);
builder.SetInsertPoint(dispatch_bb);
builder.CreateBr(condition_blocks.front());expand code

Each condition block:

Loads the current state
Optionally transforms it through SipHash or opaque operations
Compares against the block’s assigned state
Branches to the target block or the next condition block

Here’s what it looks like conceptually:

state_val = load dispatcher_state;

if (UseFunctionResolver)
    cmp = CallResolver(state_val);
else {
    MaybeTransformDispatcherState(state_val);
    cmp = (state_val == TransformedTargetState);
}

if cmp
    goto target_block;
else
    if (i < last)
        goto next_condition_block;
    else
        goto default_block;expand code

Rewriting the Original Blocks

Finally, Zyrox rewrites every terminator:

...
dispatcher_state = block_state_map[next_block];
goto dispatch_bb;expand code

...
              goto next_blockexpand code

builder.CreateStore(
    ConstantInt::get(int_ty, block_state_map[target]),
    dispatcher_state, true);
terminator->replaceAllUsesWith(builder.CreateBr(dispatch_bb));
terminator->eraseFromParent();expand code

Conditional branches get rewritten into two intermediate blocks:

true_state:
    store next_true_state
    br dispatch_bb

false_state:
    store next_false_state
    br dispatch_bbexpand code

This ensures all paths return to the dispatcher.

Optional Enhancements

Zyrox supports multiple options to make flattening harder to deobfuscate.

1. Function-Based State Resolver

Instead of comparing states inline:

cmp = (state_val == target_state);expand code

Zyrox may emit:

cmp = cff_resolve_state_check(state_val);expand code

Where the resolver is a new function:

bool __fastcall cff_resolve_state_check(unsigned int a1)
{
  unsigned __int64 v1; // rt0

  HIDWORD(v1) = __PAIR64__(HIWORD(a1), a1 << 16) >> 25;
  LODWORD(v1) = __ROL4__(a1, 16) << 7;
  return __ROL4__(v1 >> 22, 8) == 1674549860;
}expand code

This prevents pattern-based detection of comparisons.

2. Global Variable States

Instead of immediate values:

state == 0x1234ABCDexpand code

Zyrox may generate:

__state_1234ABCD = 0x1234ABCD;
state == Load(__state_1234ABCD);expand code

3. SipHash-Based State Obfuscation

This was... fun ;)

Peter, how do we implement SipHash in LLVM?

Is it secure?

We do not aim for cryptographic security here, but rather complexity and irreversibility.

And... right! LLVM does not have SipHash built-in, so I did what any other sane person would do: get it's IR and inject it into the module! (ok writing it down makes it seem not so sane, but it was fun):

const char *HashUtils::SipHashLlvmIR()
{
    return R"(
@.str = private unnamed_addr constant [38 x i8] c"mba:1 cff:1,40,70,0,70,0,0 sibr:1,100\00",
        section "llvm.metadata"
@.str.1 = private unnamed_addr constant [1 x i8]
            c"\00", section "llvm.metadata"
@llvm.global.annotations = appending global [1 x { ptr, ptr, ptr, i32, ptr }]
            [{ ptr, ptr, ptr, i32, ptr }
                  { ptr @___siphash, ptr @.str, ptr @.str.1, i32 70, ptr null }],
                        section "llvm.metadata"

define dso_local i64 @___siphash(i64 noundef %0, i64 noundef %1,
            i64 noundef %2, i64 noundef %3, i64 noundef %4,
                  i64 noundef %5, i64 noundef %6) #0 {
  %8 = xor i64 %6, %2
  %9 = xor i64 %5, %1
  %10 = xor i64 %4, %2
  %11 = xor i64 %3, %1
  %12 = xor i64 %8, %0
  br label %13
  ...expand code

To add it, we simply use linker API:

ModuleUtils::LinkModules(
    m, ModuleUtils::LoadFromIR(m.getContext(),
        HashUtils::SipHashLlvmIR()));

void ModuleUtils::LinkModules(Module &dst, std::unique_ptr<Module> src)
{
    src->setDataLayout(dst.getDataLayout());
    src->setTargetTriple(dst.getTargetTriple());
    if (Linker::linkModules(dst, std::move(src)))
    {
        Logger::Error("failed to link modules");
    }
}expand code

For every comparison, Zyrox may transform state into:

SipHash(state, random keys...)expand code

And then compare against a transformed target:

SipHash(original_target_state)expand code

output code would be something like:

if (SipHash(state) == hash_output_expected)expand code

Where SipHash is modified version of the real implementation, per invokation. This is far harder for decompilers to reason about.

This is the slowest path possible, but also one of the hardest to reverse engineer statically. it is not cryptographically secure to modify siphash like that, but it is more like customizing it per path. our goal is not protecting sensitive data, but doing irreversable (kinda) opaque transformation.

4. Opaque Arithmetic Transformation

Dispatcher state and target state can be transformed through random sequences:

The transformer simply transforms the value, then write the OPs it did with llvm builder. We get a value at compile time, and the expression at runtime to give us this value from the state as input.

int num_steps = Random::IntRanged(2, 6);
for (int i = 0; i < num_steps; i++)
{
    OpType t = static_cast<OpType>(Random::IntRanged(0, 4));
    uint64_t c = Random::IntRanged<uint64_t>(
        0x000F0000, m_isArm32 ? UINT32_MAX : UINT64_MAX);
    if (t == OpType::ROL || t == OpType::ROR)
    {
        c %= 31;
        c += 1;
    }
    m_Ops.push_back(t);
    m_Constants.push_back(c);
}expand code

These remain reversible (code below by @pitust):

from z3 import Solver, BitVec, Not
v22 = BitVec("v22", 32)
cond = (
    (
        ((((v22 ^ 0x41A0A73F) + 3399989) << 6) | (((v22 ^ 0x41A0A73F) + 3399989) >> 26))
        >> 21
    )
    | (
        ((((v22 ^ 0x41A0A73F) + 3399989) << 6) | (((v22 ^ 0x41A0A73F) + 3399989) >> 26))
        << 11
    )
) != 0x0F0EE10FA
s = Solver()
s.add(Not(cond))
print(s.check())
print(s.model())expand code

Before & After Examples

Let’s apply CFF to a simple function.

This can easily be worse ;)

__int64 sub_1F450()
{
...

  v3 = 0;
LABEL_14:
  if ( v3 < 10 )
    v4 = 1549216407;
  else
    v4 = 85336935;
  while ( 1 )
  {
    while ( 1 )
    {
      if ( (sub_1FB40(v4) & 1) != 0 )
        goto LABEL_8;
      v2 = ((v4 + dword_62FE0) << dword_62FF4) | ((v4 + dword_62FE0) >> dword_6300C);
      if ( ((((v2 >> 23) | (v2 << 9)) >> dword_62FDC) | (16 * ((v2 >> 23) | (v2 << 9)))) != 0xFFC26B4C )
        break;
      v5 = v13 - v14;
      v6 = ~(~v3 & 0xFFFFFFFD) * (v3 & 2) + (v3 & 0xFFFFFFFD) * (~v3 & 2);
      v7 = -v6;
      v8 = ~(v13 - v14);
      v4 = 1999881854;
    }
    if ( v4 == dword_62FD8 )
      return 8LL;
    if ( v4 == dword_63008 )
    {
      v0 = sub_20768(&qword_631D0, v9 + v10 + v11 * v12);
      sub_1FA94(v0, sub_1F9E0);
      v4 = 1434908897;
    }
    else
    {
      if ( (sub_1F7EC(v4) & 1) != 0 )
        goto LABEL_14;
      if ( ((v4 + 1470817521) ^ 0x60077E05) - dword_63004 + 1256162537 == dword_62FE4 )
      {
        v9 = (v8 & v7) + (v5 & ~v7) - (v8 & v7 & v5 & ~v7);
        v10 = ((v5 & -v6) + 2 - (v5 & -v6 & 2)) * (v5 & -v6 & 2);
        v11 = ~(v5 & -v6) & 2;
        v12 = v5 & -v6 & 0xFFFFFFFD;
        v4 = 940802622;
      }
      else if ( (sub_1F830(v4) & 1) != 0 )
      {
        v13 = -v3 - 8;
        v14 = (-v3 - 5 - ((-v3 - 7) & 2)) * ((-v3 - 7) & 2) + (~(-v3 - 7) & 2) * ((-v3 - 7) & 0xFFFFFFFD);
        v4 = 1142599164;
      }
      else
      {
LABEL_8:
        v3 = ((-v3 - 2) ^ ~(-v3 - 2)) & ~(-v3 - 2);
        v4 = 2002083585;
      }
    }
  }
}expand code

int __test_impl()
{
    for (int i = 0; i < 10; i++)
    {
        std::cout << i + 6 - i * 2 << std::endl;
    }
    return 5 + 3;
}expand code

Even this trivial function becomes:

JUMPOUT(0x1808LL);expand code

...or a series of unconnected blocks with no clear flow.

Results

Control-Flow Flattening is one of the strongest passes in Zyrox. Combined with MBA, BBS, and SIBR, a function’s structure becomes so fragmented that:

No recognizable control-flow graph remains
Decompilers struggle to detect loops
Branch patterns disappear
Symbolic execution becomes necessary to understand transitions

And most importantly:

The original high-level logic becomes extremely difficult to reconstruct.

More like this

You can check more blogs about Zyrox here.

And feel free to explore different blogs here.

... You can also check this page, which I wrote for a university project and fun.