Lifting Binaries, Part 0: Devirtualizing VMProtect and Themida: It's Just Flattening?
Table Of Contents
- Table Of Contents
- Intro
- Day 0
- Failed attempts
- Whiplash
- Challenges
- Action
- Whats next?
- Special thanks to
- Thanks for reading!
- Useful resources:
Intro
This is going to be the first part of multipart series in which I discuss about using compiler techniques for reverse engineering purposes. I want this first post to be more of a blog about why I decided on this approach and why I developed Mergen rather than focusing on the more technical aspects.
Let’s go back to day 0, where everything started.
Day 0
Commercial VM based obfuscators like VMProtect and Themida are considered as industry standards because of their black box implementation. Even though these solutions have been in this position for a long time and there are public projects that are able to deobfuscate a particular solution or a particular version of a solution, there are (or were) no public projects that can deobfuscate multiple versions or multiple solutions.
So bored enough to learn a new topic, stupid enough to make it wildly ambitious, I started by creating a basic executable that just returns the sum of two numbers, applied VMProtect to it, and started to mess around. When I looked at the function in IDA, I saw the function was getting replaced with a jump to VMProtect generated .???0 section with a weird looking code block.
.???0:000000014000AE7B push r10
.???0:000000014000AE7D pushfq
.???0:000000014000AE7E push rsi
.???0:000000014000AE7F bsf r10, r10
.???0:000000014000AE83 bts r10w, di
.???0:000000014000AE88 push r13
.???0:000000014000AE8A test cx, r15w
.???0:000000014000AE8E btc r10w, sp
.???0:000000014000AE93 cmc
.???0:000000014000AE94 push r8
.???0:000000014000AE96 stc
.???0:000000014000AE97 bswap r10
.???0:000000014000AE9A push rbx
.???0:000000014000AE9B bswap bx
.???0:000000014000AE9E sbb r10b, sil
.???0:000000014000AEA1 cmc
.???0:000000014000AEA2 push rdx
.???0:000000014000AEA3 btr bx, dx
.???0:000000014000AEA7 push r11
.???0:000000014000AEA9 clc
.???0:000000014000AEAA movzx r10d, bp
.???0:000000014000AEAE push rax
.???0:000000014000AEAF push r15
.???0:000000014000AEB1 sal rax, 5Ah
.???0:000000014000AEB5 push rbp
.???0:000000014000AEB6 push r9
.???0:000000014000AEB8 push rdi
.???0:000000014000AEB9 and ebp, 38D937E3h
.???0:000000014000AEBF push rcx
.???0:000000014000AEC0 push r14
.???0:000000014000AEC2 bts r9d, r9d
.???0:000000014000AEC6 btc rax, 57h ; 'W'
.???0:000000014000AECB push r12
.???0:000000014000AECD and r9b, 0CCh
.???0:000000014000AED1 shld r10w, r8w, 3Ah
.???0:000000014000AED7 add bpl, al
.???0:000000014000AEDA mov rax, 0
.???0:000000014000AEE4 push rax
.???0:000000014000AEE5 bswap rbx
.???0:000000014000AEE8 bsf bx, r14w
.???0:000000014000AEED mov rdi, [rsp+88h+arg_0]
.???0:000000014000AEF5 cmc
.???0:000000014000AEF6 shr bpl, cl
.???0:000000014000AEF9 bswap edi
.???0:000000014000AEFB add edi, 66931E79h
.???0:000000014000AF01 movzx rbp, r12w
.???0:000000014000AF05 or r11b, bpl
.???0:000000014000AF08 adc bl, r14b
.???0:000000014000AF0B bswap edi
.???0:000000014000AF0D inc r9b
.???0:000000014000AF10 sub edi, 6573517Fh
.???0:000000014000AF16 add rdi, rax
.???0:000000014000AF19 mov r10, 100000000h
.???0:000000014000AF23 bts r11, rdi
.???0:000000014000AF27 add rdi, r10
.???0:000000014000AF2A movsx r11, bx
.???0:000000014000AF2E mov rbx, rsp
.???0:000000014000AF31 movsx ebp, bp
.???0:000000014000AF34 btc r11w, r8w
.???0:000000014000AF39 mov bpl, r15b
.???0:000000014000AF3C sub rsp, 180h
.???0:000000014000AF43 bsf r9d, r15d
.???0:000000014000AF47 and bpl, 0D0h
.???0:000000014000AF4B movzx bp, r13b
.???0:000000014000AF50 and rsp, 0FFFFFFFFFFFFFFF0h
.???0:000000014000AF57
.???0:000000014000AF57 loc_14000AF57: ; CODE XREF: .???0:loc_14001D58B↓j
.???0:000000014000AF57 ; .???0:000000014001FFEC↓j ...
.???0:000000014000AF57 mov rbp, rdi
.???0:000000014000AF5A rcr r11w, 7Fh
.???0:000000014000AF5F xor r11w, 0E0Ch
.???0:000000014000AF65 xadd r9, r9
.???0:000000014000AF69 mov r9, 0
.???0:000000014000AF73 sub rbp, r9
.???0:000000014000AF76 dec r11b
.???0:000000014000AF79 or r11b, r11b
.???0:000000014000AF7C mov r9b, 0C2h
.???0:000000014000AF7F
.???0:000000014000AF7F loc_14000AF7F: ; DATA XREF: sub_14000AE7B:loc_14000AF7F↓o
.???0:000000014000AF7F lea r11, loc_14000AF7F
.???0:000000014000AF86 sar r9b, cl
.???0:000000014000AF89 bt r9, rdi
.???0:000000014000AF8D sub rdi, 4
.???0:000000014000AF94 ror r9b, cl
.???0:000000014000AF97 mov r9d, [rdi]
.???0:000000014000AF9A jmp loc_140049977
We can make sense of some instructions in this block, such as the red ones, which simply pushes the code into the (virtual) stack. So I debugged it in hopes of finding our numbers and sum of our numbers, and voila, we have the block that we do the addition operation. This is the block that does the addition:
.???0:0000000140020893 mov eax, [rbx]
.???0:0000000140020895 mov rsi, 42DB0376h
.???0:000000014002089C mov edx, [rbx+4]
.???0:000000014002089F xor si, r15w
.???0:00000001400208A3 test dl, cl
.???0:00000001400208A5 sub rbx, 4
.???0:00000001400208AC dec esi
.???0:00000001400208AE add eax, edx
.???0:00000001400208B0 movsx esi, bx
.???0:00000001400208B3 mov [rbx+8], eax
.???0:00000001400208B6 not si
.???0:00000001400208B9 xchg si, si
.???0:00000001400208BC bswap esi
.???0:00000001400208BE pushfq
.???0:00000001400208BF mov rsi, 6F022F06h
.???0:00000001400208C6 sal sil, cl
.???0:00000001400208C9 cmp r10b, sil
.???0:00000001400208CC pop qword ptr [rbx]
.???0:00000001400208CE xor sil, dl
.???0:00000001400208D1 sar si, cl
.???0:00000001400208D4 sub rdi, 4
.???0:00000001400208DB shl sil, 0E5h
.???0:00000001400208DF mov esi, [rdi]
.???0:00000001400208E1 stc
.???0:00000001400208E2 xor esi, ebp
.???0:00000001400208E4 jmp loc_14007A020
Red instruction loads the first operand, blue instruction loads the second, and purple will do the addition and store it in the slot. So essentially the handle is just this:
.???0:0000000140020893 mov eax, [rbx]
.???0:000000014002089C mov edx, [rbx+4]
.???0:00000001400208AE add eax, edx
.???0:00000001400208B3 mov [rbx+8], eax
You could also imagine that it kind of looks like this in high level implementation:
int arg1 = stack.top();
stack.pop();
int arg2 = stack.top();
stack.push_back(arg1 + arg2);
However, we don’t always have the luxury of attaching a debugger, tracing it, and analyzing each handler manually. Even though this was a good learning experience, we need to come up with a solution that requires less manual work.
Failed attempts
If we wanted to patch or change the behavior of the program, a good solution would be creating our own function. Initially, I had different approaches and different projects such as using Triton to uncover the control flow and apply existing optimizations in Triton and just pasting the code somewhere else in the binary. However, generating the output takes quite a while, and the quality of the output is not very good.
So I decided to use unicorn engine and apply my own optimizations. You can guess that it takes quite a while to implement optimization passes when you have 0 experience with compilers, so that project was also scratched.
I remember finding retdec and playing with it; while playing with it, I found a good friend that was also was interested into the project. However, retdec wasn’t sufficent for our purposes, and I wasn’t experienced enough to make the neccesary improvements to it.
Whiplash
The failed attempts made me even more determined to work on it. So I decided to get more experience and started working on a project that will lift assembly to LLVM IR. The idea was to create something like a tracer and optimize it with LLVM passes like the last project where I used unicorn engine, but instead of writing our own optimizations, LLVM would take care of it.
I also realized I could just use LLVM passes to find the next block, so we didn’t need to use unicorn engine either.
Using LLVM would introduce me to SSA (single static assignment) format, which allows us to read the output easier than asm because every variable is only defined once, making control flow and data dependencies easier to follow. Also, LLVM is maintained by other people, and it’s already a widely used project.
Assembly
add rax, rcx
sub rdx, rbx
sub rax, rdx
LLVM IR (SSA format)
%rax2 = add i64 %rax, %rcx
%rdx2 = sub i64 %rdx, %rbx
%rax3 = sub i64 %rax2, %rdx2
So the ability to produce assembly code, passes, SSA and active community made using LLVM a no-brainer.
Unlike other devirtualization oriented projects, we are going to create a generic approach that would work with everything. So no special VIP, VSP or any vm register will be tracked; we will take the assembly code and directly lift it to LLVM IR and apply optimizations. Our only assumption is the CPU is able to execute this code, so we keep track of the memory, registers and additionally assumptions that come with branches.
Challenges
In order for our project to “trace” our virtualized function correctly, there were several challenges such as:
- Partial reads between memory slots could not be resolved.
- Storing a value with higher bits symbolized and truncating again would confuse value-tracking.
This is because LLVM is a compiler, not a deobfuscator, a normal program wouldn’t use memory slots like this.
We need to implement these solutions:
- Custom aliasing system
- Pattern matching for memory tracking
I also encountered LLVM’s KnownBits class; basically, it tracks one’s and zero’s in a value and allows us to concretize values more easily. The idea is simple: we have KnownOnes and KnownZeros.
??01 represents 4 bits, KnownOnes is 0001 and KnownZeros is 0010, so we know the lower two bits and the rest is unknown. We can keep tracking the value if its bits get and’d with 0, or’d with 1, etc.
Action
After solving these challenges and implementing 60+ instructions, we could get the trace of instructions and optimize them; it was a huge breakthrough, even though it was taking 200+ seconds to trace a simple function.
This bottleneck was caused because we were creating a clone of our module and applying optimizations each time we come across a jump. So I came up with a solution that would optimize the instructions while lifting and making sure we don’t invalidate any necessary value. After doing this, our runtime dropped to 500~ ms, 400x speed up, an incredible milestone for our project!
The idea was to treat each instruction as a unique instruction. This gives us more room to analyze and modify the “original” function. Here, these graphs explain the overall idea of a VM.
If we flatten the control flow and reorder the blocks sequentially, its much easier to analyze.
Before jumping into VMProtect or Themida examples, let’s see how our lifter performs on this toy example.
#include <array>
#include <iostream>
#define VM_MAX_OPCODES 10
#define STACK_SIZE 10
enum VMOpcodes {
VM_NOP = 0,
VM_PUSH,
VM_POP,
VM_ADD,
VM_SUB,
VM_XOR,
VM_EXIT,
Invalid
};
struct VMopcode {
VMOpcodes opc;
int *v;
VMopcode(VMOpcodes opc = VM_NOP, int *v = nullptr) : opc(opc), v(v){};
};
class VMRegister {
public:
int value = 0;
VMRegister() : value(0){};
VMRegister(int v) : value(v) {}
VMRegister operator+(const VMRegister &RHS) const {
return VMRegister(this->value + RHS.value);
}
VMRegister operator-(const VMRegister &RHS) const {
return VMRegister(this->value - RHS.value);
}
VMRegister operator^(const VMRegister &RHS) const {
return VMRegister(this->value ^ RHS.value);
}
};
class VM {
public:
std::array<VMopcode, VM_MAX_OPCODES> &program;
int vsp = 0;
int vip = 0;
int vzf = 0;
VMRegister stack[STACK_SIZE]; // could be rewritten with std::stack
VM() = delete;
explicit VM(std::array<VMopcode, VM_MAX_OPCODES> &vmopcodes)
: program(vmopcodes){};
void execute() {
while (vip < VM_MAX_OPCODES) {
auto opcode = program[vip].opc;
auto arg = program[vip++].v;
switch (opcode) {
case VM_PUSH: {
stack[vsp++] = VMRegister(*arg);
break;
}
case VM_POP: {
*arg = stack[--vsp].value;
break;
}
case VM_ADD: {
auto RHS = stack[--vsp];
auto LHS = stack[--vsp];
stack[vsp++] = LHS + RHS;
break;
}
case VM_SUB: {
auto RHS = stack[--vsp];
auto LHS = stack[--vsp];
stack[vsp++] = LHS - RHS;
break;
}
case VM_XOR: {
auto RHS = stack[--vsp];
auto LHS = stack[--vsp];
stack[vsp++] = LHS ^ RHS;
break;
}
case VM_NOP: {
break;
}
case VM_EXIT: {
return; // Exit the execution loop
}
case Invalid:
break;
}
}
}
};
int callvm(int a, int b) {
// return (a ^ b) + b;
int v1;
std::array<VMopcode, VM_MAX_OPCODES> vmopcodes = {{{VM_PUSH, &a},
{VM_PUSH, &b},
{VM_XOR, nullptr},
{VM_PUSH, &b},
{VM_ADD, nullptr},
{VM_POP, &v1},
{VM_EXIT, nullptr}}};
VM ourvm(vmopcodes);
ourvm.execute();
return v1;
}If we call callvm control flow of this program somewhat looks like this:
and we just reorder the blocks to this:
The result after compiling -O3 and custom passes:
define i64 @main(i64 %rax, i64 %rcx, i64 %rdx, i64 %rbx, i64 %rsp, i64 %rbp, i64 %rsi, i64 %rdi, i64 %r8, i64 %r9, i64 %r10, i64 %r11, i64 %r12, i64 %r13, i64 %r14, i64 %r15, ptr nocapture readnone %TEB, ptr nocapture readnone %memory) local_unnamed_addr #0 {
entry:
%0 = trunc i64 %rdx to i32
%realxor-5368715542-84 = xor i64 %rdx, %rcx
%realxor-5368715542- = trunc i64 %realxor-5368715542-84 to i32
%realadd-5368715382- = add i32 %realxor-5368715542-, %0
%1 = zext i32 %realadd-5368715382- to i64
ret i64 %1
}
We can also throw this into a decompiler, but I believe LLVM IR format is easier to read because we don’t need to fix arguments. If we wanted to see this in IDA, it would look like this:
int __fastcall main(
__int64 rax,
__int64 rcx,
__int64 rdx,
__int64 rbx,
__int64 rsp,
__int64 rbp,
__int64 rsi,
__int64 rdi,
__int64 r8,
__int64 r9,
__int64 r10,
__int64 r11,
__int64 r12,
__int64 r13,
__int64 r14,
__int64 r15)
{
return (rdx ^ rcx) + rdx;
}
After that, I just needed to implement more instructions and lift flags appropriately for supporting VMProtect 3.8 and Themida.
For comparison, here is how a function protected by VMProtect 3.8 and Themida differ in terms of control flow and results.
int foo(int a, int b) {
return a + b;
}
VMProtect 3.8 control flow (39894 non-unique instructions):
Result after -O3:
define i64 @main(i64 %rax, i64 %rcx, i64 %rdx, i64 %rbx, i64 %rsp, i64 %rbp, i64 %rsi, i64 %rdi, i64 %r8, i64 %r9, i64 %r10, i64 %r11, i64 %r12, i64 %r13, i64 %r14, i64 %r15, ptr nocapture readnone %TEB, ptr nocapture writeonly %memory) local_unnamed_addr #0 {
entry:
%0 = trunc i64 %rdx to i32
%1 = trunc i64 %rcx to i32
%realadd-5370932149- = add i32 %0, %1
%3 = zext i32 %realadd-5370932149- to i64
ret i64 %3
}
Themida(Fish White) control flow (45138 non-unique instructions):
Result after -O3:
define i64 @main(i64 %rax, i64 %rcx, i64 %rdx, i64 %rbx, i64 %rsp, i64 %rbp, i64 %rsi, i64 %rdi, i64 %r8, i64 %r9, i64 %r10, i64 %r11, i64 %r12, i64 %r13, i64 %r14, i64 %r15, ptr nocapture readnone %TEB, ptr writeonly %memory) local_unnamed_addr #0 {
entry:
%0 = trunc i64 %rdx to i32
%1 = inttoptr i64 1375416 to ptr
store i32 %0, ptr %1, align 4
%2 = trunc i64 %rcx to i32
%3 = inttoptr i64 1375408 to ptr
store i32 %2, ptr %3, align 4
%4 = inttoptr i64 5368817293 to ptr
store i32 1, ptr %4, align 4
%5 = inttoptr i64 5368817402 to ptr
store i64 5368709120, ptr %5, align 4
%6 = inttoptr i64 5368817438 to ptr
store i64 4114, ptr %6, align 4
%7 = inttoptr i64 5368817092 to ptr
store i64 5368817438, ptr %7, align 4
%8 = inttoptr i64 5368817228 to ptr
store i64 0, ptr %8, align 4
%9 = inttoptr i64 5368817430 to ptr
store i64 5374802999, ptr %9, align 4
%10 = inttoptr i64 5368817305 to ptr
%11 = inttoptr i64 5368763106 to ptr
store i64 5368833026, ptr %11, align 4
%12 = inttoptr i64 5368763114 to ptr
store i64 5368833031, ptr %12, align 4
%13 = inttoptr i64 5368763122 to ptr
store i64 5369155158, ptr %13, align 4
%14 = inttoptr i64 5368763130 to ptr
store i64 5369155163, ptr %14, align 4
%15 = inttoptr i64 5368763138 to ptr
store i64 5368756062, ptr %15, align 4
%16 = inttoptr i64 5368763146 to ptr
store i64 5368756067, ptr %16, align 4
%17 = inttoptr i64 5368763154 to ptr
store i64 5369076960, ptr %17, align 4
%18 = inttoptr i64 5368763162 to ptr
store i64 5369076965, ptr %18, align 4
%19 = inttoptr i64 5368763170 to ptr
store i64 5368832102, ptr %19, align 4
%20 = inttoptr i64 5368763178 to ptr
store i64 5368832107, ptr %20, align 4
%21 = inttoptr i64 5368763186 to ptr
store i64 5368817820, ptr %21, align 4
%22 = inttoptr i64 5368763194 to ptr
store i64 5368817825, ptr %22, align 4
%23 = inttoptr i64 5368763202 to ptr
store i64 5369076312, ptr %23, align 4
%24 = inttoptr i64 5368763210 to ptr
store i64 5369076317, ptr %24, align 4
%25 = inttoptr i64 5368763218 to ptr
store i64 5369155148, ptr %25, align 4
%26 = inttoptr i64 5368763226 to ptr
store i64 5369155153, ptr %26, align 4
%27 = inttoptr i64 5368763234 to ptr
store i64 5368817454, ptr %27, align 4
%28 = inttoptr i64 5368763242 to ptr
store i64 5368817459, ptr %28, align 4
%29 = inttoptr i64 5368763250 to ptr
store i64 5368790594, ptr %29, align 4
%30 = inttoptr i64 5368763258 to ptr
store i64 5368790599, ptr %30, align 4
%31 = inttoptr i64 5368763266 to ptr
store i64 5368832092, ptr %31, align 4
%32 = inttoptr i64 5368763274 to ptr
store i64 5368832097, ptr %32, align 4
%33 = inttoptr i64 5368763282 to ptr
store i64 5368908672, ptr %33, align 4
%34 = inttoptr i64 5368763290 to ptr
store i64 5368908677, ptr %34, align 4
%35 = inttoptr i64 5368763298 to ptr
store i64 5369075736, ptr %35, align 4
%36 = inttoptr i64 5368763306 to ptr
store i64 5369075741, ptr %36, align 4
%37 = inttoptr i64 5368763314 to ptr
store i64 5368832032, ptr %37, align 4
%38 = inttoptr i64 5368763322 to ptr
store i64 5368832037, ptr %38, align 4
%39 = inttoptr i64 5368763330 to ptr
store i64 5369154176, ptr %39, align 4
%40 = inttoptr i64 5368763338 to ptr
store i64 5369154181, ptr %40, align 4
%41 = inttoptr i64 5368763346 to ptr
store i64 5368820454, ptr %41, align 4
%42 = inttoptr i64 5368763354 to ptr
store i64 5368820459, ptr %42, align 4
%43 = inttoptr i64 5368763362 to ptr
store i64 5368861334, ptr %43, align 4
%44 = inttoptr i64 5368763370 to ptr
store i64 5368861339, ptr %44, align 4
%45 = inttoptr i64 5368763378 to ptr
store i64 5368827548, ptr %45, align 4
%46 = inttoptr i64 5368763386 to ptr
store i64 5368827553, ptr %46, align 4
%47 = inttoptr i64 5368763394 to ptr
store i64 5368820444, ptr %47, align 4
%48 = inttoptr i64 5368763402 to ptr
store i64 5368820449, ptr %48, align 4
%49 = inttoptr i64 5368763410 to ptr
store i64 5368826458, ptr %49, align 4
%50 = inttoptr i64 5368763418 to ptr
store i64 5368826463, ptr %50, align 4
%51 = inttoptr i64 5368763426 to ptr
store i64 5368778594, ptr %51, align 4
%52 = inttoptr i64 5368763434 to ptr
store i64 5368778599, ptr %52, align 4
%53 = inttoptr i64 5368763442 to ptr
store i64 5369142368, ptr %53, align 4
%54 = inttoptr i64 5368763450 to ptr
store i64 5369142373, ptr %54, align 4
%55 = inttoptr i64 5368763458 to ptr
store i64 5368818708, ptr %55, align 4
%56 = inttoptr i64 5368763466 to ptr
store i64 5368818713, ptr %56, align 4
%57 = inttoptr i64 5368763474 to ptr
store i64 5368835506, ptr %57, align 4
%58 = inttoptr i64 5368763482 to ptr
store i64 5368835511, ptr %58, align 4
%59 = inttoptr i64 5368763490 to ptr
store i64 5368934020, ptr %59, align 4
%60 = inttoptr i64 5368763498 to ptr
store i64 5368934025, ptr %60, align 4
%61 = inttoptr i64 5368763506 to ptr
store i64 5369145336, ptr %61, align 4
%62 = inttoptr i64 5368763514 to ptr
store i64 5369145341, ptr %62, align 4
%63 = inttoptr i64 5368763522 to ptr
store i64 5368835496, ptr %63, align 4
%64 = inttoptr i64 5368763530 to ptr
store i64 5368835501, ptr %64, align 4
%65 = inttoptr i64 5368763538 to ptr
store i64 5369142890, ptr %65, align 4
%66 = inttoptr i64 5368763546 to ptr
store i64 5369142895, ptr %66, align 4
%67 = inttoptr i64 5368763554 to ptr
store i64 5369078460, ptr %67, align 4
%68 = inttoptr i64 5368763562 to ptr
store i64 5369078465, ptr %68, align 4
%69 = inttoptr i64 5368763570 to ptr
store i64 5368756212, ptr %69, align 4
%70 = inttoptr i64 5368763578 to ptr
store i64 5368756217, ptr %70, align 4
%71 = inttoptr i64 5368763586 to ptr
store i64 5368758102, ptr %71, align 4
%72 = inttoptr i64 5368763594 to ptr
store i64 5368758107, ptr %72, align 4
%73 = inttoptr i64 5368763602 to ptr
store i64 5368760802, ptr %73, align 4
%74 = inttoptr i64 5368763610 to ptr
store i64 5368760807, ptr %74, align 4
%75 = inttoptr i64 5368763618 to ptr
store i64 5368832062, ptr %75, align 4
%76 = inttoptr i64 5368763626 to ptr
store i64 5368832067, ptr %76, align 4
%77 = inttoptr i64 5368763634 to ptr
store i64 5369017510, ptr %77, align 4
%78 = inttoptr i64 5368763642 to ptr
store i64 5369017515, ptr %78, align 4
%79 = inttoptr i64 5368763650 to ptr
store i64 5368820904, ptr %79, align 4
%80 = inttoptr i64 5368763658 to ptr
store i64 5368820909, ptr %80, align 4
%81 = inttoptr i64 5368763666 to ptr
store i64 5368832052, ptr %81, align 4
%82 = inttoptr i64 5368763674 to ptr
store i64 5368832057, ptr %82, align 4
%83 = inttoptr i64 5368763682 to ptr
store i64 5368957214, ptr %83, align 4
%84 = inttoptr i64 5368763690 to ptr
store i64 5368957219, ptr %84, align 4
%85 = inttoptr i64 5368763698 to ptr
store i64 5368820924, ptr %85, align 4
%86 = inttoptr i64 5368763706 to ptr
store i64 5368820929, ptr %86, align 4
%87 = inttoptr i64 5368763714 to ptr
store i64 5368832082, ptr %87, align 4
%88 = inttoptr i64 5368763722 to ptr
store i64 5368832087, ptr %88, align 4
%89 = inttoptr i64 5368763730 to ptr
store i64 5369142770, ptr %89, align 4
%90 = inttoptr i64 5368763738 to ptr
store i64 5369142775, ptr %90, align 4
%91 = inttoptr i64 5368763746 to ptr
store i64 5368832042, ptr %91, align 4
%92 = inttoptr i64 5368763754 to ptr
store i64 5368832047, ptr %92, align 4
%93 = inttoptr i64 5368763762 to ptr
store i64 5368755982, ptr %93, align 4
%94 = inttoptr i64 5368763770 to ptr
store i64 5368755987, ptr %94, align 4
%95 = inttoptr i64 5368763778 to ptr
store i64 5368820914, ptr %95, align 4
%96 = inttoptr i64 5368763786 to ptr
store i64 5368820919, ptr %96, align 4
%97 = inttoptr i64 5368763794 to ptr
store i64 5368832072, ptr %97, align 4
%98 = inttoptr i64 5368763802 to ptr
store i64 5368832077, ptr %98, align 4
%99 = inttoptr i64 5368763810 to ptr
store i64 5369077210, ptr %99, align 4
%100 = inttoptr i64 5368763818 to ptr
store i64 5369077215, ptr %100, align 4
%101 = inttoptr i64 5368763826 to ptr
store i64 5369144346, ptr %101, align 4
%102 = inttoptr i64 5368763834 to ptr
store i64 5369144351, ptr %102, align 4
%103 = inttoptr i64 5368763842 to ptr
store i64 5368820464, ptr %103, align 4
%104 = inttoptr i64 5368763850 to ptr
store i64 5368820469, ptr %104, align 4
%105 = inttoptr i64 5368763858 to ptr
store i64 5369145426, ptr %105, align 4
%106 = inttoptr i64 5368763866 to ptr
store i64 5369145431, ptr %106, align 4
%107 = inttoptr i64 5368763874 to ptr
store i64 5368820804, ptr %107, align 4
%108 = inttoptr i64 5368763882 to ptr
store i64 5368820809, ptr %108, align 4
%109 = inttoptr i64 5368763890 to ptr
store i64 5369016970, ptr %109, align 4
%110 = inttoptr i64 5368763898 to ptr
store i64 5369016975, ptr %110, align 4
%111 = inttoptr i64 5368763906 to ptr
store i64 5369144830, ptr %111, align 4
%112 = inttoptr i64 5368763914 to ptr
store i64 5369144835, ptr %112, align 4
%113 = inttoptr i64 5368763922 to ptr
store i64 5368818798, ptr %113, align 4
%114 = inttoptr i64 5368763930 to ptr
store i64 5368818803, ptr %114, align 4
%115 = inttoptr i64 5368763938 to ptr
store i64 5368789714, ptr %115, align 4
%116 = inttoptr i64 5368763946 to ptr
store i64 5368789719, ptr %116, align 4
%117 = inttoptr i64 5368763954 to ptr
store i64 5368957264, ptr %117, align 4
%118 = inttoptr i64 5368763962 to ptr
store i64 5368957269, ptr %118, align 4
%119 = inttoptr i64 5368763970 to ptr
store i64 5368835566, ptr %119, align 4
%120 = inttoptr i64 5368763978 to ptr
store i64 5368835571, ptr %120, align 4
%121 = inttoptr i64 5368763986 to ptr
store i64 5368818110, ptr %121, align 4
%122 = inttoptr i64 5368763994 to ptr
store i64 5368818115, ptr %122, align 4
%123 = inttoptr i64 5368764002 to ptr
store i64 5369145276, ptr %123, align 4
%124 = inttoptr i64 5368764010 to ptr
store i64 5369145281, ptr %124, align 4
%125 = inttoptr i64 5368764018 to ptr
store i64 5368835576, ptr %125, align 4
%126 = inttoptr i64 5368764026 to ptr
store i64 5368835581, ptr %126, align 4
%127 = inttoptr i64 5368764034 to ptr
store i64 5368818350, ptr %127, align 4
%128 = inttoptr i64 5368764042 to ptr
store i64 5368818355, ptr %128, align 4
%129 = inttoptr i64 5368764050 to ptr
store i64 5368806124, ptr %129, align 4
%130 = inttoptr i64 5368764058 to ptr
store i64 5368806129, ptr %130, align 4
%131 = inttoptr i64 5368764066 to ptr
store i64 5369076710, ptr %131, align 4
%132 = inttoptr i64 5368764074 to ptr
store i64 5369076715, ptr %132, align 4
%133 = inttoptr i64 5368764082 to ptr
store i64 5369153390, ptr %133, align 4
%134 = inttoptr i64 5368764090 to ptr
store i64 5369153395, ptr %134, align 4
%135 = inttoptr i64 5368764098 to ptr
store i64 5369079940, ptr %135, align 4
%136 = inttoptr i64 5368764106 to ptr
store i64 5369079945, ptr %136, align 4
%137 = inttoptr i64 5368764114 to ptr
store i64 5369153380, ptr %137, align 4
%138 = inttoptr i64 5368764122 to ptr
store i64 5369153385, ptr %138, align 4
%139 = inttoptr i64 5368764130 to ptr
store i64 5368827868, ptr %139, align 4
%140 = inttoptr i64 5368764138 to ptr
store i64 5368827873, ptr %140, align 4
%141 = inttoptr i64 5368764146 to ptr
store i64 5368998858, ptr %141, align 4
%142 = inttoptr i64 5368764154 to ptr
store i64 5368998863, ptr %142, align 4
%143 = inttoptr i64 5368764162 to ptr
store i64 5369140266, ptr %143, align 4
%144 = inttoptr i64 5368764170 to ptr
store i64 5369140271, ptr %144, align 4
%145 = inttoptr i64 5368764178 to ptr
store i64 5368756022, ptr %145, align 4
%146 = inttoptr i64 5368764186 to ptr
store i64 5368756027, ptr %146, align 4
%147 = inttoptr i64 5368764194 to ptr
store i64 5368756052, ptr %147, align 4
%148 = inttoptr i64 5368764202 to ptr
store i64 5368756057, ptr %148, align 4
%149 = inttoptr i64 5368764210 to ptr
store i64 5369076630, ptr %149, align 4
%150 = inttoptr i64 5368764218 to ptr
store i64 5369076635, ptr %150, align 4
%151 = inttoptr i64 5368764226 to ptr
store i64 5368756012, ptr %151, align 4
%152 = inttoptr i64 5368764234 to ptr
store i64 5368756017, ptr %152, align 4
%153 = inttoptr i64 5368764242 to ptr
store i64 5368756042, ptr %153, align 4
%154 = inttoptr i64 5368764250 to ptr
store i64 5368756047, ptr %154, align 4
%155 = inttoptr i64 5368764258 to ptr
store i64 5369154586, ptr %155, align 4
%156 = inttoptr i64 5368764266 to ptr
store i64 5369154591, ptr %156, align 4
store i64 5368763106, ptr %10, align 4
%157 = inttoptr i64 5368817370 to ptr
%158 = inttoptr i64 5368817076 to ptr
%159 = inttoptr i64 5368817376 to ptr
store i32 0, ptr %159, align 4
%160 = inttoptr i64 5368817175 to ptr
%161 = inttoptr i64 5368817080 to ptr
%162 = inttoptr i64 5368817146 to ptr
%163 = inttoptr i64 5368817236 to ptr
store i16 0, ptr %163, align 2
%164 = inttoptr i64 5368817401 to ptr
%165 = inttoptr i64 5368817154 to ptr
%166 = inttoptr i64 5368817150 to ptr
%167 = inttoptr i64 5368817352 to ptr
%168 = inttoptr i64 5368817329 to ptr
%169 = inttoptr i64 5368817268 to ptr
%170 = inttoptr i64 5368817220 to ptr
%171 = inttoptr i64 5368817180 to ptr
%172 = inttoptr i64 5368817100 to ptr
%173 = inttoptr i64 5368817251 to ptr
%174 = inttoptr i64 5368817362 to ptr
store i64 %r8, ptr %174, align 4
%175 = inttoptr i64 5368817126 to ptr
store i64 %r9, ptr %175, align 4
%176 = inttoptr i64 5368817134 to ptr
store i64 %r10, ptr %176, align 4
%177 = inttoptr i64 5368817344 to ptr
store i64 %r11, ptr %177, align 4
%178 = inttoptr i64 5368817269 to ptr
store i64 %r12, ptr %178, align 4
%179 = inttoptr i64 5368817252 to ptr
store i64 %r13, ptr %179, align 4
%180 = inttoptr i64 5368817068 to ptr
store i64 %r14, ptr %180, align 4
%181 = inttoptr i64 5368817321 to ptr
store i64 %r15, ptr %181, align 4
%182 = inttoptr i64 5368817212 to ptr
store i64 %rdi, ptr %182, align 4
%183 = inttoptr i64 5368817393 to ptr
store i64 %rsi, ptr %183, align 4
%184 = inttoptr i64 5368817084 to ptr
store i64 0, ptr %184, align 4
%185 = inttoptr i64 5368817155 to ptr
store i64 %rbx, ptr %185, align 4
%186 = inttoptr i64 5368817332 to ptr
store i64 %rdx, ptr %186, align 4
%187 = inttoptr i64 5368817200 to ptr
%188 = inttoptr i64 5368817297 to ptr
%189 = inttoptr i64 5368817422 to ptr
%190 = inttoptr i64 5368817163 to ptr
%191 = inttoptr i64 5368817060 to ptr
%192 = inttoptr i64 5368817243 to ptr
%193 = inttoptr i64 5368817179 to ptr
%194 = inttoptr i64 5368817238 to ptr
%195 = inttoptr i64 5368817188 to ptr
%196 = inttoptr i64 5368817313 to ptr
%197 = inttoptr i64 5368817301 to ptr
store i64 1702573061, ptr %191, align 4
store i64 1375408, ptr %195, align 4
%198 = and i64 %rcx, 4294967295
%realadd-5369249311- = shl nuw nsw i64 %198, 1
%lsb656 = and i64 %realadd-5369249311-, 254
%pf1657 = mul nuw i64 %lsb656, 72340172838076673
%pf2658 = and i64 %pf1657, -9205322385119247872
%pf3659 = urem i64 %pf2658, 511
%pf4660 = shl nuw nsw i64 %pf3659, 2
%199 = and i64 %pf4660, 4
%200 = shl i64 %rcx, 1
%createrflag2-667 = and i64 %200, 16
%zeroflag669 = icmp eq i64 %198, 0
%createrflag2-670 = select i1 %zeroflag669, i64 64, i64 0
%201 = or disjoint i64 %createrflag2-670, %createrflag2-667
%202 = or disjoint i64 %201, %199
%creatingrflag672 = xor i64 %202, 518
%realadd-5369259651- = add nuw nsw i64 %creatingrflag672, 44
%realxor-5369259850- = xor i64 %realadd-5369259651-, 1136612388
%203 = inttoptr i64 5368817204 to ptr
%realadd-5368752762- = add nuw nsw i64 %realxor-5369259850-, 17999817424
%realadd-5368880833- = add i32 %0, %2
%add_cf1733 = icmp ult i32 %realadd-5368880833-, %2
%204 = zext i1 %add_cf1733 to i64
%lsb735 = and i32 %realadd-5368880833-, 255
%205 = zext nneg i32 %lsb735 to i64
%pf1736 = mul nuw i64 %205, 72340172838076673
%pf2737 = and i64 %pf1736, -9205322385119247871
%pf3738 = urem i64 %pf2737, 511
%pf4739 = shl nuw nsw i64 %pf3738, 2
%206 = and i64 %pf4739, 4
%lvalLowerNibble743 = and i32 %2, 15
%rvalLowerNibble = and i32 %0, 15
%add_sumLowerNibble744 = add nuw nsw i32 %rvalLowerNibble, %lvalLowerNibble743
%add_af745 = icmp ugt i32 %add_sumLowerNibble744, 15
%createrflag2-746 = select i1 %add_af745, i64 16, i64 0
%zeroflag748 = icmp eq i32 %realadd-5368880833-, 0
%createrflag2-749 = select i1 %zeroflag748, i64 64, i64 0
%207 = lshr i32 %realadd-5368880833-, 24
%208 = and i32 %207, 128
%createrflag2-751.masked = zext nneg i32 %208 to i64
%ofadd754 = xor i32 %realadd-5368880833-, %2
%ofadd1 = xor i32 %realadd-5368880833-, %0
%ofadd2 = and i32 %ofadd754, %ofadd1
%209 = lshr i32 %ofadd2, 20
%210 = and i32 %209, 2048
%createrflag2-755 = zext nneg i32 %210 to i64
%211 = or disjoint i64 %createrflag2-746, %204
%212 = or disjoint i64 %211, %createrflag2-751.masked
%213 = or disjoint i64 %212, %createrflag2-755
%214 = or disjoint i64 %213, %206
%215 = or disjoint i64 %214, %createrflag2-749
%creatingrflag756 = xor i64 %215, 518
%216 = zext i32 %realadd-5368880833- to i64
store i32 %realadd-5368880833-, ptr %187, align 4
store i32 0, ptr %203, align 4
store i64 %creatingrflag756, ptr %189, align 4
%realxor-5368894493- = xor i64 %realadd-5368752762-, 6126736
store i8 -3, ptr %193, align 1
store i8 101, ptr %194, align 1
store i64 5369144346, ptr %190, align 4
%realadd-5369240414- = add nsw i64 %216, -3404397706
%realsub-5369240475- = add nsw i64 %realadd-5369240414-, %realxor-5368894493-
store i64 %realsub-5369240475-, ptr %192, align 4
store i32 -2008313946, ptr %166, align 4
%realadd-5369249311-936 = shl nuw nsw i64 %216, 1
%lsb940 = and i64 %realadd-5369249311-936, 254
%pf1941 = mul nuw i64 %lsb940, 72340172838076673
%pf2942 = and i64 %pf1941, -9205322385119247872
%pf3943 = urem i64 %pf2942, 511
%pf4944 = shl nuw nsw i64 %pf3943, 2
%217 = and i64 %pf4944, 4
%createrflag2-951 = and i64 %realadd-5369249311-936, 16
%218 = or disjoint i64 %createrflag2-951, %217
%219 = or disjoint i64 %218, %createrflag2-749
%creatingrflag956 = xor i64 %219, 518
%realsub-5369249432-967 = add nuw nsw i64 %216, 1591862189
store i64 %realsub-5369249432-967, ptr %196, align 4
%realadd-5369259651-987 = add nuw nsw i64 %creatingrflag956, 44
%realxor-5369259850-989 = xor i64 %realadd-5369259651-987, 1136612388
%realadd-5369259926-994 = add nuw nsw i64 %realxor-5369259850-989, %realxor-5368894493-
store i32 %realadd-5368880833-, ptr %188, align 4
store i32 0, ptr %197, align 4
%realsub-5369033485-1831 = add nuw nsw i64 %realadd-5369259926-994, -49619084162
store i8 88, ptr %164, align 1
store i32 -1509841899, ptr %162, align 4
store i32 480435818, ptr %161, align 4
store i32 1317788444, ptr %160, align 4
store i8 53, ptr %168, align 1
store i8 59, ptr %169, align 1
store i8 0, ptr %173, align 1
store i64 4377670473, ptr %170, align 4
%realxor-5369031128-1967 = xor i64 %r8, 4506609098
%realadd-5369031141-1968 = add i64 %realsub-5369033485-1831, %realxor-5369031128-1967
store i64 %realadd-5369031141-1968, ptr %172, align 4
store i32 -2056377491, ptr %158, align 4
store i64 1375224, ptr %167, align 4
store i8 -35, ptr %165, align 1
%realsub-5369033485-2000 = add nuw nsw i64 %realadd-5369259926-994, -66205754248
store i64 %realsub-5369033485-2000, ptr %171, align 4
store i32 -410453815, ptr %157, align 4
store i64 5374748856, ptr %9, align 4
store i64 0, ptr %6, align 4
store i32 0, ptr %4, align 4
ret i64 %216
}After a little manual clean-up (cleaning up excess stores to .data section):
define range(i64 0, 4294967296) i64 @main(i64 %rax, i64 %rcx, i64 %rdx, i64 %rbx, i64 %rsp, i64 %rbp, i64 %rsi, i64 %rdi, i64 %r8, i64 %r9, i64 %r10, i64 %r11, i64 %r12, i64 %r13, i64 %r14, i64 %r15, ptr readnone captures(none) %TEB, ptr readnone captures(none) %memory) local_unnamed_addr #0 {
%0 = trunc i64 %rdx to i32
%1 = trunc i64 %rcx to i32
%realadd-5368880833- = add i32 %0, %1
%2 = zext i32 %realadd-5368880833- to i64
ret i64 %2
}
VM based obfuscators do not provide significantly stronger protection against static analysis compared to control flow flattening, as both techniques primarily aim to obscure control flow without altering the original program logic. However, bin2bin VM based solutions face additional challenges, such as reliably translating assembly instructions, reconstructing jump tables, and handling exceptions, which can introduce complexity without necessarily enhancing protection against static analysis. Additionally, the added layers of abstraction in VM based obfuscation often result in increased runtime overhead, making the protected program slower and less efficient.
Even though VM based obfuscation is more advanced than flattening, if a VM based obfuscator did not use dispatchers, and simply executed the handlers in sequence, it would be trivial to analyze it even without any specialized tools. That said, when combined with other protection mechanisms, VM based obfuscation remains a unique and valuable tool in software protection.
Whats next?
So, we’ve learned it’s possible to create a generic deobfuscator that even works with commercial VM based obfuscations. Even though there are several challenges, such as MBA (Mixed Boolean Arithmetics) and unrollable loops. Hopefully, we will talk about them in next posts.
In the next post, we will dive deeper into the technical details mentioned here and demonstrate devirtualization using Tigress examples.
Special thanks to
phage
sneed
Thanks for reading!
If you found this post helpful, consider supporting me on Github Sponsors!
Useful resources:
https://www.youtube.com/watch?v=9EgnstyACKA
https://www.msreverseengineering.com/blog/2014/6/23/vmprotect-part-0-basics
https://0xnobody.github.io/devirtualization-intro/
https://secret.club/2021/09/08/vmprotect-llvm-lifting-1.html
https://github.com/JonathanSalwan/VMProtect-devirtualization
https://blog.back.engineering/17/05/2021/
https://blog.thalium.re/posts/llvm-powered-devirtualization/