Skip to main content

Compiler Settings

Compiler settings govern the core behavior of the CodeDefender transformation pipeline. These include the instruction set assumptions used during lifting, how register volatility is handled across function calls, and how function call semantics are modeled in IR.

While these options provide advanced control for edge cases, the default settings are designed to be robust and safe for most use cases and typically do not require adjustment.

Lifter Settings​

These options configure how machine code is translated into the intermediate representation (IR), influencing the fidelity and granularity of the lifting process.

Calling Convention​

  • Default: WindowsAbi
  • Alternative: Conservative

This setting defines how the lifter models register volatility across function calls.

  • WindowsAbi assumes the standard Microsoft x64 calling convention.
  • Conservative treats all registers as volatile, preserving them across calls regardless of ABI. This is a safe fallback mode primarily useful when the target binary does not follow standard calling conventions or exhibits undefined behavior.
info

Most users should leave this set to WindowsAbi.

Set to WindowsAbi by default. You can switch this to Conservative which will preserve all registers across call boundaries. This is a safety feature that usually doesnt need to be enabled.

Lift Calls​

  • Default: enabled

Enables lifting of function calls into intermediate representation (IR). If disabled, call instructions are preserved as-is, outside the obfuscation pipeline—this significantly weakens protection.

Lifting a call requires allocating space in the transformed function for the original stack layout. Prior to each lifted call, a stack copy operation simulates the effect of real argument passing.

warning

Do not disable this unless you are absolutely sure you must

Max Stack Copy Size​

  • Default: 0x400 (1024 bytes)
info

Scope: This setting is only relevant when targeting kernel‑mode drivers or UEFI modules, where the available stack is extremely small and strictly bounded.

Windows kernel threads receive roughly three memory pages of stack (≈ 12–16 KB depending on platform). Running out of this space triggers an immediate bug check, so kernel code must avoid deep call chains and large stack allocations.

https://learn.microsoft.com/en-us/windows-hardware/drivers/kernel/using-the-kernel-stack

This safeguard prevents the compiler from emitting large stack‑copy operations that would overflow the kernel/UEFI stack at runtime while still allowing maximum obfuscation coverage where it is safe.

Optimization Settings​

These settings control a suite of optimization passes applied to the IR before code generation. All optimizations are run together in a fixed-point loop or for a defined number of iterations (see below). These passes ensure the IR is clean, minimal, and performant—while still preserving all obfuscation semantics.

warning

Disabling any of the optimization settings below is strongly discouraged unless you suspect a specific pass is causing instability or incorrect behavior.

If you encounter such an issue, please contact us with a detailed report. We place a high priority on the semantic correctness of our optimizations, as they are essential for robust and resilient deobfuscation efforts.

Iterations​

Defines how many times the optimization passes should run in sequence.

  • Default: 0 (run until convergence)
  • Recommendation: Leave this at 0 unless debugging optimization behavior.

Setting this to any other value besides 0 will run all enabled optimizations for that many iterations.

Constant Propagation​

Simplifies IR expressions by propagating known constant values through the IR graph.

  • Enabled by default

This pass eliminates trivially evaluable operations at compile time—e.g., folding x = 2 + 3 into x = 5. It ensures that unnecessary runtime computations are removed.

Instruction Combine​

Identifies and merges common subexpressions or redundant instruction sequences into more compact forms.

  • Enabled by default

This pass reduces instruction count and improves execution efficiency without compromising the complexity introduced by obfuscation layers. It operates after constant propagation to ensure expression folding opportunities are maximized.

Dead Code Elimination (DCE)​

Removes IR nodes that have no effect on program state—i.e., instructions with no outputs or side effects that are not referenced anywhere else.

  • Always enabled
  • Run in every optimization iteration

This is a core cleanup pass that trims unreachable or unused logic left behind by other transformations.

Prune Useless Basic Block Parameters​

Removes SSA parameters passed between basic blocks that are unused.

  • Always enabled
  • Recommendation: Do not disable unless debugging edge cases

This pass helps simplify IR function signatures by removing unused basic block parameters, which in turn enables more aggressive cleanup by downstream passes like Dead Code Elimination and Instruction Combine.

Assembler Settings​

These settings govern how the assembler generates instructions and organizes basic blocks within functions.

Shuffle Basic Blocks​

  • Enabled by default

This option randomizes the order of basic blocks within each function, significantly increasing the difficulty for attackers attempting to predict control flow after indirect branches.

By default, basic blocks are coalesced into a favored linear order, which attackers can exploit to infer control flow patterns. Enabling block shuffling breaks this predictability, strengthening control flow obfuscation and reducing pattern-based static analysis effectiveness.

Instruction Prefix​

Instructions that begin with a REX prefix (0x40–0x4F) can safely tolerate additional prefixes without altering their behavior. We take advantage of this by injecting harmless, printable prefix bytes—specifically characters in the ASCII range ‘A’ to ‘O’ (0x41–0x4F)—which overlap with valid REX prefixes.

This allows you to encode arbitrary prefix strings such as "BELAB", increasing binary entropy and confusing signature-based analysis tools, without affecting correctness.

  • Default: "" (prefix injection disabled)

Instruction Prefix Chance​

Specifies the probability (0–100%) that a given instruction will receive the configured prefix.

This value acts as a per-instruction random chance multiplier. For example:

  • 25 means ~25% of eligible instructions will be prefixed.
  • 0 disables prefix injection entirely, even if a prefix string is set.