Nick Fitzgerald

A Structure-Aware Fuzzing Experiment

Mon, 01 Jun 2026 00:00:00 -0700

Structure-aware fuzzing can better exercise the system under test (SUT) by crafting inputs in the format expected by the SUT, rather than throwing pseudorandom bytes against it. That is, it avoids “shallow” inputs that the SUT will reject early (for example, syntactically invalid source text when fuzzing a programming language’s compiler) and only produces inputs that go “deep” into the SUT (e.g. programs that type-check and exercise the mid-end optimizer and backend code generator). The Rust fuzzing ecosystem is largely built around cargo-fuzz and the libfuzzer-sys crate, which provides two methods for structure-aware fuzzing:

Generating structured inputs from scratch with the arbitrary crate
Mutating existing inputs from the fuzzer’s corpus in a structure-aware manner, thereby producing new structured inputs, via the fuzz_mutator! hook

While the two methods are not technically mutually exclusive, combining the two can be difficult and engineering resources are finite. So:

If we are only implementing one approach, is generation or mutation better?

To help answer this question, I implemented structure-aware generation and mutation of guaranteed-valid WebAssembly (Wasm) instruction sequences. This task is small enough to be easily understandable but large enough and real enough to (hopefully) be representative and applicable to other domains, or, at the very least, interesting.¹ To evaluate their effectiveness, I used Wasmtime as the SUT, libfuzzer-sys as the fuzzing engine driving everything, and then compared code coverage over time when using mutation-based fuzzing versus generation-based fuzzing.

Additionally, there are many ways we can generate pseudorandom WebAssembly instruction sequences. In this experiment, I’ve evaluated three methods:

Unconstrained instruction sequence generation followed by a fixup pass to ensure validity
Generating valid instructions in a forwards, bottom-up manner (from operands to operators)
Generating valid instructions in a backwards, top-down manner (from operators to operands)

In contrast, while there are surely many ways to mutate a given WebAssembly instruction sequence into a new, valid instruction sequence, I’ve only implemented one method: perform an arbitrary instruction insertion, deletion, or replacement, producing a new but probably-invalid instruction sequence, and then run the same fixup pass mentioned previously to ensure validity. This is the direct mutation-based equivalent of the first generation-based method.

Before continuing further, I want to disclose that I am the author of wasm-smith and mutatis, and a maintainer of Wasmtime, arbitrary, libfuzzer-sys, and cargo-fuzz. That is, while I am familiar with Wasm, fuzzing, fuzzing Wasm, and both the arbitrary and mutatis crates, I may also be propagating my own biases into these implementations.

Background

Generation-Based and Mutation-Based Fuzzing

A generation-based fuzzer uses a generator to create a pseudo-random test cases from scratch, feeds these into the system under test, and reports any failures to the user:

fn generation_based_fuzzing<T>(
    // A test-case generator.
    generator: impl Fn() -> T,
    // A function to run the system under test with a
    // generated test case, returning a result that
    // describes whether the run was successful or
    // not.
    run_system_under_test: impl Fn(&T) -> FuzzResult,
) {
    loop {
        // Generate an input.
        let input = generator();

        // Run the input through the system under test.
        let result = run_system_under_test(&input);

        // If the system crashed, panicked, failed an
        // assertion, violated an invariant, or etc...
        // then report that to the user.
        if let Err(failure) = result {
            report_to_user(&input, failure);
        }
    }
}

On the other hand, mutation-based fuzzers are given an initial corpus of inputs and create new inputs by mutating existing corpus members. They run each new input through the SUT, report failures the same as before, and if the new input was “interesting” (for example, exercised new code paths in the SUT that weren’t previously covered in any other input’s execution) then the new input is added into the corpus for use in future test iterations:

fn mutation_based_fuzzing<T>(
    // A corpus of test cases.
    corpus: &mut Corpus<T>,
    // A function to pseudo-randomly mutate an existing
    // input into a new input.
    mutate: impl Fn(&T) -> T,
    // A function to run an input in the system under
    // test, returning a result that describes whether
    // the run was successful or not.
    run_system_under_test: impl Fn(&T) -> FuzzResult,
) {
    loop {
        // Choose an old test case from the corpus.
        let old_input = corpus.choose_one();

        // Pseudo-randomly mutate that old test case,
        // creating a new one.
        let input = mutate(old_input);

        // Run the input through the system under test.
        let result = run_system_under_test(&input);

        // If the system crashed, panicked, failed an
        // assertion, violated an invariant, or etc...
        // then report that to the user.
        if let Err(failure) = result {
            report_to_user(&input, failure);
        }

        // If the input was interesting, for example if
        // it executed previously-unknown code paths,
        // then add it into the corpus for use in a
        // future iteration.
        if result.input_was_interesting() {
            corpus.insert(input);
        }
    }
}

The two approaches are not mutually exclusive and hybrid generation- and mutation-based fuzzers exist.

More resources:

Structure-Aware Fuzzing

Structure-unaware fuzzing will generate pseudorandom byte sequences and pass them directly to the SUT. If the SUT expects some sort of structured input, e.g. the source text for a programming language, it is likely that these byte sequences are invalid and will be rejected early by the SUT’s frontend. For example, when fuzzing a compiler, the input is rejected as syntactically invalid by the parser or rejected as semantically invalid by the type checker. This can be useful when hardening a tokenizer, parser, or type checker, but is less useful when hunting for misoptimization in the mid-end or bad instruction encoding in the backend because the inputs are unlikely to make it that far through the compiler’s pipeline.

Structure-aware fuzzing will produce inputs that match the SUT’s expected input format. Returning to the compiler-fuzzing example, structure-aware fuzzing lets us generate valid programs for the compiler, so we can exercise more of the mid-end and backend, rather than just the frontend.

Structure-aware fuzzing is often generation-based: for example using grammar-based fuzzing to generate pseudorandom strings from a given language grammar or language-specific tools like csmith and wasm-smith that generate C and WebAssembly programs respectively. But structure-aware fuzzing can also be mutation-based: libFuzzer’s custom mutator example implements a structure-aware mutator for zlib-compressed strings, where the raw input is decompressed, the decompressed data is mutated, and then the mutated data is recompressed to provide the new raw input. The mutator is aware of the SUT’s zlib-compressed input structure.

More resources:

The `arbitrary` Crate

The arbitrary crate helps Rust developers write custom structure-aware generators for fuzzing. It provides building blocks and abstractions for translating a raw byte sequence (usually from a fuzzing engine) into a structured type, effectively interpreting the raw bytes as a “DNA string” or set of predetermined choices for its decision tree. The library also provides a derive(Arbitrary) macro to automatically implement its functionality for a given type.

Because arbitrary is effectively implemented by combining decision trees, it is extremely easy to create imbalanced trees and unintentionally bias the distribution of generated test cases.

The `mutatis` Crate

The mutatis crate is, at a high-level, performing the same role for authoring structure-aware mutators that arbitrary plays for generators. That is, it provides Rust developers with abstractions and combinators for creating custom structure-aware mutators. It also provides a derive(Mutate) macro to automatically implement its functionality for a given type.

mutatis is designed to resist bias via a two-phase design: first, it enumerates all of the candidate mutations that could be applied to a test case, and only afterwards chooses a particular random mutation from the candidate set to actually apply.

WebAssembly

WebAssembly is a virtual instruction set designed to be safe, portable, and fast. It is a stack machine where an instruction’s operands are popped off a stack during execution and results pushed. It has sandboxed linear memories, global variables, and local variables (the latter two effectively being two kinds of virtual registers). The following instruction sequence computes a * 3 and stores the result into memory at address p:

;; []
local.get $p
;; [p]
local.get $a
;; [p, a]
i32.const 3
;; [p, a, 3]
i32.mul
;; [p, a*3]
i32.store
;; []

Generator and Mutator Implementation

The range of all three generators and the mutator is the same universe of WebAssembly programs. They are all implemented on top of the same Module and Inst types, and, given enough time, none is capable of producing an instruction sequence that another cannot. This helps ensure that our comparison is apples-to-apples. However, due to their different implementation techniques, they do produce different distributions of WebAssembly programs within that universe, and produce test cases at different speeds from one another, which ultimately affects how efficiently they exercise the SUT.

All of the generators are built on top of the arbitrary crate. The mutator is built on top of the mutatis crate.

The Module type is our structured fuzzing input. It describes a WebAssembly module containing a variable number of linear memories, a variable number and type of globals, and one function with a variable number and type of parameters and results and a variable instruction sequence:

/// A WebAssembly module of the shape:
///
///     (module
///       (memory ...)
///       (memory ...)
///       ...
///
///       (global ...)
///       (global ...)
///       ...
///
///       (func (export "run") (param ...) (result ...)
///         ...
///       )
///     )
pub struct Module {
    num_memories: u32,
    globals: Vec<Global>,
    param_types: Vec<ValType>,
    result_types: Vec<ValType>,
    instructions: Vec<Inst>,
}

The Inst type is an enum of all the WebAssembly instructions the implementations support, which is all of the integer, float, SIMD, memory, local, and global instructions. Control-flow, threading, table, and GC instructions are not supported. Here is a subset of Inst’s definition:

/// A WebAssembly instruction.
pub enum Inst {
    Drop,
    LocalGet(u32),
    GlobalGet(u32),

    // ...

    I32Const(i32),
    I32Add,
    I32Sub,
    I32Mul,

    // ...

    I64Const(i64),
    I64Add,
    I64Sub,
    I64Mul,

    // ...

    F32Const(f32),
    F32Add,
    F32Sub,
    F32Mul,

    // ...

    F64Const(f64),
    F64Add,
    F64Sub,
    F64Mul,

    // ...

    I32WrapI64,
    I64ExtendI32S,
    I64ExtendI32U,

    // ...

    V128Const(i128),
    I8x16Add,
    I8x16Sub,

    // ...

    I32Load(u32),
    I64Load(u32),

    // ...

    I32Store(u32),
    I64Store(u32),

    // ...

    MemorySize(u32),
    MemoryGrow(u32),
}

There is an Inst::operand_types method that returns the types that the instruction pops from the stack, and an Inst::result_type method that returns the type of the value that the instruction pushes onto the stack, if any. Finally, the Module::to_wasm_binary method encodes the module into WebAssembly’s binary format, so it can be fed into Wasmtime. These methods are used, directly or indirectly, in every generator and mutator implementation.

`arb`

The arb generator leverages derive(arbitrary::Arbitrary) on our structured input types to generate a pseudorandom instance of Module, unconstrained by validity. The module’s instruction sequence is almost certainly not valid at this point: it likely is missing operands for instructions, producing more results than the function’s signature describes, producing results of types that don’t match the function signature, accessing globals and locals that don’t exist, etc… Having produced an instance of Module, it next calls the Module::fixup method to mutate the Module so that it is valid.

The fixup method works by abstractly interpreting the instruction sequence to track the types of each value on the stack at every program point. Whenever an instruction’s operand types don’t match the types on top of the stack, it generates dummy values of the correct type. When the instructions produce more values than the function’s signature proscribes, it emits drop instructions.

impl Module {
    pub fn fixup(&mut self, mut make_value: impl FnMut() -> i64) {
        // ...

        // The fixed-up instructions.
        let mut fixed = Vec::with_capacity(
            self.instructions.len(),
        );

        // The types on the stack at any given program
        // point. Similar to the Wasm spec's appendix's
        // validation algorithm.
        let mut stack: Vec<ValType> = Vec::new();

        for inst in mem::take(&mut self.instructions) {
            // Special-case `drop` because it is
            // polymorphic.
            if matches!(inst, Inst::Drop) {
                if stack.is_empty() {
                    fixed.push(
                        ValType::I32.make_const(make_value()),
                    );
                } else {
                    stack.pop();
                }
                fixed.push(inst);
                continue;
            }

            // First clamp entity indices to valid
            // ranges.
            let Some(inst) = self.fixup_inst_immediates(
                &mut make_value,
                has_mutable_global,
                inst,
            ) else {
                continue
            };

            // Then make sure that the stack has
            // operands of the correct types for this
            // instruction.
            self.fixup_stack(
                &mut make_value,
                &mut fixed,
                &mut stack,
                &inst,
            );

            // Finally, apply the effects to the stack.
            let len_operands = inst.operand_types(
                &self.globals,
            ).len();
            stack.truncate(stack.len() - len_operands);
            stack.extend(inst.result_type(
                &self.param_types,
                &self.globals,
            ));

            fixed.push(inst);
        }

        // ...

        self.instructions = fixed;
    }

    fn fixup_stack(
        &mut self,
        mut make_value: impl FnMut() -> i64,
        fixed: &mut Vec<Inst>,
        stack: &mut Vec<ValType>,
        inst: &Inst,
    ) {
        let needed = inst.operand_types(&self.globals);
        let n = needed.len();

        if stack.len() >= n {
            if (0..n).all(|i| {
                stack[stack.len() - n + i] == needed[i]
            }) {
                // All needed operands are on the stack.
                return;
            }
        } else {
            if stack.iter().enumerate().all(|(i, ty)| {
                *ty == needed[i]
            }) {
                // A prefix of needed operands are on the
                // stack; make constants for the tail that
                // are missing.
                for ty in &needed[stack.len()..] {
                    fixed.push(ty.make_const(make_value()));
                    stack.push(*ty);
                }
                return;
            }
        }

        // Otherwise, just make constants for all the
        // needed operands.
        for ty in needed {
            fixed.push(ty.make_const(make_value()));
            stack.push(*ty);
        }
    }

    // ...
}

The fixup method also makes sure that for all instructions that have an immediate referencing some entity, the referenced entity is valid. For example, for a local.get $l instruction, it ensures that local $l actually exists or else rewrites the local to one that does exist.

impl Module {
    // ...

    fn fixup_inst_immediates(
        &mut self,
        mut make_value: impl FnMut() -> i64,
        has_mutable_global: bool,
        mut inst: Inst,
    ) -> Option<Inst> {
        match &mut inst {
            Inst::LocalGet(l) => *l %= self.param_types.len() as u32,

            // ...

            Inst::I32Load(m)
            | Inst::I64Load(m)
            | Inst::F32Load(m)
            | Inst::F64Load(m)
            | Inst::V128Load(m) => {
                if self.num_memories == 0 {
                    return None;
                }
                *m %= self.num_memories;
            }

            // ...

            _ => {}
        }

        Some(inst)
    }
}

After calling fixup, the arb generator invokes Module::to_wasm_binary to get the encoded Wasm program.

`bottom_up`

The bottom_up generator also uses abstract interpretation to track the types of values on the stack. It generates instructions in forwards order, from operands to operators. It begins with an empty stack, filters candidate instructions down to just those that would be valid given the types currently on the stack, randomly chooses one, updates the stack types accordingly, and repeats the process. This is the same approach that wasm-smith uses. After generating instructions this way, it then makes sure that the final types on the stack match the function signature’s results, similar to the end of fixup.

impl Module {
    pub fn bottom_up(u: &mut Unstructured<'_>) -> Result<Self> {
        // ...

        let max_insts = u.int_in_range(1..=MAX_INSTS)?;
        let mut instructions = Vec::new();
        let mut stack: Vec<ValType> = Vec::new();

        for _ in 0..max_insts {
            if stack == result_types && u.ratio(3, 4)? {
                break;
            }

            // Choose a random instruction whose operand
            // types match those currently on the stack.
            let inst = choose_inst_bottom_up(
                u,
                &stack,
                &param_types,
                &globals,
                num_memories,
            )?;

            // Apply this instruction's effects to the
            // stack.
            apply_inst(
                &inst,
                &mut stack,
                &param_types,
                &globals,
            );
            instructions.push(inst);
        }

        // ...

        Ok(Module {
            param_types,
            result_types,
            globals,
            num_memories,
            instructions,
        })
    }
}

fn choose_inst_bottom_up(
    u: &mut Unstructured<'_>,
    stack: &[ValType],
    param_types: &[ValType],
    globals: &[Global],
    num_memories: u32,
) -> Result<Inst> {
    // Build up all the valid candidate instructions.
    let mut candidates: Vec<Inst> = Vec::new();

    // Producers are always okay: [] -> [t]
    candidates.push(Inst::I32Const(0));
    candidates.push(Inst::I64Const(0));
    candidates.push(Inst::F32Const(0.0));
    candidates.push(Inst::F64Const(0.0));
    candidates.push(Inst::V128Const(0));
    if !param_types.is_empty() {
        candidates.push(Inst::LocalGet(0));
    }

    // ...

    let top = stack.last().copied();
    let second = stack.get(stack.len() - 2).copied();

    // Drop needs 1 operand of any type: [t] -> []
    if top.is_some() {
        candidates.push(Inst::Drop);
    }

    // i32 unary: [i32] -> [...]
    if top == Some(I32) {
        candidates.push(Inst::I32Clz);
        candidates.push(Inst::I32Ctz);
        candidates.push(Inst::I32Popcnt);
        // ...
    }

    // i64 unary: [i64] -> [...]
    if top == Some(I64) {
        candidates.push(Inst::I64Clz);
        candidates.push(Inst::I64Ctz);
        candidates.push(Inst::I64Popcnt);
        // ...
    }

    // ...

    // i32 binary: [i32 i32] -> [...]
    if top == Some(I32) && second == Some(I32) {
        candidates.push(Inst::I32Add);
        candidates.push(Inst::I32Sub);
        candidates.push(Inst::I32Mul);
        // ...
    }

    // i64 binary: [i64 i64] -> [...]
    if top == Some(I64) && second == Some(I64) {
        candidates.push(Inst::I64Add);
        candidates.push(Inst::I64Sub);
        candidates.push(Inst::I64Mul);
        // ...
    }

    // ...

    // Choose a random instruction from the
    // candidates.
    let mut inst = *u.choose(&candidates)?;

    // If the instruction has immediates, generate
    // them here, as they were hard-coded during
    // candidate selection.
    match &mut inst {
        Inst::I32Const(v) => *v = u.arbitrary()?,
        Inst::I64Const(v) => *v = u.arbitrary()?,
        // ...
        Inst::GlobalGet(g) => {
            *g = u.int_in_range(0..=(globals.len() as u32 - 1))?;
        }
        // ...
        Inst::I32Load(m)
        | Inst::I64Load(m)
        | Inst::F32Load(m)
        | Inst::F64Load(m)
        | Inst::V128Load(m)
        | Inst::I32Store(m)
        | Inst::I64Store(m)
        | Inst::F32Store(m)
        | Inst::F64Store(m)
        | Inst::V128Store(m)
        | Inst::MemorySize(m)
        | Inst::MemoryGrow(m) => {
            *m = u.int_in_range(0..=(num_memories - 1))?;
        }
        _ => {}
    }

    Ok(inst)
}

After constructing a Module via bottom_up, we don’t need to call fixup because the module is already valid by construction, so all that’s left is invoking Module::to_wasm_binary to get the encoded Wasm program.

`top_down`

The top_down generator is very similar to bottom_up, but instead of generating instructions forwards, from operands to operators, it generates them backwards, from operators to operands. Instead of maintaining a stack of the types of values generated thus far by the instruction sequence prefix, it maintains a stack of the types of values expected by the instruction sequence suffix. This is the approach that rgfuzz by Park, Kim, and Yun takes.²

impl Module {
    pub fn top_down(
        u: &mut Unstructured<'_>,
    ) -> Result<Self> {
        // ...

        let max_insts = u.int_in_range(1..=MAX_INSTS)?;
        let mut instructions = Vec::new();
        let mut needed = result_types.clone();
        for _ in 0..max_insts {
            if needed.is_empty() && u.ratio(3, 4)? {
                break;
            }

            // Choose a random instruction in a
            // top-down manner.
            let inst = choose_inst_top_down(
                u,
                needed.last().copied(),
                &param_types,
                &globals,
                num_memories,
            )?;

            // Pop the result type from `needed`, if
            // any, as it's been satisfied.
            let ty = inst.result_type(
                &param_types,
                &globals,
            );
            if ty == needed.last().copied() {
                needed.pop();
            }

            // Add operand type demands.
            match &inst {
                Inst::Drop => {
                    // `drop` is polymorphic; choose
                    // a random type.
                    needed.push(u.arbitrary()?);
                }
                Inst::GlobalSet(g) => {
                    needed.push(globals[*g as usize].ty);
                }
                _ => {
                    needed.extend_from_slice(
                        inst.operand_types(&globals),
                    );
                }
            }

            instructions.push(inst);
        }

        // Fill remaining needed types with
        // constants.
        for ty in needed.iter().rev() {
            instructions.push(
                ty.make_const(u.arbitrary()?),
            );
        }

        // Instructions were generated backwards, so
        // reverse.
        instructions.reverse();

        Ok(Module {
            param_types,
            result_types,
            globals,
            num_memories,
            instructions: prefix,
        })
    }
}

fn choose_inst_top_down(
    u: &mut Unstructured<'_>,
    target_ty: Option<ValType>,
    param_types: &[ValType],
    globals: &[Global],
    num_memories: u32,
) -> Result<Inst> {
    let mut candidates: Vec<Inst> = Vec::new();
    match target_ty {
        Some(I32) => {
            candidates.push(Inst::I32Const(0));
            candidates.push(Inst::I32Add);
            candidates.push(Inst::I32Sub);
            candidates.push(Inst::I32Mul);
            // ...
        }
        Some(I64) => {
            candidates.push(Inst::I64Const(0));
            candidates.push(Inst::I64Add);
            candidates.push(Inst::I64Sub);
            candidates.push(Inst::I64Mul);
            // ...
        }
        Some(F32) => {
            candidates.push(Inst::F32Const(0.0));
            candidates.push(Inst::F32Add);
            candidates.push(Inst::F32Sub);
            candidates.push(Inst::F32Mul);
            // ...
        }
        // ...
        None => {
            // Nothing needed. `drop`, `global.set`, and
            // stores add demand.
            candidates.push(Inst::Drop);
            if globals.iter().any(|g| g.mutable) {
                candidates.push(Inst::GlobalSet(0));
            }
            if num_memories > 0 {
                candidates.push(Inst::I32Store(0));
                // ...
            }
        }
    }

    let mut inst = *u.choose(&candidates)?;

    // If the instruction has immediates, generate
    // them here, as they were hard-coded during
    // candidate selection. Same as `bottom_up`.
    match &mut inst {
        // ...
    }

    Ok(inst)
}

Similar to bottom_up, after we’ve constructed a Module via top_down, we don’t need to call fixup because the module is already valid by construction. All that’s left is invoking Module::to_wasm_binary to get the encoded Wasm program.

`mutate`

mutate is, as the name implies, a mutator rather than a generator. It is the direct equivalent of the arb generator, but for mutation: it uses derive(mutatis::Mutate) on Module and Inst to automatically generate custom mutators for these types, rather than authoring them by hand. After producing a new Module by mutating an old Module, that new Module probably represents an invalid Wasm program, in the same way that derive(arbitrary::Arbitrary) produces Modules that are probably invalid. And mutate also uses the same approach that arb does to resolve this problem: the fixup method.

But first, a mutator-specific wrinkle is that fuzz_mutator! gives us a mutable byte slice to mutate, not a Module. We address this gap by deriving the serde crate’s Serialize and Deserialize traits on Module and Inst, deserializing a Module from the mutable byte slice, mutating that deserialized Module with mutatis, and then reserializing it back into the mutable byte slice. We use the postcard crate here, but could just as easily use bincode, JSON, or protobuf.

use libfuzzer_sys::{fuzz_mutator, fuzz_target, fuzzer_mutate};

fuzz_mutator!(|
    data: &mut [u8],
    size: usize,
    max_size: usize,
    seed: u32,
| {
    // With probability of about 1/8, use default
    // mutator.
    if seed.count_ones() % 8 == 0 {
        return fuzzer_mutate(data, size, max_size);
    }

    // Try to decode using postcard; fallback to
    // default input on failure.
    let mut module: Module =
        postcard::from_bytes(&data[..size])
            .ok()
            .unwrap_or_default();

    // Mutate with `mutatis`.
    let mut session = mutatis::Session::new()
        .seed(seed.into())
        .shrink(max_size < size);
    if session.mutate(&mut module).is_ok() {
        if let Ok(encoded) = postcard::to_slice(
            &module,
            data,
        ) {
            return encoded.len();
        }
    }

    // Fallback to the default libfuzzer mutator if
    // serialization or mutation fails because, for
    // example, `data` doesn't have enough capacity.
    fuzzer_mutate(data, size, max_size)
});

Finally, the fuzz target itself deserializes the Module from the raw bytes, calls fixup, encodes it to a Wasm binary via Module::to_wasm_binary, and then passes that into Wasmtime.

fuzz_target!(|data: &[u8]| {
    let Ok(mut module) = postcard::from_bytes::<Module>(data) else {
        return;
    };
    module.fixup(|| 0);
    let wasm = module.to_wasm_binary();

    // ...
});

Benchmarking

Methodology

We pair each of our generators and mutator with libfuzzer-sys and feed the resulting test cases into Wasmtime. All fuzzers start with an empty corpus.

The most important metric for a fuzzer is its bug-finding ability, but that can be difficult to measure directly. For example, Wasmtime is actively fuzzed 24/7 with more-complete fuzzers than those implemented here, so, as expected, I have not found any bugs via these benchmarks. Therefore, instead of reporting a found-bugs count, the benchmark harness reports two alternative metrics:

Coverage over time: Coverage is the cumulative code paths exercised by the fuzzer. A fuzzer cannot find bugs in code paths it does not cover. This is the most important metric reported.
Executions over time: An execution is one iteration of the fuzzing loop. This is basically measuring how fast the fuzzer can produce test cases. All else being equal, more executions is better, but all else is rarely equal. It is easy to generate poor test cases very quickly: just return an empty sequence of Wasm instructions every time. Unfortunately, that exclusively leads to useless executions. Therefore, this metric is really only useful when comparing two implementations of the same algorithm, and I’ve omitted its results in the next section.

Additionally, I report results for both 24 hours of fuzzing and 5 minutes of fuzzing. The expected behavior of long-term fuzzing, e.g. 24/7 fuzzing in OSS-Fuzz, can be extrapolated from the 24-hour results. The 5-minute results show the expected behavior of short-term fuzzing, e.g. when using mutatis::check or arbtest.

Discussion of short-term fuzzing is somewhat rare, so I feel its motivation deserves explanation. I find short-term fuzzing useful in the following scenarios, for example:

Running a quick fuzzing session locally, to catch bugs that avoid detection in the traditional unit- and integration-test suites, before opening a pull request.
Running some quick fuzzing in CI before allowing a pull request to merge, for similar reasons.

That is, short-term fuzzing is useful for the same reasons and in the same scenarios as property-based testing.³

As recommended in Evaluating Fuzz Testing by Klees, Ruef, Cooper, Wei, and Hicks and adopted in Fuzz Bench: An Open Fuzzer Benchmarking Platform and Service by Metzman, Szekeres, Simon, Sprabery, and Arya, the benchmark harness tests the statistical significance of its results with a Mann-Whitney U-test. The harness performs 20 trials per fuzzer, the same number of trials as Fuzz Bench.

Results

24 Hours of Fuzzing

arb has 1.00 ± 0.00 times more coverage than bottom_up (p = 0.01)
mutate has 1.01 ± 0.00 times more coverage than arb (p = 0.00)
top_down has 1.00 ± 0.00 times more coverage than arb (p = 0.00)
mutate has 1.02 ± 0.00 times more coverage than bottom_up (p = 0.00)
top_down has 1.01 ± 0.00 times more coverage than bottom_up (p = 0.00)
mutate has 1.01 ± 0.00 times more coverage than top_down (p = 0.00)

5 Minutes of Fuzzing

bottom_up has 1.01 ± 0.01 times more coverage than arb (p = 0.04)
mutate has 1.47 ± 0.02 times more coverage than arb (p = 0.00)
top_down has 1.06 ± 0.02 times more coverage than arb (p = 0.00)
mutate has 1.45 ± 0.01 times more coverage than bottom_up (p = 0.00)
top_down has 1.05 ± 0.02 times more coverage than bottom_up (p = 0.00)
mutate has 1.38 ± 0.02 times more coverage than top_down (p = 0.00)

Conclusion

The mutate fuzzer performs best. It vastly outperforms all the others at 5 minutes of fuzzing (36-49% more coverage), and while the rest narrow that gap after 24 hours of fuzzing, mutate maintains its lead (1-2% more coverage).

The comparison between arb and mutate is as apples-to-apples of a comparison as it gets between idiomatic test-case generation and mutation in Rust: derive(Arbitrary) and derive(Mutate). They use the same fixup method to ensure that the resulting Wasm instructions are valid. The fuzzer built with mutatis and test-case mutation provides better coverage over time than the fuzzer built with arbitrary and test-case generation. When writing structure-aware fuzzers, I used to reach for arbitrary; in the future, I will reach for mutatis instead.

The top_down fuzzer performs second-best, and is best of the generation-based fuzzers. This aligns with results from the rgfuzz paper, which found that top-down Wasm instruction generation resulted in better instruction diversity than bottom-up generation. This result is intuitive, they point out, because Wasm instructions tend to have more operands than results, which means that more candidates are filtered out from consideration when generating instructions in forward order from operands to results (bottom-up) than when generating them in backward order from results to operands (top-down).

Subjectively, none of the approaches feel significantly more-complicated nor easier to implement than the others. All approaches require a stack of types, representing the generated Wasm’s operand stack, at some point in their implementation. Some require it during instruction generation (top_down and bottom_up) while others require it during fixup (mutate and arb). Adding support for new Wasm instructions is roughly the same in all of them: add a new variant to enum Inst and define its operand and result types. top_down and bottom_up additionally require adding a line for the new instruction in their choose_inst_{top_down,bottom_up} functions, but this could be avoided with some targeted macro_rules! sugar.

The fixup method fixes instructions in a forwards order; as future work, it would be interesting to implement a backwards_fixup method that fixes instructions in a backwards order and see if mutate and backwards_fixup outperforms the current mutate and forwards fixup the same way that backwards generation (top_down) outperforms forwards generation (bottom_up).

fixup makes an attempt to reuse stack operands when it can, rather than synthesize dummy constants or drop already-computed values, but the attempt is somewhat half-hearted. Dropping operands introduces dead code, which is not very interesting for exercising deep into the compiler pipeline. Dummy constants are not that interesting either. Therefore, another potential line of follow-up work would be to investigate ways to maximize operand reuse and minimize drops and dummy constants inserted while ensuring validity. That could include storing values to memory or globals instead of droping them when possible. It could even include liberating ourselves from the stack-focused paradigm we’ve had thus far.

WebAssembly is a stack-based language and so it is natural that our approaches have focused on producing stack-y code. But, in practice, optimizing WebAssembly compilers like Wasmtime’s use a static single-assignment intermediate representation, and erase the operand stack early in their compilation pipelines. Therefore, from these compilers’ point of view, the following two WebAssembly snippets are identical:

;; `x = a + (b * c)` in a "stack-y" encoding and
;; without temporary locals.
local.get $a
local.get $b
local.get $c
i32.mul
i32.add
local.set $x

;; `x = a + (b * c)` in a "non-stack-y" encoding
;; that uses temporary locals for every operation.
;;
;; Equivalent of
;;
;;     temp0 = b * c
;;     temp1 = a + temp0
;;     x = temp1
local.get $b
local.get $c
i32.mul
local.set $temp0
local.get $a
local.get $temp0
i32.add
local.set $temp1
local.get $temp1
local.set $x

Producing code that uses many temporaries in this manner might be easier than code that doesn’t, but, more importantly, it may enable better reuse of already-computed subexpressions, emit less dead code, and ultimately produce more interesting data-flow graphs that better exercise the deep innards of the compiler.

A final vein of interesting follow-up work to mine would be comparing arbitrary-based generators and mutatis-based mutators for structured inputs that are not programming languages and when the SUT we are fuzzing is not a compiler. Do we see these same results when, for example, producing PNG images to fuzz an image-transformation library?

Here is the source code for this experiment, including the three generators, one mutator, raw benchmark data, and benchmarking harness. The README includes instructions on running the benchmarks yourself.

WebAssembly’s stack-based instructions encode an expression tree — local.get $a; local.get $b; local.get $c; i32.add; i32.mul is isomorphic to a * (b + c) — so the experiment should be relevant and applicable to any other generator or mutator for a programming language with expressions, even if it might not appear so at first glance. ↩
Ignoring its rule-guided bit, which is orthogonal and could be applied to bottom_up as well. ↩
Structure-aware fuzzing and property-based testing are basically the same: convergent evolution from different communities. ↩

A Function Inliner for Wasmtime and Cranelift

Wed, 19 Nov 2025 00:00:00 -0800

Note: I cross-posted this to the Bytecode Alliance blog.

Function inlining is one of the most important compiler optimizations, not because of its direct effects, but because of the follow-up optimizations it unlocks. It may reveal, for example, that an otherwise-unknown function parameter value is bound to a constant argument, which makes a conditional branch unconditional, which in turn exposes that the function will always return the same value. Inlining is the catalyst of modern compiler optimization.

Wasmtime is a WebAssembly runtime that focuses on safety and fast Wasm execution. But despite that focus on speed, Wasmtime has historically chosen not to perform inlining in its optimizing compiler backend, Cranelift. There were two reasons for this surprising decision: first, Cranelift is a per-function compiler designed such that Wasmtime can compile all of a Wasm module’s functions in parallel. Inlining is inter-procedural and requires synchronization between function compilations; that synchronization reduces parallelism. Second, Wasm modules are generally produced by an optimizing toolchain, like LLVM, that already did all the beneficial inlining. Any calls remaining in the module will not benefit from inlining — perhaps they are on slow paths marked [[unlikely]] or the callee is annotated with #[inline(never)]. But WebAssembly’s component model changes this calculus.

With the component model, developers can compose multiple Wasm modules — each produced by different toolchains — into a single program. Those toolchains only had a local view of the call graph, limited to their own module, and they couldn’t see cross-module or fused adapter function definitions. None of them, therefore, had an opportunity to inline calls to such functions. Only the Wasm runtime’s compiler, which has the final, complete call graph and function definitions in hand, has that opportunity.

Therefore we implemented function inlining in Wasmtime and Cranelift. Its initial implementation landed in Wasmtime version 36, however, it remains off-by-default and is still baking. You can test it out via the -C inlining=y command-line flag or the wasmtime::Config::compiler_inlining method. The rest of this article describes function inlining in more detail, digs into the guts of our implementation and rationale for its design choices, and finally looks at some early performance results.

Function Inlining

Function inlining is a compiler optimization where a call to a function f is replaced by a copy of f’s body. This removes function call overheads (spilling caller-save registers, setting up the call frame, etc…) which can be beneficial on its own. But inlining’s main benefits are indirect: it enables subsequent optimization of f’s body in the context of the call site. That context is important — a parameter’s previously unknown value might be bound to a constant argument and exposing that to the optimizer might cascade into a large code clean up.

Consider the following example, where function g calls function f:

fn f(x: u32) -> bool {
    return x < u32::MAX / 2;
}

fn g() -> u32 {
    let a = 42;
    if f(a) {
        return a;
    } else {
        return 0;
    }
}

After inlining the call to f, function g looks something like this:

fn g() -> u32 {
    let a = 42;

    let x = a;
    let f_result = x < u32::MAX / 2;

    if f_result {
        return a;
    } else {
        return 0;
    }
}

Now the whole subexpression that defines f_result only depends on constant values, so the optimizer can replace that subexpression with its known value:

fn g() -> u32 {
    let a = 42;

    let f_result = true;
    if f_result {
        return a;
    } else {
        return 0;
    }
}

This reveals that the if-else conditional will, in fact, unconditionally transfer control to the consequent, and g can be simplified into the following:

fn g() -> u32 {
    let a = 42;
    return a;
}

In isolation, inlining f was a marginal transformation. When considered holistically, however, it unlocked a plethora of subsequent simplifications that ultimately led to g returning a constant value rather than computing anything at run-time.

Implementation

Cranelift’s unit of compilation is a single function, which Wasmtime leverages to compile each function in a Wasm module in parallel, speeding up compile times on multi-core systems. But inlining a function at a particular call site requires that function’s definition, which implies parallelism-hurting synchronization or some other compromise, like additional read-only copies of function bodies. So this was the first goal of our implementation: to preserve as much parallelism as possible.

Additionally, although Cranelift is primarily developed for Wasmtime by Wasmtime’s developers, it is independent from Wasmtime. It is a reusable library and is reused, for example, by the Rust project as an alternative backend for rustc. But a large part of inlining, in practice, are the heuristics for deciding when inlining a call is likely beneficial, and those heuristics can be domain specific. Wasmtime generally wants to leave most calls out-of-line, inlining only cross-module calls, while rustc wants something much more aggressive to boil away its Iterator combinators and the like. So our second implementation goal was to separate how we inline a function call from the decision of whether to inline that call.

These goals led us to a layered design where Cranelift has an optional inlining pass, but the Cranelift embedder (e.g. Wasmtime) must provide a callback to it. The inlining pass invokes the callback for each call site, the callback returns a command of either “leave the call as-is” or “here is a function body, replace the call with it”. Cranelift is responsible for the inlining transformation and the embedder is responsible for deciding whether to inline a function call and, if so, getting that function’s body (along with whatever synchronization that requires).

The mechanics of the inlining transformation — wiring arguments to parameters, renaming values, and copying instructions and basic blocks into the caller — are, well, mechanical. Cranelift makes extensive uses of arenas for various entities in its IR, and we begin by appending the callee’s arenas to the caller’s arenas, renaming entity references from the callee’s arena indices to their new indices in the caller’s arenas as we do so. Next we copy the callee’s block layout into the caller and replace the original call instruction with a jump to the caller’s inlined version of the callee’s entry block. Cranelift uses block parameters, rather than phi nodes, so the call arguments simply become jump arguments. Finally, we translate each instruction from the callee into the caller. This is done via a pre-order traversal to ensure that we process value definitions before value uses, simplifying instruction operand rewriting. The changes to Wasmtime’s compilation orchestration are more interesting.

The following pseudocode describes Wasmtime’s compilation orchestration before Cranelift gained an inlining pass and also when inlining is disabled:

// Compile each function in parallel.
let objects = parallel map for func in wasm.functions {
    compile(func)
};

// Combine the functions into one region of executable memory, resolving
// relocations by mapping function references to PC-relative offsets.
return link(objects)

The naive way to update that process to use Cranelift’s inlining pass might look something like this:

// Optionally perform some pre-inlining optimizations in parallel.
parallel for func in wasm.functions {
    pre_optimize(func);
}

// Do inlining sequentially.
for func in wasm.functions {
    func.inline(|f| if should_inline(f) {
        Some(wasm.functions[f])
    } else {
        None
    })
}

// And then proceed as before.
let objects = parallel map for func in wasm.functions {
    compile(func)
};
return link(objects)

Inlining is performed sequentially, rather than in parallel, which is a bummer. But if we tried to make that loop parallel by logically running each function’s inlining pass in its own thread, then a callee function we are inlining might or might not have had its transitive function calls inlined already depending on the whims of the scheduler. That leads to non-deterministic output, and our compilation must be deterministic, so it’s a non-starter.¹ But whether a function has already had transitive inlining done or not leads to another problem.

With this naive approach, we are either limited to one layer of inlining or else potentially duplicating inlining effort, repeatedly inlining e into f each time we inline f into g, h, and i. This is because f may come before or after g in our wasm.functions list. We would prefer it if f already contained e and was already optimized accordingly, so that every caller of f didn’t have to redo that same work when inlining calls to f.

This suggests we should topologically sort our functions based on their call graph, so that we inline in a bottom-up manner, from leaf functions (those that do not call any others) towards root functions (those that are not called by any others, typically main and other top-level exported functions). Given a topological sort, we know that whenever we are inlining f into g either (a) f has already had its own inlining done or (b) f and g participate in a cycle. Case (a) is ideal: we aren’t repeating any work because it’s already been done. Case (b), when we find cycles, means that f and g are mutually recursive. We cannot fully inline recursive calls in general (just as you cannot fully unroll a loop in general) so we will simply avoid inlining these calls.² So topological sort avoids repeating work, but our inlining phase is still sequential.

At the heart of our proposed topological sort is a call graph traversal that visits callees before callers. To parallelize inlining, you could imagine that, while traversing the call graph, we track how many still-uninlined callees each caller function has. Then we batch all functions whose associated counts are currently zero (i.e. they aren’t waiting on anything else to be inlined first) into a layer and process them in parallel. Next, we decrement each of their callers’ counts and collect the next layer of ready-to-go functions, continuing until all functions have been processed.

let call_graph = CallGraph::new(wasm.functions);

let counts = { f: call_graph.num_callees_of(f) for f in wasm.functions };

let layer = [ f for f in wasm.functions if counts[f] == 0 ];
while layer is not empty {
    parallel for func in layer {
        func.inline(...);
    }

    let next_layer = [];
    for func in layer {
        for caller in call_graph.callers_of(func) {
            counts[caller] -= 1;
            if counts[caller] == 0 {
                next_layer.push(caller)
            }
        }
    }
    layer = next_layer;
}

This algorithm will leverage available parallelism, and it avoids repeating work via the same dependency-based scheduling that topological sorting did, but it has a flaw. It will not terminate when it encounters recursion cycles in the call graph. If function f calls function g which also calls f, for example, then it will not schedule either of them into a layer because they are both waiting for the other to be processed first. One way we can avoid this problem is by avoiding cycles.

If you partition a graph’s nodes into disjoint sets, where each set contains every node reachable from every other node in that set, you get that graph’s strongly-connected components (SCCs). If a node does not participate in a cycle, then it will be in its own singleton SCC. The members of a cycle, on the other hand, will all be grouped into the same SCC, since those nodes are all reachable from each other.

In the following example, the dotted boxes designate the graph’s SCCs:

Ignoring edges between nodes within the same SCC, and only considering edges across SCCs, gives us the graph’s condensation. The condensation is always acyclic, because the original graph’s cycles are “hidden” within the SCCs.

Here is the condensation of the previous example:

We can adapt our parallel-inlining algorithm to operate on strongly-connected components, and now it will correctly terminate because we’ve removed all cycles. First, we find the call graph’s SCCs and create the reverse (or transpose) condensation, where an edge a→b is flipped to b→a. We do this because we will query this graph to find the callers of a given function f, not the functions that f calls. I am not aware of an existing name for the reverse condensation, so, at Chris Fallin’s brilliant suggestion, I have decided to call it an evaporation. From there, the algorithm largely remains as it was before, although we keep track of counts and layers by SCC rather than by function.

let call_graph = CallGraph::new(wasm.functions);
let components = StronglyConnectedComponents::new(call_graph);
let evaoporation = Evaporation::new(components);

let counts = { c: evaporation.num_callees_of(c) for c in components };

let layer = [ c for c in components if counts[c] == 0 ];
while layer is not empty {
    parallel for func in scc in layer {
        func.inline(...);
    }

    let next_layer = [];
    for scc in layer {
        for caller_scc in evaporation.callers_of(scc) {
            counts[caller_scc] -= 1;
            if counts[caller_scc] == 0 {
                next_layer.push(caller_scc);
            }
        }
    }
    layer = next_layer;
}

This is the algorithm we use in Wasmtime, modulo minor tweaks here and there to engineer some data structures and combine some loops. After parallel inlining, the rest of the compiler pipeline continues in parallel for each function, yielding unlinked machine code. Finally, we link all that together and resolve relocations, same as we did previously.

Heuristics are the only implementation detail left to discuss, but there isn’t much to say that hasn’t already been said. Wasmtime prefers not to inline calls within the same Wasm module, while cross-module calls are a strong hint that we should consider inlining. Beyond that, our heuristics are extremely naive at the moment, and only consider the code sizes of the caller and callee functions. There is a lot of room for improvement here, and we intend to make those improvements on-demand as people start playing with the inliner. For example, there are many things we don’t consider in our heuristics today, but possibly should:

Hints from WebAssembly’s compilation-hints proposal
The number of edges to a callee function in the call graph
Whether any of a call’s arguments are constants
Whether the call is inside a loop or a block marked as “cold”
Etc…

Some Initial Results

The speed up you get (or don’t get) from enabling inlining is going to vary from program to program. Here are a couple synthetic benchmarks.

First, let’s investigate the simplest case possible, a cross-module call of an empty function in a loop:

(component
  ;; Define one module, exporting an empty function `f`.
  (core module $M
    (func (export "f")
      nop
    )
  )

  ;; Define another module, importing `f`, and exporting a function
  ;; that calls `f` in a loop.
  (core module $N
    (import "m" "f" (func $f))
    (func (export "g") (param $counter i32)
      (loop $loop
        ;; When counter is zero, return.
        (if (i32.eq (local.get $counter) (i32.const 0))
          (then (return)))
        ;; Do our cross-module call.
        (call $f)
        ;; Decrement the counter and continue to the next iteration
        ;; of the loop.
        (local.set $counter (i32.sub (local.get $counter)
                                     (i32.const 1)))
        (br $loop))
    )
  )

  ;; Instantiate and link our modules.
  (core instance $m (instantiate $M))
  (core instance $n (instantiate $N (with "m" (instance $m))))

  ;; Lift and export the looping function.
  (func (export "g") (param "n" u32)
    (canon lift (core func $n "g"))
  )
)

We can inspect the machine code that this compiles down to via the wasmtime compile and wasmtime objdump commands. Let’s focus only on the looping function. Without inlining, we see a loop around a call, as we would expect:

00000020 wasm[1]::function[1]:
        ;; Function prologue.
        20: pushq   %rbp
        21: movq    %rsp, %rbp

        ;; Check for stack overflow.
        24: movq    8(%rdi), %r10
        28: movq    0x10(%r10), %r10
        2c: addq    $0x30, %r10
        30: cmpq    %rsp, %r10
        33: ja      0x89

        ;; Allocate this function's stack frame, save callee-save
        ;; registers, and shuffle some registers.
        39: subq    $0x20, %rsp
        3d: movq    %rbx, (%rsp)
        41: movq    %r14, 8(%rsp)
        46: movq    %r15, 0x10(%rsp)
        4b: movq    0x40(%rdi), %rbx
        4f: movq    %rdi, %r15
        52: movq    %rdx, %r14

        ;; Begin loop.
        ;;
        ;; Test our counter for zero and break out if so.
        55: testl   %r14d, %r14d
        58: je      0x72
        ;; Do our cross-module call.
        5e: movq    %r15, %rsi
        61: movq    %rbx, %rdi
        64: callq   0
        ;; Decrement our counter.
        69: subl    $1, %r14d
        ;; Continue to the next iteration of the loop.
        6d: jmp     0x55

        ;; Function epilogue: restore callee-save registers and
        ;; deallocate this functions's stack frame.
        72: movq    (%rsp), %rbx
        76: movq    8(%rsp), %r14
        7b: movq    0x10(%rsp), %r15
        80: addq    $0x20, %rsp
        84: movq    %rbp, %rsp
        87: popq    %rbp
        88: retq

        ;; Out-of-line traps.
        89: ud2
            ╰─╼ trap: StackOverflow

When we enable inlining, then M::f gets inlined into N::g. Despite N::g becoming a leaf function, we will still push %rbp and all that in the prologue and pop it in the epilogue, because Wasmtime always enables frame pointers. But because it no longer needs to shuffle values into ABI argument registers or allocate any stack space, it doesn’t need to do any explicit stack checks, and nearly all the rest of the code also goes away. All that is left is a loop decrementing a counter to zero:³

00000020 wasm[1]::function[1]:
        ;; Function prologue.
        20: pushq   %rbp
        21: movq    %rsp, %rbp

        ;; Loop.
        24: testl   %edx, %edx
        26: je      0x34
        2c: subl    $1, %edx
        2f: jmp     0x24

        ;; Function epilogue.
        34: movq    %rbp, %rsp
        37: popq    %rbp
        38: retq

With this simplest of examples, we can just count the difference in number of instructions in each loop body:

12 without inlining (7 in N::g and 5 in M::f which are 2 to push the frame pointer, 2 to pop it, and 1 to return)
4 with inlining

But we might as well verify that the inlined version really is faster via some quick-and-dirty benchmarking with hyperfine. This won’t measure only Wasm execution time, it also measures spawning a whole Wasmtime process, loading code from disk, etc…, but it will work for our purposes if we crank up the number of iterations:

$ hyperfine \
    "wasmtime run --allow-precompiled -Cinlining=n --invoke 'g(100000000)' no-inline.cwasm" \
    "wasmtime run --allow-precompiled -Cinlining=y --invoke 'g(100000000)' yes-inline.cwasm"

Benchmark 1: wasmtime run --allow-precompiled -Cinlining=n --invoke 'g(100000000)' no-inline.cwasm
  Time (mean ± σ):     138.2 ms ±   9.6 ms    [User: 132.7 ms, System: 6.7 ms]
  Range (min … max):   128.7 ms … 167.7 ms    19 runs

Benchmark 2: wasmtime run --allow-precompiled -Cinlining=y --invoke 'g(100000000)' yes-inline.cwasm
  Time (mean ± σ):      37.5 ms ±   1.1 ms    [User: 33.0 ms, System: 5.8 ms]
  Range (min … max):    35.7 ms …  40.8 ms    77 runs

Summary
  'wasmtime run --allow-precompiled -Cinlining=y --invoke 'g(100000000)' yes-inline.cwasm' ran
    3.69 ± 0.28 times faster than 'wasmtime run --allow-precompiled -Cinlining=n --invoke 'g(100000000)' no-inline.cwasm'

Okay so if we measure Wasm doing almost nothing but empty function calls and then we measure again after removing function call overhead, we get a big speed up — it would be disappointing if we didn’t! But maybe we can benchmark something a tiny bit more realistic.

A program that we commonly reach for when benchmarking is a small wrapper around the pulldown-cmark markdown library that parses the CommonMark specification (which is itself written in markdown) and renders that to HTML. This is Real World™ code operating on Real World™ inputs that matches Real World™ use cases people have for Wasm. That is, good benchmarking is incredibly difficult, but this program is nonetheless a pretty good candidate for inclusion in our corpus. There’s just one hiccup: in order for our inliner to activate normally, we need a program using components and making cross-module calls, and this program doesn’t do that. But we don’t have a good corpus of such benchmarks yet because this kind of component composition is still relatively new, so let’s keep using our pulldown-cmark program but measure our inliner’s effects via a more circuitous route.

Wasmtime has tunables to enable the inlining of intra-module calls⁴ and rustc and LLVM have tunables for disabling inlining⁵. Therefore we can roughly estimate the speed ups our inliner might unlock on a similar, but extensively componentized and cross-module calling, program by:

Disabling inlining when compiling the Rust source code to Wasm
Compiling the resulting Wasm binary to native code with Wasmtime twice: once with inlining disabled, and once with intra-module call inlining enabled
Comparing those two different compilations’ execution speeds

Running this experiment with Sightglass, our internal benchmarking infrastructure and tooling, yields the following results:

execution :: instructions-retired :: pulldown-cmark.wasm

  Δ = 7329995.35 ± 2.47 (confidence = 99%)

  with-inlining is 1.26x to 1.26x faster than without-inlining!

  [35729153 35729164.72 35729173] without-inlining
  [28399156 28399169.37 28399179] with-inlining

Conclusion

Wasmtime and Cranelift now have a function inliner! Test it out via the -C inlining=y command-line flag or via the wasmtime::Config::compiler_inlining method. Let us know if you run into any bugs or whether you see any speed-ups when running Wasm components containing multiple core modules.

Thanks to Chris Fallin and Graydon Hoare for reading early drafts of this piece and providing valuable feedback. Any errors that remain are my own.

Deterministic compilation gives a number of benefits: testing is easier, debugging is easier, builds can be byte-for-byte reproducible, it is well-behaved in the face of incremental compilation and fine-grained caching, etc… ↩
For what it is worth, this still allows collapsing chains of mutually-recursive calls (a calls b calls c calls a) into a single, self-recursive call (abc calls abc). Our actual implementation does not do this in practice, preferring additional parallelism instead, but it could in theory. ↩
Cranelift cannot currently remove loops without side effects, and generally doesn’t mess with control-flow at all in its mid-end. We’ve had various discussions about how we might best fit control-flow-y optimizations into Cranelift’s mid-end architecture over the years, but it also isn’t something that we’ve seen would be very beneficial for actual, Real World™ Wasm programs, given that (a) LLVM has already done much of this kind of thing when producing the Wasm, and (b) we do some branch-folding when lowering from our mid-level IR to our machine-specific IR. Maybe we will revisit this sometime in the future if it crops up more often after inlining. ↩
-C cranelift-wasmtime-inlining-intra-module=yes ↩
-Cllvm-args=--inline-threshold=0, -Cllvm-args=--inlinehint-threshold=0, and -Zinline-mir=no ↩

New Stack Maps for Wasmtime and Cranelift

Tue, 10 Sep 2024 00:00:00 -0700

Note: This post was originally published on the Bytecode Alliance blog.

As part of implementing the WebAssembly garbage collection proposal in Wasmtime, which is an ongoing process, we’ve overhauled the stack map infrastructure in Cranelift. This post will explain what stack maps are, why we needed to change them, and how the new stack maps work.

Wasmtime is a lightweight WebAssembly runtime that is fast, secure, and standards-compliant. Cranelift is its compiler backend, which aims to strike a balance between code quality and compile time, while maintaining exceptionally high standards for correctness. The garbage collection proposal for WebAssembly extends the WebAssembly language with runtime-managed references to structs and arrays.

Background: Garbage Collection, Safepoints, and Stack Maps

In the garbage collection (GC) literature, we call the program that allocates objects and adds or removes edges between them the mutator. It mutates the heap graph.

As the mutator runs, allocating objects and filling up the GC heap, the runtime must eventually perform a GC. But the runtime can’t collect garbage at any arbitrary moment: the mutator might be in the middle of manipulating some objects, keeping them in its stack frame or in registers, and effectively hiding them from the runtime. The runtime must know about all such objects, or else it might reclaim an object that it believes is garbage, but which the mutator is still using, leading to use-after-free bugs. This is where safepoints and stack maps come in.

A safepoint is a program point in the mutator (that is, an instruction in the compiled WebAssembly program) where it is safe for the runtime to collect garbage. When the mutator calls out to a runtime routine, for example, that call instruction would typically be a safepoint, under the assumption that the runtime might need to GC while running the routine.

The mutator’s compiler does two things to make a safepoint safe, in the presence of objects that the mutator is still actively working with:

Insert instructions to move the objects from (say) volatile registers that a call safepoint might clobber to well-known locations in the stack frame.
Emit stack map records that describe, for each safepoint in the mutator program, the locations where to find those still-in-active-use objects inside the stack frame.

While collecting garbage, the runtime can walk the mutator’s stack frames and, using those stack maps, identify all of the objects that the mutator is still manipulating. No more potential use-after-collected bugs!

But there is a final twist: if the GC is a moving collector, for example a generational or compacting collector, then an object may be relocated in memory during collection. If an object is moved, then all references to the object must also be updated to refer to the new location. Failure to update a reference leads to use-after-move bugs like accessing newly-allocated objects that happened to be placed in the original object’s old location. So, in the presence of a moving collector, the runtime must additionally update the mutator’s stack frame, redirecting each reference noted in the stack map so that it now points to its object’s new memory location. The compiler must, in turn, insert reloads of each GC-managed reference from its stack map location that it was spilled to — that is, from the location where the runtime potentially overwrote the old reference with a new, redirected reference — and back into a register so that the mutator may continue to manipulate the object.

Cranelift’s Old Approach: Stack Maps During Register Allocation

Cranelift previously generated stack maps during register allocation. Our register allocator naturally tracks the live range of each value in order to determine whether the same hardware register may or may not be reused for two values. At each safepoint, it would insert spills for each of the live values that are managed by the GC, and record those spilled locations in a stack map for that program point. This approach is straightforward on the surface, but led to a number of subtle downsides and complications.

First, because register allocation is so late in the compiler pipeline, the previous system involved tracking GC-managed references through almost the entire compiler: from the WebAssembly frontend, through the ISA-agnostic mid-end, through the ISA-specific backends, and into register allocation. The only part of the pipeline that didn’t have to preserve knowledge of GC-managed references was the final instruction encoding and binary emission. In general, the fewer invariants we must maintain, the simpler our compiler’s implementation. If we could avoid precisely tracking GC references throughout most of Cranelift, we could expect less complexity and, in general, less complexity would mean fewer bugs. The complexity of the coordination required between the runtime and the compiler to correctly implement garbage collection and stack maps has bitten us before.

Second, to make tracking GC-managed values easier, we distinguished them from plain pointers and integers with dedicated types in Cranelift’s intermediate representation (CLIF). This succeeded in making them easier to track, but led to foiling instruction selection rules and mid-end peephole rewrites that otherwise should have applied. For example, checking if a pointer is null involves simply comparing it to zero, the same way we compare any other integer to zero:

v456 = icmp_imm eq v123, 0

Checking whether GC reference was null, on the other hand, required a dedicated is_null instruction, despite lowering to identical machine code when it appears in isolation:

v456 = is_null v123

This doesn’t seem like a big deal at first glance, beyond being forced to define additional, duplicate instructions that waste our opcode space and inflate our instruction set. However, we have a bunch of instruction selection rules for integer comparisons followed by conditional branches, for example, that let us avoid materializing flags on x86_64, yielding smaller and faster machine code in those contexts. However, because is_null is a different instruction, these rules’ patterns don’t match, and their optimizations are missed. The result is that we must choose between generating subpar code or duplicating the relevant set of instruction selection rules to also match against an is_null instruction followed by a conditional branch.

Third, in addition to missed optimizations, we were also seeing misoptimizations: incorrect transformations that broke the input program. From its point of view, our mid-end was performing perfectly legal optimizations. But the mid-end can’t reason about effects that it can’t see, and it couldn’t see the spills and reloads of GC references associated with a safepoint because they weren’t inserted yet. This lack of visibility became an issue when we were, for example, accessing the same field of an object before and after a safepoint:

;; `v0` is a reference to a GC-managed object.
v0 = ...
;; Bitcast it to an `i64` so we can do pointer
;; arithmetic on it.
v1 = bitcast.i64 v0
;; Get the address of the field we want to
;; access.
v2 = iadd_imm v1, 8
;; Write `11` into the object field.
v3 = iconst.i32 11
store.i32 v2, v3

;; Call some function! This might trigger a GC
;; and therefore must be a safepoint.
call $f()

;; Write to same field as above, but with the
;; value `22` this time.
v4 = bitcast.i64 v0
v5 = iadd_imm v4, 8
v6 = iconst.i32 22
store.i32 v5, v6

Our mid-end performs a number of ISA-agnostic optimizations, including global value numbering (GVN) (as part of its e-graphs pass) to eliminate duplicate computations. When the mid-end processed the CLIF snippet above, it would determine that the second field-address computation was redundant, and we could instead reuse the result of the first field computation, yielding the following optimized CLIF:

v0 = ...
v1 = bitcast.i64 v0
v2 = iadd_imm v1, 8
v3 = iconst.i32 11
store.i32 v2, v3

call $f()

v6 = iconst.i32 22
;; Reusing the field address we previously
;; computed above.
store.i32 v2, v6

To reason about why this transformation is invalid — despite only deduplicating pure, non-effectful, and otherwise-identical instructions — we must consider context that is invisible to the mid-end: the safepoint spills and reloads that will be inserted later by the register allocator. The second field-address computation is not actually reusing v0 as its base, it is using reload(v0) as the base, but that reload operation is hidden from the mid-end. If the runtime collected garbage during $f’s execution and moved the object that v0 references, then in this optimized CLIF, the second store is writing to the object’s old, vacated location, which is a use-after-move vulnerability.

We began to address these sorts of bugs in a whack-a-mole fashion, e.g. by marking bitcasts of reference types as side-effectful, but ultimately we’d rather not play this whack-a-mole game at all, reacting to bugs as we find them, never knowing how many are still lurking. Instead, we’d much prefer that the mid-end simply had the visibility it needs to optimize correctly in the first place.

A final quirk of the old system was that GC references were always represented as pointers at the machine level. You could only use 64-bit references on 64-bit ISAs and 32-bit references on 32-bit ISAs, and you definitely could not do NaN boxing. This was an annoyance for us in practice because we’ve designed Wasmtime’s GC implementation such that our GC references are always 32-bit integers, even on 64-bit ISAs. This design choice lets us leverage the same sandboxing and virtual-memory techniques to elide bounds checks that Wasmtime already implements for WebAssembly linear memories. It lets us benefit from smaller objects and denser cache locality; because of that, it is a fairly common technique. But, despite storing GC references in 32 bits on the heap, Cranelift’s old system forced us to extend those references to 64 bits when we were manipulating them on the stack or in registers when targeting a 64-bit ISA.

The New Approach: User Stack Maps

Given the complications of Cranelift’s previous system for tracking GC references across safepoints and emitting stack maps, we decided to implement a new approach. Instead of generating safepoint spills, reloads, and stack maps at the end of the compiler pipeline, the user is responsible for producing CLIF that already contains those spills and reloads, as well as annotating each safepoint with the virtual stack slots live GC references were spilled into. The mid-end, backends, and register allocator have no extra responsibilities other than forwarding these annotations through to binary emission, at which point we can translate the virtual stack slots into precise stack-pointer offsets and emit stack maps. We dubbed this new approach “user stack maps” because their correctness largely falls upon the Cranelift user, rather than the core of Cranelift itself.

User stack maps mean that we no longer need to precisely track GC references throughout the whole compiler pipeline. This simplification allows us us to remove the distinct reference types from CLIF, as well as their related instructions, like is_null. This means no more missed optimizations or duplicated instruction selection rules.

With user stack maps, we no longer have to worry about safepoint-specific miscompilations in the mid-end. Let’s revisit the previous miscompilation example, where we wrote to a GC object’s field before and after a safepoint. This is what the initial CLIF looks like with user stack maps, where the CLIF producer has already inserted the spills and reloads for the safepoint:

;; Set the field to `11`
v0 = ...
v1 = iadd_imm v0, 8
v2 = iconst.i32 11
store.i32 v1, v2

;; NEW: Spill `v0` to stack slot `ss0`
;; before the safepoint.
v3 = stack_addr ss0
store.i64 notrap v3, v0

;; NEW: stack map annotation on the safepoint.
call $f(), stack_map = [i64 @ ss0]

;; NEW: Reload what was `v0` from the
;; stack slot. Don't reuse `v0` because
;; the referenced object could have
;; moved.
v4 = stack_addr ss0
v5 = load.i64 notrap v4

;; Set the field to `22`.
v6 = iadd_imm v5, 8
v7 = iconst.i32 22
store.i32 v6, v7

Looking at the updated CLIF, it is immediately obvious to the mid-end that reusing the initial field-address computation for the second store is invalid. The base of the second field-address computation is v5, not v0, and it cannot prove that v5 and v0 are actually the same value; indeed, if the referenced object moved in memory then they will not be the same! We’ve avoided the previous misoptimization by construction.

Cranelift already must preserve the effects of accessing memory, and a safepoint’s spills and reloads are no different: they are explicitly present and modeled in the intermediate representation as loads and stores, the same as any other memory access. In the world of user stack maps, supporting safepoint spills and reloads is not an additional responsibility nor additional implementation complexity for the compiler.

User stack maps actually unlock more potential optimizations than what used to be possible. Our alias analysis pass, which performs redundant load elimination and store-to-load forwarding, can now see and operate upon safepoint-related spills and reloads. And once again, Cranelift’s surface area that is critical for correctness has not grown; the alias analysis pass must already preserve the effects of accessing memory.

While user stack maps are a big simplification for most of the compiler pipeline, they do shift some of that old complexity into the frontend. The mid-end, backends, and register allocator no longer need to precisely track GC references, and insert spills and reloads at safepoints, but now the frontend must. We believe that we’ve ended up at a design point with less overall complexity.

We, the Cranelift developers, couldn’t leave users (i.e. we, the Wasmtime developers) completely on their own, without any tools to help produce correct safepoints in CLIF. Cranelift users, including Wasmtime, generally use the cranelift-frontend crate to produce the initial CLIF that Cranelift takes as input to compilation. cranelift-frontend helps users build CLIF from their source language, handles SSA construction, and etc…; it is an appropriate place to help with user stack maps as well. While they are creating CLIF with cranelift-frontend, users can now mark any value as requiring inclusion in stack maps, and cranelift-frontend will automatically insert spills and reloads of that value around safepoints, as well as annotate those safepoints with stack map entries that record the spill locations for the runtime.

To fulfill these new obligations, the cranelift-frontend crate performs a simple liveness analysis for GC references, allocates and assigns stack slots for the GC references, and then inserts spills and reloads. If this sounds similar to register allocation, that’s because it is. The important difference is that the frontend performs this work only for GC references, not for the majority of non-GC-reference values in the program. When a program does not use any GC-managed objects, which is the vast majority of WebAssembly programs today, no overhead is imposed.

Our liveness analysis flows backwards, from uses (which mark values live) to definitions (which remove values from the live set) in the data-flow graph and from successor blocks to predecessor blocks in the control-flow graph.

We compute two live sets for each block:

The live-in set, which is the set of values that are live when control enters the block.
The live-out set, which is the set of values that are live when control exits the block.

A block’s live-out set is the union of its successors’ live-in sets. A block’s live-in set is the set of values that are still live after the block’s instructions have been processed, where uses mark values live and definitions remove them from the set.

live_in(block) = union(live_out(s) for s in successors(block))

live_out(block) = live_in(block) + uses(block) - defs(block)

Whenever we update a block’s live-in set, we must reprocess all of its predecessors, because those predecessors’ live-out sets depend on this block’s live-in set. Processing continues until the live sets stop changing and we’ve reached a fixed-point. We implement this with the classic worklist approach.

Once we have the results of the liveness analysis in hand, we do a final pass over the CLIF, assigning stack slots and inserting spills and reloads. This is also implemented as a backwards pass. Whenever the pass finds a use of a GC reference that is live across any safepoint, it allocates the reference’s stack slot if it doesn’t already have one, and replaces the use with a load from that stack slot. Upon reaching a safepoint, the pass adds user stack map entries for each of the GC references that are live across the safepoint. Finally, when the pass encounters a GC reference’s definition, it appends a spill of the reference to its associated stack slot to the definition. Then, knowing that the value will never be seen again, since we are processing SSA values in backwards order, the GC reference’s stack slot is returned to a free list and made available for any new GC references the pass has yet to encounter.

Astute readers may have noticed that replacing every use with a fresh reload may result in many redundant loads. This is not a concern because, as previously noted, our mid-end’s alias analysis can already clean up and remove these redundant loads. Separation of concerns between compiler passes lets us decrease overall complexity.

Conclusion

We’ve moved stack map generation and safepoint spills and reloads from the end of the compiler pipeline and into the frontend. It simplifies the compiler and avoids misoptimizations and missed optimizations by relying on our existing correctness invariants rather than imposing new ones to maintain. It also unlocks efficient sandboxing of the GC heap using the same techniques we use for WebAssembly’s linear memories.

We’re making steady progress implementing the WebAssembly garbage collection proposal in Wasmtime. It is not complete or ready for general use yet, but it is coming soon.

Want to start hacking on Wasmtime or Cranelift? Check out our contributor documentation!

Thanks to Trevor Elliott for partnering with me on implementing this stack maps overhaul and to Chris Fallin for design feedback, brainstorming, code review, and providing feedback on an early draft of this blog post.

Garbage Collection Without Unsafe Code

Tue, 06 Feb 2024 00:00:00 -0800

Many people, including myself, have implemented garbage collection (GC) libraries for Rust. Manish Goregaokar wrote up a fantastic survey of this space a few years ago. These libraries aim to provide a safe API for their users to consume: an unsafe-free interface which soundly encapsulates and hides the library’s internal unsafe code. The one exception is their mechanism to enumerate the outgoing GC edges of user-defined GC types, since failure to enumerate all edges can lead the collector to believe that an object is unreachable and collect it, despite the fact that the user still has a reference to the reclaimed object, leading to use-after-free bugs.¹ This functionality is generally exposed as an unsafe trait for the user to implement because it is the user’s responsibility, not the library’s, to uphold this particular critical safety invariant.

However, despite providing safe interfaces, all of these libraries make extensive use of unsafe code in their internal implementations. I’ve always believed it was possible to write a garbage collection library without any unsafe code, and no one I’ve asserted this to has disagreed, but there has never been a proof by construction.

So, finally, I created the safe-gc crate: a garbage collection library for Rust with zero unsafe code. No unsafe in the API. No unsafe in the implementation. It even has a forbid(unsafe_code) pragma at the top.

That said, safe-gc is not a particularly high-performance garbage collector.

Using `safe-gc`

To use safe-gc, first we define our GC-managed types, using Gc<T> to define references to other GC-managed objects, and implement the Trace trait to report each of those GC edges to the collector:

use safe_gc::{Collector, Gc, Trace};

// Define a GC-managed object.
struct List {
    value: u32,

    // GC-managed references to the next and previous links in the list.
    prev: Option<Gc<List>>,
    next: Option<Gc<List>>,
}

// Report GC edges to the collector.
impl Trace for List {
    fn trace(&self, collector: &mut Collector) {
        if let Some(prev) = self.prev {
            collector.edge(prev);
        }
        if let Some(next) = self.next {
            collector.edge(next);
        }
    }
}

This looks pretty similar to other GC libraries in Rust, although it could definitely benefit from an implementation of Trace for Option<T> and a derive(Trace) macro. The big difference from existing GC libraries is that Trace is safe to implement; more on this later.

Next, we create one or more Heaps to allocate our objects within. Each heap is independently garbage collected.

use safe_gc::Heap;

let mut heap = Heap::new();

And with a Heap in hand, we can allocate objects:

let a = heap.alloc(List {
    value: 42,
    prev: None,
    next: None,
});

let b = heap.alloc(List {
    value: 36,
    prev: Some(a.into()),
    next: None,
});

// Create a bunch of garbage! Who cares! It'll all be cleaned
// up eventually!
for i in 0..100 {
    let _ = heap.alloc(List {
        value: i,
        prev: None,
        next: None,
    });
}

The heap will automatically trigger garbage collections, as necessary, but we can also force a collection if we want:

// Force a garbage collection!
heap.gc()

Rather than deref’ing Gc<T> pointers directly, we must index into the Heap to access the referenced T object. This contrasts with other GC libraries and is the key that unlocks safe-gc’s lack of unsafe code, allowing the implementation to abide by Rust’s ownership and borrowing discipline.²

// Read from a GC object in the heap.
let b_value = heap[&b].value;
assert_eq!(b_value, 36);

// Write to a GC object in the heap.
heap[&b].value += 1;
assert_eq!(heap[&b].value, 37);

Finally, there are actually two types for indexing into Heaps to access GC objects:

Gc<T>, which we have seen already, and
Root<T>, which we have also seen in action, but which was hidden from us by type inference.

The Gc<T> type is Copy and should be used when referencing other GC-managed objects from within a GC-managed object’s type definition, or when you can prove that a garbage collection will not happen (i.e. you have a shared borrow of its heap). A Gc<T> does not root its referenced T, keeping it alive across garbage collections, and therefore Gc<T> should not be used to hold onto GC references across any operation that can trigger a garbage collection.

A Root<T>, on the other hand, does indeed root its associated T object, preventing the object from being reclaimed during garbage collection. This makes Root<T> suitable for holding references to GC-managed objects across operations that can trigger garbage collections. Root<T> is not Copy because dropping it must remove its entry from the heap’s root set. Allocation returns rooted references; all the heap.alloc(...) calls from our earlier examples returned Root<T>s.

Peeking Under the Hood

A safe_gc::Heap is more similar to an arena newtype over a Vec than an engineered heap with hierarchies of regions like Immix. Its main storage is a hash map from std::any::TypeId to uniform arenas of the associated type. This lets us ultimately use Vec as the storage for heap-allocated objects, and we don’t need to do any unsafe pointer arithmetic or worry about splitting large blocks in our free lists. In fact, the free lists only manage indices, not blocks of raw memory.

pub struct Heap {
    // A map from `type_id(T)` to `Arena<T>`. The `ArenaObject`
    // trait facilitates crossing the boundary from an untyped
    // heap to typed arenas.
    arenas: HashMap<TypeId, Box<dyn ArenaObject>>,

    // ...
}

struct Arena<T> {
    elements: FreeList<T>,

    // ...
}

enum FreeListEntry<T> {
    /// An occupied entry holding a `T`.
    Occupied(T),

    /// A free entry that is also part of a linked list
    /// pointing to the next free entry, if any.
    Free(Option<u32>),
}

struct FreeList<T> {
    // The actual backing storage for our `T`s.
    entries: Vec<FreeListEntry<T>>,

    /// The index of the first free entry in the free list.
    free: Option<u32>,

    // ...
}

To allocate a new T in the heap, we first get the T object arena out of the heap’s hash map, or create it if it doesn’t exist yet. Then, we check if the arena has capacity to allocate our new T. If it does, we push the object onto the arena and return a rooted reference. If it does not, we fall back to an out-of-line slow path where we trigger a garbage collection to ensure that we have space for the new object, and then try again.

impl Heap {
    #[inline]
    pub fn alloc<T>(&mut self, value: T) -> Root<T>
    where
        T: Trace,
    {
        let heap_id = self.id;
        let arena = self.ensure_arena::<T>();
        // Fast path for when we have available capacity for
        // allocating into.
        match arena.try_alloc(heap_id, value) {
            Ok(root) => root,
            Err(value) => self.alloc_slow(value),
        }
    }

    // Out-of-line slow path for when we need to GC to free
    // up or allocate additional space.
    #[inline(never)]
    fn alloc_slow<T>(&mut self, value: T) -> Root<T>
    where
        T: Trace,
    {
        self.gc();
        let heap_id = self.id;
        let arena = self.ensure_arena::<T>();
        arena.alloc_slow(heap_id, value)
    }
}

Arena<T> allocation bottoms out in allocating from a FreeList<T>, which will attempt to use existing capacity by popping off its internal list of empty entries when possible, or otherwise fall back to reserving additional capacity.

impl<T> FreeList<T> {
    fn try_alloc(&mut self, value: T) -> Result<u32, T> {
        if let Some(index) = self.free {
            // We have capacity. Pop the first free entry off
            // the free list and put the value in there.
            let index = usize::try_from(index).unwrap();
            let next_free = match self.entries[index] {
                Entry::Free(next_free) => next_free,
                Entry::Occupied { .. } => unreachable!(),
            };
            self.free = next_free;
            self.entries[index] = Entry::Occupied(value);
            Ok(index)
        } else {
            // No capacity to hold the value; give it back.
            Err(value)
        }
    }

    fn alloc(&mut self, value: T) -> u32 {
        self.try_alloc(value).unwrap_or_else(|value| {
            // Reserve additional capacity, since we didn't have
            // space for the allocation.
            self.double_capacity();
            // After which the allocation will succeed.
            self.try_alloc(value).ok().unwrap()
        })
    }
}

Accessing objects in the heap is straightforward: look up the arena for T and index into it.

impl Heap {
    /// Get a shared borrow of the referenced `T`.
    pub fn get<T>(&self, gc: impl Into<Gc<T>>) -> &T
    where
        T: Trace,
    {
        let gc = gc.into();
        assert_eq!(self.id, gc.heap_id);
        let arena = self.arena::<T>().unwrap();
        arena.elements.get(gc.index)
    }

    // Get an exclusive borrow of the referenced `T`.
    pub fn get_mut<T>(&mut self, gc: impl Into<Gc<T>>) -> &mut T
    where
        T: Trace,
    {
        let gc = gc.into();
        assert_eq!(self.id, gc.heap_id);
        let arena = self.arena_mut::<T>().unwrap();
        arena.elements.get_mut(gc.index)
    }
}

Before we get into how safe-gc actually performs garbage collection, we need to look at how it implements the root set. The root set are the set of things that are definitely alive; things that the application is actively using right now or planning to use in the future. The goal of the collector is to identify all objects transitively referenced by these roots, since these are the objects that can still be used in the future, and recycle all others.

Each Arena<T> has its own RootSet<T>. For simplicity a RootSet<T> is a wrapper around a FreeList<Gc<T>>. When we add new roots, we insert them into the FreeList, and when we drop a root, we remove it from the FreeList. This does mean that the root set can contain duplicates and is therefore not a proper set. The root set’s FreeList is additionally wrapped in an Rc<RefCell<...>> so that we can implement Clone for Root<T>, which adds another entry in the root set, and don’t need to explicitly pass around a Heap to hold additional references to a rooted object.

Finally, I took care to design Root<T> and RootSet<T> such that Root<T> doesn’t directly hold a Gc<T>. This allows for updating rooted GC pointers after a collection, which is necessary for moving GC algorithms like generational GC and compaction. In fact, I originally intended to implement a copying collector, which is a moving GC algorithm, for safe-gc but ran into some issues. More on those later. For now, we retain the possibility of introducing moving GC at a later date.

struct Arena<T> {
    // ...

    // Each arena has a root set.
    roots: RootSet<T>,
}

// The set of rooted `T`s in an arena.
struct RootSet<T> {
    inner: Rc<RefCell<FreeList<Gc<T>>>>,
}

impl<T: Trace> RootSet<T> {
    // Rooting a `Gc<T>` adds an entry to the root set.
    fn insert(&self, gc: Gc<T>) -> Root<T> {
        let mut inner = self.inner.borrow_mut();
        let index = inner.alloc(gc);
        Root {
            roots: self.clone(),
            index,
        }
    }

    fn remove(&self, index: u32) {
        let mut inner = self.inner.borrow_mut();
        inner.dealloc(index);
    }
}

pub struct Root<T: Trace> {
    // Each `Root<T>` holds a reference to the root set.
    roots: RootSet<T>,

    // Index of this root in the root set.
    index: u32,
}

// Dropping a `Root<T>` removes its entry from the root set.
impl<T: Trace> Drop for Root<T> {
    fn drop(&mut self) {
        self.roots.remove(self.index);
    }
}

With all that out of the way, we can finally look at the core garbage collection algorithm.

safe-gc implements simple mark-and-sweep garbage collection. We begin by resetting the mark bits for each arena, and making sure that there are enough bits for all of our allocated objects, since we keep the mark bits in an out-of-line compact bitset rather than in each object’s header word or something like that.

impl Heap {
    #[inline(never)]
    pub fn gc(&mut self) {
        // Reset/pre-allocate the mark bits.
        for (ty, arena) in &self.arenas {
            self.collector
                .mark_bits
                .entry(*ty)
                .or_default()
                .reset(arena.capacity());
        }

        // ...
    }
}

Next we begin the mark phase. This starts by iterating over each root and then setting its mark bit and enqueuing it in the mark stack by calling collector.edge(root).

impl Heap {
    #[inline(never)]
    pub fn gc(&mut self) {
        // ...

        // Mark all roots.
        for arena in self.arenas.values() {
            arena.trace_roots(&mut self.collector);
        }

        // ...
    }
}

trait ArenaObject: Any {
    fn trace_roots(&self, collector: &mut Collector);

    // ...
}

impl<T: Trace> ArenaObject for Arena<T> {
    fn trace_roots(&self, collector: &mut Collector) {
        self.roots.trace(collector);
    }

    // ...
}

impl<T: Trace> RootSet<T> {
    fn trace(&self, collector: &mut Collector) {
        let inner = self.inner.borrow();
        for (_, root) in inner.iter() {
            collector.edge(*root);
        }
    }
}

The mark phase continues by marking everything transitively reachable from those roots in a fixed-point loop. If we discover an unmarked object, we mark it and enqueue it for tracing. Whenever we see an already-marked object, we ignore it.

What is kind of unusual is that we don’t have a single mark stack. The Heap has no T type parameter, and contains many different types of objects, so the heap itself doesn’t know how to trace any particular object. However, each of the heap’s Arena<T>s holds only a single type of object, and an arena does know how to trace its objects. So we have a mark stack for each T, or equivalently, each arena. This means that our fixed-point loop has two levels: an outer loop that continues while any mark stack has work enqueued, and an inner loop to drain a particular mark stack.

impl Heap {
    #[inline(never)]
    pub fn gc(&mut self) {
        // ...

        // Mark everything transitively reachable from the roots.
        while let Some(type_id) = self
            .collector
            .next_non_empty_mark_stack()
        {
            while let Some(index) = self
                .collector
                .pop_mark_stack(type_id)
            {
                self.arenas
                    .get_mut(&type_id)
                    .unwrap()
                    .trace_one(index, &mut self.collector);
            }
        }

        // ...
    }
}

While the driver loop for marking is inside the Heap::gc method, the actual edge tracing and mark bit setting happens inside Collector and the arena which, because it has a T type parameter, can call the correct Trace implementation for each object.

trait ArenaObject: Any {
    fn trace_one(&mut self, index: u32, collector: &mut Collector);

    // ...
}

impl<T: Trace> ArenaObject for Arena<T> {
    fn trace_one(&mut self, index: u32, collector: &mut Collector) {
        self.elements.get(index).trace(collector);
    }

    // ...
}

pub struct Collector {
    heap_id: u32,
    // The mark stack for each type in the heap.
    mark_stacks: HashMap<TypeId, Vec<u32>>,
    // The mark bits for each type in the heap.
    mark_bits: HashMap<TypeId, MarkBits>,
}

impl Collector {
    pub fn edge<T: Trace>(&mut self, to: Gc<T>) {
        assert_eq!(to.heap_id, self.heap_id);

        // Get the mark bits for `T` objects.
        let ty = TypeId::of::<T>();
        let mark_bits = self.mark_bits.get_mut(&ty).unwrap();

        // Set `to`'s mark bit. If the bit was already set, we're
        // done.
        if mark_bits.set(to.index) {
            return;
        }

        // Otherwise this is the first time visiting this GC
        // object so enqueue it for further marking.
        let mark_stack = self.mark_stacks.entry(ty).or_default();
        mark_stack.push(to.index);
    }
}

Once our mark stacks are all empty, we’ve reached our fixed point, and that means we’ve finished marking all objects reachable from the root set. Now we transition to the sweep phase.

Sweeping iterates over each object in each arena. If that object’s mark bit is not set, then it is unreachable from the GC roots, i.e. it is not a member of the live set, i.e. it is garbage. We drop such objects and push their slots into their arena’s free list, making the slot available for future allocations.

After sweeping each arena we check whether the arena is still close to running out of capacity and, if so, reserve additional space for the arena. This amortizes the cost of garbage collection and avoids a scenario that could otherwise trigger a full GC on every object allocation:

The arena has zero available capacity.
The user tries to allocate, triggering a GC.
The GC is able to reclaim only one slot in the arena.
The user’s pending allocation fills the reclaimed slot.
Now the arena is out of capacity again, and the process repeats from the top.

By reserving additional space in the arena after sweeping, we avoid this failure mode.

We could also compact the arena and release excess space back to the global allocator if there was too much available capacity. This would additionally require a method for updating incoming edges to the compacted objects, and safe-gc does not implement compaction at this time.

impl Heap {
    #[inline(never)]
    pub fn gc(&mut self) {
        // ...

        // Sweep.
        for (ty, arena) in &mut self.arenas {
            let mark_bits = &self.collector.mark_bits[ty];
            arena.sweep(mark_bits);
        }
    }
}

trait ArenaObject: Any {
    // ...

    fn sweep(&mut self, mark_bits: &MarkBits);
}

impl<T: Trace> ArenaObject for Arena<T> {
    // ...

    fn sweep(&mut self, mark_bits: &MarkBits) {
        // Reclaim garbage slots.
        let capacity = self.elements.capacity();
        for index in 0..capacity {
            if !mark_bits.get(index) {
                self.elements.dealloc(index);
            }
        }

        // Amortize the cost of GC across allocations.
        let len = self.elements.len();
        let available = capacity - len;
        if available < capacity / 4 {
            self.elements.double_capacity();
        }
    }
}

After every arena is swept, garbage collection is complete!

Preventing Classic Footguns

Now that we know how safe-gc is implemented, we can explore a couple classic GC footguns and analyze how safe-gc either completely nullifies them or downgrades them from critical security vulnerabilities to plain old bugs.

Often an object might represent some external resource that should be cleaned up when the object is no longer in use, like an open file descriptor. This functionality is typically supported with finalizers, the GC-equivalent of C++ destructors and Rust’s Drop trait. Finalization of GC objects is usually tricky because of the risks of either accessing objects that have already been reclaimed by the collector (which is a use-after-free bug) or accidentally entrenching objects and making them live again (which leads to memory leaks). Because of these risks, Rust GC libraries often make finalization an unsafe trait and even forbid allocating types that implement Drop in their heaps.

However, safe-gc doesn’t need an unsafe finalizer trait, or even any additional finalizer trait: it can just use Drop. Drop implementations simply do not have access to a Heap, which is required to deref GC pointers, so they cannot suffer from those finalization footguns.

Next up: why isn’t Trace an unsafe trait? And what happens if you don’t root a Gc<T> and then index into a Heap with it after a garbage collection? These are actually the same question: what happens if I use a dangling Gc<T>? As mentioned at the start, if a Trace implementation fails to report all edges to the collector, the collector may believe an object is unreachable and reclaim it, and now the unreported edge is dangling. Similarly, if the user holds an unrooted Gc<T>, rather than a Root<T>, across a garbage collection then the collector might believe that the referenced object is garbage and reclaim it, leaving the unrooted reference dangling.

Indexing into a Heap with a potentially-dangling Gc<T> will result in one of three possibilities:

We got “lucky” and something else happened to keep the object alive. The access succeeds as it otherwise would have and the potentially-dangling bug is hidden.
The associated slot in the arena’s free list is empty and contains a FreeListEntry::Free variant. This scenario will raise a panic.
A new object has since been allocated in the same arena slot. The access will succeed, but it will be to the wrong object. This is an instance of the ABA problem. We could, at the cost of some runtime overhead, turn this into a loud panic instead of silent action at a distance by adding a generation counter to our arenas.

Of course, it would be best if users always rooted GC references they held across collections and correctly implemented the Trace trait but, should they fail to do that, all three potential outcomes are 100% memory safe.³ These failures can’t lead to memory corruption or use-after-free bugs, which would be the typical results of this kind of thing with an unsafe GC implementation.

Copying Collector False Start

I initially intended to implement a copying collector rather than mark-and-sweep, but ultimately the borrowing and ownership didn’t pan out. That isn’t to say it is impossible to implement a copying collector in safe Rust, but it ended up feeling like more of a headache than it was worth. I spent several hours trying to jiggle things around to experiment with different ownership hierarchies and didn’t get anything satisfactory. When I decided to try mark-and-sweep, it only took me about half an hour to get an initial prototype working. I found this really surprising, since I had a strong intuition that a copying collector, with its separate from- and to-spaces, should play well with Rust’s ownership and borrowing.

Briefly, the algorithm works as follows:

We equally divide the heap into two semi-spaces.
At any given time in between collections, all objects live in one semi-space and the other is sitting idle.
We bump allocate within the active semi-space, slowly filling it up, and when the bump pointer reaches the end of the semi-space, we trigger a collection.
During collection, as we trace the live set, we copy objects from the old semi-space that has been active, to the other new semi-space that has been idle. At the same time, we maintain a map from the live objects’ location in the old semi-space to their location in the new semi-space. When we trace an object’s edges, we also update those edges to point to their new locations. Once tracing reaches a fixed-point, we’ve copied the whole live set to the new semi-space, it becomes the active semi-space, and the previously-active semi-space now sits idle until the next collection.

Copying collection has a number of desirable properties:

The algorithm is relatively simple and easy to understand.
Allocating new objects is fast: just bumping a pointer and checking that space isn’t exhausted yet.
The act of copying objects to the new semi-space compacts the heap, defeating fragmentation.
It also eliminates the need for a sweep phase, since the whole of the old semi-space is garbage after the live set has been moved to the new semi-space.

Copying collection’s primary disadvantage is the memory overhead it imposes: we can only ever use at most half of the heap to store objects.

When I think about a copying collector, I tend to imagine Lisp cons cells, as I was first introduced to this algorithm in that context by SICP. Here is what a very naive implementation of the core copying collection algorithm might look like in safe Rust:

fn copy_collect(
    roots: &mut [usize],
    from: &[Cons],
    to: &mut Vec<Cons>,
) {
    // Contains a work list of the new indices of cons cells
    // that have been been copied to `to` but haven't had their
    // edges traced and updated yet.
    let mut stack = vec::with_capacity(roots.len());

    // The map from each live object's old location, to its new one.
    let mut old_to_new = HashMap::new();

    // Copy each root to the to-space, enqueue it for tracing, and
    // update its pointer to its new index in the to-space.
    for root in roots {
        visit_edge(from, to, &mut old_to_new, &mut stack, root);
    }

    // Now do the same for everything transitively reachable from
    // the roots.
    while let Some(index) = stack.pop() {
        let cons = &mut to[index];
        if let Some(car) = &mut cons.car {
            visit_edge(from, to, &mut old_to_new, &mut stack, car);
        }
        if let Some(cdr) = &mut cons.cdr {
            visit_edge(from, to, &mut old_to_new, &mut stack, cdr);
        }
    }
}

// Visit one edge. If the edge's referent has already been copied
// to the to-space, just update the edge's pointer so that it points
// to the new location. If it hasn't been copied yet, additionally
// copy it over and enqueue it in the stack for future tracing.
fn visit_edge(
    from: &[Cons],
    to: &mut Vec<Cons>,
    old_to_new: &mut HashMap<usize, usize>,
    stack: &mut Vec<usize>,
    edge: &mut usize,
) {
    let new_location = old_to_new
        .entry(*edge)
        .or_insert(|| {
            let new = to.len();
            // Copy the object over.
            to.push(from[*edge]);
            // Enqueue it for tracing.
            stack.push(new);
            new
        });
    *edge = new_location;
}

As written, this works and is 100% safe!⁴ So where do things start to break down? We’ll get there, but first…

The old-to-new-location map needn’t be an additional, separate allocation. We don’t need that hash map. Instead, we can reuse the from-space’s storage and write the address of each copied object’s new location inline into its old location. These are referred to as forwarding pointers. This is a super standard optimization for copying collection; so much so that it’s rare to see a copying collector without it.

Let’s implement inline forwarding pointers for our safe copying collector. Because we are mutating the from-space to write the forwarding pointers, we will need to change it from a shared borrow into an exclusive borrow. Additionally, to differentiate between forwarding pointers and actual cons cells, our semi-spaces must become slices of an enum rather than slices of cons cells directly.

enum SemiSpaceEntry {
    // The cons cell. If we see this during tracing, that means
    // we haven't copied it over to the to-space yet.
    Occupied(Cons),
    // This cons cell has already been moved, here is its new
    // location.
    Forwarded(usize)
}

fn copy_collect(
    roots: &mut [usize],
    from: &mut [SemiSpaceEntry],
    to: &mut Vec<SemiSpaceEntry>,
) {
    // Same as before, but without `old_to_new`...
}

fn visit_edge(
    from: &mut [SemiSpaceEntry],
    to: &mut Vec<SemiSpaceEntry>>,
    stack: &mut Vec<usize>,
    edge: &mut usize,
) {
    let new = match &mut from[*edge] {
        SemiSpaceEntry::Forwarded(new) => new,
        SemiSpaceEntry::Occupied(cons) => {
            let new = to.len();
            // Copy the object over.
            to.push(cons);
            // Enqueue it for tracing.
            stack.push(new);
            // !!! Write the forwarding pointer. !!!
            from[edge] = SemiSpaceEntry::Forwarded(new);
            new
        }
    };
    *edge = new;
}

Again, this copying collector with forwarding pointers works and is still 100% safe code.

Things break down when we move away from a homogeneously-typed heap that only contains cons cells towards a heterogeneously-typed heap that can contain any type of GC object.

Recall how safe_gc::Heap organizes its underlying storage with a hash map keyed by type id to get the Arena<T> storage for that associated type:

pub struct Heap {
    // A map from `type_id(T)` to `Arena<T>`.
    arenas: HashMap<TypeId, Box<dyn ArenaObject>>,

    // ...
}

My idea was that a whole Heap would be a semi-space, and if it was the active semi-space, the heap would additionally have an owning handle to the idle semi-space:

pub struct Heap {
    // ...

    idle_semi_space: Option<Box<Heap>>,
}

Given that, we would collect the heap by essentially (papering over some details) calling copy_collect on each of its internal arenas:

impl Heap {
    pub fn gc(&mut self) {
        let to_heap = self.idle_semi_space.take().unwrap();
        for (ty, from_arena) in &mut self.arenas {
            copy_collect(from_arena, to_heap);
        }
    }
}

Note that we pass the whole to_heap into copy_collect, not from_arena’s corresponding Arena<T> in the to-space, because there can be cross-type edges. A Cat object can have a reference to a Salami object as a little treat, and we need access to the whole to-space, not just its Arena<Cat>, to copy that Salami over when tracing Cats.

But here’s where things break down: we also need mutable access the whole from-space when tracing Arena<Cat>s because we need to write the forwarding pointer in the from-space’s Arena<Salami> for the Salami’s new location in the to-space. But we can’t have mutable access to the whole from-space because we’ve already projected into one of its arenas. Yeah, I guess we could use something like take the Arena<Cat> out of the from-space, and then pass both the Arena<Cat> and the whole from-space into copy_collect. But then what do we do for Cat-to-Cat edges? Have some kind of check to test for whether we need to follow a given edge into the from-space Heap or the Arena we are currently tracing?

Like I said, I don’t think it is impossible to overcome these hurdles, the question is: is overcoming them worth it? Everything I could think up got pretty inelegant pretty quickly and/or would have laughably poor performance.⁵ When compared with how easy it was to implement mark-and-sweep, I just don’t think a 100% unsafe-free copying collector that supports arbitrary, heterogeneous types is worth the headache.

Why `safe-gc`?

safe-gc is certainly a point in the design space of garbage-collection libraries in Rust. One could even argue it is an interesting — and maybe even useful? — point in the design space!

Also, it was fun!

At the very least, you don’t have to wonder about the correctness of any unsafe code in there, because there isn’t any. As long as the Rust language and its standard library are sound, safe-gc is too.

Conclusion

The safe-gc crate implements garbage-collection-as-library for Rust with zero unsafe code. It was fun to implement!

Thanks to Trevor Elliott and Jamey Sharp for brainstorming with me and thanks to Manish Goregaokar and again to Trevor Elliott for reading early drafts of this blog post.

In the garbage collection literature, we think about the heap of GC-managed objects as a graph where each object is a node in that graph and the graph’s edges are the references from one object to another. ↩
The one exception to this statement that I’m aware of is the gc-arena crate, although it is only half an exception. Similar to safe-gc, it also requires threading through a heap context (that it calls a Mutation) to access GC objects, although only for allocation and mutable access to GC objects. Getting shared, immutable borrows of GC objects doesn’t require threading in a heap context. ↩
I do have sympathy for users writing these bugs! I’ve written them myself. Remembering to root GC references across operations that can trigger collections isn’t always easy! It can be difficult to determine which things can trigger collections or whether some reference you’re holding has a pointer to another structure which internally holds onto a GC reference. The SpiderMonkey GC folks had to resort to implementing a GCC static analysis plugin to find unrooted references held across GC-triggering function calls. This analysis runs in Firefox’s CI because even the seasoned systems engineers who work on SpiderMonkey and Firefox routinely make these mistakes and the resulting bugs are so disastrous! ↩
Well, this collector works in principle; I haven’t actually compiled it. I wrote it inside this text file, so it probably has some typos and minor compilation errors or whatever. But the point stands: you could use this collector for your next toy lisp. ↩
I’m not claiming that safe-gc has incredible performance, I haven’t benchmarked anything and it almost assuredly does not. But its performance shouldn’t be laughably bad, and I’d like to think that with a bit of tuning it would be competitive with just about any other unsafe-free Rust implementation. ↩

My WasmCon 2023 Talk

Thu, 14 Sep 2023 00:00:00 -0700

I recently gave a talk at WasmCon 2023 about the measures we take to ensure security and correctness in Wasmtime and Cranelift. Very similar content to this blog post.

Here is the abstract:

WebAssembly programs are sandboxed and isolated from one another and from the host, so they can’t read or write external regions of memory, transfer control to arbitrary code in the process, or freely access the network and filesystem. This makes it safe to run untrusted WebAssembly programs: they cannot escape the sandbox to steal private data from elsewhere on your laptop or run a botnet on your servers. But these security properties only hold true if the WebAssembly runtime’s implementation is correct. This talk will explore the ways we are ensuring correctness in the Wasmtime WebAssembly runtime and in its compiler, Cranelift.

The slides are available here and the recording is up on YouTube, although unfortunately they missed the first 30 seconds or so:

How Fuzzy are Your Fuzzers?

Mon, 24 Oct 2022 00:00:00 -0700

As long as a fuzzer is uncovering a steady stream of bugs, we can have confidence it’s serving its purpose. But a silent fuzzer is harder to interpret: is our program finally free of bugs, or is the fuzzer simply unable to reach the code in which they are hidden?

Code coverage reports can help here: we can manually check which functions and blocks of code the fuzzer has executed. We can see what coverage is missing that we want or expected to be covered, and then figure out ways to help the fuzzer explore that code. We implement those changes, run the fuzzer again, check the coverage reports again, and can verify our changes had the desired effect.

But how can we be sure that the fuzzer will continue exercising these code paths — especially in evolving code bases with many developers collaborating together? Imagine this scenario: we have a generator that creates test cases that are guaranteed to be syntactically correct, but aren’t guaranteed to type check even if they do in practice 99% of the time. Therefore, our try-and-compile-the-input fuzz target intentionally ignores type errors so it can skip to the next probably-well-typed input, hoping that compiling that next input will trigger an internal compiler assertion or find some other bug. However, some change in one of the generator’s dependencies perturbed the generator so that now it only generates ill-typed programs. After this change, the fuzzer will never exercise our compiler’s mid-end optimizations and backend code generation because it always bounces off the type checker. This is a huge reduction in code exercised by the fuzzer and nothing alerted us to this regression!¹

Manually checking coverage reports every week, month, or whenever you happen to remember is tedious. Even worse, if we accidentally introduce a coverage regression, we won’t catch that until the next manual review. What if we unknowingly cut a release during one of these periods? We could ship bugs that we would otherwise have caught — not good!

This isn’t a hypothetical scenario. We’ve been bitten by it in the Wasmtime project, as detailed in the following quote from one of our security advisories:

This bug was discovered when we discovered that Wasmtime’s fuzz target for exercising GC and stack maps, table_ops, was mistakenly not performing any actual work, and hadn’t been for some time now. This meant that while the fuzzer was reporting success it wasn’t actually doing anything substantive. After the fuzz target was fixed to exercise what it was meant to, it quickly found this issue.

Catching bugs early, before you release them, is much preferable to the alternative! Exposing users to bugs isn’t good and writing security advisories and patches isn’t fun.

A more robust solution than periodic coverage reviews is to manually instrument your code with counters or other metrics and then write tests that run your fuzz target N times and assert that your metrics match your expectations within some margin of error. This technique is really low effort and high reward for fuzz targets that are specifically designed to exercise one corner of your system. Even just a single counter or boolean flag can provide lots of value!

For example, I wrote a Wasmtime fuzz target to test our backtrace capturing functionality. The fuzz target is composed of two parts:

A generator that creates pseudo-random Wasm programs consisting of a bunch of functions that arbitrarily call each other, all while dynamically maintaining a shadow stack of function activations that always reflects actual execution.
An oracle that takes these generated test cases, runs them in Wasmtime, and asserts that when we capture an actual backtrace in Wasmtime, it matches the generated program’s shadow stack of activations. Crucially, the oracle also returns the length of the deepest backtrace that it captured.

Now, we need to make sure that this fuzz target is actually exercising what we want it to, and isn’t going off the rails by, for example, returning early from the first function every time and therefore never actually exercising stack capture with many frames on the stack. To do this, I wrote a regular test that generates random buffers of data with an RNG, generates test cases from that random data, runs our oracle on those test cases, and asserts that we capture a stack trace of length ten in a reasonable amount of time. Easy!

// We should quickly capture a stack at least this deep. We
// consider this deep enough to be a "non-trivial" stack.
const TARGET_STACK_DEPTH: usize = 10;

#[test]
fn stacks_smoke_test() {
    // Use a fixed seed so the corpus of generated test cases
    // is deterministic.
    let mut rng = SmallRng::seed_from_u64(0);
    let mut buf = vec![0; 2048];

    for _ in 0..1024 {
        rng.fill_bytes(&mut buf);

        // Generate a new `Stacks` test case from the raw
        // data, using the `arbitrary` crate.
        let u = Unstructured::new(&buf);
        if let Ok(stacks) = Stacks::arbitrary_take_rest(u) {
            // Run the test case through our `check_stacks`
            // oracle.
            let max_stack_depth = check_stacks(stacks);

            // If we reached our target stack depth, then we
            // passed the test!
            if max_stack_depth >= TARGET_STACK_DEPTH {
                return;
            }
        }
    }

    panic!(
        "never generated a `Stacks` test case that reached \
        {TARGET_STACK_DEPTH} deep stack frames",
    );
}

Now we know that we won’t ever accidentally make a change that silently makes it so that we only test capturing stack traces of depth one in this fuzz target. If we tried to make that change, this test would fail, alerting us to the problem.

Of course, this technique isn’t a silver bullet. For more general fuzz targets that are testing basically the whole system, rather than a specific feature, there isn’t a single counter or metric to rely on. Some code paths might take a while to be discovered by the fuzzer, longer than you’d want to wait for in a unit test, even if it should be found eventually. But maybe there are a few counters you can implement as low-hanging fruit and get 80% of the benefits for 20% of the effort?

Finally, that earlier quote from one of our Wasmtime security advisories ends with the following:

Further testing has been added to this fuzz target to ensure that in the future we’ll detect if it’s failing to exercise GC.

We are confident we’ll detect if that fuzz target starts failing to exercise garbage collection because now we count how many garbage collections are triggered in each iteration of the fuzz target, and assert that we trigger at least one garbage collection within a small number of iterations. Simple and easy to implement, but we’ll never have that particular whoops-we-never-triggered-a-GC-in-this-fuzz-target-designed-to-exercise-the-GC egg on our faces again!

Many thanks to Alex Crichton, Chris Fallin, and Jim Blandy for reading drafts of this blog post and providing valuable feedback!

As an aside, a super neat feature for OSS-Fuzz to grow would be automatically filing an issue whenever a fuzz target’s coverage dramatically drops after pulling the latest code from upstream or something like that. ↩

Security and Correctness in Wasmtime

Tue, 13 Sep 2022 00:00:00 -0700

Note: I am cross-posting this article to my personal blog. The original is on the Bytecode Alliance blog.

The essence of software engineering is making trade-offs, and sometimes engineers even trade away security for other priorities. When it comes to running untrusted code from unknown sources, however, exceptionally strong security is simply the bar to clear for serious participation: consider the extraordinary efforts that Web browser and hypervisor maintainers take to preserve their systems’ integrity. WebAssembly runtimes also run untrusted code from untrusted sources, and therefore such efforts are also a hard requirement for WebAssembly runtimes.

WebAssembly programs are sandboxed and isolated from one another and from the host, so they can’t read or write external regions of memory, transfer control to arbitrary code in the process, or freely access the network and filesystem. This makes it safe to run untrusted WebAssembly programs: they cannot escape the sandbox to steal private data from elsewhere on your laptop or run a botnet on your servers. But these security properties only hold true if the WebAssembly runtime’s implementation is correct. This article will highlight the ways we are ensuring correctness in the Wasmtime WebAssembly runtime and in its compiler, Cranelift.

This is our second blog post leading up to Wasmtime’s upcoming 1.0 release on September 20th, 2022. The first blog post focused on Wasmtime’s performance. We’re ready to release Wasmtime 1.0 because we believe not only that it solidly clears the bar for security and correctness, but also that we have the momentum, processes, and multi-stakeholder investment in place to keep it that way in the future.

A Safe Implementation Language

Wasmtime is implemented in the Rust programming language. Google, Microsoft, and Mozilla have each independently found that around 70% of security bugs in their browsers were memory safety bugs, such as use-after-free bugs and out-of-bounds heap accesses, including security bugs within those browsers’ WebAssembly implementations. Rust helps us avoid this whole class of bugs without sacrificing the low-level control we need to efficiently implement a language runtime. Large portions of Wasmtime even have zero unsafe blocks — such as our WebAssembly parser, which is the first component to process potentially-malicious input — and the parts that necessarily use unsafe to implement primitives are carefully vetted.

Rust does not prevent all bugs, of course. It doesn’t save us from miscompilations due to logic errors in our compiler, for example, that could ultimately lead to Wasm sandbox escapes. We’ll discuss the techniques we use to address these bugs and others that Rust doesn’t catch throughout the rest of this post.

The benefits of a safe implementation language, however, extend to applications embedding Wasmtime. Even a correct WebAssembly runtime’s utility is weakened if the interface to that runtime is unsafe or so clunky that it pushes embedders towards unsafe code out of convenience or to meet performance goals. That’s why we designed Wasmtime’s user-facing API such that misusing it is nearly impossible, using it doesn’t require any unsafe Rust, and that this safety does not also sacrifice performance. Our typed function API, for example, leverages Rust’s type system to do a single type check up front when taking a reference to a WebAssembly function, and subsequent uses — such as calls into WebAssembly through that function — don’t do repeated checks. Our strongly-typed APIs let us statically maintain critical safety invariants for users, avoiding the potential for misuse and the overhead of repeated dynamic checks.

Securing Our Supply Chain

Malicious dependencies are becoming more common. An attacker gains control over a library that your application depends on, adds code to steal your SSH keys, and your world falls apart the next time you run a build. We cannot let Wasmtime — and by extension, any application that embeds Wasmtime — be compromised by malicious third-party dependencies.

The WebAssembly component model will help protect against these attacks with capabilities-based security and lightweight isolation between software components. Unfortunately that can’t be a solution for Wasmtime itself, since Wasmtime needs to implement the component model and sits below that abstraction level.

To secure Wasmtime against malicious dependencies, we are using cargo vet. Mozilla created this tool to mechanically ensure that all third-party Rust libraries used inside Firefox have been manually reviewed by a trusted auditor.

When performing an audit, reviewers double check:

the use of unsafe,
that potentially-malicious, user-supplied data is handled with care (e.g. there is no recursion over user input that could let attackers craft inputs that cause the library to blow the stack),
that a markdown-parsing library, for example, doesn’t access the file system or network when it shouldn’t need those capabilities,
and that using the crate won’t otherwise open the door to security vulnerabilities in production.

Using cargo vet we now require that a trusted Wasmtime maintainer manually reviews all new dependencies and the delta for updates to existing dependencies. At the same time, we are burning down the list of yet-to-be-reviewed libraries that Wasmtime already depended upon before we adopted cargo vet.

cargo vet benefits from network effects: it allows us to import audits from another organization, so the more trustworthy organizations that start using cargo vet and auditing dependencies, then the fewer audits we will have to perform ourselves. And the more organizations that trust our audits, the more utility each of our audits provides. Right now, Wasmtime imports and trusts Firefox’s audits, Firefox likewise imports and trusts Wasmtime’s audits, and we hope to expand this as the cargo vet community grows.

Enabling Secure Application Designs

The security of applications using Wasmtime isn’t just determined by Wasmtime’s development process. It is also determined by how Wasmtime unlocks more-secure application designs that couldn’t have been considered before, because the performance overhead was impractical. One example is our ongoing standardization and implementation work on the previously-mentioned WebAssembly component model, and composing WebAssembly programs while maintaining isolation and performance. Another is the “disposable instance” paradigm.

We’ve worked hard to make instantiating WebAssembly instances so fast that you can create a fresh instance per task, throw it away when the task is completed, and create another new instance for the next task. This means that you can instantiate a fresh WebAssembly instance per HTTP request in a serverless application, for example. It provides isolation between tasks, so if the WebAssembly module has a bug that is triggered by the input for one task, that bug can’t automatically infect all other subsequent tasks. This wouldn’t be possible without Wasmtime’s fast instantiation.

Ubiquitous Fuzzing

Fuzzing is a software testing technique used to find security and correctness issues by feeding pseudo-random data as input into the system you’re testing:

fn fuzz() {
    loop {
        // Generate some new input.
        let input = generate_pseudo_random_data();

        // Feed that input into the system under test.
        let result = run_system_under_test(input);

        // Finally, if the system under test crashed,
        // failed an assertion, etc... then report
        // that!
        if result.is_interesting() {
            report(input);
        }
    }
}

We love fuzzing. We do continuous fuzzing in the background, 24/7. We do targeted fuzzing while developing new features. We fuzz in the large (e.g. all of Wasmtime) and the small (e.g. just our WebAssembly text format parser). We contribute to and help maintain some of the core fuzzing infrastructure for the whole Rust ecosystem. Our pervasive fuzzing is probably the biggest single contributing factor towards Wasmtime’s code quality.

We fuzz because writing tests by hand, while necessary, is not enough. We are fallible humans and will inevitably miss an edge case. Our minds aren’t twisted enough to come up with the kinds of inputs that a fuzzer will eventually find.

Fuzzing can be as simple as throwing random bytes at a WebAssembly binary parser and looking for any crashes. It can be as complex as generating arbitrary, guaranteed-valid WebAssembly modules, compiling them with and without optimizations, and asserting that running them yields the same results either way. We do both.

We primarily use libFuzzer, a coverage-guided fuzzing engine developed as part of the LLVM project, for our fuzzing. Our fuzzers run 24/7 as part of the OSS-Fuzz project. We contribute to and help maintain the cargo fuzz tool that makes building and running fuzzers in Rust easy, the arbitrary crate for fuzzing with structured data, and the libfuzzer-sys crate that provides Rust bindings to libFuzzer.

We use generators to create new, pseudo-random test cases, and oracles to check security and correctness properties when evaluating those test cases in Wasmtime.

We have a variety of generators, but the one we use most is wasm-smith. We wrote wasm-smith to produce pseudo-random WebAssembly modules that are guaranteed valid. It helps us test deeper within Wasmtime and Cranelift by not bouncing off the WebAssembly parser because of a malformed memory definition or failing the validator because of a type error inside a function body. It has configuration options to avoid generating code that will trap at runtime, to only generate certain kinds of instructions such as numeric instructions, and to turn various WebAssembly proposals on and off, among many other things. We like to use swarm testing to let the fuzzer dynamically configure the kinds of test cases we generate, improving the diversity of our generated test cases. Firefox has also started using wasm-smith to exercise its WebAssembly engine.

We use a variety of oracles in our fuzzing:

Did the program crash or fail an assertion?
If we capture the WebAssembly’s stack, do we see the expected stack frames?
Do we have the expected number of garbage collector-managed allocations and deallocations? Are we unexpectedly leaking?
Can we round trip a WebAssembly module through our parser, disassembler, and assembler and get the original input again?
Do we get the same results evaluating the input in:
- Wasmtime with and without compiler optimizations enabled?
- Wasmtime and V8?
- Wasmtime and (a formally verified version of) the WebAssembly specification’s reference interpreter?
And many more.

We even have a symbolic checker for register allocation that we use as an oracle. The checker proves that a given allocation to a bounded number of physical registers correctly implements the original program that used an unbounded number of virtual registers, regardless of what actual values are given to the program or which control-flow paths are taken. We then generate arbitrary control-flow graphs of basic blocks containing instructions that operate on virtual registers, ask the register allocator to assign the virtual registers to physical registers, and finally use this checker as an oracle to assert that the assignment is correct.

When we implement new features in Wasmtime, we write generators and oracles specifically designed to exercise these new features. The symbolic register allocation checker is one example, as it was developed alongside a new register allocator for Cranelift. When implementing new WebAssembly proposals in Wasmtime, the baseline is adding support for the new proposal in wasm-smith. But we will also do things like create generators for testing the inline garbage collector write barriers that we emit in our compiled code when WebAssembly’s reference types proposal is enabled. And we developed a fuzzer for the component model’s interface functions in concert with their implementation in Wasmtime. We have fully embraced “fuzz-driven development”.

Formal Verification Efforts

Fuzzing gives us a statistical claim that our program is correct with respect to what the fuzzer is exercising. The longer we run the fuzzer, the closer that claim gets to 100%, but in general we’ll never reach 100% because our input space is way too large or even infinite. This is where our efforts to formally verify parts of Wasmtime and Cranelift come in.

The VeriWasm project — a collaboration between UCSD, Stanford, and Fastly — is a translation validator for WebAssembly programs compiled with Cranelift. It proves that the compiled program’s control-flow and memory accesses cannot escape its isolated sandbox. This is not a claim about a handful of inputs that we ran with a fuzzer, it proves this true for all inputs that could be given to the compiled program.

We’ve recently redesigned instruction selection in Cranelift to be defined via rewrite rules in a domain-specific language we call ISLE (Instruction Selection and Lowering Expressions). We have an ongoing collaboration with some folks from Cornell and CMU to formally verify the correctness of our ISLE-based instruction selection, proving that the machine instructions we emit do in fact implement the input Cranelift IR for all values they could be given. If it discovers an incorrect rule, the verifier will give us a counterexample. The counterexample is an input where the original Cranelift IR evaluates to one value, and the lowered machine instructions evaluate to a different value. The counterexample and its divergent results will help us diagnose and fix our buggy rule.

Looking further ahead, we are investigating refactoring Cranelift’s middle end to use ISLE and rewrite rules. This will let us formally verify the correctness of these rewrite rules, and further shrink our unverified, trusted compute base. We intend to keep applying this process to the whole compiler pipeline.

Spectre Mitigations

Spectre is a class of attacks exploiting speculative execution in modern processors. Speculative execution is when the processor guesses where control will flow — even though it has not actually computed branch conditions or indirect jump targets yet — and starts tentatively executing its guess. When the processor guesses correctly, the speculative execution’s results are used, speeding up the program; when it guesses incorrectly, they are discarded. Unfortunately, even discarded speculations can still affect the contents of caches and other internal processor state. An attacker can indirectly observe these effects by measuring the time it takes to perform operations that access that same internal state. Under the right conditions, this allows the attacker to deduce what happened in discarded speculative executions and effectively “see past” bounds checks and other security measures.

It is tempting to take a heavy-handed approach to defending against Spectre attacks. Operating system process boundaries are a common mitigation, however one of WebAssembly’s most enticing features is its lighter-weight-than-a-process isolation. Additionally, attackers must have access to a timer to pull off a Spectre attack, and while it is tempting to block access to timer APIs, it is surprisingly easy to find widgets that can be made into timers. The nature and severity of Spectre vulnerabilities depend greatly on context; the mitigations described below can form part of overall protection.

Wasmtime implements a number of Spectre mitigations to prevent speculative execution from leaking information to malicious programs:

Function table bounds checks are protected from speculative attack, ensuring that speculated call_indirect instructions cannot transfer control to an arbitrary location.
The br_table instruction is protected from speculative attack, ensuring that speculation cannot transfer control to an arbitrary location.
Wasmtime’s default configuration for WebAssembly linear memories elides explicit bounds checks, relying on virtual memory guard pages instead. However, when virtual memory guard pages are disabled and we must emit explicit bounds checks, we additionally emit mitigation code that prevents speculated accesses from escaping the linear memory.
We’re implementing support for hardware control-flow integrity features which could help mitigate Spectre attacks, such as BTI on aarch64.

Security researchers keep discovering new Spectre attacks and inventing better mitigations for them. Therefore, we expect we will keep expanding and refining Wasmtime’s Spectre mitigations in the future as well.

A Plan When Things Go Wrong

Even the most carefully crafted plans can go wrong, so we have backup plans for bugs that slip past our safeguards. It begins with our guidelines for reporting security bugs and our disclosure policy. Handling a security bug is a delicate matter, and we don’t want to make mistakes, so we have a vulnerability response runbook to walk ourselves through responding to security bugs in the moment. Once a patch is written, we backport security fixes to the two most-recent Wasmtime releases, as per our release process.

We’ve tested this safety net when faced with Cranelift miscompilations and incomplete stack maps for garbage collection.

Conclusion

This article detailed how Wasmtime uses language safety, fine-grained isolation, dependency auditing, fuzzing, and verification to bolster its security posture and the security postures of applications embedding Wasmtime and the WebAssembly programs Wasmtime runs. We believe that these are the minimum practices you should demand from WebAssembly runtimes when running untrusted or security-sensitive code, and we are constantly trying to raise this bar and strengthen Wasmtime’s security and correctness assurances.

Hit the Ground Running: Wasm Snapshots for Fast Start Up

Mon, 10 May 2021 00:00:00 -0700

I gave a (virtual) talk at the WebAssembly Summit this year titled “Hit the Ground Running: Wasm Snapshots for Fast Start Up”. Here is the talk’s abstract:

Don’t make your users wait while your Wasm module initializes itself! Wizer instantiates your WebAssembly module, executes its initialization functions, and then snapshots the initialized state out into a new, pre-initialized WebAssembly module. Now you can use this new module to hit the ground running, without waiting for any of that first-time initialization code to complete. This talk will cover the design and implementation of Wizer; discuss its performance characteristics and the scenarios in which it excels and when it isn’t the right tool; and finally, in the process of doing all that, we’ll take a closer look at what makes up the guts of a WebAssembly module: memories, globals, tables, etc.

You can view the slide deck here, check out the benchmarks here, and the recording is embedded below:

WebAssembly Reference Types in Wasmtime

Thu, 27 Aug 2020 00:00:00 -0700

Note: I am cross-posting this article from the Bytecode Alliance blog to my personal blog.

A few weeks ago, I finished implementing support for the WebAssembly reference types proposal in Wasmtime. Wasmtime is a standalone, outside-the-Web WebAssembly runtime, and the reference types proposal is WebAssembly’s first foray beyond simple integers and floating point numbers, into the exciting world of garbage-collected references. This article will explain what the reference types proposal enables, what it leaves for future proposals, and how it is implemented in Wasmtime.

What are Reference Types?

Without the reference types proposal, WebAssembly can only manipulate simple integer and floating point values. It can’t take or return references to the host’s objects like, for example, a DOM node on a Web page or an open connection on a server. There are workarounds: for example, you can store the host objects in a side table and refere to them by index, but this adds an extra indirection and implementing the side table requires cooperating glue code on the host side. That glue code, in particular, is annoying because it is outside of the Wasm sandbox, diluting Wasm’s safety guarantees, and it is host-specific. If you want to use your Wasm module on the Web, in a Rust host, and in a Python host, you’ll need three separate glue code implementations. This makes WebAssembly less attractive as a universal binary format.

With the reference types proposal, you don’t need glue code to interact with host references. The proposal has three main parts:

A new externref type, representing an opaque, unforgable reference to a host object.
An extension to WebAssembly tables, allowing them to hold externrefs in addition to function references.
New instructions for manipulating tables and their entries.

With these new capabilities, Wasm modules can talk about host references directly, rather than requiring external glue code running in the host.

externrefs play nice with WebAssembly’s sandboxing properties:

They are opaque: a Wasm module cannot observe an externref value’s bit pattern. Passing a reference to a host object into a Wasm module doesn’t reveal any information about the host’s address space and the layout of host objects within it.
They are unforgable: a Wasm module can’t create a fake host reference out of thin air. It can only return either a reference you already gave it or the null reference. It cannot pretend like the integer value 0x1bad2bad is a valid host reference, return it to you, and trick you into dereferencing this invalid pointer.

Example

Here’s a Wasm module that exports a hello function which takes, as an externref parameter, a reference to an open file and writes “Hello, Reference Types!” to the file. This module imports the write syscall from a hypothetical future version of WASI, the WebAssembly System Interface, that leverages reference types.

;; hello.wat

(module
  ;; Import the write syscall from a hypothetical (and
  ;; simplified) future version of WASI.
  ;;
  ;; It takes a host reference to an open file, an address
  ;; in memory, and byte length and then writes
  ;; `memory[address..(address+length)]` to the file.
  (import "future-wasi" "write"
    (func $write (param externref i32 i32) (result i32)))

  ;; Define a memory that is one page in size (64KiB).
  (memory (export "memory") 1)

  ;; At offset 0x42 in the memory, define our data string.
  (data (i32.const 0x42) "Hello, Reference Types!\n")

  ;; Define a function that writes our hello string to a
  ;; given open file handle.
  (func (export "hello") (param externref)
    (call $write
      ;; The open file handle we were given.
      (local.get 0)
      ;; The address of our string in memory.
      (i32.const 0x42)
      ;; The length of our string in memory.
      (i32.const 24))
    ;; Ignore the return code.
    drop))

This Wasm module will run in any WebAssembly environment where WASI is available.⁰ We don’t need glue code that is maintaining a side table, mapping host references to indices.

Python

For example, we can run hello.wat from a Python host environment without any glue code:

from wasmtime import *
import future_wasi

# Initial configuration, enabling reference types.
config = Config()
config.wasm_reference_types = True
engine = Engine(config)
store = Store(engine)

# Compile our `hello.wat` module.
module = Module.from_file(engine, "./hello.wat")

# Create an instance of the module and give it access
# to our `future-wasi.write` polyfill.
instance = Instance(store,
                    module,
                    [future_wasi.write(store)])

# Finally, open a file for writing and pass it into
# our Wasm instance's `hello` function!
with open("hello.txt", "wb") as open_file:
    instance.exports["hello"](open_file)

Rust

And we can run the exact same hello.wat from a Rust host and without any Rust-specific bindings or glue:

use anyhow::Result;
use std::{cell::RefCell, fs};
use wasmtime::*;

mod future_wasi;

fn main() -> Result<()> {
    // Initial configuration, enabling reference types.
    let mut config = Config::new();
    config.wasm_reference_types(true);
    let engine = Engine::new(&config);
    let store = Store::new(&engine);

    // Compile our `hello.wat` module.
    let module = Module::from_file(
        &engine,
        "./hello.wat",
    )?;

    // Create an instance of the module and give it
    // access to our `future-wasi.write` polyfill.
    let instance = Instance::new(&store, &module, &[
        future_wasi::write(&store).into(),
    ])?;

    // Get the instance's `hello` function export.
    let hello = instance
        .get_func("hello")
        .ok_or(anyhow::format_err!(
            "failed to find `hello` function export"
        ))?
        .get1::<Option<ExternRef>, ()>()?;

    // Open a file for writing and pass it into our Wasm
    // instance's `hello` function.
    let file = ExternRef::new(RefCell::new(
        fs::File::create("hello.txt")?,
    ));
    hello(Some(file))?;

    Ok(())
}

Running hello.wat — once again, without module-specific glue code — is just as easy in .NET and Go host environments, and is left as an exercise for the reader ;)

On the Web

Unlike the other host environments we’ve considered, WASI isn’t natively implemented on the Web. There’s nothing stopping us, however, from polyfilling WASI APIs with a little bit of JavaScript and a couple DOM methods! This is still an improvement because there is overall less module-specific glue code. Once one person has written the polyfills, everyone’s Wasm modules can reuse them.

There are many different things an “open file” could be modeled by on the Web. For this demo, we’ll use a DOM node: writing to it will append text nodes. This works well because we know our module is only writing text data. If we were working with binary data, we would choose another polyfilling approach, like in-memory array buffers backing the file data.

Here is the JavaScript code to run our hello.wat module and polyfill our future-wasi.write function:

// The DOM node we will use as an "open file".
const output = document.getElementById("output");

async function main() {
  // Define the imports we are giving to the Wasm
  // module: our `future-wasi.write` polyfill.
  const imports = {
    "future-wasi": {
      write(domNode, address, length) {
        // Get `memory[address..address + length]`.
        const memory = new Uint8Array(
          instance.exports.memory.buffer
        );
        const data = memory.subarray(
          address,
          address + length
        );

        // Convert it into a string.
        const decoder = new TextDecoder("utf-8");
        const text = decoder.decode(data);

        // Finally, append it to the given DOM node that
        // is our "open file".
        const textNode = document.createTextNode(text);
        domNode.appendChild(textNode);
      }
    }
  };

  // Fetch and instantiate our Wasm module.
  const response = await fetch("./hello.wasm");
  const wasmBytes = await response.arrayBuffer();
  const { instance } = await WebAssembly.instantiate(
    wasmBytes,
    imports
  );

  // Call its exported `hello` function with a DOM node.
  instance.exports.hello(output);

  // Every time the button is clicked, call the exported
  // `hello` function again.
  const button = document.getElementById("call-hello");
  button.removeAttribute("disabled");
  button.addEventListener("click", () => {
    instance.exports.hello(output);
  });
}

main().catch(e => {
  output.textContent = `${e}\n\nStack:\n${e.stack}`;
});

You can view a live demo of this code in the iframe below. It should work in Firefox 79+, or any other browser that supports reference types (at the time of writing, Chrome has support for an older version of the reference types proposal behind the --experimental-wasm-anyref flag, and Safari has in-progress support behind the JSC_useWebAssemblyReferences flag.)

What Comes After the Reference Types Proposal?

The in-progress WebAssembly type imports proposal builds on reference types to let you distinguish between different kinds of host objects: database connections and open files would be separate types. Or, on the Web, Wasm would distinguish JavaScript Promises from DOM nodes. It also adds references that point to types defined by other Wasm modules, not just host objects.

WASI will want to adopt unforgable reference types for file handles, as the examples above suggest, instead of using integers. It might make sense for WASI to wait for type imports, however, so it can use separate types for, say, an open file and a source of entropy. This is a decision for the WASI subgroup of the WebAssembly Community Group. Either way, when WASI does make this switch, it will continue to support integer file descriptors, but implemented inside the Wasm sandbox as indices into an externref Wasm table.

The interface types and module linking proposals promise to remove even more glue code and bindings. The interface types proposal lets Wasm modules exchange rich, structured types with each other or with the host. It translates between interface types and core Wasm types, while keeping modules encapsulated from each other. For example, a module can exchange in-memory strings and arrays without exporting the whole memory or exposing allocator methods, both of which are currently required without interface types. The module linking proposal makes modules and instances first class values, allowing them to be defined, imported, exported, and instantiated within other WebAssembly modules. This functionality previously relied on the host’s cooperation, requiring a snippet of JavaScript to fetch and instantiate modules, hooking up their imports and exports.

Even further in the future, full support for garbage-collected objects in WebAssembly will come eventually.

Implementation Details

Now we’ll dive into the details of Wasmtime’s reference types implementation.

With externrefs, the host gives Wasm limited access to host objects, but the host also needs to know when Wasm is finished with those objects, so it can clean them up. That clean up might involve closing a file handle or simply deallocating memory. Statically running clean up routines once Wasm returns to the host is attractive, but unfortunately flawed: Wasm can persist an externref into a table or global so that it outlives the function call. We do not, therefore, know ahead of time whether Wasm is still holding references to host objects or not. We require a dynamic method for determining which objects are still in use and which are not. This typically involves either reference counting or a tracing collector or a combination of the two.

This is all to say that implementing the reference types proposal introduced a garbage collector in Wasmtime.

Much of WebAssembly’s value rests on its predictable performance and minimal nondeterminism. Garbage collectors are infamous for their unpredictable pauses, and, if they support finalizers or weak references, nondeterministic behavior. Implementing a collector to support reference types required reconciling these contradictions as much as possible, while also balancing implementation complexity.

The design we settled upon is a deferred reference counting collector, with Cranelift, Wasmtime’s just-in-time (JIT) compiler, producing stack maps for precisely identifying roots inside of Wasm frames. This approach balances predictable latency with throughput. It is more complex than conservative stack scanning, but avoids the nondeterminism inherent in that approach. It also has the desirable property that if a Wasm module doesn’t use reference types, then it can’t trigger garbage collection and the runtime will not impose any other related overhead.

We chose reference counting over tracing garbage collection for a few reasons. First, it lends itself to short and predictable GC pauses. Second, adding reference counting to a runtime that wasn’t built from the ground up with a garbage collector in mind is easier than adding a tracing collector.

Excavating exactly what it is that makes reference counting easier to add post-facto is a worthwhile exercise, because this has implications for the whole runtime and for any application intending to embed it. Tracing collectors identify all live objects, starting from known-live roots, and then any object that wasn’t found to be live is, by definition, no longer in use, and is therefore safe to reclaim. Reference counting collectors start from known-dead roots, find all the dead objects reachable only from those roots, and then reclaim these known-dead objects. Consider what happens, with each type of collector, if an object A fails to reveal to the collector that it is referencing another object B. In a tracing collector, if no other object is referencing B, then collector will conclude that B is not live and that it is safe to reclaim B. But now A can use B after it has been reclaimed, leading to use-after-free memory unsafety. On the other hand, with a reference counting collector, if A does not reveal that it has the only reference that is keeping B alive, then B’s reference count will not be decremented, then the collector never reclaims B and the object is permanently leaked. While leaking B is not ideal, a leak is much safer than a dangling pointer. Reference counting collectors are easier to add to an existing runtime, and make embedding that runtime in larger applications easier, because they have a better failure mode than tracing collectors do. By default, reference counting collectors safely leak, while tracing collectors unsafely allow use-after-free.

Deferred Reference Counting

In normal reference counting, each time a new reference to an object is created, the object’s reference count is incremented, and each time a reference to the object is dropped, its reference count is decremented. When the reference count reaches zero, the object is deallocated. In deferred reference counting, these increments and decrements are not performed immediately, but are deferred until a later time and then processed together in bulk. This assuages one of reference counting’s biggest weaknesses, the overhead of frequently modifying reference counts, by trading prompt deallocation for better throughput.

With naïve reference counting, every single local.get and local.set WebAssembly instruction operating on a reference type needs to manipulate reference counts. This leads to many increments and decrements, most of which are redundant. It also requires code (known as landing pads) for decrementing the reference counts of the externrefs inside each frame during stack unwinding — which can happen, for example, because of an out-of-bounds heap access trap.

By using deferred reference counting for externrefs inside of Wasm frames, we don’t increment or decrement the reference counts at all, unless a reference escapes a call’s stack frame by being stored into a table or global. Additionally, we don’t need to generate landing pads for unwinding because deferred reference counting can already tolerate slack between when a reference to an object is created or dropped and when the object’s reference count is adjusted to reflect that.

Wasmtime implements deferred reference counting by maintaining an over-approximation of the set of externrefs held alive by Wasm stack frames. We call this the VMExternRefActivationsTable. When we pass an externref into Wasm, we insert it into the table. We do not update the table as Wasm runs, so as execution drops references, the table becomes an over-approximation of the set of externrefs actually present on the stack.

Garbage collection is then composed of two phases. In the first phase, we walk the stack, building a set of externrefs currently held alive by Wasm stack frames. This is the precise set that the VMExternRefActivationsTable approximates. The second phase reconciles the difference between the over-approximation and the precise set, decrementing the reference count for each object that is in the over-approximation but not in the precise set. At the end of this phase, we reset the VMExternRefActivationsTable to the new precise set.

If we are not careful with how we schedule the deferred processing of reference counts, we risk introducing nondeterminism. Using a timer-based GC schedule, for example, means that we are at the whims of the operating system’s thread scheduler and a vast variety of other factors that perturb how much we have executed in a given amount of time. Instead, we trigger GC whenever the VMExternRefActivationsTable reaches capacity, or whenever GC is explicitly requested by the embedder application. As long as the embedder triggers GC deterministically, we maintain deterministic GC scheduling, and execution remains deterministic even in the face of finalizers.

It is worth noting that outside of Wasm frames, in native VM code implemented in Rust, we use regular, non-deferred reference counting: cloning increments the count and dropping decrements it. However, Rust’s moves and borrows let us avoid some of the associated overhead. Moving an externref transfers ownership without affecting the reference count. Borrowing an externref lets us safely use the value without incrementing its reference count, or at least delays the increment until the borrowed reference is cloned.

Stack Maps

For the collector to find the GC roots within a stack frame it requires that the compiler emit stack maps. A stack map records which words in a stack frame contain live externrefs at each point in the function where GC may occur. The stacks maps, taken together, logically record a table of the form:

Instruction Address	Offsets of Live GC References
0x12345678	2, 6, 12
0x1234abcd	2, 6
...	...

The offsets denoting where live references are stored within a stack frame are relative to the frame’s stack pointer and are expressed in units of words. Since garbage collection can only occur at certain points within a function, the table is sparse and only has entries for instruction addresses where GC is possible.

Because offsets of live GC references are relative to the stack pointer, and because stack frames grow down from higher addresses to lower addresses, to get a pointer to a live reference at offset x within a stack frame, the collector adds x to the frame’s stack pointer. For example, to calculate the pointer to the live GC reference inside “frame 1” below, the collector would compute frame_1_sp + x:

          Stack
        +-------------------+
        | Frame 0           |
        |                   |
   |    |                   |
   |    +-------------------+ <--- Frame 0's SP
   |    | Frame 1           |
 Grows  |                   |
 down   |                   |
   |    | Live GC reference | --+--
   |    |                   |   |
   |    |                   |   |
   V    |                   |   x = offset of live GC ref
        |                   |   |
        |                   |   |
        +-------------------+ --+--  <--- Frame 1's SP
        | Frame 2           |
        | ...               |

Each individual stack map is associated with just one instruction address within a compiled Wasm function, contains the size of the stack frame, and represents the stack frame as a bitmap. There is one bit per word in the stack frame; if the bit is set, then the word contains a live GC reference.

The actual stack walking functionality is provided by the backtrace crate, which wraps libunwind on unix-like platforms and DbgHelp on Windows. To assist in stack walking, Cranelift emits the platform’s unwind information: .eh_frame on unix-like platforms and structured exception handling for Windows.

Inline Fast Paths for GC Barriers

Despite our deferred reference counting scheme, compiled Wasm code must occasionally manipulate reference counts and run snippets of code called GC barriers. By running GC barriers, the program coordinates with the collector, helping it keep track of objects.

Because of our deferred reference counting, Wasmtime does not need barriers for references that stay within a Wasm function’s scope, but barriers are still needed whenever a reference enters or escapes the function’s scope. Recall that a reference can escape the scope when written into a global or table. It can, similarly, enter a Wasm call’s scope when read from a global or table. Therefore, reading and writing to globals and tables requires barriers.

The barriers for writing a reference into a table slot and a global are similar so, for simplicity, I’ll just refer to tables from now on. These write barriers are responsible for ensuring that:

The new object’s reference count is incremented now that the table is holding a reference to it.
The table element’s prior value, if any, has its reference count decremented.
If the prior element’s reference count reaches zero, then its destructor is called and memory block deallocated.

There are two subtleties here. First, steps 1 and 2, although they may seem independent, must be performed in the given order. If their order were reversed, then if the new object assigned to the table slot and old object currently in the table slot are the same, we would:

Decrement the object’s reference count from one to zero.
Run the object’s destructor and deallocate it.
Re-assign a reference to the (now freed) object into the table slot.
Increment the (now freed) object’s reference count.

That’s a use-after-free bug! To avoid it, we must always increment the new object’s reference count before decrementing the old object’s reference count. This way if they are the same object then the reference count never reaches zero.

The second subtlety is that an object’s destructor can do pretty much anything, including touch the table slot we are currently running GC barriers for. If we encounter this kind of reentrancy, we want the destructor to see the new table element. We do not want it to see the half-deinitialized object and let it attempt to resurrect the object.

Most barrier executions operate on non-null references, and most executions don’t decrement a reference count to zero and destroy an object. Therefore, the JIT emits the reference counting operations inline, and only calls out from Wasm to VM code when destroying an object:

inline table_set_barrier(table, index, new_value):
    if new_value is not null:
        new_value->ref_count++

    let old_value = table[index]
    table[index] = new_value

    if old_value is not null:
        old_value->ref_count--
        if old_value->ref_count is zero:
            call out_of_line_destroy(old_value)

The other scenario for which Wasmtime requires GC barriers is when Wasm reads a reference from a table (or global), causing the reference to enter the scope of the Wasm call. Its responsibility is ensuring that these references are safely held alive by the VMExternRefActivationsTable. The VMExternRefActivationsTable has a simple bump allocation chunk to support fast insertions from inline JIT code. We maintain a next finger pointing within that bump chunk, and an end pointer pointing just after it. The next finger is where we we will insert the next new entry into the bump chunk, unless next is equal to end which means that the bump chunk is at full capacity. When that happens we are forced to call an out-of-line slow path that will trigger GC to free up space.

inline table_get_barrier(table, index):
    let elem = table[index]
    if elem is not null:
        let (next, end) = bump region
        if next != end:
            elem->ref_count++
            *next = elem
            next++
        else:
            call out_of_line_insert_and_gc(elem)
    return elem

Conclusion

The reference types proposal is WebAssembly’s first expansion beyond simple integers and floating point numbers, requiring that Wasmtime grow a garbage collector. It also cuts down on the amount of module-specific and host-specific glue code. Future proposals, like interface types and module linking, should completely remove the need for such glue.

Thanks to Alex Crichton for reviewing the bulk of this work, and exposing reference types in the wasmtime-go API. Thanks to Dan Gohman for reviewing the code that implements the inline fast paths for GC barriers and the cranelift-wasm interface changes it required. Thanks to Peter Huene for exposing reference types in wasmtime-dotnet. Finally, thanks to Jim Blandy, Dan Gohman, and Alex Crichton for reading early drafts of this article and providing valuable feedback.

⁰ However, since this is a hypothetical future version of WASI, we will need to temporarily define our own version of the write syscall. We will, furthermore, need to define this write polyfill once for each host.

Python's future-wasi.write polyfill

from wasmtime import *

def _write_impl(caller, file, address, length):
    # Get the module's memory export.
    memory = caller.get("memory")
    if not isinstance(memory, Memory):
        return -1

    # Check that the range of data to write is in bounds.
    if address + length > memory.data_len:
        return -1

    # Read the data from memory and then write it to the
    # file.
    data = memory.data_ptr[address : address + length]
    try:
        file.write(bytes(data))
        return 0
    except IOError:
        return -1

def write(store):
    return Func(store,
                FuncType([ValType.externref(),
                          ValType.i32(),
                          ValType.i32()],
                         [ValType.i32()]),
                _write_impl,
                access_caller = True)

Rust's future-wasi.write polyfill

use std::{cell::RefCell, fs, io::Write};
use wasmtime::*;

fn write_impl(
    caller: Caller,
    file: Option<ExternRef>,
    address: u32,
    len: u32,
) -> i32 {
    // Get the module's exported memory.
    let memory = match caller
        .get_export("memory")
        .and_then(|export| export.into_memory())
    {
        Some(m) => m,
        None => return -1,
    };
    let memory = unsafe { memory.data_unchecked() };

    // Get the slice of data that will be written to
    // the file.
    let start = address as usize;
    let end = start + len as usize;
    let data = match memory.get(start..end) {
        Some(d) => d,
        None => return -1,
    };

    // Downcast the `externref` into an open file.
    let file = match file
        .as_ref()
        .and_then(|file| {
            file.data().downcast_ref::<RefCell<fs::File>>()
        })
    {
        Some(f) => f,
        None => return -1,
    };

    // Finally, write the data to the file!
    let mut file = file.borrow_mut();
    match file.write_all(data) {
        Ok(_) => 0,
        Err(_) => -1,
    }
}

pub fn write(store: &Store) -> Func {
    Func::wrap(&store, write_impl)
}

These polyfills are only required until WASI is updated to leverage reference types. ↩

Writing a Test Case Generator for a Programming Language

Mon, 24 Aug 2020 00:00:00 -0700

Maxime Chevalier-Boisvert requested resources for learning about fuzzing programming language implementations on Twitter:

I’d like to learn about fuzzing, specifically fuzzing programming language implementations. Do you have reading materials you would recommend, blog posts, papers, books or even recorded talks?

@Love2Code · August 3, 2020

Maxime received many replies linking to informative papers, blog posts, and lectures. John Regehr suggested writing a simple generative fuzzer for the programming language.

A generative fuzzer combines a test case generator with the system under test (e.g. your compiler), generating new test cases and feeding them into the system:

fn generative_fuzzer() {
    loop {
        // Use the test case generator to create a new
        // input.
        let input = generate_test_case();

        // Feed that input into the system under test.
        let result = run_system_under_test(input);

        // Finally, if the system under test crashed,
        // failed an assertion, etc... then report
        // that!
        if result.is_interesting() {
            report(input);
        }
    }
}

I realized that many people might not know what it takes to write their own generative fuzzer, so this blog post shows one aspect of it: implementing a test case generator.

Our test case generator will generate WebAssembly programs. While WebAssembly has its own quirks — it’s a binary format and is generally a compilation target rather than a source language — it is a small and simple language. The techniques we use when generating WebAssembly should transfer to generating the programming language of your choice.

If you want to skip the exposition and jump head first into the code, here is the repository for our final test case generator.

What is a Test Case Generator?
Getting Set Up
Translating Grammars into Generators
Generating the Type Section
Generating the Import Section
Generating the Code Section
Using the Test Case Generator
Conclusion

What is a Test Case Generator?

Test case generators generate test cases. These test cases are always within the test domain: no cycles are wasted on invalid inputs, such as source text that fails to parse. Compare this to mutation-based fuzzing, where existing seed inputs are mutated to produce new inputs. In general, nothing guarantees that the new, mutated input is still within the test domain: the mutation may have introduced a syntax error. This property, that generated inputs are always within the test domain, is generative fuzzing’s main advantage and the test case generator’s main responsibility.

A test case generator should, additionally, support every feature of its target programming language. You won’t discover a bug in your compiler’s handling of switch statements if the test case generator doesn’t support generating switch statements. Pushing this idea even further, the test case generator should uniformly sample from the test domain. If the test case generator can technically generate switch statements but the probability of doing so is nearly zero, then you likely still won’t find that bug. However, uniformly sampling from the infinite set of all programs that can be written in a particular programming language is nontrivial and an area of active research.

A test case generator should, finally, be fast. The faster we can generate test cases, the faster we will discover bugs. If the generator is too slow, we can blow our time budget, failing to find those bugs at all.

Getting Set Up

First, we create a new crate with cargo. We’ll name this crate wasm-smith, giving a little nod to Csmith, the popular C program generator.

$ cargo new --lib wasm-smith

Second, we add the arbitrary crate as a dependency:

# wasm-smith/Cargo.toml

[dependencies]
arbitrary = { version = "0.4.6", features = ["derive"] }

The arbitrary crate helps us generate structured data from arbitrary bytes. It is typically used in combination with libFuzzer to translate the raw bytes given to use by libFuzzer into something that the system you’re testing can process. For example, a color conversion library might use arbitrary to turn the raw fuzzer-provided bytes into Rgb or Hsl color types. We will use it in a similar way for this project, translating raw bytes given to us by libFuzzer into semantically valid WebAssembly modules.

The arbitrary crate’s main export is the Arbitrary trait:

pub trait Arbitrary: Sized + 'static {
    fn arbitrary(u: &mut Unstructured) -> Result<Self>;

    // Provided methods hidden...
}

It takes an Unstructured, which is a helpful wrapper around a byte slice, and returns an instance of the type for which it is implemented.

For our wasm-smith crate, we define a Module type that represents our pseudo-random WebAssembly modules, and then we implement the Arbitrary trait for it:

use arbitrary::{Arbitrary, Result, Unstructured};

/// A pseudo-random WebAssembly module.
pub struct Module {
    // ...
}

impl Arbitrary for Module {
    fn arbitrary(u: &mut Unstructured) -> Result<Self> {
        todo!()
    }
}

Before we fill in that todo!() lets take a moment to settle on a design for what the implementation will look like.

Translating Grammars into Generators

Writing a generator is remarkably similar to hand-writing a recursive descent parser, so if you’ve done that before, then you should feel right at home. For example, given this grammar production (borrowed and lightly edited from the C++ name mangling grammar):

<class-enum-type> ::= Ts <name>
                    | Tu <name>
                    | Te <name>

A recursive descent parser will, almost mechanically, translate the production into something like this:

impl Parse for ClassEnumType {
    fn parse(p: &mut Parser) -> Result<Self> {
        // Ts <name>
        if p.peek("Ts") {
            p.consume("Ts")?;
            let name = Name::parse(p)?;
            return Ok(ClassEnumType::Ts(name));
        }

        // Tu <name>
        if p.peek("Tu") {
            p.consume("Tu")?;
            let name = Name::parse(p)?;
            return Ok(ClassEnumType::Tu(name));
        }

        // Te <name>
        p.consume("Te")?;
        let name = Name::parse(p)?;
        Ok(ClassEnumType::Te(name))
    }
}

Our generator will do something similar, except instead of peeking at the input string to decide which right-hand side of the production to parse, we will make a pseudo-random choice to generate one of those potential right hand sides.

We could use a random number generator directly to make these choices, but this has two problems:

We give up determinism unless we are careful to control the RNG’s seed and reuse the same RNG everywhere, threading it through all of our functions as a parameter. Determinism is extremely important for reproducing test failures! It’s definitely possible to do these things, but can occasionally be a little annoying.
More importantly, using an RNG precludes a mature fuzzing engine, like libFuzzer, from guiding our test case generation based on code coverage and other insights.

Instead, we use a raw input byte slice given to us by libFuzzer or AFL as a sequence of predetermined choices.⁰ This lets the fuzzer guide our test case generation, and gives us test case reduction “for free” since we can ask the fuzzer to reduce the raw input sequence, rather than write a domain-specific test case reducer. This comes as a relief because writing a reducer that understands WebAssembly is easily as much effort as writing the generator itself.

Here is the same C++ mangling example from above, but translated from a parser into a generator, using Unstructured:

fn arbitrary_class_enum_type(
    u: &mut Unstructured,
    output: &mut String,
) -> Result<()> {
    match u.int_in_range::<u8>(0..=2) {
        // Ts <name>
        0 => {
            output.push_str("Ts");
            arbitrary_name(u, output)?;
            Ok(())
        }
        // Tu <name>
        1 => {
            output.push_str("Tu");
            arbitrary_name(u, output)?;
            Ok(())
        }
        // Te <name>
        2 => {
            output.push_str("Te");
            arbitrary_name(u, output)?;
            Ok(())
        }
        _ => unreachable!(),
    }
}

Once again, this is mostly mechanical.

This pattern will generate syntactically correct test cases that can be parsed successfully but which likely contain a plethora of type errors, calls to undefined functions, etc. We’ve set out to generate semantically correct test cases that pass type checking and will exercise more than just the language implementation’s frontend.

Our final pattern maintains some extra information about the program we’ve generated thus far, so that we can consult that information when generating new forms. This extra information might include which names are in scope, the types of each variable, etc. We consult that information while dynamically building up thunks for every valid option we could generate. Once we have enumerated every option, we ask the Unstructured to choose one of them, and finally we call the chosen thunk to generate the form.

Here is an example of using this pattern for generating integer expressions, where an integer expression is either a constant integer, an arithmetic operation, a use of an integer variable, or a call of a function that returns an integer:

fn arbitrary_int_expr(
    u: &mut Unstructured,
    scope: &mut Scope,
) -> Result<Expr> {
    // We will dynamically build up all of the valid
    // options of what we can generate.
    let mut options: Vec<fn (
        &mut Unstructured,
        &mut Scope,
    ) -> Result<Expr>> = vec![];

    // It is always valid to generate a constant.
    options.push(|u, _| {
        Ok(Expr::Constant(u.arbitrary::<i32>()?))
    });

    // It is always valid to generate an addition.
    options.push(|u, scope| {
        let lhs = arbitrary_int_expr(u, scope)?;
        let rhs = arbitrary_int_expr(u, scope)?;
        Ok(Expr::Add(lhs, rhs))
    });

    // Subtraction, multiplication, division, etc look
    // similar to addition.

    // If there are integer variables in scope, we can
    // generate a use of one of them.
    if scope.has_int_variables() {
        options.push(|u, scope| {
            let var = u.choose(scope.int_variables())?;
            Ok(Expr::Var(var))
        });
    }

    // If there are any functions that return an integer
    // in scope, we can generate a call to one of them.
    if scope.has_int_funcs() {
        options.push(|u, scope| {
            let func = u.choose(scope.int_funcs())?;
            let args = arbitrary_args(u, &func.params)?;
            Ok(Expr::Call(func, args))
        });
    }

    // Choose one of our options and call the function
    // to generate the expression.
    let f = u.choose(&options)?;
    f(u, scope)
}

Finally, it is worth noting that, similar to how parser generators take a grammar and generate a parser, there are tools that will take a grammar and generate a test case generator (and even Rust implementations of those tools).

Generating the Type Section

Now we’re ready to continue generating WebAssembly!

The first section in a WebAssembly module is the type section. It declares the function type signatures used in the module and it has the following grammar:

<typesec> ::= 0x01 <size:u32> <num_funcs:u32> <functype>*

<functype> ::= <num_params:u32>
               <valtype>*
               <num_results:u32>
               <valtype>*

<valtype> ::= 0x7F     # i32
            | 0x7E     # i64
            | 0x7D     # f32
            | 0x7C     # f64

This is not context free: size is the size of the whole section in bytes, num_funcs is the number of following <functype>s, num_params defines how many parameter <valtype>s a function type has, and num_results defines how many result <valtype>s it has. But none of this comes into play until we actually serialize the module into bytes. Until then, a type section is a sequence of function type signatures, and these signatures don’t actually need access to any scope information, so we can just derive the Arbitrary trait directly for them:

#[derive(Arbitrary, Clone, Debug)]
struct FuncType {
    params: Vec<ValType>,
    results: Vec<ValType>,
}

#[derive(Arbitrary, Clone, Debug)]
enum ValType {
    I32,
    I64,
    F32,
    F64,
}

And since a module has a type section, let’s hook this up into our Module definition and its Arbitrary implementation:

pub struct Module {
    types: Vec<FuncType>,
    // ...
}

impl Arbitrary for Module {
    fn arbitrary(u: &mut Unstructured) -> Result<Self> {
        let mut module = Module::default();
        module.types = u.arbitrary()?;
        // ...
        Ok(module)
    }
}

Generating the Import Section

The import section is the first section where we will need to look at what we’ve previously generated when generating new items. An import brings an external function, table, memory, or global into scope. When importing a function, we need to specify the function’s type via an index into the types section. This means we can only generate a function import if our types section is non-empty.

The structure definitions and Arbitrary implementations for table and global types are straightforward, and nothing different from what we saw with the type section, so I’ve elided them here. The one thing to note is that the largest memory that a Wasm module can declare is 4GiB, so the Arbitrary implementation for memory types must take that into account:

#[derive(Clone, Debug)]
struct Limits {
    min: u32,
    max: Option<u32>,
}

#[derive(Clone, Debug)]
struct MemoryType {
    limits: Limits,
}

impl Arbitrary for MemoryType {
    fn arbitrary(
        u: &mut Unstructured<'_>,
    ) -> Result<Self> {
        // Note: memory sizes are in units of pages,
        // which are 16KiB in Wasm.
        let min = u.int_in_range(0..=65536)?;
        let max = if u.arbitrary().unwrap_or(false) {
            Some(if min == 65536 {
                65536
            } else {
                u.int_in_range(min..=65536)?
            })
        } else {
            None
        };
        Ok(MemoryType {
            limits: Limits { min, max },
        })
    }
}

When generating an arbitrary import, we dynamically build up a list of functions, one for each semantically valid choice we can make. Then we will use the next bit of raw input from Unstructured to choose one of those functions and call it to create an import. We keep creating more imports while the raw input tells us to — we read a bool from the Unstructured and stop adding imports once we read false.

impl Module {
    fn arbitrary_imports(
        &mut self,
        u: &mut Unstructured,
    ) -> Result<()> {
        let mut options: Vec<fn(
            &mut Unstructured,
            &mut Module,
        ) -> Result<Import>> = Vec::with_capacity(4);

        // If the module's type section is not empty, we
        // can define function imports with one of those
        // types.
        if !self.types.is_empty() {
            options.push(|u, m| {
                let max = m.types.len() as u32 - 1;
                let idx = u.int_in_range(0..=max)?;
                Ok(Import::Func(idx))
            });
        }

        options.push(|u, _| {
            Ok(Import::Table(u.arbitrary()?))
        });

        options.push(|u, _| {
            Ok(Import::Memory(u.arbitrary()?))
        });

        options.push(|u, _| {
            Ok(Import::Global(u.arbitrary()?))
        });

        loop {
            let keep_going = u.arbitrary::<bool>()
                .unwrap_or(false);
            if !keep_going {
                return Ok(());
            }

            let f = u.choose(&options)?;
            let import = f(u, self)?;
            self.imports.push(import);
        }
    }
}

Hooking this up to Module and its Arbitrary implementation is, once again, straightforward:

#[derive(Default)]
pub struct Module {
    types: Vec<FuncType>,
    imports: Vec<Import>,    // New!
    // ...
}

impl Arbitrary for Module {
    fn arbitrary(u: &mut Unstructured) -> Result<Self> {
        let mut module = Module::default();
        module.types = u.arbitrary()?;
        module.arbitrary_imports(u)?;     // New!
        // ...
        Ok(module)
    }
}

Generating the Code Section

There are a few more sections before the code section, which contains function bodies, but their implementation is similar to what we’ve already seen with the type and import sections, so we’ll skip them.

WebAssembly is a typed stack-based language, and has structured control flow. There is the operand stack, where values are pushed and popped, and there is the control stack, which keeps track of active control frames (i.e. ifs, loops, and blocks) and their labels that can be jumped to. Whether an instruction is valid depends on the types on the operand stack and, if the instruction is control instruction, the labels on the control stack. Therefore, we will explicitly model these stacks while generating instructions. Choosing which instruction to generate will consult the contents of these stacks, and, once a choice is made, generating a new instruction will update them.

Let’s begin with the basic definitions for value types and control frames:

/// The type of a value.
#[derive(Arbitrary, Clone, Copy, Debug, PartialEq, Eq)]
enum ValType {
    I32,
    I64,
    F32,
    F64,
}

/// A control frame.
#[derive(Debug)]
struct Control {
    kind: ControlKind,
    // Value types that must be on the stack when
    // entering this control frame.
    params: Vec<ValType>,
    // Value types that are left on the stack when
    // exiting this control frame.
    results: Vec<ValType>,
    // How far down the operand stack instructions
    // inside this control frame can reach.
    height: usize,
}

/// The kind of a control frame.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum ControlKind {
    Block,
    If,
    Loop,
}

We model the control stack as a vector of Controls, and the operand stack as a vector of Option<ValType>. The Option is introduced because unreachable code produces values of any type, and these are modeled as None.

type OperandStack = Vec<Option<ValType>>;
type ControlStack = Vec<Control>;

In order to generate a function body, we’ll also need to know the function’s parameter and result types, and the local variables available. We’ll collect these things and our stacks in a CodeBuilder type. To avoid thrashing the memory allocator, we factor the operand and control stacks out into a separate structure that is reused across the creation of each function body. This structure also contains the vector of options that we build up every time we generate an instruction.

pub(crate) struct CodeBuilderAllocations {
    controls: ControlStack,
    operands: OperandStack,

    // Dynamic set of instructions that would be
    // valid right now.
    options: Vec<fn(
        &mut Unstructured,
        &Module,
        &mut CodeBuilder,
    ) -> Result<Instruction>>,
}

struct CodeBuilder<'a> {
    func_ty: &'a FuncType,
    locals: &'a Vec<ValType>,
    allocs: &'a mut CodeBuilderAllocations,
}

This is basically the same information that WebAssembly’s validation algorithm requires. Our generator is similar to a WebAssembly validator for the same reasons that a generator that emits syntactically correct test cases is similar to a recursive descent parser.

Because there are so many instructions, we won’t write the validation checks and the thunks that generate the instructions inline like we did for previous sections. Instead we will write pairs of functions: the first checks whether a given instruction would be valid to generate in the current context, and the second generates that instruction and updates the context.

static OPTIONS: &[(
    // Predicate for whether this instruction is valid
    // in the given context, if any. `None` means that
    // the instruction is always valid.
    Option<fn(&Module, &mut CodeBuilder) -> bool>,
    // The function to generate the instruction, given
    // that we've made this choice.
    fn(
        &mut Unstructured,
        &Module,
        &mut CodeBuilder,
    ) -> Result<Instruction>,
)] = &[
    // ...
];

Despite this small change in code organization, we will use these function pairs to accomplish the same task as before: dynamically building up a list of functions, one for each instruction that would be valid in the current context, and then choosing one of them and calling it to generate an instruction.

`i32.add`

Let’s start with an easy example: generating the i32.add instruction. An i32.add is valid when there are two i32s on the operand stack. There are many instructions that are valid to generate when there are two i32s on the operand stack, so rather than naming the predicate function something like i32_add_is_valid, we’ll name it i32_i32_on_stack.

// Predicate for whether two `i32`s are on top of
// the operand stack, and therefore we can generate
// an `i32.add` (and many others).
fn i32_i32_on_stack(
    _: &Module,
    builder: &mut CodeBuilder,
) -> bool {
    builder.types_on_stack(&[
        ValType::I32,
        ValType::I32,
    ])
}

i32.add pops the i32s and pushes their sum. The function that generates an i32.add needs to do the same to our model of the operand stack:

fn i32_add(
    _: &mut Unstructured,
    _: &Module,
    builder: &mut CodeBuilder,
) -> Result<Instruction> {
    builder.pop_operands(&[ValType::I32, ValType::I32]);
    builder.push_operands(&[ValType::I32]);
    Ok(Instruction::I32Add)
}

`f64.const`

Here is another simple example. An f64.const instruction is always valid. It has an f64 immediate and adds that f64 onto the operand stack:

fn f64_const(
    u: &mut Unstructured,
    _: &Module,
    builder: &mut CodeBuilder,
) -> Result<Instruction> {
    let x = u.arbitrary::<f64>()?;
    builder.push_operands(&[ValType::F64]);
    Ok(Instruction::F64Const(x))
}

`if`

Things start getting more interesting when we consider instructions that introduce new control frames. An if instruction requires an i32 on top of the operand stack which determines whether its truth-y or false-y code path is executed. It can also take additional operand stack parameters, making them available to further instructions within its control frame. And while it can take stack parameters, we only require an i32 on the stack because we can always define ifs with empty stack parameters.

fn if_valid(
    _: &Module,
    builder: &mut CodeBuilder,
) -> bool {
    builder.type_on_stack(ValType::I32)
}

fn r#if(
    u: &mut Unstructured,
    module: &Module,
    builder: &mut CodeBuilder,
) -> Result<Instruction> {
    // Pop the `i32` that controls whether we take the
    // truth-y or the false-y code path.
    builder.pop_operands(&[ValType::I32]);

    // Generate an arbitrary block type that is
    // compatible with the current operand stack.
    let block_ty = builder.arbitrary_block_type(
        u,
        module,
    )?;

    // Create the control frame for this `if`.
    let (params, results) = block_ty
        .params_results(module);
    let height = builder.allocs.operands.len()
        - params.len();
    builder.allocs.controls.push(Control {
        kind: ControlKind::If,
        params,
        results,
        height,
    });

    Ok(Instruction::If(block_ty))
}

`else`

An else instruction is only valid when the top control frame was introduced by an if and the types on (the accessible portion of) the operand stack are exactly the top control frame’s result types.

fn else_valid(_: &Module, b: &mut CodeBuilder) -> bool {
    let control = b.top_control();
    control.kind == ControlKind::If
        && b.operands().len() == control.results.len()
        && b.types_on_stack(&control.results)
}

The else instruction resets the operand stack to its state at the moment that the if was introduced. It also changes the control kind to ControlKind::Block so that we don’t generate instruction sequences with two elses for the same if, like if ... else ... else ... end.

fn r#else(
    _: &mut Unstructured,
    _: &Module,
    builder: &mut CodeBuilder,
) -> Result<Instruction> {
    let control = builder.pop_control();
    builder.pop_operands(&control.results);
    builder.push_operands(&control.params);
    builder.allocs.controls.push(Control {
        kind: ControlKind::Block,
        ..control
    });
    Ok(Instruction::Else)
}

`end`

An end instruction finishes a control sequence that was introduced by an if, else, block, or a loop instruction. Similar to an else, it is only valid when we have exactly the current control frame’s result types on the operand stack. The final thing to check is that if the top control frame was introduced by an if, and the if does not leave the stack as it was upon entering (i.e. the parameter and result types are not equal) then we can’t generate an end. This scenario requires that we generate an else first, since not executing any instructions if the if’s truth-y path is not taken would leave the stack in a type mismatched state.

fn end_valid(
    _: &Module,
    builder: &mut CodeBuilder,
) -> bool {
    // Note: first control frame is the function
    // return's control frame, which never has an
    // associated `end`.
    if builder.allocs.controls.len() <= 1 {
        return false;
    }

    let control = builder.top_control();
    builder.operands().len() == control.results.len()
        && builder.types_on_stack(&control.results)
        && !(control.kind == ControlKind::If
             && control.params != control.results)
}

`call`

The final instruction we will examine is call. Whether a call is valid or not depends on the callee function’s type signature, and therefore whether we can generate a call or not depends on whether any of the module’s functions have a type signature compatible with the current operand stack.

fn call_valid(
    module: &Module,
    builder: &mut CodeBuilder,
) -> bool {
    module
        .funcs()
        .any(|(_, ty)| {
            builder.types_on_stack(&ty.params)
        })
}

To generate a call instruction, we choose which function we are calling and update the operand stack accordingly:

fn call(
    u: &mut Unstructured,
    module: &Module,
    builder: &mut CodeBuilder,
) -> Result<Instruction> {
    // Count how many of our module's functions could be
    // called with the current context.
    let n = module
        .funcs()
        .filter(|(_, ty)| {
            builder.types_on_stack(&ty.params)
        })
        .count();
    debug_assert!(n > 0);

    // Choose one of them.
    let i = u.int_in_range(0..=n - 1)?;
    let (func_idx, ty) = module
        .funcs()
        .filter(|(_, ty)| {
            builder.types_on_stack(&ty.params)
        })
        .nth(i)
        .unwrap();

    // Pop its parameters and push its results.
    builder.pop_operands(&ty.params);
    builder.push_operands(&ty.results);

    Ok(Instruction::Call(func_idx as u32))
}

Generating Instruction Sequences

Currently, the core WebAssembly specification defines 189 instructions, and I’ve implemented support for all of them in wasm-smith. That’s too many to reproduce here, but you can check them out if you’re curious. We’ll now turn our attention to using the pairs of functions we’ve defined for generating individual instructions to generate the sequences of instructions that make up a whole function body.

We keep generating new instructions until either when there aren’t any control frames left (meaning that we’ve returned from the function) or when the Unstructured tells us to stop. To generate one instruction, we filter OPTIONS down to just those options that are valid within the current context, ask the Unstructured to choose one of them, and call the chosen function to generate the instruction. This new instruction is added to our function body and we repeat the process.

impl CodeBuilder {
    pub(crate) fn arbitrary(
        mut self,
        u: &mut Unstructured,
        module: &Module,
    ) -> Result<Vec<Instruction>> {
        let mut instructions = vec![];

        while !self.allocs.controls.is_empty() {
            let keep_going =
                u.arbitrary().unwrap_or(false);
            if !keep_going {
                self.end_active_control_frames(
                    &mut instructions,
                );
                break;
            }

            // Filter `OPTIONS` down to just those that
            // are valid within the current context.
            self.allocs.options.clear();
            for (is_valid, option) in OPTIONS {
                if is_valid.map_or(true, |f| {
                    f(module, &mut self)
                }) {
                    self.allocs.options.push(*option);
                }
            }

            // Choose one of them, and call the chosen
            // function to generate the instruction.
            let f = u.choose(&self.allocs.options)?;
            let inst = f(u, module, &mut self)?;
            instructions.push(inst);
        }

        Ok(instructions)
    }
}

If the Unstructured tells us to stop generating instructions before we’ve exited all control frames, then the instruction sequence most likely is not at a valid stopping point. Either there are unfinished loops, ifs, and blocks, or we have the wrong types on the operand stack for a return. The end_active_control_frames method resolves these conflicts. When it can, it leaves the operands on the stack as a control frame’s results or a function’s return values. When the operand stack doesn’t align with the control frame’s results, it inserts an unreachable instruction. This instruction makes the sequence validate, but will trigger a trap at when executed.

impl CodeBuilder {
    fn end_active_control_frames(
        &mut self,
        instructions: &mut Vec<Instruction>,
    ) {
        while !self.allocs.controls.is_empty() {
            let num_operands = self.operands().len();
            let control = self.pop_control();

            // If we don't have the right operands on
            // the stack for this control frame, add an
            // `unreachable` instruction.
            if control.results.len() != num_operands
                || !self.types_on_stack(&control.results)
            {
                self.allocs.operands.push(None);
                instructions.push(
                    Instruction::Unreachable,
                );
            }

            // If this is an `if` that is not stack
            // neutral, then it must have an `else`.
            if control.kind == ControlKind::If
                && control.params != control.results
            {
                instructions.push(Instruction::Else);
                instructions.push(
                    Instruction::Unreachable,
                );
            }

            // The last control frame for the function
            // return does not need an `end` instruction.
            if !self.allocs.controls.is_empty() {
                instructions.push(Instruction::End);
            }

            self.allocs.operands.truncate(
                control.height,
            );
            self.allocs.operands.extend(
                control.results.into_iter().map(Some),
            );
        }
    }
}

The CodeBuilderAllocations are created in the arbitrary_code method, which generates the module’s code section. The code section contains the bodies of the functions locally defined in the module. For each function, it calls arbitrary_func_body, which creates locals for the function and a CodeBuilder that reuses the allocations and generates the instruction sequence for the function.

impl Module {
    /// Generate the module's code section.
    fn arbitrary_code(
        &mut self,
        u: &mut Unstructured,
    ) -> Result<()> {
        // Reserve space for the function bodies we
        // are about to define.
        self.code.reserve(self.funcs.len());

        // Create the allocations that are reused
        // when generating each function body.
        let mut allocs =
            CodeBuilderAllocations::default();

        // For each function defined locally,
        // generate an arbitrary function body.
        for func_ty_idx in &self.funcs {
            let func_ty =
                &self.types[*func_ty_idx as usize];
            let body = self.arbitrary_func_body(
                u,
                func_ty,
                &mut allocs,
            )?;
            self.code.push(body);
        }

        Ok(())
    }

    /// Generate a function body.
    fn arbitrary_func_body(
        &self,
        u: &mut Unstructured,
        func_ty: &FuncType,
        allocs: &mut CodeBuilderAllocations,
    ) -> Result<Code> {
        // Generate arbitrary locals for this function.
        let locals = self.arbitrary_locals(u)?;

        // Create a `CodeBuilder` that reuses the given
        // allocations.
        let builder = allocs.builder(func_ty, &locals);

        // Have the `CodeBuilder` generate an arbitrary
        // instruction sequence.
        let instructions = builder.arbitrary(u, self)?;

        Ok(Code {
            locals,
            instructions,
        })
    }
}

Finally, here is the full Arbitrary implementation for Module, including calls to each of the methods to generate sections whose description we elided in this blog post:

impl Arbitrary for Module {
    fn arbitrary(u: &mut Unstructured) -> Result<Self> {
        let mut module = Module::default();
        module.types = u.arbitrary()?;
        module.arbitrary_imports(u)?;
        module.arbitrary_funcs(u)?;
        if module.table_imports() == 0 {
            module.table = u.arbitrary()?;
        }
        if module.memory_imports() == 0 {
            module.memory = u.arbitrary()?;
        }
        module.arbitrary_globals(u)?;
        module.arbitrary_exports(u)?;
        module.arbitrary_start(u)?;
        module.arbitrary_elems(u)?;
        module.arbitrary_codes(u)?;
        module.arbitrary_data(u)?;
        Ok(module)
    }
}

That’s everything we need to generate arbitrary, valid Wasm modules!

Using the Test Case Generator

Now we are ready to use our shiny, new Wasm test case generator with a fuzzer! We’ll use the libfuzzer-sys crate, and cargo fuzz. Our new fuzz target accepts an arbitrary Module as input, serializes it into bytes, and passes these bytes into the wasmparser crate’s validator. Finally, it asserts that the module is valid. If this assertion fails, then either the test case generator or the wasmparser crate has a bug.

// fuzz/fuzz_targets/validate.rs

#![no_main]

use libfuzzer_sys::fuzz_target;
use wasm_smith::Module;

// Define a fuzz target that accepts arbitrary
// `Module`s as input.
fuzz_target!(|m: Module| {
    // Convert the module into Wasm bytes.
    let bytes = m.to_bytes();

    // Validate the module and assert that it passes
    // validation.
    let mut validator = wasmparser::Validator::new();
    validator.wasm_features(wasmparser::WasmFeatures {
        multi_value: true,
        ..wasmparser::WasmFeatures::default()
    });
    if let Err(e) = validator.validate_all(&bytes) {
        std::fs::write("test.wasm", bytes).unwrap();
        panic!("Invalid module: {}", e);
    }
});

We can run this fuzz target with cargo fuzz:

$ cargo fuzz run validate

Initially, I used this fuzz target to test my generator, making sure that it was actually generating valid Wasm modules. Then it stopped finding bugs in my generator, and soon after that it started finding bugs in wasmparser’s validation implementation. It has already found five unique validation bugs! And it isn’t like wasmparser has particularly low hanging fruit — we already fuzz wasmparser in a variety of different ways 24/7 on OSS-Fuzz.

Conclusion

Writing a test case generator for a programming language isn’t magic. It isn’t very difficult once you know the pattern to follow. And once you have the test case generator written, it can poke deep into your compiler, past the parser, helping you find a bunch of hidden bugs!

wasm-smith is already quite promising and I intend to keep developing and maintaining it. Next I want to define a bunch of fuzz targets that use wasm-smith for Wasmtime and its ecosystem of crates and tools. I also want to add support for swarm testing, toggling various Wasm proposals (such as multi-value and reference types) on and off, and add a mode that guarantees termination of its generated programs.

*Update: support for swarm testing and ensuring termination now exists. Supporting new Wasm proposals is ongoing, but you can already toggle some of them on or off.

Thanks for reading!

⁰ It should be noted that this technique, while it enables using libFuzzer and AFL with our generator, does not require their use. We can still use a purely random approach by filling a buffer with a random number generator and then using that buffer as our input sequence of predetermined choices. This would, for example, let us fuzz with our test case generator on platforms that aren’t supported by AFL or libFuzzer. ↩

Nick Fitzgerald

A Structure-Aware Fuzzing Experiment

Background

Generation-Based and Mutation-Based Fuzzing

Structure-Aware Fuzzing

The arbitrary Crate

The mutatis Crate

WebAssembly

Generator and Mutator Implementation

arb

bottom_up

top_down

mutate

Benchmarking

Methodology

Results

24 Hours of Fuzzing

5 Minutes of Fuzzing

Conclusion

A Function Inliner for Wasmtime and Cranelift

Function Inlining

Implementation

Some Initial Results

Conclusion

New Stack Maps for Wasmtime and Cranelift

Background: Garbage Collection, Safepoints, and Stack Maps

Cranelift’s Old Approach: Stack Maps During Register Allocation

The New Approach: User Stack Maps

Conclusion

Garbage Collection Without Unsafe Code

Using safe-gc

Peeking Under the Hood

Preventing Classic Footguns

Copying Collector False Start

Why safe-gc?

Conclusion

My WasmCon 2023 Talk

How Fuzzy are Your Fuzzers?

Security and Correctness in Wasmtime

A Safe Implementation Language

Securing Our Supply Chain

Enabling Secure Application Designs

Ubiquitous Fuzzing

Formal Verification Efforts

Spectre Mitigations

A Plan When Things Go Wrong

Conclusion

Hit the Ground Running: Wasm Snapshots for Fast Start Up

WebAssembly Reference Types in Wasmtime

What are Reference Types?

Example

Python

Rust

On the Web

What Comes After the Reference Types Proposal?

Implementation Details

Deferred Reference Counting

Stack Maps

Inline Fast Paths for GC Barriers

Conclusion

Writing a Test Case Generator for a Programming Language

Table of Contents

What is a Test Case Generator?

Getting Set Up

Translating Grammars into Generators

Generating the Type Section

Generating the Import Section

Generating the Code Section

i32.add

f64.const

if

else

end

call

Generating Instruction Sequences

Using the Test Case Generator

Conclusion

The `arbitrary` Crate

The `mutatis` Crate

`arb`

`bottom_up`

`top_down`

`mutate`

Using `safe-gc`

Why `safe-gc`?

`i32.add`

`f64.const`

`if`

`else`

`end`

`call`