Benchmark Results

Raw benchmark tables for the battle_bench reference suite: Ferroni (Rust) vs Oniguruma at -O3. This is the primary comparison for an Oniguruma-compatible engine. Measurements for Rust's regex crate are retained in a separate shared-syntax appendix.

Measured with Criterion on Apple M1 Pro.

Re-run with ./scripts/prepare-oniguruma-sources.sh && cargo bench --features ffi --bench battle_bench to get current numbers on your hardware, then render the tables below with ./scripts/gen_battle_tables.py.

The README intentionally rounds values for readability. This file keeps the raw numbers, and the home page derives its speed factors from the rows below (pnpm check:numbers in docs/ enforces that).

Every number on this page comes from the single run recorded in Measurement context. Releases after that commit include further performance work, so treat the tables as dated evidence: refresh them by re-running the suite on the reference machine and updating this page and the README together.

A full comparison for PR #173 records a newer run on an M1 Ultra, including seven everyday regex tasks, opt-in match-cache, and memory measurements. It also corrects the search harness to request equivalent capture-output work from Ferroni and C. Treat it as a separate snapshot, not a regression comparison with the numbers below.

Measurement context

Exact external input revisions for the publishable battle suite are pinned in benches/battle_inputs.toml.

Reference suite (battle_bench)

The main tables compare Ferroni and Oniguruma. Bold marks the lower time between those two engines; the factor is C time divided by Ferroni time. regex does not support lookarounds or backreferences, so its shared-syntax results cannot establish Oniguruma compatibility or an overall engine ranking. They remain available in the appendix. The RegSet and scanner paths have no equivalent regex measurement in this harness; that is distinct from unsupported syntax. See the benchmark scope map for the features exercised by the current suite.

Text search and log scanning

Pattern matching

Compilation

Scanner with full Shiki TextMate grammars

Full, unmodified grammars from shikijs/textmate-grammars-themes.

These rows are the warm path: every iteration hands the scanner the same OnigString, so from the second iteration on Ferroni answers fallback patterns that did not match from its per-string memo. The C scanner gets no such help here: the vscode-oniguruma wrapper only consults its per-pattern cache for strings of 1000 bytes or more, and every input in this suite is shorter, so it runs a full onig_regset_search on every call. The document rows below measure the cold path a real tokenizer sees.

Scanner on whole documents, line by line

The TypeScript, CSS and Rust documents from benches/scanner_documents.rs and benches/scanner_css_workload.rs, tokenized line by line with the trailing newline, each line handed to the scanner once, the way vscode-textmate and Shiki drive it. Ferroni sees a distinct OnigString per line and the C scanner a fresh str_cache_id per line, so neither engine answers a line from the previous one. Times are for the whole document; before measuring, the suite asserts that both engines produce the same number of tokens.

Shared-syntax context only: regex

These measurements answer a narrower question: how quickly each engine runs the particular patterns supported by all three. They do not compare the full Oniguruma feature set or establish whether regex can replace Ferroni. regex excludes lookarounds and backreferences as part of its design. Its timings remain useful when an application's patterns fit that subset. Capture groups and Unicode are supported; those features alone do not require an Oniguruma-compatible engine.

The original measurements are retained below without winner counts or an overall ranking. Only measured shared-syntax cases appear here. Lookarounds, backreferences, and the unmeasured RegSet/scanner path are excluded explicitly, not assigned a slow or zero result. Times and measurement conditions are unchanged from their corresponding main tables.

Shared syntax: text search and log scanning

Shared syntax: pattern matching

Shared syntax: compilation

Reproducing

# Reference suite for publishable Ferroni-vs-C numbers
./scripts/prepare-oniguruma-sources.sh
cargo bench --features ffi --bench battle_bench
# or build against an existing checkout at the pinned commit
FERRONI_ONIGURUMA_DIR=/path/to/oniguruma cargo bench --features ffi --bench battle_bench

# Tables in the layout of this page
./scripts/gen_battle_tables.py

# Internal Rust-only regression suite
cargo bench --bench regression_bench

# HTML report
open target/criterion/report/index.html