Compatibility

A pattern that works in C Oniguruma should work the same way in Ferroni. This page says what that promise covers, where Ferroni differs on purpose, how the promise is checked, and how the port is built to keep it.

Why a line-by-line port

Ferroni began as the regex engine for Ferriki, a Shiki-compatible syntax highlighter in Rust. TextMate grammars are written for Oniguruma: they rely on look-behind, \G anchors, backreferences and quantifier spellings that most other engines reject or read differently.

Bindings to the C library would have kept the C code, the C toolchain and its memory-safety record. A new engine would have broken grammars in subtle ways. So Ferroni ports Oniguruma itself, file by file and function by function, and verifies the result against Oniguruma's own test suite (ADR-001).

What Ferroni covers

Ferroni ports ASCII and UTF-8, two of Oniguruma's 29 encodings (ADR-003). All twelve syntax modes are included, from Oniguruma and Ruby to Perl, Python, Java and POSIX. For the projects built on Oniguruma's syntax, that means:

These parts of the C library are not ported:

• 27 of 29 encodings: UTF-16 and UTF-32, Shift_JIS, EUC-JP, EUC-KR, EUC-TW, EUC-CN, Big5, GB18030, KOI8-R, CP1251, and ISO-8859-1 to -16.
• The POSIX and GNU APIs: regcomp, regexec and regfree (ADR-012).
• 20 of the 152 functions that oniguruma.h declares, all of them C memory, string and setup plumbing that Rust does not need. The other 132 are public under their C names.
  • onig_new_deluxe, onig_new_without_alloc and onig_reg_init allocate and initialize a regex_t in C; use onig_new.
  • onig_free, onig_free_body, onig_region_free, onig_regset_free, onig_free_match_param and onig_free_match_param_content free what Rust's Drop frees.
  • onig_initialize_encoding, onig_copy_encoding, onigenc_init, onigenc_get_default_encoding, onigenc_set_default_encoding and onigenc_set_default_caseconv_table set up encoding structs and C's process-wide encoding defaults; Ferroni's two encodings are the statics ONIG_ENCODING_UTF8 and ONIG_ENCODING_ASCII, passed to onig_new.
  • onigenc_strdup, onigenc_str_bytelen_null and onigenc_get_right_adjust_char_head_with_prev are helpers over C's NUL-terminated buffers; Rust slices carry their length.
  • onig_null_warn is C's no-op warning callback; onig_set_warn_func takes any fn(&str).
  • onig_setup_builtin_monitors_by_ascii_encoded_name registers monitor callouts that print to a C FILE*.

Where Ferroni differs on purpose

• Unicode 18.0. Ferroni generates its tables from the Unicode Character Database 18.0; C Oniguruma's last release ships 16.0. Some characters change properties as a result: 672 leave Extended_Pictographic, U+0295 moves from Ll to Lo, and some \X boundaries move. U+1DF95 folds to "ss", so (?i)ss matches it (ADR-015).
• A capped backtracking stack. When no stack limit is set, C lets the backtracking stack grow until memory runs out. Ferroni stops at 10 million entries and reports the limit (untrusted input).
• Additions C never had. The Scanner, per-search limits, the compile-time backtracking check and the idiomatic Regex API sit beside the port and do not change how a pattern matches.

How compatibility is checked

Test parity

Every upstream test that targets UTF-8 is ported case by case. The EUC-JP tests are out of scope with that encoding.

The Ferroni column can be higher than the upstream one: some upstream cases are split into several Rust tests. On top of the ported cases, Ferroni adds Rust-native tests for the idiomatic API, edge cases, error paths and coverage gaps. The tree holds at least 2,500 #[test] functions, which scripts/count-tests.sh derives from the tree. The test strategy is recorded in ADR-011.

Coverage, fuzzing and benchmark checks

• Coverage gate. C Oniguruma has no coverage reporting. Ferroni measures line coverage with cargo-llvm-cov on every pull request, and scripts/coverage.sh fails the build below its threshold. Generated Unicode tables, the FFI bindings and the test files are excluded from the measurement.
• Fuzzing. Three cargo-fuzz targets cover pattern compilation, matching and the Scanner API. Each runs briefly on every pull request and for 15 minutes every week.
• Results before timings. Every production regex case must reproduce its source-runtime oracle before timing. C Oniguruma also runs the admitted tasks; all engines share explicit operation contracts (regex workloads).

The bugs found while porting, and what they teach about C-to-Rust ports, are collected in ADR-014.

How the port maps to C

Each C source file maps to one Rust module, with the same function names and control flow (ADR-001, ADR-004). A fix in upstream C can be found and compared in the corresponding Rust function.

The compilation pipeline is the same as in C:

onig_new() -> onig_compile()
  -> onig_parse_tree()     (pattern -> AST)
  -> reduce_string_list()  (merge adjacent strings)
  -> tune_tree()           (6 optimization sub-passes)
  -> compile_tree()        (AST -> VM bytecode)
  -> set_optimize_info()   (extract search strategy)

Optimizations that C does not have are recorded separately, so the parity of the ported code stays reviewable (ADR-008).