Why Benchmark

Performance claims without numbers are marketing. We implemented 6 benchmark programs in 10 languages and LOGOS, measured wall-clock time with hyperfine (3 warmup runs, 10 measured runs), and published the results on every release.

The Programs

Six programs exercise different computational patterns:

  • fib — naive recursive Fibonacci (function call overhead)
  • sieve — Sieve of Eratosthenes (array mutation, tight loops)
  • collect — hash map insert/lookup (allocation pressure)
  • strings — string concatenation (allocator throughput)
  • bubble_sort — O(n²) sort (nested loops, array mutation, swaps)
  • ackermann — Ackermann function (extreme recursion depth, stack frame overhead)

Each program is implemented idiomatically in C, C++, Rust, Go, Zig, Nim, Python, Ruby, JavaScript, and Java. No artificial handicaps or advantages.

CI Integration

A GitHub Actions workflow runs the full benchmark suite on every tagged release. The runner compiles all implementations, verifies correctness against expected output files, benchmarks runtime with hyperfine, measures compilation time separately, and assembles the results into benchmarks/results/latest.json. Results are committed back to the repository, and the frontend deploy triggers after benchmarks land.

Versioned results are archived in benchmarks/results/history/, so performance trends are recoverable across releases.

The Results Page

The /benchmarks page embeds latest.json at compile time. Each benchmark tab shows grouped bar charts split by language tier (systems, managed, transpiled, interpreted), with LOGOS highlighted. Collapsible sections show full statistics (mean, median, stddev, min, max, coefficient of variation), source code for all implementations side-by-side, and compilation time comparisons.

A cross-benchmark summary computes geometric mean speedup versus C across all 6 programs, providing a single aggregate performance number.

What It Measures

The benchmark suite tests the full compilation path: LOGOS source → parser → codegen → Rust output → rustc → binary. Interpreter mode runs separately at smaller input sizes. This means the numbers reflect codegen quality — the optimizer's job — and rustc optimization of the generated code. When LOGOS gets faster, it's because the generated Rust got better.