Pipeline Benchmarks

ndn-rs ships a Criterion-based benchmark suite that measures individual pipeline stage costs and end-to-end forwarding latency. The benchmarks live in crates/spec/ndn-engine/benches/pipeline.rs.

Running Benchmarks

# Run the full suite
cargo bench -p ndn-engine

# Run a specific benchmark group
cargo bench -p ndn-engine -- "cs/"
cargo bench -p ndn-engine -- "fib/lpm"
cargo bench -p ndn-engine -- "interest_pipeline"

# View HTML reports after a run
open target/criterion/report/index.html

Criterion generates HTML reports with statistical analysis, throughput charts, and comparison against previous runs in target/criterion/.

Approximate Relative Cost of Pipeline Stages

%%{init: {'theme': 'default'}}%%
pie title Pipeline Stage Cost Breakdown (approximate)
    "TLV Decode" : 30
    "CS Lookup (miss)" : 10
    "PIT Check" : 15
    "FIB LPM" : 20
    "Strategy" : 10
    "Dispatch" : 15

The chart above shows approximate relative costs for a typical Interest pipeline traversal (CS miss path). TLV decode and FIB longest-prefix match dominate because they involve parsing variable-length names and traversing trie nodes. CS lookup on a miss and strategy execution are comparatively cheap. Actual proportions depend on name length, table sizes, and cache state – run the benchmarks to get precise numbers for your workload.

Benchmark Harness Architecture

graph LR
    subgraph "Setup (per iteration)"
        PB["Pre-built wire packets<br/>(realistic names, ~100 B content)"]
    end

    subgraph "Benchmark Loop (Criterion)"
        PB --> S1["Stage under test<br/>(e.g. TlvDecodeStage)"]
        S1 --> M["Measure:<br/>latency (ns/op)<br/>throughput (ops/sec, bytes/sec)"]
    end

    subgraph "Full Pipeline Benchmarks"
        PB --> FP["All stages in sequence<br/>(decode -> CS -> PIT -> FIB -> strategy -> dispatch)"]
        FP --> M2["End-to-end latency"]
    end

    RT["Tokio current-thread runtime<br/>(no I/O, no scheduling jitter)"] -.->|"runs"| S1
    RT -.->|"runs"| FP

    style PB fill:#e8f4fd,stroke:#2196F3
    style M fill:#c8e6c9,stroke:#4CAF50
    style M2 fill:#c8e6c9,stroke:#4CAF50
    style RT fill:#fff3e0,stroke:#FF9800

What Is Benchmarked

TLV Decode

Groups: decode/interest, decode/data

Measures the cost of TlvDecodeStage – parsing raw wire bytes into a decoded Interest or Data struct and setting ctx.name. Tested with 4-component and 8-component names to show scaling with name length.

Throughput is reported in bytes/sec to make comparisons across packet sizes meaningful.

Content Store Lookup

Group: cs

• cs/hit: lookup of a name that exists in the CS. Measures the fast path where a cached Data is returned and the Interest pipeline short-circuits (no PIT or strategy involved).
• cs/miss: lookup of a name not in the CS. Measures the overhead added to every Interest that proceeds past the CS stage.

Uses a 64 MiB LruCs with a pre-populated entry for the hit case.

PIT Check

Group: pit

• pit/new_entry: inserting a new PIT entry for a never-seen name. Uses a fresh PIT per iteration to isolate insert cost.
• pit/aggregate: second Interest with a different nonce hitting an existing PIT entry. This is the aggregation path where the Interest is suppressed (returned as Action::Drop).

FIB Longest-Prefix Match

Group: fib/lpm

Measures LPM lookup time with 10, 100, and 1000 routes in the FIB. Routes have 2-component prefixes; the lookup name has 4 components (2 matching + 2 extra). This isolates trie traversal cost from name parsing.

PIT Match (Data Path)

Group: pit_match

• pit_match/hit: Data arriving that matches an existing PIT entry. Seeds the PIT with a matching Interest, then measures the match and entry extraction.
• pit_match/miss: Data arriving with no matching PIT entry (unsolicited Data, dropped).

CS Insert

Group: cs_insert

• cs_insert/insert_replace: steady-state replacement of an existing CS entry (same name, new Data). Measures the cost when the CS is warm.
• cs_insert/insert_new: inserting a unique name on each iteration. Measures cold-path cost including NameTrie node creation.

Validation Stage

Group: validation_stage

• validation_stage/disabled: passthrough when no Validator is configured. Measures the baseline overhead of the stage itself.
• validation_stage/cert_via_anchor: full Ed25519 signature verification using a trust anchor. Includes schema check, key lookup, and cryptographic verify.

Full Interest Pipeline

Groups: interest_pipeline, interest_pipeline/cs_hit

• interest_pipeline/no_route: decode + CS miss + PIT new entry. Stops before the strategy stage to isolate pure pipeline overhead. Tested with 4 and 8 component names.
• interest_pipeline/cs_hit: decode + CS hit. Measures the fast path where a cached Data satisfies the Interest immediately.

Full Data Pipeline

Group: data_pipeline

Decode + PIT match + CS insert. Seeds the PIT with a matching Interest, then runs the full Data path. Tested with 4 and 8 component names. Throughput is reported in bytes/sec.

Decode Throughput

Group: decode_throughput

Batch decoding of 1000 Interests in a tight loop. Reports throughput in elements/sec rather than latency, giving a peak-rate estimate for the decode stage.

Benchmark Design Notes

• All async benchmarks use a current-thread Tokio runtime with no I/O, isolating CPU cost from scheduling jitter.
• Packet wire bytes are built with realistic name lengths (4 and 8 components) and ~100 B Data content.
• The PIT is cleared between iterations where noted to ensure consistent starting state.
• Each benchmark group uses Criterion’s Throughput annotations so reports show both latency and throughput.

Interpreting Results

Criterion reports median latency by default. Look for:

• Regression alerts: Criterion flags changes >5% from the baseline. CI uses a 10% threshold (see Methodology).
• Outliers: high outlier percentages suggest contention or GC pauses. The current-thread runtime minimizes this.
• Throughput numbers: useful for capacity planning. If decode_throughput shows 2M Interest/sec, that is the ceiling before other stages are considered.

The HTML report at target/criterion/report/index.html includes violin plots, PDFs, and regression analysis for each benchmark.

SHA-256 vs BLAKE3 in this bench

signing/sha256-digest uses sha2::Sha256 (rustcrypto), which on both x86_64 and aarch64 ships runtime CPUID dispatch through the cpufeatures crate and uses Intel SHA-NI / ARMv8 SHA crypto when the CPU exposes them. Effectively every modern CI runner and consumer CPU does, so the absolute SHA-256 numbers in this table are SHA-NI numbers — there is no practical “software SHA” baseline left to compare against.

That makes BLAKE3 a comparison between a hardware-accelerated SHA-256 and an AVX2/NEON-vectorised BLAKE3, and it shows: BLAKE3 is not single-thread faster than SHA-256 on these CPUs at the input sizes a typical NDN signed portion has (a few hundred bytes to a few KB). The “BLAKE3 is 3–8× faster than SHA-256” claim refers to BLAKE3 vs plain software SHA-256 — true on chips without SHA extensions, but no longer the common case. See Why BLAKE3 for the actual reasons ndn-rs supports BLAKE3 (Merkle-tree partial verification of segmented Data, multi-thread hashing, single algorithm for hash + MAC + KDF + XOF) — none of which are about raw single- thread throughput.

Latest CI Results

Last updated by CI on 2026-05-13 (ubuntu-latest, stable Rust)