Skip to content

Commit 8622c70

Browse files
iduartgomezclaude
andcommitted
docs: update benchmark references after experimental benchmarks removal
- Update transport_perf_analysis.md to note experimental benchmarks were removed - Rename "blackbox" references to "transport" in transport_benchmark_methodology.md - Fix benchmark command examples to use current group names 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 9f7b93a commit 8622c70

File tree

2 files changed

+26
-33
lines changed

2 files changed

+26
-33
lines changed

docs/architecture/transport_benchmark_methodology.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -236,20 +236,20 @@ fn bench_serialize_pure(c: &mut Criterion) {
236236

237237
Uses channels instead of sockets - measures protocol logic without syscalls.
238238

239-
**Running the Blackbox Benchmarks:**
239+
**Running the Transport Benchmarks:**
240240

241241
```bash
242-
# Run just the blackbox transport benchmarks (uses actual transport code with mock I/O)
243-
cargo bench --bench transport_perf --features bench -- blackbox
242+
# Run just the transport benchmarks (uses actual transport code with mock I/O)
243+
cargo bench --bench transport_perf --features bench -- transport
244244
```
245245

246-
The `blackbox` benchmark group tests the **actual transport code** with mock sockets:
246+
The `transport` benchmark group tests the **actual transport code** with mock sockets:
247247

248248
| Benchmark | What it measures |
249249
|-----------|-----------------|
250-
| `level1_blackbox/connection/establish` | Full handshake: key exchange, encryption setup |
251-
| `level1_blackbox/throughput/bytes/*` | End-to-end message throughput at different sizes |
252-
| `level1_blackbox/fast_channel/*` | Our crossbeam-based channel vs tokio::sync::mpsc |
250+
| `transport/connection/establish` | Full handshake: key exchange, encryption setup |
251+
| `transport/throughput/bytes/*` | End-to-end message throughput at different sizes |
252+
| `transport/fast_channel/*` | Our crossbeam-based channel vs tokio::sync::mpsc |
253253

254254
**Using the Mock Transport Infrastructure:**
255255

@@ -550,12 +550,12 @@ echo "=== Results in target/criterion/ ==="
550550
| Encryption throughput | `level0` | Any | None | Perfect |
551551
| Serialization | `level0` | Any | None | Perfect |
552552
| Protocol state machines | `level0` | Any | None | Perfect |
553-
| **Blackbox transport** | `blackbox` | Any | **Channels only** | **High** |
553+
| **Transport (mock I/O)** | `transport` | Any | **Channels only** | **High** |
554554
| Mock I/O throughput | `level1` | Linux bare metal | Channels only | High |
555555
| Syscall overhead | `level2` | Linux bare metal | recv/send | Medium |
556556
| Full stack | `level3` | Controlled hardware | Everything | Low |
557557

558-
**Key insight**: The `blackbox` benchmark group tests the **actual Freenet transport code** (PeerConnection, fast_channel, encryption) with mock I/O. This catches real regressions in the transport pipeline without kernel noise.
558+
**Key insight**: The `transport` benchmark group tests the **actual Freenet transport code** (PeerConnection, fast_channel, encryption) with mock I/O. This catches real regressions in the transport pipeline without kernel noise.
559559

560560
To run all benchmark groups:
561561
```bash
@@ -565,7 +565,7 @@ cargo bench --bench transport_perf --features bench
565565
To run specific groups:
566566
```bash
567567
cargo bench --bench transport_perf --features bench -- level0 # Pure logic
568-
cargo bench --bench transport_perf --features bench -- blackbox # Actual transport with mock I/O
568+
cargo bench --bench transport_perf --features bench -- transport # Actual transport with mock I/O
569569
cargo bench --bench transport_perf --features bench -- level2 # Loopback
570570
```
571571

docs/architecture/transport_perf_analysis.md

Lines changed: 16 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -636,20 +636,16 @@ criterion_main!(benches);
636636

637637
### 11.4 Running the Benchmarks
638638

639-
```bash
640-
# Run packet size experiments
641-
./scripts/run_benchmarks.sh experimental_packet
642-
643-
# Run Tokio overhead experiments
644-
./scripts/run_benchmarks.sh experimental_tokio
639+
> **Note**: The experimental benchmarks referenced in sections 11.1-11.3 were one-time experiments
640+
> to inform design decisions. They have been removed from the codebase. The results are preserved
641+
> here for reference. To run current benchmarks:
645642
646-
# Run full pipeline experiments
647-
./scripts/run_benchmarks.sh experimental_combined
643+
```bash
644+
# Run transport layer benchmarks (actual code with mock I/O - used in CI)
645+
cargo bench --bench transport_perf --features bench -- transport
648646

649-
# Or run directly with cargo (requires bench feature):
650-
cargo bench --bench transport_perf --features bench -- experimental_packet
651-
cargo bench --bench transport_perf --features bench -- experimental_tokio
652-
cargo bench --bench transport_perf --features bench -- experimental_combined
647+
# Run all benchmark levels
648+
cargo bench --bench transport_perf --features bench
653649
```
654650

655651
---
@@ -761,22 +757,19 @@ This means:
761757

762758
### 12.7 Running These Benchmarks
763759

764-
```bash
765-
# Run full size×batch matrix
766-
./scripts/run_benchmarks.sh experimental_size_batch
760+
> **Note**: The size×batch experimental benchmarks referenced in section 12 were one-time
761+
> experiments to inform design decisions. They have been removed from the codebase.
762+
> The results are preserved here for reference.
767763
768-
# Individual benchmark groups (requires bench feature)
769-
cargo bench --bench transport_perf --features bench -- experimental_size_batch_matrix
770-
cargo bench --bench transport_perf --features bench -- experimental_batch_improvement
771-
cargo bench --bench transport_perf --features bench -- experimental_throughput_ceiling
764+
```bash
765+
# Run transport layer benchmarks (actual transport code with mock I/O)
766+
cargo bench --bench transport_perf --features bench -- transport
772767

773-
# NEW: Blackbox transport benchmarks (actual transport code with mock I/O)
774-
cargo bench --bench transport_perf --features bench -- blackbox
768+
# Run all benchmark levels (level0, level1, level2, level3, transport)
769+
cargo bench --bench transport_perf --features bench
775770
```
776771

777772
---
778773

779774
*Analysis performed: December 2025*
780-
*Experimental benchmarks added: December 2025*
781-
*Size×batch interaction analysis: December 2025*
782775
*Codebase version: commit c50d888*

0 commit comments

Comments
 (0)