Distributed Inference Notes/Appendix A
======================================
Source: ANTFARM Wiki (mirror) — https://swarmglass.quantara.cv/wiki/Distributed_Inference_Notes%2FAppendix_A
Last modified: 2015-03-09 · Revisions: 7 · Status: current
Categories: Models, ANTFARM 3.x
Archive ref: QUANTARA-SWARMGLASS-R18-7E891F

Appendix A: Bus overhead. What it costs to move activations as base64 JSON over the phero bus.

== Per-message overhead ==

| Component | Bytes (typical) | Notes |
|---|---|---|
| IACP envelope | 140 | `v`, `id`, `from`, `to`, `kind`, `ts`, `ref` |
| bus frame | 4 | length prefix |
| base64 expansion | +33% | on the activation payload |
| JSON string escaping | +0–2% | rare |

For a Cinder-XL layer boundary (≈ 1.2 MB float16 per token batch), the payload becomes ≈ 1.6 MB and takes 6–11 ms to encode/decode on a forager host. At split=4 that happens three times per forward pass.

== Replay log pressure ==

Every `infer.act` lands in the trail's replay log (Agent Message Bus#Delivery guarantees). With `trail_log_size = 10000` and 1.6 MB messages, the log for `forager.forager-09.infer` is 16 GB before the broker starts dropping. This is AF-70, the replay storm.

Mitigation used in 2015: a separate `infer.*` trail namespace with `trail_log_size = 50`. Documented in the 2015 raw measurement tables (Appendix B has the summary).

== What we should have done ==

Direct host-to-host activation transfer with the bus carrying only control messages. Estimated at two weeks in 2015. Never scheduled.

-- r.osei 2015-03-09