Motivation
Reviewer m2mq: "...particularly inference latency and GPU memory usage, has not been sufficiently analyzed."
Reviewer MjTC: "What fraction of step latency goes to pose scoring at the end of a 400s rollout?"
Results
Per-chunk retrieval adds 17% inference time and no peak memory. Pose scoring is about 1% of chunk time.
Table 1: Training, inference and scoring cost of Memorizon (one H200, 20 steps, CFG).
| value | |
|---|---|
| Training step, 100 → 400 s | 8.02 → 9.00 s; peak memory flat |
| Bank size | ≤ 60 slots in training; 6 slots per chunk at inference |
| Pose scoring (CPU, 1 thread) | 0.65 ms at 10 s · 11 ms at 100 s · 32 ms at 400 s · 70 ms at 800 s |
| Inference per chunk, 400 s rollout | sliding window 2.71 s · Memorizon 3.18 s (+17%), flat in history length |
| Peak GPU memory | 22.6 GB, same as sliding window |
- Pose scoring is O(n) in history length; only the attention cost is constant, and the text now says so. A spatial index would make scoring sublinear.
Conclusion
- Memory costs 17% more inference time (3.18 vs 2.71 s per chunk) and no extra peak memory (22.6 GB).
- Cost does not grow with history: chunk time is flat from 10 s to 400 s of history.
- Pose scoring is about 1% of chunk time (32 ms of 3.2 s at 400 s of history).