Response to
Reviewer m2mq
Videos for the real-capture test are on the supplementary video page.
- Novelty relative to Context-as-Memory and WorldMem
- Computational cost
- Direct comparison under a unified setting
- Evaluation data
- Longer horizons
Novelty relative to Context-as-Memory and WorldMem
Comparison on the three design axes the reviewer lists. Memorizon column is from our code and configs; [VERIFY: CaM and WorldMem columns against Sec. 2 of the paper and their papers].
| training sample | retrieval | what the loss covers | |
|---|---|---|---|
| Memorizon | span length is a sampled variable, m ~ U[9, m_max] chunks (10 s to 100 / 200 / 400 s); history is never tokenized | each of the 10 scored chunks takes its own top-6 by frustum overlap minus pose distance; the union (<= 60 slots) is a shared bank, each chunk attends only to its own six | last 10 chunks only; sequence length is bounded by 1 + bank + recent + targets whatever the span |
| Context-as-Memory | [VERIFY] | [VERIFY] | [VERIFY] |
| WorldMem | [VERIFY] | [VERIFY] | [VERIFY] |
What the framework adds: training on returns that outlast the window without paying for the intervening frames; the shared union bank, whose size does not depend on span; and a recipe in which span is one integer.
In our ablations, replacing per-chunk retrieval with one retrieval for the whole scored block (the closest analogue in our backbone to per-clip retrieval) lowers web-photo Revisit from 0.434 to 0.302, and
sharing one RoPE index across the bank lets a 10 s-trained model read frames 59 s old. We have reworded "first to decouple" claims in Sec. 1 to the union-bank mechanism.
Computational cost
- Training: step time 8.02 s -> 9.00 s from 100 s to 400 s, peak memory flat (Table 10).
- Bank size: the union is bounded by n_scored x K = 60 slots, at most, at training time; at inference one chunk is generated at a time, so the bank is that chunk's own top-K (6 slots), not kK. Per-chunk retrieval does not enlarge the sequence at inference.
- Retrieval scoring (pose-only, CPU): 0.65 ms at 10 s of history, 11 ms at 100 s, 32 ms at 400 s, 70 ms at 800 s per chunk, single thread.
- Inference along a 400 s rollout (one H200, 20 steps, CFG, one video, median per chunk): sliding window 2.71 s/chunk at every history length; Memorizon (200 s model, k = 6) 3.18 s at 10 s of history and 3.19 s at 300-400 s, i.e. +17% over a sliding window and constant in history length. Peak allocated memory 22.6 GB for both (reserved 33.0 vs 42.9 GB, an allocator effect we did not break down). One video each; no variance estimate.
Direct comparison under a unified setting
We could not run the released CaM/WorldMem models on our protocol (Appendix C.6). The mechanism-level table above and the per-segment-retrieval row of Table 2 are the closest controlled comparison we have on one backbone. [PENDING/NOT DONE: reproducing their training recipes on our backbone; stated as future work.]
Evaluation data
[VERIFY and fill: how the 20 web photos were chosen (style mix, resolution filter, ten replaced for low light / upscaled images), how the 400 s walks were designed (90 deg turns, out-and-back, returns to start), and why existing benchmarks do not fit: they lack continuous camera paths from a single photograph with minutes-long returns.]
Longer horizons
[NOT DONE: baselines to 400 s need each model's own pipeline; Fig. 6 already shows Memorizon's return intervals beyond 60 s. We state this limit explicitly.]