Hierarchical Decay Mechanisms for Infinite-Context Memory
Implement multi-scale fast weights with varying decay rates to maintain both local detail and global context in infinite-sequence memory models.
Source paper
Chen Wang, Hao Tan, Wang Yifan, Zhiqin Chen, Yuheng Liu, Kalyan Sunkavalli, Sai Bi, Lingjie Liu, Yiwei Hu
Generated by AI. The paper’s authors did not write these.
Implement multi-scale fast weights with varying decay rates to maintain both local detail and global context in infinite-sequence memory models.
Incorporate Fisher Information regularization directly into training to prevent fast weights from drifting and corrupting previously learned geometries.
Add a secondary neural residual decoding branch to explicitly composite high-frequency screen-space details over rendered 3D Gaussians.
The paper presents tttLRM, a Large Reconstruction Model that utilizes Test-Time Training (TTT) layers to perform 3D reconstruction from long sequences of images with linear computational complexity. It enables both feedforward and autoregressive (streaming) reconstruction of explicit 3D representations like Gaussian Splats, scaling efficiently to large context windows where traditional Transformers fail.
The model replaces standard self-attention with Large Chunk Test-Time Training (LaCT) blocks. Input image tokens update the layer's internal 'fast weights' (a compressed latent memory), which are subsequently queried by virtual view tokens to decode explicit 3D parameters (e.g., 3D Gaussian attributes or NeRF triplanes).