Skip to content

Source paper

A decoder-only foundation model for time-series forecasting

Abhimanyu Das, Weihao Kong, Rajat Sen, Yichen Zhou
arXiv 2310.10688 · 202310 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces TimesFM, a 200M-parameter decoder-only foundation model for time-series forecasting pretrained on a massive corpus of 100 billion real and synthetic time-points. The authors demonstrate that by using a patch-based decoder architecture and extensive pretraining, the model achieves zero-shot performance on unseen datasets that rivals or exceeds state-of-the-art supervised models trained specifically for those tasks.

How it works (AI summary)

The model uses a decoder-only Transformer architecture modified with input and output patching. Input time-series are broken into patches (tokens), processed by residual blocks, and fed into a causal transformer. Uniquely, the model predicts output patches that can be longer than input patches to reduce autoregressive inference steps. It was trained on a mixed corpus of Google Trends, Wiki Pageviews, and synthetic data using a standard Mean Squared Error (MSE) loss.