Time-Series Forecasting in Keras: Windowing, Leakage, and Honest Backtesting

Sensor traces, demand histories, energy loads, yield curves from a line of machines: most of the applied work we see outside vision is time-series. It is also where projects most often produce a great validation number and a disappointing deployment. The cause is almost never the architecture. It is the evaluation setup, and the leakage hiding inside it.

This post is the order we work in: baseline, windowing, backtest, then model.

Start with the forecasting contract

Before any code, write down four things. They determine everything downstream.

1. Horizon. How far ahead, in the units the business acts on? One step is a different problem from 24 steps, and multi-step needs a decision: direct (one model per horizon, or a multi-output head), or recursive (feed predictions back in, and accumulate error).
2. Frequency and lookback. Sampling interval, and how much history the model may see. Lookback should cover at least one full seasonal cycle you care about — hourly data with daily and weekly seasonality wants 168 steps, not 24.
3. Information cutoff. At prediction time in production, which features actually exist? Weather forecasts exist; measured weather for the target hour does not. Every feature must be answerable as "known at cutoff."
4. Metric. MAE and RMSE are not interchangeable — RMSE pays for spikes, MAE does not. On intermittent or zero-heavy series, MAPE is unusable (division by near-zero) and something like MASE or a weighted quantile loss is more honest. If the decision is asymmetric — a stockout costs more than a day of carry — say so now and use a quantile loss instead of arguing about the mean later.

Baseline first, and expect it to be good

Run these before you build anything:

• Naive: y_hat(t+h) = y(t).
• Seasonal naive: y_hat(t+h) = y(t+h-m) for season length m. On daily-cycle sensor data this is frequently very hard to beat.
• Rolling mean or drift, for slow series.
• One classical statistical model where the series count is small enough to fit them.

These cost an hour, are computed on the same backtest as the neural models, and give you the only number that matters: how much a deep model is worth. We have closed engagements by showing that a seasonal-naive baseline matched a client's LSTM within 2% MAE — the useful work was then in the features and the data pipeline, not the model.

Windowing without leaking

The standard Keras path is keras.utils.timeseries_dataset_from_array, which turns an array into batched (window, target) pairs:

import keras

LOOKBACK, HORIZON, STRIDE = 168, 24, 1

train_ds = keras.utils.timeseries_dataset_from_array(
    data=features[:-HORIZON],      # shape (T, n_features)
    targets=target[LOOKBACK + HORIZON - 1:],
    sequence_length=LOOKBACK,
    sequence_stride=STRIDE,
    batch_size=64,
    shuffle=True,
)

Get the target offset right by asserting it, not by reading it. Pull one batch, and check by hand that the last timestamp in the window is strictly before the timestamp of the target. An off-by-one here is the single most common time-series bug we find in review, and it always looks like a great result.

Three more leakage rules:

• Split by time, never shuffle before splitting. Train on the past, validate on the next block, test on the block after that. Shuffling windows across the split boundary leaks the future through overlapping windows.
• Fit scalers on train only. Normalization statistics computed over the whole series carry future information into training. Fit on the training slice and apply the frozen statistics elsewhere — keras.layers.Normalization with adapt() called on the training dataset only, so the constants ship inside the model.
• Leave a gap. Windows near the boundary overlap both sides. Drop LOOKBACK + HORIZON steps between splits.

For per-entity data (many machines, stores, patients), decide whether you are forecasting seen entities forward in time or generalizing to new entities. If it is the latter, split by entity and by time, and expect a lower, truer number.

Backtest with rolling origins

A single train/validation split on a time series is one sample of one regime. Use rolling-origin evaluation: several consecutive cutoffs, each training on everything before and scoring the next horizon window.

|--- train ---|-- test --|
|------- train -------|-- test --|
|----------- train ----------|-- test --|

Report the mean and the spread across folds, for the model and for every baseline, on identical folds. The spread is the interesting part: a model that wins on average but collapses in one fold is telling you about a regime it cannot handle — a plant shutdown, a promotion, a firmware change — and that fold will recur in production. Also score per horizon step, not just pooled; multi-step error growth is usually where recursive forecasting reveals itself as a bad choice.

Then the models, cheapest first

In Keras 3 these are all short, and all backend-portable if you write them with keras.layers and keras.ops:

1. Linear on the flattened window. Flatten then Dense(HORIZON). Astonishingly strong on many real series, and a much tougher baseline than the naive ones.
2. 1D convolutional stack. Conv1D with dilations covers long lookbacks cheaply, trains fast, and parallelizes over the window — usually our first neural attempt.
3. GRU or LSTM. Reach for these when state genuinely matters or the series is irregular. They cost more wall-clock per epoch than a conv stack and often do not repay it.
4. Small attention block. keras.layers.MultiHeadAttention over the window, with positional information added. Worth trying on long horizons with many covariates; rarely worth it on a few thousand short series.

Also fold in the calendar and lag features a neural net will not discover for you: hour-of-day and day-of-week as cyclical sine/cosine pairs, holiday flags, lagged seasonal values, rolling statistics computed strictly from the past. On most industrial datasets, feature work moves the metric more than swapping model families.

A last note on output: predict a quantile spread, not just a point. Training three heads with pinball loss at the 10th, 50th, and 90th percentiles costs almost nothing and gives downstream consumers an interval, which is what an operator actually needs to set a threshold.

What good looks like

A time-series project we would sign off on has: a written forecasting contract, baselines scored on the same folds as the model, an asserted window/target alignment test in CI, scalers fitted on train only, rolling-origin results with per-fold and per-horizon breakdowns, and a model that beats seasonal naive by a margin large enough to survive the fold spread. If the deep model does not clear that bar, the honest deliverable is the baseline plus better features — and that is a perfectly good outcome to report.