Multivariate CPI Forecasting with Real-Time Information Sets

68.1 Which Indicators Should Help Forecast Inflation? (20 min)

Before throwing every column at a model, write down why each candidate might help. Economic reasoning narrows the search and makes results interpretable. All of these already exist in your marts (Days 19, 20, 39):

Expect humility. Atkeson and Ohanian (2001) found that Phillips-curve forecasts often failed to beat a naive benchmark for US inflation, and Stock and Watson (2007) documented how inflation became harder to forecast after the mid-1980s. Your multivariate model must earn its complexity against Day 63's univariate leaderboard.

68.2 Publication Lags: Building Features Only from What Was Known (30 min)

The real-time information set

Our forecast origin is "just after CPI for month t is published", usually in the third week of month t+1. At that moment, what else is known?

Using the OECD CLI for month t at origin t would be a look-ahead leak: in reality it wasn't published yet. Each feature needs an explicit availability lag.

A reviewed feature specification

Like quality specs (Day 44) and seeds (Day 32), features belong in a reviewed configuration file. Create forecasting/feature_spec_cpi.yml:

# Predictors for CPI inflation (target: cpi_yoy, direct horizons).
# lag = months between the origin month and the latest month published at the origin.
target: cpi_yoy
origin: "just after CPI for month t is published (~3rd week of t+1)"

features:
  - {name: core_yoy,     column: core_ex_fe_yoy,     lag: 0, transforms: [level],          family: core}
  - {name: median_major, column: median_major_yoy,   lag: 0, transforms: [level],          family: core}
  - {name: diffusion,    column: diffusion_above_3,  lag: 0, transforms: [level],          family: core}
  - {name: unemp,        column: unemployment_rate,  lag: 0, transforms: [level, diff12],  family: labour}
  - {name: overnight,    column: overnight_rate,     lag: 0, transforms: [level, diff12],  family: monetary}
  - {name: real_rate,    column: real_rate,          lag: 0, transforms: [level],          family: monetary}
  - {name: slope,        column: spread_10_2_bp,     lag: 0, transforms: [level],          family: financial}
  - {name: usd_cad,      column: usd_cad_eom,        lag: 0, transforms: [pct12, pct3],    family: external}
  - {name: wti,          column: wti_eom,            lag: 0, transforms: [pct12, pct3],    family: external}
  - {name: ca_us_2y,     column: ca_us_2y_spread_bp_avg, lag: 0, transforms: [level],      family: external}
  - {name: oecd_cli,     column: oecd_cli,           lag: 2, transforms: [level, diff3],   family: cycle}

(Column names follow macro_monthly from Days 20 and 39. Check them with pd.read_parquet("data/marts/macro_monthly.parquet").columns.)

The availability-aware builder

Create forecasting/availability.py:

"""Build features from a reviewed spec, respecting publication lags (no look-ahead)."""
from pathlib import Path

import pandas as pd
import yaml

from forecasting.data import MARTS


def load_spec(path: str | Path) -> dict:
    with open(path) as f:
        return yaml.safe_load(f)


def transform(x: pd.Series, how: str) -> pd.Series:
    if how == "level":
        return x
    if how.startswith("diff"):
        return x - x.shift(int(how[4:]))
    if how.startswith("pct"):
        k = int(how[3:])
        return 100 * (x / x.shift(k) - 1)
    raise ValueError(f"unknown transform {how}")


def build_features(panel: pd.DataFrame, spec: dict) -> tuple[pd.DataFrame, dict]:
    """Row t contains only information published by origin t. Returns (X, family_of_column)."""
    X, families = {}, {}
    for f in spec["features"]:
        series = panel[f["column"]].shift(f["lag"])        # value of month t - lag, known at t
        for how in f["transforms"]:
            name = f"{f['name']}_{how}"
            X[name] = transform(series, how)
            families[name] = f["family"]
    return pd.DataFrame(X, index=panel.index), families


def load_monthly_panel() -> pd.DataFrame:
    m = pd.read_parquet(MARTS / "macro_monthly.parquet")
    m["month"] = pd.to_datetime(m["month"])
    return m.set_index("month").sort_index().asfreq("MS")

shift(lag) puts the value of month t − lag in row t. Transforms are applied after the shift, so a 3-month change of the CLI at origin t compares t−2 with t−5, both genuinely published by then.

Assemble the multivariate design matrix

from forecasting.availability import build_features, load_monthly_panel, load_spec
from forecasting.data import load_cpi
from forecasting.features import univariate_cpi_features

cpi = load_cpi(start="1992-01-01")
panel = load_monthly_panel()
spec = load_spec("forecasting/feature_spec_cpi.yml")

X_uni = univariate_cpi_features(cpi)                    # Day 61: persistence and momentum
X_mac, fam = build_features(panel, spec)
X = X_uni.join(X_mac, how="left").loc["2006-01":]       # macro panel starts in 2006
y = cpi["cpi_yoy"]
print(X.shape)
print(X.isna().mean().sort_values(ascending=False).head(8).round(2))

Look at the missing shares. Some series start later. For example, BoC's FXCADUSD begins in 2017 in our ingestion (the notebook's "Post-2017 FX API" note), so usd_cad_* is empty before then.

Missing values: let XGBoost handle them, or impute?

Our harness from Day 61 requires complete feature rows. Add a parameter so XGBoost can use incomplete ones:

def rolling_origin(X, y, model_factory, horizons, test_start, model_name,
                   refit_every=1, min_train=60, explainer=None, require_complete=True):
    feature_ok = X.notna().all(axis=1) if require_complete else X.notna().any(axis=1)
    ...                                   # the rest unchanged

Optional, recommended: lengthen the panel. Set earlier start_dates in .dlt/config.toml (Day 23) for BoC and FRED series, add DEXCAUS to the FRED list and seed_series, run a backfill (Day 27), and rebuild the marts. More history means more training data, but check for regime breaks (Day 61's 1991 discussion).

68.3 Revisions and Vintages: How "Real-Time" Is Our Backtest? (18 min)

Respecting publication lags is necessary but not sufficient. Some indicators are later revised, and a backtest using today's final values uses information no forecaster had.

A backtest that respects lags but uses final data is called pseudo-real-time (or pseudo out-of-sample). It's the standard in most applied work, as long as you say so.

Your platform can do better over time. Since Day 27 you've stored vintages (append-only snapshots in dlt) and since Day 37 dbt snapshots of CPI. As these histories grow, you can rebuild features from the data as it was on each origin date. Add this sentence to your model card (Day 70):

"The backtest is pseudo-real-time: publication lags are respected, but revised series (OECD CLI, LFS, GDP) use the latest vintage. A fully real-time evaluation becomes possible as the platform's vintage history accumulates."

Try it 68.3

Using the snapshot table from Day 37 (snapshots.snap_cpi_national), check whether any national CPI value has been revised since you began snapshotting. What does the answer imply for the CPI target in your backtest?

select month, component, count(*) as versions
from snapshots.snap_cpi_national
group by all having count(*) > 1;

68.4 Multivariate Models in the Harness (25 min)

Evaluate on a common window. The macro panel starts in 2006, so allow a training period before the first test origin (e.g. test from 2014-01), and re-score the univariate models on the same origins (score() already restricts to common origins).

from sklearn.impute import SimpleImputer
from sklearn.linear_model import RidgeCV
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

from forecasting.evaluate import rolling_origin, score
from forecasting.models import Naive, xgb_factory

H, TEST = [1, 3, 6, 12], "2014-01-01"

runs = [
    rolling_origin(X_uni.loc["2006-01":], y, Naive, H, TEST, "naive"),
    rolling_origin(X_uni.loc["2006-01":], y, xgb_factory, H, TEST, "xgb_univariate", refit_every=3),
    rolling_origin(X, y, xgb_factory, H, TEST, "xgb_multivariate", refit_every=3,
                   require_complete=False),
    rolling_origin(X.drop(columns=[c for c in X if c.startswith("usd_cad")]), y,
                   lambda: make_pipeline(SimpleImputer(strategy="median"), StandardScaler(),
                                         RidgeCV(alphas=[1, 10, 100, 1000])),
                   H, TEST, "ridge_multivariate", refit_every=3),
]
mv = pd.concat(runs, ignore_index=True)
mv.to_parquet("data/forecasts/cpi_multivariate.parquet", index=False)
print(score(mv).round(3).to_string(index=False))

Notes:

• SimpleImputer inside the pipeline learns medians from each training window only. Imputing before the backtest would leak.
• Ridge drops usd_cad_* because those columns are empty for most of the training period, and an imputed constant would be meaningless.
• Add the AR(AIC) benchmark and Chronos from Days 63 and 67 on the same window, to compare against the strongest univariate competitors, not just naive.

Feature count vs sample size

With ~100 training months at the first origin and ~30 features, overfitting is a real risk. XGBoost's regularisation (shallow trees, subsampling, Day 62) and Ridge's penalty are your protection. Out-of-sample evidence decides, not intuition.

68.5 Did the Extra Information Help? Tests and Explanations (20 min)

Statistical comparison

from forecasting.evaluate import diebold_mariano

for h in H:
    print(diebold_mariano(mv, "xgb_multivariate", "xgb_univariate", h))

This is the key test of the day: does multivariate beat univariate, with the same model class, the same origins and the same horizons? A significant negative statistic at some horizons means the macro information adds value there.

Explanations by economic family

Use Day 66's tools, with the spec's families (core, labour, monetary, financial, external, cycle) plus the univariate ones (persistence, momentum, seasonality):

from forecasting.explain import xgb_contributions

def family_of_mv(feature: str) -> str:
    if feature in fam:
        return fam[feature]
    if feature.startswith("yoy_"):
        return "persistence"
    if feature.startswith("mom_"):
        return "momentum"
    if feature == "month_of_year":
        return "seasonality"
    return "base_value" if feature == "base_value" else "other"

res = rolling_origin(X, y, xgb_factory, [12], TEST, "xgb_multivariate", refit_every=3,
                     require_complete=False, explainer=xgb_contributions)
expl = res.attrs["explanations"].assign(family=lambda d: d["feature"].map(family_of_mv))

Plot the family decomposition of the 12-month forecast over 2014–2025 (Day 66.2). Questions:

• When the 2021 inflation surge began, did external (oil, FX) or core/breadth families push the forecast up before persistence did?
• Did monetary features pull forecasts down after the 2022–23 hikes, and how quickly?

Interpret carefully: these are predictive associations learned by the model (Day 65.6). "Monetary features pulled the forecast down" doesn't prove that rate hikes lowered inflation. That causal question is Module 8.

68.6 Mini-Project: An Ablation Study by Feature Family (23 min)

An ablation study removes one family at a time and measures how much out-of-sample accuracy suffers. It's permutation importance (Day 65.5) at the level of whole information sets, with retraining.

families = sorted(set(fam.values()))
abl = []
for f in families:
    cols = [c for c in X.columns if fam.get(c) == f]
    r = rolling_origin(X.drop(columns=cols), y, xgb_factory, H, TEST, f"xgb_without_{f}",
                       refit_every=3, require_complete=False)
    abl.append(r)
ablation = pd.concat(abl + [mv[mv.model.isin(["xgb_multivariate", "naive"])]])
board = score(ablation)
full = board[board.model == "xgb_multivariate"].set_index("h")["RMSE"]
board["delta_vs_full"] = board["RMSE"] - board["h"].map(full)
print(board[board.model.str.startswith("xgb_without")]
      .pivot(index="model", columns="h", values="delta_vs_full").round(3))

Reading the table: a positive delta means removing that family made forecasts worse, so it was helping. Near-zero means it added little. Negative means it was hurting (noise or overfitting).

Write a short note in notebooks/day68.ipynb, and update MODEL_DECISIONS.md:

1. Which families help at short and long horizons?
2. Is the full multivariate model significantly better than univariate (68.5)?
3. Which families would you drop to simplify the production model, and why?

Day 68 checklist

• I can justify candidate predictors with economic reasoning, by family
• Features are built from a reviewed spec that encodes each indicator's publication lag
• I handle late-starting features correctly (native XGBoost missing values or in-window imputation)
• I can explain the difference between real-time and pseudo-real-time evaluation, and where our backtest stands
• Multivariate vs univariate is tested with Diebold–Mariano on common origins
• An ablation study shows which information families actually help

git add forecasting notebooks && git commit -m "Day 68: multivariate CPI forecasts with publication lags, ablation" && git push