Day 72: Confounders, Bad Controls and Timing · Applied Decision Intelligence & Data Architecture

Written lesson

Time	Block
0:00 – 0:23	72.1 Candidate confounders for rate hikes and housing
0:23 – 0:50	72.2 Good and bad controls: mediators, colliders, instruments
0:50 – 0:58	Break
0:58 – 1:20	72.3 Simulating bad controls
1:20 – 1:40	72.4 Timing as identification: controls must precede the decision
1:40 – 1:48	Break
1:48 – 2:10	72.5 Overlap and balance in real Canadian data
2:10 – 2:30	72.6 Mini-project: a balance and overlap report

72.1 Candidate Confounders for Rate Hikes and Housing (23 min)

A confounder is a common cause of the treatment (rate changes) and the outcome (housing starts). Brainstorm with economic reasoning, then check what your platform already provides.

Candidate	Affects rate decisions because…	Affects housing because…	Expected bias if omitted	In the platform?
inflation (headline, core)	the Bank targets 2%	correlated with construction costs and nominal activity	likely upward	✓ `macro_monthly`
real activity (GDP growth, CLI)	output gap in the reaction function	demand for housing	upward	✓ `ml_gdp_quarterly`, `oecd_cli`
labour market (unemployment)	slack in the reaction function	household income, demand	upward	✓
past housing activity	the Bank watches housing and financial stability	housing is persistent	upward	Day 78 (housing starts series)
US / global rates	Canadian rates track global conditions	global conditions affect Canadian demand	ambiguous	✓ `fed_funds`, US yields
oil / terms of trade	affects Canadian inflation and income	affects incomes in energy regions	ambiguous	✓ `wti`
exchange rate	affects imported inflation	affects relative prices and demand	ambiguous	✓ `usd_cad`
population growth / immigration	affects demand and inflation	directly affects housing demand	likely upward	✗ (candidate addition)
credit and financial conditions	affect the transmission of policy	affect mortgage availability	ambiguous	partly: yield spreads

"Expected bias" is a hypothesis from economic reasoning (Day 71.4's sign test), not a result. Writing it down before estimating is good scientific practice: it guards against rationalising whatever the data show.

72.2 Good and Bad Controls (28 min)

Not every variable correlated with both D and Y should be controlled for. The role of a variable in the causal structure decides. Cinelli, Forney and Pearl (2022), A Crash Course in Good and Bad Controls, give a systematic treatment. Here are the cases that matter for us:

Role	Structure	Control for it?	Example in our question
Confounder	X → D and X → Y	✅ yes: blocks a backdoor path	inflation, growth, unemployment
Mediator	D → M → Y	❌ no, if you want the total effect: controlling removes part of the effect	mortgage rates, 5-year bond yields, house prices (rate hikes raise mortgage rates, which reduce housing)
Collider	D → S ← Y	❌ no: conditioning creates a spurious association	a variable caused by both, e.g. "housing mentioned in the Bank's statement" if both policy and housing drive it
Post-treatment variable	caused by D	❌ generally no	anything measured after the decision
Instrument-like	Z → D only	⚠️ usually no: doesn't reduce bias and can amplify bias from unobserved confounding	a variable that moves rates but has no other link to housing
Outcome-only predictor	X → Y only	✅ optional: improves precision, no bias	regional weather affecting construction

The mediator trap is the most common mistake in monetary-policy work. "Control for mortgage rates" sounds sensible, but rate hikes affect housing largely through mortgage rates. Controlling for them asks "what's the effect of a hike holding mortgage rates fixed?", which is nearly zero by construction and isn't the policy question. If you want to study mechanisms, that's mediation analysis, a different estimand.

72.3 Simulating Bad Controls (23 min)

Add to causal/simulate.py:

def mediator_world(T=600, seed=1):
    """D → mortgage rate M → Y; cycle C confounds D and Y. Total effect of D: -6 × 0.8 = -4.8."""
    rng = np.random.default_rng(seed)
    c = np.zeros(T)
    for t in range(1, T):
        c[t] = 0.9 * c[t - 1] + rng.normal(0, 1)
    d = 0.5 * c + rng.normal(0, 0.5, T)
    m = 0.8 * d + rng.normal(0, 0.3, T)                       # mortgage rates follow policy
    y = 10 * c - 6 * m + rng.normal(0, 3, T)
    return pd.DataFrame({"cycle": c, "rate_change": d, "mortgage_rate_change": m,
                         "housing_growth": y})


def collider_world(T=600, seed=2):
    """True effect −5; S is caused by both D and Y (a collider)."""
    df = confounded_world(T=T, seed=seed)
    rng = np.random.default_rng(seed + 100)
    df["collider"] = df["rate_change"] + 0.2 * df["housing_growth"] + rng.normal(0, 1, len(df))
    return df


def amplification_world(T=2000, seed=3):
    """Unobserved U confounds D and Y; Z affects D only (instrument-like). True effect −5."""
    rng = np.random.default_rng(seed)
    u = rng.normal(0, 1, T)
    z = rng.normal(0, 1, T)
    d = 1.0 * z + 0.5 * u + rng.normal(0, 0.3, T)
    y = -5 * d + 3 * u + rng.normal(0, 1, T)
    return pd.DataFrame({"z": z, "rate_change": d, "housing_growth": y, "u_unobserved": u})

Run the experiments:

import statsmodels.formula.api as smf

from causal.simulate import amplification_world, collider_world, mediator_world

med = mediator_world()
print("TOTAL effect target: -4.80")
print("control cycle only:          ",
      smf.ols("housing_growth ~ rate_change + cycle", med).fit().params["rate_change"].round(2))
print("control cycle + mortgage rate:",
      smf.ols("housing_growth ~ rate_change + cycle + mortgage_rate_change", med)
      .fit().params["rate_change"].round(2))

col = collider_world()
print("\ncollider — true effect -5")
print("control cycle:            ",
      smf.ols("housing_growth ~ rate_change + cycle", col).fit().params["rate_change"].round(2))
print("control cycle + collider: ",
      smf.ols("housing_growth ~ rate_change + cycle + collider", col).fit().params["rate_change"].round(2))

amp = amplification_world()
print("\nbias amplification — true effect -5 (U unobserved)")
print("no controls:  ", smf.ols("housing_growth ~ rate_change", amp).fit().params["rate_change"].round(2))
print("control for Z:", smf.ols("housing_growth ~ rate_change + z", amp).fit().params["rate_change"].round(2))

What you should see:

Mediator: controlling for the mortgage rate pushes the estimate towards zero, erasing most of the real effect.
Collider: adding the collider creates bias where there was none.
Amplification: controlling for the instrument-like Z makes the estimate more biased. It removes "good" variation in D (driven by Z) and leaves a larger share of variation driven by the unobserved U.

These simulations are your intuition pumps. Whenever someone proposes "let's just add more controls", or an ML method that uses "all available variables", remember these three results. More controls aren't automatically better; the right controls are.

72.4 Timing as Identification: Controls Must Precede the Decision (20 min)

In time series, timing is a powerful tool for telling confounders from mediators. A variable measured before the rate decision can't be caused by it. Measured after, it may be a mediator or post-treatment variable.

The rule: controls must be pre-treatment, meaning known before the decision was made. You've already built exactly this logic twice:

Earlier work	What it gave you
Day 17: `ASOF` joins with the CPI publication lag	"the CPI reading known at each decision"
Day 68: the feature spec with publication lags	features built only from data published by each origin

For a monthly design:

treatment D_t: change in the overnight rate during month t (or at the decision in month t);
controls X_t: values published by the start of month t (CPI for t−2 or t−1, LFS for t−1, CLI for t−3, market data for end of t−1);
outcome Y_{t,h}: housing growth from t−1 to t+h.

     t−3      t−2      t−1    │  t   │   t+1 … t+h
 CLI ──┘  CPI ──┘  LFS, markets ┘  │  D_t │  Y measured from t−1 to t+h
         controls (pre-treatment)  │decision│

Using contemporaneous monthly values (e.g. unemployment in month t) as controls risks controlling for something already affected by the decision, or by anticipation of it. Lag your controls according to their publication delays, and reuse the Day 68 availability builder rather than writing new code.

72.5 Overlap and Balance in Real Canadian Data (23 min)

Now look at real data. For diagnostics, define a simple binary treatment: hike month = the overnight rate rose during the month. The main analysis (Days 74–80) uses the continuous change.

Create causal/balance.py:

"""Balance and overlap diagnostics for a treatment against pre-treatment controls."""
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler


def standardized_mean_differences(X: pd.DataFrame, treated: pd.Series) -> pd.DataFrame:
    """SMD = (mean_treated − mean_control) / pooled std. |SMD| > 0.25 often flags imbalance."""
    t, c = X[treated == 1], X[treated == 0]
    pooled = np.sqrt((t.var() + c.var()) / 2)
    return pd.DataFrame({"mean_treated": t.mean(), "mean_control": c.mean(),
                         "smd": (t.mean() - c.mean()) / pooled}).sort_values(
        "smd", key=np.abs, ascending=False)


def propensity_scores(X: pd.DataFrame, treated: pd.Series) -> pd.Series:
    model = make_pipeline(StandardScaler(), LogisticRegression(C=1.0, max_iter=1000))
    model.fit(X, treated)
    return pd.Series(model.predict_proba(X)[:, 1], index=X.index, name="pscore")

Build the dataset from the marts, with lagged (pre-treatment) controls:

import pandas as pd

from causal.balance import propensity_scores, standardized_mean_differences
from forecasting.availability import load_monthly_panel

m = load_monthly_panel()
d = m["overnight_rate"].diff()                         # change during month t
hike = (d > 0.001).astype(int).rename("hike")

controls = pd.DataFrame({
    "cpi_yoy_l1":     m["headline_yoy"].shift(1),
    "core_yoy_l1":    m["core_ex_fe_yoy"].shift(1),
    "unemp_l1":       m["unemployment_rate"].shift(1),
    "unemp_chg12_l1": m["unemployment_rate"].diff(12).shift(1),
    "cli_l3":         m["oecd_cli"].shift(3),
    "fed_funds_l1":   m["fed_funds"].shift(1),
    "slope_l1":       m["spread_10_2_bp"].shift(1),
    "wti_pct12_l1":   (100 * m["wti_eom"].pct_change(12)).shift(1),
    "overnight_l1":   m["overnight_rate"].shift(1),     # the policy stance going in
})
data = controls.join(hike).dropna()
print(f"{data['hike'].sum()} hike months out of {len(data)}")

print(standardized_mean_differences(data.drop(columns="hike"), data["hike"]).round(2))
data["pscore"] = propensity_scores(data.drop(columns="hike"), data["hike"])

Reading the diagnostics

SMDs: large values (e.g. inflation, the policy rate going in) confirm what the DAG predicts. Hike months happen in different conditions, which is exactly the confounding we must adjust for.
Propensity scores: the estimated probability of a hike given the controls. Plot their distributions for hike and non-hike months:

import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(8, 3.5))
ax.hist(data.loc[data.hike == 0, "pscore"], bins=30, alpha=0.6, label="no hike", density=True)
ax.hist(data.loc[data.hike == 1, "pscore"], bins=30, alpha=0.6, label="hike", density=True)
ax.set_xlabel("estimated probability of a hike"); ax.legend()
ax.set_title("Overlap: are hikes observed across comparable conditions?")
plt.tight_layout(); plt.savefig("data/forecasts/overlap_pscore.png", dpi=120)

Pattern	Meaning
distributions overlap broadly	comparisons are supported by data
hike months only at high scores, non-hikes only at low scores	poor overlap: estimates will rely on extrapolation by the model
scores near 0 or 1	some conditions always or never lead to hikes, so there's no counterfactual evidence there

Be honest about the sample. Since our panel starts in 2006, there are only a few tightening cycles (2010, 2017–18, 2022–23), so a few dozen hike months. Overlap may be thin and estimates imprecise. This is why the main design uses the continuous rate change (more informative variation) and why lengthening the panel (backfilling earlier history, Day 68) is a valuable investment before Day 78.

72.6 Mini-Project: A Balance and Overlap Report (20 min)

In notebooks/day72.ipynb, produce a short report:

Count hike, cut and hold months by tightening cycle (use seed_boc_decisions for 2022 onwards and the overnight-rate changes for earlier years).
The SMD table, with one sentence on what it implies about confounding.
The propensity-score overlap plot, with the share of hike months whose score lies within the range of non-hike scores.
Trimming: the share of observations with scores outside [0.05, 0.95], and what dropping them would mean for the estimand (the effect only for "comparable" months).
Your recommendation: is the binary-hike design viable, or should the continuous design be primary? Justify with the diagnostics.

Day 72 checklist

I listed candidate confounders with reasoning about the direction of bias, and checked the platform for each
I can classify controls as confounders, mediators, colliders, instrument-like or outcome-only
My simulations show the mediator trap, collider bias and bias amplification
Controls are pre-treatment, respecting publication lags (reusing the Day 68 logic)
I assessed balance (SMDs) and overlap (propensity scores) in real Canadian data, and reported the limitations

git add causal notebooks && git commit -m "Day 72: good and bad controls, timing, balance and overlap" && git push

Confounders, Bad Controls and Timing

What you'll learn today

Today's plan