Day 78: Housing Starts Through the Platform, and First Estimates · Applied Decision Intelligence & Data Architecture

Written lesson

Time	Block
0:00 – 0:23	78.1 Finding the right housing-starts series
0:23 – 0:50	78.2 Through the platform: ingest, seed, model, test
0:50 – 0:58	Break
0:58 – 1:15	78.3 Profiling and sanity checks
1:15 – 1:43	78.4 Running the pre-registered design
1:43 – 1:50	Break
1:50 – 2:10	78.5 Diagnostics: is the design working?
2:10 – 2:30	78.6 Robustness checks and the specification curve

78.1 Finding the Right Housing-Starts Series (23 min)

Canadian housing starts are compiled by the Canada Mortgage and Housing Corporation (CMHC) and published through Statistics Canada tables. The course notebook used table 34-10-0135-01 for a quarterly chart. For our monthly design we want a monthly, seasonally adjusted (at annual rates, SAAR), national total series.

Rather than guess a table number, search the catalogue, just as you searched the Valet catalogue on Day 8. StatCan's WDS lists all tables:

import requests
import pandas as pd

WDS = "https://www150.statcan.gc.ca/t1/wds/rest"
cubes = pd.DataFrame(requests.get(f"{WDS}/getAllCubesListLite", timeout=60).json())
hs = cubes[cubes["cubeTitleEn"].str.contains("housing starts", case=False, na=False)]
print(hs[["productId", "cubeTitleEn", "frequencyCode", "cubeStartDate", "cubeEndDate"]]
      .to_string(index=False))

frequencyCode follows StatCan's code sets (e.g. monthly vs quarterly; check the WDS documentation for the exact codes). Pick a monthly table covering all areas / Canada with seasonally adjusted at annual rates data.

Then find the vector for Canada, total units, housing starts, SAAR: use the table's web page (Add/Remove data → Display vector identifier, Day 10) or getCubeMetadata (Day 26). Verify it before using it:

vec = 12345678          # ← replace with the vector you found
info = requests.post(f"{WDS}/getSeriesInfoFromVector", json=[{"vectorId": vec}], timeout=30).json()
obj = info[0]["object"]
print(obj.get("SeriesTitleEn"), "| table", obj.get("productId"),
      "| frequency", obj.get("frequencyCode"), "| scalar", obj.get("scalarFactorCode"))

Record in causal/DAG.md (data section): table ID, vector ID, title, frequency, unit and scalar factor (e.g. thousands of units).

If you can only find quarterly data, the design still works at quarterly frequency: treatment = change in the overnight rate over the quarter, horizons in quarters, controls aggregated as in the GDP nowcast (Day 69). Record this as a deviation. Monthly data give more observations and sharper timing, so prefer them if available.

78.2 Through the Platform: Ingest, Seed, Model, Test (28 min)

Adding a series now takes configuration, not code, the payoff of Modules 3–5.

1. Ingestion (dlt, Day 23)

In .dlt/config.toml, add the vector to the StatCan vector list:

[sources.statcan]
vectors = [41690973, 2062815, 12345678]      # CPI, unemployment rate, housing starts (verified)
latest_n = 36

New vectors load their full history automatically on the first run (Day 30's initial_latest_n).

2. The series seed (Days 30, 35, 38)

Add a row to macro_dbt/seeds/seed_series.csv:

v12345678,housing_starts,statcan,monthly,last,75,true,CA,housing,

Setting in_rates_panel = true makes the monthly panel pick up the new column automatically: int_rates__monthly builds its columns from the seed with run_query (Day 35). Check that instrument_type accepts housing (Day 38's accepted_values test will fail until you add it, which is a useful reminder that tests protect reference data too).

3. Run the pipeline

source .venv/bin/activate
python -m orchestration.flows.daily_refresh            # or: trigger prod-evening from the Prefect UI

dlt loads the series; dbt rebuilds int_macro__observations → int_rates__monthly → macro_monthly (with a new housing_starts column) and exports Parquet. The relationships test confirms the vector is in the seed, and the freshness test starts watching it.

4. Quality expectations (GX, Day 44)

Create quality/specs/housing_starts.yml, with a query asset selecting the series from int_macro__observations (register it in setup_gx.py):

description: "CMHC housing starts (monthly, SAAR, Canada) — the outcome of the Module 8 causal design."
asset: housing_starts
batch_definition: full
owner: macro-team

critical:
  - type: ExpectColumnValuesToNotBeNull
    kwargs: {column: value}
    meta: {plan_id: H01, dimension: completeness}
  - type: ExpectColumnValuesToBeBetween
    kwargs: {column: value, min_value: 0, strict_min: true}
    meta: {plan_id: H02, dimension: validity, rationale: "Starts are a positive count."}
  - type: ExpectColumnValuesToBeUnique
    kwargs: {column: obs_date}
    meta: {plan_id: H03, dimension: uniqueness}

warning:
  - type: ExpectColumnValuesToBeBetween
    kwargs: {column: value, min_value: 50, max_value: 500, mostly: 0.999}
    meta: {plan_id: H04, dimension: plausibility,
           rationale: "SAAR in thousands; set the band from the Day 45-style profile of YOUR series."}

Adjust H04 to the unit you recorded in 78.1 (thousands vs units). A unit mistake here is exactly the kind of error Day 41 warned about.

5. Exposure and documentation

Add housing_starts to the column documentation of macro_monthly (a doc block, Day 40), and add the causal analysis as an exposure depending on macro_monthly (causal_housing_analysis, type analysis).

78.3 Profiling and Sanity Checks (18 min)

Before estimating anything, look at the data, as you have before every analysis in this course.

import matplotlib.pyplot as plt

from forecasting.availability import load_monthly_panel

panel = load_monthly_panel()
hs = panel["housing_starts"].dropna()
print(hs.describe().round(1))
print("first:", hs.index.min().date(), " last:", hs.index.max().date(), " n:", len(hs))

fig, ax1 = plt.subplots(figsize=(11, 4))
ax1.plot(hs.index, hs, label="housing starts (SAAR)")
ax2 = ax1.twinx()
ax2.step(panel.index, panel["overnight_rate"], color="#C95555", where="post", label="overnight rate (%)")
ax1.set_title("Housing starts and the policy rate"); fig.legend(loc="upper left")
plt.tight_layout(); plt.savefig("data/causal/housing_vs_rate.png", dpi=120)

Checks:

Units: monthly SAAR starts are an annualised rate: a value of 250 (thousand) means "at this month's pace, 250,000 starts per year". The notebook's quarterly chart (Module 05) showed quarterly counts of a different magnitude. Make sure you understand which you have.
Gaps: the Day 49 continuity logic applies. Confirm there are no missing months.
Volatility: monthly starts are noisy (multi-unit projects start in lumps). That's one reason the outcome is growth over several months, and why we expect wide intervals.
Eyeball the raw relationship: in the chart, do starts fall after tightening cycles? Remember Day 71: the raw picture mixes the causal effect with the cycle that drove the hikes.

78.4 Running the Pre-Registered Design (28 min)

This is the moment the whole module prepared for:

source .venv-ml/bin/activate
python -m causal.run_design --outcome housing_starts --outcome-type log --horizons 3,6,12,18,24

The spec's lagged-outcome control (housing_growth_l1_6: log growth of starts over the six months before the decision) now activates automatically, since the housing_starts column exists.

Read the main table in the report (data/causal/report_housing_starts_*.md):

Column	Meaning
`theta`	% change in housing starts from t−1 to t+h, per 1pp increase in the overnight rate during month t
`ci_lo`, `ci_hi`	95% interval with HAC standard errors (Day 76)
`r2_m`	out-of-fold R² of the reaction function E[D \| X]
`r2_l`	out-of-fold R² of E[Y \| X]
`n`	months used

From θ to policy language

Rate decisions usually move in 25bp steps, so translate:

effect of a 25bp hike = θ / 4 (percent of housing starts after h months)

and into units at the current level:

latest_level = hs.iloc[-1]
main = pd.read_parquet(sorted(Path("data/causal").glob("main_housing_starts_*.parquet"))[-1])
main["per_25bp_pct"] = main["theta"] / 4
main["per_25bp_units_saar"] = latest_level * main["per_25bp_pct"] / 100
print(main[["h", "theta", "ci_lo", "ci_hi", "per_25bp_pct", "per_25bp_units_saar"]].round(2))

Don't interpret yet beyond sign, magnitude and precision. Day 79 does the full interpretation, after robustness and sensitivity.

Compare with the naive regression

Also run the naive estimate (housing growth on the rate change, no controls) and the linear-controls estimate (OLS learners), and keep them for the memo:

import statsmodels.api as sm

from causal.design import estimation_frame, load_design

spec = load_design()
df = estimation_frame(panel, spec, panel["housing_starts"], h=12, log_outcome=True)
naive = sm.OLS(df["y"], sm.add_constant(df["d"])).fit(cov_type="HAC", cov_kwds={"maxlags": 12})
print(f"naive θ (h=12): {naive.params['d']:+.2f}  (HAC se {naive.bse['d']:.2f})")

If the naive estimate is less negative (or positive) and the DML estimate more negative, you're seeing the confounding of Day 71 in real data. Booms raise both rates and housing.

78.5 Diagnostics: Is the Design Working? (20 min)

Diagnostic	Where	Healthy sign	Worry
reaction-function fit	`r2_m`	moderate (policy partly predictable)	very high: little residual variation; very low: controls miss what drives policy
outcome fit	`r2_l`	positive	negative: nuisance model predicts worse than the mean, so θ̂ is unreliable
learner agreement	learner table (Day 77.3)	ML learners agree; OLS may differ	ML learners disagree widely
residual treatment variation	plot of `d − m̂(X)` over time	spread across cycles, not one episode	concentrated in one period (e.g. 2022 only): the estimate is effectively from a single episode

The last diagnostic deserves a picture. Where does the identifying variation come from?

from causal.dml import blocked_splits, cross_fit_residuals
from sklearn.ensemble import RandomForestRegressor

X = df.drop(columns=["d", "y"])
rf = RandomForestRegressor(300, min_samples_leaf=5, max_features=0.5, random_state=0, n_jobs=2)
y_res, d_res = cross_fit_residuals(df["y"].to_numpy(), df["d"].to_numpy(), X.to_numpy(),
                                   rf, rf, blocked_splits(len(df), 5, buffer=18))
v = pd.Series(d_res, index=df.index, name="policy surprise (pp)")
ax = v.plot(figsize=(11, 3), title="Residual rate changes: the variation that identifies θ")
ax.axhline(0, color="grey"); plt.tight_layout(); plt.savefig("data/causal/policy_surprises.png", dpi=120)
print((v ** 2).groupby(v.index.year).sum().div((v ** 2).sum()).round(2).sort_values(ascending=False).head())

The last line shows the share of identifying variation by year. If one year contributes most of it, say so in the memo: "the estimate is driven mainly by the 2022 tightening". It's an honest statement about external validity.

78.6 Robustness Checks and the Specification Curve (20 min)

Run the pre-registered checks (Day 73's spec, implemented on Day 77) on the real outcome. run_design.py already does this; read its robustness table and specification curve:

Check	What a sound design shows	What to do if it fails
placebo outcome (starts from t−12 to t−6)	θ ≈ 0	report residual confounding or anticipation; lean on sensitivity analysis (Day 79)
negative-control treatment (rate change at t+12)	θ ≈ 0	slow-moving confounding; consider additional trend controls, recorded as a deviation
3-month cumulative treatment	same sign and profile	discuss treatment timing
excluding the pandemic	similar estimate	report both; explain which episode drives results
linear vs ML controls	documented difference	explain via non-linear confounding
buffer sensitivity	similar estimate	report the range
specification curve	most specifications agree in sign	conclusion is fragile; say so

Copy the robustness table and specification-curve figure into notebooks/day78.ipynb with no interpretation yet, just facts. Any change you feel tempted to make goes into causal/DEVIATIONS.md with a reason, and the original result stays in the record.

Day 78 checklist

I found, verified and documented a monthly SAAR housing-starts vector (table, vector, unit, scalar)
Housing starts entered the platform via configuration: dlt vector, seed row, automatic panel column, tests, GX spec, exposure
I profiled the series and understand SAAR vs quarterly counts
I ran the pre-registered design and translated θ into per-25bp and unit terms
I checked diagnostics, including where the identifying variation comes from
Pre-registered robustness checks and the specification curve are recorded without post-hoc changes

git add .dlt/config.toml macro_dbt quality causal notebooks
git commit -m "Day 78: housing starts through the platform; pre-registered DML estimates"
git push

Housing Starts Through the Platform, and First Estimates

What you'll learn today

Today's plan