Integration and the Final Capstone

Time	Block
0:00 – 0:23	80.1 Causal results as a data product
0:23 – 0:43	80.2 Orchestrating the causal analysis
0:43 – 0:50	Break
0:50 – 1:13	80.3 The "Monetary policy & housing" dashboard
1:13 – 1:35	80.4 Capstone extension: designing a causal tariff-impact study
1:35 – 1:43	Break
1:43 – 2:13	80.5 The final capstone: assembly, demonstration and review
2:13 – 2:30	80.6 Course wrap-up

---

80.1 Causal Results as a Data Product (23 min)

Like forecasts (Day 70), causal estimates are data: kept, versioned, tested and published through the platform.

Provenance in run_design.py

Extend the output rows with metadata, so every estimate can be traced:

run_meta = {
    "run_tag": tag,
    "as_of": datetime.now(timezone.utc),
    "outcome": args.outcome,
    "spec_fingerprint": fingerprint([Path("causal/design_spec.yml")]),    # Day 70 helper
    "data_fingerprint": fingerprint([MARTS / "macro_monthly.parquet"]),
    "git_commit": subprocess.run(["git", "rev-parse", "HEAD"], capture_output=True,
                                 text=True).stdout.strip(),
    "doubleml_version": doubleml.__version__,
}
main = main.assign(**run_meta)
checks = checks.assign(**run_meta)

dbt models

macro_dbt/models/staging/causal/_causal__sources.yml:

version: 2
sources:
  - name: causal
    description: "Causal estimates from causal/run_design.py (Module 8)."
    meta:
      external_location: "../data/causal/main_*.parquet"
    tables:
      - name: effects

macro_dbt/models/marts/causal/causal_effects_latest.sql:

-- Grain: one row per outcome × horizon, from the most recent analysis run.
with ranked as (
    select *,
           row_number() over (partition by outcome, h order by as_of desc) as rn
    from {{ source('causal', 'effects') }}
)
select
    outcome, h, theta, se, ci_lo, ci_hi,
    theta / 4                         as per_25bp_pct,
    r2_m, r2_l, n, buffer, hac_lags,
    run_tag, as_of, spec_fingerprint, data_fingerprint, git_commit
from ranked
where rn = 1

Add a similar causal_checks_latest from the robustness outputs, with tests: unique_combination_of_columns on (outcome, h); ci_lo ≤ theta ≤ ci_hi (dbt_utils.expression_is_true); n ≥ 120 (warn). Then add GX expectations for the marts (Day 70.4 style), and an exposure linking them to the new dashboard page.

Keeping every run (not just the latest) means you can show how the estimate evolves as new data arrive: a policy-relevant check of stability, and protection against quietly overwriting an inconvenient result.

---

80.2 Orchestrating the Causal Analysis (20 min)

Causal analysis doesn't need to run daily. It should run when its inputs change meaningfully:

• on a monthly schedule (new housing-starts data);
• when the design spec changes (a reviewed commit), which requires a full rerun and comparison with the previous run.

Add the step policy (Day 54) and a flow, orchestration/flows/causal.py:

"""Monthly causal refresh: re-estimate, model results in dbt, validate."""
from prefect import flow

from orchestration.settings import ML_PY
from orchestration.tasks.commands import step
from orchestration.tasks.dbt import dbt_invoke


@flow(name="causal-housing")
def causal_flow(target: str = "prod", drill: dict | None = None) -> dict:
    states = {"causal_estimates": step(
        "causal_estimates",
        [ML_PY, "-m", "causal.run_design", "--outcome", "housing_starts",
         "--outcome-type", "log", "--horizons", "3,6,12,18,24"], drill=drill)}
    if states["causal_estimates"].is_completed():
        states["dbt_causal"] = dbt_invoke.with_options(name="dbt:causal")(
            ["build", "--target", target, "--select", "source:causal+"], return_state=True)
    return states

# orchestration/settings.py — STEP_POLICIES
    "causal_estimates": dict(retries=0, delays=[], timeout=7200, transient=(), warning=(2,)),

(Statistical estimation is deterministic given its inputs, so a failure isn't transient and isn't retried.)

Deployment in prefect.yaml:

  - name: causal-housing-monthly
    description: "Re-estimate the effect of rate changes on housing starts (Module 8)."
    entrypoint: orchestration/flows/causal.py:causal_flow
    work_pool:
      name: local-process
      job_variables:
        env:
          PYTHONPATH: /workspaces/macro-data-course
    concurrency_limit: 1
    schedules:
      - cron: "0 7 20 * *"              # 20th of each month, 07:00 Toronto time
        timezone: "America/Toronto"

Update STATE_POLICY.md: a failed causal run is a warning for the platform (dashboards keep showing the previous run, with its as_of date), never a blocker for the daily refresh.

---

80.3 The "Monetary Policy & Housing" Dashboard (23 min)

Add a page to dashboards/build_dashboards.py, reading only exported marts:

def policy_housing(monthly: pd.DataFrame, effects: pd.DataFrame, checks: pd.DataFrame) -> Path:
    fig = make_subplots(rows=1, cols=2, column_widths=[0.55, 0.45],
                        subplot_titles=["Housing starts and the overnight rate",
                                        "Estimated effect of a +25bp increase (% of starts)"],
                        specs=[[{"secondary_y": True}, {}]])
    m = monthly[monthly["month"] >= "2006-01-01"]
    fig.add_trace(go.Scatter(x=m["month"], y=m["housing_starts"], name="Housing starts (SAAR)",
                             line=dict(color=C["primary"])), row=1, col=1)
    fig.add_trace(go.Scatter(x=m["month"], y=m["overnight_rate"], name="Overnight rate (%)",
                             line=dict(color=C["negative"], shape="hv")), row=1, col=1,
                  secondary_y=True)

    e = effects[effects["outcome"] == "housing_starts"].sort_values("h")
    fig.add_trace(go.Scatter(x=e["h"], y=e["ci_hi"] / 4, line=dict(width=0), showlegend=False),
                  row=1, col=2)
    fig.add_trace(go.Scatter(x=e["h"], y=e["ci_lo"] / 4, fill="tonexty",
                             fillcolor="rgba(74,144,217,0.25)", line=dict(width=0),
                             name="95% CI (HAC)"), row=1, col=2)
    fig.add_trace(go.Scatter(x=e["h"], y=e["per_25bp_pct"], mode="lines+markers",
                             name="estimate", line=dict(color=C["primary"], width=3)), row=1, col=2)
    fig.add_hline(y=0, line_color=C["muted"], row=1, col=2)

    placebo = checks.query("check == 'placebo outcome'")
    placebo_ok = (not placebo.empty
                  and placebo["ci_lo"].iloc[0] <= 0 <= placebo["ci_hi"].iloc[0])
    caveat = ("Causal estimate under selection-on-observables assumptions (DoubleML, PLR). "
              f"Placebo test {'passed' if placebo_ok else 'NOT passed — interpret with caution'}. "
              f"Run {e['as_of'].max():%Y-%m-%d}. See causal/MEMO.md.")
    fig.update_layout(**LAYOUT, height=520,
                      title=dict(text=f"<b>Monetary policy & housing</b><br><sup>{caveat}</sup>"))
    fig.update_xaxes(title_text="months after the rate change", row=1, col=2)
    out = SITE / "policy_housing.html"
    fig.write_html(out)
    return out

Add the recession bands (Day 7's add_recession_shading logic, or seed_recession_bands) to the left panel, and a KPI to macro_kpis: "Estimated effect of +25bp on housing starts after 12 months: X% [CI]", with its as_of.

The dashboard banner states the assumptions and the placebo result next to the number. A causal estimate shown without its caveats invites over-interpretation, and that's the most common way causal analysis misleads decision-makers.

---

80.4 Capstone Extension: Designing a Causal Tariff-Impact Study (23 min)

The curriculum's capstone table lists "causal tariff impact models" under Model & Forecast. The course notebook tracks a 2025–26 tariff shock (its "Tariff recession" band and sector-level TARIFF_STATUS exposure table). Estimating its effects is a natural extension, and a harder design problem than the rate-hike question. Today you write a design, not results.

Why it's harder

Feature	Rate hikes (Days 78–79)	Tariff shock
treatment variation over time	many decisions over 20 years	essentially one event
comparison	months with different rate changes	units (sectors, provinces) with different exposure
natural design	selection on observables (PLR)	difference-in-differences (DiD) with exposure intensity
main threats	unobserved confounders of policy	other shocks at the same time; anticipation; spillovers; few post-treatment periods

A DiD sketch with exposure intensity

Units: provinces (or industries). Exposure Eᵢ: the share of output or exports in tariffed sectors before the shock (built from the notebook's tariff matrix and provincial or industry data you'd ingest). Outcome Yᵢₜ: provincial employment, exports or manufacturing sales.

Yᵢₜ = αᵢ + λₜ + β · (Eᵢ × Postₜ) + controls + εᵢₜ

β compares the change in outcomes after the shock for more exposed vs less exposed units. The key assumption is parallel trends: without tariffs, exposed and less-exposed units would have evolved alike. Check it with pre-period event-study coefficients (the DiD analogue of the placebo test). DoubleML offers DiD estimators that use ML to adjust for covariates (check the package documentation for the current DiD classes).

Deliverable: causal/tariff_design_spec.yml

Write the design in the same format as Day 73's spec: question, units, exposure definition and data sources (what the platform would need to ingest), outcome, treatment timing (announcement vs implementation), controls, pre-registered checks (event-study pre-trends, placebo dates, leave-one-province-out), and an explicit threats section (concurrent shocks such as interest-rate and energy-price changes; anticipation; trade diversion; small number of units). Implementing it is a strong option for an extended capstone project.

---

80.5 The Final Capstone: Assembly, Demonstration and Review (30 min)

The capstone deliverable

The curriculum's final capstone links the Prefect-orchestrated pipeline to interactive visualisation layers. Your platform now delivers each row of the capstone table:

Component	Technology	Capstone deliverable	Where you built it
Extract & Load	Python, dlt, DuckDB	automated ingestion of BoC, FRED and StatCan series	Modules 1–3; WAP and release watcher (Days 48–50, 59)
Transform & Test	dbt, Great Expectations	validated macro marts (yield spreads, real rates)	Modules 4–5 (macro_monthly, fct_open_economy_daily, …)
Model & Forecast	DoubleML, XGBoost, Chronos	causal models (rate hikes → housing; tariff design) and next-quarter GDP forecasts	Modules 7–8
Orchestration & visualisation	Prefect, Plotly	scheduled, event-driven, monitored pipeline with dashboards	Module 6; Days 60, 66, 70, 80

CAPSTONE.md: the final report

Write it at the repository root, as the entry point for anyone discovering your project:

# Canadian Macro Decision-Intelligence Platform — Capstone

## 1. What it does (one paragraph + the architecture diagram)
## 2. How to run it (setup, environments, one-command pipeline, Prefect deployments)
## 3. Data: sources, lineage, quality (links: dbt docs, Data Docs, DATA_QUALITY_PLAN.md)
## 4. Forecasts: CPI and GDP — method, backtest, live track record (MODEL_CARD.md)
## 5. Causal analysis: rate hikes and housing starts — results and caveats (causal/MEMO.md)
## 6. Dashboards: pages and what each answers
## 7. Operations: schedules, alerts, runbook, backups, readiness review (PRODUCTION_READINESS.md)
## 8. Limitations and next steps (including the tariff-impact design)

The live demonstration (15 minutes per student or team)

1. Trigger a refresh from the Prefect UI and narrate the flow: ingestion per source → WAP audit → targeted dbt build → GX gates → forecasts → dashboards.
2. Show a failure being handled: run one drill (Day 54) or chaos scenario (Day 50) and show the state, alert and unchanged production data.
3. Open the dashboards: the macro cockpit, the forecast page (with explanations and track record), and the policy-and-housing page (with caveats).
4. Answer one economic question live using the platform, e.g. "What happened to the CA–US 2-year spread during the last tightening cycle, and what did our models forecast at the time?"

Review rubric

Dimension	Excellent looks like
Reproducibility	a fresh Codespace can rebuild everything from the README; versions pinned; no secrets in Git
Data quality	contracts at the boundary and the marts; quality history and KPIs; documented incidents
Engineering	layered models, DRY code, tests, idempotent steps, clear state policy, observability
Forecasting rigour	leak-free backtests, benchmarks, significance tests, calibrated intervals, live track record
Causal rigour	explicit DAG, pre-registration, diagnostics, robustness, sensitivity, honest limitations
Communication	dashboards with dates and caveats; memo and model card a non-specialist can follow

Use the Day 60 production-readiness checklist, extended with the forecasting model card and the causal memo, for peer review: each reviewer asks for evidence behind every claim, as on Day 60.

---

80.6 Course Wrap-Up (18 min)

What you built in 80 days

Days 1–10   Python, pandas, REST APIs           → reliable extraction from BoC, FRED, StatCan
Days 11–20  DuckDB and analytical SQL           → a layered warehouse; growth rates and moving averages in SQL
Days 21–30  dlt                                 → declarative, incremental, contract-protected, resilient ingestion
Days 31–40  dbt                                 → modular, tested, documented marts; the open-economy mart
Days 41–50  Great Expectations                  → data contracts, Data Docs, Write–Audit–Publish for StatCan releases
Days 51–60  Prefect                             → scheduled, event-driven, observable orchestration; dashboards
Days 61–70  XGBoost, SHAP, Chronos              → honest CPI and GDP forecasting, explained and monitored
Days 71–80  DoubleML                            → credible causal estimates of monetary policy on housing

The habits that matter more than the tools

Tools change; the course's recurring principles don't:

1. Write down the contract before the code: quality plans, state policies, feature specs, design specs, pre-registration.
2. Never trust; verify: regression tests at every migration, chaos and failure drills, Monte Carlo coverage, placebo tests.
3. Respect time: publication lags, vintages, leak-free evaluation, pre-treatment controls.
4. Keep provenance: every number traceable to its data, code and run.
5. Say what you don't know: intervals, caveats, limitations, sensitivity, out-of-range warnings.

Where to go next

Direction	Starting point
production deployment	always-on infrastructure, managed Prefect, a hosted DuckDB-compatible warehouse (Day 56)
real-time econometrics	use the vintages and snapshots you've been collecting for fully real-time evaluation (Days 27, 37, 68)
richer macro models	Bayesian VARs, dynamic factor models for nowcasting, mixed-frequency models
more causal designs	the tariff DiD (80.4); policy shocks from high-frequency market data; regional panels
your own question	add a series (one config line, one seed row), extend the DAG, and run the playbook

A last word. You started with print("Hello, macroeconomics!"). You end with a platform that can tell a policymaker, every morning and with stated confidence, what the data say, what the models expect, and what the evidence suggests a policy change would do. That combination of economic judgement, statistical honesty and engineering discipline is what applied decision intelligence means.

Day 80 checklist

• Causal estimates are published with provenance, modelled in dbt, tested, and exposed
• A monthly Prefect deployment re-estimates the design without blocking the daily refresh
• The "Monetary policy & housing" dashboard shows estimates, intervals, placebo status and caveats
• A tariff-impact causal design specification is written, with threats and pre-registered checks
• CAPSTONE.md documents the platform end to end; the live demonstration and peer review are complete
• I can explain how every layer, from APIs to causal estimates, fits together

git add .
git commit -m "Day 80: causal results in the platform, dashboard, tariff design, final capstone"
git push

🎓 Congratulations: course complete.