Applied Decision Intelligence & Data Architecture
An 80-day technical curriculum that turns economists into full-stack data architects.
Every day is one page with a video lesson, a timed 2.5-hour plan, the outcomes you should reach, and the full written lesson with step-by-step code. You work in GitHub Codespaces on real Canadian data from the Bank of Canada, Statistics Canada and FRED, and finish with a production pipeline, forecasts and causal estimates.
Video lessons by module
One video per day. Each module has ten.
How to use this site
- Pick a module tab to see what it covers, its tools and its ten days.
- Open a day to watch the video, follow the timed plan and work through the written lesson. Each day takes about two and a half hours.
- Videos are being recorded. Until a day's video is published, its page shows a placeholder; the written lesson is already complete.
- Mark days complete to track your progress. It's saved in this browser.
Part I: Data Engineering Foundations
Days 1–30Python for Macroeconomics
Core programming, data wrangling, and API interaction.
- Days 1–3Syntax, control flow, functions, and virtual environments (.venv).
- Days 4–7Data manipulation with pandas, handling time-series indexing.
- Days 8–10Fetching indicators from the Bank of Canada, FRED and Statistics Canada via REST APIs.
Analytical SQL with DuckDB
In-process analytical database architecture and query optimization.
- Days 11–13DuckDB CLI, relational algebra, and loading external CSV/Parquet files.
- Days 14–17Advanced SQL, window functions, and Common Table Expressions (CTEs).
- Days 18–20Staging CPI and GDP; year-over-year changes and moving averages directly in SQL.
Data Ingestion with dlt
Declarative data pipelines and automated extraction.
- Days 21–23dlt pipelines, sources vs. resources, and secrets management.
- Days 24–27Schema evolution, flattening nested JSON, and incremental loading.
- Days 28–30Resilient extractors for the BoC Valet API and StatCan endpoints, loading into DuckDB.
Part II: Transformation, Validation & Orchestration
Days 31–60Data Modeling with dbt
Transforming staged data into production-ready analytical marts.
- Days 31–33Project initialization, materializations (views/tables), and basic Jinja templating.
- Days 34–37Modular modeling (staging, intermediate, marts), DRY principles, and dbt macros.
- Days 38–40A macro data mart joining yield curves with USD/CAD: an open-economy dashboard schema.
Quality Assurance with Great Expectations
Contract-driven data engineering and anomaly detection.
- Days 41–43Data quality principles, initializing contexts, and connecting to DuckDB sources.
- Days 44–47Expectation suites, profiling datasets, and automated Data Docs.
- Days 48–50Validating incoming StatCan data: unemployment rates within 0–100%, continuous dates.
Pipeline Orchestration with Prefect
Workflow scheduling, dependency management, and error handling.
- Days 51–53Flow and task decorators, local orchestration, and state management.
- Days 54–57Retries, caching, scheduling, and logging.
- Days 58–60Orchestrating the daily macro-dashboard refresh: dlt → DuckDB → dbt → GX.
Part III: Advanced Analytics & Capstone
Days 61–80Forecasting with XGBoost & Chronos
Predictive modeling on macroeconomic time-series data.
- Days 61–63Supervised learning theory, tree-based models, and univariate forecasting.
- Days 64–67Hyperparameter tuning, SHAP explainability, and zero-shot forecasting with Chronos.
- Days 68–70Forecasting Canadian CPI and real GDP growth with features engineered earlier.
Causal Inference with DoubleML
Isolating causal impacts from observational economic data.
- Days 71–73Causal inference theory, confounding variables, and Directed Acyclic Graphs (DAGs).
- Days 74–77Nuisance parameters, cross-fitting, and partially linear regression models.
- Days 78–80The causal impact of BoC overnight-rate hikes on housing starts, controlling for the business cycle.
Final capstone
The course ends with one integrated project: a Prefect-orchestrated pipeline linked to interactive dashboards, built on the architecture of the course's macro dashboard notebook.
| Component | Technology | Capstone deliverable |
|---|---|---|
| Extract & Load | Python, dlt, DuckDB | Automated ingestion of BoC, FRED and StatCan series. |
| Transform & Test | dbt, Great Expectations | Validated macro marts (yield spreads, real rates). |
| Model & Forecast | DoubleML, XGBoost, Chronos | Causal impact models and next-quarter GDP forecasts. |
| Orchestrate & Visualise | Prefect, Plotly | A scheduled, monitored pipeline feeding interactive dashboards. |