Statistics Canada · Notes from the code
How the MCP queries Statistics Canada
Statistics Canada does not have one API; it has several. This is how MapleStats MCP finds a table, turns a coordinate into a vector, slices a big table with SDMX and weights a microdata file, and what the live services taught me along the way.
Many services, one way in
Start with the list. The Web Data Service (WDS) serves tables and time series as JSON. An SDMX REST API serves the same tables as SDMX-ML. Reference Data as a Service (RDaaS) holds classifications such as NAICS. The 2021 Census Profile has its own SDMX API on another host, the 2016 profile has a separate JSON API, and the profiles from 2001 to 2016 exist as bulk downloads. Public use microdata files (PUMFs) are ZIP archives. Then come The Daily's Atom feeds, the Data catalogue, the survey directory, a census geography service and the indicator feeds behind StatCan's home page.
Each has its own identifiers, its own error habits and its own idea of what a request body looks like. An agent should not have to learn all of that before it can answer a question about prices. MapleStats wraps these services in 61 tools, and the agent does not see those as a list either. The server shows a client three tools, plan_query, search_tools and call_tool. Every StatCan tool is found by a plain-language query to search_tools and run through call_tool, so the first request of a StatCan question is not a StatCan request at all:
search_tools{
"query": "search StatCan tables"
}[
"wds_list_all_cubes",
"wds_search_cubes",
"statcan_census_tables_search",
"statcan_reference_search_data",
"ab_economic_list_tables"
]The ranking comes from the server's own BM25 index over the tools' names and docstrings, the same index the search on this site uses. wds_search_cubes is among the results, and that is where a table hunt starts.
From a question to a table
Every StatCan table has a product id, the PID. It is the table number without its dashes: table 18-10-0004-01 is PID 18100004, because the last two digits, a view of the table, are optional. The docs://statcan/addressing resource the server ships gives the anatomy of the full ten digits: two for the subject, two for the product type, four for the sequence and two for the view.
wds_search_cubes turns words into a PID, and it is less clever than it sounds. It downloads getAllCubesListLite, the list of every table in WDS, keeps it cached for an hour, and looks for the query as a substring of each English and French title. Here is the search for the table behind the CPI chart on the case studies page:
call_tool{
"name": "wds_search_cubes",
"arguments": {"query": "consumer price index", "limit": 5}
}{
"cubes": [
// 1 more
{
"product_id": 18100004,
"cansim_id": "326-0020",
"cube_title_en": "Consumer Price Index, monthly, not seasonally adjusted",
"release_time": "2026-09-14T12:30:00Z"
},
{
"product_id": 18100005,
"cansim_id": "326-0021",
"cube_title_en": "Consumer Price Index, annual average, not seasonally adjusted",
"release_time": "2026-01-19T13:30:00Z"
}
// 2 more
],
"total_count": 5,
"provenance": {
"url": "https://www150.statcan.gc.ca/t1/wds/rest/getAllCubesListLite",
"coverage": "top 5 matches of 8271 cubes searched"
// + 8 more fields
}
}A substring match is blunt. "consumer price index" finds the CPI tables because their titles say so; a question worded differently from StatCan's titles can miss them, and then the agent has to try again in StatCan's words. The provenance says how much was searched, and each result carries cansim_id, the table's number in the old CANSIM system, next to release_time, the table's latest release.
Coordinates and vectors
A StatCan table is a cube. Each dimension has a tree of members, and one member from each dimension picks out one series. wds_get_cube_metadata returns the dimensions. The CPI table has two: Geography, with 30 members, and Products and product groups, with 359.
call_tool{
"name": "wds_get_cube_metadata",
"arguments": {"product_id": 18100004}
}{
"product_id": 18100004,
"cube_title_en": "Consumer Price Index, monthly, not seasonally adjusted",
"n_series": 2139,
"dimensions": [
{
"dimension_position_id": 1,
"dimension_name_en": "Geography",
"members": [
{"member_id": 2, "parent_member_id": null, "member_name_en": "Canada"},
{"member_id": 3, "parent_member_id": 2, "member_name_en": "Newfoundland and Labrador"}
// 28 more members
]
},
{
"dimension_position_id": 2,
"dimension_name_en": "Products and product groups",
"members": [
{"member_id": 2, "parent_member_id": null, "member_name_en": "All-items"},
{"member_id": 3, "parent_member_id": 2, "member_name_en": "Food"}
// 357 more members
]
}
]
// + 13 more fields
}Member ids repeat across dimensions: in both, member 2 is the root of the tree, the one with no parent (Canada, and All-items). And not every combination exists. 30 geographies times 359 products would make 10,770 series; the table has 2,139. A coordinate is not a free choice, then: it has to name a series StatCan publishes.
A coordinate writes one member id per dimension, in dimension order, separated by dots. WDS always wants exactly ten positions, with zeros for the dimensions a table does not have. The client pads it, so asking for 2.2 sends 2.2.0.0.0.0.0.0.0.0. What comes back is the vector: a stable id for one series, the "V" number carried over from CANSIM.
call_tool{
"name": "wds_get_series_info_from_cube_pid_coord",
"arguments": {"product_id": 18100004, "coordinate": "2.2"}
}{
"product_id": 18100004,
"coordinate": "2.2.0.0.0.0.0.0.0.0",
"vector_id": 41690973,
"provenance": {
"url": "https://www150.statcan.gc.ca/t1/wds/rest/getSeriesInfoFromCubePidCoord"
// + 9 more fields
}
}With the vector, the data is one more call. This one is the request behind the CPI chart on the case studies page, the latest 84 months of v41690973:
call_tool{
"name": "wds_get_data_from_vectors",
"arguments": {"vector_ids": [41690973], "latest_n": 84}
}[
{
"product_id": 18100004,
"coordinate": "2.2.0.0.0.0.0.0.0.0",
"vector_id": 41690973,
"observations": [
// 83 earlier months
{
"ref_period": "2026-08-01",
"value": 169.8,
"decimals": 1,
"scalar_factor_code": 0,
"symbol_code": 0,
"status_code": 0,
"security_level_code": 0,
"release_time": "2026-09-14T08:30:00Z"
}
],
"provenance": {
"url": "https://www150.statcan.gc.ca/t1/wds/rest/getDataFromVectorsAndLatestNPeriods"
// + 9 more fields
}
}
]A value never travels alone. Each observation carries the codes that say how to read it:
scalar_factor_code- The power of ten the value is expressed in: 0 for units, 3 for thousands, 6 for millions. For the CPI it is 0.
decimals- The number of decimals StatCan publishes. The value already carries StatCan's rounding.
symbol_code,status_code- The symbols and status flags StatCan attaches to a value, as codes.
wds_get_code_setsdecodes them, along with scalar factors, frequencies and units of measure. release_time- The release timestamp WDS gives the data point. It is not always the first publication: August 2026 carries 14 September 2026, but September 2019, the oldest month in this recording, carries 15 September 2021.
Then the rule I care most about: the server never applies the scalar factor. value is exactly what WDS sent. The code says so twice, in the schema's docstring ("deliberately NOT applied ... WDS never auto-applies it either") and first on the list in docs://statcan/gotchas. A value in thousands stays in thousands, with its code beside it. Scaling is one line, apply_scalar_factor(value, code) multiplies by ten to the power of the code, but that line belongs to whoever uses the number, where it can be seen. What the tool returns matches what WDS returns, digit for digit.
When WDS is not enough: SDMX
WDS thinks in series: give it vectors or coordinates and it returns their observations. When a question covers a whole slice of a big table, every detailed geography or every occupation, going series by series means finding every coordinate first, and downloading the full table means taking far more than the question needs. StatCan's SDMX API slices on the server instead. A key names the members you want, dimension by dimension, and a blank position is a wildcard.
The key is the coordinate without its padding: one position per non-time dimension. sdmx_get_vector_data builds it from a vector by itself. It looks up the vector's coordinate through WDS, reads the table's SDMX structure to count its dimensions, and cuts the coordinate to that length, so v41690973 becomes the key 2.2:
call_tool{
"name": "sdmx_get_vector_data",
"arguments": {"vector_id": 41690973, "last_n_observations": 3}
}{
"dataflow_id": "DF_18100004",
"key": "2.2",
"series": [
{
"series_key": {
"Geography": "2",
"Products_and_product_groups": "2"
},
"vector_id": 41690973,
"scalar_factor": 0,
"decimals": 1,
"dguid": "2016A000011124",
"uom_code": "17",
"observations": [
{"period": "2026-06", "value": 169.0},
{"period": "2026-07", "value": 169.9},
{"period": "2026-08", "value": 169.8}
]
}
],
"row_count": 3,
"provenance": {
"url": "https://www150.statcan.gc.ca/t1/wds/sdmx/statcan/rest/data/DF_18100004/2.2"
// + 9 more fields
}
}Three things about this API are written into the client.
It answers in XML
Ask for JSON with format=jsondata or an Accept header, and StatCan's SDMX endpoint still returns SDMX-ML, for data and structure alike. The constants file records this as confirmed live, against benchmark documentation that assumed SDMX-JSON, so the client parses the XML itself, with defusedxml rather than the standard library parser.
Wildcards sample big dimensions
Leave a dimension with more than about 30 codes blank and the answer is a sparse, unpredictable sample of its codes, not all of them. sdmx_get_key_for_dimension builds the complete version instead. It reads the dimension's codelist from the structure, keeps the leaf codes (those that are no other code's parent) and joins them with + into an explicit OR key, to splice into the key at that position.
One combination is refused
lastNObservations together with startPeriod or endPeriod gets HTTP 406. The client refuses that combination before sending anything, and if a 406 comes back anyway, the error names the usual cause instead of passing the bare status through.
Microdata, weights and replicates
Tables are aggregates. A public use microdata file is the records themselves, one row per respondent, with weights that make the sample stand for the population. StatCan ships PUMFs as ZIPs, and the codebooks inside come in several formats: CSV codebooks, Stata .dct and .do files, SPSS label files and SAS files.
The table APIs cannot see PUMFs at all. The one place to discover them is StatCan's Data catalogue, which statcan_reference_search_data searches. Once a file is found, statcan_pumf_get_codebook reads its codebook from inside the ZIP with HTTP range requests, so the variables and weights are known before the data file is downloaded. statcan_pumf_tabulate then does the arithmetic on the server, with DuckDB. Its first call downloads the file into a local cache, and later calls reuse it.
This is the call behind the bachelor's degree chart on the case studies page: the 2021 Census individuals file, adults aged 25 to 64 (age groups 9 to 16), and the share holding each highest credential in each province, with the codes for "not available" and "not applicable" filtered out.
call_tool{
"name": "statcan_pumf_tabulate",
"arguments": {
"url": "https://www150.statcan.gc.ca/n1/pub/98m0001x/2023001/cen21_ind_98m0001x_part_rec21.zip",
"rows": [
"PR",
"HDGREE"
],
"statistic": "share",
"filters": {
"AGEGRP": ["9", "10", "11", "12", "13", "14", "15", "16"],
"HDGREE": ["1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13"]
}
}
}{
"data_file": "data_donnees_2021_ind_v2.csv",
"statistic": "share",
"weight": "WEIGHT",
"unweighted_n": 525504,
"weighted_total": 19463104.06541252,
"cells": [
{
"groups": [
{"variable": "PR", "code": "10", "label": "newfoundland and labrador"},
{"variable": "HDGREE", "code": "9", "label": "bachelor's degree"}
],
"estimate": 13.305555555555555,
"standard_error": 0.42801216439502804,
"cv": 0.032167928848060565,
"unweighted_n": 958,
"low_count": false
}
// 142 more cells
]
// + url, value_variable, filters, truncated, variance_method, notes, provenance
}| Province | estimate | standard_error | cv | unweighted_n | low_count |
|---|---|---|---|---|---|
| Newfoundland and Labrador | 13.31 | 0.43 | 0.032 | 958 | false |
| Prince Edward Island | 18.15 | 1.00 | 0.055 | 366 | false |
| Nova Scotia | 20.08 | 0.34 | 0.017 | 2,721 | false |
| New Brunswick | 16.67 | 0.33 | 0.020 | 1,779 | false |
| Quebec | 18.15 | 0.11 | 0.006 | 21,670 | false |
Two things decide whether a table like that is right. The first is the weight. The tool defaults to the codebook's main weight, WEIGHT here, and it checks labels and field widths as well as names, because names mislead: in the CSWC, WTQ_05 is a question about working at night, and in the CCHS, DOHWT is a height-and-weight inclusion flag. Real weights are wide numeric fields, so a known width under four columns rules a variable out. The second is the sample under each cell. Every cell carries its unweighted count, and by default cells with fewer than 30 respondents are flagged low_count.
Then the standard errors. The Census file carries 16 replicate weights, WT1 to WT16. The tool computes the estimate once with the main weight and once with each replicate, sums the squared deviations of the 16 replicate estimates from their mean, divides by 35 and takes the square root. The 35 is not a typo. It is the user guide's recipe, a Fay adjustment over 16 groups, and the comment above it in tabulate.py shows the arithmetic: (240/35) × (1/240). StatCan notes that this method overestimates the error for small estimates.
Each method is copied from its survey's user guide, with the section cited, and a file gets one only after its guide has been read, "never by analogy with another survey", in the words of the comment above the list. 3 files have one so far:
| File | Weight | Replicates | Standard error |
|---|---|---|---|
| 2021 Census, individuals file (98M0001X) | WEIGHT | WT1–WT16 16 | Random groups: squared deviations from the replicates' mean, divided by 35 |
| Employment Insurance Coverage Survey, 2024 (89M0025X) | WTPM | WRPM1–WRPM1000 1,000 | Bootstrap: squared deviations from the full-sample estimate, divided by 1,000 |
| CSWC, 2024-2025 (14-25-0001) | CSWCWT | BSW1–BSW1000 1,000 | Bootstrap: squared deviations from the full-sample estimate, divided by 1,000 |
For any other PUMF the result gives no standard error, says why, and names the replicate weights or the bootstrap file it found. The bootstrap weights in PUMFs are perturbed for confidentiality, so their standard errors are comparable to StatCan's official ones, not the same. Two case studies use the 2021 Census file this way: bachelor's degrees by province and low income across immigrant generations.
Things that bit us
Most of these were found by calling the live services, not by reading documentation. The code records how each one was confirmed, next to the fix, so that a later edit does not quietly undo it.
shared/http.py
The connection that hung
Plain httpx connections to statcan.gc.ca timed out, silently. The cause is not in this code: something on StatCan's network path, a WAF or CDN, blocks TLS handshakes whose ALPN extension offers only http/1.1, which is exactly what httpx offers by default. The diagnosis reproduced the hang with a raw ssl socket narrowed to that one value and watched it go away once h2 was back on the list. The fix is one argument, http2=True. That argument now has a comment asking the next person not to delete it, h2 is a pinned dependency because of it, and the Python scripts reproduce_code writes set it too.
statcan/wds/client.py
The overnight lock
From midnight to 8:30 a.m. Eastern, while StatCan updates its data, some WDS methods answer HTTP 409. That is a schedule, not a fault, and no retry beats a schedule, so the shared retry layer leaves 409 out of the statuses it retries (429, 500, 502, 503 and 504). The WDS and SDMX clients turn it into a DataLocked error that says to try again after 8:30.
statcan/wds/client.py
One status, three meanings
WDS answers a well-formed request for something that does not exist, such as an unknown productId, with HTTP 406, not 404. It answers 406 when a date is too short, too: reference-period ranges need full YYYY-MM-DD dates, and release ranges need YYYY-MM-DDTHH:MM. And getBulkVectorDataByRange wants its body as one flat object where every other WDS POST method takes a list; send it a list and that is a 406 as well. The client turns a WDS 406 into an InvalidInput error that asks the agent to check its identifiers.
shared/json_utils.py
Null is not an empty list
dict.get(key, []) uses its default only when the key is missing. Some WDS cubes send surveyCode and subjectCode as an explicit null instead, and a null where the schema expects a list fails validation. list_or_empty(obj, key) is obj.get(key) or [], which covers both cases, and it is now the rule for every list-typed field pulled from an external API. The numeric codes on observations get the same treatment for the same reason: int(None) raises, so decimals and the other codes are read with or 0.
statcan/reference/client.py
The search that ignored its keyword
StatCan's Reference, Analysis and Data catalogues share one Drupal search. Sent a keyword cold, it returns every document in the catalogue, unfiltered, unless the session has first visited the base page and carries the cookie that visit sets. The proof was a raw curl with a cookie jar: the same URL and query string, a different result depending only on whether the base page came first. So the client warms up each catalogue and language once. Sessions expire, though, and an expired one fails the same quiet way, so every response is checked. If the page's own search box comes back empty despite a keyword, the client warms up again and retries once, and it raises rather than return the whole catalogue as a match.
AGENTS.md
Two of thirty-two
The first version of the StatCan module called 2 of its 32 tools against the real API before it was called done. The others passed tests written against hand-made fixtures. A later pass called all 32 live and found 9 more bugs: wrong field names for 4 of the 10 getCodeSets categories, footnotes assumed to be strings that are really objects, two WDS methods that need a differently shaped request body, two with the opposite date requirements from the ones assumed, an RDaaS endpoint returning a list where a dict was expected, and 404 and 406 responses escaping as raw HTTP errors instead of typed ones. The fixtures could not catch any of it, because they were written from the same wrong assumptions as the code. The rule since then: before a client is done, a throwaway script calls every function it exports against the real API, and the test suite fails for any module without a live smoke test.
From a call to a script
An agent's answer is only as good as the check someone can run without the agent. reproduce_code takes a tool name and its arguments and writes an R, Python, Stata or Julia script that fetches the same data straight from StatCan. What the script does depends on the call:
| Call | Script |
|---|---|
A wds_ or sdmx_ call with a product_id | Downloads the full table as CSV, to filter to the rows the tool returned; in R, cansim's get_cansim(). |
wds_get_data_from_vectors | Sends the same WDS request for those vectors; in R, cansim's get_cansim_vector(). |
sdmx_get_vector_data | Fetches the same vector through WDS, which serves it as JSON. |
A statcan_pumf_ call with a ZIP url | Downloads the same ZIP, with notes on the weight variable and on reading fixed-width files with the Stata, SPSS or SAS files inside it. The weighted table itself is computed by the server. |
statcan_census_tables_get_downloads | 2016: the full-table CSV, and in R the Beyond 20/20 file read with canivt. 2006 and 2011: R only, with canivt. |
statcan_indicators_get_indicators | Downloads the same feed and repeats the tool's filters. |
| The Daily, and the documents and analysis catalogues | No script: these return documents, not data. |
| Any other StatCan tool | The tool runs once while its upstream requests are recorded, and the script repeats the data request exactly. |
For the CPI call above, the R script fetches the data with one cansim call, and the Python script repeats the WDS request, http2=True included. Neither breaks the scalar rule: both keep value as WDS sent it and put the scaled number in a column of its own, val_norm from cansim in R and value_normalized in Python. From the recorded scripts:
Python
with httpx.Client(
http2=True,
follow_redirects=True,
timeout=300,
headers={"User-Agent": "Mozilla/5.0 (compatible; research script)"},
) as client:
response = client.post('https://www150.statcan.gc.ca/t1/wds/rest/getDataFromVectorsAndLatestNPeriods', json=[{'vectorId': 41690973, 'latestN': 84}])
response.raise_for_status()
raw_path.write_bytes(response.content)
# …
# scalarFactorCode is the value's power of ten; refPer is the reference date.
data = data.with_columns(
(pl.col("value") * 10 ** pl.col("scalarFactorCode")).alias("value_normalized"),
pl.col("refPer").str.to_date(strict=False).alias("ref_date"),
)R
data <- get_cansim_vector(c("v41690973"))
# …
# cansim adds val_norm (the value times its scalar factor) and a Date column.
data <- data |>
filter(!is.na(val_norm))Everything else, briefly
Classifications. RDaaS holds NAICS, the Standard Geographical Classification and the rest: structure, category trees, index terms, exclusions, and concordances that map codes from one version to the next. It has one gap worth knowing. Asked for the category tree of the current NAICS, 2022.1.0, it answers with an empty body, while every older version tried returns the full tree. The tool says so and points to the 2017-to-2022 concordance, whose target codes are the current NAICS codes. Index terms switch to French only through an Accept-Language header, which the client sends.
The census. The 2021 Census Profile is an SDMX API on its own host: find a geography, from a province down to a dissemination area, and one of 2,631 characteristics, then fetch the values. It too picks its language from an Accept-Language header rather than a parameter. The 2016 profile has a separate JSON API, and the 2001 to 2016 profiles are CSV or TAB bulk downloads, which the archive tools resolve to direct links. Census geography comes from an ArcGIS REST service that returns boundaries and DGUIDs, and that answers every error with HTTP 200 and an error object inside.
Releases. The Daily comes from its Atom feeds, the last 100 days by subject, and from the JSON file behind the release calendar, which goes back to 14 March 2012. wds_get_changed_cube_list lists the tables that changed on a date, and statcan_delta_get_file_link finds the bulk-update ZIP for one business day.
The full list of families, with each one's prefix and number of tools, counted from the registry when this page was built:
Tables and time series
- Web Data Service: tables and vectors
wds_16 - SDMX API: filtered series
sdmx_4 - Delta files: daily bulk updates
statcan_delta_1
Census
- 2021 Census Profile
statcan_census_profile_3 - 2016 Census Profile
statcan_census_profile_2016_2 - Census Profile archive, 2001-2016
statcan_census_profile_archive_2 - Census data tables, 2006-2016
statcan_census_tables_2 - Census geography
statcan_geo_3
Microdata
- Public use microdata files (PUMF)
statcan_pumf_5
Classifications
- Classifications and concordances
rdaas_12
Indicators
- Indicators
statcan_indicators_1 - Sustainable Development Goals
statcan_sdg_3
Releases, catalogues and methods
- The Daily
statcan_daily_2 - Definitions, methods and analysis
statcan_reference_3 - Surveys and metadata
statcan_surveys_2
Where to go next
- All 61 Statistics Canada toolsEvery tool's parameters, keywords and an example call.
- The case studiesThe CPI and Census calls on this page as charts, with the calls behind each one.
- Connect MapleStats to your agentTwo steps, no account or API key.