Look-ahead bias in a macro regime backtest: real-time vs revised data, 2011–2026

Latest observation

Summary

The replay at a glance

Why a fixed rule looks free of hindsight

A rule-based regime classifier looks like the easy case for a backtest. Its thresholds are fixed in code, its inputs are official series, and its history can be recomputed for any month. Eco3min’s code reads five monthly inputs: the computed 3-month average of the Chicago Fed National Activity Index (CFNAI) for growth, the real-time Sahm rule as a recession gate, an early-warning rule on the insured unemployment rate, the Dallas Fed trimmed mean PCE inflation rate over 12 months for prices, and the Chicago Fed’s National Financial Conditions Index for an overlay. The classification methodology sets out the grid and the current regime page shows its latest output.

Two of those inputs are built to limit hindsight. The Sahm rule series that FRED publishes as SAHMREALTIME is computed from unemployment rates as they stood at the time, and FRED notes that the underlying rate is revised only with the annual update of seasonal factors in January. The trimmed mean drops, each month, the price components with the most extreme changes: 24% of the expenditure weight at the low end and 31% at the high end, in the Dallas Fed’s description. A timeline recomputed on today’s data is therefore easy to read as the timeline the code would have shown at the time, give or take a release.

The literature on real-time data has long warned against that reading for national accounts. Croushore and Stark (2001) built a real-time data set for macroeconomists so that models could be tested on the data available at each date, in quarterly vintages, and Orphanides and van Norden (2002) found that end-of-sample estimates of the output gap are revised by amounts of the same order as the gap itself. The record of US jobs data revisions shows the same mechanism on payrolls. This replay applies the idea to a narrower object: the label a published rule would have put on each month.

Three ways the gap arises

At each date the code was run three times. The revised timeline is the published one, computed on today’s data. The second run uses the same final data, cut to the observations already published on the date of the run, so it differs from the revised timeline only through release lags. The real-time run uses each series in its ALFRED vintage of that date: what the code would have read that day. Every month where the real-time run differs from the revised timeline falls in exactly one of three cases.

At month-end, in 32 of the 37 months the cut final data give the same regime as the revised timeline and the real-time run differs from both: the vintage effect alone. In 5 months the cut final data already differ and the real-time run agrees with them: the release lag alone. In 0 months do both apply at once. In 3 further months, February 2018, November 2025 and December 2025, the lag moves the cut final data away from the revised timeline but the vintages of the day bring the real-time run back onto it.

Why the labels differ: month-end gaps by cause and by axis, June 2011 to September 2026

Months where the real-time run differs from the revised timeline, by cause for the regime and by axis; the financial conditions overlay is shown for comparison and enters no count of regime gaps.

Sources: Federal Reserve Bank of St. Louis, ALFRED vintages, retrieved 16 October 2026; Federal Reserve Banks of Chicago and Dallas; Eco3min regime classifier v1.1.0. Chart: Eco3min Research.

How to read it: the first bar is the regime, the one the card displays; the next two are the axes behind it. An axis can differ without changing the regime, because the grid gives the Transition / Mixed signals label to two cells (growth neutral with inflation neutral or low): inflation states differ in 29 month-ends, but only 19 of the 37 regime gaps involve inflation. Among the 37, 18 involve growth alone, 14 inflation alone and 5 both. The decisive input is the computed CFNAI average in 22 of the 23 growth gaps and the Sahm gate in 1; it is trimmed mean PCE in all 29 inflation gaps.

Most gaps are a single step. The grid crosses three growth states with three inflation states; 32 of the 37 month-end gaps differ on one axis by one step, 5 on both axes by one step each, and none jumps two steps on an axis, from expansion to contraction for instance. 15 of the 37 sit on one boundary, between Balanced Expansion and Transition / Mixed signals.

Jumpier in real time

Over 184 month-ends the real-time run changes label 42 times, against 29 for the final data cut at publication dates and 19 for the revised timeline. 18 of the real-time changes are one-month flips, reversed the following month; 12 of those flips fall in months where the real-time run differs from the revised timeline, and 25 of the 37 gaps lie outside any flip. The revised timeline has no one-month flip; the cut final data have 7.

Part of the extra movement comes from the code itself. At a month-end run the latest month has no activity or inflation reading yet, so the code carries forward the latest available value: a lag of one month in 178 of the 184 runs for the activity index and in 149 for trimmed mean PCE, and all 184 month-end runs classify the latest month on at least one carried value. The two-month confirmation then sees the same reading twice, so a single new print across a threshold confirms a change of state at once. Final data cut at publication dates share that property, which is why they also move more than the revised timeline, 29 changes against 19. The replay measures code v1.1.0 as it runs; a version without carry-forward would be a different code, outside this study.

Regime changes caught in real time

The replay also shows where the real-time read matched. Of the 19 regime changes in the revised timeline, the real-time run made 16: 8 in the same month, 1 two months early (August 2017) and 7 late, by one to six months. At month-end it never made 2 of them, the entry into Stagflation in July 2024 and the return to Inflationary Pressure in September 2024 (the daily replay shows the first for 28 days in August), and in 1 case, November 2025, it was already in the new regime.

The Covid recession is the clearest case. The revised timeline moves to Slowdown in April 2020 and to Balanced Expansion in August 2020, and the real-time run made both moves in the same months. At the end of April 2020 it read a computed CFNAI average of −1.47 for March, against −7.56 for April in today’s data: well past the −0.50 threshold either way, while the real-time Sahm rule still read 0.30. In 2020 to 2022 the inflation axis never differs between the two runs: the move to Overheating in December 2021 came in the same month in real time, on a trimmed mean PCE reading of 2.81% against 3.23% in today’s data.

Gaps of that size between first print and final figure matter little when readings are far from the thresholds; they decide the label when readings sit next to them. The payroll revisions at turning points follow the same pattern on the jobs report.

Concentration: one threshold, two episodes

The share is not stable through time. At month-end the real-time run differs from the revised timeline in 17 of 103 months in 2011 to 2019 (16.5%), 5 of 36 in 2020 to 2022 (13.9%) and 15 of 45 since January 2023 (33.3%). The 15 recent months are two episodes, July to December 2024 and February to October 2025, and 14 of them involve the inflation axis.

The 2025 episode sits on the 2.75% threshold of trimmed mean PCE inflation. In today’s data the rate the code reads for February to October 2025 stays between 2.60% and 2.85%, and the revised timeline holds Inflationary Pressure for the nine months. In the month-end vintages the code read between 2.45% and 2.75%, never above the line, and in September and October 2025 exactly on it, which the rule does not count as above. The real-time run shows Transition / Mixed signals instead, in 8 of the 9 months, and Balanced Expansion in March 2025.

Moving the threshold shows the dependence. With the inflation threshold at 2.50% or at 3.00%, month-end gaps fall to 25 of 184 (13.6%); moving each of the code’s thresholds one at a time gives a range of 13.6% to 24.5% (table in the methodology). The window holds one recession, 19 regime changes and 17 episodes of gap: 12 last a single month, three last three or four months and account for 10 months, and the two episodes of 2024 and 2025 account for 15. The effective sample is small, and a block bootstrap puts the 95% interval of the month-end share at 11.4% to 30.4% with blocks of 6 months and 11.4% to 32.6% with blocks of 12. The test of five recession rules faces the same constraint of few episodes.

Compare the two runs, month by month

Pick a month to see what the code would have shown at the time and what changed since: the regime of each run, the state of each axis, the financial conditions overlay and the input values the code read, on the revised and the real-time side.

The module reads the published CSV directly. If it does not load, the image shown in its place is the static version of the two timelines.

Summer 2024, day by day

The revised timeline shows Stagflation in July and August 2024, the only months with that label in the window. The code’s rule for it is growth negative and inflation high; the label is that rule’s output, not a diagnosis of the US economy. Run at each month-end on the vintages of the day, the code never shows it: Inflationary Pressure at the end of July, Slowdown at the end of August, Transition / Mixed signals from September. Run on each date a regime input received a new vintage or a month began, it shows Stagflation for 28 days, from 2 to 29 August 2024. The date of the run is part of the result.

• Up to 1 August 2024: Inflationary Pressure. The Sahm rule read 0.43 on June data and trimmed mean PCE 2.82% for June, above the 2.75% line.
• 2 August: the July jobs report lifts the Sahm rule to 0.53, across 0.50. In v1.1.0 a crossing forces the growth axis negative at once, and the code shows Stagflation.
• 22 August: the July activity index comes out; the computed average, −0.07, stays in the neutral band and Stagflation holds.
• 30 August: July trimmed mean PCE is first published at 2.70%, under the line. The code carries the July reading into August, counts it twice and confirms the inflation axis at neutral: Slowdown, for two days.
• 1 September: the month changes. September is classified on the July activity reading carried forward, a second neutral reading that confirms the growth axis back to neutral while the Sahm rule still reads 0.53: Transition / Mixed signals. The August jobs report of 6 September lifts the Sahm rule to 0.57 without a new crossing, and the label stays.

At the end of July 2024 the July jobs report was not out yet: the code read June’s 0.43 where the revised timeline uses July’s 0.53. The Sahm series is not revised; the difference is the month of observation. The inflation side is a revision: 2.70% at first print for July, 2.88% in today’s data. The history of Sahm rule signals places the 2024 crossing among the rule’s other triggers.

Summer 2024: the inputs at first release against today’s data

Trimmed mean PCE inflation over 12 months and the real-time Sahm rule as read by the code at each month-end run, May to December 2024, against the revised timeline; the code’s thresholds at 2.75% and 0.50.

Sources: Federal Reserve Bank of St. Louis, ALFRED vintages of PCETRIM12M159SFRBDAL and SAHMREALTIME, retrieved 16 October 2026; Federal Reserve Bank of Dallas; Eco3min regime classifier v1.1.0. Chart: Eco3min Research.

Market outcomes, excluded by design

This study reports no market return, drawdown or strategy result by regime. It measures what a published indicator would have displayed with the data of the time, not what anyone could have earned with it, and the distribution tables that other Eco3min studies carry are left out for that reason.

Levels to watch

The replay by period, pair and run day

Each row counts month-end runs where the real-time run differs from the revised timeline, with the months where release lags alone already move the cut final data, and the months where each axis differs.

Which regime the real-time run showed instead, for the 37 month-end gaps:

The same comparison by run calendar:

The month-end and mid-month snapshots overlap only in part: 23 months differ at both runs, 14 at the month-end run only and 24 at the mid-month run only, and the real-time label itself differs between the two runs of the same month in 41 months. In the daily replay, 61 of the 184 months show a regime gap on at least one day, 48 a growth-axis gap and 34 an inflation-axis gap.

Historical turning points

November 2013 to February 2014, four months. The revised timeline moves from Transition / Mixed signals to Balanced Expansion in November 2013, when the computed CFNAI average reads 0.14 in today’s data, above +0.10. The real-time run read 0.06 at the end of November and stayed at Transition. From December 2013 to February 2014 it showed Disinflationary Expansion instead: the readings of trimmed mean PCE the code had put inflation between 1.35% and 1.38%, under the 1.50% floor, where the revised timeline reads 1.57% to 1.65%. The real-time run reached Balanced Expansion in May 2014, six months after the revised timeline.

April and August 2020, the Covid recession. Both runs move to Slowdown in April 2020 and to Balanced Expansion in August 2020. The real-time activity reading at the end of April was −1.47, against −7.56 in today’s data, past the threshold all the same; in August the real-time run read 3.58 and today’s data 3.41.

June 2022, a growth gap at high inflation. The revised timeline moves from Overheating to Inflationary Pressure on a computed CFNAI average of −0.22. The real-time run read 0.20 and stayed at Overheating, then followed one month later. Inflation was far above the line in both runs, 4.52% in today’s data and 3.96% in the vintage.

February to October 2025, the longest episode. Nine months of Inflationary Pressure in the revised timeline against Transition / Mixed signals or Balanced Expansion in real time, all on the 2.75% line of trimmed mean PCE (see Concentration). The episode ends in November 2025, when the revised timeline itself moves to Transition; the real-time run was already there. Summer 2024 has its own section above.

September 2026, the current observation. Both runs show Transition / Mixed signals on a computed CFNAI average of 0.01 and trimmed mean PCE of 2.19% for August 2026, with no gap at any month-end of 2026.

Gaps in the vintages

Release delays are kept as they were. A run where an input is more than three months old on the run date is excluded, the limit beyond which the code finds no value; it is never filled. ALFRED shows eight breaks of more than 45 days in the vintages of the regime inputs: one around each of the federal government shutdowns of October 2013 and of December 2018 to January 2019, five during or after the shutdown of October to November 2025, and one in the CFNAI between November 2022 and January 2023, outside any shutdown.

The autumn 2025 breaks are the only ones that exclude runs: 49 days of the daily replay in 12 periods between 1 December 2025 and 28 January 2026, and the mid-month runs of December 2025 and January 2026. Every month-end run stays classified, 184 of 184.

Methodology

All inputs come from ALFRED, the archive of vintages kept by the Federal Reserve Bank of St. Louis, through the official FRED API, retrieved on 16 October 2026: CFNAI and NFCI from the Federal Reserve Bank of Chicago, PCETRIM12M159SFRBDAL from the Federal Reserve Bank of Dallas, SAHMREALTIME from the Federal Reserve Bank of St. Louis on the BLS unemployment rate, and IURSA from the US Department of Labor. Eco3min publishes the current series in its CFNAI dataset, its Sahm rule dataset and its financial conditions index dataset. The code is Eco3min’s regime classifier at THRESHOLDS_VERSION 1.1.0, imported unchanged; the revised timeline is the regime_history.csv file served on 16 October 2026. The column named cfnai_ma3 is the code’s own 3-month average of the published index, not the Chicago Fed’s CFNAI-MA3.

Sensitivity. Each threshold of the code moved one at a time, the revised timeline recomputed on final data with the same setting, month-end runs:

Uncertainty and controls. A block bootstrap on the 184 month-end flags, 10,000 draws with seed 20261022, gives a 95% interval of 11.4% to 30.4% with blocks of 6 months and 11.4% to 32.6% with blocks of 12. Run on final data of 16 October 2026, the code reproduces the served timeline with 0 differences over 586 months on the regime, both axes and the overlay. Eighteen values taken from the issuers themselves, the Chicago Fed, the Dallas Fed and the BLS, for 2024 and 2026 match the ALFRED vintages at the issuer’s printed precision, 17 exactly and 1 within the rounding of the 3-month average described above.

Coverage. ALFRED holds no vintage of SAHMREALTIME before September 2019: for 99 months, June 2011 to August 2019, the real-time run takes the Sahm rule from the cut final data, on the first-release dates of the BLS unemployment rate, consistent with a series FRED documents as built in real time. The code applies the insured-unemployment rule only from May 2026, a live-only setting, so the replay reads IURSA vintages from April 2026 and the cut final data before; the rule can act in 5 months. The financial conditions overlay differs in 54 month-ends, 39 of them with no change of regime; it is left out of every count of regime gaps.

Literature. Croushore and Stark (2001) set out the case for testing models on the data available at each date and built the Philadelphia Fed’s real-time data set, in quarterly vintages, for it; Orphanides and van Norden (2002) separate, for the output gap, the part of real-time error due to data revisions from the part due to estimation at the end of the sample, and find the second the larger; Croushore (2011) surveys the field. The window contains one recession dated by the NBER, in 2020 (FRED series USREC); the accuracy of recession indicators is covered in a separate question page.

The files. One row per month in the monthly file, one row per run period in the daily file:

Reproducibility. In Python, on the published files:

Sources and references

• Federal Reserve Bank of St. Louis, ALFRED, vintages of CFNAI, PCETRIM12M159SFRBDAL, SAHMREALTIME, IURSA and NFCI, retrieved on 16 October 2026 through the FRED API: alfred.stlouisfed.org.
• Federal Reserve Bank of Chicago, Chicago Fed National Activity Index and National Financial Conditions Index: chicagofed.org.
• Federal Reserve Bank of Dallas, Trimmed Mean PCE Inflation Rate: dallasfed.org/research/pce.
• Sahm, C., “Direct Stimulus Payments to Individuals”, in Recession Ready: Fiscal Policies to Stabilize the American Economy, The Hamilton Project and Washington Center for Equitable Growth, 2019; series SAHMREALTIME on FRED.
• US Department of Labor, Employment and Training Administration, insured unemployment rate (IURSA), via FRED.
• Croushore, D. and Stark, T., “A Real-Time Data Set for Macroeconomists”, Journal of Econometrics, 105(1), 2001, pp. 111–130.
• Orphanides, A. and van Norden, S., “The Unreliability of Output-Gap Estimates in Real Time”, Review of Economics and Statistics, 84(4), 2002, pp. 569–583.
• Croushore, D., “Frontiers of Real-Time Data Analysis”, Journal of Economic Literature, 49(1), 2011, pp. 72–100.

Limitations

• An in-sample rule. The thresholds, the Sahm gate and the confirmation were chosen on data that include this window; the replay holds the rule fixed and says nothing of its behaviour out of sample.
• One classifier, one window. One recession, 19 regime changes and 17 episodes of gap from June 2011 to September 2026. The counts hold for this rule on this window only.
• Uneven vintage coverage. No Sahm vintage before September 2019 (99 months on first-release dates instead), insured-unemployment vintages read only from April 2026, the code’s rule acting from May 2026; the financial conditions overlay is left out of the counts.
• The run calendar. Month-end, mid-month and daily runs give 20.1%, 25.8% and 24.1%. The live code runs daily, not necessarily on the replay’s dates, and the replay does not reconstruct what the regime card displayed after May 2026.
• A moving reference. The revised timeline is computed on data still being revised, the latest months most of all; counts for 2025 and 2026 can change with the next vintages.
• A property of the code. Carrying the latest value forward speeds up the two-month confirmation, and the 3-month activity average is computed on the index rounded to two decimals. Both are kept as the code runs; a different version would give different counts.

Frequently asked questions