Validating forecasts: unemployment rate edition

Unemployment rate data was released on Friday, and the points continue to follow the dynamic information equilibrium model (DIEM) forecast (the model is described in my paper). I tweeted on July 31 that the result would be "4.0 ± 0.2 % (90% c.l.)" according to the DIEM which turned out to be correct (3.9%), but like for the May data (released in June) it could easily have been outside that confidence limit. At the time I didn't put too much weight on it, and you shouldn't put too much weight on this single point either. In fact, Noah Smith wrote a needed article on commentary (including his own example) that puts too much weight on the latest single data points. That's one reason why I track these forecasts over extended periods of time. Here's the latest data compared to the DIEM (in gray, all post-forecast data is in black, pre-forecast data is blue) as well as several vintages of forecasts (red) from the Federal Reserve Board of San Francisco (FRBSF) released in their FedViews publications:

However, since I've been tracking this forecast for awhile now, we can probably say with confidence that the FRBSF forecast with December 2016 data from 12 January 2017 (red) was outperformed by the DIEM of comparable vintage made 18 January 2017 (gray). Even if the data was inside the (unknown) error bands of the FRBSF model, those error bands would have to be larger than the error bands of the DIEM meaning it was a measurable improvement (two predictions that predict the same thing with one being more precise — as long as it isn't a single data point — implies the more precise one is the better model). Here's that forecast on its own:

...

Update 7 August 2018

One of the great things about the St. Louis Fed FRED data portal is that it has a related site for vintage time series called ALFRED (for ArchivaL FRED). We can see how the dynamic information equilibrium model would have performed using data available at the time (pre-revisions). Using the data from early 2016, I can show the apples-to-apples performance of the DIEM against a Minneapolis Fed VAR model:

Using the old data nudges the model prediction down a bit, and also increases the 90% confidence bands a bit. I can use the same series to compare against the various vintages of the FRBSF forecast above: