Thursday, July 31, 2014

I do not think that calculation means what you think it means

It took me a minute to figure out exactly how Sumner came up with this result. I assume that Sumner meant 2013 Q4 as the ending point of his second period starting at 2014 Q4, so that wasn't the problem. The problem is that Sumner calculated the NGDP growth at some point between 2011 Q4 and 2012 Q4 and at another point between 2012 Q4 and 2013 Q4, not the average growth during those two period (or any other meaningful number).

See, Sumner took the end points of the time periods and calculated the % change between them. I show more digits to give some evidence that I did the same calculation Sumner did:

2011 Q4 to 2012 Q4: 3.46652% (Sumner rounds to 3.47%)
2012 Q4 to 2013 Q4: 4.56636% (Sumner rounds to 4.57%)

According to the mean value theorem, taking the slope between the endpoints of a curve only means the slope of some presumably continuous curve achieved that value somewhere in between those end points. It is not the average growth for the period (or really anything at all).

Of course, this isn't the way you should do this. NGDP is a number with some measurement error, so relying on only the two end points means that you massively increase the error in the derived quantity.

What is the (omitted) error on Sumner's measurement? I assumed a log-linear model of GDP from 2009 Q4 to 2014 Q2 and calculated the error bands. The result is plotted here:


If we use these error bars to predict the error on Sumner's measurement, we get

2011 Q4 to 2012 Q4: 3.8 ± 1.6%
2012 Q4 to 2013 Q4: 3.8 ± 1.6%

That isn't a typo. The two periods have exactly the same growth rate. The difference between the two periods is entirely measurement error.

We can try a different method: average the growth rate numbers. The period starting from 2009 Q4 also has an average growth rate of 3.8% by this method, however the error is larger at ± 1.9%. Here is a graph:


If you look at this last graph, you can start to see how misleading the numbers in Sumner's post are. I see almost constant growth with random fluctuations around it. If you average these numbers for Sumner's two periods, you end up with another inconclusive result:

2011 Q4 to 2012 Q4: 3.7 ± 1.3%
2012 Q4 to 2013 Q4: 3.9 ± 1.7%

There was no failed experiment and market monetarism didn't pass any test. It's all inconclusive.

PS Sumner's commenter Kailer Mullet is incorrect: the single decimal precision (nearest 0.1%) is exactly how much precision is warranted by these numbers. Adding another decimal place is pure noise and taking one away loses information.

3 comments:

  1. You see this one?

    http://macromarketmusings.blogspot.com/2014/07/revisiting-great-experiment-of-2013.html

    ReplyDelete
    Replies
    1. I somehow missed this comment ... so I get the benefit of 20/20 hindsight :) ... models matter, and after the revised data it looks like sequestration took a big chunk out of NGDP growth:

      http://informationtransfereconomics.blogspot.com/2014/08/prediction-update-not-bad-for-five.html

      Delete

Comments are welcome. Please see the Moderation and comment policy.

Also, try to avoid the use of dollar signs as they interfere with my setup of mathjax. I left it set up that way because I think this is funny for an economics blog. You can use € or £ instead.