Mark Thoma links us to this blog post that tries to make a point about economic forecasting:
Okay, so people make fun of the bad forecasts of economists. Fair enough. But economists are trying to forecast actions of sentient, prospectively focused creatures.
Hurricanes are just big unruly piles of wind. But we can't forecast THOSE either. Check out this "ensemble" forecast for the next week .... [see the blog]
As I wrote in a comment, hurricanes are part of a system where the laws governing the micro-states are known and the macro system is known to be nonlinear and chaotic because of those microfoundations. I had two points:
- The laws governing the micro-states of economics are not known
- There cannot have been a derivation of nonlinear or chaotic behavior from unknown microfoundations
The first is directly assumed in the quote: the macroeconomy is not known to be due to the "actions of sentient, prospectively focused creatures". That would exclude emergent properties based on, say, measures of economic entropy (e.g. here). On a fundamental level, of course an economy is made up of people like an ideal gas is made of atoms. But individual atoms don't have a temperature or entropy -- these are emergent concepts. This is the problem with "obvious" complexity ... in the economic problem it is assumed.
[Update + 1.5 hr: Actually, the weather model uses the emergent concept of temperature. I am not aware of any emergent concepts from micro that are used in macro forecasts ... ]
Additionally, while hurricanes are unpredictable, climate predictions are fairly stable -- I would put predicting the path of a hurricane on par with predicting the growth of a single industry in economics. The forecasts we make fun of in economics (of future NGDP or inflation) are akin to climate forecasts, not weather forecasts.
And to really bring it home, those different hurricane paths can be used to construct a meaningful probability distribution of landfall locations. In an economic model, you can't do that. It's just the probability of NGDP = X assuming the model is correct. Symbolically, we have (via Bayes' theorem)
P(location | weather model) ~ P(location) f(P(weather model))
where f(P(weather model)) is close to 1 versus
P(NGDP = X | econ model) ~ P(NGDP = X) f(P(econ model))
where f(P(econ model)) is not close to 1. In fact, f(P(econ model)) is probably quite large as P(econ model) is likely small.
This is a good time to bring up my head-to-head with the Fed (both the FOMC predictions and the DSGE model from the NY Fed) I started almost exactly a year ago in September of 2014. Both of these projections assume complexity. The information equilibrium model (IE, aka the information transfer, IT model) does not. In fact, a constant inflation model would do almost as well as the IT model (in the third graph, the green line).
Actually, the IT model does as well as using a smoothed version of the data as the model (dashed gray line)! It's not quite 1-sigma separation from the NY Fed DSGE model, but it's getting close.
Funny thing is I was just thinking about Bayesian statistics the last couple of days, particularly with regard to natural experiments in Economics. I am referring to the devaluation of real-world evidence, and the need for "several more decades of data". I guess it's true if you fit everything to a linear regression and try to estimate parameters. Am I wrong or should Economists be using Bayesian statistics more frequently (pun intended)?
ReplyDeletehttp://informationtransfereconomics.blogspot.com/2015/09/in-case-of-deletion.html
I'm actually in the midst of writing a post that started from the idea that if the specific statistical approach matters, then you probably haven't found anything.
DeleteBut Bayesian vs frequentist is an argument over the interpretation of probabilities, not the mathematics behind them. It's very much like the different interpretations of quantum mechanics (which is rooted in the exact same question). You can calculate probability amplitudes using QM, but what they "mean" depends on Copenhagen, many-worlds, transactional ... etc interpretation.
Every economist -- not many, I am afraid -- whom I have heard express themselves on the matter are Bayesians. I don't know about the statistics they use in publications, though.
DeleteBut the differences between frequentist and Bayesian statistics are material. (Jason is right about the mathematics of probability; they remain unchanged. However, there are differences in the application of probability to data, not just the interpretation.)
As for the paucity of macroeconomic data, that is a real world problem, I think, not a problem of statistical methods. Cultural, legal, and economic institutions keep changing. I think that those changes are more important than most economists appear to, and that I think Jason does.
Institutions (and other things) keep changing, yet e.g. Interest rates seem well explained by some simple mathematical relationships:
Deletehttp://informationtransfereconomics.blogspot.com/2014/09/the-us-economy-1798-to-present.html
That makes me think institutions are more a perturbation on the trend ...
"In fact, f(P(econ model)) is probably quite large"
ReplyDeleteMeaning it's probably greater than 1?
Yes. The function f is actually
Deletef = P(B|A)/P(B)
From Bayes' theorem. But P(B) << 1 so f could be >> 1.