tag:blogger.com,1999:blog-6837159629100463303.post6055038269482791873..comments2023-06-18T01:25:08.748-07:00Comments on Information Transfer Economics: Overfitting and empirical data: qualitative and qualitative modelsJason Smithhttp://www.blogger.com/profile/12680061127040420047noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-6837159629100463303.post-15547010807030272412016-03-18T09:01:45.213-07:002016-03-18T09:01:45.213-07:00Hi, John.
What do you mean by overfitting? The t...Hi, John. <br /><br />What do you mean by overfitting? The tweaking or the testing? Or both?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6837159629100463303.post-60201537622962847202016-03-17T06:33:09.935-07:002016-03-17T06:33:09.935-07:00Bill,
You literally just described overfitting......Bill,<br /><br />You literally just described overfitting...John Handleyhttps://www.blogger.com/profile/16057855086740377031noreply@blogger.comtag:blogger.com,1999:blog-6837159629100463303.post-74720251374978935922016-03-17T06:24:31.579-07:002016-03-17T06:24:31.579-07:00Is overfitting standard in economics? I have heard...Is overfitting standard in economics? I have heard more than one economist complain that competing models can be tweaked (my word) to fit currently available data, so that new data are insufficient to eliminate them. Pardon me, but that's not the way to do it, is it? <br /><br />Let's assume that WWII was a watershed economically, after which things have been different from before. Suppose that we wish to compare two different economic models. Let's take 1950 as our first year, to allow for things to settle down post war. Then shouldn't we do something like this? Fit the models to the data for the 44 years up to 1994, and then compare them using the data for the 22 years since. Isn't that a simple way to address overfitting?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6837159629100463303.post-58512012022596846922016-03-16T20:30:49.246-07:002016-03-16T20:30:49.246-07:00Nice diagrams!
Re: modeler's compass: overfit...Nice diagrams!<br /><br />Re: modeler's compass: overfitting (prior to more data being collected) would initially move to the left I'd guess... and also with an upward component too it seems, no? But those "gains" would be reversed with more data... thus the seduction of being lured into overfitting in the 1st place.<br /><br />Also, isn't is possible to sometimes move DOWN and to the right or left? Wasn't Copernicus' concentric circle planetary orbits a case of moving down and to the right (more error than Ptolemaic epi-cycles, but also less complicated)? By the time Kepler added ellipses, it moved back up and to the left, but not as much up as to the left. That might be an interesting trajectory on the diagram.Tom Brownhttps://www.blogger.com/profile/17654184190478330946noreply@blogger.comtag:blogger.com,1999:blog-6837159629100463303.post-69436216968956174832016-03-16T20:28:07.277-07:002016-03-16T20:28:07.277-07:00"John makes the case that it is the latter: D..."John makes the case that it is the latter: DSGE models are qualitative models. I don't buy this. For one, they are way too complex to be a qualitative model. Qualitative models above the Silver criterion basically overfit the data and are useless. They're not really telling you anything besides parroting back your priors."<br /><br />So, RBC + monopolistic competition (which we basically know exists, even if the dixit-stiglitz model is a generalization) + sticky prices is too complex? I'm not sure I agree, especially since the model described above (nix capital) can be written in reduced form as three equations, at which point it basically becomes dynamic IS-LM + rational expectations. I'd definitely call that qualitative.<br /><br />Does anyone actually think that DSGE is structural? I hope not. IMHO, the whole point of having utility maximization and budget constraints is so that the model is internally consistent and nothing else. This is why I prefer DSGE to something like IS-LM or AD-AS (both of which I think you would agree are qualitative); I couldn't care less about Smets-Wouters (e.g.), since I find the idea that it is somehow structural laughable. Same goes for the NY Fed's DSGE. <br /><br />Does adding financial frictions make (super complicated and useless) DSGE models empirically successful? Maybe, but I think this falls under the point I make at the end of my post:<br /><br />"That said, the appropriate conclusion is probably that all economic models are doomed to be quantitatively unsuccessful (and even if they are, that is not necessarily an indication that the model in question is correct in the sense that it correctly matches causes with effects), even if they do capture some of the correct cause and effect relationships"<br /><br />That is, there is basically no way to tell if a model is structural given the lack of closed system, even if it is empirically accurate.<br /><br />Regarding QE, I completely agree, QE was a great experiment that proved that Market Monetarism relies entirely on unobservales and is thus unfalsifiable. Or, more seriously, that the qualitative Keynesian prediction that liquidity traps exist is correct. (Still, I much prefer any internally consistent NK model that has money demand to IS-LM, not only because of the internal consistency, but also because of the explicit modelling of AS as long as you don't cheat with the reduced form).John Handleyhttps://www.blogger.com/profile/16057855086740377031noreply@blogger.com