Thursday, February 12, 2015

Information equilibrium paper (draft) (introduction and outline)

Since I apparently can't seem to sit down and write anything that isn't on a blog, I thought I'd create a few posts that I will edit in real time (feel free to comment) until I can copy and past them into a document to put on the arXiv and/or submit to the economics e-journal (another work trip to the middle of nowhere provided some time in the evenings and H/T to Todd Zorick for helping to motivate me).

WARNING: DRAFT: This post may be updated without any indications of changes. It will be continuously considered a draft.


Information equilibrium as an economic principle

  1. Introduction: Information theory, mathematical models of economics
  2. Basic information equilibrium model: Derive the equations [link]
  3. Supply and demand: Derive supply and demand, ideal and non-ideal, elasticity of demand [link, link]
  4. Other ways to look at the equation: generalization of Fisher, long run neutrality, transfer from the future to the present [link, link, link]
  5. Macroeconomics: The price level, changing kappa, liquidity trap, hyperinflation solutions, labor market (Okun's law), ISLM model (talk about P* model)
  6. Statistical mechanics: Partition function approach, economic "entropy" and temperature
  7. Entropic forces: Nominal rigidity, liquidity trap (no microeconomic representation)
  8. Conclusions: A new way to look at economics, does not invalidate microeconomics and re-derives some known results from macro, speculate about maximum entropy principle for selecting which Arrow-Debreu equilibrium is realized among the many


In the natural sciences, complex non-linear systems composed of large numbers of smaller subunits, provide an opportunity to apply the tools of statistical mechanics and information theory. Lee Smolin suggested a new discipline of statistical economics to study of the collective behavior of economies composed of large numbers of economic agents.

A serious impasse to this approach is the lack of well-defined or even definable constraints enabling the use of Lagrange multipliers, partition functions and the machinery of statistical mechanics for systems away from equilibrium or non-physical systems. The latter -- in particular economic systems -- lack e.g. fundamental conservation laws like the conservation of energy to form the basis of these constraints.

Lee Smolin, Time and symmetry in models of economic markets arXiv:0902.4274v1 [q-fin.GN] 25 Feb 2009

In order to address this impasse, Peter Fielitz and Guenter Borchardt introduced the concept of natural information equilibrium. They produced a framework based on information equilibrium and showed it was applicable to several physical systems. The present paper seeks to apply that framework to economic systems.

Peter Fielitz and Guenter Borchardt, "A general concept of natural information equilibrium:
from the ideal gas law to the K-Trumpler effect" arXiv:0905.0610v4 [physics.gen-ph] 22 Jul 2014

The idea of applying mathematical frameworks to economic systems is an old one; even the idea of applying principles from thermodynamics is an old one.  Willard Gibbs -- who coined the term "statistical mechanics" -- supervised Irving Fisher's thesis in which he applied rigorous approach to economic equilibrium.

Mathematical models of economics: Fisher, Samuelson
Fisher, Irving. Mathematical Investigations in the Theory of Value and Prices (1892)
Fisher, Irving. The Purchasing Power of Money: Its Determination and Relation to Credit, Interest, and Crises. (1911a, 1922, 2nd ed)
  • quantity theory of money, equation of exchange
Samuelson, Paul. Foundations of Economic Analysis (1947)
  • Introduces Lagrange multipliers for economics
  • Le Chatelier's principle (general partial return to equilibrium)
  • Also cited Gibbs
The specific thrust of Fielitz and Borchardt's paper is that it looks at how far you can go with the maximum entropy or information theoretic arguments without having to specify constraints. This refers to partition function constraints optimized with the use of Lagrange multipliers. In thermodynamics language it's a little more intuitive: basically the information transfer model allows you to look at thermodynamic systems without having defined a temperature (Lagrange multiplier) and without having the related constraint (that the system observables have some fixed value, i.e. equilibrium).

Samuelson: meaningful theorems: maximization of economic agents = equilibrium conditions. This was a hypothesis from Samuelson. We don't need the equilibrium conditions (constraints), so we don't need to make this assumption, nor do we start from any notion of utility.

Samuelson didn't think thermodynamics could help out much more than he had shown:
There is really nothing more pathetic than to have an economist or a retired engineer try to force analogies between the concepts of physics and the concepts of economics. How many dreary papers have I had to referee in which the author is looking for something that corresponds to entropy or to one or another form of energy.
We hope this paper is neither pathetic nor dreary, however we do derive a quantity that corresponds to an economic entropy (actually, entropy production) of an economy that goes as $\Delta S \sim \log N!$ where $N$ is nominal output in section 6.

A word of caution before proceeding; the term "information" is somewhat overloaded across various technical fields. Our use of the word information differs from its more typical usage in economics, such as in information economics or e.g. perfect information in game theory. Instead of focusing on a board position in chess, we are assuming all possible board positions (even potentially some impossible ones such as those including three kings). The definition of information we use is the definition required when specifying a random chess board out of all possible chess positions, and it comes from Hartley and Shannon. It is a quantity measured in bits (or nats), and has a direct connection to probability.

This is in contrast to e.g. Akerlof information asymmetry where knowledge of the quality of a vehicle is better known to the seller than the buyer. We can see that this is a different use of the term information -- how many bits this quality score requires to store is irrelevant to Akerlof's argument. The perfect information in a chess board $C$ represents $I(C) \lt 64 \log_{2} 13 \simeq 237$ bits; this quantity is irrelevant in a game theory analysis of chess.

Akerlof, George A. (1970). "The Market for 'Lemons': Quality Uncertainty and the Market Mechanism". Quarterly Journal of Economics (The MIT Press) 84 (3): 488–500.Hartley, R.V.L., "Transmission of Information", Bell System Technical Journal, Volume 7, Number 3, pp. 535–563, (July 1928).
Claude E. Shannon: A Mathematical Theory of Communication, Bell System Technical Journal, Vol. 27, pp. 379–423, 623–656, 1948. 

We propose the idea that information equilibrium should be used as a guiding principle in economics and organize this paper as follows. We will begin by introducing and deriving the primary equations of the information equilibrium framework, and proceed to show how the information equilibrium framework can bee understood in terms of the general market forces of supply and demand. This framework will also provide a definition of the regime where market forces fail to reach equilibrium through information loss.

Since the framework itself is agnostic about the goods and services sold or the behaviors of the relevant economic agents, the generalization from widgets in a single market to an economy composed of a large number of markets is straightforward. We will describe macroeconomics, and demonstrate the effectiveness of the principle of information equilibrium empirically. In particular we will address the price level and the labor market where we show that information equilibrium leads to well-known stylized facts in economics. The quantity theory of money will be shown to be an approximation to information equilibrium when inflation is high, and Okun's law will be shown to follow from information equilibrium.

Lastly, we establish an economic partition function, define a concept of economic entropy and discuss how nominal rigidity and so-called liquidity traps may be best understood as entropic forces for which there are no microfoundations.

Okun, Arthur, M, Potential GNP, its measurement and significance (1962)


  1. I will include the technical details of the numerical work in an appendix

  2. I think the part I was really avoiding was digging up all the references ...

  3. Yes it is all a terrible pain, going through all the hoops you need to in order to write a true scientific report. However, it is essential, and doing so will help you clarify and solidify your thoughts- not to mention having brilliant peer reviewers pick apart your arguments. Keep up the good work!!

    1. Ha! I've written several papers (e.g. here) -- it's just that there doesn't seem to be a centralized database or Bibtex repositories for citing economics papers as far as I know :)

  4. One more thing, I would like to suggest that one way you could prove the relative utility of ITM would be to point out how it seems to do a much better job of explaining the (perhaps nonexistent) Phillips curve than anything else that has been proposed. You did a series of blog posts about the Phillips curve, basically debunking it, and then (or perhaps before) showing how ITM could explain the inflation/unemployment relationship much better. I think that would be a nice experimental verification of the likely utility of ITM, in that it can actually bring something new to economics that other approaches can't. And all with just a few degrees of freedom...

    1. I'd considered that -- however one of the problems with addressing the Phillips curve is that it requires addressing why empirical unemployment has large fluctuations relative to the model and would involve addressing non-ideal information transfer. I basically decided to address non-ideal information transfer (and things like the KL divergence) with prices falling below the ideal price and other aspects to a separate paper.

      Likewise, I also wanted to leave the discussion of interest rates (and similar non-ideal information transfer along with more about the liquidity trap and the whole post about the effects that move interest rates) to another paper.

      I also wanted to keep the number of things that go against fundamental principles held by large swaths of economists to a minimum for the first paper :)

    2. Understandable. However, paraphrasing Machiavelli, you can never avoid war, only delay it or hasten it to your advantage or disadvantage. And make no mistake my friend, you are on the warpath... (in a good way).

    3. And here comes the counter-attack...

    4. What's funny is that Noah seems to forget his own problems he has with macro ... Or for some reason he seems to think they are objections only trained experts could have. In fact, pretty much anyone with a basic grasp of math in a technical field could come up with those same objections: the HP filter, the lack of informativeness of macro data to support complex models, Atherya's assumptions economists "like" -- these are not highly technical economic objections.