Cesar Hidalgo is looking into information theory with his new book. I haven't read it; however I have read Diane Coyle's review (H/T Mike Norman) and it seems like it might be a great source of analogies for this blog. This one immediately struck me:

Hidalgo makes the same point as the final chapter of [Diane Coyle's] GDP book, that in adding things up in terms of their monetary value we are not capturing the value of diversity: three spoons are not as valuable as a knife, fork and spoon.

That is actually the exact point of this post here. We'll almost exactly. If we have

*M*dollars to spend on knives, forks and spoons, (assuming they equally cost one Euro [1] WOLOG) then all consumption possibilities (blue points) are located under the budget constraint hyperplane in the first (left side) figure:
The maximum entropy point is given by the black dot in the second (right side) figure. Now for three items, that point is actually at

*{M/4, M/4, M/4}*so if you had 4 €, you'd have 1 fork, 1 knife and 1 spoon at the maximum entropy point (and 1 € left over).
The entropy maximum is related to equilibrium in the information transfer model as well as effectively ideal markets. Under certain (ideal) conditions, it recovers all of the properties of maximizing utility.

I would disagree with the analogy of crystallizing imagination Hidalgo uses -- that is a lower entropy state for one thing. The key idea we want to capture is Jaynes'

*dither*. We want to make the world safe for people to move about the space of possibilities -- to try and fail.
PS This also maximizes entropy.

**Update +15 minutes:**
Since the maximum entropy state (3/4, 3/4, 3/4) with budget constraint M = 3 is not realizable given quantized knives, forks and spoons, you'd actually have a combination of the states (1, 1, 1), (1, 1, 0), (1, 0, 1) and (0, 1, 1) realized among a quarter of the population each.

**Footnotes:**

[1] I'm using Euros (€) here because the dollar symbol messes with mathjax.

How exactly do you compute the maximum entropy solution for a given budget constraint? Would it be possible, for example, to take the budget constraint from a stochastic neoclassical growth model and solve for the maximum entropy distribution of capital, consumption, and labor?

ReplyDeleteThat would be possible in general, but not sure how to do it without reference to a specific model.

DeleteThe maximum entropy solution in the system above is simply the average state if every state is equally likely -- the center of mass of the polytope.

https://en.wikipedia.org/wiki/Principle_of_indifference

The simplest version of the budget constraint I have in mind is:

Deleteb_t+1 + c = (1+r)*b + w*n

basically, there's a consumption good, an asset, and income comes from the product of wages and labor. The model is usually solved with a utility function like U = ln(c) - B*ln(n) and profit maximizing firms with a production function y = a*f(n).

The consumer takes wages (w) and the real interest rate (r) as given and then chooses optimal consumption and labor supply, maximizing utility.

so, given the constraint b_t+1 + c = (1+r)*b + w*n, how do you find the maximum entropy solution for c and n?

I'm pretty sure the solution should be pretty similar to a simple M = p1*c1 + p2*c2, but what math do you use to come to that solution?

Hi John,

DeleteSorry this comment fell through the cracks (by the date I see I was the first day of a conference I traveled to for work).

For the maximum entropy solution when I looked at two- or three-period models, I found it was best to look at n-period models where n >> 1 like I do here:

http://informationtransfereconomics.blogspot.com/2015/04/diamond-dybvig-as-maximum-entropy-model.html

Your budget constraint would then look at a period toward the end of the timeline (analogous to t+1) and one toward the beginning (analogous to t). In general, you would allow all possible values of c and n in your budget constraint and look at the centroid of the triangle with c on one axis and n on the other.

So, if I were to take the budget constraint above and add production technology y = a*n, would it be possible to find maximum entropy "first order conditions" for all the endogenous variables in order to have a maximum entropy version of what is normally a utility/profit maximization model? If so, would I be able to set everything up like utility maximization (aka solve a Lagrangian to solve for FOCs)?

DeleteHi John,

DeleteYes, it should be. I actually compare utility maximizing vs entropy maximizing solutions here:

http://informationtransfereconomics.blogspot.com/2015/03/utility-in-information-equilibrium-model.html

The more straightforward way to do all this is to set up a partition function as it 'automatically' maximizes entropy ... but in economics we don't necessarily know the form of the Lagrangian that you'd use to build the partition function.