With an unbiased prior, market failure is the most likely outcome

Noah Smith has a short article at Bloomberg View about how free market proponents make a play for the null hypothesis; he convincingly argues that maybe economists should be more amenable to not having to prove market failure.

I'd like to point out that using the information transfer framework while assuming we know nothing about the situation, market failure should be our default hypothesis. That's because generally in a market where A is exchanged for B, we can at best assume the information entropy of the distribution of A is a bound on the information about that distribution received at B. Symbolically:

I(A) ≥ I(B)

The intuition here is that if you send a signal about a change [0] in A to which B should respond (e.g. to clear excess supply or demand that results), at best B can receive all of the information in the signal. Given noise, irrational human actions, or sudden drops in the information content of the distributions (e.g. everyone wants to sell the same asset so that it takes less information to describe the aggregate ‒ everyone wants to sell ‒ than to describe in a typical case ‒ Alice wants to buy, Bob wants to sell, Carol wants to sell, David wants to sell, Edna wants to buy, etc [1]).

If we were completely ignorant about what is happening, then we'd expect

I(A) > I(B)

most of the time. The math can still be useful here; the resulting differential equation [2] for information equilibrium (ideal information transfer) ‒ i.e. I(A) = I(B) ‒ becomes a bound for I(A) > I(B). However, we really have to accept we know a lot less than we might think.

In a sense, in order to say that markets are working, you first have to show they are working [3]. This will involve empirical tests; my favorite are predictions like this one:

Predictions are my favorite because it's the one way you can be absolutely sure you have data you haven't trained your model on is when that data comes from the future. Actually, long term interest rates are one of the more ideal markets out there ‒ at least so far. Exchange rates (forex markets) might be one of the worst.

There are a lot more ways a market can fail than it can work. The more surprising fact is that there are working markets. When some people say that governments are responsible for functioning markets, this is one of the reasons I'm inclined to believe it.

...

Footnotes

[0] You can actually derive the information equilibrium condition by assuming an infinitesimally small signal dA and the infinitesimal response dB.

[1] Think of this as the sequence 00000... vs 10001... so that the second one requires more information to specify. A constant sequence (a light that's always on or off) carries no information.

[2] The interesting part is that this differential equation was almost written down by Irving Fisher in 1892.

[3] The micro- and macro-economic theory I've been developing on this blog actually grew out of an attempt to understand whether prediction markets were working for a vastly different application. The answer seems to be as stated above: probably not.