I was asked by Nick Rowe to explain where the information comes in:
"If I trade an apple for a banana, there is supply and demand, but where does information come into it?"
My initial reply on the post was less clear than I'd like, so I'm going to try and do better here. First we'll start with a single good (apples).
Let's say there are d potential apple buyers, labelled 1, 2 ... $d$. Selling one apple to #42 uncovers $\log_{2} d$ bits of information (the number of bits required to describe the ID number). Selling a second apple to #42 uncovers another $\log_{2} d$ bits of information for a total of
$$
\log_{2} d + \log_{2} d = 2 \log_{2} d
$$
bits of information. Selling a third apple to #1005 uncovers another $\log_{2} d$ bits of information, and so on, until we've sold $n_{d}$ apples and uncovered $n_{d} \log_{2} d$ bits.
This uncovered information is transferred from the buyers (the demand) who ostensibly know how much they'd like to buy (at a given price) to the sellers (the supply) who only have some vague idea of the size of the apple market after they start to sell some apples (or do a little market research).
The optimal way to register this information would be to keep a log of each apple and each buyer's ID number. In that case, the information captured by the supply would be
$$
\text{(1) }I_{d} = n_{d} \log_{2} d
$$
However, the suppliers don't actually know the size of their market d so they actually capture
$$
\text{(2) } I_{s} = n_{s} \log_{2} s \leq n_{d} \log_{2} d
$$
bits of information where $s$ could be e.g. the sellers' estimate of $d$ [1]. We have $I_{s} \leq I_{d}$. In an ideal world, this would be equality. If only there were something that existed to help gauge the size of the market for a product allowing a seller to capture all the information ... (hint: it's called money).
Which reminds me, we haven't actually discussed what the buyer's are buying these apples with yet. We'll start with Nick's suggestion of using a banana to buy an apple. In that case the informaton the sellers collect (by acquiring a banana from the apple buyer) is
$$
n_{b} \log_{2} b = n_{s} \log_{2} s
$$
where $b$ is the number of potential banana buyers. We can use this to determine the "exchange rate" (the price of a banana in terms of apples). If we take the smallest unit of bananas to be $dB$ so that $B/dB = n_{b}$, then
$$
B \log_{2} b = S \frac{dB}{dS} \log_{2} s
$$
where $n_{s} = S/dS$ the number of apples supplied to the apple buyers. If we assumed the market forapples bananas is about the same size as the market for apples, we could say that $\log_{2} b \sim \log_{2} s \sim \log_{2} d$, so that:
$$
B \sim S \frac{dB}{dS} \leq n_{d}
$$
where $dB/dS$ (where we let $dS$ and $dB$ become infinitesimal) is the "exchange rate" (price) of apples to bananas. What if the apple buyer gave the apple seller something else instead of bananas? Well, starting with equation (2) you'd have
$$
n_{s} \log_{2} s \leq n_{d} \log_{2} d
$$
$$
\frac{S}{dS} \log_{2} s \leq \frac{D}{dD} \log_{2} d
$$
$$
\frac{dD}{dS} \leq \frac{\log_{2} d}{\log_{2} s} \frac{D}{S}
$$
Let's say there are d potential apple buyers, labelled 1, 2 ... $d$. Selling one apple to #42 uncovers $\log_{2} d$ bits of information (the number of bits required to describe the ID number). Selling a second apple to #42 uncovers another $\log_{2} d$ bits of information for a total of
$$
\log_{2} d + \log_{2} d = 2 \log_{2} d
$$
bits of information. Selling a third apple to #1005 uncovers another $\log_{2} d$ bits of information, and so on, until we've sold $n_{d}$ apples and uncovered $n_{d} \log_{2} d$ bits.
This uncovered information is transferred from the buyers (the demand) who ostensibly know how much they'd like to buy (at a given price) to the sellers (the supply) who only have some vague idea of the size of the apple market after they start to sell some apples (or do a little market research).
The optimal way to register this information would be to keep a log of each apple and each buyer's ID number. In that case, the information captured by the supply would be
$$
\text{(1) }I_{d} = n_{d} \log_{2} d
$$
However, the suppliers don't actually know the size of their market d so they actually capture
$$
\text{(2) } I_{s} = n_{s} \log_{2} s \leq n_{d} \log_{2} d
$$
bits of information where $s$ could be e.g. the sellers' estimate of $d$ [1]. We have $I_{s} \leq I_{d}$. In an ideal world, this would be equality. If only there were something that existed to help gauge the size of the market for a product allowing a seller to capture all the information ... (hint: it's called money).
Which reminds me, we haven't actually discussed what the buyer's are buying these apples with yet. We'll start with Nick's suggestion of using a banana to buy an apple. In that case the informaton the sellers collect (by acquiring a banana from the apple buyer) is
$$
n_{b} \log_{2} b = n_{s} \log_{2} s
$$
where $b$ is the number of potential banana buyers. We can use this to determine the "exchange rate" (the price of a banana in terms of apples). If we take the smallest unit of bananas to be $dB$ so that $B/dB = n_{b}$, then
$$
B \log_{2} b = S \frac{dB}{dS} \log_{2} s
$$
where $n_{s} = S/dS$ the number of apples supplied to the apple buyers. If we assumed the market for
$$
B \sim S \frac{dB}{dS} \leq n_{d}
$$
where $dB/dS$ (where we let $dS$ and $dB$ become infinitesimal) is the "exchange rate" (price) of apples to bananas. What if the apple buyer gave the apple seller something else instead of bananas? Well, starting with equation (2) you'd have
$$
n_{s} \log_{2} s \leq n_{d} \log_{2} d
$$
$$
\frac{S}{dS} \log_{2} s \leq \frac{D}{dD} \log_{2} d
$$
$$
\frac{dD}{dS} \leq \frac{\log_{2} d}{\log_{2} s} \frac{D}{S}
$$
We'll call the left hand side the price $P$ (like the exchange rate above) and define $\log_{2} s/\log_{2} d \equiv \kappa$ (the information transfer index). Leaving us with:
$$
\text{(3) } P = \frac{dD}{dS} \leq \frac{1}{\kappa} \; \frac{D}{S}
$$
where we can call $S$ the supply and $D$ the demand.
If we take the thing exchanged to be dollars, then we could potentially take $\log_{2} s \rightarrow \log_{2} m$ where $m$ is the size of the money supply (monetary base). This would allow you to get a really good estimate of the potential market size d by looking at the price (or at least changes in the price) while knowing the size of the money supply.
In the information transfer model, I have typically assumed equality in equation (3) and taken $\kappa$ to be an unknown constant I fit to the data. One notable exception is the money market where I took $d = NGDP$ and $s = MB$ and used it to describe the price level. In many of the posts on this blog, I've used the shorthand notation $P:D \rightarrow S$ for a model that transfers information from the demand $D$ to the supply $S$ with price $P$.
PS Some notes:
1. Equation (1) assumes transactions are maximally uninformative, or equivalently, all microstates with $n_{d}$ apples sold are equally likely. One way of thinking about that is that it makes the fewest assumptions about potential microfoundations.
2. Equation (3) is the simplest possible relationship between supply and demand that maintains homogeneity of degree zero (related to the long run neutrality of money).
3. Per Nick Rowe's original post, if the network of exchanges collapsed, we'd see a fall in the total amount of information being exchanged so that if we looked at the market $P:AD \rightarrow AS$ for aggregate supply and aggregate demand, we'd see a fall in $AD$ and/or $P$.
4. Equation (3) above can be solved to recover supply and demand curves and Marshall's diagrams.
[1] Footnote added in update 3/14/2014. The a priori estimate information about the size of $d$ would actually have to come from somewhere else. Being strict about it, all the seller would know is that $s$ is at most the size of the total number of apples he or she has sold and at least the number of different people sold to. In this way, $s \rightarrow d$, eventually but only if the market for apples was dominated by a monopoly. Competitive sellers really can't get at the information about the size of $d$ without some sort of tool to measure the size of the market -- that's what money does. It allows a seller to gauge the size of the market in order to calibrate the information they receive (they don't know if $I_{s} = $ 10 bits or 100 bits) so they can use that information to supply the market. If the price suddenly goes up, the amount of information being received suddenly increases ($I_{s}$ goes from say 10 bits per apple to 20 bits per apple), telling the supplier that demand ($d$) has increased (or supply from all the suppliers has fallen).
[1] Footnote added in update 3/14/2014. The a priori estimate information about the size of $d$ would actually have to come from somewhere else. Being strict about it, all the seller would know is that $s$ is at most the size of the total number of apples he or she has sold and at least the number of different people sold to. In this way, $s \rightarrow d$, eventually but only if the market for apples was dominated by a monopoly. Competitive sellers really can't get at the information about the size of $d$ without some sort of tool to measure the size of the market -- that's what money does. It allows a seller to gauge the size of the market in order to calibrate the information they receive (they don't know if $I_{s} = $ 10 bits or 100 bits) so they can use that information to supply the market. If the price suddenly goes up, the amount of information being received suddenly increases ($I_{s}$ goes from say 10 bits per apple to 20 bits per apple), telling the supplier that demand ($d$) has increased (or supply from all the suppliers has fallen).
Nice explanation. I really like the idea of money as a medium of information transfer.
ReplyDeleteJust a quick note:
"If we assumed the market for apples is about the same size as the market for apples"
I get an impression that you've mixed bananas with apples in the middle.
Thanks! I will fix the typo :)
DeleteIn a real market the guys trading bananas for apples don't use ID numbers. Yet real markets work. Can you explain the information transfer that is going on without using ID numbers?
ReplyDeleteI used ID numbers because it is an intuitive explanation of why the amount of information is proportional to $\log d$. In information theory, all that matters is that in principle you could assign ID numbers (people are distinguishable -- a case where this fails is in quantum systems and you get corrections of order $N \log N$ where $N$ is the number of indistinguishable particles, see the Sackur-Tetrode equation for example).
DeleteThe underlying idea is that each of the $d$ states are equally probable so that all the probabilities in the Shannon information are equal ($p = 1/d$), therefore:
$$
I = - \sum_{i}^{d} p_{i} \log p_{i} = - \sum_{i}^{d} (1/d) \log (1/d)
$$
$$
I = \log d \sum_{i}^{d} (1/d) = \log d
$$
Imagine flipping a coin five times. After the first flip, you get heads (H, versus tails T) and you learn that the sequence is H????. After the second, you get heads and you learn that the sequence is HH???. After the second flip, you've eliminated all of the sequences that started with TT, so you eliminated 8 possible sequences (you gained information).
Now the real world probably doesn't have equal probability of all states, so the "ID number" argument is an approximation. However, it's one that seems to work pretty well for macroeconomics because the system is really large (that's why the assumption of maximum entropy, the principle of indifference or the ergodic hypothesis work for statistical mechanics ... I believe it is the information theory formulation of the efficient markets hypothesis).
I stopped reading at the second sentence, as soon as it became clear the guy thought there was the same amount of information in one apple out of one being sold at the end of the story and one apple out of a million being sold at the beginning.
ReplyDeleteWhy would a guy post something that's obviously senseless?
-dlj.
Is this directed at me (the author)? Because that is not what the math above says. One out of one apple conveys no information (log 1 = 0) and one out of a million conveys 6 log 10 ~ 20 bits.
DeleteYou lost me at inequality (2). What is "n_s" supposed to represent? Up to that point, it seemed that n_d was the amount acutally sold into a market of size d. But if s is, as you say, the sellers estimate of market size, then its corresponding amount sold will not be an acutally realized, observable quantity.
ReplyDeleteIf I take it that the realized sales should be on both sides of (2), so that the two n_ are the same, then the inequality holds if and only if the seller has underestimated the size of the market. That's about half the time, but you seem to be treating this as true in general.
Sorry about the long delay in getting back to your comment (I remember I was travelling Jun 20th, so that's probably why I missed this).
Deleten_d is the number of demand units
n_s is the number of widgets
These are basically proportional to each other (via the information transfer index).
And yes, in information equilibrium, these are equal (up to the IT index). However, information equilibrium only holds in large systems (and then only in the case when the market is functioning). Sometimes
n_d > k n_s
Which is the case of non-ideal information transfer.
You can think of these variables in the same way you think of variables measuring an ideal gas. The ideal gas law says that
p V = n R T
However, that is only in thermodynamic equilibrium (ideal info transfer) and in the limit of a large number of molecules. Only then can we say p = 〈p〉and the fluctuations around the average (measured by the variance) are small. Economic systems (10^6 - 10^9) don't have as many degrees of freedom as thermodynamic systems (10^23), so you'll see larger deviations from ideal in the former.
Jason, I can see why Michael was confused. You present n_d as the number of apples sold: "...we've sold nd apples..."
DeleteThus, in the post, when you introduce n_s without further explanation, it's natural to assume that the number of apples sold = the number of apples purchased, and n_s seems like it must fit this bill.
Now it's clear enough (I think) that the number of potential apple buyers (d) is going to be different than the estimate of the size of this number of buyers (s) by the sellers (as you present it in the text), but it's not at all clear whether s will be less than or greater than d.
Perhaps if you explained exactly what n_s is when you introduced it the concept would be more clear.
It's be interesting to see you present this (live) to some skeptics, with limited math skills.
I'm wondering if it would make sense to introduce the concept of apple pies here, each requiring x apples per pie.
Also, the introduction of B and S might be confusing.
"If we take the smallest unit of bananas to be dB"
I think a lot of people might jump to dB = 1 banana at that point, making B = n_b. Same for S and n_s. I know you say they get "infinitesimal" later to define the exchange rate, which makes sense, but where does that leave B and S?
I read this post when you first made it, and I was confused by some of this, but I pressed on anyway. I know I can go to your draft paper and I can refer to Fielitz and Borchardt. I love that gas law analogy... it always helps me, but the way you presented it above would leave me scratching my head I think. First off, it's not clear that D is analogous to the right hand side, and that V is analogous to S, and that p is analogous to P.
Imagine we shrunk the degrees of freedom down from 10^23 in a gas system, to such a low number that it was clear that sometimes
Iq > Iu
I.e. "large deviations from the ideal" (Iq = Iu) What would that look like?