## Sunday, November 2, 2014

### Expectations (rational or otherwise) and information loss

 The market expectation for NGDP (B, orange) and the actual distribution (A, blue).

This is going to be a rough sketch of an argument and a derivation. The one sentence summary is that there is always information loss in an economy, and that information loss is measured by a combination of NGDP and inflation.

Let's say that the market has an expectation of future NGDP measured by some distribution B; let's call the actual [1] distribution of A. Cartoons of these two distributions are shown in the graph at the top of this post. The difference between these two distributions, measured by e.g. the KL divergence, is:

$$D_{KL} (A || B) = \Delta I$$

This is the information gained by learning the real distribution A given distribution B, or alternatively, the information lost by assuming B. I previously talked about expectations destroying information -- they lead to information loss -- here (there's also bit on on the KL divergence). This information loss means that the future NGDP will be lower than the "ideal" NGDP since information in the source (aggregate demand) was not captured at the destination (aggregate supply). Another way to put this is that economic growth is lower than it could be if we knew the future as well as it could be known (i.e. knowing A).

Let's define this ideal (zero information loss) NGDP; we'll call it $N^{*}$ -- measured (observed) NGDP will be denoted $N$. Now the information loss should be equal to the difference in the "economic entropy" (see here, here) of these two NGDPs (I'll work in "natural" units where the parameter $c_{0} = 1$):

$$\Delta I = \log N^{*} ! - \log N !$$

Information and entropy are essentially the same thing, just with different units. Using Stirling's approximation for large $N$, we can write this as

$$\Delta I = N^{*} \log N^{*} - N \log N$$

Dividing through by the monetary aggregate that defines the unit of account $M$, we get:

$$\frac{\Delta I}{M} = \frac{N^{*}}{M} \log N^{*} - \frac{N}{M} \log N$$

And using the information transfer differential equation $dN/dM \sim (1/\kappa) N/M$, we can say:

$$\frac{\Delta I}{M} = \kappa \frac{dN^{*}}{dM} \log N^{*} - \kappa \frac{dN}{dM} \log N$$

Let's identify the ideal NGDP with the shock-less NGDP that defines inflation (and is the result of numerically integrating that differential equation without shocks) so that we can use $d N^{*}/dM \sim P$ where $P$ is the price level and $N^{*} \sim M^{1/\kappa}$.

$$\frac{\Delta I}{M} = P \log M - \kappa \frac{dN}{dM} \log N$$

Divide through by $\log M$

$$\frac{\Delta I}{M \log M} = P - \kappa \frac{dN}{dM} \frac{\log N}{\log M}$$

Using the definition of the information transfer index $\kappa$, we finally obtain:

$$\text{(1) } \frac{\Delta I}{M \log M} = P - \frac{dN}{dM} = \frac{dN^{*}}{dM} - \frac{dN}{dM}$$

The information loss is proportional to the difference between the growth rates of the ideal NGDP (also known as the price level $P$) and the observed NGDP ($N$) with respect to the monetary aggregate that defines the unit of account ($M$) [2].

Some observations:
• Rational expectations is the assumption that the distributions A and B are equal, that the information loss $\Delta I = 0$ and that $dN/dM = P$. This is empirically false (what we really have is $dN^{*}/dM = P$). While rational expectations may be a good first order approximation (over the short run), the market does not accurately know the distribution from which macro random variables are selected.
• An inflation target or price level target is a target for the first term in equation (1) while a NGDP level or growth rate target is a target for the second term. That is to say, an inflation or price level target is an ideal NGDP target ($N^{*}$) analogous to an NGDP target. Since $\Delta I$ is a priori unknown, setting one target sets the other. Another way to put this same information is that a given inflation target cannot be achieved simultaneously with an NGDP target (such an economy is over-determined).
• It is interesting that if one adds the assumption that $\Delta I$ is stable (e.g. is a stochastic process with unit root), this comes to the same conclusion as Nick Rowe: But we could live in a [self-stabilizing economy] ... if we adopted price level path targeting [i.e. $N^{*}$], or NGDP level path targeting [i.e. $N$]. However, there is no reason to assume $\Delta I$ is stable and the knob that the central bank would turn would be concrete step of adjusting the currency base ("M0"), not setting expectations (i.e. targeting $\Delta I$, which is impossible since the distribution A is fundamentally unknowable -- it requires knowing not only the future, but all possible futures). Nick's post was the inspiration for this one, which I previously mentioned working on here.
• If rational expectations were true (in the model above), NGDP and the price level would not be independent quantities.
Footnotes:

[1] This would be measured using a large number of identical economies and observing the measured NGDP in each economy.

[2] Note this is a growth rate with respect to the monetary aggregate that defines the unit of account unit of account, not time.

1.

I actually think keeping $N/M$ in the formula would be a better simplification (there should be a small error term introduced when replacing $N/M \rightarrow dN/dM + \epsilon$. In that case, equation (1) should read:

$$\frac{\Delta I}{M \log M} = P - \frac{N}{M}$$

2.

Due to the arbitrariness of the normalization of the price level, the relative normalization of $N^{*}$ and $N$ is not fixed (it's a free parameter).

One way to think of this is that at the point you start the model (initial conditions), you can take $N^{*} = N$ (no information loss) and project it forward (e.g. numerically solving the differential equation). Information loss accumulates from that point forward.

This represents the freedom of choosing the year to normalize the price level.

1. This comment has been removed by the author.

2. Oops

$$\frac{\Delta I}{M \log M} = P - \frac{1}{\kappa} \frac{N}{M}$$

2. Because we assumed that there is ALWAYS information loss, I would expect that the equation would work for information loss like theft or fire.

What if there was, instead of an information loss, a surplus of information (for example, additional information from an unexpected source)?

It seems to me that term M would change so we would need to leave term dM in the final formula.

1. Hi Roger,

The information we're talking about here is in the information theory sense of information, not 'facts'. See here, for example.

In this sense sending a random binary sequence through a communication channel transfers the maximum amount of information, whereas sending a constant string of 1's transfers almost zero information.

The KL divergence above is zero when the distribution B = A, and is positive for all other B's ... so our guess B of the real distribution A always represents information loss, unless B = A. There can't be a "surplus of information" -- you either have knowledge of A or you have incomplete knowledge of A.

http://informationtransfereconomics.blogspot.com/2014/07/rationality-is-beside-point.html

3. Hmmm, your work presents an interesting puzzle -- and an interesting approach to macroeconomics.

I have spent some time on the March 15, 2014 post "How Money Transfers Information". Maybe I misconstrued some of the associations.

I interpreted the Information Transfer Index as representing the number of transactions that each added to AD. I thought you reached the ITI by carrying information about transactions by enumerating in logarithmic notation, and then, when appropriate, turned the log notation back to a numerical form to get ITI = 1/k.

To me, this approach made a lot of sense because it was a reasonable attempt to carry the relatively constant number of transactions in an economy (annual basis) into the AD measurement (which is ultimately GDP).

Your posts have a lot of content--they are not easy reads. I will continue studying them. Thanks for making them available.

1. In a sense that is correct; the information transfer index measures the relative information in a unit of AD versus a unit of aggregate supply when they are matched up. The index is fundamentally a ratio of the logarithms of probabilities from Shannon's definition of information:

http://en.wikipedia.org/wiki/Quantities_of_information#Entropy

The index is first defined on this blog here:

http://informationtransfereconomics.blogspot.com/2013/04/the-information-transfer-model.html

It (and the rest of the information transfer model) comes from a paper by Fielitz and Borchardt cited at the link.

I've been working on trying to write more clearly as other commenters have also pointed out that many of my posts are not very easy to read. With each post I write I hope I'm getting a little better at it.