## Monday, June 29, 2015

### The importance of transversality conditions (more on the Ramsey model)

There has been some fun and interesting discussion of the Ramsey-Cass-Koopmans [RCK] model on this post of mine. Sorry to those in the discussion that I haven't gotten back to the comments yet -- I've been taking my time to think about what's been brought up. I noted at the end of my post that:
Now there is some jiggery-pokery in the [RCK] model -- economists include "transversality conditions" that effectively eliminate all other possible paths [in the phase diagram].
This was not some throw-away line in the conclusion; it was the key point of the post. I think LAL's comment is a really useful way to understand how Nick Rowe and I ended up talking past each other:
I think you should pay more attention to the transversality conditions...there is a lot more economic content to them than you are realizing...
I completely agree that there is a lot of economic content! I think this is what Nick Rowe thought I kept missing when he metaphorically threw the eraser at me sitting in the back of the class ("What's the difference between pendulums and people? People have plans and expectations about the future, that affect their current actions."). However, my main point was that the transversality conditions (enforcing those plans and expectations about the future) are practically all of the economic content of the RCK model -- the RCK equations are somewhat superfluous.

This system of differential equations has a "saddle path" solution that runs from from a pair of initial conditions for capital and consumption to the equilibrium point. In the next graph I show the saddle path (black), the (approximate) equilibrium (black point) along with 2000 paths randomly distributed within 1% of the initial conditions that lead to the saddle path:

As you can see, most of these paths diverge from the saddle path -- and that's just for being 1% off. So given measurement error and random events, you are unlikely to find yourself exactly on the saddle path.

One of the main purposes of the transversality conditions is to say nearly all of those 2000 paths don't make economic sense. I borrowed this particular description of the argument/intuition from these lecture notes [pdf], but in general this is what Nick was getting at:
Imagine a path along which consumption is falling and k is therefore growing very large. Along such a path the product u'(c) k would grow rapidly, probably causing the limit [to violate the transversality conditions]. Such a path could not be optimal, however, because the economy is accumulating excessive hoards of capital, the output of which never gets consumed because it is reinvested instead. It would pay for the economic planner to slightly and permanently increase consumption, an option that is perfectly feasible given the rapid growth in k.
What this means in the context of the RCK model is that if the economy finds itself on one of those 2000 paths that aren't the saddle path, the economic agents realize a better deal can be had by reducing capital (or reducing consumption) in order to bring the economy back to the optimal saddle path. The result of that (exaggerated for clarity) is a set of path segments of the RCK model (blue) along with corrections (orange dashed) intended to bring the economy back to the saddle path (black line):

You can think of the blue segments as the times when the economy is obeying the RCK model and the orange loops as the times when the economics enforcing the transversality conditions is driving the economy. And this is where we come to my point: most paths would consist of mostly those loops since nearly all paths in the neighborhood of the saddle path diverge from the equilibrium point.

That is to say the typical path would be incredibly jagged [1]. Most of the time it would not be following the RCK saddle path -- or even obeying the RCK model equations, but instead would be on some correction jog taking the economy back to the saddle solution because of the transversaility conditions [2]. A typical path would look like this:

It would be entirely orange corrections (due to transversality conditions), rather than blue RCK solution paths. An ensemble (or path integral) of such paths would average (integrate) to the RCK saddle solution (which I mentioned in my reply to Nick Rowe). But an ensemble would also do that without the transversality conditions. If we just average all the 2000 paths [3] in the graph at the top of this post, we get the result we want (the saddle path, approximately) without assuming the transversality conditions or the economics they entail:

That means the necessary transversality conditions that end up representing most of the economics of the RCK model if understood in the neoclassical sense (i.e. why only 1 of those 2000 paths turns out to be valid) are actually unnecessary. The RCK model equations (at the top of the post) should be understood as establishing all possible ways to consistently divvy up capital and consumption over time. The transversality conditions say that only one of those ways is valid by fiat. An ensemble approach says that all ways are valid, but observations should be consistent with the most likely path.

Update 6/30/2015

If I understand Nick Rowe's comment below ("If rational, [Robinson Crusoe] would jump [to the Saddle path] immediately, and stay on it forever.") the picture looks more like this:

I agree that is the rational (model-consistent) expectations view, but in that view the transversality conditions do next to nothing. They apply once when Robinson Crusoe is first stranded and calculates how much rum he has. After that, Crusoe lives a 'feet up, mind in neutral' beach bum life -- not having to worry about having drunk too much of the rum or too little. He's no longer an optimizing agent, but passively obeying the differential equations at the top of this post.

There are also the questions of a) how do you know what the saddle path is? and b) how do you determine if you're on it? Robinson Crusoe has to spend a finite amount of time figuring out the path and his location relative to it to a given (finite) accuracy ... something that gets infinitely harder as you approach the equilibrium. And valuable drinking time to boot!

Footnotes:

[1] This is actually what happens in Feynman path integrals -- the set of smooth paths has measure zero, so a typical path contributing to the path integral is a noisy path "near" the classical solution.

 Graph borrowed from here.

[2] These corrections would be the inputs to control the inverted triple pendulum in the example Tom Brown mentioned. In that example, the controller spends most of its time making tiny corrections (analogous to the orange paths in the graphs above), not letting the pendulum follow the laws of physics (analogous to the blue paths).

[3] It becomes numerically unstable in the brute force way that I've implemented the model and the actual solution wasn't exact so I was unable to show the whole path on account of an outbreak of laziness in finding the exact solution and making sure the random path initial conditions weren't biased (as they seem to be ... towards the all-consumption solutions).

1. Hm, I'm not sure I'm convinced that the ensemble average is independent of the transversality conditions...does the typical path deviate much from them?

1. I'm not sure I understand your question -- are you asking if the ensemble average path obeys the transversality conditions? It probably does because it's near the optimal RCK path which obeys the conditions.

2. I guess yes I am asking that..I find that fact interesting...I thought you said somewhere in the above post that it doesn't have to obey the transversality conditions and hence doesn't have to imply the economics of it...but I'll probably take another look

3. What I tried to say was you didn't have to assume them ... additionally any single given path doesn't have to obey them.

4. Jason, for clarification:

Are you saying that the average difference between the ensemble path and the RCK path will be small?

5. Hi Bill,

Sorry for the delay in answering your question (I actually thought I did, but somehow the comment didn't appear), but the short answer is yes. The average of paths that start near a point on the saddle path will on average have a starting point on the saddle path and will therefore on average follow the saddle path (the difference will be small).

2. Jason, very interesting. Your footnote [2] answered the question I was going to have for you. But I have another: in your 3rd and 4th figures (i.e. the 2nd and 3rd plots resp.) are the orange paths hand drawn or did you "stabilize" the system with a feedback control law and calculate them? I'm guessing the former.

Although I was never involved in the project (or anything quite like it) I understand that the Apollo missions (moon shots) were determined via some optimal control solution off-line as an open loop solution for the best trajectory, and then real-time linear feedback control solutions (linearized around different parts of the optimal trajectory) were applied locally during the actual flight of the rocket to try to keep it on the pre-calculated optimal path. Or at least that's what's now in my imagination after hearing a professor mention this subject decades ago in one of my engineering classes. Whether that's true or not, that's kind of what the orange paths remind me of.

1. Hi Tom,

You're right; I drew them in by hand.

Regarding the Apollo missions, that's basically true. One of the biggest advancements was more accurate inertial guidance systems. A system that could guide astronauts to a landing site within a few miles on the moon could guide a nuclear weapon to a few hundred meters on earth... Hence the interest at the time.

2. Hemispherical resonator gyroscope? I have a co-worker with one on his bookshelf (made into a trophy of sorts) from a previous job where he worked on that. Actually a few people here were involved with that project... now relocated from here (Goleta) down to Woodland Hills (where by brother works, by chance). My brother told me last week they sell about 12 of them a year, but they have something like 30 million hours w/o a failure, so if that's what you're after you don't have too many other choices. He says the proximity of the freeway forced some expensive modifications to the facility (because of vibrations). The Wikipedia article wasn't that helpful (in terms of dates of service), but I can ask my co-workers...

3. I think that is a more recent gyroscope design; the story I am thinking of involves MIT/Draper labs and is summarized here:

I've been down to Goleta -- once did a work trip down there. Beautiful area.

4. Thanks Jason. Goleta: yeah, I went to UCSB and never left town. I really enjoy it here (although I do prefer next door in Santa Barbara... one of the few moderately sized cities in California (~90k) where you can take a pleasant, sensory-appealing walk, nearly any time of the day or year, to nearly everywhere you might need to go. Too bad we don't have any fresh water left. )c:

5. ... my co-worker says 1996 was when the HRGs were first used. They did have one failure, but it was in the supporting electronics (a bad component). And he accidentally broke his desktop one some time ago...

3. Suppose we were literally talking about a one-agent version of the model. Why would Robinson Crusoe ever be off the saddle path? If he were off the saddle path, the longer he delays returning to the saddle path, the lower his lifetime utility. If rational, he would jump there immediately, and stay on it forever.

The only thing that matters for RC's current decision is his stock of K today. What previous path of C he took to get to that current K is irrelevant. Unlike a pendulum, he has no momentum. Unlike a pendulum, he looks only looks forward.

1. What if his estimate of his time remaining on the island has "inertia" and is noisy? Perhaps he sees a ship on the horizon and get's excited at his prospect for rescue for a day or two. Or bad weather makes him doubt if he'll live much longer... for a week or two.

2. Not so sure that people do not have something akin to momentum. After all, they have drives, they have wishes, they have plans. Drives like hunger and thirst have short cycles, while plans tend not to be cyclical.

As for history, I think that humans are more affected by that than pendulums.

3. BTW, what did the people on Pitcairn Island do?

4. Nick,

I understand that the RCK model is an optimizing model. All paths, including non saddle path ones, result from an optimization set up. So, I don't think your argument is correct.

One can think of the transversality conditions as intertemporal constraints. However, they are not as innocuous as generally assumed. Lance Taylor has a good discussion on this topic.

5. Srini: for simplicity, assume U=log(C), and time preference rho, and a single agent.

We know that C(t)/C(t+1) = (1+rho)/(1+r(t)) is a *necessary* condition for optimisation, but it is not sufficient. For example, with a finite-lived agent, who dies at time T, we also know that K(T)=0 is a necessary condition for optimisation. The (implicit) constraint is that K(t) >= 0 for all t.

Tom and Bill: those are different models. For Tom's model we have to maximise E(U), and specify the probability distribution, and how new information arrives. For Bill's model we need to introduce some cost of change, like a different utility function, so current utility depends not just on current consumption but on past consumption too (rum is habit-forming).

6. Thanks, Nick.

7. Now assume r = K^-0.5 (or something like that), and take the limit as T approaches infinity. As T gets larger and larger, Robinson Crusoe will spend an increasingly large fraction of his life increasingly close to the stationary state where C(t)=C(t+1) and K = rho^-2.

This is in line with what Bill says below.

I think there's something called a "turnpike theorem" that's related to this. But I never understood the math. On a very long journey, starting at K(0) being whatever its initial value is, and ending at K(T)=0, you spend a very long time driving very close to the turnpike in the middle of your journey. At T = infinity, you keep driving closer and closer to the turnpike.

8. Wiki is more authoritative than me on this subject: https://en.wikipedia.org/wiki/Turnpike_theory

9. Hi Nick,

I added an update above with what I think you are talking about. In that case, however, it seems agents optimize once and then live feet up, mind in neutral. The RCK equations are more an accounting identity (all the 2000 paths satisfy it), while the transversality conditions are doing the optimizing (selecting which path).

Srini,

Thanks for the reference -- I will look into it.

Tom,

I agree -- Robinson Crusoe has to spend valuable drinking time to determine not only his location but the location of the saddle path every time he is subject to shocks.

Bill,

History is probably important, but not so much for this model -- once you've moved to a given path, your previous path doesn't matter much anymore.

10. Jason: re your update: I think you understand it right.

" In that case, however, it seems agents optimize once and then live feet up, mind in neutral."

Well, I would say they make their optimal plan once, and then just carry out that plan. Like a driver who plots his route only once, at the beginning of the journey. He only changes his plan if new information arrives, but even her you could say he has only one *contingent* plan, that says what he will do conditional on all possible news.

But if he did re-examne his optimal plan, halfway along the route, he would continue on as before. See Bellman's Principle https://en.wikipedia.org/wiki/Bellman_equation (Though there are cases where optimal plans are "time-inconsistent", like when you make a promise and want to renege on it later.)

" The RCK equations are more an accounting identity..."

Oh no they aren't. They are a *necessary* (not sufficient) condition for optimality. An accounting identity is like "number of my sons + number of my daughters = number of my children."

11. Nick,

I stand corrected -- only one of the RCK equations is an accounting identity (for k' = production - consumption - depreciation = change in capital). The Euler equation (for c') is a necessary condition for utility maximization/optimality.

12. Jason: Yep. The output = consumption + investment thing is a true accounting identity (in this model, though it could also be reinterpreted as an equilibrium condition, in other models, just to confuse things!)

4. My impression is that the transversality condition is in line with Jaynes's recommendation to derive infinite cases by taking the limit of finite cases. Since all of the finite cases end with the complete consumption of capital, that should happen in the limit, as well. That is not jiggery-pokery.

Whether that is a realistic condition is another question. I am reminded of this line from "Von Armen B. B." by Bertolt Brecht,

"Froelich machet das Haus den Esser: er leert es."

Gleeful the house makes the eater: he empties it. Homo economicus, probably; real humans, probably not.

Wait. The transversality condition is more than that, isn't it? The central planner adapts by increasing consumption when the people are saving too much. How she does that is unclear. Is this a well defined model?

Also, under a random consumption model I suppose that on average half of what is available is consumed in the last period, instead of half of it. If the only "transversality condition" is that everything is consumed in the last period, then that may well not matter in the limit. But if under some incompletely specified conditions the central planner burns part of the crops or something, in order to guarantee that everything is consumed in the last period, then that may be a significant difference. Now, the transversality condition implies that there is a limit to what a person can consume in one period. Perhaps setting a top limit to capital would work in the random model.

1. Please ignore obvious typo in the last paragraph.

2. Bill: "My impression is that the transversality condition is in line with Jaynes's recommendation to derive infinite cases by taking the limit of finite cases."

Sounds sensible. Dumb question: who is Jaynes?

3. Nick,

This is ET Jaynes of maximum entropy fame ...

https://en.wikipedia.org/wiki/Principle_of_maximum_entropy

He took a wild swing at how economics might look as an entropy maximization problem:

http://informationtransfereconomics.blogspot.com/2015/02/jaynes-on-entropy-in-economics.html

4. I also toyed around with your Wicksellian roundabout idea as a MaxEnt problem afterwards:

5. Thanks Jason.

I did take a look at your roundabout post. Couldn't get my head around it, but that's probably my fault.

6. Yes, E. T. Jaynes. He was talking about probabilities, but I think that the idea applies in general. There are any number of probabilities related to infinities where people disagree. Keynes mentions some in his discussion of the principle of indifference in his book on probability. Jaynes realized that taking finite models to the limit could let you prove that a certain probability was correct in the limit. If the same infinite case could be reached in the limit in a different way, then a different probability would be valid in that case.

Certain paradoxes, such as two envelopes paradox, do not have finite models. For instance, if there are only two possibilities for the amounts of money in the envelopes, keep the envelope if it has the higher amount, otherwise switch envelopes. With Jaynes's approach the paradox is simply incoherent.

In the infinite Robinson Crusoe case Crusoe wants to eat everything on the last day, but does not know when the last day will be. If you simply allow Crusoe to eat everything on the last day of the finite cases, with no other restrictions, then the last day hardly matters. OTOH, if the next to last day needs to be like the last day, and the day before needs to be like it, etc., you can reason backwards. But won't that give you a higher rate of consumption, by comparison with the anything goes model? Froelich machet das Haus den Esser: er leert es.

7. Thanks Bill.

If Robinson Crusoe has diminishing Marginal Utility of consumption d2U/dC2 < 0, which is what we normally assume, then he won't want to wait till the last day. He will smooth his consumption over time (to some extent).

Here's how we handle uncertain lifetime:

If Robinson Crusoe had a probability of death p(t) per period, then the Euler equation becomes C(t)/C(t+1) = (1+rho)/((1-p(t))(1+r)

(That's assuming U=log(C), so dU/dC = 1/C )

One way to approach the transversality condition: assume there's a constant risk of death p per period up to time T, but a cutoff at date T, so p(T)=1 if he's still alive at that point. Then take the limit as T approaches infinity.

8. Thanks, Nick.

Crusoe is still going to want to have no savings if he dies on day omega, which means that he is going to want to have infinitesimal savings for the day before day omega, and so on. Homo sapiens might view that as a model for overconsumption.