Wednesday, May 11, 2016

Lyapunov exponents and the information tranfer index

As some of you may know, I'm in the process of writing a paper with frequent commenter (and MD) Todd Zorick on applying the information equilibrium model to neuroscience -- in particular: can you distinguish different states of consciousness by different information transfer indices that characterize EEG data? It's been something of a slog, but one reviewer brought up the similarity of the approach with a "scale dependent Lyapunov exponent" [pdf].

You can consider this post a draft of a response to the reviewer (and Todd, feel free to use this as part of the response), but I thought it was interesting enough for everyone following the blog. Let's start with an information equilibrium relationship $A \rightleftarrows B$ between an information source $A$ and an information destination (receiver) $B$ (see the paper for more details on the steps of solving this differential equation):

\frac{dA}{dB} = k \; \frac{A}{B}

If we have a constant information source (in economics, partial equilibrium where $A$ moves slowly with respect to $B$), we can say:

\frac{A - A_{ref}}{k A_{0}} \equiv \frac{\Delta A}{k A_{0}} = \log \frac{B}{B_{ref}}

Let's define $B$ and $B_{ref}$ as $B_{A_{ref}+\Delta A}$ and $B_{A_{ref}}$, respectively, and rewrite the previous equation:

B_{A_{ref}+\Delta A} = B_{A_{ref}} \; \exp \left( \frac{\Delta A}{k A_{0}} \right)

This is exactly the form of the Lyapunov exponent [wikipedia] $\lambda$ if we consider $A$ (the information source) to be the time variable and $\lambda = 1/k A_{0}$

B_{t+\Delta t} = B_{t} \; \exp \lambda \Delta t

[Update 13 June 2016: As brought up in peer review, we should consider the $B$ to be some aggregation of a multi-dimensional space (in economics, individual transactions; in neuroscience, individual neuron voltages) because $\lambda$ measures the separation between two paths in that phase space.]

This is interesting for many reasons, not the least of which is that a positive $\lambda$ (and it is typically in the economics case) is associated with a chaotic system. Additionally, the Lyapunov dimension is directly related to the information dimension. (See the Wikipedia article linked above.)

I want to check this out in this context.


In general equilibrium we have

B_{t+\Delta t} = B_{t} \; \exp \left( \frac{1}{k} \; \log \left( \frac{t + \Delta t}{t} \right) \right)

which reduces the the other form for $t \gg \Delta t$ (i.e. short time scales).


  1. Thanks, I will add some of this! Kind of reminds me of a warped version of the TV show "Jeopardy": "Please phrase your answer in the form of a blog post..."

    1. Ha! And sorry for the delay in getting back to you -- my primary job has intruded on the fun work.