Friday, February 17, 2017

Generative adversarial networks and information equilibrium


This is kind of a placeholder for some thoughts when I get a bit more time. There was a tweet today about Generative Adversarial Networks (GANs) leading to this Medium post. I immediately retweeted it:
This is very interesting for a couple of reasons. It demonstrates how a simple "price" (the judgments of the discriminator D) can cause the supply distribution (G, the generated distribution) to match up with the demand distribution (R, the real distribution). In this setup, however, the discriminator doesn't aggregate information (as it does in the traditional Hayekian view of the price mechanism), but merely notes that there are differences between G and R (the information equilibrium view). See more here.

As such, this sets up (an abstract) information equilibrium model:

$$
`` \; D \equiv \frac{dR}{dG} = k \; \frac{R}{G} \; "
$$

with D as the detector, except we have a constant information source (R, the real data). I hope to put this in a more formal statement soon. Real markets would also have demand (R) also adjust to the supply (G).

The example shown at the Medium post is exactly the generator attempting to match to a real distribution, which is one way to see information equilibrium operating. Here are the results for the mean and standard deviation of the distribution R:


The other thing I noticed is that there is a long tail towards zero mean in the final result:
Not bad. The left tail is a bit longer than the right, but the skew and kurtosis are, shall we say, evocative of the original Gaussian.


Is this some non-ideal information transfer?

$$
`` \; D \equiv \frac{dR}{dG} \leq k \; \frac{R}{G} \; "
$$

Does this have anything to do with this?


Or this (distribution of stock returns)?


Overall, looks very interesting!

3 comments:

  1. Wow, very interesting Jason! Thanks.

    I noticed you got rid of your "Anonymous" login capability. Or is that just a change in Blogger? I used to sometimes select "Name/URL" from the list (also not there anymore) to save me from having to log in to Google. Was somebody abusing it somehow?

    ReplyDelete
    Replies
    1. Thanks.

      And yes, there was an anonymous commenter who was suffering from Dunning Kruger, the fog of which could not be pierced.

      Delete
    2. Lol... I think I saw a bit of that.

      Delete