This is a nice article by unlearning economics on Pieria about some assumptions in microeconomics. I have a couple of things to add to this (I'll leave the great discussion of game theory for another time) ... as well as a possible solution to the issues raised.
First, the assumption of transitivity (or the different but related assumption of GARP) is actually equivalent to the idea that humans have a measure called "utility". Basically all that is involved is that the manifold of [transitive] preferences has a structure that allows them to be related by a diffeomorphism to the manifold of real numbers ... numbers we call utility. A real number we can maximize.
There is no difference between assuming "well-behaved preferences" or "transitivity" and assuming there is a property of economic agents called "utility" measured by a real number.
Another assumption mentioned in the article is monotonicity of utility functions. This is required in order for the utility maximum to saturate the budget constraint and be a single point on that constraint. Again, this assumption is only required because you want to use utility (to come up with a single solution).
So here we are making strange assumptions that are only required because we want to use utility to analyze economic problems.
It reminds me of the history of physics where physicists came up with a bunch of odd assumptions in order to continue to use a theoretical construct. It was called the aether and it asked physicists to assume partial aether dragging and length contraction.
What if I told you you could get the same general results as utility without assuming transitivity or monotonicity? And what if I told you it had a single assumption: the principle of indifference.
Let's start with a basket of consumption goods (or intertemporal consumption periods or both) C₁, C₂, C₃, ... Cn subject to the budget constraint Σ Ci ≤ M.
Let's assume every consumption "state" Ci = pi xi in this d-dimensional space is equally likely (the principle of indifference). This could be intertemporal consumption (i ≤ d indexing time) or different goods (i ≤ d indexing goods) or even intertemporal consumption of different goods (i ≤ d indexing goods and time).
And here is d = 3:
How can I saturate the budget constraint (and select a point on it) using this? Dimensionality. As d → ∞, the distribution of points becomes highly concentrated around Σ Ci = Σ pi xi = M as can be seen in this graph:
For an infinite number of goods and/or an infinite number of time periods, the 'representative' (average) point approximately saturates the budget constraint. The (emergent) representative agent spends all of its money. However individual agents can vary from spending all to saving all of their money.
This representative agent also appears to engage in consumption smoothing (if you look at i indexing time in the intertemporal problem, all time periods are roughly equal in terms of the value of consumption ... [a symmetry that can be broken by the rate of interest]). Consumption smoothing is an emergent property of the ensemble of agents that are free to choose any point in the domain (and any given agent is unlikely to have very smooth consumption).
This maximum entropy view reproduces the basics of the utility maximization model without the utility. In fact, utility can be seen as emergent . And since utility, a real number, can be used to describe the solutions (equilibria) we see in the maximum entropy view we see that transitivity (or GARP, both equivalent to real number utility) is an emergent property of the emergent representative agent. This is important: transitivity is explicitly not true of the individual agents -- they have random consumption baskets that they have revealed they prefer! Their preferences are not transitive -- they aren't even stable! Agent 9000 prefers A to B one day and B to A another.
So here's a list of some emergent properties of the emergent representative agent:
- Transitive preferences (a consequence of emergent utility)
- Monotonicity of utility (satiation)
- Consumption smoothing
And here's a list of properties of the underlying individual agents:
- Preferences are not transitive and are unstable (random preferences)
- No preference to more or less of a good (random preferences)
- Consumption fluctuates (random consumption)
The idea of a rational utility maximizing representative agent is an emergent construct  in the entropy maximization paradigm; real individual people need not have any of these properties.
 Noah Smith references emergent representative agents in a recent post:
For example, suppose psychologists find that most human beings are incapable of forming the kind of expectations that time-varying utility models say they do. That would mean one of two things. It could mean that the economy as a whole behaves qualitatively differently than the individuals who make it up (in physics jargon, that would mean that the representative agent is "emergent"). Or it could mean that time-varying utility models must not be the reason for excess volatility.
 This is a bit subtle -- entropy maximization chooses a particular utility function (or really a class of utility functions that are maximized at the entropy maximization point).