My view of why you should include math is best captured by Paul Krugman:
My point is that there seems to be a lot of implicit theorizing going on here — and at least at first glance, the implicit theorizing doesn’t make a lot of sense. I could be wrong, but that’s the whole point of simple models: to lay bare what you’re assuming, and make it clear what, specifically, is driving your conclusions.
Krugman says "models", and that's what I mean by math -- some sort of formal construct that organizes the various factors you are trying to explain. It doesn't have to be a precise DSGE model. Generic supply and demand diagrams are math. Working with orders of magnitude and asymptotic behavior qualify as math. Basically, math lets other people use your model.
Math is the basic universal language for talking about how things relate to each other. If you are saying X is related to Y, you are implicitly using math. If you don't want to use math, you're not allowed to say X is related to Y.
Of course, some people think math doesn't help illuminate anything. In its most dogmatic form, we can see the opposition to mathematics from Ludwig von Mises:
All of mathematical economics, with its beautiful curves and equations, is idle flirtation. The setting up of equations and the drawing of curves must be preceded by nonmathematical considerations; the setting up of equations does not broaden our understanding. Mechanical equations can be used to solve practical problems through the introduction of empirically acquired constants and data; but equations of mathematical catallactics cannot in the same way be of service to practical problems in the area of human action where constant relations do not exist.
This is of course completely disproved by the success of the theories of how auctions work. As well as the information equilibrium model ...
But in general the reason for mathematics is not that humans are mathematical, but that economists' models of the macroeconomy are mathematical -- whether they want them to be or not. When you start saying effect E has cause C, then you are assuming a functional relationship between E and C: E = f(C). From that we can say that if C is small, we have E ~ f(0) + a C ... and we're off to the races. I've just turned your statement of C causes E into a linear model with an equilibrium value E0 = f(0) and an "elasticity" a. If you say "but a and/or f(0) changes" ... then is it really C that is causing E? Note that linear function E ~ f(0) + a C has exactly the same form if E = f(a) where C is the elasticity. And f(0) changing means that something else is causing E .
The reason people don't use math is either a) they can't think clearly enough to use math or b) they want to pull the wool over your eyes. The stated defense is that math doesn't clarify (that's false; it does) or that human actions don't have constant relations (my question to von Mises would be "then why even write anything ... once written down as math or words, human actions will change"). If human actions don't have constant relations, then what does it matter? Even history is studied because people think certain things can cause e.g. wars in a fairly consistent way like competition for natural resources. People study WWI and WWII because they think the things that happened then could have benefit in future conflicts (preventing wars or fighting them). That is the entire point behind "history repeating itself". If history never repeated itself, then there wouldn't really be any reason to study it.
Not using mathematics for your model:
- Puts an upper bound on the complexity of your model (makes it easier to compete with smart people)
- Allows you to say what you're saying is novel even if it is equivalent to existing models (allows you to fool people into thinking you have new insight)
- Makes it impossible for anyone else to use your model (limits the use of your model to yourself)
- Allows your model to change in respose to new information without saying you are changing it in response to new information (so you can fudge any data)
- Allows you to get around stating clear predictions and precise conditions on those predictions (so you can fudge any prediction)
- Allows you to stay away from quantitative statements about the data (so you can fudge any data)
- Allows you to have effects of different magnitudes from causes of the same size (and vice versa) (so you can fudge any data)
A lack of math is not depth of argument or intuitive understanding, it is flim flam.
Update 7 September 2016
I should really have rephrased the part about "history repeating" in terms of "lessons of history" instead. People tend to take historical analogies too literally sometimes. An historical situation may enlighten you about the possibilities in a current situation (lessons), but there is almost never an exact analog of some historical event (repeating).
 This is my opinion of those expectation operators in models. An E[u(c)] term may look like your model depends on consumption or utility, but really it just depends on E.