On the problem of theory selection in economics:
I was initially attracted to Hayek not for his politics or his view on markets, but for what he had to say about the limitations of economics. Hayek developed his ideas about the dispersion of knowledge during the socialist calculation debate of the 1930s. His ideas led him to reject, of course, the assumption of perfect information that inhabited most of the models of his day, but also the rationality assumption and even the use of equilibrium theorizing, which is one reason he is so little taught today in the standard economics curriculum. Hayek viewed the economy as a complex adaptive system, and thought that when we confront such a phenomenon, our ability to control it is severely limited. Often the best we can do is to make broad pattern predictions or provide explanations of the principles by which such phenomena behave.
If Hayek was right, this implies severe limitations on our ability to adjudicate among rival theories. This fallibalist conclusion is evident in a paper first published back in 1955. As he put it:
But if it is true that in subjects of great complexity we must rely to a large extent on such mere explanations of the principle, we must not overlook some disadvantages connected with this technique. Because such theories are difficult to disprove, the elimination of inferior rival theories will be a slow affair, bound up closely with the argumentative skill and persuasiveness of those who employ them. There can be no crucial experiments which decide between them. There will be opportunities for grave abuses: possibilities for pretentious, over-elaborate theories which no simple test but only the good sense of those equally competent in the field can refute. There will be no safeguards even against sheer quackery. Constant awareness of these dangers is probably the only effective precaution. But it does not help to hold up against this the example of other sciences where the situation is different. It is not because of a failure to follow better counsel, but because of the refractory nature of certain subjects that these difficulties arise. There is no basis for the contention that they are due to the immaturity of the sciences concerned. It would be a complete misunderstanding of the argument of this essay to think that it deals with a provisional and transitory state of the progress of those sciences which they are bound to overcome sooner or later.
Hayek’s words explain why progress has been so slow in economics, why very bright people can still continue to disagree about the best way forward for our economy, and why it will ever be so. The hoped for effect of this is to make us all a bit more humble about our proposals, like Keynes’ dentists.