I apologize ahead of time for the rant that's ahead. The rant is motivated by daily interactions with my colleagues who mostly have PhD's in statistics, computer science and mathematics. This gives me a different and less-economisty view of model building and validating in general, so bare with me.
When you sit down and read your average economics paper you generally know what's coming in terms of statistics. You're going to see a regression (often a variation of least squares), there's going to be some fancy talk about the error term and the author will likely base most of the conclusions on what the coefficients look like. This is actually true of many social science papers and it has been the primary means of social science modeling for quite a while.
However, when I see some of the cool new tools coming out of statistics or computer science I ask myself "why can't economists use these?" Those who follow finance relatively closely are aware of neural nets and some boosting techniques for time series forecasting but as a general rule many economists seem to turn a blind eye to these cutting edge techniques. I'm sure I'll get some comments mentioning Hal Varian's paper on machine learning and economics or Matthew Jackson's work with networks, but these seem to be exceptions rather than a trend. I've come up with a list of reasons why this may be, along with some comments:
1) Economists are obsessed with error terms. Anecdotally true - I've heard that it's hard to get a paper accepted to a decent journal that doesn't contain error terms for coefficients. With many machine learning models, you're generally concerned with some sort of cross-validation error rather than coefficient standard errors.
2) There isn't much interaction between economists and statisticians/mathematicians/computer scientists. This also is (sadly) a factor. I've seen a decent amount of interaction between econometricians and statisticians, but it's generally only frequentist statisticians. Not the new-fangled statisticians that are interested in some interesting Bayesian classifiers or machine learning techniques. Additionally, I rarely see economists publishing with authors from any of these three fields. There's a lot we could learn from them (and them from us!) so I see no reason why this should be binding.
3) It's harder to explain results from models more complicated than regression techniques. I can see this being an issue, particularly when some of these new techniques are first introduced and economists are not used to them. I can imagine the early seminars for these papers with no standard errors will lead to a lot of grumbling in the audience. However, I've seen economists explain some fairly dense material in a user-friendly way. It will take some practice, but I see nothing wrong with economists getting better at explaining models - it will add to our audience!
4) Economists don't have access to large enough data sets to use these techniques. I'm not sure about this one. After spending time in the tech field it seems like everyone has "big data," but I realize for many important topics, that's just not true. Some of it is due to not having the data "all in one place," but that is what research assistants are for! Particularly for revealed preference data, it seems like there should be some low-hanging fruit in terms of larger data sets where these new modeling methods could be used.
5) The math is too hard. No! Well, I guess it's different math, but that's nothing to be afraid of. For many methods such as trees and model boosting and bagging, there's not a lot of math involved that would be too different from existing econometrics.
6) Economists aren't always trying to predict something. Yes and no - sometimes we're trying to describe a system and sometimes we're trying to predict what the system will do. Decision trees are a perfect example of a method that lends itself relatively well to descriptive studies. However, I can understand why neural nets might not be as useful. It's a fair scientific question: what are our models capable of and what should they be used for? I don't think changing economics to a discipline that produces only predictive studies would be a good progression (and then we'd get even more comparisons to weathermen!) but I do think that we should take the time to evaluate whether existing models or new models can be used for the purposes we intend.
I'd love to hear comments - I would be happy to be proven wrong about this trend.
