This review is a primer for those who wish to familiarize themselves with nonparametric econometrics. Though the underlying theory for many of these methods can be daunting for some practitioners, this article will demonstrate how a range of nonparametric methods can in fact be deployed in a fairly straightforward manner. Rather than aiming for encyclopedic coverage of the field, we shall restrict attention to a set of touchstone topics while making liberal use of examples for illustrative purposes. We will emphasize settings in which the user may wish to model a dataset comprised of continuous, discrete, or categorical data (nominal or ordinal), or any combination thereof. We shall also consider recent developments in which some of the variables involved may in fact be irrelevant, which alters the behavior of the estimators and optimal bandwidths in a manner that deviates substantially from conventional approaches.

But story time can’t be avoided. On one hand, there are real questions to be answered and real decisions to be made in development economics (and elsewhere), and researchers and policymakers can’t simply sit still and say they can’t do anything because the data aren’t fully persuasive. (Remember the first principle of decision analysis: Not making a decision is itself a decision.)

From the other direction, once you have an interesting quantitative finding, of course you want to understand it, and it makes sense to use all your storytelling skills here. The challenge is to go back and forth between the storytelling and the data. You find some interesting result (perhaps an observational data summary, perhaps an analysis of an experiment or natural experiment), this motivates a story, which in turn suggests some new hypotheses to be studied. Yu-Sung and I were just talking about this today in regard to our article on public opinion about school vouchers.

The question is: How do quantitative analysis and story time fit into the big picture? Mike McGovern writes that he wishes Paul Collier had been more modest in his causal claims, presenting his quantitative findings as “intriguing and counterintuitive correlations” and frankly recognizing that exploration of these correlations requires real-world understanding, not just the rhetoric of hard-headed empiricism.

I agree completely with McGovern–and I endeavor to follow this sort of modesty in presenting the implications of my own applied work–and I think it’s a starting point for Coliier and others. Once they recognize that, indeed, they are in story time, they can think harder about the empirical implications of their stories.

Cookbook econometrics has few fans.

I am one of them.

Cookbook econometrics provides clear algorithms for solving econometrics problems, without providing detailed explanations of why these algorithms work, or why specific steps in that algorithm are required.

For example, cookbook econometrics says "if you are trying to explain variable that can take on just two possible values, for example, smoker/non-smoker, use probit," but skips the formula for the inverse cumulative density function of the standard normal distribution, any discussion of how probit estimates are actually calculated, and proofs of the properties of the probit estimator.

David Giles sets out the case against cookbook methods in his blog:

My contention is that if you've been taken through the proof, and seen the assumptions "in action", you're more likely to pay proper attention to those assumptions being satisfied when you use the result, day to day, in your empirical work.

My position is the exact opposite: When you have struggled with empirical work, and seen econometrics "in action", you're more likely to pay proper attention to the proof, and understand the underlying assumptions. 

The difference stems from contrasting views about how people learn. His position appears to be that it is possible -- indeed desirable -- to grasp and understand abstract concepts before applying them and working with them. 

Real world data is messy. Dirty. Untidy.

Before you can even think about using all of those pretty techniques you learned in econometrics class, you need to clean up the data.

Here is my nine step approach.

Four basic steps constitute statistical hypothesis testing. First, one develops a null hypothesis about some phenomenon or parameter. This null hypothesis is generally the opposite of the research hypothesis, which is what the investigator truly believes

A page of resources on hierarchical linear models.

An incredibly useful method is to fit a statistical model repeatedly on several different datasets and then display all these estimates together. For example, running a regression on data on each of 50 states (see here as discussed here), or running a reg