What Can We Learn from Randomized Controlled Trials?

February 25, 2010

Randomized control trials (RCT) are becoming a standard tool in economic research, but they pose potential pitfalls for the research, according to MFI Fellow Angus Deaton. In social science, randomization is not a foolproof path to scientific validity, and may actually inhibit thinking about the question studied, Deaton cautioned in a lecture for more than 100 faculty and students on Feb. 25, 2010.

Randomized trials are considered the the most credible way for testing the efficacy of new drugs, medical devices, or surgical protocols. They have been widely adopted in economics research, but Deaton points out that too many researchers use RCTs exclusively—abandoning an integrated empirical and theoretical approach.

Deaton, the Dwight D. Eisenhower Professor of International Affairs and Professor of Economics at Princeton University, pointed out potential problems with the methodology. The randomization might fail, for example, and without some understanding of the mechanism, he said, "there's no validity to generalizations about the data."

"In economics, we are getting too dependent upon RCTs. Good observational studies often yield the same results," Deaton noted. "Randomized controlled trials actually inhibit thinking in some cases."

Politicians sometimes base policy decisions based on the outcome of RCT research. "If politicians do things we don't like with research based on RCTs, that's another argument for being careful about which RCTs we use or which questions we ask," said Deaton.

Deaton said he wants to raise economists' consciousness about the pitfalls of relying too heavily on RCTs. When it comes to research methodologies, there is "no method that is a magic bullet, but RCTs should be scrutinized carefully."

His advice to economists: "You work the theory to the point where you've got sharp predictions—preferably predictions that can be tested without having to do incredible econometric acrobatics that nobody actual believes. It's a lot of work, and it's not what we train our students to do. We don't often train them how to develop the theory to the point where it is yielding a sharp, refutable hypothesis."

Deaton also recommends that economists with different approaches should work more closely together. "The problem is not with empiricists or with theorists," he said, "the problem is that we don't talk with one another nearly as much as what used to be the case. That may be less true in macroeconomics than it is in microeconomics, but there seems to be very little common work."

Watch Video of the event

View Slideshows

Other Papers Presented