“Despite all my rage, I am still just a rat in a cage”

Rachel Strohm links to an interesting piece in the Boston Globe on the trend toward using randomized controlled trials in developing regulations and legislation. Set aside the fact that it sounds ethically monstrous to deny people homelessness-prevention services for the sake of an experiment. What we really don’t get is how incentives work in a situation like this when the subjects are able to know that they’re subjects. The article mentions business owners who would act a certain way during the experiment to get the regulation they want. Duh. The other way that people are different from rats (besides, uh, not being rats) is that they can read the Internet. No way this kind of thing flies once the right-wing blogo-talko-sphere gets a hold of it. They’d be singing the Smashing Pumpkins song quoted in the subject line….

Two things though: one is the difference between monitoring and evaluation. One of the proponents of experimentation says in the article:

We study regulations only at “the very moment when we know least about the consequences,” Greenstone says. “There is no culture of trying to understand ex post what the consequences are.”

I would be happy if we studied the consequences of regulations and tried to figure out from there what the mechanisms are that lead people to act in certain ways. In other words, map out the incentives. What do businesses end up doing in terms of providing health care for employees as a result of different pieces of health-care reform? We don’t even get that in many cases; what exactly are we going to experiment on? What’s being evaluated? Let’s start by monitoring and go from there.

The other is the difference between laws and regulation. Laws are passed by democratically elected bodies like legislatures and executives; regulations are drafted by appointed bodies in the civil service. It would seem there would be more incentive to test laws because of the greater potential accountability. Perhaps all the more reason for there to be testing of regulations, to build in greater accountability?

There’s a lot of interest in philanthropy and the nonprofit sector in the use of RCTs in evaluation and the design of programs, so this debate is worth following.


Tags: , , ,

Leave a Reply