With environmental programs, science often stops when implementation starts

It’s impossible to imagine the medical community in the United States rolling out a new drug for obesity, or hypertension, or diabetes without first completing a robust series of experimental clinical trials. Even during a public health emergency, they conduct research before approving and deploying measures.

But that’s exactly what happens with environmental programs in the United States, says Paul Ferraro, Bloomberg Distinguished Professor of Human Behavior and Public Policy at Johns Hopkins. Ferraro says formal experimentation is “virtually absent” when the federal government enacts environmental programs.

Case in point: Over the past 30 years, the U.S. Environmental Protection Agency and the U.S. Department of Agriculture have embedded formal experimentation in their environmental programs fewer than half a dozen times, note Ferraro and colleagues in a recent policy forum article published in Science.

“We just roll along, year after year, completely unsure of how much of a difference these policies are really making because science typically stops when implementation starts,” said Ferraro, who has a joint faculty appointment in the Carey Business School, Whiting School of Engineering, and Bloomberg School of Public Health.

That represents a huge, missed opportunity, he says. “With environmental programs, we need to stretch scarce resources as far as possible, but we can’t do that unless we can accurately test how these interventions are actually working within complex systems.”

The messy complexity of human behavior

Largely what makes these systems so complex, Ferraro explains, is human behavior — which is a major factor in how environmental programs play out. But it goes overlooked when the government establishes environmental programs without deliberate variation, like experimentation, in how they are implemented.

“Human behavior is complex and thus very hard to predict. That’s why testing is so important,” said Ferraro.

Consider the real-life case of the Ogallala Aquifer, one of the world’s largest aquifers, which lies beneath the Great Plains in the western United States. Large-scale extraction for agriculture, which began in earnest after World War II, now threatens to deplete the aquifer that supplies drinking water to millions of people. Once depleted, the Ogallala will take thousands of years to replenish through rainfall.

As a solution, the U.S. government stepped in with new technology aimed at making water use more efficient for farmers. “The thinking was that if the farmers could use water more efficiently, they wouldn’t need to use so much water,” Ferraro said.

The flaw in this plan? “In their water savings calculations, planners held the farmers’ behavior constant,” he said. “The assumption was that farmers, after adopting the new technology, would behave the same way they had behaved before. Rather than run an experiment to test that assumption, the government spent a lot of taxpayer money to get as many farmers as possible to adopt the new techniques.”

In fact, according to observational data, the farmers did change their behavior. With more efficient water extraction, water use increased as farmers expanded their areas of irrigation and changed to more water-intensive crops.

“In the Ogallala case, through careful testing of the intervention, policymakers might have seen that their policy would have the opposite intended impact and that we shouldn’t spend taxpayer money on technology that would increase pressure on the aquifer,” Ferraro said.

Sometimes we can afford to wait

In their viewpoint article, Ferraro and his colleagues address — and diffuse — some key obstacles to increasing experimentation in environmental programs.

Among those obstacles: A belief that, when it comes to the accumulating damages of pollution or climate change, we can’t afford to delay taking action. And a related concern around feedback lags: We can’t measure the full effects of any environmental program, such as long-term climate impacts, for years.

The researchers note that for many environmental problems, “the culprit is human behavior, for which the desired changes can be measured on a shorter time scale.” Measuring short-term environmental indicators, such as energy consumption by households, could give early feedback on whether an intervention is working as intended.

“We believe that in many cases, experimentation embedded in program implementation will improve outcomes in the long run, even at the costs of some delay in the short run,” they note.

Ethical considerations: not black and white

Perhaps the most problematic concern associated with experimentation, at least superficially, surrounds the ethics of treating some people or species differently than others.

“But this concern presumes that people exposed to an environmental program are sure to benefit from it,” said Ferraro. He and his co-authors argue that, in fact, environmental organizations have an ethical obligation to understand the effects of untested programs before subjecting large groups of people to them, particularly vulnerable communities.

Ferraro offers the example of “brownfields,” tracts of land that have been polluted and then abandoned by industry. In cities like Baltimore, these areas are disproportionately found in low-income communities.

When deciding when and how to clean up these brownfields, Ferraro says, the argument is: How can you ethically create an experiment in which some vulnerable communities serve as the control group, with their neighborhoods left untouched by mitigation efforts?

His rejoinder: “It’s not necessarily clear that cleaning up brownfields is a ‘win-win’ for vulnerable populations — and in fact, it might make things worse for them.” Once communities are cleaned up, gentrification often follows. As amenities improve and the price of housing soars, low-income families can be pushed out, which can lead to loss of employment, food insecurity, and decreased access to health care.

“No one says, ‘I am going to expose the entire population of Maryland to an untested intervention,’ but from an ethical standpoint that’s what we’re doing,” Ferraro said. He added that even programs that don’t directly harm people or the environment may simply be ineffective, wasting scarce taxpayer money and delaying more impactful actions.

“If environmental organizations were guided by an ethical precept that required evidence before changing or scaling up a program, the science and practice of environmental protection would look different and be more successful,” write Ferraro and colleagues in their viewpoint article.

Making room for experimentation is crucial

The researchers note that scientists and environmental program directors can legitimately argue about the benefits and opportunity costs of allocating scarce time and financial resources to formal experimentation in environmental programs.

“Should half of the environmental programs include experimentation? Is 10 percent the right amount? Although the optimal share is debatable,” they write. “We believe that the current allocation of roughly 0 percent is suboptimal.”