Banning sodas from school vending machines, building walking paths and playgrounds, adding supermarkets to food deserts and requiring nutritional labels on restaurant menus: Such changes to the environments where people live and work are among the growing number of solutions that have been proposed and attempted in efforts to stem the rising obesity epidemic with viable, population-based solutions. But which of these changes actually make an impact?
To answer that question, many public health researchers take advantage of “natural experiments”—looking at people’s calorie consumption or physical activity levels, either comparing before and after a policy or environmental change, or comparing against a similar group of people not affected by that change. But not all natural experiments are created equal.
“Rigorous science is needed to evaluate these natural or quasi-experiments,” said Amy Auchincloss, PhD, an associate professor at the Drexel University School of Public Health who was a member of a research team that authored a new study published online ahead of print in Obesity Reviews: “Impact of policy and built environment changes on obesity-related outcomes: a systematic review of naturally occurring experiments.” The review was led by Stephanie Mayne, a doctoral student supervised by Auchincloss, and also co-authored by Yvonne Michael, ScD, an associate professor and associate dean for academic and faculty affairs in the Drexel University School of Public Health.
The Drexel team reviewed the state of the science on this topic, evaluating the results and methods of all previous such studies published in the medical literature, in particular:
- Which policies and built environment changes have been evaluated via natural- or quasi-experiments and what are the results from these studies?
- Are there issues of concern with the studies’ design, including methods of assessment?
- What are the limitations of these studies and areas where additional science is needed?
This is the first review that has examined the use of natural- or quasi-experiments to evaluate the efficacy of policy and built environment changes on obesity-related outcomes (body mass index, diet or physical activity). The review included PubMed (Medline) articles published 2005–2013; 1,175 abstracts and 115 papers were reviewed and ultimately 37 studies were included in the review.
The review identified certain types of interventions that are more successful than others in improving obesity-related outcomes, and identified areas where more research is needed to draw conclusions about obesity-related outcomes:
| Diet & Food Policy Changes |
Physical Activity Focused Changes |
|
Changes with strong impacts were ones that improved the nutritional quality of foods:
- Trans-fat bans
- Sugary food and beverage availability limits
- Higher-fat food availability limits
Changes that had smaller or no impacts in the research to date included:
- Nutritional information requirements
- Supermarkets built in underserved areas
|
Changes with stronger impacts included:
- Active transportation infrastructure improvements
- Changes studied after longer-term follow-up periods
More research is needed to look at physical activity effects (not just use of amenities) for built environment changes including:
- Park improvements
- Trails
- Active transportation infrastructure
|
The researchers noted that a common shortcoming in many studies is that they only measured process outcomes such as food purchases or use of bike/transit infrastructure, rather than measuring the desired health outcomes, such as weight loss.
“Research suggests that people will use new amenities like bike shares, and limit purchases of unhealthy foods in specific contexts like schools,” said Mayne. “But it is less clear whether these changes translate into overall improvements in diet and physical activity.”
Likewise, only a few studies directly assessed impacts on BMI or weight; thus, the authors concluded that evidence is lacking on whether environmental and policy modifications are successful in maintaining healthy weight or reducing excess weight.
A key value of a natural experiment is that it can narrowly focus on the direct impact of a change in policy or infrastructure on an affected population—making natural experiments an important way to check on what kinds of public policies and investments make real-world impacts on health, and to what degree. The authors concluded that more natural experiments are needed to strengthen the evidence base about obesity-related policies and interventions. They also recommend more natural experiments to explore whether the timing of a change or repeated exposure to the changed condition enhances or reduces impacts on obesity-related outcomes.
The authors generally found stronger results in studies that had longer follow-up periods after a policy change or other intervention.
Both Auchincloss and Michael are members of the Department of Epidemiology and Biostatistics as well as principal investigators of Drexel’s Urban Health & Design team, an interdisciplinary effort to evaluate the impact of design projects on public health. The team’s work encompasses multiple research projects that are embedded within community-based design projects in Philadelphia that are believed to impact health, such as an urban greenway, community gardens and a school playground.