Broader impacts in NSF's Division of Environmental Biology leave hope for improvement

Since 1997, researchers applying for grant funds from the National Science Foundation (NSF) have been asked to give an account of the broader societal effects of their proposed research. The Broader Impacts Criterion was intended as a supplement to the traditional "intellectual merit" criterion, with the hope of creating an incentive for principal investigators to consider and enhance the further-reaching benefits of their work.

In an article to be published in the April 2015 issue of BioScience, conservation ecologist Sean M. Watts, of the University of Washington, and his colleagues describe the results of an investigation into the broader impacts activities outlined in proposals made to the NSF's Division of Environmental Biology (where one of the authors is employed). The authors examined a total of 596 proposals and arrived at three major findings:

1. Publicly available abstracts often failed to reflect the broader impacts activities, so "the public at large might easily conclude that the [criterion] is not being implemented."
2. Past reviewers often failed to comment on the broader impacts activities outlined during the review process.
3. Project reports often lacked sufficient detail to measure the principal investigators' performance against the criterion.

One category of proposed broader impacts activities stood out: Watts and his colleagues discovered that efforts to bolster the participation of members of underrepresented groups were comparatively rare. Perhaps even more worrisome, "the [principal investigators] proposed more than twice the number of underrepresented [activities] than they subsequently reported." In contrast, among the other categories, the number of proposals roughly matched the number of reported outcomes. According to the authors, this may "provide further evidence that the underrepresented category, in particular, is more fundamentally challenging than were the teaching, infrastructure, and dissemination categories."

Despite these troubling trends, efforts are under way to improve adherence to and the usefulness of the Broader Impacts Criterion. A 2013 revision of the NSF's Proposal Guide discourages principal investigators from treating broader impacts activities as a rote checklist, in part by requiring that they be assessed for their novelty, impact, and feasibility. The authors see cause for hope in the new Proposal Guide: "If its requirements are well-implemented, they will bring much needed recognition to a generation of scientists who have toiled to engage society despite limited incentive from their peers, sponsoring institutions, or the review process."