Ecologists warn of overreliance on unvetted computer source code by researchers

May 17, 2013 by Bob Yirka report

( —A team of scientists, led by ecologist Lucas Joppa of Microsoft Research, has published a commentary piece in the journal Science, highlighting what they say is a growing problem in research efforts. They suggest that an overreliance on source code that has not been properly vetted is increasingly leading to incorrect research effort results.

The problem, Joppa et al, say, is that increasingly, researchers are relying on existing software to perform their research, despite the fact that no one has peer reviewed the software itself. It's a problem, they say, that is particularly troubling when big applications are used because small coding errors can be compounded. A rounding error in a spreadsheet generally won't cause much problem, they note, but when a rounding error is repeated over and over again, perhaps millions of times, it can lead to completely inaccurate results.

In a Podcast interview with Science, Joppa explains the problems with software use in research have come about mainly due to the software being written by researchers themselves, rather than by trained . Software written by one research group can very easily become the standard for use by many other groups, despite the fact that it has never been thoroughly tested to ensure it's giving accurate results.

He said another problem is that sometimes, there is a mismatch between equations that have been worked out by researchers and the way they are implemented in software. It can become truly problematic, he points out, when a catch-22 situation arises—when researchers use a to find answers to questions they have no other way to find, or verify. If it's the only way to get the answer, how do they know it's correct?

Resarchers for the current study pulled data from a survey conducted among fellow ecologists. It's a field, they note, that relies very heavily on big number-crunching applications. Among other findings, the team reports that just 8 percent of 400 scientists who responded reported validating results (from a black-box computer system) with more than one system.

The researchers don't just point out problems with the way is used in current research efforts; they offer ways to improve the situation as well. The first are the most obvious—make open-source and require it to be peer reviewed before journals will accept research articles based on their use. They also suggest journals could help by publishing more articles educating researchers about the problem and how to deal with it. Encouraging colleges and universities to educate students on the issue (and perhaps require more computer science courses) would be helpful too, they add.

Explore further: NIST improves tool for hardening software against cyber attack

More information:

Related Stories

NIST improves tool for hardening software against cyber attack

November 23, 2011

( -- Computer scientists at the National Institute of Standards and Technology (NIST) have dramatically enlarged a database designed to improve applications that help programmers find weaknesses in software. This ...

Checking out open access

January 30, 2013

From Wikipedia to shareware, the Internet has made information and software more widely available than ever. At the heart of this explosion is the simple idea that information should be open and free for anyone. Yet with ...

Software glitch delays 660,000 tax refunds

March 14, 2013

The Internal Revenue Service says 660,000 taxpayers will have their refunds delayed by up to six weeks because of a problem with the software they used to file their tax returns.

Recommended for you

Inferring urban travel patterns from cellphone data

August 29, 2016

In making decisions about infrastructure development and resource allocation, city planners rely on models of how people move through their cities, on foot, in cars, and on public transportation. Those models are largely ...

How machine learning can help with voice disorders

August 29, 2016

There's no human instinct more basic than speech, and yet, for many people, talking can be taxing. 1 in 14 working-age Americans suffer from voice disorders that are often associated with abnormal vocal behaviors - some of ...

Auto, aerospace industries warm to 3D printing

August 25, 2016

New 3D printing technology unveiled this week sharply increases the size of objects that can be produced, offering new possibilities to remake manufacturing in the auto, aerospace and other major industries.


Adjust slider to filter visible comments by rank

Display comments: newest first

2.1 / 5 (7) May 17, 2013
What a joke...
i work in research computing, and i discuss this problem with close colleagues
but i am stuck in a bad office, with pc bs, politics and so on
so the prlblem is that management wont let us software engineers with 30 years experience and knowlege to do the work (as they ahve done for years in other places) and thanks to high functioning autism. ie. prior employers in wall street and others used my special abilities and liked them... go to medicine, and they do not get autism, and instead treat you like your retarded, not like you went to bronx science a year early have a 175 IQ and dont communicate as well... but can write programs like no one else can...

they dont want the fixes.
and management dont read the ideas in this article.
ie. the people he is appealing to, are not hte people who make the choices to make outcomes

i have software that grows multicellular mathematical soltuions
as i solved that problem in math... but i cant present it...
they would love it
Dark Lord Xenu
1 / 5 (4) May 17, 2013
2 / 5 (4) May 17, 2013

Oh dread lord Xenu, do not release thine anger upon yon hapless software engineer, though he hath shourne his body thetans from his vessel in pursuit of coding purity, I implore thee!

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.