Ecologists warn of overreliance on unvetted computer source code by researchers

May 17, 2013 by Bob Yirka report

(Phys.org) —A team of scientists, led by ecologist Lucas Joppa of Microsoft Research, has published a commentary piece in the journal Science, highlighting what they say is a growing problem in research efforts. They suggest that an overreliance on source code that has not been properly vetted is increasingly leading to incorrect research effort results.

The problem, Joppa et al, say, is that increasingly, researchers are relying on existing software to perform their research, despite the fact that no one has peer reviewed the software itself. It's a problem, they say, that is particularly troubling when big applications are used because small coding errors can be compounded. A rounding error in a spreadsheet generally won't cause much problem, they note, but when a rounding error is repeated over and over again, perhaps millions of times, it can lead to completely inaccurate results.

In a Podcast interview with Science, Joppa explains the problems with software use in research have come about mainly due to the software being written by researchers themselves, rather than by trained . Software written by one research group can very easily become the standard for use by many other groups, despite the fact that it has never been thoroughly tested to ensure it's giving accurate results.

He said another problem is that sometimes, there is a mismatch between equations that have been worked out by researchers and the way they are implemented in software. It can become truly problematic, he points out, when a catch-22 situation arises—when researchers use a to find answers to questions they have no other way to find, or verify. If it's the only way to get the answer, how do they know it's correct?

Resarchers for the current study pulled data from a survey conducted among fellow ecologists. It's a field, they note, that relies very heavily on big number-crunching applications. Among other findings, the team reports that just 8 percent of 400 scientists who responded reported validating results (from a black-box computer system) with more than one system.

The researchers don't just point out problems with the way is used in current research efforts; they offer ways to improve the situation as well. The first are the most obvious—make open-source and require it to be peer reviewed before journals will accept research articles based on their use. They also suggest journals could help by publishing more articles educating researchers about the problem and how to deal with it. Encouraging colleges and universities to educate students on the issue (and perhaps require more computer science courses) would be helpful too, they add.

Explore further: 'Halo' makers shed light on live-action series

More information: www.sciencemag.org/content/340/6134/814

Related Stories

Checking out open access

Jan 30, 2013

From Wikipedia to shareware, the Internet has made information and software more widely available than ever. At the heart of this explosion is the simple idea that information should be open and free for anyone. Yet with ...

Software glitch delays 660,000 tax refunds

Mar 14, 2013

The Internal Revenue Service says 660,000 taxpayers will have their refunds delayed by up to six weeks because of a problem with the software they used to file their tax returns.

Recommended for you

Watching others play video games is the new spectator sport

Aug 29, 2014

As the UK's largest gaming festival, Insomnia, wrapped up its latest event on August 25, I watched a short piece of BBC Breakfast news reporting from the festival. The reporter and some of the interviewees appeared baff ...

SHORE facial analysis spots emotions on Google Glass

Aug 28, 2014

One of the key concerns about facial recognition software has been over privacy. The very idea of having tracking mechanisms as part of an Internet-connected wearable would be likely to upset many privacy ...

User comments : 3

Adjust slider to filter visible comments by rank

Display comments: newest first

ArtflDgr
2.1 / 5 (7) May 17, 2013
What a joke...
i work in research computing, and i discuss this problem with close colleagues
but i am stuck in a bad office, with pc bs, politics and so on
so the prlblem is that management wont let us software engineers with 30 years experience and knowlege to do the work (as they ahve done for years in other places) and thanks to high functioning autism. ie. prior employers in wall street and others used my special abilities and liked them... go to medicine, and they do not get autism, and instead treat you like your retarded, not like you went to bronx science a year early have a 175 IQ and dont communicate as well... but can write programs like no one else can...

they dont want the fixes.
and management dont read the ideas in this article.
ie. the people he is appealing to, are not hte people who make the choices to make outcomes

i have software that grows multicellular mathematical soltuions
as i solved that problem in math... but i cant present it...
they would love it
Dark Lord Xenu
1 / 5 (4) May 17, 2013
DARK LORD XENU THINKS THIS IS NOT A JOKE. MY BREVITY IS THE SOUL OF WIT AND YOUR MOTHER.
thefurlong
2 / 5 (4) May 17, 2013
DARK LORD XENU THINKS THIS IS NOT A JOKE. MY BREVITY IS THE SOUL OF WIT AND YOUR MOTHER.

Oh dread lord Xenu, do not release thine anger upon yon hapless software engineer, though he hath shourne his body thetans from his vessel in pursuit of coding purity, I implore thee!