Research brings cloud costs back to earth

Jul 13, 2011

(PhysOrg.com) -- Researchers from Swinburne University of Technology are looking for ways to reduce the high cost of internet data storage and retrieval in cloud computing.

While – which relies on remote, rather than local servers – offers almost unlimited capacity for data storage and processing, current usage charges mean the costs are expanding at the same near-limitless rate.  

Social media such as Facebook and Flickr are simple examples of cloud computing, but the drain on resources from these sites doesn't compare to the volumes of high-end data generated by the world’s research institutions, healthcare systems and industries.

Government agencies such as the Australian Taxation Office, Bureau of Statistics, and Treasury are all potential heavy users of cloud computing services, and the costs to them are high and rising. An estimated $1 billion could be saved if the Australian government develops a data centre strategy – the core for cloud computing – for the next 15 years.

This is why, using funding from an Australian Research Council Discovery Project Grant, researchers from Swinburne’s Centre for Computing and Engineering Software Systems (SUCCESS), are developing more cost-effective models for cloud computing’s heavy users.

Professor Yun Yang and Professor John Grundy (from Swinburne) and Dr. Jinjun Chen (now with the University of Technology, Sydney) have been exploring the management of raw data and intermediate data sets, which are generated from processing this initial data.

“The trade-off is going to be between storage cost and computation cost,” Professor Grundy said. “Finding this balance is complex, and there are currently no decision-making tools to advise on whether to store or delete intermediate datasets, and if to store, which ones.”

To overcome this, the researchers have developed a mathematical model which factors in the size of the initial datasets, the rates charged by the service provider and the amount of intermediate data stored in the specified time.

“The formula can be used to find the best deals for storing data in the cloud,” Professor Yang said.

They have also developed an Intermediate Data-dependency Graph (IDG) which helps users decide whether they are better off spending money on storage or computation for intermediate datasets.

“IDG records how each intermediate dataset is generated from the one before it and shows the generation relationship between them. This means if a deleted intermediate dataset needs to be regenerated, the IDG could find the nearest predecessor of the dataset. This can save computation cost, time and electricity consumption,” Professor Grundy said.

The researchers have been evaluating these solutions by simulating a pulsar survey used to crunch information from radio telescopes.

“Searching for pulsars – rapidly spinning stars that beam light – is a typical scientific application,” Professor Yang said. “It generates vast amounts of data – typically at one gigabyte per second. That data will be processed and may be reanalysed by astronomers all over the world for years to come.

“We used the prices offered by Amazon cloud’s cost model for this evaluation. For example, 15 cents per gigabyte per month for storage, and 10 cents per hour for computation.”

From one set of raw beam data collected by the telescope, the pulsar application generated six milestone intermediate datasets. The model generated three different cost scenarios. The minimum cost for one hour of observation data from the telescope and storing intermediate data for 30 days was $200; for storing no data and regenerating when needed, $1000; and for storing all intermediate data, $390.

This gave the researchers options for which data to keep, and which to delete. “We could delete the intermediate datasets that were large in size but with lower generation expenses, and save the ones that were costly to generate, even though small in size,” Professor Yang said.

These are only a few of the solutions the researchers have come up with so far. To cater to different sectors, the group is also working on models that will allow users to determine the minimum cost on-the-fly, and as frequently as they wish.

Explore further: MU researchers develop more accurate Twitter analysis tools

Provided by Swinburne University of Technology

not rated yet
add to favorites email to friend print save as pdf

Related Stories

How energy-efficient is cloud computing?

Oct 08, 2010

(PhysOrg.com) -- Conventionally, data storage and data processing are done at the user's own computer, using that computer's storage system and processor. An alternative to this method is cloud computing, ...

Mitsubishi, Hitachi eye disc for cloud computing era

Aug 06, 2009

Hitachi Ltd., Mitsubishi Chemical Corp. and some other organizations plan to jointly develop a next-generation optical disc that can store 25 times more data than a Blu-ray Disc, with the aim of putting the technology into ...

Cloud computing: The good, the bad, and the ugly

Nov 24, 2010

A survey of 31 Cloud computing contracts from 27 different providers has found that many include clauses that could have a significant impact, often negative, on the rights and interests of customers.

New device may revolutionize computer memory

Jan 20, 2011

(PhysOrg.com) -- Researchers from North Carolina State University have developed a new device that represents a significant advance for computer memory, making large-scale "server farms" more energy efficient and allowing ...

CeBIT 2011: Administration in the cloud

Feb 10, 2011

The emerging field of cloud computing is an interesting one, and not just for businesses. The field of public administration benefits from the technology as well. Fraunhofer Institutes are developing solutions ...

Recommended for you

Chameleon: Cloud computing for computer science

Aug 26, 2014

Cloud computing has changed the way we work, the way we communicate online, even the way we relax at night with a movie. But even as "the cloud" starts to cross over into popular parlance, the full potential ...

User comments : 0