Research brings cloud costs back to earth

July 13, 2011

( -- Researchers from Swinburne University of Technology are looking for ways to reduce the high cost of internet data storage and retrieval in cloud computing.

While – which relies on remote, rather than local servers – offers almost unlimited capacity for data storage and processing, current usage charges mean the costs are expanding at the same near-limitless rate.  

Social media such as Facebook and Flickr are simple examples of cloud computing, but the drain on resources from these sites doesn't compare to the volumes of high-end data generated by the world’s research institutions, healthcare systems and industries.

Government agencies such as the Australian Taxation Office, Bureau of Statistics, and Treasury are all potential heavy users of cloud computing services, and the costs to them are high and rising. An estimated $1 billion could be saved if the Australian government develops a data centre strategy – the core for cloud computing – for the next 15 years.

This is why, using funding from an Australian Research Council Discovery Project Grant, researchers from Swinburne’s Centre for Computing and Engineering Software Systems (SUCCESS), are developing more cost-effective models for cloud computing’s heavy users.

Professor Yun Yang and Professor John Grundy (from Swinburne) and Dr. Jinjun Chen (now with the University of Technology, Sydney) have been exploring the management of raw data and intermediate data sets, which are generated from processing this initial data.

“The trade-off is going to be between storage cost and computation cost,” Professor Grundy said. “Finding this balance is complex, and there are currently no decision-making tools to advise on whether to store or delete intermediate datasets, and if to store, which ones.”

To overcome this, the researchers have developed a mathematical model which factors in the size of the initial datasets, the rates charged by the service provider and the amount of intermediate data stored in the specified time.

“The formula can be used to find the best deals for storing data in the cloud,” Professor Yang said.

They have also developed an Intermediate Data-dependency Graph (IDG) which helps users decide whether they are better off spending money on storage or computation for intermediate datasets.

“IDG records how each intermediate dataset is generated from the one before it and shows the generation relationship between them. This means if a deleted intermediate dataset needs to be regenerated, the IDG could find the nearest predecessor of the dataset. This can save computation cost, time and electricity consumption,” Professor Grundy said.

The researchers have been evaluating these solutions by simulating a pulsar survey used to crunch information from radio telescopes.

“Searching for pulsars – rapidly spinning stars that beam light – is a typical scientific application,” Professor Yang said. “It generates vast amounts of data – typically at one gigabyte per second. That data will be processed and may be reanalysed by astronomers all over the world for years to come.

“We used the prices offered by Amazon cloud’s cost model for this evaluation. For example, 15 cents per gigabyte per month for storage, and 10 cents per hour for computation.”

From one set of raw beam data collected by the telescope, the pulsar application generated six milestone intermediate datasets. The model generated three different cost scenarios. The minimum cost for one hour of observation data from the telescope and storing intermediate data for 30 days was $200; for storing no data and regenerating when needed, $1000; and for storing all intermediate data, $390.

This gave the researchers options for which data to keep, and which to delete. “We could delete the intermediate datasets that were large in size but with lower generation expenses, and save the ones that were costly to generate, even though small in size,” Professor Yang said.

These are only a few of the solutions the researchers have come up with so far. To cater to different sectors, the group is also working on models that will allow users to determine the minimum cost on-the-fly, and as frequently as they wish.

Explore further: How energy-efficient is cloud computing?

Related Stories

How energy-efficient is cloud computing?

October 8, 2010

( -- Conventionally, data storage and data processing are done at the user's own computer, using that computer's storage system and processor. An alternative to this method is cloud computing, which is Internet-based ...

Mitsubishi, Hitachi eye disc for cloud computing era

August 6, 2009

Hitachi Ltd., Mitsubishi Chemical Corp. and some other organizations plan to jointly develop a next-generation optical disc that can store 25 times more data than a Blu-ray Disc, with the aim of putting the technology into ...

Cloud computing: The good, the bad, and the ugly

November 24, 2010

A survey of 31 Cloud computing contracts from 27 different providers has found that many include clauses that could have a significant impact, often negative, on the rights and interests of customers.

New device may revolutionize computer memory

January 20, 2011

( -- Researchers from North Carolina State University have developed a new device that represents a significant advance for computer memory, making large-scale "server farms" more energy efficient and allowing ...

CeBIT 2011: Administration in the cloud

February 10, 2011

The emerging field of cloud computing is an interesting one, and not just for businesses. The field of public administration benefits from the technology as well. Fraunhofer Institutes are developing solutions to create such ...

Recommended for you

When words, structured data are placed on single canvas

October 22, 2017

If "ugh" is your favorite word to describe entering, amending and correcting data on the rows and columns on spreadsheets you are not alone. Coda, a new name in the document business, feels it's time for a change. This is ...

Enhancing solar power with diatoms

October 20, 2017

Diatoms, a kind of algae that reproduces prodigiously, have been called "the jewels of the sea" for their ability to manipulate light. Now, researchers hope to harness that property to boost solar technology.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.