Google gives itself a jolt of 'Caffeine'

Google is getting faster and more comprehensive with a new Web indexing system called "Caffeine."

Carrie Grimes, a Google software engineer, said the Internet search giant had completed the system which "provides 50 percent fresher results for Web searches" and the "largest collection of Web content we've offered."

"Whether it's a news story, a blog or a forum post, you can now find links to relevant content much sooner after it is published than was possible ever before," Grimes said in a blog post late Tuesday.

She said Google had built Caffeine to "keep up with the evolution of the Web and to meet rising user expectations."

"Searchers want to find the latest relevant content and publishers expect to be found the instant they publish," Grimes said.

She explained that Google would previously index the Web in layers "some of which were refreshed at a faster rate than others."

"To refresh a layer of the old index, we would analyze the entire Web, which meant there was a significant delay between when we found a page and made it available to you," Grimes said.

"With Caffeine, we analyze the Web in small portions and update our search index on a continuous basis, globally," she said. "As we find new pages, or new information on existing pages, we can add these straight to the index."

Caffeine processes hundreds of thousands of pages a second in parallel, Grimes said, and adds hundreds of thousands of gigabytes per day. "You would need 625,000 of the largest iPods to store that much information," she said.

Grimes said Google built Caffeine "with the future in mind."

"Not only is it fresher, it's a robust foundation that makes it possible for us to build an even faster and comprehensive search engine that scales with the growth of information online, and delivers even more relevant search results to you," she said.

(c) 2010 AFP