Team develops software for automatic summarization of long texts

Aug 07, 2014

While long-form writing, epic cinematic tales and hefty tomes have had something of a renaissance recently, the continued popularity of the so-called microblogging platform Twitter and other such tools highlights the fact that many people still like to be very succinct. The terse commentary, the abstract, the executive summary: all still favored by many of us at some time or in some context.

Moreover, who has the time to read long texts when a chunk of pithy sound bites is needed. Thankfully, researchers in India are developing new software that can make longwinded prose short and sweet.

Esther Hannah of St. Joseph's College of Engineering, in Sholinganallur and Saswati Mukherjee of Anna University, Guindy Campus, both in Chennai, have developed a classification-based summarization model that performs automatic summarization of . The direct application of the software will be to remove extraneous noise sentences from bulk text allowing much more efficient and faster text mining to be carried out. Of course, the same summarization would allow a reader to extract the salient points from any given text too. The team suggests that the automatic summarization is comparable to that which might be carried out by an expert editor in terms of removing the redundancy and irrelevance.

The team "trained" their software with 60% of the 105 English-language documents from the Document Understanding Conference (DUC-2002), checking and correcting errors the algorithm makes along the way and thus teaching the software what would be an appropriate summarization and what would not. They then tested the remainder at various levels of summarization – 10%, 20% and 30%. The system works well with grammatically well-constructed, even very long, documents, has problems if there are extensive mathematical and scientific symbols in the text.

Precision was optimal between 20 and 30% percent, which is a significant reduction in text length for parsing by text-mining software. They obtained a precision value of about 0.65, which is significantly better than fuzzy logic summarization , which scores around 0.47 and far better than Microsoft Word 2007 inbuilt summarization tool, which is a little over 0.46.

Explore further: Simple technique may help older adults better remember written information

More information: Esther Hannah, M. and Mukherjee, S. (2014) 'A classification-based summarization model for summarising text documents', Int. J. Information and Communication Technology, Vol. 6, Nos. 3/4, pp.292–308.

add to favorites email to friend print save as pdf

Related Stories

Software provides a clear overview in long documents

Jul 25, 2014

In the future, a software will help users better analyze long texts such as the documents for calls for bids, which are often more than one thousand pages long. Experts at Siemens' global research unit Corporate ...

Recommended for you

Wireless sensor transmits tumor pressure

3 hours ago

The interstitial pressure inside a tumor is often remarkably high compared to normal tissues and is thought to impede the delivery of chemotherapeutic agents as well as decrease the effectiveness of radiation ...

Tim Cook puts personal touch on iPhone 6 launch

4 hours ago

Apple chief Tim Cook personally kicked off sales of the iPhone 6, joining in "selfies" and shaking hands with customers Friday outside the company's store near his Silicon Valley home.

Team improves solar-cell efficiency

19 hours ago

New light has been shed on solar power generation using devices made with polymers, thanks to a collaboration between scientists in the University of Chicago's chemistry department, the Institute for Molecular ...

Calif. teachers fund to boost clean energy bets

19 hours ago

The California State Teachers' Retirement System says it plans to increase its investments in clean energy and technology to $3.7 billion, from $1.4 billion, over the next five years.

User comments : 0