Software detects and extracts text from within video frames, makes it searchable

Jan 02, 2013
Text written on signs captured within video frames can now be detected and extracted using software that will make video footage searchable. © Hemera Technologies/AbleStock.com/Thinkstock

As video recording technology improves in performance and falls in price, ever-more events are being captured within video files. If all of this footage could be searched effectively, it would represent an invaluable information repository. One option to help catalogue large video databases is to extract text, such as street signs or building names, from the background of each recording. Now, a method that automates this process has been developed by a research team at the National University of Singapore, which also included Shijian Lu at the A*STAR Institute for Infocomm Research.

Previous research into automated detection within images has focused mostly on document analysis. Recognizing background text within the complex scenes typically captured by is a much greater challenge: it can come in any shape or size, be partly occluded by other objects, or be oriented in any direction.

The multi-step method for automating text recognition developed by Lu and co-workers overcomes these challenges, particularly the difficulties associated with multi-oriented text. Their method first processes video frames using 'masks' that enhance the contrast between text and background. The researchers developed a process to combine the output of two known masks to enhance text pixels without generating . From the contrast-enhanced image, their method then searches for characters of text using an algorithm called a Bayesian classifier, which employs probabilistic models to detect the edges of each text character.

Even after identifying all characters in an image, a key challenge remains, explains Lu. The software must detect how each character relates to its neighbors to form lines of text—which might run in any orientation within the captured scene. Lu and his co-workers overcame this problem using a so-called 'boundary growing' approach. The software starts with one character and then scans its surroundings for nearby characters, growing the text box until the end of the line of text is found. Finally, the software eliminates false–positive results by checking that identified 'text boxes' conform to certain geometric rules.

Tests using sample video frames confirmed that the new method is the best yet at identifying video text, especially for text not oriented horizontally within the image, says Lu. However, there is still room for refinement, such as adapting the method to identify text not written in straight lines. "Document analysis methods achieve more than 90% character recognition," Lu adds. "The current state-of-the-art for video text is around 67–75%. There is a demand for improved accuracy."

Explore further: Innovative new supercomputers increase nation's computational capacity and capability

More information: Shivakumara, P., Sreedhar, R. P., Phan, T. Q., Lu, S. & Tan, C. L. Multioriented video scene text detection through Bayesian classification and boundary growing. IEEE Transactions on Circuits and Systems for Video Technology 22, 1227–1235 (2012). dx.doi.org/10.1109/TCSVT.2012.2198129

add to favorites email to friend print save as pdf

Related Stories

New surveillance camera system provides text feed

Jun 03, 2010

(PhysOrg.com) -- Scientists at the University of California in Los Angeles (UCLA) have developed a prototype surveillance camera and computer system to analyze the camera images and deliver a text feed describing ...

SAP, Open Text Pen Reseller Agreement

May 03, 2007

SAP and Open Text have signed a reseller agreement that will deliver enterprise content management software, enabling businesses to have data archiving platforms provided by Open Text.

Recommended for you

Forging a photo is easy, but how do you spot a fake?

Nov 21, 2014

Faking photographs is not a new phenomenon. The Cottingley Fairies seemed convincing to some in 1917, just as the images recently broadcast on Russian television, purporting to be satellite images showin ...

Algorithm, not live committee, performs author ranking

Nov 21, 2014

Thousands of authors' works enter the public domain each year, but only a small number of them end up being widely available. So how to choose the ones taking center-stage? And how well can a machine-learning ...

Professor proposes alternative to 'Turing Test'

Nov 19, 2014

(Phys.org) —A Georgia Tech professor is offering an alternative to the celebrated "Turing Test" to determine whether a machine or computer program exhibits human-level intelligence. The Turing Test - originally ...

Image descriptions from computers show gains

Nov 18, 2014

"Man in black shirt is playing guitar." "Man in blue wetsuit is surfing on wave." "Black and white dog jumps over bar." The picture captions were not written by humans but through software capable of accurately ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.