Supercomputers could generate warnings for stock crashes

April 19, 2013 by Lisa M. Krieger

Powerful computers can wreak havoc on U.S. stock markets, creating hair-raising volatility and eroding investor confidence in the lightning-fast search for profit. But far more powerful computers could help save it.

High-speed trading now dominates U.S. stock markets, buying and selling in a fraction of the time that it takes to blink. Computers detecting rapid swings in prices and instantly reacting builds volatility and more trades, generating a sea of chaotic data and a vicious feedback loop that can lead to nightmares far worse than May 2010's infamous "flash crash."

Faster still is "Edison," a tended by Lawrence Berkeley Laboratory scientists in a former Wells Fargo Bank building in downtown Oakland, Calif. Edison-like computers could track ultrafast trading across the nation's many markets, detecting precursors to a crash - and sounding early warnings for regulators seeking to avert a gruesome economic wreck.

It would be like a NASCAR race yellow flag warning drivers to slow down, scientists say.

"If improved monitoring and regulation can build some greater trust in the market, everyone benefits," said David H. Bailey, director of the lab's new Center for Innovative Financial Technology, which is building a bridge that links and financial market communities.

Edison loves big data. Its idea of an average day is simulating a supernova explosion, measuring the expanding universe's rate of acceleration. Or modeling 150 years of Earth's future climate change - in three dimensions.

When fully deployed later this year, Edison will perform as many as 2 quadrillion operations a second. How big is that number? Two quadrillion cups of water would fill Lake Tahoe twice.

Tracking every trade, in real time, on every U.S. stock exchange? No big deal.

"That data size - we routinely do 10 times that much. Easily. It's a trivial matter," said Bailey, a leading figure in both high-performance scientific computing and computational mathematics.

Not so long ago, life was simpler. U.S. moved at a human pace, simply matching buyers with sellers. But now many exchanges take place. And more than half of all trading is done by high-speed computer "traders" that live their electronic lives in server parks.

Most agree that computer trading is good for the average investor because it's inexpensive. But it also triggers unpredictably large price swings - causing widespread Maalox moments. It's breeding distrust in the market.

The "flash crash" of 2010 was triggered by a single firm using algorithms to rapidly sell 75,000 futures contracts.

In moments, the Dow Jones industrial average fell more than 700 points, or almost 10 percent, and quickly recovered.

Since then, numerous mini-flash crashes and other anomalies have slapped around stock values and investor confidence.

"Electronic markets ... seem capable of impressively flaky behavior," said Lawrence Berkeley Lab's David Leinweber, a computer scientist, former investment manager and algorithmic trading specialist.

"We are lost in the jungle when it comes to our ability to understand ever-faster markets well enough to keep them safe, stable and secure," he said.

Regulators with weak and incompatible computer systems have set safeguards. One uses shutdown switches - "circuit breakers" - to halt all trading. A second, called "limit up, limit down," cancels trades outside a normal price range.

But these, asserted Leinweber, are like applying the rules of the road to aircraft. Slowing, rather than suddenly halting, markets is less traumatic.

In early 2011, using the lab's Cray XE6 "Hopper" supercomputer, Leinweber's team found that a supercomputer could use a recently identified measure to warn of a looming flash crash.

Called Volume-synchronized Probability of INformed trading, or VPIN, it detects an imbalance between buy and sell orders, and growing volatility, about 45 minutes before a crash. It reveals "flow toxicity," that is, when high-tech traders' computers generate so many buy or sell orders that it becomes all but impossible to match orders. Amid this volatility, one side will stop trading to stem its losses, causing a sudden drop of prices that triggers an avalanche of similar withdrawals called a "flash crash."

A second measure of market instability, the Herfindahl-Hirschman Index, or HHI, also rose sharply for some stocks, although not for others.

These two signals of instability - and that uneasy feeling that someone else has more information than you - do not give a clear direction of specific trades, said the lab's John Wu. So they're not likely to be exploited by someone hoping to jump in and profit.

But they might be useful "for regulators to impose some rules that might slow down the market so we don't get into a sort of feeding frenzy," he wrote in an email.

Market experts said they saw promise in the idea. Leinweber "is well situated to speak to the role that machine learning and artificial intelligence can play for detecting patterns in markets," said David Andre of San Francisco's Cerebellum Capital.

A venture firm leader sees Leinweber's work in the forefront of understanding market complexities. "Leveraging 'smart' machines to parse and extract signals from massive quantities of textual data is hard, and David's work has put him at the vanguard of the next wave of the alpha generation," said Roger Ehrenberg of the New York City-based venture firm IA Ventures.

Bryan Harkins, chief operating officer at Direct Edge, the fourth-largest stock exchange, was more cautious: "It is great that the academic community has chimed in - not that we choose any specific winning idea, but these different ideas and themes are woven into what we are seeing as we look to improve the market."

The Securities and Exchange Commission, which went before Congress to ask for a 2013 budget increase of $245 million to "strengthen oversight of market stability and expand the agency's information technology systems," has not commented on the lab's research.

U.S. Commodity Futures Trading Commission Commissioner Scott O'Malia supports their work, writing in response to publication of their paper: "I am certain that our national labs can improve our market surveillance capabilities and open our eyes to new and innovative means to improve our real-time oversight of market."

The scientists say they don't want Edison to do the SEC's work - the computer belongs to the U.S. Department of Energy.

But they hope to study the possibilities more, marking the path to the next generation of market surveillance.

"We have got to find a way to make the system work and monitor it," Bailey said. "If this science is better understood, perhaps another flash crash can be averted."

Explore further: US imposes new rules on high-speed traders


Related Stories

US imposes new rules on high-speed traders

July 26, 2011

US regulators on Tuesday announced new rules aimed at shedding light on the secretive industry of computerized high-speed trading, which has been blamed for destabilizing financial markets.

Metric predicts stock market 'flash' crashes

December 3, 2010

The May 6, 2010, stock market crash briefly erased almost $1 trillion in value and plunged the Dow Jones Industrial Average into its biggest intraday fall ever. The market recovered most of its losses within the hour, but ...

High-frequency stock trade risky, unfair: experts

September 21, 2012

The increasing power of computerized high-frequency trades on US markets has been assailed in Congress as dangerous and unfair, as pressure builds to reel in the powerful industry.

Study links ultrafast machine trading with risk of crash

February 19, 2012

( -- In the United States, ultrafast trading in financial markets between 2006 and 2011 was the underlying factor for over 18,000 extreme price changes, according to a new study. Neil Johnson, a professor in the ...

Wall Street's super traders come under fire

September 9, 2010

A handful of traders who master stock markets using ultra-fast computers may soon face a clampdown by US watchdogs as they try to prevent freak electronic glitches.

Algorithmic trading to replace humans in the stock market

September 14, 2011

( -- The UK Government’s Foresight panel, led by Dame Clara Furse, has released a working paper that points out that algorithmic trading, or high frequency trading, will soon replace human decision making ...

Recommended for you

Top takeaways from Consumers Electronics Show

January 13, 2018

The 2018 Consumer Electronics Show, which concluded Friday in Las Vegas, drew some 4,000 exhibitors from dozens of countries and more than 170,000 attendees, showcased some of the latest from the technology world.

Finnish firm detects new Intel security flaw

January 12, 2018

A new security flaw has been found in Intel hardware which could enable hackers to access corporate laptops remotely, Finnish cybersecurity specialist F-Secure said on Friday.


Adjust slider to filter visible comments by rank

Display comments: newest first

3.7 / 5 (3) Apr 20, 2013
Most agree that computer trading is good for the average investor because it's inexpensive.

It's still not a level playing field. An average investor might spend a thousand or so dollars buying a computer, connectivity, and a trading account. For perspective, large investors routinely spend hundreds of millions in exotic data links and custom processors to shave a few milliseconds off the time it takes to complete transactions. Then, the transactions are derivatives, something that sucks money out of the market, but which aren't useful unless you devote all your time to trading (even then, maybe not). Joe Six-Pack as an investor, is naively looking for something that reflects the potential future value of a company because of what it produces. When ever a company's stock loses value because of derivative induced instability, Joe loses. In some ways, computers have helped the average investor. In many other ways, not so.
1.7 / 5 (6) Apr 20, 2013
US stock marked is funded with narcodollars. A lot of Israelis deal in drugs and "intelligence" associates handle the laundering and their tribal buddies at the bench ensure the case is buried. With their dual nationality passports and connections it's a lucrative lifestyle. That's why the equity market is staying high even while the bond market is surging. Of course the two are incompatible but the banksters make the most $ from crashed markets. For every loser in a crash there is a winner. The MSM insures those stories remain only stories

In my opinion the quote system of stocks is a broken model, and unfilled orders create illiquidity which is behind these major crashes
3.7 / 5 (3) Apr 20, 2013
The article seems to ignore the obvious, that when the first hint of an "Edison" warning got out, the very crash it was warning about would then occur, everyone would scramble to sell before the event . . . causing the event.
1.6 / 5 (7) Apr 20, 2013
The article seems to ignore the obvious, that when the first hint of an "Edison" warning got out, the very crash it was warning about would then occur, everyone would scramble to sell before the event . . . causing the event.

PPT will stop trading after "signal" got it? There is only one direction allowed - upward. That`s because nowadays market is driven not by predictiions of real economy but by capital transfers genereted by money printing Central Banks of G7.
1 / 5 (4) Apr 20, 2013
The supercomputers can generate stock crashes too with their high frequency trading..
1.8 / 5 (5) Apr 20, 2013
The supercomputers can generate stock crashes too with their high frequency trading
Good point. And who could be using these capabilities to trade on impending movement? These markets have never been free.

Notice how every day the news makes a special point of assigning a reason for every move? Ask any pro and if you can get them to speak candidly they will tell you that they really have no idea what makes the markets move.
US stock marked is funded with narcodollars
They are far bigger than that. They were made that way when bush created the 401k system. The huge increase in size was necessary to immunize them from Unauthorized Players.

Only the People who created them for the Purpose, are allowed to manipulate them.
1.7 / 5 (6) Apr 20, 2013
Notice how every day the news makes a special point of assigning a reason for every move? Ask any pro and if you can get them to speak candidly they will tell you that they really have no idea what makes the markets move.
News is an accelerant, moving price toward the desired point. For example Americans don't care about Chechen wandering around freely, and the FBI coddles them for casus belli staged events. Citizens are expendable commodities useful for generating tales of suffering and death counts. FBI had a "drill" in Boston with exactly the scenario with which they trained the Chechens. In that way they could have claimed an innocent exercise was taking place, had they been found out prematurely

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.