Researchers investigate decision-making by physical phenomena

June 2, 2017 by Lisa Zyga, Phys.org feature
Experimental configuration of laser chaos-based reinforcement learning. Credit: Naruse et al.

(Phys.org)—Decision-making is typically thought of as something done by intelligent living things and, in modern times, computers. But over the past several years, researchers have demonstrated that physical objects such as a metal bar [video], liquids [paper], and lasers can also "make decisions" by responding to feedback from their environments. And they have shown that, in some cases, physical objects can potentially make decisions faster and more accurately than what both humans and computers are capable of.

In a new study, a team of researchers from Japan has demonstrated that the ultrafast, chaotic oscillatory dynamics in lasers makes these devices capable of making and reinforcement learning, which is one of the major components of machine learning. To the best of the researchers' knowledge, this is the first demonstration of ultrafast photonic decision making or reinforcement learning, and it opens the doors to future research on "photonic intelligence."

"In our demonstration, we utilize the computational power inherent in physical phenomena," coauthor Makoto Naruse at the National Institute of Information and Communications Technology in Tokyo told Phys.org. "The computational power of physical phenomena is based on 'infinite degrees of freedom,' and its resulting 'nonlocality of interactions' and 'fluctuations.' It contains completely new computational principles. Such systems provide huge potential for our future intelligence-oriented society. We call such systems 'natural Intelligence' in contrast to artificial intelligence."

In experiments, the researchers demonstrated that the optimal rate at which laser chaos can make decisions is 1 decision per 50 picoseconds (or about 20 decisions per nanosecond)—a speed that is unachievable by other mechanisms. With this fast speed, decision making based on laser chaos has potential applications in areas such as high-frequency trading, data center infrastructure management, and other high-end uses.

The researchers demonstrated the laser's ability by having it solve the multi-armed bandit problem, which is a fundamental task in . In this problem, the decision-maker plays various slot with different winning probabilities, and must find the slot machine with the highest winning probability in order to maximize its total reward. In this game, there is a tradeoff between spending time exploring different slot machines and making a quick decision: exploring may waste time, but if a decision is made too quickly, the best machine may be overlooked.

A key to the laser's ability is combining laser chaos with a strategy known as "tug of war," so-called because the decision-maker is constantly being "pulled" toward one slot machine or another, depending on the feedback it receives from its previous play. In order to realize this strategy in a laser, the researchers combined the laser with a threshold adjustor whose value shifts so as to play the with the higher reward probability. As the researchers explain, the laser produces a different output value depending on the threshold value.

"Let us call one of the slot machines 'machine 0' and the other 'machine 1'," said coauthor Songju Kim, at the National Institute for Materials Science in Tsukuba, Japan. "The output of the laser-based decision maker is '0' or '1.' If the signal level of the chaotic oscillatory dynamics is higher than the threshold value (which is dynamically configured), then the output is '0,' and this directly means that the decision is to choose 'machine 0.' If the signal level of the chaotic oscillatory dynamics is lower than the threshold value (which is dynamically configured), then the output is '1,' and this directly means that the decision is to choose 'machine 1.'"

The researchers expect that this system can be scaled up, extended to higher-grade machine learning problems, and lead to new applications of chaos in the field of artificial intelligence.

Explore further: Researchers show that an iron bar is capable of decision-making

More information: Makoto Naruse, Yuta Terashima, Atsushi Uchida, and Song-Ju Kim. "Ultrafast photonic reinforcement learning based on laser chaos." To be published. arXiv:1704.04379 [physics.optics]

Abstract
Reinforcement learning involves decision making in dynamic and uncertain environments, and constitutes one important element of artificial intelligence (AI). In this paper, we experimentally demonstrate that the ultrafast chaotic oscillatory dynamics of lasers efficiently solve the multi-armed bandit problem (MAB), which requires decision making concerning a class of difficult trade-offs called the exploration-exploitation dilemma. To solve the MAB, a certain degree of randomness is required for exploration purposes. However, pseudo-random numbers generated using conventional electronic circuitry encounter severe limitations in terms of their data rate and the quality of randomness due to their algorithmic foundations. We generate laser chaos signals using a semiconductor laser sampled at a maximum rate of 100 GSample/s, and combine it with a simple decision-making principle called tug-of-war with a variable threshold, to ensure ultrafast, adaptive and accurate decision making at a maximum adaptation speed of 1 GHz. We found that decision-making performance was maximized with an optimal sampling interval, and we highlight the exact coincidence between the negative autocorrelation inherent in laser chaos and decision-making performance. This study paves the way for a new realm of ultrafast photonics in the age of AI, where the ultrahigh bandwidth of photons can provide new value.

Related Stories

Quantum dots make efficient decisions

November 22, 2013

Even the simplest forms of life face an endless barrage of decisions—where to search for sustenance, for example, or how to avoid predators. Various mathematical models can mimic these decision-making processes, coming ...

How quantum effects could improve artificial intelligence

October 17, 2016

(Phys.org)—Over the past few decades, quantum effects have greatly improved many areas of information science, including computing, cryptography, and secure communication. More recently, research has suggested that quantum ...

Recommended for you

Information engine operates with nearly perfect efficiency

January 19, 2018

Physicists have experimentally demonstrated an information engine—a device that converts information into work—with an efficiency that exceeds the conventional second law of thermodynamics. Instead, the engine's efficiency ...

Team takes a deep look at memristors

January 19, 2018

In the race to build a computer that mimics the massive computational power of the human brain, researchers are increasingly turning to memristors, which can vary their electrical resistance based on the memory of past activity. ...

Artificial agent designs quantum experiments

January 19, 2018

On the way to an intelligent laboratory, physicists from Innsbruck and Vienna present an artificial agent that autonomously designs quantum experiments. In initial experiments, the system has independently (re)discovered ...

5 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

zave
1 / 5 (1) Jun 03, 2017
I wonder how much more effecient will the computers with this have when they make them?
Eikka
5 / 5 (1) Jun 03, 2017
So the system is basically an oscillatory analog of a scale, which counts wins from one machine in cup A, and the other in cup B, and tips in the direction of more wins as more games are played. A multi-choice version of the system would simply extend the scale on further dimensions, like a top that can tip in any direction between 0 to 360 degrees.

It is simply rhetorics to say the system decides anything, because it is a physical inevitability to reach a certain state - within the probabilities of the wins. You could just as well conclude that a pile of sand in a hourglass "knows" what time it is.

Dougsinc
not rated yet Jun 04, 2017
Eikka is right, there is nothing intelligent about this system, not even among the designers.
However, the article is useful in that it is an clear example of fake science in what appears to be responsible news outlet. Shame on you Lisa Zyga, Phys.org and ScienceX!
Spaced out Engineer
1 / 5 (1) Jun 04, 2017
Eikka, the dichotomy is far stranger. The information of the problem space correlates with the mechanistic instrument. From one perspective the question provides the information that constrains the light. From another there was only forever of fatalistic surreal branching. What is reported is a nonobjective quantitative model, from multiple minds, of their exchange.

Also some hourglasses have no grains of sand, but whose surfaces are hidden from every one. Do they tell time? No one seems to be able to look away, when there maybe no top. Just a drum whose hole and handles are too useful to be denied despite shape.

Where do you draw the line of organism with membrane and fluid? A body possesses correlates.
https://www.quant...0170119/
When efficiency is a proposition after condition and classical probability isn't the only lens.
Eikka
5 / 5 (1) Jun 05, 2017
Eikka, the dichotomy is far stranger. The information of the problem space correlates with the mechanistic instrument. From one perspective the question provides the information that constrains the light. From another there was only forever of fatalistic surreal branching. What is reported is a nonobjective quantitative model, from multiple minds, of their exchange.


Are you trying to pull off a new Sokal affair, or are you just playing with Markov Chains?

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.