Network traffic anomaly detection

December 27, 2016, University of Electro-Communications
Basic architecture and procedures of the proposed algorithm. Credit: University of Electro-Communications

"Diagnosing unusual events (called "anomalies") in a large-scale network like Internet Service Providers and enterprise networks is critical and challenging for both network operators and end users," explain Hiroyuki Kasai from The University of Electro-Communications in Japan, and co-authors Wolfgang Kellerer Martin Kleinsteuber at the Technical University of Munich in Germany in a recent report. In their latest work they devise a computationally efficient and effective algorithm to identify network level anomalies by exploiting the state-of-the-art machine learning algorithms, especially the large-scale higher-order tensor tracking technique.

Kasai, Kellerer and Kleinsteuber describe their system as data flows from origin to destination along courses that cross at various links. Measuring the traffic volume of each flow is incredibly data intensive, so instead the researchers focus on the directly observable but coarse link matrix, for which they then need to identify how they can estimate the unobservable flow matrix for the full from the link matrix.

The researchers also avoid storage issues for large sets of archive data by developing the algorithm to operate online. They formulate their system with a latent structure of normal flows with noise, and they can then estimate abnormal flows as outlier sparse flows by leveraging sparse modelling.

As the researchers point out in their report, network anomalies can be caused by deliberate malicious operations, or misconfigurations and failures of network equipment, all of which are important to identify. They add, "Extensive numerical evaluations show that the proposed algorithm achieves faster convergence per iteration of model approximation, and better volume anomaly detection performance compared to state-of-the-art algorithms."

Explore further: Big data technique shrinks data sets while preserving their fundamental mathematical relationships

More information: Hiroyuki Kasai et al. Network Volume Anomaly Detection and Identification in Large-Scale Networks Based on Online Time-Structured Traffic Tensor Tracking, IEEE Transactions on Network and Service Management (2016). DOI: 10.1109/TNSM.2016.2598788

Related Stories

A friend of a friend is... a dense network

December 1, 2016

It's a familiar request in the digital age: one of your friends on social media has a friend who wants to be your friend. Frequent linking among friends of friends can cause a rapid increase in social network connectivity.

Recommended for you

Cryptocurrency rivals snap at Bitcoin's heels

January 14, 2018

Bitcoin may be the most famous cryptocurrency but, despite a dizzying rise, it's not the most lucrative one and far from alone in a universe that counts 1,400 rivals, and counting.

Top takeaways from Consumers Electronics Show

January 13, 2018

The 2018 Consumer Electronics Show, which concluded Friday in Las Vegas, drew some 4,000 exhibitors from dozens of countries and more than 170,000 attendees, showcased some of the latest from the technology world.

Finnish firm detects new Intel security flaw

January 12, 2018

A new security flaw has been found in Intel hardware which could enable hackers to access corporate laptops remotely, Finnish cybersecurity specialist F-Secure said on Friday.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.