MPEG hammers out codec that halves bit rate

Aug 15, 2012 by Nancy Owano report

(Phys.org) -- A new international standard for a video compression format was announced today. The draft was issued by the influential Moving Picture Experts Group (MPEG) which met in Stockholm in July. MPEG, formed by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC), drew 450 participants at the meeting, from 26 countries, representing telecoms, computer, TV and consumer electronics industries. MPEG discussions and standards affect these industries. In other words, the standard is a big deal.

“High-Efficiency Video Coding (HEVC) was the main focus and was issued as a Draft International Standard (DIS) with substantial compression efficiency over AVC, in particular for higher resolutions where the average savings are around 50 percent,” said Ericsson’s meeting notes.

In video alone, almost all digital terrestrial, satellite and cable TV services rely on video codecs that MPEG has standardized. The new standard issued this week is all about bandwidth, and the reduction thereof. This is a draft standard for High Efficiency Video Coding, to enable compression levels twice as high as the current H.264/AVC standard. The format may launch in commercial products next year.

The news is especially good for mobile networks, where spectrum is costly. Service providers will be able to launch more video services with the spectrum that is currently available.

“You can halve the bit rate and still achieve the same visual quality, or double the number of television channels with the same bandwidth, which will have an enormous impact on the industry,” said Per Fröjdh, Manager for Visual Technology at Ericsson Research, Group Function Technology.

He was a key figure at the event as chairman of the Swedish MPEG delegation. Ericsson, by nature of its business, is actively involved with MPEG. (Fröjdh’s Visual Technology team is working with MPEG in a new kind of 3-D format, which would do away with 3-D glasses. Fröjdh said the technology could be standardized by 2014.)

Anything to do with video compression over mobile broadband is a key concern to Ericsson, said another executive at Ericsson.

Out of all data sent over networks, a good proportion is video. According to statistics in a study dated this year, across all geographies, video is the leading driver of total data traffic on mobile networks at an average of 50 percent, but, in some networks, the data volume due to content is approaching 70 percent.

Explore further: Singapore moves to regulate taxi booking apps

Related Stories

Recommended for you

Singapore moves to regulate taxi booking apps

Nov 21, 2014

Singapore on Friday announced new rules for mobile taxi booking apps, including US-based Uber, in the latest move by governments around the world to regulate the increasingly popular services.

Protecting personal data in the cloud

Nov 20, 2014

IBM today announced it has patented the design for a data privacy engine that can more efficiently and affordably help businesses protect personal data as it is transferred between countries, including across private clouds.

Gift Guide: Dragons, aliens, heroes for the gamer

Nov 19, 2014

Sony's PlayStation 4 video-game console has built an impressive lead over its competitors. That's good news for holiday shoppers because it has driven Microsoft and Nintendo to offer more budget-friendly ...

User comments : 35

Adjust slider to filter visible comments by rank

Display comments: newest first

GSwift7
2.3 / 5 (17) Aug 15, 2012
in particular for higher resolutions where the average savings are around 50 percent


That's very cool. I wonder if there's an absolute limit to compression. We already surpass what we once thought of as the theoretical limit.

The format may launch in commercial products next year


Most people don't even notice things like this. Every single one of the people I work with would look at me like I had grown a third eyeball if I tried to tell them about this at lunch tomorrow.
Vendicar_Decarian
3.8 / 5 (18) Aug 15, 2012
Excellent. Twice the porn in the same space.
Eikka
1.9 / 5 (14) Aug 15, 2012
You can halve the bit rate and still achieve the same visual quality, or double the number of television channels with the same bandwidth


There's only the problem that sometimes you can't, because sometimes there's more actual information in the picture that you can't toss out, like panning through a scenery instead of just filming one spot. Or filming an outdoors scene and suddenly the wind starts to flutter thousands of tree leaves.

It's cases like these where the bandwidth runs out and you get a big glitch in the picture because the difference between compression efficiency in the scenarios is too large to fit your alloted bandwidth.

That's one thing I hate about digital television. It looks excellent most of the time, but every now and then it glitches, especially on live broadcasts that have to be encoded on the fly. Streaming media isn't affected as badly because the buffers are longer, but still you can see when they've gone over-zealous with the low bitrates.
chardo137
1 / 5 (14) Aug 15, 2012
Recently it has been shown that some meta-materials can have a negative index of refraction and may revolutionize the field of optics. We can only hope for meta-programming and a negative index of compression so that you could download a movie and end up with more room on your hard drive than when you started. Only kidding, this is very cool!
BIG COCK
2 / 5 (12) Aug 15, 2012
This is pretty great; still-image compression efficiency has essentially flatlined since the introduction of JPEG (JPEG2000 and other later formats were only small improvements and have yet to become mainstream, if ever) suggesting that image compression has hit a theoretical limit based on the mathematics currently employed. But video still keeps getting better and better even nowadays, with most of the improvement coming from better time-axis compression (motion estimation, etc).

A high-school level explanation for why this is the case is that still image compression requires capture of 2D features, whereas video compression requires capture of 3D features (two dimensions from still frames, and time adds a third dimension); the extra dimension in video makes things harder to perfect. Apparently 3D compression still has room for improvements like this!
BIG COCK
2 / 5 (12) Aug 15, 2012
That's very cool. I wonder if there's an absolute limit to compression. We already surpass what we once thought of as the theoretical limit.

There's always an absolute efficiency limit to any compression algorithm designed to compress things that aren't stupid. This may sound like a silly way to say it, but it's actually quite accurate: a compression system can be viewed as a map from one finite space of bit-sequences to another. If the output space is "smaller" than the input space (which it must be if it is actually compressing signals) then the map must not be injective. However, if you're a smart programmer, you design your map so that it is injective (ie, safe to use) on some subspace of "interesting" input signals.

If your compression was incredibly good, your output space would be tiny, and thus must correspond to a uselessly tiny subspace of "interesting" input signals. Therefore, no arbitrary strength compression scheme exists for signal spaces which aren't stupid.
Parsec
5 / 5 (2) Aug 15, 2012
in particular for higher resolutions where the average savings are around 50 percent


That's very cool. I wonder if there's an absolute limit to compression. We already surpass what we once thought of as the theoretical limit.

The format may launch in commercial products next year


Most people don't even notice things like this. Every single one of the people I work with would look at me like I had grown a third eyeball if I tried to tell them about this at lunch tomorrow.

Yes there is an absolute theoretical limit. The work of Shannon and Hoffman as well as many early compression pioneers were aimed at figuring out just that question.
eachus
1.6 / 5 (7) Aug 15, 2012
Yes there is an absolute theoretical limit. The work of Shannon and Hoffman as well as many early compression pioneers were aimed at figuring out just that question.


1) Yes, there is a limit.
2) It depends on the assumptions you make about the set of inputs to be compressed.
3) If the assumptions are wrong, there are two cases:
a) with lossless compression, the compressed signal is larger than the original.
b) with lossy compression, the information discarded includes necessary or meaningful data.

So what is going on here? This is a lossy algorithm, and the additional savings are from iterating the H.264 algorithms. In other words, the algorithm is choosing similar compressions for (subtiles) of the images, and using that in further compression.
Sanescience
1 / 5 (10) Aug 15, 2012
There have been some rigorous studies involving lossless codecs. But consumer grade codecs does not reproduce an image pixel for pixel.

Compression "limits" for so called "lossy" codecs are a tricky topic. Having worked with digital codecs it is not a simple "how low of a bit rate can they go" but also a "what quality visually is acceptable." So it becomes a "what you can get away with" engineering problem. For example you could compress a video stream down to a hand full of pixels, then stretch it back into fuzzy blobs onto any size display. Your bit rate would be miniscule, but cable subscribers wouldn't tolerate it.

With more powerful (esp. video) processors and memory you can do crazy stuff like arbitrary screen region shifts, scales, rotations, color operations, and whole and partial frame caching for use in future frames.

The size of the video stream then also depends on what kind of resources are placed on the encoding front side to find all the possible optimizations.
Sanescience
1.4 / 5 (11) Aug 15, 2012
@eachus: There are plenty of lossless codecs that reduce the data size of a video stream. (Note, a compressed signal isn't what is going on, signal tends to have a different specific meaning in AV)

Here is a bit of whacky compression "philosophy". If you had an entire video library stored in a device and your online connection consists of accessing the video provider for a video identification number and a decryption key, you could say that they have implemented a compression scheme that has reduced the movie down to a hand full of bytes, and its a "lossless" reproduction at that. Granted that doesn't quite fit a lot of what people think of as compression, but some basic forms of it are already being used by what we consider today's compression techniques (like a bit code for identifying a solid color screen that is obtained from the decoder box) But it's all just a black box to most people.
Deathclock
1.3 / 5 (14) Aug 15, 2012
in particular for higher resolutions where the average savings are around 50 percent


That's very cool. I wonder if there's an absolute limit to compression. We already surpass what we once thought of as the theoretical limit.


yes, there is, and I know this because it is impossible to represent a trillion bytes in a single byte, so that at least is a limit, if not the tightest limit.
Vendicar_Decarian
5 / 5 (3) Aug 15, 2012
Every movie can pretty much be compressed to the ascii content of it's title.

Exactly how is irrelevant detail.

"I know this because it is impossible to represent a trillion bytes in a single byte" - Deathclock
Deathclock
1.2 / 5 (14) Aug 15, 2012
Every movie can pretty much be compressed to the ascii content of it's title.

Exactly how is irrelevant detail.

"I know this because it is impossible to represent a trillion bytes in a single byte" - Deathclock


What are you talking about? If that were true you would implement it and become a millionaire overnight.

Unless you're talking about compression wherein almost all of the detail is lost... but you wouldn't be talking about that because that would be fucking stupid and have nothing at all to do with the topic of this conversation, which is compression that results in a file with a large majority of the fidelity of the original.
bluehigh
1.7 / 5 (12) Aug 16, 2012
Vendi is almost correct and likely relies on the comments of sanescience, who described the system in an earlier comment. Website Flash objects (among others) use this technique to extract an object from a library (a user dll) and embed a token in the data stream to retrieve the object rather than transfer contents in real time. i dont see in the article anything about fidelity. its more lossy, you just might not notice it, except sometimes.
antialias_physorg
5 / 5 (3) Aug 16, 2012
There's a lot of approaches towards compression. The major decision you want to make first is:
Lossy vs. no-loss compression.
Obviously lossy compression gives you a lot more leway, but you can do lossy compression and still get very god results.

E.g. for audio MP3 is a lossy compression format, but the loss is not noticeable (even by experts) because it makes use of psychoaccoustics (the human ear is incapable of hearing a sound if another sound thatis 6dB louder is played at the same time - so MP3 does not encode the fainter sound). The only way you can notice it is if you have a hearing defect (e.g. if you cannot hear the frequency of the loud sound at all - and so would notice the missing fainter sound...for example the missing 'tick' of a metronome in the background)

Most of todays' compression schemes are dynamic: They compress stuff that would be noticeable less and non-noticeable stuff more.
antialias_physorg
5 / 5 (2) Aug 16, 2012
I wonder if there's an absolute limit to compression.

There is (if you want loss-less comprssion). You can calculate the information content of the data stream you wish to compress. If you are not willing to accept losses then that gives you a lower limit for the number of bits you need. (If you're interested in how to calculate the information content of data go to wikipedia and search for: Shannon information theorem)

The bits can be distributed among the stream or the compression software (e.g. in the form of lookup tables), but you can't cheat that lower limit.

If you accept losses then there's really no limit how low you can go. You could go as low as one (or even zero bit) but that would just give you a black (or white) screen - whatever the original was. For lossy compression it's all about what type of quality you're willing to accept.
Husky
4 / 5 (2) Aug 16, 2012
what is the meaning of having ultra-hd screens if they pump lossy stream to it, a bit of loss is acceptable (or unnoticed) there has to be a point where it defeats the purpose of having all these extra pixels available, supposedly for unprecedented actual detail.
Eikka
1 / 5 (7) Aug 16, 2012
what is the meaning of having ultra-hd screens if they pump lossy stream to it


You need twice the pixel density of the smallest detail you want to show. Lossy stream or not. Otherwise you get aliasing errors, or you have to blur the picture which is a more noticeable error.
antialias_physorg
5 / 5 (2) Aug 16, 2012
where it defeats the point of having all these extra pixels available for detail.

As I said compression schemes can be dynamic.

The following a hypothetical compression algorithm. I know various algorithms (e.g. jpeg) use these methods but I don't know if any one uses exactly this combination:

Example: the scene on your HD monitor features a very 'busy' foreground (e.g. an amusement park scene) and also some not-so-busy background (e.g. night sky).
an algorithm can split the picture into regions (e.g. by an octree method) and assign each region a color palette based on the color range in the region. The foreground will get a large color palette (requiring many bits per pixel) the background not so many (requiring less bits per pixel). Voila: Compression without loss.

Other algorithms use (adaptive) wavelets/cosine transforms, reference frames and differential encoding and whatnot. The aim is always to get the most 'loss' where you don't notice it.
antialias_physorg
5 / 5 (1) Aug 16, 2012
Another thing to consider is the application you're compressing for.

For example images that will be scientifically processed, like the one from Curiosity, should not be compressed with a lossy method. For medical images lossy compression is downright forbidden (though the imaging/reconstruction method may be lossy itself and any filters used introduce information loss - but those are different issues altogether.)

Lossy compression always leads to artifacts in the image
E.g. early jpg methods led to block artifacts, of which you can see a good example here:
http://en.wikiped...artifact

A good picture of how increased compression reduces quality can be seen here
http://en.wikiped...iki/JPEG
As you can see, even in the compressed parts of about 2/3 along the picture artifacts become practically unnoticeable.

Compression is a fascinating subject - and I've never met anyone interestd in information theory who has NOT dabbled in it at one time or another.
GSwift7
1.4 / 5 (9) Aug 16, 2012
It seems that most of you do not understand modern compression very well. While some techniques do involve trimming information from the data before sending it, that's not all that's going on. When sending information to and from satellites, or from one internet hub to another, they cannot use compression that involves data loss. The ideas currently in use are rooted in the techniques of cryptography. It's like creating a cipher, where a small message can be a code that represents a longer one. You can even do this multiple times on the same data, reducing the bits without losing any information, as long as the person receiving your message has the cipher to decode your hidden message.

The theoretical limit may or may not come into play as the size of the cipher increases relative to the size of the average message you are encoding. There may be a point where the number of possible combinations you account for makes the number of digits in a single "word" of your code too long.
antialias_physorg
not rated yet Aug 16, 2012
the number of possible combinations you account for makes the number of digits in a single "word" of your code too long

There was actually an April Fool's article in Scientific American once that posited 'fractal compression'. The idea went like this: Since every sequence of numbers will eventually occur in a fractal (or in a transcendental number like Pi) all you need to do to compress is tell the other side the offset/coordinate and the length of the sequence (e.g. start at the digit 667834521 of Pi and use the next 2 megabytes and you have the image of an F16 over the grand canyon)

Of course this is BS because, while any sequence WILL occur in some mathematical constructs the information to say WHERE it occurs is, with an overwhelming likelyhood, longer than the information contained in the sequence (i.e transmitting the sequence itself would be shorter)

NB: Compression needs also be chosen for the noise characteristics of the channel. Especially when streaming data.
BLAST OF SHIT IN THE FACE
1.3 / 5 (12) Aug 16, 2012
BIG COCK
1.4 / 5 (10) Aug 16, 2012
Every movie can pretty much be compressed to the ascii content of it's title.

"I know this because it is impossible to represent a trillion bytes in a single byte"-Deathclock


What are you talking about? If that were true you would implement it and become a millionaire overnight.

This is an excellent example of what I meant when I said that arbitrary-strength compression must necessarily be "stupid". Vendicar's statement is essentially a restating of what Sanescience said. What Sanescience proposed is a scheme where the input space is tiny (simply a couple-byte digital pointer to a movie stored in some medium, rather than the data of the movie itself, which is gigabytes). The "compression" (if you are willing to call it that!) is thus essentially infinitely efficient, but necessarily stupid for the reasons I outlined earlier.

My earlier "proof" is vague, but can be made precise if mathematical rigor is desired.

Who the hell is "BLAST OF SHIT IN THE FACE"?
blengi
1 / 5 (8) Aug 16, 2012
One wonders how much content datawise is redundant to your average eyeball in a realtime setting. Given that detail outside the eye's fovea is less concerning and people aren't always attentive, perhaps there could be some sort compression standard for smart devices capable of tracking eye movements which returns/encodes data packets on the basis of what the user is looking at at any given moment. That is, weight bandwidth towards areas to focus, drop data during saccades/blinks/aversion, reduced data depending on distance from screen etc etc Might be a bit tricky with multiple viewers, and any lag would destroy the illusion but heck might be worth it...
antialias_physorg
5 / 5 (1) Aug 16, 2012
One wonders how much content datawise is redundant to your average eyeball in a realtime setting


The average bitrate processed by the brain from the eyes is surprisingly low. I do seem to remember reading that it's 'merely' something like 60-75kByte per second.

While there are a lot of nerve cells in the eye (about 160million cells) and each of these cells can fire several times per second they are highly integrated (meaning that there are clusters where the information from cells get summed, edge detection is processed and the like). So your brain doesn't really get a 'bitmap' of what the eye sees but a highly integrated set of information from which it then reconstructs the mental image. There's a lot of fill-in-the-blanks (spatially and temporally) happening.

The average CONSCIOUS processing speed (i.e. information gained via conscious processing of input) is quite a bit lower. Usual numbers given in literature are in the 100 bit to 1k bit per second range.
GSwift7
1.4 / 5 (10) Aug 16, 2012
The "lossy" types of compression are usually based on a math trick (fourier transform) where you convert the digital signal to the equivalent of an analog wave form. Then you find a limited number of overlapping simple waves which create a close fit to the original complex wave. Then you can just send instructions on how to re-create those simple waves at the receiving end. It's not really eliminating unneeded information, rather you create new information that 'looks' a lot like the original information. NONE of the actual information in the original survives though.

For those with a curious mind:
http://en.wikiped...ransform
antialias_physorg
not rated yet Aug 16, 2012
Wavelets need not lose any more information than any other method. Depending on how many wavelets you use you can do lossless wavelet compression.

Transforming (an image) to Fourier space and back is mathematically loss-free

Matching it to wavelets is lossy if you don't use enough sets of them - but then again: the color space/bit depth of your original image and the resolution of the pixels are already a subsampling of what it "really there".

That said: The point of wavelet transform is compression. If you'd use enough wavelets to get a bit-perfect representation of the original then the wavelet data will often be exactly as large as the original data (so: no compression).

But wavelets do allow for very little apparent information loss at very good compression rates in a wide variety of applications (not so much in others)

Which compression scheme is best really depends on what kind of data you have: smooth, periodic, block noisy, gaussian noisy, transient, ...
Vienna
1.1 / 5 (12) Aug 16, 2012
Apparently none of the commenters here explaining the issues at hand have actually worked in high level video production for broadcast and high definition formats.

Having worked in television and video in ntsc and pal formats before the digitization and codec issues were needed I can tell you that I can easily detect the visual degradation in a satellite, cable, or even DVD high definition video -- what you experts are claiming is "undetectable".

I still produce high definition video in today's "professional" codecs.

ALL CODECS are LOSSY. They are "dumb" as to what the human eye can or can not detect. It is humans themselves who guide the development of a video codec. In the development of a new video codec, from what I can very plainly see, the codec must ALWAYS SACRIFICE quality. EVERY SINGLE FRAME is missing serious chunks of information that would be easy to spot if the original quality of video were compared before and after the codec is applied.
Vienna
1 / 5 (10) Aug 16, 2012
The biggest joke to me is that "high definition" video is so far, far from what it was intended to convey when the standard was proposed years ago. Just like the quality of "digital" music formats, what kids have now listened to all their young lives is a chopped up codec upon codec degradation of the original music track in a lossless format.

But in both video and audio manufacturing today the LOSSY Codecs are applied to the content even as it is burned into a disc. What they burn in is already MISSING huge chunks of video or audio data.

This is NOT like a "code" where a string of characters ultimately represent only one distinct value such as "c". Once your video is processed by a video codec it is "blowing up" one bit of information and telling the video card to smear it across a bunch of pixels because the viewer supposedly cannot tell the difference.

I CAN. Easily. Even without seeing the original footage. And then, folks, to get to your home the provider strips out even more!
Vienna
1.7 / 5 (12) Aug 16, 2012
Furthermore, the compression is done on the fly when you watch an HD television program. To squeeze more channels into their bandwidth spectrum the provider, such as SKY or HBO, etc, squeeze the video even more, de-coding and re-encoding it even further to get it to the viewer.

I routinely see "HD" (lie) programs where the dynamic decoding and encoding is "misinterpreting" the data. Whole backgrounds will suddenly shift, even distorting a wall's shape or lines, patterns, or textures for seconds at a time because the processing gear "decided" for itself what was "relevant" or "unimportant".

I will see people's faces suddenly lose all pores and texture for one second or more, suddenly as blandly featureless as a porcelain doll, and then immediately "regain" their skin texture and pores.

Today's HD programming is a hodge-podge of constantly shifting line and texture distortions, artifacts, "stuttering", "fluttering", etc.

HD is a big lie. This is not the quality promised us!
GSwift7
1.8 / 5 (11) Aug 16, 2012
to Vienna:

Yes, good points, all of them. However, there just isn't a market demand for that right now. People get what they pay for, and only pay for what they feel is worth the price. Mass distribution is at the mercy of the weakest links needed to reach enough people to make a profit.

BTW, I can't speak for anyone else, but when I talk about loss-less compression, I am not talking about video or audio compression like MPEG. I was talking about data compression that is used in internet backbone and data centers and such. They must use zero loss compression because they are dealing with other people's data, such as banking transactions, etc. They don't use fourier transform on that, just multi-pass ciphers, as far as I know, which are true zero loss compressions.
Torbjorn_Larsson_OM
4 / 5 (3) Aug 16, 2012
@ "Who the hell is "BLAST OF SHIT IN THE FACE"?"

What a fanboi gets when the BIG COCK turns to other business?

As humor goes I don't think he or she nailed it, but it was an amusing response to a bad handle.

@ Vienna:

"This is not the quality promised us!"

Yes, yes, we heard that with audio, we will hear it when people colonize Mars ("not the planet promised us!") and I bet it was heard when fire was invented ("not the cooking promised us!").

No one promised any specific quality (how could they?), they promised an experience that would sell better on the markets compared to earlier products. Codecs improves both quality and cost usually, because you would have to wait for the data otherwise and/or pay more for the hardware.

baudrunner
1 / 5 (10) Aug 16, 2012
The human eye resolves continuity out of thirty frames per second. This means that there is a time delay between frame loading (called the "hammerhead" in the old days of TV) during which calculations can be done, and in fact, unrelated data transmissions were (are) taking place in that black space between frame refresh. The faster the processor, the more complex the calculation can be. Furthermore, in the seventies, Philips introduced a "sharpness" control that effectively inserted black "pixels" between two "pixels", discarding a pixel in the process. This actually provided the illusion of a sharper picture. The same could be applied to compression techniques, ie. averaging pixel attributes using an algorithm contained in the information header of the video file between successive frames, with the objective of reducing the size of the transmitted file.
david_king
1 / 5 (10) Aug 19, 2012
Why can't everybody just adopt their own cat (or pussy) and leave the bandwidth for junk emails?

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.