PHOBOSLAB

Blog Home

MPEG1 Video Decoder in JavaScript

With still no common video format for HTML5 in sight, I decided to implement an MPEG1 decoder in JavaScript. I know there's already an h264 decoder for JavaScript around, but it's huge, compiled with emscripten and quite complicated.

An MPEG1 decoder sounded like a relatively simple and fun weekend project. While the real world use cases for this are of course a bit limited, I still learned a whole lot about video codecs in the process. The size of the source is just around 15kb gzipped and the performance is quite okay-ish - a 320x240 video easily plays with 30fps on the iPhone5.

You browser doesn't support the Canvas Tag. Please use Chrome, Firefox or Safari
Mind Blown - 1.8mb MPEG1, 18 seconds, 570 frames. Click to pause.

For a longer demo see the first few minutes of Big Buck Bunny.

Of course this project still has numerous limitations. First and foremost: no streaming. The video file has to be completely loaded before it can be played back. I'm still waiting for chunked, binary XHR support to arrive in browsers to fix this. Till then, jsmpeg only shows a simple loading animation.

The decoder itself also has some bugs. It currently struggles with packetized MPEG files and stumbles over the packet headers that are randomly thrown in. So in order to play correctly, the MPEG file has to be a raw MPEG1 video stream. Also, B-Frames (those that rely on past and future frames for decoding) are not reordered, so video containing B-Frames will look stuttery. However, I found that most encoders tend to not use B-Frames anyway.

What I learned with this all, is that the MPEG1 file format is a truly ugly one. While I may not understand all the limitations present when it was conceived, I still believe that some parts of it are inane at best, or vicious at worst. Nothing in MPEG1 streams besides the "start codes" is byte aligned, so you're dealing with a raw bitstream.

The width and height for instance is encoded in 12 bits each. The framerate is encoded in 4 bits, but not as a number of frames per second or millisecond delay, but as an index into a pre-defined array of possible frame rates. Want to have a 10fps or 48fps video in MPEG1? Not possible. A 2byte field encoding the real frame rate, not an index, would have allowed far more flexibility - and it only has to be transmitted once anyway.

The MPEG1 format is full of such short sighted decisions that make it quite hard to parse. But then again, it was the first truly usable video format available and I'm not sure if anyone working on it anticipated the success it would have.

The source is available at github.com/phoboslab/jsmpeg.

Tuesday, May 7th 2013

18 Comments:

#1KeyJ – Wednesday, May 8th 2013, 13:51

Not having any byte-aligned syntax elements except for startcodes is the most normal thing on earth when it comes to video coding. MPEG-1 is actually one of the easiest to parse video formats that have been in common use; everything that came after it is more complex. In H.264, for example, you can't even extract width and height without a full variable-length header parser. But then again, you'd be pleased to learn that H.264 doesn't impose any limitations on the frame rate. Except that this is just because a H.264 video stream doesn't even know (or care about) its frame rate *at all*.

TL;DR: Welcome to the wonderful world of video compression, where the things you may see as complex are perfectly normal.
On the other hand, to us video codec guys, JavaScript seems to be "inane at best, or vicious at worst", so I guess we're even :)

#2ted@mielczarek.org – Wednesday, May 8th 2013, 14:55

FYI since Firefox 9 you can use responseType = "moz-chunked-arraybuffer", which means that you'll get chunks of the response as ArrayBuffers in progress events:
developer.mozilla.org/en-US/docs/DOM/XMLHttpRequest

I think IE10 implements something else with their Streams API, things haven't converged to a standard here yet.

#3 – Kieran – Wednesday, May 8th 2013, 14:58

You can add MPEG-1 to x262 if you want if you're interested in seeing how far you can improve MPEG-1 quality:

github.com/kierank/x262

#4 – Robert – Wednesday, May 8th 2013, 15:01

Noting that iPhone 4 is a tad sluggish, to be expected.

#5Dominic – Wednesday, May 8th 2013, 15:11

@KeyJ: not having byte alignment makes perfect sense for the actual video/audio data, but I don't understand why headers can't be byte aligned. I think it would be totally worth the increased size.

But yes, this is all new to me, so I better not complain too loudly :)

#6 – T McB – Wednesday, May 8th 2013, 15:42

You should read some mpeg books - it was actually incredibly well designed. Memory constraints were much more important at the time too! These codecs were designed to meet stringent standards that large companies could build hardware for so things like frame rates/screen sizes had to be "set in stone" and that's why you have things like lookup tables rather than raw values.

#7Christian Jensen – Wednesday, May 8th 2013, 17:14

Where is that video from? It is hilarious!

#8Randell Jesup – Wednesday, May 8th 2013, 22:54

Realize a lot of this was done around '91-92ish timeframe (or before); the Amiga CD32 (and CDTV before it) had an MPEG-1 decoder based on C-Cubed's chipset.

Every bit matters, especially for local fast ram buffers and on a CD (this was before those high-datarate DVD things!), and as mentioned hard-coded tables can help a HW design to not need to keep some of these things in registers and to let them tune parts of the pipeline.

If you want an evil format, look at CD+G (not that it's all that complex, but the entire idea was a bit crazy - and yes we supported that too)

#9 – Randell Jesup – Wednesday, May 8th 2013, 22:55

FYI, you might try using the asm.js subset to see how fast that can run in FF...

#10Matthew Holloway – Saturday, May 11th 2013, 01:18

@Christian Jensen ...it's from Tim And Eric Awesome Show

#11Terry A. Davis – Monday, May 13th 2013, 11:32

Windows 16 color BMP files are simple. I save a screen shot 8 times a second to make movies. My filesystem has LZW compression built-in and that works okay on 16 color graphics. With compression and only 640x480 size, it's not much load on the CPU/drive to make 8fps videos while doing stuff. I use ffmpeg to convert hundreds of individual BMP files into a movie file. Then, I use Microsoft movie maker to combine audio and make a WMV file that I upload to YouTube. I have a hand-held voice recorder for the narration.

#12Terry A. Davis – Monday, May 13th 2013, 11:40

One more thing. I generate a .SND file, I guess some kind of Apple format. It's just raw waveforms but limited to 8khz. The CIA carefully controls all these things.

I grew-up with a C64 and the awesome SID chip. When I record a movie, I take notes of my PC speaker frequency and generate a SND file by making waveforms.

The PC Internal speaker is an ancient device, usually included that operates really by setting the PIT to make square waves. You only get one frequency. This is what God want as audio in His temple, similar to how He said 640x480 16 color video.

#13panzi – Monday, May 27th 2013, 04:04

Nice!

Because asm.js was mentioned: Future Chrome releases will probably also support asm.js.

I wonder if one could write this decoder as a GLSL program. Then it should be at least as fast as C, I'd think. And running on the GPU the CPU is free to do other stuff.

#14 – Gabor – Sunday, September 15th 2013, 22:01

Nice job. Can i stream only 1 video in same time? So.. it possible create multi channel stream?

#15 – unknown – Monday, September 30th 2013, 15:44

How can I make this work cross domain?

#16 – BR – Monday, March 17th 2014, 21:33

How would I go about creating a seek function for this? Would I need to manipulate the current buffer index somehow?

#17Ingvar Stepanyan – Friday, April 11th 2014, 10:01

Great work! For those interested in similar topics, check out also rreverser.github.io/mpegts/ JavaScript realtime HTTP Live Streaming convertor and player, handported from MPEG-TS and H.264 specifications.

#18 – 0917436251 – Thursday, September 11th 2014, 21:16

0917436251

Post a Comment:

Comment: (Required)

(use <code> tags for preformatted text; URLs are recognized automatically)

Name: (Required)

URL:

Please type phoboslab into the following input field or enable Javascript. This is an anti-spam measure. Sorry for the inconvenience.