PHOBOSLAB

Blog Home

Drawing Pixels is Hard

Way harder than it should be.

Back in 2009 when I first started to work on what would become my HTML5 game engine Impact, I was immediately presented with the challenge of scaling the game screen while maintaining crisp, clean pixels. This sounds like an easy problem to solve – after all Flash did this from day one and "retro" games are a big chunk of the market, especially for browser games, so it really should be supported – but it's not.

Let's say I have a game with an internal resolution of 320×240 and I want to scale it up 2x to 640×480 when presented on a website. With the HTML5 Canvas element, there are essentially two different ways to do this.

a) Creating the Canvas element in the scaled up resolution (640×480) and draw all images at twice the size:

var canvas = document.createElement('canvas');
canvas.width = 640;
canvas.width = 480;

var ctx = canvas.getContext('2d');
ctx.scale( 2, 2 );
ctx.drawImage( img, 0, 0 );

b) Using CSS to scale the Canvas – In my opinion this is the cleaner way to do it. It nicely decouples the internal canvas size from the size at which it is presented:

var canvas = document.createElement('canvas');
canvas.width = 320;
canvas.width = 240;
canvas.style.width = '640px';
canvas.style.width = '480px';

var ctx = canvas.getContext('2d');
ctx.drawImage( img, 0, 0 );

Both methods have a problem though – they use a bilinear (blurry) filtering instead of nearest-neighbor (pixel repetition) when scaling.

For the internal scaling approach (method a), you can set the context's imageSmoothingEnabled property to false in order to have crisp, nearest-neighbor scaling. This has been supported in Firefox for a few years now, but Chrome only just recently implemented it and it is currently unsupported in Safari (including Mobile Safari) and Internet Explorer (test case).

When doing the scaling in CSS (method b), you can use the image-rendering CSS property to specify the scaling algorithm the browser should use. This works well in Firefox and Safari, but all other browsers simply ignore it for the Canvas element (test case).

Of course Internet Explorer is the only browser that currently doesn't support any of these methods.

Not having crisp scaling really bothered me when I initially started to work on Impact. Keep in mind that at the time no browser supported either of the two methods described above. So I experiment a lot to find a solution.

And I found one. It's incredibly backwards and really quite sad: I do the scaling in JavaScript. Load the pixel data of each image, loop through all pixels and copy and scale the image, pixel by pixel, into a larger canvas then throw away the original image and use this larger canvas as the source for drawing instead.

var resize = function( img, scale ) {
    // Takes an image and a scaling factor and returns the scaled image
    
    // The original image is drawn into an offscreen canvas of the same size
    // and copied, pixel by pixel into another offscreen canvas with the 
    // new size.
    
    var widthScaled = img.width * scale;
    var heightScaled = img.height * scale;
    
    var orig = document.createElement('canvas');
    orig.width = img.width;
    orig.height = img.height;
    var origCtx = orig.getContext('2d');
    origCtx.drawImage(img, 0, 0);
    var origPixels = origCtx.getImageData(0, 0, img.width, img.height);
    
    var scaled = document.createElement('canvas');
    scaled.width = widthScaled;
    scaled.height = heightScaled;
    var scaledCtx = scaled.getContext('2d');
    var scaledPixels = scaledCtx.getImageData( 0, 0, widthScaled, heightScaled );
    
    for( var y = 0; y < heightScaled; y++ ) {
        for( var x = 0; x < widthScaled; x++ ) {
            var index = (Math.floor(y / scale) * img.width + Math.floor(x / scale)) * 4;
            var indexScaled = (y * widthScaled + x) * 4;
            scaledPixels.data[ indexScaled ] = origPixels.data[ index ];
            scaledPixels.data[ indexScaled+1 ] = origPixels.data[ index+1 ];
            scaledPixels.data[ indexScaled+2 ] = origPixels.data[ index+2 ];
            scaledPixels.data[ indexScaled+3 ] = origPixels.data[ index+3 ];
        }
    }
    scaledCtx.putImageData( scaledPixels, 0, 0 );
    return scaled;
}

This worked surprisingly well and has been the easiest way to scale up pixel-style games in Impact from day one. The scaling is only done once when the game first loads, so the performance hit isn't that bad, but you still notice the longer load times on mobile devices or when loading big images. After all, it's a stupidly costly operation do to, even in native code. We usually use GPUs for stuff like that.

All in all, doing the scaling in JavaScript is not the "right" solution, but the one that works for all browsers.

Or rather worked for all browsers.

Meet the retina iPhone

When Apple introduced the iPhone 4, it was the first device with a retina display. The pixels on the screen are so small, that you can't discern them. This also means, that in order to read anything on a website at all, this website has to be scaled up 2x.

So Apple introduced the devicePixelRatio. It's the ratio of real hardware pixels to CSS pixels. The iPhone 4 has a device pixel ratio of 2, i.e. one CSS pixel is displayed with 2 hardware pixels on the screen.

This also means that the following canvas element will be automatically scaled up to 640×480 hardware pixels on a retina device, when drawn on a website. Its internal resolution, however, still is 320×240.

<canvas width="320" height="240">

This automatic scaling again happens with the bilinear (blurry) filtering by default.

So, in order to draw at the native hardware resolution, you'd have to do your image scaling in JavaScript as usual but with twice the scaling factor, create the canvas with twice the internal size and then scale it down again using CSS.

Or, in recent Safari's, use the image-rendering: -webkit-optimize-contrast; CSS property. Nice!

This certainly makes things a bit more complicated, but devicePixelRatio was a sane idea. It makes sense.

Meet the retina MacBook Pro

For the new retina MacBook Pro (MBP), Apple had another idea. Instead of behaving in the same way as Mobile Safari on the iPhone, Safari for the retina MBP will automatically create a canvas element with twice the internal resolution than you requested. In theory, this is quite nice if you only want to draw shapes onto your canvas - they will automatically be in retina resolution. However, it significantly breaks drawing images.

Consider this Canvas element:

<canvas width="320" height="240"></canvas>

On the retina MBP, this will actually create a Canvas element with an internal resolution of 640×480. It will still behave as if it had an internal resolution of 320×240, though. Sort of.

This ingenious idea is called backingStorePixelRatio and, you guessed it, for the retina MBP it is 2. It's still 1 for the retina iPhone. Because… yeah…

(Paul Lewis recently wrote a nice article about High DPI Canvas Drawing, including a handy function that mediates between the retina iPhone and MBP and always draws in the native resolution)

Ok, so what happens if you now draw a 320×240 image to this 320×240 Canvas that in reality is a 640×480 Canvas? Yep, the image will get scaled using bilinear (blurry) filtering. Granted, if it wouldn't use bilinear filtering, this whole pixel ratio dance wouldn't make much sense. The problem is, there's no opt-out.

Let's say I want to analyze the colors of an image. I'd normally just draw the image to a canvas element retrieve an array of pixels from the canvas and then do whatever I want to do with them. Like this:

ctx.drawImage( img, 0, 0 );
var pixels = ctx.getImageData( 0, 0, img.width, img.height );
// do something with pixels.data...

On the retina MBP you can't do that anymore. The pixels that getImageData() returns are interpolated pixels, not the original pixels of the image. The image you have drawn to the canvas was first scaled up, to meet the bigger backing store and then scaled down again when retrieved through getImageData(), because getImageData() still acts as if the canvas was 320×240.

Fortunately, Apple also introduced a new getImageDataHD() method to retrieve the real pixel data from the backing store. So all you'd have to do is draw your image to the canvas with half the size, in order to draw it at the real size. Confused yet?

var ratio = ctx.webkitBackingStorePixelRatio || 1;
ctx.drawImage( img, 0, 0, img.width/ratio, img.height/ratio );

var pixels = null;
if( ratio != 1 ) {
    pixels = ctx.webkitGetImageDataHD( 0, 0, img.width, img.height );
}
else {
    pixels = ctx.getImageData( 0, 0, img.width, img.height );
}

(Did I say it's called getImageDataHD()? I lied. You gotta love those vendor prefixes. Imagine how nice it would be if there also was a moz, ms, o and a plain variant!)

The "Good" News

Ok, take a deep breath, there are only 3 different paths you have to consider when drawing sharp pixels on a scaled canvas.


The CSS image-rendering property and the Canvas' imageSmoothingEnabled really make things a bit easier, but it would be nice if they were universally supported. Especially Safari is in desperate need for imageSmoothingEnabled-support, with all the crazy retina stuff they have going on.

Let me also go on record saying that backingStorePixelRatio was a bad idea. It would have been a nice opt-in feature, but it's not a good default. A comment from Jake Archibald on Paul Lewis' article tells us why:

<canvas> 2D is a bitmap API, it's pixel dependent. An api that lets you query individual pixels shouldn't be creating pixels you don't ask for.

Apple's backingStorePixelRatio completely breaks the font rendering in Impact, makes games look blurry and breaks a whole bunch of other apps that use direct pixel manipulation. But at least Apple didn't have to update all their dashboard widgets for retina resolution. How convenient!

Update September 18th 2012: To demonstrate the bug in Safari, I build another test case and filed a report with Apple.

Friday, September 14th 2012 / Comments (29)

Where the Innovation Stops

Some time ago a rather large US entertainment company bought a few licenses for my HTML5 Game Engine Impact. They used it for internal prototyping and, as they told me, were quite happy with it.

A few month later I got an email from the company's legal department. They asked me to sign an amendment to the Impact Software license agreement with three additional terms. I assume they wanted to publish a game they made with Impact.

The first term stated that I would not be allowed to use their company's name in marketing material. Fine with me. I wouldn't do that without asking beforehand anyway.

The other two terms however felt a bit strange.

But let's back off for a second. As you may know, I'm the sole author of Impact. I'm based in Germany and self employed. It's a one man show. I'm providing Impact without any warranty and I'm not liable for any damages my software may cause (6. & 7. in the license agreement). Pretty standard.

Now, the license amendment I was to sign stated that my software "does not use, embed or incorporate any software which is subject to any open source or other similar types of license terms". What? Why? How? Is there any software out there that truly honors this term?

At this point it's already clear that I can't sign this. Impact uses John Resig's Simple Inheritance, Array.erase and Function.bind as found in MooTools, parts of DOMReady as found in jQuery and some more snippets and boilerplate code that I would consider public domain.

Typical Huge Company™ I thought. Kind of cute.

The last term however is where it gets truly frightening. In short, I would be held liable for "all damages, liabilities, losses, costs and expenses (including attorneys' fees) relating to any claim, action, suit or proceeding brought by a third party based on any actual or alleged infringement or misappropriation of such third party's intellectual property rights in connection with the use of the software." This goes on for a few more paragraphs.

Let's ignore for a minute that this legalese is written so vaguely that I could be held liable if the company published a game, using my game engine, with art assets they stole from another company. Let's ignore that this multi-billion-dollar company only bought software worth a few hundred USD from me, yet they still want me to pay their legal fees. Ignore that.

The core of the problem is another one: The US Legal System and Patent Law.

Quite frankly, the US Legal System scares me. A Legal System that is vague enough, emotional enough to have spawned a whole subcategory of dramatic movies is not a good indicator for true justice. The notion that the party with more money wins a trial, the whole jury selection process, the fact that there even is a jury. It all seems so absurd from our perspective.

Maybe I have watched too many Hollywood movies. But maybe it really is like this. Just look at the recent Apple vs. Samsung case – a supposedly boring patent trial made convoluted and emotional.

If I ever get sued in the US, who knows what will happen. It's truly unpredictable.

And in the US there's always a reason to get sued. Impact probably (unknowingly) infringes a whole lot of US software patents. All trivial, all with prior art. Yet, proving so in a court case would absolutely ruin my business and me financially.

It's understandable that this entertainment company wanted me to sign their license amendment. It absolutely made sense from their perspective. They are deep in this circus and wanted a bit of certainty that their legal system couldn't provide.

I told them I couldn't sign their amendment and never heard back.

Sometimes I have the feeling that I'm missing out by not being in Silicon Valley, in the epicenter of Startup culture. I was there for a visit and it was extremely energizing and motivational. I loved the people, the mentality, the atmosphere. Instead, I'm in a small and boring town in Germany.

But then again, I feel safer here; I have the freedom to experiment, to innovate. I'm glad that we don't have impeding software patents. Glad that our Legal System is still sane. Glad that my business is based in Germany and not in the US.

Friday, August 31st 2012 / Comments (11)

Letter to John Carmack

The following is an email I sent to John Carmack of Id Software after I had watched his Quakecon 2012 Keynote Speech.

Edit: John Carmack posted his response below.

Hi John,

I watched the live stream of your keynote speech yesterday, and – as every year – I enjoyed it very much. I applaud your openness; it's always refreshing to hear some insights from the gaming industry that is otherwise very secretive. Thank you!

However, I believe your views on JavaScript and WebGL are short sighted.

Nobody pretends that the next AAA-title will be written in JavaScript. The community understands that it's magnitudes slower than native code, but the thing is: it just doesn't matter.

Many, many types of games don't need the absolute best and fastest environment to offer interesting gameplay and graphics. Take a look at X-Type on your iPad or iPhone for instance.

You yourself talked about how frustrating it is that you need 200 lines of code just to set up your screen for rendering on Windows, whereas in the olden days you could just push pixels to the screen with 3 lines of code.

We're there again: take 3 lines of JavaScript and you're drawing an image to a canvas element. Take 20 more lines and you have a rendering loop and a sprite that moves with the arrow keys. And all you need is a text editor and your favorite Browser. That is what JavaScript is all about.

When you're new to programming and still figuring out whether you like it, you don't want to read thick books. You want to see results. Fast. You want pictures dancing on the screen.

JavaScript is the language you should teach your son.


I backed the Oculus Rift Kickstarter yesterday and I fully expect to see significant development for it going on with WebGL. It doesn't matter that JS is "slow" when you're just trying stuff; ease of development is far more important.

Coupled with the openness of the Web, I wouldn't be surprised if much, if not most, of the research for applications of the Rift comes out of the JavaScript community.

Right-Click -> View Source is what made the web so successful and it's awesome that we now have this for 3D graphics as well.

Don't hate JavaScript. It's our generation's Apple II.

Regards,
Dominic

Friday, August 3rd 2012 / Comments (53)

What the Fucking Fuck, Apple?

Update Tuesday, July 16th 2012: The Bug appears to be fixed in iOS 6 Beta 3.

Almost two years ago I noticed a strange multitouch problem in Mobile Safari with one of my games. This problem was new; it was introduced with the iOS 4.2.1 update and wasn't present on older iOS versions.

I built a test case and filed a bug report with Apple. It was marked as a duplicate of a bug they already knew about. I have no idea what that original bug report said, because fuck you, Apple's bug tracker is private. That was almost two years ago.

Two years.

I re-submitted this bug report 4 more times, after it wasn't fixed in the subsequent iOS updates. It was marked as a duplicate and closed each time. It still hasn't been fixed in the iOS 6 Beta.

Two fucking years.

Annoyed by this bug, Jayson Potter recently contacted me, asking for the bug id and description of my report and re-submitted it by himself. This time it wasn't marked as a duplicate. Instead, he got this reply from Apple a week later:

Please know that our engineers have not been able to reproduce this reported behavior with iOS 5.x

WHAT THE FUCKING FUCK, APPLE? You need two fucking years to decide that you can't reproduce a bug with a test case so simple that a three year old could understand it?

This is beyond frustrating. And there's no alternative to Mobile Safari on iOS; Apple doesn't allow it. On the iPhone, you're as locked in with HTML5 as you were with Flash everywhere else before - you're completely at the mercy of one fucking company.

So I'm really desperate to get this bug fixed – I recorded a nice video and voice over, explaining the bug itself and why it's such a big deal. I re-submitted the bug report again this morning (Bug #11796586), with a link to it:

Demonstrating the bug in the test case and my game Biolab Disaster

All in all I probably spent about 40 hours dealing with this shit: Building the test case, reporting it to Apple, writing documentation that states where this bug is and why it exists, explaining to my customer's that it's not their fault, nor mine, that their on-screen button doesn't work and apologizing to them for the fact that Apple is a fucking ignorant piece of shit company.

Fucking fix it, Apple.

Tuesday, July 3rd 2012 / Comments (90)

X-Type – Making Of

For this year's Google IO, Google asked me to do a Chrome experiment for Mobile for them. They initially wanted me to vamp up Biolab Disaster – it's still a good game, but because of it's retro style it wouldn't be that impressive. Modern mobile browsers can do a lot more.

I suggested I would try to take another game of mine – X-Type – and make it work on mobile browsers. The game was made with my JavaScript Game Engine, so it mostly "just worked" on mobile browsers already. Yet, I still had a lot of work to do.

Have a look at X-Type over at chromeexperiments.com.

X-TYPE running on various mobile devices

Screen Size

One of the most difficult things for HTML5 games is dealing with the vast amount of different screen sizes and aspect ratios out there. I experimented a lot with different solutions and ended with a fairly simple one: the internal width the game's viewport is always 480px. These 480px get scaled to whatever is available on the device. The height of the viewport is variable, so that it fills the screen with the same scaling factor as the width.

// The internal width for our canvas is fixed at 480px.
// The internal height is set so that it fills the screen when scaled
canvas.width = 480;
canvas.height = window.innerHeight * (canvas.width / window.innerWidth);

// Scale the canvas via CSS to fill the screen 
canvas.style.width = window.innerWidth + 'px';
canvas.style.height = window.innerHeight + 'px';

In older browsers (Mobile and Desktop), scaling the <canvas> element was a horrible idea – it decreased performance to a tenth of what it would be unscaled. I'm happy to report that this is no longer true; Mobile Safari on iOS5 and the Chrome Beta on Android work just fine with scaled canvas. It still makes the game unplayable in Android's "Browser", though.

I also took care to only display the game in portrait mode and show a "Please Rotate the Device" message otherwise. Mobile Safari and Chrome both support the orientationchange event, which makes this easy. However, we can not rely on window.orientation, which reports the rotation in degrees (0, 90, 180 or 270), because some devices report 0° for portrait mode, while others report 0° for landscape. How convenient!

The solution is to just check if the window height is bigger than the width – if so, we're obviously in portrait mode! But as this would be too easy, Chrome's Browser offers another challenge for us: it only updates the window dimensions after it has fired the orientationchange event. So we listen for orientationchange and resize events. Sigh.

var wasPortrait = -1;
var checkOrientation = function() {
    var isPortrait = (window.innerHeight > window.innerWidth);
    if( isPortrait === wasPortrait ) { return; // Nothing to do here }
    wasPortrait = isPortrait;
    
    // Do your stuff...
};
window.addEventListener( 'orientationchange', checkOrientation, false );
window.addEventListener( 'resize', checkOrientation, false );

Performance

Since iOS 5 the <canvas> element is hardware accelerated and it really shows. You can draw hundreds of sprites on the screen without any slowdowns at all. The same is true for Chrome on Android – to a certain degree.

All drawing is scheduled via requestAnimationFrame and thus bound to the display's refresh rate. This works nicely on iOS, but Chrome refuses to draw at the 60hz even for the simplest scenes. You can use setInterval to process more frames, but only a portion of them is really presented on the screen.

So while Chrome's JavaScript engine is fast enough to process and render 60 FPS, it fails to display all of the rendered frames. I have no doubt that this bug(?) will get fixed.

While the game works in a multitude of other browser's on Android, such as Firefox, Dolphin or Opera, none of them provided good performance. I suspect the <canvas> element is not hardware acclerated in any of these, but didn't investigate further.

Controls

In the desktop version of the game you move the player with the arrow keys and aim and shoot with the mouse. Of course this doesn't work on touch screens, so I opted for dual virtual analog sticks. This worked out surprisingly well – with a bit of practice, you can control your spaceship quite precisely.

I also tried to make the analog sticks appear where you touch the screen first, so that you can always change the position. This made everything quite a bit confusing; it's easier to grasp the concept when the position of the analog sticks is fixed. Providing the dynamic positioning is probably more of "pro gamer" feature that should be optional, if at all.

Sound

This is the sad part. I complained about support for the <audio> element in mobile browsers last year already – and guess what: it's still the same shit. Apple hasn't done anything at all to improve the situation; same goes for Android's Browser. The Chrome Beta on Android seems to have some support for Audio, but it's not really usable for real time games at the moment. I'll investigate this further.

As always, I have high hopes though. Never give up, never surrender!

All in all, I'm very pleased with the results. Rendering performance in modern mobile browsers is really awesome and the quirks I encountered were workaroundable.

I learned a lot with this project and will use this new gained knowledge to make mobile browser support much more easy in the next version of Impact.

Wednesday, June 27th 2012 / Comments (23)

Measuring Input Lag in Browsers

For games, the time between a key-press and the appropriate action happening on the screen can often decide over virtual life or death.

It is no surprise that game developers go to great length to reduce input lag as much as possible. Guitar Hero for instance even has an option to compensate for lag. One of the biggest time hogs these days seem to be displays, but a poorly written application or game can introduce a lot of lag as well.

The typical path a key-press travels – Keyboard » USB Driver » OS » Application – is bad enough as each of these layer can introduce some lag. Yet, JavaScript games have to go through an additional layer they have no control over: the browser – which in itself may have several layers that introduce lag until a JavaScript callback for a keydown event can be called.

I decided to try and measure this lag for different browsers to see if there's room for improvement. The good news: most browsers do a good job delivering input to JavaScript events as fast as possible. The bad news: Chrome does not.

I build a simple website that draws to a canvas element at 60 frames per second, counts the number of frames and captures input events. I used my camera (a Samsung NX200) to record video at 120 frames per second, then slowed the video down even further to have a close look at the results.

The response time from the command prompt was used as a baseline (I measured the same lag with Quake 3 and other native games): 5 frames or about ~83ms. This is pretty awful to begin with. I'm not really sure where all this lag comes from, but I blame the display. Since I measured all (Windows) browsers with the same setup, results are still comparable.

Firefox 14 had the fastest response time of only 5 frames, followed by IE9, Opera 12 and Safari 5.1.5 (Mac) with 6 frames. The real surprise here is Chrome's response time of 8 frames or 133ms – 50ms more than Firefox. This doesn't sound like much, but if you directly compare a real JavaScript game in Chrome and Firefox, you can definitely feel the difference.

The input lag for Mobile Safari on iOS6 is around the 5 frame, ~83ms mark as well. I also tried to measure the Android's "Browser" and the Chrome Beta on my Galaxy Nexus, but couldn't get accurate results.

Chrome on Android refused to render at more than 20 frames per second and the "Browser", while proclaiming to render at 60 frames per second, only really presented every fourth frame. This is clearly visible in the slow motion video – 4 boxes appear at the same time, instead of one after the other. This is also why HTML5 games in the "Browser" (damn, I hate that "name") still seem to stutter, even though they are "rendered" at 60 frames per second.

Android has a lot of catching up to do.

I build another simple website that behaves like Guitar Hero's lag calibration. As humans, we try to compensate for the lag ourselves by pressing a bit earlier, so the results from this aren't as accurate as measured with a camera, but you can clearly see and feel the difference between Chrome and about any other desktop browser.

Try it here: phoboslab.org/inputlag/

Update June 26th 2012

As suggested by Filip Svendsen in the comments, I made another video to show mouse input lag in Firefox 14 and Chrome 22. To make the lag easier to spot, I wrote a Python script to move the mouse cursor at constant speeds.

From the video, it's quite obvious that the mouse lag in Chrome is much higher than in Firefox. However, the movement looks much smoother in Chrome. I guess you can't have everything - at least not yet.

Tuesday, June 19th 2012 / Comments (8)

What the requestAnimationFrame API Should Have Looked Like

Chrome, Firefox and IE10 now support requestAnimationFrame – a noble idea, with a totally backwards API.

Imagine you have this JavaScript animation that you want to constantly update. An animation that constantly updates, needs to be updated more than once, i.e. constantly. After all it's an animation. So why on earth do I have an API to request one update at a time?

Mozilla's documentation even warns developers that the API might not work as expected:

Note: Your callback routine must itself call requestAnimationFrame() unless you want the animation to stop.

Ah, yes.

Car analogy time: I have this very nice car that makes a beeping noise whenever I open the door with the ignition off and the headlights still on. You know, to warn me that my headlights might run the battery dry in my absence. "Thank you car, that's very kind of you. But let me ask you something: why didn't you just turn the headlights off yourself instead of notifying me?"

The requestAnimationFrame API is modeled after setTimeout when it should've been modeled after setInterval instead.

Here's my proposal:

// Install an animation that continously calls the callback
var animationId = setAnimation( callback, element );

// Stop the animation
clearAnimation( animationId );

And the code to make this happen:

(function(w) { 
    "use strict";
    
    // Find vendor prefix, if any
    var vendors = ['ms', 'moz', 'webkit', 'o'];
    for( var i = 0; i < vendors.length && !w.requestAnimationFrame; i++ ) {
        w.requestAnimationFrame = w[vendors[i]+'RequestAnimationFrame'];
    }
    
    // Use requestAnimationFrame if available
    if( w.requestAnimationFrame ) {
        var next = 1, 
            anims = {};
                
        w.setAnimation = function( callback, element ) {
            var current = next++;
            anims[current] = true;
            
            var animate = function() {
                if( !anims[current] ) { return; } // deleted?
                w.requestAnimationFrame( animate, element );
                callback();
            };
            w.requestAnimationFrame( animate, element );
            return current;
        };
        
        w.clearAnimation = function( id ) {
            delete anims[id];
        };
    }
    
    // [set/clear]Interval fallback
    else {
        w.setAnimation = function( callback, element ) {
            return w.setInterval( callback, 1000/60 );
        }
        w.clearAnimation = w.clearInterval;
    }
}(window));

Gist on Github

Monday, May 7th 2012 / Comments (16)

JavaScript on the XBox 360

Yes, it works. No, it's not very fast.

As a part of my HTML5 Game Engine I previously wrote a framework that lets you run a JavaScript/Canvas game on the iPhone without using the browser engine, drawing everything in OpenGL directly. This worked out pretty well. The iOS version of Biolab Disaster is indistinguishable from natives apps.

My HTML5 Game Biolab Disaster running on the XBox 360

I wanted to do the same for the Xbox360 – how hard could it be? I got a JavaScript engine running on iOS, so I'm used to jumping through hoops, but nothing could prepare me for what I was up to.

Microsoft offers a developer program for the XBox 360, where you can write games in C# with a framework called XNA. These games are compiled into NET bytecode and work on Windows and the XBox without any modifications. However, in contrast to Windows, you can't execute unmanaged code on the XBox. This ruled out all the popular JavaScript engines that are written in C++, such as V8 or JavaScriptCore.

Luckily, a guy called Paul Bartrum wrote Jurassic - a JavaScript engine for NET. It compiles JavaScript code into NET bytecode at runtime, allowing far better performance than a pure interpreter (e.g. Jint). It works almost seamlessly with C# classes and fully supports the ECMAScript 5 standard. I wrote some quick code to get Biolab Disaster up and running with Jurassic and XNA on Windows and got a steady 60hz with lots of headroom.

But there's a catch: Even though Jurassic is written in C#, it doesn't work on the XBox. The XBox doesn't run the real NET Framework, but something called NET CF or Compact Framework. The functionality to create bytecode at runtime ("Reflection.Emit") is completely missing.

Now here's is where it gets funky. If Jurassic can compile JavaScript at runtime, I should be able to rig it to compile everything into a NET Assembly at compile time. The XBox then only needs to load this Assembly (a DLL) and execute it. I pitched my idea to Paul Bartrum:

It seems doable, though I think you underestimate the work required to generate the assembly

I sure did.

Fast forward three weeks of me trying to wrap my head around C# – a language I've never worked with – modifying a real world compiler – something I had no knowledge of whatsoever – and sitting for hours in the debugger, stepping through the code line by line.

I finally got it working. The Biolab Disaster Title screen came on my TV. Smoothly animated and in glorious 1cm pixels. I pressed a button to start the game and was greatly disappointed to see it running with only 3 frames per second.

After a bit of profiling, I found no obvious bottleneck. The compiled JavaScript code was just slow overall. Good enough for some scripting, but not for a whole game. Maybe I'm missing something that makes the compiled JavaScript so much slower on the XBox. A certain operation that has to be "emulated" by the NET CF; something that requires extensive 64bit math? I don't know. I called it quits.

I put all the sources and a small JavaScript demo on GitHub. It should compile and run out of the box on Windows and the XBox 360 with the Windows Phone SDK (don't be fooled by the name, it includes the XBox stuff as well).

JS360 on Github

If anyone from Microsoft is reading this: Don't be Apple. Give us a speedy JavaScript engine as part of XNA.

Friday, April 27th 2012 / Comments (19)