Posts tagged with “iPhone”
Ejecta is a fast JavaScript, Canvas & Audio implementation for iOS. Today, I'm releasing it under the MIT Open Source license.
Visit the Ejecta website for more info on what it is and how to use it. I will talk a bit more about some implementation details for the Canvas API here.
Implementing a general purpose drawing API, such as the HTML5 Canvas API, on top of OpenGL is by no means an easy endeavor. Before I decided to roll my own solution (you know, I have this problem), I looked at a number of graphic libraries including Google's skia and OpenVG.
I discovered exactly what I feared beforehand: these libraries do way too much, are too large and too hard to implement. You can't just use them here and there to draw – instead they replace your whole drawing stack. Getting them to compile alone is a huge pain; getting them to compile on the iPhone and then get them do what you wanted to seemed close to impossible.
So I began working on my own solution. Implementing the path methods for moveTo(), lineTo(), bezierCurveTo(), etc. was fairly straight forward: have an array of subpaths where each subpath is an array of points (x,y). Each call to the API methods pushes one or more points to the subpath or closes it.
However, I struggled a bit with getting bezier curves to behave in a manner that makes sense for the current scale; i.e. push more points for large bezier curves and at sharp corners, fewer points for smaller ones and straight lines. After a few days of reading and experimenting, I found this excellent article on adaptive bezier curves and adopted its solution.
The hard part was getting that array of points on the screen. For drawing lines (.stroke()) I didn't want to go with the obvious solution of just using GL_LINES, because it has a number of drawbacks, especially on iOS: no anti aliasing, limited line width and no miters or line caps.
So instead of using GL_LINES to draw, I ended up creating 2 triangles for each line segment and calculate the miter values myself. This correctly honors the APIs .miterLimit property, though the bevel it then draws is still a bit off. The code I ended up with is a bit on the ugly side, because it handles a lot of edge cases, but all in all this solution worked very well and is extremely fast.
Implementing .fill() proved to be yet another challenge. With OpenGL, before you can draw a primitive to the screen, you have to break it down into triangles first. This is quite easy to do for convex polygons, but not so much for concave ones that potentially have holes in them.
I spent a few days looking for triangulation library and soon realized that this is serious business. Triangle for instance, sports 16k loc – I'm quite allergic to libraries that need that much code to solve seemingly simple problems. Poly2Tri looked much more sane, but apparently has some stability problems.
After a bit of searching, I found libtess2, which is based on OpenGL's libtess and is supposed to be extremely robust and quite fast. The code base is excellent and I had no problem implementing it with Ejecta.
However, some tests showed that it's much slower than I hoped it would be. Realtime triangulation of complex polygons isn't very feasible on the iPhone.
In the end, I found a trick that lets you draw polygons in OpenGL without triangulating them first. It is so simple and elegant to implement, yet so ingenious: You can draw polygons with a simple triangle fan and mark those areas that you overdraw in the stencil buffer. See Drawing Filled, Concave Polygons Using the Stencil Buffer. It's a hacker's solution – thinking outside the box – and it fills me with joy.
There's still some parts missing in my Canvas implementation, namely gradients, shadows and most notably: text. I believe the best solution for drawing text in OpenGL, while honoring the Canvas spec, would be drawing to a texture using the iPhone's CG methods. This will make it quite slow, but should be good enough for a few paragraphs of text.
If you want to help out with anything grab the Ejecta source code on github – I'd be honored.
Update Tuesday, July 16th 2012: The Bug appears to be fixed in iOS 6 Beta 3.
Almost two years ago I noticed a strange multitouch problem in Mobile Safari with one of my games. This problem was new; it was introduced with the iOS 4.2.1 update and wasn't present on older iOS versions.
I built a test case and filed a bug report with Apple. It was marked as a duplicate of a bug they already knew about. I have no idea what that original bug report said, because fuck you, Apple's bug tracker is private. That was almost two years ago.
Two years.
I re-submitted this bug report 4 more times, after it wasn't fixed in the subsequent iOS updates. It was marked as a duplicate and closed each time. It still hasn't been fixed in the iOS 6 Beta.
Two fucking years.
Annoyed by this bug, Jayson Potter recently contacted me, asking for the bug id and description of my report and re-submitted it by himself. This time it wasn't marked as a duplicate. Instead, he got this reply from Apple a week later:
Please know that our engineers have not been able to reproduce this reported behavior with iOS 5.x
WHAT THE FUCKING FUCK, APPLE? You need two fucking years to decide that you can't reproduce a bug with a test case so simple that a three year old could understand it?
This is beyond frustrating. And there's no alternative to Mobile Safari on iOS; Apple doesn't allow it. On the iPhone, you're as locked in with HTML5 as you were with Flash everywhere else before - you're completely at the mercy of one fucking company.
So I'm really desperate to get this bug fixed – I recorded a nice video and voice over, explaining the bug itself and why it's such a big deal. I re-submitted the bug report again this morning (Bug #11796586), with a link to it:
All in all I probably spent about 40 hours dealing with this shit: Building the test case, reporting it to Apple, writing documentation that states where this bug is and why it exists, explaining to my customer's that it's not their fault, nor mine, that their on-screen button doesn't work and apologizing to them for the fact that Apple is a fucking ignorant piece of shit company.
Fucking fix it, Apple.
For this year's Google IO, Google asked me to do a Chrome experiment for Mobile for them. They initially wanted me to vamp up Biolab Disaster – it's still a good game, but because of it's retro style it wouldn't be that impressive. Modern mobile browsers can do a lot more.
I suggested I would try to take another game of mine – X-Type – and make it work on mobile browsers. The game was made with my JavaScript Game Engine, so it mostly "just worked" on mobile browsers already. Yet, I still had a lot of work to do.
Have a look at X-Type over at chromeexperiments.com.
X-TYPE running on various mobile devices
Screen Size
One of the most difficult things for HTML5 games is dealing with the vast amount of different screen sizes and aspect ratios out there. I experimented a lot with different solutions and ended with a fairly simple one: the internal width the game's viewport is always 480px. These 480px get scaled to whatever is available on the device. The height of the viewport is variable, so that it fills the screen with the same scaling factor as the width.
// The internal width for our canvas is fixed at 480px.
// The internal height is set so that it fills the screen when scaled
canvas.width = 480;
canvas.height = window.innerHeight * (canvas.width / window.innerWidth);
// Scale the canvas via CSS to fill the screen
canvas.style.width = window.innerWidth + 'px';
canvas.style.height = window.innerHeight + 'px';
In older browsers (Mobile and Desktop), scaling the <canvas> element was a horrible idea – it decreased performance to a tenth of what it would be unscaled. I'm happy to report that this is no longer true; Mobile Safari on iOS5 and the Chrome Beta on Android work just fine with scaled canvas. It still makes the game unplayable in Android's "Browser", though.
I also took care to only display the game in portrait mode and show a "Please Rotate the Device" message otherwise. Mobile Safari and Chrome both support the orientationchange event, which makes this easy. However, we can not rely on window.orientation, which reports the rotation in degrees (0, 90, 180 or 270), because some devices report 0° for portrait mode, while others report 0° for landscape. How convenient!
The solution is to just check if the window height is bigger than the width – if so, we're obviously in portrait mode! But as this would be too easy, Chrome's Browser offers another challenge for us: it only updates the window dimensions after it has fired the orientationchange event. So we listen for orientationchange and resize events. Sigh.
var wasPortrait = -1;
var checkOrientation = function() {
var isPortrait = (window.innerHeight > window.innerWidth);
if( isPortrait === wasPortrait ) { return; // Nothing to do here }
wasPortrait = isPortrait;
// Do your stuff...
};
window.addEventListener( 'orientationchange', checkOrientation, false );
window.addEventListener( 'resize', checkOrientation, false );
Performance
Since iOS 5 the <canvas> element is hardware accelerated and it really shows. You can draw hundreds of sprites on the screen without any slowdowns at all. The same is true for Chrome on Android – to a certain degree.
All drawing is scheduled via requestAnimationFrame and thus bound to the display's refresh rate. This works nicely on iOS, but Chrome refuses to draw at the 60hz even for the simplest scenes. You can use setInterval to process more frames, but only a portion of them is really presented on the screen.
So while Chrome's JavaScript engine is fast enough to process and render 60 FPS, it fails to display all of the rendered frames. I no doubt that this bug(?) will get fixed.
While the game works in a multitude of other browser's on Android, such as Firefox, Dolphin or Opera, none of them provided good performance. I suspect the <canvas> element is not hardware acclerated in any of these, but didn't investigate further.
Controls
In the desktop version of the game you move the player with the arrow keys and aim and shoot with the mouse. Of course this doesn't work on touch screens, so I opted for dual virtual analog sticks. This worked out surprisingly well – with a bit of practice, you can control your spaceship quite precisely.
I also tried to make the analog sticks appear where you touch the screen first, so that you can always change the position. This made everything quite a bit confusing; it's easier to grasp the concept when the position of the analog sticks is fixed. Providing the dynamic positioning is probably more of "pro gamer" feature that should be optional, if at all.
Sound
This is the sad part. I complained about support for the <audio> element in mobile browsers last year already – and guess what: it's still the same shit. Apple hasn't done anything at all to improve the situation; same goes for Android's Browser. The Chrome Beta on Android seems to have some support for Audio, but it's not really usable for real time games at the moment. I'll investigate this further.
As always, I have high hopes though. Never give up, never surrender!
All in all, I'm very pleased with the results. Rendering performance in modern mobile browsers is really awesome and the quirks I encountered were workaroundable.
I learned a lot with this project and will use this new gained knowledge to make mobile browser support much more easy in the next version of Impact.
Some weeks ago I released a compiled version of the JavaScriptCore library for iOS as part of my game engine. Since then I had many people asking for the project files necessary to build the library - and I promised to release them as well. But before I do so, let me just rant about Apple's politics real quick:
They suck.
You see, JavaScriptCore is an Open Source library. In fact the whole WebKit project is. It's licensed under BSD and LGPL licenses. The latter of which requires that if you modify the software you have to release your modified version – including the complete source – under the LGPL as well. Furthermore, it states: "For a library, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the library."
Yet the JavaScriptCore sources that you get from Apple's Open Source page come without a project file. So while you get the source code for the library, it's useless because you can't compile it.
Luckily you can also get the current JavaScriptCore sources directly from the SVN repository – and behold, they even come with an Xcode project file, ready to be build for MacOSX with the touch of a button. But – and here's the kicker – this project file curiously misses the iOS platform target. You can't build it for the iPhone or iPad.
Adding this iOS target to the project file is by no means a trivial endeavor, mind you. Especially not in Xcode. I ended up duplicating the MacOSX target, setting the Base SDK to Latest iOS, changing the Supported Platforms to iphoneos iphonesimulator and poking around in the project file with a text editor to change the productType to library.static, because you're not allowed to build a framework for iOS.
After some more changes that I can't remember (I'll be sure to document this next time I do it), I was finally able to compile the library for iOS. I submitted my game Biolab Disaster to the App Store only to see that it was rejected the next day.
JavaScriptCore utilizes the libicucore library to sort strings in a unicode fashion. Apparently libicucore is a "private" API on iOS – which is curious because I can add this library to my iOS project without any dirty hacks, I'm just not allowed to use any functions of it. So I compiled JavaScriptCore again, this time setting the UCONFIG_NO_COLLATION preprocessor macro to disable unicode sorting.
Did I mention Apple's politics suck?
Update December 2nd 2012: an up-to-date version can be found on github. The old version and instructions follow.
Anyway, here's the statically compiled libiOSJavaScriptCore.a that you can use in your iOS projects and the source and project file if you want to build it yourself for whatever reason:
This all is based on the slightly outdated 534.27 version of JavaScriptCore, but at least this version seems to be AppStore compatible.
Using libiOSJavaScriptCore.a in your Project:
- Copy the JavaScriptCore directory with the header files from the ZIP to your project folder and add them to your Xcode project
- Copy the libiOSJavaScriptCore.a into your project folder
- Add the
libstdc++.dylib, libicucore.dylib from the list and the libiOSJavaScriptCore.a from "Add other..." like so
import <JavaScriptCore/JavaScriptCore.h>
Building JavaScriptCore from the Source:
- Select iOSJavaScriptCore iPhone as the active Scheme and build
- Select iOSJavaScriptCore iPhone Simulator as the active Scheme and build
- Navigate your terminal to the build directory: depending on your Xcode settings either
build/Products/ directly in your project directory or ~/Library/Developer/Xcode/DerivedData/JavaScriptCore-xxx/Build/Products/
- Combine the ARM6/7 and x86 (Simulator) libs into one:
lipo -create Production-iphoneos/libiOSJavaScriptCore.a Production-iphonesimulator/libiOSJavaScriptCore.a -output libiOSJavaScriptCore.a
Btw.: AppCelerator maintains their own version of JavaScriptCore that they use in Titanium, but I have no idea how to build it. Any hints are greatly appreciated!
If you want to support me, please consider buying a license of my game engine. It also makes a great gift ;-)
Less talk, more action – Apple just approved two of my JavaScript games for the AppStore: Biolab Disaster and Drop. Both are free; go check them out. You can also play them in your browser here and here.
Both games are pretty simple (the source for Drop is only 300 odd lines long) and written with my JavaScript Game Engine Impact.
These are certainly not the first games written in JavaScript to be available in the AppStore. Tools like AppMobi, PhoneGap or Titanium make it easy to bundle some HTML pages and JavaScript together in an App and display them in a UIWebView, which is basically just a browser window (correction: Titanium doesn't use a UIWebView but instead has some native bindings). Games written with Impact already work okay-ish in the iPhone's browser and thus also in AppMobi and PhoneGap.
So what's so special about these two games now? They don't use PhoneGap or Titanium. They don't even use a UIWebView. Instead, they bypass the iPhone's browser altogether and use Apple's JavaScript interpreter (JavaScriptCore) directly. All graphics are rendered with OpenGL instead of in a browser window and all sound and music is played back with OpenAL instead of… well, having no sound at all.
What makes this possible is a compatibility layer that mimics the HTML5 Canvas and Audio APIs but is implemented with OpenGL and OpenAL behind the scenes. Think of it as a browser that can only display a Canvas element and play Audio elements, but does not render generic HTML pages. A browser perfectly suited for HTML5 games.
This means you can take your JavaScript games written for Impact and run them on iOS with perfect sound and touch input and way better drawing performance than with Mobile Safari. Now, to be clear, I only implemented a bare minimum of the Canvas API - just enough to be able to run Impact. The whole thing is still in a very experimental state.
If you have a license for Impact you will find the complete source code for this all on your download page. I also wrote some basic documentation to get you started. In theory, you don't have to know anything about Objective-C to use this, but at this stage some Objective-C knowledge will sure come in handy. Again, this is very experimental. Don't expect it to work at all.
If you've been following this blog for a while, you might remember a post from October 2010 where I attempted the exact same thing. Back then I used the JavaScriptCore library that is already available on iOS and used by Apple in Mobile Safari. The problem was, that this library is "private", meaning that Apple does not want you to use it. And since you can't publish anything in the AppStore that uses private libraries I abandoned the idea.
However I recently revisited the project, because there was still a chance to make this work: JavaScriptCore is a part of the open source WebKit project. Instead of using the private library that comes with iOS, you theoretically could compile your own version of this library and bundle it together with your App. Which is exactly what I did.
Since Apple does not provide any project files to compile JavaScriptCore for iOS (presumably to annoy us) and JavaScriptCore itself uses some of iOS' private APIs, compiling this beast into a static library - in an App Store compatible fashion - took me a few days.
I also had to make a small sacrifice: JavaScriptCore uses libicu to sort strings according to a unicode locale. Sadly, libicu is also private on iOS and bundling it is not an option because of its size. So I got rid of libicu completely. This means that only ASCII strings are now sorted correctly (e.g. the umlaut "Ä" will come after "Z", not after "A" as it should). Other than that, the JavaScript library should behave exactly as the private one that comes with iOS.
Also, the JavaScriptCore library bundled with iOSImpact does not use the JIT compiler (Nitro). You can't allocate executable memory on the iPhone and if Apple doesn't lift that restriction, there's nothing I can do about it. However, Apple recently made an exception for Mobile Safari – if they would make their JavaScriptCore API public, they probably could enable the JIT for everyone. That's a big if though; I don't see it happening, because Apple loves native code (Objective-C) and hates scripting languages.
I was afraid Apple would reject the two games for some obscure reason, but they didn't. Which leaves me to wonder why the JavaScriptCore library that comes with iOS is private in the first place. Bundling JavaScriptCore with your App adds about 2MB in size – not much, but I fail to see how this can be in Apple's interest.
Anyway, the performance of both games is pretty good. I still get some occasional slowdowns (~20fps) on my iPhone3GS in Biolab Disaster when there are too many particles on the screen, but it remains playable at all times. It's also nice to have perfectly working sound on iOS now - something even some desktop browsers still struggle with.
I know you're waiting for the release of the Impact Game Engine, and I promise you, it's coming. I just get distracted too easily. So here's my game Biolab Disaster running on the iPhone 3GS with 60 frames per second:
The game is running in its own process and is not using the iPhone's browser at all. Instead, it's just using the JavaScriptCore Framework to run the game. All the necessary calls to the Canvas API have been reimplemented with OpenGL-ES and the touch input is passed over to JavaScript to be evaluated by the engine. I of course had to make some changes to the engine, but the game source code is exactly the same as for the web version.
The JavaScriptCore Framework is still private on iOS. So as it is now, I won't be allowed to distribute the game in the AppStore. But my understanding is, that if I bundle my own copy of the JavaScriptCore Framework (which is part of WebKit and thus freely available) with my game, I should be on the safe side. Let's see how this works out.
Yuckfu has been in the App Store for almost two month now and I got some good feedback from numerous people. The sales numbers however have been disappointing to say the least. The main problem was the lack of exposure.
When Yuckfu went live in the App Store, its release date was still set to the upload date a few days prior. Therefore the game appeared on page 6 of the "new Apps" in the AppStore – the chances of someone stumbling over the game without explicitly searching for it were pretty much zero. I also came to the sad conclusion that a price of $2.99 is much too high for a simple arcade game. I don't like this at all and I don't think this is a healthy state for the App Store to be in, but for now this is just how things work.
Getting my game reviewed was another challenge. Most bigger sites didn't respond to my emails, others required at least 3 promo codes to even consider reviewing. Some reviews were positive, while others failed to understand the concept of inertia - and I can't even blame them. Controlling a rocket powered robot isn't supposed to be easy. Learning to control the robot is the game, it's part of the challenge. It just takes some time to learn - time which most review sites simply don't have.
In the meantime I worked on the first update for Yuckfu, which has some minor UI improvements, but most importantly added the completely new Challenge Mode. This is something I thought of from the beginning – having pre-designed “container mazes” and spawn points. I also created some new animated backgrounds to have some more eye candy. One more advantage of the Challenge Mode is that the game now has a more shallow learning curve. The new version of Yuckfu (iTunes Link) is now available for only $0.99!
So, to gain some more exposure, I also decided to built a free “Lite” version of the game, which just cuts some of the levels from the Challenge Mode and the Highscore upload functionality. Let's see how this works out. Yuckfu Lite (iTunes Link) also made it's way to the App Store.
For some screenshots and more info head over to yuckfu.net.
It has been a while since my last update on my iPhone game. I haven’t had much time in the previous month to work on Yuckfu, but development is now on full speed again!
As you can see, I refined the graphics quite a bit. The edges of the playing area are now colored to make it clearer where your robot is and where the walls are. I also added the effect for spawning containers which looks great in motion. It’s actually a real 3D model with a rotating spiral.
For the HUD, I wanted a clean and simple solution that is showing only the information you really need. I ended up with mainly two elements: the score on the left, your energy on the right. As your energy is decreasing over time, the energy bar tells you at a glance how much you’ve got left. The four little boxes next to the score shows the “multiplier”. If you collect several containers in a row your multiplier increases and you get more points.
Read complete post »
