FairyEngine's Dev Log

Postmortem: Ace of Dynamites

2011-07-17T22:01:00.000-07:00

After the bad experience I had developing on Android (and equally poor commercial results: with about 300-350 new or updated apps in the puzzle category alone, less than 48 hours after I released Jigsaw Guru, my game got buried in the depths of the marketplace immediately, and it made about $5 of ad revenue in the first 3 months), it was time to go back to my beloved C# and XNA, and make another game for the Windows Phone 7 (WP7 for short), and XBox Live Indie Games channel (XBLIG). This time, I didn't completely start from scratch though: I decided to remake a game I initially created on the Amstrad CPC 6128 twenty years ago, when I was a student with some spare time on his hands.

The project

The original game was brutal: because I was using some hardware trick to scroll the screen at 50 fps, I couldn't do it by less than 4 pixels at a time, which means every move was extremely fast (since, in mode 0, the resolution was only 160x200. For those who are interested, I was also using an overscan technique to cover the whole monitor, and another CRTC reprogramming to have an area at the bottom of the screen that wasn't scrolling, and used the more detailed mode 1 to display times and high scores). Of course nowadays you just redraw the screen each frame and make your level scroll at the speed you want, and I certainly slowed it down a lot in the new version, but the principle of the game hasn't changed: you control a character in 2D tile-based levels, there are enemies bothering you on your way to the exit, and you can push crates and boxes full of dynamite sticks to block or blow these enemies up. I don't have a screenshot of the CPC game (and I haven't actually been able to play it in many years, as the computer's drive belt is dead and needs to be replaced), but here is one of the PC prototype I made with the exact same graphics (hence the low resolution blockiness), next to what is now level 3.1:

The CPC game was called Dedalis (dédale being another word in French to say maze), but I changed the title to try to find something that would describe a bit better what the gameplay is about. Also, unlike in Ace of Dynamites, there was no gem to collect, and no award to unlock, but the rest is pretty much the same, especially the different tiles and what they do.

What went right

Working on a game I already knew and played, even years ago, definitely had some advantages. Of course I had to make many adjustments because of the different platforms, input systems, audiences, etc; but since I'm currently thinking about my next project idea, and trying to figure out what is going to be in it, how it's going to look, and how the different elements will fit together, it's easy to see I have a lot more question marks this time than I did when I started Ace of Dynamites. I wish I could play my next game before starting to work on it, and have a better idea of where I'm going with it; that's basically what happened here.

Developing the WP7 and XBLIG versions at the same time was also a good thing. When I created Jigsaw Guru, I made the WP7 version first, then an Xbox360 version, and finally ported it to Android. The problem with that is each platform has its own characteristics , in terms of performance and resolution for example, but also regarding what you need to provide when you submit the game. Reworking the UI of Jigsaw Guru to make it match completely different aspect ratios took a few days; needing an extra logo to publish a free version wasn't something I had planned for; discovering the 3rd platform required a thumbnail in a much higher resolution than the ones I had wasn't a nice surprise. So, being able to cover all the bases from the beginning, and verify I wasn't putting myself in a corner at any time in the gameplay or UI departments, was definitely a plus on Ace of Dynamites. And if I ever wanted to do an Android version (not that I'm thinking about it at all!), I made sure I have all the iconography I would need.

This one is going to sound like a joke, but it's true: having a phone to test the game was great! As I explained in the postmortem of Jigsaw Guru, I developed my first WP7 game entirely in the emulator, since that was before the phones went on sale. I said the emulator was very reliable, but still: only real hardware can give you a good idea of the performance, it's reassuring to see the game run correctly on one of the devices people will use to play it, and more importantly the emulator can't replace the touch screen when it comes to deciding if some elements have the right size and are not too close to each other to be tapped. Go back to the previous screenshots: on the right one, it's obvious I moved the camera closer, this has the drawback of showing a smaller portion of the level, but without this change I found it was very difficult to tap the exact tile where you want your character to go. In the emulator, you almost have pixel perfect precision all the time with the mouse, and your finger doesn't get in the way, this is a big difference.

Pretty early in the project, I decided I would submit an Xbox360 build of Ace of Dynamites to the Dream Build Play (DBP) competition. That gave me a deadline to shoot for, without adding too much pressure since the game didn't necessarily have to be finished. Of course I wanted to put as much polish as possible into that build, and I think that helped me staying focused.

Working with the same artist as for Jigsaw Guru was a big plus. I knew what Aimee could do, and that we would hit the ground running instantly since we knew what to expect from each other, and even if she wasn't available immediately when I contacted her, I waited rather than look for somebody else. That meant she had less time to do all the art than I had initially planned to give her, but she managed to finish everything a couple of days early nonetheless, which was great. And maybe I also learned my lesson from the previous project: this time the list of what I needed was already finalized before Aimee started to work, I made sure all the question marks had been removed.

Finally, I spent more time polishing this game, and adding little details here and there. For example, when a menu command is selected, in addition to changing color, the dynamite stick that represents the command (Aimee's idea) also generates sparks like if the wick was burning, which emphasizes what item is selected even more, and makes the screen look a bit less static. This is a direct consequence of not having a hard deadline to submit the game, unlike on Jigsaw Guru when I was trying to have a launch title.

Guess what command is selected? (Xbox360 version)

What went wrong

I believed I would be able to reuse most levels from the original CPC game; what a mistake! It actually didn't work that way for several reasons: some levels were too big, especially now that the character moves slower; some levels were too long to complete, in particular for a phone game where play sessions can be quite short; some levels relied on a couple of tiles I didn't keep in the remake, because what they did was a bit confusing; and finally, some levels were just too hardcore. I was still able to use a few maps as is, but in many cases I had to make modifications, or build something different based on the same idea or puzzle. And I had to create and test a bunch of brand new levels, those are usually the ones that are smaller.

On the CPC, handling the character movement and collisions was simple: because one step of the hardware scrolling was exactly half the size of a tile, the character was always either not on a tile at all, half on that tile, or exactly on the tile, on each axis of the 2D world. This is not the case anymore, I implemented real 2D collisions in Ace of Dynamites, but I had a lot of problems with the arrow tiles. As soon as you overlap one of those, even by a very small amount, it's going to start pulling you in the direction pointed by the arrow. In some levels where arrow tiles are next to each other and form some sort of treadmill, that has turns and corners, it was very easy for the character to not take the turns perfectly and end up bumping into some static skull and die. You can imagine how frustrated the testers were.

On the WP7, it's a bit like I have 3 versions: the free game, the paid game when you're playing it in trial mode, and the paid game once you bought it. I didn't think that would be much of a problem, but actually it was. The deal is: the free game has 10 levels, the paid game has 20, but in trial mode you can only play the first 10 (so that it's equivalent to the free game), and at some point I'll add some bonus levels to all versions. So, the question was: for each version, what happens when the player completes the 10th level? The answer is: in the free game, the player goes back to level 1 and the difficulty mode increases, unless there are some bonus levels in which case the player goes to the first of them and the difficulty stays the same; in the paid game, the player goes to level 11 and so on, and after 20 is when he either loops back to level 1, or starts the bonus maps; in trial mode, the game jumps from level 10 to the 1st bonus (which is basically level 21) since levels 11 to 20 are not accessible, or to level 1 if there is no bonus yet. Yeah, that's quite a mess indeed, having just one single case would sound easier, but I wanted to give another shot at having both paid and free versions of the same game in the marketplace, since I didn't do it right with Jigsaw Guru (the free game was an afterthought, a different experiment, that wasn't even advertising its paid counterpart).

The end of this project wasn't fun. After I entered it into the DBP competition, and finished the last few tasks related to polishing the game, all that was left on the XBox360 was submission related (trial mode, a few bugs, the evil checklist...). Then I needed to deal with a bunch of WP7 specific tasks, since I had focused on the console version for a while because of DBP, and those were not exciting either: a few bugs only happening on the phone, certification requirements, trial mode, adding ads, tombstoning, that sort of things. I was hoping to be done at the very end of June, but as often in software development it took a bit more time, and what I'm going to talk about in the next paragraph played a part in that too.

I don't use much middleware really, only the EasyStorage library because it makes life easier to handle saved games on the Xbox360. But this time I wanted to have online leaderboards in the WP7 version, and also try adding analytics, without implementing those things myself. Since I knew some free and battle tested solutions existed, my mistake was to think integrating them would be a piece of cake, and to do those tasks last. Integrating Preemptive Solutions' analytics was in fact easy, but somehow none of my data was showing up on their website, except for the name of my game that was there to prove some messages had indeed been received. When statistics finally appeared 4 days later, I saw they didn't give me the information I wanted, made some modifications, played the game again to generate new data, saw it was queued on the website... and one week later, it still hasn't been processed. I don't know what's up with that, I was certainly hoping for something much closer to real time information, but this is my experience so far. For the leaderboards, I chose Mogade, and it was only a question of bad timing: the code just got rewritten, I ran into a few bugs, and I spent some time verifying my code wasn't the culprit, and looking for workarounds. Fortunately Karl, the author, was very responsive, and fixed the issues very quickly. But the lesson is: when you integrate some external library you haven't seen before, don't wait until the last minute, even if it seems like it's something trivial to use. That's basically why I'm not using the latest version of the Microsoft ads SDK yet: it was released very recently, I was reading in the forums some people had trouble using it, and I didn't want to postpone my submission just to figure out what changed; I'll look into it when I'll make an update for the game.

The Android experiment: porting my XNA game to Google's platform

2011-05-30T19:08:00.000-07:00

In this post, I'm going to talk about my experience porting my XNA game Jigsaw Guru Free from the Windows Phone 7 platform to Android. I will focus on the development side of things only, marketing a game on Android and the fact that there are multiple market places (including the quite recent Amazon Appstore for Android) would be a totally different topic.

Why?

Why try a new platform?

Once I released Jigsaw Guru on WP7 at the end of October 2010, I started working on an update, and prototyping a couple of other ideas. When the first sale numbers became available in December, and I saw how low they were, I stopped what I was doing and spent a few days making Jigsaw Guru Free, a free ad-supported version that got published just before the New Year. And I finished the Xbox Live Indie Game version of the program.

Then what? I could have worked on another WP7 game immediately, but the initial results from the ads were neither bad nor very encouraging, until I changed my ad categories and eCPM went up a few weeks later. Since I already had a full game implemented, with all the necessary assets (that I didn't make myself, since my artistic skills are extremely limited), another option was to port that game to a different platform, with a larger user base.

Why Android?

I basically had to choose between Android and the iPhone. Since I don't own a Mac (I know there are a few ways to develop on iOS without one, but ultimately if you want to have access to all the libraries and not be limited to some specific middleware SDK, I think you need one), I decided to try the Android tools for Windows. Android is supposed to be the fastest growing mobile market these days, and lots of tablets are going to be sold this year, so why not?

Why Java?

On Android, you can write code either in Java, or in native C/C++. On one hand I have many years of C++ experience, on the other hand I only read a book about Java a few years ago to see what the syntax looked like, so why did I choose to use Java? One reason was learning a new language can be fun :). Another one is C# and Java both have garbage collection, and therefore Java was a better match for the code of Jigsaw Guru, that doesn't care about deleting objects manually all the time. I would have used C++ if there had been a good reason for that, but what are the reasons to use native code on Android?

reusing an existing C/C++ code base. Well, Jigsaw Guru is written in C#.

performance. My game sometimes has to draw a bunch of stuff, but on the CPU side there's not much happening, I thought Java would be fine.

access to OpenGL ES 2.0. Yes, on Android you can only use GL 1.0 and 1.1 from Java, if you want to call 2.0 you have to do it from native code. But since I only used 2 different shaders from XNA (BasicEffect and DualTextureEffect), it was no big deal emulating them with the fixed function pipeline.

So, in the end, I didn't have any good reason to use native code, and reading the online documentation gave me the impression installing the native SDK (called NDK) on top of the rest was not super fun, and that debugging native code was not well integrated into Eclipse (the development environment).

Why not MonoDroid?

When I started developing on Android, MonoDroid (an SDK for running C# code on Android) was not ready yet. Even if it had been, I don't think it would be that useful for a program like Jigsaw Guru: sure, it would take care of running the C# logic of the game, but it wouldn't auto-magically remap the WP7 input, sound, file system, and rendering to the Android platform (and I'm not even talking about tombstoning, or the WP7 Chooser Tasks). For that, you would need something like ExEn, which wasn't ready for primetime either.

Installation

The installation process can't be easier for the WP7 tools: one executable, and you get Visual Studio Express 2010 if you don't already have it, XNA 4.0 (which also allows you to develop on Windows and Xbox360), the WP7 emulator, etc. It takes a while, but you can go do something else in the meantime.

For Android, there are a few more steps, since I had to install: the Java Development Kit (JDK), Eclipse, the Android SDK, the components for the versions of Android I wanted to be able to use (for example: 2.1 + 2.2 + 2.3, and now 3.0), and ADT (the Android Development Tools, which are basically a plug-in for Eclipse). Overall it went quite well, the only problem I had was with the JDK: there are 32 bit and 64 bit flavors, guess which one I picked since I run Windows7 64 bits; but apparently some of the tools are hard coded to use the 32 bit version, and I had to install that one in the end or nothing would work.

Eclipse versus Visual Studio

I've been using Visual Studio for a very long time now, since version 4.2 I believe. It's a very good code editor in my opinion, and probably the best debugger used in video game development. So, how does Eclipse (which is free) compare to Visual Studio Express 2010 (also free)? Let's start with the things I definitely liked:

the suggestion window. If there is an error in the code, and you put the mouse cursor on it, a floating window shows up with suggestions on how to fix the problem. Here is an example, where I started declaring a local variable "m" of an unknown type "Matrix":

As you can see, each of the suggestions is a link, that you just have to click to do what it says. I don't know if and how the suggestions are sorted, but the one I needed was always in the top 2 as far as I can remember. As a comparison, here is what I get in Visual Studio:

Then I can right-click and choose to resolve the missing reference, or generate a new class or type called Matrix. But this is really not as informative and convenient as what Eclipse does.

the bookmark window. I use bookmarks quite frequently to switch between 2 to 5 different places in my code. In Eclipse, I can give a name to each of them when I create them, see them all in a window, and use that window to go to any of them at any time. This seems so simple, and similar to the breakpoint window in Visual Studio, that I was surprised I never saw it there. Quick search on the internet: this window actually does exist in Visual Studio, but is not included in the Express versions of the product. Annoying, but fair enough. By the way, the breakpoint window is apparently included in VS C++ Express (so I read), but not in VS C# Express.

the File Explorer tab in the DDMS (Dalvik Debug Monitor Server) perspective. As your game is running in the emulator or on a real phone, you can see what files it creates, updates or deletes, and transfer them to your hard drive to have a look at their content. That's very useful to verify your save game code works correctly, etc.

In Visual Studio, I don't know a good way of doing that when running a WP7 game (I use the PC version I also have, since XNA is cross-platform, to check my files). And although this is not the subject of this article, this would also be a great feature to have when developing on the Xbox360 with XNA.

the DDMS perspective also allows you to take screenshots of your application, whether it's running in the emulator or on a device! I haven't checked the Mango beta of the WP7 tools yet (I cannot install it and modify my setup until I'm done with my current project), but this is something that has been missing since the beginning.

the SD card emulation. It's not totally trivial to set it up, but it allowed me to transfer some photos and pretend they had been taken with the phone's camera. This was important to me because Jigsaw Guru allows players to use their own pictures, and in the WP7 emulator you only have a few predefined ones for testing, and they don't necessarily cover all the cases you want to verify as far as naming or dimensions go.

this one is more of a detail, but when a source file imports libraries it doesn't need, I get warnings. And when I use the aforementioned suggestion window, or some other automatic way of adding imports without typing them, they are grouped according to the first part of their namespace, and then ordered alphabetically inside each group. Perhaps I'm a maniac, but I like this stuff to stay clean, and basically Eclipse does for me what I manually do in Visual Studio.

Now let's talk about the things I didn't like as much:

when an exception is thrown, it's not as easy as it should to know what happened. Maybe it's because in my case it happened most of the time in the OpenGL thread, but anyway: there doesn't seem to be a call stack window like in Visual Studio, in Eclipse I found after a while that the call stack gets printed in a window in the middle of some other log messages, and that's a mess. Seriously. Especially compared to Visual C#, where the program automatically puts me on the line that generated the exception, with a floating window telling me what the problem was.

when you're stepping into your Java code, you cannot move the instruction pointer. I so could not believe it that I did an online search, which confirmed it. Actually, I even ran into some programmers that are so used to it in Eclipse, that they were asking how this can be useful anyway, and why you would want this feature. Oh boy!

if you want to change the value of some variables while you're debugging your code, it's a pain. I don't exactly remember the details, I just remember I gave up after a while.

when your program is stopped on a breakpoint, you often want to check the value of some variables, don't you? Most of the time it works, but I got this "JDI thread evaluations" error a lot each time I tried to look at the content of arrays.

auto-completion is not the greatest. First, I find it kind of slow. Second, it only kicks in after you've typed a dot character, as in "variable.something". I'm sorry, but Visual C# is an order of magnitude better, and will auto-complete about anything: a language keyword, a function name, or the name of a variable in the above example - if what I'm typing starts like the name of an existing local variable, parameter of the function I'm in, class member, etc. I'm so used to it that I never type "public void" for example, but "pu" + space + "vo"+ space instead. In the long run, this probably saves me from a lot of typos :)

I'm not 100% sure about this one, but I got the impression changing a resource file (texture, sound, etc) doesn't trigger a rebuild of the project.

when you have both a warning and a breakpoint on the same line of code, you basically can't see the breakpoint anymore.

in the DDMS file explorer, it is not possible to multi-select files and delete them, you have to do it one by one.

To sum up these pros and cons, I would say that editing code with Eclipse was OK (although improving auto-completion would make it even better), but debugging was rather painful.

Java versus C#

I'm afraid this comparison is a bit unfair: not because I've been using C# for several years whereas I was learning Java with this project, but because I was porting from one language to the other. This means I immediately noticed every feature I use in C# that is not available in Java, but I'm not aware of some other features that may be supported in Java and not C#. So, you've been warned, and hopefully the simple fact that I was ready to learn Java somehow shows that I'm not into language flame wars and that sort of things (even if I don't hide I love C#).

The first thing I found out is Java and C# are more similar than I thought. They don't just have many features in common (such as garbage collection, even if it's not necessarily based on the same implementation), the syntax is often almost identical. Some keywords are different of course, but lots of times I could just take my C# source file, copy it into the Java project, do some renaming (replacing "const" in C# with "final" in Java, bool with boolean, override with @Override, etc), compile, and then fix the few errors I was getting. That said, there are also some real differences, and I found some of them truly annoying.

The main one is structs. Unlike C++ and C#, Java doesn't have structs (everything is a class). Why is that annoying? Because of the copy semantics of structs versus references. In C#, if I do something like this:

Vector2 position = otherPosition;
position.X = 1f;

and Vector2 is a struct, I'm basically copying the value of otherPosition to initialize my "position" variable, and then I'm modifying the latter. Now since Java doesn't have structs, and I wrote a Vector2 class instead, the same code puts a reference pointing to the same object as otherPosition in my new variable, and then modifies the value of the object referenced both by position and otherPosition. I just modified the object accessed through otherPosition in Java, although I was absolutely not doing that in C#!

This ended up being the source of most bugs I had to fix in my Android port, because XNA relies on a few structs you use all the time: Vector2, Vector3, Rectangle, Color and Matrix are the ones that come to mind. Of course, I would have overloaded the assignment operator (and a few others) of my Java Vector2 class to make it behave like a struct if it was possible; but Java doesn't support operator overloading.

Java doesn't have out and ref parameters. Therefore, if you want a function to return more than one value, for example two integers representing the row and column indices of an item in a grid, you need to do it another way. One suggestion I read online is to pass an array to the function (in my example, an array of 2 ints), and I used that method in a few places. But it's kind of a hack, and of course the compiler won't guarantee that values have been set for all the elements of the array, whereas the .NET compiler will tell you if an out parameter isn't assigned a value for some code path in the function.

Java doesn't have the equivalent of C#'s properties. It's obviously easy to rewrite them as functions since properties are nothing more than a writing convenience, it just takes some time since any C# program usually has a lot of them (and you also need to change the places where they are called, and add parentheses everywhere).

Every function is virtual in Java, and everything is public by default. Seriously? These two points didn't create any problem for me, but I still felt like adding the private keyword everywhere I omitted it in C#. I don't know who had this brilliant idea, but to me it just seems wrong that broken encapsulation is the default, I'm sorry.

Java doesn't have unsigned types. I didn't see that one coming, and I'm still surprised that if you're only dealing with positive non-fractional numbers, you would want to lose half of the range of values you can store in a given type. Weird.

Java cannot do a switch statement on a string. This is a nice addition in C#, for example when you're loading an xml file and need to do different things depending on the value of an attribute you just read.

Java doesn't have delegates (more or less the equivalent of C++ function pointers, but type safe). I'm sure there are a bunch of ways to work around that, mine was to use an interface with just one function, and a nested class implementing it. Then, instead of passing the delegate to a function, you pass an object of the nested class.

Java forces you to declare all the different types of exceptions a function may throw. I found it pretty annoying: if you modify a function, and a new type of exception can now be thrown by the system, all the functions calling the one you modified also have to declare that exception, and so on. Except of course if you catch it at the lowest level, but that's very often not the place where you want to do it. I don't know, C# exceptions work the same way but without these extra declarations, and I can't think of a case where having this feature would have helped me as a programmer.

Finally, simple enumerations are fine, but enums where you want to assign a specific value to each element are not supported. Some people online suggest using EnumSet instead, that seemed a bit overkill to me and I replaced them with a series of constants (which is of course not as good as the original enum, since values that didn't exist in the enum can be passed to a function that now accepts an integer, without the compiler telling me something is wrong).

Like I said before, there may be some awesome features in Java that don't exist in C#; but from what I saw, C# is basically a more advanced, more convenient to use, version of Java, thanks to some of the things I just detailed. Anyway, since those languages share a lot of similarities, porting from one to the other is not very difficult, as long as you stay away from the latest additions to C# (such as Linq), and know what to pay attention to.

OpenGL versus XNA

XNA is based on DirectX, and Android uses OpenGL. Which means I had to rewrite my rendering code. I had not done any OpenGL in about 10 years, but it wasn't too difficult to get back to it: OpenGL ES 1.0 and 1.1 on mobile phones are basically the same thing as what I used on desktops at the beginning of this century, except for glBegin/glEnd that have disappeared for performance reasons. Anyway, I don't really have a preference between DirectX or OpenGL (at least one section where I shouldn't hurt anybody's feelings ;)), but here are a few notes I took:

the OpenGL way of passing predefined parameters to functions is just awful. I mean, seriously, on Android there is this one long list of values such as GL10.GL_PROJECTION or GL10.GL_BLEND, and you can pass any of them to any function since they're all integers, even if most combinations don't make any sense (for example, glMatrixMode(GL10.GL_BLEND)). Where are the enums? (I know, old interface, historical reasons, backward compatibility...)

on Android, you can call the GLSurfaceView.setDebugFlags function to get some logs and add some error checking when OpenGL methods are executed. That's great, until you move from GL ES 1.0 to GL ES 1.1 (to use vertex buffer objects), and discover that gl11.glGenBuffers throws an UnsupportedOp exception when the debug flags are enabled. Meaning they become totally unusable :(

when your application is paused, because another application moves to the foreground, its OpenGL context is destroyed. This means that if it gets resumed later on, you have to recreate all your textures and vbo's, or you will crash. This doesn't happen on WP7, because when the game is tombstoned and resumed, the whole application is restarted, it's not just the rendering context that needs to be recreated! Anyway, since I have written code to handle the Android "application life cycle", and tombstoning on WP7, I don't believe one is more difficult to deal with than the other, both are necessary evils.

OpenGL commands cannot be issued outside of the OpenGL thread. That makes sense, especially once you've been bitten.

That's about it. The main difference in fact, is that XNA adds a small extra layer on top of DirectX, and that's what makes it nicer: there are a few classes that are very useful to have, that I had to partly rewrite to make my game run on Android. I'm talking about SpriteBatch (used to batch and draw anything that uses screen coordinates, for example your game's UI), and SpriteFont (used to manage fonts). Yes, unfortunately, if you need to render text with OpenGL ES, it's Do It Yourself.

The emulator

That's where things become very painful. At first everything looked OK, while I was trying to render a static menu screen, and hook up touch input to navigate to another one. But when I finally animated something, I was shocked to see I was getting 4-5 frames per second, with only a few draw calls and almost no overdraw! Of course I went online, thinking there was something I needed to set up to fix the problem, or something along those lines. Unfortunately, this wasn't the case.

The Android emulator does a lot of things, and I suppose it's a decent or even good tool to write apps. Although, I found out many people complain it takes a very long time to start, and isn't particularly fast: this is because it emulates a real ARM architecture, instead of running an x86 version on Windows. But here is the real kicker: it doesn't use hardware acceleration for OpenGL! So, forget about your brand new ATI/AMD or nVidia video card, and suck it up: everything is very slow, even in 480*320, and some 16 bit like banding is visible if you have gradients. Unbelievable.

And that's if you use OpenGL ES 1.0 or 1.1. If you'd rather go the native route, and rely on ES 2.0, it gets worse: the emulator doesn't support it, period. Wow, just wow. So, how do people write OpenGL games on Android? Well, the only "work-around" is to use a real device, and never use the emulator. I've put work-around between quotes though, because to me it's a very limited one, for the following reasons:

deploying to a device cannot be a very fast process, which is why you normally want to do most of your work in an emulator to start with. Even if this is not as true on Android, since deploying to the emulator is not really fast either (more on that later).

the emulator allows you to test your program in different resolutions, how do you do that with a single phone? I don't know, I suppose you buy more phones.

the emulator allows you to test your program on different versions of the OS. My phone has Android 2.2, maybe I could retrofit it to 2.1 but even if that's possible I don't want to keep reinstalling it, and what about 2.3 and 3.0 if it doesn't support them? Buy more phones?

So, developing on a real device does work, but it won't allow you to test your game in as many configurations as the emulator, and that's a problem. Except maybe if you work for a company, and can get several different phones.

Now let's talk about productivity. I noticed two problems with the Android emulator:

launching it the first time is long, but after that you keep it running and just restart your application, which is much faster: this is fine, and the WP7 emulator works the same way. Well, this is fine until the deployment starts failing for some unknown reason, and you have to re-launch the emulator all the time. This happened to me more and more, and I think it has something to do with the size of the apk file: when I finally added music to my game (3 big mp3 files), it basically became impossible to not reboot the emulator each time. I looked online and tried to add some command line parameters, but that didn't fix it.

even when I didn't have to kill and restart the emulator each time, I think I noticed something else: it looks like if you make even the simplest change to your code, let's say you change the value of a boolean variable, the deployment takes 35 seconds. Since recompiling that change takes no time, I'm under the impression the whole (and eventually big) apk file is redeployed, the same way it would after a rebuild all. Whether that's what's happening or not, it sucks: on WP7, if I do a similar change, I don't know how things work (since the xap file is modified, and as big as the Android apk file), but I'm getting a real incremental deployment, meaning none of the assets are sent to the emulator, and that's very fast. On Android, once you know this won't be the case and will probably take at least 30 seconds, you switch to another program, maybe check your emails or start reading something on the internet, and when you finally come back to the emulator, the first thing you do is ask yourself: "what was I going to test, again?". This interruption totally kills your workflow, and productivity drops.

There are a few other things I could complain about, that are more related to the libraries than the emulator, such as the pinch gesture only appearing in Android 2.2, which means you have to implement it yourself if you want to support Android 2.1. But those are smaller details, and overall the libraries are very good, so let's move on to the last topic.

Compatibility

Hallelujah! Despite the big framerate issues, you made an OpenGL game and everything works fine in the Android emulator. Maybe you even already tried it on a real device, or did the whole development that way, and all you have left is to release the game on the Android market, which is a very fast operation since there is no certification process (unlike on WP7 and iOS). And then you're done (with the development side of things, as long as you're not making an update)! Or are you?

No you're not. Even if the program runs fine in the emulator, and on your phone, and on your friend's tablet, there are still many devices that won't run it correctly, or at all. This is the main development problem with Android imho: the platform is open, no two phones are identical, they don't necessarily run the exact same software, some have terrible OpenGL drivers, and therefore being compatible with a decent percentage of the devices currently available on the market is a huge challenge. And new hardware keeps being released every week...

To illustrate this point, let me give you two real cases I had with my game:

once the Android port was advanced enough, and I knew I would finish it (rather than give up), I bought a Nexus One, the phone that was recommended for development by Google at that time. First time I tried to run the game on it: everything was white, with a couple of grey squares where I normally have disabled UI buttons. It wasn't too hard to figure out what was happening: the Nexus One cannot handle textures with dimensions that are not powers of 2. Fixing it wasn't difficult enough (I rescale the textures on the fly when I load them), but this shows the fact that the program runs in the emulator, or on some phones, doesn't mean it will run on all of them (even the one recommended by Google!). What I don't understand is why the framework doesn't take care of that, it could totally do under the hood the same rescaling I do, and I would never had known about this limitation.

a player emailed me, saying he used to play on his Galaxy Tab and everything was fine, but now that he moved to a Xoom tablet he cannot save his settings or games. I don't know if this comes from the tablet running Android 3.0, or some different type of storage being used, but it's pretty surprising the file API isn't compatible. I still have to look into it, and since I don't own a 3.0 device, I'll do it in the emulator, even if the OpenGL software rendering is going to make this a painful experience.

Conclusion

Developing on Android wasn't a fun experience for me. Like I said at the beginning, it is or is going to be a big market (although, not many independent developers seem to be able to make money on it - yet?), so it's tough to ignore it; but the tools are not at the same level as on the competing platforms, by quite some margin. If Google wants to have more games on Android, and I'm talking about original games and not just ports of iPhone ones many months later, the first thing to do is fix the emulator. I don't care if it fully emulates an ARM CPU, I want it to start faster, run faster, support incremental deployment, use OpenGL hardware acceleration, and that would actually also allow it to support ES 2.0. As far as compatibility goes, this is a more difficult problem to address, and I don't believe it will be fixed anytime soon, since it's basically a consequence of the platform being so (too?) open.

Before and after screenshots

2011-03-25T18:36:00.000-07:00

I haven't written anything recently, as I was busy porting Jigsaw Guru Free to Android. I have a lot to say about this experience, but before that, here is another post I've wanted to share for quite a while: the "before and after screenshots" comparison.

I suppose this is not something developers show very often, but I think it can be interesting to see how a game evolves during its development, and what a difference real content, as opposed to placeholders, can make (this article only talks about graphics, but this is the same for music and sound effects).

Some people have multiple talents, and can create the different elements required in a game (design, code, art, sound...) on their own. I'm not one of them, I was born a programmer, and all the art I can make is coder art with underappreciated colors. As soon as I started working on Jigsaw Guru, I knew I would contract somebody to make my game look prettier (which means I didn't even try to make my best coder art ever ;)), let's see the result.

Title screen

before

after

This is the title screen and main menu. The old background (top) was generated programmatically, by separating and randomly rotating the pieces of a completed puzzle, and using a different photo for each of them. At this stage of development, "best times", "credits", and "settings" were simply not implemented yet. "Support" was reusing the Help icon, since I didn't know what to draw. The round buttons (bottom) with little blanks and tabs making them look like puzzle pieces were the artist's idea.

Picture selection

before

after

This is the first rotating menu the player sees in the game. Having a background was always part of the plan, but the black border was not; this forced me to push everything (buttons, texts, photos) a little bit away from the sides of the screen, and you may notice the photos are a bit smaller. Lesson learned: don't put your UI elements too close to the borders of the screen, even if the platform you're working on doesn't have a "title safe area" like the Xbox360.

Puzzle screen

before

after

On the puzzle screen, I used a checkerboard pattern for the longest time as the background, and also in the piece browser on the left. Initially, this was because it wasn't easy to see the shape of dark pieces on a black background, or the shape of bright pieces on a white background, you get the idea. After a while, I added an outline to the pieces to solve this issue, but somehow the checkerboard remained, until only a few days before submission. You can also notice the 4 buttons are in a different place, when the photo is horizontal like here they move to the bottom of the screen so that they don't hide the picture.

Pause screen

before

after

I still kind of like the way the old version looked: even if the image was a bit stretched (from a 4/3 to a 5/3 ratio), it was nice to see it in full screen. But depending on the colors of the photo, the round buttons, and the texts under them, were not always very readable, and the old layout didn't work well with vertical pictures. And since the photo is smaller, I got some extra space to add tips, and encourage players to discover more features (such as the helps: I was under the impression nobody wanted to open that menu, maybe people feel like they failed if they use helps? Personally, I use them all the time!)

Helps

before

after

Speaking of the devil... This is another rotating menu, that combined the problem of the photo selection one (UI had to be moved to make room for the border) with the checkerboard pattern of the puzzle screen.

Overwrite confirmation

before

after

This is an example of a false good idea: you're on the pause screen, you hit the save button to save the current game, and instead of replacing the pause screen, this confirmation shows up on top of it (if you didn't see it, look carefully at the 'before' screenshot: you can see the full screen photo, and the 'save game' and 'help' buttons, 'behind' the current screen). In the best case scenario, this doesn't make much difference, in the worst case, it's confusing and less readable (not to mention: inconsistent with the rest of the application).

That's all folks

Hopefully this post convinced you presentation is important, and there's nothing wrong with working with other individuals who are more talented than you, while you focus on the things you like and do best. Also, when you program menus and other non-gameplay screens, try not to use a black uniform background: once you replace it with something else, you might discover some UI elements or texts don't work well with the new colors or patterns, and you'll have to spend some extra time to fix them. Believe me, I've been there.

Free is good

2011-02-07T15:00:00.000-08:00

At the end of the postmortem I wrote for my Windows Phone 7 game Jigsaw Guru, I mentioned I was about to submit a free, ad supported version of the application, since the downloads and sales of the paid game were extremely low. When Jigsaw Guru Free was released on December 29, I said I would write another post after a few weeks, and share some results. This is the post you are reading, I now have all the data related to the first month (12/29/2010-01/28/2011) of Jigsaw Guru Free on the marketplace, so let's get started.

Downloads

This should speak for itself:

Figure 1: downloads for Jigsaw Guru (10/23-01/28)

Figure 2: downloads for Jigsaw Guru Free (12/29-01/28)

The paid version got 103 downloads in over 3 months, the free one got 13 109 in its first month. That's 127 times more, in a shorter period of time. And if you look carefully at the first figure, you will notice the download rate for Jigsaw Guru increased a bit before the beginning of the year: that is because of the free version, even if it doesn't contain any information or link to the paid game (which was a mistake on my part), some people searched for and found the initial product after seeing the new one.

I don't want to state the obvious, but this shows your game has a much bigger chance to find its audience if it's free, and if it's not you probably should make a light but free version of it to let people know about your paid application. This is the same principle as on the iPhone, and as I mentioned in the postmortem, the trial mode implemented by a lot of Windows Phone 7 games (including mine) is not a good substitute for a free version, that system as it is right now unfortunately doesn't work (because it doesn't give your game any exposure outside of the section for paid apps, which has much less traffic).

Ad impressions

Of course it's good if a lot of people download your game, but if they try it once and uninstall it immediately, or never play it again, then what's the point? I don't have any analytics system built into Jigsaw Guru Free, to keep track of the number of active players, and sessions, and a bunch of other things, but I have ad impressions. That basically tells me how many ads were displayed by the game every hour, I just can't see if it's a small number of people playing for the whole hour, or a lot more players only spending a few minutes in the program.

What's interesting is the number of ad impressions has been very stable for me so far. Of course it goes up a bit during the week end (people have more free time), and down in the middle of the week, but the changes are not drastic. Jigsaw Guru Free started just under 9000 impressions per day, is now doing between 16k and 17k most of the time, and served close to 420k ads in its first month.

eCPM

The eCPM (effective cost per thousand - shouldn't that be eCPT?) is basically how much money you get for 1000 ad impressions, so, that's a very important number. Unfortunately, unlike the impressions themselves, it can vary a lot from one hour to the next (or at least that's what I saw in January). Which makes it difficult, if not impossible, to predict how much ad revenue you're going to get the next day, since revenue = impressions / 1000 * eCPM.

Many factors have an influence on eCPM, I only know a few of them, and some are out of our control: for example, if a lot of advertisers are running big campaigns at the same time, eCPM will go up, since there is more demand for ad space; on the opposite, if it's a more quiet period, eCPM goes down, like it did in January. As developers, the main parameters we have access to with the Microsoft ad SDK and the corresponding pubCenter website are ad categories, application keywords, and phone location.

When you create an ad unit, you can select up to 3 categories of ads you want to receive. These can be really large categories such as "Sports", or more targeted subcategories like "Sports - Skiing". This selection seems to be the most important step in getting a good eCPM, but there is no list of what the 3 best categories are, and if there was one it could change over time anyway. The good thing is: categories are not hard coded in the application, they are defined on pubCenter, which means you can change them at any time without submitting an update, and the new categories are used instantly or in the next hour as far as I can tell (I changed mine one time in the middle of January). You can experiment as much as you want, maybe changing one category at a time, but don't forget that many other factors affect eCPM, and you're not necessarily always comparing apples with apples.

I have one piece of advice though: don't choose categories based on what your application or game does. That's what I initially did, because that sounds like a good idea to have a better click-through rate (CTR, percentage of ads being clicked / tapped when they are displayed), and for example one of my categories was "Travel", since the photos provided in Jigsaw Guru Free all show places you may want to visit while you're on vacation. I'm not saying Travel is a bad category, or Astronomy is a bad category (someone in the forums mentioned he was using that one), but how often do you see advertisement about travel and astronomy, compared to some other products? So, my advice is to just look at commercials you see a lot on TV or in magazines, and pick the categories they would fall into: if companies making these products spend a lot of money advertising them in other media, there's a good chance they (or their competitors) will do the same on mobile phones. CTR may be one of the factors secretly influencing eCPM, but even if it does, categories have so much weight right now that it makes more sense to focus on them first.

Application keywords can be specified in your code, which means you cannot change them without submitting an update. This is not a problem for now: they're not taken into account by the current version of the Microsoft ad SDK anyway (this could obviously change in a future release).

Finally, your program can provide the phone's location to the ad manager (XNA) or control (Silverlight), and this can increase your eCPM if some campaigns are specifically targeting the area where the user is when he's playing your game. I have no idea how much of a difference this can make, but it cannot hurt. Just be careful and re-read the certification requirements if you implement this: you need to ask customers if they want to allow your app to use the location or not, and they should be able to opt out at any time.

Ad revenue

How much money did I make in one month with Jigsaw Guru Free? I won't give the exact amount, but here is what I can say:

much more than I did with the paid Jigsaw Guru. Especially since the latter is still very far from reaching the payout threshold, and I may very well never see a check for it.

with about 2 million phones in circulation right now, and 13k downloads of my only game on the WP7 so far, the ad revenue doesn't pay all my bills. This is to be expected, let's be realistic, I'm not disappointed at all.

on the contrary, the good news is what I made in January will pay for my rent in Southern California, in March (since I should get that money at the end of February). This is a pretty good start, and with a couple more projects on the market, or some ports of the same game to other platforms, I could eventually be able to balance my budget in a few months if everything goes well. It's certainly not a done deal though, this will depend a lot on how eCPM evolves in the future, how successful my next game is, etc. But there is hope :-)

Free is good

Making a game available for free has a couple more advantages I want to mention. The first one is related to piracy: since the game is free, I don't have to worry about people eventually managing to install and play it without buying it, actually the more people have it the happier I am.

The second one has to do with Android: I don't know how that happened, but apparently customers on that platform are used to getting everything for free, to the point that it's now an expectation, and even Angry Birds was released as a free ad supported game. I find it kind of sad that people are not willing to spend a few dollars on a high quality product (after all, the persons who developed it need to earn a living, don't they?), but at least if I ever want to port something to Android I won't have a problem with the $0 price tag, since that's the one I would choose anyway.

Free is good. My next game will definitely follow that path, the only question is whether I will also have a paid version with some extra features and without ads, or not.

The main differences when porting a game from Windows Phone 7 to Xbox360

2011-01-30T21:40:00.000-08:00

Since the Xbox360 version of my game Jigsaw Guru was released on the marketplace a week ago, I thought I could write a post about the main changes I had to make to go from a mobile device to a living room console. Those changes basically fall under 3 categories: screen, input, and storage.

Screen

A TV screen is obviously bigger than the screen of a phone, but more importantly it has a different resolution and aspect ratio. The difference can actually be huge, if the game runs in portrait mode on the phone:

Jigsaw Guru on Xbox360, 1280*720

Jigsaw Guru Free on WP7, 480*800

Fortunately for me, the initial paid version of Jigsaw Guru runs in landscape mode on the phone, but going from 800*480 to 1280*720 still means all the graphics elements have to be scaled and positioned according to the width and height of the screen, which I do with code looking like this:

successRect.X = (int)(Renderer.Instance.DeviceWidth * 0.1f);

successRect.Width = (int)(Renderer.Instance.DeviceWidth * 0.8f);

successRect.Y = (int)(Renderer.Instance.DeviceHeight * 0.4f);

successRect.Height = (int)(successRect.Width * completedText.Height / completedText.Width);

spriteBatch.Draw(completedText, successRect, Color.White);

These lines are used to display the "Puzzle Completed" texture; successRect is an XNA Rectangle, Renderer.Instance is a singleton I use for all my rendering code, and as you can see all the dimensions I need are expressed as a percentage of the width or height of the screen. This is a pretty convenient system, for example tweaking the value returned by the DeviceHeight property was enough to make room at the bottom of the screen for the ad banner of the free version, and force the game to only use the top 90%. This won't solve every single case though, sometimes scaling the UI is not enough and a different layout needs to be used; but even if you're only working on 1 platform, it's better to not hardcode your sizes and positions in pixels, in case you want to change the resolution or support several aspect ratios (such as 4/3 versus widescreen) later on.

Televisions have a very annoying property: they don't display the whole image your console sends them. Depending on the model, up to 10% of the screen can be missing on each side, and you should therefore only draw important information in the remaining 80% zone in the middle, which is known as the "title safe area" (this is a certification requirement, if you don't follow it your game will not be approved on Xbox360). This actually took a big portion of the time I spent porting Jigsaw Guru to the console: I couldn't use the previous trick to make the game fit in the title safe area, because 1) you don't want to have a big black border on televisions that can display the whole image or most of it, 2) you don't want to only use 80% of a phone screen that's small enough already.

For this particular problem, I still tried to avoid having too much platform specific code everywhere I draw text and icons, by using a mix of small functions and constants like the following ones:

#if XBOX

public const float textTop = 0.10f;

public const float textBottom = 0.90f;

#else

public const float textTop = 0.07f;

public const float textBottom = 0.93f;

#endif

For example, the "Select Game to Load" title in the previous screenshots is positioned like this:

textPosition.Y = Renderer.Instance.DeviceHeight * textTop;

instead of directly using DeviceHeight * 0.07f like I could do it if I was only developing for the phone. This makes sure the text is low enough to be entirely contained in the title safe area on the Xbox360. Also, to verify everything is correct on all the screens, I have a boolean I can toggle in debug mode to draw the title safe area limit as a red line:

Input

Playing with a gamepad is of course totally different than playing with a touch screen, no surprise here. Therefore, the main input related change in the Xbox360 game is that the player moves a cursor on the screen to drag and drop puzzle pieces, instead of moving her finger. Also, some icons became unnecessary: for example, you don't tap a pause icon to go to the pause menu, you just press the Start button on the gamepad. But the modifications don't stop here.

The console can have up to 4 controllers connected to it, and a player must be able to pick any of them to play. That's why every Xbox360 game has a "start screen" before its main menu, where the controller used to press Start is detected, and remembered for the rest of the session (changing controller mid-game doesn't have to be supported, it's not a requirement).

In menus, you have to keep track of a "currently selected" item: on the phone, any icon can be tapped at any time, but on the console the player can only navigate to the previous or next item one step at a time.

Finally, games usually have a "controls screen" like the following one, and I didn't have anything like it on the WP7:

Storage

Saving settings and games to isolated storage and reloading them is fairly easy on WP7. On Xbox360 however, this is a different story: players may or may not have memory units, if they do they need to be able to select which storage device (including the hard drive) they want to use, and the game should never crash even if the selected device is removed at any time (while playing, loading, saving, you name it). In my opinion, this is the most annoying requirement to fulfill, it's not particularly difficult to implement it, but it's hard to think about all the different cases and places where something might go wrong and test them. Here are two things you can do to make your life easier, and hopefully avoid failing during peer review.

Use the EasyStorage library. Lots of Xbox Live Indie Games developers do it, and that's what I did too, even if I wouldn't have needed it for the WP7 version. I don't use the asynchronous functions because the files I read and write are very small and don't require any kind of progress bar, but I know they're here in case I want them in a future project.

Protect all your file accesses with try/catch blocks. Even checking if a file exists can fail if the storage device was removed, so don't be shy, and better be safe than sorry. Try to give some feedback to the players if you can, so that they don't assume their game was saved if it wasn't, they would hate you for that the next time they play (I know I would).

The other way around

What about going the other way around, and porting an Xbox360 game to the WP7 platform? Well, I haven't done that so I don't have much experience with it, but from the top of my head there are two phone specific things you'll have to pay attention to, in addition to what I have mentioned in this article: tombstoning, and the use of the Back button. Those are actually 2 of the main reasons for failing submission on the WP7, even when you're writing a game directly for that platform, so don't wait until the last minute to take care of them, the earlier you know what modifications they will require in your project the better.

Parallel.For: easy multithreading in C#

2011-01-17T18:54:00.000-08:00

Quest for speed

Last December, I experimented for a while with SPH (Smoothed Particle Hydrodynamics), using C# and XNA as usual. Of course, the more particles you can simulate the better, and I tried to optimize for speed the different parts of the method as much as possible, which even led me to use a SOA (Structure of Arrays) rather than the more natural AOS (Array of Structures) for my data, since I basically ended up being limited by the speed of memory accesses and cache misses. Anyway, I thought I had done my best and I had no plans to multithread this code, when I ran into an explanation of Parallel.For and Parallel.ForEach on the internet, and decided to give it a try.

The original code

Let's start with a simple function:

private void MoveParticlesSOA(float deltaTime)

{

Vector2[] positions = particleSystemSOA.Position;

Vector2[] forces = particleSystemSOA.Force;

float xMax = dimensions.X;

float yMax = dimensions.Y;

for (int p = 0; p < particleSystemSOA.NbParticles; p++)

{

Vector2 position = positions[p];

if (position.X < 0)

position.X = dimensionEpsilon;

else if (position.X > xMax)

position.X = xMax - dimensionEpsilon;

if (position.Y < 0)

position.Y = dimensionEpsilon;

else if (position.Y > yMax)

position.Y = yMax - dimensionEpsilon;

positions[p] = position;

particleSystemSOA.UpdateParticle(p, deltaTime);

forces[p] = Vector2.Zero;

}

What this code does is for each particle, it first makes sure the position is inside a 2D rectangle (these are the if/else lines), then it asks the particle system to move the particle, and finally it resets the forces applied to the particle to zero (for the next frame). Since there are potentially a lot of particles to deal with, how would we go about multithreading this function, so that it can take advantage of 2, 4, or more cores?

Parallel.For

This is what it looks like with Parallel.For, after I added some #if/#else/#endif statements to easily enable or disable multithreading:

private void MoveParticlesSOA(float deltaTime)

{

Vector2[] positions = particleSystemSOA.Position;

Vector2[] forces = particleSystemSOA.Force;

float xMax = dimensions.X;

float yMax = dimensions.Y;

#if PARALLEL

Parallel.For(0, particleSystemSOA.NbParticles, p =>

#else

for (int p = 0; p < particleSystemSOA.NbParticles; p++)

#endif

{

Vector2 position = positions[p];

if (position.X < 0)

position.X = dimensionEpsilon;

else if (position.X > xMax)

position.X = xMax - dimensionEpsilon;

if (position.Y < 0)

position.Y = dimensionEpsilon;

else if (position.Y > yMax)

position.Y = yMax - dimensionEpsilon;

positions[p] = position;

particleSystemSOA.UpdateParticle(p, deltaTime);

forces[p] = Vector2.Zero;

}

#if PARALLEL

);

#endif

}

Simple, isn't it? The 'for' line is replaced with Parallel.For, and what used to be the loop's code is now a delegate that can be called on several threads simultaneously for different particles. All the synchronization is taken care of automatically, it's pretty awesome if you ask me!

Shared memory

Parallel.For and the other methods of the Task Parallel Library don't do anything special regarding shared memory, and it's the programmer's job to protect the latter from data races and simultaneous accesses from different threads. Let's see another function from my SPH class as an example (sorry it's a bit long, there's no need to read it line by line as I will explain the important parts right after the code):

private void UpdateDensityAndPressureSOA()

{

Vector2[] positions = particleSystemSOA.Position;

float[] densities = particleSystemSOA.Density;

float[] pressures = particleSystemSOA.Pressure;

short[] gridIndex = particleSystemSOA.GridIndex;

for (int p = 0; p < particleSystemSOA.NbParticles; p++)

{

densities[p] = 0f;

neighborCount[p] = 0;

}

#if PARALLEL

ParallelOptions options = new ParallelOptions();

options.MaxDegreeOfParallelism = -1;

Parallel.For(0, particleSystemSOA.NbParticles, options, p =>

#else

for (int p = 0; p < particleSystemSOA.NbParticles; p++)

#endif

{

int nbNeighbors;

int[] neighbors = grid.GetNeighbors(particleSystemSOA, gridIndex[p], out nbNeighbors); // NOT GOOD FOR MULTITHREADING

for (int n = 0; n < nbNeighbors; n++)

{

int neighborIndex = neighbors[n];

if (neighborIndex < p)

continue;

if (neighborIndex == p)

{

densities[p] += selfDensity;

continue;

}

float deltaX = positions[p].X - positions[neighborIndex].X;

float deltaY = positions[p].Y - positions[neighborIndex].Y;

float r2 = deltaX * deltaX + deltaY * deltaY;

if (r2 < h2)

{

float diff2 = h2 - r2;

float density = poly6FactorMass * diff2 * diff2 * diff2;

densities[p] += density;

densities[neighborIndex] += density;

float r = (float)Math.Sqrt(r2);

if (neighborCount[p] < maxNeighbors)

{

int tableIndex = p * maxNeighbors + neighborCount[p];

neighborTable[tableIndex] = (short)neighborIndex;

neighborDist[tableIndex] = r;

neighborCount[p]++;

}

if (neighborCount[neighborIndex] < maxNeighbors)

{

int tableIndex = neighborIndex * maxNeighbors + neighborCount[neighborIndex];

neighborTable[tableIndex] = (short)p;

neighborDist[tableIndex] = r;

neighborCount[neighborIndex]++;

}

const float restDensity = 100f;

const float gasConstant = 0.1f;

pressures[p] = gasConstant * (densities[p] - restDensity);

}

#if PARALLEL

);

#endif

}

I won't go into the details of this function since they're not relevant to the current discussion, as the name indicates it basically calculates the density and pressure of the fluid at each particle's position. But there are two interesting things we haven't seen yet that I want to discuss.

The first one is just after the first #if PARALLEL line: as you can see, options can be passed to Parallel.For, such as the maximum number of threads you want to be used. Here I specified -1, which is the default and means all the available cores will be used, but you could pass 2, 4, etc.

The second topic I need to talk about is the line with the NOT GOOD FOR MULTITHREADING comment. This line in itself has no problem, but it calls a function of a grid that's used to find the neighbors of a particle (that is, other particles that are within a given radius of the current one) without having to test all the particles each time. Since a particle usually has a bunch of neighbors, the GetNeighbors function would initially store their indices in an array that was part of the grid object, and return a reference to that array. Once this function can potentially be called by several threads at the same time, this approach doesn't work anymore: the threads all try to access the same piece of memory simultaneously, and you can trust me when I tell you the SPH simulation became totally unstable as a result.

How do we fix that? Well, since I was only trying Parallel.For for the sake of trying it, I did the simplest thing I could think of just to verify that would fix the simulation and there wasn't any other problem: GetNeighbors allocates and returns a new array each time it is called, which means each thread gets its own piece of memory, each time it calls the function (that's the annoying thing, since one thread could totally reuse the same array for all the particles it handles). That's a lot of allocations per frame, that sounds crazy, and I don't recommend doing it in real code, although we're going to see in the next section that it wasn't so bad after all.

Performance

So far we've seen that Parallel.For is very easy to use, definitely much much easier than managing a bunch of threads manually, managing a bunch of BackgroundWorker objects, or using the ThreadPool class (even if I haven't shown these methods in action, and I'm not saying they don't have some valid usage cases). But what about performance?

Unfortunately, I don't have numbers to give you, since like I mentioned before, my algorithm was already memory bound when running on one core, and processing power was not the issue. But let me say this: despite the memory bottleneck, and all the extra allocations made in the GetNeighbors function, the multithreaded version of my SPH simulation still managed to run faster than the non-multithreaded one, which is a great sign in my opinion. And eventually also shows allocating memory is crazy fast in C#, and the garbage collector does a good job at quickly getting rid of short lived objects, at least on PC.

I really encourage you to read a bit more documentation about the Task Parallel Library, and try it for yourself on some existing code that could use more multithreading, I think you'll be happily surprised.

Jigsaw Guru Free released today

2010-12-29T12:26:00.000-08:00

The free, ad supported version of my Windows Phone 7 game Jigsaw Guru was approved today, and is ready to be downloaded from the marketplace. It's basically the same game, with no limitation (meaning: you can use any picture from your phone as a jigsaw puzzle, in addition to those provided in the application), and it's free. But there are actually a few more differences between the two versions:

The most obvious one is that Jigsaw Guru is a lanscape mode game, whereas Jigsaw Guru Free is in portrait mode. I made this change for two reasons: first, the Microsoft ad SDK only serves horizontal banners for now, which were clearly designed for portrait mode applications and are easier to fit in the screen that way. Second, the submission process and the Zune software only support vertical screenshots, which makes it less convenient to look at landscape mode games in the catalog. Reorganizing all the screens took a bit of work, but I think it was worth it.

I added tips to the pause screen. They rotate automatically, but can also be glanced through by tapping them.

The piece browser displays 8 pieces instead of 6.

I removed one of the icons from the puzzle screen, since none of my testers was using it, they couldn't remember what it was for, and I had a hard time explaining it.

I'm curious to see how many people will download Jigsaw Guru Free, and I'll try to write a post with some numbers in a few weeks. If you've read the game's postmortem, you already know it can only get better :)

Jigsaw Guru for the Windows Phone 7 postmortem

2010-12-22T22:50:00.000-08:00

New beginning

When my job position got terminated last summer, I decided I would take a break from working for big companies, and try to make a game on my own at home. I've been in the video game industry since the late nineties, I've written code for a few released PC and console AAA titles and probably as many canceled ones, and I wanted to do something different at least for a few months. Go back to the roots of game development, when one or two guys could create something in their garage or bedroom, without spending years and millions of dollars on it.

Platform choice

I'm a C# fan, I won't deny it, I even explained why back in 2007. I also toyed with XNA each time a new version was released, and I like how straightforward it is to use. When the beta version of the tools to develop for the Windows Phone 7 (WP7 for short) became available, just at the right time for me, it seemed like the obvious choice for what I wanted to do. And developing for a brand new platform sounded exciting as well, even if it was perhaps a riskier bet than targeting well established markets.

For those who don't know much about WP7 development, I just want to highlight a few key points:

Applications are written in Visual Studio using C#, and either XNA or Silverlight. This may come in the future, but at this point you are not allowed to mix them, and use the controls of Silverlight with the faster rendering of XNA, for example.

In App Purchases are not supported yet.

Programmable shaders are not available for WP7 (they obviously are for the PC and XBox360) in the current version of XNA. Instead, you have 5 predefined shaders with features that can be toggled on and off (such as fog or per pixel lighting). These shaders should cover most of the usual needs, but if you want something very specific that they don't support, you're out of luck (or have to fall back to multiple passes and alpha blending).

Paid applications have a trial mode, that allows users to evaluate them before making a purchase. Unlike on XBox Live Indie Games (XBLIG), where the game just abruptly stops after 8 minutes, developers have a complete control over what they want to show in trial mode (for example, the first few levels of a game, with no time limit, or without the ability to save, etc). In theory, this means you shouldn't have to make a free version of your game in addition to the paid one, but I'll come back to this later.

The certification requirement document is only 27 pages long, table of content and change history included.

The project

I wrote a few ideas on paper, but they all looked too ambitious for a first project. One thing in particular that I wanted to avoid was to require a lot of content, since I didn't even know how I was going to find an artist yet. During a bike ride, I thought about using photos, and that's how I decided to make a jigsaw puzzle game. Not super original, I know, there are several of those on the iPhone and iPad, but there was none on the WP7 since it wasn't released yet :), and it looked like a good project of the right scope to keep things under control and learn the platform. For example, I had never worked with a touch screen device, and it was pretty obvious from the start that I was going to spend quite some time iterating on the user interface part of the game.

The game is called Jigsaw Guru, and you can find more screenshots and a video at http://www.fairyengine.com/.

What went right

I actually already mentioned some of the things that went right in my opinion, but here is the full list:

Working on a short project, without too much content to integrate, instead of picking another more ambitious and time consuming idea, was the right choice. My goal was to have Jigsaw Guru on the marketplace the day of the WP7 launch, I submitted it on 10/18 and it got approved on 10/23 (the update I submitted in December passed even faster than that), two weeks before the US launch and two days after the other territories started getting their phones. At times I still had more items on my to-do list than I would have liked, and I made a good push in the last 3 weeks to get everything wrapped up in time, but it never got too crazy.

Knowing the tools beforehand, and using C# and XNA, definitely made me feel productive. The phone emulator was surprisingly stable in the beta, and the XNA forums were very helpful when I wasn't sure about something. It's also worth noting that the website used to submit bug reports allows everybody to see what other people have written, and vote on issues you think are important and should be fixed first.

Garbage collection is the one thing that can hurt your framerate really bad when using XNA on XBox360 and WP7. But it's not that hard to avoid: when I had a look at my memory usage late in the project, I only found two spots where I was creating some amount of garbage each frame (both related to strings). Basically what I do is each time the player goes from one screen to another, I load everything I need for the new screen, and I force a garbage collection. This gets rid of temporary objects that may have been created during the loading process, and objects from the previous screen that are not used anymore. After that, I don't allocate anything, and the garbage collector never has to do any work.

Finally: testing. Very few people tested my game while I was developing it, but my friend Alexis alone kept giving me more feedback than I thought I could handle. He never said anything was good just to make me happy, he reported every little detail he didn't like, even and especially the ones I was aware of but never had time to fix (and maybe thought nobody would notice). Even if I sometimes felt he was nitpicking, this was exactly what I needed.

What went wrong

Developing for a new platform has its share of drawbacks, it's just how it is. The final version of XNA 4.0 only came out mid-September, and before that I was working with the beta version. As mentioned before, it was pretty solid already, but there was no redistribution of the runtime. Which means you had to install all the tools to be able to just run the game, and you could only install them under Vista or Windows 7 because the phone emulator requires DirectX10, and some of my potential testers were still using Windows XP. So, even if what I was sending was a PC version of the game, they couldn't run it on XP just because the phone emulator, which they didn't need, requires a more recent operating system. To work around this problem, I made an XNA 3.1 PC version of Jigsaw Guru, but maintaining two versions side by side for several weeks and merging back and forth between them was not fun.

Coding for a device that wasn't released yet also meant I could not test what I was doing on real hardware. I unsuccessfully tried to get a phone prototype, and basically the only way I found to run my game on a WP7 at the end of September was to go to a Microsoft event. Even there, I thought there would be phones locked to the tables, that developers could connect to their laptop and use for the whole day, but I was wrong: there was no such thing, and the 4-5 persons who already had a working application to test like me had to do it on the phone of the event's organizer, when he wasn't using it for something else. I was able to run my application twice for about 5 minutes each time, and fortunately everything seemed to be working fine, which is a testament of the reliability of the emulator.

When the game began to be really playable, with most of the screens that are in the final version, I started looking for an artist and a musician. I did it way too late, and I was lucky to find Aimee and Tim through the XNA forums, and that the time estimates they gave me just matched my own programming deadline. So, in the end everything worked out well, but even if I wanted to wait and be sure the UI elements were final before asking somebody to redo them, I believe I also waited just because I was more comfortable doing my programming tasks first, and that was a mistake.

Marketing was a difficult problem: I knew I needed to do some, but how? When Jigsaw Guru got approved, the WP7 wasn't out yet in the US, and had just been released in a few countries of Europe and Asia, as well as Australia and New Zealand. Basically, nobody had that phone yet, and I couldn't find any website reviewing its games. Even when I had a second look for the US launch, I only saw a couple of sites testing 2 or 3 of the official XBox Live games (that's how they're called even if they run on a phone), but nothing related to indie games. So, what ended up happening is I did nothing, and just hoped that because there weren't too many applications on the marketplace yet (which is the reason for being a launch title, right?), people interested in jigsaw puzzle games would fine mine. Of course I was wrong, and the fact that the search in the Zune software didn't seem to care about the keywords you enter when submitting your app, or couldn't find my game when searching for 'jigsaw', didn't help (this seems to be fixed now). I don't know how other developers releasing WP7 games around the same time as me handled their marketing, but I would be curious to hear it, that question is still a mystery to me.

Downloads and sales

This could have been another paragraph in the "what went wrong" category, but I think this subject deserves its own section. Download and sale numbers for WP7 apps became available on Wednesday 12/08, and I don't think anybody was thrilled. Don't get me wrong: WP7 is a very young platform, there aren't tens of millions of phones in circulation, sale expectations have to be low for now, and can certainly not be compared with those of competing platforms. No, what really surprised me is not the number of sales, but the number of downloads.

On the "top selling" tab of the Zune software (which is more likely based on downloads, like the Bing visual search), Jigsaw Guru has always been in the middle of the paid games from the puzzle & trivia category, the biggest category by far (about 45% of all games). Today for example, it's 174th out of 363. How many downloads do you think this means on average? About 0.65 per day. This totally blows my mind, but is in line with a graph a developer made with the numbers from his 4 apps, that shows the relationship between the marketplace rank and the number of downloads.

One potential problem is it's not obvious enough that lots of paid games support the free trial mode, and some players don't know they can download them for free. But really, it all comes down to most players only checking the XBox Live and the free sections, and rarely visiting the paid one. Which is why although I said earlier that in theory, and thanks to the trial mode, you shouldn't need to make a free version of your paid game, in practice it's obvious this doesn't work, and you have to do it. My friend Noel, who is an experienced iPhone developer, had actually told me I should release my game as a free, ad supported app; unfortunately, the ad SDK's for XNA didn't exist at launch and are still in their infancy, the one from Microsoft got released this month and only serves ads in the US, the other ones are either not finished or not stable yet.

So, what's next? I'm submitting Jigsaw Guru Free (with the Microsoft ad SDK) this week, because as a developer, I still want people to play my game and hopefully have a good time with it. The two projects I started working on after Jigsaw Guru might follow the same path and be free apps from the start, whether it's on the WP7 or another platform.