The Android experiment: porting my XNA game to Google's platform

In this post, I'm going to talk about my experience porting my XNA game Jigsaw Guru Free from the Windows Phone 7 platform to Android. I will focus on the development side of things only, marketing a game on Android and the fact that there are multiple market places (including the quite recent Amazon Appstore for Android) would be a totally different topic.

Why?

Why try a new platform?

Once I released Jigsaw Guru on WP7 at the end of October 2010, I started working on an update, and prototyping a couple of other ideas. When the first sale numbers became available in December, and I saw how low they were, I stopped what I was doing and spent a few days making Jigsaw Guru Free, a free ad-supported version that got published just before the New Year. And I finished the Xbox Live Indie Game version of the program.

Then what? I could have worked on another WP7 game immediately, but the initial results from the ads were neither bad nor very encouraging, until I changed my ad categories and eCPM went up a few weeks later. Since I already had a full game implemented, with all the necessary assets (that I didn't make myself, since my artistic skills are extremely limited), another option was to port that game to a different platform, with a larger user base.

Why Android?

I basically had to choose between Android and the iPhone. Since I don't own a Mac (I know there are a few ways to develop on iOS without one, but ultimately if you want to have access to all the libraries and not be limited to some specific middleware SDK, I think you need one), I decided to try the Android tools for Windows. Android is supposed to be the fastest growing mobile market these days, and lots of tablets are going to be sold this year, so why not?

Why Java?

On Android, you can write code either in Java, or in native C/C++. On one hand I have many years of C++ experience, on the other hand I only read a book about Java a few years ago to see what the syntax looked like, so why did I choose to use Java? One reason was learning a new language can be fun :). Another one is C# and Java both have garbage collection, and therefore Java was a better match for the code of Jigsaw Guru, that doesn't care about deleting objects manually all the time. I would have used C++ if there had been a good reason for that, but what are the reasons to use native code on Android?

• reusing an existing C/C++ code base. Well, Jigsaw Guru is written in C#.

• performance. My game sometimes has to draw a bunch of stuff, but on the CPU side there's not much happening, I thought Java would be fine.

• access to OpenGL ES 2.0. Yes, on Android you can only use GL 1.0 and 1.1 from Java, if you want to call 2.0 you have to do it from native code. But since I only used 2 different shaders from XNA (BasicEffect and DualTextureEffect), it was no big deal emulating them with the fixed function pipeline.

So, in the end, I didn't have any good reason to use native code, and reading the online documentation gave me the impression installing the native SDK (called NDK) on top of the rest was not super fun, and that debugging native code was not well integrated into Eclipse (the development environment).

Why not MonoDroid?

When I started developing on Android, MonoDroid (an SDK for running C# code on Android) was not ready yet. Even if it had been, I don't think it would be that useful for a program like Jigsaw Guru: sure, it would take care of running the C# logic of the game, but it wouldn't auto-magically remap the WP7 input, sound, file system, and rendering to the Android platform (and I'm not even talking about tombstoning, or the WP7 Chooser Tasks). For that, you would need something like ExEn, which wasn't ready for primetime either.

Installation

The installation process can't be easier for the WP7 tools: one executable, and you get Visual Studio Express 2010 if you don't already have it, XNA 4.0 (which also allows you to develop on Windows and Xbox360), the WP7 emulator, etc. It takes a while, but you can go do something else in the meantime.

For Android, there are a few more steps, since I had to install: the Java Development Kit (JDK), Eclipse, the Android SDK, the components for the versions of Android I wanted to be able to use (for example: 2.1 + 2.2 + 2.3, and now 3.0), and ADT (the Android Development Tools, which are basically a plug-in for Eclipse). Overall it went quite well, the only problem I had was with the JDK: there are 32 bit and 64 bit flavors, guess which one I picked since I run Windows7 64 bits; but apparently some of the tools are hard coded to use the 32 bit version, and I had to install that one in the end or nothing would work.

Eclipse versus Visual Studio

I've been using Visual Studio for a very long time now, since version 4.2 I believe. It's a very good code editor in my opinion, and probably the best debugger used in video game development. So, how does Eclipse (which is free) compare to Visual Studio Express 2010 (also free)? Let's start with the things I definitely liked:

• the suggestion window. If there is an error in the code, and you put the mouse cursor on it, a floating window shows up with suggestions on how to fix the problem. Here is an example, where I started declaring a local variable "m" of an unknown type "Matrix":
  
  
  
  
  As you can see, each of the suggestions is a link, that you just have to click to do what it says. I don't know if and how the suggestions are sorted, but the one I needed was always in the top 2 as far as I can remember. As a comparison, here is what I get in Visual Studio:
  
  
  
  
  Then I can right-click and choose to resolve the missing reference, or generate a new class or type called Matrix. But this is really not as informative and convenient as what Eclipse does.

• the bookmark window. I use bookmarks quite frequently to switch between 2 to 5 different places in my code. In Eclipse, I can give a name to each of them when I create them, see them all in a window, and use that window to go to any of them at any time. This seems so simple, and similar to the breakpoint window in Visual Studio, that I was surprised I never saw it there. Quick search on the internet: this window actually does exist in Visual Studio, but is not included in the Express versions of the product. Annoying, but fair enough. By the way, the breakpoint window is apparently included in VS C++ Express (so I read), but not in VS C# Express.

• the File Explorer tab in the DDMS (Dalvik Debug Monitor Server) perspective. As your game is running in the emulator or on a real phone, you can see what files it creates, updates or deletes, and transfer them to your hard drive to have a look at their content. That's very useful to verify your save game code works correctly, etc.
  
  In Visual Studio, I don't know a good way of doing that when running a WP7 game (I use the PC version I also have, since XNA is cross-platform, to check my files). And although this is not the subject of this article, this would also be a great feature to have when developing on the Xbox360 with XNA.

• the DDMS perspective also allows you to take screenshots of your application, whether it's running in the emulator or on a device! I haven't checked the Mango beta of the WP7 tools yet (I cannot install it and modify my setup until I'm done with my current project), but this is something that has been missing since the beginning.

• the SD card emulation. It's not totally trivial to set it up, but it allowed me to transfer some photos and pretend they had been taken with the phone's camera. This was important to me because Jigsaw Guru allows players to use their own pictures, and in the WP7 emulator you only have a few predefined ones for testing, and they don't necessarily cover all the cases you want to verify as far as naming or dimensions go.

• this one is more of a detail, but when a source file imports libraries it doesn't need, I get warnings. And when I use the aforementioned suggestion window, or some other automatic way of adding imports without typing them, they are grouped according to the first part of their namespace, and then ordered alphabetically inside each group. Perhaps I'm a maniac, but I like this stuff to stay clean, and basically Eclipse does for me what I manually do in Visual Studio.

Now let's talk about the things I didn't like as much:

• when an exception is thrown, it's not as easy as it should to know what happened. Maybe it's because in my case it happened most of the time in the OpenGL thread, but anyway: there doesn't seem to be a call stack window like in Visual Studio, in Eclipse I found after a while that the call stack gets printed in a window in the middle of some other log messages, and that's a mess. Seriously. Especially compared to Visual C#, where the program automatically puts me on the line that generated the exception, with a floating window telling me what the problem was.

• when you're stepping into your Java code, you cannot move the instruction pointer. I so could not believe it that I did an online search, which confirmed it. Actually, I even ran into some programmers that are so used to it in Eclipse, that they were asking how this can be useful anyway, and why you would want this feature. Oh boy!

•  if you want to change the value of some variables while you're debugging your code, it's a pain. I don't exactly remember the details, I just remember I gave up after a while.

• when your program is stopped on a breakpoint, you often want to check the value of some variables, don't you? Most of the time it works, but I got this "JDI thread evaluations" error a lot each time I tried to look at the content of arrays.

• auto-completion is not the greatest. First, I find it kind of slow. Second, it only kicks in after you've typed a dot character, as in "variable.something". I'm sorry, but Visual C# is an order of magnitude better, and will auto-complete about anything: a language keyword, a function name, or the name of a variable in the above example - if what I'm typing starts like the name of an existing local variable, parameter of the function I'm in, class member, etc. I'm so used to it that I never type "public void" for example, but "pu" + space + "vo"+ space instead. In the long run, this probably saves me from a lot of typos :)

• I'm not 100% sure about this one, but I got the impression changing a resource file (texture, sound, etc) doesn't trigger a rebuild of the project. 

• when you have both a warning and a breakpoint on the same line of code, you basically can't see the breakpoint anymore.

• in the DDMS file explorer, it is not possible to multi-select files and delete them, you have to do it one by one.

To sum up these pros and cons, I would say that editing code with Eclipse was OK (although improving auto-completion would make it even better), but debugging was rather painful.

Java versus C#

I'm afraid this comparison is a bit unfair: not because I've been using C# for several years whereas I was learning Java with this project, but because I was porting from one language to the other. This means I immediately noticed every feature I use in C# that is not available in Java, but I'm not aware of some other features that may be supported in Java and not C#. So, you've been warned, and hopefully the simple fact that I was ready to learn Java somehow shows that I'm not into language flame wars and that sort of things (even if I don't hide I love C#).
 

The first thing I found out is Java and C# are more similar than I thought. They don't just have many features in common (such as garbage collection, even if it's not necessarily based on the same implementation), the syntax is often almost identical. Some keywords are different of course, but lots of times I could just take my C# source file, copy it into the Java project, do some renaming (replacing "const" in C# with "final" in Java, bool with boolean, override with @Override, etc), compile, and then fix the few errors I was getting. That said, there are also some real differences, and I found some of them truly annoying.

• The main one is structs. Unlike C++ and C#, Java doesn't have structs (everything is a class). Why is that annoying? Because of the copy semantics of structs versus references. In C#, if I do something like this:
  
  
  Vector2 position = otherPosition;
  position.X = 1f;
  
  
  and Vector2 is a struct, I'm basically copying the value of otherPosition to initialize my "position" variable, and then I'm modifying the latter. Now since Java doesn't have structs, and I wrote a Vector2 class instead, the same code puts a reference pointing to the same object as otherPosition in my new variable, and then modifies the value of the object referenced both by position and otherPosition. I just modified the object accessed through otherPosition in Java, although I was absolutely not doing that in C#!
  
  
  This ended up being the source of most bugs I had to fix in my Android port, because XNA relies on a few structs you use all the time: Vector2, Vector3, Rectangle, Color and Matrix are the ones that come to mind. Of course, I would have overloaded the assignment operator (and a few others) of my Java Vector2 class to make it behave like a struct if it was possible; but Java doesn't support operator overloading.
   

• Java doesn't have out and ref parameters. Therefore, if you want a function to return more than one value, for example two integers representing the row and column indices of an item in a grid, you need to do it another way. One suggestion I read online is to pass an array to the function (in my example, an array of 2 ints), and I used that method in a few places. But it's kind of a hack, and of course the compiler won't guarantee that values have been set for all the elements of the array, whereas the .NET compiler will tell you if an out parameter isn't assigned a value for some code path in the function.

• Java doesn't have the equivalent of C#'s properties. It's obviously easy to rewrite them as functions since properties are nothing more than a writing convenience, it just takes some time since any C# program usually has a lot of them (and you also need to change the places where they are called, and add parentheses everywhere).

• Every function is virtual in Java, and everything is public by default. Seriously? These two points didn't create any problem for me, but I still felt like adding the private keyword everywhere I omitted it in C#. I don't know who had this brilliant idea, but to me it just seems wrong that broken encapsulation is the default, I'm sorry.

• Java doesn't have unsigned types. I didn't see that one coming, and I'm still surprised that if you're only dealing with positive non-fractional numbers, you would want to lose half of the range of values you can store in a given type. Weird.

• Java cannot do a switch statement on a string. This is a nice addition in C#, for example when you're loading an xml file and need to do different things depending on the value of an attribute you just read.

• Java doesn't have delegates (more or less the equivalent of C++ function pointers, but type safe). I'm sure there are a bunch of ways to work around that, mine was to use an interface with just one function, and a nested class implementing it. Then, instead of passing the delegate to a function, you pass an object of the nested class.

• Java forces you to declare all the different types of exceptions a function may throw. I found it pretty annoying: if you modify a function, and a new type of exception can now be thrown by the system, all the functions calling the one you modified also have to declare that exception, and so on. Except of course if you catch it at the lowest level, but that's very often not the place where you want to do it. I don't know, C# exceptions work the same way but without these extra declarations, and I can't think of a case where having this feature would have helped me as a programmer.

• Finally, simple enumerations are fine, but enums where you want to assign a specific value to each element are not supported. Some people online suggest using EnumSet instead, that seemed a bit overkill to me and I replaced them with a series of constants (which is of course not as good as the original enum, since values that didn't exist in the enum can be passed to a function that now accepts an integer, without the compiler telling me something is wrong).

Like I said before, there may be some awesome features in Java that don't exist in C#; but from what I saw, C# is basically a more advanced, more convenient to use, version of Java, thanks to some of the things I just detailed. Anyway, since those languages share a lot of similarities, porting from one to the other is not very difficult, as long as you stay away from the latest additions to C# (such as Linq), and know what to pay attention to.

OpenGL versus XNA

XNA is based on DirectX, and Android uses OpenGL. Which means I had to rewrite my rendering code. I had not done any OpenGL in about 10 years, but it wasn't too difficult to get back to it: OpenGL ES 1.0 and 1.1 on mobile phones are basically the same thing as what I used on desktops at the beginning of this century, except for glBegin/glEnd that have disappeared for performance reasons. Anyway, I don't really have a preference between DirectX or OpenGL (at least one section where I shouldn't hurt anybody's feelings ;)), but here are a few notes I took:

• the OpenGL way of passing predefined parameters to functions is just awful. I mean, seriously, on Android there is this one long list of values such as GL10.GL_PROJECTION or GL10.GL_BLEND, and you can pass any of them to any function since they're all integers, even if most combinations don't make any sense (for example, glMatrixMode(GL10.GL_BLEND)). Where are the enums? (I know, old interface, historical reasons, backward compatibility...)

• on Android, you can call the GLSurfaceView.setDebugFlags function to get some logs and add some error checking when OpenGL methods are executed. That's great, until you move from GL ES 1.0 to GL ES 1.1 (to use vertex buffer objects), and discover that gl11.glGenBuffers throws an UnsupportedOp exception when the debug flags are enabled. Meaning they become totally unusable :(

• when your application is paused, because another application moves to the foreground, its OpenGL context is destroyed. This means that if it gets resumed later on, you have to recreate all your textures and vbo's, or you will crash. This doesn't happen on WP7, because when the game is tombstoned and resumed, the whole application is restarted, it's not just the rendering context that needs to be recreated! Anyway, since I have written code to handle the Android "application life cycle", and tombstoning on WP7, I don't believe one is more difficult to deal with than the other, both are necessary evils.

• OpenGL commands cannot be issued outside of the OpenGL thread. That makes sense, especially once you've been bitten.

That's about it. The main difference in fact, is that XNA adds a small extra layer on top of DirectX, and that's what makes it nicer: there are a few classes that are very useful to have, that I had to partly rewrite to make my game run on Android. I'm talking about SpriteBatch (used to batch and draw anything that uses screen coordinates, for example your game's UI), and SpriteFont (used to manage fonts). Yes, unfortunately, if you need to render text with OpenGL ES, it's Do It Yourself.


The emulator

That's where things become very painful. At first everything looked OK, while I was trying to render a static menu screen, and hook up touch input to navigate to another one. But when I finally animated something, I was shocked to see I was getting 4-5 frames per second, with only a few draw calls and almost no overdraw! Of course I went online, thinking there was something I needed to set up to fix the problem, or something along those lines. Unfortunately, this wasn't the case.

The Android emulator does a lot of things, and I suppose it's a decent or even good tool to write apps. Although, I found out many people complain it takes a very long time to start, and isn't particularly fast: this is because it emulates a real ARM architecture, instead of running an x86 version on Windows. But here is the real kicker: it doesn't use hardware acceleration for OpenGL! So, forget about your brand new ATI/AMD or nVidia video card, and suck it up: everything is very slow, even in 480*320, and some 16 bit like banding is visible if you have gradients. Unbelievable.

And that's if you use OpenGL ES 1.0 or 1.1. If you'd rather go the native route, and rely on ES 2.0, it gets worse: the emulator doesn't support it, period. Wow, just wow. So, how do people write OpenGL games on Android? Well, the only "work-around" is to use a real device, and never use the emulator. I've put work-around between quotes though, because to me it's a very limited one, for the following reasons:

• deploying to a device cannot be a very fast process, which is why you normally want to do most of your work in an emulator to start with. Even if this is not as true on Android, since deploying to the emulator is not really fast either (more on that later).

• the emulator allows you to test your program in different resolutions, how do you do that with a single phone? I don't know, I suppose you buy more phones.

• the emulator allows you to test your program on different versions of the OS. My phone has Android 2.2, maybe I could retrofit it to 2.1 but even if that's possible I don't want to keep reinstalling it, and what about 2.3 and 3.0 if it doesn't support them? Buy more phones?

So, developing on a real device does work, but it won't allow you to test your game in as many configurations as the emulator, and that's a problem. Except maybe if you work for a company, and can get several different phones.

Now let's talk about productivity. I noticed two problems with the Android emulator:

• launching it the first time is long, but after that you keep it running and just restart your application, which is much faster: this is fine, and the WP7 emulator works the same way. Well, this is fine until the deployment starts failing for some unknown reason, and you have to re-launch the emulator all the time. This happened to me more and more, and I think it has something to do with the size of the apk file: when I finally added music to my game (3 big mp3 files), it basically became impossible to not reboot the emulator each time. I looked online and tried to add some command line parameters, but that didn't fix it.

• even when I didn't have to kill and restart the emulator each time, I think I noticed something else: it looks like if you make even the simplest change to your code, let's say you change the value of a boolean variable, the deployment takes 35 seconds. Since recompiling that change takes no time, I'm under the impression the whole (and eventually big) apk file is redeployed, the same way it would after a rebuild all. Whether that's what's happening or not, it sucks: on WP7, if I do a similar change, I don't know how things work (since the xap file is modified, and as big as the Android apk file), but I'm getting a real incremental deployment, meaning none of the assets are sent to the emulator, and that's very fast. On Android, once you know this won't be the case and will probably take at least 30 seconds, you switch to another program, maybe check your emails or start reading something on the internet, and when you finally come back to the emulator, the first thing you do is ask yourself: "what was I going to test, again?". This interruption totally kills your workflow, and productivity drops.

There are a few other things I could complain about, that are more related to the libraries than the emulator, such as the pinch gesture only appearing in Android 2.2, which means you have to implement it yourself if you want to support Android 2.1. But those are smaller details, and overall the libraries are very good, so let's move on to the last topic. 

Compatibility

Hallelujah! Despite the big framerate issues, you made an OpenGL game and everything works fine in the Android emulator. Maybe you even already tried it on a real device, or did the whole development that way, and all you have left is to release the game on the Android market, which is a very fast operation since there is no certification process (unlike on WP7 and iOS). And then you're done (with the development side of things, as long as you're not making an update)! Or are you?

No you're not. Even if the program runs fine in the emulator, and on your phone, and on your friend's tablet, there are still many devices that won't run it correctly, or at all. This is the main development problem with Android imho: the platform is open, no two phones are identical, they don't necessarily run the exact same software, some have terrible OpenGL drivers, and therefore being compatible with a decent percentage of the devices currently available on the market is a huge challenge. And new hardware keeps being released every week...

To illustrate this point, let me give you two real cases I had with my game:

• once the Android port was advanced enough, and I knew I would finish it (rather than give up), I bought a Nexus One, the phone that was recommended for development by Google at that time. First time I tried to run the game on it: everything was white, with a couple of grey squares where I normally have disabled UI buttons. It wasn't too hard to figure out what was happening: the Nexus One cannot handle textures with dimensions that are not powers of 2. Fixing it wasn't difficult enough (I rescale the textures on the fly when I load them), but this shows the fact that the program runs in the emulator, or on some phones, doesn't mean it will run on all of them (even the one recommended by Google!). What I don't understand is why the framework doesn't take care of that, it could totally do under the hood the same rescaling I do, and I would never had known about this limitation.

• a player emailed me, saying he used to play on his Galaxy Tab and everything was fine, but now that he moved to a Xoom tablet he cannot save his settings or games. I don't know if this comes from the tablet running Android 3.0, or some different type of storage being used, but it's pretty surprising the file API isn't compatible. I still have to look into it, and since I don't own a 3.0 device, I'll do it in the emulator, even if the OpenGL software rendering is going to make this a painful experience. 

Conclusion

Developing on Android wasn't a fun experience for me. Like I said at the beginning, it is or is going to be a big market (although, not many independent developers seem to be able to make money on it - yet?), so it's tough to ignore it; but the tools are not at the same level as on the competing platforms, by quite some margin. If Google wants to have more games on Android, and I'm talking about original games and not just ports of iPhone ones many months later, the first thing to do is fix the emulator. I don't care if it fully emulates an ARM CPU, I want it to start faster, run faster, support incremental deployment, use OpenGL hardware acceleration, and that would actually also allow it to support ES 2.0. As far as compatibility goes, this is a more difficult problem to address, and I don't believe it will be fixed anytime soon, since it's basically a consequence of the platform being so (too?) open.