F# for game development

In an earlier post entitled Why inheritance for code reuse should be avoided I pointed out that the key problem was conforming to the Liskov Substitution Principle.

Although I guess there is nothing controversial in what I wrote, I fear I probably did not convince anyone who wasn't already convinced.

Today at work, I had the pleasure of being hit by a bug that is a wonderful example of inheritance gone wrong.

Let me give a bit of context. Once upon a time there was an application that allowed viewing and editing a database in a nice and user-friendly somewhat buggy GUI. As it was felt the application could be user-friendlier and more stable, additional development efforts were planned to improve it. This logically led to the decision of distributing what was a stand-alone file-oriented program over a server-client architecture.

In case you hadn't noticed, I'm being a bit sarcastic, but the fact is that the original plans for the application included extensions to a server-client architecture, and the source code was cleanly organized in modules, keeping the GUI and the database separated by a middle layer.

When moving to the new architecture, it felt natural to turn the middle layer into a web service. This required all data types in the API of the middle layer to be serializable so that they could be sent over the net between the client (the GUI) and the server. This was fairly easily achieved using WCF and DataContract.

[DataContract]
class Item {
   [DataMember]
   public int Data { get; set; }
}

Today, I opened a bug report with an unexpected exception stating that a type named "...Dialog+GuiItem" could not be serialized. Indeed, I could recognize the "Item" part, which was serializable thanks to DataContract, but the GUI component most certainly wasn't! What was going on there?

Looking at the definition of Item, I could see it was a fairly simple type GuiItem augmenting a simple data-type (called Item) with GUI thingies.

class GuiItem : Item {
   public Gui SomeGuiStuff;
}

The problem is, the GUI-augmented type couldn't provide all features the basic type did: It could not be sent over the net.

In the first version of the software, passing the GUI-augmented instance to the middle layer was not a problem. The middle layer expected a basic Item or anything that can be implicitly converted to such an item. As long as all passing of arguments occurs within the same process, no additional effort or processing was needed.

class MidLayer {
   public void DoSomething(Item item);
}
...
void OnClick(GuiItem item) {
  midlayer.DoSomething(item); // Fine, GuiItem is also an Item.
}

In the new version, passing arguments is a significantly more complex operation involving doing a deep dump of the content of the instance, streaming over the net, and summoning a new instance stuffed with the dump on the other side of the wire. This only works for types which are purely data-oriented. Inject any kind of "live" resource in there, and you are out of luck.

[WebService]
class MidLayer {
   [WebMethod]
   public void DoSomething(Item item);
}
...
void OnClick(GuiItem item) {
  midlayer.DoSomething(item); // Exception: can't serialize GuiItem!
}

The solution was simple enough, simply include the basic Item as a member of the GUI-augmented type, and provide a getter to easily send it from the GUI to the server.

class GuiItem {
   Item item;
   Item Item {
      get {
         return item;
      }
   }

   public Gui SomeGuiStuff { get; set; }
}

void OnClick(GuiItem item) {
  midlayer.DoSomething(item.Item); // Fine again.
}

When inheriting from a type, you don't just get its nice features, but also its responsibilities.

At work we are a couple people who have been using F#. All people who have used it are enthusiastic, yet I was surprised today when I heard a project leader had asked to limit the use of F#.

There is a question on programmers' stack exchange which touches the subject titled "Real world pitfalls of introducing F# into a large codebase and engineering team". I'm quoting:

1) Everyone has to learn F#, and it's not as trivial as switching from, say, Java to C#. Team members that have not learned F# will be unable to work on F# parts of the codebase.
2) The pool of hireable F# programmers, as of now (Dec 2010) is non-existent. Search various software engineer resume databases for "F#", way less than 1% of resumes contain the keyword.

The fear is that maintaining F# code might be difficult because current employees and job applicants lack functional programming skills.

It seems some decision makers are still stuck at the time when C# 1.0, C++ and Java 2 were the languages of choice. Since then, a few things have happened.

Firstly, Python has become hugely popular. Although many were horrified by its use of white space to delimit blocks, it would be difficult to take seriously a programmer who claims he can't adjust to Python's syntax. Some love it, some hate it, but all can write Python code.
Why would the syntax of F# be any harder to adopt?

Secondly, C# has introduced a number of functional programming and other high-level concepts in each new version that has come out. Modern C# programmers are expected to master LINQ to objects, lambdas and yield return. The next version will introduce a specific syntax for asynchronous programming. Would you consider hiring a programmer would admits he knows nothing about these features and is unwilling to learn about them?
F#'s higher order functions, functions as first-class citizen, sequence expressions and asynchronous workflows match directly the modern features of C#. Why would these be harder to use in F# than in C#?

Dear reader, if you are a decision maker, I hope these two simple remarks dissipated your fears. You may be wondering why adopt F# when C# is evolving? You won't hear it from Microsoft, but the fact is that C# is breaking at the seams in its attempts to bring in new constructs. I won't give an exhaustive list, but here are a few examples of the limits of C#:

1. Limits of iterator blocks (yield return). No such restriction in F#.
2. Closures capture variables, not values. In F#, closures capture values, which is in my opinion a much saner design decision.
3. Language-integrated queries are nice, but they don't sound as generic as they actually are. F#'s notion of quotation is more elegant, and makes it easier to see the applicability of LINQ beyond SQL.
4. Although it's nice that C# is "stealing" async from F#, why not introduce the more general concept of custom workflows?

Moreover, F# supports a few data-types that somehow got forgotten when class-based OOP became dominant, namely records and discriminated unions. Together with pattern-matching, they are doing their come-back in the main stream and I have no doubt they will shortly become tools top programmers are expected to master.
 
Now, I'm not saying you should ditch C# and replace it "en masse" by F#. C# has better tooling support, features implicit conversions and a lighter syntax for implicit interface inheritance. I don't think these features are a must-have in all situations, but they can be handy at times.

Adopting all new hyped technologies has its dangers, but if you refrain from getting on with the times because of a few dinosaurs among your developers, your business will eventually go the way of the dinosaurs.

I have just heard about a new project being started by a game developer, NPerformant.

The developer writes

At various times in my “day job/career” I’ve used commercial profilers with varying results, but I don’t want or need to profile the whole application or even a whole library.  I also don’t want to code two solutions into my product in order to let a normal profiler test them side by side in order to remove the loser later.

The problem about creating additional solutions also occurs in testing and when writing small utilities and editors while developing your game.

F# scripts help avoid this issue. You can add number of .fsx files to your project, and executing them using F# interactive (aka fsi) is very easy. For my current game, I have a script to render parts of my engine data using XNA and winforms, and another script to test some of the game logic. I don't know how long I will need these, but the day I lose the use for them, removing them is as easy as removing a single file from your project. Compare that to removing an entire project, which in my case would involve getting out of Visual Studio and using Windows Explorer then TortoiseHg.


Back to the topic of measuring performance, did you know about the #time command you can use in fsi:

> #time "on";;

--> Timing now on

randomTest();;
Real: 00:00:15.761, CPU: 00:00:54.725, GC gen0: 1917, gen1: 2, gen2: 0

Note that you get information about garbage collection too, which is interesting information when developing games for the Xbox 360.

Regarding testing, I just found out yet another case when pattern matching saves the day. In the code extract below, I'm using it to express the expected result of the code being tested.

match gs'.player_units with
    | [| _; [| { health = x } |] |] when x < 1.0f -> true // Injured
    | [| _; [||] |] -> true  // Killed
    | _ -> false

player_units is an array of player units, where players units are represented using arrays of units. My test has an artillery belonging to the first player fire at a helpless infantry of the second player.

The first branch of the pattern matching says: "An array composed of two elements where the first element can be anything, and the second element is a one-element array of units where x is the value of field health, and x is strictly less than 1".

I don't know how I would have written that in C# in a similarly concise fashion.

Informally, my pattern match checks that the unit of the second player was injured or killed.

Every now and then, a question about how to transfer data between game screens pops up on the App Hub forums. The source of the problem is that many try to solve this problem using synchronous programming. This does not work because game screens require interaction with users, which cannot be handled in a synchronous manner without freezing the game.

Although F# gives you tools to solve that problem using e.g. async workflows, most people are using C#.

How do you solve that problem in this language? Usually, people use a combination of events and public fields. This solution is fine, but I just don't like events much. They make bugs tricky to track, because I lose track of where handlers fit in the expected flow of execution of my code. Moreover, the only practical way that I know of to pass data from an event handler to the main code is through a shared variable (which I call "I'll Whack My Own Fingers"-oriented programming).

For this reason, I suggested to use Begin-End-style asynchronous programming. This requires an implementation of IAsyncResult if you want to do it in a nice .NET-ish kind of way. I am not aware of any such implementation (EDIT: Here is one, thanks mausch), so I wrote my own:

/// 
 
  /// An implementation of IAsyncResult that wraps result data. 
  /// 
 
  /// Type of the result data. 
  class AsyncResult<T> : IAsyncResult, IDisposable { 
    public AutoResetEvent signal = new AutoResetEvent(false); 
    public bool isDone = false; 
    public T result = default(T); 
 
    public object AsyncState { 
      get { return state; } 
    } 
 
    public WaitHandle AsyncWaitHandle { 
      get { return signal; } 
    } 
 
    public bool CompletedSynchronously { 
      get { throw new NoneOfYourBusinessException(); } 
    } 
 
    public bool IsCompleted { 
      get { return isDone; } 
    } 
 
    public void Dispose() { 
      signal.Dispose(); 
    } 
  } 

Did you notice how I made all fields public? I think many a modern programmer would wince when reading this. What if some careless programmer accessed isDone and modified it? Bad things can follow, and surely we should restrict access to these fields using the private keyword.

I like to call this "I'll whack your fingers if you touch this"-oriented programming. Beside "private" and "protected", which were probably invented to tempt stressed programmers to replace them with "public", enthusiasts of this approach enjoy using "sealed" to punish users who might be tempted to reuse your code.

There are better ways to prevent accidental tempering with an object's internal state: access objects via interfaces.

For instance, here is the code for the Begin-End pair of methods that use my AsyncResult:

/// 
 
        /// Open a MessageBoxScreen asynchronously. 
        /// 
 
        /// The screen manager to which the screen is to be added. 
        /// The index of the player who has control, or null. 
        /// The text of the message to show. 
        /// An object which can be used to wait for the request to complete and retrieve the result. 
        public static IAsyncResult BeginShowMessage(ScreenManager sm, PlayerIndex? player, string msg) { 
          var result = new AsyncResult(); 
 
          var screen = new MessageBoxScreen(msg); 
          screen.Accepted += (src, args) => { 
            result.isDone = true; 
            result.result = true; // User accepted the message box. 
            result.signal.Set(); 
          }; 
          screen.Cancelled += (src, args) => { 
            result.isDone = true; 
            result.result = false; // User cancelled the message box. 
            result.signal.Set(); 
          }; 
 
          sm.AddScreen(screen, player); 
 
          return result; 
        } 
 
        /// 
 
        /// Wait for the user to complete interaction with a MessageBoxScreen opened with BeginShowMessage. 
        /// 
 
        /// The object returned by BeginShowMessage. 
        /// Whether the user accepted or cancelled the message screen. 
        public static bool EndShowMessage(IAsyncResult r) { 
          var result = r as AsyncResult; 
          if (result == null) 
            throw new ArgumentException("Wrong type or null", "r"); 
 
          result.signal.WaitOne(); 
          result.Dispose(); 
          return result.result; 
        } 

Note how BeginShowMessage and EndShowMessage return and take respectively an IAsyncResult. Unless users of these methods really want to take risks getting intimate with AsyncResult, there is no risk of tampering. And if they really want to work directly with the inner parts of AsyncResult, why prevent them to do so?

I wonder, what's the point with "private" and "protected" anyway?

UPDATE:

Thinking a bit more on the subject, I think another overused feature resides in non-virtual public methods (and fields). Anything public in the API of a class should belong to an interface, with the exception of constructors and construction helpers (which maybe should be internal to assemblies).

I would however not advise to use abstract data types in all situations. Navigating in the call tree is a good way to learn a new code base, and typically abstract methods defeat this. One could say abstraction through interfaces is a form of obfuscation, as it effectively hides implementations.

Introduction

Asteroid Sharpshooter is a game published on the Xbox LIVE Marketplace, under the independent games section.

It is written in F# and C#, using XNA Game Studio 3.1. The game falls in the category of 3d space shooters. Inspired by a classic, Asteroids, the game puts the player in space, in a field filled with rocks. Some of these can be destroyed by shooting at them. The goal of the game is to shoot and destroy a number of rocks. To make things challenging, the controls of the ship are consistent with what you would expect from a ship drifting in space. There is no friction, and the ship does not necessarily point where it's headed. Controlling the ship to collect power-ups and approach asteroids to shoot requires skill. The higher difficulty levels feature enemies in the form of drones that track the player's ship and detonate themselves when they come close enough.

In this article, I present my thoughts about the development process of the game and the reception of the game by players.

Development

The game was written using Visual Studio in C# and in F#. C# is used for the higher levels of the software, which consist of menus, parts of the rendering, loading assets, loading and saving user progress.
Menus use the game state management sample from the App Hub.

Using F#

F# is a good programming language in general, and contributed positively to the development. Features such as discriminated unions and pattern matching are lacking in C#.

Although the game uses multiple threads, it does not use async workflows (these are primarily intended to ease the development of asynchronous code, but they also have interesting properties for parallel programming, as shown in my earlier blog entries).

Some of F# features were at the time not supported on Xbox and resulted in run-time errors: sprintf and async workflows. I haven't had the occasion to try async workflows with the current version of F#, but sprintf now works!

Building the game was tricky before I managed to hack project files.

I initially used the free versions of Visual Studio: the Express edition for XNA Game Studio and C#, Visual Studio Shell for F#. This worked OK, even for debugging, but wasn't as practical of using Visual Studio Pro, which I ended up using at the end. You can use the free editions to try and see if F# works for you, but for regular development, you will probably want to use the Pro edition or higher.

I was a bit worried that using F# on the Xbox might not fully work, and might even be impossible as new versions of XNA or F# are released. Although there have been problems, all could be resolved, and the mix got better or more stable with every new release. Although F# is still not officially supported on Xbox, the fact that Microsoft Research has developed an XBLA game that uses F# sounds positive.

F# uses reference types for most of its data types: lists, tuples, records, classes, discriminated unions, function values, mutable cells (ref). This can cause problems because of the limitations of the current garbage collector on Xbox. I decided to design my code so that garbage collections would have a low latency, and not care too much about how often they would occur. This worked well enough. The game triggers a garbage collection every 6 frames, which each last for 3.5ms. The remaining time was enough to update the physics and render all objects, but a more complex game with more models and more complex class hierarchies could have difficulties.

F# does not allow circular dependencies between types unless they are declared in the same file and marked as mutually dependent. I was aware of this restriction, and it did not cause me any trouble. I started the project with all C# code in one project, and all F# code in another. Toward the end, I started moving reusable code into separate libraries. For my F# code, this was little more work than moving files to a new project and changing namespaces. The task was notably more difficult in C#, as the language supports circular dependencies within assembly bounds. Breaking apart an assembly will require the introduction of interfaces at the bounds. Although F# and C# do not differ on that point, the fact that F# forces you to work that way has benefits the day you want to split your code, in addition to all other benefits that non-circular designs have (better tool support, easier to understand...).

I don't know of any way to declare pass-by-reference parameters in F#, the way you can in C# using the "ref" keyword. In the XNA framework, some methods use passing by reference to save time. Although it is possible to call such functions from F# code, I don't know of any way to declare new functions.
There is however an F# way of avoiding copying large structs when calling functions: use inlining. Last time I checked, it was not fully reliable though, as the F# compiler tends to introduce lots of unnecessary variables in inlined code. Whether the .net CLR notices that and removes these variables isn't clear to me.

Project planning and tracking

I have used a free account on fogbugz to organize my work and keep track of bugs. Although it may seem to be overkill, it allowed me to look at what went wrong when writing this article, as I had a record of most of the bugs. It also simplifies working on a project part-time. During my day job I can focus on my job and forget about the project. When the week-end comes, I can pick up the project where I left it, and start new tasks which were planned but not started.

Obstacles encountered

Although the game state management sample was very helpful to get the menu system up and running in no time, it's not obvious at first how to integrate user interaction forced by the StorageDevice API. One needs to keep track whether a device is selected, whether the user is currently selecting one... The first solution that comes to mind, using Boolean variables isn't maintainable when the number of variables grows. For instance, the device selection screen had to keep track of whether the user was signed in, currently signing in, if a title-specific storage device was chosen, being chosen, whether a user-specific storage device was chosen, being chosen. Mix that with event handlers that may need to display alert boxes, and it becomes tricky. I used handlers to deal with storage disconnection and I/O operation notification.
Even after writing my own version of EasyStorage in F#, cleaning up code, the Update() method of my game object still looked too complicated for its own good:

protected override void Update(GameTime gameTime)
        {
            base.Update(gameTime);

            if (titleStorageLost == GuideStates.Requested)
            {
                if (!Guide.IsVisible) try
                {
                    Guide.BeginShowMessageBox("Storage device disconnected",
                        "The storage device used for scores was disconnected. " +
                        "Scores will not be saved unless a new device is selected. " +
                        "Would you like to select a device now?",
                        new string[] { "Yes", "No" }, 0, MessageBoxIcon.Alert,
                        (result) =>
                        {
                            var choice = Guide.EndShowMessageBox(result);
                            if (choice.HasValue && choice.Value == 0)
                                requestTitleStorage = true;
                            titleStorageLost = GuideStates.None;
                        },
                        null);
                    titleStorageLost = GuideStates.Pending;
                }
                catch (GuideAlreadyVisibleException) { }
            }

            if (titleStorageLost == GuideStates.None && requestTitleStorage)
            {
                storage.RequestTitleStorage();
                requestTitleStorage = false;
            }

            if (titleStorageLost == GuideStates.None && !requestTitleStorage && userStorageLost == GuideStates.Requested)
            {
                if (!Guide.IsVisible) try
                {
                    Guide.BeginShowMessageBox("Storage device disconnected",
                        "The storage device used for player progress was disconnected. " +
                        "Progress will not be saved unless a new device is selected. " +
                        "Would you like to select a device now?",
                        new string[] { "Yes", "No" }, 0, MessageBoxIcon.Alert,
                        (result) =>
                        {
                            var choice = Guide.EndShowMessageBox(result);
                            if (choice.HasValue && choice.Value == 0)
                                requestUserStorage = true;
                            userStorageLost = GuideStates.None;
                        },
                        null);
                    userStorageLost = GuideStates.Pending;
                }
                catch (GuideAlreadyVisibleException) { }
            }

            if (titleStorageLost == GuideStates.None && userStorageLost == GuideStates.None && !requestTitleStorage && requestUserStorage)
            {
                var screens = screenManager.GetScreens();
                if (screens.Length > 0)
                {
                    var topScreen = screens[screens.Length - 1];
                    if (topScreen.ControllingPlayer.HasValue)
                        storage.RequestUserStorage(topScreen.ControllingPlayer.Value);
                }
                requestUserStorage = false;
            }

        }

Since then, I have developed a better way of solving that kind of problem. All this code now becomes:

task {
  do! storage.CheckPlayerStorage
}

Shorter, isn't it? The point is that F# has features that make it possible to compose code using a notation similar to traditional function calls, yet the execution can differ from a traditional call. The idea is so neat that the normally conservative C# design team decided to add a variant of it to the next version of the language.

Back to the post-mortem, another related problem is to deal with screen transitions. There are several approaches: Hard-coding, events and delegates.

Using hard-coding, screen A creates and displays screen B when a certain action is performed.
This requires the class for screen A to know about the class for screen B, and must have the data needed during instantiation of B at hand. I found this approach was not very flexible, and caused me some trouble when I added support for local multiplayer (which wasn't planned from start).

The two other approaches, events and delegates make it easier to forward the data needed to instantiate new screens, as it's typically available in the class which registers the event handler, or creates the delegates which captures the data in question.

All these approaches share the same problem: the transition code is spread out all over the place, making it hard to debug, modify and extend. Of the 50 bugs I registered in fogbugz, 13 involved screen transitions at some level. For a game programmer who is interested in getting the gameplay right, getting 26% extra bugs because of menus is a very frustrating, even if most of those bugs are easy to fix.


Art assets

Asteroid models, ship models and textures were provided by Benoit Roche. The title and menu music is from Partners in Rhyme, sounds from Soundsnap.  The game backgrounds and the box art were done by myself using The Gimp and a tutorial on starfields. When doing sky boxes, it's a good idea to test them on a PC screen. I failed to notice on my TV that the sides of the box had light levels that did not match. Happily, a tester on the App Hub noticed that and reported the problem.

The community on App Hub...

... was very helpful. Thanks to all of you fellow developers for your feedback and suggestions!

Due to my earlier involvement in free software and Linux, I thought that sending your game to playtest early and often was a good thing. While it was, don't expect feedback for every release. Other developers will not test your game time and again every month. Getting comments from other is a motivation boost, but you should not rely on that. I think it's a good idea to send to playtest as soon as your gameplay is done, to see how well it's received. After that, sending updates every time won't get you much feedback. It may actually be better to wait until a new milestone is reached, e.g. menus are done, art is done, game is ready for evil-checklist testing.

Reception of the game

The mix between a classic 2d game and a 3d space shooter was not well received by the market. After five weeks, the game sold 143 copies with a conversion rate of 8.06%
Few reviews were written, most of them judging the game as dull and hard to control. This is what xboxindiegames had to say about the game:

You can't steer, you can't see, you can't aim and you can't shoot. You can avoid this game, though... 

The Yellow Pages from Pencil Shavings sounded more positive:

Looks fantastic and plays well, just a little redundant [...]. Nice game, enjoyable, but lacking variety.

I also registered the game for Dream-Build-Play 2010, but the game did not make it to the best 20.
The game is rated 3 stars of 5 in the American Xbox LIVE Marketplace, which I think is characteristic of well-done but uninspiring games.

Conclusions

Technically, the game was a success and showed the feasibility of using F#. It took me way too much time (about two years), though. I hope I will be able to increase the production rate for my upcoming games.