fasterthanlime

Dec 24, 2012 19 min #programming · #ooc · #gamedev

Cross-platform game distribution

ooc makes it easy to compile your application on all major platforms (Windows, OSX, Linux) - the compiler itself runs there, and the SDK supports all these platforms with basic functionality: data structures, file handling, time handling, networking, etc.

But between getting your application running on your dev environment with all the libraries installed, and getting it into a neat package for your users to run without having to install any dependencies by hand, there's a bag of tricks. Fortunately, I have found the time to figure most of them out. I'll try to explain these in detail here as clearly as possible, here in this article.

All your .ooc files are compiled by rock (and gcc/clang) into a single executable file. Let's pretend we're working on a game project named 'foobar'. On Windows, the resulting executable will be named 'foobar.exe', and on Linux/OSX, it'll be named 'foobar'.

But wait, that's not all! Chances are, your game is loading assets (music, graphics, levels). That's a dependency. For those to load correctly, the CWD (current working directory) should be set correctly when running your executable. We'll see how to do that correctly on all platforms.

And even when the CWD is correct, we're not done yet: your game also probably depends on external libraries, such as SDL, freeimage, and so on. If you compile it naively on your dev environment, they'll be dynamically linked. When running the application, your OS is looking for the libraries in the default path, such as /usr/lib, /usr/lib64, /Library/Frameworks and so on.

However, you can't assume that your target user will have all dependencies installed like you did, especially if he's on a distribution or version other than the one you compiled the game on. For this too, we have OS-specific tricks that I'll expose here.

I'll be handling Windows first, since it seems to be the easiest case - however it is also the hardest to automate (from my current workflow at least).

The best way to build C (and thus ooc) programs from the command line on Windows is to use MinGW and MSYS. MinGW is a version of the GCC compiler collection for Windows, with a runtime based on msvcrt, ie. native to Windows. You'll find that some functions are missing from your comfy unix-y environment, unlike cygwin - but do not be afraid! The ooc sdk is versioned specifically to use the Win32 API when needed (for file handling and time routines, for example).

MSYS is a port of some command-line utilities, such as the bash shell, and a cmd.exe-based command line. It also handles path translation, so that in your shell, /usr/ refers to an actual location on your Windows drive, for example C:\MinGW\MSYS\1.0\usr\ and so on.

Installing MinGW

However, the MinGW shell is less than convenient for every-day use. My biggest gripes with it are the way it handles resizing windows (spoiler: cmd.exe style, where you have a separate "buffer size" that you can adjust in the options, and a "window size" that cannot be bigger than the buffer size), and the way it handles copy/paste (spoiler: cmd.exe style, where you have to go: window contextual menu, Edit -> Mark, then draw a rectange in your window by click-dragging with your mouse, then Edit -> Copy, and it's finally in your clipboard).

A much neater alternative is Console, an acceptable command-line for Windows. You might want to install that, and take a look at the preferences to make it as comfy as possible. I made it gnome-terminal like (Ctrl-Page Up/Down to navigate tabs, Ctrl-Shift-T to open a new tab, Ctrl-Shift-W to close one, Ctrl-Shift-C/V to copy/paste, the only difference is that Console2 requires shift to be pressed while highlighting text for copy - but I can live with that).

You need to set the startup command to C:\MinGW\MSYS\1.0\bin\sh.exe -login (or wherever sh.exe lives) to get an MSYS environment in there.

Then, you'll want to grab msysgit so you get the full git goodness in your unix-like environment. You'll want to add the path to msysgit's bin directory to your PATH environment variable in your ~/.bashrc on MSYS. Here's what mine looks like:

export OOC_LIBS="/c/Dev"
export PATH="/usr/local/bin:/c/Dev/rock/bin:/c/Program Files (x86)/Git/bin:$PATH"

If you don't do that, git will only be accessible from the 'Git Bash' terminal that msysgit installs. But I don't recommend using this, you should have all you need in a single place so that you can use as many terminals as you want in Console2 without worrying if it's a "git" terminal or an "everything else" terminal.

Once you've done all that, you can go ahead and install rock to compile ooc code.

The basic process goes like this: I suggest you decide on a single directory where to put all your ooc development files - for this example, I'm going to go with /c/Dev/. Note: if you don't have wget on your system, grab it with mingw-get install msys-wget

cd /c/Dev/
git clone https://github.com/fasterthanlime/rock.git
cd rock
make quick-rescue

If everything goes well, you should have a bin/rock.exe that works. And if you followed my .bashrc advice earlier, it's even already in your path!

Most cross-platform libraries have MinGW 'dev packages' that you can get if you look around, or they have MSVC dev packages that you can turn into MinGW-compatible dev packages with a bit of fiddling. However, that's not satisfactory for me - I want to build everything from source, and get/install any package with a single command instead of having to search for it online.

For that purpose, I've ported homebrew to run in an MSYS environment, on Windows. To grab it, simply follow these instructions in your bash shell:

cd /usr
git clone https://github.com/nddrylliog/homebrew-mingw.git local

Note: homebrew-mingw is still very much experimental, it assumes a few things (that you have msysgit in your path, that MinGW is installed to C:\MinGW, etc.). You can work with it and it built over 40 packages flawlessly for me, but there's work to do. Contributions, bugfixes, and package upgrades are welcome!

Perhaps one of the first packages to install with homebrew is pkg-config. It'll allow you to work with most C libraries, setting the correct paths for your C compilers so that it finds headers and libraries. To install it, simply do:

brew install pkg-config

Optionally, you can pass -v or --verbose to brew, so that it shows which commands it is running. I find that it helps time pass by faster, otherwise staring at a "Compiling" line for a few minutes is unsettling. One thing you'll notice is that the same packages will take a lot longer (I'm talking 3-4x longer) to compile with MinGW (ie. GCC for Windows) than they did with Clang on OSX. I have no idea why MinGW is so slow (even on an SSD). If someone finds out, please, please tell me. The executables produced by MinGW seem to be running speedily, though!

Building your program in ooc is as simple as running:

rock

(Optionally, pass -v or --verbose to see the commands rock runs)

In your program's directory, as long as you follow our conventions. The conventions for a program named 'foobar' and that uses ooc-sdl, should be as follows: the directory should be named 'foobar' and have a file named 'foobar.use' in it. The contents of the file would look like this:

Name: foobar
Description: Foobar is a game of some sort
SourcePath: source
Main: foobar.ooc
Requires: sdl

That way, rock can find your .ooc source files (all shoved in source/, possibly neatly organized in packages), and knows which one is the main executable. The Requires clause specifies which ooc libraries are need, and in turn, the .use files of those libraries give instructions to the C compiler on where to find the headers and link to the right libraries.

If everything went fine, you should have a foobar.exe in your foobar directory. You can run it either in bash with ./foobar or by simply double clicking it in the Windows explorer!

However, that .exe depends on .dll files (dynamic libraries) that are scattered across your system. That's fine when developing, but not when you package your application! Hence the next section.

Windows has a few dangerous but convenient conventions in place. For example, when launching an executable when double-clicking on it, or from a shortcut, it will look for .dll files first in the directory of the executable.

We'll use that to our advantage to package our application. The first step to package your application for Windows is to find out on which .dll files your application relies to launch. Steve P. Miller released a convenient application called depends.exe.

Opening it on your application will let you know the .dlls it's loading at runtime. Then, you can just distribute them along (in the same folder as) your .exe file, and you're good to go!

For minimal distribution, just releasing a .zip of your assets, .exe and .dll files is enough. For a full installer, you might be interested in NSIS

You can pretty much follow the instructions from the Windows section, except everything is much simpler: you basically have to install Homebrew, and then you can install rock with brew install rock, and compile your ooc program as usual.

On OSX, the common way to distribute a game is to have an "App". Apps are just folders, in our example named foobar.app, with a standard directory structure. Here's an example directory structure for the foobar application bundle:

foobar.app
    Contents
  Info.pList
  MacOS
      assets
      foobar
      wrapper
  Resources
  libs
      somelib.dylib

The pList file should look something like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>CFBundleGetInfoString</key>
  <string>Foobar</string>
  <key>CFBundleExecutable</key>
  <string>wrapper</string>
  <key>CFBundleIdentifier</key>
  <string>com.yourdomain.www</string>
  <key>CFBundleName</key>
  <string>legithief</string>
  <key>CFBundleIconFile</key>
  <string>legithief.icns</string>
  <key>CFBundleShortVersionString</key>
  <string>1.0</string>
  <key>CFBundleInfoDictionaryVersion</key>
  <string>6.0</string>
  <key>CFBundlePackageType</key>
  <string>APPL</string>
  <key>IFMajorVersion</key>
  <integer>0</integer>
  <key>IFMinorVersion</key>
  <integer>1</integer>
</dict>
</plist>

There's still two tricks up our sleeve, though. First off, although we have created an app bundle, our dynamic libraries are still not in there. We've made a directory for it, but we're not going to copy them by hand: there's a much better solution in our case.

Enter dylibbundler, a utility that will:

1. Identify the libraries your program uses
2. Copy these libraries to a libs directory within your app bundle (as seen above in the directory structure)
3. Modify your executable so that the runtime library search path points to the libs directory inside your app bundle.

Before messing with dylibbundler, we need to compile our ooc program with an additional flag passed to the C compiler, to reserve enough room to set the library search path. That flag is -headerpad_max_install_names. As you may or may not know, you can pass options to the C compiler by prefixing them with + in the rock command line. So our build command becomes:

rock +-headerpad_max_install_names

And we can copy that executable to Foobar.app/Contents/MacOS. Then, we can use dylibbundler to copy the libs and alter the executable. That command should look like:

dylibbundler -od -b -x ./Foobar.app/Contents/MacOS/helloworld \
    -d ./Foobar.app/Contents/libs/

If everything went right, you should have a bunch of .dylib files in the libs directory.

Finally, we have one additional trick that we need to apply: we're all set on dynamic libraries (.dylib files) when your application is launching, but what about the CWD (current working directory)? As-is, it won't be the Contents/MacOS directory inside your app bundle, but it needs to be.

For that purpose, we use a wrapper shell script that will cd to the right directory and then launch the actual executable. Here's an example wrapper file:

#!/bin/bash
cd "${0%/*}"
./foobar

Don't forget to make it executable (chmod +x ./wrapper) and put it into Contents/MacOS. Also, keep in mind your Info.pList's CFBundleExecutable should be set to wrapper and not to foobar.

When all that is done, your .app should launch on double-click from Finder. Time to package it into a zip (zip -r Foobar.zip Foobar.app) and upload it somewhere!

For more advanced install methods on OSX, you might want to take a look at how to create a .dmg, or how to use PackageMaker.

A final note for OSX: when people encounter errors running your application, ask them to run it from the console with open Foobar.app to see error messages. And before blaming your app, run plutil Info.pList to check your info file for syntax errors.

On Linux, we face the same challenges as on Windows and OSX: 1) making sure your application can find dynamic libraries, and 2) making sure it can find its assets.

In my case, I had an additional challenge: I'm building from a 64-bit version of Ubuntu 12.10, but to be as universal as possible, I want to produce a 32-bit executable.

Getting and building rock is as simple as cloning it, building it with make quick-rescue, and adding it to your path. (For more detailed instructions, see the Windows section above).

Once we've done that, we can compile our application with the same commands as on other platforms. However, if you're on 64-bit and want to compile for 32-bit, you have a few modifications to make.

First, you want to install gcc-multilib and ia32-libs on Ubuntu. Then, you want to pass -m32 to rock, which will then pass it to the C compiler. And finally, since we'll use the same trick as on OSX to distribute dynamic libraries along the executable, we'll need to pass -Wl,-R,libs to the compiler (-Wl is a trick to pass options from the C compiler to the linker... it's layers upon layers).

So, our final ooc build command becomes:

rock -m32 +-Wl,-R,libs

Note that there are some peculiarities depending on the libs. For example, for SDL, the 32-bit dev libs on Ubuntu expose not -lSDL, but -lSDL1.2. And what's more, even with the correct path, it wouldn't link. So in my case, I had to tweak ooc-sdl/sdl.use to have this Libs line:

Libs: /usr/lib/i386-linux-gnu/libSDL-1.2.so.0

And then it compiled and ran fine. Note that you wouldn't have this problem if instead of relying on Ubuntu's dev packages, you compiled your own SDL version. Perhaps it is time to port homebrew to Linux? I know I'm fed up with the discrepancies between distributions and the whims of packagers.

And then we have one task left: actually copying dynamic libraries (that happen to be .so files on linux - so much for consistency) to our newly-created libs directory.

For that, we don't have a tool like dylibbundler for OSX, at least not that I know of. So we'll have to do with a few basic tools. Don't be afraid though, it's easier than it looks!

The first thing we want to do is to know which libraries our program depends on. Thanks to ldd, that's dead easy. Running ldd foobar should output something like:

  linux-gate.so.1 =>  (0xf76df000)
  libSDL-1.2.so.0 => /usr/lib/i386-linux-gnu/libSDL-1.2.so.0 (0xf761a000)
  libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf7470000)
  libasound.so.2 => /usr/lib/i386-linux-gnu/libasound.so.2 (0xf737d000)
  libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf7351000)
  libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf734c000)
  libpulse.so.0 => /usr/lib/i386-linux-gnu/libpulse.so.0 (0xf72f9000)
  libX11.so.6 => /usr/lib/i386-linux-gnu/libX11.so.6 (0xf71c2000)
  *snip*

Thanks to a wee bit of command-line trickery, we can copy all these libraries into our libs folder:

cp $(ldd foobar | cut -d ' ' -f 3) libs/

What are we doing here? We're breaking down the output of ldd by fields separated by spaces, using the cut command. Then we're passing all these paths to cp, and telling it where to copy them. If it went fine, you should have a bunch of .so files in libs/.

Since we compiled our executable with the -R linker option, running ldd foobar again from the directory of our executable should know display this kind of output instead:

  linux-gate.so.1 =>  (0xf77ab000)
  libSDL-1.2.so.0 => libs/libSDL-1.2.so.0 (0xf770f000)
  libc.so.6 => libs/libc.so.6 (0xf7565000)
  libasound.so.2 => libs/libasound.so.2 (0xf7472000)
  libm.so.6 => libs/libm.so.6 (0xf7446000)
  libdl.so.2 => libs/libdl.so.2 (0xf7441000)
  libpulse.so.0 => libs/libpulse.so.0 (0xf73ee000)
  libX11.so.6 => libs/libX11.so.6 (0xf72b7000)
  *snip*

One last thing: you want to make sure that whatever happens, your executable is ran from the right directory. In order to do that, we can create a launcher script, just like we did for OSX. Our wrapper file might look like this:

#!/usr/bin/env bash

SCRIPTPATH=$(cd $(dirname $0); pwd -P)
cd $SCRIPTPATH && ./foobar

Don't forget to chmod +x wrapper, and voila! Portable linux app, cross-distribution, and runs on 32-bit and 64-bit, normally. If you want to provide both a 32-bit and a 64-bit version of your app, nothing prevents you from making two separate builds. A word of warning though: it's easy to cross- compile from 64-bit to 32-bit, but I'm almost certain the inverse is not.

For starters (demos, etc.) I'd slap that binary, libs, and assets directory in a 'foobar' directory, itself in a .tar.bz2 archive (tar cjvf foobar.tar.bz2 foobar) and it'd be enough. For more advanced installation means... well, stay tuned. I have explored it a little bit but between Loki installer (obsolete), 0install (requires a client), autopackage (website down), and listaller (requires a client), I don't know where to turn.

You might want to take a look at makeself to make self-extracting archives, but anything that requires using the command-line is probably a bad idea. I'd suggest publishing on Desura and Steam, but I must admit I still have no experience there!

And on a final note, I would recommend against having anything to do with the Ubuntu Store, seeing as they seem to have dishonest business practices

Astute readers might have noticed the visible absence of static linking anywhere in this article. Here's the short version: static linking is unsupported on all the platforms we talked above.

And here's the long version: most libraries and platforms have given up on static linking. For OSX, applications have been for a long while depending on Frameworks, which are just a bunch of headers and dynamic libraries shoved in a folder. For Windows, everybody's using DLLs because they've never been a big issue since you can just put them alongside your executable (trust me, DLL "hell" is nothing compared to its OSX and Linux variants...)

So, you might succeed statically linking your application, but it's almost guaranteed that you will run into strange issues and in general, you'll be swimming against the current: the methods outlined above just work, without hassle.

Distributing games mean making sure it can find assets and the dynamic libraries it needs. Often, it means distributing dynamic libraries (.dll on Windows, .dylib on OSX, .so on Linux) in your application's folder or a subfolder.

On Windows, the OS will look for .dlls alongside your executable. On Mac, the best thing to do is to create an app bundle, put the libs in there and use dylibbundler to adjust your executable's runtime library search path. On Linux, you can simulate an app bundle with a directory, and directly pass -Wl,-R,libs to the C compiler when building your app, using ldd, cut and cp to copy the relevant libraries to your libs subfolder.

If you're building on a 64-bit OS, you should produce at least 32-bit binaries. On Ubuntu, it requires installing additional packages. On other distributions, your mileage may vary. You may end up having to work in a 32-bit chroot.

Often, a .zip or .tar.bz2 with your application it will be enough, especially if the executable runs on a double-click in the most used file manager on the target OS. More advanced options exists, and digital publishing platforms each have their own requirements.

All in all, I hope you learned something today and that this article will be a point of reference from now on for people in need.

As a reminder, there's a small, but growing, community around making games with the ooc programming language, hosted at oocgaming.org. The folks there will be happy to discuss packaging - and other issues - with you at length.

Comment on /r/fasterthanlime

Here's another article just for you:

Surviving Rust async interfaces

Aug 09, 2020

1 hour 2 min #rust · #async

I used to be afraid of async Rust. It's easy to get into trouble!

But thanks to the work done by the whole community, async Rust is getting easier to use every week. One project I think is doing particularly great work in this area is async-std.

Let's say we want to compute the SHA3-256 hash of a file. It's very easy to do with synchronous I/O:

Read more