Animative Blogging, Part 2: Sprite Extraction

My previous blog entry on animated GIF creation received some productive feedback in the comments, so I want to elaborate a bit more on my processes for creating the animations you see in this blog.  The focus of this article is sprite extraction, which I made use of in my study of the early evolution of sprites in arcade games.

Note that there are other techniques for sprite extraction, such as ROM hacking and tile-map editors that I won't cover here.  This article focuses on extracting from image video data produced by emulators.

Step 1: Configure the Emulator

Before you actually start capturing game data, it's important to be aware of what you're capturing from.  The video games that are the subject of this blog are all captured on emulators, and by default, emulators typically alter the game images in some way.  One reason that they do this is that old video games were produced at extremely low resolution compared to modern computer monitors.  Pac-Man, for example, had a native resolution of 224x288, and looks like this on my screen when brought down to its original size:

Image of the arcade version of Pac-Man being played on a typical desktop at the original screen resolution.

So that you don't need a magnifying glass to play, most emulators will enlarge the image to fit your screen by default.  Unfortunately, increasing the resolution requires adding data, which alters the image from its original form.  We don't want that if we're going to extract the original sprite.

Aspect Ratio

Another thing that emulators often change by default is the aspect ratio.  The goal of an emulator is to provide a reproduction of the game as it was experienced by players of the original system.  In the early days of gaming, just about everything was played on a cathode ray tube monitor with a 3:4 (or 4:3) aspect ratio, so emulators will usually present the games at the same aspect.  However, the resolution of the video data in many games was not at an exact 3:4 ratio -- for example, Donkey Kong (1981) has a resolution of 224 x 256, which corresponds to an aspect of 7:8 if your pixels are square (as they are in computer monitors).  

So does this mean that these arcade games were being played with effectively rectangular pixels?  Yep, and the designers would have known about that and drawn the graphics to account for it.  Unfortunately, this means that an emulator will have to manipulate the image in order to achieve the "correct" 3:4 aspect on your monitor.

For sprite extraction, you'll want to disable this correction, if possible.  It may result in sprites that appear slightly different from their intended shape, but the problem with extracting from the forced-aspect images is that the sprite no longer has a fixed shape; that is, it will appear different on different parts of the screen.  The image below was extracted from a clip of Donkey Kong played at a forced 4:3 aspect.  You can clearly see repeated lines throughout the image -- thicker rungs of the ladder, Mario's tall head, uneven platform blocks, etc.  

For comparison, here is a similar screen with a 7:8 aspect:

The barrel is wider and Mario is a bit fatter than he would be otherwise, but the image is at least self-consistent, and the sprite of Mario shown above will look the same even if he moves to a different part of the screen.  It's true that Mario would have looked thinner on the original machine, but if you want to reproduce that effect in your game or image, I suggest applying it to the bitmap in post-processing.

To put it another way, you're capturing an accurate reproduction of the original bitmap, not the original image as it appeared on the monitor.

MAME-Specific Configuration

In MAME, there are two ways to capture images in an unprocessed form.  First, there is a built-in game recorder that you can start with Shift-F12.  It should produce an MNG file in the "snap" subdirectory that records the raw game data at its native resolution.  If you want it in GIF format, you'll need another program to perform the conversion.  Also, be aware that some games have multiple "native" resolutions and MAME may end up outputting at the wrong one for your purposes.  Galaxian (1979), for example, has a different resolution for its star field than its sprites and tilemap data, but the MAME recorder will only recognize the star field resolution and give you highly distorted images.  There may be ways around this, but I haven't looked into too deeply.

The other option is to configure MAME so that the game appears in its native, unprocessed form on your screen and then use a screen recorder, like ScreenToGif, to capture the data.  This is my preferred approach, both because I can actually see things as I'm capturing them and the data feeds directly into ScreenToGif for more detailed manipulation without having to perform a conversion first.  Here are the options I had to change in MAME to make this happen:

The MAME parameters that I had to change to view the game's video data without correction.
MAME Parameter
If this is off, the game will go right to fullscreen mode.
Disabling this will start the game at its original resolution
When on, this makes the picture less pixelated, the opposite of what we want here.
Makes sure the graphics card and monitor are in sync.
This is the option that forces the 3:4 or 4:3 aspect ratio.
When off, MAME will stick to integer scaling in the video aspect.

There may be other image manipulation techniques that are options in the MAME or any other emulator you're using, so make sure they're all disabled.

Step 2:  Capture the Game

In the first part of Animative Blogging, I detailed the use of the Windows XBOX Game Bar for capturing gameplay, but for what follows, I'm going to suggest you use ScreenToGif instead.  Most video capture utilities produce compressed data that are "lossy", meaning that they only retain as much information as the software thinks is necessary to produce a visually pleasing video.  This is usually fine if all you want is to show a section of gameplay, but more detailed manipulation and study of the images often requires "lossless" capture.  ScreenToGif can do this for you and, as a bonus, provides a wide range of tools for frame-by-frame manipulation of the resulting animation.  So I recommend it even if you're not interested in extracting sprites.

Now that you have the game displaying at its original resolution, you can capture frames using ScreenToGif.  The recording pane has a  icon that you can drag to the MAME window to match the size.  Sometimes the pane won't be able to get quite as small as the game window, but you can crop it in ScreenToGif after capture.

Depending on your system, you may also have the option of decreasing the resolution of your computer monitor to make it easier to see the game data at its original resolution.  I don't find this necessary for sprite extraction, however -- if you need to play the game to get to the point where the sprite appears, just maximize to play, and minimize again when you need to capture.

Step 3:  Isolating the Sprite

Once you have captured still frames of your game of interest, you'll want to select ones where the sprite is clearly distinguished.  In very old games, like the ones I discuss here, this usually isn't very difficult, since most of them are set against a uniform background.  However, I will address more complex backgrounds at the end of the article.

In the example below (enlarged 4x for demonstration purposes), three of the four ghost sprites are set against a black background and would be easy to extract, but Pinky and Pac-Man himself are both partially overlapping background objects.  I wouldn't use this frame for Pac-Man or Pinky unless you really have nothing else.

Still image from the arcade version of Pac-Man 1980, pixellated.

The first thing you'll want to do is crop the picture to remove as much of the non-sprite pixels as possible.  Here is one such example from the above frame:

Red ghost from original Pac-Man set against a black background, pixellated.

If you're unable to capture an instance of your sprite without intervening background, you can still use a pixel editor to remove the offending pixels.  I do my pixel editing with a homegrown Python script, so I don't have any specific recommendations, but there should be plenty of resources on the web.  Here's Pac-Man from the aforementioned frame, both before and after manual pixel editing:

Open-mouthed Pac-Man from original arcade version, set against a black background, pixellated.

Open-mouthed Pac-Man from original arcade version, set against a black background, pixellated.

Step 4: Make Non-Sprite Pixels Transparent

In order to prepare your sprite for insertion into an arbitrary scene, you'll want some way of marking which pixels are part of the sprite and which pixels aren't.  In practice, this is usually done with transparency, where the sprite pixels are completely opaque and the non-sprite pixels are completely transparent.  This is stored in the image data as RGBA, where RGB are the red-green-blue color values and A is the alpha channel.  You can think of A as an opacity value, so for 8-bit image data the non-sprite pixels are assigned A = 0 and the sprite pixels have A = 255.

There are a range of methods you can use to add transparency to your extracted cutouts.  One such method was already demonstrated in the last installment of Animative Blogging, where you make use of ezGIF to replace the background pixels with transparency.  If you don't want to use a web-based tool, you can achieve a similar result with GIMP or Photoshop.  See here step-by-step instructions on how to do it in GIMP.

If I replace the black pixels with transparency in the images above, I get the final sprites:

Red ghost from original arcade version of Pac-Man, set against a transparent background, pixellated.

Original arcade version of Pac-Man, set against a transparent background, pixellated.

This is a relatively simple case in which the background is a single color.  I'll address more advanced techniques in a future entry.