Decoding Doom RPG

I first played Doom RPG on a Sony Ericsson j300a in late 2006. I wasn't expecting much more than a functional Doom-themed Wolfenstein port with finicky controls. (I clearly didn't know much about it before buying it!) What I got was so much more than I thought possible from a feature phone game.

Unfortunately, Doom RPG never found its way to smartphones, or any other platform. EA Mobile doesn't even seem to have it listed in their 'Legacy' section. There aren't many games in the Doom franchise, so it's surprising to see this one get so little attention.

Actually playing the game today is a bit tricky. I've had success running it on KEmulat0r, configuring it as a Sony Ericsson k800. Of course, KEmulat0r appears to be a dead project, nearly 9 years without an update. I couldn't get it to run at all in Microemulator (an open source project), but that hardly matters as it also appears to be dead.

Doom RPG isn't nearly as popular as its older siblings, Doom and Doom II, but there does seem to be some interest. Kyle873 and Kate Fox (Pink Silver) developed a GZDoom Mod inspired by the game. While they're not the same type of game, DoomRL (now just DRL) and Jupiter Hell both offer turn-based gameplay in a Doom-like setting.

The goal here is to puzzle-out and document as much of Doom RPG as possible to empower interested modders and other developers looking to get Doom RPG running on modern systems. There's a lot of stuff here, so I won't say much, if anything, about the disovery process.

Contents

• Before we begin
• Wall Textures (wtexel.bin)
• Sprites (stexel.bin, bitshapes.bin)
• Palettes (palettes.bin)
• Entities (entities.db, entities.str)
• Mappings (Complete textures and sprites) (mappings.bin)
• Levels, Maps, Commands and Scripts (*.bsp)
• Sprite Data Appendix
• Thing Data Appendix
• Entities Data Appendix

Before we begin

Some important things to remember before we get started:

• I'm using version 1.0.92 for everything here. There may be differences between older and new versions.
• All numbers are unsigned integers.
• All integers are little-endian.

Doom RPG uses three coordinate systems. If you're familiar with raycasting engines like the one used in Wolfenstein 3D and other early first-person games, you'll likely be familiar with the "coarse grid" and "fine grid" they employ. The coarse grid represents the "blocks" or "grid squares" that compose the level. In Doom RPG, the coarse grid is 32x32. The fine grid divides each coarse grid square in to smaller blocks, typically matching our texture width. In Doom RPG, the fine grid divides coarse grid blocks in to 64x64 blocks and, consequently, the map in to 2048x2048 blocks.

Doom RPG places objects and enemies (things) on a third grid we'll call the "medium grid", which is somewhere between the coarse grid and fine grid. This divides the map in to 256x256 blocks. You can convert medium grid coordinates to fine grid coordinates by multiplying them by 8.

Wall Textures (wtexels.bin)

All the wall textures are stored in wtexels.bin Textures are all 64x64 pixels in size. Each pixel is represented by a 4-bit index in to a color palette.

The first four bytes in wtexels.bin represent a uInt32 that tells us the remaining length of the file. As each texture is 2048 bytes in size, dividing 53248 by 2048 tells us that there are 26 textures in the file.

There is no palette information associated with the textures in this file.

Doom RPG renders textures in vertical strips, from left to right, just like old raycasters like Wolfenstein 3D and Doom. It would be convenient, then, to store the textures on their side, beginning with the left-most strip, so that each strip could be read sequentially, which is precisely how they're laid out. A naive rendering (each pixel in sequence from left-to-right, top-to-bottom) will produce a mirror image of the texture, rotated 90-degrees to the left.

One additional quirk: While each pixel is represented by 4-bits, meaning each byte contains the color index for two pixels, they're reversed from what you'd expect. The least significant nybble represents the left-most pixel in the group and the most significant nybble represents the right-most pixel.

For example: If you encounter [01 23 45 67], the output would look like [1 0 3 2 5 4 7 6]

Sprites (stexels.bin, bitshapes.bin)

Sprites are more complicated than textures, as they're spread out across two files: stexels.bin and bitshapes.bin

stexels.bin, like wtexels.bin has a 4-byte header, which is just a uInt32 that tells us the remaining length of the file. The rest is just pixel data, 4-bits per pixel, with the same quirks we saw in wtexels.bin In a confusing twist, only the visible pixels are included. There isn't a value to represent a 'transparent' color, so only the visible parts of the sprite have texels. To produce the sprites, we need to know where those invisible pixels belong. This is where bitshapes.bin comes in.

bitshapes.bin contains the "shape" of our sprites. In the sprite data segments, each bit represents one pixel with a 0 meaning transparent and a 1 meaning colored. The file itself starts of with 4 bytes, a uInt32, representing the remaining length of the file. This is followed by a bunch of variable length records representing individual sprites. Each sprite record contains a 12-byte header followed by the sprite shape data.

I've only managed to decipher one small part of the Sprite Record Header, the length of the data segment + 4, which is enough to move to the next record. The rest is unknown:

Sprite Record Header (12 bytes):

	4 bytes (unknown)
	2 bytes (uInt16) Length of Sprite Shape Data + 4
	6 bytes (unknown)

Sprites are of variable size. They aren't all 64x64 like the wall textures. In order to properly render the sprite, you'll need to know how many bytes are used per vertical strip, as well as the number of strips. This information does not seem to be in the header, but I've managed to work-out the correct sizes for all the sprites. This is included in the Sprite Data Appendix.

To clarify, each 1 bit represents 4-bits in stexels.bin. You'll need to read them together to compose an entire sprite. Additionally, if a sprite in bitshapes.bin contains an odd number of 1's, you'll need to advance your pointer to the next byte for the next sprite. That is, a sprite with an odd-number of visible pixels will waste four bits in stexels.bin.

There is one additional quirk. Like the nybbles in our wall texture data, the individual bits composing our sprite shape are reversed for each byte, with the least significant bit representing the left-most pixel and the most significant bit representing the right-most pixel.

Conveniently, like textures, a naive rendering (each pixel in sequence from left-to-right, top-to-bottom) will produce a mirror image of the sprite, rotated 90-degrees to the left.

You'll notice that neither stexels.bin or bitshapes.bin contain an index of offsets for sprite records. While you can rather quickly walk through bitshapes.bin to find the sprite you want, you won't know where to start reading stexels.bin to get the texels. That data doesn't appear to be anywhere. You can find it by walking through each sprite shape in sequence in tandem with stexels.bin and making a note of each offset. I've done this already and included those values in the Sprite Data Appendix under "stexels.bin Offset".

Again, like textures, no palette information is included. This makes sense as sprites are frequently used with different palettes.

Palettes (palettes.bin)

The first four bytes of palettes.bin gives us the remaining length of the file. This is immediately followed by the palette records, each of which are 32 bytes in length.

Each palette contains 16 colors. Each color is 16 bits, in BGR565 format (not RGB565). You'll want to convert this in to the more familiar 24 or 32 bit color.

The red and blue components are 5 bits wide. The green component is 6 bits wide. We need to do more than just copy the values, as our colors would be very dark. White (0xFFFF) would become 0x1F3F1F, which is a very dark green. Shifting the components up gets us a lot closer, but they still won't be quite correct. White would turn in to 0xF8FCF8, which is both duller, and a little green.

The most obvious way to do this is to find the ratio between 0x1F and 0xFF and 0x3F and 0xFF and multiply the values. This is better, but with integer multipication we still end up with a very slightly green white (0xF8FCF8). If we try to correct this by adding 1, multiplying, and subtracting one, we preserve white, but end up with the same problem for black (0x0000), which will turn in to a slightly purple 0x070307.

While you could work around those problems, there is a much faster and simpler way. For red and blue, after shifting the values left by 3, copy the three most-significant bits to the least significant positions. Do the same for green, but with the two most significant bits. Whites and blacks are preserved, and none of our colors are tinted green. This is the way the game does this internally:

	// separate each component
	red   = color & 0x1F;
	green = (color >> 5 ) & 0x3F;
	blue  = (color >> 11) & 0x1F;
	
	// convert to RGB888
	red   =   (red << 3) | (red >> 2);
	green = (green << 2) | (green >> 4);
	blue  =  (blue << 3) | (blue >> 2);

	// put them back together
	color24 = (red << 16) & (green << 8)  &  blue;

There are 143 palettes contained in palettes.bin

Entities

Some entity information is stored in entities.db and entities.str. Not all in-game 'things' are stored here, with 'wall decorations' and 'fences' being the more obvious omissions. No sprite or palette information is included in either file.

The entities.str file contains the in-game description of each entity. The file begins with a uInt16 that tells us the number of records in the file. Records are variable length. Each record begins with a uInt16 that tells us the length of the string data that follows. These records are matched with the records in entities.db by the order in which they appear.

	entities.str
		uInt16: number of Records to follow
		
	Records:
		uInt16: length of string
		varies: string data

The entities.db file contains some information about in-game entities. What information it contains varies by entity type. The file begins with a uInt16 that tells us the number of records in the file. Records are 8 bytes in length.

	entities.db
		uInt16: number of Records to follow
	Records
		uInt16: id
		uInt8:  type
		uInt8:  p1
		uInt32: p2

Entity records are divided in to 4 parts: a 2-byte id, a type, and two parameters. The id's match the id's used for 'things' in .bsp files. The 'type' field groups related entities. Entities with a shared type (mostly) share the same purpose for p1 and p2 (keys and medkits being exceptions).

	Type 0 - Doors
		P1: Normal: 0, Locked: 1, Unlocked: 2
		P2: not used

	Type 1 - Enemy
		P1: Class
		P2: Enemy 'level' within each class Level1: 0x126, Level2: 0x1D3, Level3: 0x280

	Type 2 - Humans
		P1: not used
		P2: not used

	Type 3 - Keys, health, armor, credits
		P1: Class (Keys: 0x18, Health: 0x14, Armor: 0x15, Credits: 0x16 & 0x17)
		P2: Keys: Green - 0, Yellow - 1, Blue - 2,  Red - 3
		    Health, armor, credits: Amount to add

	Type 4 - Inventory Consumables
		P1: Sm Medkit: 0x19, Lg Medkit: 0xA1, SoulSphear: 0xA2, Berzerker: 0xA3, Dog Collar: 0xA4 
		P2: Only for Medkits, amount of health to add when used

	Type 5 - Weapons
		P1: Cycle order
		P2: Amount of ammo added on pickup	
	
	Type 6, 16 - Ammo Pickups (6 - single, 16 - multiple)
		P1: Ammo Type (Halon: 0, Bullet: 1, Shell: 2, Rocket: 3, Cell:4)
		P2: Amount of ammo added on pickup

	Type 7 - Furniture, portal, computers, unnamed
		P1: unknown
		P2: not used
	
	Type 8, 9, 14, 15 - Unknown (Unused?)
		P1: not used
		P2: not used
	
	Type 10, 11 (10 - Fire, 11 - Lava)
		P1: not used
		P2: not used
	
	Type 12 - Breakable Items
		P1: Class (Barrel: 1, Crate: 2, Door/Wall: 3, Power Coupling: 4)
		P2: unknown
	
	Type 13 - Furniture
		P1: unused
		P2: unused

There are 115 entities included in the file. See the Entities Data Appendix for more details.

Mappings (mappings.bin)

mappings.bin contains four unrelated groups of records. The file starts out with a 16 byte header, composed of 4 uInt32's that indicate the length of each of these groups.

Group 1 maps wall textures to color palettes. As each texture is used with different palettes, we end up with 51 records instead of 26. Each record is 8 bytes in length, being composed of two uInt32's. The first uInt32 indicates the wall texture (from wtexels.bin), the second indicates the palette (from palettes.bin). How, in both cases, is not obvious. To find the texture index, divide by 4096. To find the texture offset, divide by 2 and add 4. To find the palette index, divide by 16. To find the palette offset, add 4.

We can now extract all of the wall textures.

Group 2 maps sprites to color palettes. There are 207 8-byte records, each composed of two uInt32's. The first indicates the sprite (from bitshapes.bin), the second indicates the color palette (from palettes.bin). To find the offset in to bitshapes.bin, add 4. To find the palette index, divide by 16. To find the palette offset, add 4.

Using some additional information about the dimentions of each sprite (from the Sprite Data Appendix), we can finally extract all of the sprites.

Group 3 contains 93 records. Each record consists of a single uInt16. This maps the wall texture id from the line segment portion of .bsp files to the wall textures in Group 1. For example, The wall texture assigned to the Portal (in reactor.bsp) is 34, but the wall texture is at index 24. The value at Group 3 index 34 is 24.

Group 4 contains 252 records. Each record consists of a single uIn16. Values range from 0 to 205. Its purpose is unknown, but appears to be related to sprites.

Levels and Maps (*.bsp)

As luck would have it, Simon Howard (Fraggle) has worked out a good bit of the Doom RPG bsp file format [Mirror]. I'll expand on his work here, filling in quite a few of the missing details. You can also take a look at Richard Walmsley's Map Viewer.

.bsp files start off with a 13-byte header:

	.bsp header:
		uInt24: Floor Color BGR888
		uInt24: Ceiling Color BGR888
		uInt24: unknown (always 0)
		uInt8:  Level id
		uInt24: unknown

The Level id number starts at 1 for intro.bsp and increases by one for each file in the natural level order: junction.bsp, level01.bsp, level02.bsp, level03.bsp, level04.bsp, level05.bsp, level06.bsp, level07.bsp, junction_destroyed.bsp, reactor.bsp. The enemy montage at the end of the game endgame.bsp and the test items.bsp both have an id of 0.

BSP Tree

Following the header is the BSP tree. It starts with a uInt16 representing the number of nodes. Each node record is 48 bytes in length, so you can quickly skip over this section if you don't need it.

The first four bytes of each node record represent the two coordinate pairs of a bounding box (x1, y1, x2, y2 in that order). (These are positioned on the "medium grid" mentioned earlier. You can multiply each value by 8 to convert them to the "fine grid".)

The last six bytes represent different things, depending on the value of the first byte. We'll use Fraggle's naming here. The first byte is the "nodetype", the second is named "splitpos", the remaining four represent two uInt16's, the purpose of which varies depending on nodetype, called "arg1" and "arg2".

If nodetype is zero, the current node is a leaf-node. The splitpos byte is ignored. arg1 tells us the number of line segments in the node, and arg2 gives us the index of the first line segment, in the line segment record set, of the first line segment in the node. (The remaining line segment indices are in sequence from the first segment.)

If nodetype is 1, the current node is split in to two subnodes along the y-axis. splitpos contains the position along the y-axis to split the node. Arg1 contains the index of the first subnode, arg2 the index of the second.

If nodetype is 2, the current node is split in to two subnodes along the x-axis. splitpos contains the position along the y-axis to split the node. Arg1 contains the index of the first subnode, arg2 the index of the second. (This is identical to a nodetype of 1, but split along the x-axis instead of the y-axis.)

Line Segments

The line segment section immediately follows the bsp tree. Line segments are used for walls and doors. The line segment section begins with a uInt16 that tells us the number of line segment records that follow.

Line segment records are 36 bytes in length. The first 4 bytes represent the start and end coordinate pairs for each line segment on the medium grid. This is followed by a uInt16 for the wall texture, then a uInt16 for various flags.

The value for the wall texture is an index to group 3 in mappings.bin which has the actual wall texture index.

For consistancy with Fraggle's work, I'll use LSB 0 numbering. From most significant to least significant:

	15: Flip texture horizontally
	14: East-facing wall
	13: West-facing wall
	12: North-facing wall
	11: South-facing wall
	10: Locked door
	 9: Shift horizontal
	 8: Shift vertical
	 7:  not used
	 6:  not used
	 5: Secret / Hidden
	 4: Shift West or North
	 3: Shift East or South
	 2: Door (with bit 0)
	 1:  not used
	 0: Door (with bit 2)

Doors, short walls, and walls that are not perfectly vertical or horizontal do not have an orientation flag (bits 14-11) set.

Doors all have either bit 8 or 9 set, but none have bit 4 or 3 set.

Things

Immediately following line segments are 'things'. Things is a broad category that covers everything to enemies and pickups to wall decorations and 'fences'. There are 107 things in the game. See the Thing Data Appendix for details.

The things section starts with a uInt16 that tells us the number of thing records that follow.

Thing records are 5 bytes in length. The first two bytes represent the x and y coordinates of the thing on the medium grid. Things are centered on their coordinates. The third byte identifies the thing id (not necessarily the entity id) of the thing being described or the wall texture if flag bit 2 is set. The remaining two bytes (uInt16) are flags which provided additional information about the thing. Flags are mostly unused by enemies and pickups. Wall decorations and 'fences' use most of the flags.

Using LSB 0 numbering, from most to least significant:

	15:  not used
	14:  not used
	13:  not used
	12:  not used
	11: Fence (with bit 1)
	10:  not used 
	 9: North or South-facing wall decoration
	 8: Dead Enemy
	 7: Wall Decoration
	 6: West-facing
	 5: East-facing
	 4: South-facing
	 3: North-facing
	 2: Treat as Wall *
	 1: Fence (with bit 11)
	 0: Hidden

	 * When set, the id is used as the wall texture id.
	   Orientation bits also change:
	   6: East-facing
	   5: West-facing
	   4: North-facing
	   3: South-facing

Events

The events section immediately follows the things section. Fraggle calls these 'interactions'. Events mark every scripted action in the game. They can be triggered with the action button like activating doors, using computers, and talking to people or they can be triggered simply by stepping on or off a coarse grid square. They can even be triggerd by scripts.

The events section starts with a uInt16, which tells us how many event records follow.

Event records are 4 bytes long, with all of the fields packed tightly in to a uInt32. From most to least significant:

	bits 31-25:  unknown (7 bits) [sometimes set to 1, e.g. the portal in reactor.bsp]
	bits 24-19: Number of commands in script (6 bits)
	bits 18-10: Index of first command (9 bits)
	bits 9-5  : Event Y position on coarse grid (5 bits)
	bits 4-0  : Event X position on coarse grid (5 bits)

Commands and Scripts

Following events are commands. The command section starts with a uInt16 which tells us how many commands follow.

Commands are 9 bytes in length. The first byte is the command. The next 8 bytes are two uInt32's, the first of which (arg1) acts as arguments to the command. The second (arg2) deals with the execution of the command. Each event has a value associated with it I'll call the Event Variable. It defaults to zero, and can be modified by other commands. That variable is referenced by the event's position on the coarse grid (typically 16 bits with bits 15-8 representing the y coordinate and bits 7-0 representing the x coordinate).

Arg2:
	Bits 31-16: Only execute command IF this Value matches the Event Variable

	Bits 15-0:
	15:
	14:
	13:
	12:
	11:
	10:
	 9: things/walls are added/removed by this command
	 8: Execute command on 'action button pressed' 
	 7: East  (execute command on 'leave', in directions set by bits 7-4)
	 6: North 
	 5: West  
	 4: South 
	 3: West (execute command on 'enter' in directions set by bits 3-0)
	 2:
	 1: East
	 0:

Commands:

	 1: Change Location
		arg1: Orientation (bits 23-20) (N: - S: - E: - W: 8)
		      Location Y,X on coarse grid (bits 15-0)

	 2: Change Level 
		arg1: Flags, level
		      Completed current level (bit 31)
		      unknown (bits 30-8)
		      Level filename (local string index, bits 7-0)
	
	 3: Run Event Script
		arg1: Event location Y,X (bits 15-0)

	 4: Show Status Bar Message
		arg1: Message (string index, bits 15-8)

	 5: Not Used

	 6: Not Used

	 7: Show Thing 
		arg1: Thing state (bits 15-8) (state 0: normal, state 2: dead)
		      Thing id (bits 7-0)

	 8: Show Message
		arg1: Message (string index, bits 15-8)
	
	 9: Get Map Data

	10: Enter Passcode (Halt on failure)
		arg1: Prompt (string index, bits 15-8)

	11: Set Event Variable
		arg1: Value (bits 31-16)
		      Event Variable to Modify (bits 15-0, coarse grid Location)

	12: Lock Door
		arg1: Door (line segment index)

	13: Unlock Door
		arg1: Door (line segment index)

	14: Not Used

	15: Open Door
		arg1: Door line segment index

	16: Close Door
		arg1: Door line segment index

	17: Not Used

	18: Hide Things at Location
		arg1: coarse grid Location (bits 15-0)

	19: Incriment Event Variable
		arg1: event variable (bits 15-0)
		      (increases by 2x after zero: 0, 1, 2, 4, 8, etc.)

	20: Not Used

	21: Increase Status Item Count
		arg1: Amount to gain (bits 11-8)
		      Status item (bits 7-0) health: 0, armor: 1, credits: 2

	22: Decrease Status Item Count
	    arg1: Amount to lose (bits 11-8)
	          Status item (bits 7-0) health: 0, armor: 1, credits: 2

	23: Show Message and Halt IF Status Item Count is less than Amount
		arg1: Message string index (bits 31-16)
		      Amount (bits 15-8)
		      Status item (bits 7-0) health: 0, armor: 1, credits: 2

	24: Show Status Bar Message
		arg1: Message (string index, bits 15-8)

	25: Explosion
		arg1: type? (bits 31-24), location on coarse grid (bits 23-0)

	26: Show Message
		arg1: Message (string index, bits 15-8)

	27: Change Level Parameters (called before command 2) 
		arg1: unknown (bits 31-24)
		      Start Level At coarse grid Location Y,X (bits 23-8)
		      unknown (bits 7-0)
	
	28: Not Used

	29: Shake Effect
		arg1: Intensity (bits 31-24), duration in ms (bits 23-0)
	
	30: Set Floor Color
		arg2: Color BGR888 (bits 23-0)
	
	31: Set Ceiling Color
		arg1: Color BGR888 (bits 23-0)

	32: Give/Take All Collected Weapons
		arg1: 0 - Take, 1 - Give

	33: Open Store
		arg1: store id (0, 1, 2, or 3)
	
	34: Set State of Thing at coarse grid Location (used only to kill humans)
		arg1: State (Bits 23-16) (state 2: dead)
		      unknown (bits 15-10)
		      Human coarse grid location Y (bits 9-5)
		      Human coarse grid location X (bits 4-0)
	
	35: Particle Effect
		arg1: effect (bits 31-24)
		      color BGR888 (bits 23-0)
		      (Effects: 0x20 blood spurt, 0x23 teleport/transformation)

	36: Draw Frame

	37: Wait
		arg1: time to wait in ms (bits 23-0)

	38: Unknown. Used Only Once (in reactor.bsp in the "gate lift" script) Related to the portal?

	39: Show message IF [some level] not complete
		arg1: Level id (bits 31-24) from bsp header
		      Message (string index, bits 15-8)

	40: Add Notebook Entry
		arg1: Note (string index, bits 15-8)
	
	41: Halt IF missing keycard
		arg1: Keycard id (bits 1-0)
		(Green: 0, Yellow: 1, Blue: 2, Red: 3)

Block Map

The last 256 bytes of the file represent the block map. The blocks are ordered from left to right, top to bottom. The block map is 32x32 blocks in size, meaning that each block is represented by two bits. The MSB is a flag that tells us if a block is part of a secret area. The LSB tells us if the square is impassable. Bit pairs within each byte are not ordered as you'd expect, but 'reversed'. Each byte represents 4 blocks, but the left most block is represented by the least significant pair, the most significant pair representing the rightmost block.

	Bit pairs:
		00 - Floor
		01 - Wall 
		10 - Floor (secret)
		11 - Wall (secret)

This block map of Level 2 shows all four block types:

Sprite Data Appendix

Thing Data Appendix

Entities Data Appendix