Wednesday, February 24, 2010

Starcraft 2 Model Format Pt. 1

Due to a number of requests I've received, I've decided to write down some of my findings in the .m3 model format. Here we go:

Overall structure

The .m3 format is a mixture of the World of Warcraft .m2 format and the Warcraft 3 .mdx format in that it has a parsing structure similar to the former, but uses tags, like the later. The list of tags with offsets are located at the end of the .m3 file. Specifically, the file header is

struct M3Header {
fourcc header_tag
uint32 tagindex_offset
uint32 tagindex_size
uint32 unknown1
uint32 unknown2
}

The header_tag should be 'MD33'. The tagindex is the aforementioned list of tags, and starts at tagindex_offset and has tagindex_size elements. I suspect that unknown1 and unknown2 may point to the tag where to start the recursive parsing, which is the 'MODL' tag. The elements of the tagindex have the form:

struct Tag {
fourcc tag
uint32 offset
uint32 repetitions
uint32 version
}

The first two elements, tag and offset, are pretty obvious I guess. The interesting part starts at the repetitions part. Each tag describes a fixed-size structure, which may be a little different for different files, hence the version number. The number of repetitions tells us how often this
structures is repeated in the chunk. As an example, a string chunk (with a 'CHAR' tag) describes a string that consists of repetitions many characters of size 1 byte (unless it's UTF8, but I've not encountered any special characters yet). This is great for loading, because we don't need dynamic parsing. We can just allocate and read the structure as many times as requested, without worrying about dynamically sized content.

If dynamic sizes are required, they are placed in a seperate chunk and referenced via the tag. Again, a good example are strings, which you can find all over the file, usually directly after the chunks where they are needed.

All chunks are padded to 16 byte boundaries using 0xaa.

Parsing

Once the tag directory has been read, my parsing starts at the 'MODL' tag (and I suspect Blizzards parsing does, too). The 'MODL' chunk is essentially the root of the parsing tree for the .m3 format, just like the header of the .m2 format, but instead of referencing the other chunks by offsets, they are referenced by their index in the tagindex. The references usually also contain the number of repetitions, which make them quite easy to spot.

That's it for today and the general parsing and file structure. Next time I'll get into the details of detecting the vertex format, reading out the vertices and faces and finally rendering a rough version of the model.

As an outlook, here are some more pictures (Templar, Ultralisk, Hydralisk):



Blizzards art team is just incredible!

Sunday, February 21, 2010

Starcraft 2

By now most of my regular readers probably know that I'm going to pause working at the university by then end of February and start working full-time for Limbic soon after. That means I'll also start to blog more, again.

By now most of you also have heard the exciting news that the Starcraft 2 Beta launched last week. As I have done many times before, I couldn't resist to dig a little into the game data formats and see what I could find. Of key interest to me is the .m3 model format that is being used in SC2. It's a natural evolution of the Warcraft 3 .mdx format, and the World of Warcraft .m2 format.

Since I can think, I've been dissecting formats of games that I liked from a technical point of view, in order to learn how they accomplish those great games (some prime examples are Half Life, Quake 3, Black&White, Warcraft 3, World of Warcraft). Hence, for the sake of the good old times and out of pure curiosity, I couldn't resists opening up the SC2 files in my Hex Editor and dig a little around. To my surprise, the .m3 format is quite a bit different from .mdx and .m2. It didn't take too long though to find myself through the file and extract the meaningful bits of data I needed to get a better understanding of the whole thing. So far I can say that I already love the SC2 engine, even more than the WC3 and WoW engines.

Here are two examples of cool models I rendered with my python tool, only vertices+normals for now (Hatchery and Hydralisk):


If someone is interested in my findings or if I find more time to make the tool more worth-while, I may upload it somewhere (github, probably) and I'm happy to share my insights with anyone interested.