Will XNA tools be able to help reduce game sizes?
In this article, Aaron Stanton speculates that XNA technologies might be capable of helping game developers reduce the size of their games in order to fit more game content onto DVD's. I'm pleased to confirm that based on our tests with early versions of XNA technology on actual game sources we have indeed been able to identify some fairly substantial size reductions on several games. I should point out that reducing the size of games is certainly not the only benefit we will be enabling with XNA, but it does happen to be a nice side effect of using some of the new technologies we're building.
For the rest of this entry I'll provide a bit more context into how this game size reduction is achieved. I won't attempt to describe in full detail everything that our technology will enable since we are going to be presenting a lot more information at the upcoming Game Developer's Conference (don't worry if you're not attending GDC - the same content will also be available on our Web site after GDC).
The particular XNA technology which enables game size reduction (along with a slew of other benefits) is one we are currently calling XNA Build. XNA Build is a tool for helping game developers manage their game asset pipeline(s). If you're not familiar with a game asset pipeline, this is basically the way in which content finds its way from source assets (files created by tools such as 3D Studio Max and Maya) into the final format which is expected by the game engine. Along the way there are typically several tasks which are performed as files progress along the asset pipeline, resulting in a format that is consumable by the game engine.
To describe what exactly happens to a file as it moves through the asset pipeline I'll use as an example a canonical task known as a triangle stripper. A triangle stripper can efficiently reduce the extremely high-fidelity source assets from millions of polygons into a more digestible polygon count (thousands or tens of thousands of polygons), usually with the goal of yielding a higher frame rate in the game by reducing the number of polygons which must be processed. Now you might wonder why the content creation tool doesn't just save the file using fewer polygons and in an engine-consumable format to begin with, thus eliminating the need for a pipeline (at least in our simplified example). There are actually several good reasons why a game asset pipeline exists. A very basic reason is that you don't want to lose the high-fidelity version of the artwork that your artists spent hours or days creating because you might need it later. You could later decide to target a more capable engine or platform where you can afford to process more polygons while maintaining a high frame rate. In this case you would want to run the high-fidelity source art through the triangle stripper again, but on a more conservative setting. Another reason is that when you preview your game you might discover that you need to brighten a set of textures so they are easier to see. As anybody who works with audio or video editing can attest to, it's much better to start with the highest possible quality of an asset when performing such operations because there might be quality degradation suffered during the various edit steps. Yet another reason to have a game asset pipeline is that game development is a very iterative process - you may decide that you're willing to have a slightly slower frame rate in exchange for better detail, and hence you want to instruct your polygon stripper task not to strip out as much detail and to generate a new build to preview. Rinse, wash, repeat until you find the right balance - it's much easier to do this with a build pipeline than by having to go into the content creation tool and re-exporting everything each time. There are several other reasons for having a game asset pipeline which I won't go into here, but if you're interested in this topic I would recommend Ben Carter's book called The Game Asset Pipeline.
Now that we have the lengthy description of a game asset pipeline out of the way, let's get back to XNA Build. XNA Build is designed to help studios with their game asset pipeline by providing a tool for defining, maintaining, debugging, and optimizing their pipelines. XNA Build can also provide a great deal of additional information a studio can benefit from by capturing build pipeline dependencies. XNA Build can gather a lot of dependency information simply by "watching" files move through the pipeline. XNA Build also provides API access to its built-in dependency store so that tools such as Max and Maya can provide additional dependency data. Once you start collecting this dependency information, XNA Build can begin to give a studio a clearer view of the files going into their pipeline, the relationship of those files to each other, and the files coming out of the pipeline. This dependency information can unlock several benefits for the studio, such as enabling incremental builds, or giving an artist insight into where a given texture is used in the game. Now this might sound like studios should already have this information, but very few studios we talked to actually did. Most studios are stuck using primitive batch files or Perl scripts to model their build processes, and they rarely have the time or resources to invest in building out the infrastructure of their build pipeline. There are also other reasons studios don't have centralized dependency information, but I won’t go into all of them here. XNA Build, in addition to providing a productive way of creating and maintaining the build pipeline, is designed to start collecting and surfacing all of that dependency information to enable a studio to be more productive.
So how do we enable game size reductions? Well it turns out that once you have good dependency information (again, something that most studios don't benefit from today) you can begin to generate a list of files which are being deployed as part of your game but never actually used. This orphaned content can result from obsolete files, misspelled copies of files, or any number of factors. Without a reliable dependency tree, it's usually not worth risking removing files even if they appear like they might not be in use. Hence, the common practice is just to leave them in the pipeline, and ultimately these unused files could get deployed along with the game which leads to a larger game size. With XNA Build, since we can efficiently collect and report on this dependency information, it allows a studio to finally clean up their pipeline and game layouts by looking for files which are in the pipeline but never used by the game.
So does it work? The theory sounds right, but can we really reduce the size of games? Our results from looking at several titles with early versions of XNA Build indicate that we certainly can. Results will vary from game to game, of course, but one of the games I can share information for is a PC title called MechCommander 2. It turns out that 40% of textures which shipped on the final MechCommander 2 game disk were actually never used in the game! And MechCommander 2 is an older title with even fewer assets and less pipeline complexity than most modern titles, so for newer titles and next-gen titles we expect that we’ll see even better results. There were also some additional files besides textures which were never used in the game, such as a handful of mech models themselves which were never used by the game. I should point out that a 40% reduction in textures does not necessarily equate to a 40% reduction in overall game size, but it is a good proof point that we're headed in the right direction.
But should we care? As Aaron's article concludes, the games he sampled aren't yet coming close to filling up a DVD9 disk, which is the format used by the Xbox 360 and the original Xbox. But as tomorrow's games become bigger with more and more of that HD content we've come to love and expect, we want developers to concentrate on building a great game and worrying less about being constrained by game size. And for PC games, where CD media is still the norm to appeal to the broadest customer base possible, this can help a studio reduce the overall number of CD’s they have to manufacture while delivering as much game content as possible. That's why we believe that technologies such as XNA Build will play a very useful role in this and other game development scenarios.
There's also another way we benefit from reducing game sizes which is worth mentioning. The size of downloadable games and demos, such as those which have been available for the PC for years and are now on console with the introduction of the Xbox 360 Marketplace, can be reduced as well. Smaller download sizes obviously mean that gamers can spend less time waiting for a download and more time actually playing the game. And after all, enabling a great game playing experiences is the ultimate goal of XNA.