Content Streaming

Overview

The next generation of console platforms promises an incredible leap in graphics and computing power. However, memory is at a premium on these platforms. To achieve maximum detail, next-generation games will need to use streaming and seamless world support to load game content on-demand.

Streaming

Here, we take streaming to mean: optimized loading of large (typically, multi-megabyte) chunks of raw content on-demand according to a visibility prediction scheme, directly into memory in the platform's native format such that no on-the-fly data conversion is required.

The streaming system does not aim to support complex object-oriented data such as UObject-derived classes; a level's UTexture objects (the 200-byte UObject-derived classes describing textures) will be resident in memory at all times that the level is loaded, while the bulk mipmap data associated with the texture might be streamed. In other words, streaming support is aimed only on bulk content, and not complex data structures.

We only aim to stream texture mips this way and use package streaming for other content streaming needs. We do not plan to stream audio.

Seamless Worlds

Unreal Engine 3 supports seamless worlds, and the seamless world system is aimed at dynamically background-loading and unloading the complex object-oriented data associated with levels. This works in conjunction with streaming so that, together, all of the data associated with worlds may be dynamically loaded.

The world (a UWorld object) in Unreal Engine 3 may consist of many levels (ULevel objects); a typical game may contain several hundred individual levels. These levels may be dynamically loaded and unloaded based on proximity, explicit load/unload triggers, and other criteria.

A level defines a set of actors and other complex object-oriented data that can be loaded and unloaded atomically. Conceptually, either all of the objects referenced by a given level are loaded and visible to C++ and script code (i.e. FindObject?), or none are. This atomicity makes it possible for objects referenced by a level to contain pointers to other arbitrary objects, a pattern that occurs extremely frequently in complex hierarchies of actors, components, materials, and gameplay scripts. Thus C++ and script code never has to deal with the case of references to objects which have not yet finished loading.

While the ULevel abstraction exists to support loading and other atomic per-file operations, they are generally invisible to gameplay code. Actors and gameplay functionality are exposed through the UWorld abstraction, which aggregates all of the currently loaded static and dynamic actors into a single list. Thus ordinary gameplay code does not need to worry about level "boundaries", and actors don't "change levels" as they move through the world.

Asynchronous Package Loading

The seamless world loading code is based on the ability to fully load a package in the background, which can be used to (pre)load arbitrary combinations of other data and have garbage collection take care of removing it again.

Detailed Explanation

Package Loading

At the core of Unreal's loading code are its packages, which are modeled closely after DLLs for their dependencies. A package always contains the following in this order:

• package file summary
• name table
• import table
• export table

The package file summary contains some basic information and offset/ size of the various tables stored in the package. The name table contains all serialized names. The export table contains all objects serialized into the package, while the import table contains all direct dependencies of objects serialized into the package.

Each export in the export table contains an offset into the file where its data is located. The exports are saved into the package in an order that allows them to be created / loaded in a linear fashion. This means that an object's class or outer is always sorted before the object itself.

The C++ classes directly dealing with packages are the ULinker subclasses ULinkerLoad and ULinkerSave. ULinkerLoad is what ties a UObject to the package it has been loaded from. There are a few kind of UObjects that never have a linker associated with them like e.g. intrinsic classes and top-level packages. For those GetLinker() will return NULL. This is not the only time an object is dissociated from a linker though. Saving a package under the same filename will dissociate it from its linker to allow the saving code to safely overwrite the file and renaming will do the same. Another case where loaders are reset is if a package is being loaded into a different outer. On console platforms linkers are not kept around to save on memory.

In general, loading an object with an associated linker will cause the load operation to return the object already in memory while loading an object that doesn't have an associated linker will cause the object to be replaced in-place from disk. The code is more complicated in practice due to the fact that the Editor needs to support manually reloading packages to undo changes but not clobber changes to a package if it has been indirectly referenced after having been saved and therefore not have a linker associated with it. Script compilation also adds another layer of complexity there. On console platforms the engine will always try to find an object in memory first and in the case of single object loading via StaticLoadObject will not load it from disk as blocking operations on load from DVD are not practical there.

Loading a single object follows a slightly different codepath than loading objects via packages and is not covered in this document. The following list describes what goes on when the engine loads an entire package.

• UObject::LoadPackage calls BeginLoad
• UObject::GetPackageLinker is called which
  • checks whether there is an existing linker for the package
  • potentially resolves the not fully qualified filename
  • creates a ULinkerLoad object using a "real" filename
• the returned ULinkerLoad is used to "load all objects"/ "create all exports" in order
• EndLoad is called which
  • in a loop routes the actual serialization and PostLoad calls
  • dissociates cached import objects from the import table to avoid dangling pointers
  • dissociates forced exports from the export table to avoid dangling pointers

To elaborate a bit more on the work that goes on when a ULinkerLoad object gets created:

• package file summary is read
• name table is read
• import table is read
• export table is read
• existing objects are potentially hooked up to export table in the Editor
• all imports are verified (matched with export indices in their respective source linkers)

Verifying all imports can be deferred by specifying LOAD_NoVerify to speed up creating the linker object, though this also means that error handling might not be as graceful.

Asynchronous Package Loading

The engine supports loading a package in an asynchronous fashion. More precisely, the engine supports loading all objects in a passed in linker in an async fashion. This functionality can be found in UObject::PreloadLinkerAsync and does the following:

• create all imports via ULinkerLoad::CreateImport which maps to ULinkerLoad::CreateExport on the appropriate linker
• create all export objects and serialize their data (one by one)
• ensure that PreLoad (the actual serialization) has been routed to all exports
• route PostLoad to objects

Those operations are spread out across several frames, with a time limit. The next step is only executed if the previous one completed.

This process is done in a loop as PostLoad might cause new objects to be constructed / loaded. Once no more work is left, RF_AsyncLoading is removed from all objects that have been loaded / created in an async fashion. Then import objects and forced exports are dissociated (like in EndLoad) and the completion callback is called.

Objects created during async loading are marked as RF_AsyncLoading as they need to be hidden from the rest of the engine until they are finished loading. The async loading code cannot be used to in-place reload existing objects and will assert if it is used this way.

Seek-free Aspects

In the case of seek free loading, it is guaranteed that all imports are already loaded so the import creation phase doesn't result in any disk activity. All objects required to load the package will be found in the export table even though they might not be part of the package. This is accomplished by an object flag called RF_ForceTagExp which upon save forces objects that would otherwise reside in the import table to be serialized into the package and henceforth reside in the export table. These objects are flagged specially so when exports are created they are created as if they had been loaded from their original package, which means they will have an "outer most" that is NOT the loaded package.

Before creating the exports for "forced exports" the engine checks whether the object already exists in memory and reconciles it. This allows several maps in the multi-level case to share the same content that only needs to be loaded once.

A quick side note about async loading is that regular loading cannot occur while async background loading is taking place, which is why any background activity is being blocked on until completion if there is a non-async load request. The same holds true for garbage collection, which cannot occur during async loading and henceforth will block until the loading has finished before performing its work. The automatic timed garbage collection code will avoid calling the garbage collector while there are any outstanding requests in order to not end up blocking on load.

Seek free loading relies on duplicating dependencies that are not guaranteed to already have been loaded into the to be loaded package to avoid any potential seeks when resolving them aka creating imports. The vast majority of seeks is going to be caused by little helper objects like e.g. UMaterialExpression that have virtually no payload and are very cheap to duplicate. Static mesh and sound data on the other hand is probably going to be the most expensive commonly duplicated data from a size point of view.

Textures usually don't have their payload / bulk data duplicated but rather only have the UObject data and the lowest miplevels be put into the seek free package and have the rest streamed in via the texture streaming code.

Duplication can be reduced without sacrificing much of the seek free loading part by creating packages with shared content and loading those first. This is a mostly manual process and we didn't make much use of this for Gears of War with the exception being audio for multi-player maps.

Networking

The networking code relies on a structure called package map which is used to communicate object references via indices. The package map code uses the object linker index and toplevel package name to convert objects to indices and back. Objects loaded via the seek free loading code won't have a linker associated with them so the code needs to store the original linker index in the seek free package and use that to build the package map.

Initial loading

The initial loading requires a bit more handholding. During the cooking step:

• each script package containing native classes is fully loaded
• all content referenced by it is put into it via forced exports

The code does this in a way so forced exports are sorted after all regular exports for mixed script/ content packages. This is to allow loading all data for regular exports out of a single block of memory without having to deal with cross references that couldn't be resolved otherwise.

CreateExport is going to be called on all script package exports, which in turn is going to create and serialize the classes and also create but not serialize the content. Then once all classes are created the memory for the script block can be freed and content can have its exports created and serialize being called on it like regular seek free packages using a sliding window, allowing async prefetching of data.

All this is done before the obligatory code making sure that all native classes are loaded. This is done to ensure that any content referenced by native classes was loaded in a seek free way and there shouldn't be any disk access during this phase. As a quick important side note, ALL native classes are ALWAYS loaded at engine startup which means that content references via e.g. default properties or direct code references will also be loaded. Native classes are NOT garbage collected so any content referenced by them will ALWAYS be around. This makes it very important to not have any content references in native classes that don't always need to be loaded if they weren't native. A good workaround for this case is to subclass the native class and reference the content in a subclass residing in a different script package.

On initial load all script packages containing native classes are then fully loaded. Script only classes in packages not containing native classes and content referenced is going to be put into the map files using them. There are various ways to reduce duplication if content is guaranteed to be loaded for all maps in a given world by e.g. only moving the content into the persistent level of a world or having always loaded packages.

Console Cooking

As far as cooking data for consoles goes, separating the bulk data from the forced exports requires that regular packages are being cooked before maps depending on them and that the engine also tracks the offset, size and other assorted information about the bulk data in those packages so it can correctly patch up the bulk data being serialized into the seek free level package. This also requires that all maps depending on a package are being recooked after said package changes. The cooker has been adapted for this purpose by allowing a list of map names to be specified on the command line and it will do all the work required to only cook what needs to be cooked and make sure to automatically handle all dependencies.

See the Content Cooking page for more details.

Miscellaneous Notes

The concept of load flags has a few issues related to their propagation and is slated to be removed / cleaned up. The main problem is that the load flags used to load each object inside a package is the one used when originally creating the linker which could e.g. happen implicitly when serializing a package that has a reference to said package.

Both initial loading and the per frame amount of work can be reduced by improving performance of UObject::Serialize and derived versions and the various << operators. The engine supports a concept of bulk serialization which relies on every single member of a struct to be serialized in the order it is declared in memory and the data being serialized mapping directly to the layout in memory which means potential gaps due to alignment are serialized as well and different byte order is not an issue. This relies on the saving code to perform any potential byte order conversion and also deal with the fact that struct member alignment might be different for platforms. All in all bulk serialization is a very fragile construct but works quite well for simple data types like array of colors or vectors. The engine currently makes use of it in a couple of places and ideally we are going to extend the concept of TTypeInfo to allow natively serialized arrays to automatically use bulk serialization for simple cases.

Another fruitful optimization is going to be special case handling of common data types that are currently serialized via SerializedTaggedProperties like e.g. FMatrix if they use script serialization. The 'immutablewhencooked' property flag can be used in many cases to automate this without requiring custom serialization code for the data type.

Texture Streaming

For static objects, the texture streaming code looks at the screen space size of the bounding box of the object it is applied on, taking into account UV scaling factors and is streaming in the texture data, which resides compressed on disk in the case of consoles, into video memory.

Dynamic objects have their textures streamed based on visibility and there are various overrides to force streaming in miplevels of objects to avoid texture popping. We currently plan to extend the texture streaming code on skeletal meshes to be better suited for a fast paced game like Unreal Tournament 2007 with its many uniquely textured characters.

Audio Streaming

There is no current way to stream audio content. However, there is a cooking solution implemented to get a greater diversity of dialog, which may meet your requirements.