User Interface System Overview

The UI System and its functionality are no longer supported. Please see the Scaleform GFx documentation for information on the currently supported user interface system.

Overview

This document describes the major features of the UI system. The design for the UI system is broken down into two parts - the UI system itself and the UI editor.

UI System

The UI is an artist-driven system, where the artist creates a layout for the UI controls, binds those controls to game data, and creates scripted events and actions via Kismet to perform various operations on the data. The UI system is composed of three major components - data source, presentation, and interaction. The data source component represents how the UI interacts with game data; the presentation component represents how the UI displays that data to the user; and the interaction component represents how the UI system processed events (both user generated events as well as system generated events). These three components are described in more detail below.

Data Store

Data stores are how the UI retrieves data from the game and publishes changes to data to the game. A data store can be persistent, in which case it is registered and attached to the UI controller object and is available to all widgets, or it can be temporary, in which case it is attached the current scene and is only accessible to the widgets contained by that scene. Data stores can provide any type of data to the UI, though the interface for binding and retrieving data from a data store is the same regardless of the type of data the data store provides.

Persistent data stores might track information such as gametype and character resource or state data, while per-scene data stores may track information such as the value that was entered into some widget in the scene, or information about the state of a pending connection. Data stores may provide static information, such as the names of all gametypes or character resource data, as well as dynamic information, such as the name of the current gametype.

There are also a few special types of data stores used by the UI. The currently active UISkin contains the list of styles available to the current scene, and can be referenced by widgets using the 'Styles' tag. The 'Attributes' data store is a virtual data store that modifies a UIString's text styles (such as bold, italic, etc.).

Data store features

• Responsible for providing data to widgets, accepting changes to data from widgets, and propagating that data to the appropriate location, if applicable.
• Capable of accessing arbitrary data types through a class-based design.
• Capable of real-time interaction with the underlying data source, for example relaying changes to game state values as soon as they occur.
• Capable of delayed, buffered, or batched transactions when publishing changes to data.

Data store categories

There are four main categories of data stores.

1. Game State - Tracks all data about the current game state, such as players, objectives, time remaining, current scores, etc. Game data stores can be nested, in that a GameState data store can contain references to other game state data stores. This is useful for isolating the weapon data store associated with a particular player, for example. Game data stores are further divided into two components:
   • Game state data providers: Provides state and static data about a particular instance of a data source, such as a player, weapon, pickup, or game objective. Data providers can generally not be referenced directly by the UI. Instead, they are normally accessed through a game state data store, such as the game state data store associated with the owning player, or the current game info instance.
   • Game state data stores: Acts as the first layer between the game and the UI. Each data store contains a collection of game state data providers, which provide the data for instances of a game object.
2. Game Settings - Provides the UI with access to the game-specific or global configurable settings. The Game Settings data store may expose such data as a gametype's configured MaxPlayers, GoalScore, etc, or user-specific data such as the the user's configured button layout. The Game Settings data store is also responsible for publishing user selections to the appropriate persistent location so that these values are used by the gameplay code when the game is played.
3. Remote Data - Provides the UI with access to data received remote machines across a network. One usage of this type of data store would be for a server browser, where game and player data about internet game sessions is retrieved from the master server.
4. Game Resources - Tracks all information about resources available in the game, such as the game modes, weapons selections, character selections, maps, etc.

Referencing data stores

Each data store has a unique tag which allows the UI to identify that data store. The specifics of how a data store tag is associated with an actual data store will generally be dictated by the needs or purpose of the type of data store. To expose data values to the UI, each data store provides a list of names corresponding to properties that can be bound to widgets.

Data stores and their values are referenced by the UI using markup syntax. The syntax for this is <DataStoreName:PropertyName>. For example, let's assume your game provided the user with a choice of three controller layouts, and you'd like the name of the current layout to appear on a label in a configuration screen. To bind the label to the property that controls this setting, you might set the label's value to something like <Controls:LayoutType> (assuming that the data store responsible for tracking this value used the tag 'Controls' and that it exposes a property named 'LayoutType' which controls which gamepad layout is used).

This method for binding widgets allows the UI to reference game data in a safe manner, since the data stores encapsulate lifetime management, ensuring that there are never any references between widgets and the underlying data represented by the data store which might interface with garbage collection.

Interaction

Each widget contains an EventProvider component which provides the widget with the list of events which are available to that widget. EventComponents contain an array of event classes (called UIEvent, though in reality these will probably be children of the SequenceEvent class from the Kismet system). Each of the UIEvent instances contained by the EventComponent has a collection of actions which are performed when the event is triggered. The list of actions assigned to a particular event can be different for each widget that implements that event.

There is a specialized UIEvent class for handling input events (which for the sake of simplicity, we'll simply call UIEvent_ProcessInput). In order for a widget to be able to process input, its EventComponent must contain a UIEvent_ProcessInput in its list of events. For the built-in set of widgets, all widgets which should handle input will already have a input processor in their list of events. For custom widgets that designers create (via prefabs), if they want that widget to process input, they can add an event processor to that widget. The UIEvent_ProcessInput class will be able to handle an arbitrary number of input key events. For each input event that it handles, it can define one or more actions that should occur as the result of that input. So for any given scene, only widgets that contain input processors are capable of handling input.

Since we can determine which input events the widget handles by looking at its event processor, we can determine which widgets are capable of responding to which input key events at any given time, and ignore all other widgets in that scene. Whenever an event occurs which alters the set of widgets that can process input, the scene's list of pollable widgets is updated. This will be encapsulated using an InputEventSubscription, a structure that maps an input keyname to an array of widgets. The widgets contained by an InputEventSubscription are widgets that are currently eligible to process input, that contain a UIEvent_ProcessInput event in its EventComponent's list of events which respond to that input key. Typically widgets are only eligible to process input if they are part of the focus chain, but widgets which are configured to accept unfocused input would also be considered eligible. Each scene has a mapping of input keynames to InputEventSubscriptions. When an input event is received, the scene retrieves the InputEventSubscription struct that corresponds to that key name. This gives the scene immediate access to all widgets that could potentially process that input event. The order in which the eligible widgets occur in the array is the order in which they are given the opportunity to process input. So, the currently focused control might be the first widget in the array, followed by its parent, followed by any of its parent's children that can process that input, so on.

A scene's InputSubscribers is not a static data structure. During the lifetime of the scene, widgets will constantly add and remove themselves from the scene's list of eligible widgets. When widgets lose focus, they will remove themselves from the scene's list of eligible widgets. When widgets gain focus, they will add themselves to the scene's list of InputSubscribers. Additionally, a widget may add or remove new input keys to its list of events that it can handle, based on a change in state or some internal logic of the widget (for example, an editbox might require that the first letter entered into a textbox is an alphanumeric character, but any characters after that are OK). When this happens, the widget notifies the scene and the scene adjusts its InputSubscribers accordingly. The point is that a scene's InputSubscribers is modified only in response to some event; there is no polling necessary. When an input event occurs, the InputSubscribers is already up to date and can be used as a reliable source for determining what to do with the input.

When an input event is received, the scene looks up the appropriate InputEventSubscription from its InputSubscribers list. It iterates through the list of widgets that are currently eligible to respond to that input event, giving each widget the opportunity to process the event. These widgets are all guaranteed to be capable of responding to that input event (though they may of course for whatever reason decide not to). If none of the widgets in the InputEventSubscription want to respond to that input, or if they did respond, but are configured to allow that input event to continue being passed around, the UI system allows the GameViewportClient to pass that input event to the remaining interactions in its array. (When an input event is handled, but passed on anyway, the event object representing the input event will be marked to indicate that the input was handled already, possibly including information on what action was taken or who handled the input. This will be used to allow other game objects to work together with widgets to perform some mutual task. Again, this is only if the widget is configured to not capture an input event that it responds to).

Presentation

The presentation subsystem is responsible for displaying the UI to the user. This includes rendering widgets and their resources as well as rendering data provided by data stores.

Scenes

The outermost container for a group of widgets is called a UIScene. Scenes contain a collection of widgets, and the relationship between scenes and widgets is very similar to the relationship between a map and the actors placed in that map, in that all widgets must be contained by a UIScene. Widgets cannot be rendered directly - they are always rendered by their container scene. There can be more than one scene active at one time, and some scenes should be able to persist across map changes. UIScenes are created in the editor and stored in an unreal package file; opening a UIScene is exactly like loading any other Unreal resource during a game. UIScenes can be associated with a single player, or they can be globally accessible to all players. The responsibilities for a UIScenes include tracking and updating the positions of its widgets, routing input events, and rendering its widgets. While the widgets in a scene may be organized hierarchically (widgets may contain other widgets), the order in which input and rending is passed to each widget is not necessarily determined by the organizational structure of the scene's widgets (though they should, for the most part).

Widgets

The base class for all widgets is called UIObject (an in many places widget and UIObject are used interchangeably).

Features:

• Every widget is capable of acting as a container for other widget
• Each widget has a unique GUID.
• Each widget can specify any number of "states", such as pressed, active, focused, disabled, etc.
• Widgets can be docked to any other widget, provided that the dock target is contained within the same scene. Widgets are able to dock to more than one widget, but the maximum number of docking-sets any widget can have is equal to the maximum number of faces that widget has. 2D widgets can only have 4 docking sets (one for each side). A single widget can be a target in multiple dockings sets, with the limitation that you can not bind two sides of the source widget to the same side of the target widget (i.e. You can't make a widget dock both its top and right side to the target widget's right side). You are allowed to dock faces that are perpendicular; the target face's length will divided in half, and the source face will dock to that point on the target face. You are not allowed to create a circular relationship between two docking sets (WidgetA docks its left face to WidgetB's right face, and WidgetB's right face docks to WidgetA's left face). A widget cannot contain itself in one of its own docking sets. However, it is perfectly legal for WidgetA to dock to WidgetB, and WidgetB to dock to WidgetA, provided that the docking relationship is not circular. Consider a container-child relationship (WidgetA is the container, WidgetB is a child of WidgetA). You may want WidgetB to dock its left side to WidgetA's left side, then have WidgetB's right side dock to WidgetA's right side. This would result in WidgetA expanding or contracting to contain WidgetB, and WidgetB would remain left-aligned to the left edge of WidgetA. You can also specify a padding value for each docking set.
• The position for widgets can be specified in either pixels or percentage. This value can be relative to the entire viewport, the owning scene, or the UIObject's container widget.
• Any widget can be configured to respond to any event (both user or system). The programmer of the widget class determines which events that particular class is capable of responding to. The designer chooses which of those events are implemented in instances of that widget class, or the programmer of the widget can make the widget always implement one or more event types.
• Each widget can execute a different action in response to a particular event.
• Every widget can potentially process input. Widgets can be configured to always process input or only when focused. Widgets can also choose whether to swallow both processed and unprocessed input.
• Any attribute of a widget can be animated using Matinee, including rotation, scale, position, color, anchor, UV-coordinates, etc.
• Hierarchial context menu support for each widget
• All widgets will support drag-n-drop, where applicable.
• Can be configured to automatically align and position the widgets it contains

(The following is not a reference of all built-in widget classes that will be available; it's just an overview of a few of the more interesting classes).

Label

A basic widget for displaying text in the UI.

• Supports auto-sizing itself to accommodate the string it contains.
• Supports three clip modes - normal (clip), wrap, and ellipsis (the last few characters of the string are replaced with ellipsis)
• UIString is the underlying workhorse.

List

UIList is designed to support very complex types of lists in a general fashion so that it can be abstracted enough to be configurable via a UI. A list is composed of three components: data source, container widget, and presenter.

• Container - The container acts as the conduit for the data to the UI. The container knows nothing about the type of data it contains. The container is responsible for tracking the number of elements it has, the size of each cell, handling input (including tracking which elements are selected, changing the selected element, etc.), adding and removing elements from the list, and passing data back and forth between the data source and the presenter. The container is represented in the UI by the UIList widget.
• Data - The data source is where the list gets the data values that will be rendered into the list. There can be multiple data sources for any given list. The list has access to any data store that is accessible by its parent scene. The data source for lists are handled by the UIString class.
• Presenter - The presenter is what controls how the data is represented in the list container. The presenter is responsible for formatting the data according to the layout configured by the designer. The presenter handles adjusting the values for any dynamically adjustable parameters in the list container, according to the non-dynamic constraints of the list container.

UIString

UIString is the core renderable entity for all data that is presented by the UI. UIStrings are divided into one or more UIStringNodes, where each node corresponds to either normal text or markup data. Markup data is defined as text that will be replaced by some data retrieved from a data store. As mentioned in the section on data stores, data stores are referenced by <DataStoreName:PropertyName>. Markup can change the current style: <Styles:NormalText>, can enable or disable a style attribute: <Attributes:B> <Attributes:/B>, or it can indicate that the markup should be replaced by the value of the property from the data store specified in the markup: <SomeDataStoreName:PropertyName>. UIStrings dynamically generate UIStringNodes by parsing the input text. For example, passing the following string to a UIString generates 7 tokens:
The name specified '<SceneData:EnteredName>' is not available. Press <ButtonImages:IMG_A> to continue or <ButtonImages:IMG_B> to cancel.
The tokens generated are:

[0] = "The name specified '"
[1] = "<SceneData:EnteredName>"
[2] = "' is not available.  Press "
[3] = "<ButtonImages:IMG_A>"
[4] = " to continue or "
[5] = "<ButtonImage:IMG_B>"
[6] = "to cancel."

These tokens are then converted into UIStringNodes. Markup which references a data store is processed by that data store, which creates the appropriate UIStringNode prefilled with the appropriate values. There are two types of UIStringNodes - one for rendering text, and one for rendering images. UIStringNode_Text nodes are used for strings, as well as data values that are rendered as text, such as ints, bools, etc. UIStringNode_Image nodes are used for data store markup which represents data that is only renderable as an image, such as objects or textures. Each string node is capable of calculating and tracking its own precise extent value (width and height), based on the post-processed version of the text or markup. This is only done when certain elements of the node's state changes, such as the data itself, scaling or kerning, etc. The extent of the complete UIString is then very easy to calculate - it's just the sum of the extents of all nodes. A UIString may also be configured to manually control the extents of each UIStringNode, such as when contained by a UIListElementCell, where the designer wishes a particular region of the cell to have a certain width or be right aligned or something.

Widget styles

Styles control how a widget is displayed. A style might contain information about how to render a texture (scaled, stretched, etc.), and it may also contain information for drawing a string (font, color, attributes, etc.). There will be a style browser in the UI editor. In order to apply a style to a widget, the user will be able to select a style from the list of existing styles, or create a new style (either based on an existing style or from scratch). When a style is created, it is assigned a persistent GUID. All styles for a particular widget skin are stored in a single unreal package file. The resources required by the style may also be stored in the skin file, or they might be located in another package. A complete game UI is required to have at least one style package that will serve as the default style package. Any additional skin sets are based on this default skin, and only the changes from the base skin are stored in the custom skin file. When a widget is assigned a style from a skin package, a mapping between the widget's GUID and the style's GUID is placed into the skin file. For custom skin sets, by default widgets will automatically map to the customized version of the style contained in the custom style set, but the user may choose to assign a completely different style to a particular widget. This only changes the style of that widget for that skin set, and any skin set that is based on the custom set. Custom sets can be hierarchical, in that custom skin sets can be based from other custom skin sets.

Skins

Animation

UI Editor

A document describing the major features of the UI Editor, and how to use them may be found in the UIEditorUserGuide.