Creating Animations for the Unreal Engine

Document Summary: A description of the content pipeline for getting animations into the Unreal Engine.

Document Changelog: Created by Scott Dossett and Jay Hosfelt; maintained by Richard Nalezynski.

Overview

This document will serve as a starting point for identifying the different types of art assets that are used in animation creation for the Unreal Engine content pipeline.

Development Tools

ActorX

All skeletal characters and animations must be exported from your 3D package and imported into the Unreal Editor through ActorX.

See the ActorX page for download information.

Characters

Character Setup and Rigging

Character models should be created in a fairly neutral pose, usually in a “T-Pose”, but it is okay to pose the joints in a relaxed, natural position. The bone structure should be a single hierarchy of bones, mimicking the human skeleton, posed to fit within the character mesh, and parented under a main “root” bone placed at the origin.

Although the Editor can import skeletal meshes created with the Y-Up axis orientation, we prefer to work with Z-Up in all 3D Packages. This makes things consistent across the board and allows us to transfer meshes and animations between Max and Maya with no axis issues.

With Z-Up being the standard, the characters should be modeled and rigged facing down the +Y axis in Max, or the –Y axis in Maya. (This axis is flipped between the two packages). In the Editor we apply a rotation offset to the characters to make them face the +X axis in the Editor.

The skeleton can also contain extra bones weighted to parts like cloth, armor or hair, and can also contain bones to be used just as attachment points, such as weapon bones or IK bones.

It’s best to keep your character animation rig completely separate from the skeletal hierarchy. Adding locators and nodes within the skeletal hierarchy can cause issues when exporting.

If multiple characters are modeled and proportioned to fit around the same skeleton, these characters can then share animation sets, reducing the amount of animations needed. Even characters with slightly different proportions can usually share animations.

Technical note:

• The maximum number of mesh-influencing bones in a skeleton is 75.
• The maximum number of total bones in a skeleton is 256.

(Increasing either of these numbers beyond their maximum will cause a second rendering pass on the character, reducing optimization)

• For the mesh, the maximum number of bone influences per vertex is 4. (Upon import, any vertex that is influenced by more than 4 bones will have its lowest influences removed and normalized among the remaining 4 influences for that vertex.) Note that there is a cost based on the max you use for a particular LOD. So if you can get away with using no more than 3, it will be faster. We generally try and use just 1 for LOD 1 and below.

Optimization note: If multiple characters are modeled and proportioned to fit around the same skeleton, these characters can then share animation sets, reducing the amount of animations needed. Even characters with slightly different proportions can share animations.

3DStudioMax note: We will setup a Biped character first, znd then build a skeleton proportioned to match that biped, then constrain that skeleton to the Biped. This allows us to animate onto the Biped, but then bake out the animation to the skeleton and delete the Biped for exporting in ActorX.

Maya note: When setting up bone rotations for the bind pose, the initial rotation values of the bind pose can either be put in the main Rotate fields, or in the Joint Orient fields. Putting these values in the main Rotate fields will let you always see the true rotation values of each joint. Putting these values in the Joint Orient fields acts as an offset, and allows you “zero out” the main Rotate values of each joint and still retain the proper bind pose. In the figure below, the characters are in the exact same pose. On the left, though, the arm bone has an initial rotation value, and on the right, the values are zeroed out without changing the pose by applying the initial rotation to the Joint Orient value. It's completely personal preference as to which method you choose to use, but it is good to choose one and make sure everyone sticks with it. Switching between these methods can cause inconsistencies in the pipeline.

Bones Hierarchy

Having a root bone separate of the pelvis is important. It doesn't matter that much where it is, although it may make things a bit more complicated for certain things. We find it easier to make sure animations will work together when the root is at the feet and origin. If it's somewhere up in the air, it's harder to detect when animations won't chain/blend properly and create a visible pop. For example, when doing root motion animations where Marcus climbs up something, it is easier to visualize where the root bone should be put because it should be where his feet are going to be touching the ground.

For IK, we use IK Bones for the feet and hands, direct children of the root bone. This eliminates position/rotation errors when blending animations with a long bone hierarchy.

Depending on the animations involved, the location of weapon bones in the hierarchy always has pros and cons. Most of the time our characters are holding guns so it makes sense for them to be children of the hand since no matter how heavily we compress the animations or how low of a framerate we animate at, the gun will maintain a solid position in the hands. The disadvantage is when weapons have to leave the hand. During certain reload animations like the sniper rifle, or when holstering the gun, you might get a lot of fighting between the bones and need to have a higher framerate so it isn't noticeable. Since those types of animations are rare in our case, we decided that having them as children of the hands would give us the best memory savings. If we were animating a bunch of melee weapons where they are constantly spinning them around or something, we would have have chosen to put the weapon bones higher in the hierarchy, maybe even as children of the root.

The weapon end bones were a put in as a way to change the pivot points of the guns to the center, but still have all of the animations look correct without having to change the weapon bone position in all the animations and then re-export them. There are some minor rig advantages with animating, but it would be better not to have them at all. It was a change that had to happen when we finished how mirroring animations worked without much time to get it implemented. It can be useful to reduce foot sliding. When you blend 2+ animations with a long hierarchy of bones you may get some errors; IK on the feet was a way to ensure the feet would remain at the right place. This was quite noticeable in directional blends. We did keep it on for much larger creatures, like the Brumak boss fight in Gears of War (PC).

The armroll bones - direct children of the upper arm bones - are there just to twist half the amount of the upperarm bone to help the verts deform better. This works better in-game than having roll bones going straight down the arm in the hierarchy, so that way we can still have a two jointed arm for working with IK.

Character Export

Once the character mesh is bound to the skeleton and properly weighted in your 3D package, it can be exported as a PSK file using ActorX. If there are extra bones or weighted meshes in the scene that you do not wish to export, you must either remove them from the scene, or (in Maya only) you can rename individual bones with the postfix _noexport, or rename the topmost hierarchy bone with the postfix _ignore.

Character Import and Setup

With the Unreal Editor open, in the Asset Browser, open the package you wish to import your character into. Right-click on the… (to be continued).

Weapons

First Person Weapon Setup and Rigging

1st Person weapons are the weapons you see yourself holding in-game. Since they are placed very close to the player's view, and are usually in the view a majority of the time, the models are usually much higher poly, and have much higher resolution textures on them.

Tip: Since the 1st person weapon is usually only seen from one side, the model can be optimized by removing any polygons or objects that are never seen by the viewer (usually on the front/right side of the model.). Only optimize the model after the final animations are complete, so you know which parts are never seen.

In your 3D package, place a camera at the origin and align it down the +X axis. This will represent the in-game view of the player. Position the 1st person weapon mesh in front of this camera view as it would be seen in-game. Adjust the camera FOV and mesh position to get the desired look.

Technical note: The 1st person models are displayed with a different rendering pass, so view offsets and FOV changes can be made that only affect the 1st person objects without affecting the entire scene view. Work with a programmer to adjust the FOV, view offset, and clipping planes for 1st person items.

The first major decision when setting up 1st person weapons is whether you will be able to see your own arms. The decision to have 1st person arms in view can add levels of complexity to the setup. On top of that, the decision to have multiple types of arms (male, female, robotic, creature, etc) with each weapon, adds yet another level of complexity.

1) 1st person weapon only. No 1st person arms. This is the least complex option and only requires the rigging and animation of the weapon. The weapon should be bound to a heirarchy of bones. This bone structure should be parented under a “root” bone that remains at the origin. Animate the weapon from the point of view of the camera at the origin. Once the animations are complete, export the skeletal mesh out as a PSK file, and export the animations out as a PSA file,

2) 1st person weapon and 1st person arms, but no arm variants. If all characters have similar arms, and you only require one model of the 1st person arms, the arms and weapons can be part of a single skeleton. This single skeleton should be parented under a “root” bone that remains at the origin. Animate the weapon and arms from the point of view of the camera at the origin. Once the animations are complete, export the weapon and arms meshes together as a single PSK file, and export the animations as a PSA file.

3) 1st person weapon and multiple variations of 1st person arms. If your characters are vastly different, and you require multiple 1st person arm meshes that need to work with each weapon, you will need two separate skeletons, one for the weapon, and one for the arms. Each skeleton needs to be parented under its own “root” bone at the origin of the scene. From a modeling standpoint, all arm mesh variants should be modeled in the same pose with the same proportions and bound to the same skeleton. You will export out each arm mesh variant into individual PSK files, and separately export out each weapon mesh into individual PSK files; and then import them into the Unreal Editor. These individual arm skeletal meshes will then be able to share a single AnimSet.

Once you’ve animated the weapon and arms together in your 3D package, bake out the animations and remove the constraints between them. Save this file but rename it as an “ExportReady” verson, so you have a version with both skeletons. Delete the arms skeleton and export the weapon animations into a PSA file. Reopen the “ExportReady” file with both skeletons. Now delete the weapon skeleton and export the arm animations with the same animation names and frame ranges, but into a different PSA file. Import each PSA into it own AnimSet. You will end up with an Arm AnimSet and a Weapon AnimSet for each weapon.

In-game, the weapon skeletal mesh and the arm skeletal mesh will be put back together and play synchronized animations, and any Arm skeletal mesh will then be able to play the animations from any Arm AnimSet.

Tweaking in the Engine

It’s best to work with a programmer to get the 1st person weapons and arms working in game with the correct camera FOV, loading the correct arm mesh variant, and get the weapon and arm animations synchronized. Since the animations were based on the origin of the scene, no offset should be needed to get them in view. You can either apply offsets to the mesh, or a programmer can hard-code offsets if they need some slight tweaking.

Third Person Weapon Setup and Rigging

3rd person weapons are the weapons you see other characters holding. To setup a 3rd person weapon model, it should first be bound to its own small hierarchy of bones and exported as a skeletal mesh. The weapon’s “root” bone should be positioned so that when it is constrained to the character’s “weapon” bone, the weapon lines up properly in the character’s hands. In your 3D package, animate the character and weapon together in the same scene with the weapon’s “root” bone constrained to the character’s “weapon” bone. You will eventually export out two separate but synchronized animations. One for the weapon and one for the character.

To animate the weapon and character together, such as a reload animation, bring the rigged character and weapon into the same scene, and constrain the weapon’s “root” bone to the character’s “weapon” bone. You will eventually export out the weapon and character as two separate, but synchronized, animations.

Animate the character and weapon as you desire, using constraints to keep hands locked to weapon parts or vice versa. Once this character-plus-weapon animation is ready, bake the animation out onto both skeletal hierarchies, and remove all constraints between them. Remove all animation keys from only the weapon’s “root” bone, and set it’s position and rotation values to <0,0,0>. This will retain animation on all the animated weapon bits, but keep the weapon at the scene’s origin. Save this file but rename it as an “ExportReady” version.

Weapon Export

Open your "Export Ready" file containing the baked animations for the weapon and the character. Delete the character skeleton and mesh from the scene, and export the weapon animation as a PSA file. Reopen the “ExportReady” file with both skeletons, and this time delete the weapon skeleton and mesh. Export the character animation with the same animation name and frame-range, but into a different PSA file.

Weapon Import

In the Unreal Editor, import the character’s PSA into the character’s AnimSet, and import the weapon’s PSA into the weapon AnimSet.

Tweaking in the Engine

Work with a programmer to constrain the weapon skeletal mesh to the character’s “weapon” bone during gameplay. This may require the creation of a Socket onto the character’s “weapon” bone. When the animation is then played on both the character and weapon, they should be synchronized.

Morph Targets

Morph Target Setup

Morph Targets (also known as Blend Shapes) are used to change the shape of a mesh, or part of a mesh by offsetting a group of vertices. This vertex deformation can be a good alternative when bone deformation will not suffice. Morph Targets can have many applications, including facial animation, muscle simulation, or even vehicle damage.

Morph Target Export

For use in the editor, Morph Targets are exported as individual PSK files (The same files used to export skeletal meshes) and imported in the Unreal Editor onto a previously imported skeletal mesh. If you have 20 morphs for a particular mesh, you will need to export 20 PSK files.

In your 3D package, target all morphs onto the same mesh. With this target mesh selected, turn on the first morph and export the target mesh as a .PSK file using the ActorX exporter. Turn off the first morph and then turn on the next one, and export out that same target mesh as another .PSK. Repeat this until each morph is exported into individual .PSK files.

Morph Note: All morphs should be exported from, and imported into, the same mesh to ensure that the vertex order remains the same. Importing a morph onto a mesh with a different vertex order will most likely cause problems. Be sure not to move the mesh, or change the skeletal bind pose when exporting each morph. Re-importing the source skeletal mesh can also cause vertex order problems and may require re-importing the morph targets.

Morph Target Import and Setup in the Editor

To import and setup morph targets in the Unreal Editor, you must create a MorphTargetSet Asset. Then import your PSKs into that MorphTargetSet to create a list of morphs. Then attach the morph list to your skeletal mesh in the Animtree. At that point, a programmer can access and apply the morphs, or they can be applied and modified in Matinee.

To create a MorphSet asset, double click on the skeletal mesh you wish to apply the morphs to. This will open the Animset Editor. In the Animset Editor, select the menu item File -> NewMorphTargetSet. Enter the Package, Group, and Name of the new MorphTargetSet asset you would like to create.

Click on the "Morph" tab at the top of the Browser section of the Animset Editor. Your new MorphTargetSet should be selected. If not, click on the drop-down menu and find your new MorphTargetSet.

To import your PSK files into the morph list, select the menu item File -> Import MorphTarget. Select your list of PSKs (you can shift select multiple PSK files), and click on "OK". Each morph will bring up a dialog box allowing you to either double-check or change the name of the morph. Morphs for high polygon meshes can take a while to import. After import, click on each morph in the MorphTargets list. This will preview the selected morph onto the source mesh in the 3D window, so you can verify that they imported correctly.

Technical Note: When importing new morphs, the editor compares vertices, in vertex order, between the imported PSK and the source skeletal mesh, and finds the ones where the positions have changed, and only saves data for the vertices that have moved.

To connect the morph targets to your skeletal mesh, open the Animtree for your mesh. This will open the Animtree Editor window. In the Animtree Editor select the Animtree node.

You first need to connect the Animtree to the MorphTartetSet you created. In the Properties section, expand the "PreviewMorphSets" list, and select the green "plus" icon to create an entry.

With the Animset Editor window open, go back to the Generic Browser and select your MorphTargetSet. Then come back to this Animtree Editor window and click on the green arrow to add that MorphTargetSet to the Animset properties.

Now you must connect the indivdual morphs to the Animtree. Click on a blank area of the Animtree Editor window to display a list of nodes. Select "MorphNodeWeight" to create that node. Right click again on a blank area, and select "MorphPose" to create that node. Link the morph nodes and the animtree as shown.

Select on the MorphPose node and, in the properties section, enter the name of an existing morph in your MorphTargetSet. You should now be able to drag the value bar below the MorphNodeWeight node and see the Morph blend in and out in the 3D window. Rename the MorphNodeWeight node. The name of the MorphNodeWeight node will be used through Matinee or by a programmer to apply the morph in-game.

To quickly duplicate this tree, and add more morphs to the Animtree, select both the MorphNodeWeight node and the MorphPose node and press ctrl+w to duplicate the nodes and links. Change the morph name in the new MorphPose node, and rename the MorphNodeWeight node.

Faces

Vehicles

For information on setting up and rigging vehicles, please see the Setting up Vehicles page.

Cameras

For information on setting up and rigging cameras, please see the Setting up Cameras page.

Creatures

Editor Tools

Understanding PSKs and PSAs

When importing skeletal meshes and animations into the editor, you will be working with two file types: PSK's and PSA's

A PSK file contains all the information about the skeletal mesh, including the mesh itself, material numbers, edge smoothing, bone hierarchy, bone position and orientation at the bind pose, as well as the vertex weighting information. It contains no animation info. When imported into the editor, this will create a mesh weighted to a hierarchy of bones. The mesh will be exported in the pose that it was in at the beginning of the timeline of your 3D package, and in the Editor, this will be its Bind Pose.

A PSA file contains a list of animations. These animations contain bone position and orientation data over a range of time. This data is relative to its parent bone, so if the hierarchy changes, animations may have to be re-exported. When imported into the editor, this will add the list of animations into an Animset, containing its own list of bone tracks, and position and rotation data for those tracks.

PSA Note: Keep in mind that this list of bone tracks is completely independent of the bones in the skeletal mesh it is linked to, and will only apply animation to the skeletal mesh bones that match the animset's track list. Bones can be added to the track list by importing an animation containing the extra bones. Bones can be removed from the track list by selecting the "Delete tracks" menu item in the animation browser. Deleting tracks from an animset will not add or remove bones from any skeletal mesh. Linking an animset to a different skeletal mesh will not change the animset's track list of bones.

Content Import and Setup in the Editor

To import your PSK file into the Unreal Editor, you must first open the Asset Browser, and load the package you wish to import your character into.

Right click on the package in the list, and choose "Import..."

Select your PSK file, and click "Open"

In the import dialog box that opens, verify the package name is the correct one. Typing a new, unused name in this field will create a new package upon import. You can also add a name to the "Group" field, if you wish to add a new sub group, or import into an existing sub group. In the "Name" field, verify that the name is correct. By default, it will use the name of the PSK file. Type the name of an existing skeletal mesh to overwrite it.

If you are using Y-Up in Maya, you can check on the "bAssumeMayaCoordinates" box, and this will apply offsets to reorient the mesh to look correct in the Editor.

Click "OK", and this will create a new skeletal mesh in the package. High polygon meshes may take a while to import.

To view and edit this mesh, double click on it. This will open the Animset Editor window. This window is used to modify animsets, skeletal meshes and morph targets.

Animation Set Editor

The Animset Editor Window is used to view and modify animsets, skeletal meshes and morph targets.

Preview Buttons: for manipulating what is shown in the 3D view
Browser: for listing and selecting meshes, animations, or morphs.
Skeleton Tree: for viewing the hierarchy and hiding bones in the 3D view
Properties: for modifying settings for meshes, animations, or morphs.
3D View: for viewing meshes, animations, and morphs
Time Slider: for playing and pausing animations, and viewing animnotifies

In the 3D view, use the left mouse button to orbit the view around your mesh. The right button moves your view closer and farther away, and the middle mouse button pans your view to the left, right, up or down.

Holding the "L" key while dragging in the 3D window, will allow you to move the light.

General buttons:

Show Skeleton

This displays the bones structure in the 3D window, overlaid over the mesh. With this turned on, you can show and hide certain bones by right clicking on them in the Skeleton Tree section.

Show Bone Names

This displays the names of each bone in the 3D window.

Show Wireframe

This toggles between wireframe view and shaded view

Show Reference Pose

This will force the skeletal mesh into its bind pose (the pose it was imported in).

Show Mirror

This will mirror the animation along the XZ plane. It is for preview only. It does not actually modify the animation.

Socket Manager

This opens the Socket Manager window allowing you to add Sockets to bones. Sockets are use to attach items such as weapons or particles. Sockets can be offset from the bone if needed.

New Notify

This adds an AnimNotify to the timeline of the selected animation. Animnotifies are points in the timeline that can trigger an event, such as a sound, a particle, attachment swap, or any number of things that a programmer can hook up.

Toggle Cloth

This allows you to view vertex cloth simulation.

Generate Soft-Body Tetrahedron Mesh

(?)

Toggle Soft-Body Preview Simulation

(?)

Show Uncompressed Animation

When animation compression is applied, and Show Skeleton is toggled on, this will overlay another skeleton playing uncompressed version of the selected animation. So you can visually compare the difference between the two skeletons.

Animation Compression...

This opens the Animation Compression window

LOD

This set of buttons will change the Level of Detail viewed in the 3D window. AUTO will show the LOD's as they would be displayed in game, and will change the LOD mesh depending on the viewers distance from the mesh. The current LOD level number can be seen in the top left corner of the 3D window. The B, 1, 2, 3 Buttons will force the 3D window to display that LOD level. "B" refers to Base, or LOD-0.

Playback Speed

This set of buttons will change the playback speed of an animation in the 3D window. But will only change the preview rate, and will not affect its play rate in game. The "RateScale" setting, which will affect the speed of the animation in-game, can be modified in the AnimSequence tab in the Properties section.

See the AnimSet Editor User Guide for more information.

Animation Tree Editor

See the AnimTree Editor User Guide for more information.

PhAT

See the Physics Editor User Guide for more information.

Motion Capture

This section will discuss Epic’s particular motion capture equipment and pipeline. But keep in mind there are many types of motion capture systems on the market. Each type has its advantages and disadvantages. Use the system that will work best for you. Whatever motion capture system or pipeline you use, the final animation must be exported from a 3D Package such as Max, Maya or Softimage, using the ActorX exporter, in order to be imported into the Unreal Editor.

We use a Vicon MX system (http://www.vicon.com/). This is a “Passive Optical” system which tracks reflective markers attached by Velcro to the actor. Our system has 36 cameras arranged fairly evenly throughout the room. We also record synchronized reference video at the same time.

Our motion capture pipeline is as follows:

1. Calibrate System
2. Capture the movement.
3. Clean marker data with Vicon IQ or Blade.
4. Target on to characters in Motionbuilder.
5. Polish animation in Max or Maya.
6. Import into the Unreal Editor using ActorX.

We currently use Vicon Blade software to capture, clean, and label the marker data. The marker placement on the actors can vary based on your needs, but we use 51 markers per actor in the following layout:

We also place markers on weapons and props – trying to put 4 or 5 markers on each weapon or object that needs to be tracked. You really only need 3 markers, but it helps to add one or two more in case of occlusion. It also makes it easier to determine the front of the object when you’re cleaning and labeling the data.

After cleaning and labeling the marker data in Vicon IQ, we then import that labeled marker data as well as a fully rigged character into a Motionbuilder scene, and then target the motion onto that character. Once Motionbuilder’s “Character” and “Actor” options have been adjusted to create the desired movement and posture, we plot the animation (also known as “baking” the animation) either to the character skeleton or to a Motionbuilder character rig. At that point we can further clean the animation, or adjust the timing, or setup loops, all within Motionbuilder.

We then import that animation from Motionbuilder onto a Max Biped or into Maya, to apply final polish to the animation.

Once the animation is exactly how we want it, we use ActorX to export that animation out of Max or Maya into the Unreal Editor.

Tip: We have a camcorder mounted at one corner of the room. Our system is setup to capture video from that camcorder and synchronize it with the data. Be sure the camcorder has a wide enough lens to cover all of the capture space!

Tip: It is easier to clean the data if all characters always stay within the capture volume, and if they start and end each performance in a T-Pose. The only time we let an actor exit the captures space is if he is sprinting. Once an actor leaves the capture space, he should remain out and not return to a T-Pose at the end of the scene.

Tip: We will attach colored Velcro straps to each actor so that we have an easy way to identify actors in the reference video.

Mo-capping for in-game animations versus mo-capping for cinematics.

Mo-capping for in-game animations: When capturing in-game animations you will probably only capture a single actor at a time. Possibly two actors if there is any type of interaction. Many times the animator himself will get suited up and be captured because he best knows what type of animation is required. Keep in mind which animations this will eventually blend to or blend from, and try to come close to that pose at the beginning and end of the motion. Also keep in mind whether the animation will be looped, such as idles or idle-breaks, and try to minimize any unique movements that will make the loop obvious. In-game animations will probably not be acting to a specific camera, so they will need to look good from every angle.

Mo-capping for cinematic animations: When capturing animations for Cinematics or Cut-Scenes, you will capture all actors that will be in the scene. In our case, we usually capture up to five professional actors at once. It is best to use pipelines similar to film production, including scripts, storyboards, animatics, and pre-recorded voiceover. All of these will help set up the scene for the actors and animators involved when recording the mo-cap. We will usually marker up a camcorder, and have a Cinematic Director walk around filming the scene from the intended view, at the same time the scene is mo-capped. Your actors will perform to this camera, and in the end, you will end up with reference video that can be used to setup the camera work in Matinee. And if a particular cinematic requires a handheld camera look, you will also have the camera movement captured in the mo-cap session.

Tip: You can adjust the actor’s stance and walk by adding weight to parts of the body. For the Boomer animations in Gears of War, we added ankle weights and a diving weight belt to the actor to achieve a “lumbering” walk cycle.

Development Assets

Animation Data

Motion Capture Data

Typically, human motion is best captured by motion capture.

At Epic we capture using Vicon IQ, then target the data using Motionbuilder, and do any time shifting or looping in Motionbuilder. We then export the targeted animation into 3D Studio Max using BVH's, or into Maya using the .fbx importer and do the final cleanup in either of those two packages.

Keyframed Data

Exporting

Importing