---

Web Server

---

Document Summary: Describes the usage and API of the HTTP WebServer implementation in UT3.

Document Changelog: Posted by Michiel Hendricks.

Overview

---

This document will explain how to use the Web Server and how to create new Web Applications. For low level information on the network connection used by the web server see TcpLink.

This document will assume you have basic knowledge of the HTTP protocol. It is not really required to know the details of the HTTP, but it will help.

The web server only supports a subset of the HTTP protocol. But web applications can extend the functionality. Only GET and POST requests are supported.

Structure

---

The following classes make up the web server functionality:

WebServer

This is the entry point for the web server, it manages connectivity and the applications.

WebConnection

This class is the acceptor class for the web server. For each client connecting to the web server an instance of this class is created. It manages the connection and preprocesses the request before handing it over to a web application.

WebRequest

This class embodies the HTTP request the web server received.

WebResponse

This class implements functionality to provide a response to the request.

WebApplication

An implementation of this class is responsible for handling the request and to produce a response. When using the web server, this is the class you will be going to extend.

HTTP Request process

---

1. Client connects to the web server.
2. A new WebConnection is created, connection counter in WebServer is increased.
3. WebConnection.Accepted is called.
   1. Start shutdown timer. A connection will only remain open for 30 seconds.
4. WebConnection.ReceivedText. The connection is receiving the request
   1. When still processing the request headers, split up the data into lines.
   2. Process the request headers
      1. Process request variables provided in the request URL
      2. If a GET request:
         1. Create WebRequest and WebResponse instances.
         2. Find the associated web application.
         3. If no web application was found, respond with a HTTP redirect to the default application (if set).
      3. If a POST request:
         1. Create WebRequest and WebResponse instances.
         2. Find the associated webapplication.
      4. Process the headers, done by WebRequest.ProcessHeaderString.
         1. If Authorization: Basic, decode the authentication username and password.
         2. If Content-Length, set the ContentType variable.
         3. If Content-Length, set the ContentLength variable.
   3. Check the request:
      1. If no WebResponse was created, respond with an HTTP 400 error.
      2. If no webapplication was found, response with an HTTP 404 error.
   4. If WebRequest.ContentLength is larger than 0 and it is a POST request, read the additional data.
   5. †Call WebApplication.PreQuery. If the result was true:
      1. †Call WebApplication.Query.
      2. †Call WebApplication.PostQuery.
   6. Initiate cleanup:
      1. Unset all variables.
      2. Close the connection.

† This is the part you are going to build

Creating a Web Application

---

To extend the functionality of the web server you generally create a web application. You do this by subclassing the WebApplication class and implement at least the WebApplication.Query method.

In the IpDrv you will find an example implementation called HelloWeb.

Your web application will only be used by the web server when it is registered. To register a web application you need to update the web server configuration. Edit the ??Web.ini file and update the [IpDrv.WebServer] section:

Applications[x]=MyPackage.MyWebApplication
ApplicationPaths[x]=/myweb

Where x is a free slot. You can have up to 10 web applications. Your web application will be reachable through http://ip:port/myweb.

When the web server starts it will create and initialize all web applications. The path variable will be set to the ApplicationPaths value for this web application. Implement the init() function to perform some initialization of your web application. When the web server is shut down it will call the CleanupApp() function. In that function you should nullify all actor references you created (i.e. set the variables to none. ).

As shown in the request process, the first function of the web application which is called is PreQuery(..). If this function returns true (the default behavior) it will subsequently call the Query(..) and PostQuery(..) functions. All three functions receive the same 2 arguments: a WebRequest and WebResponse instance. As the class names hint, the WebRequest instance contains information about the HTTP request which the user made. And the WebResponse instance can be used to produce a response to the request. The URI variable of the WebRequest instance will contain the relative URL which was made. The web application path has been stripped from the original URI.

Processing the request

The client's request is stored in the WebRequest instance passed to the Query functions of your WebApplication.

Probably the post important variable in WebRequest is the URI variable. It will contain the request page within the web application domain. For example, if the client made the request http://ip:port/myweb/some/page?foo=bar then the URI variable will have the value /some/page. The /myweb prefix of the web application has been removed. And request variables are stored elsewhere.

In order to retrieve request variables, for either GET or POST request, you need to use the GetVariable, GetVariableCount, GetVariableNumber, and GetVariables functions. The GetVariable function allow you to retrieve the first value for a given request available. In the previously posted example GetVariable("foo") would return bar. In some cases you have multiple request variables of the same name, for example: http://ip:port/myweb/some/page?foo=bar&quux. In this case you can use GetVariableCount("foo") to get the number values for that variable, and use GetVariableNumber("foo", X) to get each value.

You can also read the request headers through the GetHeader and GetHeaders functions. The HTTP headers Content-Type, Content-Length, Authentication are processed and stored in special variables. Content-Type and Content-Length are only used in POST request. The Authentication request header is only processed for basic authentication requests. In that case the username and password variables in the WebRequest object will contain the decoded values.

A typical method of processing the request is shown in the following example:

function Query(WebRequest Request, WebResponse Response)
{
    // Each "page" is handled by a separate function
    switch (request.URI)
    {
        case "/page1"
            doPage1(request, response);
            break;
        case "/page2"
            doPage2(request, response);
            break;
        // etc.
    }
}

function doPage1(WebRequest Request, WebResponse Response)
{
    if (request.getVariable("action") ~= "save")
    {
        // save incomming settings
    }
    // produce a response
}

Producing a response

The web server doesn't provide any default response when your web application doesn't. Only in some rare cases it will produce something useful for a client. So it is up to your web application to produce the content.

The WebResponse object is used to generate a response to the client. The most basic method of generating a response is by using the SendText method to return content. But generally you want to do a bit more or use more or less static content instead of constantly generating the response through Unrealscript.

The basic flow of generating a response through the WebResponse object is the following:

1. Define content: AddHeader, Subst, ClearSubst, LoadParsedUHTM, headers, CharSet
2. HTTPResponse
3. SendStandardHeaders
4. Send content: IncludeUHTM, SendText, IncludeBinaryFile

The first three steps are completely optional.

The WebResponse generates a few standard HTTP headers, which are send before custom headers. These are:

Server

"UnrealEngine IpDrv Web Server Build ..."

Content-Type

the value of ContentType argument provided in the SendStandardHeaders call

Cache-Control

"max-age=...", the value of ExpirationSeconds in the web server

Expires

"...", the value of ExpirationSeconds in the web server

Connection

"Close". The web server only processes one request at the time.

There are three common methods to create the response data:

1. Manual text sending
2. Sending files AS-IS
3. Processed UHTM files

Manual text sending

Using the SendText function you can set manually construct a document to response to the web browser. See the HelloWeb class for an example web application using this method.

Sending files AS-IS

The WebResponse object provides the IncludeBinaryFile function which will allow you to send a file as it is stored on disk to the client. This function is commonly used to send static files like images, see the ImageServer class.

The files you can include have to be contained within the UDK/Web directory.

Note: The IncludeBinaryFile does not make sure that the response headers are send. So you need to call SendStandardHeaders before calling IncludeBinaryFile.

Processed UHTM files

UHTM is a templating system which processing text files and replaces certain variables and directives. These files do not have to be HTML files, any text file can be used.

There are two special constructions. First there are the variables which will be substituted. You create a substituted variable by using the following construction: <%var_name%>, where var_name is the name of the variable. You can set the value for this variable by using the Subst function. It is important to note that within SGML comments no variables are substituted. So <!-- <%var_name%> --> will stay like that.

The second construction is <!-- #include file="file.inc" -->. This will include the file file.inc at that place in the current file. The included file will also be processed for additional variables and include directives.

The LoadParsedUHTM and IncludeUHTM functions will process the file. The LoadParsedUHTM function will return the processed file as a string, where as the IncludeUHTM will send the processed UHTM file to the web browser. Generally you use LoadParsedUHTM to create the value for a new substitution variable.

API

---

WebApplication

Fields

Path

This field will be set to the configured base path of the web application. Use this field to create proper links.

Functions

Init

function Init()

Called right after the WebApplication instance is created. Use this to perform some important initialization. It is advised not to perform too much stuff during this method. It might be better to delay full initialization until the first request for this web application.

CleanupApp

function CleanupApp()

Called before the web server is destroyed. In this method you should set all references to actors in your web application to none in order to prevent memory from leaking.

PreQuery

function bool PreQuery(WebRequest Request, WebResponse Response)

Called to check if this web application wants to handle the given query. If false is returned the Query and PostQuery methods will not be called.

Query

function Query(WebRequest Request, WebResponse Response)

Called after a call to PreQuery returned true.

PostQuery

function PostQuery(WebRequest Request, WebResponse Response)

Called after the call to Query returned.

WebRequest

Fields

RemoteAddr

The address of the client which produced this request,

URI

The request URL. This field has been stripped from the web application prefix when it is handled by a web application.

Username

The decoded username when the request contained a basic HTTP Authentication header. See RFC 2617 for more information about this.

Password

The decoded password when the request contained a basic HTTP Authentication header.

ContentLength

The number of bytes in the POST request.

ContentType

The content mime type for the POST data.

RequestType

Will be set to either Request_GET or Request_POST depending on the kind of request.

Functions

DecodeBase64

native final function string DecodeBase64(string Encoded)

Decode a base64 encoded string.

EncodeBase64

native final function string EncodeBase64(string Decoded)

Encode a string to base64.

AddHeader

native final function AddHeader(string HeaderName, coerce string Value)

Add a request header to the data structure. This function is used during the request processing. It is not meant to call this function from your web application.

GetHeader

native final function string GetHeader(string HeaderName, optional string DefaultValue)

Get the value of a given request header. If the header was not present in the request the default value is returned. Note that if there are multiple request headers with the same name, only the last received header is returned. The header name is case insensitive.

GetHeaders

native final function GetHeaders(out array headers)

Retrieve the names all received header.

AddVariable

native final function AddVariable(string VariableName, coerce string Value)

Register a received request variable. This function is used internally during the request processing. It is not intended to be used from a web application.

GetVariable

native final function string GetVariable(string VariableName, optional string DefaultValue)

Get the value of a request variable. If the request variable does not exist the default value is returned. The variable name is case insensitive. If multiple variables with the same name are received only one value is returned. Use GetVariableNumber and GetVariableCount to retrieve individual values.

GetVariableCount

native final function int GetVariableCount(string VariableName)

Return the number of variables for a given request variable. If the value is more than 1 you should use GetVariableNumber to retrieve each value.

GetVariableNumber

native final function string GetVariableNumber(string VariableName, int Number, optional string DefaultValue)

Retrieve a given request variable value. If the variable does not exist, or does not have that number of values the default value is returned.

GetVariables

native final function GetVariables(out array varNames)

Get the names of received request variables.

WebResponse

Fields

headers

This array contains all headers which will be send to the remote host. It contains the headers as they are send. It is allowed to modify this array directory, or by using the AddHeader function. When content is send changes to this field has not effect.

IncludePath

The relative path from the base directory where the files that the web server can send or process are located.

Connection

The WebConnection instance through which this response will be send. You should not use this variable to write data.

Functions

FileExists

native final function bool FileExists(string Filename)

Returns true when the given filename exists in the web include directory. This can be used in case of a static file server like the ImageServer class to determine if a given file exists so that the WebApplication can return a HTTP 404 Not Found error.

Subst

native final function Subst(string Variable, coerce string Value, optional bool bClear)

Modify a substitution variable which will be used during the IncludeUHTM and LoadParsedUHTM functions. The third optional argument allows you to remove a previously declared variable.

ClearSubst

native final function ClearSubst()

Remove all registered substitution variables.

IncludeUHTM

native final function bool   IncludeUHTM(string Filename)

Load a UHTM file and process it. The processed result will directly be send to the remote host. This function returns true when the file was properly included and processed. The filename provided has to live within the subtree as declared by the IncludePath field.

IncludeBinaryFile

native final function bool   IncludeBinaryFile(string Filename)

Send a file as is to the remote host. Note that unlike IncludeUHTM this function will not make sure the response headers have been send. Thus before calling this method make sure the headers are send.

LoadParsedUHTM

native final function string LoadParsedUHTM(string Filename)

Similar to the IncludeUHTM function, with the major exception that it returns the precessed file as a result, and does not send it to the remote host. You can use this to create more complex UHTM templates.

GetHTTPExpiration

native final function string GetHTTPExpiration(optional int OffsetSeconds)

Create a RFC 1123 date of now + OffsetSeconds. This is used to construct the Expires response header.

SendText

event SendText(string Text, optional bool bNoCRLF)

Send a string of text data to the remote host. If bNoCRLF is true no DOS line terminator will be added (0x13 0x10). This function ensures the response headers will be send before the text data is send.

SendBinary

event SendBinary(int Count, byte B[255])

Send binary data to the remote host. This function does not ensure that the headers are send.

HTTPResponse

function HTTPResponse(string Header)

Send a HTTP response code. By default a 200 OK response will be send, using this function you can send an alternative response. See the HTTP RFC for more details on the HTTP response codes. The HTTP response should be given in the header argument. For example:

HTTPResponse("HTTP/1.0 404 Not Found");

HTTPHeader

function HTTPHeader(string Header)

Send a response header. This will make sure a HTTP response code has been send before sending the header. You should not call this method to send headers. It is better to enqueue headers through the AddHeader function, or by directly editing the Headers field.

AddHeader

function AddHeader(string header, optional bool bReplace=true)

Add/update a header to the headers list. It will be send at the first possible occasion. To completely remove a given header simply give it an empty value, e.g. "X-Header:".

To add multiple headers with the same name (needed for the Set-Cookie header) you'll need to edit the headers array directly.

SendHeaders

function SendHeaders()

Send the enqueued headers. This function is called internally by SendStandardHeaders. You should not use this function.

HTTPError

function HTTPError(int ErrorNum, optional string Data)

A short hand method to send some common HTTP response codes like 400 Bad Request, 401 Unauthorized, 404 Not Found.

It will include an inline HTML response with a error message. You probably want to handle these kinds of responses yourself with a more user friendly response. This is just a convenience method.

SendStandardHeaders

function SendStandardHeaders( optional string ContentType, optional bool bCache )

Send the standard and all custom headers. After calling this function you can safely include the response data.

The ContentType argument is used to set the "Content-Type" response header. It defaults to "text/html".

If bCache is true it will also include some headers with caching instructions. For this it uses the value of WebServer.ExpirationSeconds.

Only the "Connection: Close" header is enforced. All other default headers will only be used if that header was not already set.

Redirect

function Redirect(string URL)

A convenience method to send a 302 Document Moved HTTP response to redirect the client to a different URL.

SentText

function bool SentText()

True if response data has been send. At this point you can no longer send headers.

SentResponse

function bool SentResponse()

True when the HTTP response code has been send. After this you can not change the response code. You can still send headers.

WebServer

This class is not meant to be subclassed, only notable elements will be described.

Fields

ServerName

This optional configuration field can be used to assign a host name to the web server. This does not influence the behavior of the server at all. It will just updated the ServerURL field with this name rather than the local IP.

Applications

The fully qualified class name of an web application to load. This variable is closely related to the ApplicationPaths field. You can define up to ten web applications.

ApplicationPaths

The path to associate with the web application. The order is significant, the first matching path will take priority.

bEnabled

This configurable field allows you to easily enable or disable the web server.

ListenPort

Defines the port for the web server to listen on. The web server will only accept this port to bind to.

MaxConnections

The maximum number of concurrent connection the web server will support. This number should not be too high because it can impact the performance of the game.

DefaultApplication

The default web application to use. This is an index in the Applications field. When no web application could be found for the requested URL the user is redirected to this web application. This field is used by the WebConnection class.

ExpirationSeconds

A configurable field to provide a generic location to define a expiration time. It is used by the WebResponse class to set the expiration time for responses than may be cached.

ServerURL

This field is populated during the startup of the webserver. It will contain the base url of the webserver.

Functions

GetApplication

function WebApplication GetApplication(string URI, out string SubURI)

This method returns a web application for the given URI. The second argument of this method will contain the URI without the prefix of the web application path.

WebConnection

This class is not meant to be subclassed, only notable elements will be described.

Fields

MaxValueLength

This configurable item defines the maximum length a request value is allowed to have. Values will be truncated to this length. It is important that this value is large enough to process the request. This value is used by the WebRequest class.

MaxLineLength

The maximum length of a single line is the request. This limits the length of the variables in the GET request (everything after the '?'). But it also limits the length of a single line within the POST data.