HTTP Transformations

Lua is a lightweight, high-level, multi-paradigm programming language designed primarily for embedded use in applications.

HTTP transformation rules that are used to control the processing of a request by IBM Application Gateway (IAG) can be easily written as a Lua script.

Detailed information on Lua scripting can be found in the Lua reference manual: Lua 5.4 Reference Manual. In addition to this, due to the fact that Lua is a widely used scripting language, there are many resources available on the Web to assist you with learning Lua. You can start learning Lua with Programming in Lua. It is available in paperback and as an e-book. The first edition is available for free at Programming in Lua (first edition).

IAG provides custom Lua modules to control the processing of the request. See Custom Lua modules.

Since Lua is a lightweight and performant scripting language, and the custom Lua modules operate on 'live' data, the impact performance of calling simple Lua transformation rules is negligible. This means that it is acceptable to invoke Lua transformation rules on every request to handle things such as the insertion of custom HTTP headers into responses.

Configuring HTTP Transformation Rules

Transformation rules can be applied to HTTP requests or responses by the application gateway. Rules themselves can be attached based on a combination of host, paths and methods. See HTTP Transformations/Request, HTTP Transformations/Postazn and HTTP Transformations/Response for complete details of the matching properties.

Different triggers are used to indicate that the transformation should take place at different stages during process:

TriggerDescription
requestThe rule is triggered as soon as the request is received by IAG.
postaznThe rule is triggered immediately after the authorization decision has been made. This option should be used if you wish to use credential or session information in your rule.
responseThe rule is triggered after the response has been received from the resource server.
policies:
  http_transformations:
    request:
      - name: insert-custom-request-identifier-header
        host: www.test.com
        paths:
          - "/reports/*"
          - "/myapp/*"
        method: GET
        rule: "@req_insert_header.lua"
    postazn:
      - name: insert-credential-attribute-header
        host: www.test.com
        paths:
          - "/reports/generate"
          - "/reports/export"
        method: POST
        rule: "@req_insert_header.lua"
    response:
      - name: apply-content-security-policy
        host: www.test.com
        paths:
          - "*"
        method: GET
        rule: "@resp_csp.lua"

📘

Legacy XSLT HTTP Transformation Rules

IAG also supports legacy XSLT HTTP Transformation Rules from IBM Security Verify Access. To configure an XSLT rule, just provide the XSL rule in place of the Lua rule. Lua transformation rules execute significantly faster than the legacy XSL engine rules and are much simpler to author. It is strongly recommended to consider rewriting any legacy XSL rules in Lua.

Installed Lua modules

LuaRocks is a package manager for Lua scripts. It allows you to create and install Lua modules as self-contained packages called rocks.

It is not possible to install additional rocks into the container, but several rocks have been preinstalled. The following table lists the preinstalled rocks that are available to the transformation rule scripting:

RockDescription
basexxA Lua library which provides base2(bitfield), base16(hex), base32(crockford/rfc), base64(rfc/url), base85(z85) decoding and encoding.
binaryheapBinary heaps are an efficient sorting algorithm. This module implements a plain binary heap (without reverse lookup) and a 'unique' binary heap (with unique payloads and reverse lookup).
cqueuescqueues is a type of event loop for Lua, except it's not a classic event loop. It does not use callbacks—neither as part of the API nor internally—but instead you communicate with an event controller by the yielding and resumption of Lua coroutines using objects that adhere to a simple interface.
fifoA Lua library/'class' that implements a FIFO. Objects in the FIFO can be of any type, including nil.
httpA HTTP library for Lua.
lpegLPeg is a new pattern-matching library for Lua, based on Parsing Expression Grammars (PEGs). The nice thing about PEGs is that it has a formal basis (instead of being an ad-hoc set of features), allows an efficient and simple implementation, and does most things we expect from a pattern-matching library (and more, as we can define entire grammars).
lpeg_patternsA collection of LPEG patterns.
luaosslluaossl is a comprehensive binding to OpenSSL for Lua. It includes support for certificate and key management, key generation, signature verification, and deep bindings to the distinguished name, alternative name, and X.509v3 extension interfaces. It also binds OpenSSL's bignum, message digest, HMAC, cipher, and CSPRNG interfaces.
luasocketLuaSocket is a Lua extension library that provides support for the TCP and UDP transport layers, commonly needed by applications that deal with the Internet.
lua-cjsonLua CJSON provides fast UTF-8 JSON parsing/encoding support for Lua.
redis-luaA Lua client library for the redis key value storage system.
urlencodeA URL encoder/decoder with native extension.

Custom Lua modules

IAG provides a number of custom modules which can be used to control the processing of the request.

ModuleDescription
HTTPRequestThis module allows you to manage the HTTP request.
HTTPResponseThis module allows you to manage the HTTP response.
SessionThis module allows you to access details of the current user session. The user session information will be available for response triggers and will not be available for request triggers.
ControlThis module allows you to control the processing of the request.

HTTPResponse

This module allows you to retrieve details of the current HTTP response, and modify the current HTTP response.

Header Access

boolean containsHeader(string name)

Used to determine whether the specified HTTP header exists in the response.

Parameters
NameDescription
name The name of the HTTP header.
Return value

true if the header exists, otherwise false.

Example
if HTTPResponse.containsHeader("my-header") then
  print(HTTPResponse.getHeader("my-header"))
end

table getHeaderNames()

Used to retrieve the header names from the response.

Return value

A table of HTTP header names.

Example
table = HTTPResponse.getHeaderNames()

string getHeader(string name)

Used to retrieve the specified header from the response.

Parameters
NameDescription
name The name of the HTTP header.
Return value

The value of the HTTP header, or \a nil if the header was not found.

Example
print(HTTPResponse.getHeader("my-header"))

nil setHeader(string name, string value)

Used to set the specified header in the response.

Parameters
NameDescription
name The name of the HTTP header.
value The value of the HTTP header.
Return value

nil

Example
HTTPResponse.setHeader("my-header", "my-header-value")

nil removeHeader(string name)

Used to remove the specified header from the response.

Parameters
NameDescription
name The name of the HTTP header.
Return value

nil

Example
HTTPResponse.removeHeader("my-header")

nil clearHeaders()

Used to remove all headers from the response.

Return value

nil

Example
HTTPResponse.clearHeaders()

Cookie Access

boolean containsCookie(string name)

Used to determine whether the specified cookie exists in the response.

Parameters
NameDescription
name The name of the cookie.
Return value

true if the cookie exists, otherwise false.

Example
if HTTPResponse.containsCookie("my-cookie") then
  print(HTTPResponse.getCookie("my-cookie"))
end

table getCookieNames()

Used to retrieve the cookie names from the response.

Return value

A table of HTTP cookie names.

Example
table = HTTPResponse.getCookieNames()

string getCookie(string name)

Used to retrieve the specified cookie from the response.

Parameters
NameDescription
name The name of the cookie.
Return value

The value of the cookie, or \a nil if the cookie was not found.

Example
print(HTTPResponse.getCookie("my-cookie"))

nil setCookie(string name, string value)

Used to set the specified cookie in the response.

Parameters
NameDescription
name The name of the cookie.
value The value of the cookie.
Return value

nil

Example
HTTPResponse.setCookie("my-cookie", "cookie-value; Secure; HttpOnly")

nil removeCookie(string name)

Used to remove the specified cookie from the response.

Parameters
NameDescription
name The name of the cookie.
Return value

nil

Example
HTTPResponse.removeCookie("my-cookie")

nil clearCookies()

Used to remove all cookies from the response.

Return value

nil

Example
HTTPResponse.clearCookies()

Body Access

string getBody()

Used to retrieve the body of the response. You will not be able to use this function to retrieve the body if the body is in binary format. The maximum size of the body which is returned is controlled by the '[server] request-body-max-read' WebSEAL configuration entry.

Return value

The body, or \a nil if the body does not exist.

Example
print(HTTPResponse.getBody())

nil setBody(String body)

Used to replace the body of the response.

Parameters
NameDescription
body The new response body.
Return value

nil

Example
HTTPResponse.setBody("<html><body><h1>Hello World!</h1></body></html>")

number getContentLength()

Used to retrieve the content length of the body.

Return value

The body content length, or \a nil if the content length is unknown (e.g. for a chunked response).

Example
print(HTTPResponse.getContentLength())

Response Line Access

string getVersion()

Used to retrieve the HTTP version from the response.

Return value

The HTTP version of the response.

Example
print(HTTPResponse.getVersion())

number getStatusCode()

Used to retrieve the status code of the response.

Return value

The status code of the response.

Example
print(HTTPResponse.getStatusCode())

nil setStatusCode(number code)

Used to set the status code of the response.

Parameters
NameDescription
code The new status code for the response.
Return value

nil

Example
HTTPResponse.setStatusCode(200)

string getStatusMsg()

Used to retrieve the status message from the response.

Return value

The status message of the response.

Example
print(HTTPResponse.getStatusMsg())

nil setStatusMsg(string msg)

Used to set the status message for the response.

Parameters
NameDescription
msg The new status message for the response.
Return value

nil

Example
HTTPResponse.setStatusMsg("OK")

Session

This module allows you to access details of the current user session. The user session information will only be available for postazn and response triggers, and will not be available for request triggers. Used to retrieve the session identifier associated with the current user session. The 'user-session-ids' configuration entry, within the '[session]' stanza must be set to 'yes' in order for the user session identifier to be available.

string getSessionId()

Return value

The session identifier.

Example
print(Session.getSessionId())

string getUsername()

Used to retrieve the name of the user associated with the current session.

Return value

The name of the user.

Example
print(Session.getUsername())

boolean containsCredentialAttribute(string name)

Used to determine whether the specified credential attribute exists in the session.

Parameters
NameDescription
name The name of the credential attribute.
Return value

true if the credential attribute exists, otherwise false.

Example
if Session.containsCredentialAttribute("AZN_CRED_PRINCIPAL_NAME") then
  print(Session.getCredentialAttribute("AZN_CRED_PRINCIPAL_NAME"))
end

table getCredentialAttributeNames()

Used to retrieve the attribute names from the credential.

Return value

A table of attribute names.

Example
table = Session.getCredentialAttributeNames()

string getCredentialAttribute(string name)

Used to retrieve the specified attribute from the credential. If the attribute is multi-valued each attribute value will be returned in the string, separated by a comma.

Parameters
NameDescription
name The name of the credential attribute.
Return value

The value of the credential attribute, or \a nil if the credential attribute was not located.

Example
print(Session.getCredentialAttribute("AZN_CRED_PRINCIPAL_NAME"))

HTTPRequest

This module allows you to retrieve details of the current HTTP request, and modify the current HTTP request. The request information, with the exception of the body, is available on all triggers, but the request can only be modified on a request or postazn trigger.

Header Access

boolean containsHeader(string name)

Used to determine whether the specified HTTP header exists in the request.

Parameters
NameDescription
name The name of the HTTP header.
Return value

true if the header exists, otherwise false.

Example
if HTTPRequest.containsHeader("my-header") then
  print(HTTPRequest.getHeader("my-header"))
end

table getHeaderNames()

Used to retrieve the header names from the request.

Return value

A table of HTTP header names.

Example
table = HTTPRequest.getHeaderNames()

string getHeader(string name)

Used to retrieve the specified header from the request.

Parameters
NameDescription
name The name of the HTTP header.
Return value

The value of the HTTP header, or \a nil if the header was not found.

Example
print(HTTPRequest.getHeader("my-header"))

nil setHeader(string name, string value)

Used to set the specified header in the request.

Parameters
NameDescription
name The name of the HTTP header.
value The value of the HTTP header.
Return value

nil

Example
HTTPRequest.setHeader("my-header", "my-header-value")

nil removeHeader(string name)

Used to remove the specified header from the request.

Parameters
NameDescription
name The name of the HTTP header.
Return value

nil

Example
HTTPRequest.removeHeader("my-header")

nil clearHeaders()

Used to remove all headers from the request.

Return value

nil

Example
HTTPRequest.clearHeaders()

Cookie Access

boolean containsCookie(string name)

Used to determine whether the specified cookie exists in the request.

Parameters
NameDescription
name The name of the cookie.
Return value

true if the cookie exists, otherwise false.

Example
if HTTPRequest.containsCookie("my-cookie") then
  print(HTTPRequest.getCookie("my-cookie"))
end

table getCookieNames()

Used to retrieve the cookie names from the request.

Return value

A table of HTTP cookie names.

Example
table = HTTPRequest.getCookieNames()

string getCookie(string name)

Used to retrieve the specified cookie value from the request.

Parameters
NameDescription
name The name of the cookie.
Return value

The value of the cookie, or \a nil if the cookie was not found.

Example
print(HTTPRequest.getCookie("my-cookie"))

nil setCookie(string name, string value)

Used to set the specified cookie value in the request.

Parameters
NameDescription
name The name of the cookie.
value The value of the cookie.
Return value

nil

Example
HTTPRequest.setCookie("my-cookie", "my-cookie-value")

nil removeCookie(string name)

Used to remove the specified cookie from the request.

Parameters
NameDescription
name The name of the cookie.
Return value

nil

Example
HTTPRequest.removeCookie("my-cookie")

nil clearCookies()

Used to remove all cookies from the request.

Return value

nil

Example
HTTPRequest.clearCookies()

Body Access

string getBody()

Used to retrieve the body of the request. You will not be able to use this function to retrieve the body if the body is in binary format. The maximum size of the body which is returned is controlled by the '[server] request-body-max-read' WebSEAL configuration entry.

Return value

The body, or \a nil if the body does not exist.

Example
print(HTTPRequest.getBody())

nil setBody(String body)

Used to replace the body of the request.

Parameters
NameDescription
body The new request body.
Return value

nil

Example
HTTPRequest.setBody("field1=value1&field2=value2")

number getContentLength()

Used to retrieve the content length of the body.

Return value

The body content length, or \a nil if the content length is unknown (e.g. for a chunked request).

Example
print(HTTPRequest.getContentLength())

Request Line Access

string getMethod()

Used to retrieve the method from the request line.

Return value

The HTTP method of the request.

Example
print(HTTPRequest.getMethod())

nil setMethod(string method)

Used to set the method for the request.

Parameters
NameDescription
method The new method for the request.
Return value

nil

Example
HTTPRequest.setMethod("GET")

string getURL()

Used to retrieve the URL from the request line.

Return value

The HTTP URL of the request.

Example
print(HTTPRequest.getURL())

nil setURL(string url)

Used to set the URL for the request.

Parameters
NameDescription
url The new URL for the request.
Return value

nil

Example
HTTPRequest.setURL("/index.html?arg=value")

string getVersion()

Used to retrieve the HTTP version from the request line.

Return value

The HTTP version of the request.

Example
print(HTTPRequest.getVersion())

number getProtocol()

Used to retrieve the HTTP protocol used in the request.

Return value

The protocol used in the request, 0 for HTTP and 1 for HTTPS.

Example
print(HTTPRequest.getProtocol())

Control

This module allows you to control the processing of the request by WebSEAL.

nil includeInRequestLog(boolean includeInLog)

Used to control whether the request should be included in the request log or not.

Parameters
NameDescription
includeInLog Should the request be included in the log?
Return value

nil

Example
Control.includeInRequestLog(false)

nil trace(number level, string message)

Used to trace a string using the WebSEAL tracing framework. Use the pdweb.http.transformation tracing component to activate the tracing.

Parameters
NameDescription
level The trace level
message The string which is to be traced.
Return value

nil

Example
Control.trace(5, string.format("value: %s", value))

string dumpContext()

Used to return a YAML representation of the request context. This includes information associated with the HTTP request, current HTTP response and session. The resultant YAML can be used as input into the isva_lua_transformation_test binary.

Return value

A YAML representation of the request context.

Example
print(Control.dumpContext())

nil returnErrorPage(string message)

Used to return an error to the client. As a result of calling this function an error page will be generated and returned by WebSEAL. The supplied message can be included in the WebSEAL generated error page using the %FAILREASON% macro.

Parameters
NameDescription
message The error message.
Return value

nil

Example
Control.returnErrorPage(string.format("An error occurred: %s", reason))

nil responseGenerated(boolean generated)

Used to indicate that a response has been created by the transformation rule. This will ensure that the response is returned directly to the client and WebSEAL will not forward the request onto the junctioned application.

Parameters
NameDescription
generated Has a response been generated by the rule?
Return value

nil

Example
Control.responseGenerated(true)

Disabled APIs

To help protect the integrity of the gateway the following functions from the Lua standard libraries have been disabled and are not available to HTTP transformation scripts.

Standard LibraryFunction
Basic Libraryloadfile
Operating System Facilitiesexecute
exit
remove
setlocale
tmpname
Input and Outputall functions
Package Libraryloadlib

Debugging

There is a command line tool which allows Lua transformation rules to be developed and tested offline, iag_lua_transformation_test. This command is available in the IAG container image.

The input to this command is a YAML representation of the context and the Lua transformation rule script. The context of the request includes the HTTP request, HTTP response and session information.

The output from the binary is a description of the transformation actions which would be performed by IAG.

Example Usage of the iag_lua_transformation_test Tool

The following scenario tests a simple transformation rule using the command line tool.

The example YAML representation of the context, named context.yaml:

http:
  request:
    method: "GET"
    uri:    "/test.html"
    headers:
      - name:  "User-Agent"
        value: "curl"
      - name:  "Host"
        value: "www.ibm.com"
  response:
    status: 201
    headers:
      - name:  "Origin"
        value: "https://www.ibm.com"
session:
    id:       "<Session ID>"
    username: "testuser"
    credential:
        attributes:
          - name:  "AZN_CRED_PRINCIPAL_NAME"
            value: "testuser"

The example transformation rule, named rule.lua:

if Session.containsCredentialAttribute("AZN_CRED_PRINCIPAL_NAME") then
    HTTPRequest.setHeader("IV-USER", Session.getCredentialAttribute("AZN_CRED_PRINCIPAL_NAME"))
end

if HTTPResponse.containsHeader("origin") then
  if not (HTTPResponse.containsHeader("access-control-allow-origin")) then
    HTTPResponse.setHeader("access-control-allow-origin", HTTPResponse.getHeader("origin"))
  end

  if not (HTTPResponse.containsHeader("access-control-allow-methods")) then
    HTTPResponse.setHeader("access-control-allow-methods", "GET,POST,OPTIONS")
  end

  if not (HTTPResponse.containsHeader("access-control-max-age")) then
    HTTPResponse.setHeader("access-control-max-age", "86400")
  end
end

In this docker example, the context and rule are present in the current directory, which is mounted into a temporary container at /lua_test. Passing in the context and rule produces output which describes the changes IAG would make when applying this rule in this context:

~/test$ ls
context.yaml  rule.lua
~/test$ docker run --rm -v "${PWD}":/lua_test \
  --entrypoint iag_lua_transformation_test \
  icr.io/ibmappgateway/ibm-application-gateway:22.07 \
  -c /lua_test/context.yaml -s /lua_test/rule.lua
Request header: set (IV-USER,testuser)
Response header: set (access-control-allow-origin,https://www.ibm.com)
Response header: set (access-control-allow-methods,GET,POST,OPTIONS)
Response header: set (access-control-max-age,86400)

Generating Sample Context YAML

To assist with the generation of the YAML representation of the context of the request, the iag_lua_transformation_test command can be started with the -t option to produce a template context YAML file. For example:

docker run --rm \
  --entrypoint iag_lua_transformation_test \
  icr.io/ibmappgateway/ibm-application-gateway:22.07 \
  -t

The produced template YAML context:

http:
  request:
    method:   <method>
    url:      <url>
    version:  <version>
    protocol: <protocol>
    body:     <body>
    headers:
      - name:  <hdr-name>
        value: <hdr-value>
    cookies:
      - name:  <cookie-name>
        value: <cookie-value>
  response:
    version:  <version>
    body:     <body>
    status:
      code:    <code>
      message: <message>
    headers:
      - name:  <hdr-name>
        value: <hdr-value>
    cookies:
      - name:  <cookie-name>
        value: <cookie-value>
session:
  id:       <id>
  username: <username>
  credential:
    attributes:
      - name:  <attribute-name>
        value: <attribute-value>

Capturing Context YAML

The dumpContext function within the Control Lua module can be called from within a running IAG instance to capture a YAML representation of the current request context. For example:

print(Control.dumpContext())

Tracing

Print and trace statements can be used in in Lua transformation rules scripts.

Printing to Container Standard Out

The output from any print statements will be sent to the container standard out. For example:

print(string.format("Host header: %s",
                    HTTPRequest.getHeader("host")))

Tracing Using the Tracing Framework

The Control custom module contains a trace function which can be used to trace a string using the tracing framework. Use the pdweb.http.transformation tracing component to activate the tracing.

For example:

  1. Configure pdweb.http.transformation in the IAG YAML:
  2. Use the Control.trace function to write to the tracing framework.
logging:
  tracing:
    - file_name: /var/iag/lua_trace.log
      component: pdweb.http.transformation
      level: 5
Control.trace(5, string.format("Host header: %s",
                   HTTPRequest.getHeader("host")))

Example Transformation Rules

Modifying the Body of a Response

-- This script is used to test the modification of a JSON based response.  It
-- will change the 'modified' field in the JSON payload from false to
-- true.

local cjson = require "cjson"

-- Only modify the response if the content-type is 'application/json'
if HTTPResponse.getHeader("content-type") == "application/json" then
    local payload = cjson.decode(HTTPResponse.getBody())

    payload["modified"] = true

    HTTPResponse.setBody(cjson.encode(payload))
end

Creating a Custom Response

-- This script is used to create a custom HTTP response.

url = HTTPRequest.getURL()

-- Only send our custom response if the URL contains the string:
--  'custom.response'

if string.find(url, "custom.response") then
    HTTPResponse.setHeader("content-type", "text/html")
    HTTPResponse.setCookie("my-cookie", "cookie-value; Secure; HttpOnly")
    HTTPResponse.setBody("<html><body><h1>This is an example transformation response!</h1></body></html>")
    HTTPResponse.setStatusCode(200)
    HTTPResponse.setStatusMsg("OK")

    Control.responseGenerated(true)
end

Setting the Content Security Policy

-- This script is used to set the CSP header based on the value of the
-- incoming host header.

host = HTTPRequest.getHeader("host")

-- Validate the host header.
local validHosts = {
  "www.ibm.com",
  "gateway.ibm.com"
}

local isValid = false

for index, data in pairs(validHosts) do
    if data == host then
        isValid = true
        break
    end
end

-- If the host header is accepted, we add the CSP header to the response,
-- otherwise we return an error page.
if isValid then
    HTTPResponse.setHeader("Content-Security-Policy", 
                                string.format("script-src %s;", host))
else
    Control.returnErrorPage(
        string.format("An invalid host header was received: %s", host))
end