Build Your Own Access Gateway

Roll out your own access gateway on top of OpenResty (Nginx) to enable seamless on-boarding for non-Siteminder applications—with no rewriting.

 min read

In the previous post, we discussed how containerization smooths the transition of legacy identity systems, allowing us to easily and rapidly build environments that can run seamlessly on Amazon EC2, Google Cloud, Azure, Virtualbox, and others. We showcased how that works specifically, with CA Siteminder. Moving beyond that, in this post we'll explore a no-code solution to onboarding non-Siteminder applications.

Standards-based Cloud SSO can help, but...

When we "lift-and-shift" a legacy identity system, the cloud environment will host new applications and services that were not part of the previous setting. Enabling single sign-on (SSO) for the more modern web applications that feature either SAML or OpenID Connect (OIDC) should be pretty straightforward, since Siteminder plays nice with both of those standards.

In this scenario, Siteminder will play the role of the identity provider (IdP) and the application will take the role of service provider (SP). This way, we can avoid the effort of application rewriting, while extending SSO to any new application. That means the user doesn't have to reenter their credentials when attempting to access any of the new applications or services. We've gained two enterprise advantages: decreased effort and improved user experience.

When this falls short

By default, the way Siteminder secures applications is to front them with an Access Gateway, also known as a Secure Proxy Server (SPS). In a nutshell, the Access Gateway intercepts requests, authenticating and authorizing users before they are granted access to the target application. The application is not aware of Siteminder, since it relies on standard HTTP headers employed by the Access Gateway to identify the user, as well as several other claims.

Ideally, this should serve as a workaround, meant to minimize disruption while addressing longer term migration tasks. The tradeoff is that the technical debt—as far as IAM modernization is concerned—will increase, since the legacy you're trying to migrate away from is now doing more than before. Additionally, as part of the new ecosystem, there are very likely applications which do not support standards-based SSO (so, no SAML or OIDC) or that are not web-based (for instance, Windows applications performing Active Directory authentication). Finally, the cloud-based legacy environment will increase operational costs due to maintenance.

Risks incurred might include:

* Increased technical debt
* Loss of support for standards-based SSO
* Increased operational expense

How can we keep disruption to the minimum while decreasing the technical debt of our IAM deployment?

Taking ownership of the "identity glue"

It's worth noting that the Access Gateway is essentially a reverse proxy built around an Apache web server, configured with a set of Siteminder-specific modules. The inner core is really within those plugins, which take care of doing authentication and authorization on behalf of the downstream applications.

In the past few years the Nginx web server has been steadily gaining in popularity due to its lightweight nature and compatibility with Microservices architecture. So, Nginx is likely to be part of your cloud architecture. Given that it is functionally equivalent and shines in terms of extensibility, why not implement our own access gateway on top of it?

The good news is that you can extend Nginx down to the core without the added burden of writing a plugin. You can simply implement what you wish, using plain and simple Lua scripts. The name for this Nginx and Lua—more precisely LuaJIT—combination is called OpenResty. It's open source and has a huge community.

Reference Siteminder Architecture with Custom Access Gateway

But, how is Nginx expected to talk to Siteminder (more specifically, the Policy Server)?
FFI is your friend here. Per Wikipedia:

A foreign function interface (FFI) is a mechanism by which a program written in one programming language can call routines or make use of services written in another.

In layman's terms, what this means for our scenario is this: We can invoke the Siteminder Agent SDK from LuaJIT, allowing us to impersonate an upstream Siteminder Agent—protecting downstream applications, just as the access gateway did. To enforce access gateway proxy rules, we can just take the programmatic route: write the corresponding Lua if-then-else conditions.

Goodbye, Access Gateway, XML files, ProxyUI and friends: now it's just the OpenResty-based reverse proxy and the Policy Server. Applications will not even notice the change, since the usual contract—namely through HTTP headers—is honored.

Last but not least, once we’ve got Lua interfacing with Siteminder, we can re-use the same implementation to secure not only web applications, but also headless ones. No need to re-implement everything in Perl.

Case Study: Generate a valid Siteminder SSO token

Set up your development environment

In order to create a portable environment, we recommend using a container-based approach. In this case, we’ll use Docker.
Here's an example dockerfile which defines an image containing a Siteminder development environment:

FROM openresty/openresty:centos

ENV \ \
    BASE_DIR=/opt/CA/siteminder \


# Creation of User, Directories and Installation of OS packages
# ----------------------------------------------------------------
RUN dnf config-manager --set-enabled powertools
RUN yum install -y which unzip rng-tools java-1.8.0-openjdk-1: \
    ksh openldap-clients openssh-server xauth libnsl gcc gcc-c++ openmotif
RUN groupadd smuser && \
    useradd smuser -g smuser
RUN mkdir -p ${BASE_DIR} && \
    chmod a+xr ${BASE_DIR} && \ 
    chown smuser:smuser ${BASE_DIR} 
RUN mkdir -p ${INSTALL_TEMP} && \
    chmod a+xr ${INSTALL_TEMP} && chown smuser:smuser ${INSTALL_TEMP} 

# Increase entropy
# ----------------
RUN mv /dev/random /dev/ && \
    ln -s /dev/urandom /dev/random

# Copy packages and scripts
# -------------------------
COPY --chown=smuser:smuser install/* ${INSTALL_TEMP}/
COPY --chown=smuser:smuser ${INSTALL_TEMP}/
COPY --chown=smuser:smuser ${INSTALL_TEMP}/

# Install Policy Server
# -------------------------
    chmod +x ${INSTALL_TEMP}/ca-ps-12.8-sp05-linux-x86-64.bin && \
    ${INSTALL_TEMP}/ca-ps-12.8-sp05-linux-x86-64.bin -i silent -f ${INSTALL_TEMP}/

RUN echo ". /opt/CA/siteminder/ca_ps_env.ksh" >> /home/smuser/.bash_profile

# Install the SDK
# -----------------------------------------------
    chmod +x ${INSTALL_TEMP}/ca-sdk-12.8-sp05-linux-x86-64.bin && \
    ${INSTALL_TEMP}/ca-sdk-12.8-sp05-linux-x86-64.bin -i silent -f ${INSTALL_TEMP}/

USER smuser

# Define default command to start bash.
ENTRYPOINT ["/bin/bash"]

The generated image will be based on the OpenResty CentOS one instead of the stock image, so we only need to bundle the Siteminder-specific packages into the image. Note: We're installing the Policy Server package, but we're not going ahead with the setup phase. This is because we only need the dynamic libraries it provides, as well as the tooling, to connect with an external and functioning Policy Server.

Create the image using the following command:

$ docker build -t mysmdev .

Then run it using:

$ docker run -t -i mysmdev

In order to be able to connect to the Policy Server, we need to generate a SmHost.conf descriptor and reference it from the Lua script.

Bundle the LuaJIT FFI stubs for the Agent API

Let's first define our FFI types for the Agent API:

local _M = {
  -- Function return codes
  SM_AGENTAPI_NOCONNECTION                = -3,
  SM_AGENTAPI_TIMEOUT                     = -2,
  SM_AGENTAPI_FAILURE                     = -1,
  SM_AGENTAPI_SUCCESS                     = 0,
  SM_AGENTAPI_YES                         = 1,
  SM_AGENTAPI_NO                          = 2,
  SM_AGENTAPI_CHALLENGE                   = 3,
  SM_AGENTAPI_UNRESOLVED                  = 4,
  SM_AGENTAPIINIT_FAILURE                 = 6,
  SM_AGENTAPI_ERR_HCO_NOT_OURS            = 9,
  SM_AGENTAPI_YES_DLP                     = 11,
  SM_AGENTAPI_INVALID_ARGS                = 12,
  -- Attributes
  SM_AGENTAPI_ATTR_AUTH_DIR_OID           = 151,
  SM_AGENTAPI_ATTR_USERMSG                = 216,
  SM_AGENTAPI_ATTR_USERDN                 = 218,
  -- Well-known attributes used by the Single Sign-On APIs
  -- not previously defined above                 
  SM_AGENTAPI_ATTR_DEVICENAME             = 200,
  SM_AGENTAPI_ATTR_SESSIONID              = 205,
  SM_AGENTAPI_ATTR_CLIENTIP               = 208,
  SM_AGENTAPI_ATTR_USERNAME               = 210,
  SM_AGENTAPI_ATTR_SSOZONE                = 228,
  -- Well-known attributes used by the Authentication Chain 
  -- not previously defined above                 

local mt = {__index = _M}

ffi.cdef [[
    typedef struct Sm_AgentApi_Server_s
        char    lpszIpAddr[256];       
        long    nConnMin;                                
        long    nConnMax;                                
        long    nConnStep;                               
        long    nTimeout;                                 
        long    nPort[3];                                 
        void*   pHandle[3];                               
        long    nClusterSeq;                              
    } Sm_AgentApi_Server_t;

    typedef struct Sm_AgentApi_Init_s
      long    nVersion;                                 
      char    lpszHostName[256];      
      char    lpszSharedSecret[256];  
      long    nFailover;                               
      long    nNumServers;                              
      Sm_AgentApi_Server_t*    pServers;               
    } Sm_AgentApi_Init_t;

    typedef struct Sm_AgentApi_Attribute_s
      long    nAttributeId;
      long    nAttributeTTL;
      long    nAttributeFlags;
      char    lpszAttributeOid[64];
      long    nAttributeLen;
      char*   lpszAttributeValue;
    } Sm_AgentApi_Attribute_t;

    typedef struct Sm_AgentApi_Session_s
      int32_t nReason;
      int32_t nIdleTimeout;
      int32_t nMaxTimeout;
      int32_t nCurrentServerTime;
      int32_t nSessionStartTime;
      int32_t nSessionLastTime;
      char  lpszSessionId[64];
      char  lpszSessionSpec[2048];
    } Sm_AgentApi_Session_t;

    typedef struct Sm_AgentApi_ResourceContext_s
      char    lpszAgent[256];
      char    lpszServer[256];
      char    lpszAction[256];
      char    lpszResource[8192];
    } Sm_AgentApi_ResourceContext_t;

    typedef struct Sm_AgentApi_Realm_s
      char    lpszDomainOid[64];
      char    lpszRealmOid[64];
      char    lpszRealmName[256];
      long    nRealmCredentials;
      char    lpszFormLocation[8192];
    } Sm_AgentApi_Realm_t;

    typedef struct Sm_AgentApi_UserCredentials_s
      int32_t nChallengeReason;
      char    lpszUsername[256];
      char    lpszPassword[4096];
      char    lpszCertUserDN[1024];
      char    lpszCertIssuerDN[1024];
      int32_t nCertBinaryLen;
      char*    lpszCertBinary;
    } Sm_AgentApi_UserCredentials_t;

    int Sm_AgentApi_GetConfig (Sm_AgentApi_Init_t* pInit, const char *lpszAgentName, const char *lpszPath);
    int Sm_AgentApi_Init (const Sm_AgentApi_Init_t* pInitStruct, void** ppHandle);
    int Sm_AgentApi_SetDefaultAgentId(const char *pszAgentIdentity, void* pHandle);
    int Sm_AgentApi_IsProtected (
      const void* pHandle,
      const char* lpszClientIpAddr,
      const Sm_AgentApi_ResourceContext_t* pResourceContext,
      Sm_AgentApi_Realm_t* pRealm);
    int Sm_AgentApi_Authorize (
      const void* pHandle,
      const char* lpszClientIpAddr,                                /* optional */
      const char* lpszTransactionId,                               /* optional */
      const Sm_AgentApi_ResourceContext_t* pResourceContext,
      const Sm_AgentApi_Realm_t* pRealm,
      Sm_AgentApi_Session_t* pSession,
      long* pNumAttributes,
      Sm_AgentApi_Attribute_t** ppAttributes);

    int Sm_AgentApi_CreateSSOToken (const void* pHandle, Sm_AgentApi_Session_t* pSession, long nNumAttributes,
      Sm_AgentApi_Attribute_t* pTokenAttributes, long* pNumSSOTokenLength , char* lpszSSOToken); 
    int Sm_AgentApi_Login (
        const void* pHandle,
        const char* lpszClientIpAddr,                                /* optional */
        const Sm_AgentApi_ResourceContext_t* pResourceContext,
        const Sm_AgentApi_Realm_t* pRealm,
        const Sm_AgentApi_UserCredentials_t* pUserCredentials,
        Sm_AgentApi_Session_t* pSession,
        long* pNumAttributes,
        Sm_AgentApi_Attribute_t** ppAttributes);
    void Sm_AgentApi_FreeAttributes (const long nNumAttributes, const Sm_AgentApi_Attribute_t* pAttributes);
    int Sm_AgentApi_DecodeSSOToken(
        const void* pHandle,
        const char* lpszSSOToken,
        long* nTokenVersion,
        long* pThirdPartyToken,
        long* pNumAttributes,
        Sm_AgentApi_Attribute_t** ppTokenAttributes,
        long nUpdateToken,
        long* pNumUpdatedSSOTokenLength,
        char* lpszUpdatedSSOToken);

Although these resemble declarations typically found in 'C' header files, in this context they are used for Lua scripts, to be able to define the shape of data to be exchanged with the Agent dynamic library.

Next, we declare agent functions that operate on these types:

function _M.getconfig(self, smhostpath)
    local agentapi ="Sm_AgentApi_Init_t")
    smagentapilib.Sm_AgentApi_GetConfig(agentapi, nil, smhostpath)
    return agentapi

function _M.init(agentapi, phandle)
    return smagentapilib.Sm_AgentApi_Init(agentapi, phandle)

function _M.boot(smhostpath, agentname)
    local pSmApiHandle ="void*[1]")
    local agentapi = _M.getconfig(smhostpath)
    _M.init(agentapi, pSmApiHandle)
    if smagentapilib.Sm_AgentApi_SetDefaultAgentId(agentname, pSmApiHandle[0]) ==
        _M.SM_AGENTAPI_FAILURE then return -1 end
    return pSmApiHandle[0]

function _M.is_protected(file, method, pSmApiHandle)
    local resourceContext ="Sm_AgentApi_ResourceContext_t")
    local realm ="Sm_AgentApi_Realm_t")

    ffi.copy(resourceContext.lpszResource, file)
    ffi.copy(resourceContext.lpszAction, method)
    ffi.copy(resourceContext.lpszServer, "extapp")

    local res = smagentapilib.Sm_AgentApi_IsProtected(pSmApiHandle, "",
                                                      resourceContext, realm)
    return res, resourceContext, realm

function _M.login(loginName, password, resourceContext, realm, pSmApiHandle)
    local userCreds ="Sm_AgentApi_UserCredentials_t")
    local session ="Sm_AgentApi_Session_t")
    local iNumAttributes ="long[1]", 0)
    local pAttributes ="Sm_AgentApi_Attribute_t*[1]")

    ffi.copy(userCreds.lpszUsername, loginName)
    ffi.copy(userCreds.lpszPassword, password)

    local res = smagentapilib.Sm_AgentApi_Login(pSmApiHandle, "",
                                                resourceContext, realm,
                                                userCreds, session,
                                                iNumAttributes, pAttributes)
    return res, session, iNumAttributes, pAttributes

function _M.sso(pSmApiHandle, pszUserDN, session, numAttributes)
    if pSmApiHandle == nil then return -1 end

    local pAttr ="Sm_AgentApi_Attribute_t[3]")
    local pszIPAddress = ""
    local pszSSOToken ="char[2048]")
    local nTlen ="long[1]", 2048)
    local result = 0
    local attrCount ="long", 1)
    ffi.fill(pszSSOToken, ffi.sizeof("char[2048]"))

    pAttr[0].nAttributeId = _M.SM_AGENTAPI_ATTR_USERDN
    pAttr[0].lpszAttributeValue ="char[?]", string.len(pszUserDN) + 1) -- new char[strlen(pszUserDN)+1];
    ffi.copy(pAttr[0].lpszAttributeValue, pszUserDN)

    local res = smagentapilib.Sm_AgentApi_CreateSSOToken(pSmApiHandle, session,
                                                         numAttributes, pAttr,
                                                         nTlen, pszSSOToken)
    return res, ffi.string(pszSSOToken)

function _M.decode_sso_token(pSmApiHandle, ssoToken)
    local nTokenVer ="long[1]", 0)
    local nNumDecAttr ="long[1]", 0)
    local nNumThdParty ="long[1]", 0)
    local nNumUpdateToken = 1
    local nNumUpdateTokenLength ="long[1]", 2048)
    local pszUpdatedSSOToken ="char[2048]")
    local pTokenAttributes ="Sm_AgentApi_Attribute_t*[1]")

    if pSmApiHandle == nil then return -1 end

    local result = smagentapilib.Sm_AgentApi_DecodeSSOToken(pSmApiHandle,
                                                            ssoToken, nTokenVer,

    local tokenAttrs = {}                                                        
    for i = 1, tonumber(nNumDecAttr[0]) do
        local pTemp = pTokenAttributes[0][i - 1]
        tokenAttrs[tonumber(pTemp.nAttributeId)] = tostring(ffi.string(pTemp.lpszAttributeValue))

    smagentapilib.Sm_AgentApi_FreeAttributes(nNumDecAttr[0], pTokenAttributes[0]);

    return result, nNumDecAttr, tokenAttrs, ffi.string(pszUpdatedSSOToken)

return _M

As with the previously declared types, these also map one-to-one to the Agent library functions.

Invoke the Agent from OpenResty

One interesting feature of OpenResty is that the code can be run in a headless fashion, either from the command line or your test suite. This allows for a quick turnaround when implementing behavior that doesn't require any rendering. Once you've fleshed out your library you can then consume as-is from your UI.

local agent = require "resty.siteminder.agent"

local smhost = arg[1]
local agentname = arg[2]
local pSmApiHandle = agent.boot(smhost, agentname)

if pSmApiHandle ~= -1 then
    local _, resourceContext, realm = agent.is_protected("/private/index.html",
                                                         "GET", pSmApiHandle)
    local res, session, iNumAttributes, pAttributes =
        agent.login("admin", "secret", resourceContext, realm, pSmApiHandle)
    local res2, token = agent.sso(pSmApiHandle,
                                  "cn=admin,ou=Contoso,o=psdsa,c=US", session,
    print("Token: " .. token)
    local res3, attrsnum, tokenAttrs, updatedToken =
        agent.decode_sso_token(pSmApiHandle, token)
    print("User DN: " .. tokenAttrs[agent.SM_AGENTAPI_ATTR_USERDN])
    for akey, aval in pairs(tokenAttrs) do
        if #aval > 0 then
            print(akey .. "=" .. aval)

Before running the script, make sure that you've configured the corresponding Realm in the Policy Server. For instance, "/private/index.html" is one of the protected resources.

Build and Deploy

You might want to use the following Makefile for simplifying the construction and deployment of our scripts.


PREFIX ?=          /usr/local
LUA_LIB_DIR ?=     $(PREFIX)/lib/lua/$(LUA_VERSION)
INSTALL ?= install

.PHONY: all test install

all: ;

install: all
	$(INSTALL) -d $(DESTDIR)$(LUA_LIB_DIR)/resty/siteminder
	$(INSTALL) lib/resty/siteminder*.lua $(DESTDIR)$(LUA_LIB_DIR)/resty/
	$(INSTALL) lib/resty/siteminder/*.lua $(DESTDIR)$(LUA_LIB_DIR)/resty/siteminder/
example: install
	resty -I /usr/local/lib/lua/5.3 examples/sso.lua conf/SmHost.conf spsapacheagent 

test: all
	PATH=$(OPENRESTY_PREFIX)/nginx/sbin:$$PATH prove -I../test-nginx/lib -r t


Run the following code from your shell:

$ make example

It should print the User DN and generated SSO token.


This was quite a journey. We've shown how the Siteminder SDK can be leveraged to create your own access gateway on top of FOSS software, such as the modern Nginx web server, without reinventing the wheel; and using Lua scripts instead of having to deal with all the nuances of native code, such as manual memory management.

Subscribe to our newsletter now!

Thanks for joining our newsletter.
Oops! Something went wrong.