SPNEGO based Single sign-on (SSO) setup for Webtop

[update Aug 2, 2010] I just finished SSO solution to resolve the security issue with authentication plug-in. You may check it out.

[update June 13, 2010] I posted Security Consideration to resolve the security flaw due to the shortcut implemenation of authentication plug-in. Further details will come soon.

[update April 08, 2010] As Deniel pointed out in his comment, the setup has serious flaw, which would compromise the security of the repository. I will work on a solution to address it, hopefully soon.

Webtop has default setup that can be configured to support SSO using either RSA Access Manager(formerly known as ClearTrust) or using CA SiteMinder Policy Server. The Documentum download site for EMC RAS plug-in from Powerlink lists documents for how to setup SSO for Webtop. The following link presented a document about the setup using SiteMinder.

While searching for Webtop SSO solutions, I also found out that IBM presented a solution using Tivoli Access Manager (http://www-01.ibm.com/support/docview.wss?uid=swg24007434).

Needless to say, all these SSO solutions need additional proprietary components (either RSA Access Manager, or CA SiteMinder Policy Server, or IBM Tivoli Access Manager) to make it happen. Here I will present a way to set up SSO for Webtop where no extra component is necessary. The approach is to rely on SPNEGO support from Internet Explorer or Firefox. The underlying authentication protocol is Kerberos


Even the solution presented here should be working with broader environments, the following is the system configuration I have been working with:

– Webtop 6.5 SP1 on Tomcat 6.0.18 under Windows 2003 SR2.
– Windows 2003 Server with Active Directory 2003
– JDK 1.6 used for Tomcat
– JRE 1.6+ used for IE or Firefox


The only thing I can think of is that the users in Documentum repositories should have their login name set the same as their AD login name. How the users in Documentum were created was irrelevant here, and as long as they are not using inline password.

Preparation for the setup

As for understanding the basics about the approach, please implement the setup from the following two links (where each link has detailed explanation and references to other resources):


Please note when testing, the browser instance has to be on a different host other than the web server. That’s the requirement of Kerberos support from IE and Firefox. Configuration is also needed for the browser to support SPNEGO(see here).

After you have successfully run the above two samples, you may follow the following the steps to set up SSO for Webtop (actually the solution is an application of SpnegoHttpFilter, many setup already done will be re-used here):

Service Account

Make sure to get a service account from AD, say myssoaccount. Actually I was using the LDAP binding account that already exists for my Documentum setup. The account and its password will be used later. 

Service Principal Name 

The following commands are run to against the AD, which created the service principal names for the above service account (probably you need to ask your AD admin to do that for you):

setspn -a HTTP/mytomcatserver.mydomain.com myssoaccount 
setspn -a HTTP/mytomcatserver myssoaccount


– Gets the source from SPNEGO SourceForge project

– Modify the implementation for SpnegoHttpFilter as following:

public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException 
    HttpServletRequest httpRequest = (HttpServletRequest)request;
    // go through the negotiation process.
    SpnegoHttpServletResponse spnegoResponse =
         new SpnegoHttpServletResponse((HttpServletResponse)response);
    SpnegoPrincipal principal;
    try {
        principal = authenticator.authenticate(httpRequest, spnegoResponse);
    } catch(GSSException gsse) {
        LOGGER.severe((new StringBuilder("HTTP Authorization Header=")).append(
        throw new ServletException(gsse);
    if(spnegoResponse.isStatusSet()) {
    if(principal == null) {
        LOGGER.severe("Principal was null.");
        spnegoResponse.setStatus(500, true);
    } else {
        LOGGER.fine((new StringBuilder("principal=")).append(principal).toString());
        // chain.doFilter(new SpnegoHttpServletRequest(httpRequest, principal), response); 
        HttpSession hs = httpRequest.getSession(); 
        if (hs != null) { 
            String principalAttr = "spnego_principal_"; 
            hs.setAttribute(principalAttr, principal); 
        chain.doFilter(request, response); 


The reason to just modify the original source was for the purpose of simplicity. We could make a sub-class if class SpnegoHttpFilter were not defined as final class. Please note the hard-coded attribute name for SpnegoPrincipal spnego_principal_, which will be used later in the authentication scheme and could be a point to improve by using configuration parameter.

The compiled classes should be put under Webtop’s class path.


Create file kbr5.conf as the following:

     default_realm = mydomain.com 
     default_tkt_enctypes = aes128-cts rc4-hmac des3-cbc-sha1 des-cbc-md5 des-cbc-crc  
     default_tgs_enctypes = aes128-cts rc4-hmac des3-cbc-sha1 des-cbc-md5 des-cbc-crc  
     permitted_enctypes   = aes128-cts rc4-hmac des3-cbc-sha1 des-cbc-md5 des-cbc-crc
[realms]   = {
     kdc =  myactivedirectoryserverhost 
     default_domain = mydomain.com
     .mydomain.com = mydomain.com

And put the file under Tomcat boot folder.


create a file, login.conf with the following content:

spnego-client {  
     com.sun.security.auth.module.Krb5LoginModule required;

spnego-server {  
    com.sun.security.auth.module.Krb5LoginModule required  

and put the file under Tomcat root folder.

web.xml for Webtop

Add the following section








           <param-value>your service account</param-value>

           <param-value>password for your service account</param-value>




before the following section in web.xml file


which would result in the SpnegoHttpFilter being called before the WDKController. The placeholders for account name and password should be replaced by the service account, myssoaccount and its password.

Also add the filter mapping to web.xml before the filter mapping for WDKController.


The  above initial parameter definition for SpnegoHttpFilter showed that NTLM and basic authentication are actually supported as well as Kerberos. When Kerberos support is not available (usually due to the browser configuration), it will fall back to NTLM, or then Basic authentication.  


Create a java class as the following and compile it. The class file needs to put under Webtop WEB-INF/classes folder. The package definition for the class can be anything you like. 

public class MySSOAuthenticationScheme implements IAuthenticationScheme 
    public SpnegoSSOAuthenticationScheme() {
        if(Trace.SESSION) {
             System.out.println(getClass().getName() + " starting ...");
    public String authenticate(HttpServletRequest httpservletrequest,
          HttpServletResponse httpservletresponse, String docbase) throws DfException 
        String principalAttr = "spnego_principal_";
        HttpSession hs = httpservletrequest.getSession();
        SpnegoPrincipal spnego_principal = (SpnegoPrincipal)hs.getAttribute(principalAttr);
        String domain = null;
       String strUser = null;
       String s3 = "noop";
       if (spnego_principal == null) {
           return null;
       } else {
           String username[] = spnego_principal.getName().split("@", 2);
           strUser = username[0];
       } if (Trace.SESSION) {
           System.out.println("Trying to log in: " + strUser + " into " + docbase);
       HttpSession httpsession = httpservletrequest.getSession();
       IAuthenticationService authservice = AuthenticationService.getService();
       String sTempDocbase = docbase;
       docbase = null;
       try {
           authservice.login(httpsession, sTempDocbase, strUser, getServiceLogin(s3), domain);
           docbase = sTempDocbase;
           if (Trace.SESSION) {
               System.out.println(getClass().getName() + " - Authentication succeed.");
       } catch (DfException dfe) {
           docbase = null;
       return docbase;
    public String getLoginComponent(HttpServletRequest httpservletrequest,
       HttpServletResponse httpservletresponse, String s, ArgumentList argumentlist)
       String principalAttr = CookieStringFinder.cookieString(httpservletrequest);
       HttpSession hs = httpservletrequest.getSession();
       SpnegoPrincipal spnego_principal = (SpnegoPrincipal)hs.getAttribute(principalAttr);
       if (spnego_principal == null) {
           if (Trace.SESSION) {
              System.out.println(getClass().getName() + " - null.");
           return null;
       } else {
       if (Trace.SESSION) {
           System.out.println(getClass().getName() + " - sso_login .");
       return "sso_login";
    private String getServiceLogin(String s)
       StringBuffer stringbuffer = new StringBuffer(128);
       return stringbuffer.toString();
    private static final String DM_PLUGIN = "DM_PLUGIN";

Please note the plug-in id was named as ‘myspnegossoauth’, which will be matched in the authentication plug-in definition for Content Server later. The hard-coded attribute name, spnego_principal_ was used to retrieve the SpnegoPrincipal in the scheme.


Update the properties file from Webtop installation under /WEB-INF/classes/com/documentum/web/formext/session/ as following:

scheme_class.2=<the package name>.MySSOAuthenticationScheme

app.xml for Webtop

So far I have found out no changes to app.xml are necessary for the SSO setup to work. However, if you only have one repository, or want to use a different authentication service class, definitely you can do so by customizing the authentication section from the file.

Authentication plug-in

From your Content Server, make a backup of several files under $DM_HOME/install/external_apps/authplugins/sampleauth, such as sampleauth.cpp, sampleauth.so (sampleauth.dll if the CS is on Windows), and sampleauth.o.

Open file sampleauth.cpp and locate the line,

static const char pluginId[] = "sampleauth";

and change it to:

static const char pluginId[] = "myspnegossoauth";

The matching plug-in id to that from the authentication scheme would ensure that users would be validated by this plug-in.

In the same file, locate method dm_authenticate_user and make changes as the following:

  // Add the following two lines
dmTrace(traceEnabled, isTraceInitialized, traceFilePath, "Authentication-%s: ALWAYS Success.", userOsName);
return true;
// dmTrace(traceEnabled, isTraceInitialized, traceFilePath,
// "Authentication-%s: Failure. Unknown user or password", userOsName);
// dmSetProperty(outPropBag, DM_ERROR_MSG, "Sorry, I couldn't authenticate this user!");
// return false;

Yes, the changes would result in all the users who reach to this plug-in will be accepted by the plug-in. I will explain this later. Then compile the plug-in with available C++ compiler on your CS platform and set it up for the repository you want to run SSO. The CS administration guide provides details about how to implement authentication plug-in and how to set it up.

Internet Explorer and Firefox setup

The link SPNEGO SSO Browser configuration shows how to set up IE or Firefox to support SPNEGO.

For now, you should be able to make your Webtop SSO enabled. congratulations!

Further explanation

You may have noticed the authentication plug-in would accept any users. Is it a security concern? I would say No.
The SPNEGO protocol would make sure the users are trustworthy between Webtop server and the client (IE or Firefox). Only users who passed the security checking by SPNEGO with Kerberos (via KDC from the setup) will reach the authentication plug-in. The following is an exception for an user login from docbase log (the timestamp for each line was removed for clarity. Space lines were added as well):

.. AT 61102: Start-AuthenticateUser: ClientHost(), LogonName(myuserloginname),
   LogonOSName(SYSTEM), LogonOSDomain(), UserExtraDomain(), ServerDomain()

.. AT 61102: Start-AuthenticateUserName:
.. AT 61102: dmResolveNamesForCredentials: auth_protocol()
.. AT 61102: End-AuthenticateUserName: dm_user.user_login_domain(myldapconfigname)
.. AT 61102: success
.. AT 61102: Found dm_user.user_login_name(myuserloginname),

.. AT 61102: Start-AuthenticateDomain:LogonName(myuserloginname),
    UserExtraDomain(), auth_protocol()
.. AT 61102: AuthenticateDomain - no domain required:domainOverride(False),
    user_login_domain(myldapconfigname), serverAuthTarget(), userAuthTarget()
.. AT 61102: End-AuthenticateDomain:
.. AT 61102: success

.. AT 61102: Start-AuthenticateUserState:UserLoginName(myuserloginname), UserExtraDomain()

.. AT 61102: Start-AuthenticateUserState:
.. AT 61102: dmStateForUser: auth_protocol()
.. AT 61102: End-AuthenticateUserState:
.. AT 61102: success

.. AT 61102: Start-AuthenticateByTrust:OSLogonName(SYSTEM),
    UserLoginName(myuserloginname), OSLogonDomain(), UserExtraDomain()
.. AT 61102: End-AuthenticateByTrust:
.. AT 61102: failure

.. AT 61102: Start-AuthenticateByPassword:UserLoginName(myuserloginname), UserExtraDomain()

.. AT 61102: Start-authenticateByPlugin: UserLogonName(myuserloginname), PluginId(myspnegossoauth)
.. AT 61102: End-authenticateByPlugin:
.. AT 61102: success

.. AT 61102: End-AuthenticateByPassword:
.. AT 61102: success

.. AT 61102: Final Auth Result=T, LOGON_NAME=myuserloginname, AUTHENTICATION_LEVEL=8,
userID=xxxx xxxx xxxx xxxx, USER_NAME=My User Name, USER_OS_NAME=myuserloginname,
USER_LOGIN_NAME=myuserloginname, USER_LOGIN_DOMAIN=myldapconfigname,

The user authentication log shows there are several steps for an user to be authenticated to a repository:

– First of all, the user has to be existing in the repository.

– Secondly the user domain is checked (I guess this step would branch into different path based on user’s setup method, i.e. inline password, LDAP authentication, or authentication plug-in, and so on).

– Then the user state is checked.

– And then the trusted login is tried with the user. Usually it would fail unless the user is the installation owner.

– Lastly, the user is checked with his/her login credential.

From the log, only the last step would involve the authentication plug-in when necessary. Apparently the Content Server employes other means to make sure the user has to pass the previous several steps to reach the last checking.

During my testing, only a valid user would be able to reach the last step where the plug-in is being called to verify the user. Since the SPNEGO protocol has made sure the user is from our trusted domain, it will be safe for our authentication plug-in to accept anyone who has ‘entitled’ with the ‘myspngossoauth’ identifier and come this far.

Hopefully I have clearly stated the necessary steps to implement the SSO solution. Please leave your questions or comments if you run into any issues.



Tags: , , , ,

33 Responses to “SPNEGO based Single sign-on (SSO) setup for Webtop”

  1. nguyen daniel Says:

    yes, this would work …
    however, given such plugin implementation, couldn’t i access the server with the whatever identity i want by just providing as password: “DM_PLUGIN=myspnegossoauth/xxx” ?

    • Allen Says:

      Hi Daniel
      I have to admit you were absolutely right. It took me a while to try your trick with both Webtop and DFC. The plugin has to be re-worked to deal with the situation. I will work on a solution to address it.
      Really appreciate your reading the post and pointing this out!

  2. Utilize SASL GSSAPI mechanism to achieve Single Sign-on (SSO) for JNDI/LDAP client « Documentum DAA Says:

    […] the procedure to work on SSO solution for Documentum Webtop and Kerberos setup, the word SASL popped up many times during my searching and cought my eyes. […]

  3. Koby Maru Says:

    I can think of two approaches that may work:
    1) Take the SPNEGO username, superuser credentials, and use principal mode to get a documentum ticket and submit that to the content server. This moves superuser credentials out to the Webtop server and creates the problem of securing them.
    2) Modify the Content Server authentication to validate the Kerboros ticket from SPNEGO and submit that as the password.

    I think a lot of people would be interested in a SSO solution that didn’t require 3rd party software.

    • Allen Says:

      Hi Koby
      I totally agree with you about the two suggestions.
      Personally, I prefer the second one. The reason is the same as you mentioned, the superuser credential has to be used and secured. But the first approach is still a feasible way to make the SSO solution work (even without much extra effort as the second). I hope someone can follow the suggestion and give us some feedback about it.
      My thinking right now for the second approach is to passing the SPNEGO token (Kerberos ticket inside) as the password to the authentication plug-in. Then the plug-in would decode the ticket to locate the client principal and match it with the user id passed from authentication service by Webtop. Since the token is encrypted by the private secret key of the service (myssoaccount), the plug-in kind of needs to play the role of the service, which will be provided with the secret key in the form of keytab file for myssoaccount. In this way, the plug-in structure can still be utilized.
      I never tried any 3rd party software (no access to any). From their docs, it seemed all of them need to login to an extra layer before reaching to Webtop. Looks like ‘Second’ Sign-on 🙂 to me, instead of Single Sign-on.

      Thanks a lot for your comments and suggestions!

  4. Mark Says:

    Did you ever solve the security issue? I’m trying to determine if I should spend time on this as a possible SSO solution or if it is ultimately a dead-end.

    • Allen Says:

      Hi Mark
      Thanks for your interest in the solution.
      I have not finalized the fix for the security issue. From what I have found so far, it will be a very feasible solution ( in my opinion :)).
      Best Regards

  5. Security consideration about SPNEGO based SSO solution for Webtop « Documentum DAA Says:

    […] about SPNEGO based SSO solution for Webtop By Allen Due to the security flaw presented in my previous post, it is hardly a workable solution for anyone interested. Here is the consideration to address the […]

  6. Single sign-on (SSO) solution for Webtop using SPNEGO « Documentum DAA Says:

    […] everyone who is interested. If you first reach to this post, I would suggest you to take a look at the first and the second, since they have established a foundation for the approaches described here, and […]

  7. Single Sign-on (SSO) implementation for Webtop 6.6 with Content Server 6.5 « Documentum DAA Says:

    […] well for me,. Comparing the support by Webtop 6.6 with the usage of SpnegoHttpFilter (described in SSO setup), Webtop 6.6 triggers SPNEGO mechanism in KerberosSSOAuthenticationScheme, which saved […]

  8. Usability of Single Sign-on for Webtop with multiple repositories « Documentum DAA Says:

    […] the solution using SpengoHttpFilter I initially presented did not bear those two issues. After carefully examining what […]

  9. Usability of Single Sign-on for Webtop with multiple repositories « Documentum DAA Says:

    […] the solution using SpengoHttpFilter I initially presented did not bear those two issues. After carefully examining what […]

  10. Felik Says:


    Currently I’m working on the MySSOAuthenticationSchema class
    but I can’t find Trace.SESSION and CookieStringFinder class

    can you please tell us from which package it was imported?
    And how can I find those class anyway?


    • Allen Says:

      Hi Felik

      The CookieStringFinder was intended to create a string from the request.
      Later I found it not that significant. It should be changed to the following:
      String principalAttr = “spnego_principal_”;
      ( need match it with another occurrence of princialAttr though)
      It’s my bad not cleaning it completely.

      The Trace.SESSION is just a boolean flag to control whether spitting out log or not. It must be from WDK framework if I recall correctly. You may just take it out for simplicity.

      Good luck.

      • Felik Says:

        Hi Allen,

        Really appreciate your reply.
        I have another query regarding location of the kbr5 and login.conf.
        Are they supposed to be located in C:\Tomcat\bin? Where is the root folder anyway? is it inside the webapps folder?

        And when I startup Tomcat service I got this error message:
        SEVERE: Exception starting filter SpnegoHttpFilter
        javax.servlet.ServletException: java.security.PrivilegedActionException: GSSException:

        I’m stucked at this point at the moment, please kindly advice 🙂

        Thanks and Regards,


  11. Felik Says:

    Hi Allen,

    I see a ROOT folder inside webapps folder in tomcat. Is it the one you’re referring into? I’ve changed the code, and I’m facing this error in the tomcat console, which causing webtop failed to startup:

    Exception in thread “Resource Housekeeper” java.lang.NullPointerException at org.apache.log4j.LogManager.getLogger(LogManager.java:188) at org.apache.log4j.Logger.getLogger(Logger.java:104) at com.documentum.fc.common.DfLogger.getLogger(DfLogger.java:397) at com.documentum.fc.common.DfLogger.error(DfLogger.java:113) at com.documentum.fc.internal.reshousekeeper.ResourceHousekeeper$1.run(ResourceHousekeeper.java:49)

    • Allen Says:

      Hi Felik,

      The root folder I referred to is c:\tomcat for example, not ROOT under webapps.
      It’s better you first follow setup samples from SpnegoHttpFilter to get them working. Then you can isolate the issues from SPNEGO and Webtop.

      Good luck.

      • Felik Says:

        Hi Allen,

        After almost 1 day trying, we already managed to have the opening page of the webtop become sso_login.jsp
        however I think I stuck on the Authentication plug-in step
        I open up the CS administration guide but no luck.
        Is it possible to send you the cpp file by email?

        Thanks a lot

      • Allen Says:

        Hi Felik

        I am very glad you got it working with the login page. One day is really an exceptional job in my view.

        Actually, if you strictly follow the approach from this post, there is not much with the Authentication plug-in.
        The way I showed here was just letting anyone reaching the plug-in got authenticated.
        Perhaps you need find a way to compile the C code of authentication plug-in on whatever platform you have for your Content Server.
        You could start with the sample provided by CS default installation. Once it’s done, it would be just a small change to make it accept anyone.

        If you want, you may also take a look at my other posts that presented with a secure approach, which employed delegation.

        Good luck

  12. Wout Says:

    Hi, thanks for the very helpfull tutorial.

    I’m using apache httpd with mod_jk. Is there any way to pass to user to tomcat?
    I get a NullPointerException at SpnegoAuthenticator.isLocalhost(..) when I’m browing through apache.

  13. Gustavo Says:

    Does this SSO mechanism supports authentication in more than one domain? In example: I have 8 different LDAP configurations in one repository, each configuration points to a different domain. Will the plugin be able to do the SSO login in the correct domain/server?

    I’m asking because Taskspace 6.7 native SSO/Kerberos doesn’t support multiple domains.


    • Allen Says:

      Hi Gustavo

      Personally, I have not verified and tested with multiple domain environment. As you have all the steps/code for the solution presented here, you may explore how to improve it to support multiple domain.

      As my other posts talked, you may need first check how to authenticate users from different domain to Taskspace server.
      I would think it involves how to generate the keytab files and how to setup the trust relationship between users’ domains and the domain where TaskSpace server is running.

      After the authentication to Taskspace server is achieved, the plug-in implementation should follow the same approach. Only with a bit attention to the delegation token created by Taskspace server to the CS plug-in.

      If you ever found the solution, could you please share it?

      Good luck

  14. Thanh Says:

    I’m encountering errors and would really like to get the spnego setup properly if anyone can assist. at this time, the preflight is good. helloKDC works. however, putting data in web.xml results in bad version number. please help.

    • Allen Says:

      Hi Thanh

      Not sure what error you got.
      Can you post the error message and specify from which log file you got it?

      What changes did you make to web.xml?

  15. Figure out how to manage Search engine marketing Wordpress Plugin Says:

    Figure out how to manage Search engine marketing WordPress Plugin…

    […]SPNEGO based Single sign-on (SSO) setup for Webtop « Documentum DAA[…]…

  16. Baljeet Says:

    I am implementing SSO and i performed all steps as given above.
    Now i am developing CS plugin. I created a DLL by referring SAMAUTHPLUG.cpp code and other files and copy that DLL into C:\Documentum\dba\auth folder. When i am starting docbase services ,i am getting an error during initializing the plugin. During debugging , i found that i am getting an error from dm_init function of DLL.
    Following statement of dm_init function creating problem..
    dmSetProperty(outPropBag, DM_PLUGIN_ID, _pluginId)

    this function is defined into dmauthplug.lib library . Please help

    • Allen Says:

      Hi Baljeet

      You did not mention what error you got.
      I would guess it’s just the environment settings for the c code compiler.

      You may first follow section ‘Implementing a custom authentication plug-in’ from Content Server Administration configuration guide to make the compiling of the sample plug-in work.
      Then you can work on your custom plug-in.

      Good luck

  17. Siteminder Authentication | DiscVentionsTech Says:

    […] support from Internet Explorer or Firefox. The underlying authentication protocol is Kerberos.: HTTP://DMDAA.WORDPRESS.COM/2009/12/24/SPNEGO-BASED-SINGLE-SIGN-ON-SSO-SETUP-FOR-WEBTOP/         this is the […]

  18. Violet Says:

    Doing that would mean lack of entry to your account and unauthorized
    purchases as someone may change the password after which use your account to buy items using your linked
    bank card. Should you share your Google Play info you might additionally lose entry to your Gmail account.

  19. khbkbv Says:

    you are really a just right webmaster. The web site loading pace is incredible.
    It sort of feels that you are doing any unique trick.
    Furthermore, The contents are masterwork. you’ve done a wonderful
    job in this subject!

  20. Henriko Says:

    Fascinating article. I’ll be sticking around to uncover much more from you guys. Thanks a lot!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: