Troubleshooting RSA Access Manager Single Sign-on · PDF fileRSA SSO cookie configuration in a...

White Paper

Abstract

This white paper explains how to diagnose and troubleshoot issues in the RSA® Access Manager single sign-on solution for Web-based applications. March 2011

TROUBLESHOOTING RSA ACCESS MANAGER SINGLE SIGN-ON FOR WEB-BASED APPLICATIONS

2 Troubleshooting RSA Access Manager Single Sign-On for Web-Based Applications

Copyright © 2011 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate of its publication date. The information is subject to change without notice. The information in this publication is provided “as is”. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All other trademarks used herein are the property of their respective owners. Part Number h8199


Table of Contents

Executive summary.................................................................................................. 4

Audience ............................................................................................................................ 4

Integration overview ................................................................................................ 4

RSA Access Manger for Web-based applications....................................................... 4

Software requirements ............................................................................................ 5

Product configuration .............................................................................................. 5

Before you begin ................................................................................................................ 5

Integration overview ........................................................................................................... 6

Prerequisites ........................................................................................................... 6

Configuration and installation.................................................................................. 7

aserver.conf file .................................................................................................................. 7

webagent.conf file .............................................................................................................. 7

Request processing flow .......................................................................................... 8

Login ...................................................................................................................... 9

Diagnosis and troubleshooting steps ..................................................................... 11

RSA SSO cookie configuration in a web-based client (for example, Webtop) configuration files .................................................................................................................................. 11

Configure the RSA SSO Authentication Service Class to perform login .............................. 11

Set the trace level ............................................................................................................. 11

HTTP/HTTPS Web debug/network files ............................................................................. 12

Test with the out-of-the-box Webtop application .............................................................. 13

Configure RSA Web Agent and configuration files ............................................................. 13

RSA Access Manager configuration file (aserver.conf) in Webtop .................................. 13

RSA Web Agent configuration file (webagent.conf) in Webtop ....................................... 14

Webtop Authentication Scheme ....................................................................................... 15

Timeout configuration ...................................................................................................... 17

RSA support .......................................................................................................... 17

Known issues ........................................................................................................ 17

Conclusion ............................................................................................................ 18


Executive summary This white paper explains how to diagnose and troubleshoot issues in the RSA® Access Manager single sign-on (SSO) solution for Web-based applications.

This paper lists the related configuration steps and installation files for RSA Access Manager and RSA Web Agent, the Web server, application server, Content Server, and Web-based clients. It also explains RSA Access Manager deployment for Web-based applications and lists known issues with resolution steps.

Audience

This white paper is intended for those who need to troubleshoot or diagnose issues related to Web-based applications integrated with the RSA Access Manager SSO solution.

Integration overview RSA provides SSO functionality to EMC® Documentum® (DCTM) Webtop through the RSA ClearTrust – Documentum SSO Module (CDSM). CDSM binds Documentum content, process management functionality, RSA ClearTrust security, and SSO functionality together. The system consists of two separate modules, the first running as part of the Webtop client process on the application server, and the second as part of the DCTM Content Server process. This paper will refer to the former as the Application Server Module (ASM), and the latter as the Content Server Module (CSM).

When a user attempts to access DCTM data through a Web browser, the RSA ClearTrust Web Server Agent intercepts the request and redirects the user to the ClearTrust login page. After authenticating with RSA ClearTrust, the ASM extracts the authenticated user and the SSO token from the RSA ClearTrust session cookie and connects to Content Server.

The CSM is an extension of the standard Documentum Content Server process. It accepts and validates the authenticated user and the SSO token. If ClearTrust servers validate the user and token, Content Server grants the user access to the resource through Webtop. Alternatively, the CSM sends an error message to the client, and the user is denied access to the requested resource.

RSA Access Manger for Web-based applications The following figure illustrates the supported deployment environment. It includes an RSA ClearTrust (RSA Access Manager) and RSA ClearTrust Web Server Agent and a Web server proxy to the Webtop application server host.


Figure 1. Deployment environment

Software requirements

Webtop host Content Server host

RSA ClearTrust (RSA Access Manager) Content Server

RSA ClearTrust Web Server Agent

Supported application server

Web server proxy to application server

Web-based application (Webtop)

The installation of these products is outside the scope of this paper. Please refer to the documentation provided with the relevant Documentum product for more information.

Product configuration

Before you begin

This section provides instructions for integrating and troubleshooting partners’ products with RSA ClearTrust. This paper is not intended to suggest optimum installations or configurations. It is assumed that the reader has both working knowledge of the two products to perform the tasks outlined in this section, and access to the documentation for both products to install the required software components. All prerequisite products and components must be installed and working prior to this integration. Perform the required tests to confirm the prerequisite components are up and running before proceeding.


Integration overview

This section provides instructions for integrating and troubleshooting RSA ClearTrust and Documentum Webtop-Content Server. For the following example, it is assumed that the deployment environment consists of a Windows/Tomcat environment with an Apache Web server/IIS proxy for the Webtop host, and a Windows environment for the Content Server host.

Figure 2 illustrates a fully configured environment.

Figure 2. A fully configured environment

Prerequisites

Ensure that the following prerequisites are satisfied before you start the installation and configuration process:

Confirm that your Documentum environment is running and that all necessary components including Content Server and the Webtop client are installed, deployed, and properly configured.

Confirm that your RSA ClearTrust environment is running and that all necessary components are installed and properly configured.

Configure your Apache/IIS Web server as a proxy to the Webtop Tomcat host. Please refer to Apache Tomcat documentation for instructions.


Install the RSA ClearTrust Web Agent on the Apache/IIS Web server. Create RSA ClearTrust users for Documentum.

Note: - Usernames must be the same in RSA ClearTrust and Documentum. - Usernames are case-sensitive in DCTM. For example, jsmith and JSmith are two distinct users. Keep this in mind when you create RSA ClearTrust users.

Configuration and installation

aserver.conf file

Open a current copy of your aserver.conf file. The following list contains mandatory and optional aserver.conf directives. You must restart your ClearTrust servers if you modify the file.

Mandatory directives

Ensure that the "cleartrust.aserver.dispatcher_info_list" directive is set to a valid list of one or more dispatcher process hosts for runtime API clients. The CSM must have this value to connect to the RSA ClearTrust servers and authenticate the SSO token. Refer to the aserver.conf file for more information.

Optional directives

You can set the “cleartrust.runtime_api.security” directive to the minimum security required on the connection between the CSM and the authorization server—“clear” or “anon”. If you do not set this directive, the default value for CSM is “anon”. Refer to the aserver.conf file for more information.

webagent.conf file

Open a current copy of your webagent.conf file. The following list contains mandatory and optional webagent.conf directives. You must restart your IIS server if you make any changes to the file.

Mandatory directives

Ensure that the "cleartrust.agent.cookie_name" directive is set to the name of the RSA ClearTrust SSO cookie. The CSM must have this value in order to read the RSA ClearTrust SSO token value. Refer to the webserver.conf file for details.

Ensure that the "cleartrust.agent.retain_uri" directive is set to “Yes”. Refer to the webserver.conf file for details.

Ensure that the "cleartrust.agent.sso" directive is set to “Yes”. Refer to the webserver.conf file for details.


Optional directives

You can set the “cleartrust.agent.user_header_list” directive to a list of one or more HTTP headers for the RSA ClearTrust agent to set with the value of the authenticated user’s name. See the webserver.conf file for details.

Navigate to %WEBTOP_HOME% and create the rsaConfig directory. For example, if Webtop is running on an instance of Tomcat 6.0 on Windows and Tomcat has been installed in its default home directory, the new directory path is as follows: C:\Program Files\Apache Group\Tomcat 6.0\webapps\webTop\rsaConfig)

Copy the webagent.conf and aserver.conf files to this directory. Log in to the RSA ClearTrust Web administration tool, add the Apache/IIS server,

create Documentum resources, and set the appropriate entitlements for the current users.

On the Webtop host: Ensure that all files are placed and configured properly.

AuthenticationService.class %WEBTOP_HOME%\WEBINF\classes\com\Documentum\web\formext\session\

RSASSOAuthenticationScheme.class %WEBTOP_HOME%\WEBINF\classes\\com\Documentum\web\formext\session\

sso_login_component.xml %WEBTOP_HOME%\wdk\config

sso_login.jsp %WEBTOP_HOME%\wdk\system\login

Restart the Tomcat application server. On the Content Server host:

Copy the dm_rsa.dll file to the %DOCUMENTUM_HOME%\dba\auth directory. Copy the dmauthplug.lib file to the

%DOCUMENTUM_HOME%\product\6.5\install\external_apps\authplugins\lib directory.

Install the RSA ClearTrust Runtime API by placing ct_runtime_api.dll in the %DOCUMENTUM_HOME%\product\6.5\bin directory.

Install the RSA ClearTrust Runtime API lib file by placing ct_runtime_api.lib in the directory. %DOCUMENTUM_HOME%\product\6.5\install\external_apps\authplugins\rsa\lib

Restart Content Server.

Request processing flow During a standard flow of events, the application server and Content Server modules are engaged in the following sequence as per Figure 3:

A user request for an RSA ClearTrust protected Webtop resource is redirected to the CDAM. The request is intercepted by an RSA Web Server Agent that is installed on a Web server proxy to the Webtop host server. After RSA ClearTrust


authenticates the user, it creates an SSO token and promotes the process to Step 2.

The authenticated user selects a repository from a list of available Documentum repositories. The ASM uses this repository to create a Content Server session.

The ASM reads RSA ClearTrust configuration files to establish information on how to establish a connection with the Content Server Module.

The ASM collects parameters to prepare for the Content Server connection. For example, using information from the aforementioned configuration files, ASM reads the authenticated username and extracts the SSO token from the user’s session cookie.

The ASM connects to the CSM. CDAM then pushes the SSO token from the web client (Webtop) process to its server (Content Server) process.

After receiving the ClearTrust token, the CSM verifies its validity. An invalid token results in an error.

If the token is valid, the CSM uses it to verify the validity of the username parameter.

If the token and username are valid, the process displays the requested resource.

Figure 3. Request processing flow

Login When a user attempts to access an RSA ClearTrust protected Documentum resource after successful installation and configuration of the module, the ClearTrust login screen appears.


Figure 4. ClearTrust login

Note: The RSA ClearTrust user ID must be a valid Documentum user ID.

When RSA ClearTrust authenticates the ClearTrust user ID and password and determines that the user is authorized to access the requested Documentum resource, the user is prompted to select a repository.

Note: The Repository drop-down list is dynamically updated at runtime with the available Documentum repositories.

After a Content Server connection has been established and the CSM has validated the user ID and SSO token, the user is redirected to the requested Documentum resource.

When the user logs out of the application, the application logoff jsp pages are displayed.

Note: Remember the following points when you create RSA ClearTrust users: - Usernames must be the same in RSA ClearTrust and Documentum. - Usernames are case-sensitive in DCTM. For example, jsmith and JSmith are two distinct users. - For session management, RSA ClearTrust sessions are managed separately from Documentum sessions. The administrator must synchronize session timeout periods.

Diagnosis and troubleshooting steps

RSA SSO cookie configuration in a web-based client (for example, Webtop) configuration files

Ensure that the user has specified valid RSA-related configuration values in the custom/app.xml or wdk/app.xml files and the content is as follows: <config>

<scope>

<application extends="webtop/app.xml">

<authentication>



<domain/>

<docbase>pe65</docbase>



<service_class>com.documentum.web.formext.session.AuthenticationService<

/service_class>



<sso_config>

<ecs_plug_in>dm_rsa</ecs_plug_in>

<ticket_cookie>CTSESSION</ticket_cookie>

<user_header>CT_REMOTE_USER</user_header>

</sso_config>

</authentication>

</application>

</scope>

</config>

Configure the RSA SSO Authentication Service Class to perform login

Configure RSASSOAuthenticationScheme to perform the login operation. RSASSOAuthenticationScheme is the first authentication scheme in the AuthenticationSchemes.properties file, which resides in the <web-app-root>\ WEB-INF\classes\com\documentum\web\formext\session folder and the content is as follows: scheme_class.1=com.documentum.web.formext.session.RSASSOAuthenticationSc

heme

Set the trace level

You can enable trace on Webtop and Content Server to troubleshoot issues:


To enable RSASSOAuthenticationService.class process tracing in Webtop, set the Webtop SESSIONENABLEDBYDEFAULT and SESSION trace levels to “true”.

To enable dm_rsa.dll process tracing in Content Server, include the flag in the Content Server configuration screen.

Enable DFC tracing from dfc.properties.

HTTP/HTTPS Web debug/network files

Capture the Charles/Fiddler logs for any cookie/proxy/SSL-related problems.

Charles/Fiddler is an HTTP proxy/HTTP monitor/reverse proxy that enables an engineer to view all HTTP and SSL/HTTPS traffic between the engineer’s machine and the Internet. This includes requests, responses, and HTTP headers (that contain cookies and caching information).

If the application is running in HTTPS mode, you must ensure that HTTPS traffic is decrypted. Add the host and port as illustrated in Figure 5.

Figure 5. Host and port details

You must capture Wireshark logs to track network-related issues.


Test with the out-of-the-box Webtop application

If you use another client application apart from Webtop, you must point the client application from Digital Asset Manager or Web Publisher to Webtop and test the issue in the environment. You may need to extend the Webtop layer instead of DAM or WP in custom/app.xml to point DAM or WP to Webtop as follows: <config>

<scope>

<application extends="webtop/app.xml">

</application>

</scope>

</config>

If you have performed any customizations, you must point the customized application to Webtop and test the issue.

Configure RSA Web Agent and configuration files

The application must contain the rsaConfig folder. In addition, ensure that the aserver.conf and webagent.conf files are available in the rsaConfig folder.

RSA Access Manager configuration file (aserver.conf) in Webtop

This configuration file contains the default information. We need to configure the following properties.

cleartrust.aserver.dispatcher_info_list=rsaaxm.dctmlabs.com:5608

This specifies the locations of all Dispatchers so that the Authorization Server can find other Authorization Servers and inform them that a user's status has changed and must be reloaded. For example, this occurs when a user is locked out or when his failed login count changes.

Allowed Values: A comma-separated list of Dispatchers, in the form host:port,host:port, where host is the hostname of the Dispatcher machine and the port is the Dispatcher list port. The list port is the Dispatcher's port for Runtime API communication.

Default Value: localhost:5608 Dependencies: Each port number in this list is a Dispatcher registration port and

must match the cleartrust.dispatcher.reg_port set in the dispatcher.conf of each Dispatcher.

Example: cleartrust.aserver.dispatcher_info_list=saturn:5608,venus:5608

cleartrust.aserver.dispatcher_info_list=localhost:5608

cleartrust.runtime_api.security=anon

This is the minimum network connection security level allowing connected clients to request some operations of this Authorization Server, including SSO token creation


and user property retrieval. This is set to the same value as cleartrust.net.ssl.use. However, in environments where these operations are not required, disabling them can improve security.

Note: This parameter must be set to a value equal to or lower than that of the cleartrust.net.ssl.use parameter for clients (including ClearTrust agents) to create tokens.

Allowed Values: clear | anon | auth

clear = cleartext, no encryption

anon = anonymous SSL, SSL encryption only

auth = mutually authenticated SSL, SSL encryption with certificate-based authentication

Dependencies: The value of this parameter must be the same for all RSA ClearTrust Authorization Servers deployed in your environment.

Cleartrust.runtime_api.security=auth

default from Webtop code : anon

RSA Web Agent configuration file (webagent.conf) in Webtop

cleartrust.agent.cookie_domain=.corp.dctmlab.com

This specifies the domain name in the HTTP “Set-Cookie” header for SSO tokens. Use the domain name or the fully qualified domain name (FQDN) of the Web server.

Allowed Values:

A valid domain name, prefixed with a period (requirement for some browsers). An FQDN. Do not prefix with a period. No value means that a site-specific cookie will be set.

Examples:

To return the SSO token to all servers in the “rsa.com” domain: cleartrust.agent.cookie_domain=.rsa.com

To return the SSO token to the “finance.rsa.com” server only: cleartrust.agent.cookie_domain=finance.rsa.com

cleartrust.agent.path=/

Specifies the path on the Web server where the SSO cookie applies. If only a subset of the content tree is protected with RSA ClearTrust, this option can be specified to prevent browsers from sending the cookie when requesting files from other parts of the tree. An empty value means that the current URL path will be used (that is, “/cleartrust” will be the path set for the cookie after successful authentication). This is not recommended.


Allowed Values:

A valid content directory path on this Web server. The SSO cookie is applicable to all resources on the Web server. This is the

recommended value.

Example: cleartrust.agent.path=/webapps

cleartrust.agent.cookie_name=CTSESSION

This specifies the name of the RSA ClearTrust SSO cookie. When SSO is enabled, the agent creates a cookie and sets it in the browser.

Allowed Values: Any string with no spaces or special characters. Dependencies:

If SSO is enabled, all Web servers participating in SSO must use the same cookie name.

To enable backward-compatibility with RSA ClearTrust 4.6.1 agents, set the cookie name to “ctrust-session-v002d” and parameter “cleartrust.aserver.token_version=0” in aserver.conf.

cleartrust.agent.user_header_list=

This specifies the list of HTTP request headers to set with the username value of the current RSA ClearTrust user. This is provided for sites that rely on the username specified in a header other than CT_REMOTE_USER.

Allowed Values: Any string that denotes an HTTP header. Multiple headers are comma delimited.

Example: cleartrust.agent.user_header_list=CTUSER

Note: This parameter is deprecated by cleartrust.agent.exported_headers.

Webtop Authentication Scheme

The RSASSOAuthenticationScheme.class file is the Webtop-based scheme file to initiate the authentication request of RSA Access Manager SSO with the repository. RSASSOAuthenticationScheme.class

/**

* The relative directory path to local copies of the RSA ClearTrust

aserver.conf and webagent.conf configuration files.

*/

// chee todo: should this be defined in app.xml


private final static String CONFIG_PATH = "/rsaConfig";

/**

* A String token that contains the name of the RSA ClearTrust aserver

configuration file.

*/


private final static String CT_ASRV_CONF = "aserver.conf";

/**

* A String token that contains the name of the RSA ClearTrust web

server agent configuration file.

*/


private final static String CT_WSRV_CONF = "webagent.conf";

/**

* The aserver.conf directive that contains a list of hosts on which the

dispatcher process(es) are running for runtime API communication. This

is in the form host:port,host:port for all the dispatchers that the

server should contact for runtime API communication.

*/

private final static String CT_DISPATCH_INFO =

"cleartrust.aserver.dispatcher_info_list";

/**

* The aserver.conf directive that contains the minimum security required

on the connection between Runtime API clients and the authorization

server in order to create/manipulate SSO tokens and to retrieve user

properties.

* Note that it isn't mandatory for the user to set this value. If it is

not set, this class will use the default value. See CT_SSL_DEFAULT_VAL.

*/

private final static String CT_SSL =

"cleartrust.runtime_api.security";

/**

* The default value for the minimum security required on the

connection between Runtime API clients and the authorization server in

order to create/manipulate SSO tokens and to retrieve user properties.

*/

private final static String CT_SSL_DEFAULT_VAL = "anon";


/**

* The webagent.conf directive that contains the name of the RSA

ClearTrust SSO cookie.

*/

private final static String CT_COOKIE_NAME =

"cleartrust.agent.cookie_name";

/**

* The webagent.conf directive that contains a list of HTTP request

headers to set with the value of the user name.

*/

private final static String CT_USER_LIST =

"cleartrust.agent.user_header_list";

Timeout configuration

You must verify the timeout parameters configured with RSA as follows:

HTTP Time Out is less than the RSA Idle Time out HTTP Time Out is equal to RSA Idle Time out.

RSA support RSA Access Manager 6.0.4 is supported for all supported Web application servers and operating systems.

Known issues

Web Workflow Manager stops responding when the RSA session times out (142160/WEBTOP-2753, WEBTOP-7225)

When you open a Workflow Template from Workflow Manager the RSA session times out, and the application page stops responding. The RSA login screen is not displayed.

Workaround: Close the browser window and open the browser again, and log in to the Webtop application.

When DAM is integrated with SSO there is an issue with synchronizing timeout for RSA Access Manager and DAM, and with “History” when SSO is enabled (WEBTOP-19846)

When the RSA session times out or when a user clicks the Logout button, the DAM application displays a message that the history has been released.

http://cmajira.lss.emc.com/browse/WEBTOP-19846



Workaround: Perform the following steps to ensure the history released page is not displayed:

1. Configure the Intermediate Form History Released page, which is a configurable property to any custom page or show the login page by modifying the "historyReleasedURL" entry in the file FormProcessorProp.properties. This file is available in the <web-app-root>/WEB-INF/classes/com/documentum/web/form folder.

2. Pass the main component URL or custom jsp that redirects to the Login page to the "historyReleasedURL" property.

Example (using a relative URL): historyReleasedURL=/webtop/component/main

OR

historyReleasedURL=/webtop/customjsp/customPage.jsp

The intermediate SSO landing page is not displayed when the application is opened in another tab in the browser (WDK-4622)

When an application is opened in a separate tab in the browser, the user is not redirected to the intermediate SSO landing page. Instead, the user is redirected to the main component.

Workaround: This problem does not occur in the out-of-the-box functionality. Verify whether the customer has customized the logout component and logout.jsp pages. Ensure that the session is invalidated in the logout customization files.

When a user logs in again after the session times out, the user is not taken to the last folder location (WEBTOP-20292)

When a user logs out of the application and logs in again, the user is not redirected to the same component from where the user was logged out of the application. Alternatively, the user is redirected to the Home cabinet.

Workaround: This is a defect in the application. Apply the available hotfix.

Conclusion You can use the information in this paper to troubleshoot and diagnose issues with RSA Access Manager for Web-based applications.

The task of troubleshooting issues involves capturing application-related wdk/webtop logs with the SESSIONENABLEDBYDEFAULT and SESSION flags, application server logs, proxy logs, RSA Access Manager logs, RSA Web agent logs, and Content Server logs with the –O trace authentication flag enabled.

You must also capture the Charles/Fiddler logs for HTTP/HTTPS debugging issues and Wireshark for network-related issues.



Troubleshooting RSA Access Manager Single Sign-on · PDF fileRSA SSO cookie configuration in a...

Documents

Transcript of Troubleshooting RSA Access Manager Single Sign-on · PDF fileRSA SSO cookie configuration in a...