SAP CRM Monitoring

Best Practice: mySAP CRM Monitoring

© 2002 SAP AG 1

mySAP CRM Monitoring Best Practice for Solution Management

Version Date: November 2002 The newest version of this Best Practice can always be

obtained through the SAP Solution Manager

CRM 3.0

Contents Applicability, Goals and Requirements _________________________________________________________3

1.1 Goal of Using this Service ___________________________________________________________3 1.2 Staff and Skills Requirements _______________________________________________________3 1.3 Example of a Business Scenario ______________________________________________________3

2 Best Practice Procedure and Verification ___________________________________________________6 2.1 Preliminary Tasks _________________________________________________________________6

2.1.1 Monitoring Concepts____________________________________________________________6 2.2 Procedures ______________________________________________________________________10

2.2.1 R/3 Back-End System Monitoring ________________________________________________10 2.2.2 CRM Server Monitoring ________________________________________________________15 2.2.3 Field Sales Specific Monitoring __________________________________________________23

2.2.3.1 CRM Server________________________________________________________________23 2.2.3.2 Communication Station_______________________________________________________25 2.2.3.3 Mobile Client Monitoring _____________________________________________________28

2.2.4 Interaction Center Specific Monitoring_____________________________________________29 2.2.4.1 CRM Server________________________________________________________________29

2.3 Verification _____________________________________________________________________31 2.3.1 Middleware Portal – SMWP _____________________________________________________32 2.3.2 Monitoring CRM Outbound Queues – SMQ1________________________________________33 2.3.3 QOUT Scheduler – SMQS ______________________________________________________36 2.3.4 Monitoring CRM Inbound Queues – SMQ2 _________________________________________37 2.3.5 Scheduler Status – SMQR_______________________________________________________39 2.3.6 TRFC Monitoring – SM58 ______________________________________________________40 2.3.7 Monitoring the Middleware Message Flow Statistics – SMWMFLOW ____________________41 2.3.8 Check Flow Definitions – SMO8FD_______________________________________________42 2.3.9 Middleware Trace _____________________________________________________________43 2.3.10 Monitor Request – R3AR3 ______________________________________________________43 2.3.11 Monitor Initial Load – R3AM1 ___________________________________________________43 2.3.12 Monitoring the Replication & Realignment Queues – SMOHQUEUE ____________________44 2.3.13 Queue Information for CRM Mobile Sites – SMWMQUEUES __________________________45 2.3.14 Communication Monitor – SMWMCOMM _________________________________________46 2.3.15 Message Recovery – CMWQ ____________________________________________________47 2.3.16 DCOM Connector Monitor ______________________________________________________47 2.3.17 Communication Station Log File: TransferService.Log ________________________________48 2.3.18 Windows Performance Monitor __________________________________________________49 2.3.19 SAP Connect Monitor – SCOT ___________________________________________________50 2.3.20 SAP Phone Administration – SPHB _______________________________________________51

Best Practice: mySAP CRM Field Sales Monitoring

© 2002 SAP AG 2

2.3.21 Internet Communication Manager Monitor – SMICM _________________________________52 2.3.22 Gateway Monitor – SMGW _____________________________________________________53 2.3.23 R/3 Buffer Monitor – ST02 ______________________________________________________53 2.3.24 Workload Monitor – ST03N _____________________________________________________53 2.3.25 Database Performance Analysis – ST04 ____________________________________________54 2.3.26 Operating System Monitor – ST06 ________________________________________________55 2.3.27 Local and System-Wide Work Process Overview – SM50 / SM66 _______________________55 2.3.28 Display Statistical Records – STAD _______________________________________________56 2.3.29 ABAP Dump Analysis – ST22 ___________________________________________________56 2.3.30 System Log – SM21 ___________________________________________________________57 2.3.31 Update Error Monitor – SM13 ___________________________________________________58

2.4 Troubleshooting__________________________________________________________________59 2.4.1 Problems during Data Exchange __________________________________________________59

2.4.1.1 Error Detection in Delta Download______________________________________________59 2.4.1.2 Error Detection in Initial Download _____________________________________________65 2.4.1.3 Error Detection in Upload _____________________________________________________68

2.4.2 Problems with E-Mail Sending (Outbound Direction) _________________________________71 2.4.3 CRM Server Performance problems or high I/O load due to excessive traces and logs being written 72 2.4.4 Mass changes of data (creating, changing) on the OLTP system leads to reduced system performance__________________________________________________________________________74 2.4.5 Performance problems due to statistics updates on tRFC/qRFC tables_____________________74 2.4.6 System performance degrades as the size of the tRFC/qRFC tables increases _______________74 2.4.7 Problem situation 1 ____________________________________________________________75 2.4.8 Problem situation 2 ____________________________________________________________75 2.4.9 Problem situation 3 ____________________________________________________________75 2.4.10 Problem situation 4 ____________________________________________________________75

2 ______________________________________________________________________________________77 Feedback and Questions ________________________________________________________________77


© 2002 SAP AG 3

Applicability, Goals and Requirements A Best Practice may not be applicable in some situations. To ensure that this Best Practice is the one that is needed in your situation, consider the following goals and requirements.

1.1 Goal of Using this Service The goal of this Best Practice is to document and to recommend procedures for:

• Identifying system conditions that can lead to business process standstill, loss of production or inefficient system operations

• Facilitating good system operations practices

• Troubleshooting system problems The general monitoring goals are:

• Detect documents in an error status

• Refer to reliable procedures for error handling

• Measure performance of data exchanges Monitoring objects and their attributes are usually displayed in the alert monitors. However, some interface points must be checked using generic tools and procedures. These are also described in this document.

1.2 Staff and Skills Requirements System administrators with experience for your installed my SAP CRM-Scenarios like e.g. mySAP CRM Field Sales, mySAP CRM Field Service solutions, mySAP Interaction Center solutions and CRM Online.

1.3 Example of a Business Scenario The example below describes a business process typical for companies with a field force organization and covers the basic tasks in the daily work of a sales representative like e.g. entering new sales orders. The business process flow for Sales Order Processing in the Field Sales environment is depicted on the following page.

Business Process Flow Description

1. Sales person creates an order for one of his customers.

2. Once a day the sales person connects to the CRM Server to upload his sales orders.

3. The sales orders are transferred to the CRM Server and passed through the validation process, which then decides to either update the Consolidated Database (CDB) or to forward rejections. In case of an update, all subscribed receivers are provided with the order. One of the receivers can be e.g. the R/3 back-end system.

4. If the R/3 back-end system is subscribed to these data, an SD sales order is created based on the information provided by the CRM Server.

5. During this process within the R/3 back-end system, the sales order is created or rejected, e.g. because of the exceeding the credit limit.

6. The R/3 back-end system sends confirmation or refusal information to the CRM Server.


© 2002 SAP AG 4

7. The CRM Server updates the CRM Online database and after replication to the CDB puts the information to update the mobile client’s database into the client’s outbound queue on the CRM Server.

8. The mobile client picks up its data from the CRM Server.

IPCIPC

Create inquiry

Create quotation

Create sales order

Print sales order

Mark for upload

Configure product

Determine price

Perform upload (connTrans)

Create sales order

Create sales order

Save sales order to CDB

Replicate to mobile clients

Perform download (connTrans)

Create quotation

Create inquiry

Configure product

Determine price

SAP CRMSAP CRM Mobile SAP R/3

Technical Process Flow Overview The business process described above is technically represented by the message flow used for transferring data from the mobile client to the CRM Server and to the R/3 back-end system. The flow is based on the standard SAP business process steps of the “Standard Sales Order Processing” Scenario.

This general description can be applied to most communications between the mobile clients and the R/3 back-end and/or CRM Servers.

“Sales orders” are represented by the corresponding BDoc message type being transferred between the mobile client and the CRM Server. BDoc types can also transfer all types of data between the mobile client and the CRM Server (such as system information and administrative information) and then to all other receivers.

Message Flow Within CRM In all CRM scenarios, message flow is used to supply all components of the CRM landscape with necessary information in accordance with subscription rules. The figure below shows message flow for CRM Field Sales scenario.


© 2002 SAP AG 5

CRM ServerApplication

Mobile Client

Mobile Client

Mapping: Sync BDoc ->

Msg BDoc

XIF Adapter

Routing (Simple Replication)

CRM Adapter

XIF Adapter

R/3 Adapter

External interfaces (XML - SOAP, IDoc)

External interfaces (XML – SOAP, IDoc)

Mobile Bridge:Msg BDoc -> Sync BDoc

Replication/ Realignment

CDB Service

Outbound Adapter

R/3 Adapter

Both XML/SOAP and IDocs can be generated from the ABAP complex data type used to define the external interface

CRM AdapterMsg BDoc Flow

Sync BDoc Flow

Sync BDoc Flow

R/3

R/3

CRM Middleware Adapters and Services

• The CRM Adapter is called by the message flow to pass inbound BDoc messages to the CRM Server application for validation. In case of a successful validation by the CRM Server applications, the content of the BDoc message is written (from the extension part) to the corresponding tables of the CRM database. If the validation was not successful, the BDoc message is sent back to the sender.

• The Replication and Realignment Service determines whether a replication and/or a realignment is necessary or not. If a realignment has to be performed for a BDoc message, this message is copied into a separate realignment queue for further processing. If realignment is not required, the receiving sites for a BDoc message are determined.

• The Mapping Service maps synchronization BDoc types to messaging BDoc types. The reverse direction is mapped using the Mobile Bridge. The Mobile Bridge takes a messaging BDoc type and creates one or more synchronization BDoc types (1:n relationship). It also takes one or more synchronization BDoc message and produces exactly one messaging BDoc message of one predefined type (n:1 relationship).

• The CDB Service saves the content of a synchronization BDoc message in the corresponding CDB tables.

Note: Mobile Bridge and CDB Service are only active in Field Sales (=MSA) or Field Service (=MSE) Scenarios.

(CDB = Consolidated Database on the CRM Server, MTS = Message Transfer Service. MTS client is on the Mobile Client, MTS server - on the Communication Station)


© 2002 SAP AG 6

2 Best Practice Procedure and Verification

2.1 Preliminary Tasks Before applying this Best Practice, ensure that you perform the following preliminary tasks or checks in the system:

• Complete all installation and post-installation actions and procedures including customizing.

• Ensure that the initial load has been successfully executed.

• Apply all SAP recommendations from SAP Service Sessions and any SAP recommendations resulting from customer problem messages.

• Implement all current SAP Support Packages upon availability.

2.1.1 Monitoring Concepts Monitoring in the mySAP CRM can be divided into the following areas:

• Monitoring the end-to-end message flow in the CRM Server

• Monitoring transactional RFC requests

• Monitoring the queues located on the respective server systems

• Operating system performance monitoring

• Database system performance monitoring

• Error analysis

ActionsDetected

errors

Undetectederrors

Preventionby regular monitoring

E.g. messageto system

administrator

StandardR/3 tools

Use ofprocessingmonitors

Automaticallygenerated

Computing Center

Management System

� Middleware Portal� Display BDoc Messages � Monitoring of queued RFC� Middleware Trace� RFC log files

The following table briefly lists the Software Components and the monitoring functions associated with them:


© 2002 SAP AG 7

Server Types

Monitoring Function Scenario

Mobile Client 1. Monitor the outbound queue of the mobile client

2. Monitor the inbound queue of the mobile client

3. Monitor database

4. Monitor the operating system

5. Monitor network throughput

6. Message Recovery

Only MSA/ MSE

Communication Station

1. Monitor message flow from all mobile clients to the CRM Server


3. Monitor operating system

Only MSA/ MSE

R/3 Back-End System

1. Monitor outbound queue

2. Monitor R/3 Basis System

3. Monitor database



all, if R/3-Back end is connected

CRM Server 1. Start with the Middleware Portal. In case of an error go to the appropriate transactions.

2. Monitor outbound queue, data transfer from the CRM Server to mobile clients

3. Monitor inbound queue, data transfer from mobile clients to the CRM Server

4. Monitor outbound queue, data transfer from the CRM Server to the R/3 back-end system

5. Monitor inbound queue, data transfer from the R/3 back-end system to the CRM Server

6. Monitor replication and realignment queues (only MSA/ MSE)

7. Monitor Middleware message flow

8. Monitor flow control

9. Monitor the Middleware log

10. Monitor status of BDoc types

11. Monitor initial/delta load / requests from the R/3 back-end system to the CRM

or from the CRM to the CDB (only MSA/ MSE)

In addition to the Middleware monitors, the SAP basis has to be monitored:

1. Monitor R/3 Basis System

2. Monitor SAP Connect interface for mailing and/or fax (if used in CRM Online or Interaction Center Scenarios)

3. Monitor CTI interface (if used in Interaction Center scenario)

4. Monitor database


© 2002 SAP AG 8

Server Types

Monitoring Function Scenario



Best Practice: mySAP CRM Monitoring

© 2002 SAP AG 9

CDBCDB

Inbound Queue

Outbound Queue

SAPBusinessProcessingInbound

Queue

Outbound Queue

R/3 System

Comm.Station

CRM MWSystem

Log file

Mobile Client(s)

Inbound Queue

Outbound Queue

BusinessProcessing

R/3 ApplicationDatabase

DCOMConnector

Flow / Process Control Engine & other services

Replication and Realignment Queue

Business Process Monitoring in the CRM – MSA Server Landscape Example

ST03 DB02ST06 SM50 / SM66SM58 SMGWST22

SMQ2

SMQ1 / SMQSSMWMFLOWSMO8FTSMW01/SMW02

SMOHQUEUESM58

SMQ1/SMQS

SMQ2SMQRSMWMQUEUES

MS Windows NT Performance Monitor

QMTCHECK

Non CRM-specific SAP System monitoring transactions: ST02, ST04, SM21, SMGW, SM13, STAD

DCOM Connector MonitorTransferService.log


DB & Log


© 2002 SAP AG 10

2.2 Procedures

2.2.1 R/3 Back-End System Monitoring

After the CRM Server has been installed and the initial load has been successfully performed, system load (data exchange) to and from the R/3 back-end system can be categorized as follows:

R/3 Back-End System � CRM Server

• Delta load (business data and conditions)

• Request of specific data

• Synchronization load (customizing data only)

• “Regular” communications using RFC calls

CRM Server � R/3 Back-End System

• Delta uploads from CRM via outbound adapter

• “Regular” communications using RFC calls

The following diagram shows the main points of interface on the R/3 back-end system. The transactions used to monitor these interface points are listed along with a brief label describing their function.


© 2002 SAP AG 11

CDBCDB

Inbound Queue

Outbound Queue


Queue

Outbound Queue

R/3 System

Comm.Station

CRM MWSystem

Log file

Mobile Client(s)

Inbound Queue

Outbound Queue

BusinessProcessing


DCOMConnector



Monitoring R/3 System

ST03N DB02ST06 SM50 / SM66SM58 SMGWST22

SMQ2

SMQ1 / SMQS

DB & Log


© 2002 SAP AG 12

The table below lists transactions for monitoring the R/3 back-end system:

Middleware Specific Monitoring

R/3 Back-End System Monitoring Queue Monitoring Activities / Functions

Transaction Code Monitoring Frequency Periods and Events

qRFC Outbound Queue Monitor: • Monitor data exchange from the R/3

back-end system to the CRM Server

• Queues should be relatively short and quickly processed

• Check, if the qRFC Version is up to date (see SAP Note 438015)

• To prevent data inconsistencies, you need to monitor the interfaces regularly for aborted or stopped data transfer

SMQ1 • Several times a day depending on the business process

• In case of an error message

• During mass updates

• Use Alert Monitoring

Status of Queue Scheduler • Monitor status of the QOUT

Scheduler

SMQS • In case of a status error

• Use Alert Monitoring

Basis Monitoring: SAP Performance Monitors

R/3 Back-End System General Monitoring Activities / Functions


R/3 System Buffer Monitor: • Monitor memory resource usage for

specific R/3 application servers

• Implement parameter recommendations for memory management per EarlyWatch analysis

• Use “normal” R/3 system monitoring practices

ST02 • Weekly

• In case of performance problems

R/3 System Workload Analysis: • Monitor RFC response time

statistics for CRM

• Monitor the Dialog response time for online transactions

ST03N • Daily


Database Performance Analysis: • Monitor database statistics • Monitor the buffer cache hit ratio

ST04 • Daily



© 2002 SAP AG 13



Database Performance Monitor • Monitor the growing of tables and

indices, especially on tRFC- and qRFC-tables

DB02 • Daily


Operating System Monitor: • Monitor hardware load during high

RFC transmission times

ST06 • Several times a day depending on the business process


Basis Monitoring: General System Checks


Transaction Code Monitoring Frequency Periods and

Local Work Process Overview: • Monitor current state of individual

work processes

• Ensure that enough work process capacity is available at peak times

SM50 • Several times a day depending on the business process

• Daily

• In case of performance problems or error messages

System-wide Work Process Overview: • Similar to SM50 but for system-wide

statistics



ABAP Dump Analysis: • Analyze aborted programs



System Log: • Check for general system errors

SM21 • Daily


Update Errors: • Check for entries that have “ERR”

status

SM13 • Daily


Display Statistical Records: • Determine the performance of single

statistical records

STAD • In case of an error message


© 2002 SAP AG 14

R/3 Back-End System Parameter Settings The parameters for data exchange with an R/3 back-end are defined in tables CRMCONSUM and CRMRFCPAR in the R/3 back-end. These tables are customized during system installation and are usually not changed during system operations. CRMRFCPAR controls the RFC traffic between the R/3 back-end system and the CRM Server.

The following table gives an example how to fill the CRMRFCPAR table. The actual CRMRFCPAR entries are client dependent and significantly larger than this example. If you want to, for example, provide a CRM system with data, you have to create the following entries:

Example for CRMRFCPAR Settings

Parameter Name Value Description

RFC QUEUE CRM Consumer that uses the CRM plug-in functions as a data receivers (e.g. CRM)

OBJNAME * Object name

* = valid for all objects

RFCDEST CRM_<SID>_<client> Specifies the RFC destination of the CRM Server.

DOWNLOAD * Restricts CRMRFCPAR entries to the initial or delta load. Possible values are:

* valid for all kinds of data transfer

I only for initial data transfer

D for delta data transfer

R for request

With initial transfers, data can be sent to two destinations but with delta transfers data can only be sent to one destination. Standard value: *

USE IN Q X Controls whether qRFC inbound queues are used on the CRM Server

SEND XML M “M”* – mixed mode (recommended)

“space”* - no XML-conversion. This is only possible if the sending and receiving CPUs have the same architecture (homogeneous system landscape).

“X” -XML will be used.

To improve performance in data exchange and further information on this setting, refer to note 442277 in the SAP Service Marketplace


© 2002 SAP AG 15

Scheduling Background Jobs There are no CRM-specific background jobs to schedule on the R/3 back-end system.

2.2.2 CRM Server Monitoring The following diagram shows the interfaces on the CRM Server. The transactions used to monitor these interface points are listed along with a brief label describing their function.


© 2002 SAP AG 16

CDBCDB

Inbound Queue

Outbound Queue


Queue

Outbound Queue

R/3 System

Comm.Station

CRM MWSystem

Log file

Mobile Client(s)

Inbound Queue

Outbound Queue

BusinessProcessing


DCOMConnector



Monitoring CRM Middleware Server

SMWMFLOWSMO8FTSMW01/SMW02

SMOHQUEUESM58

SMQ1/SMQS

SMQ2SMQRSMWMQUEUES

DB & Log


© 2002 SAP AG 17

The table below lists transactions used for monitoring the CRM Server:

CRM Server Central Monitoring Activities / Functions

Monitoring Type Monitoring Frequency Periods and Events

Middleware Portal Monitoring the CRM Middleware Portal with transaction SMWP can be subdivided into the following areas:

• Generation Information containing: BDocs Types: Generation of structuresBDocs Types: Generation of other runtime objects Replication objects Publications: Missing Indexes, Status of Delta/Initial Generation, Flow Definitions

• Runtime Information containing: Adapter Status Information Replication and Realignment Queues BDoc Messages in the Flow

SMWP • Several times a day depending on the business process

• After implementing transports

qRFC Outbound Queue Monitor:

• Monitor data transfer between the R/3 back-end and the CRM Server and between the CRM Server and mobile clients and other connected systems

• Queues destined for the R/3 back-end and other stady connected systems should be relatively short and quickly processed, queues to destination NONE too

• Entries destined for the mobile clients and BW remain in the queue until each receiver fetches its data

• When the queue entries for a client reach 10,000, the queue should be closely monitored (e.g. issue warning). When the entries reach 100,000, severe problems may occur and performance will be affected. Administrative actions must be taken in these cases, any deletion of queues causes data inconsistencies

• If a queue that is “in use” between a mobile client and the CRM Server is deleted, it will cause data inconsistency between the CRM


• Daily


© 2002 SAP AG 18



Server and the mobile client.

• In severe cases when a client cannot be manually rebuilt, it can be brought back into a consistent state by rebuilding the client data from the CDB (AC_EXTRACT)

QOUT Scheduler

• Ensure that all destinations are registered. Only destination NONE must be excluded. See also SAP Note 400330

SMQS • Daily


qRFC Inbound Queue Monitor:

• Monitor data transfer between the R/3 back-end system and the CRM Server and between mobile clients and the CRM Server and all other queues, which have to be stored in the CRM Online DB

• Messages in the inbound queue are processed according to the capacity of the CRM Server.

• High number of entries in the inbound queue can indicate insufficient capacity on the CRM Server


QIN Scheduler Status • This transaction runs the scheduler to

check the inbound queues on the CRM Server

• Ensure that all inbound queues are registered (type R). Register them, if necessary, using the transaction SMQR

• If a queue is registered and has not been processed for a long time, the administrator has to check the reason for it. Possible a scheduler problems may exist.

• Check the maximum processing time of the inbound queues.

• Set the maximum processing time of a queue ("Max-time") to 60 seconds for all queues during normal operations

SMQR • Daily


TRFC Monitoring • Monitor all transactional RFCs

SM58 • Several times a day depending on the b i


© 2002 SAP AG 19



(tRFCs) processed on the CRM Server (e.g. workflow)

SM58 is linked to the Replication and Realignment queues in error status in the EXTRACT and/or AC_EXTRACT queue (click on the error symbol)

business process

Message Flow Statistics • Collects statistical data about the

workload on the CRM Server caused by BDocs messages

• Use this as a starting point for analyzing performance problems

• The message flow is a central function in the CRM Server and uses BDocs

• Display statistics of the message flow within the CRM Server

• Ensure that the middleware message flow statistics are switched on

SMWMFLOW Only for power users

• Daily • In case of an error

message

BDoc Messages / Summary • Application error analysis • SMW01: Display the data of a BDoc

message and possible errors • SMW02: Display BDocs message

summary in dependency on the site ID

SMW01 / SMW02 • Several times a day depending on the business process


Middleware Trace • Developer Trace

SMWT • In case of an error message

Summary of Unprocessed BDoc Messages

SMW03

Check Flow Definitions Only after changes in the customizing • Consistency Check for Flow

Definitions

SMO8FD • After BDoc type changes or changes in services or in the flow are made



© 2002 SAP AG 20

CRM Server R/3 Adapter / Load Monitoring Activities / Functions


Monitor Load Status • Checks, whether the initial load was

successfully completed • Object status and actions:

• RED: Refer to SAP Notes 429423 and 350176 for initial load problem analysis

• YELLOW: initial download must still be done / is running

• GREEN: OK

R3AM1 • In case of an error messages during/after initial load

• Depending on the object status

Monitor Request • Used in exceptional cases to bring the

database into a consistent state after a data lost for instance in the CDB

R3AR3 • In case of an error message, if the databases are not consistent and a request from the R/3 back-end, the CRM or the CDB is necessary

Basis Monitoring: General System Checks CRM Server General Monitoring Activities / Functions


R/3 System Workload Analysis: • Monitor RFC response time statistics

for CRM • Monitor the dialog response time for

Online transactions (e.g. CIC0)

ST03N • Daily • In case of an error

message

Local Work Process Overview: • Monitor current state of individual work

processes • Ensure that enough work process

capacity is available at peak times

SM50 • Hourly • In case of an error

message or performance problems

System-wide Work Process Overview: • Similar to SM50, but for system-wide

statistics


• In case of an error message or performance problems

Internet Communication Manager (ICM) Monitor • Monitor current state of ICM and

services

SMICM • Upon error • Daily


© 2002 SAP AG 21

CRM Server General Monitoring Activities / Functions


• ICM administration

Buffer Monitor: • Monitor memory resource usage for

specific R/3 application servers • Use “normal” SAP System monitoring

practices

ST02 • Weekly

Operating System Monitor: • Monitor hardware load during high

dialog user load / RFC transmission times


Database Performance Monitor: • Monitor indices on tRFC and aRFC

tables

DB02 • Daily • In case of an error

message

Database Performance Analysis: • Monitor the growth of tables and

indices

ST04 • Daily • In case of

performance problems

ABAP Dump Analysis: • Analyze aborted programs

ST22 • Daily • In case of an error

message System Log: • Check for general system errors

SM21 • Hourly • In case of an error

message Gateway Monitor / Connection List: • Monitor the active connections to

other servers • Display gateway trace • Check the “SAP return code” and

“CPI-C return code” values for errors

SMGW • In case of an error message

Update Errors: • Check for entries that have an “ERR”

status



Display Statistical Records: • Determine the performance of

statistical records

STAD • In case of an error message

CRM Server Parameter Settings For data exchange with an R/3 back-end the parameters are defined in the CRMCONSUM table on the CRM Server side.



© 2002 SAP AG 22


Consumer CRM / CDB / EXT / ONL Consumer of adapter functionality

Active X Flag if application is activated

Q_Prefix CDB

R3A

EXT

ONL

Prefix for queue names in QRFC

The table SMOFPARSFA contains control parameters of the CRM Server.

Other settings of the parameters depend on the business scenario you are using, see CRM Installation Guide.

Current, experience-based parameter settings are checked during each SAP Service Session (EarlyWatch, GoingLive, etc.). The SAP recommendations should be applied according to the instructions given in the SAP Service report. The following parameters are important for CRM Field Sales and are provided here as an example:

Parameter key Parname Description

CRMGENERAL TRACE-LEVEL ENV=G

CRMGENERAL TRACE-LEVEL ENV=*

CRMGENERAL TRACE-LEVEL ENV=I

RRS_COMMON MAX_PACKAGE_SIZE Note 453882 (only MSA/MSE)

RRS_COMMON RRQUEUE_PARALLEL Note 453882 (only MSA/MSE)

FLOW USE_INQUEUE_ALWAYS Note 529764 (for distributed systems)

Scheduling Batch Jobs Middleware Trace Reorganization For tracing, you can use various standard settings or your own settings; see above in section settings in table SMOFPARSFA. Depending on the selected trace level, the system writes entries into trace tables. Such entries must be deleted in regular intervals to prevent these tables from becoming too large. Some possibilities to handle this would be to keep trace information (especially errors) in the log for e.g. 1 day, or 1 week and to delete the data afterwards. You have to schedule reorganization jobs, called SMO6_REORG (SAP Note 206439) and MW_REORG (SAP Note 487915)

Middleware Portal To be able to use the centralized status monitoring for the generation steps, you must call up the Middleware Portal (transaction SMWP) and activate the background job for status processing by clicking on Scheduled Background Job.

Note that status monitoring will be available only during the next day.


© 2002 SAP AG 23

BDoc Flow Statistics Activation Ensure that the middleware kernel application statistics are switched on. Call transaction SMWMFLOW. Execute Goto -> Activate Statistics. Select “message flow statistics”. Activate monitoring of the middleware message flow.

To activate the application statistics, select “Kernel application statistics”, select the change mode, mark the field “MW” (Middleware Message Hub Statistics”. Save.

Communication Monitor Collector Activation For activating transaction SMWMCOMM, the Conntrans Performance monitoring, you have to choose “Environment”-> “Run Collector”

2.2.3 Field Sales Specific Monitoring

CRM Adapter

qRFC Inbound

Mapping Service: Sync BDoc ->

Msg BDoc

XIF Adapter

Routing (Simple Replication)

XIF Adapter

R/3 Adapter

Mobile Bridge: Msg BDoc -> Sync BDoc Replication/

Realignment

CDB Service

Outbound Adapter

R/3 Adapter

Messaging Flow

Synchronization Flow

Synchronization Flow

qRFC Inbound

qRFC Inbound

qRFC Outbound

qRFC Outbound

qRFC Outbound

Inbound and Outbound Queue

Monitoring

Display BDoc Messages,

Middleware Trace

Replication & Realignment

Queues

In this section, all CRM Middleware monitors specific for Mobile scenarios are described.

2.2.3.1 CRM Server Mobile Bridge settings: For all BDoc types that you are planning to use in MSA/ MSE scenario you have to enable the Mobile Bridge: You have to set field ACTIVE to ‘X’ in table SMW3FDCUST according SAP Note 430392.


© 2002 SAP AG 24

Monitoring Activities / Functions Monitoring Type Monitoring Frequency Periods and Events

Queue Information for CRM Client Sites

• Display all mobile sites defined in the CRM Server together with the name of the queue assigned to each of these sites

SMWMQUEUES Only for error tracking in seldom cases

Replication Queues Monitor • Displays information about the status

and contents of the replication and realignment queues in the clients defined in the CRM Server

• The EXTRACT queue shows to which clients a specific BDoc type will be distributed

• To check the status of the queues • Deselect the flag: Display current

client only • Press the Refresh button

• Search for queues with • Status = Hold • number of entries > 0 and a • flash icon button in the right-most

column • Correct the error situation for the

indicated queues (error handling in SM58)

Note: Do not press the trash icon button if it appears next to an entry.

SMOHQUEUE • Several times a day depending on the business process

• Daily

Communication monitor • Communication monitor: monitoring

individual sessions, statistics of the data exchange for each site

SMWMCOMM Middleware � Monitoring � Mobile Client � Communication Monitor

• Daily • In case of


Mobile Client Message Recovery Unprocessed message recovery: reporting messages informing the CRM Server about errors during the import on the mobile clients

CMWQ

Monitoring -> Mobile Client -> Message Recovery

• Daily

Operating System / Gateway The SAP system statistic collector daemons, SAPOSCOL and RFCOSCOL, run on the Communication Station and gather hardware resource consumption data. Complementary programs run on the CRM Server and collect and display statistical data.

OS07 • Daily



© 2002 SAP AG 25


While the Communication Station is running, data is continually being collected via existing connections, and system data is also collected for subsequent evaluation. This data is called up periodically by the CRM Server and can be displayed and analyzed there using monitoring tools. The gateway, which is installed on the Communication Station, is used to call up the collected data. Windows performance monitor

2.2.3.2 Communication Station In the CRM Field Sales implementation, the monitoring possibilities for the Communication Station can be divided into three areas:

• DCOM Connector monitor

• Communication Log File: TransferService.Log

• Windows NT Performance Monitor / WIN 2000 Perfmonitor

Communication Station Monitoring Activities / Functions


CRM Transfer Service The SAP CRM Queued Transfer Service component logs the communication sessions between the mobile clients and the CRM Server in the TransferService.log file. The default file path is as follows:

C:\WINNT\TransferService.log QmtCnfg.exe, a Release 3.0 tool, which can be used to view the current trace level and log file location

Tracing the data transfer on the Communication Station is not required, only for troubleshooting.

TransferService.log QmtCnfg.exe

• Daily


© 2002 SAP AG 26

Communication Station Monitoring Activities / Functions


DCOM Connector: Use the following menu path to access the DCOM Connector component monitor: Start > Programs > Middleware > DCOM Connector. Start the application and choose Monitor. The following tables explain the different areas of the monitor.

• On demand

The following diagram shows the monitoring areas in respect to the Communication Station:

The Communication Station (CommStation) is the link between the CRM Server and the mobile clients.


© 2002 SAP AG 27

CDBCDB

Inbound Queue

Outbound Queue


Queue

Outbound Queue

R/3 System

Comm.Station

CRM MWSystem

Log

Mobile Client(s)

Inbound Queue

Outbound Queue

BusinessProcessing


DCOMConnector



Monitoring Communication StationDCOM Connector MonitorTransferService.log


DB & Log


© 2002 SAP AG 28

2.2.3.3 Mobile Client Monitoring When a traveling sales person has data to up- or download from his laptop, he must establish a connection to the CRM Server via the Communication Station. This may be done at the customer site using a telephone line. The Communication Station maintains a connection to the CRM Server that in turn maintains a connection to the R/3 back-end system (LAN configuration).

The CONTRANS.EXE program is executed on the mobile client to establish the connection. Data from the outbound queue of the mobile client is transferred through the Communication Station to the CRM Server. This data can be customer orders and/or customer data, such as newly created customers on the mobile client.

For monitoring the mobile clients, you have to consider four different areas:

• The operating system

• The local IDES database

• Trace files

• Data transfer log file Mobile Client Monitoring Activities / Functions


Queued Transfer Service The QmtCnfg program displays the connection status between the mobile client and the Communication Station.

QmtCnfg.exe TransferService.log

• In case of an error message in the data transfer phase

Client Console, metadata check, generation, comparison of BDoc structures

Data bound from the CRM outbound queue to a specific mobile client is copied to the inbound queue of that mobile client.

Data from the mobile client outbound queue is transferred to the inbound queue on the CRM Server.

The inbound and outbound queues of the mobile client can be displayed using the Client Console.

Trace.log • In case of an error message in the import phase

Operating system Windows NT Performance Monitor

Perfmon.exe In case of an error message

Database

You can monitor the SQL-Server database on the Mobile client with the remote SQL-Server Monitor

SAP Note 358507 SAP Note 433401 SAP Note 530317


© 2002 SAP AG 29

2.2.4 Interaction Center Specific Monitoring

CRM ServerApplication

E-Mail Server

XIF Adapter

SAP Connect

R/3 Adapter

SAPphone

Telephony Gateway

qRFC Inbound / Outbound

qRFC Inbound / Outbound

Inbound and Outbound Queue

Monitoring

Monitor Interfaces to External

Components

In ST03, Create Transaction Detail

Profile for CIC0

In this section, CRM Middleware monitors specific for Interaction Center scenarios are described.

2.2.4.1 CRM Server Monitor interfaces to the external components If you are using telephony / E-mail functionality in your Interaction Center scenario, you have to monitor the corresponding SAPphone and SAP connect interfaces.

Monitoring Activities / Functions Monitoring Type Monitoring Frequency

Periods and Events Monitor outgoing e-mails / faxes

• Nodes: List of all nodes defined for your System and their status.

• Jobs: List of all jobs defined for sending mails of faxes from your system.

• Routing: List of all routes defined for your system.

• Send orders: overview of the send orders (e-mails, faxes) in a given time

SCOT

View > Nodes or

View > System status

View > Routing

Utilities > overview of send orders

• Upon error

• Daily


© 2002 SAP AG 30


period and/or send status.

Monitor SAPphone Interface • Trace Files: switch the trace on or off

and display the trace files.

• Alert Monitor: starting the collection method and view the alert monitor data (Option: Integration of the Alert Monitor into CCMS)

• SAPphone version: get the SAPphone version

SPHB

Utilities > Trace > internal Trace

Utilities > Alert Monitor > Start collection method or Display

Utilities > SAPphone version

• Upon error

Transaction Detail Profile for CIC0 • monitor transaction CIC0 by using the

transaction detail profile

ST03N

• In case of



© 2002 SAP AG 31

2.3 Verification After adapting the recommendations described in this Best Practice, you can monitor your mySAP CRM solution using the transactions described in this section.

Many CRM-specific transactions are not available in the R/3 back-end system. Those common to both the R/3 back-end system and the CRM Server are described together. Those handled or interpreted differently are explained separately.


© 2002 SAP AG 32

2.3.1 Middleware Portal – SMWP


© 2002 SAP AG 33

2.3.2 Monitoring CRM Outbound Queues – SMQ1 Functional Description The CRM outbound queues contain all objects, which should be sent to the R/3 back-end, the mobile clients and other connected systems.

• Each queue contains single tRFCs in a specific sequence, which is called qRFC.

• Each client has a dedicated queue number.

• It is normal that some entries exist in the queues for the individual mobile clients. These have to connect asynchronously to the CRM-system and “pick up” their messages from the dedicated queue, which waits in SMQ1 (only MSY/MSE).

The R/3 back-end system, the APO and the BW system should always be connected (except during the downtime for backup)

The following table shows the different prefixes used for the different queues:

Queue Name Systems Description

R3AI<Object name> R/3 → CRM Initial. BAPIs destined for the R/3 back-end

R3AD<Object name> R/3 → CRM Delta

R3AR_<Object name> R/3 → CRM Request

R3AU<Object name> R/3 → CRM Upload

CRM_SITE<Queue number> CRM-> Mobile Client Synchronization BDoc types from the CRM Server going to the mobile client

CSA_<Object name>

CSA_MASS_<Object name>

CRM-> CRM (simple replication)

Outbound for simple replication of BDocs to receivers

BW.. <Object name> CRM-> BW Queues to BW systems

EXT.. <Object name> CRM -> extern Queues to external systems

When mass updates are carried out (e.g. application data updates) in the R/3 back-end system, the data is shipped via the outbound queue to the CRM Server. This data is controlled via qRFC settings.

Choose Information → Version in transaction SMQ1 to determine the current qRFC version.

For a detailed description of qRFC, refer to SAP Notes 193515 and 438015.

Usage

Menu path: Middleware → Monitoring → Queues → Display Outbound RFC Queues

Transaction: SMQ1

Handling Procedure

• Deleting a queue will lead to data inconsistencies between the sender and the receiver. E.g. deleting a queue that is “in use” between a mobile client and the CRM Server will lead to data inconsistencies between the CRM Server and the mobile client.


© 2002 SAP AG 34

In this case, you must bring the client back to a consistent state. In some cases, you can do this by extracting the data from the CRM Server and sending it to the client. This restores the client state, as it is known on the server. Data that was not yet successfully transferred from the client to the CRM Server must be manually reconstructed. Local data that is not usually sent to the server may be also lost and you may need to reconstruct it manually.

• If you accidentally delete data from a queue that is not assigned to a mobile client, data may be lost and cannot be recovered (e.g. in APO, BW, R/3 System).

• In case of error, first try to solve the problem. If all other attempts to fix the problem fail, you may need to delete an entry in a queue.

In this case, you must bring the different databases (R/3, CRM Server, CRM Mobile Client) manually into a consistent state. To do so, you must be familiar with the technical and business logic of the process involved.

Problems and errors can occur during data exchange (such as program crash, lost connection between R/3 back-end and the CRM Server, and so on).

Queues for the R/3 back-end should not wait too long. Possible causes for waiting can be e.g.:

• Error in the first BAPI in queue,

• Resource bottlenecks in the R/3 System,

• Logical sequence.

Check the status. If it is SYSFAIL or CPICERR, an error has occurred for the corresponding queue (most likely on the CRM Server). Double-click on the queue name and look at the status text of the first entry in the queue: you will find a short error message describing the error. After correcting the error, you can restart the queue.

SAP Note 443900 contains useful information on how to analyze the error situations during the data exchange.

Monitoring CRM Outbound Queues - SMQ1 – Queue Details Functional Description In the Details view of the outbound queue entries you can see the oldest queue entry. This gives you information about when the queue was running last time.

For MSA/MSE this gives you an estimation, how long a Mobile Client was not connected. It is possible that clients are not used anymore. The end user for such clients should be contacted and the site deleted or the client connected for running Conntrans.

Note: If the queue is deleted for the mobile client without deleting the mobile client, an inconsistency can occur. If data is sent to this mobile client later, it does not have the data that was deleted from the queue.

The outbound queues information is stored in the following tables: Table Name Description TRFCQOUT Queue Information (Header of the Queue) ARFCSSTATE Status table of the LUWs in the tRFC/qRFC

target system (Header of the record) ARFCSDATA Data To display the function modules of the queue, double-click on the line.

The following table describes the different information shown for a specific queue:

Field Name Description


© 2002 SAP AG 35

Field Name Description

Client Client

Queue name Queue name

Destination Logical destination (specified in function call)

Entries Number of logical units of work (LUWs)

Normal status READY: The first entry of the queue is ready for sending

RUNNING: The first entry of the queue is just processing EXECUTED: The first entry of the queue is already processed, just waiting for confirmation of the receiver system

NOSEND: no active sending, receiver has to fetch the records, deleting the record only after confirmation of the receiver

Error status SYSLOAD: No dialog work process free, the record will be automatically reprocessed by a background job SYSFAIL: error on receiver, look for a short dump there

CPICERR: network or communication error, record will be automatically reprocessed by a background job

Status

Wait status STOP: Application locks the first entry of the queue temporarily

WAITSTOP: Locked because of dependencies to other queues, because the other queue is just locked

WAITING: locked because of dependency to another queues, there is the linked record not the first entry in the queue WAITUPDA: First record in the queue contains update function, for which the queue is just waiting

NOSENDS: Queue is just waiting for debugging

MODIFY: The data of the first LUW are just been modified

Detailed information about the status can be found in note 378903.

1. Date Oldest date of a queue entry

1. Time Time of oldest queue entry

NxtDate Most recent date of queue entry

NxtTim Time of most recent date of queue entry

Wait for queue Only used for linked queues, not in CRM

Handling Procedure


© 2002 SAP AG 36

If you recognize many entries in one or more queues, take care of the following points:

• Only MSA/MSE: The corresponding client of the queue has not been connected to the CRM Server for a long time. You can check this in the column 1. Date. You should avoid situations when the mobile client has not synchronized his data for a long time. Many queues with a large amount of data can lead to a rapid growth of communication tables (e.g. ARFCSDATA) and have an impact on the system performance. The client should regularly connect to the CRM Server, in exceptional cases at least every two weeks.

• Only MSA/MSE: To reduce the amount of data the subscriptions and publications for a client should be as few as possible necessary from a business point of view.

• For all other queues not assigned to a client errors should be monitored hourly at least several times per day. Errors should be resolved and the queue restarted afterwards.

Monitoring CRM Outbound Queues - SMQ1 – Transaction Details

Functional Description Only the first and last entries are displayed. To display the full queue you can double-click on the dots in the mid of the screen.

To get detailed data about qRFC, double-click on a line and by selecting a client or user or queue name column. By selecting the destination you are linked to transaction SM59. Double-click on the function module column shows you the coding of the function.

2.3.3 QOUT Scheduler – SMQS

Functional Description

This transaction monitors the scheduler to process the outbound queues per destination (see SMQ1).

Usage

Here you can register, deregister or exclude the destinations from scheduling. You can stop all outbound queues in transaction SMQS.

Menu path: > Edit>Registration/ Deregistration

Transaction: SMQS

Handling Procedure

For registration of destination NONE refer to SAP Note 496826 dependent on your QRFC version.

Setup for Optimizing the Performance.

You can configure the “Max_time” parameter per queue by deregistering the queue and registering them again. Please do not set it too small (<60s), otherwise it causes a scheduling overhead in your system.

If you need to specify a special RFC server group for sending please maintain it in transaction RZ12 and specify the RFC-server group name in transaction SMQS>edit>change AS-group.

Refer to SAP Note 400330 for getting more information about the QOUT Scheduler.


© 2002 SAP AG 37

2.3.4 Monitoring CRM Inbound Queues – SMQ2

Functional Description The transmission of delta information is performed from the R/3 back-end system or other systems to the CRM Server. When the delta information arrives at the CRM Server, the data is forwarded to the inbound queue.

• The delta data can be found for instance in the R3AD* queues.

• The inbound queues are normally empty or small, if the CRM Server has enough resources and no errors occur.

• Mark an entry and press Change View (F8) to detect stopped or hanging queues.

• Double-click on the queue to display the LUWs belonging to the queue.

It is strongly recommended to avoid deleting the queue or queue entries, because this can lead to data inconsistencies!

The qRFC Inbound Queue Monitor, SMQ2, is not a critical monitor on the R/3 back-end system because there is normally no data to be displayed. Instead, monitor the outbound queue (SMQ1 or R3AM1) on the CRM Server.

When the CRM Server tries to send data to the R/3 back-end and the system is not “up”, the data stays in the outbound queue of the CRM Server in status CPICERR.

As soon as the data is in the R/3 back-end in-queue, it gets processed through to the respective application or to its appropriate end location.

Data is not deposited in the R/3 back-end’s in-queue and picked up later. It is processed immediately.

The inbound queue information is stored in the following tables: Table Name Description TRFCQIN Status inbound Queue - Header of queue TRFCQSTATE QRFC call condition TRFCQDATA Data of records ARFCRSTATE Status table of the LUWs in the tRFC/qRFC

target system (Header of the records) Usage Menu path: Middleware > Monitoring > Queues > Display Inbound RFC Queues

Transaction: SMQ2

1) Select the entry you want to view.

2) To view the details, press the Change View button (F8) and check for stopped or hanging queues.


© 2002 SAP AG 38

Field Names Descriptions Client Client Queue name Queue name Entries Number of logical units of work (LUWs)

Normal status READY: The first entry of the queue is ready for sending

RUNNING: The first entry of the queue is just processing

Error status SYSFAIL: error on receiver, look for a short dump there

CPICERR: network or communication error, record will be automatically reprocessed by a background job

ARETRY: Temporary error on receiver, record will be automatically reprocessed by a background job

ANORETRY: Fatal error, processing by QRFC manager stopped, check together with application Status Options

Wait status STOP: Application locks the first entry of the queue temporarily

WAITSTOP: Locked because of dependencies to other queues, because the other queue is just locked

WAITING: Locked because of dependency to another queues, there is the linked record not the first entry in the queue

NOEXEC: Queue is waiting for debugging

MODIFY: The data of the first LUW are just been modified

Read SAP Note 378903 for further information related to the status. 1. Date Oldest date of a queue entry 1. Time Time of oldest queue entry n. Date Most recent date of queue entry n. Time Time of most recent date of queue entry Source ID Destination of the sender Waiting for queue Only used for linked queues, not in CRM

Handling Procedure An error has occurred, if during an object load with status Running (light yellow) the date, time and the number of blocks remain constant over the time or grows, if new entries are written to the end of the queue. No queue entries are processed here.

General procedure: In the case of stopped queues, first search for the error causes and then fix them. Here are some possible error situations:

• If the status of the queue is SYSFAIL or CPICERR, an error is occurred.


© 2002 SAP AG 39

- By double clicking on the queue name and looking at the status text of the first entry in the queue, you will find a short error message describing the error reason. After the correction of the error, you can restart the queue.

- Check the short dumps (transaction ST22) on the receiving system, here the CRM system. An error in the queue data processing generates a short dump. Here you can find information about the cause and location of the error.

- Check further information about errors in the system log (transaction SM21) on the CRM Server. Have a look at the CRM Server Log.

• The input queue on the CRM Server has status Ready, but the number of entries does not decrease:

Transactions cannot be performed because the system is overloaded. Check if enough system resources are available and check queue registration in SMQR.

If the queue cannot be activated again, check the application log for possible application error messages.

• Data arrive from the R/3 back-end system but is still hanging in the inbound queue of the CRM Server:

• Check that the scheduler is set to active on the CRM Server (transaction SMQR)

• Follow-up problems can be found in the tRFC (transaction SM59) or in case of short dumps in transaction ST22.

• If no delta data arrives from the R/3 back-end (no data in the inbound queue of the CRM Server), or the delta queue has the status STOP you can proceed as follows:

• At least one object has load status Running or Abort and has not finished. Wait until the object is finished or you have to force termination.

2.3.5 Scheduler Status – SMQR

Functional Description This transaction runs the scheduler to activate the inbound queues if tRFC resources are free on the CRM Server. It also controls the runtime of a queue. Using transaction SMQR you can register or deregister the inbound queues “to be scheduled” by the scheduler.

If you have more than one client in your system, per client one scheduler is running.

You can stop all inbound queues in the CRM Server by de-registering all queues in transaction SMQR. Only registered queues are processed. These are registrations with any generic queue name with type R (R = registered, U = unregistered). So an entry must exist in transaction SMQR for the queue names, to get the queues processed by the scheduler in SMQR.

In the upper part of the display, you find the scheduler status, which is either INACTIV, ACTIVE or WAITING, STARTING (or SYSFAIL or CPICFAIL in error case). If the scheduler status is INACTIV: This status means that the qRFC scheduler has no work list at the moment. Therefore no registered Inbound queues in the READY status.

The scheduler is ready for queues are to be periodically checked for scheduling. The period can be set by the parameter “Max_time” per queue. The scheduler runs also if a new queue comes in (see SAP Note 369007).

Usage Menu path: Monitoring -> Queues -> Register/Deregister Queues


© 2002 SAP AG 40

Transaction: SMQR

Handling Procedure You can configure the “Max_time” parameter per queue by deregistering the queue and registering them again. Please do not set it too small (<60s), otherwise it causes a scheduling overhaed in your system.

If the scheduler status is ACTIVE or WAITING, select Refresh (F9) at short intervals (approx. 1 second) and monitor the scheduler status as well as date and time of the field “Last update (every 2 minutes)”.

If the status of the scheduler is occasionally WAITING, and the date, time in field “Last update (every 2 minutes)” changes every few seconds without a decreasing number of queue entries in SMQ2 this indicates that the qRFC scheduler does not get enough resources. The qRFC scheduler may then not start any asynchronous RFC calls (all dialog work processes are blocked) with the result that the qRFC inbound queues are not processed.

You can check whether the system can provide enough resources by using transaction SM50. A few dialog work processes should remain in status waiting. Otherwise the system is overloaded.

Important: perform the setup depending on the number of workprocesses as well as CPU resources.

If you need to specify a special RFC server group for sending please maintain it in transaction RZ12 and specify the RFC server group name in transaction SMQS>edit>change AS-group.

2.3.6 TRFC Monitoring – SM58

Functional Description Monitor all transactional RFCs (tRFC) processed on the CRM Server (such as workflow or replication/realignment for MSA/MSE).

Execution generates a selection screen where you can choose the display period, the user name, function, destination and status of transactional RFCs. Almost any combination (single or several values, ranges, and exclusions) of these parameters is possible.

If a system or application exception is raised during the processing of a tRFC LUW the target system will return this error to the sending system. The status of this LUW will be updated with the exception error message (red background color of status message).

You get information about the caller, function module, target system, date, time and status text. Additionally you get information about the transaction ID, Host, current transaction code, program, client and number of attempts establishing the connection.

Usage

Menu path: Middleware � Monitoring � Transactional RFC � Display Transactional RFC

Transaction SM58

Handling Procedure When error messages appear in the SM58 transaction, the error condition(s) must be resolved before re-execution of the tRFC procedure.

When the problem is solved, the tRFC should be manually re-executed / re-scheduled. If the error is fixed, the entry does no longer appear in the SM58 display.

Do not delete an entry before resolving the problem because this will lead to data inconsistency.


© 2002 SAP AG 41

2.3.7 Monitoring the Middleware Message Flow Statistics – SMWMFLOW Functional Description With this function you can display statistics on the BDoc message flow within the CRM Server.

The statistics provided by this function are divided into two main areas:

• BDoc type/ service kernel application statistics: Application statistics from the R/3 kernel for BDoc types and services.

• BDoc type/ site / queue statistics: Statistics for BDoc types, sites and queues.

Usage

Menu Path: Middleware → Monitoring → Message Flow

Transaction: SMWMFLOW

Activation

To activate the statistics, choose the transaction SMWMFLOW and execute Goto → Activate statistics.

• If you want to check whether the application statistics from the R/3 kernel have been activated, choose Kernel application statistics. The checkbox for Middleware Message Hub Statistic must be active. So that the statistics are actually activated, you must have first scheduled the background job SAP_COLLECTOR_FOR_PERFMONITOR.

• Choose Middleware Message Flow Statistics. You can toggle the creation of statistics on and off with On/Off. When On is selected, the background job RSMWM_BSTAT_COLLECTOR is automatically scheduled (status: Collector: On).

Workload from database Choose Workload�Workload from database. Select one or all (TOTAL) R/3 instance(s) and specify the required time period. This statistics records the progress of BDoc messages in the CRM Server (including the retention period in inbound queues). The retention period in the outbound queues is not recorded. The single records are directly written to the message flow before the completion of processing. In contrast to the application statistics, these statistics also contain data from the message header of the BDoc messages concerned.

The report RSMWM_CHECK_STATISTIC_RECORDS displays single records of the application statistic. It displays related statistical data records together (i.e. a single BDoc type with its services). You can also view the entire statistical data record here by double clicking.

Just as in case of transaction ST03N, the single records created in this way are available at least for the current day or the last 2-3 days, depending on settings in ST03N.and can also be displayed.

These records are automatically aggregated by the CCMS data collector to form the following statistics (the sum of the total and detailed response times, as well as the average total and detailed response times):

• By BDoc type (including the times for the services)Per service (for all BDoc types)Per BDoc type and service. Just as in the case of ST03N these statistics are available for days, weeks and months. Column Records displays the numbers of flow runs of the different BDoc types.

Alternatively, in SMWMFLOW, choose Workload from database �Per service

Here you can get the workload for individual services per BDoc type. You can also display the workload per BDoc type (Detail).


© 2002 SAP AG 42

This overview does not include the processing time for the following:

• Inbound adapter

• R/3 inbound adapter

• Flow itself

Therefore, the sum of the total response times of all services can be significantly smaller than the total response time of the BDoc type you have chosen before.

Last minutes workload Choose Workload� last minutes workload (Set time interval: Analyze last 15 min for this Instance) Here you can get statistics about the current workload of the message flow in your system (for BDoc types and for CRM services).

To display statistics for the Middleware services, you can also choose Goto � Service statistics. In the service statistics you can see Detail statistics for individual services as well as the workload for individual services per BDoc type.

In all detail statistics, to display individual statistics for the selected BDoc type or service, choose Single steps

Alternatively, in SMWMFLOW, choose Last minutes workload � Per service

Message-flow statistics Press the Message flow statistics button.

The Data Collector starts automatically every hour to provide the required data. If you need the latest data for your analysis, you can manually start the data collector at any time by choosing Run Collector.

The statistics relate to one week and to different instances.

The header data shows the display and your selection criteria.

The results show the single records that the statistics refer to. For each record the BDoc type, client, site, queue and name of the CRM Server are displayed, as well as the retention time of the corresponding BDoc type on the CRM Server.

2.3.8 Check Flow Definitions – SMO8FD Functional Description

Consistency Check for Flow Definitions Usage

Menu path: Middleware → Message Flow → Display and Check Flow Definitions

Transaction: SMO8FD

Handling procedure You developed one or several new BDoc types. You generated the flow definition for these BDoc types.

• Flow definition errors may occur using newly developed BDoc types.


© 2002 SAP AG 43

• Search for SAP Notes that solve the problem. If there is no solution, create a customer message.

Note: Use SMO8FD after a new BDoc definition has been transported.

2.3.9 Middleware Trace Functional Description The table SMWT-TRC stores all BDOC trace information and is linked to transaction SMW01.

Usage This information is only necessary for error handling in transaction SMW01.

Handling Procedure Trace level is important: during the implementation phase should be high. In the production system should be reduced, otherwise you may get performance problems. Refer to note 206439, see setting in table SMOFPARSFA.

The trace should be used withSMW01/SMW02 to check the content and find the error and fix it.

2.3.10 Monitor Request – R3AR3

Functional Description Data requests can be defined and triggered manually from an R/3 back-end or from the CRM database using the transactions R3AR2 and R3AR4. The request monitor then monitors these requests. If there are data in the CRM Server or CDB or R/3 back-end missing or incomplete, in exceptional cases, you can use the request to bring your database into a consistent state after a data loss in one the databases.

Usage

Menu path: Middleware � Monitoring �Data Exchange-> Monitor Requests

Transaction: R3AR3

2.3.11 Monitor Initial Load – R3AM1 Functional Description Checks whether the initial load was successfully completed.

Usage Menu path: Middleware � Monitoring � Data Exchange-> Monitor Objects

Transaction: R3AM1

Call transaction R3AM1 (Monitor Objects) on the CRM Server and search for objects with a status other than green

Handling Procedure

• RED: If the status monitor displays objects with a status other than green, then problems occurred during the initial load

• YELLOW: If the display is empty - initial load hat not been performed yet


© 2002 SAP AG 44

• GREEN: Status OK

If the status symbol for an object is RED, refer to SAP Note 429423 for initial load problem analysis.

2.3.12 Monitoring the Replication & Realignment Queues – SMOHQUEUE

Functional Description This transaction must only be monitored for MSA/MSE. Displays information about the status and contents of the replication and realignment queues in the Mobile Sites defined on your CRM Server.

• To stop or start replication queues manually, click on the traffic light: The light changes between red and yellow or green.

• Normally all queues should be running or waiting: Particular processes are automatically triggered to process queue entries when such entries are entered into the queue.

• The Number of entries displays the number of entries that are currently in the queue. This number should be continuously decreasing, unless new entries are entered into the queue at the same time. Double-click on the field number to view the entries in the respective queue.

• If you interrupt queue processing, the processing of the current entry is completed and then the queue is set to status Hold.

Usage Menu path: Middleware � Monitoring � Queues � Monitor R&R Queues

Transaction SMOHQUEUE

Field Names Descriptions Name • AC_EXTRACT • EXTRACTBLK • EXTRACT • REALIGN • SUBCHECK

Name of queue: Represents those extract jobs triggered directly from the Administration Console, for example for a data reload to a mobile client to recover the local SQL database Contains extract jobs broken down by BDoc types of the type bulk. Contains extract jobs broken down by BDoc types (the jobs are created by realignment) Contains realignment jobs for individual BDoc types whose responsibilities must be checked Contains subscription checker jobs (these are generated in the Administration Console, if a subscription is changed)

Number of entries Current number of entries (messages) in the queue Active tasks Current number of active tasks

By setting the status icon in the status column you can:


© 2002 SAP AG 45

• Release queues for processing by setting their status to Released (yellow light)

• Reset released queues to Hold (red light)

• Interrupt queue processing (Status Running – green light) by setting the status to Hold. Handling procedure Normal condition: all lights green or yellow, number of queue entries is decreasing, if no new queue entries are created.

• If an error occurs during processing the Queue demon automatically sets the queue to hold. An error symbol appears in the last column. You can select it to display the error message. When you release the queue again, the error status is reset.

• If you release a queue after the Queue demon has started, the status of the queue changes to Running, provided it contains entries for processing. If all entries have been processed (number of entries is zero), the status changes to Released again.

If the number of entries in a queue remains constant over a period of time, it may be necessary to manually trigger the processing jobs by clicking the green flag corresponding to the queue.

If a queue contains entries, you can display details of the individual entries (such as the number of sites affected) by clicking on the number of entries. You can delete individual entries by selecting the checkbox in the first column of the record and choosing Delete. To update the information choose Refresh, but:

Try to avoid deleting entries; this leads to inconsistencies of the mobile client local database. Refer to note 429627 for implementing the authorization checks necessary to avoid this kind of situations.

2.3.13 Queue Information for CRM Mobile Sites – SMWMQUEUES

Functional Description Display all mobile sites defined in the CRM System together with the name of the queue assigned to each site. This transaction collects some queue information to Mobile Sites. Call this transaction:

• If you want to know which queue is assigned to a specific mobile client site

• If you want to check whether this queue has been registered for the qRFC scheduler

• If you want to check when data was last entered into this queue.

Field Names Descriptions

Client Displays the respective CRM Server client

Site Name Site name assigned to the mobile client

Queue Name Queue name assigned to the mobile client

Status

Displays the queue status

In the case of inbound queues, a yellow symbol indicates that the queues have either been stopped or are waiting for another queue to be processed, while a red symbol indicates an error.

Entries Indicates the number of entries currently in this queue.


© 2002 SAP AG 46

Registered This column is displayed only in the case of inbound queues.

It indicates, if a queue has been registered (R), has never been registered (empty) or has been deregistered again (U).

A red symbol indicates that there are entries in the respective queue, that the queue is however not registered or that there are problems with the Queue Scheduler (for example status SYSFAIL or CPICFAIL).

A yellow symbol refers to the same problems, but indicates that there are no entries in the queue.

A green symbol indicates that there are no problems.

First Date / First Time Informs you when the oldest entry was placed in the queue.

Last Date / Last Time Informs you of when an entry was last placed in this queue.

Site description Describes the respective site and was defined when this site was created in the Administration Console.

Usage

Menu path: Middleware� Monitoring � Queues � Display Mobile Site Queue Information

Transaction SMWMQUEUES Handling Procedure Shows the last time of connection. Here you can see whether a client did connect for a longer time (also shown in transaction SMQ1), Assignment of site name and queue name (also shown in transaction SMOEACSMOEAC).

2.3.14 Communication Monitor – SMWMCOMM Functional Description This transaction is only for MSA/MSE-monitoring. You can use this for checking the performance of the Conntrans sessions of the Mobile Sites. The transfer rates, MW server’s answer time and the time consumed by the database read and writes during a session can be monitored using the transaction SMWMCOMM. The session monitoring functions will be used for the logging the time intervals. The module SMWM_START_SESSION will be called soon after the client attempts an authentication with the CRM server. Then after the transfer completes all the relevant session data and the session completion will be sent to the CRM server using the DCOM Connector interface (The Gateway installed on the Communication Station is not needed to execute this operation). A session ID, which is a 32-byte GUID generated at the time of instantiation of the server component will be passed in all the above session monitoring function module calls as one of the parameters. These data is reorganized within the middleware reorganization job. Usage

Menu path: Middleware � Monitoring � Mobile Client � Commstation Monitor Handling Procedure


© 2002 SAP AG 47

You can monitor the Conntrans performance per site, queue or Commstation. Otherwise you can check the performance per time range, for instance, day or week. The statistic you get shows upload and download times and transferred mount of data. It is also possible to get information about failed session between Commstation and CRM server.

2.3.15 Message Recovery – CMWQ


The Mobile Client Message Recovery displays the status of BDoc messages sent from the CRM Server to the Mobile Clients (inbound processing) and allows their reprocessing: Usage

Menu path: Middleware � Monitoring � Mobile Client >Message Recovery Handling Procedure Processed messages have successfully been processed from the inbound queue into the database of the Mobile Client.

Unprocessed messages could not be saved on the client database. These BDoc messages are deleted by the system, a reporting message is sent to the CRM Server. The corresponding BDoc messages can be sent again to the Mobile Client by calling the transaction Message Recovery.

The complete information on the BDoc message status on the Mobile Client can be accessed via the Client Console.

2.3.16 DCOM Connector Monitor

Usage To access the DCOM Connector component monitor choose Start� Programs�Middleware � DCOM Connector. Start the application and choose Monitor. The following tables explain the different areas of the monitor.

View: Active Processes Shows the active processes of the DCOM Connector

Column Headings Descriptions Process ID

State Either active or terminated Up since Time since process started Page faults Number of page faults since startup Working Set Size of the process’ actual working set in KB

(kilobytes) Peak Working Set Maximum working set of the process since startup Virtual Memory Set actual size of the page file usage in KB Peak Virtual Memory Maximum usage of the page file since startup in KB

View: Open Connections


© 2002 SAP AG 48

Shows the open connections of the DCOM Connector

Column Headings Descriptions Destination

State Initial: created but not yet allocated Connecting: connecting to server Active: allocated by an object Pool: resource was returned to pool without cleanup Cleaning: R/3 context is being released Clean: resource was returned to pool in a clean state Closed: resource is closed and not available

Sys Name of application server Host Hostname of application server ID System number (port number is 33## and 32##) SAP User User ID in R/3 Rem. Vers. Version on application server Rem. Codep. Code page on application server. Proc. Process ID hRfc Handle to RFC connection Loc. Vers. Version of DCOM Connector Loc. Codep. Code page of DCOM Connector Conv. ID Conversation ID like in SAP gateway monitor T Trace

Calls Number of calls since startup Last Function Name of last function being called

2.3.17 Communication Station Log File: TransferService.Log Functional description The SAP CRM Queued Transfer Service component logs the communications between the mobile clients and the CRM Server in the TransferService.log file. Usage The default file path is as follows: <windows temp folder >\TransferService.log

The TransferService.log file supplies the following data for each session. This data is stored and can be displayed in text format on the local Communication Station server.

Session header data

• Name and/or IP address of the DCOM Server

• Date and time when the session was started (in UTC time)

• Date and time when the session was closed

• PID of the DCOM process

• TID (thread ID) of the current session (PID and TID together are unique IDs for session)


© 2002 SAP AG 49

• Object ID (session ID)

• Site name

• Queue name

• Name of the PC from which the session was opened

• Name of the Windows user

• Session status

Session performance data

• Number of transactions sent to the R/3 System

• Number of table entries sent to R/3

• Number of transactions received from R/3

• Number of table entries received from R/3

• Compression rate for data received from client

• Compression rate for data sent to client

• Processing time on the DCOM Server for data received from R/3 and sent to the client

• Processing time on the DCOM Server for data received from the client and sent to R/3

• Session status

• Detailed information for the functions Read, ReadConfirm, Send and SendConfirm:

• Number of calls to R/3

• R/3 response time

• Network time between Mobile Client and DCOM Server

• Response time of the client database

• Transmission rate for communication of DCOM Server with R/3

• Transmission rate for RFC communication between client and DCOM server

• Access rate in the client database (bytes per second)

2.3.18 Windows Performance Monitor Functional description The Microsoft Windows Performance Monitor (PERFMON) is a performance-monitoring tool on Windows NT / 2000 systems.

PERFMON can be used on the Communication Station to analyze performance problems due to:

• Total response time

• Network time

• Browser generation time

• CPU and Memory Usage

When analyzing network conditions, PERFMON should be run locally on the Communication Station rather than from a remote server. When running PERFMON from a remote server,


© 2002 SAP AG 50

network statistics can be affected due to the network load caused by PERFMON itself. Other network-based applications, such as PCAnywhere, also create a significant network load that may invalidate any network statistics measured while connected.

Usage

• Verify that the Performance Monitor is installed.

• Set up the counters and the log file (modify/set the log file and chart settings)

• Ensure that no other services or programs are running that may impact the measurement (such as programs causing network or CPU load).

• Perform the measurement

• Extract the relevant counters (export them to a file)

• Calculate the relevant quantities

• Interpret the results

For further details, go to www.microsoft.com � Support Knowledgebase and see the White Paper Measuring performance-relevant data using PERFMON on Windows NT.

In Windows NT/2000 start PERFMON by choosing Start � Run and enter PERFMON. Handling Procedures

Set up PERFMON as follows.

1. Measure the CPU utilization on the server or PC. From the PERFMON menu, choose Edit � Add to Chart / Add Item Select in PERFMON: Object, Processor, Counter, % Processor Time

2. Determine the size of dataset transferred for analysis. Select in PERFMON: Object, Network Interface, Counter, Bytes Received/sec Note: Depending on the network connection, the name of the category and item of the above-mentioned examples may differ. When selecting the network interface counter it is possible that various instances (= lines) are offered. If you are not sure on which network line the traffic takes place, choose all instances. You may have to start processes at the NT level monitoring network traffic.

3. To start your analysis, monitor during the peak time when Mobile Clients are uploading to the CRM Server. To save the performance data and transfer it into a spreadsheet program, choose File � Export Chart.

4. Evaluate your analysis.

2.3.19 SAP Connect Monitor – SCOT

Functional description

SAP connect provides a standard interface for external communication, which supports sending using telecommunication services, such as faxing, pagers (SMS), Internet and X.400, as well as sending to printers and between several SAP Systems. It enables external communication components to be connected to the SAP System.


© 2002 SAP AG 51

As of Release 6.10, SAP connect also provides a direct connection to the Internet using the SMTP plug-in of the Web application server. This enables you to send messages directly to Internet, fax, and paging addresses without using an additional external communication system.

Transaction SCOT can be used for SAP connect administration.

Handling Procedures

SAP connect administration provides various views of your communications environment. Each view shows the environment from a certain viewpoint:

• Nodes / System Status: You see all nodes defined for your System and their status when you choose View > System status or View > Node. Choose the DISPLAY Button to view the node data.

• Jobs: You see all Jobs defined for sending mails of faxes from your System. The list shows for each job the time for the next run and the last runs that transmitted mails or faxes. When you mark one entry and chose the DISPLAY button you jump into the job details where you can find the job logs.

• Routing: The view of routing (View > Routing) shows, for each communication method, which address areas are processed by which node. The display also tells you if a communication method is not available in your system or is processed using another communication type.

• Alert Monitor: Choose Utilities > Alert monitor > Start data collection method to start the data collection for the monitor and then Utilities > Alert monitor > Display to see the results of this collection. (Option: Integration of the Alert Monitor into CCMS)

• Send orders: to get an overview of the send orders choose Utilities > overview of send orders. You can specify a time period, the send status or the sender.

Field Names Descriptions

Status Status of send process (i.e. transmitted, waiting, send, errors,…)

Trans. Method Transmission Method (i.e. via Telefax, via Remote SAP, via Internet,…)

Doc. Title Document Title

Sender Sender

Recipient Recipient

Send date Send date

Send time Send time

Msg. Message: Link to further details like status text, long text and transmission history

(Double-click on the entries to see more details.)

For error analysis, you can use the trace functionality by choosing Utilities > Trace > Internal Trace. Here you can set the trace for inbound or outbound direction and display results.

2.3.20 SAP Phone Administration – SPHB


SAPphone integrates telephony functions in SAP applications. This allows data to be exchanged between computer and telephone processes.


© 2002 SAP AG 52

Third-party manufactures can program a direct connection to the RFC interface or use the SAPphone server delivered with the SAP system. The SAPphone server serves as a nexus between the SAPphone RFC interface and the TAPI standard interface that Microsoft has defined for telephone integration solutions. If the SAPphone server is used, manufacturers of TAPI-compatible telephony products do not have to program an individual connection to the SAPphone RFC interface.

The functions for starting and monitoring the telephony environment are available in transaction SPHB. The existing work centers are displayed there for each telephony server in the navigation tree. You can also display other information about each telephony server. In the standard view, the telephony servers are displayed with name and description, the work centers with extension and user name of the work center user.

Handling Procedures

To enable communication the SAP System and the SAPphone server (or the self-programmed gateway software) you need a configured RFC connection between them. The following settings are required:

• For outbound calls, an RFC destination should be maintained in transaction SM59

• For inbound calls, an RFC destination should be maintained in the Saprfc.ini or the gateway software

You can see the name of RFC connection for the corresponding telephony server in transaction SPHB in tab strip ‘Server attributes’. By double-clicking on the destination name you can jump into the destination settings in SM59 and perform e.g. connection test or maintain required settings.

• Trace Files: Choose Utilities > Trace > internal Trace. Here you can switch the trace on or off and display the trace files.

• Alert Monitor: Choose Utilities > Alert Monitor > Display or start collection method. After starting the collection method you can view the alert monitor. (Option: Integration of the Alert Monitor into CCMS)

• SAPphone version: Choose Utilities > SAPphone version to get the SAPphone version

2.3.21 Internet Communication Manager Monitor – SMICM


Use this transaction to monitor and administrate the ICM (Internet Communication Manager). The ICM receives and sends requests (i.e., in a server role, incoming HTTP requests) from or to the internet. The SAP Web Application Server can serve both as a (WEB) server and as a client. As a server, ICM accepts incoming HTTP requests and forwards them to the Internet Communication Framework for processing. As a client, requests (i.e. SMTP e-mails) are sent from the SAP server via ICM to any Internet server (i.e. Mail server).

Handling Procedures

Status: you can check the status of Internet Communication Manager by calling transaction SMICM. The ICM status should be "runs" with green lights.

Services: Choose Goto > Services to obtain the service list (e.g. SMTP, HTTP(S)) with corresponding ports. The column ‘Active’ shows you whether the service is running.

Trace Files: Choose Goto > Trace file or Trace level. Here you can display or reset the trace file and set the trace level (values between 0 and 3 are permitted, default is 1). For normal operation,


© 2002 SAP AG 53

the ICM trace level should be set to 1. Higher trace levels should only temporarily be used during active error monitoring.

2.3.22 Gateway Monitor – SMGW


Analyzing the gateway trace gives you information about the RFC connection between the CRM Server and the Communication Station.

Handling Procedures

Check if the connection is broken or still existing and check for CPIC errors.

2.3.23 R/3 Buffer Monitor – ST02

Functional Description The Buffer Monitor shows the current status and the memory resource usage for a specific R/3 application server.

As part of the SAP Basis, the Buffer Monitor does not display any CRM specific monitoring data.

It should be analyzed using the same procedures as those used in a non-CRM system environment.

General Threshold Values - ST02

Name Threshold value Hit ratio (PXA, TABLP, EIBUF) >= 98% Swaps (PXA) <= 10.000 Swaps (other than PXA) = 0 Roll area “max used” < “in memory” Extended memory <80% of “in memory”

2.3.24 Workload Monitor – ST03N

Functional Description The Workload Monitor is used to analyze statistical data from the R/3 kernel. When analyzing the performance of a system, you should normally start by analyzing the workload statistics. For example, you can display the totals for all instances and compare the performance of individual instances over specific periods of time.

Workload� choose ‘Expert mode’: Check the time of day when the longest wait times are listed. Check on both CRM Server and on the R/3 systems.

You can use the workload monitor to display the:

• Data per application instance

• Data for all application instances, not just the one you have logged on to

Further you can use the Analysis Views to display:

• Transactions used and the users that call them


© 2002 SAP AG 54

• The time profile

• The ranking list

• The RFC profiles

General Threshold Values - ST03

Name SAP Threshold values Dispatcher wait time <= 50 ms Database time < 40% of (response time - dispatcher time) Processing time < 2 x CPU time Avg. response time < 1000 ms (for records without RFC-subrecords)

Handling procedure As part of the SAP Basis, the Workload Monitor does not display any CRM specific monitoring data. However the RFC response time statistics should be monitored relative to the RFC communications between the R/3 back-end system and the CRM Server.

In general, it should be analyzed using the same procedures as those used in a non-CRM system environment.

Monitor online transactions by using called transaction detail profiles If you are implementing Interaction Center and/or CRM Online Scenarios, you have to check the response times for transaction CIC0 or for the corresponding CRM Online transactions.

In order to be able to monitor these transactions by using the so-called transaction detail profiles we recommend to configure them in the transaction ST03(N). Recommendation: 1. Call transaction ST03(N) 2. Change the user mode to 'Expert mode' 3. In upper launch pad choose 'Collector and Performance DB' -> 'Workload Collector' -> 'Statistics to be Created' 4. In section 'Create transaction detail profiles for' enter the transactions you would like to have analyzed (e.g. CIC0). 5. Click button 'Save values'

Afterwards, you can display and analyze collected data in Transaction Profiles -> Details by double-clicking on the corresponding transaction name.

Analyzing RFC Profiles In all CRM Scenarios, large part of processing in CRM is done by RFC. The RFC statistics should be therefore analyzed in detail.

Choose: RFC Profiles, you can check client (sender) and server (receiver) records in detail. Keep in mind, that QRFCs and TRFCs are processed as function ARFC_DEST_SHIP. All other RFCs are displayed by function module in this statistics. You can double –click to get details of the call type. Select the function module and look into SE37 to get the short description.

• Check to see which functions are time consuming or very often executed or transport high mount of data.

2.3.25 Database Performance Analysis – ST04



© 2002 SAP AG 55

The Database Performance Monitor does not display any CRM specific monitoring data.


Exception: Check the indices on the tRFC and aRFC tables (ARFC* and TRFC*). For MSA/MSE: monitor the disk I/O in high-load phases to find possible bottlenecks.

See also transaction DB02.

Refer to R/3 Monitoring Best Practice.

2.3.26 Operating System Monitor – ST06

Functional Description Transaction ST06 for monitoring the available swap space in the host system.

Compare the hardware load for times of high RFC transmissions with the times of “normal” operations.

You can detect the following bottlenecks from the following values:

• CPU bottleneck by CPU idle time of less than 5% or a CPU load greater than 3% A very high load and a CPU idle time of about 50% normally indicates an I/O bottleneck.

• Memory bottleneck

• UNIX page-out rate greater than 60 000 pages/hour

• NT page-in rate greater than 60 0000 pages/hour

• High file system utilization and wait times can indicate a disk bottleneck.

• A high number of collisions can indicate network problems on your shared Ethernet network.

General Threshold Values

Name Threshold Value CPU idle >= 20%, or better >= 35% (UNIX) page-out rate <= 60 000 page/h (NT) page-in rate <= 60 000 page/h

2.3.27 Local and System-Wide Work Process Overview – SM50 / SM66

Functional Description This transaction enables you to detect long running transactions. Check it occasionally or if you suspect that there is a sudden performance bottleneck.

Threshold Values - SM66/SM50

Name Threshold Value Work processes in PRIV mode > 20 % of all WP (memory management

problem?) Work processes in database action > 40% of all WP (database problem?) Work processes reading the same table at same time

> 20% of all WP (exp. SQL or excl. lock waits?)


© 2002 SAP AG 56

By monitoring work processes in the SAP System (transaction SM50) or from outside the system (program DPMON at operating system level), you can determine the status of the work processes in relation to the PRIV mode. If work processes are often switched to the PRIV mode, you must increase the extended memory and/or adjust the limit for the extended memory). In this case, you must consult with SAP.

• You can use transaction SM66 to evaluate this information for all the work processes.

• You can determine the swap space requirements at R/3 and operating system level.

• Many monitoring tools are operating system dependent.

Transaction SM66 shows the SAP System work process overview for the local application server and for all other SAP application servers that are connected to the same SAP database (such as all application servers connected to the database and central SAP instance).

Usage With help of this transaction you can detect long running transactions. Monitor periodically, and also check if there are sudden performance problems.

Handling Procedures SM66 – check for ARFC programs running

Make sure that there are enough work processes free for online users, if all processes are blocked, check the RFC parameters.

Are there work-processes that are stopped or locked by CPIC to determine whether there are enough work processes configured for the system?

In SM50, ARFC jobs show up as entries for the background work processes of rescheduled tRFCs..

2.3.28 Display Statistical Records – STAD

Functional Description Shows who has done what and when.

Expansion

• RFC statistic

• Time analysis

Check whether it says “no subrecords”

Check “RFC subrecords” to see who has done what.

2.3.29 ABAP Dump Analysis – ST22 Refer to partner systems for short dumps.


© 2002 SAP AG 57

2.3.30 System Log – SM21

Check the syslog for error messages referring to RFC and gateway with message number Q23 and Q22.

Example: Deletion of single message in the queue (here in the outbound queue) (Q23, Q26)

Delete a single message, with TID 0A1089630DEC3D491F782EA9 The following system log entry is created 15:32:09 DIA 3 800 D036792 SMQ1 Q23 Missng:TSL1TE,Q23):D036792&0A1089630DEC3D491F782EA9 Error in SM21: Q23, user deleting and from which transaction as well, which client If something is deleted in system log

Technical details File 170136 Position 0000329040 Entry type Message ID Q2 3 Variable parts D036792&0A1089630DEC3D491F782EA9&CRM_SITE_000000000000100,CRM_SI


© 2002 SAP AG 58

Example: Deletion of whole queue (here from the inbound queue) (Q22, Q25)

If a whole queue is deleted, this information is displayed (the TID are not shown anymore): 15:40:14 DIA 3 800 D036792 Q22 Missng:TSL1TE,Q22):D036792&CRM_SITE_000000000000100 Example: Deletion of a single message from the R&R Queues (CM1)

Example: deletion of all messages from the R&R Queues (CM1)

Client information is missing on this screen, but it is in the process. Example: Release and hold R&R Queues (CM0)

2.3.31 Update Error Monitor – SM13

Functional description The Update Error Monitor does not display any CRM specific monitoring data.



© 2002 SAP AG 59

2.4 Troubleshooting This chapter describes problems that can arise in the CRM environment and helps handling and solving these problems.

The possible problem solving strategies are described either as flow diagrams or as problem descriptions.

2.4.1 Problems during Data Exchange Problems and errors during data exchange can occur at several points in the CRM landscape. Unsuccessful data transfers may have one of the following causes:

• System capacity overload

• Program crashes due to database problems, e.g. tablespace overflow

• RFC connection problems (network related)

• Incorrect middleware specific configuration settings

• Incorrect application specific configuration settings

Whatever the reason, the exact cause of the problem must be determined and the problem fixed before the data transfer can be properly completed. The download or upload of objects should only be restarted after the errors have been found and repaired.

The following problem analysis diagrams should be used as guides in correcting errors occurring during the initial download, during delta downloads and in troubleshooting upload problems. The basic recommendation is to first ensure that the download works properly before trying to correct any upload problems. Middleware specific situations are covered in the following flow diagrams - the CRM application settings may need to be handled by the application personnel.

2.4.1.1 Error Detection in Delta Download If you experience problems with delta load of some objects, you can refer to the flow diagram below. This is a logical sequence of checks to analyze the problems and helps to determine and to eliminate the causes of the errors.

For further information, refer to:

• SAP Note 430980 and/or to

• SAP Library CD – Error Detection in Delta Download.


© 2002 SAP AG 60

➼ Table TBE11 Parameter: BC_MID, ACTIVE: X Parameter: NDI, ACTIVE: X

NO

YES

NO

THE DELTA DOWNLOAD OF SOME OBJECTS IS NOT SUCCESSFUL - R3AM1 on the CRM server shows objects with

- Status RUNNING - Date, Time, and Block Number remaining constant for longer than 5 minutes

OLTP System?

Event Control Active?

Parameter Settings OK?

➼ Table CRMCONSUM CONSUMER: CRM (or consumer application, e.g. BBP) ACTIVE: X TEXT: CRM (or a descriptive text)

➼ Table CRMRFCPAR CONSUMER: CRM (see table CRMCONSUM) OBJNAME: * RFCDEST: <RFC Dest. of CRM System> DOWNLOAD: * INACTIVE: <blank> DISCARDDAT: <blank> USE_IN_Q: X, SEND_XML: X HOLD_DATA: <blank>

(CONTINUED ON THE NEXT PAGE)

Error Detection in Delta Download

Note 430980


© 2002 SAP AG 61

(Continued from the previous page)

Check RFC Communication

Queues CRM

➼ SMQ1 Check R3AD* queues for errors

➼ SMQ2 Check R3AD* queues for errors

OLTP

YES?

NO? ➼ CRM System – ST22

➼ OLTP System – ST22 Analyze and fix problems

Short Dumps Exist?

YES?

NO?

STOP Entries in the CRM

Inbound Queue?

➼ SMW01: Analyse FLOW information on CRM system

YES?

NO?

Error messages from CRM Services in

FLOW?

Refer analysis to application people




© 2002 SAP AG 62

YES

NO

YES?

NO?

➼ OLTP – SMQ1

➼ CRM – SMQ2 Restart the queues manually

➼ CRM – SMW01 Release queue (manually unblock queue)

YES

NO

Delta data passed-on from OLTP to CRM

System?

➼ CRM - R3AM1 Ensure that all objects have GREEN status

➼ OLTP – SMQ1 Ensure that all initial download objects are completed

Initial Download completed

successfully?

Events for all application

relevant objects activated?

➼ CRM - R3AC4 Activate the missing entries manually by using change mode and then save





© 2002 SAP AG 63

(Continued from the previous page) Error Detection in Delta Download

YES

NO

➼ CRM – SMQR All R3AD* queues must have “R“ in Type column

R3AD* Delta queues registered for processing?

YES

NO➼ CRM – SM30 – table SMO8_LQU Delete the lock entries

➼ CRM – SMQ2 Manually restart the queue(s) (ACTIVATE)

➼ OLTP – SMQ1 Check for possible follow-on problems

➼ ST22 Analyze possible short-dumps

Do LOCK entries exist for inbound queue entries in table SMO8_LQU?

YES

NO

➼ CRM – R3AC1 Delete unwanted filters that may be preventing an object to download from OLTP Check with the application people before deleting any filters!

Do FILTER criteria

exist for objects that have not been downloaded?



© 2002 SAP AG 64

YES

NO

➼ SMO8FD Check whether the Message Flow has been generated.➼ GNRWB Generate the Message Flow for all objects

Has a Message Flow been

generated for the application

relevant

(Continued from the previous page) Error Detection in Delta Download

YES

NO

➼ CRM – SMQ2 Check for errors

➼ CRM – Production client – table SMOFPARSFACheck for BREAK-POINTS (parameter BREAK_POINT) Delete the BREAK POINTS or deactivate them by entering a non-existent user-name for each

Does the CRM Inbound queue stop

after processing each object?

YES

NO

Call SAP Support

Are there still Delta

Download problems?

DONE


© 2002 SAP AG 65

2.4.1.2 Error Detection in Initial Download References:

• SAP Notes 429423 and 350176

• SAP Library CD Error Detection and Correction


Error Detection in Initial Download

YES

NO

➼ Check SAPNet for newest Support Package releases- HTTP://Service.SAP.com/swcenter-main - Upgrade to newest versions

Are the newest Support Packages installed

- R/3 Plug-In? OLTP System? CRM System?

THE INITIAL DOWNLOAD WAS NOT FINISHED SUCCESSFULLY - R3AM1, Download Monitor, shows non-GREEN status for some objects

- R3AM1, Download Monitor, shows GREEN status but no Objects are downloaded

- SMW01, BDoc Display, shows error segments for downloaded objects

OLTP System Parameter

Settings OK?

➼ Table CRMCONSUM CONSUMER: CRM (or consumer application, e.g. BBP))ACTIVE: X TEXT: CRM (or a descriptive text)

➼ Table CRMRFCPAR CONSUMER: CRM (see table CRMCONSUM) OBJNAME: * RFCDEST: <RFC Dest. of CRM System> DOWNLOAD: * INACTIVE: <blank> DISCARDDAT: <blank> USE_IN_Q: X, SEND_XML: X HOLD_DATA: <blank>

➼ have you maintained the logical system of R/3- and CRM-System

NO

YES

Tablespace problems?

➼ CRM – SM21 the system log shows the affected tablespaces

➼ Extent tablespace NO

YES


© 2002 SAP AG 66

YES?

NO

➼ Test RFC connection between the systems OLTP System �� CRM System CRM System �� OLTP System

➼ CRM - SM58 check for errors in tRFC

Does the RFC connection

between systems work?

YES

➼ R/3 - SMQ1 check for errors, restart queue in case of capacity overload

➼ CRM- ST22 check for short dumps

Are there R3AI* queues waiting on the OLTP system?

NO?

NO

YES

➼ CRM – SMQ2 check for errors, often there are customizing errors

➼ CRM - SMW01 analyze errors, correct the customizing settings

➼ CRM – SMQ2 Restart the processing of the queue

➼ Check note 309734

Are there R3AI* (R3AMATERIAL) inbound queues

on the CRM system?

➼ CRM – SMQ2 restart the queue

➼ CRM - SMQR check registration of queue

Are there Entries with error

messages? NOYES

(Continued from the previous page) Error Detection in Initial Download



© 2002 SAP AG 67

Furthermore, there are several other problems related to the initial download:

- R3AM1: the Download Manager shows that some objects remain in status WAIT: The reason is that this object has a parent object, and the initial load of the parent object was not yet completed successfully. Whether or an object has a parent object can be seen in transaction R3AM1 by the "HasPar" indicator. When it is set to "X", then the respective object has parent objects. These parent objects can be seen in table SMOFOBJPAR.

- R3AM1: the Download Manager shows only few Objects in status RUNNING: The maximum allowed number of loads and requests is already being processed. This number can be set in table SMOFPARSFA via the MAX_PARALLEL_PROCESSES parameter (default = 5). If the number of running loads and the number of running requests in total is as much as or higher than this number, the remaining loads have to wait until a process is free again.

- The R3AM1 transaction shows that an object from the initial download has a status of RED The initial load has not finished successfully.

Analysis Errors occurred in the CRM System during the initial download and as a result the data was not posted into the CRM database. In order to be able to restart the data from the queue after elimination of the error an exception was probably triggered. Normally, the problem is caused by missing/incorrect customizing of the CRM online application.

Handling: See SAP notes 429423, 350176

The reason why the initial load was not successful must be determined. Error messages and traces should be examined to see where the problem(s) might be.

Use transactions SMW01 (BDoc Viewer) or SM08FT (Flow Trace) to view the error messages

Correct the error based on the error message text

Use transaction SMQ2 to restart the queue

Use a similar procedure for subsequent errors


YES

NO

Call SAP Support

Are there still Initial

Download problems?

DONE

Error Detection in Initial Download


© 2002 SAP AG 68

Do not a complete restart of the initial download. The download will resume from the point where it stopped after the error has been corrected.

Customizing of the CRM online application as specified in the “SAP CRM Setup and Load Guide” may have to be done to avoid subsequent errors.

2.4.1.3 Error Detection in Upload References:

• SAP Note 431345

• SAP Library CD Error Detection in Upload


© 2002 SAP AG 69

Error Detection in Upload

THE UPLOAD OF SOME OBJECTS HAS NOT FINISHED SUCCESSFULLY- SMW01 shows YELLOW or RED status for some objects

- SMW01 shows GREEN Status, but the upload causes no changes

Does the Delta Download

➼ R/3 change i.e. the address of a business partner

➼ Analyze errors using Error Detection in Delta Download

NO

YES

YES

NO

Are there error messages?

➼ CRM – SMW01 analyze all entries marked red or yellow find out which steps the message has already run through

➼ CRM - SMQ1, SMQ2 check for dependant queue entries

➼ R/3 – SMQ1, SMQ2 check for dependant queue entries

➼ CRM + R/3 – ST22 check for short dumps

➼ CRM – SMQ1 Debug LUW (Note 337753)



© 2002 SAP AG 70

Did the upload cause changes?

(Continued from the previous page) Error Detection in Upload

➼ CRM – SMW01 analyze relevant entries

➼ CRM – SMQ1 analyze similar message by using Debugging (Note 337753)

YES

NO

Call SAP Support

Are there still Upload

problems?

DONE

YES

NO


© 2002 SAP AG 71

2.4.2 Problems with E-Mail Sending (Outbound Direction)

Error Detection for sending Mails via SMTP

NO MAILS GOING OUTSIDE

SMTP Plugin active?

➼ Check the profile parameters – RZ10:

• icm/plugin_x: PROT=SMTP,PLG=/usr/sap/<SID>/SYS/exe/run/smtpplugin.dll

• Icm/server_port_x: PROT=SMTP,PORT=8025 (default Port)

➼ Check path for the Plugin (AL11)

NO?

YES?

NO?SMTP node active?

➼ Check the settings for the SMTP node – SCOT:

• Mail host and Mail port correct

• Address area correct

• Node active

Send job running?

YES?

NO?

➼ Check the send job – SCOT:

• View > Jobs

YES?


© 2002 SAP AG 72

2.4.3 CRM Server Performance problems or high I/O load due to excessive traces and logs being written

The performance of the CRM server is poor and may have some of the following symptoms

• Slow response times

• High I/O load

• Disk space increases relatively fast

• Great data quantities are written to the database logs

• Database is in a standstill condition

• The Middleware Log and Flow control Trace tables are larger than 100 MB (transaction DB02 can be used to check the size of the tables)

Affected tables:

Error messages in CRM?

YES?

➼ Analyze errors -SCOT:

• Alert Monitor, send Orders (see above)

NO?

Are there still problems while

sending?

➼ External Software: Look for error messages on the mail server)

➼ Call SAP Support

YES?

DONE

NO?



© 2002 SAP AG 73

SMW3_BDOC, SMW3_BDOC1, SMW3_BDOC2, SMW3_BDOC4, SMW3_BDOC5, SMW3_BDOC6,

SMW3_BDOC7 (Tables for Administration of BDoc messages)

SMWT_TRC (Table for Middleware Trace).

Analysis

Possible causes

• High trace level settings are activated on the CRM server

• The log level may be set incorrectly

Handling

Check the relative sizes of the above listed tables using transaction DB02 � Detailed Analysis � Tables and Indexes

Check the following traces, optimize their settings, and schedule a periodic reorganization of the tables as described in note 206439:

As of Release 3.0: Set default trace level '1' for all trace environments except for 'G' (generation) of the Middleware trace. For this purpose, choose 'Middleware' -> 'Administration' -> 'Define Middleware Parameters' (R3AC6) from the SAP Menu and check entries with the following structure in the displayed table (SMOFPARSFA):

Column Value

Key CRMGENERAL

ParamName TRACE-LEVEL

ParamName2 ENV=m

Param.Wert LEVEL=n

Where m is the trace environment and n the trace level that can be a number between 1 and 4. If this is higher than 1 in certain entries, except for those where the 'ParamName2' column contains the string 'ENV=G' (trace environment generation), set the default value 1 and then save the changes. The default value of the trace level for the trace environment generation (the entry where the ‘ParamName2' column contains the string 'ENV=G') is 4.

Scheduling the reorganization Reference SAP Note 206439

Create a background job MW_REORG using variant SAP_MW_REORG and schedule the SMO6_REORG program as periodic job for execution every night.

When you use the variant SAP_MW_REORG all trace and log data that is older than a week will be deleted. If you want to keep the data for a longer or shorter period create a variant for the SMO6_REORG report via Transaction SE38 and use it instead of SAP_MW_REORG for the job scheduling.


© 2002 SAP AG 74

2.4.4 Mass changes of data (creating, changing) on the OLTP system leads to reduced system performance

Due to all of the dialog and background work processes becoming occupied the OLTP system has poor performance.

Analysis: The outbound queue (SMQ1) of the OLTP system shows that there are between several hundred and several thousand different queues waiting to be processed. A large amount of the changed data from the OLTP system is intended for the delta download to the CRM Middleware Server and is therefore stored in the outbound queue of the OLTP before being transferred to the middleware server. For performance reasons qRFC tries to transfer as many queues as possible, as fast as possible. The result is that many parallel queues (with different queue names) are generated.

The total number of queues that can be generated for parallel processing is limited by the possible names that can be assigned to the queues.

Handling See SAP Note: 356228.

2.4.5 Performance problems due to statistics updates on tRFC/qRFC tables

This applies only to SAP Systems using the Oracle database.

Performance problems on the tRFC and qRFC tables

ARFCSSTATE ARFCSDATA ARFCRSTATE TRFCQDATA TRFCQIN TRFCQINS TRFCQOUT TRFCQSTATE

Analysis The size of the tRFC and qRFC tables can vary for applications (such as CRM) that frequently use this function. For databases that use a cost-based optimizer and do not keep the table statistics current, incorrect access paths may be chosen. This can cause a high database load or lead to excessive runtimes.

Handling Update statistics should be switched-off (and deleted) for the database tables listed below

Transaction DB21 should be used.

For details see SAP Notes 330059 and 371068.

2.4.6 System performance degrades as the size of the tRFC/qRFC tables increases

There is a high system interface load because 1 or more of the tRFC/qRFC tables have become too large.

Affected tables:


© 2002 SAP AG 75

ARFCSSTATE ARFCSDATA ARFCRSTATE

TRFCQIN TRFCQOUT

TRFCQSTATE TRFCQDATA

Analysis

1 or more of the tRFC and qRFC tables have become bigger than 500 MB. The large tRFC and qRFC tables affect the performance of the tRFC and the qRFC interfaces.

Handling Use transaction DB02 – Detail Analysis to check the relative sizes of the tables.

Reorganize these tables regularly to avoid future performance problems. Reorganization will decrease the sizes of the tables.

Refer to the SAP Note 375566 for details concerning administration of these tables

Note: Never delete the contents of this tables otherwise data inconsistencies might occur.

2.4.7 Problem situation 1

Roll-Out of new mobile clients with productive mobile client. Organization of the Roll-Out timeframe to avoid performance problems. No parallelization of the R&R is possible.


Do not restart a new Initial load


BDOC Fehlerhandling procedure, consistency of data


Mobile Brdge activation and deactivation


© 2002 SAP AG 76

Checklists SAP Basis (CRM Server and R/3 Back-end Server)

SMGW

ST02

ST03N

ST04

DB02

ST06

SM50

SM66

ST22

SM21

SM13

STAD

R/3 Back-end System

SMQ1

SMQS

BGD --

CRM Server

SMWP

SMQ1

SMQS

SMQ2

SMQR

SM58

SMWMFLOW

SMW01/SMW02

SMWT

SMW03

SMO8FD

R3AM1

R3AR3

SMWMQUEUES

SMOHQUEUE

SMWMCOMM

CMWQ


© 2002 SAP AG 77

Feedback and Questions Send any feedback by composing an SAP customer message to component SV-GST. You can do this at http://service.sap.com/message.

© Copyright 2001 SAP AG. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft®, WINDOWS®, NT®, EXCEL®, Word®, PowerPoint® and SQL Server® are registered trademarks of Microsoft Corporation. IBM®, DB2®, OS/2®, DB2/6000®, Parallel Sysplex®, MVS/ESA®, RS/6000®, AIX®, S/390®, AS/400®, OS/390®, and OS/400® are registered trademarks of IBM Corporation. ORACLE® is a registered trademark of ORACLE Corporation. INFORMIX®-OnLine for SAP and Informix® Dynamic ServerTM are registered trademarks of Informix Software Incorporated. UNIX®, X/Open®, OSF/1®, and Motif® are registered trademarks of the Open Group. HTML, DHTML, XML, XHTML are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology. JAVA® is a registered trademark of Sun Microsystems, Inc. JAVASCRIPT® is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape. SAP, SAP Logo, R/2, RIVA, R/3, ABAP, SAP ArchiveLink, SAP Business Workflow, WebFlow, SAP EarlyWatch, BAPI, SAPPHIRE, Management Cockpit, mySAP.com Logo and mySAP.com are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other products mentioned are trademarks or registered trademarks of their respective companies. Disclaimer: SAP AG assumes no responsibility for errors or omissions in these materials. These materials are provided “as is” without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP shall not be liable for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. SAP does not warrant the accuracy or completeness of the information, text, graphics, links or other items contained within these materials. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third party web pages nor provide any warranty whatsoever relating to third party web pages.

SAP CRM Monitoring

Documents

Transcript of SAP CRM Monitoring