Integrating IBM Tivoli Monitoring with Tivoli Netcool/OMNIbus Multi-Tier Architecture V1.0 Stefania Oliverio & Randall Allen Introduction Integrating ITM and OMNIbus with a Multi-Tier Architecture presents the administrative user with certain challenges that would not be encountered when integrating ITM with a simple, single-tier OMNIbus installation.The flow of events from one layer to the next requires that configuration steps specific to each layer and each OMNIbus system be performed in order for events to properly be synchronized between ITM and OMNIbus installs. (Correct event sharing, updating, deleting etc.) Additional steps are needed when there are OMNIbus failover pairs in the mix as well. This paper describes the main concepts and steps necessary to configure event sharing between IBM Tivoli Monitoring (ITM) and Tivoli Netcool/OMNIbus Multi Tier Architecture.These instructions assume that ITM 6.2.3 or later versions and OMNIbus7.3.1 have been deployed in your environment and that the multi - tier architecture has been configured with failover capabilities as described herein. While this paper covers all of the steps needed to configure this integration, it is by no means a complete guide and should be used in addition to the published OMNIbus and ITM documentation. Table Of Contents Environment Configuration .......................................................................................................................... 3 Multi-Tier Architecture Machine Details & Diagram.............................................................................. 3 Event flows between ITM and OMNIbus multi tiers architecture............................................................ 5 Configuring the omni.dat file (aggp, aggb, colp1, colb1, dis1, & dis2) ................................................... 6 Configuring hosts file on HUB TEMS and Aggregation Servers............................................................. 8 Aggregation Layer Configuration................................................................................................................. 9 Configure Process Control (aggp & aggb)................................................................................................ 9 Installing the ITM Event Synchronization component (aggp & aggb)................................................... 12 Configuring Bi-Directional Gateway (aggb) .......................................................................................... 17 Update bi-directional gateway map file on aggb................................................................................ 17 Update bi-directional gateway on aggb to replicate tables................................................................ 22 Update OMNIbus Data Base schema(aggp & aggb)........................................................................ 22 Updating the OMNIbus deduplication triggers (aggp & aggb) ......................................................... 23 Bi-Directional Gateway stop and restart (aggb) ................................................................................ 27 Collection Layer Configuration .................................................................................................................. 28 Update uni-directional gateway map file (colp1 & colb1) ..................................................................... 28 Update OMNIbus ObjectServer Database schema (colp1 & colb1)....................................................... 28 Configure probes (colp1 & colb1) .......................................................................................................... 29 Stop and restart colp1 and colb1 Gateways and Probes: ........................................................................ 29 Display Layer Configuration ...................................................................................................................... 30 Update uni-directional gateway map file (dis1 & dis2).......................................................................... 30 Update OMNIbus Data Base schema (dis1 & dis2) ............................................................................... 30 Stop and restart dis1 and dis2 Gateways:................................................................................................ 31 Install and configure OMNIbus Web GUI (TIP) on (dis1 & dis2)......................................................... 31 Configure WEBGUI for Active Event List............................................................................................. 37 Configure ITM TEMS to share situational events with the EIF probes ..................................................... 38 OMNIbus ObjectServer, Gateways, and Probes Commands Reference .................................................... 39 OMNIbus Start ObjectServer Commands: ............................................................................................. 39 OMNIbus Start Gateway Commands: .................................................................................................... 40 OMNIbus Start EIF Probe Command:.................................................................................................... 40 OMNIbus Process Control Commands:.................................................................................................. 40 Environment Configuration This white paper is based on a real deployment that can be taken as example and consisted primarily of virtual machines (VMs); the Tivoli Monitoring components were installed on VMs in the a domain specified later as domain1 and the OMNIbus Multi-tier Architecture was installed on VMs in the Integration Test Enablement (ITE) environment specified later as domain2.Tivoli Monitoring agents used to generate situational events were installed on a mix of physical machines and VMs in the domain1 domain. Multi-Tier Architecture Machine Details & Diagram Netcool / OMNIbus 7.3.1 was previously deployed for high availability using the multi-tiered architecture to increase performance and event handling capacity. This software was deployed in the default locations of /opt/IBM/Tivoli/netcool/omnibus on linux and C:\IBM\Tivoli\Netcool\OMNIbus on Windows. The OMNIbus multi-tier components were deployed on 6 different systems as described in the table below: ComponentSystem Installed ComponentsComponent name Primary Collection ObjectServerCOL_P_1 uni-directional ObjectServer Gateway that connects Primary Collection server to the Aggregation layer C_TO_A_GATE_P_1 Primary Collection ObjectServer Hostname: colp1 SUSE Linux Enterprise Server 10 Master probe nco_p_tivoli_eif Backup Collection ObjectServerCOL_B_1 uni-directional ObjectServer Gateway that connects Backup Collection server toAggregation layer C_TO_A_GATE_B_1 Backup Collection ObjectServer Hostname: colb1 SUSE Linux Enterprise Server 10 Slave probe nco_p_tivoli_eif Primary Aggregation ObjectServer Hostname: aggp Red Hat Enterprise Linux Server release 5.7 Primary Aggregation ObjectServerAGG_P Backup Aggregation ObjectServer AGG_B Backup Aggregation ObjectServerHostname: aggb Red Hat Enterprise Linux Server release 5.7 Bi-directional ObjectServer gatewayAGG_GATE Primary Display ObjectServer DIS_1 Primary Display ObjectServers Hostname: dis1Widows Server 2008 R2 uni-directional ObjectServer Gateway that connects Display Server toAggregation layer A_TO_D_GATE_1 Backup Display ObjectServer DIS_2 Backup Display ObjectServers Hostname: dis2 Widows Server 2008 R2 uni-directional ObjectServer Gateway that connects Display Server toAggregation layer A_TO_D_GATE_2 The following diagram details the standard multi-tier architecture as found in the IBM Netcool/OMNIbus InfoCenter: Display layer cloudAggregation layer (AGG_V)Collection layer pairDIS_1 DIS_2AGG_P AGG_BCOL_P_1 COL_B_1C_TO_A_GATE_P_1 C_TO_A_GATE_B_1A_TO_D_GATE_1 A_TO_D_GATE_2 All OMNIBUS components were installed using netcool user-id belonging to ncoadmin group: uid=500(netcool) gid=500(ncoadmin) groups=0(root),500(ncoadmin) Components were installed under the default OMNIBUS install directory as follows: /opt/IBM/tivoli/netcool For more information on the OMNIbus Multi-Tier Architecture consult the OMNIbus references or InfoCenter: http://publib.boulder.ibm.com/infocenter/tivihelp/v8r1/index.jsp?topic=%2Fcom.ibm.netcool_OMNIbus.doc_7.3.1%2FOMNIbus%2Fwip%2Finstall%2Fconcept%2Fomn_ins_multitieredhighavailability.html Event flows between ITM and OMNIbus multi tiers architecture The following picture describes the typical events flow. TEPSOmibus EIF ProbeCollection LayerDisplay LayerSituation Update ForwarderAggregation layerData Base TriggersStandby Hub TEMSHub TEMS12345678910 1.Events are generated on Hub TEMS and sent to EIF Probe2.ITM slots are mapped to ObjectServer attributes through ITM rules file and sent to Collection Layer 3.Collection Layer collects all events and alerts and send them to Aggregation Layer 4.Events are inserted into the ObjectServer or ignored in case of loopback events by using IBM Tivoli Monitoring provided triggers 5.Events are sent to Display Layer and displayed in the OMNIbus Web GUIActive Event List (AEL). 6.Events can be acknowledged, de-acknowledged, deleted or cleared. 7.Using IBM Tivoli Monitoring provided triggers Aggregation Layer forward events status changesto the Situation Update Forwarder 8.Situation Update Forwarder sends changes to Hub TEMS using a SOAP request 9.Status changes are propagated through the Tivoli Enterprise Portal Server and shown in the Tivoli Enterprise Portal 10.The complete list of open, acknowledged and de-acknowledged events is shown in the Situation Event Console workspace of the Tivoli Enterprise Portal Broswer/Desktop. Configuring the omni.dat file (aggp, aggb, colp1, colb1, dis1, & dis2) Each of the OMNIbus installations should utilize the same omni.dat file (detailed below). After configuring this file ononesystemitshouldbecopied toeachoftheother5machinesonwhichthedifferentOMNIbuscomponents havebeeninstalled.(Formoreinformationabouttheomni.datfileorOMNIbusinterfacesconsulttheNetcool OMNIbus InfoCenter.) The following is the omni.dat configuration file(/opt/IBM/tivoli/netcool/etc/omni.dat) used in this environment: # # omni.dat file as prototype for interfaces file # # Ident: $Id: omni.dat 1.5 1999/07/13 09:34:20 chris Development $ # [AGG_P] { Primary: aggp 4100 } [AGG_B] { Primary: aggb 4100 } [AGG_V] { Primary: aggp 4100 Backup: aggb 4100 } [AGG_GATE] { Primary: aggb 4300 } [COL_P_1] { Primary: colp1 4100 } [COL_B_1] { Primary: colb1 4100 } [DIS_1] { Primary: dis1 4100 } [DIS_2] { Primary: dis2 4100 } [C_TO_A_GATE_P_1] { Primary: colp1 4300 } [C_TO_A_GATE_B_1] { Primary: colb1 4300 } [A_TO_D_GATE_1] { Primary: dis1 4300 } [A_TO_D_GATE_2] { Primary: dis2 4300 } [NCO_PA] { Primary: aggb 4200 } [NCO_PROXY] { Primary: aggb 4400 } Please note the [AGG_V] definition in the file. It is the Virtual Aggregation pair definition. All incoming CollectionGatewayconnectionsandalloutgoingDisplayGatewayconnectionsconnecttotheVirtual AggregationpairAGG_VsothattheycanfailoverandfailbackoncetheprimaryAggregation ObjectServer.

Configuring hosts file on HUB TEMS and Aggregation Servers Tivoli Monitoring and OMNIbus have been installed in two different Network domains as shown in picture below. In ITE environment machines have two network cards and hostnames are not resolved by DNS: Backup Hub TEMSnc049044Primary Hub TEMS nc049043Remote TEMSITM/ITCAM agents.domain1.ibm.com9.x.x.xBackup Agg ServeraggbPrimary Agg ServeraggpBackup Col Servercolb1Primary Col Servercolp1Backup Dis Serverdis2Primary Dis Serverdis1ITE .domain2.ibm.comInternal IP: 192.x.x.xExternal IP: 9.x.x.xProbes For this reason hosts files on Hub TEMS and Hot Standby Hub TEMS need to include entries for systems where probes are running: 9.x.x.1 colp1.domain2.ibm.com colp1 external IP 9.x.x.2 colb1.domain2.ibm.com colb1 external IP and hosts files on Aggregation Servers where ITM Event Synchronization components are running need entries for the two Hub TEMS systems: 9.x.x.3nc049044.domain1.ibm.comnc049044 9.x.x.4nc049043.domain1.ibm.com nc049043 Considering no DNS resolution is in place hosts file on OMNIbus systems need to list all machines and this is the reason why later in this document you see commands run with short hostname:

192.x.x.3aggp.domain2.ibm.com aggp192.x.x.4aggb. domain2.ibm.comaggb 192.x.x.1colp1. domain2.ibm.com colp1 192.x.x.2colb1. domain2.ibm.com colb1 192.x.x.5dis1. domain2.ibm.com dis1 192.x.x.6dis2. domain2.ibm.com dis2 Aggregation Layer ConfigurationConfigure Process Control (aggp & aggb) OMNIbus Server components on the Aggregation Layer must be configured under process control (PA) to allow bi-directional synchronization between OMNIbus and Tivoli Monitoring.Process control manages the external procedures that communicate with the ITM Event Synchronization component installed in the next section. The steps below would be used to configure the OMNIbus ObjectServer and bi-directional Gateway on the aggb box but the same steps can be followed to configure the OMNIbus ObjectServer on the aggp box as well. 1.Login to aggb with netcool user. (Make sure netcool user is a member of linux group ncoadmin.) 2.Edit /opt/IBM/tivoli/netcool/omnibus/etc/nco_pa.conf 3.Under the List of processes section modify the MasterObjectServer entry, on the Command line, changethe run as value to the uid value for the netcool user (500). 4.Next, change the Host value from omnihost to the hostname of the system (aggb).The entry should now look similar to the following: Command '$OMNIHOME/bin/nco_objserv -name AGG_B -pa NCO_PA' run as 500 5.Add an additional nco_process entry for the bi-directional OMNIbus Server Gateway, calling the OMNIHOME/bin/nco_g_objserv_bi command. 6.Under the List of Services section, change the Core service entry, set ServiceStart to Auto, and add the processes created above.7.Save the file. The following is the file as customized on aggb:

#NCO_PA3 # # Process Agent Daemon Configuration File 1.1 # # # # List of processes # nco_process 'MasterObjectServer' { Command '$OMNIHOME/bin/nco_objserv -name AGG_B -pa NCO_PA' run as 500 Host= 'aggb' Managed = True RestartMsg= '${NAME} running as ${EUID} has been restored on ${HOST}.' AlertMsg= '${NAME} running as ${EUID} has died on ${HOST}.' RetryCount= 0 ProcessType = PaPA_AWARE } nco_process 'bigate' { Command '$OMNIHOME/bin/nco_g_objserv_bi -propsfile OMNIHOME/etc/AGG_GATE.props' run as 500 Host= 'aggb' Managed = True RestartMsg= '${NAME} running as ${EUID} has been restored on ${HOST}.' AlertMsg= '${NAME} running as ${EUID} has died on ${HOST}.' RetryCount= 0 ProcessType = PaPA_AWARE } # # List of Services # # NOTE: To ensure that the service is started automatically, change the # "ServiceStart" attribute to "Auto". # nco_service 'Core' { ServiceType = Master ServiceStart= Auto process 'MasterObjectServer' NONE process 'bigate' NONE } # # This service should be used to store processs that you want to temporarily # disable. Do not change the ServiceType or ServiceStart settings of this # process. # nco_service 'InactiveProcesses' { ServiceType = Non-Master ServiceStart= Non-Auto } # # ROUTING TABLE # # 'user' - (optional) only required for secure mode PAD on target host #'user' must be member of UNIX group 'ncoadmin' # 'password' - (optional) only required for secure mode PAD on target host #use nco_pa_crypt to encrypt. nco_routing { host 'omnihost' 'NCO_PA' 'user' 'password' } 8.Next, edit the ObjectServer props files, $OMNIHOME/etc/AGG_P.props on aggp and AGG_B.props on aggb. Edit the PA. properties specifying the operating system user and password of the user who will run process control and then save the file: PA.Name: 'NCO_PA' PA.Password: 'passw0rd' PA.Username: 'netcool' 9.Execute the following command to start the process control daemon: opt/IBM/tivoli/netcool/omnibus/bin/nco_pad The following is a sample of the normal nco_pad output displayed on the console: [netcool@aggp /]$ opt/IBM/tivoli/netcool/omnibus/bin/nco_pad Netcool/OMNIbus Process Agent Daemon - Version 7.3.1 Netcool/OMNIbus PA API Library Version 7.3.1 Sybase Server-Library Release: 15.0 Server Settings : Name of server: NCO_PA Path of used log file : /opt/IBM/tivoli/netcool/omnibus/log/NCO_PA.log Configuration File: /opt/IBM/tivoli/netcool/omnibus/etc/nco_pa.conf Child Output File : /dev/null Maximum logfile size: 1024 Thread stack size : 69632 Message Pool size : 45568 PID Message Pool size : 50 Rogue Process Timeout : 30 Truncate Log: False Instantiate server to daemon: True Internal API Checking : False No Configuration File : False Start Auto-start services : True Authentication System : UNIX Trace Net library : False Trace message queues: False Trace event queues: False Trace TDS packets : False Trace mutex locks : False Host DNS name : aggp PID file (from $OMNIHOME) : ./var/nco_pa.pid Kill Process group: False Secure Mode : False Administration Group Name.: ncoadmin Forking to a Daemon Process............. 10.After the process control daemon has started check that it is running correctly by running the following command from the $OMNIHOME/bin directory: /opt/IBM/tivoli/netcool/omnibus/bin/nco_pa_status server NCO_PA user netcool password passw0rd 11.After you have confirmed that NCO_PA is running properly you can use the following commands to stop/start OMNIbus processes using process control: To stop the bi-directional gateway/opt/IBM/tivoli/netcool/omnibus/bin//nco_pa_stop -process bigate To stop the ObjectServer /opt/IBM/tivoli/netcool/omnibus/bin/nco_pa_stop -process MasterObjectServer To start the ObjectServer/opt/IBM/tivoli/netcool/omnibus/bin/nco_pa_start -process MasterObjectServer To start the bi-directional gateway/opt/IBM/tivoli/netcool/omnibus/bin//nco_pa_start -process bigate Or if every OMNIbus process is down you may re-launch them with the following command to start everything in one shot:/opt/IBM/tivoli/netcool/omnibus/bin/nco_pad Installing the ITM Event Synchronization component (aggp & aggb) 1.The ITM Event Syncronization component is included in the IBM Tivoli Monitoring Tools offering available on Passport Advantage. 2.Untar the tools product package and copy the ESync2300Linux.bin file from the ITM Tools tec directory to the desired software images directory on both the aggp and aggb systems. NOTE:The ITM EventSync componentcan be installed using the same user that OMNIbus was installed with or as root. It should be noted that if the programs executable (in step 11) has been run as root complications may arise if you try to subsequently run it as a non-root user as certain files will then be owned by root. 3.Run the command :./ESync2300Linux.bin 4.The following panel should be displayed, click Next. 5.The panel for License Agreement is displayed. Accept the License and click Next. 6.On the panel to select installation directory leave the default and click Next. 7.In the next 2 panels leave the defaults (or increase debugging level for logs if needed) and click Next. 8.Enter Tivoli Monitoring TEMS values: hostname (both short and long names) and operating system user-id and password to login to it. Click Add and then Next. NOTE: You may need to use the fully qualified domain name for your Tivoli Monitoring TEMS server as the Event Synchronization component may not be able to communicate with the TEMS system otherwise. If you are unsure of which name to use you should add entries for both the fully qualified domain name and simple hostname. 9.Click Next on the summary panel. 10.Once the install has completed a panel with the installation result is displayed. Click Finish. 11.The ITM Event synchronization component has now been installed. Use the following commands to start/test/stop ITM Event Synchronization component when needed: To Start the Event Sync component: /opt/IBM/SitForwarder/bin/startSUF.sh To test TEMS configuration has been done correctly use the command: /opt/IBM/SitForwarder/bin/test.sh A successful test should output the following:Successfully connected to Tivoli Enterprise Monitoring Server nc049043.ibm.com To stop ITM Event Synchronization component: /opt/IBM/SitForwarder/bin/stopSUF.sh Configuring Bi-Directional Gateway (aggb) Update bi-directional gateway map file on aggb 1.Login to aggb where bi-directional gateway has been configured with netcool user 2.Copy the attribute mappings from /opt/IBM/SitForwarder/omnibus/multitier/GATE_itm.map into /opt/IBM/tivoli/netcool/omnibus/etc/AGG_GATE.map under the section : ############################################################################# # # CUSTOM alerts.status FIELD MAPPINGS GO HERE # ############################################################################# 3.Copy the table mappings from/opt/IBM/SitForwarder/omnibus/multitier/GATE_itm.mapto the end of the/opt/IBM/tivoli/netcool/omnibus/etc/AGG_GATE.map file: 4.The following is the resulting file as sample: ############################################################################# # Netcool/OMNIBUS Bi-directional ObjectServer Gateway 7.3.1 # # AGG_GATE Multitier map definition file. # # Revision History: # 1.1:Initial revision. # # Notes: # # Fields that are marked as 'ON INSERT ONLY' will only be passed when an event # is inserted for the first time. (ie. they will not be updated). The ordering # of the fields is not important as the gateway will use named value insertion. ####################################################################### CREATE MAPPING StatusMap ( 'Identifier' ='@Identifier' ON INSERT ONLY, 'Node' ='@Node' ON INSERT ONLY, 'NodeAlias'='@NodeAlias'ON INSERT ONLY NOTNULL '@Node', ####################################################################### # # CUSTOM alerts.status FIELD MAPPINGS GO HERE # ####################################################################### 'ITMStatus' = '@ITMStatus', 'ITMDisplayItem'= '@ITMDisplayItem', 'ITMEventData'= '@ITMEventData', 'ITMTime' = '@ITMTime', 'ITMHostname' = '@ITMHostname', 'ITMPort' = '@ITMPort', 'ITMIntType'= '@ITMIntType', 'ITMResetFlag'= '@ITMResetFlag', 'ITMSitType'= '@ITMSitType', 'ITMThruNode' = '@ITMThruNode', 'ITMSitGroup' = '@ITMSitGroup', 'ITMSitFullName'= '@ITMSitFullName', 'ITMApplLabel'= '@ITMApplLabel', 'ITMSitOrigin'= '@ITMSitOrigin', 'TECHostname' = '@TECHostname', 'TECFQHostname' = '@TECFQHostname', 'TECDate' = '@TECDate', 'TECRepeatCount'= '@TECRepeatCount', 'ServerName' ='@ServerName' ON INSERT ONLY, 'ServerSerial' ='@ServerSerial' ON INSERT ONLY ); CREATE MAPPING JournalMap ( . ); CREATE MAPPING DetailsMap ( .. ); CREATE MAPPING IducMap ( .. ); ####################################################################### # NOTE: If replication of the user related system tables is required, uncomment # the table mapping definitions below. The associated table replication # definitions will also need to be uncommented. ####################################################################### CREATE MAPPING SecurityUsersMap ( .. ); CREATE MAPPING SecurityGroupsMap ( ); CREATE MAPPING SecurityRolesMap ( . ); CREATE MAPPING SecurityRoleGrantsMap ( ); CREATE MAPPING SecurityGroupMembersMap ( .. ); CREATE MAPPING CatalogRestrictionFiltersMap ( .. ); CREATE MAPPING SecurityRestrictionFiltersMap ( ); CREATE MAPPING SecurityPermissionsMap ( ); ####################################################################### # NOTE: If replication of desktoprelated system tables is required, uncomment # the replication definitions below. The associated maps will also need to be # uncommented. ####################################################################### CREATE MAPPING ToolsMenusMap ( ); CREATE MAPPING ToolsMenuITEMSMap ( ); CREATE MAPPING ToolsActionsMap ( ); CREATE MAPPING ToolsActionAccessMap ( . ); CREATE MAPPING ToolsMenuDefsMap ( ); CREATE MAPPING ToolsPromptDefsMap ( .. ); CREATE MAPPING AlertsConversionsMap ( . ); CREATE MAPPING AlertsColVisualsMap ( . ); CREATE MAPPING AlertsColorsMap ( ... ); ####################################################################### # NOTE: If replication of the master.servergroups is is required, uncomment # the table mapping definitions below. The associated table replication # definitions will also need to be uncommented. ####################################################################### CREATE MAPPING MasterServergroupsMap ( ); ####################################################################### # # CUSTOM table mappings # ####################################################################### CREATE MAPPING ItmLoopbackMap ( 'Identifier'= '@Identifier' ON INSERT ONLY, 'itmstatus' = '@itmstatus' ); CREATE MAPPING ItmHeartbeatMap ( 'Identifier'= '@Identifier' ON INSERT ONLY, 'LastOccurrence'= '@LastOccurrence', 'Agent' = '@Agent'ON INSERT ONLY, 'AlertGroup'= '@AlertGroup' ON INSERT ONLY, 'Node'= '@Node', 'NodeAlias' = '@NodeAlias', 'ITMSitOrigin'= '@ITMSitOrigin', 'Manager' = '@Manager', 'Class' = '@Class'ON INSERT ONLY, 'HeartbeatInterval' = '@HeartbeatInterval', 'ExpirationTime'= '@ExpirationTime', 'Type'= '@Type' ); ### This is needed only if using cache for cleared sampled events CREATE MAPPING ItmEventCacheMap ( 'Identifier'= '@Identifier' ON INSERT ONLY, 'Node'= '@Node', 'NodeAlias' = '@NodeAlias', 'AlertGroup'= '@AlertGroup' ON INSERT ONLY, 'AlertKey'= '@AlertKey', 'Summary' = '@Summary', 'ExtendedAttr'= '@ExtendedAttr', 'ITMEventData'= '@ITMEventData', 'ITMTime' = '@ITMTime', 'ITMHostname' = '@ITMHostname', 'ITMThruNode' = '@ITMThruNode', 'ITMDisplayItem'= '@ITMDisplayItem', 'ITMSitOrigin'= '@ITMSitOrigin', 'InsertTime'= '@InsertTime' ); Update bi-directional gateway on aggb to replicate tables 1.Login to aggb where bi-directional gateway has been configured with netcool user 2.Copy the contents of /opt/IBM/SitForwarder/omnibus/multitier/GATE_itm.tblrep.def file at the end of /opt/IBM/tivoli/netcool/omnibus/etc/AGG_GATE.tblrep.def under the section ####################################################################### # # CUSTOM table replications ####################################################################### Update OMNIbus Data Base schema(aggp & aggb) The steps below must be executed both on aggp and aggb systems: 1.Login to the system with netcool user (if not already done) 2.Run command id to take the IDs of netcool user and ncoadmin group. In this environmentnetcoolID=500 and ncoadminID=500 [netcool@aggp ~]$ id uid=500(netcool) gid=500(ncoadmin) groups=0(root),100(users),500(ncoadmin) 3.Edit the /opt/IBM/SitForwarder/omnibus/itm_proc.sql file and change accordingly all the host, userid and groupid occurrences : host 'aggp' user 500 group 500 4.Save the file 5.Run the nco_sql command to create objects defined in the itm_proc.sql file: /[netcool@aggb OMNIbus]$ /opt/IBM/tivoli/netcool/omnibus/bin/nco_sql -user root -server AGG_P < /opt/IBM/SitForwarder/omnibus/itm_proc.sql Password: Default Password for internal OMNIbus root user is blank. On aggb system run the same command changing the name of the OMNIbus ObjectServer from AGG_P to AGG_B. 6.Run the nco_sql command to create objects defined in the itm_db_update.sql file: [netcool@aggp /]$ opt/IBM/tivoli/netcool/omnibus/bin/nco_sql -user root -server AGG_P < /opt/IBM/SitForwarder/omnibus/itm_db_update.sql Password: Default Password for internal OMNIbus root user is blank On aggb system run the same command changing the name of the OMNIbus ObjectServer from AGG_P to AGG_B. 7.Run the nco_sql command to create objects defined in the itm_sync.sql file: [netcool@aggp /]$ opt/IBM/tivoli/netcool/omnibus/bin/nco_sql -user root -server AGG_P < /opt/IBM/SitForwarder/omnibus/itm_sync.sql Password: Default Password for internal OMNIbus root user is blank. On aggb system run the same command changing the name of the OMNIbus ObjectServer from AGG_P to AGG_B. 8.Run the nco_sql command to create objects defined in the itm_event_cache.sql file: [netcool@aggp /]$ opt/IBM/tivoli/netcool/omnibus/bin/nco_sql -user root -server AGG_P < /opt/IBM/SitForwarder/omnibus/itm_event_cache.sql Password: Default Password for internal OMNIbus root user is blank. On aggb system run the same command changing the name of the OMNIbus ObjectServer from AGG_P to AGG_B. Updating the OMNIbus deduplication triggers (aggp & aggb) To modify the agg_deduplication triggers on each ObjectServer in the aggregation tier. the following steps must be executed both on aggp and aggb. 1.On each windows machine where Display ObjectServers have been installed run the Netcool Administrator:

2.When the Administrator console opens. Expand the Navigator tree on the left of the console, select the ObjectServer you have to work with (AGG_P in this case), right click on the mouse and selectthe Connect as option: 3.Insert root for the Username, leave blank in the Password field (or enter your ObjectServer root password if one has been configured) and click OK. 4.The Configuration console for AGG_P opens. 5.When the Admin GUI opens,select Automation and then Triggers to display the list of all triggers installed on this ObjectServer: 6.Select the agg_deduplication trigger, right click on the mouse and select Edit Trigger: 7.Click on the When tab and write the following line: (new.Type != 20) and (new.Type != 21) As shown below: 8.Click OK to save the change. 9.Repeat the above steps connecting to OMNIbus ObjectServer AGG_B. Bi-Directional Gateway stop and restart (aggb) After repeating these steps on both aggregation systems you must stop and restart the bi-directional server gateway (only on aggb) for changes to take effect. Theres no need to restart OMNIbus ObjectServers. To stop the bi-directional gateway/opt/IBM/tivoli/netcool/omnibus/bin/nco_pa_stop -process bigate To start the bi-directional gateway/opt/IBM/tivoli/netcool/omnibus/bin/nco_pa_start -process bigate Collection Layer Configuration Update uni-directional gateway map file (colp1 & colb1) The steps below must be executed on both the 2 Objects Servers in collection layer (colp1 and colb1) 1.Login to colp1 with netcool user 2.Copy the attribute mappings file from aggp machine where ITM Event Synchronization component has been installed to colp1 machine using scp command: scp /opt/IBM/SitForwarder/omnibus/multitier/GATE_itm.map netcool@colp1:/tmp 3.Copy just the attribute definitions from /tmp/GATE_itm.map into /opt/IBM/tivoli/netcool/omnibus/etc/C_TO_A_GATE.map under the section : ############################################################################# # # CUSTOM alerts.status FIELD MAPPINGS GO HERE # ############################################################################# 4.Login to colb1 with netcool user 5.Copy the attribute mappings file from aggp machine where ITM Event Synchronization component has been installed to colb1 machine using scp command: scp /opt/IBM/SitForwarder/omnibus/multitier/GATE_itm.map netcool@colb1:/tmp 6.Copy just the attribute definitions from /tmp/GATE_itm.map into /opt/IBM/tivoli/netcool/omnibus/etc/ C_TO_A_GATE.map under the section : ############################################################################# # # CUSTOM alerts.status FIELD MAPPINGS GO HERE # ############################################################################# Update OMNIbus ObjectServer Database schema (colp1 & colb1) The steps below must be executed both on colp1 and colb1 systems 1.Login to the system with netcool user (if not already done) 2.Copy the sql collection file from aggp machine where ITM Event Synchronization component has been installed to colp1 machine using scp command: scp /opt/IBM/SitForwarder/omnibus/multitier/collection_itm.sql netcool@colp1:/tmp 3.Run the nco_sql command to create objects defined in the collection_itm.sql file: /[netcool@colp1]$ /opt/IBM/tivoli/netcool/omnibus/bin/nco_sql -user root -server COL_P_1 < /tmp/collection_itm.sql Password: Default Password for internal OMNIbus root user is blank. On colb1 system run the same commands changing the name of the OMNIbus ObjectServer from COL_P_1 to COL_B_1. Configure probes (colp1 & colb1) Main configuration settings for the probes are in tivoli_eif.props under /opt/IBM/tivoli/netcool/omnibus/probes/linux2x86: Server: 'COL_P_1' ServerBackup: 'COL_B_1' PortNumber : 9998 Inactivity : 0 The steps below must be executed for both probes installed on colp1 and colb1 systems. 1.Login to the system with netcool user (if not already done) 2.Copy the rule files from aggp machine where ITM Event Synchronization component has been installed to colp1 machine under the probes directory using scp command: scp /opt/IBM/SitForwarder/omnibus/itm_event.rules netcool@colp1:/opt/IBM/tivoli/netcool/omnibus/probes/linux2x86 3.Uncomment the include statement for itm_event.rules in the tivoli_eif.rules file under/opt/IBM/tivoli/netcool/omnibus/probes/linux2x86 directory Stop and restart colp1 and colb1 Gateways and Probes: After completing the configuration steps on colp1 and colb1 stop and then restart the uni-directional gateways and probes to make the changes effective. (ObjectServer restart is not necessary) To Stop the processes here run ps ef | grep omnibus and search for the specific process id number for the gates and probes and kill the processes. To restart ObjectServer Uni-directional Gateway (change GW property file name on backup server colb1 from C_TO_A_GATE_P_1.propsto C_TO_A_GATE_B_1.props) execute the following command: /opt/IBM/tivoli/netcool/omnibus/bin/nco_g_objserv_uni -propsfile /opt/IBM/tivoli/netcool/omnibus/etc/C_TO_A_GATE_P_1.props & To start probe on both primary and backup systems: /opt/IBM/tivoli/netcool/omnibus/probes/nco_p_tivoli_eif & Display Layer Configuration Update uni-directional gateway map file (dis1 & dis2) The steps below must be executed on both the 2 Objects Servers in display layer (dis1 and dis2) 1.Login to the machine as Administrator 2.Copy just the ITM attribute definitions from the GATE_itm.map under/opt/IBM/SitForwarder/omnibus/multitier directory found on aggp machine where ITM Event Sync component has been installed into C:\IBM\Tivoli\Netcool\OMNIbus\etc\ A_TO_D_GATE.map under the section : ############################################################################# # # CUSTOM alerts.status FIELD MAPPINGS GO HERE # ############################################################################# Update OMNIbus Data Base schema (dis1 & dis2) The steps below must be executed both on dis1 and dis2 systems 1.Login to the system with Administrator (if not already done) 2.Copy the sql display file /opt/IBM/SitForwarder/omnibus/multitier/display_itm.sqlfrom aggp machine where ITM Event Synchronization component has been installed to dis1 machineunder C:\TEMP 3.Run the following command to update data base schema C:\TEMP\display_itm.sql | %OMNIHOME%\..\bin\redist\isql U root S DIS_1 Password: Default Password for internal OMNIbus root user is blank. 4.On dis2 system run the same commands changing the name of the OMNIbus ObjectServer from DIS_1 to DIS_2. Stop and restart dis1 and dis2 Gateways: After completing the previous configuration steps on dis1 and dis2 restart the uni-directional gateways to make changes effective. You may do this via the Windows Services window or by using the commands below: To start Uni-directional gateway (on backup Display machine dis2 change property file from A_TO_D_GATE_1.props to A_TO_D_GATE_2.props) %OMNIHOME%\bin\nco_g_objserv_uni -propsfile %OMNIHOME%\etc\A_TO_D_GATE_1.props & Install and configure OMNIbus Web GUI (TIP) on (dis1 & dis2) 1.Click on OK when the Netcool/OMNIbus splash screen appears. 2.Click on Next at the Introduction window. 3.Accept the Licensing Agreement and then click on Next. 4.Click Next if presented with the following window. 5.Keep the default TIP Directory path and click on Next. 6.Keep the default product install path and click Next. 7.Select Default installation type and click on Next. 8.Enter and confirm a password for the tipadmin user and then click on Next. 9.Select ObjectServer for the user registry. 10.Enter the root password , ObjectServer name, hostname, and port for the aggp server and then click Next. 11.Enter the ObjectServer name, hostname, and port for the aggb server and then click Next. 12.On the Pre-Installation Summary window click Install

13.When the Install Complete window appears observe and note the listed URL for accessing the and then click on Done. Configure WEBGUI for Active Event List To work with events using Active Event List in such an environment it is needed to configure Multiple Data Source with Failover on WEBGUI following the steps below: 1.Edit the file: C:\IBM\Tivoli\Netcool\omnibus_webgui\etc\datasources\ncwDataSourceDefinitions.xml 2.Define a ncwDataSourceEntry giving a name of your choice 3.Leave all default values except the following to be set accordingly with your environment: oncwDataSourceCredentials oncwFailOverPairDefinition with Primary and Backup Aggregation ObjectServers oncwReadCloudDefinition with Primary and Backup Display ObjectServers 4.Save the file 5.Restart TIP / Webgui from the Windows Services utlity. The following is the file configured for the described environment. (You may wish to paste this into an XML editor and use it in place of the default file):

- -

- -

-

- -

- -

-

-

Configure ITM TEMS to share situational events with the EIF probes Configure colp1 as default eif receiver on both Hub TEMS and Hot Standby Hub TEMS. In this integration scenario both TEMS were Linux systems with hostnames nc049043 and nc049044 respectively and the ITM home directory on both was /data/IBM/itm. To configure the TEMS for EIF Probe failover and follow steps below on the primary Hub TEMS (nc049043) first: 1.Stop the Hot standby Hub TEMS. 2.Run command to reconfigure TEMS: /data/IBM/itm/bin/itmcmd config -S -t HUB_nc049043 3.Enter 1 when asked to configure Tivoli EIF and insert the hostname and port of EIF probe of colp1 as follows: Tivoli Event Integration Facility? [1=YES, 2=NO] (Default is: 2): 1 EIF Server Host Name or type 0 for "none" :(Default is:): colp1 EIF Port? (Default is: 9999): 9998 4.Save the configuration and choose to restart TEMS to make changes effective. 5.Once restarted add the backup probe definition with the tacmd createEventDest command as shown below: [root@nc049043 bin]# ./tacmd createEventDest -i 3 -p host1=colb1:9998 default=Y description="backup eif probe" name=colb1 KUICCE004I: Are you sure you want to create the event destination server definition colb1 with server ID 3 on the server? Enter Y for yes or N for no: Y KUICCE007I: The event destination server definition colb1 with server ID 3 was successfully created on the server athttps://nc049043:3661.Note that Hub TEMS needs to be recycled or refreshed for this to take effect. 6.Restart TEMS to make changes effective Repeat steps above on Hot Standby Hub TEMS nc049044. OMNIbus ObjectServer, Gateways, and Probes Commands Reference OMNIbus Start ObjectServer Commands: To start the ObjectServer on Linux /opt/IBM/Tivoli/netcool/omnibus/bin/nco_objserv name & e.g. /opt/IBM/Tivoli/netcool/omnibus/bin/nco_objserv name AGG_B & To start the ObjectServer on Windows C:\IBM\Tivoli\netcool\omnibus\bin\nco_objserv name e.g. C:\IBM\Tivoli\netcool\omnibus\bin\nco_objserv name DIS_1 Note: On Windows platforms the OMNIbus ObjectServer and Gateways can be configured as Windows Services and controlled via the Windows Services control panel. The OMNIbus InfoCenter should be referencedif you wish to configure them in this manner. OMNIbus Start Gateway Commands: To start the bi-directional gateway on Linux /opt/IBM/tivoli/netcool/omnibus/bin/nco_g_objserv_uni -propsfile /opt/IBM/tivoli/netcool/omnibus/etc/C_TO_A_GATE_P_1.props & To start the uni-directional gateway on Linux/opt/IBM/tivoli/netcool/omnibus/bin/nco_g_objserv_uni -propsfile /opt/IBM/tivoli/netcool/omnibus/etc/C_TO_A_GATE_P_1.props & To start the uni-directional gateway on WindowsC:\IBM\Tivoli\netcool\omnibus\bin\nco_g_objserv_uni -propsfile %OMNIHOME%\etc\A_TO_D_GATE_1.props

OMNIbus Start EIF Probe Command: To start the EIF probes on Linux /opt/IBM/tivoli/netcool/omnibus/probes/nco_p_tivoli_eif & OMNIbus Process Control Commands: To start the Process Control(nco_pad) daemon/opt/IBM/tivoli/netcool/omnibus/bin/nco_pad To check the Process Control Status/opt/IBM/tivoli/netcool/omnibus/bin/nco_pa_status server NCO_PA user netcool password passw0rd To stop the bi-directional gateway/opt/IBM/tivoli/netcool/omnibus/bin//nco_pa_stop -process bigate To stop the ObjectServer /opt/IBM/tivoli/netcool/omnibus/bin/nco_pa_stop -process MasterObjectServer To start the ObjectServer/opt/IBM/tivoli/netcool/omnibus/bin/nco_pa_start -process MasterObjectServer To start the bi-directional gateway/opt/IBM/tivoli/netcool/omnibus/bin//nco_pa_start -process bigate