Om flex mgmt

Using Operations Manager 8 in aFlexible Management Environment

(The “MoM Cookbook”)

Version 3.02008-02-20

MoM Cookbook Version 3.0

_____________________________________________________________________________

Legal NoticesCopyright Notices

©Copyright 2007 Hewlett-Packard Development Company, L.P.

No part of this document may be copied, reproduced, or translated toanother language without the prior written consent of Hewlett-PackardCompany. The information contained in this material is subject tochange without notice.

Warranty

The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

The information contained herein is subject to change without notice.

Trademark Notices

Microsoft® and Windows® are U.S. registered trademarks of Microsoft Corporation.UNIX® is a registered trademark of The Open Group.


_____________________________________________________________________________

Preface

This cookbook is mainly intended for HP support personnel.It provides some steps for setting up a “Manager of Manager” (MoM) configuration.

Feedback to [email protected]

The hostnames used in this cookbook are purely fictitious and reflect the usage for the various purposes:

su1 management server with OMU 8 (HP-UX PA-RISC)su2 management server with OMU 8 (HP-UX PA-RISC)su3 management server with OMU 8 (HP-UX PA-RISC)su7 management server with OVOU 7 (HP-UX PA-RISC)sw75 management server with OVOW 7.5 (Windows 2003)sw80 management server with OMW 8.0 (Windows 2003)hn1 managed node with OMU 8 agent (HTTPS)hn2 managed node with OMU 8 agent (HTTPS)hn3 managed node with OMU 8 agent (HTTPS)dn4 managed node with OMU 8 agent (DCE)hn5 managed node with OMU 8 agent (HTTPS)c1n1 physical cluster node 1 of cluster 1 with OMU 8 agent (HTTPS)c1n2 physical cluster node 2 of cluster 1 with OMU 8 agent (HTTPS)c1v virtual node of cluster 1; management server with OMU 8c2n1 physical cluster node 1 of cluster 2 with OMU 8 agent (HTTPS)c2n2 physical cluster node 2 of cluster 2 with OMU 8 agent (HTTPS)c2v virtual node of cluster 2; management server with OMU 8c3n1 physical cluster node 1 of cluster 3 with OVOW 7 agent (DCE)c3n2 physical cluster node 2 of cluster 3 with OVOW 7 agent (DCE)c4n1 physical cluster node 1 of cluster 4 with OMW 8 agent (HTTPS)c4n2 physical cluster node 2 of cluster 4 with OMW 8 agent (HTTPS)c4v virtual cluster node of cluster 4vpool1 virtual node for server poolingvpool2 virtual node for server pooling

The hostnames here are also used without domain names. When using these commands it is advised to use the "Fully Qualified Domain Name" (FQDN), especially when the name resolution is not setup such that all short hostnames and aliases always resolve to the FQDN.

Sometimes color is used for hostnames or core IDs to emphasize changes. The colors are consistent within one toplevel chapter but not across toplevel chapters.


_____________________________________________________________________________

History:

Date Version Changes2008-02-20 3.0 Name changes from OVOU 8 to OMU 8, from s1 to su1, etc.

Added chapters:Agent-based message forwarding from OVOW 7 to OMU 8Agent-based message forwarding from OMU 8 to OVOW 7Server based message forwarding from OVOW 7.5 to OMU 8Message forwarding from OMU 8 to OVOW 7.5Message forwarding from OMW 8 to OMU 8Message forwarding from OMU 8 to OMW 8

2008-01-11 2.10 Fixed typo, removed unnecessary certificates in ovcert outputs2007-11-23 2.9 Added legal notices2007-11-06 2.8 Added chapters:

Message forwarding between two OMU 8 standalone serversServer Pooling


_____________________________________________________________________________

1 Switching the management server from primary to backup server in OMU 8 .....................71.1 Setup on the 2 management servers............................................................................71.2 Setup for the managed node .....................................................................................12

2 Message forwarding from OMU 8 to OVOU 7 server and vice versa ...............................162.1 Add servers to each other’s node bank .....................................................................162.2 Setup and test the message forwarding from OMU 8 to OVOU 7.............................172.3 Setup and test the message forwarding from OVOU 7 to OMU 8.............................172.4 Add source server's managed nodes to target server nodebank .................................182.5 Restrictions in message forwarding between OVOU 7 and 8....................................18

3 Message forwarding between two OMU 8 standalone servers ..........................................193.1 Verify certificates on both servers ............................................................................193.2 Setup certificate trust between the two servers .........................................................203.3 Add servers to each other’s node bank with correct ovcoreid....................................233.4 Setup the message forwarding template on server A.................................................243.5 Add managed nodes of server A to nodebank of server B.........................................243.6 Test the message forwarding from server A to server B............................................243.7 Add managed nodes of server B to nodebank of server A.........................................243.8 Setup the message forwarding template on server B.................................................253.9 Test the message forwarding from server B to server A............................................253.10 Configure managed nodes for switching primary manager .......................................25

4 Message forwarding between OMU 8 standalone and OMU 8 cluster ..............................264.1 Verify certificates on both servers ............................................................................264.2 Setup certificate trust between the two servers .........................................................284.3 Add servers to each other’s node bank with correct ovcoreid....................................314.4 Setup the message forwarding template on server A.................................................334.5 Add managed nodes of server A to nodebank of server B.........................................334.6 Test the message forwarding from server A to server B............................................334.7 Add managed nodes of server B to nodebank of server A.........................................344.8 Setup the message forwarding template on server B.................................................354.9 Test the message forwarding from server B to server A............................................354.10 Configure managed nodes for switching primary manager .......................................35

5 Message forwarding between two OMU 8 clusters ..........................................................365.1 Verifying certificates................................................................................................365.2 Setup certificate trust between the two servers .........................................................405.3 Add servers to each other's nodebank with correct ovcoreid .....................................435.4 Setup message forwarding template and test forwarding ..........................................445.5 Configure managed nodes for switching primary manager .......................................45

6 Server Pooling in OMU 8 ................................................................................................466.1 Configuring Management Server Nodes...................................................................466.2 Configuring Virtual Interfaces..................................................................................476.3 Configuring Primary Manager..................................................................................516.4 Configuring Message Forwarding ............................................................................526.5 Configuring Managed Nodes ...................................................................................526.6 Moving Virtual Interface to Another Physical Server ...............................................53

7 Agent-based message forwarding from OVOW 7 to OMU 8 ...........................................567.1 Create an agent-based flexible management policy ..................................................567.2 Configure Agents to communicate with OpenView Operations for UNIX ................577.3 Verification of correct message forwarding ..............................................................59

8 Agent-based message forwarding from OMU 8 to OVOW 7............................................60


_____________________________________________________________________________

8.1 Create an agent-based flexible management policy ..................................................608.2 Configure OMU Agents to communicate with OVOW 7..........................................618.3 Prepare the OVOW 7 management server ................................................................628.4 Verification of correct message forwarding ..............................................................62

9 Server based message forwarding from OVOW 7.5 to OMU 8 ........................................639.1 Configure OVOW 7.5 source server.........................................................................639.2 Configure OMU 8 target server................................................................................649.3 Verification of forwarded messages .........................................................................64

10 Message forwarding from OMU 8 to OVOW 7.5.........................................................6611 Message forwarding from OMW 8 to OMU 8..............................................................67

11.1 Verify certificates on both servers ............................................................................6711.2 Setup certificate trust between the two servers .........................................................6811.3 Add servers to each other’s node bank with correct ovcoreid....................................7111.4 Add managed nodes of source server to nodebank of target server ...........................7211.5 Setup the message forwarding template on the source server....................................7411.6 Verify correct forwarding of messages.....................................................................7511.7 Configure the managed nodes to accept action requests from target server ...............76

12 Message forwarding from OMU 8 to OMW 8..............................................................7812.1 Add managed nodes of source server to nodebank of target server ...........................7812.2 Setup the message forwarding template on the source server....................................78

13 Troubleshooting...........................................................................................................8013.1 Trying to distribute a new mgrconf file from OMU 8 ...............................................8013.2 Trying to distribute a new mgrconf file from OMU 8 (again) ...................................8113.3 Trying to distribute mgrconf file (mixture of OVOU 7 and OMU 8) ........................8213.4 Trying to switch primary manager to backup server on OMU 8................................8313.5 Trying bbcutil -ping to another management server on OMU 8 ................................8613.6 Trying opcragt from a backup server on OMU 8 ......................................................8813.7 Trying opcragt from a OMU 8 server to a OMW 8 controlled node..........................9013.8 Tring to forward a message from OVOU8 to OMW8...............................................92

14 Defects and Workarounds ............................................................................................94


_____________________________________________________________________________

1 Switching the management server from primary to backup server in OMU 8

The 2 management servers in this example are (both OMU 8):server A su1 (primary)server B su2 (backup)

The managed node to be switched from su1 to su2 is:hn1

All 3 systems have HTTPS agents.

1.1 Setup on the 2 management servers

1.1.1 Export trusted certificates on server su1

su1 # ovcert -exporttrusted -file /tmp/`hostname`.cert -ovrg serverINFO: Trusted certificates have been successfully exported to file '/tmp/

su1.cert'.su1 #

1.1.2 Export trusted certificates on server su2


su2.cert'.su2 #

1.1.3 Exchange the 2 `hostname`.cert files to the other server

Copy /tmp/su1.cert to su2, and /tmp/su2.cert to su1.


_____________________________________________________________________________

1.1.4 Import trusted certificates from server su2 to server su1

su1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+

su1 # ovcert -importtrusted -file /tmp/su2.cert -ovrg serverINFO: Import operation was successful.su1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de (*) || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 |+---------------------------------------------------------+

su1 #


_____________________________________________________________________________

1.1.5 Update trusted certificates on the server su1

As ovbbccb is running on the agent side of the management server, it needs to know the certificate of server su2 as well.

su1 # ovcert -updatetrustedINFO: Trusted certificate update was successful.su1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de (*) || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 |+---------------------------------------------------------+

su1 #


_____________________________________________________________________________

1.1.6 Import trusted certificates from server su1 to server su2

su2 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7958cdb8-5cad-7506-1d7e-dbea390a7cd8 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 7958cdb8-5cad-7506-1d7e-dbea390a7cd8 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 (*) |+---------------------------------------------------------+

su2 # ovcert -importtrusted -file /tmp/su1.cert -ovrg serverINFO: Import operation was successful.su2 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: |

| 7958cdb8-5cad-7506-1d7e-dbea390a7cd8 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 7958cdb8-5cad-7506-1d7e-dbea390a7cd8 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 (*) |+---------------------------------------------------------+

su2 #


_____________________________________________________________________________

1.1.7 Update trusted certificates on the server su2

As ovbbccb is running on the agent side of the management server, it needs to know the certificate of server su1 as well.

su2 # ovcert -updatetrustedINFO: Trusted certificate update was successful.su2 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7958cdb8-5cad-7506-1d7e-dbea390a7cd8 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 |+---------------------------------------------------------+


su2 #

1.1.8 Add server su2 to node bank of server su1

1.1.9 Obtain OVCoreID from server su2

su2 # ovcoreid -ovrg server7958cdb8-5cad-7506-1d7e-dbea390a7cd8su2 #

1.1.10 Add it to the node data for su2 in server su1

su1 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=su2id='7958cdb8-5cad-7506-1d7e-dbea390a7cd8'Operation successfully completed.su1 # /opt/OV/bin/OpC/utils/opcnode -list_id node_list='su2'List of IDs for node(s):Name = su2 ID = 7958cdb8-5cad-7506-1d7e-dbea390a7cd8Operation successfully completed.su1 #


_____________________________________________________________________________

1.2 Setup for the managed node

1.2.1 Update trusted certificates on the managed node hn1

hn1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || e342588e-f9f4-7508-1d48-aeedd15c855b (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

hn1 # ovcert -updatetrustedINFO: Trusted certificate update was successful.hn1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || e342588e-f9f4-7508-1d48-aeedd15c855b (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 |+---------------------------------------------------------+

hn1 #

1.2.2 Create the mgrconf file

This has to be created on server su1 in the directory/etc/opt/OV/share/conf/OpC/mgmt_sv/respmgrs. This can be done either as allnodes (for DCE agents) or as allnodes.bbc (for HTTPS agents) or as as node specific file with the hex IP address of the managed node (if special settings are used that don't apply for all managed nodes). In this example a node specific file will be used (but for no partcular reason).

The file name to be used can be found using opc_ip_addr:

su1 # /opt/OV/bin/OpC/install/opc_ip_addr hn1hn1 = 15.140.10.236 = f8c0aecsu1 #

The file /etc/opt/OV/share/conf/OpC/mgmt_sv/respmgrs/f8c0aec then can be created with e.g. this contents:


_____________________________________________________________________________

## Responsible Manager Configurations for a backup server#RESPMGRCONFIGS

RESPMGRCONFIGDESCRIPTION "responsible mgrs for agents in ..."

SECONDARYMANAGERS SECONDARYMANAGERNODE IP 0.0.0.0 "su1"DESCRIPTION "Managment Server su1"SECONDARYMANAGERNODE IP 0.0.0.0 "su2"DESCRIPTION "Backup Server for su1"

ACTIONALLOWMANAGERSACTIONALLOWMANAGERNODE IP 0.0.0.0 "su1"DESCRIPTION "Managment Server su1"ACTIONALLOWMANAGER

NODE IP 0.0.0.0 "su2"DESCRIPTION "Backup Server for su1"

1.2.3 Distribute the mgrconf file

On the server su1:

su1 # opcragt -distrib -templates hn1Node hn1:Create distribution data and inform agent...Done.

su1 #

Additionally to distributing the assigned templates, OMU will select and download the correct file from the directory /etc/opt/OV/share/conf/OpC/mgmt_sv/respmgrs:

First it checks for a node specific file named after the hex IP address of the node.

If such a file is not present, then either the allnodes or allnodes.bbc file (depending on the type of the managed node) is downloaded.

In any case the file will be given a fixed on the managed node, see next chapter.


_____________________________________________________________________________

1.2.4 Verify mgrconf file on managed node hn1

The mgrconf file is named differently, depending on the type of the agent.In case of a DCE agent, it will be /var/opt/OV/conf/OpC/mgrconfFor a HTTPS agent the file will be in the mgrconf directory:

hn1 # cat /var/opt/OV/datafiles/policies/mgrconf/*data## Responsible Manager Configurations for a backup server#RESPMGRCONFIGS

RESPMGRCONFIGDESCRIPTION "responsible mgrs for agents in ..."

SECONDARYMANAGERSSECONDARYMANAGERNODE IP 0.0.0.0 "su1" ID "7681325c-c1a9-7508-0441-

a54412c264de"DESCRIPTION "Managment Server su1"SECONDARYMANAGERNODE IP 0.0.0.0 "su2" ID "7958cdb8-5cad-7506-1d7e-

dbea390a7cd8" DESCRIPTION "Backup Server for su1"

ACTIONALLOWMANAGERSACTIONALLOWMANAGERNODE IP 0.0.0.0 "su1" ID "7681325c-c1a9-7508-0441-

a54412c264de"DESCRIPTION "Managment Server su1"ACTIONALLOWMANAGERNODE IP 0.0.0.0 "su2" ID "7958cdb8-5cad-7506-1d7e-

dbea390a7cd8"DESCRIPTION "Backup Server for su1"

hn1 #

Note the additional ID in this file.

This has been added on the mgmt sv and stored as additional file f8c0aec_data:

su1 # pwd/etc/opt/OV/share/conf/OpC/mgmt_sv/respmgrssu1 # ll f8c0aec*-r--r--r-- 1 root sys 578 Jan 21 10:36 f8c0aec-rw------- 1 root sys 746 Jan 21 11:03 f8c0aec_datasu1 #

1.2.5 Add node hn1 to server su2 into node bank and node group(s)


_____________________________________________________________________________

1.2.6 Obtain OVCoreID of managed node hn1

hn1 # ovcoreide342588e-f9f4-7508-1d48-aeedd15c855b

hn1 #

1.2.7 Set the id of the node hn1 in server su2

su2 # /opt/OV/bin/OpC/utils/opcnode -list_id node_list='hn1'List of IDs for node(s):Name = hn1 ID = NONEOperation successfully completed.su2 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=hn1id='e342588e-f9f4-7508-1d48-aeedd15c855b'Operation successfully completed.su2 # /opt/OV/bin/OpC/utils/opcnode -list_id node_list='hn1'List of IDs for node(s):Name = hn1 ID = e342588e-f9f4-7508-1d48-aeedd15c855bOperation successfully completed.su2 #

1.2.8 switch the primary manager to server su2

su2 # opcragt -primmgr hn1Node hn1:Setting OpC primary manager...Done.

su2 #

1.2.9 Verify message communication

Create a test message on the node hn1 and verify that it is displayed in the browser of the server su2:

hn1 # opcmsg a=a o=o msg_t=hello


_____________________________________________________________________________

2 Message forwarding from OMU 8 to OVOU 7 server and vice versa

For the 1st test the configuration is:

Source server is su1 (OMU 8)Target server is su7 (OVOU 7)

In the 2nd test later in this chapter the message forwarding is configured from su7 to su1.

2.1 Add servers to each other’s node bank

2.1.1 Add target server to node bank of source server

In “Modify Node” the machine type of the newly added node (target server su7) has to be the DCE version for that OS choice, e.g. for a HP-UX 11.x PA-RISC system it has to be “HP PA-RISC” instead of “HP PA-RISC (HTTPS)”.

This is clear because the target server is OVOU 7, i.e. it only knows the DCE protocol.

su1 # opcnode -add_node node_name=su7 node_label=su7 \net_type=NETWORK_IP mach_type=MACH_HP11_PA_RISC group_name=hp_ux

Note that you have to move the newly added node in the GUI from the Holding Area to the correct place in the node bank.

The node entry can be verified with:

su1 # opcnode -list_nodes node_list=’su7’List of all Nodes in the OVO database:Name = su7Label = su7IP-Address = 16.58.24.87Network Type = NETWORK_IPMachine Type = MACH_HP11_PA_RISCComm Type = COMM_DCE_TCPDHCP enabled = no (0x22)

2.1.2 Add source server to node bank of target server

In “Modify Node” the machine type of the newly added node (source server su1) has to be the DCE version for that OS choice, e.g. for a HP-UX 11.x PA-RISC system it has to be “HP PA-RISC” instead of “HP PA-RISC (HTTPS)”.

su7 # opcnode -add_node node_name=su1 \> node_label=su1 net_type=NETWORK_IP \> mach_type=MACH_HP11_PA_RISC group_name=hp_ux


_____________________________________________________________________________

Operation successfully completed.su7 #

Verify the new node as above.

2.2 Setup and test the message forwarding from OMU 8 to OVOU 7

Add a suitable msgtargetrule to the msgforw template, check it with opcmomchk(1m) and place the file into /etc/opt/OV/share/conf/OpC/mgmt_sv/respmgrs on the source server su1, e.g.:

MSGTARGETRULEDESCRIPTION "to OVO7"

MSGTARGETRULECONDS MSGTARGETRULECOND

DESCRIPTION "OVO7"OBJECT "OVO7"

MSGTARGETMANAGERSMSGTARGETMANAGER

TIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "su7"MSGCONTROLLINGMGR

MSGTARGETMANAGERTIMETEMPLATE "$OPC_ALWAYS"

OPCMGR IP 0.0.0.0 "su1"

Then restart the management server processes on su1 to activate the message forwarding:

su1 # ovstop opcsu1 # ovstart

On source server su1 create a suitable message to be forwarded to the target server su7:

su1 # opcmsg a=a o=OVO7 msg_t="forwarded from su1"

Verify that the message is displayed in the message browser of the target server su7.

2.3 Setup and test the message forwarding from OVOU 7 to OMU 8


MSGTARGETRULEDESCRIPTION "to OMU8"


DESCRIPTION "OMU8"OBJECT "OMU8"


TIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "su7"



_____________________________________________________________________________

OPCMGR IP 0.0.0.0 "su1" MSGCONTROLLINGMGR



On source server su7 create a suitable message to be forwarded to the target server su1:

su7 # opcmsg a=a o=OMU8 msg_t="forwarded from su7"


2.4 Add source server's managed nodes to target server nodebank

For each managed node whose messages shall be forwarded, add the node to the node bank of the target server. Be sure to use the correct mach_type, as outlined above. For the simple test done here (message forwarding from the source mgmt sv itself) this has been done already.

Please note that in each case the machine type has to be the DCE version for that OS choice, e.g. for a HP-UX 11.x PA-RISC system it has to be “HP PA-RISC” instead of “HP PA-RISC (HTTPS)”.

In the case of adding OVOU 7 nodes to the OMU 8 server node bank this is clear because these really are DCE nodes.

In the case of adding OMU 8 nodes to the OVOU 7 server node bank there is just no other choice.

2.5 Restrictions in message forwarding between OVOU 7 and 8

If a message from a HTTPS node is forwarded to a OVOU 7 server and the message has an operator-initiated action configured, then this action can't be initiated from the OVOU 7 server. This would require a direct communication from the OVOU 7 server to the HTTPS node but OVOU 7 doesn't have HTTPS communication.


_____________________________________________________________________________

3 Message forwarding between two OMU 8 standalone servers

In this setup we will use the following systems:

server A su1 management server with OMU 8server B su3 management server with OMU 8

First a message forwarding from A to B will be configured, then the other way 'round.

Note that the message forwarding process opcforwm will normally use the DCE protocol to forward messages to the target server. Since the A.08.21 server patch also the HTTPS communication is possible for opcforwm but has to be enabled in a variable:

# ovconfchg –ovrg server –ns opc –set OPC_HTTPS_MSG_FORWARD TRUE

Also it makes sense to set these variables:

# ovconfchg –ovrg server –ns opc –set OPC_DONT_FORW_MSGKEY_ACK TRUE# ovconfchg –ovrg server –ns opc –set OPC_MOM_SEND_OP_ACK TRUE

3.1 Verify certificates on both servers

3.1.1 Certificates on server A

su1 # ovcoreid7681325c-c1a9-7508-0441-a54412c264desu1 # ovcoreid -ovrg server7681325c-c1a9-7508-0441-a54412c264desu1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+


su1 #


_____________________________________________________________________________

3.1.2 Certificates on server B

su3 # ovcoreidae33c7ea-94b0-7525-04ef-cbab70bb7252su3 # ovcoreid -ovrg serverae33c7ea-94b0-7525-04ef-cbab70bb7252su3 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || ae33c7ea-94b0-7525-04ef-cbab70bb7252 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || ae33c7ea-94b0-7525-04ef-cbab70bb7252 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 (*) |+---------------------------------------------------------+

su3 #

3.2 Setup certificate trust between the two servers

3.2.1 export trusted certificates on server A


su1.cert'.su1 #

3.2.2 export trusted certificates on server B


su3.cert'.su3 #


Copy /tmp/su1.cert to su3, and /tmp/su3.cert to su1.


_____________________________________________________________________________

3.2.4 Import trusted certificates from server B to server A

su1 # ovcert -importtrusted -file /tmp/su3.cert -ovrg serverINFO: Import operation was successful.su1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de (*) || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 |+---------------------------------------------------------+

su1 #

3.2.5 Update trusted certificates in server A

su1 # ovcert -updatetrustedINFO: Trusted certificate update was successful.su1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 |+---------------------------------------------------------+


su1 #


_____________________________________________________________________________

3.2.6 Import trusted certificates from server A to server B

su3 # ovcert -importtrusted -file /tmp/su1.cert -ovrg serverINFO: Import operation was successful.su3 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || ae33c7ea-94b0-7525-04ef-cbab70bb7252 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || ae33c7ea-94b0-7525-04ef-cbab70bb7252 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 (*) |+---------------------------------------------------------+

su3 #

3.2.7 Update trusted certificates in server B

su3 # ovcert -updatetrustedINFO: Trusted certificate update was successful.su3 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || ae33c7ea-94b0-7525-04ef-cbab70bb7252 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 |+---------------------------------------------------------+


su3 #


_____________________________________________________________________________

3.3 Add servers to each other’s node bank with correct ovcoreid

3.3.1 Add server B to node bank of server A

su1 # /opt/OV/bin/OpC/utils/opcnode -add_node node_name=su3 \> node_label=su3 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.su1 #

Be sure to use the HTTPS version of mach_type (MACH_BBC_*) for the corresponding OS platform and move the node from the holding area to the correct node layout hierarchy afterwards.

3.3.2 Add B's ovcoreid to the node data for server B in server A

su1 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=su3 \> id='ae33c7ea-94b0-7525-04ef-cbab70bb7252'Operation successfully completed.su1 # /opt/OV/bin/OpC/utils/opcnode -list_id node_list='su3'List of IDs for node(s):Name = su3 ID = ae33c7ea-94b0-7525-04ef-cbab70bb7252Operation successfully completed.su1 #

3.3.3 Add server A to node bank of server B

su3 # /opt/OV/bin/OpC/utils/opcnode -add_node node_name=su1 \> node_label=su1 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.su3 #


3.3.4 Add A's ovcoreid to the node data for server A in server B

su3 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=su1 \> id='7681325c-c1a9-7508-0441-a54412c264de'Operation successfully completed.su3 # /opt/OV/bin/OpC/utils/opcnode -list_id node_list='su1'List of IDs for node(s):Name = su1 ID = 7681325c-c1a9-7508-0441-a54412c264deOperation successfully completed.su3 #


_____________________________________________________________________________

3.4 Setup the message forwarding template on server A


MSGTARGETRULEDESCRIPTION "to su3"

MSGTARGETRULECONDS MSGTARGETMANAGERS

MSGTARGETMANAGERTIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "su3"MSGCONTROLLINGMGR




3.5 Add managed nodes of server A to nodebank of server B

For each managed node whose messages shall be forwarded, add the node to the node bank. Be sure to use the correct mach_type and also set the correct ovcoreid, as outlined above. For the simple test below (message forwarding from server A) this is not necessary. In general, however, it is desired to have all managed nodes in each other's nodebank. This can easiest be achieved with 'opccfgdwn -backup', followed by 'opcmgrdist' on server A and 'opccfgupld' on server B.

3.6 Test the message forwarding from server A to server B

On source server su1 create a suitable message to be forwarded to the target server c1v:

su1 # opcmsg a=a o=o msg_t="forwarded from su1"

Verify that the message is displayed in the message browser of the target server c1v.

3.7 Add managed nodes of server B to nodebank of server A

This is only necessary if server B has managed nodes of its own. It is not necessary if server B is used as backup server for A.


_____________________________________________________________________________

3.8 Setup the message forwarding template on server B

Add a suitable msgtargetrule to the msgforw template, check it with opcmomchk(1m) and place the file into /etc/opt/OV/share/conf/OpC/mgmt_sv/respmgrs on the source server c1v, e.g.:


MSGTARGETRULECONDSMSGTARGETRULECONDDESCRIPTION "su1"

OBJECT "su1"MSGTARGETMANAGERS

MSGTARGETMANAGERTIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "su3"




3.9 Test the message forwarding from server B to server A

su3 # opcmsg a=a o=su1 msg_t="forwarded from su3"


3.10Configure managed nodes for switching primary manager

Usually the concepts of message forwarding and switching primary manager are combined, i.e. it is desired to switch the controlling manager of the managed nodes to the target manager of the forwarded messages, or at least to allow operator-initiated actions from that target manager.

For this the certificate trust of the target manager has to be extended to the managed nodes, and suitable mgrconf templates have to be setup / distributed. See chapter 1 for an example of how to do this.


_____________________________________________________________________________

4 Message forwarding between OMU 8 standalone andOMU 8 cluster

In this setup we will use the following systems:

server A su1 management server with OMU 8server B c1v virtual node of cluster 1; management server with OMU 8

c1n1 physical cluster node 1 of cluster 1 with OMU 8 agent (HTTPS)c1n2 physical cluster node 2 of cluster 1 with OMU 8 agent (HTTPS)


Note that the message forwarding process opcforwm will normally use the DCE protocol to forward messages to the target server. Since the A.08.21 server patch also the HTTPS communication is possible for opcforwm but has to be enabled in a variable:

ovconfchg –ovrg server –ns opc –set OPC_HTTPS_MSG_FORWARD TRUE





su1 #

Note that 'ovcoreid' and 'ovcoreid -ovrg server' have the same output on a standalone server.


_____________________________________________________________________________

4.1.2 Certificates on server B cluster node 1

c1n1 # ovcoreidf7996602-d96c-750a-19f1-972b895012fcc1n1 # ovcoreid -ovrg serverf7996602-d96c-750a-19f1-972b895012fcc1n1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || f7996602-d96c-750a-19f1-972b895012fc (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: |+---------------------------------------------------------+| Trusted Certificates: |+---------------------------------------------------------+

c1n1 #

This is the cluster node currently NOT running the ov-server package. Therefore 'ovcoreid' and 'ovcoreid -ovrg server' have the same output, and the keystore of the server is empty.

4.1.3 Certificates on server B cluster node 2, currently running ov-server

c1n2 # ovcoreid681b477e-3def-750b-02a3-d3cdd199a983c1n2 # ovcoreid -ovrg server659b2fa0-d93b-750a-0aab-b285232fc049c1n2 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 681b477e-3def-750b-02a3-d3cdd199a983 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 659b2fa0-d93b-750a-0aab-b285232fc049 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 (*) |+---------------------------------------------------------+


_____________________________________________________________________________

c1n2 #

This is the cluster node currently running the ov-server package. Note the different ovcoreid outputs for the physical and the virtual cluster node, and the keystore of the server.


4.2.1 export trusted certificates on server A


su1.cert'.su1 #

4.2.2 export trusted certificates on server B

c1n2 # ovcert -exporttrusted -file /tmp/`hostname`.cert -ovrg serverINFO: Trusted certificates have been successfully exported to file '/tmp/

c1n2.cert'.c1n2 #


Copy /tmp/su1.cert to c1n2, and /tmp/c1n2.cert to su1.


_____________________________________________________________________________

4.2.4 Import trusted certificates from server B to server A

su1 # ovcert -importtrusted -file /tmp/c1n2.cert -ovrg serverINFO: Import operation was successful.su1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 || CA_7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+

su1 #

4.2.5 Update trusted certificates in server A

su1 # ovcert -updatetrustedINFO: Trusted certificate update was successful.su1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 || CA_7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+

su1 #


_____________________________________________________________________________

4.2.6 Import trusted certificates from server A to server B

c1n2 # ovcert -importtrusted -file /tmp/su1.cert -ovrg serverINFO: Import operation was successful.c1n2 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 681b477e-3def-750b-02a3-d3cdd199a983 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 659b2fa0-d93b-750a-0aab-b285232fc049 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 (*) || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

c1n2 #

4.2.7 Update trusted certificates in server B cluster node 1

c1n1 # ovcert -updatetrustedINFO: Trusted certificate update was successful.c1n1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || f7996602-d96c-750a-19f1-972b895012fc (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+


c1n1 #


_____________________________________________________________________________

4.2.8 Update trusted certificates in server B cluster node 2

c1n2 # ovcert -updatetrustedINFO: Trusted certificate update was successful.c1n2 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || f7996602-d96c-750a-19f1-972b895012fc (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 659b2fa0-d93b-750a-0aab-b285232fc049 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 (*) || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

c1n2 #


4.3.1 Add server B to node bank of server A

su1 # opcnode -add_node node_name=c1v \> node_label=c1v-ov-server \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.su1 #


4.3.2 Add B's ovcoreid to the node data for server B in server A

su1 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=c1v \> id='659b2fa0-d93b-750a-0aab-b285232fc049'Operation successfully completed.su1 # /opt/OV/bin/OpC/utils/opcnode -list_id node_list='c1v'List of IDs for node(s):Name = c1v ID = 659b2fa0-d93b-750a-0aab-b285232fc049Operation successfully completed.su1 #


_____________________________________________________________________________

4.3.3 Add server A to node bank of server B

c1n2 # /opt/OV/bin/OpC/utils/opcnode -add_node node_name=su1 \> node_label=su1 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.c1n2 #



_____________________________________________________________________________

4.3.4 Add A's ovcoreid to the node data for server A in server B

c1n2 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=su1 \> id='7681325c-c1a9-7508-0441-a54412c264de'Operation successfully completed.c1n2 # /opt/OV/bin/OpC/utils/opcnode -list_id node_list='su1'List of IDs for node(s):Name = su1 ID = 7681325c-c1a9-7508-0441-a54412c264deOperation successfully completed.c1n2 #

4.4 Setup the message forwarding template on server A


MSGTARGETRULEDESCRIPTION "to c1v-ov-server"

MSGTARGETRULECONDSMSGTARGETRULECONDDESCRIPTION "c1v-ov-server"

OBJECT "c1v-ov-server"MSGTARGETMANAGERS

MSGTARGETMANAGERTIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "c1v"MSGCONTROLLINGMGR

MSGTARGETMANAGERTIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "su1"



4.5 Add managed nodes of server A to nodebank of server B

For each managed node whose messages shall be forwarded, add the node to the node bank. Be sure to use the correct mach_type and also set the correct ovcoreid, as outlined above. For the simple test below (message forwarding from server A) this is not necessary. In general, however, it is desired to have all managed nodes in each other's nodebank. This can easiest be achieved with 'opccfgdwn -backup', followed by 'opcmgrdist' on server A and 'opccfgupld' on server B.

4.6 Test the message forwarding from server A to server B

On source server su1 create a suitable message to be forwarded to the target server c1v:

su1 # opcmsg a=a o=c1v-ov-server msg_t="forwarded from su1"


_____________________________________________________________________________

Verify that the message is displayed in the message browser of the target server c1v.

4.7 Add managed nodes of server B to nodebank of server A

As server B is a virtual cluster node but the messages are generated e.g. on the physical cluster nodes, at least those physical nodes have to be made known to server A. For the simple test below (forwarding messages from physical node where the server is currently running) it is sufficient to add just that one physical node.

4.7.1 Add cluster node 2 to node bank of server A

su1 # opcnode -add_node node_name=c1n2 \> node_label=c1n2 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.su1 #


4.7.2 Add ovcoreid to the node data for cluster node 2 in server A

su1 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=c1n2 \> id='681b477e-3def-750b-02a3-d3cdd199a983'Operation successfully completed.su1 # /opt/OV/bin/OpC/utils/opcnode -list_id node_list='c1n2'List of IDs for node(s):Name = c1n2 ID = 681b477e-3def-750b-02a3-d3cdd199a983Operation successfully completed.su1 #


_____________________________________________________________________________

4.8 Setup the message forwarding template on server B

Add a suitable msgtargetrule to the msgforw template, check it with opcmomchk(1m) and place the file into /etc/opt/OV/share/conf/OpC/mgmt_sv/respmgrs on the source server c1v, e.g.:


MSGTARGETRULECONDSMSGTARGETRULECONDDESCRIPTION "su1"

OBJECT "su1" MSGTARGETMANAGERS

MSGTARGETMANAGERTIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "c1v"

MSGTARGETMANAGER TIMETEMPLATE "$OPC_ALWAYS"

OPCMGR IP 0.0.0.0 "su1"MSGCONTROLLINGMGR

Then restart the management server processes on c1v to activate the message forwarding. Note that this is a cluster, so normally this would fail over the HA package, unless the cluster servive monitoring isn't stopped in before:

c1n2 # /opt/OV/lbin/ovharg -monitor ov-server disablec1n2 # opcsv -startc1n2 # /opt/OV/lbin/ovharg -monitor ov-server enable

4.9 Test the message forwarding from server B to server A

c1n2 # opcmsg a=a o=su1 msg_t="forwarded from c1v"


4.10Configure managed nodes for switching primary manager




_____________________________________________________________________________

5 Message forwarding between two OMU 8 clusters

In this chapter we will use the following systems:

server A c1v virtual node of cluster 1; management server with OMU 8c1n1 physical cluster node 1 of cluster 1 with OMU 8 agent (HTTPS)c1n2 physical cluster node 2 of cluster 1 with OMU 8 agent (HTTPS)

server B c2v virtual node of cluster 2; management server with OMU 8c2n1 physical cluster node 1 of cluster 2 with OMU 8 agent (HTTPS)c2n2 physical cluster node 2 of cluster 2 with OMU 8 agent (HTTPS)


5.1 Verifying certificates

Please note that in an actual configuration there may be certificate trusts from other servers configured already. These are not relevant for the current task, and for clarity not shown here.


c1n1 # ovcoreidf7996602-d96c-750a-19f1-972b895012fcc1n1 # ovcoreid -ovrg serverf7996602-d96c-750a-19f1-972b895012fcc1n1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || f7996602-d96c-750a-19f1-972b895012fc (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 |+---------------------------------------------------------+


c1n1 #

So c1n1 is the inactive node. Note that the output of ovcoreid and ovcoreid -ovrg server is the same because the server data is currently not available.


_____________________________________________________________________________

c1n2 # ovcoreid681b477e-3def-750b-02a3-d3cdd199a983c1n2 # ovcoreid -ovrg server659b2fa0-d93b-750a-0aab-b285232fc049c1n2 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 681b477e-3def-750b-02a3-d3cdd199a983 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 659b2fa0-d93b-750a-0aab-b285232fc049 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 (*) |+---------------------------------------------------------+

c1n2 #

So c1n2 is the active node.


_____________________________________________________________________________

5.1.2 Certificates on server B

c2n1 # ovcoreid3f22e490-d54b-7507-0012-8b15c6ae224dc2n1 # ovcoreid -ovrg server3f22e490-d54b-7507-0012-8b15c6ae224dc2n1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 3f22e490-d54b-7507-0012-8b15c6ae224d (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_260a0712-d533-7507-1c68-e5d0d06b2196 |+---------------------------------------------------------+


c2n1 #

So c2n1 is the inactive node. Note that the output of ovcoreid and ovcoreid -ovrg server is the same because the server data is currently not available.


_____________________________________________________________________________

c2n2 # ovcoreidaba6d168-d58b-7507-1477-cb518338c12fc2n2 # ovcoreid -ovrg server260a0712-d533-7507-1c68-e5d0d06b2196c2n2 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || aba6d168-d58b-7507-1477-cb518338c12f (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_260a0712-d533-7507-1c68-e5d0d06b2196 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 260a0712-d533-7507-1c68-e5d0d06b2196 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_260a0712-d533-7507-1c68-e5d0d06b2196 (*) |+---------------------------------------------------------+

c2n2 #

So c2n2 is the active node.


_____________________________________________________________________________


Export trusted certificates on both servers, exchange the 2 files and import them to the other servers, then update the trusted certificates on the agent side:

Server A:


c1n2.cert'.c1n2 #

Server B:


c2n1.cert'.c2n1 #

Server A:

c1n2 # ovcert -importtrusted -file /tmp/c2n1.cert -ovrg serverINFO: Import operation was successful.c1n2 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 681b477e-3def-750b-02a3-d3cdd199a983 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_659b2fa0-d93b-750a-0aab-b285232fc049 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 659b2fa0-d93b-750a-0aab-b285232fc049 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_260a0712-d533-7507-1c68-e5d0d06b2196 || CA_659b2fa0-d93b-750a-0aab-b285232fc049 (*) |+---------------------------------------------------------+

c1n2 # ovcert -updatetrustedINFO: Trusted certificate update was successful.c1n2 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 681b477e-3def-750b-02a3-d3cdd199a983 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_260a0712-d533-7507-1c68-e5d0d06b2196 || CA_659b2fa0-d93b-750a-0aab-b285232fc049 |


_____________________________________________________________________________

+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 659b2fa0-d93b-750a-0aab-b285232fc049 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_260a0712-d533-7507-1c68-e5d0d06b2196 || CA_659b2fa0-d93b-750a-0aab-b285232fc049 (*) |+---------------------------------------------------------+

c1n2 #

Repeat the 'ovcert -updatetrusted' on the other cluster node which is currently not running the ov-server package.

Server B:

c2n1 # ovcert -importtrusted -file /tmp/c1n2.cert -ovrg serverINFO: Import operation was successful.c2n1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || aba6d168-d58b-7507-1477-cb518338c12f (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_260a0712-d533-7507-1c68-e5d0d06b2196 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 260a0712-d533-7507-1c68-e5d0d06b2196 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_260a0712-d533-7507-1c68-e5d0d06b2196 (*) || CA_659b2fa0-d93b-750a-0aab-b285232fc049 |+---------------------------------------------------------+

c2n1 # ovcert -updatetrustedINFO: Trusted certificate update was successful.c2n1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || aba6d168-d58b-7507-1477-cb518338c12f (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_260a0712-d533-7507-1c68-e5d0d06b2196 || CA_659b2fa0-d93b-750a-0aab-b285232fc049 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |


_____________________________________________________________________________

+---------------------------------------------------------+| Certificates: || 260a0712-d533-7507-1c68-e5d0d06b2196 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_260a0712-d533-7507-1c68-e5d0d06b2196 (*) || CA_659b2fa0-d93b-750a-0aab-b285232fc049 |+---------------------------------------------------------+

c2n1 #

Repeat the 'ovcert -updatetrusted' on the other cluster node which is currently not running the ov-server package.


_____________________________________________________________________________

5.3 Add servers to each other's nodebank with correct ovcoreid

Normally it is desired to have all managed nodes in each other's nodebank. This can easiest be achieved with 'opccfgdwn -backup', followed by 'opcmgrdist' on server A and 'opccfgupld' on server B. For the purpose of testing here only the bare minimum nodes (i.e. the physical and virtual cluster nodes) are added manually via opcnode. Also the virtual node is defined as such 'opcnode -set_virtual' although that is not necessary for the purpose of message forwarding.

Server A:

c1n2 # /opt/OV/bin/OpC/utils/opcnode -add_node node_name=c2n1 \> node_label=c2n1 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.c1n2 # /opt/OV/bin/OpC/utils/opcnode -add_node node_name=c2n2 \> node_label=c2n2 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.c1n2 # /opt/OV/bin/OpC/utils/opcnode -add_node node_name=c2v \> node_label=c2v-ovserver \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.c1n2 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=c2n1 \> id='3f22e490-d54b-7507-0012-8b15c6ae224d'Operation successfully completed.c1n2 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=c2n2 \> id='aba6d168-d58b-7507-1477-cb518338c12f'Operation successfully completed.c1n2 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=c2v \> id='260a0712-d533-7507-1c68-e5d0d06b2196'Operation successfully completed.c1n2 # /opt/OV/bin/OpC/utils/opcnode -list_id \> node_list='c2n1 c2n2 c2v'List of IDs for node(s):Name = c2n1 ID = 3f22e490-d54b-7507-0012-8b15c6ae224dName = c2n2 ID = aba6d168-d58b-7507-1477-cb518338c12fName = c2v ID = 260a0712-d533-7507-1c68-e5d0d06b2196Operation successfully completed.c1n2 # /opt/OV/bin/OpC/utils/opcnode -set_virtual node_name=c2v \> cluster_package=ov-server \> node_list='c2n1 c2n2'Operation successfully completed.c1n2 #


_____________________________________________________________________________

Server B:

c2n1 # /opt/OV/bin/OpC/utils/opcnode -add_node node_name=c1n1 \[2>] node_label=c1n1 \[2>] net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.c2n1 # /opt/OV/bin/OpC/utils/opcnode -add_node node_name=c1n2 \[2>] node_label=c1n2 \[2>] net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.c2n1 # /opt/OV/bin/OpC/utils/opcnode -add_node node_name=c1v \[2>] node_label=c1v-ov-server \[2>] net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.c2n1 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=c1n1 \[2>] id='f7996602-d96c-750a-19f1-972b895012fc'Operation successfully completed.c2n1 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=c1n2 \[2>] id='681b477e-3def-750b-02a3-d3cdd199a983'Operation successfully completed.c2n1 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=c1v \[2>] id='659b2fa0-d93b-750a-0aab-b285232fc049'Operation successfully completed.c2n1 # /opt/OV/bin/OpC/utils/opcnode -list_id \[2>] node_list='c1n1 c1n2 c1v'List of IDs for node(s):Name = c1n1 ID = f7996602-d96c-750a-19f1-972b895012fcName = c1n2 ID = 681b477e-3def-750b-02a3-d3cdd199a983Name = c1v ID = 659b2fa0-d93b-750a-0aab-b285232fc049Operation successfully completed.c2n1 # opcnode -set_virtual node_name=c1v cluster_package=ov-server node_list='c1n1 c1n2'Operation successfully completed.c2n1 #

5.4 Setup message forwarding template and test forwarding

Add a suitable msgtargetrule to the msgforw template, check it with opcmomchk(1m) and place the file into /etc/opt/OV/share/conf/OpC/mgmt_sv/respmgrs on the source server A, e.g.:

MSGTARGETRULEDESCRIPTION "to c2v"

MSGTARGETRULECONDSMSGTARGETRULECOND

DESCRIPTION "c2v"OBJECT "c2v"


TIMETEMPLATE "$OPC_ALWAYS" OPCMGR IP 0.0.0.0 "c1v"

MSGTARGETMANAGERTIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "c2v"MSGCONTROLLINGMGR


_____________________________________________________________________________

Then restart the management server processes on server A to activate the message forwarding. Note that this is a cluster, so normally this would fail over the HA package, unless the cluster servive monitoring isn't stopped in before:

c1n2 # /opt/OV/lbin/ovharg -monitor ov-server disablec1n2 # opcsv -startc1n2 # /opt/OV/lbin/ovharg -monitor ov-server enable

Send a message and verify that it is displayed in the message browser of the target server:

c1n2 # opcmsg a=a o=c2v msg_t="forwarded from c1v"

5.5 Configure managed nodes for switching primary manager




_____________________________________________________________________________

6 Server Pooling in OMU 8

The concept of Server Pooling is described in this White Paper available from the doc server http://ovweb.external.hp.com/lpe/doc_serv/ in the product Operations Manager for UNIX, version 8.0:

High Availability through OVO/UNIX Server Pooling

The page numbers refer to the edition 2 of that White Paper from January 2007.

The chapter here merely makes some additions / clarifications, especially for the case where 2 virtual interfaces (here named vpool1 and vpool2) are configured. The servers used here are su1and su3, both HP-UX PA-RISC.

6.1 Configuring Management Server Nodes

Page 19, Configuring Management Server Nodes: This means to setup trusted certificates between both management servers, as outlined in this cookbook, chapter “Message forwarding between two 8.x standalone servers”.

The certificates afterwards are:

su1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 |+---------------------------------------------------------+


su1 #


_____________________________________________________________________________

And on the server B:

su3 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || ae33c7ea-94b0-7525-04ef-cbab70bb7252 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 |+---------------------------------------------------------+


su3 #

6.2 Configuring Virtual Interfaces

Page 20, Configuring Virtual Interface: The White Paper describes the scenario for adding just a single virtual interface. In this cookbook the scenario 2 with two virtual interfaces (the virtual hostnames shall be vpool1 and vpool2) is outlined. The new resource groups are named virt1 and virt2.

1. Create resource groups

su1 # /opt/OV/lbin/xpl/init_ovrg.sh virt1su1 # /opt/OV/lbin/xpl/init_ovrg.sh virt2

su3 # /opt/OV/lbin/xpl/init_ovrg.sh virt1su3 # /opt/OV/lbin/xpl/init_ovrg.sh virt2

2. Create ovcoreid for both resource groups

su1 # /opt/OV/bin/ovcoreid -create -ovrg virt1NOTE: OvCoreId was set to '17e53638-921e-7529-11d0-cce2939436c5'.su1 # /opt/OV/bin/ovcoreid -ovrg virt1 > /tmp/virt1.coreidsu1 # /opt/OV/bin/ovcoreid -create -ovrg virt2NOTE: OvCoreId was set to 'd12263a4-921f-7529-039b-cc8b008de26d'.su1 # /opt/OV/bin/ovcoreid -ovrg virt2 > /tmp/virt2.coreidsu1 #

Both files are copied to su3, then:


_____________________________________________________________________________

su3 # /opt/OV/bin/ovcoreid -set `cat /tmp/virt1.coreid` -ovrg virt1NOTE: OvCoreId was set to '17e53638-921e-7529-11d0-cce2939436c5'.su3 # /opt/OV/bin/ovcoreid -set `cat /tmp/virt2.coreid` -ovrg virt2NOTE: OvCoreId was set to 'd12263a4-921f-7529-039b-cc8b008de26d'.su3 #

3. Issue certificates

su1 # /opt/OV/bin/ovcm -issue -file /tmp/virt1.cert -name vpool1 \> -pass virt1 -coreid `cat /tmp/virt1.coreid`INFO: Issued certificate was written to file '/tmp/virt1.cert'.su1 # /opt/OV/bin/ovcm -issue -file /tmp/virt2.cert -name vpool2 \> -pass virt2 -coreid `cat /tmp/virt2.coreid`INFO: Issued certificate was written to file '/tmp/virt2.cert'.su1 #

4. Import certificates

su1 # /opt/OV/bin/ovcert -importcert -ovrg virt1 -file /tmp/virt1.cert \> -pass virt1WARNING: The common name field (CN) in the certificate

'17e53638-921e-7529-11d0-cce2939436c5' does not match the OvCoreId'7681325c-c1a9-7508-0441-a54412c264de' of the system.

INFO: Import operation was successful.su1 # /opt/OV/bin/ovcert -importcert -ovrg virt2 -file /tmp/virt2.cert \> -pass virt2WARNING: The common name field (CN) in the certificate

'd12263a4-921f-7529-039b-cc8b008de26d' does not match the OvCoreId '7681325c-c1a9-7508-0441-a54412c264de' of the system.

INFO: Import operation was successful.su1 #

Verify certificates:

su1 # /opt/OV/bin/ovcert -list -ovrg virt1+---------------------------------------------------------+| Keystore Content (OVRG: virt1) |+---------------------------------------------------------+| Certificates: || 17e53638-921e-7529-11d0-cce2939436c5 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 |+---------------------------------------------------------+

su1 # /opt/OV/bin/ovcert -list -ovrg virt2+---------------------------------------------------------+| Keystore Content (OVRG: virt2) |+---------------------------------------------------------+| Certificates: || d12263a4-921f-7529-039b-cc8b008de26d (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 |+---------------------------------------------------------+

su1 #


_____________________________________________________________________________

Copy certificate files to server B (su3) and import them there:

su3 # /opt/OV/bin/ovcert -importcert -ovrg virt1 -file /tmp/virt1.cert \> -pass virt1WARNING: The common name field (CN) in the certificate

'17e53638-921e-7529-11d0-cce2939436c5' does not match the OvCoreId'ae33c7ea-94b0-7525-04ef-cbab70bb7252' of the system.

INFO: Import operation was successful.su3 # /opt/OV/bin/ovcert -importcert -ovrg virt2 -file /tmp/virt2.cert \> -pass virt2WARNING: The common name field (CN) in the certificate

'd12263a4-921f-7529-039b-cc8b008de26d' does not match the OvCoreId'ae33c7ea-94b0-7525-04ef-cbab70bb7252' of the system.

INFO: Import operation was successful.su3 #

Verify certificates:

su3 # /opt/OV/bin/ovcert -list -ovrg virt1+---------------------------------------------------------+| Keystore Content (OVRG: virt1) |+---------------------------------------------------------+| Certificates: || 17e53638-921e-7529-11d0-cce2939436c5 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 |+---------------------------------------------------------+

su3 # /opt/OV/bin/ovcert -list -ovrg virt2+---------------------------------------------------------+| Keystore Content (OVRG: virt2) |+---------------------------------------------------------+| Certificates: || d12263a4-921f-7529-039b-cc8b008de26d (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 |+---------------------------------------------------------+

su3 #

5. Bind the virtual IP addresses to the virtual resource groups

The IP addresses are those of the hosts vpool1 and vpool2.

su1 # /opt/OV/bin/ovconfchg -ovrg virt1 -ns bbc.cb \> -set SERVER_BIND_ADDR 16.58.25.121 -set SERVER_PORT 383su1 # /opt/OV/bin/ovconfchg -ovrg virt2 -ns bbc.cb \> -set SERVER_BIND_ADDR 16.58.25.122 -set SERVER_PORT 383su1 #

su3 # /opt/OV/bin/ovconfchg -ovrg virt1 -ns bbc.cb \> -set SERVER_BIND_ADDR 16.58.25.121 -set SERVER_PORT 383su3 # /opt/OV/bin/ovconfchg -ovrg virt2 -ns bbc.cb \> -set SERVER_BIND_ADDR 16.58.25.122 -set SERVER_PORT 383su3 #


_____________________________________________________________________________

6. Add the virtual interfaces to the node banks:

su1 # /opt/OV/bin/OpC/utils/opcnode -add_node \> node_name=vpool1 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC \> group_name=hp_uxOperation successfully completed.su1 # /opt/OV/bin/OpC/utils/opcnode -add_node \> node_name=vpool2 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC \> group_name=hp_uxOperation successfully completed.su1 #

su3 # /opt/OV/bin/OpC/utils/opcnode -add_node \> node_name=vpool1 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC \> group_name=hp_uxOperation successfully completed.su3 # /opt/OV/bin/OpC/utils/opcnode -add_node \> node_name=vpool2 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC \> group_name=hp_uxOperation successfully completed.su3 #

7. Set ovcoreid for both virtual interfaces:

su1 # /opt/OV/bin/OpC/utils/opcnode -chg_id \> node_name=vpool1 id=`cat /tmp/virt1.coreidÒperation successfully completed.su1 # /opt/OV/bin/OpC/utils/opcnode -chg_id \> node_name=vpool2 id=`cat /tmp/virt2.coreidÒperation successfully completed.su1 #

su3 # /opt/OV/bin/OpC/utils/opcnode -chg_id \> node_name=vpool1 id=`cat /tmp/virt1.coreidÒperation successfully completed.su3 # /opt/OV/bin/OpC/utils/opcnode -chg_id \> node_name=vpool2 id=`cat /tmp/virt2.coreidÒperation successfully completed.su3 #

8. Activate virtual interfaces

In this example vpool1 is activated on server A, vpool2 on server B.

su1 # ifconfig lan0:1 inet 16.58.25.121 \> netmask 255.255.254.0 upsu1 #

su3 # ifconfig lan0:1 inet 16.58.25.122 \> netmask 255.255.254.0 upsu3 #


_____________________________________________________________________________

9. Start the resource groups

su1 # /opt/OV/bin/ovbbccb -start virt1

NOTE: The HP OpenView resource group 'virt1' on node 'localhost' started.

su1 #

su3 # /opt/OV/bin/ovbbccb -start virt2


su3 #

6.3 Configuring Primary Manager

1. Create the node-specific mgrconf file

cp /etc/opt/OV/share/conf/OpC/mgmt_sv/tmpl_respmgrs/backup-server \/etc/opt/OV/share/conf/OpC/mgmt_sv/work_respmgrs/allnodes

The file is then edited to:

## Responsible Manager Configurations for a backup server#RESPMGRCONFIGS

RESPMGRCONFIGDESCRIPTION "responsible mgrs for agents reporting to vpool*"

SECONDARYMANAGERSSECONDARYMANAGER

NODE IP 0.0.0.0 "su1"SECONDARYMANAGERNODE IP 0.0.0.0 "su3"SECONDARYMANAGERNODE IP 0.0.0.0 "vpool1 "SECONDARYMANAGER

NODE IP 0.0.0.0 "vpool2 "ACTIONALLOWMANAGERS

ACTIONALLOWMANAGERNODE IP 0.0.0.0 "su1"ACTIONALLOWMANAGERNODE IP 0.0.0.0 "su3"ACTIONALLOWMANAGERNODE IP 0.0.0.0 "vpool1 "ACTIONALLOWMANAGERNODE IP 0.0.0.0 "vpool2 "

2. Verify syntax correctness

su1 # /opt/OV/bin/OpC/opcmomchk /etc/opt/OV/share/conf/OpC/mgmt_sv/work_respmgrs/allnodesusing singlebyte mode

Syntax of /etc/opt/OV/share/conf/OpC/mgmt_sv/work_respmgrs/allnodes is OK.su1 #


_____________________________________________________________________________

3. Copy the allnodes file

su1 # cd /etc/opt/OV/share/conf/OpC/mgmt_sv/work_respmgrssu1 # cp allnodes ../respmgrssu1 #

4. Distribute the files on server A

su1 # /opt/OV/bin/OpC/opcragt -distrib -templates su1Node su1:Create distribution data and inform agent...Done.

su1 # /opt/OV/bin/ovpolicy -list* List installed policies for host 'localhost'.

Type Name Status Version--------------------------------------------------------------------…mgrconf "OVO authorization" enabled 1

…

5. Distribute the files on server B

After copying the allnodes file from su1 to su3:

su3 # /opt/OV/bin/OpC/opcragt -distrib -templates su3Node su3:Create distribution data and inform agent...Done.

su3 # /opt/OV/bin/ovpolicy -list* List installed policies for host 'localhost'.

Type Name Status Version--------------------------------------------------------------------…mgrconf "OVO authorization" enabled 1

…

su3 #

6.4 Configuring Message Forwarding

The message forwarding template msgforw is setup like documented in the chapter “Message forwarding between two 8.x standalone servers” and also like documented in the White Paper “Server Pooling”.

6.5 Configuring Managed Nodes

There is one step missing in the “Server Pooling” White Paper for the case where existing HTTPS agent are switched over to the virtual interface:


_____________________________________________________________________________

After distributing the mgrconf authorization in steps 1 and 2, it is necessary to update the trusted certificates on the nodes, e.g.:

hn2 # ovcert -updatetrustedINFO: Trusted certificate update was successful.hn2 #

If this is not done, an error may happen later on, see the chapter Troubleshooting, Trying opcragt from a backup server.

After that is done, the step 3 (Instruct each managed node that its primary manager is the Vvirtual interface) from the White Paper succeeds, regardless from which server it is done:

su3 # /opt/OV/bin/OpC/opcragt -set_config_var \> eaagt:OPC_PRIMARY_MGR=vpool1 hn3Node hn3:Done.

su3 # /opt/OV/bin/OpC/opcragt -get_config_var eaagt:OPC_PRIMARY_MGR hn3Node hn3:OPC_PRIMARY_MGR = vpool1Done.

su3 #

6.6 Moving Virtual Interface to Another Physical Server

If everything works correct, then the nodes send their messages to their configured virtual management server now, and through message forwarding they are seen in the message browser of both physical servers.

To see which virtual interface is currently running where, these commands can be used:

su1 # /opt/OV/bin/ovbbccb -listovrg

NOTE: HP OpenView resource groups on node 'localhost':

virt1 383 16.58.25.121 1

You have mail in /var/mail/rootsu1 # netstat -inName Mtu Network Address Ipkts Ierrs Opkts Oerrs Colllan0:1 1500 16.58.24.0 16.58.25.121 19459 0 16343 0 0lan0 1500 16.58.24.0 16.58.24.137 7823163 0 7721471 0 0lo0 4136 127.0.0.0 127.0.0.1 4030141 0 4030141 0 0su1 #

And on the other server:

su3 # /opt/OV/bin/ovbbccb -listovrg

NOTE: HP OpenView resource groups on node 'localhost':

virt2 383 16.58.25.122 0


_____________________________________________________________________________

duplo # netstat -inName Mtu Network Address Ipkts Ierrs Opkts Oerrs Colllan0:1 1500 16.58.24.0 16.58.25.122 16 0 0 0 0lan0 1500 16.58.24.0 16.58.24.82 31473830 0 17523809 0 0lo0 4136 127.0.0.0 127.0.0.1 17475239 0 17475239 0 0su3 #

If the physical server to which that virtual interface is currently bound, doesn’t have the OMUserver processes running (e.g. after a ovstop), then the messages can’t be sent to the virtual server, and the message agent on the node starts buffering:

su3 # ovstopsu3 # opcragt -status hn2Node hn2:OVO Managed Node status :-------------------------OV Control ovcd (25917) is runningOV Communication Broker ovbbccb (25921) is runningOV Config and Deploy ovconfd (26228) is runningOV Performance Core coda (26215) is runningSubagent EA:Message Agent opcmsga (buffers) (26232) is runningAction Agent opcacta (26233) is runningMessage Interceptor opcmsgi (26234) is running

Message Agent buffering for the following servers :---------------------------------------------------vpool2Done.

su3 #

So the virtual interface can now be moved to the other server, to have the messages displayed.

Stopping resource group and virtual interface:

su3 # /opt/OV/bin/ovbbccb -stop virt2

NOTE: The HP OpenView resource group 'virt2' on node 'localhost' was stopped.

su3 # ifconfig lan0:1 inet 0.0.0.0 downsu3 #

Start resource group and virtual interface on the other server:

As there is already the resource group virt1 running on interface lan0:1, the resource group virt2 needs to be assigned a new index for the interface:

su1 # netstat -inName Mtu Network Address Ipkts Ierrs Opkts Oerrs Colllan0:1 1500 16.58.24.0 16.58.25.121 72 0 0 0 0lan0 1500 16.58.24.0 16.58.24.137 367788542 0 330003785 0 0lo0 4136 127.0.0.0 127.0.0.1 181539801 0 181539799 0 0su1 # ifconfig lan0:2 inet 16.58.25.122 netmask 255.255.254.0 upsu1 # /opt/OV/bin/ovbbccb -start virt2



_____________________________________________________________________________

su1 # netstat -inName Mtu Network Address Ipkts Ierrs Opkts Oerrs Colllan0:1 1500 16.58.24.0 16.58.25.121 72 0 0 0 0lan0:2 1500 16.58.24.0 16.58.25.122 12 0 4 0 0lan0 1500 16.58.24.0 16.58.24.137 367788542 0 330003785 0 0lo0 4136 127.0.0.0 127.0.0.1 181539801 0 181539799 0 0su1 #

After a short while, the message agent on the managed node successfully sends the buffered messages to vpool2 again, i.e. the messages are displayed.


_____________________________________________________________________________

7 Agent-based message forwarding from OVOW 7 to OMU 8

There are several scenarios possible, when, why, and how to send some messages from a OVOW 7 managed node to a OMU server, e.g. follow-the-sun control, or competence centers. For description see the OVOW 7.5 online help, Administering your environment, Scalable architecture for multiple management servers, Agent-based flexible management.

Here an example of a competence center is shown:Two Unix cluster nodes, c3n1 and c3n2, are managed by the OVOW 7.5 server sw75, and have a clustered SAP system installed. The messages from SAP SPI, i.e. all messages from the SAP application, shall be sent to the OMU 8 server su1, all other messages to the normal OVOW 7 server sw75.

The following notes are based on the above mentioned chapter in the OVOW 7.5 online help.

7.1 Create an agent-based flexible management policy

The template created in /etc/opt/OV/share/conf/OpC/mgmt_sv/work_respmgrs is called competence-center:

# This template sets the following configuration:## - send SAP related messages to Unix management server su1# - send other messages to the primary manager of that node# - allow both servers to run actions on the node#

TIMETEMPLATES# none

RESPMGRCONFIGSRESPMGRCONFIGDESCRIPTION "responsible mgrs for messages and agents"SECONDARYMANAGERSSECONDARYMANAGER

NODE IP 0.0.0.0 "su1"DESCRIPTION "HP OpenView Operations for Unix Management Server"

SECONDARYMANAGERNODE IP 0.0.0.0 "sw75"DESCRIPTION "HP OpenView Operations for Windows Management Server"

ACTIONALLOWMANAGERSACTIONALLOWMANAGER


ACTIONALLOWMANAGERNODE IP 0.0.0.0 "sw75"DESCRIPTION "HP OpenView Operations for Windows Management Server"

MSGTARGETRULESMSGTARGETRULEDESCRIPTION "SAP responsibility for matched SAP SPI messages"

MSGTARGETRULECONDSMSGTARGETRULECONDDESCRIPTION "SAP messages"APPLICATION "R/3 EP7 78"



_____________________________________________________________________________


MSGTARGETRULEDESCRIPTION "SAP responsibility for unmatched SAP SPI messages"

MSGTARGETRULECONDSMSGTARGETRULECONDDESCRIPTION "SAP messages"APPLICATION "OPC/SAP"



MSGTARGETRULEDESCRIPTION "all other messages"MSGTARGETRULECONDS

MSGTARGETMANAGERSMSGTARGETMANAGERTIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "$OPC_PRIMARY_MGR"

7.2 Configure Agents to communicate with OpenView Operations for UNIX

7.2.1 Prepare the OpenView Operations for UNIX server

Ensure to have the managed node(s) in the node bank (and suitable node group bank) in the OMU 8 server. Be careful to choose the DCE versions (i.e. the ones that *don’t* have the‘HTTPS’ comment in the machine type) in the ‘Add Node’ operation. Also the node type needs to be ‘Controlled”.

su1 # /opt/OV/bin/OpC/opcsw -installed c3n1103943a8su1 # /opt/OV/bin/OpC/opcsw -installed c3n2103943a9su1 # /opt/OV/bin/OpC/opchbp -start c3n1 c3n2

Node c3n1 ... unchanged - HBP already on.Node c3n2 ... unchanged - HBP already on.su1 #

After checking the syntax of the template with opcmomchk, copy the file to the resp_mgrs directory, either as allnodes file, or with the hex name of the managed node:

su1 # opcmomchk competence-centerusing singlebyte mode

Syntax of competence-center is OK.su1 # /opt/OV/bin/OpC/install/opc_ip_addr c3n1 c3n2c3n1 = 16.57.67.168 = 103943a8c3n2 = 16.57.67.169 = 103943a9su1 # cp competence-center ../respmgrs/103943a8 ../respmgrs/103943a9su1 #


_____________________________________________________________________________

7.2.2 Change the management server that is responsible for the agent

The path noted in the OVOW 7.5 online help for mgmt_sv.sh (on a UNIX node) is wrong. The correct path is: /opt/OV/bin/OpC/install/mgmt_sv.sh

The script has to be executed on each managed node. It will ask interactively for the new management server (su1 in this example). It will then change the opcinfo file on the managed node and change this line:OPC_MGMT_SERVER su1

7.2.3 Distribute the agent-based flexible management template(s) to the appropriate managed nodes

su1 # opcragt -distrib -templates -force c3n1 c3n2Node c3n1:Create distribution data and inform agent...Done.

Node c3n2:Create distribution data and inform agent...Done.su1 #

The correct distribution can be verified on the managed node with e.g.:

[root@c3n1]# ls -l /var/opt/OV/conf/OpC/mgrconf-rw------- 1 root sys 1554 Jan 22 14:24 /var/opt/OV/conf/OpC/mgrconf[root@c3n1]#

7.2.4 To switch agent back to OpenView Operations for Windows

After the switching back, the agent processes should be restarted with ‘opcagt –start’:

[root@c3n1]# /opt/OV/bin/OpC/install/mgmt_sv.shFully qualified system name of Management Server: sw75Updating Management Server info in OpC version file /opt/OV/bin/OpC/install/opcinfo

############################################################################### File: opcinfo# Description: Installation Information of ITO Managed Node# Package: HP OpenView IT/Operations##############################################################################OPC_INSTALLED_VERSION A.07.33PERF_INSTALLED_VERSION A.07.27SVCDISC_INSTALLED_VERSION A.07.28COMM_INSTALLED_VERSION 2.6.8.0OPC_MGMT_SERVER sw75OPC_INSTALLATION_TIME 09/19/07 17:48:09OPC_SG FALSEOPC_IP_ADDRESS 16.57.67.169OPC_SET_PROXY_FLAG_FOR_IP_ADDRESSES 16.57.67.170

everything done - bye, bye[root@c3n1]# opcagt –start[root@c3n1]#


_____________________________________________________________________________

The same has to be done for the other managed node(s) as well.

7.3 Verification of correct message forwarding

If the messages from the OMW managed node are not arriving in the OMU message browser, then at first the agent can be traced. In opcinfo these settings could be used:

OPC_TRACE TRUEOPC_TRACE_TRUNC FALSEOPC_TRACE_AREA MSGOPC_TRC_PROCS opcmsga

The trace should then contain for a message that is to be sent to the OMU server (the real message text has been replaced by the string “msg text” here for clarity; and 16.58.24.137 is the IP address of the destination server su1):

01/28 15:00:07.410 opcmsga(21263:001)[MSG]: Message/Act.Resp. received from agents: 550178fe-cda9-71dc-17ac-103943aa0000 R3_CCMS 'msg text' 16.57.67.17001/28 15:00:07.413 opcmsga(21263:001)[MSG]: OpC mgr for msg: 550178fe-cda9-71dc-17ac-103943aa0000 R3_CCMS 'msg text' 16.57.67.170 opcmgr : su1 16.58.24.13701/28 15:00:07.413 opcmsga(21263:001)[MSG]: forwarding msg: 550178fe-cda9-71dc-17ac-103943aa0000 R3_CCMS 'msg text' 16.57.67.170 opcmgr : su1 16.58.24.13701/28 15:00:07.496 opcmsga(21263:001)[MSG]: Sending msg (len = 851): msg text01/28 15:00:07.532 opcmsga(21263:001)[MSG]: Message forwarded: 550178fe-cda9-71dc-17ac-103943aa0000 R3_CCMS 'msg text' 16.57.67.170 opcmgr : su1 16.58.24.137


_____________________________________________________________________________

8 Agent-based message forwarding from OMU 8 to OVOW 7

Communication from agents, managed by OMU 8 management servers, to OVOW 7 management servers, is of course possible only for DCE (OVOU 7) agents, as OVOW 7 knows only that communication protocol. As example the node dn4 is used here.

8.1 Create an agent-based flexible management policy

As in the example of the previous chapter, a competence center approach is shown here. The responsible manager configuration is enhanced to include the "OMW responsibility" message target rule. This way the same file can be used for both OMU and OVOW nodes, but for this specific chapter the message target rules "SAP responsibility for matched SAP SPI messages"and "SAP responsibility for unmatched SAP SPI messages" are not used.

Create the file competence-center in /etc/opt/OV/share/conf/OpC/mgmt_sv/work_respmgrs:

# This template sets the following configuration:## - send SAP related messages to Unix management server su1# - send OMW related messages to Windows management server sw75# - send other messages to the primary manager of that node# - allow both servers to run actions on the node#

TIMETEMPLATES# none

RESPMGRCONFIGSRESPMGRCONFIGDESCRIPTION "responsible mgrs for messages and agents"SECONDARYMANAGERSSECONDARYMANAGER






MSGTARGETRULESMSGTARGETRULEDESCRIPTION "SAP responsibility for matched SAP SPI messages"

MSGTARGETRULECONDSMSGTARGETRULECONDDESCRIPTION "SAP messages"APPLICATION "R/3 EP7 78"

MSGTARGETMANAGERS MSGTARGETMANAGER


MSGTARGETRULEDESCRIPTION "SAP responsibility for unmatched SAP SPI messages"


_____________________________________________________________________________

MSGTARGETRULECONDSMSGTARGETRULECOND

DESCRIPTION "SAP messages"APPLICATION "OPC/SAP"



MSGTARGETRULEDESCRIPTION "OMW responsibility"

MSGTARGETRULECONDSMSGTARGETRULECONDDESCRIPTION "OMW messages"APPLICATION "OMW"


TIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "sw75"

MSGTARGETRULEDESCRIPTION "all other messages"

MSGTARGETRULECONDSMSGTARGETMANAGERSMSGTARGETMANAGER

TIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "$OPC_PRIMARY_MGR"

Then check its syntax:

su1 # /opt/OV/bin/OpC/opcmomchk competence-centerusing singlebyte mode

Syntax of competence-center is OK.su1 #

8.2 Configure OMU Agents to communicate with OVOW 7

Copy the file to the respmgrs directory, either as “allnodes” or as node-specific file:

su1 # /opt/OV/bin/OpC/install/opc_ip_addr dn4dn4 = 16.58.24.142 = 103a188esu1 # cp competence-center ../respmgrs/103a188esu1 #

And distribute it to the node, then verify that the file has arrived correctly on the managed node:

su1 # opcragt -distrib -templates dn4Node dn4:Create distribution data and inform agent...Done.

su1 # ll 103a188e-r--r--r-- 1 root sys 2369 Jan 31 13:28 103a188esu1 #

dn4 # ll /var/opt/OV/conf/OpC/mgrconf-rw------- 1 root sys 2369 Jan 31 13:29 /var/opt/OV/conf/OpC/mgrconfdn4 #


_____________________________________________________________________________

8.3 Prepare the OVOW 7 management server

Add the node dn4 to the OVOW 7 management server. In the “Configure Nodes” add the new node, in the Network tab ensure to have the “Disable Auto Deployment” checkbox enabled. Check the System tab to be correct.

8.4 Verification of correct message forwarding

Send a message with application OMW from the node dn4. If it doesn’t arrive in the OVOW 7 browser, then enable tracing on the node dn4 in opcinfo:

OPC_TRACE TRUEOPC_TRACE_TRUNC FALSEOPC_TRACE_AREA MSGOPC_TRC_PROCS opcmsga

# opcagt –trace# opcmsg a=a o=o msg_t="msg for su1"# opcmsg a=OMW o=o msg_t="msg for sw75"#

The trace then should contain:

01/31 14:54:56.599 opcmsga(22265:001)[MSG]: Message/Act.Resp. received from agents: 1b0440fc-d004-71dc-1dbc-103a188e0000 <empty> 'msg for su1' 16.58.24.14201/31 14:54:56.619 opcmsga(22265:001)[MSG]: OpC mgr for msg: 1b0440fc-d004-71dc-1dbc-103a188e0000 <empty> 'msg for su1' 16.58.24.142 opcmgr : su1 16.58.24.13701/31 14:54:56.619 opcmsga(22265:001)[MSG]: forwarding msg: 1b0440fc-d004-71dc-1dbc-103a188e0000 <empty> 'msg for su1' 16.58.24.142 opcmgr : su1 16.58.24.13701/31 14:54:56.645 opcmsga(22265:001)[MSG]: Sending msg (len = 163): msg for su101/31 14:54:56.651 opcmsga(22265:001)[MSG]: Message forwarded: 1b0440fc-d004-71dc-1dbc-103a188e0000 <empty> 'msg for su1' 16.58.24.142 opcmgr : su1 16.58.24.13701/31 14:55:10.180 opcmsga(22265:001)[MSG]: Message/Act.Resp. received from agents: 231c94ec-d004-71dc-1dbc-103a188e0000 <empty> 'msg for sw75' 16.58.24.14201/31 14:55:10.181 opcmsga(22265:001)[MSG]: OpC mgr for msg: 231c94ec-d004-71dc-1dbc-103a188e0000 <empty> 'msg for sw75' 16.58.24.142 opcmgr : sw75 16.58.24.8801/31 14:55:10.195 opcmsga(22265:001)[MSG]: forwarding msg: 231c94ec-d004-71dc-1dbc-103a188e0000 <empty> 'msg for sw75' 16.58.24.142 opcmgr : sw75 16.58.24.8801/31 14:55:10.200 opcmsga(22265:001)[MSG]: Sending msg (len = 168): msg for sw7501/31 14:55:10.206 opcmsga(22265:001)[MSG]: Message forwarded: 231c94ec-d004-71dc-1dbc-103a188e0000 <empty> 'msg for sw75' 16.58.24.142 opcmgr : sw75 16.58.24.88


_____________________________________________________________________________

9 Server based message forwarding from OVOW 7.5 to OMU 8

This mainly is described in the OVOW 7.5 online help, Administering your environment, Scalable architecture for multiple management servers, Server-based flexible management.Some additional explanations:

Basically the steps mentioned in “Configure server-based flexible management” have to be followed.

The OMU server patch level needs to be at least A.08.12. Then the “Unix Compatibility” should be set to 0 in the registry (default is 1). For details see the “Configure using registry keys” in the above mentioned online help chapter.

9.1 Configure OVOW 7.5 source server

This is described in the subchapters “Configure MsgForwarding.ini” with the several subchapters immediately following, about details, examples, syntax of the MsgForwarding.ini file.

There is a typo: The example files for MsgForwarding.ini are located in:C:\Program Files\HP OpenView\examples\OvOW\Server-based flexible management

In the example used here, messages from the OSSPI application and from a specific list of nodes (the cluster nodes c3n1 and c3n2) are forwarded from the OVOW 7.5 server sw75 to the OMU 8 server su3.

The file %OvShareDir%\conf\MsgActSrv\MsgForwarding.ini is then created as:

TIMETEMPLATES # none as time templates are not supported on OVOWRESPMGRCONFIGS

RESPMGRCONFIG DESCRIPTION "msg-forwarding target specification"SECONDARYMANAGERS# for server-based flexible management on OVOW 7.50 this is the list of servers

to which all message operations are forwarded.SECONDARYMANAGER NODE IP 0.0.0.0 "su1"SECONDARYMANAGER NODE IP 0.0.0.0 "su3"

ACTIONALLOWMANAGERS # empty and disregarded, but syntactically required sectionMSGTARGETRULESMSGTARGETRULE DESCRIPTION "OSSPI"MSGTARGETRULECONDS

MSGTARGETRULECOND DESCRIPTION OSSPI messages"APPLICATION "HP OSSPI"NODE IP 0.0.0.0 "c3n1"

IP 0.0.0.0 "c3n2"MSGTARGETMANAGERS

MSGTARGETMANAGERTIMETEMPLATE "$OPC_ALWAYS" OPCMGR IP 0.0.0.0 "su3"

Use the ovowmomchk.exe tool to verify that the MsgForwarding.ini has a valid syntax. Then the Message Action Server has to be restarted such that the new MsgForwarding.ini is read.


_____________________________________________________________________________

9.2 Configure OMU 8 target server

All managed nodes from the OVOW 7.5 server, for which messages shall be forwarded, have to be present in the OMU 8 server’s node bank. Ensure to use the correct OS type and machine type, also the DCE versions (i.e. those that have no “HTTPS” in the machine type) has to be selected. The nodes may be of type “Message Allowed”. Also ensure to put these nodes into suitable node groups, for which the responsibility matrix of the operators is set to see the messages.

Also the source server sw75 has to be entered this way. For this the node type should be “Controlled”. Adding this source server to the node bank of the target server is not necessary for forwarding the messages to the target server. But if the messages are modified (e.g. owned or acknowledged) on the target server, this message change event can only be routed back to the source server, i.e. the synchronization can only take place, if the source server is present in the target’s node bank.

9.3 Verification of forwarded messages

On the managed node trigger a message that should then be first sent to sw75, and from there forwarded to su3.

If the messages are shown in the browser of sw75 but not shown in the browser of su3, thenfirst check whether the message has been forwarded from sw75 to su3. For this the tracemon utility can be used. As application select the OvEpMsgActSrv process. The relevant component for forwarding then is OvEpDceSnd. The trace then should contain for this message:

"Application" "Component" "TraceMsg",OvEpMsgActSrv OvEpDceSnd "COvEpDceSnd::Indicate(IOvEpMessage*)(su3)"OvEpMsgActSrv OvEpDceSnd "COvEpDceSnd::TransMsg(IOvEpMessage*)"OvEpMsgActSrv OvEpDceSnd "Node Name: c3n2"OvEpMsgActSrv OvEpDceSnd "COvEpDceSnd::AddServerList()"OvEpMsgActSrv OvEpDceSnd "Node Name: sw75"OvEpMsgActSrv OvEpDceSnd "Server List added"OvEpMsgActSrv OvEpDceSnd "COvEpDceSnd::DceConnect(su3)"OvEpMsgActSrv OvEpDceSnd "DCE-Forwarder: Connected to target server su3"OvEpMsgActSrv OvEpDceSnd "COvEpDceSnd::DceSendMsg(su3)"OvEpMsgActSrv OvEpDceSnd "Flexible attribute handling"OvEpMsgActSrv OvEpDceSnd "Message sent to su3"OvEpMsgActSrv OvEpDceSnd "DCE Sender stopped, yes(1)/no(0): 0x0"

Also in before enable tracing of the target server su3 with “ovconfchg -ovrg server –edit” and set these entries in the [opc] name space:

OPC_TRACE=TRUEOPC_TRACE_TRUNC=FALSEOPC_TRACE_AREA=MSGOPC_TRC_PROCS=opcmsgm,opcdispm

In the /var/opt/OV/share/tmp/OpC/mgmt_sv/trace file there then should be entries like this:


_____________________________________________________________________________

02/01 15:47:59.472 opcmsgm(26696:001)[msg]: Message received from message receiver: ba0475f8-d0d3-71dc-09da-c0a800020000 OS 'msg text' IP 16.57.67.169 c3n202/01 15:47:59.473 opcmsgm(26696:001)[msg]: message accepted: ba0475f8-d0d3-71dc-09da-c0a800020000 OS 'msg text' IP 16.57.67.169 c3n202/01 15:47:59.513 opcmsgm(26696:001)[msg]: Message forwarded to DM: ba0475f8-d0d3-71dc-09da-c0a800020000 OS 'msg text' IP 16.57.67.169 c3n202/01 15:47:59.531 opcdispm(26699:00a)[msg]: Message received: ba0475f8-d0d3-71dc-09da-c0a800020000 OS 'msg text' 16.57.67.16902/01 15:47:59.531 opcdispm(26699:00a)[msg]: Forwarding message ba0475f8-d0d3-71dc-09da-c0a800020000 to user opc_adm


_____________________________________________________________________________

10 Message forwarding from OMU 8 to OVOW 7.5

This works very similar to the message forwarding from OMU 8 to OVOU 7, see the chapter:Message forwarding from OMU 8 to OVOU 7 server and vice versa

The basic steps are:

On the OMU 8 server setup /etc/opt/OV/share/conf/OpC/mgmt_sv/respmgrs/msgforw with e.g.:

TIMETEMPLATES# none

RESPMGRCONFIGSRESPMGRCONFIGDESCRIPTION "msg-forwarding target specification"

MSGTARGETRULESMSGTARGETRULEDESCRIPTION "to sw75 (OVOW 7.5)"


DESCRIPTION "OVOW75"OBJECT "OVOW75"


TIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "sw75"MSGCONTROLLINGMGR


OPCMGR IP 0.0.0.0 "s3"MSGCONTROLLINGMGR

Check the syntax with opcmomchk(1m), then restart the server processes (ovstop opc; ovstart).

Add the managed notes, whose messages shall be forwarded, to the node bank of the OVOW 7.5 target server.

Test the successful forwarding with a suitable message from the managed node(s).


_____________________________________________________________________________

11 Message forwarding from OMW 8 to OMU 8

For general information about this please also see the OMW 8 Online Help: Administering your environment, Scalable Architecture for Multiple Management Servers, Server-based flexible management.

The servers being used in this example are:

source server: sw80 management server with OMW 8.0 (Windows 2003)target server: su1 management server with OMU 8 (HP-UX PA-RISC)


Note that each server may contain more trusted certificates (on the agent as well as on the server side). For clarity they have been removed from the output shown below.

11.1.1 Certificates on source server

C:\>ovcoreid2c1cf5f2-98f0-752b-0c72-ee508271ab3b

C:\>ovcoreid -ovrg server2c1cf5f2-98f0-752b-0c72-ee508271ab3b

C:\>ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 2c1cf5f2-98f0-752b-0c72-ee508271ab3b (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_2c1cf5f2-98f0-752b-0c72-ee508271ab3b |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 2c1cf5f2-98f0-752b-0c72-ee508271ab3b (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_2c1cf5f2-98f0-752b-0c72-ee508271ab3b (*) |+---------------------------------------------------------+

C:\>


_____________________________________________________________________________

11.1.2 Certificates on target server



su1 #


11.2.1 export trusted certificates on source server

C:\>ovcert -exporttrusted -file \temp\sw80.cert -ovrg serverINFO: Trusted certificates have been successfully exported to file '\temp\

sw80.cert'.

C:\>

11.2.2 export trusted certificates on target server


su1.cert'.su1 #


Copy /tmp/su1.cert to sw80 into \temp, and \temp\sw80.cert to su1 into /tmp.


_____________________________________________________________________________

11.2.4 Import trusted certificates from target server to source server

C:\>ovcert -importtrusted -file \temp\su1.cert -ovrg serverINFO: Import operation was successful.

C:\>ovcert –list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 2c1cf5f2-98f0-752b-0c72-ee508271ab3b (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_2c1cf5f2-98f0-752b-0c72-ee508271ab3b |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 2c1cf5f2-98f0-752b-0c72-ee508271ab3b (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_2c1cf5f2-98f0-752b-0c72-ee508271ab3b (*) || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

C:\>

11.2.5 Update trusted certificates in source server

C:\>ovcert -updatetrustedINFO: Trusted certificate update was successful.

C:\>ovcert –list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 2c1cf5f2-98f0-752b-0c72-ee508271ab3b (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_2c1cf5f2-98f0-752b-0c72-ee508271ab3b || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 2c1cf5f2-98f0-752b-0c72-ee508271ab3b (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_2c1cf5f2-98f0-752b-0c72-ee508271ab3b (*) || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

C:\>


_____________________________________________________________________________

11.2.6 Import trusted certificates from source server to target server

su1 # ovcert -importtrusted -file /tmp/sw80.cert -ovrg serverINFO: Import operation was successful.su1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_2c1cf5f2-98f0-752b-0c72-ee508271ab3b || CA_7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+

su1 #

11.2.7 Update trusted certificates in target server

mephisto # ovcert -updatetrustedINFO: Trusted certificate update was successful.mephisto # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_2c1cf5f2-98f0-752b-0c72-ee508271ab3b || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_2c1cf5f2-98f0-752b-0c72-ee508271ab3b || CA_7681325c-c1a9-7508-0441-a54412c264de (*) |+---------------------------------------------------------+

mephisto #


_____________________________________________________________________________


11.3.1 Add target server to node bank of source server

Use the node editor to add the target server. Be sure to set these properties:- In the ‘General’ tab, Advanced Configuration, enable the ‘Modify Agent ID/Core-ID’

and enter the ovcoreid of the target server.- In the ‘Network’ tab, disable the ‘Enable Auto Deployment’.- In the ‘System’ tab, disable the ‘Automatically grant certificate’.

After adding the node, enter the node editor again, edit the properties of the newly added node:In the ‘General’ tab, Advanced Configuration, enable the ‘Modify Certificate State’ and set the state to ‘Granted’.

11.3.2 Add source server to node bank of target server

su1 # /opt/OV/bin/OpC/utils/opcnode -add_node node_name=sw80 \> node_label=sw80 layout_group="" \> net_type=NETWORK_IP mach_type=MACH_BBC_WINNT_X86 group_name=windowsOperation successfully completed.su1 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=sw80 \> id='2c1cf5f2-98f0-752b-0c72-ee508271ab3b'Operation successfully completed.su1 # /opt/OV/bin/OpC/utils/opcnode -list_id node_list='sw80'List of IDs for node(s):Name = sw80 ID = 2c1cf5f2-98f0-752b-0c72-ee508271ab3bOperation successfully completed.su1 #

The empty string “” for the layout_group adds the new node to the top level of the node bank.

Be sure to use the HTTPS version of mach_type (MACH_BBC_*) for the corresponding OSplatform. See also the OMW 8 Online Help: Administering your environment, Scalable Architecture for Multiple Management Servers, Server-based flexible management, Configure communication protocols for server-based flexible management:

“By default, with HPOM for Windows 8.00 and above, the management server uses the HTTPS protocol to communicate with other management servers securely.”


_____________________________________________________________________________

11.4 Add managed nodes of source server to nodebank of target server

For each managed node whose messages shall be forwarded, add the node to the node bank of the target server. Be sure to use the correct mach_type and also set the correct ovcoreid, as outlined above. As example, the HP-UX PA-RISC cluster nodes c4n1 and c4n2 are used here. Both are HTTPS nodes managed by OMW 8.

To find out the ovcoreid of a OMW 8 managed node, either one of these methods may be used:• Login to the managed node itself, and use the ovcoreid command:

[root@c4n1]# ovcoreidd23fef0a-9d42-752c-16f4-ee46eacb1fad[root@c4n1]#[root@c4n2]# ovcoreid3fe5ab1a-a344-752c-133c-cf9e2e6a8df4[root@c4n2]#

• From the OMW 8 management console, use the node editor to view the properties of the managed node, ‘General’ tab, Advanced Configuration, enable the ‘Modify Agent ID/Core-ID’ and copy the core id to e.g. notepad. Be sure to exit the Advanced Configuration window with ‘Cancel’ to avoid any accidental change of the core id.

• From the command prompt of the OMW 8 management server, use this command to ping the managed node. Upon successful execution, it returns the ovcoreid of the managed node:

C:\>bbcutil -ping c4n1

false: status=eServiceOK coreID=d23fef0a-9d42-752c-16f4-ee46eacb1fadbbcV=06.10.050 appN=ovbbccb appV=06.10.050 conn=2 time=765 ms

C:\>bbcutil -ping c4n2

true: status=eServiceOK coreID=3fe5ab1a-a344-752c-133c-cf9e2e6a8df4bbcV=06.10.050 appN=ovbbccb appV=06.10.050 conn=2 time=437 ms

C:\>

On the target server, add the nodes and put them into the existing node layout group SAP_EP7, and set the correct core id:

su1 # /opt/OV/bin/OpC/utils/opcnode -add_node node_name=c4n1 \> node_label=c4n1 layout_group=SAP_EP7 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.su1# /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=c4n1 \> id='d23fef0a-9d42-752c-16f4-ee46eacb1fad'Operation successfully completed.su1# /opt/OV/bin/OpC/utils/opcnode -add_node node_name=c4n2 \> node_label=true layout_group=SAP_EP7 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.su1# /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=c4n2 \> id='3fe5ab1a-a344-752c-133c-cf9e2e6a8df4'Operation successfully completed.su1#


_____________________________________________________________________________

As in this case the managed nodes belong to a cluster, add the virtual node c4v as well:

su1 # /opt/OV/bin/OpC/utils/opcnode -add_node node_name=v4v \> node_label=c4v layout_group=SAP_EP7 \> net_type=NETWORK_IP mach_type=MACH_BBC_HPUX_PA_RISC group_name=hp_uxOperation successfully completed.su1 # /opt/OV/bin/OpC/utils/opcnode -set_virtual node_name=c4v \> cluster_package=SAPEP7 node_list='c4n1 c4n2'Operation successfully completed.su1 # /opt/OV/bin/OpC/utils/opcnode -list_virtual node_name=c4vAttributes of virtual node 'c4v'==========cluster_package=SAPEP7node_list="c4n2 c4n1"

Operation successfully completed.su1 #


_____________________________________________________________________________

11.5 Setup the message forwarding template on the source server

See also the OMW 8 Online Help: Administering your environment, Scalable Architecture for Multiple Management Servers, Server-based flexible management, Create a server-based flexible management policy.

In the policy editor, the following example is used:

# Example competence center forward configuration for HPOM for Windows Server-based flexible management# Forward SAP related messages to mephisto#TIMETEMPLATES# none

RESPMGRCONFIGSRESPMGRCONFIG DESCRIPTION "responsible mgrs for messages and agents"SECONDARYMANAGERS

SECONDARYMANAGERNODE IP 0.0.0.0 "mephisto.deu.hp.com"DESCRIPTION "HP OpenView Operations for Unix Management Server"

SECONDARYMANAGERNODE IP 0.0.0.0 "crcdiga84.deu.hp.com"DESCRIPTION "HP OpenView Operations for Windows Management Server"

ACTIONALLOWMANAGERSACTIONALLOWMANAGERNODE IP 0.0.0.0 "mephisto.deu.hp.com"DESCRIPTION "HP OpenView Operations for Unix Management Server"

ACTIONALLOWMANAGERNODE IP 0.0.0.0 "crcdiga84.deu.hp.com"DESCRIPTION "HP OpenView Operations for Windows Management Server"

MSGTARGETRULESMSGTARGETRULE DESCRIPTION "SAP responsibility for matched SAP SPI messages"MSGTARGETRULECONDSMSGTARGETRULECOND DESCRIPTION "matched SAP messages"

APPLICATION "SAP"MSGTARGETMANAGERSMSGTARGETMANAGER TIMETEMPLATE "$OPC_ALWAYS"

OPCMGR IP 0.0.0.0 "mephisto.deu.hp.com"MSGTARGETRULE DESCRIPTION "SAP responsibility for unmatched SAP SPI messages"MSGTARGETRULECONDSMSGTARGETRULECOND DESCRIPTION "unmatched SAP messages"

APPLICATION "OPC/SAP"MSGTARGETMANAGERSMSGTARGETMANAGER TIMETEMPLATE "$OPC_ALWAYS"

OPCMGR IP 0.0.0.0 "mephisto.deu.hp.com"

Check the syntax with the button provided in the policy editor, then save the policy (give it a suitable name and description), and deploy it to the OMW 8 management server.


_____________________________________________________________________________

11.6 Verify correct forwarding of messages

If the messages from OMU 8 are not shown in the message browser of the target manager, then check:

Are the managed nodes on the target server in a suitable node group, and are the OMU user responsibilities setup such that the node group / message group combination should be displayed?

Are the messages received in the target server at all? For that it is best to trace on the target server in the [opc] section:

mephisto # ovconfget -ovrg server opc…OPC_TRACE=TRUEOPC_TRACE_AREA=MSGOPC_TRACE_TRUNC=FALSEOPC_TRC_PROCS=opcmsgm,opcdispm

Then obtain the message id from the OMW 8 console, message properties, ‘General’ tab, in the example it is ba25a9ba-dfc1-71dc-1902-103943aa0000.

The OMU 8 trace should then contain for a new message arriving from OMW 8:Not all trace lines for this message are shown here, the trace lines are truncated (…), and it is assumed that the user opc_adm is logged in to the Motif GUI:

02/20 15:44:19.130 opcmsgm(4786:001)[msg]: Message received from message receiver: ba25a9ba-dfc1-71dc-1902-103943aa0000 …02/20 15:44:19.130 opcmsgm(4786:001)[msg]: message accepted: ba25a9ba-dfc1-71dc-1902-103943aa0000 …02/20 15:44:19.138 opcmsgm(4786:001)[msg]: csm_db_msg_add called with msg ba25a9ba-dfc1-71dc-1902-103943aa0000.02/20 15:44:19.176 opcmsgm(4786:001)[msg]: csm_db_msg_add finished for msg ba25a9ba-dfc1-71dc-1902-103943aa0000. Last err: 0-002/20 15:44:19.176 opcmsgm(4786:001)[msg]: Message forwarded to DM: ba25a9ba-dfc1-71dc-1902-103943aa0000 …02/20 15:44:19.202 opcdispm(4789:008)[msg]: Message received: ba25a9ba-dfc1-71dc-1902-103943aa0000 …02/20 15:44:19.202 opcdispm(4789:008)[msg]: Forwarding message ba25a9ba-dfc1-71dc-1902-103943aa0000 to user opc_adm


_____________________________________________________________________________

11.7 Configure the managed nodes to accept action requests from target server

If the target server shall be able to execute actions on the node managed by the source server, then some more configuration steps have to be done first.

11.7.1 Update trusted certificates

[root@c4n1]# ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || d23fef0a-9d42-752c-16f4-ee46eacb1fad (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_2c1cf5f2-98f0-752b-0c72-ee508271ab3b |+---------------------------------------------------------+

[root@c4n1]# ovcert -updatetrustedINFO: Trusted certificate update was successful.[root@c4n1]# ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || d23fef0a-9d42-752c-16f4-ee46eacb1fad (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_2c1cf5f2-98f0-752b-0c72-ee508271ab3b || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

[root@c4n1]#

The same has to be done on the other managed nodes.

11.7.2 Setup and deploy a mgrconf file

See also the OMW 8 Online Help: Administering your environment, Scalable Architecture for Multiple Management Servers, Agent-based flexible management, Create an agent-based flexible management policy, and also Configure action-allowed and secondary managers.

The policy created in this example is:

RESPMGRCONFIGSRESPMGRCONFIG DESCRIPTION "responsible mgrs for messages and agents"SECONDARYMANAGERS

SECONDARYMANAGERNODE IP 0.0.0.0 "su1"DESCRIPTION "HP OpenView Operations for Unix Management Server"




_____________________________________________________________________________



MSGTARGETRULESMSGTARGETRULE DESCRIPTION "always send all messages to current primary

manager"MSGTARGETRULECONDSMSGTARGETMANAGERSMSGTARGETMANAGER

TIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "$OPC_PRIMARY_MGR"

Then deploy the policy to the managed node(s).

Verify the correct deployment on the managed node:

[root@c4n1]# pwd/var/opt/OV/datafiles/policies/mgrconf[root@c4n1]# lltotal 32-r--r----- 1 root sys 1202 Feb 20 17:30 44307656-E32C-4445-9585-C9D9B9B1064A_data-r--r----- 1 root sys 2728 Feb 20 17:30 44307656-e32c-4445-9585-c9d9b9b1064a_header.xml[root@c4n1]#

The *_data file will have the same content as the policy written on the OMW 8 management server.


_____________________________________________________________________________

12 Message forwarding from OMU 8 to OMW 8

Please also see the previous chapter “Message forwarding from OMW 8 to OMU 8”. It is assumed that the first 3 subchapters have already been done, i.e.:

• Verify certificates on both servers• Setup certificate trust between the two servers• Add servers to each other’s node bank with correct ovcoreid

The source server now is su1, the target server is sw80.

12.1 Add managed nodes of source server to nodebank of target server

As example the node hn5 is used, which is managed by su1. Its properties can be obtained with:

su1 # opcnode -list_nodes node_list='hn5'List of all Nodes in the OVO database:====================================================================Name = hn5Label = hn5IP-Address = 16.58.24.213Network Type = NETWORK_IPMachine Type = MACH_BBC_HPUX_PA_RISCComm Type = COMM_BBCDHCP enabled = no (0x22)====================================================================Operation successfully completed.su1 # opcnode -list_id node_list='hn5'List of IDs for node(s):Name = hn5 ID = 7eeeaa28-4729-751c-1ef9-91417d3caff8Operation successfully completed.su1 #

Use the node editor to add the node hn5. Be sure to set these properties:- In the ‘General’ tab, Advanced Configuration, enable the ‘Modify Agent ID/Core-ID’

and enter the ovcoreid of hn5.- In the ‘Network’ tab, disable the ‘Enable Auto Deployment’.- In the ‘System’ tab, disable the ‘Automatically grant certificate’.

After adding the node, enter the node editor again, edit the properties of the newly added node:In the ‘General’ tab, Advanced Configuration, enable the ‘Modify Certificate State’ and set the state to ‘Granted’.

12.2 Setup the message forwarding template on the source server

Edit the file /etc/opt/OV/share/conf/OpC/mgmt_sv/respmgrs/msgforw on the OMU8 server, and add a suitable MSGTARGETRULE for forwarding the desired messages. The example used here is:


_____________________________________________________________________________

TIMETEMPLATES# noneRESPMGRCONFIGSRESPMGRCONFIG DESCRIPTION "msg-forwarding target specification"

MSGTARGETRULESMSGTARGETRULE DESCRIPTION "to sw80 (OMMW 8.0)"

MSGTARGETRULECONDSMSGTARGETRULECOND DESCRIPTION "OMW80"OBJECT "OMW80"

MSGTARGETMANAGERSMSGTARGETMANAGERTIMETEMPLATE "$OPC_ALWAYS"OPCMGR IP 0.0.0.0 "sw80"MSGCONTROLLINGMGR


Check the syntax with opcmomchk(1m), then restart the server processes (ovstop opc; ovstart).

Test the successful forwarding with a suitable message from the managed node(s).


_____________________________________________________________________________

13 Troubleshooting

13.1 Trying to distribute a new mgrconf file from OMU 8

su1 # opcragt -distrib -templates hn1Node hn1:Create distribution data and inform agent...Distribution of configuration failed (templates, actions, monitors, commands) (OpC40-482)No according data found in the database. (OpC50-3)Can't get info for node su2 in the database.Can't convert / distribute responsible-manager file needed on HTTPS nodestherefore. (OpC20-3181)No according data found in the database. (OpC50-3)Can't get info for node su2 in the database.Can't convert / distribute responsible-manager file needed on HTTPS nodestherefore. (OpC20-3181)Failed.

su1 #

Reason:

The node su2 (mentioned in mgrconf as SECONDARYMANAGER) is not in the node bank.


_____________________________________________________________________________

13.2 Trying to distribute a new mgrconf file from OMU 8 (again)

su1 # opcragt -distrib -templates hn1Node hn1:Create distribution data and inform agent...Distribution of configuration failed (templates, actions, monitors, commands) (OpC40-482)Found no valid OVCoreID entry for node su2 in the database.Can't convert / distribute responsible-manager file needed on HTTPS nodestherefore. (OpC20-3180)Found no valid OVCoreID entry for node su2 in the database.Can't convert / distribute responsible-manager file needed on HTTPS nodestherefore. (OpC20-3180)Failed.

su1 #

Reason: The node su2 (mentioned in mgrconf as SECONDARYMANAGER) is now in the node bank, but its OVCoreID is not known, see in "modify node, communication options", and also with:

su1 # /opt/OV/bin/OpC/utils/opcnode -list_id node_list='su2'List of IDs for node(s):Name = su2 ID = NONEOperation successfully completed.su1 #

Fix: obtain OVCoreID from the other manager su2:

su2 # ovcoreid -ovrg server7958cdb8-5cad-7506-1d7e-dbea390a7cd8su2 #

and enter it to the node data for su2 in the mgmt sv su1:

su1 # /opt/OV/bin/OpC/utils/opcnode -chg_id node_name=su2id='7958cdb8-5cad-7506-1d7e-dbea390a7cd8'Operation successfully completed.su1 # /opt/OV/bin/OpC/utils/opcnode -list_id node_list='su2'List of IDs for node(s):Name = su2 ID = 7958cdb8-5cad-7506-1d7e-dbea390a7cd8Operation successfully completed.su1 #


_____________________________________________________________________________

13.3 Trying to distribute mgrconf file (mixture of OVOU 7 and OMU 8)

If you have OVOU7 and OMU8 management servers in your responsible manager file (allnodes), the distribution fails and you get an error like this:

0: ERR: Tue Jan 25 13:03:41 2005: opcuiadm (1962/1): [momconv.c:103]: Node su7 from responsible-manager file must have communication type HTTPS. Can't convert / distribute responsible-manager file needed on HTTPS nodes therefore. For details see instruction text. (OpC20-3182)

The solution:You need to have two files, allnodes contains all the managers while allnodes.bbc contains only OMU8 managers.

The server will now distribute the allnodes file to OVOU7 (DCE) managed nodes, while the allnodes.bbc file will be distributed to OMU8 managed nodes.


_____________________________________________________________________________

13.4 Trying to switch primary manager to backup server on OMU 8

su2 # opcragt -primmgr hn1Node hn1:Setting OpC primary manager...Cannot become primary manager. Please check mgrconf of the node. (OpC40-362)Network communication problems occurred. (OpC40-427)Failed.

su2 #

Troubleshooting:

on managed node:

hn1 # cat /var/opt/OV/datafiles/policies/mgrconf/*data## Responsible Manager Configurations for a backup server#RESPMGRCONFIGS RESPMGRCONFIG DESCRIPTION "responsible mgrs for agents in ..."SECONDARYMANAGERSSECONDARYMANAGERNODE IP 0.0.0.0 "su1" ID "7681325c-c1a9-7508-0441-a54412c264de"DESCRIPTION "Managment Server su1"

SECONDARYMANAGERNODE IP 0.0.0.0 "su2" ID "7958cdb8-5cad-7506-1d7e-dbea390a7cd8"DESCRIPTION "Backup Server for su1"

ACTIONALLOWMANAGERSACTIONALLOWMANAGERNODE IP 0.0.0.0 "su1" ID "7681325c-c1a9-7508-0441-a54412c264de"

DESCRIPTION "Managment Server su1"ACTIONALLOWMANAGERNODE IP 0.0.0.0 "su2" ID "7958cdb8-5cad-7506-1d7e-dbea390a7cd8"DESCRIPTION "Backup Server for su1"

hn1 #

So this includes the certificate IDs of both managers. Both are trusted on the node:

hn1 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || e342588e-f9f4-7508-1d48-aeedd15c855b (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 |+---------------------------------------------------------+

hn1 #

The backup mgmt sv su2 knows the node hn1 with its ID:

su2 # /opt/OV/bin/OpC/utils/opcnode -list_id node_list='hn1'List of IDs for node(s):Name = hn1 ID = e342588e-f9f4-7508-1d48-aeedd15c855bOperation successfully completed.su2 #


_____________________________________________________________________________

The trusted certificates on the backup mgmt sv su2 are:


+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: || 7958cdb8-5cad-7506-1d7e-dbea390a7cd8 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 (*) |+---------------------------------------------------------+

su2 #

So this means there is no trust for the primary mgmt sv su1 here.

Fix:

- export trusted certificate from su1:

su1 # ovcert -exprttrusted -file /tmp/`hostname`.cert -ovrg server

- transfer the file to su2 and import it there:

su2 # ovcert -importtrusted -file /tmp/su1.cert -ovrg serverINFO: Import operation was successful.su2 #

- Verify the import:




_____________________________________________________________________________

su2 #

- switch primary manager:

su2 # opcragt -primmgr hn1Node hn1:Setting OpC primary manager...Done.

su2 #


_____________________________________________________________________________

13.5 Trying bbcutil -ping to another management server on OMU 8

To verify the trust between 2 management servers it is useful to check with 'bbcutil -ping':

su2 # bbcutil -ping su1

su1: (bbc-289) status=eSSLError time=54 ms

su2 #

Troubleshooting:



su2 #

This means that there is a trust for server su1 in the server keystore but not in the agent keystore.

Fix:

As ovbbccb is running on the agent side, it needs the trusted certificate as well:

su2 # ovcert -updatetrustedINFO: Trusted certificate update was successful.su2 # ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 7958cdb8-5cad-7506-1d7e-dbea390a7cd8 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 |+---------------------------------------------------------+

+---------------------------------------------------------+| Keystore Content (OVRG: server) |+---------------------------------------------------------+| Certificates: |


_____________________________________________________________________________

| 7958cdb8-5cad-7506-1d7e-dbea390a7cd8 (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_7958cdb8-5cad-7506-1d7e-dbea390a7cd8 (*) |+---------------------------------------------------------+

su2 # bbcutil -ping su1

su1: status=eServiceOK coreID=7681325c-c1a9-7508-0441-a54412c264de bbcV=05.20.050 appN=ovbbccb appV=05.20.050

conn=20 time=155 ms

su2 #


_____________________________________________________________________________

13.6 Trying opcragt from a backup server on OMU 8

If a HTTPS node (here hn3) had originally been installed from server A (here su1) and message forwarding between servers A and B (here su3) has been setup, and the mgrconf authorization has been downloaded to hn3, allowing access from server B, the opcragt from server B may still fail, e.g.:

su3 # opcragt -status hn3Node hn3:

Cannot get status information from node hn3. (OpC40-428)Network communication problems occurred. (OpC40-427)

-----------------------------------------------------------------------------CTRL - CommunicationException:-----------------------------------------------------------------------------(ctrl-21) Communication error when executing 'Status' method.(sec.core-116) An SSL connection IO error has occurred. This may be due to a network problem or an SSL handshake error. Possible causes for SSL handshake errors are that no certificate is installed, an invalid certificate is installed, or the peer does not trust the initiator's certificate. (OpC40-2130)Probably the certificates don't fit together or a certificate ismissing. Check whether the OVO server certificate is trusted on the nodeand vice versa as follows. On the OVO server call:"/opt/OV/bin/ovcert -certinfo `/opt/OV/bin/ovcoreid -ovrg server`"Check whether the ID mentioned in the line beginning with "Issuer CN:"appears in the output of command "ovcert -list" called on the node in thetrusted certificates section.On the node call "ovcert -certinfo `ovcoreid`". Check whether the IDmentioned in the line beginning with "Issuer CN:" appears in the outputof command "ovcert -list" called on the OVO server. The ID must be listedin the trusted certificates for the "Keystore Content (OVRG: server)" section. (OpC40-2174)Failed.

su3 #

Troubleshooting:

The System.txt file on the node hn1 contains for that time:

0: WRN: Wed Nov 14 10:04:16 2007: ovbbccb (3832/2112): (bbc-90) The incoming HTTPS client connection from host 16.58.24.82 failed due to the SSL error:1: WRN: Wed Nov 14 10:04:16 2007: ovbbccb (3832/2112): (sec.core-113) SSL certificate verification error (The presented peer certificate is not trusted. The certificate verification chain could not be built.).

So both information (from the opcragt command as well as in the node’s System.txt) hint towards a problem with trusted certificates.

The server’s coreid is:

su3 # ovcoreidae33c7ea-94b0-7525-04ef-cbab70bb7252su3 #

The trusted certificates on the node hn3 are:


_____________________________________________________________________________

C:\Documents and Settings\Administrator>ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 3d22c0c2-f956-7508-1f64-8a689337afca (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de |+---------------------------------------------------------+

C:\Documents and Settings\Administrator>

So this means that the node hn3 doesn’t trust the certificate of su3.

Fix:

Update the trusted certificates on hn3, i.e. get them from the installation server su1:

C:\Documents and Settings\Administrator>ovcert -updatetrustedINFO: Trusted certificate update was successful.

C:\Documents and Settings\Administrator>ovcert -list+---------------------------------------------------------+| Keystore Content |+---------------------------------------------------------+| Certificates: || 3d22c0c2-f956-7508-1f64-8a689337afca (*) |+---------------------------------------------------------+| Trusted Certificates: || CA_7681325c-c1a9-7508-0441-a54412c264de || CA_ae33c7ea-94b0-7525-04ef-cbab70bb7252 |+---------------------------------------------------------+

C:\Documents and Settings\Administrator>

Now the opcragt from server B succeeds:

su3 # opcragt -status hn3Node hn3:OVO Managed Node status :-------------------------OV Control ovcd (3964) is runningOV Communication Broker ovbbccb (3832) is runningOV Config and Deploy ovconfd (3004) is running…Done.

su3 #


_____________________________________________________________________________

13.7 Trying opcragt from a OMU 8 server to a OMW 8 controlled node

If a HTTPS node (here c4n1) had originally been installed from server A (here OMW 8 ‘sw80’) and message forwarding between servers A and B (here OMU 8 ‘su1’) has been setup, and the trusted certificates have been updated on hn5 to include the certificate of su1, the opcragt from server B may still fail, e.g.:

su1 # opcragt -status c4n1Node false.deu.hp.com:

Cannot get status information from node c4n1. (OpC40-428)Network communication problems occurred. (OpC40-427)

-------------------------------------------------------------------------------CTRL - CommunicationException:-------------------------------------------------------------------------------(ctrl-21) Communication error when executing 'Status' method.SoapFaultException:faultcode: ctrlauthorizationfaultstring: Authorization failed (OpC40-2130)Probably the OVO server isn't authorized to access the node.Check whether the server's core ID (on server call "ovcoreid -ovrg server")is mentioned on the node in the MANAGER_ID setting or in the mgrconf policy(on the node call "ovconfget sec.core.auth MANAGER_ID" and lookinto the file $OvDataDir/datafiles/policies/mgrconf/*_data (if any)). (OpC40-2173)Failed.

su1 #

Troubleshooting:

As this node is not managed by su1, access has to be allowed via a mgrconf file on the node.

[root@c4n1]# ll /var/opt/OV/datafiles/policies/mgrconftotal 0[root@c4n1]#

So there doesn’t exist a mgrconf definition on this node.

Fix:

A suitable mgrconf policy has to be defined on the OMW 8 management server sw80.An example is shown in the chapter “11.7.2 Setup and deploy a mgrconf file”.After deploying the policy, the command from su1 succeeds:

su1 # opcragt -status c4n1Node c4n1:OVO Managed Node status :-------------------------OV Control ovcd (18917) is runningOV Communication Broker ovbbccb (18918) is runningOV Config and Deploy ovconfd (18919) is runningOV Performance Core coda (18952) is runningSubagent EA:Message Agent opcmsga (27734) is runningAction Agent opcacta (18971) is running


_____________________________________________________________________________

Message Interceptor opcmsgi (18972) is runningLogfile Encapsulator opcle (18979) is runningMonitor Agent opcmona (18980) is runningSubagent AgtRep:OV Discovery Agent agtrep (19000) is runningDone.

su1 #


_____________________________________________________________________________

13.8 Tring to forward a message from OVOU8 to OMW8

The node is hn5 (HTTPS agent) managed by su1 (OMU 8.2x), and the message shall be forwarded from su1 to sw80 (OMW 8.0).

The message shows up in the message browser of su1, but not in sw80.A suitable msgforw template has been setup and activated on su1.The node hn5 is in the node bank of sw80.

Troubleshooting:

Tracing the server su1 with these trace settings:

su1 # ovconfget -ovrg server opc…OPC_TRACE=TRUEOPC_TRACE_AREA=MSGOPC_TRACE_TRUNC=FALSEOPC_TRC_PROCS=opcmsgm,opcforwm

The trace then shows:

02/22 11:59:24.906 opcmsgm(11795:001)[msg]: Message received from message receiver: 31150abe-e136-71dc-10a5-103a18d50000 <empty> 'hallo' IP 16.58.24.213 hn502/22 11:59:24.906 opcmsgm(11795:001)[msg]: message accepted: 31150abe-e136-71dc-10a5-103a18d50000 <empty> 'hallo' IP 16.58.24.213 hn502/22 11:59:24.915 opcmsgm(11795:001)[msg]: csm_db_msg_add called with msg 31150abe-e136-71dc-10a5-103a18d50000.02/22 11:59:24.931 opcmsgm(11795:001)[msg]: csm_db_msg_add finished for msg 31150abe-e136-71dc-10a5-103a18d50000. Last err: 0-002/22 11:59:24.931 opcmsgm(11795:001)[msg]: sending message to forward-mgr: 31150abe-e136-71dc-10a5-103a18d50000 <empty> 'hallo' IP 16.58.24.213 hn502/22 11:59:24.931 opcmsgm(11795:001)[msg]: Message forwarded to DM: 31150abe-e136-71dc-10a5-103a18d50000 <empty> 'hallo' IP 16.58.24.213 hn502/22 11:59:24.932 opcforwm(11797:001)[msg]: Handling HTTPS message.02/22 11:59:24.932 opcforwm(11797:001)[msg]: Message received: 31150abe-e136-71dc-10a5-103a18d50000 <empty> 'hallo' IP 16.58.24.213 hn502/22 11:59:24.932 opcforwm(11797:001)[msg]: Forwarding msg using HTTPS to 2 servers02/22 11:59:24.940 opcforwm(11797:007)[msg]: Handling DCE message.02/22 11:59:27.039 opcforwm(11797:001)[msg]: SnF reports message delivered to target https://sw80:383/com.hp.ov.opc.msgr/rpc#NewMessage, ctrl = FALSE.02/22 11:59:27.040 opcforwm(11797:001)[msg]: sent message to mgr sw80 (id=31150abe-e136-71dc-10a5-103a18d50000)02/22 11:59:27.078 opcforwm(11797:007)[msg]: Message received: 31150abe-e136-71dc-10a5-103a18d50000 <empty> 'hallo' IP 16.58.24.213 hn502/22 11:59:27.078 opcforwm(11797:007)[msg]: Forwarding msg using DCE to 2 servers02/22 11:59:27.078 opcforwm(11797:007)[msg]: sent message to mgr sw80 (id=31150abe-e136-71dc-10a5-103a18d50000)02/22 11:59:27.079 opcforwm(11797:001)[msg]: Handling HTTPS message returns 0.02/22 11:59:27.098 opcforwm(11797:007)[msg]: Handling DCE message returns 0.

On the target server sw80 the process OvEpMsgActSrv is traced.After saving the trace to a file and converting it to text, this can be seen:

2008-02-22 12:52:47,906,OvEpMsgActSrv,OvEpMsgFilter,ActFlow,0(2924),3844,"Discarding message (found node is NOT using AgentId of message) (OvEpMessage: Id=\"31150abe-e136-71dc-10a5-103a18d50000\", Text=\"hallo ...\").",…


_____________________________________________________________________________

2008-02-22 12:52:47,921,OvEpMsgActSrv,CclMsgObj,Operation,0(2924),3844,"Format message 06328240 returns:\n(MS673) Discarding message (OvEpMessage: Id=\"31150abe-e136-71dc-10a5-103a18d50000\", Text=\"hallo ...\") because the found node (key: {16182158-FEF7-4308-B37D-0E26DBE9B548}; name: hn5) is not \nusing the AgentID '7eeeaa28-4729-751c-1ef9-91417d3caff8' contained in the message.",

So this means that on the target server sw80 the node hn5 exists, but its stored Core ID is not the one contained in the message. Therefore the message is discarded and not shown in browser.

Fix:

Verify the core id of the managed node:

grcdg515 # ovcoreid7eeeaa28-4729-751c-1ef9-91417d3caff8grcdg515 #

Set the core id on the target server sw80:Use the node editor to edit the node hn5. In the ‘General’ tab, Advanced Configuration, enable the ‘Modify Agent ID/Core-ID’ and enter the ovcoreid of hn5.

Then send a new such message, it should be displayed in the target server now.


_____________________________________________________________________________

14 Defects and Workarounds

This is a list of currently known problems that may occur in the area of MoM configuration. For a final fix see the QXCRnnn defect ID. When a patch is released, it lists the fixed defect IDs.

QXCR1000222525 ovcert -updatetrusted works only if ovcs is restarted

Problem:

'ovcert -updatetrusted' on the managed node can't update newly added trusted certificates on the management server until the Certificate Server (ovcs) is restarted.

Workaround:

After importing trusted certificates, re-start the Certificate server:

# ovc -stop ovcs# ovc -start ovcs

Now, update the certificates on the managed nodes using ovcert -updatetrusted.

Fix:

The fix is included in the A.08.14 Core Agent patches for each supported platform.

QXCR1000227787 ovcd is not notified of a change when mgrconf policy is deployed to the node

Problem:

Remote commands cannot be issued from non-primary managers in a MoM scenario even after installing a new mgrcongf policy on all nodes.

Workaround:

1. Create and deploy mgrconf policy to the node (it must show up in "/var/opt/OV/datafiles/policies/mgrconf/" directory)

2. Run "/opt/OV/lbin/xpl/config/update/ctrlconfupd"

QXCR1000388238 Secondary mgmt_sv cannot reinstall agent on node

Problem:

A Secondary OVO Management Server cannot install an Agenton its Windows / HP-UX Managed Nodesalthough the switch from Primary to Secondary Managementserver via opcragt -primmgr works withoutproblems (opcragt -status <node> works properly).

Fix:

The fix is included in the A.08.25 Server patch.

Om flex mgmt

Technology

Transcript of Om flex mgmt