Post on 28-Oct-2019
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP ArcSight ESM 24/7 Aparna Varanasi, Sr. Software Engineer Bill Alexander, Sr. Software Engineer #HPProtect
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3
This is a rolling (up to three year) Roadmap and is subject to change without notice.
Forward-looking statements
This document contains forward looking statements regarding future operations, product development, product capabilities and availability dates. This information is subject to substantial uncertainties and is subject to change at any time without prior notification. Statements contained in this document concerning these matters only reflect Hewlett Packard's predictions and / or expectations as of the date of this document and actual results and future plans of Hewlett-Packard may differ significantly as a result of, among other things, changes in product strategy resulting from technological, internal corporate, market and other changes. This is not a commitment to deliver any material, code or functionality and should not be relied upon in making purchasing decisions.
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4
This is a rolling (up to three year) Roadmap and is subject to change without notice.
HP confidential information
This Roadmap contains HP Confidential Information. If you have a valid Confidential Disclosure Agreement with HP, disclosure of the Roadmap is subject to that CDA. If not, it is subject to the following terms: for a period of 3 years after the date of disclosure, you may use the Roadmap solely for the purpose of evaluating purchase decisions from HP and use a reasonable standard of care to prevent disclosures. You will not disclose the contents of the Roadmap to any third party unless it becomes publically known, rightfully received by you from a third party without duty of confidentiality, or disclosed with HP’s prior written approval.
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5
ESM 24/7
Next ESM release • What is ESM HA (High Availability)?
– HA features overview – HA architecture
• HA features – Installation – Monitoring
• Failover demo • Key takeaways
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6
What is ESM HA?
ESM HA (High Availability) is a two server installation of the ESM product for improved reliability and availability. ESM HA is an active/passive cluster. • Primary – the server running ESM. • Secondary – the other server (on hot standby).
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7
HA features overview
• Secondary backs up Primary disk • Automatic Failover
− System failures are automatically detected − Secondary switches to primary and runs ESM
• Monitoring − Audit events and notifications − Console Content − arcsight_cluster script – monitoring and
maintenance • Simplified installation
This is a rolling (up to 3 year) roadmap and is subject to change without notice
ESM HA
Connectors Clients
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HA architecture
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9
Server room layout
Interconnected pair of ESM servers • Primary and Secondary connected by one or more 1G or 10G
Ethernet cables. • Optional HP iPDU (Intelligent Power Distribution Unit)
– Only external device that ESM HA supports for forced reboot of servers. • To reboot a server, commands are sent to the iPDU to
turn power on and off. – Optionally redundant power supplies supported.
• Switch(es) provide network access. – Connectors – ESM Clients – iPDUs
HA architecture
iPDU
Primary
Secondary
Network Switch
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10
Software architecture
• Cluster Control (Pacemaker/Heartbeat) – Monitors software and communications – Determines which software runs on which server. – Restarts software when needed
• Disk Mirroring (DRBD) – Makes disk available on primary – Sends changes made on primary to secondary
• ESM, Service IP, and File System – only on primary – File System containing ESM installation only
mounted on primary. – Service IP – ESM IP Address – dynamically moves
between servers. – ESM runs where its files are mounted.
HA architecture Intranet
Primary Secondary
eth0 eth0
eth1 eth1 Disk Mirroring
Disk Mirroring
Cluster Control
ESM
File System
Service IP
Cluster Control
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11
HA architecture
STONITH (Shoot The Other Node In The Head)
Enabling technology for failover • Needed when primary is crippled and will not release resources
– Communication problems – primary cannot receive stop request – Software problems (e.g. out of memory or other resources)
• Ideally STONITH mechanism should be independent of primary hardware/software
– Power control like iPDU – In some clusters cutting the server off from the network (I/O fencing) is used.
• Default SSH based fallback reboot control far from ideal.
– Will only work if SSH to server, reboot is possible.
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Installation
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13
Installation overview
• Designed to be easy – Installation questions via Wizards – Run installation on primary • Secondary installation done automatically
• Major data inputs covered in the next slides
Installation
Hopefully this won’t be too bad …
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 14
Cluster setup parameters
• Shared disk – mount point for ESM installation (/opt or /opt/arcsight)
• Metadata volume – small partition with disk sync status
• Service hostname – ESM hostname or IP – moves between servers.
• Secondary hostname – hostname of secondary server.
• Primary cable IP – IP address of primary via interconnect cable.
• Secondary cable IP – IP address of secondary via interconnect cable.
Installation
This is a rolling (up to 3 year) roadmap and is subject to change without notice
All product views are illustrations and might not represent actual product screens
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15
Cluster configuration parameters
• Preferred primary – this one will be selected as primary.
• Connected hosts – hosts to ping to see if this machine is connected to the internet.
• Connectivity down timeout – how long the communication between the network and the primary should be down before failover.
• Time between failovers – if a failover has occurred recently, wait this long before failing over again.
• Ping timeout – how long to wait before considering a ping to have failed.
• Ping attempts – how many pings to try before concluding cannot reach this host.
Installation
This is a rolling (up to 3 year) roadmap and is subject to change without notice
All product views are illustrations and might not represent actual product screens
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 16
iPDU parameters Installation
This is a rolling (up to 3 year) roadmap and is subject to change without notice
All product views are illustrations and might not represent actual product screens
Only fill out this screen if you have iPDU • iPDU hostnames – hostname of iPDU(s) • Wait time for reboot – reboot is accomplished by
doing power off, followed by power on. The iPDU should wait this long to turn on power after turning it off.
• Primary iPDU outlets – the outlet(s) to which the primary is connected.
• Secondary iPDU outlets – the outlet(s) to which the secondary is connected.
• iPDU login – user to log into iPDU. • iPDU password – password to log into iPDU
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HA monitoring
“Trust, but verify” Ronald Reagan 40th President of the United Statesrerson’s name, title and)
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18
HA monitoring
• HA audit events and notifications • HA monitoring and maintenance script • HA content
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HA audit events and notifications
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20
HA audit events and notifications
Audit event and notification types • HA:100 – Primary Manager Started • HA:200 – HA Status Failed
– Secondary is offline – Disk is not syncing or possible disk failure on
secondary – Network communication on primary or secondary
is down • HA:300 – Sync in Progress • HA:400 – iPDU Status Failed • HA:500 – HA Status OK
Frequency • HA:100 – Created on change of HA state • HA:200, HA:300, HA:400 - Created on change of
HA state and/or about every 5 mins, if the same state exists. Notifications sent as well
• HA:500 – Created on change of HA state. Notification sent
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21
HA audit events and notifications This is a rolling (up to 3 year) roadmap and is subject to change without notice
All product views are illustrations and might not represent actual product screens
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 22
Configuration
HA audit events and notifications
HA properties in server.properties highavailability.monitor.on=true • Turn on/off the HA Notification feature. True is On and False is Off.
highavailability.notification.interval=300 • Set notification interval for failure conditions. It is configured in seconds and the default is 5 mins.
whine.check.interval.HASubsystemChecker=30 • Set the polling interval of the tracker/checker that checks arcsight_cluster status. It is configured in seconds
and the default is 30s.
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HA monitoring and maintenance script
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 24
arcsight_cluster script
HA monitoring and maintenance script
/usr/lib/arcsight/highavail/bin/arcsight_cluster Options • status • prefer • offline • online • diagnose • clusterParameters • increaseDisk • tuneDiskSync
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 25
arcsight_cluster status output This is a rolling (up to 3 year) roadmap and is subject to change without notice
All product views are illustrations and might not represent actual product screens
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HA content
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 27
HA Monitoring Use Case - All Use Cases/ArcSight Administration/ESM/HA Monitoring
HA content
• Active Channel – HA Monitoring
• Dashboard – ESM HA Status
• Query Viewers and Queries – System Status Changes – Last
24 hours – Current Primary Server – System Status Changes – Current Primary
• Report – ESM HA Status Updates –
last 7 days • Filter
– ESM HA Status
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 28
HA Monitoring Use Case - All Use Cases/ArcSight Administration/ESM/HA Monitoring
HA content
• Rules – ESM SystemStarted – Alert – HA Status Change • Notification sent
• Active List – Current Primary System
• Data Monitor – Last 10 HA Status Changes – ESM HA Status
• Field Set – HA Management
• Session List – Current Primary System Status
Change
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 29
HA content
All product views are illustrations and might not represent actual product screens
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Failover demo
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 31
How is a failover done?
1. Cluster determines failover is necessary. 2. Cluster brings down resources on old primary
a. Stops ESM b. Unconfigures Service IP c. Unmounts Disk d. Puts Disk Mirroring Software into Secondary Mode
3. If any of the steps in 2 fail, STONITH (reboot) the primary. 4. Cluster brings up resources on new primary
a. Puts Disk Mirroring Software in Primary Mode b. Mounts Disk c. Configures Service IP d. Starts ESM
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 32
Cover title over with Failover video This is a rolling (up to 3 year) roadmap and is subject to change without notice
All product views are illustrations and might not represent actual product screens
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Key takeaways
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 34
Key takeaways
• ESM HA can help by – Protecting ESM from hardware failures – Minimizing downtime for scheduled maintenance
• Steps to a successful implementation – System is only as reliable as weakest link – improve reliability everywhere • Power • Network
– Use application monitoring software to catch cases when both Primary and Secondary go down • HP Operations Manager, NAGIOS
– HA Best Practices • Use Logical Volume Manager (LVM) to simplify creating, resizing partitions. • Bonded Interfaces for speed, reliability in interconnect. • Use iPDU for cleaner failovers
This is a rolling (up to 3 year) roadmap and is subject to change without notice
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 35
For more information
Attend these sessions
• TT3058, Building a highly available HP ArcSight solution
Visit these demos
• HP ArcSight ESM
After the event
• Contact your sales rep
Your feedback is important to us. Please take a few minutes to complete the session survey.
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 36
Please fill out a survey. Hand it to the door monitor on your way out.
Thank you for providing your feedback, which helps us enhance content for future events.
Session TB3069 Speakers Aparna Varanasi and Bill Alexander
Please give me your feedback
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you