Post on 21-Mar-2020
myAMC™ FA Suite
Version 9.0
myAMC.FA_LogAgent - Concept and Usage
Fujitsu Limited
© Copyright Fujitsu Technology Solutions 2011
FlexFrame® and PRIMERGY™ are trademarks or registered trademarks of Fujitsu Limited in
Japan and other countries.
SAP® and NetWeaver™ are trademarks or registered trademarks of SAP AG in Germany and
in several other countries
Linux® is a registered trademark of Linus Torvalds
SUSE® Linux is a registered trademark of Novell, Inc., in the United States and other coun-
tries
Oracle™ and Java™ are trademarks of ORACLE Corporation and/or its affiliates
Intel® and PXE
® are registered trademarks of Intel Corporation in the United States and other
countries
MaxDB® is a registered trademark of MySQL AB, Sweden
MySQL® is a registered trademark of MySQL AB, Sweden
NetApp® and the Network Appliance® logo are registered trademarks and Network Ap-
pliance™ and Data ONTAP™ are trademarks of NetApp, Inc. in the U.S. and other countries.
EMC®, CLARiiON
®, Symmetrix
® and Celerra™ are trademarks or registered trademarks of
EMC Corporation in the United States and other countries
VMware®, ESX
®, ESXi, VMware vCenter, VMware vSphere are registered trademarks or
trademarks of VMware, Inc. in the United States and/or other jurisdictions.
Ethernet® is a registered trademark of XEROX, Inc., Digital Equipment Corporation and Intel
Corporation
Windows® and Word
® are registered trademarks of Microsoft Corporation
All other hardware and software names used are trademarks of their respective companies.
All rights, including rights of translation, reproduction by printing, copying or similar methods,
in part or in whole, are reserved.
Offenders will be liable for damages.
All rights, including rights created by patent grant or registration of a utility model or design,
are reserved. Delivery subject to availability. Right of technical modification reserved.
FlexFrame™ Autonomy LogAgent Guide Page 3 von 59 Version 9.0 confidential Stand: 15.04.2013
Contents
1 Introduction ..................................................................................................... 5 1.1 Overview ........................................................................................................... 5 1.2 Summary of Contents ........................................................................................ 5 1.3 Requirements .................................................................................................... 5 1.4 Related Documentation ..................................................................................... 6 1.5 Features in this Version ..................................................................................... 6 1.6 Notational Conventions ..................................................................................... 6 1.7 Document History .............................................................................................. 7
2 Getting Started ................................................................................................ 9 2.1 Installation ......................................................................................................... 9 2.2 Configuration ..................................................................................................... 9 2.2.1 Base Configuration ............................................................................................ 9 2.2.2 Log Definitions ................................................................................................. 10 2.2.3 Log Filters........................................................................................................ 12 2.2.4 Notification Destination .................................................................................... 13 2.3 Running the LogAgent ..................................................................................... 13 2.4 Testing the Scenario ....................................................................................... 14
3 Functional Concepts ..................................................................................... 15 3.1 Object Model ................................................................................................... 15 3.1.1 Sources and Templates .................................................................................. 15 3.1.2 Log Records .................................................................................................... 16 3.1.3 Log Readers .................................................................................................... 16 3.1.4 Processing Engine .......................................................................................... 16 3.1.5 Log Processors ............................................................................................... 16 3.2 Generating Log Records ................................................................................. 16 3.3 Processing Log Records ................................................................................. 17 3.4 Generating Notifications .................................................................................. 17
4 FlexFrame Integration ................................................................................... 21 4.1 Basic Configuration ......................................................................................... 21 4.2 Pool-specific Configuration .............................................................................. 21 4.3 Monitoring CCMS Alerts .................................................................................. 21 4.3.1 Prerequisites ................................................................................................... 21 4.3.2 Configuration ................................................................................................... 22 4.3.3 Filtering Alerts ................................................................................................. 23
5 Reference ....................................................................................................... 25 5.1 Starting and Stopping the LogAgent ................................................................ 25
Contents
FlexFrame™ Autonomy LogAgent Guide Page 4 von 59 Version 9.0 confidential Stand: 15.04.2013
5.1.1 Windows .......................................................................................................... 25 5.1.2 Linux ................................................................................................................ 25 5.2 Logging ............................................................................................................ 26 5.3 Configuration ................................................................................................... 26 5.4 Log Definitions ................................................................................................. 27 5.4.1 Overview.......................................................................................................... 27 5.4.2 Log Templates ................................................................................................. 28 5.4.3 Log Sources .................................................................................................... 29 5.4.4 Log Variables and Log Arguments .................................................................. 29 5.4.5 Generic Log Processors .................................................................................. 32 5.5 Log Readers and Arguments ........................................................................... 36 5.5.1 Common Log Arguments ................................................................................. 36 5.5.2 File Log Reader ............................................................................................... 36 5.5.3 Windows EventLog Reader ............................................................................. 38 5.6 Structure of a Log Record................................................................................ 39 5.7 Log Filters ........................................................................................................ 40 5.7.1 Filter configuration ........................................................................................... 40 5.7.2 Filter syntax ..................................................................................................... 40 5.7.3 Available Syntaxes .......................................................................................... 43 5.7.4 Filter actions .................................................................................................... 45 5.7.5 Complete Filter Example ................................................................................. 45 5.8 Log Processors ................................................................................................ 46 5.8.1 Stop Log Processor ......................................................................................... 46 5.8.2 Sink Log Processor ......................................................................................... 46 5.8.3 Reject Log Processor ...................................................................................... 47 5.8.4 Generic Log Processor .................................................................................... 47 5.9 Some Notes on XML Syntax ............................................................................ 50 5.9.1 Structure of a XML document .......................................................................... 50 5.10 Datatypes and Encodings ................................................................................ 53 5.10.1 Datatypes ........................................................................................................ 53 5.10.2 Encodings ........................................................................................................ 53 5.10.3 Data types, encodings and possible combinations .......................................... 54
6 Troubleshooting ............................................................................................ 57
7 Index ............................................................................................................... 58
FlexFrame™ Autonomy LogAgent Guide Page 5 von 59 Version 9.0 confidential Stand: 15.04.2013
1 Introduction
1.1 Overview
The myAMC.FA_LogAgent belongs to the Siemens myAMC Enterprise IT Management
suite. It enables an IT manager to monitor arbitrary applications without special instru-
mentation based on log messages written from the application to a number of different
output channels. Whenever an important event occurs a notification message is generat-
ed and sent to an application management console, e.g. myAMC.Messenger. Important
events of monitored applications can be sent to a central management console, which
saves the need for manual monitoring of applications on many different servers or
workstations.
The myAMC.FA_LogAgent is a universal tool for monitoring all kinds of log, trace and
system messages, such as application log files, Windows EventLog entries, or syslog
messages. Most server applications write log files with important messages, which have
to be monitored for errors and warnings. The myAMC.FA_LogAgent scans these log files
and processes all entries to find important messages to be forwarded to an event con-
sole. To avoid information overload a filter mechanism determines whether a log entry
should generate a notification.
The myAMC.FA_LogAgent is integrated into the myAMC Enterprise IT Management suite
for managing systems and applications. The integration of the myAMC.FA_LogAgent with
vendor-specific network, application, and system management products results in a com-
prehensive set of management functionality.
1.2 Summary of Contents
This manual describes components, functional concepts, and usage scenarios of the
myAMC.FA_LogAgent.
Chapter 2 helps the user to get started quickly, while chapter 3 gives an introduction to
the functional concepts of the LogAgent. Chapter 4 describes the integration into the
Fujitsu FlexFrame environment. Details on configuration and using the product are pro-
vided in chapter 5. Chapter 6 gives hints in case of unexpected results.
1.3 Requirements
This document is intended for users of myAMC as well as for administrators who wish to
integrate components of the Siemens Application Management Center into an Enterprise
IT Management System.
Introduction Related Documentation
FlexFrame™ Autonomy LogAgent Guide Page 6 von 59 Version 9.0 confidential Stand: 15.04.2013
1.4 Related Documentation
FlexFrame® for SAP
® – Administration and Operation
FlexFrame® for SAP
® – HW Characteristics Quickguides
FlexFrame® for SAP
® – Installation and Configuration LVM 1.0
FlexFrame® for SAP
® – Installation Guide for SAP Solutions
FlexFrame® for SAP
® – Installation of a FlexFrame Environment
FlexFrame® for SAP
® – Management Tool
FlexFrame®
for SAP®
– myAMC.FA_Agents Installation and Administration
FlexFrame®
for SAP®
– myAMC.FA_Messenger Installation and Administration
FlexFrame®
for SAP®
– myAMC.FA_LogAgent Installation and Administration
FlexFrame® for SAP
® – Network Design and Configuration Guide
FlexFrame® for SAP
® – Security Guide
FlexFrame® for SAP
® – Technical White Paper
FlexFrame® for SAP
® – Upgrading FlexFrame 5.0A or 5.1A to 5.2A
ServerView Documentation
SUSE Linux Enterprise Server Documentation
1.5 Features in this Version
Benefits of myAMC.FA_LogAgent
● Integration in the FlexFrame landscape with improved installation tools
● myAMC.FA_LogAgent helps
● to get control of a distributed network, system, and applications environments
● to reduce network administrations and downtime costs
● to detect application faults
● to reduce network traffic because of autonomous operation
1.6 Notational Conventions
The following conventions are used in this manual:
Additional information that should be observed.
Warning that must be observed.
fixed font Names of paths, files, commands, and system output.
<fixed font> Names of variables.
Document History Introduction
FlexFrame™ Autonomy LogAgent Guide Page 7 von 59 Version 9.0 confidential Stand: 15.04.2013
1.7 Document History
Document Version Changes Date
1.0 First Edition 2007-03-01
1.1 Update 2008-11-26
1.2 Update 2010-08-02
1.3 Update 2011-11-04
1.4 Update 2012-04-05
1.5 Update for 5.2A 2012-10-01
FlexFrame™ Autonomy LogAgent Guide Page 9 von 59 Version 9.0 confidential Stand: 15.04.2013
2 Getting Started
This chapter describes the steps needed to start the LogAgent and monitor a simple log
file. The examples cover the Windows version of the LogAgent, but the concepts and
configuration settings are the same for all supported platforms.
The first step is to install the LogAgent.
We will create the configuration needed to monitor a simple log file then.
Running the LogAgent or applying configuration changes is described in section 2.3,
while the last section in this chapter gives hints in case of anything is not working as
expected.
2.1 Installation
On Windows, the LogAgent is installed using the Setup program myAMC.
Agent.version.exe. Simply run the setup by double clicking the program and follow
the instructions.
On Linux systems, the LogAgent is installed using rpm with the package myAMC.FA_LogAgent-version.i386.rpm. The command to install the agent is as
follows:
root@linux> rpm –i /path/to/myAMC.FA_LogAgent-version.i386.rpm
On both Windows and Linux machines, the LogAgent is started automatically on system
start-up.
2.2 Configuration
The configuration is split up into three different kinds of configuration data:
● Base configuration: specifies log definition files, log filter definition files and other
information, e.g. SNMP trap targets and the cycle time of the processing engine.
● Log definitions: specify which data sources to monitor
● Log filters: specify what to do with log messages found by the Log Agent
2.2.1 Base Configuration
The base configuration has reasonable default values and is therefore left unchanged for
the purpose of this quick start manual. For details on all configuration parameters see
section 5.1.
Getting Started Configuration
FlexFrame™ Autonomy LogAgent Guide Page 10 von 59 Version 9.0 confidential Stand: 15.04.2013
2.2.2 Log Definitions
The log definition files specify which data sources the Log Agent should monitor. Per
default log definitions can be found in the config/logdefinitions.xml file.
Log definitions are structured into log templates and log sources. Log templates act as
containers for common attributes for several log sources, whereas a log source specifies
a single data source to be monitored. A log definitions file can contain any number of log
templates and sources. Log sources of unknown type are ignored.
The supplied file already contains several log definitions. Templates both for simple log
files and for Windows EventLog entries are ready to be used without further changes.
To monitor a sample log file for certain entries we add a log template and a log source definition to the file config/LogAgent/logdefinitions.xml, so that it looks like the
following example:
<logmonitor>
<logtemplate name="GenericLogFile" type="file">
<description>Template for generic log files</description>
<!-- check file every 30 seconds -->
<logarg name="CycleTime" type="UnsignedInteger" encoding="standard">
30
</logarg>
<!-- start from top of log file -->
<logarg name="StartFromTOF" type="Boolean" encoding="standard">
false
</logarg>
</logtemplate>
<logsource name="myLogFile" template="GenericLogFile">
<description>Inherit all defaults from template except
location</description>
<!-- path to log file -->
<logarg name="Location" type="String" encoding="standard">
c:\temp\mylogfile.txt
</logarg>
</logsource>
<logmonitor>
Configuration Getting Started
FlexFrame™ Autonomy LogAgent Guide Page 11 von 59 Version 9.0 confidential Stand: 15.04.2013
Make sure that the name of the log template (GenericLogFile) is the same as
specified in the template attribute of the log source element.
For details on all possible parameters see section 5.4.
Getting Started Configuration
FlexFrame™ Autonomy LogAgent Guide Page 12 von 59 Version 9.0 confidential Stand: 15.04.2013
2.2.3 Log Filters
From all configured log sources entries will be read and forwarded to an internal queue.
The entries in this queue are processed at periodic intervals (see section 5.1 for details).
A filter mechanism is used to determine what to do with the newly found log records. Per default filter definitions can be found in the file config/LogAgent/logfilters.xml.
The filter definitions are structured into filter domains, filter sets, filter entries, and filter
args. For details on filtering see sections 5.6 and 5.7.
For the purpose of this quick start manual we just want to forward all log file entries con-
taining the words WARNING, ERROR, and CRITICAL (case-insensitive) as notifications to
a SNMP trap listener such as myAMC.Messenger or an other enterprise management
suite.
The filter rules needed to perform this sample task are specified below:
<filter>
<filterdomain name="LogFilter" defaultset="default">
<filterset name="default" defaultaction="reject">
<filterentry name="WARNING" action="sink">
<filterarg name="rawmessage" syntax="containsi">
Warning
</filterarg>
</filterentry>
<filterentry name="CRITICAL" action="sink">
<filterarg name="rawmessage" syntax="containsi">
Critical
</filterarg>
</filterentry>
<filterentry name="ERROR" action="sink">
<filterarg name="rawmessage" syntax="containsi">
Error
</filterarg>
</filterentry>
</filterset>
</filterdomain>
</filter>
The filter syntax containsi used in this example looks for every occurrence of the
specified expression of the filter argument. The i indicates case-insensitive search. Each
new log entry will now be filtered using these rules. The action of the first rule matching
our log entry specifies what to do with it. If there are no matching filter entries, the default
action specifies what to do. We want important lines to be forwarded to our management station so we use action sink. Lines not containing any of the words WARNING,
CRITICAL, or ERROR are to be ignored, so the default action is reject.
Running the LogAgent Getting Started
FlexFrame™ Autonomy LogAgent Guide Page 13 von 59 Version 9.0 confidential Stand: 15.04.2013
2.2.4 Notification Destination
For the purpose of this quick start manual we want to forward the filtered log entries as
SNMP traps to our management station.
In order to achieve this, we specify the network address of our management station to tell
the LogAgent where to send our traps to.
The file config/TrapTargets.xml contains a config section TrapTargets, which
holds configuration data for trap destinations and communities. We add a new config
section for our management station, which looks like below:
<!-- ... rest of file -->
<configsection name="TrapTargets">
<configsection name="myManagementStation">
<configentry name="Host">
<value type="String" encoding="standard">
THE NAME OR IP ADDRESS OF YOUR MANAGEMENT STATION GOES HERE
</value>
</configentry>
<configentry name="Community">
<value type="String" encoding="standard">
public
</value>
</configentry>
</configsection>
<!-- ... next trap target goes here -->
</configsection>
<!-- ... rest of file -->
Your config file probably contains default values and descriptions as well, but they are left
out for the sake of brevity. The value of the config entry has to be changed to reflect the
host name or ip address of your management station.
2.3 Running the LogAgent
Now that the LogAgent is configured we start it. On Windows the LogAgent is imple-
mented as Windows Service and can be started using the Service Control Manager,
which can be found in the Control Panel:
Start Settings Control Panel Services
Getting Started Testing the Scenario
FlexFrame™ Autonomy LogAgent Guide Page 14 von 59 Version 9.0 confidential Stand: 15.04.2013
The LogAgent is registered with the Service Control Manager under the name myAMC.FA_LogAgent. To start the LogAgent select the corresponding entry in the
service list and click on the start button.
An alternative way to start the LogAgent is to open a command shell and enter the follow-
ing command:
Windows:
prompt> net start myAMC.FA_LogAgent
Linux:
prompt> /etc/init.d/myAMC.FA_LogAgent start
The LogAgent now periodically checks all data sources specified in the configuration. It
now monitores the specified file and sends a message whenever a log record passed the
filter. If the file does not exist yet, the LogAgent will ignore it and try again during the next
cycle.
The steps required to start and stop the LogAgent is described in section 5.1.
2.4 Testing the Scenario
To test our scenario we can now write arbitrary lines of text to the specified log file.
Whenever one of the words WARNING, CRITICAL, or ERROR appears in the file, the cor-
responding line is sent as SNMP trap to the management station.
In order to quickly check whether traps arrive at a destination, the Windows version of the LogAgent provides a tool called TrapListener, which can be used to view incoming
SNMP traps.
This tool is unsupported and provided for your convenience only. Use it at your
own risk!
According to the configuration created in this chapter the log source is checked
every 30 seconds. The log records in the input queue are per default processed
periodically at an interval of 30 seconds as well so it may require up to 60
seconds for a trap to arrive at the management station.
In case of unexpected results see chapter 6 for possibilities to diagnose errors.
FlexFrame™ Autonomy LogAgent Guide Page 15 von 59 Version 9.0 confidential Stand: 15.04.2013
3 Functional Concepts
This chapter describes the basic concepts of the LogAgent and gives a detailed overview
of the functionality.
General overview of generating and processing log records
3.1 Object Model
3.1.1 Sources and Templates
The basic components in the LogAgent‟s object model are log sources and templates.
A log source corresponds to a data source creating message entries of arbitrary type,
e.g. application log files, Windows EventLog, UNIX Syslog, etc. In order to read the mes-
sages from the data source, a log source contains attributes called log arguments, which
specify parameters needed to access the information and controlling the data retrieval.
Examples for log arguments are the location of a log file and the time interval between
two read cycles.
A log template acts as a container of log arguments and can be used to group log
sources. It also specifies the type of a log source, e.g. a text-based log file or the Win-
dows EventLog. Each log source references a log template from which it inherits the type
and all attributes.
Functional Concepts Generating Log Records
FlexFrame™ Autonomy LogAgent Guide Page 16 von 59 Version 9.0 confidential Stand: 15.04.2013
See section 5.5 for details on types and log readers.
3.1.2 Log Records
A log record is a data object containing a single message entry from a log source. A pre-
defined set of attributes holds all information gathered by the log reader (see below). It
contains both message data and information about the origin of the message entry, i.e.
the corresponding log source and template. The log record‟s attributes can be used for
filtering and further processing via hard-coded and custom log processors.
For a list of all attributes see section 5.6.
3.1.3 Log Readers
A log reader is a component to read log records from a specific source. The log reader to be used for a log source is specified with the type attribute of the corresponding log
template. A log reader gets all information needed to access the log messages from the
attributes specified by both log source and log template.
See section 5.5 for details on log readers.
3.1.4 Processing Engine
The processing engine is responsible for filtering and processing all log records in the
input queue. At periodic time intervals all records from the input queue are filtered and
processed according to the filter results. The filter action is interpreted as the name of a
log processor, which is then called to process the current log record. A log record may be
processed by multiple log processors. This feature is especially useful if a log entry is to
be split up into application-specific fields before sending it as notification.
See section 5.7 for details on filtering and section 5.8 for details on log processors.
3.1.5 Log Processors
A log processor is able to perform a specific operation on a log record. The log proces-
sors are called via log filters. The action attribute of the filter entry reflects the name of
the log processor to call. The LogAgent provides several hard-coded log processors as
well as the ability to use soft-coded or custom log processors. An example for a hard-coded log processor is the sink log processor, which can be used to forward a log
record to a managemant station as a notification.
3.2 Generating Log Records
Log records can be read from any number of log sources of different types, including
plain text files and the Windows EventLog (see section 5.5 for a list of supported log
readers). A log source has to be configured in the log definition file (see section 5.4 for
Processing Log Records Functional Concepts
FlexFrame™ Autonomy LogAgent Guide Page 17 von 59 Version 9.0 confidential Stand: 15.04.2013
details on log definitions). Each log source is periodically checked for new entries. If a
new entry is found, a log record with a standard set of attributes, which depends on the
log reader, is created and appended to the input queue.
3.3 Processing Log Records
The processing engine periodically processes all log records in the input queue. They are
filtered according to the merged rules defined in the log filter definition files and all other
filter definitions files loaded by the agent framework (see section 5.7). The log records are
removed from the queue and the resulting actions are translated into calls to log proces-
sors, which perform some processor-specific operation on the log records. For a list of
supported log processors see section 5.8. The action specified in the filter definition cor-
responds to the name of the log processor to be called.
A (generic) log processor can alter the attributes of a log record but this will not
affect the filtering process because the filtering process is completed before any
further processing takes place. In order to alter the record before filtering is per-
formed, a preprocessor can be used. See section 5.5.1 for details.
3.4 Generating Notifications
The usual reaction on an important event is to send a notification to an automated man-
agement system or an administrator. The LogAgent provides a universal notification me-
chanism, which allows notifying a management station or application in case of an unex-
pected or important event. The notification mechanism is protocol independent.
A notification request is send to all installed notification service providers, which map the
request to their specific notification protocols.
An example of a notification protocol is the Simple Network Management Protocol
(SNMP), where notifications are mapped to SNMP Traps.
The notification generator is implemented as a log processor with the name “sink” (see
also section 5.8).
All valid attributes of a log record are part of the notification message (for a list of all
attributes see section 5.6).
The mapping of a log record to a SNMP trap is shown in the following table.
Enterprise OID: 1.3.6.1.4.1.231.694.<ApplicationId> (see the following table)
Major trap number: 6 (enterprise-specific)
Minor trap number: 4 (fixed)
Functional Concepts Generating Notifications
FlexFrame™ Autonomy LogAgent Guide Page 18 von 59 Version 9.0 confidential Stand: 15.04.2013
Some fields can be specified both as text and as numeric id (Type, Severity, AppID).
When myAMC.Messenger receives such a trap, it automatically sets the numeric id ac-
cording to the text and vice versa if one of the two fields is missing.
Generating Notifications Functional Concepts
FlexFrame™ Autonomy LogAgent Guide Page 19 von 59 Version 9.0 confidential Stand: 15.04.2013
All variable bindings (VB) are prefixed with 1.3.6.1.4.1.231.694.
VB Type Description Comment
10 Long Version number currently 1
20 Long Application Id always 20
21 String Application as text LogAgent
30 Long Timestamp (UNIX Time) Seconds since Jan 1st, 1970, 0:00:00h
31 String Date as text YYYY-MM-DD
32 String Timestamp as text HH:MM:SS
111 String group name
121 String System name
131 String Device name (Host)
132 IPAddr Device IP address
140 Long Type 0=Command,1=Alarm, 2=Event, 3=Log,...
141 String Type as string “Command”, ”Alarm”, “Event”, ”Log”
152 String Category Type of log source: “file” or “nteventlog”
160 String Symbolic instance name
161 Long Instance number
162 String Instance name
170 Long Severity -1, 50, 150, 250
171 String Severity as string “Unknown”, “Normal”, “Warning”, “Critical”
190 String Pool name
191 String Service class
192 Long Priority
200 String InfoStrg00 LogAgent; location of log file
201 String InfoStrg01 LogAgent: name of log source
202 String InfoStrg02 LogAgent: name of log template
203 String InfoStrg03
204 String InfoStrg04
Functional Concepts Generating Notifications
FlexFrame™ Autonomy LogAgent Guide Page 20 von 59 Version 9.0 confidential Stand: 15.04.2013
VB Type Description Comment
205 String InfoStrg05
206 String InfoStrg06
207 String InfoStrg07
208 String InfoStrg08
209 String InfoStrg09
210 Long InfoLong00
211 Long InfoLong01
212 Long InfoLong02
213 Long InfoLong03
214 Long InfoLong04
215 Long InfoLong05
216 Long InfoLong06
217 Long InfoLong07
218 Long InfoLong08
219 Long InfoLong09
500 String Short Message short message
501 String Long Message long message
All fields which are not preset by LogAgent can be set specific for a log source using a
log processor.
FlexFrame™ Autonomy LogAgent Guide Page 21 von 59 Version 9.0 confidential Stand: 15.04.2013
4 FlexFrame Integration
For better integration into the Fujitsu FlexFrame environment, there is a specific package
myAMC.FA_LogAgent, which is installed at /opt/myAMC/FA_LogAgent.
4.1 Basic Configuration
The default configuration contains basic monitoring out of the box:
● All traps a forwarded to the local Control Node.
● The default configuration contains log definitions for the RMS switch log.
4.2 Pool-specific Configuration
In order to add pool-specific configuration, the LogAgent checks all pool directories for
additional configuration and log definition files:
/opt/myAMC/vFF/vFF_<poolname>/config/logfiles/config-<name>.xml
Additional config file to include
/opt/myAMC/vFF/vFF_<poolname>/config/logfiles/logdefinitions-<name>.xml
Additional log definitions to include
/opt/myAMC/vFF/vFF_<poolname>/config/logfiles/logfilters-<name>.xml
Additional log filters to include
All configuration files, log definitions and log filters are merged at startup of the
LogAgent. The order of the files within a pool-specific directory and the order of
the pools specify which settings may possibly overwrite other settings.
4.3 Monitoring CCMS Alerts
The LogAgent comes preconfigured for monitoring SAP CCMS alerts.
4.3.1 Prerequisites
As a prerequisite, the agent SAPCCMSR provided by SAP must be installed and confi-
gured to write an alert log file. The only option required is AlertLog, which tells the
agent to create a logfile for all CCMS alerts. A suitable configuration file sapccmsr.ini
could look like this:
AlertLog /usr/sap/ABC/DVEBMGS10/log/sapccm4x/alerts.txt
Note that ABC specifies the system id (SID) of the SAP system and DVEBMGS10 is the
name of the SAP start profile.
FlexFrame Integration Monitoring CCMS Alerts
FlexFrame™ Autonomy LogAgent Guide Page 22 von 59 Version 9.0 confidential Stand: 15.04.2013
The agent SAPCCMSR has to be configured for each SAP instance to be monitored.
4.3.2 Configuration
In order to monitor the CCMS alerts for a specific SAP instance, the LogAgent needs to
know about these instances. They are specified in a pool-specific configuration file. A
template can be found in
/opt/myAMC/FA_LogAgent/config/logdefinitions-example-sapccms.xml
It has to be copied to the pool‟s config directory as
/opt/myAMC/vFF/vFF_<poolname>/config/logfiles/logdefinitions-sapccms.xml
From there it will be automatically included (see section 4.2 for details).
For each SAP instance to be monitored, a <logsource> section has to be created by
copying the sample section in the configuration file and adjusting the values. An example
can be seen below:
<logsource name="app05wst" template="SAPCCMSlog">
<logvar name="profile">
DVEBMGS05
</logvar>
<logvar name="poolname">
myPool
</logvar>
<logvar name="groupname">
myGroup
</logvar>
<logvar name="systemname">
WST
</logvar>
<logvar name="instancename">
app05wst
</logvar>
<logvar name="instancenumber">
05
</logvar>
<logvar name="hostname">
app05wst
</logvar>
</logsource>
Please note, that the field hostname contains the virtual hostname of the SAP
instance.
Monitoring CCMS Alerts FlexFrame Integration
FlexFrame™ Autonomy LogAgent Guide Page 23 von 59 Version 9.0 confidential Stand: 15.04.2013
4.3.3 Filtering Alerts
In order to reduce the amount of alerts forwarded to a trap destination, the LogAgent is
able to filter alerts to exclude unimportant or unwanted notifications. See section 5.7 for
details on filters.
An example filter file is provided in
/opt/myAMC/FA_LogAgent/config/logfilters-example-sapccms.xml
It can be copied to
/opt/myAMC/vFF/vFF_<poolname>/config/logfiles/logfilters-sapccms.xml
From there it will be automatically included (see section 4.2 for details).
Please note, that all filters are merged into one big filter set. So it‟s not currently possible
to have different filters for different pools except by specifying the pool name as filter
argument like this:
<filterentry name="alertVanished" action="reject">
<filterarg name="poolname" syntax="containsi" syntaxargs="">
mypool
</filterarg>
<filterarg name="template" syntax="strcmp" syntaxargs="">
SAPCCMSlog
</filterarg>
<filterarg name="rawmessage" syntax="containsi" syntaxargs="">
vanished
</filterarg>
</filterentry>
FlexFrame™ Autonomy LogAgent Guide Page 25 von 59 Version 9.0 confidential Stand: 15.04.2013
5 Reference
5.1 Starting and Stopping the LogAgent
5.1.1 Windows
On Windows the LogAgent is implemented as Windows Service and can be controlled
using the Service Control Manager, which can be found in the Control Panel:
Start Settings Control Panel Services
The LogAgent is registered with the Service Control Manager under the name myAMC.FA_LogAgent. To start or stop the LogAgent select the corresponding entry in
the service list and click on the start button or stop button correspondingly.
An alternative way to start the LogAgent is to open a command shell and enter the follow-
ing command:
prompt> net start myAMC.FA_LogAgent
The LogAgent can be stopped with the following command:
prompt> net stop myAMC.FA_LogAgent
5.1.2 Linux
On Linux machines, the LogAgent is integrated in the start/stop system managed by
init using the script /etc/init.d/myAMC.FA_LogAgent.
To start the LogAgent, simply run the following command:
prompt> /etc/init.d/myAMC.FA_LogAgent start
The LogAgent can be stopped with the following command:
prompt> /etc/init.d/myAMC.FA_LogAgent start
Reference Logging
FlexFrame™ Autonomy LogAgent Guide Page 26 von 59 Version 9.0 confidential Stand: 15.04.2013
5.2 Logging
The LogAgent writes log information containing errors and hints. The log information is
written to the file <installdir>/log/myAMC.FA_LogAgent_<hostname>.log. If
the file gets too big, it is automatically renamed and a new file is created. Several genera-
tions of log files are kept, before the oldest one is overwritten.
5.3 Configuration
The LogAgent needs some basic configuration parameters, which are stored in the config
sub-directory of your myAMC.LogAgent distribution. The configuration is split up into
separate files to allow modularization. For details on configuration see chapter “Configu-
ration” in manual ” General section: myAMC Agents, Collectors, Managers”.
The LogAgent specific configuration files are contained in the folder <install-
dir>/config:
LogMonitor.xml
This file contains basic configuration settings (see below).
logdefinitions.xml
This file contains log definitions (see section 5.4).
logfilters.xml
This file contains filter rules for log alerts (see section 5.7).
TrapTargets.xml
This file contains destinations for SNMP traps (see section 2.2.4)
The other files serve as examples.
The config section LogMonitor in the main configuration file contains all relevant config
settings, which are described in the following paragraphs (for details on configuration see
chapter 5.1).
CycleTimeProcessingEngine (type: UnsignedInteger, default: 30)
This config entry specifies how often (in seconds) the log processing engine
processes log records.
IncludeUnknownFilterDomains (type: Boolean, default: false)
This config entry indicates whether filter domains other than the LogMonitor ones are
to be included.
Log Definitions Reference
FlexFrame™ Autonomy LogAgent Guide Page 27 von 59 Version 9.0 confidential Stand: 15.04.2013
LogFilters
This config section contains config entries of type string each of which specify the
URI of a filter definition file to load (for details on filtering see sections 5.6 and 5.7).
All referenced filter definitions are loaded and merged.
If the switch IncludeUnknownFilterDomains is set to true, non-LogMonitor filter
definitions contained in these files are loaded as well.
The config section contains per default a config entry with the name
UriLogFilters and the value ../config/LogMonitor/logfilters.xml.
The path to a filter definition file can either be a plain file name (path absolute
or relative to bin directory), or a valid file:// or http:// URL.
LogDefinitions
This config section contains config entries of type string each of which specify the
URI of a log definition file to load (for details on log definitions see section 5.4). All re-
ferenced filter definitions are loaded and merged.
The config section contains per default a config entry UriLogDefinitons and the
value file:///../config/LogMonitor/logdefinitions.xml.
The path to a filter definition file can either be a plain file name (path absolute
or relative to bin directory), or a valid file:// or http:// URL.
5.4 Log Definitions
5.4.1 Overview
LogMonitor reads information about all log sources to watch from one or more log defini-
tion files. See section 5.1 for details on how to specify the log definition files.
The path to a log definition file can either be a plain file name (path absolute or relative to
bin directory), or a valid file:// or http:// URL.
The log definition is based on an XML format. The LogAgent only accepts well formed XML files (see chapter 5.9 and http://www.w3.org/XML). The provided utility agentcfg
can be used to check an XML file for well-formedness. See section “Utilities” in the ma-
nual “General section: myAMC Agents, Collectors, Managers” for details on agentcfg.
The log definitions are composed of log templates and log sources. The names of all
elements are case sensitive and should not contain spaces or special characters.
Leading and trailing whitespaces (tabs, blanks, linebreaks) are ignored.
The definitions of log templates and log sources may be spread over several files.
Reference Log Definitions
FlexFrame™ Autonomy LogAgent Guide Page 28 von 59 Version 9.0 confidential Stand: 15.04.2013
Example:
<logmonitor>
<logtemplate name="GenericLogFile" type="file">
<description>Template for generic log files</description>
<!-- check file every 30 seconds -->
<logarg name="CycleTime" type="UnsignedInteger" encoding="standard">
30
</logarg>
<!-- start from top of log file -->
<logarg name="StartFromTOF" type="Boolean" encoding="standard">
true
</logarg>
</logtemplate>
<logsource name="myFile" template=" GenericLogFile ">
<!—- path to file -->
<logarg name="Location" type="String" encoding="standard">
Application
</logarg>
</logsource>
</logmonitor>
For some more examples see the provided log definition files in the directory config/LogMonitor of the agent distribution.
5.4.2 Log Templates
A log template contains common properties for all log sources derived from this template.
A log template may contain arguments (logarg) and variables (logvar), which are used
by the application. A description element may contain a short explanation of its meaning.
Parameters:
name
Unique name of the log template
type
Log reader type
See section 5.5 for available log readers.
Log Definitions Reference
FlexFrame™ Autonomy LogAgent Guide Page 29 von 59 Version 9.0 confidential Stand: 15.04.2013
Example:
<logtemplate name="GenericLogFile" type="file">
..<description>Some descriptive text</description>
</logtemplate>
5.4.3 Log Sources
A log source corresponds to a single source of logging information, e.g. a text file. It inhe-
rits information (arguments and variables) from a log template, but might overwrite this
information with specific values. A log source may contain arguments (logarg) and
variables (logvar), which are used by the application. A description element may con-
tain a short explanation of its meaning.
Parameters:
name
Unique name of the log source
template
Name of the corresponding template
Example:
<logsource name="myLogFile" template="GenericLogFile">
<description>Some descriptive text</description>
</logsource>
5.4.4 Log Variables and Log Arguments
Log variables and log arguments are elements used for both log sources and templates.
5.4.4.1 Log Variables
A log variable contains arbitrary information, usually of type string. The variable may be
referenced from other variables and arguments. Using this feature, a log template could
e.g. specify the important parts of a log definition, with the log source overwriting just the
variable parts like the log file location. Variables are referenced using the expression
${<varname>}, where <varname> is the name of the variable, which should be re-
placed by its content.
The names of variables are case sensitive and should not contain spaces or special cha-
racters. They may not be used recursively calling each other.
Reference Log Definitions
FlexFrame™ Autonomy LogAgent Guide Page 30 von 59 Version 9.0 confidential Stand: 15.04.2013
Parameters:
name
Name of the log variable
type (optional; default: String)
Data type of the value
encoding (optional; default: standard)
Encoding of the string representation of the value
See section 5.10 for details on data types and encodings.
Example:
<logvar name="var1">
server.log
</logvar>
<logvar name="var2">
d:\program files\myapp\${var1}
</logvar>
A special variable with the name date can be used to format a timestamp. The content of
this variable is used as format string for the timestamp and whenever the variable is
referenced it is re-evaluated to reflect the current time. This feature can be used to create
file names with date information.
Example:
<logvar name="date">
%Y%m%d
</logvar>
<logvar name="var2">
server-${date}.log
</logvar>
would evaluate to server-20080717.log.
Log Definitions Reference
FlexFrame™ Autonomy LogAgent Guide Page 31 von 59 Version 9.0 confidential Stand: 15.04.2013
The format string syntax is the same as the one used by the C function strftime():
%a Abbreviated weekday name
%A Full weekday name
%b Abbreviated month name
%B Full month name
%c Date and time representation appropriate for locale
%d Day of month as decimal number (01-31)
%H Hour in 24-hour format (00-23)
%I Hour in 12-hour format (01-12)
%j Day of year as decimal number (001-366)
%m Month as decimal number (01-12)
%M Minute as decimal number (00-59)
%p Current locale‟s A.M./P.M. indicator for 12-hour clock
%S Second as decimal number (00-59)
%U Week of year as decimal number, with Sunday as first day of week (00-53)
%w Weekday as decimal number (0 - 6; Sunday is 0)
%W Week of year as decimal number, with Monday as first day of week (00-53)
%x Date representation for current locale
%X Time representation for current locale
%y Year without century, as decimal number (00-99)
%Y Year with century, as decimal number
%z, %Z Time-zone name or abbreviation; no characters if time zone is unknown
%% Percent sign
Reference Log Definitions
FlexFrame™ Autonomy LogAgent Guide Page 32 von 59 Version 9.0 confidential Stand: 15.04.2013
5.4.4.2 Log Arguments
A log argument contains control information for the log reader specified by the enclosing
template or the template corresponding to the enclosing log source. The names of argu-
ments are case sensitive and log reader specific (see section 5.5 for a list of supported
arguments). They are explained in the section describing the specified log reader. A log
argument can contain references to log variables (see section 5.4.4.1 for details on log
variables).
Parameters:
name
Name of the log argument
type
Data type of the value
encoding (optional; default: standard)
Encoding of the string representation of the value
See section 5.10 for details on data types and encodings.
Example:
<logvar name="mypath">
d:\program files\myapp\logs
</logvar>
<logarg name="Location" type="String" encoding="standard">
${mypath}\server.log
</logarg>
5.4.5 Generic Log Processors
A generic log processor is a “soft” log processor as it is defined in a log definition file. It
can be used to manipulate attributes of log records with user defined settings, e.g. to split
the log entry of a log file into separate fields like timestamp, severity, and message.
Example:
<logprocessor name="ifproc">
<description>Split a log record into separate parts</description>
<logstatement syntax="regex" syntaxargs="">
<logexpression input="rawmessage">
<![CDATA[\[.*\][[:blank:]]+(.*)]]>
Log Definitions Reference
FlexFrame™ Autonomy LogAgent Guide Page 33 von 59 Version 9.0 confidential Stand: 15.04.2013
</logexpression>
<!-- message -->
<logformat output="formattedmessage" type="String"
encoding="standard">
($1)
</logformat>
</logstatement>
<logstatement syntax="regex" syntaxargs="">
<logexpression input="rawmessage">
<![CDATA[\[.*[[:blank:]]+([^[:blank:]]+)\].*]]>
</logexpression>
<!-- severity (text) -->
<logformat output="severitytext" type="String" encoding="standard">
($1)
</logformat>
</logstatement>
</logprocessor>
5.4.5.1 Log Processor
A log processor describes a set of processing rules. The element contains none, one, or
more log statements. A description element may contain a short explanation of its mean-
ing.
Parameter:
name
Unique name of the log processor; this name is used to call the log processor from
within a filter (as filter action)
Example:
<logprocessor name="myproc">
<description>Split an log record into separate parts</description>
</logprocessor>
Reference Log Definitions
FlexFrame™ Autonomy LogAgent Guide Page 34 von 59 Version 9.0 confidential Stand: 15.04.2013
5.4.5.2 Log Statement
A log statement contains a specific processing rule. The element contains a log expres-
sion and a log format element.
Parameters:
syntax
Syntax to use for expression and format (see section 5.4.5.5 for details)
syntaxargs
Arguments for the specified syntax (see section 5.4.5.5 for details)
Example:
<logstatement syntax="regex" syntaxargs="">
<logexpression input="rawmessage">
...
</logexpression>
<logformat output="formattedmessage"
type="String" encoding="standard">
...
</logformat>
</logstatement>
5.4.5.3 Log Expression
A log expression contains an expression of the syntax specified in the enclosing log
statement element. This expression is used with the current log record‟s attribute speci-fied as input field.
Parameter:
input
Name of the attribute of the current log record, whose value is used with the expres-
sion specified by this element
Example:
<logexpression input="rawmessage">
<![CDATA[\[.*\][[:blank:]]+(.*)]]>
</logexpression>
Log Definitions Reference
FlexFrame™ Autonomy LogAgent Guide Page 35 von 59 Version 9.0 confidential Stand: 15.04.2013
5.4.5.4 Log Format
A log processor describes a specific processing rule. The element contains none, one, or
more log statements. A description element may contain a short explanation of its mean-
ing.
Parameters:
output
Name of the attribute of the current log record, the value of which will become the re-
sult of the transformation
type
Data type to convert the resulting value to
encoding
Encoding of the resulting value
Example:
<logformat output="formattedmessage" type="String" encoding="standard">
($1)
</logformat>
5.4.5.5 Regular Expression Syntax for Log Processors
The regular expression syntax processes text and allows control characters to be used as
placeholders. Substrings can be extracted from any string and used to form new attri-
butes. An introduction to regular expression and the corresponding formatting rules can
be found in the files doc/LogMonitor/regex_syntax.htm and
regex_format_string.htm of your myAMC Agent distribution.
Currently, there are no syntax arguments defined.
Example:
<logstatement syntax="regex" syntaxargs="">
<logexpression input="rawmessage">
<![CDATA[\[(.*)\][[:blank:]]+(.*)]]>
</logexpression>
..<logformat output="formattedmessage"
type="String" encoding="standard">
($2)
</logformat>
</logstatement>
Reference Log Readers and Arguments
FlexFrame™ Autonomy LogAgent Guide Page 36 von 59 Version 9.0 confidential Stand: 15.04.2013
In this example the input string
[2001/07/17 16:49:00 WARNING] Can’t find file /tmp/abcdefg.txt
would evaluate to
Can’t find file /tmp/abcdefg.txt
A format string of ($1) would evaluate to
2001/07/17 16:49:00 WARNING
5.5 Log Readers and Arguments
A log reader is a component to read log records from a specific source. Log sources
include ordinary text files and the Windows EventLog.
5.5.1 Common Log Arguments
The log arguments described in this section are common to all log readers.
CycleTime (type: UnsignedInteger; default: -)
This argument specifies how often (in seconds) the corresponding log sources are
checked for new log records.
Application (type: String; default: unknown)
This argument specifies the application name used for log record creation. See sec-
tion 5.6 for details on log record attributes.
Class (type: String; default: unknown)
This argument specifies the class name used for log record creation. See section 5.6
for details on log record attributes.
PreProcessor (type: String; default: -)
This argument specifies the name of a generic log processor to be used for
processing a log record before filtering is performed. See section 5.8.4 for details on
generic log processors.
5.5.2 File Log Reader
The file log reader reads log records from plain text files. The file is read line by line, so a
log record must fit into a single line. The file log reader recognizes two special cases:
1. If the LogAgent is (re)started and the file to be monitored already contains some log
records, the LogAgent can start at the top of file or at the current end of file. This be-
havior is controlled by the argument StartFromTOF.
Log Readers and Arguments Reference
FlexFrame™ Autonomy LogAgent Guide Page 37 von 59 Version 9.0 confidential Stand: 15.04.2013
2. If the log file has been rewritten by the logging application since the last log file
check, the LogAgent can start at the top of recreated file or at the current end of file.
This behavior is controlled by the argument RestartPositionAtTOF.
Reference Log Readers and Arguments
FlexFrame™ Autonomy LogAgent Guide Page 38 von 59 Version 9.0 confidential Stand: 15.04.2013
Type Name: file
Log Arguments:
Location (type: String; default: -)
This argument specifies the path of the file to be monitored. The name of the log file
is case sensitive for UNIX platforms and case insensitive for Windows.
StartFromTOF (type: Boolean; default: false)
This argument specifies the behavior of the log monitor at startup (see case 1 above
for details). If set to true, the file is scanned from the top, otherwise the file position is
set to the end of the file and only new entries are read.
RestartPositionAtTOF (type: Boolean; default: true)
This argument specifies the behavior of the log monitor at startup (see case 2 above
for details). If set to true, the file is rescanned from the top, otherwise the file position
is set to the end of the file and only new entries are read.
5.5.3 Windows EventLog Reader
The EventLog reader reads log records from the Windows EventLog. It recognizes one
special case:
● If the LogAgent is (re)started and the event log to be monitored already contains
some log records, the LogAgent can start at the top of the event log or at the current
end of the log. This behavior is controlled by the argument StartFromTOF.
Type Name: nteventlog
Log Arguments:
UncHostName (type: String; default: local host name)
Name of the host to read the event log from. The event log must be accessible with-
out any user credentials. Access can be checked with the Windows Event Viewer
(„Select Computer…“ from the Log Menu).
Location (type: String; default: -)
Specifies the name of the event log to use. Usually Windows maintains three event logs: Application, System, and Security. The name of the log file is case sensi-
tive.
StartFromTOF
This argument specifies the behavior of the log monitor at startup (see the special
case above for details). If set to true, the event log is scanned from the top, otherwise
the read position is set to the end of the log and only new events are read.
Structure of a Log Record Reference
FlexFrame™ Autonomy LogAgent Guide Page 39 von 59 Version 9.0 confidential Stand: 15.04.2013
5.6 Structure of a Log Record
A log record consists of a set of attributes, some of which are mandatory and some of
which are optional. The attributes can be accessed from the filter definitions (via the
name filed of the filterarg element, and from the log definitions (via the input and
output fields of the logexpression and logformat elements; see sections 5.4.5.3 and
5.4.5.4).
This section gives an overview and a short explanation of all attributes.
Name Explanation Comment
rawmessage Raw message as read from log source always
available
formattedmessage Formatted (or splitted) message optional
hostname Name of host the log monitor runs on always
available
application Name of the application which created the
log record
optional
source Name of log source as specified in the log
definitions
always
available
template Name of log template as specified in the log
definitions
always
available
severity Severity code optional
type Log reader type as specified in the log defini-
tions
always
available
numprocessingloops Number of internal filter loops (this attri-bute
is used to protect the filter engine from infi-
nite loops)
always
available
timestamp Timestamp of log record creation always
available
infostrg00-infostrg10 Arbitrary text field, which may be polouated
by a generic log processor (10 fields)
optional
infolong00-infolong10 Arbitrary number field, which may be po-
louated by a generic log processor (10
fields)
optional
location Location of the log source as specified in the
log definitions
always
available
Reference Log Filters
FlexFrame™ Autonomy LogAgent Guide Page 40 von 59 Version 9.0 confidential Stand: 15.04.2013
Name Explanation Comment
<logarg> All logargs specified in the log definitions optional
5.7 Log Filters
The LogAgent filters all log records in the input queue to determine further processing.
5.7.1 Filter configuration
Filters definitions are read from one or more filter definition files. Filters can be spread
over multiple filter definition files; in this case all filter definitions are merged. See section
5.1 for details on how to specify the log filter files.
5.7.2 Filter syntax
5.7.2.1 Filter file format
The filter definition is based on a XML format. The agent only accepts well formed XML
files (see chapter 5.9 and http://www.w3.org/XML).
The filter rules are hierarchically organized in filter domains, sections, entries, and argu-
ments according to the following rules:
• each filter definitions file can contain none, one, or more filter domains, which specify
filter rules for different kind of application data
• each filter domain can contain none, one, or more filter sets, which define a rule set
for a specific type of application data
• each filter set can contain none, one, or more filter entries, which define a single rule
for a specific type of application data
• each filter entry can contain none, one, or more filter arguments, which defines a
single filter argument for one named value
The names of all elements are case sensitive and should not contain spaces or special
characters. The filter rules are applied in the order of appearance in the filter definition
file.
Example:
<filter>
<filterdomain name="LogFilter" defaultset="default">
<description>text</description>
<filterset name="default" defaultaction="reject">
<description>Default filter set</description>
<filterentry name="IfTrace" action="ifproc"
Log Filters Reference
FlexFrame™ Autonomy LogAgent Guide Page 41 von 59 Version 9.0 confidential Stand: 15.04.2013
ifmatch="continue">
<description>Text</description>
<filterarg name="source" syntax="strcmp">
IFTraceTest
</filterarg>
</filterentry>
</filterset>
</filterdomain>
</filter>
For each object being filtered the result is one or more actions. The filter actions are ap-
plication specific and described in the corresponding manual.
5.7.2.2 Filter domains
A filter domain contains filter rules for application objects of similar type. All objects being
filtered by a filter domain have comparable properties. Not all properties must be availa-
ble for all objects, though. A filter domain contains none, one, or more filter sets. A de-
scription element may contain a short explanation of its meaning.
Parameters:
name
unique name of the filter domain
defaultset
filter set, which is used if the application does not specify, which filter set to use
The name of the filter domain used by the LogAgent is LogFilter.
Example:
<filterdomain name="LogFilter" defaultset="default">
</filterdomain>
5.7.2.3 Filter sets
A filter set contains a named set of rules. Different rule sets may be applied for different
objects. This features allow different filter rules e.g. for production systems and test sys-
tems. A filter set contains none, one, or more filter entries. If the set has no filter entries, it
uses the default action. A description element may contain a short explanation of its
meaning.
Parameters:
Reference Log Filters
FlexFrame™ Autonomy LogAgent Guide Page 42 von 59 Version 9.0 confidential Stand: 15.04.2013
name
unique name of the filter set
defaultaction
default action, which is used if no filter entries match; see section 5.7.4 for poss-
ible values
Example:
<filterset name="default" defaultaction="reject">
</filterset>
5.7.2.4 Filter entries
A filter entry describes a specific filter rule. The filter entry matches if all contained filter
arguments match (see section 2.2.1.6.3.4). The resulting filter actions are application
specific and described in the corresponding manual. A filter entry contains none, one, or
more filter args. If the entry has no filter args, it always matches. A description element
may contain a short explanation of its meaning.
Parameters:
name
unique name of the filter entry
action
action to be applied by the application; see section 5.7.4 for possible values
ifmatch (optional, default: "stop")
specifies whether to stop filtering if the entry matches or to continue with the
next entry. Allowed values: "stop" or "continue"
Example:
<filterentry name="default" action="sink" ifmatch="stop">
</filterentry>
5.7.2.5 Filter arguments
A filter argument describes a specific filter rule for a single attribute. The attribute is
processed by the specified expression, which follows the specified syntax. The syntax is
defined by a plugable module. All available syntaxes are described below. The filter ar-
gument contains the filter expression according to the specified syntax.
Log Filters Reference
FlexFrame™ Autonomy LogAgent Guide Page 43 von 59 Version 9.0 confidential Stand: 15.04.2013
Parameters:
name
name of the attribute to filter
syntax
name of the syntax which is used to filter the value; see section 5.7.3 for possi-
ble values
syntaxargs (optional, default: "")
arguments for the syntax module; the arguments are syntax specific; see section
5.7.3 for possible values. Usually this attribute is not used and therefore empty
or missing.
The attribute names of the log records can be used as input values for the filtering
process (name field of the filterarg element). The names and meanings of all
attributes are listed in section 5.6.
Example:
<filterarg name="severity" syntax="stricmp" syntaxargs=””>
WARNING
</filterarg>
5.7.3 Available Syntaxes
The available syntaxes are described in the following paragraphs.
5.7.3.1 Syntaxes strcmp, stricmp, strncmp, and strnicmp
These syntaxes are direct calls to the corresponding C functions. They all compare the
expression with the textual representation of the specified attribute.
There are some small semantic differences between the syntaxes:
• strcmp and strncmp compare case sensitively
• stricmp and strnicmp compare case insensitively
• strcmp and stricmp compare the whole string so the whole attribute must match
• strncmp and strnicmp compare only the start of the attribute so only the first n
characters must match where n is the length of the expression
Example:
Reference Log Filters
FlexFrame™ Autonomy LogAgent Guide Page 44 von 59 Version 9.0 confidential Stand: 15.04.2013
<filterarg name="severity" syntax="stricmp" syntaxargs="">
WARNING
</filterarg>
5.7.3.2 Syntaxes contains and containsi
These syntaxes search for occurrences of the expression in the string representation1 of
the specified attribute.
There is one small difference between the two syntaxes:
• contains compares case sensitively
• containsi compares case insensitively
Example:
<filterarg name="severity" syntax="containsi">
WARNING
</filterarg >
5.7.3.3 Syntax wildcard
The wildcard syntax searches for occurrences of the expression in the string represen-
tation of the specified attribute and allows wildcards to be used like in path names. The
wildcards may appear as often as needed anywhere within the expression.
Allowed wildcards are:
• '*' for any sub string of any length
• '?' for any single character
• '\?' for a question mark
• '\*' for an asterisk
• '\\' for a backslash
Example:
<filterarg name="rawmessage" syntax="wildcard">
[*] ?arning*c:\\temp\\*.txt
</filterarg>
Log Filters Reference
FlexFrame™ Autonomy LogAgent Guide Page 45 von 59 Version 9.0 confidential Stand: 15.04.2013
5.7.3.4 Syntax regex
The regular expression syntax searches for occurrences of the expression in the textual
representation of the specified attribute and allows control characters to be used as
placeholder. An introduction to regular expression can be found in the doc directory of
your LogAgent distribution.
Example:
<filterarg name="rawmessage" syntax="regex" syntaxargs="">
\[([^[:blank:]]+)[[:blank:]]+([0-9A-Za-z]+).*\].*
</filterarg>
5.7.4 Filter actions
The resulting filter action for each log record is interpreted as name of a log processor.
The list of log processors include the provided log processors described in section 5.8 as
well as any generic log processors defined in all log definition files.
5.7.5 Complete Filter Example
<?xml version="1.0" encoding="ISO-8859-1"?>
<filter>
<filterdomain name="LogFilter" defaultset="default">
<description>text</description>
<filterset name="default" defaultaction="reject">
<description>Default filter set</description>
<filterentry name="IfTrace" action="ifproc" ifmatch="continue">
<description>Split IF records</description>
<filterarg name="source" syntax="strcmp" syntaxargs="">
IFTrace
</filterarg>
</filterentry>
<filterentry name="NTEventlogWarning" action="sink">
<description>Sink all events with severity Error</description>
Reference Log Processors
FlexFrame™ Autonomy LogAgent Guide Page 46 von 59 Version 9.0 confidential Stand: 15.04.2013
<filterarg name="sourcetemplate" syntax="strcmp" syntaxargs="">
NTEventLog
</filterarg>
<filterarg name="severitytext" syntax="containsi" syntaxargs="">
ERROR
</filterarg>
</filterentry>
</filterset>
</filterdomain>
</filter>
5.8 Log Processors
A log processor is able to perform a specific operation on a log record. This chapter gives
an overview on the provided standard processors. The log processors are called via log
filters. The action attribute of the filter entry reflects the name of the log processor to
call (see section 5.7.2.4 for details on filter actions).
5.8.1 Stop Log Processor
The stop log processor stops any further processing of the current log record.
Example:
<filterentry name="entry1" action="stop">
...
</filterentry>
5.8.2 Sink Log Processor
The sink log processor sends the current log record as notification (see section 3.4 for
details).
Example:
<filterentry name="entry1" action="sink">
...
</filterentry>
Log Processors Reference
FlexFrame™ Autonomy LogAgent Guide Page 47 von 59 Version 9.0 confidential Stand: 15.04.2013
Several log arguments are supported. They are optional, as the defaults are usually suffi-
cient:
NotificationOriginatorApp (type: String; default: LogMonitor)
Sets the name of the originating application
NotificationOriginatorOid
(type: ObjectIdentifier; default: .1.3.6.1.4.1.231.694.20)
Sets the originator oid of the notification
NotificationId (type: UnsignedInteger; default: 2)
Sets the notification id
NotificationDomain (type: String; default: Log)
Sets the notification domain
NotificationVarBindBaseOid
(type: ObjectIdentifier; default: .1.3.6.1.4.1.231.694.20)
Base object identifier for all notification objects. The last identifier can‟t be changed
but is fixed for each object (see also the table in section 3.4)
Example:
<logarg name=" NotificationOriginatorOid" type="ObjectIdentifier" encod-
ing="standard">
.1.3.6.1.4.1.1000.5
</logarg>
5.8.3 Reject Log Processor
The reject log processor acts as a no-operation or nop. No action is performed.
Example:
<filterentry name="entry1" action="reject">
...
</filterentry>
5.8.4 Generic Log Processor
A generic log processor is a “soft” log processor as it is defined in the log definition file. It
can be used to manipulate attributes of log records with user defined settings, e.g. to split
the log entry of a log file into separate fields like timestamp, severity, and message.
The syntax of a generic log processor is explained in section 5.4.5.
Example:
The following generic log processor is used to split a log line of the form
Reference Log Processors
FlexFrame™ Autonomy LogAgent Guide Page 48 von 59 Version 9.0 confidential Stand: 15.04.2013
[date time <channel> (pid/tid) severity] message
into the separate fields date (info1), time (info2), channel (class), pid (source1), tid
(source2), severity (severitytext), and message (formattedmessage):
<logprocessor name="myproc">
<description>Split an log record into separate parts</description>
<logstatement syntax="regex" syntaxargs="">
<logexpression input="rawmessage">
\[.*\][[:blank:]]+(.*)
</logexpression>
<!-- message -->
<logformat output="formattedmessage"
type="String" encoding="standard">
($1)
</logformat>
</logstatement>
<logstatement syntax="regex" syntaxargs="">
<logexpression input="rawmessage">
\[.*[[:blank:]]+([^[:blank:]]+)\].*
</logexpression>
<!-- severity (text) -->
<logformat output="severitytext" type="String" encoding="standard">
($1)
</logformat>
</logstatement>
<logstatement syntax="regex" syntaxargs="">
<logexpression input="rawmessage">
\[.*<(.*)>.*\].*
</logexpression>
<!-- category -->
<logformat output="class" type="String" encoding="standard">
($1)
</logformat>
</logstatement>
<logstatement syntax="regex" syntaxargs="">
<logexpression input="rawmessage">
Log Processors Reference
FlexFrame™ Autonomy LogAgent Guide Page 49 von 59 Version 9.0 confidential Stand: 15.04.2013
\[.*\((.*)/(.*)\).*\].*
</logexpression>
<!-- process id -->
<logformat output="source1" type="String" encoding="standard">
($1)
</logformat>
</logstatement>
<logstatement syntax="regex" syntaxargs="">
<logexpression input="rawmessage">
.*\((.*)/(.*)\).*\].*
</logexpression>
<!—- thread id -->
<logformat output="source2" type="String" encoding="standard">
($2)
</logformat>
</logstatement>
<logstatement syntax="regex" syntaxargs="">
<logexpression input="rawmessage">
\[([^[:blank:]]+)[[:blank:]]+([^[:blank:]]+).*\].*
</logexpression>
<!-- date -->
<logformat output="info1" type="String" encoding="standard">
($1)
</logformat>
</logstatement>
<logstatement syntax="regex" syntaxargs="">
<logexpression input="rawmessage">
\[([^[:blank:]]+)[[:blank:]]+([^[:blank:]]+).*\].*
</logexpression>
<!-- time -->
<logformat output="info2" type="String" encoding="standard">
($2)
</logformat>
</logstatement>
</logprocessor>
Reference Some Notes on XML Syntax
FlexFrame™ Autonomy LogAgent Guide Page 50 von 59 Version 9.0 confidential Stand: 15.04.2013
5.9 Some Notes on XML Syntax
The myAMC Agent configuration files are all based on XML syntax. The agent only ac-
cepts well formed XML files (for details on XML see http://www.w3.org/XML). On Unix
systems, the utility xmllint can be used to check a XML file for well-formedness (see
“man xmllint” for details), on Windows systems, the file can simply be opened in Inter-
net Explorer, which will report XML errors as well.
This chapter provides some basic notes on XML syntax.
5.9.1 Structure of a XML document
5.9.1.1 XML Header
A XML document always contains the XML header, which states the XML version used
for the current document. An optional attribute specifies the encoding.
Example:
<?xml version="1.0" encoding="ISO-8859-1" ?>
5.9.1.2 Root element
A XML document always contains one root element. All other elements must be children
of (i.e. must be enclosed by) this element.
Example:
<?xml version="1.0" encoding="ISO-8859-1" ?>
<rootelement>
</rootelement>
5.9.1.3 Contents of a XML document
Comments
A XML document may content any number of comments, which may stretch over multiple
lines of text. The comment starts with the sequence <!-- and ends with -->. Comments
must not be nested.
Some Notes on XML Syntax Reference
FlexFrame™ Autonomy LogAgent Guide Page 51 von 59 Version 9.0 confidential Stand: 15.04.2013
Example:
<!-- I am a XML comment -->
Structure of an element
An XML element (also called “tag”) consists of the start tag, which may contain arbitrary
attributes of the form name="value", and an end tag. Its body may contain other elements
or arbitrary text. The end tag may be omitted if the element has no content. In this case
the closing "/" has to appear at the end of the start tag. The names of tags and attributes
are case sensitive and application specific.
Example:
<myelement attr1=”value1” attr2=”value2”>
some arbitrary content.
<myelement2>
this is an inner element
</myelement2>
</myelement>
<myelement3 attr1="val1" />
Order and spacing of elements
A XML parser usually ignores all line breaks, tabs, spaces, etc. which often make up the
structured layout of a XML document. Indentation is just a means to get a good reading of
the document‟s content.
The following XML fragments are equivalent:
Variant 1:
<filter>
<filterdomain name="xyz" defaultset="abc">
<filterset name="abc" defaultaction="someaction">
...
</filterset>
</filterdomain>
</filter>
Variant 2:
Reference Some Notes on XML Syntax
FlexFrame™ Autonomy LogAgent Guide Page 52 von 59 Version 9.0 confidential Stand: 15.04.2013
<filter><filterdomain name="xyz" defaultset="abc"><filterset name="abc"
defaultaction="someaction">
...</filterset></filterdomain></filter>
The parser ignores all whitespaces (tabs, line breaks, blanks) before and after tag values,
but not within attribute values (e.g. name="", defaultset="", defaultaction="",...).
Special characters
XML uses some special characters for control purposes, e.g. the character „<‟ to start a
tag. These characters must not appear within tag or attribute names, or element text. If
they are contained in element text, they must be escaped using the &<name>; construct,
called entities.
Character Escaped form Meaning
& & Ampersand
" " Quotation mark
' ' Apostrophe
< < Less than
> > Greater than
As an alternative, string containing special characters can be embedded in a CDATA
section. This section starts with the sequence <![CDATA[ and ends with ]]>.
Example:
<![CDATA[ My text containing the special characters &"<>' ]]>
Whole XML- or HTML fragments can be embedded into XML documents within a CDATA
section.
Datatypes and Encodings Reference
FlexFrame™ Autonomy LogAgent Guide Page 53 von 59 Version 9.0 confidential Stand: 15.04.2013
5.10 Datatypes and Encodings
Configuration values, log definitions and filter definitions contain user-specified parame-
ters, which are represented in a specific data type and encoding. This chapter describes
used data types and encodings in detail.
5.10.1 Datatypes
Data type Description
Null Null value
Integer 32 bit Integer (signed or unsigned) in the range -231
to
(231
)-1
Integer64 64 bit Integer (signed or unsigned) in the range -263
to
(263
)-1
UnsignedInteger 32 bit Integer (unsigned) in the range 0 to (232
)-1
UnsignedInteger64 64 bit Integer (unsigned) in the range 0 to (264
)-1
Float 64 bit floating point value in the range 1.7E-308 to
1.7E+308 (11-bit excess-1023 exponent and a 52-bit
mantissa)
String Character string
Boolean true or false
Binary Any binary data, see possible encodings for details
DataType One of the datatypes described in this table (case-
sensitive!)
ObjectIdentifier ASN.1 object identifier in the form id1.id2.id3.id4. Each id
is in the range 0 to 65535. If the id contains a leading dot
(„.‟) it is ignored.
5.10.2 Encodings
Data type Description
standard Standard encoding for each data type. This is usually the
encoding which is used intuitively.
Reference Datatypes and Encodings
FlexFrame™ Autonomy LogAgent Guide Page 54 von 59 Version 9.0 confidential Stand: 15.04.2013
Data type Description
base64 Base64-Encoding for binary data (s. RFC2045, section
6.8)
hex Hex-Encoding: each byte is encoded using two charac-
ters, which represent the printable form of the hex value
of this byte
Example:
Binary value Encoded String:
255 FF
254 FE
... ..
128 80
127 7F
... ..
016 10
015 0F
... ..
001 01
000 00
5.10.3 Data types, encodings and possible combinations
Data type Description
Null Null value
Integer 32 bit Integer (signed or unsigned) in the range -231
to
(231
)-1
Integer64 64 bit Integer (signed or unsigned) in the range -263
to
(263
)-1
UnsignedInteger 32 bit Integer (unsigned) in the range 0 to (232
)-1
UnsignedInteger64 64 bit Integer (unsigned) in the range 0 to (264
)-1
Float 64 bit floating point value in the range 1.7E-308 to
1.7E+308 (11-bit excess-1023 exponent and a 52-bit
mantissa)
Datatypes and Encodings Reference
FlexFrame™ Autonomy LogAgent Guide Page 55 von 59 Version 9.0 confidential Stand: 15.04.2013
Data type Description
String Character string
Boolean true or false
Binary Any binary data, see possible encodings for details
DataType One of the datatypes described in this table (case-
sensitive!)
ObjectIdentifier ASN.1 object identifier in the form id1.id2.id3.id4. Each id
is in the range 0 to 65535. If the id contains a leading dot
(„.‟) it is ignored.
Data type Encoding Example
Null standard
Integer standard
hex
-1234567890,
-1234567890
Integer64 standard
hex
1234567890,
-12345678901234
UnsignedInteger standard
hex
1234567890
UnsignedInteger64 standard
hex
12345678901234567890
Float standard 1234.5678,
-1.23456E-7
String standard abcDEF 123 !"§$%&?{[]}\,;.:
Boolean standard true, false
Binary base64
hex
AKOSSQZDCC=,
AAFF0099
DataType standard Null, String
ObjectIdentifier standard .1.3.6.1.4.1.231.694.9, 0.0
FlexFrame™ Autonomy LogAgent Guide Page 57 von 59 Version 9.0 confidential Stand: 15.04.2013
6 Troubleshooting
If the log monitor shows unexpected behavior the following methods may help to find and
correct the problem.
● Open the log file
● Look for log entries starting with WARNING or ERROR
● For each log source a log reader job should run every <CycleTime> seconds
● The log reader prints a message with the number of new records found in a log
source
● If there are no log reader jobs being run the LogAgent probably failed to read the log
definition files. Restart the LogAgent and look for messages from the module Log-
Monitor and/or LogMonitorExtensionModule. They should give hints what is going
on.
FlexFrame™ Autonomy LogAgent Guide Page 58 von 59 Version 9.0 confidential Stand: 15.04.2013
7 Index
F
FlexFrame integration 19
basic configuration 19
monitoring CCMS alerts 19
configuration 20
filtering alerts 21
prerequisites 20
pool-specific configuration 19
L
LogAgent
configuration 24
base configuration 6
log definitions 6
log filters 8
notification destination 9
functional concepts 13
generating log records 15
generating notifications 16
object model 13
processing log records 15
getting started 5
configuration 5
running 10
testing the scenario 10
log definitions 25
generic log processors 31
log arguments 28
log expression 33
log format 34
log processor 32
log sources 28
log statement 33
log templates 27
log variables 28
regular expression syntax 34
log filters 39
log processors 45
generic log processor 47
reject log processor 46
sink log processor 46
stop log processor 45
log readers and arguments
common log arguments 35
file log reader 35
NT EventLog reader 37
log record structure 38
object model
log processors 15
log readers 14
log records 14
processing engine 14
sources 13
templates 13
T
troubleshoting 57
FlexFrame™ Autonomy LogAgent Guide Page 59 von 59 Version 9.0 confidential Stand: 15.04.2013