Post on 24-Apr-2015
description
APRIL 11, 2023 | SLIDE 1
APRIL 11, 2023 | SLIDE 2
www.realdolmen.com
APPLICATION LOGGING
APRIL 11, 2023 | SLIDE 3
ABOUT ME
Sander De Vos Software Engineer RealDolmen 2008 Past projects
MaTMa (M*) E-Metro (FOD-ECO) SP10B (FOD-FIN) MIK (FOD-MOB)
APRIL 11, 2023 | SLIDE 4
QUOTES FROM THE FIELD
“Who needs logs? I sure don’t!” “Debugger is better! Sysout works too!” “Isn’t that what infrastructure does?” “It slows down disk and memory!” “It isn’t in the spec!” “We don’t know what to log – we’ll just log something!” “We don’t know how – we’ll just do it somehow!”
APRIL 11, 2023 | SLIDE 5
AGENDA
What is logging Definition and Challenges
Why logging Who uses logs What logs are used for Functional or non-functional
What to log What will (not) help you Log message decomposed Guidelines
How to log in Java Logging in production Log processing and analysis
APRIL 11, 2023 | SLIDE 6
WHAT IS LOGGING
APRIL 11, 2023 | SLIDE 7
DEFINITION
“Logs may be referred to as log files, audit logs, or audit trails.
Logs also cover alerts, alarms and event records.“
“Logs are the recordings of one or more events occurring on information systems.”
“Logs can be organized based on the program, day, severity, host, or a number of other categories.“
APRIL 11, 2023 | SLIDE 8
LOG CHALLENGES
Decentralization Cloud SOA
Volatility Multiple tiers and layers Archival Accessability Absence of critical information Random log formats
APRIL 11, 2023 | SLIDE 9
WHY LOGGING
APRIL 11, 2023 | SLIDE 10
WHO USES LOGS
Managed Services Security Teams
Monitor Detect Investigate Track Analyze
Auditors Developers
Debugging
Customer
APRIL 11, 2023 | SLIDE 11
WHAT LOGS ARE USED FOR
Debugging and Forensics Fault monitoring Performance monitoring Troubleshooting Feature usage Security / Incident detection Regulatory and standards compliance
APRIL 11, 2023 | SLIDE 12
FUNCTIONAL – NON-FUNCTIONAL LOGGING
Functional Audit Governance Security
Non-functional Faults, Errors, Exceptions Execution context Performance Component usage
APRIL 11, 2023 | SLIDE 13
PUTTING IT ALL TOGETHER
Functional Logs Customer
Audit Always on Requirements
Scope known Years ‘Clutter, yuck, slow’
Non-Functional Logs System operator
Developer Sometimes on Errors, Exceptions,
Debug Scope not known Hours, Days ‘Might come in handy’
APRIL 11, 2023 | SLIDE 14
WHAT TO LOG
APRIL 11, 2023 | SLIDE 15
KEEP IN MIND
”What would help YOU at 3 AM in the morning? “
Image: Ambro / FreeDigitalPhotos.net
APRIL 11, 2023 | SLIDE 16
WHAT WILL HELP YOU
Exact notice of what happened When Where How Who
Clear concise context information Analysis possible without application
Manual Semi-automated Automated
Remotely collectible files Low footprint Proven reliable and authentic
APRIL 11, 2023 | SLIDE 17
WHAT WILL HELP YOU EXAMPLES
2011-05-31 08:46:11,308 +0200 INFO [eventlog.security] ACCESS OF ProfileServiceImpl.findProfiles BY [login: some_username]; Password: [PROTECTED]; Authenticated: true; Details: RemoteIpAddress: 192.168.156.17; SessionId: 8XyrNkLWv7FdXS… Granted Authorities:ROLE_USER,ROLE…
APRIL 11, 2023 | SLIDE 18
WHAT WON’T HELP YOU
No details about the record No context information Inconsistent format or bad use Non-correlatable Subjective/interpretative message Too much useless log records Logging to one single big file
APRIL 11, 2023 | SLIDE 19
WHAT WON’T HELP YOU – EXAMPLE #1
log4j: setFile called: ./logs/be/mobistar/network-status/log_functional.log, false
log4j: setFile ended tis nen default tis nen default Activations : 15 Deactivations : 0 In straigt to resource filter
APRIL 11, 2023 | SLIDE 20
WHAT WON’T HELP YOU – EXAMPLE #2
INFO [dao.ProductDao] [] - FOUND 40 INFO [web.listeners.PhaseLogger] [] -
+RENDER_RESPONSE INFO [service.ProductFamilyService] [] - product in
family is true INFO [.dao.ProfileDao] [] - [OBJECTS] find profiles
APRIL 11, 2023 | SLIDE 21
LOG MESSAGE DECOMPOSEDContext Meaning
WHEN Timestamp
WHERE System/Application/Component
WHO User
WHAT Action
RESULT Status
PRIORITY Severity, rank, level
APRIL 11, 2023 | SLIDE 22
“A log record should be understandable by a human, and easily machine
processable.”
APRIL 11, 2023 | SLIDE 23
LOGGING GUIDELINES: PRIORITIES
Fatal Application crash&burn
Error Exceptions Monitoring team alert
Warn Monitor health, performance
Info Informative Context Audit trail
Debug Extensive Context Development, Testing Troubleshooting
APRIL 11, 2023 | SLIDE 24
LOGGING GUIDELINES: EXCEPTIONS
Handling Recover Log the details if critical
Not Handling, Translating Translate Handling logic if not propagating
Not Handling Propagate Someone else will handle (and log!)
Choose exceptions to log Defend from stacktrace chaos Choose what to log for each exception
Catching block has all context details!
APRIL 11, 2023 | SLIDE 25
LOGGING GUIDELINES: CORRELATION
1 transaction = * log records Correlation necessary
Execution trail
Shared context variable Shared per transaction Reuse of context variables
Nested Diagnostic Context Map Diagnostic Context
Only one correlation Use a context unique variable Correlate transactions
APRIL 11, 2023 | SLIDE 26
HOW TO LOG IN
APRIL 11, 2023 | SLIDE 27
HOW LOGGING FRAMEWORKS WORK
APRIL 11, 2023 | SLIDE 28
JAVA LOGGING FRAMEWORKS
Implementation System.out – System.err
Redirect output Java Util Logging Log4j Logback
Abstraction Commons Logging SLF4J
APRIL 11, 2023 | SLIDE 29
JAVA UTIL LOGGING
Java 1.4+ Based on Log4J Less features out of the box
Console File Stream
APRIL 11, 2023 | SLIDE 30
LOG4J
First framework Introduction of Logger, Appender, Level Hiearchical Loggers Configurable appenders, e.g.
MailAppender FileAppender SMTPAppender SocketAppender SyslogAppender
Leader quit and started logback
APRIL 11, 2023 | SLIDE 31
COMMONS LOGGING
Apache Project API Bridge API-calls forward to
Log4j if on CP Falls back to JUL
Plagued Double Configuration Runtime dependency check Classpath Scanning
Discontinued v1.1.1 dates from end 2007
APRIL 11, 2023 | SLIDE 32
SLF4J - LOGBACK
Two libraries to rule them all Slf4j
Comparable with commons-logging Parameter parsing Framework bridges Much more powerful
Logback Evolved log4j Implements Slf4j “Picking up where log4j leaves off”
Testing Filters on-need-base
APRIL 11, 2023 | SLIDE 33
SLF4J BINDING
APRIL 11, 2023 | SLIDE 34
SLF4J BRIDGING
APRIL 11, 2023 | SLIDE 35
KEEPING IT PERFORMANT
String concatenation LOGGER.debug("Hello " + name); Protected block
if (logger.isDebugEnabled()) { LOGGER.debug("Hello " + name);}
Slf4j LOGGER.debug("Hello {}", name);
APRIL 11, 2023 | SLIDE 36
CONTEXT VARIABLES IN PRACTICE
NDC.push(correlationId); app.logMe("Hello world!"); NDC.pop();
Use %x in Pattern ConversionPattern = %x %d %m %n
correlationId 2011-06-21 09:20:41 [INFO] Hello world!
APRIL 11, 2023 | SLIDE 37
SPECIALIZED LOGGERS
Logger for class Fine-tune log Execution trace How distinguish
Security Performance Business
Specialized Loggers Priorities Output handlers
Grouped logging
APRIL 11, 2023 | SLIDE 38
LOGGING IN PRODUCTION
Only necessary logging Situation -> turn on debug logging
No restart: Dynamic Reloading ‘Watch’ file for changes
Log file management Rolling file Seperation of concerns
Clustering Different systems generate messages Synchronization of timestamps Correlation on host-specific files
APRIL 11, 2023 | SLIDE 39
MY OPINION
Importance of logging will grow Mobile / offline applications Cloud integration System interaction
Logging standards enterprisewide Standardization of logging effort Inexpensive exercise
Draft a standard Build adapters / patterns Enforce standards
APRIL 11, 2023 | SLIDE 40
LOG PROCESSING AND ANALYSIS
APRIL 11, 2023 | SLIDE 41
LOG ANALYSIS
Bang for your buck XpoLog
http://www.xpolog.com Licensed Faster
Splunk http://www.splunk.com Free Edition ‘Slow’ AJAX feel Nice graphics Modular (http://www.splunkbase.com)
Tuning your pattern is important?
APRIL 11, 2023 | SLIDE 42
DEMO
APRIL 11, 2023 | SLIDE 43
APRIL 11, 2023 | SLIDE 44
THANK YOU
For more information:visit our website WWW.REALDOLMEN.COM
Follow us on:
Selected presentations are available on:
Or scan this QR code with your Smartphone to immediately go to the website