Muhammad Siddiqi

Event Source API &

Semantic Logging

@SiddiqiMuhammad

http://shujaat.net

MARQUEE SPONSOR

PLATINUM SPONSOR

PLATINUM SPONSOR

GOLD SPONSORS

SILVER SPONSORS

Ask the Logs!!!

176 [main] INFO examples.Sort - Populating an array of 2 elements in reverse order.225 [main] INFO examples.SortAlgo - Entered the sort method.262 [main] DEBUG SortAlgo.OUTER i=1 - Outer loop.276 [main] DEBUG SortAlgo.SWAP i=1 j=0 - Swapping intArray[0] = 1 and intArray[1] = 0290 [main] DEBUG SortAlgo.OUTER i=0 - Outer loop.304 [main] INFO SortAlgo.DUMP - Dump of integer array:317 [main] INFO SortAlgo.DUMP - Element [0] = 0331 [main] INFO SortAlgo.DUMP - Element [1] = 1343 [main] INFO examples.Sort - The next log statement should be an error message.346 [main] ERROR SortAlgo.DUMP - Tried to dump an uninitialized array.467 [main] INFO examples.Sort - Exiting main method.

When should we deploy?

When should we schedule maintenance?

What is the effect of our recent email advertising campaign?Should we support IE or Windows XP

anymore?

Why does application response gets slower between 6 – 11 PM?

Is the application flow as we expect? (I see a lot of items getting added to the cart but no orders, what is going on?)

We need…

Meaningful structured logging

Minimum impact on performance

Session Agenda?

-What is Semantic??

- Event Source API

- Semantic Logging Application Block

- Event Source & ETW Tools

- Event Source & Transfer Events

- Event Source & Windows Event Logs

Semantic [Webster]“Of or relating to meaning, especially meaning in language.”

-------Log What you mean!!! ------ What, Where & How??

Semantic Web { Tim Berners Lee – Scientific American 2001 }[ converting the unstructured / semi-structured data on web to structured one ]

[ a web of data that can be processed by machines ]

--- Creating logs which can be processed by machines ---

Other Names In the Industry- Structured Logging- Strongly typed events

Semantic Logging is to create logs with machine’s consumption in mind

Log4Net Exam

pleLog File

EventSource APIProvides the ability to create events for event tracing for Windows (ETW).

.net framework 4.5

EventSourceProvides the ability to create events for event tracing for Windows (ETW).

Helps with…

• Allows to separate the event authoring request from writing the event. + Simpler Logging site.

• Make it easier to work with ETW.

• Takes care of • generating the manifest for your events• converting your method parameters to event data.• XCOPY deployment now possible.

automatic

Performance – ETW based Structured Payload Cross-application

Benefits

EventSource & EventListener

Logging Call site

RFC4122

Listener ConsumerController

In-Proc Usage

Controller (Attachment with EventSource)

Consumer [ OnEventWritten() ]

EventSource’s [Event] MethodsLevel Keyword

Task & Opcodes Versioning

Max level of events to enable

VIWECL Very Intelligent & Wise Elfs Can Learn

Power of 2

Task oriented grouping of events

Task

OpcodesUsed in conjunction

Integer Value

Operation within a taskTask Opcodes

Versions cannot co-exist…duplicate Event Ids not allowed

Demo EventSource with attributes

Event Source Proxy

ETW : Event Tracing for Windows

Event Provider

Event Payload

.net framework ETW

Event Type

Method Parameters

Event Method

Event Source

Event Controller

Event Consumer

Event Listener

EventSource & ETW Mapping

Semantic Logging Application Block [SLAB]

- Provides- Customized Listener- Sinks- Unit Testing

Destination specific

Console

Flat File

Rolling Flat File

Database

Windows Azure Table Storage

- Can be consumed- In-Proc- Out-Proc

Semantic Logging Service

SLAB is about making your log talk!!! EventSource : Authoring LogsSLAB: Example for Logs Consumption

SLAB & Rx

ObservableEventListener : IObservable<EventEntry>

ConsoleLog

FlatFileLog

RollingFlatFileLog

SqlDatabaseLog

WindowsAzureTableLog

XLog

LogToX (extension methods

- creates subscription)

ConsoleSink

FlatFileSink

RollingFlatFileSink

SqlDatabaseSink

WindowsAzureTableSink

XSink

IObs

erve

r<T>

SLAB Nuget Packages

Semantic Logging Service Separate Download

Unit Testing Event Source

Why? EventSource.EventMethod[Parameters]

ETW Event Type

EventSourceAnalyzer

EventSourceInspect

• Error when the given EventSource is enabled by an EventListener?• Can event schema be requested from the EventSource?• Can all Event based methods be invoked in the specified EventSource

class?

Checks Performed

Event Text Formatting

- Flat File Sink- Rolling Flat File Sink- Console Sink

Supported Sinks

+ Custom Event Text Formatter

Semantic Logging Service

Can run as Console Application

Windows Service

Out-Proc Event Listener

Must run on the same machine as source

Xml based configurationsDynamically updated

No restart required

Available as a separate download

Increased fault tolerance for Application crashes

Can monitor Event Providers from multiple processes

SLAB Customizations

Custom Text formatters

Custom Sinks

Rx In-Proc filtering

Console color and level mapping

Application Buffer1

ETW Sessions

Semantic Logging Service

S3

S3 Sink

SLAB Custom Sinks

Demo Using Semantic Logging Service

with Windows Azure Table Storage Sink &

Sql Database Sink

ETW Tools

ETW Manifests

Installed[Static Registration]

Inline [EventSource][Dynamic Registration] Manifest based Provider

Classic Provider

TraceEvent Package

EventSource & ETW Tools

ETW Tools must incorporate support using Trace Event API

WPR

XPerf

PerfView

…..…..…..…..

ETL FilePerf Monitor

LogMan

WPA TraceRpt

Performance Monitor [Perfmon.exe]

PerfMonitor 2.01Not a next version of PerfMon but a different utility.Also called command line version of PerfViewBased on Trace Event Library

PerfMonitor listSources MyApp.exe PerfMonitor monitor MyApp.exe <args>

Single file deployment

Perfmonitor listSessions

PerfView

Based on Trace Event Library

Can run using Command Line

Shows Event Data on User Interface

Vance Morrison’s for WPA support

Windows Performance Analyzer

WPR

XPerf

PerfView

…..…..…..…..

ETL File

Collectors

Log Manager [LogMan]

TraceRpt

- Processing binary ETL files into human readable formats including XML / CSV

- Xml (default) or Html based report for generated events data

- Command line utility

Activity Id for Events [Transfer Event]Grouping related events from more than one component

Helps in understanding End-to-End scenarios

Supported in both Manifest based and Classic ETW provider

EventSource

Additional WriteEvent like methods

EventWrittenEventArgs EventListener.OnEventWritten()

.net framework 4.5.1

Correlation between calls and tiers. Can span applications

Semantic Logging Service

Application 1

Application 2

Application 3

ETW Infrastructure

Event Store

ActivityId = X, RelatedActivityId = A

ActivityId = X, RelatedActivityId = C

ActivityId = X, RelatedActivityId = B

EventSource & Windows Event Log

EventSource doesn’t support Windows Event Log by default

No Channel support!

Microsoft.Diagnostics.Tracing.EventSource (Standalone New definition)Provides

Compile time checks

Supports .net framework 4.0

Naming Event Source – Hyphenated Friendly names Consult IT

Consult IT[Channel]

Step I : Implementing EventSource

Step II : Compile the Project

Step III : Registering Provider

Using the Package…

Unified Enterprise Logging

Questions ???

http://shujaat.net

@SiddiqiMuhammad