GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application...

20
GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik Data Analyst

Transcript of GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application...

Page 1: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

GO BEYOND DATAReal-time Analytics for Application

Performance Management

Yury OleynikData Analyst

Page 2: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 2

AgendaModern applications

Monitoring challenges

INSTANA apploach

Page 3: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 3

Business Demand: Write and Deploy code faster!

Drives adoption of - Cloud - Containers / Microservice- Reactive / Polyglot Technologies

- Agile - Continuous Delivery

Drives adoption of

Page 4: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 4

Monitoring

Why monitor systems and the applications?- to obtain information needed to guide whether the system is working properly

Reality of monitoring- produce data - data is not information- current insight tools are system oriented- built from the perspective of the system providing the metrics- require specialised knowledge to use and interpret

How it should be- information about the quality of service- help diagnose what is causing the problem- suggest what to do to fix the problem

Page 5: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

NetflixJanuary 2014

“And at our scale, humans cannot continuously monitor the status of all of our systems. To maintain high availability

across such a complicated system, and to help us continuously improve the experience for our customers, it is

critical for us to have exceptional tools coupled with intelligent analysis to proactively detect and communicate system faults and identify areas of

improvement.”

Page 6: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 6

Daily/Hourly code and configuration changes.

AGILE

Monitoring Challenges: Intense operational complexity

Shared infrastructure.

CLOUD

Throw away infrastructure.

CONTAINERS

Experts knowledge needed.

POLYGLOT

Non-deterministic code path.

REACTIVE

Page 7: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 7

Vision

Page 8: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 8

Creating a Virtual DevOps Expert

Page 9: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 9

Creating a Virtual DevOps Expert

Intelligent Sensor Technology- Dynamic Component Discovery- Realtime Sensoring

Page 10: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential

Data viewed as 5 second running average followed by 1 second data points. INSTANA collects 1 second resolution data.

Data viewed as 1 minute running average

Aggregation = loss of information

10

CURRENTAPM

Intelligent Sensor TechnologyRealtime Sensoring

Page 11: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 11

Creating a Virtual DevOps Expert

Intelligent Health Management- Dynamic Dependency Graph- Adaptive Learning- Predictive Alerting & Optimization

Page 12: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 12

Persistence

Stream Processing

Dynamic Graph

Communication

Sensor Data

Realtime Stream Processing

Machine Learning

Knowledge Base

Predictive Alerting

Health Management

Raw Store

Memory

Result Store

Health Signatures

3D Map

Intelligent Health MangementStreaming, Analytics, Learning & Knowledge

Page 13: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 13

Dynamic Dependency Graph

JV

JV

JV

JV

JV JV

JV

JV

JV

CassandraCluster

ServiceA

TomcatCluster

JVM JVM

App1

App2

JVM JVM

App3

ServiceB

Schema1

Schema2

Page 14: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential

JV

JV

JV

JV

JV JV

JV

JV

JV

CassandraCluster

ServiceA

TomcatCluster

JVM JVM

App1

App2

JVM JVM

App3

ServiceB

Schema1

Schema2

JV

CassandraCluster

CassandraCluster

Dynamic Dependency Graph

14

Page 15: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential

JV

JV

JV

JV

JV JV

JV

JV

JV

CassandraCluster

ServiceA

TomcatCluster

JVM JVM

App1

App2

JVM JVM

App3

ServiceB

Schema1

Schema2

CassandraCluster

CassandraCluster

15

Dynamic Dependency Graph

App2

JV

Page 16: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 16

Predictive Alerting & Optimization

Realtime Event Stream Intelligent Health Management Alterts & Optimizations

Severe Situation Detected JVM GC Overhead too high - Impact on Service >20%.

Knowledge Base recommendations:1. Update to Java 1.6.62 2. Increase Eden Space to -XX:NewRatio=2

at 10:43am.

Optimization Detected.com.mycomp.Calc.calc() consumes 20% of Clock time.

Optimization will have high impact on Service response time for Shop service.

a few seconds ago

Page 17: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 17

Creating a Virtual DevOps Expert

3D Map- Runtime Visualization- Persona Optimized Perspectives

Page 18: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Demo

Page 19: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 19

STAN - a Virtual DevOps Expert

Page 20: GO BEYOND DATA - performance-symposium.org · GO BEYOND DATA Real-time Analytics for Application Performance Management Yury Oleynik ... Machine Learning Knowledge Base Predictive

Instana, Inc. Proprietary and Confidential 20

THANK YOU!