TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data...

39
TIBCO Analytics Meetup TIBCO Data Science Team October 30 th 2018

Transcript of TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data...

Page 1: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

TIBCO Analytics Meetup

TIBCO Data Science Team

October 30th 2018

Page 2: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

The following information is confidential information of TIBCO Software Inc. Use, duplication, transmission, or republication for any purpose without the prior written consent of TIBCO is expressly prohibited.

CONFIDENTIALITY

Page 3: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. This document is provided for informational purposes only and its contents are subject to change without notice. TIBCO makes no warranties, express or implied, in or relating to this document or any information in it, including, without limitation, that this document, or any information in it, is error-free or meets any conditions of merchantability or fitness for a particular purpose. This document may not be reproduced or transmitted in any form or by any means without our prior written permission.

The material provided is for informational purposes only, and should not be relied on in making a purchasing decision. The information is not a commitment, promise or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for our products remains at our sole discretion.

During the course of this presentation TIBCO or its representatives may make forward-looking statements regarding future events, TIBCO’s future results or our future financial performance. These statements are based on management’s current expectations. Although we believe that the expectations reflected in the forward-looking statements contained in this presentation are reasonable, these expectations or any of the forward-looking statements could prove to be incorrect and actual results or financial performance could differ materially from those stated herein. TIBCO does not undertake to update any forward-looking statement that may be made from time to time or on its behalf.

DISCLAIMER

Page 4: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

TIBCO Analytics Meetup – opportunities to learn and network

Page 5: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

• Welcome and TIBCO update by Dan Rope• Python Data Function in Spotfire by Vinoth Manamala• Data Science Survival Skills by DivyaJyoti (DJ) Rajdev• TIBCO Community update by Heleen Snelting• Live Q&A

Please submit your questions at any time via Q&A option

We will answer at the end or will get back to you via email

Agenda

© Copyright 2000-2018 TIBCO Software Inc.

Page 6: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Dan Rope

Spotfire X

AI Using Spotfire &Spotfire Using AI

© Copyright 2000-2018 TIBCO Software Inc.

Page 7: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

7

Spotfire X with the A(X) Experience

© Copyright 2000-2018 TIBCO Software Inc.

AugmentedSearch & AI-Powered Insights

Start in seconds, get instant insights

AutomatedAutomagical Dataflows

Author & audit with automaticallyrecorded dataflow steps

AgileReimagined User Interface

Agile exploration made even easier

AcceleratedReal-time Data Streams

Real-time awareness and action

Analytics, Accelerated.

Page 8: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

8

Spotfire X with the A(X) Experience

© Copyright 2000-2018 TIBCO Software Inc.

AugmentedSearch & AI-Powered Insights

Start in seconds, get instant insights

AutomatedAutomagical Dataflows

Author & audit with automaticallyrecorded dataflow steps

AgileReimagined User Interface

Agile exploration made even easier

AcceleratedReal-time Data Streams

Real-time awareness and action

Analytics, Accelerated.

Page 9: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

AI Apps Using Spotfire

© Copyright 2000-2018 TIBCO Software Inc.

Page 10: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Critical business moments occur in every facet of enterprise operations.

They drive competitive differentiation, customer satisfaction and business success.

Inventory Management

Customer Engagement

Recommender Systems

Anomaly Detection

Patient Diagnosis and Treatment

Fraud DetectionPrice

Optimization

Intrusion Detection

AI Success Stories – Critical Business Moments

Risk Management

© Copyright 2000-2018 TIBCO Software Inc.

Page 11: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

• Supervised & unsupervised learning combine to recognize known risk patterns and identify anomalies

• Models deployed and applied to new incoming data

• Suspicious cases are case managed and then absorbed into the data for future model updates

AI on-demand / Continuous Learning

Augments People - Create / Operate Systems that actively manage Risk

AI Using Spotfire: Risk Management

© Copyright 2000-2018 TIBCO Software Inc.

Page 12: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Spotfire X Using AI

© Copyright 2000-2018 TIBCO Software Inc.

Page 13: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

13

Spotfire X: AI Suggestions Automated, Augmented insight discovery & display

Variable Relationships Algorithm• User selects target variable • AI algorithm discovers variable relationships to target• Spotfire displays in order of strength - best practices graphics• 4 clicks to brush-linked dashboard

Users and Data• Broad user persona• Scalable – including wide data• Flexible – iterative, rapid configuration

© Copyright 2000-2018 TIBCO Software Inc.

Page 14: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Select a Target

UX Workflow: AI Suggestions and beyond

See Graphs

Data preparation

Feature relationships

Ranking

+MORE LIKE THIS shows extra graphs with the same features Adds graph to the pageAnalyst Business User

Tell me about “Target”

Continue on the journey: adorn, manipulate, explore, drill

© Copyright 2000-2018 TIBCO Software Inc.

Page 15: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

AI & Variable Relationships

AI + People = Smarter Together

• AI-engine finds potentially interesting insights→ Less work to explore data (esp wide data)

• Human determines actual useful insights→ Curate final analysis, visuals, dashboard

AI Engine

• Based on well-established Data Science algorithms

• Customized for robustness and scale• Background processing for performance &

scalability

© Copyright 2000-2018 TIBCO Software Inc.

Page 16: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

TIBCO Runtime for R(TERR™)

TIBCO Data Science™ Python

Spotfire Data Functions - Enable AI / ML Applications

© Copyright 2000-2018 TIBCO Software Inc.

Page 17: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Vinoth Manamala

Python Data Function in Spotfire - Demo

© Copyright 2000-2018 TIBCO Software Inc.

Page 18: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Summary of Python Demo

© Copyright 2000-2018 TIBCO Software Inc.

Page 19: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

DivyaJyoti Rajdev

Data Science Survival Skills

© Copyright 2000-2018 TIBCO Software Inc.

Page 20: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

20

1. Ask the right questions

20

• Define clear objectives with domain experts

• What is more important:

insight discovery or prediction

Page 21: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

21

2. Figure out what to analyze

21

• Are some points more influential than others?

• Consider data relationships

• Use data transforms for predictors and featurization tricks

Outlier

Leverage points

Page 22: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

22

• Create baseline models

3. Explore to uncover insights

22

• Dive into visual analytics

• Sampling is your friend

Page 23: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

23

4. Build trustworthy algorithms

23

• Look at features critically

• Don’t just trust RMSE or accuracy use residual analysis

• Try variable importance

FeatureOutcome Var

Purchasing Power inc_index * zip_cat * hist.qty^2

Not Good

Page 24: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

24

• Prepare the data both before and after modeling

• Do you need all the predictors?

5. Make the most of your model

24

• Neural nets aren’t “God”

Garbage In = Garbage out

Page 25: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

25

6. DevOps in analytics

25

• Testing isn’t just for software engineers

• Agile principles for data science

• Knowledge and frameworks are transferable

SAP HANASAP HANA

Page 26: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

26

• Visuals are more than just reporting

7. Bridge the talk divide

26

• Build consumable solutions for personas

Tech DashboardExecutive Dashboard

• Interpretability is important

Page 27: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

27

Key Takeaways

27

1. Ask the right questions

2. Figure out what to analyse

3. Explore to uncover insights

4. Build trustworthy algorithms

5. Make the most of your model

6. DevOps in analytics

7. Bridge the talk divide

Page 28: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Heleen Snelting

TIBCO Community Update

© Copyright 2000-2018 TIBCO Software Inc.

Page 29: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Swap Logoif neededIn master slide

TIBCO Community the platform for our users

Page 30: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Data Literacy - things you want to know and teach

© Copyright 2000-2018 TIBCO Software Inc.

Experts sharing knowledge on terminology you and your teams are probably using and learning about. Theory and practical ideas.

● Data Literacy● Statistical Literacy● Visual Literacy & Storytelling

Learn, join in, take the Quiz, contribute!

Page 31: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Statistica and Python Data Functions

Statistica

Spotfire

Spotfire Data Function Tips & Tricks section

Page 32: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Spotfire Tips & Tricks page - continues to develop

© Copyright 2000-2018 TIBCO Software Inc.

Access & Visualize Graph data from Neo4j in Spotfire

Custom Maps - with Dr SpotfireCreate R Graphics in Spotfire

Page 33: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Analytic Development Languages

TIBCO provides a collaborative analytics work environment for everyone: Data Engineer, Data Scientist, Developer, ..

• Spotfire gives the interactive visualizations which can call and communicate with TIBCO Data Science tools including TERR

• Analysts can work in no-code workflow based environment

• But can also use open source codes (R, Python, C, Scala, Spark,…)

• and use external knowledge from analytic market places (Azure ML, Apervita, Algoritmia, H2O,…)

Page 34: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Customer Orientation

Customer Orientation page and linked content is guiding new users through a self-service on-boarding process showing how to access resources and how to use Spotfire and other TIBCO Analytics software.

Content built by TIBCO’s customer success team!

Example of one sub-section shown on left

Page 35: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

What’s new in… Spotfire Data ScienceLatest features for every new release

Explained for users with examplesIncluding interoperability with other TIBCO Analytics software

Page 36: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

TIBCO NOW 2018

© Copyright 2000-2018 TIBCO Software Inc.

and 2019

Page 37: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

Google Search is also a great way to find TIBCO Community content

What’s new in… for example Spotfire for soon to be released Spotfire X info

https://community.tibco.com/wiki/whats-new-tibco-spotfire

Customer Orientation and Customer Success Center - ideal for on-boarding and staying up to date

https://community.tibco.com/wiki/tibco-analytics-new-customer-orientation

https://community.tibco.com/wiki/tibco-spotfire-customer-success-center

Essential Spotfire Analytics Resources including Doctor Spotfire & Tips & Tricks - really useful for adoption, new users

https://community.tibco.com/wiki/essential-spotfire-analytics-resources

TIBCO Geo-Analytics capabilities

https://community.tibco.com/wiki/tibco-spotfire-location-analytics-mapping-geoanalytics-and-spatial-statistics

TIBCO Analytics Meetup pages with recordings and presentations

https://community.tibco.com/wiki/tibco-analytics-meetup

TIBCO Blogs - Data Science & AI

https://www.tibco.com/blog/category/analytics/© Copyright 2000-2018 TIBCO Software Inc.

Your top TIBCO Community links to bookmark

© Copyright 2000-2018 TIBCO Software Inc.

Page 38: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

38

Questions & Contact

© Copyright 2000-2018 TIBCO Software Inc.

Thank you!Heleen Snelting [email protected]@HeleenSnelting

Dan [email protected]

TIBCO Communitycommunity.tibco.com

TIBCO Exchangecommunity.tibco.com/exchange

Spotfire Trialspotfire.tibco.com/trial

Page 39: TIBCO Data Science Team TIBCO Analytics Meetup...2018/10/30  · TIBCO Analytics Meetup TIBCO Data Science Team October 30th 2018 The following information is confidential information

© Copyright 2000-2018 TIBCO Software Inc.