All Together Now: Connected Analytics for the Internet of Everything
-
Upload
inside-analysis -
Category
Technology
-
view
10 -
download
1
Transcript of All Together Now: Connected Analytics for the Internet of Everything
Grab some coffee and enjoy the pre-show banter
before the top of the
hour!
The Briefing Room
All Together Now: Connected Analytics for the Internet of Everything
Twitter Tag: #briefr The Briefing Room
Reveal the essential characteristics of enterprise software, good and bad
Provide a forum for detailed analysis of today’s innovative technologies
Give vendors a chance to explain their product to savvy analysts
Allow audience members to pose serious questions... and get answers!
Mission
Twitter Tag: #briefr The Briefing Room
Topics
August: REAL-TIME DATA
September: HADOOP 2.0
October: DATA MANAGEMENT
Twitter Tag: #briefr The Briefing Room
Outside In
Ø Centrifugal Reality
Ø Responsiveness Required
Ø Customization Paramount
Ø Context is King
Twitter Tag: #briefr The Briefing Room
Analyst: Mark Madsen
Mark Madsen is president of Third Nature, a technology research and consulting firm focused on business intelligence, data integration and data management. Mark is an award-winning author, architect and CTO whose work has been featured in numerous industry publications. Over the past ten years Mark received awards for his work from the American Productivity & Quality Center, TDWI, and the Smithsonian Institute. He is an international speaker, a contributor to Forbes Online and on the O’Reilly Strata program committee. For more information or to contact Mark, follow @markmadsen on Twitter or visit http://ThirdNature.net
Twitter Tag: #briefr The Briefing Room
Cisco
Cisco Systems is a known leader in the design, manufacturing and sales of networking equipment
Cisco offers Connected Analytics, a comprehensive platform designed to deliver a complete infrastructure for streaming analytics
The platform supports open APIs for custom solutions and application development
Twitter Tag: #briefr The Briefing Room
Guest: Kim Macpherson
Kim Macpherson is the Director of Analytics Engineering in the Data and Analytics Business Unit at Cisco Systems. She is developing analytics technologies and architectures for devices and fog nodes for platforms, applications and solutions that deliver business value for infrastructure, operations and the line of business. She is contributing to Cisco’s software platform strategy by embedding analytics software into Cisco devices, creating analytics services and delivering analytics applications to service providers, enterprises and vertical industries through digitization and hyper-distributed processing. Kim joined Cisco via the 2012 acquisition of Truviso, Inc., a venture-funded startup company providing streaming real-time analytics, where Kim was Senior Vice President of Engineering and Business Operations. Before joining Cisco, Kim served as Vice President of Engineering for Consumer Security products at EMC and Vice President of Global Professional Services at RSA Security Systems.
Kim MacPherson
August 18, 2015
Data and Analytics Briefing Room
2 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Data & Analytics Unlock the Value in the Internet of Everything
Data Monetization
Customer Engagement
Optimized Operations
Organizational Effectiveness
Threat Minimization
Source: Cisco Consulting Services primary research, 2013
of the $19T IoE Opportunity over the Next 10 Years will be Realized Through Data & Analytics $7.3T
3 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Getting insight from the Data is the Client’s Biggest Challenge
Data Effectively capturing, storing, and analyzing data generated by connected “things” (e.g., machines, devices, equipment)
Things Connecting the right “things”
(e.g., machines, devices, equipment) to capture useful data
Process Updating our business and operational processes to benefit from IoT solutions
People Enabling workers to effectively
use IoT solutions through means such as training and providing
user-friendly systems
1230 Respondents
Source: Cisco Consulting Services Global IoT Study, 2014
20%
27%
13% 40%
In Which of the Following Areas Does Your Organization Need to Improve the Most to Make Effective Use of IoT Solutions?
4 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Widely Distributed, Streaming, Short Shelf Life, Too Big to Move
“Most data will be processed at the edge” (mobile devices, appliances, routers)
Digital Enterprises See an Explosion of Data at the Edge
Three years from now, where will most data generated by IoT solutions be processed?
1230 respondents Source: Cisco Consulting Services Global IoT Study, 2014
37%
5 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Connecting Diverse Types of Data is Required to Uncover the Most Valuable insight
Structured Data Unstructured Data Data Streaming at the Edge
Traditional Data Warehouse
Big Data Store
6 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
However, a New Approach is Needed to Reach and Analyze That Data
Structured Data Unstructured Data Data Streaming at the Edge
Traditional Data Warehouse
Big Data Store
Analytics 1.0 Analytics 2.0 Hours/Minutes/Seconds Days/Hours
Analytics 3.0 Seconds/Milliseconds
7 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Operational Consistency Data Mobility Optimized Form Factors
UCS Mini
UCS Mini
UCS for Enterprise
UCS for Hadoop
Nexus Family
ISR
APIC EM
AP MS
CGR
IE
Video
Cloud Services and Applications
Partner Clouds
Intercloud Fog & Edge Core Data Center
Only Cisco Delivers the Connected Infrastructure You Need to Reach from the Data Center to the Edge
8 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Integrated Infrastructure From Data Center to Edge
On-Demand Performance & Scalability
Integrated Security
Unified Management
Providing End-to-end Consistency, Performance, and Security Across Analytic Applications
Data Center to Edge Analytics Infrastructure, Lowest Analytics TCO
9 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
We are Investing in Three Ways to Turn Data from Connected Infrastructure into Extraordinary Outcomes
Real-time Connected Solutions 3
Analytics Packages for Optimized Infrastructure
2
Agile & Pervasive Data Access 1
Network
Collaboration
Contact Center
Video
Sensor-based Real-time
Location / Mobility
Energy Management
Threat
Analytics from Core Cisco Competencies
10 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Cisco Solution Support Services
Cisco Advanced Services
Data Science Services
Partner Designed Solutions
Cisco Consulting Services
Streaming Analytics Intellectual Capital Data Virtualization Work-load Automation
Agile & Pervasive Data Access
Integrated Infrastructure and Networking
Big Data & Analytics Ready Infrastructure
Real-time Connected Solutions Connected
Analytics for Events Connected Analytics
for IT Connected
Analytics for Retail Connected
Analytics for SP
Analytics Packages for Optimized Infrastructure
Cisco Connected Analytics for the Internet of Everything
1
3
Contact Center
Video Analytics
Threat Analytics
Location Analytics
Collaboration Analytics
Network Analytics
Energy Analytics 2
NEW
NEW
Devices Wi-Fi Video POS Sensors Social 3G/4G
Access Wired | Small Cell | Wi-Fi
Data Center | Intercloud Core/Edge | IoT | IoX
Mobile Packet Core/SON I Unified Communication
Collaboration | Contact Center
11 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Integrated Streaming query processing supports active, continuous monitoring of live data. Get
instantaneous real-time analysis and action, and efficient use of compute
resources.
Scalable Offers the framework and interfaces for use case development across a
wide variety of business and network management functions and
industries.
React intelligently Provides real-time insights with Big Data views to support actionable
events and dynamic dashboards to help you get more value out of your
data.
What is Cisco Connected Streaming Analytics? An analytics platform delivering predictive, actionable insights in high velocity streams of live data
from multiple sources, enabling real-time governance and immediate actions. It adapts to customer environments and use cases for a wide range of industries and business applications.
12 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Ultra-fast performance, real-time analytics
for multiple data sources
Flexible data integration framework including
infrastructure-ready adapters
Customizable live dashboards and reporting framework that let
you look at data your way
Reliability with high availability, failover, and online backup and
restore
A variety of deployment architectures for cloud, data
center and fog
Cisco Connected Streaming Analytics Offers
Seamless persistent data management for historical
reporting, big data integration
13 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
• Dynamic dashboards • Real-time trending • Predictive analytics integration • Continuous queries • Event generation
• Combine information from network and applications
• Seamlessly query live and historic data
• Historic reporting framework
• Complex queries from fact streams and dimensional data
• North/south and east/west interfaces for customization
• Multivendor extensibility
Cisco Connected Streaming Analytics Drives
Big Data Flexibility Real-Time
14 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Value of Real-Time Connected Streaming Analytics
… Enabling closed-loop use cases:
Make it actionable in real time
Make it scale without sacrificing latency
Integrate advanced predictive analytics and machine learning
Transparently combine both live and historic data
Collect Analyze Alert and store Adjust
Make your data work for you
Usage data
15 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Real-time Streaming Analytics
• Mine (fetch) • Analyze • Report Usage Data
!
Generate an Actionable
Event
that is sent to the Policy System, Management System, etc. to allow immediate control
Next Generation Analytics Applies predicates, aggregations, and joins with metadata tables and contextual data to
identify and match trends.
Querybase Waiting for Data Store raw data or filtered data for further mining.
Database Waiting for Queries Store raw data for further mining.
Traditional Analytics Model Store first, and query later.
Usage Data
• Mine (fetch) • Analyze • Report
16 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Alerts
Predictive
Pattern Detection
Aggregation engine for pre-computation and reporting
Massive data reduction
Connected Streaming Analytics Capabilities Live Data Expertise
Decouple “Data Processing” from “Result Consumption”
Immediate Consumption True real-time streaming analytics
Delayed Consumption Heavy lifting in real time Pre-aggregated results on demand
High-cardinality analytics: Internet-scale dimensions • Compressed bitmap aggregates for uniques • Per-actor analysis Location
Deviations based on history, thresholds and cohort analysis
Dynamic model scoring, classification and categorizing
Finding patterns and detecting complex events
High-performance spatial indexing
17 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Prebuilt Adapters Available & Framework to easily create
other adapters
Business Data
Operations Data
Infrastructure Data
Sensor Data
Analytics for Streaming & Historical Data
Multiple Data Sources
Continuous & Interactive Query Processing over Streaming &
Historical Data
Intelligent Actions / Events
XML, alarms …
Customizable visualizations
18 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Sample Cisco Streaming Analytics Dashboard
Customizable Dashboards Look at the Data Your Way Live and Historical Dashboards
• Web-based drag and drop dashboard design including rich navigation, drilling and a library of filter controls.
• Custom build dashboards uniquely tailored for your business.
• Portal and mash-up integration seamlessly connects business analytics with other applications.
19 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Historical Reporting Capabilities • Self-service report creation allows you to easily
select data; sort, filter, and group data; and generate reports from a web browser (prepackaged sample reports are provided)
• Reporting administration capabilities include:
Export to PDF, CSV, XLS, and other formats
Report scheduling
Publishing by email or on a local server
• Customization: Drag-and-drop report creation and impromptu reporting
20 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Customization and Application Building • Cisco Connected Streaming Analytics
supports open APIs and commercial database methodologies to develop solutions and custom applications
• Developers can use commercial tools and standard APIs such as SQL, XML, and Java to develop use cases, applications, and system integrations
• Cisco knows the infrastructure data, and enterprises and service providers know their business data
Each organization is unique and has it’s own specific analytics needs:
• Use Cases • Data sources • Customers • Infrastructure
21 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Use Case - Mobility KPI Dashboard Connected Analytics for Mobility (CAM) • Unique and new clients trend analysis by date, week or month
• Real-time active session information in real-time to identify network usage pattern or manage promotional programs
• Authenticated vs. unauthenticated associations to identify potential customers
• At-a-glance information on total data usage to identify network anomaly or usage spikes
• Drill down for further analysis
Product currently under development. Actual Production UI may vary.
22 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Use Case - Shopper Insights: Dwell Time Analysis Connected Analytics for Retail (CAR)
Shopper insights create competitive advantage for retailers- shows where, how and why buyers make decisions on path to purchase • Measure shopper
engagement with product and promo displays
• Insights into store areas where shoppers spend most time
23 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
What can you do?
• Correlate information from multiple sources to derive
best representation of what is occurring in your stadium
• Customer Behavior Analytics, Attendance Trends & Wi-
Fi Body Language of your Fans
Benefits:
• Unprecedented insight & trends into your Fans
• Better understanding of who your Fans are
• Measure sponsor & advertisement impact and response
Use Case - Fan Behavior and Device Usage Connected Analytics for Events (CAE)
Product currently under development. Actual Production UI may vary.
24 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Horizontally & hierarchically scalable yet embeddable
Integrate advanced predictive analytics and machine learning
Immediate Insight: True real-time with low-latency
Data acquisition & mediation from the Internet of Everything
Blends live & historical sources from Big Data ecosystem
Open APIs and highly customizable: SQL, R, JDBC etc.
Connected Streaming Analytics Advantage
25 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Want to learn more?
Register and attend: Cisco Data and Analytics Conference October 20-22, Chicago Hilton http://cisco.com/go/daconf Kim MacPherson, Cisco Dir. Engineering, Data and Analytics [email protected]
Twitter Tag: #briefr The Briefing Room
Perceptions & Questions
Analyst: Mark Madsen
11
All Together Now: Connected Analy4cs for the Internet of Everything Analyst comments August, 2015 Mark Madsen Third Nature @markmadsen
Copyright Third Nature, Inc.
The data warehouse. More of a CostCo, really. Came about in a pre-‐event streaming market
Copyright Third Nature, Inc.
Core idea: centralize: that solves all problems!
But there are tradeoffs:
Creates bottlenecks
Causes scale problems
Enforces a single model
Injects latency
Copyright Third Nature, Inc.
New terminology: Data is the new oil
Copyright Third Nature, Inc.
New terminology: Data Exhaust
Copyright Third Nature, Inc.
New terminology: the data lake
Copyright Third Nature, Inc.
There is a different assump4on with these terms
“Data isn’t something we make, it’s just there.” But the prevailing model for management and use is sJll “centralize”
Copyright Third Nature, Inc.
There is a different assump4on with these terms
“Data isn’t something we make, it’s just there.” But the prevailing model for management and use is sJll “centralize” This won’t work
Copyright Third Nature, Inc.
In a mostly-‐connected world, events occur in different 4me frames, follow different cycles of use
19 Source: Noumenal
Disconnected
Milliseconds Minutes Hours+
Copyright Third Nature, Inc.
In a mostly-‐connected world, events occur in different 4me frames, follow different cycles of use
20 Source: Noumenal
Disconnected Every event is
persisted for some period of Jme before
it is forgoRen or forwarded
Milliseconds Minutes Hours+
Copyright Third Nature, Inc.
In a mostly-‐connected world, events occur in different 4me frames, follow different cycles of use
21 Source: Noumenal
Disconnected
Local context and control,
local decisions, local latency
Milliseconds Minutes Hours+
Copyright Third Nature, Inc.
In a mostly-‐connected world, events occur in different 4me frames, follow different cycles of use
22
Disconnected
Source: Noumenal Milliseconds Minutes Hours+
Bigger context, likely correlated, more complex rules, external monitoring
Copyright Third Nature, Inc.
In a mostly-‐connected world, events occur in different 4me frames, follow different cycles of use
23
Disconnected
Source: Noumenal Milliseconds Minutes Hours+
Broad context, human intervenJon, diagnosis and analyJcal tasks that have to be coordinated.
Copyright Third Nature, Inc.
In a mostly-‐connected world, events occur in different 4me frames, follow different cycles of use
24
Disconnected
Source: Noumenal Milliseconds Minutes Hours+
Data lives in mulJple places, at mulJple levels of detail, for differing duraJons. Unlikely to all be in one place. Nor should it be.
Copyright Third Nature, Inc.
Flowing Persisted
Sliding window of “now”
Persisted but not yet loaded into a platform
Queryable history
Managed history
Streaming isn’t either-‐or, it’s part of IT architecture
A DB can get you to within minutes (at large scale) but it won’t be easy or cheap, mainly lives in the realm of history
Event streams, in-mem stores, CEP streaming SQL can be used for these
Real time monitoring doesn’t use only real time data: windows, restarts, detecting deviation, so the above boundaries are crossed.
ESB Cache/Queue Database / platform
Copyright Third Nature, Inc.
Stream
If you want to do real4me and s4ll manage your data effec4vely then you need to rethink data architecture
Collect Refine Manage Deliver
Flowing Managed history Persisted
Microservices Metadata Metadata & reuse?
Flow, persisted, managed define different access, processing, storage and retrieval requirements
Copyright Third Nature, Inc.
Rethinking soLware and data architectures is required. Enterprise IT needs to be renovated.
Twitter Tag: #briefr The Briefing Room
Twitter Tag: #briefr The Briefing Room
Upcoming Topics
www.insideanalysis.com
August: REAL-TIME DATA
September: HADOOP 2.0
October: DATA MANAGEMENT
Twitter Tag: #briefr The Briefing Room
THANK YOU for your
ATTENTION!
Some images provided courtesy of Wikimedia Commons and http://lylawarren.com/2012/06/denver-comic-con/