Post on 18-Mar-2020
The Reality of Real-Time
The "what, why, and how“ of
accessing and using real-time data
from your enterprise systems.
Moving the Needle 2014
Thursday, Nov. 6th - 9:40 am
Brian Parish – CEO and Founder, IData
The primary takeaway from this presentation?
Your Goal is “A Clear Path from Question to Answer.”
How does real-time data fit in this path?
Agenda – The Reality of Real-time
Agenda – The Reality of Real-time
When?
When is real-time? What do we mean by real-time data?
Why?
Understanding your business needs/questions
What?
Understanding what data you need
How?
Architecture and Connectivity options for real time.
So?
Getting people to use and trust the data
Who?
How IData can help you on the path.
Introductions
About the IData Presenters
Brian S. Parish:
• President, IData Inc.
• 19 years in Higher Ed Tech
• Worked with well over 150 schools on technology and reporting projects
About IData Incorporated
• Higher Education Data Management
• Started in 2004
• Based in Alexandria, VA
• Technology Services
• Reporting and Data Governance
• System Integration
• Custom Application Development
• Implementation and Migration Services
• Innovative Software
Web services integration platform for building connectors to higher education ERP systems
Collaborative data dictionary and data management tool for higher education
IData Products
Who is here?
• From an IT office?
• From an IR office?
• From a functional office?
• From a vendor?
• Other?
WHAT DO WE MEAN BY REAL-TIME?
When?
When is real-time?
Some Definitions first:
• What is Real-time data?
• What is near real-time data?
• What is snap-shot data?
• What is Longitudinal data?
• Do people sometimes mean “Automated?”
“real-time”
UNDERSTANDING YOUR BUSINESS NEEDS/QUESTIONS
Why?
Why?
• There are compelling business reasons for real-
time.
• Are these needs well understood.
• Thought – All data request should be a
conversation that start with the business need or
the question being asked.
Why?
Examples of real-time data
requirements?
Why?
Think critically about the need.
• Some solutions will require real-time data
• Some will not (even though it was asked for)
• Some can be solved by looking at different
architecture options
UNDERSTANDING WHAT DATA YOU NEED
What?
Data Management Practices?
• Data Governance Group
• Data Dictionary
• Data request process
• Report Specification templates or tools
• IR and IT collaboration
• BI Group
• Report/Data documentation or transparency
• Trust?
What?
The key is good data definitions
• Your functional definitions (business rules)
• Your technical definitions (data systems)
• Multiple technical definitions for different data
sources (real-time vs warehouse)
Understand and document both as part of any
reporting project.
Observation:
• Reporting is Hard.
• Institutions continue to struggle with reporting.
• Some schools are successful with a wide range
of reporting tools.
• Many schools have been unsuccessful with the
same tools.
Conclusion:
ARCHITECTURE AND CONNECTIVITY OPTIONS FOR REAL TIME.
How?
Reporting Architecture
Reporting Environment?
• Data Warehouse?
• Homegrown
• Commercial (COTS)
• Operational Data Store?
• Transactional Reporting tools?
• Census Data Extracts
• Aggregate reporting (combining systems)
• Standard ETL Tools?
Direct Reporting vs Warehousing
• Real-time reporting is simple if you are primarily
reporting directly from the production
transactional system.
• Things get more complicated for real-time if you
are primarily reporting out of a data warehouse
• Hybrid approaches can work.
• Real-time ETL can also work
5 architecture approaches
• Direct to Transaction reporting
• Direct to warehouse with scheduled ETL
• Direct to warehouse with real-time (triggered) ETL
• Reporting that connects to both warehouse and
transactional system
• Separate tools for real-time and warehouse
reporting
Transactional Systems
Architecture Options
• Direct to Transaction reporting
Transactional Systems
Architecture Options
Data Warehouse
• Direct to warehouse with scheduled ETL
Transactional Systems
Architecture Options
Data Warehouse
• Direct to warehouse with real-time (triggered) ETL
Transactional Systems
Architecture Options
Data Warehouse
• Reporting that connects to both
warehouse and transactional system
Transactional Systems
Architecture Options
Data Warehouse
• Separate tools for real-time and
warehouse reporting
Actually connecting to the systems
• ETL Tools
• Native Tools
• Third Party Tools
• Direct Database connections
• ODBC/Queries
• Web Services
• InBound or Outbound
• Master Data Management tools
Data SystemsREAL-TIME CONNECTIVITY OPTIONS
Systems?
• ERP/SIS
• Ellucian Banner
• Oracle/PeopleSoft Campus Solutions
• Ellucian Colleague
• Jenzabar (EX, CX)
• Campus Vue
• Ellucian Power Campus
• Homegrown
• Others…
• Any Plans for Workday?
Ellucian Banner
Mobile
Portal
Reporting
Integration
Modules:
Languages/Tech:
Database/OS:
Hosting or SaaS?
• Student• Finance• HR
• Anything that compiles in Oracle• No specific toolkit• New UI being developed in Groovy
• Oracle• Unix
Hosting available
• Banner Mobile Connect
• Banner delivered a customizable package
• Luminus based on UPortal
Banner ODS, Banner EDW (Enterprise Data Warehouse), Partner with IBM Cognos and eVisions Argos for reporting tools
• Most integration is done through Oracle
• New Ellucian XE Framework for WS
• Alumni Development
Ellucian Colleague
Mobile
Portal
Reporting
Integration
Modules:
Languages/Tech:
Database/OS:
Hosting or SaaS?
• Student• Finance• HR
• Proprietary Envision toolkit—Colleague Studio
• Unidata, SQL Server, Oracle
• Unix , Linux,Windows
Hosting Available
• Partnership with DubLabs to implement MOX mobile app
• Provides portal support based on
SharePoint
• BusinessObjects• Bb Analytics for
warehouse• ODS
• Import/export batch files using ELF
• A few APIs through Sharepoint
• Advancement
PeopleSoft Campus Solutions
• No standard• May partner with
DubLabs AT&T offering
Mobile
Portal
Reporting
Integration
Modules:
Languages/Tech:
Database/OS:
Hosting or SaaS?
• Student• HR• Finance
No
• PeopleSoft Campus solutions warehouse
• BbAnalytics• Oracle Discoverer
• Integration Broker• SQR imports/exports• SAIP
• Alumni (Contributor Relations)
PeopleTools
• Oracle, SQL Server
• Unix, Linux, Windows
• Oracle Enterprise Portal
CampusVue Student
Mobile
Portal
Reporting
Integration
Modules:
Languages/Tech:
Database/OS:
Hosting or SaaS?
• Student• Finance• HR-use Talisma for fundraising
• Built on .net platform• CampusVue forms
• SQL Server• Windows
Yes
• Talisma CRM for sending notices and alerts to mobile devices
• SharePoint-CampusVueportal
• CampusVuePerformance Analytics for warehouse
• CampusVue Reporting Services with Crystal Reports
• SSRS
• CampusLink
Jenzabar EX
Mobile
Portal
Reporting
Integration
Modules:
Languages/Tech:
Database/OS:
Hosting or SaaS?
• Student• Finance• HR
• PowerBuilder• C++, C#
• SQL Server• WIndows
Yes
• JICS Go-browser based
• .net Portal-JICS
• Reporting through SSRS
• Provides data access using SSIS
• Alumni Development• PowerFAIDS
Ellucian Power Campus
Mobile
Portal
Reporting
Integration
Modules:
Languages/Tech:
Database/OS:
Hosting or SaaS?
• Student• Finance-Microsoft
Dynamics• HR-Microsoft Dynamics
• Visual Basic• Visual Studio
• SQL Server• Windows
No
• No standard mobile option
• SharePoint
• Microsoft reporting tools
• Provides data access using SSIS
• Financial Aid-PowerFAIDS
• Advancement
Unit 4-Agresso
Mobile
Portal
Reporting
Integration
Modules:
Languages/Tech:
Database/OS:
Hosting or SaaS?
• Student• Finance• HR
• Web-based architecture using .net framework, supports XML-based data sharing
• SQL Server, Oracle
• Windows, Linux, Unix
Yes
• Agresso Reporting and Analytic tools
• Can be integrated with third-party systems at basic level, CSV, or XML
• Financial Aid-PowerFAIDS
• No standard portal offering
• Provide reporting apps
WorkDay
Mobile
Portal
Reporting
Integration
Modules:
Languages/Tech:
Database/OS:
Hosting or SaaS?
• HR• Finance
• Java-based• WorkDay business process
framework for workflow
• MySQL• Linux
Fully SaaS
• Built-in BI• WorkDay
Actionable Analytics
• WorkDayintegration
• Cloud to integrate other systems to WorkDay
• No standard portal offering
• Mobile apps for IPad, IPhone and Blackberry
Three Rivers CAMS
Mobile
Portal
Reporting
Integration
Modules:
Languages/Tech:
Database/OS:
Hosting or SaaS?
• Student• Finance-Microsoft Dynamics• HR-Microsoft Dynamics
• Built using Microsoft technology and tools
• SQL Server• Windows
Yes
• Provides the BYOR (Build Your Own Reports)
• CAMS Student Mobile Portal
• Microsoft technology
• No clear integration strategy
TopSchool
Mobile
Portal
Reporting
Integration
Modules:
Languages/Tech:
Database/OS:
Hosting or SaaS?
• Student• Financials
• Microsoft tools for development• .Net framework• SharePoint
• SQL Server• All browser-
based, possibly Internet Explorer
Yes
•Microsoft Reporting Services
• TopX set of web services that include APIs for customers to build their own integrations
• No standard mobile options
• SharePoint
Kuali
Mobile
Portal
Reporting
Integration
Modules:
Languages/Tech:
Database/OS:
Hosting or SaaS?
• Finance• Student• HR
• Java-based• Eclipse
Workbench
• Database-independent, primarily MySQL
• Linux, Unix, Windows
Yes
• Cognos
• Kuali Rice-middleware component
• Google Web toolkit
• UPortal for UI
• KualiMobiliityEnterprise
• No standard offering currently
• Will be based on Uportal or Google widget
Jenzabar JX
Mobile
Portal
Reporting
Integration
Modules:
Languages/Tech:
Database/OS:
Hosting or SaaS?
• FrontRunner Advancement
• Google Widget Toolkit• Java
• Database independent
• OS dependent on database option
Not yet
• GWT optimized for different browsers
• Cognos
Client Counts (SIS)
System Total FTE Total Campuses
Banner 7M 1444
Colleague 3.2M 792
PeopleSoft 4.5M 875
CampusVue 276K 473
Jenzabar EX 352K 258
Jenzabar CX 281K 125
PowerCAMPUS 165K 116
Agresso 7K 4
WorkDay - -
Three Rivers 132K 89
Top School 2K 3
Kuali 41K 5
Jenzabar JX - -
SunGard Matrix 113K 120
GETTING PEOPLE TO USE AND TRUST THE DATA
So?
Simple illustration of Reporting Process
Data Request and
Requirements Gathering
Report Development
Report Consumption
Simple illustration of Reporting Process
Data Request and
Requirements Gathering
Report Development
Report Consumption
What happens here?
The Reaction…
Where the hell did these numbers come
from?
Root causes of lack of trust…
• Inconsistent numbers – “Dueling Data”
• Lack of understanding of data definitions
• Lack of involvement in decision making
• Previous failed projects
• Perceived (or real) data quality issues
• No transparency in to the data rules
• Personal conflict
9 Steps for Building Trust and Adoption
1. Ask more questions
• Data requests should be a conversation
2. Place importance on the purpose of each
report. Ask “Why” not “What”
3. Train in the Data more than the Tool
9 Steps for Building Trust and Adoption
4. Start documenting your own reports
• Be very specific in defining reporting terms
• Document for the functional and technical user
5. Create an institutional knowledge base
• Knowledge is developed one question at a time.
• It is happening now at your institution.
• Are you capturing it?
6. Establish and empower Data Stewards
9 Steps for Building Trust and Adoption
7. Create a report request/change process
with a standard for report specifications
• Include the data stewards in this process
8. Make your knowledge base easily shared
and accessible: “Liberate the knowledge”
9. Link your delivered reports to the
knowledge base.
HOW IDATA CAN HELP YOU ON THE PATH.
Who?
Web services integration platform for building connectors to higher education ERP systems
Collaborative data dictionary and data management tool for higher education
IData Products
The Data Cookbook
What is the Data Cookbook?
DataCookbook.com is an online and
collaborative data management tool
for higher education
What is the Data Cookbook?
It combines a good reporting process with
a governed knowledge base
What is the Data Cookbook?
Features include:
• Data Dictionary
• Report Specifications/Documentation
• Report Request/Change Workflow
• Data Governance Workflow
• Community for collaboration with other
institutions, vendors, and agencies
Data Cookbook Features
The IDataHub
ERP Integration – an illustration
Current Institution Architecture With the IDataHub
School Servers
High-Level Architecture
Web Servicesor Batch
IDataHubERP
Connectors
Data Providers and Consumers• Third Party Software• External Agencies• Mobile devices• Other external systems
Transaction Web servicesand/or Batch Files
Institution Network
Administrative Systems • ERP (Datatel, Banner, etc.)• LMS (Bb, Moodle, etc.)• Other internal Systems
Custom API
Standard API
InstitutionFirewall
• Procedures/Tasks• Scheduling• Logging• System Connections
Admin System Connectors
Application Database
IDataHub Institution Admin Site
• Administer transactions• View logs• View/Edit configuration • Create/Edit Procedures & tasks
“The Cloud”Admin web services
Service Oriented Front-End Architecture (SOFEA)
Admin UI
IData Services
IData Services
• Reporting and Data Governance
• Data management assessments and workshops
• Building data governance organizational structures
• Implementing data management and reporting processes
• Reporting and data warehouse development (agile iterations and
project-based)
• Report migrations (moving to new reporting systems)
• Requirements and design for reporting and data analysis projects
• Implementation and training for the Data Cookbook (IData’s data
management tool)
IData Services
• System Integration Services
• Develop custom web services
• Streamline batch processing
• Automate and monitor data integrations
• Standardize system integration API’s
• Agile Technical Staffing
• Developer/Analysts
• Project Managers
• Strategic Consulting
Find Out More…
www.idatainc.com
www.datacookbook.com
Brian Parish
bparish@idatainc.com
Wrap Up
Revisiting the Presentation Goals
Final Thought
Understand how to access real-time data
when you need it.
However…
Real-time data is only one potential method
to a useful answer.
Discussion
Thank YouVISIT WWW.IDATAINC.COM OR WWW.DATACOOKBOOK.COM
FOR MORE INFORMATION