Itaipu A Business Activity Monitoring system designed for business users Prof. Jacob Slonim June 27,...

ItaipuA Business Activity Monitoring system designed for business users

Prof. Jacob Slonim June 27, 2009

Student: Azza Abouzied, MCSc

Motivation Related work Itaipu Usability Future workStream Cubes

Largest hydroelectric dam in the world, provides 20% of Brazil’s and 94% of Paraguay’s power

Outline

• Motivation

• Related Work

• Research Questions

• Our vision of BAM: Itaipu

• Multi-dimensional analysis

using Stream Cubes

• Usability

• Future Work

• Provide just-in-time analysis of business events– Business Intelligence (BI)

• Business Objects– Initiated research– Different approaches

Research Goal

• Enterprise’s Global Responsiveness and Agility

• Wal-Mart sales employees– Oil prices increase– Weather predictions– Canadian tire discounts garden products– Will we make the quarter?

Motivation

Business Activity Monitoring (BAM)

• Continuous queries over data streams– Continuous query

• has implications on multiple query optimization– Infinite, unbounded, data streams

• Single-pass processing, processing bounds, or approximation techniques required

• Ad-hoc queries

• Dynamic environment

• Target business end-users NOT technical analysts

• Complex Event Processing Systems– Cayuga– Origin: Business Process Management– Business Objects Approach

• Data Stream Management Systems– STREAM– Aurora, Borealis– TelegraphCQ– Origin: Monitoring sensor networks

Related Work

Research Questions1. Could we reuse DSMS for a BAM system?

2. Could we provide multi-dimensional analysis of data streams?

3. Could we simplify and expedite query write-up for business users?

Largest hydroelectric dam in the world, provides 20% of Brazil’s and 94% of Paraguay’s power

Itaipu• Spring

– Data Source

• Dam– Query processing

• Delta– Results display and

storage

Itaipu’s big picture

• Brief design overview of Itaipu

• Two more questions left:

– Multidimensional data analysis of stream data

– Enhancing user-experience or usability

Recap & What’s Next?

city, sub_category

city, categorycountry, sub_category

country, category

country, allall, category

all, all

city, allall, sub-category

Cuboid

Base cuboid

Apex cuboid

Cube: a multi-dimensional data model

• Dimensions– Region: (city < country

< all)– Product: (sub-category

< category < all)

• Measures– Sum, count,

average

• SizeProduct

dimensionRegion dimension

Stream Cubes

• Han et. al introduced stream cubes

• Minimally-interesting (m-layer) and observation layers (o-layer)

• Partial Materialization– Popular drilling path

Conceptual representation of a tilted time frame

Stream Cubes

• Tilted Time frame– Time dimension– Distant aggregates

provided at larger granularities in comparison to more recent ones

Difference in perspective

• Popular drilling path– Users interested in o-layer and occasionally drill down– Path could be selected by an expert– Users drilling path is not likely to change

• Dynamic materialization path– User requests are distributed at different abstraction

layers– User requests are likely to change or evolve over the

course of the cube’s lifetime in the system

city, sub_category

country, category

country, allall, category

all, all

city, allall, sub-category

Drivers for change• External factors

– Oil prices increase– Export and oil-based products

affected– Shift towards viewing more

product information, less regional detail

• Competition• Seasonal• Market or economic

Dynamic Materialization Path

• Simple and efficient approach– Linear cost-effectiveness function– Dynamic programming approach

• Cost-effectiveness function– Memory cost– Popularity effect– Update potential

• Adapts to different operating environments– Weighted cost-effectiveness =- α(Memory cost) + β(Popularity) + γ(update potential)

Update potential?

α = 0.5,

β = 1.0 ,

γ = 0.75

Simulated example

city, sub_category

city, allcountry, category

country, all

all, all

all, category

all, sub_category

Resetting Path

Resetting paths on demand

Benefit of dynamic path materialization?

• Flexible– Follows user requests

• Low reliance on IT staff– Suitable for just-in-time operation

• How does it affect data stream processing?– Queue Size measure

• How does it affect memory consumption– Memory usage

Data rate ≈ 12,000

Experimental Setup

Results – Queue size

Results – Memory consumption

So, is it better?

• Conclusion– Situation dependent: more flexible solution– Not suitable when fixed memory guarantees are

required

• Are their better dynamic path materialization algorithms?– Probably– Our aim: simple, efficient, something that works!– Future improvements?

Recap & What’s Next?

• Brief design overview of Itaipu

• Optimization work with stream cubes

• Last question:– Enhancing user-experience or usability for Itaipu’s

business end-users

BAM user experience

• Bottleneck to the adoption of BAM– Operational decision making requires query

enabling technology

– The entire organization utilizes BAM: training effort needs to be minimal

A usability framework

Jarke’s & Vassiliou’s model:• Training Effort

• Repeated Efforts– Thinking

• Self-construction• Assisted-construction

– Input– Correction

Lessons Learned from database usability

• The simpler the task, the less query formulation effort is required.– Corollary: Limiting the functionality of a query

interface, makes it easier to use– Query by example– Query by forms

• Collaboration reduces the training effort– C-TORI– SearchTogether

Redrawn from a table in Jarke et Vassiliou, 1985

BAM’s user profile

Our usability solution

• Query Frames– Created by data analysts– Simplify query formulation to completing a query outline– Transformed easily to a form interface

• Query search– Each query or query frame is associated with textual

descriptions– Users and analysts could share queries and query frames– Passive collaboration– Keyword-based Google-like search interface

Query Frames

Dimension Query Frame

Itaipu’s Search Interface

Analytical Evaluation

Goal Query Frames Search

Training Thinking

Self construction Assisted construction

Input Correction

Hypothesis Result

A DSMS could be used as the core of a BAM system Dynamic path materialization is better suited for the data stream model Query entry could be simplified with the help of query frames and query sharing ≈

Final Recap

• Further studies to validate some our hypotheses

• A vision for BI

What’s in the future?

Conclusion

• Business Objects was convinced with our solution:– Itaipu’s architecture– Stream Cube extension– Query frame and query search

Thank for the support of:

NSERC, Business Objects,

Business Intelligent

Network

Questions ?

Spring

• Framework, based on data sources,– define

• Properties (allows drop)

• Data rate• Push/pull

– create:• Data Transformer• Data Streamer

• Data specification

Dam: Execution Engine

• Execution Engine– Variant of pipe &

filter architecture

– Similar to Fjords model

– Works like an Operating system

• Thread Manager• Queue Manager• Operator Builder

Execution Engine: Queue Manager

Execution Engine: Thread Manager

Execution Engine: Operator Builder

• XML multiple-query plan– Decouple query planning/optimizing from execution engine

• Java to XML Binding (JAXB)– Builds a content object model for each XML element

representing an operator

• Framework for building operators– Hierarchy of builders (Operator -> Aggregators -> SUM)– Update existing network of operators

Dam: Query Processing Unit

• Query Language– SQL-like with windowing constructs– Industry standard

• Current implementation– Extremely naïve– Open source tools

• End-user interface– Displaying and storing results– Searching for queries in the query catalog or inserting

new queries using SQL or modifying query frames

• Communicates with execution engine – Delta is thick client, execution engine is the server– XML messages over a TCP channel

• Memory cost

Example c1 = country, c2 = product category, |c1| = 5,

|c2| = 10, then |(c1, c2)| = 5 * 10 = 50

A good cuboid

• Popularity

• Update Potential

Cost-effectiveness function

• Cuboid

• Path

Itaipu A Business Activity Monitoring system designed for business users Prof. Jacob Slonim June 27,...

Documents

Transcript of Itaipu A Business Activity Monitoring system designed for business users Prof. Jacob Slonim June 27,...

MARINE CORPS SYSTEMS COMMAND (MCSC) AND ......MARINE CORPS SYSTEMS COMMAND (MCSC) AND PROGRAM EXECUTIVE OFFICER LAND SYSTEMS (PEO LS) 2012 ADVANCED PLANNING BRIEFING TO INDUSTRY (APBI)

Hidroelectrica de Itaipu Diapo Final

NATURAL RESOURCES IN SLONIM REGION

Krzysztof Guzek GCB Presentation Itaipu Event Eng_reduzida_cd

Itaipu Reservoir limnology: eutrophication degree and the ... · Itaipu Reservoir limnology 891 2.3. Trophic state index (TSI) In order to evaluate the trophic state of the Itaipu

Sustainability of Hydropower Global Challenges and a Global …...Aug-15 Itaipu Itaipu Binacional Brazil / Paraguay 14,000 MW Operation Feb-16 Mangdechhu Mangdechhu Project Authority

web.mit.eduweb.mit.edu/skd/www/itaipu-web.pdf · established ITAIPU Binacional provided continuity to the work conducted by the ñ-ažyan-arazilian Joint Technical Commission, by

Member and Chapter Services Newsletter March 2020mideasternapco.org/wp-content/uploads/2020/03/MCSC... · MCSC - The Member & Chapter Services Committee (MCSC) supports the membership

Itaipu Binacional’s 2006 Sustainability Report Report_2006.pdfThe Relatório de Sustentabilidade 2006 da Itaipu (Itaipu’s 2006 Sustainability Report), as the document published

Prize-Linked Savings Accounts and Moral Licensing K. Atalay, F. Bakhtiar, S. Cheung, R. Slonim; & K. Atalay, E. Garbarino, S. Guney and R. Slonim Workshop.

CBIRF initiated, TSWG & MCSC Supported Enhanced Personal ...

REPRESA HIDROELECTRICA ITAIPU

PERSONALISING HISTORY Sonia Slonim Leibler Yavneh College

Engineering studies for Itaipu convertor station design · Engineering studies for Itaipu convertor station design ... by Brazil and Paraguay on the Parana River, ... transformer

MCSC Newsletter 2020-06

web.mit.eduweb.mit.edu/skd/Desktop/skd/www/itaipu-web.pdfMarketing the Power From Itaipu The Treaty and terms for Power supply Administration and operation of agreements Bookkeeping

ITAIPU TECHNOLOGY PARK - PTI

Goals and Challenges of Modernizing ITAIPU PI System ... · Goals and Challenges of Modernizing ITAIPU PI System Infrastructure Roger Daniel F. Ferreira ... Project PI-SIRI Latest

ABOUT MCSC

Cop Itaipu 2014