Monetization and Optimization Index (MOI) - Global Smart Home Forecast
Processing and Optimization of Forecast Queries
description
Transcript of Processing and Optimization of Forecast Queries
© Prof. Dr.-Ing. Wolfgang Lehner |
Processing and Optimization of Forecast Queries
Ulrike Fischer
© Ulrike Fischer | | 2
> Motivation
Time series data appears in many domains
Processing and Optimization of Forecast Queries
High accuracy possible Sophisticated models Sophisticated estimators
Runtime restrictions Large number of time series Short amount of time available
Two Optimization Dimensions: Accuracy and Runtime
Renewable energy ressourcesSales and inventory
© Ulrike Fischer | | 3
> Outline
Motivation
Integration of Forecasting inside a DBMS
Processing of Forecast Queries
Optimization of Forecast Queries in Hierarchies
Summary
Processing and Optimization of Forecast Queries
© Ulrike Fischer | | 4
>
1. Model Creation Model Identification Parameter Estimation
Model-based Time Series Forecasting
3. Model Maintenance Model Evaluation
Threshold-based, time-based … Model Adaption
Parameter Re-estimation
Processing and Optimization of Forecast Queries
!
!
2. Model Usage
Forecasting Model Triple Exponential Smoothing
© Ulrike Fischer | | 5
> Time Series Forecasting in DBMS
Processing and Optimization of Forecast Queries
exportexportM
M
MM
M
SQL
SQL Reuse of models and
results
Transparency and Effienciency
© Ulrike Fischer | | 6
> Project Overview
Processing and Optimization of Forecast Queries
FlexOffers
Scheduling
Forecasting Aggregation
Supply Demand
SELECT date, quantityFROM salesWHERE … FORECAST …
DWH
date quantity2012 34,0002013 38,000
… …
EU FP7 project
© Ulrike Fischer | | 7
> Overview F2DB
Processing and Optimization of Forecast Queries
Model Index
Base Tables
Model Pool
Time Series Time Series Time Series
ModelModel
ModelModel
Query Interface
Forecast Queries Inserts
Query Processing & Optimization
QP in Hierarchies
Publish Subscribe Queries
Model Usage
Hybrid Maintenance
On-Demand Estimation
Model Maintenance
Ensemble Models
Physical Design
Model Creation
© Ulrike Fischer | | 8
> Outline
Motivation
Integration of Forecasting inside a DBMS
Processing of Forecast Queries
Optimization of Forecast Queries in Hierarchies
Summary
Processing and Optimization of Forecast Queries
© Ulrike Fischer | | 9
> Forecast Query Processing
Extension of SQL language Horizon, measure and time column,
model type and parameters, …
Logical query plan Forecast operator Ψ
Processing and Optimization of Forecast Queries
SELECT date, SUM(quantity)FROM salesWHERE product = ‘HTC‘GROUP BY dateFORECAST 3
Physical query plan
σ product= 'HTC'
sales
Ψk=3
πdate, quantity
γdate:AGG(sales)
Scan
Aggregate
BuildModel
Forecast
sales
Forecast
MHTC
© Ulrike Fischer | | 10
> Advanced Forecast Query Processing
Data warehouse contains multidimensional data
Processing and Optimization of Forecast Queries
HTC
HD2
Mobiles
Smart
Nokia
2. Aggregation
3. Disaggregation
Forecast
MMobiles
DisAgg
KeyForecast
MHD2
Forecast
MSmart
Aggregation
SELECT date, SUM(quantity)FROM salesWHERE product = ‘HTC‘GROUP BY dateFORECAST 3 days
1. Direct
© Ulrike Fischer | | 11
> Aggregation vs. Disaggregation
Processing and Optimization of Forecast Queries
Complete(Direct)
Bottom-Up(Aggregation)
Top-Down(Disaggregation)
Efficiency
Accuracy No information lossModel creation easier
Grunfeld and Griliches (1960)Gross and Sohl (1990)
Zellner and Tobias (2000)….
Edwards and Orcuss (1969)Schwarzkopf et. al. (1988)
Hubrich (2005)…
Depends on data set, quality of forecast model, correlation …
© Ulrike Fischer | | 12
> Outline
Motivation
Integration of Forecasting inside a DBMS
Processing of Forecast Queries
Optimization of Forecast Queries in Hierarchies
Summary
Processing and Optimization of Forecast Queries
© Ulrike Fischer | | 13
> Configuration Advisor
Processing and Optimization of Forecast Queries
Problem: Exponential search space Greedy Algorithm (monotonic maintenance costs)
Start one model at the top, add models step-by-step
DWH Model Pool
Query Interface
Updates Forecast Queries
Model Advisor
Workload W Preference α
Analyze
Cost BW + Error EW
Configuration + Strategy
CreateConfiguration CW
© Ulrike Fischer | | 14
> Performance Comparison
Processing and Optimization of Forecast Queries
Complete(C) All models, only direct forecasts Bottom-Up (B) Only models at level one, others use aggregation Top-Down (T) Only one model for top element, others use
disaggregation Greedy (G)
© Ulrike Fischer | | 15
> Extensions
Observation: aggregation (bottom-up) hardly used in real data sets Reason: large number of child time series
Processing and Optimization of Forecast Queries
? ?VirtualGroup
Sample Aggregation Use sample of child models
aggregation + estimation
Group Design Relax fixed aggregation groups
?
Estimate using historical proportion Weighted sampling
support of disjunctive queries
© Ulrike Fischer | | 16
> Outline
Motivation
Integration of Forecasting inside a DBMS
Processing of Forecast Queries
Optimization of Forecast Queries in Hierarchies
Summary
Processing and Optimization of Forecast Queries
© Ulrike Fischer | | 17
> Summary
DBMS Integration Sophisticated models computationally expensive DBMS integration for reuse, transparency and optimization
Forecast Queries New query type with forecast horizon Face two otimization dimensions
Hierarchical Forecasting Reduce maintenance costs with derivation schemes Possible increase of accuracy Large search space
Processing and Optimization of Forecast Queries
© Prof. Dr.-Ing. Wolfgang Lehner |
Processing and Optimization of Forecast Queries
Ulrike Fischer