Data Economy for Prosumers in a Smart Grid Ecosystem · Smart grids, demand response, energy...

Data Economy for Prosumers in a Smart Grid EcosystemRicardo J. Bessa

INESC TECPorto, Portugal

[email protected]

David RuaINESC TEC

Porto, [email protected]

Cláudia AbreuINESC TEC and FEUP


Paulo MachadoINESC TEC


José R. AndradeINESC TEC


Rui PintoINESC TEC and FEUP


Carla GonçalvesINESC TEC and FCUP


Marisa ReisINESC TEC and FEUP


ABSTRACTSmart grids technologies are enablers of new business models fordomestic consumers with local flexibility (generation, loads, stor-age), where access to data is a key requirement in the value stream.However, legislation on personal data privacy and protection im-poses the need to develop local models for flexibility modeling andforecasting, to exchange models and not personal data. This paperdescribes the functional and hardware architecture of an home en-ergy management system (HEMS) and its optimization functions. Aset of data-driven models, embedded in the HEMS, are discussed forimproving renewable energy forecasting accuracy and modelingmulti-period flexibility of distributed energy resources.

CCS CONCEPTS• Computing methodologies → Learning paradigms; • Ap-plied computing→ Forecasting; • Hardware→ Smart grid;

KEYWORDSSmart grids, demand response, energy management, flexibility, dataanalytics

ACM Reference Format:Ricardo J. Bessa, David Rua, Cláudia Abreu, Paulo Machado, José R. Andrade,Rui Pinto, Carla Gonçalves, and Marisa Reis. 2018. Data Economy for Pro-sumers in a Smart Grid Ecosystem. In Proceedings of International Workshopon Energy Data and Analytics (e-Energy Workshop 2018). ACM, New York,NY, USA, Article 4, 8 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from [email protected] Workshop 2018, June 2018, Karlsruhe, Germany© 2018 Association for Computing Machinery.ACM ISBN 978-x-xxxx-xxxx-x/YY/MM. . . $15.00https://doi.org/10.1145/nnnnnnn.nnnnnnn

1 INTRODUCTIONThe deployment of smartmeters at the domestic consumer/prosumerlevel is enabling emergent regulated and non-regulated data-basedservices that can help to boost distributed energy resources (DER)integration and promote customer empowerment [21].

At the regulated level, the ongoing discussion about the futureroles of Distribution SystemOperators (DSO) is focused in smartme-tering data management (data manager role) and electricity marketfacilitation (neutral market facilitator role) [18, 23]. In this context,the following paragraphs present examples of recent initiativeswithin the smart grid ecosystem.

The European Union (EU) project UPGRID developed and demon-strated a Neutral Market Access Platform (NMAP) and a RetailerPlatform (RP). The NMAP is hosted by a DSO and encompasses theexchange of information, including consumption profiles from theDSO and flexibility profiles from the Home Energy ManagementSystem (HEMS) [3]. The RP is responsible for receiving informa-tion or requests from the UPGRID platform, process and send thisinformation to its HEMS. The EU project FLEXICIENCY developedand demonstrated pan-European Market Place that aims at deliver-ing services and exchange of data, tools, methodologies, in a stan-dardized way across Europe [26]. The platform receives/submitsdata/service request and readdresses requests to the platforms (e.g.,DSO platforms, service platforms) where the data, services, softwareand tools are located [6]. Green Button (U.S.A.) is an industry-ledwork that provides a common format for electrical energy meter-ing data so that electricity consumers can access their data in aneasily readable and secure format via a “Green Button” on theirelectric utilities’ website. Once customers access their data, theycan share it as they choose, by independent choice and action, withthose they trust. Services from third parties are also emerging withthis initiative, e.g. a solar developer could use customer-meteringdata to determine optimal system size with a more accurate cost-benefit analysis [12]. A review of from Council of European EnergyRegulators (CEER) of eight European countries in terms of datamanagement models can be found in [23].

In terms of non-regulated services or third-party services, severalworks in the literature describe data-driven services and business

https://doi.org/10.1145/nnnnnnn.nnnnnnn

https://doi.org/10.1145/nnnnnnn.nnnnnnn

e-Energy Workshop 2018, June 2018, Karlsruhe, Germany R.J. Bessa et al.

cases to boost demand response potential and promote energy effi-ciency. Smart meter data can be used by a third-party (e.g., energyretailer, energy services company) for segmentation of customersand identification of temporal consumption patterns [10], predict-ing customer response and reliability in response to price signals[17], estimating the price elasticity of customers [13] and deriveoptimal bidding strategies (bidding curves) under dynamic electric-ity tariffs [25]. In fact, total energy consumption at the residentiallevel is enough to derive a ranking of thermal-load flexibility andsub-metering is not required [2].

Smart meter and HEMS data can also be used to induce behav-ioral changes in energy consumption [28], for instance throughgamification techniques [14] or broadcast of information (e.g., pricesignal) [11]. In [22], a detailed analysis of the requirements fordifferent applications of smart meter data (e.g., balancing, demandresponse, network planning) is conducted considering the Europeanlegislation about personal data protection.

The General Data Protection Regulation (GDPR) approved bythe EU Parliament on 14 April 2016, harmonizes data privacy lawsacross Europe, protect and empower all EU citizens data privacy[29]. The new regulation introduces a new fine (i.e., up to 4% ofannual global turnover or 20 Me), intelligible and easily accessibleform for data access consent, right to access data and be forgotten(erase personal data and cease further dissemination of the data).Moreover, it requires “privacy by design”, which means inclusion ofdata protection from the start in systems designing, rather than anaddition. Therefore, in order to build data economy or data market-places focused electrical energy consumption data, it is necessaryto design knowledge extraction methods that ensure data privacyfrom the beginning.

Presently, some researchers are starting to design demand re-sponse algorithms that avoid sharing and transfer of personal data.For instance, data-driven pricing schemes for load shifting withoutaccess to personal load requirements [31] or information exchangemodels where consumers keep their load levels private and partici-pate in a real-time price scheme [11].

This paper presents the Horizon 2020 InteGrid project’s visionfor the HEMS and its embedded intelligent functions. The core goalis to design forecasting algorithms (load, solar, flexibility) keepingdata local and private and at the same time create economic valuefor stakeholders like retailers and aggregators. The paper starts bydescribing the functional architecture and hardware/software inte-gration of the HEMS (section 2), proposes distributed forecastingservices (section 3) and a flexibility forecast/modelling representa-tion (section 4). The potential for future work is discussed in theend (section 5).

2 HOME ENERGY MANAGEMENT SYSTEMDavid, Cláudia, Paulo The concept of Home Energy ManagementSystem (HEMS) was initially introduced as a central unit locatedwithin a domestic building with the capability of performing anoptimized control of energy resources behind the meter. This op-timization was loosely defined as a cost reduction procedure, typ-ically achieved through a lower cost use of energy consideringinternal factors (e.g., devices and system that are able to provide

energy use flexibility) and external factors (e.g. weather conditions,discriminated energy prices).

The main characteristic of current HEMS implementations is theability to allow the monitoring of energy consumption by meansof existing metering devices and a user interface that allows therepresentation of data in a user-oriented way (e.g. dashboards).Another characteristic is the ability to automate existing resourcesand by using very simple control mechanisms allow the operationof devices and systems at more convenient hours (e.g. pre-programload activation to hours in which the energy costs are lower). Theuse of sensor systems to allows the remote disconnection of devicesand system based on a threshold or an user-defined activations thatallows exploiting energy savings potential with the avoidance ofenergy waste. Optimization schemes are currently being sought toallow the optimization of existing flexible energy resources accord-ing to multiple criteria and multiple restrictions.

To enhance the capability of fully optimizing the energy usein households HEMS have been incorporating energy models ofdevices, systems and spaces so that their specific characteristicsalong with user preferences can in fact be properly considered whendefining their optimal activation. Users are thus able to insert con-figurations and preferences and take advantage of local flexibilityto activate their energy resource at an optimal time. This involvesa planning stage and a time-ahead operation, being an example thecase where the end-user sets the preferences and configurations,an optimal operation schedule is determined, and in the next daythe existing resources are activated according to the schedule.

2.1 Functional ArchitectureHEMS are currently flexible and modular HW and SW platforms,capable of supporting a wide variety of features and functionali-ties the allow end-users to take advantage of existing incentivesand define a customizable operation schedule. The functionalitiesof a HEMS are typically grouped into: user interface, energy op-timization and automation. User interface (UI) is responsible forpresenting data and relevant information to the user and allow theinsertion of configurations and preferences. The energy optimiza-tion creates an energy representation of the household, throughthe parameterization of the energy models of devices and systemsaccording to their configuration, and according to the selected cri-teria establish a time-ahead operation schedule. The automationtakes care of the communication between the HEMS central systemand the associated devices and systems to retrieve monitoring andoperation state data and to set remote configurations or actionsthat allow an optimal schedule to be implemented.

Other groups can be defined ...

2.2 Hardware Integration2.3 Optimization StrategiesOne of the distinguishable feature that a HEMS must provide isthe capability of computing optimal (or suboptimal) energy useschedules that provide added-value to the end-users. There aredifferent optimization strategies that can be exploited through aHEMS and they largely depend on the criterion (or set of criteria)that might be established. Typically, a cost reduction benefit issough, and that can be achieved either through energy consumption

Data Economy for Prosumers in a Smart Grid Ecosystem e-Energy Workshop 2018, June 2018, Karlsruhe, Germany

reduction or through an optimal load allocation considering energyconsumption prices. While the former is associated to behavioralchanges the latter is more related to the technical aspects of theenergy management. In [1] a HEMS implementation, based onthe work carried out in the AnyPLACE project, is presented, withthe mathematical formulation of optimization strategies for costreduction considering the variable and fixed components of theenergy consumption costs.

In

0,10

0,11

0,12

0,13

0,0

0,5

1,0

1,5

2,0

2,5

3,0

3,5

4,0

4,5

5,0

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Pri

ces(€/k

Wh

)

Pow

er (

kW)

Hours

Total AC Other loads EWH REF CD DW WM PV €/kWh

Figure 1: Models hub concept for renewable energy forecast-ing.

3 FORECASTING SERVICES3.1 ConceptThe works in [5, 9, 27] showed that geographically distributed timeseries data can improve the renewable energy forecasting skill upto six lead-times ahead. Time series data from different sources orowners are combined in a vector autoregression (VAR) model andthe LASSO penalty structure is used to uncover sparse structuresin the model’s coefficients matrix. The VAR process captures linearinterdependencies amongmultiple time series, enabling each HEMSto model its PV time series evolution based not only on its ownlagged values but also in the lagged values collected by other HEMS.Themain challenge is on how to combine data frommultiple ownersandmaintain data privacy, which is analyzed from themathematicalpoint of view in subsection 3.3.

Furthermore, information from a spatial grid of NumericalWeatherPredictions (NWP) is valuable to improve renewable energy fore-cast accuracy for multiple days [4]. Here, the challenge is to processthe NWP grid data and run the statistical forecasting algorithmslocally at the HEMS level. Subsection 3.2 presents computationalresults of forecasting algorithms running in the HEMS described insection 2.

Figure 2 illustrates the models hub concept that serves two goals:(a) central node in a collaborative forecasting scheme (i.e. VARmodel) where each HEMS exchanges the matrices from the B andH-update steps of the Alternating Direction Method of Multipliers(ADMM) method [7]; (b) apply feature engineering techniques thatextract information from the NWP grid and send the post-processedvariables to the HEMS where a embedded gradient boosting treealgorithm is used to produce solar power forecasts.

Features

extracted

from NWP

grid

[B,H]

data

remains

local

HEMS

HEMS

HEMS

HEMS

HEMS

HEMS

HEMS

HEMS

Models Hub

Figure 2: Models hub concept for renewable energy forecast-ing.

3.2 Embedded ForecastingAndrade and Bessa in [4] presented a novel methodology that com-bines the gradient boosting trees (GBT) algorithm with featureengineering techniques capable of extracting information from aspatial grid of NWP and resuming it in a new smaller subset ofvariables.

Besides advantages regarding the dimensionality reduction ofdatasets to be fed into the forecasting algorithms, the introductionof this new spatial-temporal features on the forecasting models ledto substantial improvements on point and probabilistic forecastsfor short-term horizons (i.e., 1 to 72 hours ahead).

However, the high quantity of information to process representsa major constraint and makes it prohibitive to be run in smallcomputational units such as local HEMS (Raspberry Pi 3 Model B).Table 1 presents a comparison between the technical specificationsof the HEMS and a conventional desktop computer.

Table 1: Computational resources comparison between theHEMS (Raspberry Pi) and a conventional desktop computer.

Component Raspberry Pi 3 Desktop

CPU

Broadcom BCM2837 Intel(R) Core(TM)ARM Cortex-A53, 1.2GHz i7-6700, 3.4GHz

4 cores 4 cores4 threads 8 threads

RAM1GB LPDDR2 12 GB DDR4(900 MHz) (2133 MHz)

Op. System Linux-Raspbian Windows 10

Table 1 reveals that the RAM size and processing power of theRaspberry Pi are a clear limitation when trying to process largequantities of NWP data in an acceptable computational time. Forthis reason, the envisioned solution uses a centralized distributedcomputing framework, nested in the Models Hub platform, to pro-cess the raw NWP grid data and create relevant features representa-tive of the NWP grid information. These variables, combined with


NWP forecasts for the client location, feed a GBT model that runslocally in each HEMS.

For the HEMS, the challenge of embedded forecasting is summa-rized in four phases:

• Initial request to retrieve historical post-processed NWP gridvariables from the Models Hub platform, for the timespan ofhistorical observed PV data.

• Apply feature engineering techniques to extract temporalinformation from NWP variables for the HEMS location.

• Fit the GBT model using the historical of observed PV gen-eration values and respective NWP variables. The model isupdated in a daily or weekly basis.

• Generate solar power forecasts based on the NWP variablesfor an horizon up to 48 hours ahead and the post-processedoperational forecasts requested to the Models Hub platform.

3.2.1 Accuracy Results Analysis. In this subsection, improvements on the solar power forecastingskill resulting from the introduction of spatial-temporal featuresextracted from the NWP grid are demonstrated. The metrics MeanAbsolute Error (MAE) and Root Mean Squared Error (RMSE) areconsidered to assess the point forecast quality. The ContinuousRanked Probability Score (CRPS) is preferred to evaluate the qualityof probabilistic forecasts (quantiles between 5% and 95%, with a5% increment). An extended description of the feature-engineeringprocess can be found in [4].

The importance of variables extracted from the NWP grid is hereevaluated by comparing two models that contain extra temporaland spatial information with a base reference model exclusivelycomposed by seasonal and NWP variables for the grid central point.Table 2 describes the input information of the base model.

Table 2: Base model input variables.

Type Variables

Seasonal Month of the yearHour of the day

NWP

swflx [W/m2] - surface downwelling shortwave fluxtemp [K] - ambient temperature at 2 meterscfl [0,1] - cloud cover at low levelscfm [0,1] - cloud cover at mid levelscfh [0,1] - cloud cover at high levelscft [0,1] - cloud cover at low and mid levels

The following two models accommodate the new information.Model T is representative of the information that can be extractedfrom the NWP series for the HEMS location, and Model F comprisesa selection of the best spatial and temporal variables that were ableto maximize the forecast skill.

• Model T - Temporal information extracted from the NWPseries to the HEMS location:– Lags and Leads.– Temporal variance with centered windows of 3h, 7h, 11h.– Information from different NWP runs.

• Model F - Spatial information extracted from the NWP grid:– Hourly spatial standard deviation of NWP grid variables.

– Hourly spatial weighted average of NWP grid variables.– Principal components applied individually to the grid in-formation of each NWP variable mentioned in Table 2.

The performance of these models is evaluated over a timespanof two years (from May 1st 2015 to 28th June 2016) with a slid-ing window of 12 months. Considering ϵ as a metric score, andbase as a the base model, the improvement of a model is given by(1 − ϵm

ϵbase

)× 100 (%).

Figure 3 depicts the average improvements of every aforemen-tioned model over the reference base model for forecasts within anhorizon of 24 hours ahead. The evaluation is computed on out-of-sample datasets, guaranteeing that in each fold the training data isnot contaminated by test samples. The night periods were removedfor the PV generation.

� ��

��

�!

%�

� �

�!�� !��

'� � �� "#!&�� %�! ��$��!��

��

'� � ��

��

Figure 3: Monthly relative improvements of models T and Fover base model.

An analysis of the figure shows that the two new models greatlyoutperform the base model in every month of the evaluation period.By itself, the collection of temporal variables (Model T) alreadyprovides great average improvements of 13.73%, 10.35%, 11.33%on MAE, RMSE and CRPS that peak in November 2015 at 34.48%,24.67%, 23.32%. However, the maximum forecast skill is only ob-tained by including the spatial information (Model F), which ledto average improvements of 16.09%, 12.85%, 13.11% for the samemetrics, that peak at 37.75%, 28.43%, 27.97% in January 2015.

As final remark, it is important to underline that a good fore-cast quality can be achieved by solely depending on the temporalinformation extracted from the NWP runs for the HEMS location.However, to maximize point and specially probabilistic forecastsquality, the NWP grid information is necessary.


3.2.2 Computational Times Evaluation. Although the forecasting algorithms demonstrated to successfullyrun in the Raspberry Pi 3 Model B, a significant impact on thecomputational time of each model run is verified when comparedto a conventional desktop computer. It is important to underlinethat the GBT regression model requires a separated training foreach quantile of the probabilistic forecast. Table 3 shows the totalcomputational times necessary to train all the regression modelsand to compute forecasts for the two systems described in Table 1.

Table 3: Comparision of computational time results.

Device GBT Fitting Operational ForecastDesktop 42s 0.03s

Raspberry Pi 3 320s 0.49s

3.3 Collaborative Forecasting3.3.1 VAR-LASSO model and ADMM

. Here, a brief review of the VAR-LASSO model is presented, as wellas the ADMM formulation for distributed parameter estimation.

Letyi,t be the time series of an i–th HEMS, in time t , and {Yt } ={(y1,t , . . . ,yn,t )} a n-dimensional vector time series. Then, a VARmodel of order p describes the trajectory of Yt as

Yt = η +p∑ℓ=1

B(ℓ)Yt−l + εt , (1)

where (B(ℓ))i, j represents the parameters for time series i , associ-ated with lag ℓ of time series j; η = (µ1, . . . , µn )T is the vector ofconstant terms; and εt = (ε1,t , . . . , εn,t )T is a white noise term. Bysimplification, Yt is assumed to be a centered process, i.e., η = 0.

In order to formulate a matrix representation of VAR(p) model,let Y = (Y1, . . . ,YT ) ∈ R(n×T ) the response matrix; B = (B(1),. . . ,B(p)) ∈ R(n×np), the coefficient matrix; Z = (Z1, . . . ,ZT ) ∈R(np×T ) the explanatory variables matrix, with Zt = (Yt−1, . . . ,Yt−p ) ∈ R(n×T ); and E = (ε1, . . . , εT ) ∈ R(n×T ) the error matrix.Then,

Y = BZ + E. (2)Commonly, the system (2) is solved computing the coefficients

that minimize ∥Y − BZ∥22 , where ∥.∥r will represent both vectorand matrix Lr norms. However, high-dimensional data can intro-duce irrelevant or redundant information making convenient theapplication of the LASSO framework, which is a regularized versionof least squares that introduces an L1 penalty on the coefficients. Instandard VAR-LASSO approach, the coefficients are estimated by

argminB

( 12∥Y − BZ∥22 + λ∥B∥1

), (3)

where λ > 0 is a scalar penalty parameter.Since the cost function in (3) is non-differentiable, the ADMM

constitutes a powerful algorithm to solve this problem, making itpossible to perform a parallel optimization. Summarily, the ADMMrewrites the VAR-LASSO objective function (3) replicating the Bvariable using the H variable ( 12 ∥Y − BZ∥22 + λ∥H∥1) and addingan equality constraint imposing B = H. Then, based in the aug-mented Lagrangian of this reformulated minimization problem

B Z

Main formulation B1 . . . BN

Z1

. . .

ZN

Splitting across predictors – equation system (5)

B1. . .

BNZ1. . .ZN

Splitting across examples – equation system (6)

Figure 4: Two main scenarios for distributed computation.

(with penalty parameter ρ > 0), the ADMM formulation of (3)consists in the following iterations [9]:

Bk+1 := argminB(12 ∥Y − BZ∥22 +

ρ2 ∥B − Hk + Uk ∥22

)Hk+1 := argminH

(λ∥H∥1 + ρ

2 ∥Bk+1 − H + Uk ∥22)

Uk+1 := Uk + Bk+1 − Hk+1.

(4)

Given that ∥Y − BZ∥22 and ∥H∥1 are decomposable, the minimiza-tion problem over B and H can be separately solved for distributeddata. Therefore, the ADMM provides a desirable formulation forparallel computing.

Figure 4 illustrates the two most commonly used approaches tosplit the optimization problem, in which Z is partitioned into N rowblocks (splitting across predictors) or N column blocks (splittingacross examples). Conventionally, this two generic formulations arecalled Consensus Optimization and Sharing Optimization, respec-tively. The corresponding ADMM formulation using (4) is presentedin the systems of equations (5) and (6),

Bk+1i := argminBi(ρ2 ∥Bki Zi + H

k − BZk − Uk − BiZi ∥22+

λ ∥Bi ∥)

Hk+1 := 1

N+ρ

(Y + ρBZ

k+1+ ρUk

)Uk+1 := Uk + BZ

k+1 − Hk+1

(5)

Bk+1i := argminBi

(12 ∥Yi + BiZi ∥22 +

ρ2 ∥Bi − Hk + Uki ∥22

)Hk+1 := S1

(Bk+1+ U

k, λN ρ

)Uk+1i := Uki + B

k+1i − Hk+1

(6)

where BZk = 1N

∑Nj=1 B

kj Zj , B

k= 1

N∑Nj=1 B

kj and S1 is the scalar

soft thresholding operator, defined as S1(x ,a) = x|x | max{0, |x | −a}.

3.3.2 Can ADMM ensure data privacy in collaborative forecast-ing?The main challenge is to investigate the potential of these iter-ative systems of equations to develop a collaborative forecastingwith privacy preserving data. However, as will be concluded, thedirect application of these two alternatives cannot fulfill the privacyrequirement.


If the problem is divided across predictors, then Y =∑i BiZi and

each HEMS will estimate Bi using Zi , which is composed uniquelyby the lags of its own time series. Hence, each HEMS computes thecorresponding Bi without share data. The problem is the updateof H where each HEMS should share its BiZi + (0,Yi , 0), allowingthe more curious HEMS to recover Y.

On the other hand, if the problem is divided across examples, theneach Bi (computed in parallel) is estimated using Zi , which in thiscase consists of specific lags of all time series, meaning data is sharedbetween all participants. Nevertheless, this division by examplescan be interesting for single-output problems in the energy sector,where private data from multiple consumers is needed.

Notice that in both situations, the existence of a neutral elementmay be imposed, through which all HEMS communicate, i.e., as-suming a centralized process so that HEMS do not exchange resultsdirectly. However, this would imply that the neutral element canreconstruct the original data. Furthermore, if in system (5) the neu-tral element provides the i–th HEMS H

k ,BZk and Uk , then i–thHEMS can reconstruct Y.

Ideally, the algorithmwould be adapted in a way that nobody canreconstruct the original data, be they HEMS or the neutral element.Even better, the algorithm should be decentralized (i.e. peer-to-peer),asynchronous (to cope with delays or communication failures) andtime-adaptive (enabling the assimilation of new samples as theybecome available in order to improve the model with time, updatingthe coefficients of the model, without starting from scratch).

In the literature, related works may be found. a decentralizedstructure with asynchrony and delays was proposed in [30], inwhich the workers can communicate independently with theirneighbors, at different times and for different durations. In termsof online ADMM, a general approach may be explored in [16].Latter work proposed an asynchronous time-adaptive ADMM forconsensus optimization [19].

With respect to algorithms that take into account data privacy:the authors of [32] proposed a privacy-preserving decentralized(but neither asynchronous nor online) optimization method, intro-ducing time-varying penalty matrices on ADMMmethod and usingpartially homomorphic cryptography.

4 SURROGATE MODELS FOR ENERGYFLEXIBILITY

This section describes two surrogate representations of energyflexibility from behind-the-meter controllable appliances and small-scale storage. The main idea is to exchange flexibility models, in-stead of behind-the-meter data, which can be flexibility trajectories,machine learning based or virtual batteries.

It is important to stress that other representations are possible,such as simplified models to quantity thermal-based demand re-sponse potential and the HEMS only sends upstream the estimatedmodel and not the consumption data [2].

4.1 Flexibility TrajectoriesHEMS can include diverse equipment such as electric water heaters(EWH), domestic small-scale batteries, photovoltaic (PV) panels,and HVAC systems. The flexibility that HEMS can offer relates todeviations from the net-load profiles that were expected to occur,

and has its explanation based on the flexible nature of the previouslyenumerated HEMS equipment.

The concept of flexibility trajectory embraces the multi-periodHEMS flexibility potential instead of independent single-periodsformulation [24] . A feasible flexibility trajectory represents theHEMS potential of reshaping its expected net-load profile whileguaranteeing technical operation viability of all equipment andinternal constraints such as EWH water temperature, battery state-of-charge, or even room air-temperature. Additionally, user-definedconstraint regarding the use of the battery can be also implemented(e.g., aiming at minimizing the energy spilling during PV gener-ation surplus). The uncertainty in the net-load profile was alsomodeled, namely by considering different PV generation scenarios.Ultimately, a feasible trajectory must comply with the defined con-straints for a pre-establish percentage of PV generation scenarios(e.g. feasible in 90% of the PV scenarios).

Defining the feasible domain for the HEMS flexibility potentialis a complex task. An Evolutionary Particle Swarm Optimization(EPS) based algorithm was proposed in [24] to search and samplethe mentioned feasible domain. The final output is a set of feasibletrajectories that can be used to describe the HEMS multi-temporalflexibility potential. Figure 5 depicts a set of 20 feasible flexibilitytrajectories encompassing a period where PV generation surplusoccurs.

07:45 08:15 08:45 09:15 09:45 10:15 10:45 11:15 11:45 12:15 12:45Hour

2.0

1.5

1.0

0.5

0.0

0.5

1.0

1.5

2.0

Powe

r [M

W]

Feasible Flexibility Trajectories

Figure 5: Illustrative set of 20 EPSO-generated feasible flexi-bility trajectories.

4.2 Machine Learning RepresentationThe set of flexibility trajectories described in the previous sectioncan be learned (or presented) by a support vector data description(SVDD) algorithm [8, 24]. SVDD is a machine learning one-classsupport vector machine that can be used to classify “feasible” or“unfeasible” HEMS operating trajectories upon request from anoptimization or control algorithm.


In SVDD, the flexibility trajectories set (X ) is summarized witha combination of support vector (xi ) and respective coefficients(βi ). This describes a high-dimension sphere delimiting the feasibledomain. A new trajectory (x ) is classified by comparing the sphereradius with its radius calculated as follows:

R2(x) = 1 − 2∑i

β ik(x i,x) +∑i, j

β iβ jk(x i,x j) (7)

where R2 is the square of the radius being calculated, xi and x jare support vectors, k is the kernel function. For this problem, itwas found that sigmoid kernel is the most suitable type [24].

To be classified as feasible, a trajectory must present a radiuslower or equal to the sphere’s radius.

The results for the test case described in [24] are presented inTable 4.3.

The main limitation of this representation is that it only allowsto classify trajectories as feasible or unfeasible and Interpretability(in terms of energy flexibility) is low. However, Eq. 7 can be easilyintegrated in optimization or control problems, while maintainingdata privacy.

4.3 Virtual Battery RepresentationAn alternative to the SVDD representation for the set of flexibilitytrajectories by means of a virtual battery model (used in [15]). Thevirtual battery representation might be seen as a linear systemresembling the operation of a battery parameterized by powerlimits for charge (Ptmax) and discharge (Ptmin) cycles, by maximum(SOCtmax) and minimum (SOCtmin) limits for the state of charge(SOC), and also by the initial SOC level (SOCini).

The power and SOC limits modeled in this work are dynamicthroughout the time horizon that is being considered, instead ofusing a fixed value for all the periods considered. This way, thefeasible domain represented by the virtual battery will be moreadapted regarding the period-based information coming from theset of feasible flexibility trajectories.

The problem formulation for the virtual battery model aims atminimizing the battery size (SOC), as well as minimizing, for eachperiod, the power amplitude represented by the difference betweenthe charge and discharge limits. Eq. 8-10 summarize the problemformulation, where trajk, ti refers to the power value representedby the trajectory k from the flexibility set, for the time period ti;and T refers to the number of periods considered.

minT∑t=1

SOCtmax +

T∑t=1

SOCtmin +

T∑t=1

P tmax −

T∑t=1

P tmin (8)

SOCtimin <= SOC ini+

t i∑t=1

trajk, tt <= SOCtimax, ∀t i ∈ [1,T ] , ∀k

(9)

P timin <= trajk, ti <= P ti

max, ∀t i ∈ [1,T ] and ∀k (10)

Compared to the SVDD-based flexibility model, the virtual batterymodel can be easily integrated in a optimal power flow formulation,

Table 4: Classification performance: SVDD versus VirtualBattery.

Feasible trajectories Unfeasible trajectoriesSVDD 95 % 80 %

Virtual Battery 100 % 70 %

interpreted by an end-user in terms of available flexibility and it isscalable for prosumers aggregation algorithms.

Regarding the performance on correctly classifying trajectories,the virtual battery model shows a better performance comparedto the SVDD model when classifying feasible trajectories. On theother hand, for unfeasible trajectories the virtual battery presentsan higher number of wrongly classified trajectories. Table 4 presentsthe percentage of correctly classified trajectories for both modelson two different sets: feasible and unfeasible trajectories. Resultsrefer to an 8-hours time horizon.

5 CONCLUSIONSThe increasing awareness of prosumers about data protection andbusiness value of their smart meter data demands for a revision ofthe current data management paradigms and business use cases.

Forecasting algorithms of renewable energy and net-load benefitfrom exchange of information between peers (e.g., geographicallydistributed observations), which shows significant challenges inmaintaining data private. A recent trend for mobile devices is theFederated Learning concept1 that keeps all training data on thedevice and only the statistical or machine learning model is shared.Only the updated model is sent to the cloud, using encrypted com-munication. Furthermore, technologies like TensorFlow Lite formobile and embedded device developers will enable machine learn-ing tasks in data that does not leave the device and with faster localcomputation. The advantages are clear, no dependency on networkconnection and training with less data. However, collaborativeforecasting with data privacy remains an open area of research.

The aggregation of flexibility from prosumers also demands fora model-based approach with local computations to extract flexi-bility parameters. In this context, the interpretability of the virtualbattery model is appealing . However, several challenges arise formodel-driven representation: inclusion of forecast uncertainty inthe flexibility quantification; multi-temporal nature of the flexibilityactivation; combination of different flexible resources. This typeof models have relevant information to implement economic de-mand response scheme and it is possible to create a marketplacefor such type of models. For instance, virtual batteries flexibilitycan be traded through the combination of blockchain and smartcontracts [20] or integrated in a distributed optimal power flow(OPF) problem.

An interesting initiative is the OpenMined2 deep learning mar-ketplace that combines federated machine learning, blockchain,multi-party computation, and homomorphic encryption. In thisecosystem, deep learning algorithms fitted in distributed data blocks

1https://research.googleblog.com/2017/04/federated-learning-collaborative.html (ac-cess on February 2018)2https://openmined.org/ (accessed on February 2018)

https://research.googleblog.com/2017/04/federated-learning-collaborative.html

https://openmined.org/


are traded through smart contracts. The SVDD parameters of a flex-ibility set or the virtual batteries parameters can be traded in asimilar platform.

The future research directions are: (a) a non-linear peer-to-peervector autoregression model with data fully private and that can beapplied to different collaborative forecasting problems; (b) aggrega-tion of virtual battery models that combine prosumer’s flexibilityand forecast uncertainty and its integration with the HEMS hard-ware. The HEMS and its intelligent functions will be installed inreal prosumers and the value of the proposed approaches will beassessed in a real-world scenario.

ACKNOWLEDGEMENTSThe research leading to this work is being carried out as a part ofthe InteGrid project (Demonstration of INTElligent grid technologiesfor renewables INTEgration and INTEractive consumer participa-tion enabling INTEroperable market solutions and INTErconnectedstakeholders), which received funding from the European Union’sHorizon 2020 Framework Programme for Research and Innovationunder grant agreement No. 731218.

REFERENCES[1] C. Abreu, D. Rua, P. Machado, J.A. Peças Lopes, and M. Heleno. 2017. AnyPLACE

- An Energy Management System to Enhance Demand Response Participation.In IEEE PowerTech’17. Manchester.

[2] A. Albert and R. Rajagopal. 2018. Finding the Right Consumers for ThermalDemand-Response: An Experimental Evaluation. IEEE Transactions on SmartGrid In Press (2018).

[3] A. Alonso, R. Couto, H. Pacheco, R.J. Bessa, C. Gouveia, L. Seca, J. Moreira,P. Nunes, P. Matos, and A. Oliveira. 2017. Towards New Data ManagementPlatforms for a DSO as Market Enabler – UPGRID Portugal demo. CIRED - OpenAccess Proceedings Journal 2017, 1 (Oct. 2017), 2926–2930.

[4] J.R. Andrade and R.J. Bessa. 2017. Improving Renewable Energy Forecastingwith a Grid of Numerical Weather Predictions. IEEE Transactions on SustainableEnergy 8, 4 (Oct. 2017), 1571–1580.

[5] R.J. Bessa, A. Trindade, and V. Miranda. 2015. Spatial-temporal Solar PowerForecasting for Smart Grids. IEEE Transactions on Industrial Informatics 11, 1(Feb. 2015), 232–241.

[6] K. Boukir, D. Waestlund, B. Traverson, F. Schwarzlaender, and S. Defrancisci.2017. Providing Smart Metering Data Services Through an EU Market Place.CIRED - Open Access Proceedings Journal 2017, 1 (2017), 2848–2851.

[7] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. 2011. Distributed Optimiza-tion and Statistical Learning via the Alternating Direction Method of Multipliers.Foundations and Trends in Machine Learning 3, 1 (2011), 1âĂŞ122.

[8] J. Bremer, B. Rapp, and M. Sonnenschein. 2011. Encoding Distributed SearchSpaces for Virtual Power Plants. In 2011 IEEE Symposium on ComputationalIntelligence Applications In Smart Grid (CIASG). Paris, France.

[9] L. Cavalcante, R. J. Bessa, M. Reis, and J. Dowell. 2017. LASSO Vector Autore-gression Structures for Very Short-term Wind Power Forecasting. Wind Energy20, 4 (April 2017), 657–675.

[10] C. Chelmis, J. Kolte, and V. K. K. Prasanna. 2015. Big Data Analytics for DemandResponse: Clustering Over Space and Time. In IEEE International Conference onBig Data (IEEE Big Data 2015). Santa Clara, CA, USA.

[11] C. Eksin, H. Deliç, and A. Ribeiro. 2018. Demand Response with CommunicatingRational Consumers. IEEE Transactions on Smart Grid 9, 1 (Jan. 2018), 469–482.

[12] A. Gerza. 2015. The Value of Interval Meter Data in Solar PVProject Analysis. Technical Report. Energy Toolbase, ETB ResourceGuide. https://www.energytoolbase.com/newsroom/resourceguides/TheValueOfIntervalMeterDataInSolarPvProjectAnalysis

[13] V. Gómez, M. Chertkov, S. Backhaus, and H. J. Kappen. 2012. Learning Price-Elasticity of Smart Consumers in Power Distribution Systems. In 2012 IEEE ThirdInternational Conference on Smart Grid Communications (SmartGridComm).

[14] A. Gustafsson, C. Katzeff, and M. Bang. 2009. Evaluation of a Pervasive Game forDomestic Energy Engagement Among Teenagers. Computers in Entertainment(CIE) 7, 4 (2009).

[15] He Hao, Borhan M. Sanandaji, Kameshwar Poolla, and Tyrone L. Vincent. 2015.Aggregate Flexibility of Thermostatically Controlled Loads. IEEE Transactions onPower Systems 30, 1 (jan 2015), 189–198. https://doi.org/10.1109/TPWRS.2014.2328865

[16] Saghar Hosseini, Airlie Chapman, and Mehran Mesbahi. 2014. Online distributedADMM via dual averaging. In Decision and Control (CDC), 2014 IEEE 53rd AnnualConference on. IEEE, 904–909.

[17] Jungsuk Kwac and Ram Rajagopal. 2016. Data-Driven Targeting of Customers forDemand Response. IEEE Transactions on Smart Grid 7, 5 (Sept. 2016), 2199–2207.

[18] P. Mallet, P. Granstrom, P. Hallberg, G. Lorenz, and P. Mandatova. 2014. Power tothe People!: European Perspectives on the Future of Electric Distribution. IEEEPower and Energy Magazine 12, 2 (March 2014), 51–64.

[19] Javier Matamoros. 2017. Asynchronous online ADMM for consensus problems.In Acoustics, Speech and Signal Processing (ICASSP), 2017 IEEE International Con-ference on. IEEE, 5875–5879.

[20] J. Mattila, T. Seppala, C. Naucler, R. Stahl, M. Tikkanen, A. Badenlid, and J. Seppala.2016. Industrial Blockchain Platforms : An Exercise in Use Case Development in theEnergy Industry. Technical Report. ETLA Working Papers 43.

[21] M. McGranaghan, D. Houseman, L. Schmitt, F. Cleveland, and E. Lambert. 2016.Enabling the Integrated Grid: Leveraging Data to Integrate Distributed Resourcesand Customers. IEEE Power and Energy Magazine 14, 1 (Jan. 2016), 83–93.

[22] E. McKenna, I. Richardson, and M. Thomson. 2012. Smart meter data: balancingconsumer privacy concerns with legitimate applications. Energy Policy 41 (Feb.2012), 807–814.

[23] Council of European Energy Regulators. 2016. Review of current and future datamanagement models. Technical Report C16-RMF-89-03. CEER.

[24] R.B. Pinto, R.J. Bessa, and M.A. Matos. 2017. Multi-period Flexibility Forecast forLow Voltage Prosumers. Energy 141 (Dec. 2017), 2251–2263.

[25] M. Song and M. Amelin. 2017. Purchase Bidding Strategy for a Retailer withFlexible Demands in Day-ahead Electricity Market. IEEE Transactions on PowerSystems 32, 3 (May 2017).

[26] J. Stromsather, L. Marretta, and P. Soderstrom. 2015. Opening Up for a MoreCompetitive Energy Market with New Energy Services by Making “Real time”MeteringData Accessible toMarket Players. InCIRED 2015 - The 23rd InternationalConference and Exhibition on Electricity Distribution. Lyon, France.

[27] J. Tastu, P. Pinson, P. Trombe, and H. Madsen. 2014. Probabilistic Forecasts ofWind Power Generation Accounting for Geographically Dispersed Information.IEEE Transactions on Smart Grid 5, 1 (Jan. 2014), 480–489.

[28] K. Tsuda, M. Uwasu, K. Hara, and Y. Fuchigami. 2017. Approaches to InduceBehavioral Changes with Respect to Electricity Consumption. Journal of Envi-ronmental Studies and Sciences 7, 1 (March 2017), 30–38.

[29] European Union. 2016. Regulation (EU) 2016/679 of the European Parliamentand of the Council of 27 April 2016 on the protection of natural persons withregard to the processing of personal data and on the free movement of such data,and repealing Directive 95/46/EC (General Data Protection Regulation). OfficialJournal of the European Union. (2016).

[30] Tianyu Wu, Kun Yuan, Qing Ling, Wotao Yin, and Ali H Sayed. 2016. Decentral-ized consensus optimization with asynchrony and delays. In Signals, Systems andComputers, 2016 50th Asilomar Conference on. IEEE, 992–996.

[31] Z. Xu, T. Deng, Z. Hu, Y. Song, and J. Wang. 2018. Data-Driven Pricing Strategyfor Demand-Side Resource Aggregators. IEEE Transactions on Smart Grid 9, 1(Jan. 2018), 57–66.

[32] Chunlei Zhang and Yongqiang Wang. 2017. Privacy-preserving DecentralizedOptimization Based on ADMM. arXiv preprint arXiv:1707.04338 (2017).

https://www.energytoolbase.com/newsroom/resourceguides/TheValueOfIntervalMeterDataInSolarPvProjectAnalysis

https://www.energytoolbase.com/newsroom/resourceguides/TheValueOfIntervalMeterDataInSolarPvProjectAnalysis

https://doi.org/10.1109/TPWRS.2014.2328865

https://doi.org/10.1109/TPWRS.2014.2328865

Data Economy for Prosumers in a Smart Grid Ecosystem · Smart grids, demand response, energy...

Documents

Transcript of Data Economy for Prosumers in a Smart Grid Ecosystem · Smart grids, demand response, energy...