LITERATURE REVIEW SENSOR FUSION TECHNOLOGY
Transcript of LITERATURE REVIEW SENSOR FUSION TECHNOLOGY
Bachelor of Science Thesis
KTH School of Industrial Engineering and Management
Energy Technology EGI-2017
SE-100 44 STOCKHOLM
Literature review of sensor fusion technology -
For improved occupancy information in indoor spaces
Mahmoud Samara
i
Bachelor of Science Thesis EGI-2017
Literature review of sensor fusion technology –
For improved occupancy information in indoor spaces
Mahmoud Samara
Approved
2017-06-06
Examiner
Per Lundqvist
Supervisor
Marco Molinari
Commissioner
Contact person
ii
Abstract
As the energy consumption within the building sector is projected to steadily increase in regards to heating
and cooling of the buildings, the importance of improving the principle sensor technology that obtains
occupancy information to manage these control systems is prominent. This report aims to provide a basic
literature review of the commercially available single-sensor technology applied for occupancy detection in
buildings for control systems of heating, cooling and for monitoring the use of indoor spaces. Moreover,
detailed information on the researched case studies implementing sensor fusion technology to increase
detection accuracy, and the possibility of acquiring the people count within buildings will be provided and
discussed. From the articles reviewed, a use of multi-sensory technology systems, and extensive data accu-
mulation, the occupancy estimation accuracies are increasing as well as verified energy savings of the Heat-
ing, Cooling and Air Condition (HVAC) systems in several experiments. The parameters of success rate
obtained in the reviewed sensor fusion studies are occupancy estimation accuracies ranging between 73-
78%, occupancy detection accuracies ranging from 74-98%, Root Mean Square Errors (RMSE) of the
model performance ranging between 0.084-0.1842, and total energy savings by implementing the articles’
sensory model ranging between 21-39%.
iii
Acknowledgements
The author would like to thank supervisor Marco Molinari for fair and deliberate guidance.
iv
List of figures Figure 1. A possible combination of single sensors for sensor fusion technology...............................................5
Figure 2. An overview of the experimental ground of Article 1. (Source: Ekwevugbe et al. 2017).................6
Figure 3. The ambient sensors and BLEMS sensor box (Source: Yang et al., 2014). ..........................................8
Figure 4. An overview of the experimental ground of Article 2. (Source: Yang et al., 2014). ...........................9
Figure 5. View of the smart-door, including the sensor positioning and flow of sensor data. (Source: Chil
Prakash et al., 2015)............................................................................................................................................................. 10
Figure 6. An overview of the experimental ground of Article 4. (Source: Zikos et al., 2016). ....................... 11
Figure 7. An overview of the experimental ground of Article 5. (Source: Agarwal et al., 2011). .................. 12
Figure 8. An overview of the experimental ground of Article 6. (Source: Dong et al., 2010) ........................ 13
Figure 9. An overview of the experimental ground of Article 7. (Source: Meyn et al., 2009). ....................... 14
Figure 10. Graphs depicting occupancy levels at zone level in certain times. (Source: Meyn et al., 2009). 15
Figure 11. An overview of the experimental ground of Article 8. (Source: Wang et al., 2017)...................... 15
Figure 12. An overview of the experimental ground of Article 9. (Source: Zhu et al., 2017). ....................... 16
Figure 13. An overview of the experimental ground of Article 10. (Source: Vaccarini et al., 2016)............. 17
List of tables Table 1. Table of single sensor systems ............................................................................................................................6
Table 2. Table of reviewed articles.................................................................................................................................. 20
v
Table of Contents
Abstract........................................................................................................................................ ii
Acknowledgements....................................................................................................................... iii
List of figures ............................................................................................................................... iv
List of tables ................................................................................................................................ iv
1. Introduction .............................................................................................................................. 1
1.1 Sensor-based control systems ................................................................................................. 1
1.2 Purpose ............................................................................................................................... 1
2. Method ..................................................................................................................................... 2
2.2 Classification success rates ..................................................................................................... 2
2.2.1 RMSE ........................................................................................................................... 2
3. Analysis .................................................................................................................................... 3
3.1 Commercially available technologies ....................................................................................... 3
3.1.1 Passive infrared (PIR) sensors .......................................................................................... 3
3.1.2 Carbon Dioxide (CO2) sensor .......................................................................................... 3
3.1.3 Ultrasonic sensors........................................................................................................... 4
3.1.4 Image sensors ................................................................................................................ 4
3.1.5 Acoustic sensors ............................................................................................................. 4
3.2 Sensor fusion ....................................................................................................................... 5
3.2.1 Literature review............................................................................................................. 5
4. Results .................................................................................................................................... 19
5. Discussion............................................................................................................................... 21
References .................................................................................................................................. 23
1
1. Introduction
The U.S. Energy Information Administration (EIA) have projected an increase in energy consumed within
the residential building sector by 1.4% annually, increasing by 48% between 2012 and 2040 worldwide
(‘International Energy Outlook 2016-Buildings Sector Energy Consumption - Energy Information Admin-
istration’ 2017). Most the energy used within the sector is mainly delivered to control systems for space
heating and cooling, but also for lighting and other energy consuming products such as fridges and chargers.
With such a rapid growth rate, there is a demand on decreasing the energy delivered to the Heating, Venti-
lation and Air Conditioning (HVAC) control systems as well as upholding the thermal comfort of the
occupants.
1.1 Sensor-based control systems
A use of sensors to detect occupants in a specified indoor environment has been an essential part for
minimising energy consumption in buildings by implementing sensor-based lighting control systems in
buildings (Guo et al. 2010; Delaney, O’Hare, and Ruzzelli 2009), regulating the ventilation to supply the
occupancy load (Emmerich and Persily 2003) and important for this article; less delivered energy to HVAC
control systems as a result of more accurate sensor-based occupancy information (Labeodan et al. 2015;
Ekwevugbe 2013).
The commercially available technologies for acquiring occupancy information on the presence/absence
(binary classification) of occupants on room- or building level use single sensor technologies. However, as
discussed in chapter 3, the single sensor-based control systems prove to be very limited in acquiring further
information on the count, location and identity of the detected occupants. Research claims that the addition
of more sensors generally has a better success rate in occupancy information as they minimise their respec-
tive drawbacks and increase their strengths (Ekwevugbe et al. 2017; Yang et al. 2014). By using a combina-
tion of different types of sensors, estimation of indoor occupancy can be improved. resulting in HVAC
control systems acting on more accurate occupancy loads.
1.2 Purpose
This report aims to give an overview on sensor fusion as a feasible approach to counting people in buildings
and how this may contribute to the energy management, without compromising the occupants thermal
comfort. Ultimately, this review aims to serve as a guideline for experiments conducted in the Live-In Lab
on KTH campus (https://www.liveinlab.kth.se/). The KTH Live-in Lab is a test-bed building projected to
be finalised in the summer of 2017 that will be used for innovative and practical studies within the field of
environmental engineering. It will house residents and include other work spaces for scientists who aim to
further research and validate products or services for application in a residential building. A number of
articles on sensor fusion based HVAC systems have been reviewed, analysed and briefly summarised to
provide the reader with both practical and research-level studies within the field.
2
2. Method
Established scientific databases (http://www.sciencedirect.com/, https://www.scopus.com/) have been
used to find relevant articles within the field of sensor based occupancy information that are reviewed and
analysed for this report. It is in the interest of the author to enlighten certain studies that are befitting to be
implemented into, and further studied in the KTH Live-in Lab based on classifications discussed below.
2.2 Classification success rates
The occupancy information on detection refers to the sensor systems’ ability to send a binary output of the
occupants’ presence or absence in an indoor space. The success rate of the system is mainly obtained in
a figure of accuracy in percent, defined as the obtained result in comparison to the true result inside a given
indoor space. Information on occupancy estimation or count, the ability to provide an exact figure of the
number of occupants’ in a building is required for occupancy driven HVAC systems. For the count, the
success rate in most of the articles are given as a figure of occupancy estimation accuracy or the Root Mean
Squared Error (RMSE) for measuring the model systems’ performance.
2.2.1 RMSE
RMSE measures a difference between the estimated count and the real observations (commonly: ground
truth result), providing the reader with the standard deviation of the model systems error, where smaller
values mean that the model system is better (Yang et al. 2014). The total data range of the occupants is
denoted by 𝑛. It can be explained as the sample size of the model, e.g. in some studies the occupancy load
is sampled into, low-, medium-, and high making 𝑛 = 3.
√1
n∑(OE(i)− OR(i))
2n
i=1
(𝟏)
Where 𝑂𝐸 = Estimated occupancy count of the model
𝑂𝑅 = Real occupancy count
𝑛 = Total data range
3
3. Analysis
In the following chapter the analysis of reviewed articles will be briefly provided on commercially available
single-sensor technology, followed by case studies on the application of sensor fusion for occupancy esti-
mation in indoor spaces.
3.1 Commercially available technologies
Most of the demand driven control systems such as lighting, electrical devices (computers, printers etc.) or
heating and cooling consist of data from single-sensor technology. Since this report has more interest in
the combination of single sensors, there will only be a brief overview of the most common single sensors
and some applications in reviewed literature, with greater focus on their respective limitations.
3.1.1 Passive infrared (PIR) sensors
PIR sensors detect heat energy which is emitted by humans and everything with an above-zero absolute
temperature. It is passive because it does not itself send out any energy or radiation; it sends a signal to the
control system whenever it senses a change in infrared energy within its field of vision.
It is used in demand controlled electrical appliances based on occupancy detection, for example the lighting
system of a building (Guo et al. 2010; Wahl, Milenkovic, and Amft 2012), as it will turn appliances off if
there is no change in heat energy in its field of vision, thus saving energy on unoccupied rooms.
For ventilation control, the PIR sensor is not as effective as its not able to obtain occupancy count in a
room due to its binary sensor output (Li, Calis, and Becerik-Gerber 2012), thus providing no information
on the occupancy load. Its limitations concern occupants that remain still for a longer period resulting in
the sensor changing room or building status from occupied to unoccupied, and that two or more occupants
entering the sensors’ field of vision simultaneously can be registered as one.
3.1.2 Carbon Dioxide (CO2) sensor
Carbon dioxide measurement is a prominent sensor technology for occupancy detection in an indoor area,
specifically when a Demand Controlled Ventilation (DCV) is implemented in the building. CO2-based DCV
systems are an alternative method to constant ventilation that can lower the energy consumption of a
building as well as providing the occupants with better Indoor Air Quality (IAQ), by controlling the air
flow into the room in accordance to occupancy levels. A literature review (Emmerich and Persily 2003),
provides a summarisation of mainly CO2-based DCV systems that were tested by case/field studies, proving
to yield great energy savings and can therefore decrease energy costs.
Drawbacks due to the slow response between sample intervals of the sensor mean that the ventilation might
set off after the occupants already are in discomfort (Apte, Fisk, and Daisey 2000; Labeodan et al.
4
2015). A purge of the system would be needed to “rinse” the room of old air that should be scheduled
when e.g. a lecture hall is scheduled to be empty between lectures. However, this wastes a lot of the free
heat that is provided from the buildings systems (computers etc.).
Other drawbacks include disturbances in regards to open windows or other sources of ventilation that may
result in false concentration values which in turn obtains false occupancy information.
3.1.3 Ultrasonic sensors
Unlike the passive nature of the PIR sensor, the ultrasonic sensor consists of both a transmitter and receiver
with which it sends and receives ultrasonic sound waves in the environment in which it is placed. They can
provide occupancy information on the presence and location by the difference in the echo of the signal
emitted by the sensor when a soundwave hits an occupant (Caicedo and Pandharipande 2012). The limita-
tions for implementing the ultrasonic sensor to HVAC control systems are its inability to provide an occu-
pancy count as it also gives off a binary output, and the sensitivity to vibrations from e.g. air turbulence
giving false readings of presence. (Labeodan et al. 2015).
3.1.4 Image sensors
Video recording has been used primarily for surveillance in the form of CCTV cameras, of open outdoor
areas or inside buildings containing important goods/information e.g. a bank. Unlike the aforementioned
sensors, it can provide occupancy information on presence, location, count, activity and even identity in
some cases (Erickson, Achleitner, and Cerpa 2013), which is suitable for occupancy based HVAC systems.
However, the limitations are the cost of installation and maintenance of equipment, having its line of sight
blocked by other occupants providing false counts (like the PIR sensor) and that the privacy of the occupant
would be compromised if it were to be implemented in any building, specifically residential buildings. This
is mainly due to the unsettling nature of being recorded in the comfort of one’s home.
3.1.5 Acoustic sensors
Like the PIR sensors, acoustic sensors are of a passive nature, where they get triggered by energy received
in the form of audible sound. The sensors are best suited for industrial buildings or warehouses (Guo et al.
2010) with constant sources of noise. The sensor can be falsely triggered by disturbances other than occu-
pants or be subject to false-OFF values where it doesn’t register a silent occupant. For further occupancy
information needed for HVAC control systems the acoustic sensor reveals the following limitations
1. The sensor is not triggered if there is no sound made for a longer period, even if the room is
occupied.
2. It is subject false triggers from electrical appliances, or other non-occupant sources.
5
3.2 Sensor fusion
Figure 1. A possible combination of single sensors for sensor fusion technology.
As it becomes clear that single sensor technologies have drawbacks for heating and cooling control, an
approach of sensor fusion based technology has arisen by combining sensors with each other and attempt-
ing to compensate for the respective single sensors’ drawbacks.
This approach is widely researched and aims to obtain more accurate occupancy information figures as
opposed to using single sensor systems. In more recent studies, there have been less indications of im-
provements being made on the single sensors per se, in regards to their respective performance; however,
when combining several already existing means of occupancy detection, they can provide great accuracy
increases further discussed in the chapter.
3.2.1 Literature review
Below is a review of several articles ranging from 2009 to 2017 (not in chronological order) conducting
experiments to improve occupancy information by adding sensors to already existing single-sensor tech-
nologies, or implementing own configurations of multi-sensory systems to buildings. A table summarising
the sensors used in the article as well as in which type of building it has been used is also provided. The
table is based on the sensors provided in the method chapter, however to not have rows with only one
sensor-box ticked, the article-specific sensors have been excluded. The number of excluded sensors are
listed in the last column and explicit information of them can be found in either each respective article or
in Table 2. of the Results. The article sub-chapters will provide general information in regards to the meth-
odology, sensors used and an underlying discussion in regards to limitations or possible application to the
KTH Live-In Lab.
6
Table 1. Summary of the articles below based on the sensors used and in what type of building it is tested. Last column refers to
number of other sensors used in experiment. Note: more detailed information found for each respective article and in the Results.
3.2.1.1 Article 1. ‘Improved Occupancy Monitoring in Non-Domestic Buildings’
This experiment (Ekwevugbe et al. 2017) was conducted in an admissions office in a university in England,
with an associated small kitchen that can house two occupants. The aim was to use a sensor fusion approach
for obtaining accurate occupancy information for control systems to adjust heating and cooling based on
occupancy load. There is normally a working staff of six occupants, however with a varying occupancy load
due to frequent visits of e.g. students and lecturers.
Figure 2. An overview of the admissions office, revealing the positions of the sensors. (Source: Ekwevugbe et al. 2017).
Commercial
building
Residential
buildingPIR
Tempera-
ture
Ultra-
sonicImage Acoustic
Other sensors
(See chapter 3)
Article
1 x x x x 1
2 x x x x x 3
3 x x 1
4 x x x x 2
5 x x 1
6 x x x x x 3
7 x x x x 0
8 x x x 0
9 x x x 2
10 x x x x 4
CO2
7
Sensors used:
1. PIR
2. 𝐂𝐎𝟐
3. Temperature (computer desktop cases)
4. VOC
5. Sound
The PIR sensors were placed close to already existing sensors which was used by the Building Energy
Management Systems (BEMS), with the rest of the sensors placed as viewed in Figure 1. The testing took
place from 27/11/2012 – 20/12/2012 and were all recorded to data loggers and compared to the ground
truth estimation provided by data from the infrared camera with one minute intervals.
The overall estimation accuracy ranged between 62-74%. It was discussed in the article that the lower range
(62%) had a possible link to the slow decay rate of CO2, which can last till the next morning. In comparison
to the multi-sensory estimation accuracy, if only two sensors from the model were tested e.g. case temper-
ature and sound, the estimation accuracy decreased to 59-62%. This proves that the use of a multi-sensory
system is superior to single-sensor systems in this experiment as the sensors combined can compensate for
their respective drawbacks.
This study is the only one in this report that records ground truth occupancy information with an un-
intrusive sensor for privacy concerns. Adding to the positive remarks of the study is that it aims to use less
and low cost sensor combinations making it suitable for an experiment on a smaller budget. The suggest
implementing the model specifically on non-domestic buildings, however there is no clarity to why.
3.2.1.2 Article 2. ‘A Systematic Approach to Occupancy Modeling in Ambient Sensor-Rich Buildings’
A study was conducted by (Yang et al. 2014) attempting to find the best sensor-software fusion in order to
model occupancy profiles that can be applied to an HVAC system in a test-bed building. Since this article
is applying the sensor fusion approach to a building with similar features to the KTH Live-in Lab, this
review will be more detailed. The Building Level Energy Management System (BLEMS) project has built
an experimental building on the University of South California campus. The building has three stories with
indoor spaces such as offices, classrooms and auditoriums. It houses 50 permanent residents annually (staff,
graduate students, and faculty), and 2000 temporary residents (undergraduate students and graduate stu-
dents) per semester. It is an experimental ground for residents to put theoretical experiments of a buildings
energy management into practice for validation.
The experiment tested two single- and two multi-occupancy rooms with a size of around 18 and 40 square
meters respectively. The smaller rooms have had up to three visitors, whilst the larger rooms that normally
is shared by 5-8 students has had up to 10 visitors in the experiment. The actual occupancy was obtained
by mounting cameras in the ceiling of the single-occupancy rooms, and touch screen devices that the oc-
cupants logged in/out of upon entrance/exit in the multi-occupancy rooms.
8
The study acquires occupancy estimation by using 50 sensor boxes mounted throughout the building with
a single-board microcontroller computer that support Wi-Fi, as well as a variety of sensors listed below
Figure 3. (a) The ambient sensors alongside the microcontroller and WiFi module inside the box. (b) Shows the BLEMS sensor box. (Source: Yang et al., 2014).
Sensors used:
1. PIR (detection of occupants that pass through a door)
2. Light
3. 𝐂𝐎𝟐
4. Temperature & Humidity
5. Door (detection of doors status; open or closed)
6. Sound
7. Motion (in room)
The boxes handle 11 sensor variables which were categorized into three types:
1. Instant variables or instant data received from sensors: lighting level, binary motion, 𝐂𝐎𝟐
concentration, temperature, humidity, binary infrared and door status.
2. Count variables or the net output change of the sensor in the last minute: Net motion
count, net infrared count and net door count.
3. Average variables or the average of the output over given time: Sound average taken every
5 seconds.
9
Figure 4. Third storey view of the deployment of sensor boxes and existing thermostats. (Source: Yang et al., 2014).
At a sampling rate of 1 minute intervals, sensor data was collected from 12/09/2011 to 01/10/2011 in the
multi-occupancy rooms, and from 15/05/2012 to 15/06/2012 in the single-occupancy rooms. A combi-
nation of all sensors yielded the best binary (present/absent) occupancy accuracy of 98.2% and RMSE
0.109 for one of the tested single-occupancy rooms, and an accuracy & RMSE of 97.8% & 0.141 respec-
tively for one of the tested multi-occupancy rooms.
The success rates of the performance were of high standard with further studies on possible the use of a
global model that can, with no required ground truth, and other indoor sensing data uphold same success
rates. However, there are drawbacks due to the heterogeneity in the data caused errors when creating global
models. This experimental method would be interesting to apply to the KTH Live-In Lab as the success
rates are good, and due to the clear similarities between the two buildings.
3.2.1.3 Article 3. ‘Demo Abstract: Demonstration of Using Sensor Fusion for Constructing a Cost-Effective
Smart-Door’
(Chil Prakash et al. 2015) looks at the possibility of occupancy estimation by using a smart-door. The use
of a smart-door for recording height and weight of entries and exits can by storing the data, create unique
profiles for occupants for that specific session. During the training phase, each enter/exit is tagged with
the actual occupants’ identity by tablets installed at the door, where this ground truth is collected to the real
height and weight of each occupant.
As the occupant enters or exits the door, either of the lasers are cut changing the voltage in the phototran-
sistor and sending a signal to the Raspberry Pi, which in turn triggers the sensor mat and ultrasonic sensor
to start measuring the occupants’ weight and height respectively.
10
Figure 5. View of the smart-door, including the sensor positioning and flow of sensor data. (Source: Chil Prakash et al., 2015).
Sensors used:
1. Ultrasonic
2. Weight mat
The use of different software classifiers provides different accuracy results, ranging from 71.76-93.7%. The
Random forest classifier, used for learning and predicting occupants’ identity based on the ground truth
values from the learning phase, yielded results between 90.72-93.7% in measurements of height, weight,
shoulder width, hip width and torso length.
The study is very basic, and even though it obtains quite accurate results, the experiment has not covered
aspects such as two students walking through the door simultaneously. The article has not stated any occu-
pancy estimation figure either, however it is reviewed for its applicable nature for residential entrances with
good detection success rate and the interest of occupancy profiling for HVAC control systems upon a
specific occupants’ thermal comfort preferences.
3.1.2.4 Article 4. ‘Conditional Random Fields - Based Approach for Real-Time Building Occupancy Esti-
mation with Multi-Sensory Networks’
(Zikos et al. 2016) experiments using an array of sensors within different indoor spaces in a building; a
kitchen, meeting room, multi-occupant office and an open space area to primarily test the estimation and
occupancy accuracy of different sensor combinations for ventilation controls. The occupancy estimation
was tested in three of the indoor spaces with success rates provided in NRMSE which is the RMSE divided
by the range of the occupancy class. For the exact number of occupants, the range is from empty to maxi-
mum recorded occupants, which was 0-13 occupants.
11
Figure 6. View of the four indoor spaces, with the associated sensor placement. (Source: Zikos et al., 2016).
Sensors used:
1. PIR
2. Acoustic
3. 𝐂𝐎𝟐
4. Active infrared beams (AIR, double beam sensor)
5. Pressure mats
The meeting room provided the lowest NRMSE 0.084 when using a combination of double beam-, acous-
tic-, and PIR sensor. The office, using double beam sensor + pressure mats and PIR sensors obtained a
lowest NRMSE of 0.113, whilst the kitchen had an NRMSE of 0.111 when using double beam sensor +
mats, acoustic- and PIR sensor. In conclusion, the authors found that a larger set of sensors in the combi-
nation, the higher the overall accuracy, however in some cases the gain in performance might be negligibly
increased.
The trial of different combinations of sensors as well as the test-bed building being a multi-zoned floor
with different purposes makes the study a valid experimental framework for a residential building as it yields
the highest occupancy estimation and lowest NRMSE of the studies reviewed.
3.2.1.5 Article 5. ‘Duty-Cycling Buildings Aggressively: The next Frontier in HVAC Control’
An interesting experiment (Agarwal et al. 2011) in a building on UC San Diego campus was reviewed due
to the inexpensive sensor nodes deployed on a floor with several offices to control the HVAC system.
12
Figure 7. View of the floor model showing how the HVAC system is controlled by the nodes sending out signals to the base stations. (S ource: Agarwal
et al., 2011).
Sensors used:
1. PIR
2. Magnetic reed
The sensor nodes consist of a wireless module with salvaged PIR sensors from AirWick air fresheners (4$
each) for measuring movement inside each thermal zone, combined with a magnetic reed switch for binary
information of the doors’ status (open/closed). After deploying 33 nodes, 29 of the nodes (4 erroneous
ones due to false activation due to poor placement) worked with a 96% estimation accuracy compared to
the ground truth which was acquired by a head count every 15 minutes.
The naïve assumptions that most the occupants will close their office door after entering/exiting, and that
the number of occupants doesn’t vary within 15 minutes of ground truth measurements removes some of
this articles credibility. However, they have achieved energy saving results ranging from 9.54% to 15.73%
and 7.59% to 12.85% in HVAC electrical energy and HVAC thermal energy use respectively.
3.2.1.6 Article 6. ‘An Information Technology Enabled Sustainability Test-Bed (ITEST) for Occupancy
Detection through an Environmental Sensing Network’
(Dong et al. 2010) used an array of sensors in an open-plan office consisting of 16 rooms and 1 conference
room. Many visitors frequent at this indoor space and classes are held in the conference room, making the
occupancy load considered dynamic.
13
Sensors used:
1. CO, 𝐂𝐎𝟐 , TVOC (Total Volatile Organic Compound), 𝑷𝑴𝟐.𝟓
2. Acoustic
3. Temperature, Relative Humidity
4. 𝐂𝐎𝟐 (independent)
5. Motion
6. Pressure
A success rate in occupancy estimation accuracy of 73% was obtained. For ground truth values of occu-
pancy, the authors used a camera network. Being one of the two older studies of this review, it is interesting
for purposes of viewing the scientific progress in comparison to the recent studies, as well as the use of
mainly environmental sensors.
3.2.1.7 Article 7. ‘A Sensor-Utility-Network Method for Estimation of Occupancy in Buildings’
This article (Meyn et al. 2009) handles estimation count on both building level and zone level in a commer-
cial building using occupancy data from several sensors and historical data of the buildings’ occupancy
patterns. 10 video cameras, 6 pairs of PIR sensors, and 15 CO2 sensors were placed according to the figure.
Figure 8. View of the open-plan office with a sensor key to see the sensor placement. (Source: Dong et al., 2010)
14
Figure 9. Building- and zone-view of the sensor placement within the office building with sensor-key in a separate box. (Source: meyn et al., 2009).
Sensors used:
1. PIR
2. 𝐂𝐎𝟐
3. Image (Digital video cameras)
The study compares success rate of occupancy information using a sensor-data fusion compared to the
sensor systems independent estimation. The results yielded were in the form of an error of estimation in
percent, where the sensor count independently yielded 70% error on building level and 30% on zone level
due to higher estimated occupancy than the observed ground truth. The large building levels’ error may be
interpreted as the slow rate of CO2 measurements when many occupants enter/exit at once, compared to
the zonal level where the changes in occupancy load are smaller. whilst the introduction and combination
of sensor- and historical data lowered the error to 11% and 21% on building- and zone-level respectively.
As for the success rate, this study achieves significant decreases in error of occupancy estimation, however
from the graph (Figure 10.) viewing the results in comparison to the ground truth value, the estimator using
sensor-data fusion underestimates the count on zone-level when there is a higher occupancy level for a
longer period. This might affect the thermal comfort of the occupants before the HVAC system can cool
or heat the building/zone based on occupancy load which, in residential buildings would most probably
result in the occupants’ interaction with control systems or e.g. windows. Also, a ground truth value was
obtained by manually counting the video data which is not feasible for implementation, especially in resi-
dential buildings as the control system must work independently without interaction of outsiders.
15
Figure 10. Graphs depicting the different occupancy levels at zone level in certain times. Naive = independent sensor data, SUN = sensor -data fusion. (Source: meyn et al., 2009).
3.2.1.8 Article 8. ‘Predictive Control of Indoor Environment Using Occupant Number Detected by Video
Data and 𝐶𝑂2 Concentration’
(Wang et al. 2017) introduced the use of sensor-fusion based data for control of Air Conditioning (AC) and
Outdoor Air Handling Units (OAHU) that regulates the indoor CO2 concentration, as well as controlling
the lighting of an office space with 12 occupants. The temperature, humidity, illuminance, CO2 concentra-
tion and the electrical consumption of the AC/OAH units was measured using the sensor boxes seen in
Figure 11.
Figure 11. View of office space with placement of the video- and CO_2 concentration-occupancy based sensors as well as the facilities for studying the performance of the predictive control in the associated Legend. (Source: Wang et al., 2017).
Sensors used:
1. 𝐂𝐎𝟐 (occupancy estimation)
2. Image (occupancy estimation)
16
3. Temperature (performance)
4. Illuminance (performance)
The success rate was an indication of the energy savings of the predictive control of AC/OAH units based
on the occupancy estimation yielded from the video- and CO2 concentration data. Implementing the set-
up in the office, an occupancy detection accuracy of 91% was obtained, and the predictive control achieved
combined energy savings of 39.4% for the heating and cooling units.
The results are positive; however, comments arise regarding the use of video camera as means of occupancy
information. Although the authors of the article point the camera downwards to the floor to record enter-
ing/exiting occupants for privacy manners, it is still of privacy concern for occupants if implemented in a
residential building. In addition, the digital camera relies on a well-lit space, for it to provide less erroneous
estimations. A ground truth value was obtained by manually counting the video data which is not feasible
for implementation, especially in residential buildings. This is due to the control systems’ dependency on
the scientists’ ability of acquiring ground truth values.
3.2.1.9 Article 9. ‘Occupancy Estimation with Environmental Sensing via Non-Iterative LRF Feature
Learning in Time and Frequency Domains’
(Zhu et al. 2017) promoted the use of the environmental sensing parameters for achieving better occupancy
estimation results. The experiment was conducted in a research laboratory in Nanyang Technological Uni-
versity (NTU) with an occupancy level of more than 20 occupants. The aim was to increase accuracy of
detection and lower the NRMSE value as the improved occupancy information is an important parameter
for energy efficient Air Conditioning and Mechanical Ventilation (ACMV) systems.
Figure 12. A view of the research laboratory with the placement of sensors, as well as the cameras recording ground truth values at each entrance. (Source:
Zhu et al., 2017).
17
Sensors used:
1. 𝐂𝐎𝟐
2. Temperature
3. Humidity
4. Air pressure
The experiment yielded a detection accuracy of 95.63% and an NRMSE of 0.1842. The ground truth values
were recorded by image sensors (cameras), which affects the privacy of the occupants due to collecting
occupancy information that can disclose identity and behaviour. Implementing the presented occupancy
detection approach in a residential building can be a feasible occupancy detection approach, considering
how few sensors that were used for a large research laboratory, while still achieving a high success rate. The
experiment was conducted using raw sensor data rather than empirical knowledge lowering the level of
expertise needed, making this experiment a suitable choice for an experimental application for graduate
students within the KTH Live-In Lab.
3.2.1.10 Article 10. ‘Model Predictive Energy Control of Ventilation for Underground Stations’
The authors of (Vaccarini et al. 2016) have enlightened the problem of energy consumption in underground
stations. Due to major thermal exchanges through openings and the surrounding ground, the building is of
a more complex nature than residential buildings. A model predictive controller (MPC) based on data from
weather forecast services, schedules of trains and external fans, and occupancy detection for real number
of occupants in the station is used. The aim was to lower the energy consumption of the ventilation control
as well as maintaining comfort levels, and ultimately to have it applied in an actual underground station.
The importance of a well-functioning sensory network is stressed in the article.
Figure 13. View of an underground station with placement of sensors . Also visible is a bit of the outside to show that the sensors inside work with
comparable data from the outside. (Source: Vaccarini et al., 2016).
18
Sensors used:
1. Image (CCTV)
2. 𝐂𝐎𝟐
3. Air temperature
4. PM10 (Pollutants with a diameter less than 10 µm, which can pass through occupants’
lungs)
5. Wind direction
6. Air speed
The sensor model was applied to the underground station Passeig de Gràcia in Barcelona and yielded an
average of 33% energy savings during the year of 2014. As there were substantial energy savings in the
underground station, the authors of this article are interested to attempt application on residential buildings.
However, being that the wireless sensor network (WSN) includes a CCTV and has high cost of installation
and maintenance, privacy issues will arise, as well as economic burdens. The study has relied on another
study for occupancy density estimation in underground stations (Chow, Yam, and Cho 1999) reaching 90%
accuracy. This article is interesting due to its use of sensor fusion technology put into practice and yielding
such positive energy savings results, and the ambition to later apply this to residential buildings, however it
would be interesting to see the model in combination with a more recent occupancy detection method with
higher accuracy than the one used, possibly yielding higher energy savings results.
19
4. Results
The articles reviewed have shown an overall edge to sensor fusion technology over single sensor technology
for occupancy information in buildings due to its ability to lower each sensors’ respective drawbacks when
combined. As sensor fusion technology has been studied, but not yet extensively enough, there are still
discussion points concerning limited, and in some cases absent information on the occupancy detection
setup’s success rate. However, most of them have at least a figure representing the performance of the
sensor fusion model in a percentage of occupancy accuracy or a lowest RMSE/NRMSE. Furthermore,
most of the studies have been conducted on non-residential buildings which must do with the fact that the
experiments are more easily conducted in e.g. universities. Table 2. provides the reader with an overview of
the reviewed articles in chapter 3.
20
Table 2. Result table of the articles reviewed. RH = Relative Humidity, NP = Not provided or clear in the article. Best results for
each category highlighted.
ArticleNon-
Residentia
l building
Residentia
l building
Technologies used for
sensor fusion
Highest
occupancy
detection
accuracy
Highest
occupancy
estimation
accuracy
Lowest
RMSE/
NRMSE
Highest
energy
savings
1. Ekwevugbe et al. 2017;
‘Improved Occupancy Monitoring in Non-
Domestic Buildings’x
CO2, PIR, VOC,
Acoustic74.67% NP RMSE: 0.815 NP
2. Yang et al., 2014;
‘A Systematic Approach to Occupancy
Modeling in Ambient Sensor-Rich Buildings’x
PIR, Light, CO2,
Temperature + RH,
Door, Acoustic,
Motion
98.20% NP RMSE: 0.109
21.3%
(3
months)
3. Chil Prakash et al., 2015;
‘Demo Abstract: Demonstration of Using
Sensor Fusion for Constructing a Cost-
Effective Smart-Door’
xUltrasonic, Weight
mat93.70% NP NP NP
4. Zikos et al., 2016;
‘Conditional Random Fields - Based
Approach for Real-Time Building Occupancy
Estimation with Multi-Sensory Networks’
xPIR, AIR, Pressure
mat, CO2, Acoustic93% 78%
NMRSE:
0.084NP
5. Agarwal et al., 2011;
‘Duty-Cycling Buildings Aggressively: The
next Frontier in HVAC Control’x Magnetic Reed, PIR 96% NP NP 15.73%
6. Dong et al., 2010;
‘An Information Technology Enabled
Sustainability Test-Bed (ITEST) for
Occupancy Detection through an
Environmental Sensing Network’
x
CO2 + CO + TVOC +
PM2.5, CO2 (indep.),
Acoustic,
Temperature + RH,
Motion, Pressure
NP 73% NP NP
7. Meyn et al., 2009;
‘A Sensor-Utility-Network Method for
Estimation of Occupancy in Buildings’x Image, PIR, CO2 NP NP
Provided a
lowest error
percentage
of 11%
NP
8. Wang et al., 2017;
‘Predictive Control of Indoor Environment
Using Occupant Number Detected by Video
Data and CO2 Concentration’
x CO2, Image 91% NP NP39.4% (5
days)
9. Zhu et al., 2017;
‘Occupancy Estimation with Environmental
Sensing via Non-Iterative LRF Feature
Learning in Time and Frequency Domains’
x
CO2, Relative
humidity,
Temperature, Air
pressure
95.63% NPNRMSE:
0.1842NP
10. Vaccarini et al., 2016;
‘Model Predictive Energy Control of
Ventilation for Underground Stations’ x
Image, CO2, Air
temperature, PM10,
Wind direction, Air
speed
NP
Based on
study
achieving 90%
in
underground
stations
NP33%
(annual)
21
5. Discussion
As the project aimed to study the possibilities of occupancy detection based on sensor fusion systems in a
residential building for controlling heating and cooling, one cannot consider the results from studies made
on non-residential buildings directly applicable. Despite this, the buildings can have similarities for instance
the sizes of different rooms or similar occupancy loads. However, the primary difference that arises is the
behaviour of the occupants themselves. In recent years, occupant profiling has become a more interesting
topic, as one cannot disregard how their behaviour can influence the resulting energy consumption (Diraco,
Leone, and Siciliano 2015; O’Neill and Niu 2017; Luo et al. 2017). However, these studies have not been
included in the review as they do not intricately discuss the success rates of the implementation of sensor
fusion as concretely as is sought after for the purpose of this literature review.
A key feature in need of improvement is the means of measuring ground truth values. These are commonly
obtained during the learning phase of the predictive control model in which the actual occupancy estimation
is collected. The most common way to collect this data is by using image sensors, as seen in the reviewed
articles. This however, conflicts with the privacy of the occupants as it collects and stores information on
identity and in some cases the behaviour. The authors in article 2. (Yang et al. 2014) had difficulties imple-
menting the image sensors in all the rooms, due to some occupant’s unwillingness of having their move-
ment recorded, even in the name of science. This makes application within residential buildings as a stand-
ard seem more difficult. In addition, the camera will not acquire the accurate ground data independently
but requires someone to manually process the data and count the images, which would result in a need of
frequent human interaction with private images of the occupants. The few studies that obtained this value
by visiting the different test-areas and manually counted the occupants periodically did not record any im-
ages or store information of the occupant for a longer time making it a good alternative to the use of image
sensors. However, there are issues in regards to the time between each measurement where the occupants
can have moved throughout the building and returned to their respective rooms or offices without changing
the recorded ground truth value. Moreover, this would also require somebody to collect new ground truth
data for every learning phase of the system when e.g. there is an exchange of residents, making it an im-
practical alternative that is too dependent on human interaction. Article 1. uses an infrared camera which
is preferable due to not having ones’ identity disclosed as there is no facial recognition, however the data
recorded might not be 100% accurate which interferes with the overall acquired occupancy information.
For further research, the use of mobile phones for collecting occupancy information is of interest as most
data storage is regulated by authorities removing the stigma around occupancy-data collecting. Examples
of these regulated applications are Facebook and Snapchat.
The most implementable study to be tested and verified for the KTH Live-In Lab project is the article with
a similar test-bed building (Yang et al. 2014), according to the author of this review. The methodology may
well work as an experimental framework for the application of such technology in the lab due to the simi-
larities in the test-bed building. It is stated that inexpensive sensors are used further justifying the application
22
of their methodology to the Lab. However, there is a need of a predictive model of occupancy load as the
temperature from direct heating and cooling of a building cannot be instantaneously altered. The tempera-
ture is regulated based on the occupancy load as well as maintaining the thermal comfort of the occupants.
Thermal comfort is primarily country-based and specifies temperature requirements in certain spaces. The
national board of housing, building and planning in Sweden have regulations of minimum and maximum
indoor temperatures ranging between 18-20 degrees Celsius for upholding the occupants’ thermal comfort
(‘Boverket´s Building Regulations – Mandatory Provisionsand General Recommendations, BBR’ 2017)
These predictive models or software are prevalent in recent studies discussed in the analysis section. How-
ever, the focus of this report was on the sensor-based occupancy information and did therefore not include
detailed information of the improvement based on the use of software. The data-sensor-fusion, or use of
sensor fusion technology to acquire occupancy information, combined with data from predictive models
based on ideal occupancy information or occupancy profiles is a more complete approach for HVAC con-
trol systems in residential buildings. Based on the articles reviewed in this report, the data-sensor fusion
experiments might not have yielded the best success rates in all studies, due to still being widely researched,
however the overall trend proves the efficacy of the data-sensor fusion. In regards to the rest of the articles
that are studied on different indoor spaces with a majority resulting in positive success rates, it can be
concluded that several of the multi-sensory networks might be implementable, but would need more testing
than (Yang et al. 2014) on which sensor-combination that will render the best success rate in occupancy
estimation specifically for a building with the likeness of the KTH Live-in Lab.
For future work, a separate comprehensive review of the use of sensor-data fusion, studying the different
software used may be written and then compared to literature reviews of sensor-based occupancy estima-
tion such as this one, for experimental framework of which ideal sensor combinations and software that
can be studied and later validated in a test-bed building.
23
References
Agarwal, Yuvraj, Bharathan Balaji, Seemanta Dutta, Rajesh K. Gupta, and Thomas Weng. 2011. ‘Duty-Cycling Build-
ings Aggressively: The next Frontier in HVAC Control’. In Information Processing in Sensor Networks
(IPSN), 2011 10th International Conference on, 246–257. IEEE. http://ieeexplore.ieee.org/abstract/docu -
ment/5779042/.
Apte, M. G., W. J. Fisk, and J. M. Daisey. 2000. ‘Associations between Indoor CO2 Concentrations and Sick Building
Syndrome Symptoms in U.S. Office Buildings: An Analysis of the 1994-1996 BASE Study Data’. Indoor Air
10 (4): 246–57.
‘Boverket´s Building Regulations – Mandatory Provisionsand General Recommendations, BBR’. 2017. Boverket. Ac-
cessed May 21. http://www.boverket.se/en/start-in-english/publications/2016/boverkets-building-regula-
tions--mandatory-provisionsand-general-recommendations-bbr/.
Caicedo, D., and A. Pandharipande. 2012. ‘Ultrasonic Array Sensor for Indoor Presence Detection’. In 2012 Proceed-
ings of the 20th European Signal Processing Conference (EUSIPCO), 175–79.
Chil Prakash, Vivek, Anand Krishnan Prakash, Uddhav Arote, Vitobha Munigala, and Krithi Ramamritham. 2015.
‘Demo Abstract: Demonstration of Using Sensor Fusion for Constructing a Cost-Effective Smart-Door’. In
, 191–92. ACM Press. doi:10.1145/2768510.2770938.
Chow, Tommy WS, JY-F. Yam, and S.-Y. Cho. 1999. ‘Fast Training Algorithm for Feedforward Neural Networks:
Application to Crowd Estimation at Underground Stations’. Artificial Intelligence in Engineering 13 (3):
301–307.
Delaney, Declan T., Gregory MP O’Hare, and Antonio G. Ruzzelli. 2009. ‘Evaluation of Energy-Efficiency in Lighting
Systems Using Sensor Networks’. In Proceedings of the First ACM Workshop on Embedded Sensing Sys-
tems for Energy-Efficiency in Buildings, 61–66. ACM. http://dl.acm.org/citation.cfm?id=1810293.
Diraco, Giovanni, Alessandro Leone, and Pietro Siciliano. 2015. ‘People Occupancy Detection and Profiling with 3D
Depth Sensors for Building Energy Management’. Energy and Buildings 92 (April): 246–66.
doi:10.1016/j.enbuild.2015.01.043.
Dong, Bing, Burton Andrews, Khee Poh Lam, Michael Höynck, Rui Zhang, Yun-Shang Chiou, and Diego Benitez.
2010. ‘An Information Technology Enabled Sustainability Test-Bed (ITEST) for Occupancy Detection
through an Environmental Sensing Network’. Energy and Buildings 42 (7): 1038–46.
doi:10.1016/j.enbuild.2010.01.016.
Ekwevugbe, Tobore. 2013. ‘Advanced Occupancy Measurement Using Sensor Fusion’.
https://v00dor00001d.dmu.ac.uk/handle/2086/10103.
Ekwevugbe, Tobore, Neil Brown, Vijay Pakka, and Denis Fan. 2017. ‘Improved Occupancy Monitoring in Non -
Domestic Buildings’. Sustainable Cities and Society 30 (April): 97–107. doi:10.1016/j.scs.2017.01.003.
Emmerich, Steven J., and Andrew K. Persily. 2003. State-Of-The-Art Review of Co2 Demand Controlled Ventilation
Technology and Application. DIANE Publishing.
Erickson, Varick L., Stefan Achleitner, and Alberto E. Cerpa. 2013. ‘POEM: Power-Efficient Occupancy-Based En-
ergy Management System’. In Proceedings of the 12th International Conference on Information Processing
in Sensor Networks, 203–216. ACM. http://dl.acm.org/citation.cfm?id=2461407.
Guo, X., D. Tiller, G. Henze, and C. Waters. 2010. ‘The Performance of Occupancy -Based Lighting Control Systems:
A Review’. Lighting Research and Technology 42 (4): 415–31. doi:10.1177/1477153510376225.
24
‘International Energy Outlook 2016-Buildings Sector Energy Consumption - Energy Information Administration’.
2017. Accessed April 28. https://www.eia.gov/outlooks/ieo/buildings.cfm.
Labeodan, Timilehin, Wim Zeiler, Gert Boxem, and Yang Zhao. 2015. ‘Occupancy Measurement in Commercial
Office Buildings for Demand-Driven Control applications—A Survey and Detection System Evaluation’.
Energy and Buildings 93 (April): 303–14. doi:10.1016/j.enbuild.2015.02.028.
Li, Nan, Gulben Calis, and Burcin Becerik-Gerber. 2012. ‘Measuring and Monitoring Occupancy with an RFID Based
System for Demand-Driven HVAC Operations’. Automation in Construction 24 (July): 89–99.
doi:10.1016/j.autcon.2012.02.013.
Luo, Xuan, Khee Poh Lam, Yixing Chen, and Tianzhen Hong. 2017. ‘Performance Evaluation of an Agent -Based
Occupancy Simulation Model’. Building and Environment 115 (April): 42–53. doi:10.1016/j.build-
env.2017.01.015.
Meyn, Sean, Amit Surana, Yiqing Lin, Stella M. Oggianu, Satish Narayanan, and Thomas A. Frewen. 2009. ‘A Sensor-
Utility-Network Method for Estimation of Occupancy in Buildings’. In Decision and Control, 2009 Held
Jointly with the 2009 28th Chinese Control Conference. CDC/CCC 2009. Proceedings of the 48th IEEE
Conference on, 1494–1500. IEEE. http://ieeexplore.ieee.org/abstract/document/5400442/.
O’Neill, Zheng, and Fuxin Niu. 2017. ‘Uncertainty and Sensitivity Analysis of Spatio -Temporal Occupant Behaviors
on Residential Building Energy Usage Utilizing Karhunen-Loève Expansion’. Building and Environment
115 (April): 157–72. doi:10.1016/j.buildenv.2017.01.025.
Vaccarini, M., A. Giretti, L.C. Tolve, and M. Casals. 2016. ‘Model Predictive Energy Control of Ventilation for Un-
derground Stations’. Energy and Buildings 116 (March): 326–40. doi:10.1016/j.enbuild.2016.01.020.
Wahl, F., M. Milenkovic, and O. Amft. 2012. ‘A Distributed PIR-Based Approach for Estimating People Count in
Office Environments’. In 2012 IEEE 15th International Conference on Computational Science and Engi-
neering, 640–47. doi:10.1109/ICCSE.2012.92.
Wang, Fulin, Qingqing Feng, Zheliang Chen, Qianchuan Zhao, Zhijing Cheng, Jianhong Zou, Yufeng Zhang, Jinbo
Mai, Yun Li, and Hayden Reeve. 2017. ‘Predictive Control of Indoor Environment Using Occupant Number
Detected by Video Data and CO 2 Concentration’. Energy and Buildings 145 (June): 155–62.
doi:10.1016/j.enbuild.2017.04.014.
Yang, Z., N. Li, B. Becerik-Gerber, and M. Orosz. 2014. ‘A Systematic Approach to Occupancy Modeling in Ambient
Sensor-Rich Buildings’. SIMULATION 90 (8): 960–77. doi:10.1177/0037549713489918.
Zhu, Qingchang, Zhenghua Chen, Mustafa K. Masood, and Yeng Chai Soh. 2017. ‘Occupancy Estimation with En-
vironmental Sensing via Non-Iterative LRF Feature Learning in Time and Frequency Domains’. Energy and
Buildings 141 (April): 125–33. doi:10.1016/j.enbuild.2017.01.057.
Zikos, Stylianos, Apostolos Tsolakis, Dimitrios Meskos, Athanasios Tryferidis, and Dimitrios Tzovaras. 2016. ‘Con-
ditional Random Fields - Based Approach for Real-Time Building Occupancy Estimation with Multi-Sensory
Networks’. Automation in Construction 68 (August): 128–45. doi:10.1016/j.autcon.2016.05.005.