Counting versus Detection in an FM Application that Deals with Rooms Reservation

Introduction

The rapid development in recent years of the sensor systems performance and the increasing speed of networks through which data are transmitted in the cloud allows enterprise applications to display and use more and more real time data. These can be used to improve the reports, operations, and decisions that the application already had implemented.

The IoT enables physical objects to see, hear, think, and perform jobs by having them “talk” together, share information and coordinate decisions (Al-Fuqaha et al. 2015). In the last period, the number of applications that make use of the IoT has increased dramatically and it spans areas from business and manufacturing to home, health care, and knowledge management (Nord et al. 2019).

According to the definition given by the International Facility Management Association (IFMA), Facility Management (FM) is a profession that includes several disciplines helping to ensure the functionality of the built environment by integrating people, space, processes, and technology (EuroFM 2019). In the field of Facility Management (FM), the impact of IoT is growing and IoT is playing an important role in improving FM services.

Among the information obtained using IoT technology, people counting has implications not only in the efficient use of space but also in many other areas of FM. For example, in the study of passenger activity and traffic patterns at airports, the authors found that accurate data on people counting are essential information for people flow analysis and management, space use, service monitoring, security enhancement, productivity assessment, and resource optimization to improve or maintain the operating means at airport facilities (Gerland and McDonald, 2016). The Wireless Sensor Networks (WSN) is viewed as an information service provider not only to building management systems but also to wider applications in the enterprise infrastructure (Malatras et al. 2008).

There is a need on the market to see if the meeting rooms are optimally reserved. At the organizational level, this need was transposed through the requirement to monitor the meeting rooms with the help of sensors. In order to demonstrate the reliability of such a solution, a small project, POC type is first needed. From such a project, in fig. 1, is presented a Requirements Diagram, where is taken and developed one of the central business requirements that states: “For Proof of Concept (POC) project it is necessary to monitor the presence in two meeting rooms of the client”. The synthesis diagram of the entire Requirements Model is presented in another article, written by some of the authors of this paper, also sent to this conference (Udrea et al. 2020). Enterprise Architect software was used to create the entire Requirements Model and other models of the project (EA 2016).

This work realizes the requirement “A study must be done to find the best people count solution in a room” from Requirements Diagram presented in fig. 1. It contains a study of choosing the best people count solution for a meeting room.

Regarding the structure of Requirements Diagram, when modeling using requirement elements, there are numerous Unified Modeling Language (UML) connector types that can be used, however there are two types of relationships that are commonly used with requirement management. One for setting relationships between peer requirements (Aggregation), and another for representing how they will be implemented (for example a Realization by a Use Case). Requirements linked by Aggregation relationships form a composition hierarchy. High level requirements may be composed of lower-level requirements, which in turn are made up of finer and more specialized requirements. This hierarchical structure helps manage the complexity of large systems with thousands of requirements and many elements being employed to implement the requirements (EAReqM 2016). It should be noted that the Aggregation relationship is not specified in (BABOK 2015), in the subchapter Trace Requirements, where the types of relationships between requirements are defined. As can be seen from fig. 1, the requirements of the presented model are suitable for the Aggregation relation.

From the tracking requirements methods available in Enterprise Architect, the method of creating and visualizing the requirements using diagrams was chosen. It is simple to use this method and it is best applicable for small models of requirements. The traceability of the requirements in fig. 1 is made using diagrams

Fig 1. Requirements Diagram, made with Enterprise Architect software

In the context in which there was a need to monitor the presence in meeting rooms in order to improve the room reservation reports of an FM application, there was a requirement to find the best method of people counting. To achieve this, this article aims to find the best method of people counting and human detection by studying various methods and sensor systems.

Method and procedure

The building, represented by its BIM model, is a main if not central element of FM applications. There are “fixed” features that are determined once and remain valid as long as the building does not change, such as heat transfer and its simulation (Natephra et al. 2018, Pop et al. 2018), building ventilation (Sporr et al. 2019, Chitaru et al. 2018), and features that need to be monitored in real time, such as indoor air quality (Diodiu and Dogeanu, 2016) and the counting of the number of people (Ye, 2010, Gerland and McDonald, 2016).

Regarding the monitoring of people, the identifiable elements can be classified according to several properties. Thus, there can be spatio-temporal properties, classified hierarchically, from fewer to more attributes as follows: presence, count, location, track, identity. There can be behavioral properties, also classified hierarchically: pose, action, activity, behavior, group behavior. And there can be physiological properties: blood pressure, temperature, weight and other properties (Teixeira et al. 2010).

A classification of different people counting technologies is made according to the generation to which the measurement system belongs (Sruthi, 2019). Thus, there are first generation technologies (infrared beam counters), second generation (thermal counters) and third generation technologies (video & Wi-Fi counters).

A comparative analysis of localization technologies inside buildings is performed in (Chen et al. 2015). Occupancy detection systems can be categorized as individualized and non-individualized, based on whether an occupant can be identified and tracked in the sensed area. Passive sensors, such as infrared sensors, CO2 sensors, light sensors, and ultrasonic sensors are widely used for non-individualized detection. Most passive sensors use indirect sensing results as an indication of occupancy. Passive infrared (PIR) sensors are commonly used for lighting control in commercial buildings. Another category of detection is the individualized system, which usually uses videos, electromagnetic or radio signals, such as wireless fidelity (Wi-Fi), RFID tags, and Ultra-Wideband (UWB), to track a unique device. The advantages of these methods are validated and compared in the study conducted by (Taneja et al. 2012). Such detection systems are capable of providing higher resolution occupancy information on location, presence, counting, identification, and tracking (Chen et al. 2015).

Another aspect to consider when monitoring people is whether people are on the move or not. In this sense, one can talk about techniques that are capable of modeling static or dynamic objects (Teixeira 2010).

In this study, it is intended to determine the number of people in a meeting room. This will be done through direct counting or through detection, following that by processing the detection results to obtain people counting. With two motion sensors working together on either side of a door, it is possible to count the number of people entering and leaving a room. Thus, two methods of people counting will be studied, one using a pair of PIR sensors and another using a pair of ultrasonic sensors. The proposed detection method will use infrared technology and a Grid-EYE sensor (Grid-EYE 2020).

The solution that uses two PIR sensors

The sensor system used in this study is composed of a Raspberry Pi board, a multiplexing connection board and a pair of two identical PIR Motion Sensors (PIR 2020). PIR sensors allow you to sense motion, almost always used to detect whether a human has moved in or out of the sensors range. They are small, inexpensive, low-power, easy to use and do not wear out. For that reason, they are commonly found in appliances and gadgets used in homes or businesses.

The Raspberry Pi used is a Model 4, B (Raspberry Pi4 2019). It is a credit-card-sized single-board computer. The CPU on the board is an ARM v8 processor with 1.5 GHz speed. It has a variety of interfacing peripherals, including 2 USB 3.0 ports, 2 USB 2.0 ports, 2 micro-HDMI ports, 4 GB RAM, Micro-SD Card storage and 40 pin GPIO header. For GPIO port expansion, a multiplexing connection board is used.

Two monitors, keyboard, and mouse can be connected to Raspberry Pi through micro-HDMI and USB connectors and it can be used like a desktop computer. It supports a number of operating systems including a Debian-based Linux distro, Raspbian, which is used in our design. Raspberry Pi can be connected to a local area network through Ethernet cable or USB Wi-Fi adapter, and then it can be accessed through SSH remote login.

It is desired to count the persons in a room with two motion sensors working together on either side of a door. The laboratory mounting (on a desk) of the sensors simulates very well the sensors working together in the real situation, on either side of a door. In fig. 2 can be seen the image of the system used in this study and its electrical diagram.

Fig 2. Two PIR sensors and Raspberry PI: a) photo of the assembly; b) assembly scheme

In the schematic representation in fig. 2 b) the following connections are done: the 3-5VDC power pins of the PIR sensors are connected to the 5-volt power supply (2 and 4) of Raspberry PI. The Ground pins of PIR sensors are connected to GND (6 and 14) on Raspberry PI. The digital out (SDA) of one PIR sensor is connected to GPIO7 (11) on Raspberry PI and of the other SDA of the PIR sensor to GPIO17 (11). The calculation algorithm of people count is presented in table 1. A software written in Python was developed to implement these rules.

Table 1: Rules for determining people count using the information received from the two PIR sensors.

In Python code, to check the status of a PIR sensor, an infinite loop can be used (of course with a certain pause set between repetitions) in which it is checked if there has been any input at the sensor pin. But there is also the possibility in which the code is told to wait until an event occurs e.g., when the pin detects a signal. The advantage of the second approach is that processor resources (CPU) are not consumed by repeating a loop. In the case of this study, the variant of waiting to appear, for the event that represents the signal detected by the pin, was used.

Another aspect to be specified is that the PIR sensors used were too sensitive, so their adjustment was made according to the datasheet and the user manual (PIR 2020).

The solution that uses two ultrasonic sensors

This solution uses a Raspberry Pi, Model 4, B (Raspberry Pi4 2019), which was presented in the previous subchapter, a multiplexing connection board and a pair of two identical ultrasonic sensors (Ultrasonic 2020). Ultrasonic sensor works on the principle that is similar to RADAR and SONOR i.e., transmits a signal and analyzes the target by capturing the reflected signals. HC-SR05 is an ultrasonic ranging sensor with good accuracy. Ultrasonic sensors overcome many of the weaknesses of IR sensors – they provide distance measurement regardless of color and lighting of obstacles. They also provide lower minimum distances and wider angles of detection to guarantee that obstacles are not missed by a narrow sensor beam. It has 5 pins and can be used in 1-pin trigger/echo or 2-pin.

It is desired to count the persons in a room with two ultrasonic sensors working together on either side of a door. In this study, the ability to measure the distance that ultrasonic sensors have will not be used but only the capability that they can detect movement. The laboratory mounting (on a desk) of the sensors simulates very well the sensors working together in the real situation, on either side of a door. In fig. 3 can be seen the image of the system used in this study and its electrical diagram.

In the schematic representation in fig. 3 b) the following connections are done: the VCC power pins of the PIR sensors are connected to the 5-volt power supply (2 and 4) of Raspberry PI. The Ground pins of PIR sensors are connected, using a breadboard to GND (6) on Raspberry PI. The Trig pin of one sensor is connected to GPIO4(7) of Raspberry PI and the Trig pin of the other sensor to GPIO23(16) of Raspberry PI. It is realized, for each sensor, a voltage divider that is set up for the echo pin (it drops the voltage going to the GPIO pins down to 3.3v from 5v). 1k Ω resistor (R1) and a 2k Ω resistor (R2) are used to achieve this. Echo pin of the sensor is connected to R1, R2 is connected to Ground of Raspberry PI, and wire from R1 and R2 are connected to GPIO17(11) of Raspberry PI, respectively GPIO24(18) for the other sensor.

Fig 3. Two ultrasonic sensors and Raspberry PI: a) photo of the assembly; b) assembly scheme

Rules for determining people count using the information received from the two ultrasonic sensors are the same as rules used for PIR sensors. Regarding the Python code, to check the status of an ultrasonic sensor, the solution that checks if the sensor has detected something uses an infinite loop

The solution that uses a Grid-EYE sensor

This solution uses a Raspberry Pi, Model 4, B (Raspberry Pi4 2019), which was presented in the previous subchapter, a multiplexing connection board and Grid-EYE Infrared sensor (Grid-EYE 2020). Grid-EYE is a high precision infrared array sensor based on advanced MEMS technology. It has the feature of temperature detection of two-dimensional area: 8×8 (64 pixels), digital output (capability of temperature value output), and a compact package. Temperature range of measuring object is from 0 °C to 80 °C.

Fig 4. Grid-EYE sensor and Raspberry PI: a) photo of the assembly; b) photo of the assembly with the Grid-EYE sensor mounted in the ceiling; c) assembly scheme

In fig. 4 can be seen the image of the system used in this study, a photo of the assembly on the desk (a) and a photo with it in working position, mounted on the ceiling (b) and its electrical diagram (c).

In the schematic representation in fig. 4 c) the following connections are done: the VCC (3.3V) power pins of the Grid-EYE sensor are connected to the 3.3-volt power supply(1) of Raspberry PI. The Ground pin of Grid-EYE sensor is connected to GND(9) on Raspberry PI, the SDA pin of the sensor to GPIO2-SDA1(3) on Raspberry PI and the SDA pin of the sensor to GPIO3-SCL1(5) on Raspberry PI.

It is desired to count the persons in a room using this solution. First, Python code was used to display the 8×8 pixels grid given by the Grid-EYE sensor. Then a people count was performed based on the obtained image.

Python has been chosen as the programming language because of its vast data processing toolbox and the existence of an AMG88xx library. The AMG88xx library handles the I2C protocols and returns the temperature values of the 64 individual IR detectors (Hrisko, 2018). The pyplot interface was also used in Python. matplotlib.pyplot is a collection of functions that make matplotlib work like MATLAB. Each pyplot function makes some change to a figure: e.g., creates a figure, creates a plotting area in a figure, plots some lines in a plotting area, decorates the plot with labels, etc. (Matplotlib 2012).

In order to determine the number of people based on an infrared photo, OpenCV-Python (OpenCVTut 2013) was used. OpenCV-Python is the Python API of OpenCV. OpenCV was started at Intel in 1999 by Gary Bradsky and the first release came out in 2000. Right now, OpenCV supports a lot of algorithms related to Computer Vision and Machine Learning and it is expanding day-by-day (OpenCV 2013). Using the scripy.interpolate module, function griddata, that deals with the OpenCV-Python interface, and the solution proposed by (Keith 2018), a code that makes blob detection was developed in Python. A Blob is a group of connected pixels in an image that share some common property (e.g., grayscale value). OpenCV provides a convenient way to detect blobs and filter them based on different characteristics (Mallick 2015).

Python code for all three solutions can be downloaded from http://www.cladirieficiente.ro.

Results and Discussions

Regarding the first two methods of determining people count, using PIR sensors and ultrasonic sensors, the calculation algorithm is relatively simple. Following the tests performed, no errors occurred. However, here is a limitation with this approach: if two people enter a room at the same time, there is no way of detecting the two people apart. Or if people who enter and people who leave meet at the access door or other situations that are easy to imagine.

In this project, the display of the code in Python console was sufficient, in later stages, after choosing the optimal solution for determining the number of people, functions will be developed and, depending on the requirements, adequate methods will be found to make the API of the application on the Raspberry Pi PCB.

Fig 5. The image obtained with Grid-EYE: a) three people in the room; b) one person in the room; c) two people in the room (8×8 grid given by Grid-EYE, without interpolation)

In fig. 5 is shown the grid given by the Grid-EYE sensor. One can see the presence of three people (a), one person (b) and two people (c). The grid display can be done with or without interpolation, in fig (c) it is missing. Note that the people count algorithm works without the graphical display of the image (but uses it without displaying it). The display of the image is made to better understand what is happening and for didactic purposes.

In the images in fig. 5, the number of people in the room is quite clear, but not all the images are so clear. The result can be influenced by the distance people are from the sensor, if they are sitting or standing, if people are moving, it can also be influenced by various other heat sources.

Regarding the testing, there were some errors in this method, as expected for an information of 8×8 pixels. The calculation method and algorithm are very good and other sensors are recommended, such as FLIR Radiometric Lepton (FLIRCamera 2020) which has a relatively affordable price. It

should be noted that People Counting Software using Grid-EYE is developed by Panasonic (the sensor manufactures also) (Grid-EYECount 2017), but it is not an open-source software.

Conclusions

In the context of the exponential growth of IoT in the last period and of the need of FM to optimize its activity based on the growth of information received from the field, there was a need to count the presence in the meeting rooms of various companies. Thus, the present paper fulfills a requirement that is part of a POC project through which it is desired to monitor the presence and light intensity in two meeting rooms of a customer. The requirement is to establish the optimal method of people counting in the meeting rooms.

Two methods of people counting are studied, one using a pair of PIR sensors and another using a pair of ultrasonic sensors and a detection method, using infrared technology by using a Grid-EYE sensor, then establishing the people count based on the information received by the Grid-EYE sensor.

In the first two methods, there are no reading errors due to the sensors, however certain situations that exceed the limit of the possibilities of people counting through this system may occur, such as the simultaneous entry or exit of two people. Furthermore, in case of monitoring the presence in the meeting rooms, attention must be paid to the cumulation of possible errors that would occur for people counting using PIR and ultrasound sensors. That is, the wrong counting of an entrance or exit will be perpetuated at the next meetings; the system has no way to reset (to set the number of people to zero) at the beginning of a session. Although this would not be a problem from a technical point of view, from a business point of view, it would be a mistake (it is possible that the meeting started earlier, at another time, for example). A system reset at the end of a working day is not enough.

In the case of determining people count using Grid-EYE, there were sometimes noticed differences in counting compared to the number of people actually present in the room. This may be due to the low resolution of the information sent by the Grid-EYE sensor, or a grid of only 8×8 pixels. In this situation, higher resolution sensors are recommended.

In this paper, people counting is considered in relation to the entire trace of requirements it satisfies. For example, the level of accuracy that is required in an FM application that deals with Rooms Reservation. A very high degree of accuracy would be needed if, depending on the people count, decisions could be made such as canceling a meeting. But if there is interest in the evolution over time of the occupancy rate of the meeting rooms (real number of participants vs. planned number of participants and room capacity), on the basis of which statistical evaluations would be made, then small deviations from the people count could be admitted.

Acknowledgment

This article benefits from funding from the University POLITEHNICA of Bucharest, through the project “Engineer in Europe” online, registered at MEC under no. 457 / GP / 06.08.2020, by using the fund for financing special situations that cannot be integrated in the form of financing state higher education institutions.

References

• Al-Fuqaha A., Guizani M., Mohammadi M., Aledhari M. and Ayyash M. (2015), Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications, in IEEE Communications Surveys & Tutorials, 17(4), 2347-2376, Fourthquarter, doi: 10.1109/COMST.2015.2444095.
• BABOK (2015), A Guide to the Business Analysis Body of Knowledge, IIBA (International Institute of Business Analysis, Toronto, Ontario, Canada, version 3.0, 2015, ISBN-13: 97978-1-927584-03-3.
• Chen K., Lu W., Peng Y., Rowlinson S., Huang G. (2015), Bridging BIM and building: From a literature review to an integrated conceptual framework, International Journal of Project Management, 33(6), 1405-1416.
• Chitaru, Berville C. and Dogeanu A. (2018), Numerical simulation and comparison of two ventilation methods for a restaurant – displacement vs mixed flow ventilation, E3S Web of Conferences 32, 01012, https://doi.org/10.1051/e3sconf/20183201012.
• Diodiu R. and Dogeanu A. (2016), Development and validation of an analytical method for quantitative determination of carboxylic acids in air samplers, Sustainable Solutions for Energy and Environment, Energy Procedia 85, 201 – 205, doi: 10.1016/j.egypro.2015.12.217.
• EA (2016), Enterprise Architect Version 13 – visual modelling and design tool, SPARK Systems, [accessed September 2020], https://community.sparxsystems.com/news/1002-sparx-systems-releases-enterprise-architect-13.
• EAReqM (2016), Requirements Management with Enterprise Architect 13, SPARK Systems, Visual Modelling Tool, Requirements_Management_in_Enterprise_Architect-2.pdf, https://www.sparxsystems.com/resources/webinar/requirements-management/ea13/requirements-in-enterprise-architect.html, [accessed September 2020].
• EuroFM (2019), European Facility Management Network, [accessed September 2020], http://www.eurofm.org/index.php/what-is-fm.
• FLIRCamera (2020), FLIR Radiometric Lepton Thermal Camera Breakout, SparkFun Electronics, https://www.sparkfun.com/products/15948, [accessed September 2020].
• Gerland H. and McDonald I. (2016), bCounted: Automatic People Counting and People Flow Management at Airports, 15th International Conference on Automated People Movers and Automated Transit Systems, April 17–20, 2016 | Toronto, Ontario, Canada, https://doi.org/10.1061/9780784479797.007.
• Grid-EYE (2020), SparkFun Grid-EYE Infrared Array Breakout – AMG8833 (Qwiic), https://www.sparkfun.com/products/14607, [accessed September 2020].
• Grid-EYECount (2017), Grid-EYE People Counting Software Manual, https://eu.industrial.panasonic.com, 20170314_grid-eye_people_counting_software_manual-1.pdf, [accessed September 2020].
• Hrisko J. (2018) Heat Mapping with a 64-Pixel Infrared Detector (AMG8833), Raspberry Pi, and Python, MakerPortal, posted on 2018, https://makersportal.com/blog/2018/1/25/heat-mapping-with-a-64-pixel-infrared-sensor-and-raspberry-pi, [accessed September 2020].
• Keith R. (2018), Environmental Monitoring and Presence Detection with Helium, blog helium, posted on 2018, https://blog.helium.com/environmental-monitoring-and-presence-detection-with-helium-4db48d95fceb, [accessed September 2020].
• Malatras A., Asgari A. and BaugÉ T. (2008), Web Enabled Wireless Sensor Networks for Facilities Management, in IEEE Systems Journal, 2 (4) pp. 500-512, doi: 10.1109/JSYST.2008.2007815.
• Mallick S. (2015), Tutorial – Blob Detection Using OpenCV ( Python, C++ ), https://www.learnopencv.com/blob-detection-using-opencv-python-c/, posted on 2015, [accessed September 2020].
• Matplotlib (2012), Matplotlib Version 3.3.2, Pyplot tutorial, https://matplotlib.org/tutorials/introductory/pyplot.html, [accessed September 2020].
• Natephra W., Yabuki N. and Fukuda T. (2018), Optimizing the evaluation of building envelope design for thermal performance using a BIM-based overall thermal transfer value calculation, Building and Environment, 136, 128-145, ISSN 0360-1323, https://doi.org/10.1016/j.buildenv.2018.03.032.
• Nord J. H., Koohang A., Paliszkiewicz J. (2019), The Internet of Things: Review and theoretical framework, Expert Systems With Applications, 133 97–108, doi: https://doi.org/10.1016/j.eswa.2019.05.014.
• OpenCV (2020), Open Source Computer Vision Library [OpenCV], https://opencv.org/about/, [accessed September 2020].
• OpenCVTut (2013), OpenCV – Python Tutorials, Introduction to OpenCV-Python Tutorials, https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_setup/py_intro/py_intro.html, [accessed September 2020].
• PIR (2020), PIR Motion Sensor Datasheet, [accessed September 2020], https://datasheetspdf.com/pdf/1418686/Adafruit/PIR_Motion_Sensor/1.
• Pop O. G., Abrudan A. C., Dogeanu A. M., Pocola A. G., Tutunaru L. F. and Balan M. C. (2018), Dynamic thermal modeling of buildings and application to a hospital, 2018 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR), Cluj-Napoca, Romania, 2018, volume 1, pp. 1-6, doi: 10.1109/AQTR.2018.8402769.
• Raspberry Pi4 (2019), desktop computer, [accessed September 2020], https://www.raspberrypi.org/products/raspberry-pi-4-model-b/.
• Sporr A., Zucker G. and Hofmann R. (2019), Automated HVAC Control Creation Based on Building Information Modeling (BIM): Ventilation System, IEEE Access, 7, 74747-74758, 2019, doi: 10.1109/ACCESS.2019.2919262.
• Sruthi M. S. (2019), IoT based real time people counting system for smart buildings, International Journal of Emerging Technology and Innovative Engineering, 5(2), available at SSRN: https://ssrn.com/abstract=3340446.
• Taneja S., Akcamete A., Akinci B., Garrett J.H., Soibelman L., East E.W. (2012), Analysis of three indoor localization technologies for supporting operations and maintenance field tasks, J. Comput. Civ. Eng. 26, 708–719, https://doi.org/10.1061/(ASCE)CP.1943-5487.0000177.
• Teixeira , Dublon G., Savvides A. (2010), A Survey of Human-Sensing:Methods for Detecting Presence, Count,Location, Track, and Identity, ENALAB Technical Report 09-2010, Vol. 1, No. 1.
• Udrea I., Kraus V.F. and Popescu-Cuta A. (2020), Facility Management Platform Sensors Monitoring for Improving Workplace Services, accepted paper at IBIMA 2020 Conference – online.
• Ultrasonic (2020), HC-SR05 / HY-SRF05 Precision Ultrasonic Sensor, https://www.tindie.com/products/upgradeindustries/hc-sr05-hy-srf05-precision-ultrasonic-sensor/, [accessed September 2020].
• Ye Q. (2010), A robust method for counting people in complex indoor spaces, 2010 2nd International Conference on Education Technology and Computer, Shanghai, 2010, pp. V2-450-V2-454, doi: 10.1109/ICETC.2010.5529346.