[Smart Building x AI] 3 examples of AI utilization for social issues in smart buildings

● Recommended for: ●

・ Those who want to know AI cases that can be applied to smart buildings
・ Those who want to know the AI architecture announced with the spread of new coronavirus infection
・ Those who are interested in virtual safety training

Time needed to finish reading this article

5 minutes

Introduction

The growth speed of the smart building market scale is now on par with the speed of technological evolution of AI and IoT. This is because the more advanced AI and IoT technologies are, the more they can be applied to various areas of smart buildings.

Furthermore, at present, voluntary activities to achieve the SDGs and responses to social issues related to the COVID-19 are required, and there are many research reports on AI and technology that address these social issues. You can

In this article, we will introduce case studies and research that provide hints on how to utilize technology in various areas of smart buildings.

Human Behavior Analysis Using AI Applied to COVID-19

ソーシャルディスタンス検出

With the outbreak of the novel coronavirus infection (COVID-19), social distancing has been cited as a means to prevent the spread of infection, in addition to wearing masks. To prevent the spread of the virus, it is important to maintain social distancing in public areas such as train stations, shopping malls, and university campuses.

A paper, A Vision-Based Social Distancing and Critical Density Detection System for COVID-19, submitted by Ohio State University, explores critical pedestrian density for detecting social distancing conditions and regulating influx into congested areas. We propose a warning system that can identify a threshold for .

Source: A Vision-Based Social Distancing and Critical Density Detection System for COVID-19
Caption: Figure 1. Overview of the proposed system.
https://www.mdpi.com/1424-8220/21/13/4608/htm

Human detection is performed based on the video taken by the camera, and the human detection area such as a bird's eye view is generated from 3D to a 2D plane, and the position of the person is plotted on the coordinates of the 2D plane to obtain the person's position. .
Then, the distance between people is obtained from each person's position, and the social density is defined from the distance between people. is raised on a two-dimensional plane, and the position information of the person is also plotted on the two-dimensional plane (right figure).

Source: A Vision-Based Social Distancing and Critical Density Detection System for COVID-19
Caption: Figure 3. Illustration of pedestrian detection using Faster R-CNN [13] and the corresponding social distancing.
https://www.mdpi.com/1424-8220/21/13/4608/htm

In today's Japan, there are still scenes where social distancing is required. In addition to the needs for COVID-19-19 countermeasures, this technology can also be used for density monitoring and prediction in commercial buildings and event facilities. It may even help keep comfort in smart buildings.

Detection of good or bad mask wearing

Earlier, I introduced a system implementation proposal for social distancing. With the spread of the COVID-19, the number of public facilities that require wearing masks as well as social distancing has increased. However, it's not just whether you're wearing a mask or not, it's actually very important to wear a mask correctly to prevent infection.

Therefore, this time, we will introduce the AI model "RetinaFaceMask" of the mask wearing detector announced at IEEE 2021, an academic conference on system theory and application and AI technology, which can detect no mask wearing, correct mask wearing, and incorrect mask wearing. We will introduce a research example that created a dataset for detecting more realistic mask wearing conditions.

Source: RetinaFaceMask: A Single Stage Face Mask Detector for Assisting Control of the COVID-19 Pandemic
Caption: Fig. 2. Architecture of RetinaFaceMask.
https://arxiv.org/pdf/2005.03950.pdf

RetinaFaceMask uses a powerful feature extraction network as its backbone (backbone) to support various scenes in mask detection. Furthermore, compared to face-only detection, mask detection requires both face location and mask wearing status identification. By using a unique block called CAM to give higher identifiability, it is possible to distinguish whether the mask is worn.

In this research, we created a dataset called MAFA-FMD based on the dataset called MAFA. MAFA-FMD is data that categorizes not only “no mask wearing” and “mask wearing”, but also “correct mask wearing” and “wrong mask wearing” among mask wearing. Using the publicly available data set AIZOO and the MAFA-FMD data set, the result of predicting the wearing state of the mask shows that "wrong mask wearing" is also detected with high accuracy.

Source: RetinaFaceMask: A Single Stage Face Mask Detector for Assisting Control of the COVID-19 Pandemic
Caption: Fig. 4. Qualitative Results on AIZOO and MAFA-FMD Datasets.
https://arxiv.org/pdf/2005.03950.pdf

I think it will continue to be recommended to wear a mask correctly during events, etc., but this technology, which does not require human hands for confirmation at that time and can also detect people walking, will be used and applied. This will bring many benefits in the future.

Work improvement by visualizing construction progress

Understanding the construction progress of building construction is very important in construction management according to the construction schedule. However, measuring construction progress is often a manual process that imposes a manual, time-consuming, error-prone, and labor-intensive process.

Therefore, we would like to introduce an example of using AI to monitor construction progress.

Schedule management with 3D data using drones and GIS

As many of you may know, drones are unmanned aircraft that can be flown by remote control or automatic control, and there are various types from toys to industrial and military use. GIS (Geographic Information System) refers to a geographic information system, and based on digital data of geographic information such as digital maps, editing such as superimposing road and building data on top of it, and such various data can be analyzed and managed on the system. By using GIS, it is possible to associate and visualize geographical information and various types of information.

"BUILDING CONSTRUCTION PROGRESS MONITORING USING UNMANNED AERIAL SYSTEM (UAS), LOW-COST PHOTOGRAMMETRY, AND GEOGRAPHIC" is a practical paper that can be used as a reference when systematizing construction schedule management using these drones and GIS. INFORMATION SYSTEM (GIS)" is a progress monitoring method in which many ingenuity can be seen in the process of creating 3D building data from image processing.

The proposed method uses clear images, but not distorted or blurred images, from videos acquired from drones. Select two images from the images acquired by the drone, and automatically detect matching features using software such as VisualSFM that creates 3D shape data.

Source: BUILDING CONSTRUCTION PROGRESS MONITORING USING UNMANNED AERIAL SYSTEM (UAS), LOW-COST PHOTOGRAMMETRY, AND GEOGRAPHIC INFORMATION SYSTEM (GIS)
Caption: Figure 5. Image to image matching using SIFT algorithm
https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-2/41/2018/isprs-annals-IV-2-41-2018.pdf

It keeps the detected matching features as keypoint data and finally creates a 3D model like below. Then, the CAD drawings of the building to be constructed are converted into 3D models using GIS. By comparing the 3D data acquired and created from this drone with the 3D data of the building to be constructed, it will be possible to monitor the progress of the construction work.

Source: BUILDING CONSTRUCTION PROGRESS MONITORING USING UNMANNED AERIAL SYSTEM (UAS), LOW-COST PHOTOGRAMMETRY, AND GEOGRAPHIC INFORMATION SYSTEM (GIS)
Caption: Figure 8. Processed LAS files for flight observations Mar. 05,2017, Figure 9. Isometric View of the As-Planned Model of UPCA
(walls - yellow, columns - red, beams - green, slab/floor - gray).
https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-2/41/2018/isprs-annals-IV-2-41-2018.pdf

As mentioned above, this technology can reduce man-hours for manually grasping the progress, and during construction work, it is necessary to keep track of the schedule of nearby facilities and the environment and obtain consent. Environmental risks can be reduced.

Summary

We introduced three papers based on technology utilization for social issues. The last technology is also an example of utilizing the recently popular Metaverse for human resource development. In the past few years, the use of AI has taken root in Japanese society, and now it has become commonplace for AI to be included as one of the elements that create these new technologies.
I would like to continue to send out various information so that it can be used for issues that are more relevant to the times!

■ Sources of content and papers introduced on this page / References

Dongfang Yang, Ekim Yurtsever, Vishnu Renganathan , Keith A. Redmill and Ümit Özgüner, “A Vision-Based Social Distancing and Critical Density Detection System for COVID-19”, Figure 1.:Overview of the proposed system., Figure 3.: Illustration of pedestrian detection using Faster R-CNN [13] and the corresponding social distancing.,
https://www.mdpi.com/1424-8220/21/13/4608/htm

Xinqi Fan, Mingjie Jiang, “Retina Face Mask: A Single Stage Face Mask Detector for
Assisting Control of the COVID-19 Pandemic”,Fig. 2. Architecture of RetinaFaceMask.,Fig. 4. Qualitative Results on AIZOO and MAFA-FMD Datasets.,
https://arxiv.org/pdf/2005.03950.pdf

Jezekiel R. Bognot, Christian G. Candido, Ariel C. Blanco, and Jore Rene Y. Montelibano, “BUILDING CONSTRUCTION PROGRESS MONITORING USING UNMANNED AERIAL SYSTEM (UAS), LOW-COST PHOTOGRAMMETRY, AND GEOGRAPHIC INFORMATION SYSTEM (GIS)”, Figure 5. Image to image matching using SIFT algorithm,Figure 8. Processed LAS files for flight observations Mar. 05,2017, Figure 9. Isometric View of the As-Planned Model of UPCA
(walls - yellow, columns - red, beams - green, slab/floor - gray).,
https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-2/41/2018/isprs-annals-IV-2-41-2018.pdf