The Design and Implementation of Mobile Application Solution for Forest Fire based on Drone Photography and Amazon Web Service (AWS)

Abstract

Last year's Goseong-Sokcho forest fires have highlighted the limitations of extinguishing work for night-time forest fire and the importance of quick identification for information on the spread of forest fire. However, it is not easy to find services that take into account the characteristics of forest fires, as most existing disaster-related mobile applications and research assume various disaster situations rather than just forest fire situations. Therefore, a system that can provide information quickly is needed, taking into account the characteristics of forest fires and the limitations of extinguishing work. In this paper, we propose evacuation route guidance services that bypass areas where fire has already spread, supplement existing methods of extinguishing work, and provide general information on forest fire situations in real time, by putting drones into forest fire situations. It has been implemented to automate image analysis using the Rekognition service of Amazon Web Service (AWS), and the results of fire detection and the T Map API guide the evacuation path. It is expected that the results of this paper will allow efficient and rapid rescue and extinguishing work to be carried out, and further reduce the damage of human life caused by forest fires.

1. Introduction

Forest fires occur a lot in dry times, and in Korea, they occur a lot in spring and fall. Forest fires spread to the direction of the wind, which can rapidly speed up the spread depending on the wind speed and surrounding conditions. Therefore, if forest fires spread to residential areas, the damage of human life and property will quickly increase. Therefore, it is very important to check the spread of forest fires in real time and quickly inform residents near the site of the forest fire. However, due to the nature of forest fires, it is not easy to grasp all the situations at a fast pace of spread.

In the case of the Goseong-Sokcho forest fire on April 4, 2019, 1,757 hectares of forest were destroyed and about 900 houses and facilities were damaged. At the time of the forest fire in Goseong-Sokcho, the maximum wind speed reached 30m/s, causing forest fires to spread rapidly to various regions, and the fire broke out at night, which caused many difficulties in extinguishing the fire. Night-time forest fires require a "helicopter for extinguishing the night-time forest fire" that allows pilots to gain visibility, not a regular helicopter. However, few fire departments currently have the night-time forest fire extinguishing helicopters. This means that it is difficult for regular helicopters for fire to be deployed if a forest fire broke out at night. However, whether or not to deploy helicopters has a significant impact on the speed of the spread of forest fires.

In this paper, we would like to utilize drones by taking into account the characteristics of forest fires and the limitations of the current extinguishing work. Drones can fly at night and record the site situation in real time. Thus, drones can collect information of current situation and spread regardless of when a forest fire occurs. This information can be used to supplement the limitations of forest fire extinguishing work at night, and the collected information can be used as analysis data after extinguishing work. The fire department can efficiently deploy firemen considering the direction and speed of the spread of forest fires, and residents nearby can evacuate safely based on information about the current situation. In the event of a forest fire in Goseong-Sokcho, the fire spread to downtown of the city, filling the road with fire trucks. In this situation, if there is a service that excludes areas where the fire has already spread and guides the evacuation route, we can evacuate more quickly and even expect to reduce the damage of human life. [1]

Therefore, in this paper, we propose 'Sanbul Alimi,' an application that collects information on forest fires through drones and guides them on site situations and evacuation routes based on them, and helps nearby residents make quick rescue requests.

The reminder of the paper is organized as follows. Section 2 introduces the related works on mobile application for the event of a disaster. Section 3 addresses the application development for ‘Sanbul Alimi. Finally, Section 4 presents the conclusions.

2. Related Works

An example of an existing mobile application that guides the optimal evacuation route in the event of a disaster is '2e SEMS'. This application, launched by company "2e Solution", is a real-time evacuation route guidance solution based on the Bluetooth Beacon. In addition, '2e SEMS' provides facility guidance and facility notice guidance. This application mainly focuses on identifying users' locations and guiding optimal evacuation routes to nearby exits based on indoor positioning technology using beacons in case of emergency situations such as fire in multi-use facilities. However, there is a limit to the need to install beacons in advance and to incur additional costs due to installation. [2]

Another related application is "Safety Stepping Stones." The application is a disaster safety portal application supported by government that provides various disaster-related information. It provides information on natural disaster such as earthquakes, landslides, typhoons, etc., facility information such as shelter and emergency medical center, daily safety rules and behavioral tips, and provides disaster news and weather information. They can also make an emergency call to 119 via this application, such as the 'Sanbul Alimi' application, which means forest fire alarm. The "Safety Stepping Stone" provides behavioral tips on fire situations, but does not provide features specific to forest fire situations, such as evacuation tips, information on situations, and guidance on evacuation routes. [3] The following are examples of existing studies on evacuation and bypass route guidance in the event of a disaster. One study developed an algorithm that provides evacuation and bypass routes for vehicles in emergency situations. The paper presented evacuation and bypass route algorithms based on dynamic shortest route search techniques and dynamic traffic assignment models, and later evaluated the performance of the algorithms by applying a hypothetical scenario called "terrorist attacks." However, the study focused on vehicle simulation, so public transportation and pedestrians were not considered, and is characterized by guiding the route mainly on the road. [4]

Another related study developed a bypass and evacuation route guidance system to prevent secondary damage that could occur during a natural disaster attempt to enter a disaster area due to a lack of information. The study introduced a mobile application that analyzed the route and evacuation route that bypassed the disaster area when the manager entered the disaster area and provided it to the user. However, the paper featured that it was difficult to provide real-time information because the manager had to enter the disaster information directly. [5]

In addition, a variety of sensors are applied to research on systems that guide evacuation routes in case of an emergency of indoors and to guide real-time evacuation routes in the direction of the least cumulative damage by using information collected from sensors through mobile applications. [6][7]

Various studies are currently under way because the damage caused by forest fires, as well as general fires, is too great. In addition to efforts to improve the function of the fire detection system applied to CCTVs, research is being conducted on low-cost fire monitoring systems using thermal imaging cameras and smartphones. Research and development of a real-time system for fire prevention in a wider area for forest fire monitoring is also underway. [8][9]

This paper guides the evacuation route bypassing the spread area of forest fire as in the existing study, but both way information is collected on the disaster-causing area and data is entered on the current situation, are different from the existing one. The "Sanbul Alimi" consists of receiving information on the situation by analyzing drone aerial photographs taken in real time using the Rekognition service of Amazon Web Service, rather than by an administrator entering disaster areas. Existing studies also focus on assuming various disaster situations or indoor fires rather than guiding evacuation routes, taking into account the specificity of forest fire situations. However, in this paper, we introduce 'Sanbul Alimi', an application that assumes forest fire situations and provides various functions considering the characteristics of the situation.

3. Development of ‘Sanbul Alimi’ Application

3.1 Overview of Application Development

According to the news of the recent frequent forest fires, the need for solutions is being highlighted, that would help to quickly inform nearby residents in the event of a forest fire and to help them evacuate safely, and make requests for rescue easier. Forest fires can change the pace of progress rapidly, and it is not easy to accurately identify these changes in real time. Considering the characteristics of these forest fires, we develop the 'Sanbul Alimi' application to help users make evacuation and rescue requests efficiently and quickly by organizing of the application with the most necessary functions in the forest fire situation.

3.2 System Configuration

The system configuration of the 'Sanbul Alimi' is as shown in Figure 1. First, the photos taken in real time in the forest fire area are sent to Amazon S3 for storage. Photos stored in the specified S3 Bucket are analyzed using Rekognition service of AWS on the server. This service detects whether objects related to word 'fire' exist within the photo and stores these result values in a JSON file on the server. The analysis file for each image is sent to the "Sanbul Alimi" application, and if objects related to word ‘fire’ are found, the location of fire in the photo is displayed using the T Map API.

(Figure 1) System Configuration Diagram

3.3 Screen Configuration and Features

3.3.1 Main Page

The main page that appears first when you launch the "Sanbul Alimi" application is like Figure 2. The main menu consists of 'SOS', 'Injury/Isolation', 'Evacuation Route Guidance', 'Current Situation', 'Emergency Procedures Manual', and 'Emergency Shelters', and has a total of six menus. At the top of the main menu there is a navigation bar, and in the middle of this page there is a map indicating the user's current location and information about mountains within a 5-kilometer radius of the user. The bottom of the main page shows real-time weather information such as temperature and wind speed based on the current location.

(Figure 2) Home Screen of ‘Sanbul Alimi’

3.3.2 ‘Current Situation’ Menu

The "Current Situation" menu provides users with real-time information on the overall situation of forest fires. This page shows a map based on the user's current location, and if a mountain near the user is on fire, it is also displayed on the map. The flowchart of the function is as shown in Figure 3.

(Figure 3) Flowchart for ‘Current Situation’

When the user clicks this menu, the server first identifies the existence of the analysis result files generated within 24 hours for the mountain within a 5-kilometer radius of the user's current location. If the analysis shows that the file does not exist, it means that there was no forest fire, so it shows a map based on the current location information. However, if the result file exists, it means that the mountain caught fire within 24 hours. Therefore, in this case, the most recently created files are reviewed to determine if objects such as 'Fire', 'Mountain Fire', and 'Forest Fire' have been detected. If an object related to fire is detected, the location information of the photo is sent to the application and displayed on the map.

3.3.3 ‘Evacuation Route Guidance’ Menu

The "Evacuation Route Guidance" menu provides a function to guide safe evacuation routes, excluding areas where forest fires have occurred. The flowchart of the function can be described in Figure 4. Figure 5.a appears when the user clicks on the 'Evacuation Route Guidance' menu. The default starting point for route guidance is the user's current location, and the default destination is set to the nearest shelter. The origin and destination can be changed according to the user's situation, and by pressing the 'Start' button, the shortest route between the destination and the destination is explored. It is required whether there is an analysis result file with location information included in the explored route, for the purpose of guiding the safe evacuation route rather than simply guiding the shortest route. If a fire is detected within the explored path, the area is excluded and the shortest path is explored again. This process is repeated and then this menu starts to guide the appropriate path if the analysis result file corresponding to the discovered path location does not exist or if an object related to fire is not found within the result file. Figure 5.b shows that a safe route is guided, linked to T Map application.

(Figure 4) Flowchart for Evacuation Route Guidance​​​​​​​

(Figure 5) Screen for Evacuation Route Guidance and Screen linked to T Map Application​​​​​​​

3.3.4 Other Menus

If you click the SOS menu among the main menus, you can call 119 without shutting down the application. The 'Injury / Isolation' menu is also useful in situations of injury or isolation, and can be used to make rescue requests in situations where phone connections are difficult. Figure 6 is the screen when you click on this menu. In the event of an injury to the user, a brief description of the extent and status of the injury can be sent to 119, along with location information. If the user is not injured but is isolated, a rescue request can be made by sending the user's current location (latitude, longitude) via text message to 119.

(Figure 6) Screen for ‘Injury/Isolation’​​​​​​​

The "Emergency Shelters" menu is a menu that allows you to check the list of shelters, and you can check the location of designated shelters and nearby shelters by region. Finally, the 'Emergency Procedures Manual' menu provides action tips on the various situations that can occur in the event of a forest fire. [10]

3.4 Implementation for Object Detection Feature related to ‘fire’

The core of the "Sanbul Alimi" application is to identify the current situation based on aerial photographs taken by drones in real time. This process does not involve a person finding a fire in a photo, but rather automating image analysis with the Rekognition service of AWS, a pre-learned deep learning service. Figure 7 is a part of the function 'detect_fire' that automatically detect objects related to fire. [11] This functions to detect objects in aerial photographs stored in Amazon S3 Bucket. The value of the result variable is set to True if the objects associated with the fire, i.e. 'Fire', 'Flame', 'Forest Fire', 'Mountain Fire' and 'Bonfire' are presented in a given image.

(Figure 7) Code for fire detection​​​​​​​

Figure 8 is an example of analysis result file for an aerial photograph, indicating the value and filename of the result variable, the label of the object detected within that photograph, and the reliability in terms of the accuracy of each label.

(Figure 8) Example for result file of fire detection​​​​​​​

3.5 Development Environment and Tool used

This research was conducted using the Rekognition service of AWS, Amazon S3 and Python, Android Studio, and T Map APIs from AWS.

S3 is an object storage service that can easily store and retrieve the desired amount of data, and was used to store pictures taken by drones. The Rekognition service is a pre-learned deep running model that can be used for image and video analysis, and is a service that provides functions such as object detection, text detection, face detection and analysis, face search and verification, celebrity recognition, etc. Python 3.7 was used in PyCharm 2019.1.2 IDE to utilize the Rekognition service. The overall ‘Sanbul Alimi,’ application is developed, using Android studio 3.5.2 and T Map API is used for the map based function in application and the function of evacuation route guidance is implemented to guide, while linking to T Map application.

4. Conclusions

In this paper, to improve the limitations of night-time forest fire extinguishing method and the difficulty of real-time identification of spread situation for forest fire, we propose a 'Sanbul Alimi' application that provides situation information based on drone aerial photographs and the Rekognition service of AWS. Unlike previous studies that guide detour and evacuation routes in disaster situations, this study collects real-time situation and spread information by deploying drones into forest fire-causing areas, based on this, guiding users to the evacuation route. In addition, it provides various functions necessary in forest fire situations, such as SOS, Evacuation Shelter, Emergency Procedure Manual, and Injury/Isolation. The fire department can use the results of this study to deploy firemen more efficiently, and to quickly identify the condition and location of injured or isolated people and rescue.

The ‘Sanbul Alimi’ application, developed so far, is a prototype that will complement its function and stability through future research. Additional experiments will be conducted to analyze each process so that everything from photographing a large area through drones to providing various information to users will be carried out smoothly, and based on this, we will supplement and implement additional deficiencies in each function.

☆ A preliminary version of this paper was presented at ICONI 2019 and was selected as an outstanding paper.

☆ This work was supported by a research grant from Seoul Women’s University (2020)