| Student: | N. Zhang |
|---|---|
| Timeline: | December 2019 - 1 December 2023 |
My Ph.D. project is a part of the EU Horizon 2020 project INGENIOUS (https://ingenious-first-responders.eu/), which aims at designing novel devices to help First Responders (FR) improve their work efficiency. UAV (Unmanned Aerial Vehicles) have shown to be a valuable instrument to inspect indoor spaces, such as damaged buildings. The images collected by these platforms are useful to understand the environment, detect relevant details such as the presence of possible hazards and localize trapped people before First Responders (FR) enter inside. So far, this useful information needs to be extracted by human operators from long image and video sequences, requesting an additional effort for the rescue team on the field. In this regard, an automated procedure to automatically extract the useful information would make the use of UAV more effective, giving a tangible support to FR’s prompt action.
My research focuses on designing situational awareness algorithms using cutting-edge artificial intelligent (AI) algorithms. The images captured by camera sensors on the UAV are processed by deep learning algorithms to provide high-level semantic map and autonomously interpret the scenes selecting useful information for First Responders [1, 2]. A semantic map (Figure 1) is a visual representation of the scene where each pixel is classified into one of a pre-defined set of object classes. From a semantic map, we can easily extract the information on the presence of certain objects and allow First Responders to focus on specific classes they are interested in.
Figure 1. The semantic segmentation map corresponding to a picture.
Besides, I design a victim detection algorithm [3, 4], which is able to detect partially buried victims (Figure 2). After a house collapses or an earthquake, autonomous UAV is used for detecting victims automatically. To solve the shortage of real training data I devise a novel method to generate harmonious composite victim images.
Figure 2. Visualization of the victim detection algorithm.
References:
1. Ning Zhang*, Francesco Nex, Norman Kerle, George Vosselman. ”LISU: Low-light indoor scene understanding with joint learning of reflectance restoration.” ISPRS Journal of Photogrammetry and Remote Sensing, 2022.
2. Ning Zhang*, Francesco Nex, Norman Kerle, George Vosselman. ”Towards Learning Low-Light Indoor Semantic Segmentation with Illumination-Invariant Features.” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.
3. Ning Zhang*, Francesco Nex, George Vosselman, Norman Kerle. ”Training a Disaster Victim Detection Network for UAV Search and Rescue Using Harmonious Composite Images.” Remote Sensing, 2022.
4. Ning Zhang*, Francesco Nex, George Vosselman, Norman Kerle. ”Unsupervised harmonious image composition for disaster victim detection.” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.
Meet the team
