PhD Defence Caroline Gevaert

unmanned aerial vehicle mapping for settlement upgrading

Caroline Gevaert is a PhD student in the department of Earth Observation Science. Her supervisors are M.G. Vosselman and prof.dr. R.V. Sliuzas from the faculty of Geo-Information Science and Earth Observation.

Informal settlements, or slums, are considered to be one of the major development challenges of our time. One of the major obstacles for slum upgrading projects is the lack of data regarding current slum conditions such as the existing housing situation, accessibility through road networks, and hazardous environments. Informal settlements are often literally and symbolically “empty spots on the map”. Unmanned Aerial Vehicles (UAVs) are capable of providing imagery at a higher resolution but lower cost than imagery from satellites or manned aircraft. The present research explores the use of this exciting new technology to help fill these gaps on the map. The work entails an exploration of how to tailor machine learning methods to the characteristics of UAV data and ensure their performance despite the challenging characteristics of informal settlements. By working in the field together with actual informal settlement upgrading projects, it is also possible to investigate how well UAVs match the practical needs of upgrading projects and understand its societal impact.

The first part focusses on the use of machine learning methods. For example, supervised classification methods can be used to recognize patterns in data from some labeled training samples, enabling a class label to be assigned to new data. The first step in supervised classification is usually to define relevant features to describe the samples. The first objective aimed to identify synergies between 2D and 3D data provided by UAVs. Experiments using UAV data of unplanned settlements in Kigali, Rwanda and Maldonado, Uruguay indicated that buildings, roads, vegetation, structures and clutter could be discriminated with accuracies over 90% when combining 2D features from the imagery with 3D features from the point cloud.

In recognition of the statistical differences between 2D and 3D features, the next step aimed to adapt supervised classification methods to deal with heterogeneous data. Support Vector Machines (SVMs) are a successful machine learning method but generally use a single kernel to describe the non-linear similarity between training samples. Multiple Kernel Learning, however, uses different kernels for different feature groups which allows it to identify more subtle similarities and differences between samples. This manuscript presents an algorithm which can automatically group the features and provide tailored kernel parameters. Experiments show that the proposed MKL method achieves higher accuracies than conventional single-kernel methods applied to the 2D and 3D features while requiring less user-interaction than previous MKL methods.

As supervised classification methods are improving, obtaining training samples is proving to be a bottleneck as it generally requires much manual work. Therefore, research was done to analyze how reliable training labels can be obtained from existing geospatial data. Translating existing maps into training labels for newly-acquired UAV data will introduce errors due to (1) changes in the scene itself such as building constructions or demolitions, or (2) misalignments due to digitization at a lower spatial resolution or other geo-referencing issues. Experiments demonstrate the effectiveness of using these ‘noisy’ labels to train a classifier, remove samples with unreliable labels based on local and global contextual cues, train another classifier, etc. in an iterative process. An accuracy of above 90% could be obtained even if 30% of the initial training samples were mislabeled. This method can easily be applied to classify recurrent UAV imagery in projects which require frequent data coverage and check/improve the quality of manual digitization campaigns.

In addition to maps, detailed elevation models are important sources of information to support urban upgrading projects. Overlapping aerial images can provide a Digital Surface Model (DSM) which describes the elevation of the tops of objects, whereas many planning activities require the Digital Terrain Model (DTM) which provides the elevation of the underlying terrain. Unfortunately, unplanned settlements are often characterized by densely built-up areas and are often located in less desired areas such as steep slopes which cause difficulties. Therefore, it was also analyzed how to extract Digital Terrain Models in challenging settings. A method specifically tailored to aerial photogrammetric datasets was developed using cutting-edge deep learning techniques. Firstly, a simple rule uses the DSM to propose pixels which are likely to be ground or off-ground without any manual intervention. These samples are used to train a Fully Convolutional Network (FCN) specifically designed for this task, enabling it to differentiate between terrain and off-ground objects using imagery and DSM-based features. The proposed method was shown to significantly outperform two reference DTM extraction techniques, thus enabling DTM extraction to be performed in challenging settings while eliminating the requirement of collecting costly training samples.

Apart from generating maps automatically and extracting DTMs, the UAV data can be useful for upgrading projects in many ways. One task was to observe the use of the UAV data by stakeholders in Kigali, Rwanda to identify opportunities of UAVs to support urban upgrading workflows. Important observations included that even without advanced machine learning techniques, the images were considered to be highly valuable and were used by the upgrading projects in various ways. The higher resolution and recency of the imagery facilitated manual digitization exercises. Additional information required for upgrading projects which are not typically identifiable in satellite or aerial imagery, such as solid waste accumulations, was visible. The imagery enabled consultants to prepare better for the field and navigate the complex network of footpaths more effectively during operations. The data was also valuable as a communications platform, enabling communication between stakeholders in understanding the existing situation and prioritizing interventions.

Some concerns regarding the use of UAVs is the possible capture of objects considered as private in the imagery, the distribution of this sensitive information, and possible misuse of it. A final sub-objective was therefore to analyze the social impacts of using UAVs in the context of urban upgrading projects. Residents of unplanned areas in Kigali, Rwanda and Dar es Salaam, Tanzania which was subject to UAV flights were asked what their perceptions of the flights were. They were also asked to point out objects in the imagery and maps which they considered to be sensitive. These consisted of avoidable objects which could be removed by residents if they are aware UAV flights will take place, unavoidable but removable objects which are captured in the UAV imagery but can be blurred before distribution to other stakeholders, and unavoidable and irremovable objects. The later causes the most concern as the objects considered as sensitive are exactly those who are targeted by the UAV operations. For example, houses located in hazardous areas may be subject to expropriation. The research further illustrates the importance of local context regarding these concerns and which actions can be taken to ensure more ethical UAV operations and equitable distribution of the benefits.

In sum, this manuscript illustrates how UAVs and machine learning methods can be manipulated to provide accurate and up-to-date geospatial information. The simultaneous provision of 2D imagery and 3D point clouds proves to be quite useful and stresses the importance of developing targeted geoinformatic workflows which make use of these synergies rather than applying standard algorithms developed for either imagery or point clouds. This enables automatic algorithms to return highly accurate maps, despite the challenging characteristics of unplanned neighborhoods. Secondly, involvement with existing urban upgrading projects throughout the research has enabled a unique view of the actual usage and effectiveness of the imagery for current urban upgrading projects by local governments, engineering consultants and residents. At the data collection phase, residents were intrigued but often unable to think of practical uses for the UAV imagery. Returning years later, it appeared that the images were being used for a wide range of unexpected applications. As UAVs becoming increasingly available and as data processing simplifies, it is feasible to imagine a future where UAVs become increasingly used to support urban upgrading projects.