End-to-end predictive models for remote sensing applications
Due to the COVID-19 crisis measures the PhD defense of John Ray Bergado will take place online in the presence of an invited audience.
The PhD defence can be followed by a live stream.
John Ray Bergado is a PhD student in the department of Earth Observation Science (EOS). His supervisor is prof.dr.ir. A. Stein from the Faculty of Geo-Information Science and Earth Observation (ITC).
Prediction plays a crucial role in mapping urban land cover, land use, and wildfire risks. Data-related challenges concern the spectral confusion of the classes of interest, noisy classification results, highly abstracted class definitions, and the large amount of spatial, temporal, structural, and typological variation. Standard methods tackle these issues by introducing pre- and post-prediction steps such as image fusion, feature extraction, and label regularization. These additional steps are often approximate and detached from the prediction task at hand, making them sub-optimal. This research explores end-to-end predictive models for several remote sensing applications, integrating independent additional steps within a single prediction pipeline.
The first objective addresses image fusion in the context of urban land cover classification. I present a single-stage framework embedding feature extraction and image fusion in a multiresolution convolutional network, called FuseNet. The network matches the resolution of the panchromatic and multispectral bands in a very high resolution (VHR) image using convolutional layers with corresponding downsampling and upsampling. I compared FuseNet with the use of separate processing steps for image fusion, such as pansharpening and resampling through interpolation. Results show quantitative improvements on the accuracy of the land cover classification results when using FuseNet.
The second objective addresses label regularization in the context of urban land cover classification. I propose a novel end-to-end classification model integrating image fusion, feature extraction, and label regularization within a recurrent multiresolution convolutional network, called ReuseNet. ReuseNet extends FuseNet by incorporating contextual label information via recurrent connections. This extension is similar to a standard post-classification label regularization step. I designed experiments on land cover classification where I compared ReuseNet with using separate processing steps for both image fusion and map regularization. Experimental results show both quantitative and qualitative improvements on the classified land cover maps produced by ReuseNet.
The third objective addresses predicting abstracted urban land use classes. Features extracted from land cover maps are helpful on performing land use classification. Such prior information can be incorporated in the design of a deep learning based land use classifier by applying a multitask learning setup, i.e. simultaneously solving a land use and a land cover classification task. I explored a fully convolutional multitask network to classify urban land use from VHR imagery. I experimented with three different setups: a standard network only predicting the land use class of each pixel in the image, a multitask network concatenating the land use and land cover class labels in the same output layer, and a multitask network accepting as an input the land cover that was predicted by a subpart of the network, concatenated to the original input image patches. Comparing the three against a standard random forest classifier, I found that the two convolutional multitask networks outperform the other two classifiers by at least 30% in the average F1-score.
The fourth objective deals with integrating a big set of geodata to produce daily maps of the probability of wildfire burn. I designed a deep fully convolutional network, called AllConvNet, to produce daily maps of the probability of a wildfire burn over the next 7 days in Victoria, Australia for the period of 2006--2017. Fifteen factors that were extracted from six different datasets and resulted into 29 quantitative features, were selected as input. I compared AllConvNet with three baseline methods: SegNet, multilayer perceptron, and logistic regression. AllConvNet outperformed the three baseline methods in four of the six quantitative metrics considered. Total precipitation, lightning flash density, and land surface temperature were consistently highly weighted by all models while terrain aspect components, wind direction components, certain land cover classes (such as crop field and woodland), and distance from power lines were lowly weighted.
In summary, this thesis presents end-to-end predictive models for three different remote sensing applications: urban land cover, urban land use, and wildfire prediction. These models demonstrate that standard methods adding independent pre- and post-prediction steps can be integrated into a single end-to-end framework that streamlines the prediction task. The urban land cover and land use maps can be used for regular updating of geospatial database layers, while the wildfire probability maps can either directly serve as a proxy measure for wildfire risk or may serve as an input for probabilistically quantifying wildfire risk.
