position estimation of mobile laser scanner using aerial imagery
Zille Hussnain is a PhD student in the department of Earth Observation Science (EOS). His supervisor is prof.dr.ir. M.G. Vosselman from the Faculty of Geo-Information Science and Earth Observation (ITC).
Mobile Laser scanning is a technique to obtain geo-information for large areas using Laser scanners mounted on a car or another vehicle. The obtained product is called a mobile laser scanning point cloud. Under normal conditions, accurate positioning is provided by the integration of Global Navigation Satellite Systems (GNSS) and Inertial Navigation Systems (INS). However, especially in urban areas building structures impede a direct line-of-sight to navigation satellites and lead to multipath effects. In this situation the mobile mapping derived products, such as mobile laser scanning point clouds, lack the expected reliability and inherit an unknown positioning error.
This issue has been addressed by many researchers, whose aim to mitigate these effects mainly concentrates on utilizing datasets, such as digital maps, orthoimages or airborne lidar. These datasets serve as a reliable source of orientation and are being used subsidiarily or as the basis for adjustment.
This research aims to improve the position estimation of mobile laser scanning platforms by employing high-resolution multiview nadir aerial imagery as a reference dataset. The aerial images have highly accurate exterior orientations as their positioning is not suffering from the GNSS signal occlusion and estimated by aerial triangulation as well. The highly accurate positioned images can therefore provide precise locations of features. These features can be measured in the mobile laser scanning data as well.
The first task in the automatic correction pipeline is the matching of the point cloud images with aerial images, after a preprocessing step generates images from the point clouds. The conventional image matching techniques are tailor-made for specific applications, perform poorly and are unable to obtain substantial matches for our dataset, let alone the fulfilment of the near decimetre-level accuracy. Due to the unavailability of a suitable 2D matching technique which can also detect geometrically same feature across both datasets, we invest our efforts in the development of a new feature matching technique. The realization of feature matching technique starts from defining geometrically consistent features viable for matching.
Therefore, for feature detection, we have developed a variant of the Harris corner detector which adopts a threshold regardless of diverse image information. The developed technique detects clusters of keypoints on the vertices of road markings with pixel-level accuracy. For the feature descriptor, we use the LATCH binary descriptor, which is robust while matching the datasets from dissimilar sensors. Due to a large number of outliers, we also developed an outlier filtering technique which exploits the relative Euclidean-distances and relative angles between the corresponding topologies of feature sets. In its entirety, the developed feature matching extracts the 2D correspondences with pixel-level accuracy. However, the pixel-level correspondences are not sufficient to improve the MLS dataset to a near decimetre accuracy level. Therefore, we map pixel-level correspondences to subpixel based on features detected from the Förstner-operator in the last phase of the feature matching.
Once the subpixel-level stereo correspondences are known, the remaining tasks are the multiview matching and the computation of the 3D tie points by 3D triangulation of the multiview matches. The 3D tie points determine the 3D-3D correspondences between the point cloud and aerial images, and are normally sufficient for the adjustment and the correction of the MLS dataset. For the evaluation of 3D tie point accuracy, it is necessary to develop a method which first performs an adjustment of the six degrees of freedom trajectory using 3D-3D correspondences and then evaluate the enhanced trajectory.
For the enhancement of the trajectory, changes of the six pose parameters over time are represented by B-spline functions and enhanced by updating the spline coefficients using three types of observations. The main observation for the trajectory enhancement is the 3D-3D correspondences between MLS point cloud and aerial images: it contains high local and global consistency. The high global consistency is necessary to achieve accuracy in the absolute coordinate system, while the local consistency is important to achieve the relative accuracy in the dataset. The second type of observation is based on data from the Inertial Measurement Unit (IMU). It provides only local consistency, but at a high frequency. The third type of observation is based on constraints applied to the pitch and yaw rotations of the vehicle, which reduces the degrees of freedom. To enhance the trajectory, these observations are added to a normal matrix and solved for the unknown changes to the coefficients. The computed changes to the coefficients are then added back to the coefficients of trajectory in an iterative process. The iterative process stops when the square sum of the residuals no longer improves. The procedure to select a trajectory segment for the experiments is based on several checks. First, we verify the high positioning accuracy of the start and end positions of the trajectory segment using checkpoints. Alternatively, if checkpoints are not available exactly at the start and end locations, we make sure that a sufficient amount of 3D tie points exist at those locations. The latter check also means that the trajectory’s start and end positions are always located within the areas covered by more than one aerial image. Second, we verify that all other parts of the trajectory segment are as well covered by the aerial imagery. Third, depending on the evaluation criteria, we select the trajectory segment which has the availability of checkpoints after short intervals. The interval of the checkpoints availability should be shorter for the evaluation of the trajectory constructed only with the IMU observations, while this interval can be larger for the evaluation of the trajectory enhanced by all observations.
When comparing with other available methods, the developed method is fully automatic while successful in obtaining a near decimetre-level accuracy. The quality of the trajectory enhancement is analysed under several conditions. Experiments are designed to test the influence of the number and quality of 3D-3D correspondences and try to determine the accuracy in the areas where checkpoints are not available. The experiment conducted on two datasets demonstrated that the enhanced trajectory achieved near decimetre-level accuracy. The RMSE values of the original Kalman filter results at the checkpoints were 0.26 m, 0.30 m and 0.47 m for the X-, Y-, and Z-coordinates in one of the two datasets and 1.10 m, 1.51 m, and 1.81 m in another dataset. After the trajectory adjustment these RMSE values were reduced to 0.09 m, 0.11 m, and 0.16 m for the first dataset and 0.12 m, 0.14 m and 0.18 m for the second dataset. In this project, the complete pipeline of the MLS dataset correction is implemented and as well delivered to the stakeholders.
