Assessing the thematic accuracy of area-class soil maps

D G Rossiter

PREPRINT of 31-August-2001

Download Complete Preprint (334Kb)

In Preparation This is the version that was rejected by Geoderma. If you use it, please cite as "(unpublished)". I am leaving it here for your reference while I prepare a new version. I hope it is useful; any comments are welcome. Copyright ©2001 D G Rossiter To be assigned on publication.

Abstract

This paper is meant as a guide for data users who need to quantify the accuracy of area-class maps of soil classes or properties, and for data producers who need to provide quantified accuracy measures in metadata for digital datasets derived from such maps. The thematic accuracy of a soil map is the degree to which the attribute information in a map agrees with reality, and is assessed by statistics of a partial field sample or a complete reference map. Accuracy may be assessed on the basis of legend categories, taxonomic classes, single or multiple soil properties, interpretations, or interpretive classes. Binomial tests may be used to determine the error frequency, and multinomial tests to also identify which classes or properties are confused. Multinomial tests are based on the confusion matrix, and include naïve measures of agreement and the kappa and tau indices. Partial credit for mapping errors may be given by weighted variants of these tests, with weights being assigned by utility functions or by the producer's subjective judgement of class similarity. A c² test may be used to simultaneously verify the overall accuracy and the agreement with stated proportions of a compound map unit. Field sampling for accuracy assessment may be random or stratified by legend category or major landscape. The required sample size depends on prior estimates of class proportions, required relative accuracy, and acceptable chance of Type I error. A reference map to be used for comprehensive sampling should be discretized to a grid cell size equal to one quarter of a minimum legible delineation.

Contents

1. Thematic accuracy as an aspect of adequacy
   1. Background
   2. Conceptual models of soil spatial variability
   3. Geometric accuracy, symbols, and boundaries
   4. Types of thematic accuracy
2. What is `agreement' between map and ground?
   1. Simple map units
   2. Compound map units
3. Binomial tests of accuracy
   1. Comparing the relative accuracy of two maps
4. Multinomial tests of accuracy
   1. Naive measures of agreement
      1. The weighted confusion matrix
      2. Assigning weights
         1. Assigning weights with a utility function
         2. Assigning weights with a utility function
         3. Assigning weights by expert judgement
         4. Assigning weights by expert judgement
   2. The kappa index of agreement
      1. Unweighted kappa
      2. Conditional kappa
      3. Weighted kappa
   3. The tau index
5. Multinomial tests applied to compound map units
6. Sampling schemes for accuracy assessment
   1. Sample design
   2. Sample distribution for multinomial tests
   3. Sample size
      1. Sample size for binomial tests
      2. Sample size for multinomial tests
   4. Sampling within a reference map
7. Conclusions

Downloads

• Preprint (334Kb)
• R functions and sample data (tar, gzip)
• R functions and sample data (ZIP)
• Matlab functions and sample data (zip)

Links to Software

• R
• MATLAB

URL: http://www.itc.nl/personal/rossiter/pubs/accuracy.html

2003_337