Land Cover Classification based on different UAV-acquired data types

Table of Contents

1. Introduction

This blog post compares land cover classifications performed on different datasets. The images were acquired using a DJI Phantom 4 on the 25th of November 2021, using the standard on-board RGB camera as well as a Near-Infrared (NIR) and Red Edge (RE) sensor near Lancieux, Bretagne. Since this specialized sensor also measures the sun radiance, the actual reflectance values are automatically obtained on-chip by subtracting these values from the captured pixel values.

The goal of this comparison is to see how the aforementioned Orthoimages as well as the DSM of the area contribute to a successful image classification. The goal is not to refine the classification algorithm, but instead see how different data types impact the classification.

2. Data

2.1 Spectral Data

Map 1: RGB Orthoimage.
Map 2: Digital surface Model.
Map 2: Digital surface Model.
Map 3: Near-Infrared Image.
Map 3: Near-Infrared Image.
Map 4: Red-Edge Image
Map 4: Red-Edge Image

The datasets were processed using Agisoft Metashape. 

2.2. Classification Dataset for Training and Validation

Similarily sized training and validation regions were created to serve the supervised machine learning process. Three polygons were created for the training dataset, two to serve as validation set. The pixel values and their class are fed into the classification algorithm to train the model and then assess its performance. 
Map 5: Training and Verification Areas.

3. Classification Algorithm

For the classification, a standard Random Forest model is used. The code can be found here.

3.1 Classification Based on RGB

Map 6: Classification based on RGB values.
Map 6: Classification based on RGB values.
Matrix 1: Confusion Matrix.

The classification based on RGB values alone returns an accuracy of 0.74 on the validation set. The main problem seems to be the distinction between the bush and sediment classes, since the bushes have a similar color to the sediment. Very clearly visible are the training areas, since the algorithm has seen these data points before and classifies them accordingly.

Graph 1: RGB feature importances with stdev.

3.2 Classification based on RGB and DSM

Map 7: Classification based on RGB and DSM values.
Map 7: Classification based on RGB and DSM values.

Including the DSM results in a better classification in many areas, for example the roof for which height is a good discriminator. On the other hand, the spectral signature looses importance, leading to confusions especially between the bush and road classes. Still, the overall accuracy is slightly increased to 0.81.

Graph 2: RGB and DSM feature importances with stdev.

3.3 Classification based on RGB and NIR

Map 8: Classification based on RGB and NIR values.
Map 8: Classification based on RGB and NIR values.
Matrix 3: Confusion Matrix.

The RGB+NIR combination leads to a better distinction between vegetation and non-vegetation classes. Detecting the bushes and trees is still a problem, since the spectral signature seem to be similar to the other vegetation classes. The algorithm might be quite heavily overfitted, since the training dataset polygons are highly visible in the final classified image. The final accuracy is 0.80, with all spectral bands having a similar feature importance.

Graph 3: Feature importance for RGB + NIR with stdev.

3.4 Classification based on RGB and RE

Map 10: Classification based on RGB and RE values.
Map 10: Classification based on RGB and RE values.
Matrix 4: Confusion Matrix.

The combination of RGB and RE performs slightly worse, since the vegetational aspect is lost a bit compared to the RGB+NIR combination. Still, the overall accuracy is 0.79.

Graph 4: Feature Importances for RGB+RE combination with stdev.

3.5 Classification based on RGB, DSM, IR, and RE

Map 11: Classification based on RGB, DSM, NIR, and RE.
Map 11: Classification based on RGB, DSM, NIR, and RE.
Matrix 5: Confusion Matrix.

The final combination, including all data sources, leads to the best result. The grass, sediment and water are classified very well. Some problems are still present in the distinction between the tree and bush classes, which is a problem for all combinations. The accuracy is 0.85. The building is also accurately delineated, while segments of the road are misclassified as bush.

Interestingly, the DSM is still the most important feature. In my opinion, this is due to the difference in height of the house and the other areas. If the difference in the spectral bands is not too high, an important feature might be the altitude of the pixel. With the domain knowledge in mind, potentially not taking the DSM into account would lead to a better result.

Graph 5: Feature importances for a classification based on all available data with stdev.

4. Conclusion

In this test, the combination of all data types lead to the best result. The NIR band helped identify and separate vegetation classes from the other classes, having a higher impact on the correct identification than the RE band. While the DSM had some high feature importance scores and also lead to improvements in accuracy, but also introduced some noise into the classification.

While some overfitting was observed, the optimization of the classifier was not the main motivation but rather the differences in accuracy of the different data combinations.

  • Accuracy of RGB: 0.74
  • Accuracy of RGB + DSM: 0.81
  • Accuracy of RGB + RE: 0.79
  • Accuracy of RGB + IR: 0.80
  • Accuracy of RGB + DSM + NIR + RE: 0.85

5. Appendix 1 – Improved Verification Dataset

The verification result was only based on two small polygons per class. These polygons might not accurately capture the variance within the class, leading to inaccurate validation results. For that reason, a verification dataset covering almost the whole study area was created. Based on these polygons, the accuracy of the classification can be better assessed.
Map 12: Extended verification areas.
Map 12: Extended verification areas.

This verification method came to the following results:

  • Accuracy of RGB: 0.65
  • Accuracy of RGB + DSM: 0.73
  • Accuracy of RGB + RE: 0.68
  • Accuracy of RGB + IR: 0.68
  • Accuracy of RGB + DSM + NIR + RE: 0.73

In general, the accuracies are lower than the accuracies of the previous method. This is most likely due to the larger verification areas, capturing a bigger internal difference if the classes. This method is more likely to accurately reflect the actual prediction quality.

While the absolute accuracy values are different in this verification method than the previous one, the relative accuracies still follow the same patterns as previously. This shows that the underlying quality of the data and its power to represent the underlying phenomena is the driver behind the differences in accuracy, not the classification method itself.

6. Appendix 2 – Neural Network Classifier

Just out of curiosity a Neural Network was implemented as a classifier instead of the Random Forest for the same dataset and verification areas as described in Appendix 1. Since the data is the same, the resultsare comparable and diferences in accuracies are only due to tue classifiers themselves. Some of the combinations unfortunately suffer from ‘neuron death’, which at this stage is to time-consuming to fix.

See the code for the NN here.

  • Accuracy of RGB: 0.67
  • Accuracy of RGB + DSM: 0.79
  • Accuracy of RGB + RE: 0.40 (Neuron Death)
  • Accuracy of RGB + IR: 0.40 (Neuron Death)
  • Accuracy of RGB + DSM + NIR + RE: 0.68

Leave a Reply

Your email address will not be published. Required fields are marked *

Further Reading
Recent Updates