Generalization of image classification models using fair optimization techniques

M-GEO
M-SE
ACQUAL
M-SE Core knowledge areas
Spatial Information Science (SIS)
Spatial Planning for Governance (SPG)
Additional Remarks

Suggested elective courses: Advanced image analysis

Programming skills required

Topic description

Deep learning algorithms can achieve great accuracies, but there are also concerns about biases and generalizability to unseen data. Biases can arise for many reasons, one of them being that certain classes are better represented in training datasets than others. As the classifier sees examples of the majority class more than minority classes, it will also learn to recognize the majority class better than the minority classes. One way to reduce this bias is to use optimization functions that not only maximize the accuracy but also minimize the difference in performance amongst the different classes. The TensorFlow Constrained Optimization Library is one example of a tool to do this.

One possible use case is the identification of the minority crops cultivated in a diverse agricultural landscape. This use case is, however, not cast in stone. Minority crop class can be replaced by any other land cover/land use class of interest.

Topic objectives and methodology

The main objective of this work is to implement “fair” optimization techniques to improve the generalizability of deep learning workflows. Satellite images (either single-date or time series) together with one of the following deep learning algorithms can be used to identify the minority classes from the target study area(s): Recurrent Neural Network, (Fully) Convolutional Neural Network or Transformers. The dataset will be provided by the supervisors.

References for further reading

· Waldner, F. Chen, Y., Lawes, R., & Hochman, Z. (2019) Needle in a haystack: Mapping rare and infrequent crops using satellite imagery and data balancing methods. Remote Sensing of Environment. 233, 111375. https://doi.org/10.1016/j.rse.2019.111375 .