Grad-SLAM: Explaining Convolutional Autoencoders’ Latent Space of Satellite Image Time Series

Abstract

This paper introduces a tool for explaining the latent space generated by applying convolutional autoencoders to satellite image time series, entitled Grad-SLAM. We rely on backpropagated gradient interpretation combined with network activation localization. We use the proposed formula for multiple layers of the encoder, then scale and merge the results to generate a single date contribution metric for the generation of the latent space. We illustrate the potential of this method with the study of the unsupervised classification of agricultural Sentinel-1 time series. We show that critical characterizing dates for unsupervised retrieval of a given class are conditioned by the crop type’s radiometric signature and class count. We also present how Grad-SLAM can be used to enhance the understanding of unsupervised classification confusion.