Latent space interpretation [R]

Hi all, I have trained a convolutional autoencoder on a set of medical images. Further classified latent feature maps using random forest to find the top scoring feature map. Now my goal is to understand which input image is captured in top scoring latent feature map. Any suggestions? I have tried encoding one image at a time while other images were muted. I then checked spearman between top scoring feature map with the original top scoring feature map. While I see some expected results, I still have some false positives. I have also tried decoding only top scoring latent feature map by setting others feature maps to 0. But I believe, the decoder entanglement is giving me many false positive results.