Using HSDT Model to de-noise images in our data-cube

• Github Repo: https://github.com/zeqiang-lai/hsdt
• Paper: https://arxiv.org/pdf/2303.09040
• HSIR: https://github.com/bit-isp/HSIR (For testing, training and visualization)

Ways to Integrate into Project

1. Using their pre-trained model, and training our own data on top of it.
2. Importing their ‘API’ from the library, train them with our data.
3. Importing their ‘API’ from the library, manually edit the layers in order to better fit our data

I suggest starting out with method-1, especially since this HSDT model is made specifically to work on different spectral dimensions, so having pre-trained model on publicly available data can enable us to test its performance quickly without much time investment.

If we need custom integrations as part of the processing pipeline, we could look into using its API directly

If the testing yields less than satisfactory results, we can look into tweaking its layers manually from their library.

HSDT

A 3D hybrid spectral denoising transformer that effectively

• Integrates local spectral-spatial features (S3 Conv)
• Capture long-range global spectral correlation (GSSA)

It also allow us to train our model with data of different spectral dimensions.

Architecture of HSDT