Self-supervised denoising of dynamic fluorescence images via temporal gradient-empowered deep learning

Woojin Lee; Minseok A. Jang; Hyeong Soo Nam; Jeonggeun Song; Jieun Choi; Joon Woo Song; Jae Yeon Seok; Pilhan Kim; Jin Won Kim; Hongki Yoo

doi:10.1186/s43074-025-00173-8

Article Contents

Article Navigation > PhotoniX > 2025 > Accepted Manuscript

Woojin Lee, Minseok A. Jang, Hyeong Soo Nam, Jeonggeun Song, Jieun Choi, Joon Woo Song, Jae Yeon Seok, Pilhan Kim, Jin Won Kim, Hongki Yoo. Self-supervised denoising of dynamic fluorescence images via temporal gradient-empowered deep learning[J]. PhotoniX. doi: 10.1186/s43074-025-00173-8

Citation:

PDF( 0 KB)

Self-supervised denoising of dynamic fluorescence images via temporal gradient-empowered deep learning

doi: 10.1186/s43074-025-00173-8

1.
Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea
2.
Graduate School of Medical Science and Engineering, KAIST, 291 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea
3.
KAIST Institute for Health Science and Technology, 291 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea
4.
Multimodal Imaging and Theranostic Lab
5.
Department of Pathology, Yongin Severance Hospital, Yonsei University College of Medicine, 363 Dongbaekjukjeon-Daero, Giheung-gu, Yongin-si, Gyeonggi-Do, 16995, Republic of Korea

Funds:

This work was supported by the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (NRF-RS-2023-00208888 and NRF-RS-2024-00401786), and by a Korea Medical Device Development Fund grant funded by the Korean government (Ministry of Science and Information and Communication Technologies, Ministry of Trade, Industry and Energy, Ministry of Health and Welfare, Ministry of Food and Drug Safety) (RS-2023-00254566).

Received Date: 2024-11-07
Accepted Date: 2025-05-10
Rev Recd Date: 2025-04-22

Available Online: 2025-05-23

Abstract

Abstract

Fluorescence microscopy has become one of the most widely employed in vivo imaging modalities, enabling the discovery of new biopathological mechanisms. However, the application of fluorescence imaging is often hindered by signal-to-noise ratio issues owing to inherent noise arising from various systemic and biophysical characteristics. These limitations pose a growing challenge, especially with the desire to elucidate dynamic biomechanisms at previously unreachable rapid speeds. Here, we propose a temporal gradient (TG)-based self-supervised denoising network (TeD) that could enable an unprecedented advance in spatially dynamic fluorescence imaging. Our strategy is predicated on the insight that judicious utilization of spatiotemporal information is more advantageous for denoising predictions. Adopting the TG, which intrinsically embodies spatial dynamic features, enables TeD to prudently focus on spatiotemporal information. We showed that TeD can provide new interpretative opportunities for understanding dynamic fluorescence signals in in vivo imaging of mice, representing cellular flow. Furthermore, we demonstrated that TeD is robust even when fluorescence signals exhibit temporal kinetics without spatial dynamics, as seen in neuronal population imaging. We believe that TeD’s superior performance even with spatially dynamic samples, including the complex behavior of cells or organisms, could make a substantial contribution to various biological studies.

FullText(HTML)

References(44)

References

[1]	Lichtman JW, Conchello J-A. Fluorescence microscopy. Nat Methods. 2005;2(12):910–9.
[2]	Verweij FJ, Balaj L, Boulanger CM, Carter DR, Compeer EB, D’angelo G, et al. The power of imaging to understand extracellular vesicle biology in vivo. Nat Methods. 2021;18(9):1013–26.
[3]	Calarco JA, Samuel AD. Imaging whole nervous systems: insights into behavior from worms to fish. Nat Methods. 2019;16(1):14–5.
[4]	Han J, Kim S, Jung Kim H, Soo Nam H, Lee MW, Song JW, et al. Label-free characterization of atherosclerotic plaques via high-resolution multispectral fluorescence lifetime imaging microscopy. Arterioscler Thromb Vasc Biol. 2023;43(7):1295–307.
[5]	Lee S, Lee MW, Cho HS, Song JW, Nam HS, Oh DJ, et al. Fully integrated high-speed intravascular optical coherence tomography/near-infrared fluorescence structural/molecular imaging in vivo using a clinically available near-infrared fluorescence–emitting indocyanine green to detect inflamed lipid-rich atheromata in coronary-sized vessels. Circ Cardiovasc Interv. 2014;7(4):560–9.
[6]	Chen B-C, Legant WR, Wang K, Shao L, Milkie DE, Davidson MW, et al. Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution. Science. 2014;346(6208):1257998.
[7]	Klonis N, Rug M, Harper I, Wickham M, Cowman A, Tilley L. Fluorescence photobleaching analysis for the study of cellular dynamics. Eur Biophys J. 2002;31:36–51.
[8]	Li X, Li Y, Zhou Y, Wu J, Zhao Z, Fan J, et al. Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit. Nat Biotechnol. 2023;41(2):282–92.
[9]	Song J, Kang J, Kang U, Nam HS, Kim HJ, Kim RH, et al. SNR enhanced high-speed two-photon microscopy using a pulse picker and time gating detection. Sci Rep. 2023;13(1):14244.
[10]	Yokota H, Fukasawa A, Hirano M, Ide T. Low-light photodetectors for fluorescence microscopy. Appl Sci. 2021;11(6):2773.
[11]	Meiniel W, Olivo-Marin J-C, Angelini ED. Denoising of microscopy images: a review of the state-of-the-art, and a new sparsity-based method. IEEE Trans Image Process. 2018;27(8):3842–56.
[12]	Ryu J, Kang U, Kim J, Kim H, Kang JH, Kim H, et al. Real-time visualization of two-photon fluorescence lifetime imaging microscopy using a wavelength-tunable femtosecond pulsed laser. Biomed Opt Express. 2018;9(7):3449–63.
[13]	LeCun Y, Bengio Y, Hinton G. Deep learning nature. 2015;521(7553):436–44.
[14]	Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial networks. Commun ACM. 2020;63(11):139–44.
[15]	Jain V, Seung S. Natural image denoising with convolutional networks. Advances in neural information processing systems. 2008;21.
[16]	Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, et al. Photo-realistic single image superresolution using a generative adversarial network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2017. p. 4681–90.
[17]	Lee W, Nam HS, Seok JY, Oh W-Y, Kim JW, Yoo H. Deep learning-based image enhancement in optical coherence tomography by exploiting interference fringe. Communications Biology. 2023;6(1):464.
[18]	Liang J, Cao J, Sun G, Zhang K, Van Gool L, Timofte R. Swinir: Image restoration using swin transformer. In:Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). IEEE; 2021. p. 1833–44.
[19]	Chaudhary S, Moon S, Lu H. Fast, efficient, and accurate neuro-imaging denoising via supervised deep learning. Nat Commun. 2022;13(1):5165.
[20]	Tian C, Fei L, Zheng W, Xu Y, Zuo W, Lin C-W. Deep learning on image denoising: An overview. Neural Netw. 2020;131:251–75.
[21]	Weigert M, Schmidt U, Boothe T, Müller A, Dibrov A, Jain A, et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat Methods. 2018;15(12):1090–7.
[22]	Fan C-M, Liu T-J, Liu K-H. SUNet: Swin Transformer UNet for image denoising. In: Proceedings of the IEEEInternational Symposium on Circuits and Systems (ISCAS). IEEE; 2022. p. 2333–7.
[23]	Lecoq J, Oliver M, Siegle JH, Orlova N, Ledochowitsch P, Koch C. Removing independent noise in systems neuroscience data using DeepInterpolation. Nat Methods. 2021;18(11):1401–8.
[24]	Li X, Zhang G, Wu J, Zhang Y, Zhao Z, Lin X, et al. Reinforcing neuron extraction and spike inference in calcium imaging using deep self-supervised denoising. Nat Methods. 2021;18(11):1395–400.
[25]	Huang T, Li S, Jia X, Lu H, Liu J. Neighbor2Neighbor: Self-supervised denoising from single noisy images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2021. p. 14781–90.
[26]	Krull A, Buchholz T-O, Jug F. Noise2Void: Learning denoising from single noisy images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2019. p. 2129–37.
[27]	Lehtinen J, Munkberg J, Hasselgren J, Laine S, Karras T, Aittala M, et al. Noise2Noise: Learning image restorationwithout clean data. arXiv preprint arXiv:1803.04189. 2018.
[28]	Eom M, Han S, Park P, Kim G, Cho E-S, Sim J, et al. Statistically unbiased prediction enables accurate denoising of voltage imaging data. Nat Methods. 2023;20(10):1581–92.
[29]	Zhang G, Li X, Zhang Y, Han X, Li X, Yu J, et al. Bio-friendly long-term subcellular dynamic recording by self-supervised image enhancement microscopy. Nat Methods. 2023;20(12):1957–70.
[30]	Li X, Hu X, Chen X, Fan J, Zhao Z, Wu J, et al. Spatial redundancy transformer for self-supervised fluorescence image denoising. Nature Computational Science. 2023;3(12):1067–80.
[31]	Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, et al. Swin Transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). IEEE; 2021. p. 10012–22.
[32]	Qu L, Zhao S, Huang Y, Ye X, Wang K, Liu Y, et al. Self-inspired learning for denoising live-cell super-resolution microscopy. Nat Methods. 2024;21(10):1895–908.
[33]	Lequyer J, Philip R, Sharma A, Hsu W-H, Pelletier L. A fast blind zero-shot denoiser. Nature Machine Intelligence. 2022;4(11):953–63.
[34]	Song A, Gauthier JL, Pillow JW, Tank DW, Charles AS. Neural anatomy and optical microscopy (NAOMi) simulation for evaluating calcium imaging methods. J Neurosci Methods. 2021;358: 109173.
[35]	Choi J, Choi M-S, Jeon J, Moon J, Lee J, Kong E, et al. In vivo longitudinal 920 nm two-photon intravital kidney imaging of a dynamic 2, 8-DHA crystal formation and tubular deterioration in the adenine-induced chronic kidney disease mouse model. Biomed Opt Express. 2023;14(4):1647–58.
[36]	Kang J, Kang U, Nam HS, Kim W, Kim HJ, Kim RH, et al. Label-free multimodal microscopy using a single light source and detector for biological imaging. Opt Lett. 2021;46(4):892–5.
[37]	Luu P, Fraser SE, Schneider F. More than double the fun with two-photon excitation microscopy. Communications Biology. 2024;7(1):364.
[38]	Rahaman N, Baratin A, Arpit D, Draxler F, Lin M, Hamprecht F, et al. On the spectral bias of neural networks. International conference on machine learning: PMLR; 2019:5301–10.
[39]	Xia X, Li J, Wu J, Wang X, Xiao X, Zheng M, et al. TRT-ViT: TensorRT-oriented vision transformer. arXiv preprint arXiv:220509579. 2022.
[40]	Chen X, Qiao C, Jiang T, Liu J, Meng Q, Zeng Y, et al. Self-supervised denoising for multimodal structured illumination microscopy enables long-term super-resolution live-cell imaging. PhotoniX. 2024;5(1):4.
[41]	Krull A, Basevi H, Salmon B, Zeug A, Müller F, Tonks S, et al. Image denoising and the generative accumulation of photons. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE; 2024. p. 1528–37.
[42]	Chen X, Wang X, Zhou J, Qiao Y, Dong C. Activating more pixels in image super-resolution transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2023. p. 22367–77.
[43]	Kranig SA, Lajqi T, Tschada R, Braun M, Kuss N, Pöschl J, et al. Leukocyte infiltration of cremaster muscle in mice assessed by intravital microscopy. JoVE (Journal of Visualized Experiments). 2020;158: e60509.
[44]	Jang MA, Song JW, Kim RH, Kang DO, Kang U, Kim HJ, et al. Real-time imaging assessment of stress-inducedvascular inflammation using heartbeat-synchronized motion compensation. Arteriosclerosis, Thrombosis, and Vascular Biology. 2024;44(12):2493–506.