Surmounting photon limits and motion artifacts for biological dynamics imaging via dual-perspective self-supervised learning

1.
Key Laboratory of Optoelectronic Devices and Systems of Guangdong Province and Ministry of Education, College of Physics and Optoelectronic Engineering, Shenzhen University, Shenzhen, 518060, China
2.
Med-X Research Institute and School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200030, China
3.
Department of Chemistry, Institute of Molecular Aggregation Science, Tianjin University, Tianjin, 300072, China

Funds: We thank the National Natural Science Foundation of China (62225505/61935012/ 62175163/61835009/62127819/62205220), Shenzhen Key Projects (JCYJ20200109105404067), Shenzhen International Cooperation Project (GJHZ20190822095420249), and Shenzhen Medical Research Project (A2303018) for financial support.

摘要

Abstract:
Visualizing rapid biological dynamics like neuronal signaling and microvascular flow is crucial yet challenging due to photon noise and motion artifacts. Here we present a deep learning framework for enhancing the spatiotemporal relations of optical microscopy data. Our approach leverages correlations of mirrored perspectives from conjugated scan paths, training a model to suppress noise and motion blur by restoring degraded spatial features. Quantitative validation on vibrational calcium imaging validates significant gains in spatiotemporal correlation (2.2×), signal-to-noise ratio (9–12 dB), structural similarity (6.6×), and motion tolerance compared to raw data. We further apply the framework to diverse in vivo experiments from mouse cerebral hemodynamics to zebrafish cardiac dynamics. This approach enables the clear visualization of the rapid nutrient flow (30 mm/s) in microcirculation and the systolic and diastolic processes of heartbeat (2.7 cycle/s), as well as cellular and vascular structure in deep cortex. Unlike techniques relying on temporal correlations, learning inherent spatial priors avoids motion-induced artifacts. This self-supervised strategy flexibly enhances live microscopy under photon-limited and motion-prone regimes.

HTML全文

下载: 全尺寸图片幻灯片

参考文献(65)

[1]	Drew PJ, Shih AY, Kleinfeld D. Fluctuating and sensory-induced vasodynamics in rodent cortex extend arteriole capacity. P Natl Acad Sci USA. 2011;108:8473–8.
[2]	Wang Z, et al. Real-time volumetric reconstruction of biological dynamics with light-field microscopy and deep learning. Nat Methods. 2021;18:551–6.
[3]	Andrade-Talavera, Y., Fisahn, A. & Rodríguez-Moreno, A. Timing to be precise? An overview of spike timing-dependent plasticity, brain rhythmicity, and glial cells interplay within neuronal circuits. Mol Psychiatry. 2023;28:2177–88.
[4]	Fan JL, et al. High-speed volumetric two-photon fluorescence imaging of neurovascular dynamics. Nat Commun. 2020;11:6020.
[5]	Shen Z, Lu Z, Chhatbar PY, O’Herron P, Kara P. An artery-specific fluorescent dye for studying neurovascular coupling. Nat Methods. 2012;9:273–6.
[6]	O’Herron P, et al. Neural correlates of single-vessel haemodynamic responses in vivo. Nature. 2016;534:378–82.
[7]	Fan J, et al. Video-rate imaging of biological dynamics at centimetre scale and micrometre resolution. Nat Photonics. 2019;13:809–16.
[8]	Mickoleit M, et al. High-resolution reconstruction of the beating zebrafish heart. Nat Methods. 2014;11:919–22.
[9]	Huang XS, et al. Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy. Nat Biotechnol. 2018;36:451–9.
[10]	Zhang, Y.D. et al. A Poisson-Gaussian Denoising Dataset with Real Fluorescence Microscopy Images. 2019 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr 2019), 2019: 11702-11710.
[11]	Lee S, Negishi M, Urakubo H, Kasai H, Ishii S. Mu-net: Multi-scale U-net for two-photon microscopy image denoising and restoration. Neural Netw. 2020;125:92–103.
[12]	Audier, X., Heuke, S., Volz, P., Rimke, I. & Rigneault, H. Noise in stimulated Raman scattering measurement: From basics to practice. Apl Photonics. 2020;5:011101.
[13]	Weigert M, et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat Methods. 2018;15:1090–7.
[14]	Lin HN, et al. Microsecond fingerprint stimulated Raman spectroscopic imaging by ultrafast tuning and spatial-spectral learning. Nat Commun. 2021;12:3052.
[15]	Shen B, et al. Deep learning autofluorescence-harmonic microscopy. Light. 2022;11:76.
[16]	Liu S-B, et al. Deep learning enables parallel camera with enhanced- resolution and computational zoom imaging. PhotoniX. 2023;4:17.
[17]	Wang K, et al. Deep learning wavefront sensing and aberration correction in atmospheric turbulence. PhotoniX. 2021;2:8.
[18]	Li, Y.P. et al. Fast denoising and lossless spectrum extraction in stimulated Raman scattering microscopy. J Biophotonics. 2021;14:e202100080.
[19]	Jiang M, et al. Fiber laser development enabled by machine learning: review and prospect. PhotoniX. 2022;3:16.
[20]	Zhao Z, et al. Two-photon synthetic aperture microscopy for minimally invasive fast 3D imaging of native subcellular behaviors in deep tissue. Cell. 2023;186:2475-2491.e2422.
[21]	Sarder P, Nehorai A. Deconvolution methods for 3-D fluorescence microscopy images. Ieee Signal Process Mag. 2006;23:32–45.
[22]	Figueiredo MAT, Bioucas-Dias JM. Restoration of Poissonian images using alternating direction optimization. Ieee Trans Image Process. 2010;19:3133–45.
[23]	Chen JJ, et al. Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes. Nat Methods. 2021;18:678–87.
[24]	Qiao C, et al. Evaluation and development of deep neural networks for image super-resolution in optical microscopy. Nat Methods. 2021;18:194–202.
[25]	Chaudhary, S., Moon, S. & Lu, H. Fast, efficient, and accurate neuro-imaging denoising via supervised deep learning. Nat Commun. 2022;13:5165.
[26]	Zhang Y, et al. Multi-focus light-field microscopy for high-speed large-volume imaging. PhotoniX. 2022;3:30.
[27]	Junchao F, Xiaoshuai H, Liuju L, Shan T, Liangyi C. A protocol for structured illumination microscopy with minimal reconstruction artifacts. Biophys Rep. 2019;5:80–90.
[28]	Xue F, et al. Hessian single-molecule localization microscopy using sCMOS camera. Biophys Rep. 2018;4:215–21.
[29]	Lehtinen, J. et al. Noise2Noise: Learning Image Restoration without Clean Data. Int Conference Machine Learn. 2018;80:2965–74.
[30]	Lecoq J, et al. Removing independent noise in systems neuroscience data using DeepInterpolation. Nat Methods. 2021;18:1401–8.
[31]	Li XY, et al. Reinforcing neuron extraction and spike inference in calcium imaging using deep self-supervised denoising. Nat Methods. 2021;18:1395–400.
[32]	Li X, et al. Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit. Nat Biotechnol. 2023;41:282–92.
[33]	Song A, Gauthier JL, Pillow JW, Tank DW, Charles AS. Neural anatomy and optical microscopy (NAOMi) simulation for evaluating calcium imaging methods. J Neurosci Methods. 2021;358:109173.
[34]	Zhang Y, et al. Rapid detection of neurons in widefield calcium imaging datasets after training with synthetic data. Nat Methods. 2023;20:747–54.
[35]	Meiniel W, Olivo-Marin JC, Angelini ED. Denoising of Microscopy Images: A Review of the State-of-the-Art, and a New Sparsity-Based Method. Ieee Trans Image Process. 2018;27:3842–56.
[36]	Lorenz KS, Salama P, Dunn KW, Delp EJ. Digital correction of motion artefacts in microscopy image sequences collected from living animals using rigid and nonrigid registration. J Microsc. 2012;245:148–60.
[37]	Lee S, Vinegoni C, Sebas M, Weissleder R. Automated motion artifact removal for intravital microscopy, without a priori information. Sci Rep-Uk. 2014;4:4507.
[38]	Vinegoni C, Lee S, Aguirre AD, Weissleder R. New techniques for motion-artifact-free in vivo cardiac microscopy. Front Physiol. 2015;6:147.
[39]	Pnevmatikakis, Eftychios A, et al. Simultaneous denoising, deconvolution, and demixing of calcium imaging data. Neuron. 2016;89:285–99.
[40]	Pnevmatikakis EA, Giovannucci A. NoRMCorre: An online algorithm for piecewise rigid motion correction of calcium imaging data. J Neurosci Methods. 2017;291:83–94.
[41]	Culley S, et al. Quantitative mapping and minimization of super-resolution optical imaging artifacts. Nat Methods. 2018;15:263–6.
[42]	Wang H, et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat Methods. 2019;16:103–10.
[43]	Secomb TW. Hemodynamics. Compr Physiol. 2016;6:975–1003.
[44]	Peyrounette M, Davit Y, Quintard M, Lorthois S. Multiscale modelling of blood flow in cerebral microcirculation: Details at capillary scale control accuracy at the level of the cortex. Plos One. 2018;13:e0189474.
[45]	Munoz CJ, Lucas A, Williams AT, Cabrales P. A review on microvascular hemodynamics. Crit Care Clin. 2020;36:293–305.
[46]	Kamoun WS, et al. Simultaneous measurement of RBC velocity, flux, hematocrit and shear rate in vascular networks. Nat Methods. 2010;7:655–60.
[47]	Mannam V, et al. Real-time image denoising of mixed Poisson-Gaussian noise in fluorescence microscopy images using ImageJ. Optica. 2022;9:335–45.
[48]	Kirillov, A. et al. Segment anything. arXiv:2304.02643. 2023. https://doi.org/10.48550/arXiv.2304.02643.
[49]	Cheng, Y. et al. Segment and track anything. arXiv:2305.06558. 2023. https://doi.org/10.48550/arXiv.2305.06558.
[50]	Zhang X, et al. High-resolution mapping of brain vasculature and its impairment in the hippocampus of Alzheimer’s disease mice. Natl Sci Rev. 2019;6:1223–38.
[51]	Hove JR, et al. Intracardiac fluid forces are an essential epigenetic factor for embryonic cardiogenesis. Nature. 2003;421:172–7.
[52]	Hoage T, Ding Y, Xu X. Quantifying cardiac functions in embryonic and adult zebrafish. Methods Mol Biol (Clifton, NJ). 2012;843:11–20.
[53]	De Luca E, et al. ZebraBeat: a flexible platform for the analysis of the cardiac rate in zebrafish embryos. Sci Rep-Uk. 2014;4:4898.
[54]	Debarre D, Olivier N, Supatto W, Beaurepaire E. Mitigating Phototoxicity during Multiphoton Microscopy of Live Drosophila Embryos in the 1.0–1.2 μm Wavelength Range. Plos One. 2014;9:e104250.
[55]	You S, et al. Intravital imaging by simultaneous label-free autofluorescence-multiharmonic microscopy. Nat Commun. 2018;9:21–5.
[56]	Lefort C. A review of biomedical multiphoton microscopy and its laser sources. J Phys D Appl Phys. 2017;50:423001.
[57]	Liu H, et al. Visualizing astrocytes in the deep mouse brain in vivo. J Biophotonics. 2019;12:e201800420.
[58]	Samanta S, et al. AIE-active two-photon fluorescent nanoprobe with NIR-II light excitability for highly efficient deep brain vasculature imaging. Theranostics. 2021;11:2137–48.
[59]	Poon C, Teikari P, Rachmadi MF, Skibbe H, Hynynen K. A dataset of rodent cerebrovasculature from in vivo multiphoton fluorescence microscopy imaging. Sci Data. 2023;10:141.
[60]	Qin W, et al. Bright and Photostable Organic Fluorescent Dots with Aggregation-Induced Emission Characteristics for Noninvasive Long-Term Cell Imaging. Adv Funct Mater. 2014;24:635–43.
[61]	Wu W, et al. Polymerization-Enhanced Photosensitization. Chem. 2018;4:1937–51.
[62]	Florence CM, Baillie LD, Mulligan SJ. Dynamic volume changes in astrocytes are an intrinsic phenomenon mediated by bicarbonate ion flux. Plos One. 2012;7:e51124.
[63]	Schindelin J, et al. Fiji: an open-source platform for biological-image analysis. Nat Methods. 2012;9:676–82.
[64]	Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T. & Ronneberger, O. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016. (eds. S. Ourselin, L. Joskowicz, M.R. Sabuncu, G. Unal & W. Wells) 424-432 (Springer International Publishing, Cham; 2016).
[65]	Cortinas-Lorenzo B, Perez-Gonzalez F. Adam and the Ants: On the Influence of the Optimization Algorithm on the Detectability of DNN Watermarks. Entropy. 2020;22:1379.