Multi-focus light-field microscopy for high-speed large-volume imaging

Yi Zhang^{1, 2},
Yuling Wang^{1, 2},
Mingrui Wang^{2, 3, 4, 5},
Yuduo Guo^{2, 3, 4},
Xinyang Li^{1, 2, 3, 6},
Yifan Chen^{1, 2},
Zhi Lu^{1, 2},
Jiamin Wu^{1, 2, 4, 5},
Xiangyang Ji^{1, 2},
Qionghai Dai^{1, 2, 4, 5}

1.
Department of Automation, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, 100084, China
2.
Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing, 100084, China
3.
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
4.
Beijing Key Laboratory of Multi-dimension & Multi-scale Computational Photography (MMCP), Tsinghua University, Beijing, 100084, China
5.
IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China
6.
Hangzhou Zhuoxi Institute of Brain and Intelligence, Hangzhou, 311100, China

Funds: We would like to acknowledge Zheng Jiang and Dong Jiang for preparing the zebrafish larvae. We thank Zhifeng Zhao, Jiaqi Fan and Yuwei Huang for providing the Drosophila embryo used for long-term volumetric imaging. We thank Guihua Xiao, Guoxun Zhang, Runan Ji, Rujin Zhang and Qingwei Li for providing transparent mouse brains used for network training and testing, and Qiyu Zhu for his support in mouse surgery.

摘要

Abstract: High-speed visualization of three-dimensional (3D) processes across a large field of view with cellular resolution is essential for understanding living systems. Light-field microscopy (LFM) has emerged as a powerful tool for fast volumetric imaging. However, one inherent limitation of LFM is that the achievable lateral resolution degrades rapidly with the increase of the distance from the focal plane, which hinders the applications in observing thick samples. Here, we propose Spherical-Aberration-assisted scanning LFM (SAsLFM), a hardware-modification-free method that modulates the phase-space point-spread-functions (PSFs) to extend the effective high-resolution range along the z-axis by ~ 3 times. By transferring the foci to different depths, we take full advantage of the redundant light-field data to preserve finer details over an extended depth range and reduce artifacts near the original focal plane. Experiments on a USAF-resolution chart and zebrafish vasculatures were conducted to verify the effectiveness of the method. We further investigated the capability of SAsLFM in dynamic samples by imaging large-scale calcium transients in the mouse brain, tracking freely-moving jellyfish, and recording the development of Drosophila embryos. In addition, combined with deep-learning approaches, we accelerated the three-dimensional reconstruction of SAsLFM by three orders of magnitude. Our method is compatible with various phase-space imaging techniques without increasing system complexity and can facilitate high-speed large-scale volumetric imaging in thick samples.

HTML全文

参考文献(41)

[1]	Keller PJ, Ahrens MB. Visualizing whole-brain activity and development at the single-cell level using light-sheet microscopy. Neuron. 2015;85:462–83.
[2]	Zong W, Wu R, Chen S, et al. Miniature two-photon microscopy for enlarged field-of-view, multi-plane and long-term brain imaging. Nat Methods. 2021;18:46–9.
[3]	Fan JL, Rivera JA, Sun W, Peterson J, Haeberle H, Rubin S, et al. High-speed volumetric two-photon fluorescence imaging of neurovascular dynamics. Nat Commun. 2020. https://doi.org/10.1038/S41467-020-19851-1.
[4]	Chen Z, Özbek A, Rebling J, Zhou Q, Deán-Ben XL, Razansky D. Multifocal structured illumination optoacoustic microscopy. Light Sci Appl. 2020. https://doi.org/10.1038/S41377-020-00390-9.
[5]	Winkelmann JA, Eid A, Spicer G, Almassalha LM, Nguyen TQ, Backman V. Spectral contrast optical coherence tomography angiography enables single-scan vessel imaging. Light Sci Appl. 2019. https://doi.org/10.1038/S41377-018-0117-7.
[6]	Stegmaier J, Amat F, Lemon WC, McDole K, Wan Y, Teodoro G, et al. Real-time three-dimensional cell segmentation in large-scale microscopy data of developing embryos. Dev Cell. 2016;36:225–40.
[7]	Truong TV, Supatto W, Koos DS, Choi JM, Fraser SE. Deep and fast live imaging with two-photon scanned light-sheet microscopy. Nat Methods. 2011;8:757–62.
[8]	Chakraborty T, Chen B, Daetwyler S, Chang BJ, Vanderpoorten O, Sapoznik E, et al. Converting lateral scanning into axial focusing to speed up three-dimensional microscopy. Light Sci Appl. 2020. https://doi.org/10.1038/S41377-020-00401-9.
[9]	Pantazis P, Supatto W. Advances in whole-embryo imaging: a quantitative transition is underway. Nat Rev Mol Cell Biol. 2014;15:327–39.
[10]	Liu TL, Upadhyayula S, Milkie DE, et al. Observing the cell in its native state: imaging subcellular dynamics in multicellular organisms. Science. 2018. https://doi.org/10.1126/SCIENCE.AAQ1392.
[11]	Winter PW, Shroff H. Faster fluorescence microscopy: advances in high speed biological imaging. Curr Opin Chem Biol. 2014;20:46–53.
[12]	Huang Q, Cohen MA, Alsina FC, et al. Intravital imaging of mouse embryos. Science. 2020;368:181–6.
[13]	Nakano A. Spinning-disk confocal microscopy — a cutting-edge tool for imaging of membrane traffic. Cell Struct Funct. 2002;27:349–55.
[14]	Yang B, Chen X, Wang Y, et al. Epi-illumination SPIM for volumetric imaging with high spatial-temporal resolution. Nat Methods. 2019;16(6):501–4.
[15]	Wu J, Liang Y, Chen S, Hsu CL, Chavarha M, Evans SW, et al. Kilohertz two-photon fluorescence microscopy imaging of neural activity in vivo. Nat Methods. 2020;17(3):287–90.
[16]	Zhou Z, Huang J, Li X, Gao X, Chen Z, Jiao Z, et al. Adaptive optical microscopy via virtual-imaging-assisted wavefront sensing for high-resolution tissue imaging. PhotoniX. 2022;3(1):1–20.
[17]	Levoy M, Ng R, Adams A, Footer M, Horowitz M. Light field microscopy. In: ACM SIGGRAPH 2006 Papers. New York: Association for Computing Machinery; 2006. p. 924–34.
[18]	Wu J, Guo Y, Deng C et al. An integrated imaging sensor for aberration-corrected 3D photography. Nature. 2022.
[19]	Nöbauer T, Skocek O, Pernía-Andrade AJ, Weilguny L, Martínez Traub F, Molodtsov MI, et al. Video rate volumetric Ca2+ imaging across cortex using seeded iterative demixing (SID) microscopy. Nat Methods. 2017;14:811–8.
[20]	Wagner N, Norlin N, Gierten J, de Medeiros G, Balázs B, Wittbrodt J, et al. Instantaneous isotropic volumetric imaging of fast biological processes. Nat Methods. 2019;16:497–500.
[21]	Cong L, Wang Z, Chai Y, Hang W, Shang C, Yang W, et al. Rapid whole brain imaging of neural activity in freely behaving larval zebrafish ( Danio rerio). Elife. 2017. https://doi.org/10.7554/ELIFE.28158.
[22]	Zhang Z, Bai L, Cong L, Yu P, Zhang T, Shi W, et al. Imaging volumetric dynamics at high speed in mouse and zebrafish brain with confocal light field microscopy. Nat Biotechnol. 2021;39:74–83.
[23]	Cohen N, Yang S, Andalman A, Broxton M, Grosenick L, Deisseroth K, et al. Enhancing the performance of the light field microscope using wavefront coding. Opt Express. 2014;22:24817.
[24]	Wu J, Lu Z, Jiang D, et al. Iterative tomography with digital adaptive optics permits hour-long intravital observation of 3D subcellular dynamics at millisecond scale. Cell. 2021;184:3318–3332.e17.
[25]	Bo E, Ge X, Luo Y, et al. Cellular-resolution in vivo tomography in turbid tissue through digital aberration correction. https://doi.org/10.1186/s43074-020-00009-7.
[26]	Huang L, Chen H, Luo Y, Rivenson Y, Ozcan A. Recurrent neural network-based volumetric fluorescence microscopy. Light Sci Appl. 2021. https://doi.org/10.1038/S41377-021-00506-9.
[27]	Zhang Y, Lu Z, Wu J, et al. Computational optical sectioning with an incoherent multiscale scattering model for light-field microscopy. Nat Commun. 2021. https://doi.org/10.1038/S41467-021-26730-W.
[28]	Park J, Papoutsi A, Ash RT, Marin MA, Poirazi P, Smirnakis SM. Contribution of apical and basal dendrites to orientation encoding in mouse V1 L2/3 pyramidal neurons. Nat Commun. 2019. https://doi.org/10.1038/S41467-019-13029-0.
[29]	Zhou P, Resendez SL, Rodriguez-Romaguera J, et al. Efficient and accurate extraction of in vivo calcium signals from microendoscopic video data. Elife. 2018. https://doi.org/10.7554/ELIFE.28728.
[30]	Weissbourd B, Momose T, Nair A, Kennedy A, Hunt B, Anderson DJ. A genetically tractable jellyfish model for systems and evolutionary neuroscience. Cell. 2021;184:5854–5868.e20.
[31]	Liu C, Adelson EH, Wilson D. Beyond pixels: exploring new representations and applications for motion analysis; 2009.
[32]	Ihle SJ, Reichmuth AM, Girardin S, Han H, Stauffer F, Bonnin A, et al. Unsupervised data to content transformation with histogram-matching cycle-consistent generative adversarial networks. Nat Mach Intell. 2019;1(10):461–70.
[33]	Li X, Zhang G, Qiao H, et al. Unsupervised content-preserving transformation for optical microscopy. Light Sci Appl. 2021. https://doi.org/10.1038/S41377-021-00484-Y.
[34]	Zhu J-Y, Park T, Isola P, Efros AA, Research BA. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision; 2017. p. 2223–32.
[35]	Li X, Li Y, Zhou Y, et al. Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit. Nat Biotechnol. 2022. https://doi.org/10.1038/s41587-022-01450-8.
[36]	Li X, Zhang G, Wu J, et al. Reinforcing neuron extraction and spike inference in calcium imaging using deep self-supervised denoising. Nat Methods. 2021;18:1395–400.
[37]	Lecoq J, Oliver M, Siegle JH, Orlova N, Ledochowitsch P, Koch C. Removing independent noise in systems neuroscience data using DeepInterpolation. Nat Methods. 2021;18:1401–8.
[38]	Broxton M, Grosenick L, Yang S, Cohen N, Andalman A, Deisseroth K, et al. Wave optics theory and 3-D deconvolution for the light field microscope. Opt Express. 2013;21:25418–39.
[39]	Lu Z, Wu J, Qiao H, Zhou Y, Yan T, Zhou Z, et al. Phase-space deconvolution for light field microscopy. Opt Express. 2019;27:18131.
[40]	Boyd S, Parikh N, Chu E, Peleato B, Eckstein J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and trends®. Mach Learn. 2011;3:1–122.
[41]	Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics): Springer Verlag; 2015. p. 234–41.