A planar compound eye based microsystem for high precision 3D perception

1.
Department of Precision Instrument, Tsinghua University, Beijing, 100084, China
2.
State Key Laboratory of Precision Measurement Technology and Instrument, Tsinghua University, Beijing, 100084, China
3.
Beijing Advanced Innovation Center for Integrated Circuits, Tsinghua University, Beijing, 100084, China

Funds: This work was financially supported by the National Key Research and Development Program of China (2023YFB3906300).

摘要

- /
- /
- /
- /
- /
- /
- /
- /
Abstract: Three-dimensional (3D) panoramic vision system plays a fundamental role in the biological perception of external information, and naturally becomes a key system for embodied intelligence to interact with the outside world. A binocular vision system with rotating eyeball has long baseline, large volume and weak sensitivity to motion. A compound eye system has small volume, high sensitivity to motion but poor precision. Here, a planar compound eye microsystem for high precision 3D perception is proposed by combining semiconductor manufacturing process and biological compound eye structure. Using a semiconductor planar image sensor as the sensing unit, a space-coded planar sub-eye array is designed and its sub field of view (FOV) is dynamically mapped to the image sensor. It solves the problem that a traditional vision system cannot simultaneously accommodate wide FOV with long focal length and high sensitivity to motion with high resolution. The parallax among different sub-eyes enables the system to accurately perceive and dynamically track the 3D position of the target in the range of 10 m and within the FOV of 120 ° in a single compound eye. This system is of great significance in the fields of intelligent robot and intelligent perception.

HTML全文

下载: 全尺寸图片幻灯片

参考文献(43)

[1]	Lee LP, Szema R. Inspirations from biological, optics for advanced phtonic systems. Science. 2005;310:1148–50. https://doi.org/10.1126/science.1115248.
[2]	Sarkar, M., Theuwissen, A. in A Biologically Inspired CMOS Image Sensor Vol. 461 Studies in Computational Intelligence 1-11 (Springer-Verlag Berlin, 2013).
[3]	Toates FM. Accommodation function of Human eye. Physiol Rev. 1972;52:828–000. https://doi.org/10.1152/physrev.1972.52.4.828.
[4]	Jagger WS. The optics of the spherical fish Lens. Vision Res. 1992;32:1271–84. https://doi.org/10.1016/0042-6989(92)90222-5.
[5]	Easter SS, Hitchcock PF. The myopic eye of the black moor goldfish. Vision Res. 1986;26:1831–000. https://doi.org/10.1016/0042-6989(86)90135-5.
[6]	Rossel S. Binocular stereopsis in an insect. Nature. 1983;302:821–2. https://doi.org/10.1038/302821a0.
[7]	Sarkar, M. & Theuwissen, A. in A Biologically Inspired CMOS Image Sensor Vol. 461 Studies in Computational Intelligence 13-48 (Springer-Verlag Berlin, 2013).
[8]	Land MF, Nilsson DE, Land MF, Nilsson DE. Apposition compound eyes. New York: Oxford Univ Press; 2012.
[9]	Song YM, et al. Digital cameras with designs inspired by the arthropod eye. Nature. 2013;497:95–9. https://doi.org/10.1038/nature12083.
[10]	Keum D, et al. Xenos peckii vision inspires an ultrathin digital camera. Light-Science & Applications. 2018;7:7. https://doi.org/10.1038/s41377-018-0081-2.
[11]	Kim K, Jang KW, Ryu JK, Jeong KH. Biologically inspired ultrathin arrayed camera for high-contrast and high-resolution imaging. Light-Science & Applications. 2020;9:7. https://doi.org/10.1038/s41377-020-0261-8.
[12]	Kogos LC, et al. Plasmonic ommatidia for lensless compound-eye vision. Nature Communications. 2020;11:9. https://doi.org/10.1038/s41467-020-15460-0.
[13]	Zhu, L Z., Y. L. Sun, H B. Miniaturising artificial compound eyes based on advanced micronanofabrication techniques. Light Advanced Manufacturing.2021;(2):84-100 https://doi.org/10.37188/lam.2021.007
[14]	Floreano D, et al. Miniature curved artificial compound eyes. Proc Natl Acad Sci U S A. 2013;110:9267–72. https://doi.org/10.1073/pnas.1219068110.
[15]	Shi CY, et al. SCECam: a spherical compound eye camera for fast location and recognition of objects at a large field of view. Optics Express. 2017;25:32333–45. https://doi.org/10.1364/oe.25.032333.
[16]	Zheng YL, Song L, Huang JX, Zhang HY, Fang FZ. Detection of the three-dimensional trajectory of an object based on a curved bionic compound eye. Opt Lett. 2019;44:4143–6. https://doi.org/10.1364/ol.44.004143.
[17]	Ma MC, et al. Target orientation detection based on a neural network with a bionic bee-like compound eye. Optics Express. 2020;28:10794–805. https://doi.org/10.1364/oe.388125.
[18]	Dai B, et al. Biomimetic apposition compound eye fabricated using microfluidic-assisted 3D printing. Nature Communications. 2021;12:11. https://doi.org/10.1038/s41467-021-26606-z.
[19]	Feng X, et al. A Meniscus Multifocusing Compound Eye Camera Based on Negative Pressure Forming Technology. Micromachines. 2023;14:12. https://doi.org/10.3390/mi14020420.
[20]	Hu ZY, et al. Miniature optoelectronic compound eye camera. Nat Commun. 2022;13:5634. https://doi.org/10.1038/s41467-022-33072-8.
[21]	Zhang H, et al. Development of a low cost high precision three-layer 3D artificial compound eye. Optics Express. 2013;21:22232–45. https://doi.org/10.1364/oe.21.022232.
[22]	Chung-You, L., Jun-Fu, C., Ting-Chieh, Y., Wang, K. Use bionic microlens array and CMOS image sensor for three-dimensional motion detection. 2012 7th IEEE International Conference on Nano/Micro Engineered and Molecular Systems (NEMS). 2021;388-391 https://doi.org/10.1109/nems.2012.6196800.
[23]	Chen AH, He BW, Gao CH. In: 3rd International Conference on Mechatronics and Control Engineering (ICMCE). Switzerland: Trans Tech Publications Ltd; 2014. p 290–4.
[24]	Frost SA, Gorospe GE, Teubert C. Compound eye sensor for real-time aircraft wing deflection measurement. AIAA Guidance Navigation and Control Conference. 2016;9–9. https://doi.org/10.2514/6.2016-1136.
[25]	Xian, D., Su, Q., Fengwen, M. & Weiqi, J. Development of a 3x3-channel bionic compound eyes imaging system for target positioning. Proc. SPIE (USA) 11885, 118850V (118810 pp.)-118850V (118810 pp.), https://doi.org/10.1117/12.2602394 (2021).
[26]	Ma MC, Guo F, Cao ZL, Wang KY. Development of an artificial compound eye system for three-dimensional object detection. Applied Optics. 2014;53:1166–72. https://doi.org/10.1364/ao.53.001166.
[27]	Jian HJ, He JZ, Jin XY, Chen XC, Wang KY. Automatic geometric calibration and three-dimensional detecting with an artificial compound eye. Applied Optics. 2017;56:1296–301. https://doi.org/10.1364/ao.56.001296.
[28]	Pang K, Fang FZ, Song L, Zhang Y, Zhang HY. Bionic compound eye for 3D motion detection using an optical freeform surface. J Opt Soc Am B-Opt Phys. 2017;34:B28–35. https://doi.org/10.1364/josab.34.000b28.
[29]	Li L, Hao YP, Xu JL, Liu FL, Lu J. The Design and Positioning Method of a Flexible Zoom Artificial Compound Eye. Micromachines. 2018;9:14. https://doi.org/10.3390/mi9070319.
[30]	Kral K, Poteser M. Motion parallax as a source of distance information in locusts and mantids. J Insect Behav. 1997;10:145–63. https://doi.org/10.1007/bf02765480.
[31]	Kapustjansky A, Chittka L, Spaethe J. Bees use three-dimensional information to improve target detection. Naturwissenschaften. 2010;97:229–33. https://doi.org/10.1007/s00114-009-0627-5.
[32]	Zhang L, Zhan HY, Liu XY, Xing F, You Z. A wide-field and high-resolution lensless compound eye microsystem for real-time target motion perception. Microsyst Nanoeng. 2022;8:9. https://doi.org/10.1038/s41378-022-00388-w.
[33]	Zhan H, et al. Analyzing the Effect of the Intra-Pixel Position of Small PSFs for Optimizing the PL of Optical Subpixel Localization. Engineering. 2023. https://doi.org/10.1016/j.eng.2023.03.009.
[34]	Braddick O. VISUAL HYPERACUITY. Nature. 1984;308:228–9. https://doi.org/10.1038/308228a0.
[35]	Liu B, He Z. Genetic Algorithm based MIMO Radar Polyphase Code Design. Journal of Electronic Measurement and Instrument. 2008;22:62–6.
[36]	Asim B. Advances in Theory and Applications of Stereo Vision. Croatia: INTECH d.o.o.; 2011. p 39–43.
[37]	Okutomi M, Kanade T. A MULTIPLE-BASE-LINE STEREO. IEEE Trans Pattern Anal Mach Intell. 1993;15:353–63. https://doi.org/10.1109/34.206955.
[38]	Wang DK, Watkins C, Xie HK. MEMS Mirrors for LiDAR: A Review. Micromachines. 2020;11:24. https://doi.org/10.3390/mi11050456.
[39]	Zhang, X. S., Kwon, K., Henriksson, J., Luo, J. H. & Wu, M. C. A large-scale microelectromechanical-systems-based silicon photonics LiDAR. Nature 603, 253-+, https://doi.org/10.1038/s41586-022-04415-8 (2022).
[40]	Muralikrishnan B, Phillips S, Sawyer D. Laser trackers for large-scale dimensional metrology: A review. Precis Eng-J Int Soc Precis Eng Nanotechnol. 2016;44:13–28. https://doi.org/10.1016/j.precisioneng.2015.12.001.
[41]	Vikas & Sahu, R. K. A review on application of laser tracker in precision positioning metrology of particle accelerators. Precis. Eng.-J. Int. Soc. Precis. Eng. Nanotechnol. 2021;71, 232-249, https://doi.org/10.1016/j.precisioneng.2021.03.015.
[42]	Katoch S, Chauhan SS, Kumar V. A review on genetic algorithm: past, present, and future. Multimed Tools Appl. 2021;80:8091–126. https://doi.org/10.1007/s11042-020-10139-6.
[43]	Zhang, Y. F. et al. Conceptual design of the optical system of the 6.5m wide field multiplexed survey telescope with excellent image quality. PhotoniX. 2023;4:25, https://doi.org/10.1186/s43074-023-00094-4