Research progress in optical neural networks: theory, applications and developments

Jia Liu; Qiuhao Wu; Xiubao Sui; Qian Chen; Guohua Gu; Liping Wang; Shengcai Li

doi:10.1186/s43074-021-00026-0

Article Contents

Article Navigation > PhotoniX > 2021 > Accepted Manuscript

Jia Liu, Qiuhao Wu, Xiubao Sui, Qian Chen, Guohua Gu, Liping Wang, Shengcai Li. Research progress in optical neural networks: theory, applications and developments[J]. PhotoniX. doi: 10.1186/s43074-021-00026-0

Citation:

Jia Liu, Qiuhao Wu, Xiubao Sui, Qian Chen, Guohua Gu, Liping Wang, Shengcai Li. Research progress in optical neural networks: theory, applications and developments[J]. PhotoniX. doi: 10.1186/s43074-021-00026-0

Citation:

PDF( 3127 KB)

Research progress in optical neural networks: theory, applications and developments

doi: 10.1186/s43074-021-00026-0

1.
School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China
2.
Institute of Armored Forces, Army Research Institute, Beijing, China

Funds:

This work was supported in part by the National Natural Science Foundation of China under Grant 11773018 and Grant 61727802, in part by the Key Research and Development programs in Jiangsu China under Grant BE2018126, in part by the Fundamental Research Funds for the Central Universities under Grant 30919011401 and Grant 30920010001, and in part by the Leading Technology of Jiangsu Basic Research Plan under Grant BK20192003.

Received Date: 2020-12-23
Accepted Date: 2021-03-09

Available Online: 2021-04-19

Abstract

Abstract

With the advent of the era of big data, artificial intelligence has attracted continuous attention from all walks of life, and has been widely used in medical image analysis, molecular and material science, language recognition and other fields. As the basis of artificial intelligence, the research results of neural network are remarkable. However, due to the inherent defect that electrical signal is easily interfered and the processing speed is proportional to the energy loss, researchers have turned their attention to light, trying to build neural networks in the field of optics, making full use of the parallel processing ability of light to solve the problems of electronic neural networks. After continuous research and development, optical neural network has become the forefront of the world. Here, we mainly introduce the development of this field, summarize and compare some classical researches and algorithm theories, and look forward to the future of optical neural network.
- Optical neural network,
- Deep learning,
- Optical linear operation,
- Optical nonlinearity,
- Training method

FullText(HTML)

References(93)

References

[1]	Hines ML, Carnevale NT. The neuron simulation environment. Neural Comput. 1997;9(6):1179–209. https://doi.org/10.1162/neco.1997.9.6.1179.
[2]	Schwabe RJ, Zelinger S, Key TS, Phipps KO. Electronic lighting interference. IEEE Ind Appl Mag. 1998;4:46–8.
[3]	Markram H, Muller E, Ramaswamy S. Reconstruction and simulation of neocortical microcircuitry. Cell. 2015;163(2):456–92. https://doi.org/10.1016/j.cell.2015.09.029.
[4]	Tsai F-CF, O'Brien CJ, Petrović NS, Rakić AD. Analysis of optical channel cross talk for free-space optical interconnects in the presence of higher-order transverse modes. Appl Optics. 2005;44(30):6380–7. https://doi.org/10.1364/AO.44.006380.
[5]	Hu W, Li X, Yang J, Kong D. Crosstalk analysis of aligned and misaligned free-space optical interconnect systems. J Opt Soc Am A. 2010;27(2):200–5. https://doi.org/10.1364/JOSAA.27.000200.
[6]	Goodman JW, Dias AR, Woody LM. Fully parallel, high-speed incoherent optical method for performing discrete fourier transforms. Opt Lett. 1978;2(1):1–3. https://doi.org/10.1364/OL.2.000001.
[7]	Hu X, Wang A, Zeng M, Long Y, Zhu L, Fu L, et al. Graphene-assisted multiple-input high-base optical computing. Sci Rep. 2016;6:32911.
[8]	Caulfield HJ, Dolev S. Why future supercomputing requires optics. Nat Photon. 2010;4(5):261–3. https://doi.org/10.1038/nphoton.2010.94.
[9]	Mosca EP, Griffin RD, Pursel FP, Lee JN. Acoustooptical matrix-vector product processor: implementationissues. Appl Optics. 1989;28(18):3843–51. https://doi.org/10.1364/AO.28.003843.
[10]	Sun C-C, Chang M-W, Hsu KY. Matrix-matrix multiplication by using anisotropic self-diffraction in batio3. Appl Optics. 1994;33:4501X507.
[11]	Nasr MB, Chtourou M. A hybrid training algorithm for feedforward neural networks. Neural Process Lett. 2006;24(2):107–17. https://doi.org/10.1007/s11063-006-9013-x.
[12]	de Lima TF, Shastri BJ, Tait AN, Nahmias MA, Prucna PR. Progress in neuromorphic photonics. Nanophotonics. 2017;6(3):577–99. https://doi.org/10.1515/nanoph-2016-0139.
[13]	Chen Y. 4f-type optical system for matrix multiplication. Optim Eng. 1993;32.
[14]	PIAGGIO HTH. The mathematical theory of huygens' principle. Nature. 1940;145(3675):531–2. https://doi.org/10.1038/145531a0.
[15]	Young T. The Bakerian lecture. Experiments and calculations relative to physical optics. Abstr Pap Print Philos Transactions Royal Soc Lond. 1832;1:131–2.
[16]	Mandel L, Wolf E. Some properties of coherent light*. J Opt Soc Am. 1961;51(8):815–9. https://doi.org/10.1364/JOSA.51.000815.
[17]	Porter MB. Concerning green's theorem and the cauchy-riemann differential equations. Ann Math Sec Ser. 1905;7(1):1–2. https://doi.org/10.2307/1967189.
[18]	AL-Jawary MA, Wrobel LC. Numerical solution of the two-dimensional helmholtz equation with variable coefficients by the radial integration boundary integral and integro-differential equation methods. Int J Comput Math. 2012;89:1463–87.
[19]	Umul YZ. Young-kirchhoff-rubinowicz theory of diffraction in the light of sommerfeld's solution. J Opt Soc Am A. 2008;25(11):2734–42. https://doi.org/10.1364/JOSAA.25.002734.
[20]	Sommerfeld A. Optics. Lectures on theoretical physics, vol. iv. Am J Physiol. 1955;23(7):477–8. https://doi.org/10.1119/1.1934064.
[21]	Goodman J. Introduction to Fourier optics: 2rd Edition, Roberts and Company Publishers, Englewood; 1995. p. 35.
[22]	Karczewski B. Fraunhofer diffraction of an electromagnetic wave. J Opt Soc Am. 1961;51(10):1055–7. https://doi.org/10.1364/JOSA.51.001055.
[23]	Wang X, Xu Q, Liu E. Angular spectrum theory to calculate coupling efficiency in rectangular waveguide resonators. Opt Laser Technol. 2000;32(3):177–81. https://doi.org/10.1016/S0030-3992(00)00037-2.
[24]	Lin X, Rivenson Y, Yardimci NT, Veil M, Luo Y, Jarrahi M, et al. All-optical machine learning using diffractive deep neural networks. Science. 2018;361(6406):1004–8. https://doi.org/10.1126/science.aat8084.
[25]	Lu L, Zhu L, Zhang Q, Zhu B, Yao Q, Yu M, et al. Miniaturized diffraction grating design and processing for deep neural network. IEEE Photon Technol Lett. 2019;31(24):1952–5. https://doi.org/10.1109/LPT.2019.2948626.
[26]	Qian C, Lin X, Xu J, Sun Y, Li E, Zhang B, et al. Performing optical logic operations by a diffractive neural network. Light Sci Appl. 2020;9(1):59. https://doi.org/10.1038/s41377-020-0303-2.
[27]	Luo Y, Mengu D, Yardimci NT, Rivenson Y, Veli M, Jarrahi M, et al. Design of task-specific optical systems using broadband diffractive neural networks. Light Sci Appl. 2019;8(1):112. https://doi.org/10.1038/s41377-019-0223-1.
[28]	Liao D, Chan KF, Chan CH, Zhang Q, Wang H. All-optical diffractive neural networked terahertz hologram. Opt Lett. 2020;45(10):2906–9. https://doi.org/10.1364/OL.394046.
[29]	Blackwell CA, Simpson RS. The convolution theorem in modern analysis. IEEE Transact Educ. 1966;9(1):29–32. https://doi.org/10.1109/TE.1966.4321930.
[30]	Lu T, Wu S, Xu X, Yu FTS. Two-dimensional programmable optical neural network. Appl Optics. 1989;28(22):4908–13. https://doi.org/10.1364/AO.28.004908.
[31]	Gao S, Yang J, Feng Z, Zhang Y. Implementation of a large-scale optical neural network by use of a coaxial lenslet array for interconnection. Appl Optics. 1997;36(20):4779–83. https://doi.org/10.1364/AO.36.004779.
[32]	Kuratomi Y, Takimoto A, Akiyama K, Ogawa H. Optical neural network using vector-feature extraction. Appl Optics. 1993;32(29):5750–8. https://doi.org/10.1364/AO.32.005750.
[33]	Chang J, Sitzmann V, Dun X, Heidrich W, Wetzstein G. Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification. Sci Rep. 2018;8:12324.
[34]	Zuo Y, Li B, Zhao Y, Jiang Y, Chen Y-C, Chen P, et al. All-optical neural network with nonlinear activation functions. Optica. 2019;6(9):1132–7. https://doi.org/10.1364/OPTICA.6.001132.
[35]	Breit G. The interference of light and the quantum theory. Proc Natl Acad Sci. 1923;9(7):238–43. https://doi.org/10.1073/pnas.9.7.238.
[36]	Shen Y, Harris NC, Skirlo S, Prabhu M, Baehr-Jones T, Hochberg M, et al. Deep learning with coherent nanophotonic circuits. Nat Photon. 2017;11:44H46.
[37]	Elson JM, Rahn JP, Bennett JM. Light scattering from multilayer optics: comparison of theory and experiment. Appl Optics. 1980;19(5):669–79. https://doi.org/10.1364/AO.19.000669.
[38]	Rochon P, Bissonnette D. Lensless imaging due to back-scattering. Nature. 1990;348(6303):708–10. https://doi.org/10.1038/348708a0.
[39]	Vellekoop IM, Mosk AP. Focusing coherent light through opaque strongly scattering media. Opt Lett. 2007;32(16):2309–11. https://doi.org/10.1364/OL.32.002309.
[40]	Katz O, Small E, Silberberg Y. Looking around corners and through thin turbid layers in real time with scattered incoherent light. Nat Photon. 2012;6(8):549–53. https://doi.org/10.1038/nphoton.2012.150.
[41]	Vellekoop IM, Lagendijk A, Mosk AP. Exploiting disorder for perfect focusing. Nat Photon. 2010;4(5):320–2. https://doi.org/10.1038/nphoton.2010.3.
[42]	Bertolotti J, van Putten EG, Akbulut D, Vos WL, Lagendjk A, Mosk AP. Scattering optics resolve nanostructure. In: Proc. SPIE 8102, Nanoengineering: fabrication, properties, optics, and devices VIII; 2011. p. 810206.
[43]	Huang D, Swanson EA, Lin CP, Schuman JS, Stinson WG, Chang W, et al. Optical coherence tomography. Science. 1991;254(5035):1178–81. https://doi.org/10.1126/science.1957169.
[44]	Katz O, Heidmann P, Fink M, Gigan S. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations. Nat Photon. 2014;8(10):784–90. https://doi.org/10.1038/nphoton.2014.189.
[45]	Yaqoob Z, Psaltis D, Feld MS, Yang C. Optical phase conjugation for turbidity suppression in biological samples. Nat Photon. 2008;2(2):110–5. https://doi.org/10.1038/nphoton.2007.297.
[46]	Ando T, Horisaki R, Tanida J. Speckle-learning-based object recognition through scattering media. Opt Express. 2015;23(26):33902–10. https://doi.org/10.1364/OE.23.033902.
[47]	Pierangeli D, Marcucci G, Moriconi C, Perini G, Spirito MD, Papi EAM. Deep optical neural network by living tumour brain cells. Physis. 2018.
[48]	Khoram E, Chen A, Liu D, Ying L, Wang Q, Yuan M, et al. Nanophotonic media for artificial neural inference. Photon Res. 2019;7(8):823–7. https://doi.org/10.1364/PRJ.7.000823.
[49]	Qu Y, Zhu HZ, Shen YC, Zhang J, Tao CN, Ghosh P, et al. Inverse design of an integrated-nanophotonics optical neural network. Sci Bull. 2020;65(14):1177–83. https://doi.org/10.1016/j.scib.2020.03.042.
[50]	Koester CJ. Wavelength multiplexing in fiber optics. J Opt Soc Am. 1968;58(1):63–70. https://doi.org/10.1364/JOSA.58.000063.
[51]	Paquot Y, Duport F, Smerieri A, Dambre J, Schrauwen B, Haelterman M, et al. Optoelectronic reservoir computing. Sci Rep. 2012;2:287.
[52]	Duport F, Schneider B, Smerieri A, Haelterman M, Massar S. All-optical reservoir computing. Opt Express. 2012;20(20):22783–95. https://doi.org/10.1364/OE.20.022783.
[53]	Cheng T-Y, Chou D-Y, Liu C-C, Chang Y-J, Chen C-C. Optical neural networks based on optical fiber-communication. Neurocomputing. 2019;364:239–44. https://doi.org/10.1016/j.neucom.2019.07.051.
[54]	Zang Y, Chen M, Yang S, Chen H. Electro-optical neural networks based on time-stretch method. IEEE J Sel Top Quantum Electron. 2020;26(1):1–10. https://doi.org/10.1109/JSTQE.2019.2957446.
[55]	Zhang H, Feng X, Li B, Wang Y, Cui K, Liu F, et al. Integrated photonic reservoir computing based on hierarchical time-multiplexing structure. Opt Express. 2014;22(25):31356–70. https://doi.org/10.1364/OE.22.031356.
[56]	Nguimdo RM, Verschaffelt G, Danckaert J, der Sande GV. Simultaneous computation of two independent tasks using reservoir computing based on a single photonic nonlinear node with optical feedback. IEEE Transact Neur Netw Learn Syst. 2015;26(12):3301–7. https://doi.org/10.1109/TNNLS.2015.2404346.
[57]	Maass W. Networks of spiking neurons: the third generation of neural network models. Neural Netw. 1997;10(9):1659–71. https://doi.org/10.1016/S0893-6080(97)00011-7.
[58]	Tait AN, de Lima TF, Zhou E, Wu AX, Nahmias MA, Shastri BJ, et al. Neuromorphic photonic networks using silicon photonic weight banks. Sci Rep. 2017;7:7430.
[59]	Shastri BJ, Nahmias MA, Tait AN, Rodriguez AW, Wu B, Prucnal PR. Spike processing with a graphene excitable laser. Sci Rep. 2016;6:19126.
[60]	Chakraborty I, Saha G, Sengupta A, Roy K. Toward fast neural computing using all-photonic phase change spiking neurons. Sci Rep. 2018;8:12980.
[61]	Feldmann J, Youngblood N, Wright CD, Bhaskaran H, Pernice WHP. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature. 2019;569(7755):208–14. https://doi.org/10.1038/s41586-019-1157-8.
[62]	Nahmias MA, Peng H, de Lima TF, Huang C, Tait AN, Shastri BJ, Prucnal PR. A TeraMAC neuromorphic photonic processor. In: 2018 IEEE photonics Conf. (IPC); 2018. p. 1–2.
[63]	Tait AN, Nahmias MA, Shastri BJ, Prucnal PR. Broadcast and weight: an integrated network for scalable photonic spike processing. J Light Technol. 2014;32(21):4029–41. https://doi.org/10.1109/JLT.2014.2345652.
[64]	Shainline JM, Buckley SM, McCaughan AN, Chiles J, Jafari-Salim A, Mirin RP, et al. Circuit designs for superconducting optoelectronic loop neurons. J Appl Phys. 2018;124(15):152130. https://doi.org/10.1063/1.5038031.
[65]	Selden AC. Pulse transmission through a saturable absorber. Br J Appl Phys. 1967;18(6):743–8. https://doi.org/10.1088/0508-3443/18/6/306.
[66]	Braunstein R. Nonlinear optical effects. Phys Rev. 1962;125(2):475–7. https://doi.org/10.1103/PhysRev.125.475.
[67]	Cotton A. Recherches Sur l'absorption et la dispersion de la lumiere par les milieux doues du pouvoir rotatoire. J Phys Theor Appl. 1896;5(1):237–44. https://doi.org/10.1051/jphystap:018960050023700.
[68]	Skinner SR, Steck JE, Behrman EC. Optical neural network using Kerr-type nonlinear materials. In: Proceedings of the fourth international conference on microelectronics for neural networks and fuzzy systems: IEEE; 1994. p. 12–5.
[69]	Dejonckheere A, Duport F, Smerieri A, Fang L, Oudar J-L, Haelterman M, et al. All-optical reservoir computer based on saturation of absorption. Opt Express. 2014;22(9):10868–81. https://doi.org/10.1364/OE.22.010868.
[70]	Cheng Z, Tsang HK, Wan X, Xu K, Xu J. In-plane optical absorption and free carrier absorption in graphene-on-silicon waveguides. IEEE J Sel Top Quant Electron. 2013;20:43–8.
[71]	Soljacic M, Ibanescu M, Johnson SG, Fink Y, Joannopoulos JD. Optimal bistable switching in nonlinear photonic crystals. Phys Rev E. 2002;66(5):055601. https://doi.org/10.1103/PhysRevE.66.055601.
[72]	Coarer FD, Sciamanna M, Katumba A, Freiberger M, Dambre J, Bienstman P, et al. All-optical reservoir computing on a photonic chip using silicon-based ring resonators. IEEE J Sel Top Quant Electron. 2018;24(6):1–8. https://doi.org/10.1109/JSTQE.2018.2836985.
[73]	Serber R. The theory of depolarization, optical anisotropy, and the Kerr effect. Phys Rev. 1933;43(12):1003–10. https://doi.org/10.1103/PhysRev.43.1003.
[74]	Weinberger P. John Kerr and his effects found in 1877 and 1878. Philos Mag Lett. 2008;88(12):897–907. https://doi.org/10.1080/09500830802526604.
[75]	Mesaritakis C, Kapsalis A, Syvridis D. All-optical reservoir computing system based on ingaasp ring resonators for high-speed identification and optical routing in optical networks. Quant Sens Nanophoton Devices XII. 2015;9370:608–14.
[76]	Steinbrecher GR, Olson JP, Englund D, Carolan J. Quantum optical neural networks. NPJ Quant Inf. 2019;5:60.
[77]	Amin R, George J, Khurgin J, El-Ghazawi T, Prucnal PR, Sorger VJ. Attojoule modulators for photonic neuromorphic computing. In: Conference on lasers and electro-optics: Optical Society of America; 2018. p. ATh1Q.4.
[78]	Amin R, Khan S, Lee CJ, Dalir H, Sorger VJ. 110 attojoule-per-bit efficient graphene-based plasmon modulator on silicon. In: Conference on lasers and electro-optics: Optical Society of America; 2018. p. SM1I.5.
[79]	George JK, Mehrabian A, Amin R, Meng J, de Lima TF, Tait AN, et al. Neuromorphic photonics with electro-absorption modulators. Opt Express. 2019;27(4):5181–91. https://doi.org/10.1364/OE.27.005181.
[80]	George J, Amin R, Mehrabian A, Khurgin J, El-Ghazawi T, Prucnal PR, Sorger VJ. Electrooptic nonlinear activation functions for vector matrix multiplications in optical neural networks. In: Advanced photonics 2018 (BGPP, IPR, NP, NOMA, sensors, networks, SPPCom, SOF): Optical Society of America; 2018. p. SpW4G.3.
[81]	Miscuglio M, Mehrabian A, Hu Z, Azzam SI, George J, Kildishev AV, et al. All-optical nonlinear activation function for photonic neural networks. Opt Mater Express. 2018;8:3851–63.
[82]	Fleischhauer M, Imamoglu A, Marangos JP. Electromagnetically induced transparency: optics in coherent media. Rev Mod Phys. 2005;77(2):633–73. https://doi.org/10.1103/RevModPhys.77.633.
[83]	Williamson IAD, Hughes TW, Minkov M, Bartlett B, Pai S, Fan S. Reprogrammable electro-optic nonlinear activation functions for optical neural networks. IEEE J Sel Top Quantum Electron. 2020;26(1):1–12. https://doi.org/10.1109/JSTQE.2019.2930455.
[84]	Mengu D, Luo Y, Rivenson Y, Ozcan A. Analysis of diffractive optical neural networks and their integration with electronic neural networks. IEEE J Sel Top Quantum Electron. 2020;26(1):1–14. https://doi.org/10.1109/JSTQE.2019.2921376.
[85]	Zhou T, Fang L, Yan T, Wu J, Li Y, Fan J, et al. In situ optical backpropagation training of diffractive optical neural networks. Photon Res. 2020;8(6):940–53. https://doi.org/10.1364/PRJ.389553.
[86]	Hughes TW, Minkov M, Shi Y, Fan S. Training of photonic neural networks through in situ backpropagation and gradient measurement. Optica. 2018;5(7):864–71. https://doi.org/10.1364/OPTICA.5.000864.
[87]	Hughes TW, Williamson IAD, Minkov M, Fan S. Wave physics as an analog recurrent neural network. Sci Adv. 2019;5(12):eaay6946.
[88]	Ba A, Kovalenko A, Aristegui C, Mondain-Monval O, Brunet T. Soft porous silicone rubbers with ultra-low sound speeds in acoustic metamaterials. Sci Rep. 2017;7:40106.
[89]	Qiu J, Si J, Hirao K. Photoinduced stable second-harmonic generation in chalcogenide glasses. Opt Lett. 2001;26(12):914–6. https://doi.org/10.1364/OL.26.000914.
[90]	Karmarkar UR, Najarian MT, Buonomano DV. Mechanisms and significance of spike-timing dependent plasticity. Biol Cybern. 2002;87(5-6):373–82. https://doi.org/10.1007/s00422-002-0351-0.
[91]	Xiang S, Ren Z, Zhang Y, Song Z, Guo X, Han G, et al. Training a multi-layer photonic spiking neural network with modified supervised learning algorithm based on photonic STDP. IEEE J Sel Top Quantum Electron. 2020;27:1–9.
[92]	Vivien L, Polzer A, Marris-Morini D, Osmond J, Hartmann JM, Crozat P, et al. Zero-bias 40Gbit/s germanium waveguide photodetector on silicon. Opt Express. 2012;20(2):1096–101. https://doi.org/10.1364/OE.20.001096.
[93]	Radulaski M, Bose R, Tran T, Van Vaerenbergh T, Kielpinski D, Beausoleil RG. Thermally tunable hybrid photonic architecture for nonlinear optical circuits. ACS Photon. 2018;5(11):4323–9. https://doi.org/10.1021/acsphotonics.8b00376.