Journal of the Korean Mathematical Society (대한수학회지)

Korean Mathematical Society (대한수학회)
권호 표지
DOI QR코드

DOI QR Code

SHADOWING PROPERTY FOR ADMM FLOWS

  • Yoon Mo Jung (Department of Mathematics Sungkyunkwan University) ;
  • Bomi Shin (Institute of Basic Science Sungkyunkwan University) ;
  • Sangwoon Yun (Department of Mathematics Education Sungkyunkwan University)
  • Received : 2023.05.31
  • Accepted : 2023.11.11
  • Published : 2024.03.01
Download PDF
⟨ Previous Next ⟩

Abstract

There have been numerous studies on the characteristics of the solutions of ordinary differential equations for optimization methods, including gradient descent methods and alternating direction methods of multipliers. To investigate computer simulation of ODE solutions, we need to trace pseudo-orbits by real orbits and it is called shadowing property in dynamics. In this paper, we demonstrate that the flow induced by the alternating direction methods of multipliers (ADMM) for a C2 strongly convex objective function has the eventual shadowing property. For the converse, we partially answer that convexity with the eventual shadowing property guarantees a unique minimizer. In contrast, we show that the flow generated by a second-order ODE, which is related to the accelerated version of ADMM, does not have the eventual shadowing property.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2016R1A5A1008055). The work of Y. M. Jung was supported by NRF of Korea (No. 2022R1A2C1010537). The work of B. Shin was supported by NRF of Korea (No. 2021R1C1C2005241). The work of S. Yun was supported by NRF of Korea (No. 2022R1A2C1011503).

References

  1. N. Aoki and K. Hiraide, Topological Theory of Dynamical Systems, North-Holland Mathematical Library, 52, North-Holland, Amsterdam, 1994.
  2. R. Bhatia, Positive Definite Matrices, Princeton Series in Applied Math., 2007.
  3. A. Bielecki and J. Ombach, Shadowing property in analysis of neural networks dynamics, J. Comput. Appl. Math. 164/165 (2004), 107-115. https://doi.org/10.1016/S0377-0427(03)00486-2
  4. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends in Machine Learning 3 (2011), 1-122.
  5. M. H. Dong, K. Lee, and N.-T. Nguyen, Expanding measures for homeomorphisms with eventually shadowing property, J. Korean Math. Soc. 57 (2020), no. 4, 935-955. https://doi.org/10.4134/JKMS.j190453
  6. G. Fran,ca, D. P. Robinson, and R. E. Vidal, ADMM and accelerated ADMM as continuous dynamical system, Proceedings of the 35th International Conference on Machine Learning 80 (2018), 1559-1567.
  7. G. Fran,ca, D. P. Robinson, and R. E. Vidal, A nonsmooth dynamical systems perspective on accelerated extensions of ADMM, IEEE Trans. Automat. Control 68 (2023), no. 5, 2966-2978. https://doi.org/10.1109/tac.2023.3238857
  8. M. Garg and R. Das, Continuous semi-flows with the almost average shadowing property, Chaos Solitons Fractals 105 (2017), 1-7. https://doi.org/10.1016/j.chaos.2017.10.005
  9. C. Good and J. Meddaugh, Orbital shadowing, internal chain transitivity and ω-limit sets, Ergodic Theory Dynam. Systems 38 (2018), no. 1, 143-154. https://doi.org/10.1017/etds.2016.30
  10. L. F. He, Z. H. Wang, and H. Li, Continuous semiflows with the shadowing property, Acta Math. Appl. Sinica 19 (1996), no. 2, 297-303.
  11. A. Jung, A fixed-point of view on gradient methods for big data, Frontiers in Applied Mathematics and Statistics 3 (2017), 18.
  12. W. Jung, K. Lee, and C. A. Morales, Dynamics of G-processes, Stoch. Dyn. 20 (2020), no. 1, 2050037, 30 pp. https://doi.org/10.1142/S0219493720500379
  13. Y. M. Jung, B. Shin, and S. Yun, Global attractor and limit points for nonsmooth ADMM, Appl. Math. Lett. 128 (2022), Paper No. 107890, 8 pp. https://doi.org/10.1016/j.aml.2021.107890
  14. P. E. Kloeden and M. Rasmussen, Nonautonomous dynamical systems, Mathematical Surveys and Monographs, 176, Amer. Math. Soc., Providence, RI, 2011. https://doi.org/10.1090/surv/176
  15. M. Komuro, One-parameter flows with the pseudo-orbit tracing property, Monatsh. Math. 98 (1984), no. 3, 219-253. https://doi.org/10.1007/BF01507750
  16. Y. Nesterov, A method for solving the convex programming problem with convergence rate O(1/k2), Dokl. Akad. Nauk SSSR 269 (1983), no. 3, 543-547.
  17. A. Orbieto and A. Lucchi, Shadowing properties of optimization algorithms, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.
  18. B. G. Pachpatte, Inequalities for differential and integral equations, Mathematics in Science and Engineering, 197, Academic Press, Inc., San Diego, CA, 1998.
  19. R. Vasisht and R. Das, Specification and shadowing properties for non-autonomous systems, J. Dyn. Control Syst. 28 (2022), no. 3, 481-492. https://doi.org/10.1007/s10883-021-09535-4