• Title/Summary/Keyword: ADMM

Search Result 21, Processing Time 0.024 seconds

Parallel Learning System Optimization using ADMM (ADMM을 이용한 병렬 학습 시스템 최적화)

  • Kim, Min-Woo;Lim, Hwan-Hee;Lee, Byung-Jun;Kim, Kyung-Tae;Youn, Hee-Yong
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2018.07a
    • /
    • pp.49-50
    • /
    • 2018
  • 인공지능의 급격한 발전으로 빅 데이터의 활용이 증가되었지만 이로 인해 머신 러닝에서 일어나는 문제들 또한 해결해야할 과제이다. 본 논문에서는 이에 따라 초래되는 문제들 중 학습 데이터가 많아질 경우의 문제들을 방지하기 위해, 알고리즘의 수정 대신 병렬 처리 기반 시스템을 제안한다. 본 논문에서는 Alternating Direction Method of Multiplier(ADMM) 알고리즘을 소개하고 ADMM 기반의 최적화 기법을 적용하여 병렬 학습 시스템 최적화를 제안하였다.

  • PDF

ADMM for least square problems with pairwise-difference penalties for coefficient grouping

  • Park, Soohee;Shin, Seung Jun
    • Communications for Statistical Applications and Methods
    • /
    • v.29 no.4
    • /
    • pp.441-451
    • /
    • 2022
  • In the era of bigdata, scalability is a crucial issue in learning models. Among many others, the Alternating Direction of Multipliers (ADMM, Boyd et al., 2011) algorithm has gained great popularity in solving large-scale problems efficiently. In this article, we propose applying the ADMM algorithm to solve the least square problem penalized by the pairwise-difference penalty, frequently used to identify group structures among coefficients. ADMM algorithm enables us to solve the high-dimensional problem efficiently in a unified fashion and thus allows us to employ several different types of penalty functions such as LASSO, Elastic Net, SCAD, and MCP for the penalized problem. Additionally, the ADMM algorithm naturally extends the algorithm to distributed computation and real-time updates, both desirable when dealing with large amounts of data.

Fully Distributed Economic Dispatching Methods Based on Alternating Direction Multiplier Method

  • Yang, Linfeng;Zhang, Tingting;Chen, Guo;Zhang, Zhenrong;Luo, Jiangyao;Pan, Shanshan
    • Journal of Electrical Engineering and Technology
    • /
    • v.13 no.5
    • /
    • pp.1778-1790
    • /
    • 2018
  • Based on the requirements and characteristics of multi-zone autonomous decision-making in modern power system, fully distributed computing methods are needed to optimize the economic dispatch (ED) problem coordination of multi-regional power system on the basis of constructing decomposition and interaction mechanism. In this paper, four fully distributed methods based on alternating direction method of multipliers (ADMM) are used for solving the ED problem in distributed manner. By duplicating variables, the 2-block classical ADMM can be directly used to solve ED problem fully distributed. The second method is employing ADMM to solve the dual problem of ED in fully distributed manner. N-block methods based on ADMM including Alternating Direction Method with Gaussian back substitution (ADM_G) and Exchange ADMM (E_ADMM) are employed also. These two methods all can solve ED problem in distributed manner. However, the former one cannot be carried out in parallel. In this paper, four fully distributed methods solve the ED problem in distributed collaborative manner. And we also discussed the difference of four algorithms from the aspects of algorithm convergence, calculation speed and parameter change. Some simulation results are reported to test the performance of these distributed algorithms in serial and parallel.

SHADOWING PROPERTY FOR ADMM FLOWS

  • Yoon Mo Jung;Bomi Shin;Sangwoon Yun
    • Journal of the Korean Mathematical Society
    • /
    • v.61 no.2
    • /
    • pp.395-408
    • /
    • 2024
  • There have been numerous studies on the characteristics of the solutions of ordinary differential equations for optimization methods, including gradient descent methods and alternating direction methods of multipliers. To investigate computer simulation of ODE solutions, we need to trace pseudo-orbits by real orbits and it is called shadowing property in dynamics. In this paper, we demonstrate that the flow induced by the alternating direction methods of multipliers (ADMM) for a C2 strongly convex objective function has the eventual shadowing property. For the converse, we partially answer that convexity with the eventual shadowing property guarantees a unique minimizer. In contrast, we show that the flow generated by a second-order ODE, which is related to the accelerated version of ADMM, does not have the eventual shadowing property.

ADMM algorithms in statistics and machine learning (통계적 기계학습에서의 ADMM 알고리즘의 활용)

  • Choi, Hosik;Choi, Hyunjip;Park, Sangun
    • Journal of the Korean Data and Information Science Society
    • /
    • v.28 no.6
    • /
    • pp.1229-1244
    • /
    • 2017
  • In recent years, as demand for data-based analytical methodologies increases in various fields, optimization methods have been developed to handle them. In particular, various constraints required for problems in statistics and machine learning can be solved by convex optimization. Alternating direction method of multipliers (ADMM) can effectively deal with linear constraints, and it can be effectively used as a parallel optimization algorithm. ADMM is an approximation algorithm that solves complex original problems by dividing and combining the partial problems that are easier to optimize than original problems. It is useful for optimizing non-smooth or composite objective functions. It is widely used in statistical and machine learning because it can systematically construct algorithms based on dual theory and proximal operator. In this paper, we will examine applications of ADMM algorithm in various fields related to statistics, and focus on two major points: (1) splitting strategy of objective function, and (2) role of the proximal operator in explaining the Lagrangian method and its dual problem. In this case, we introduce methodologies that utilize regularization. Simulation results are presented to demonstrate effectiveness of the lasso.

Adaptive OLSR Protocol Based on Average Node Distance in Airdropped Distributed Mobility Model (분산 낙하 이동 모델에서의 평균 노드 거리 기반 적응적 OLSR 프로토콜)

  • Lee, Taekmin;Lee, Jinhae;Wang, Jihyeun;Yoo, Joonhyuk;Yoo, Seong-eun
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.13 no.2
    • /
    • pp.83-91
    • /
    • 2018
  • With the development of IT (Information Technology) technology, embedded system and network technology are combined and used in various environments such as military environment as well as everyday life. In this paper, we propose a new airdropped distributed mobility model (ADMM) modeling the dispersion falling of the direct shot of a cluster bomb, and we compare and analyze some representative MANET routing protocols in ADMM in ns-3 simulator. As a result of the analysis, we show OLSR routing protocol is promising in ADMM environment in the view points of packet delivery ratio (PDR), end to end delay, and jitter. In addition, we propose a new adaptation scheme for OLSR, AND-OLSR (Average Node Distance based adaptive-OLSR) to improve the original OLSR in ADMM environment. The new protocol calculates the average node distance, adapts the period of the control message based on the average node distance increasing rate. Through the simulation study, we show that the proposed AND-OLSR outperforms the original OLSR in PDR and control message overhead.

Image Deblurring Based on ADMM and Deep CNN Denoiser Image Prior (ADMM과 깊은 합성곱 신경망 잡음 제거기 이미지 Prior에 기반한 이미지 디블러링)

  • Kwon, Junhyeong;Soh, Jae Woong;Cho, Nam Ik
    • Proceedings of the Korean Society of Broadcast Engineers Conference
    • /
    • 2020.07a
    • /
    • pp.680-683
    • /
    • 2020
  • 오래 전부터 모델 기반 최적화 방법이 이미지 디블러링을 위해 널리 사용되어 왔고, 최근에는 학습 기반 기술이 영상 디블러링에서 좋은 성과를 보이고 있다. 본 논문은 ADMM과 깊은 합성곱 신경망 잡음 제거기 이미지 prior를 이용하여 모델 기반 최적화 방법의 장점과 학습 기반 방법의 장점을 모두 활용할 수 있는 방법을 제안한다. 본 방법을 이용하여 기존 방법보다 더 좋은 디블러링 성능을 얻을 수 있었다.

  • PDF

Mean-shortfall portfolio optimization via sorted L-one penalized estimation (슬로프 방식을 이용한 평균-숏폴 포트폴리오 최적화)

  • Haein Cho;Seyoung Park
    • The Korean Journal of Applied Statistics
    • /
    • v.37 no.3
    • /
    • pp.265-282
    • /
    • 2024
  • Research in the area of financial portfolio optimization, with the dual goals of increasing expected returns and reducing financial risk, has actively explored various risk measurement indicators. At the same time, the incorporation of various penalty terms to construct efficient portfolios with limited assets has been investigated. In this study, we present a novel portfolio optimization formula that combines the mean-shortfall portfolio and the SLOPE penalty term. Specifically, we formulate this optimization expression, which differs from linear programming, by introducing new variables and using the alternating direction method of multipliers (ADMM) algorithms. Through simulations, we validate the automatic grouping property of the SLOPE penalty term within the proposed mean-shortfall portfolio. Furthermore, using the model introduced in this paper, we propose and evaluate four different types of portfolio compositions relevant to real-world investment scenarios through empirical data analysis.

A depth-based Multi-view Super-Resolution Method Using Image Fusion and Blind Deblurring

  • Fan, Jun;Zeng, Xiangrong;Huangpeng, Qizi;Liu, Yan;Long, Xin;Feng, Jing;Zhou, Jinglun
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.10
    • /
    • pp.5129-5152
    • /
    • 2016
  • Multi-view super-resolution (MVSR) aims to estimate a high-resolution (HR) image from a set of low-resolution (LR) images that are captured from different viewpoints (typically by different cameras). MVSR is usually applied in camera array imaging. Given that MVSR is an ill-posed problem and is typically computationally costly, we super-resolve multi-view LR images of the original scene via image fusion (IF) and blind deblurring (BD). First, we reformulate the MVSR problem into two easier problems: an IF problem and a BD problem. We further solve the IF problem on the premise of calculating the depth map of the desired image ahead, and then solve the BD problem, in which the optimization problems with respect to the desired image and with respect to the unknown blur are efficiently addressed by the alternating direction method of multipliers (ADMM). Our approach bridges the gap between MVSR and BD, taking advantages of existing BD methods to address MVSR. Thus, this approach is appropriate for camera array imaging because the blur kernel is typically unknown in practice. Corresponding experimental results using real and synthetic images demonstrate the effectiveness of the proposed method.

Pairwise fusion approach to cluster analysis with applications to movie data (영화 데이터를 위한 쌍별 규합 접근방식의 군집화 기법)

  • Kim, Hui Jin;Park, Seyoung
    • The Korean Journal of Applied Statistics
    • /
    • v.35 no.2
    • /
    • pp.265-283
    • /
    • 2022
  • MovieLens data consists of recorded movie evaluations that was often used to measure the evaluation score in the recommendation system research field. In this paper, we provide additional information obtained by clustering user-specific genre preference information through movie evaluation data and movie genre data. Because the number of movie ratings per user is very low compared to the total number of movies, the missing rate in this data is very high. For this reason, there are limitations in applying the existing clustering methods. In this paper, we propose a convex clustering-based method using the pairwise fused penalty motivated by the analysis of MovieLens data. In particular, the proposed clustering method execute missing imputation, and at the same time uses movie evaluation and genre weights for each movie to cluster genre preference information possessed by each individual. We compute the proposed optimization using alternating direction method of multipliers algorithm. It is shown that the proposed clustering method is less sensitive to noise and outliers than the existing method through simulation and MovieLens data application.