Description
Yurii Nesterov (CORE/INMA, UCLouvain, Belgium) professzor a Budapesti Corvinus Egyetem, Corvinus Institute for Advanced Studies (CIAS) vendége lesz februárban. Ennek keretében:
CCOR minikurzus: Modern Theory of Second-Order Methods
február 8., szerda 9:30 - 11:00
február 9., csütörtök 10:00 - 11:30
február 14., kedd 10:00 - 11:30
Az előadásokat hibrid formában szervezzük, online Zoomon lehet majd csatlakozni.
For Zoom access please contact E.-Nagy Marianna (marianna.eisenberg-nagy[at]uni-corvinus.hu).
In this course we present the latest achievements in the theory of the second-order schemes. We start from the global complexity estimates for the new second-order methods, based on the cubic regularization of quadratic model of the objective function. We consider several complexity bounds for different classes of nonconvex functions. Next topic is the accelerated second-order methods for convex functions and the lower complexity bounds. We finish the course with the universal second order schemes.
Lecture 1: Global complexity bounds for 2nd-order methods. Systems of nonlinear equations.
Historical remarks
Trust region methods
Cubic regularization of second-order model
Local and global convergence
Solving the system of nonlinear equations
Numerical experiments
Lecture 2: Accelerated second-order methods. Lower complexity bounds.
Composite convex optimization problem
Composite Cubic Newton Method
Complexity bound
Non-degeneracy for 2nd-order methods
Estimating sequences
Accelerated Cubic Newton
Lecture 3: Universal 2nd-order methods.
Problem formulation
Holder classes for second derivative
Main inequalities
Regularized methods for particular Holder class
Accelerated method
Universal accelerated method
References
1. Yu. Nesterov, B. Polyak. Cubic regularization of Newton's method and its global performance. Mathematical Programming, 108(1), 177-205 (2006).
2. Yu. Nesterov. Accelerating the cubic regularization of Newton's method on convex problems. Mathematical Programming, 112(1) 159-181 (2008)
3. Yu. Nesterov. Modified Gauss-Newton scheme with worst-case guarantees for its global performance. Optimization Methods and Software, 22(3) 469-483 (2007)
4. Yu. Nesterov. Complexity bounds for primal-dual methods minimizing the model of objective function. Mathematical Programming, 171, 311-330 (2018)
5. G.N. Grapiglia, Yu. Nesterov. Regularized Newton methods for minimizing functions with Holder continuous Hessians. SIOPT, 27(1), 478-506 (2017).
8. Yu. Nesterov. Superfast 2nd-order methods for unconstrained convex optimization. JOTA, 191, 1-30 (2021).
OR (instead of 1-5): sections 4.1-4.4, and section 6.4.6 of
Yu. Nesterov. Lecture notes on Convex Optimization. Springer (2018).