6.4 Convergence Analysis¶
Global Convergence of The BFGS Method¶
Superliner Convergence of the BFGS Method¶
Convergence Analysis of the SR1 Method¶
No global results like Theorem 6.5 or local superlinear results like Theorem 6.6 have been established, except the results for quadratic functions discussed earlier.
- Theorem 6.7. Suppose that the iterates \(x_k\) are generated by Algorithm 6.2. Suppose also that the following conditions hold:
The sequence of iterates does not terminate, but remains in a closed, boudned, convex set \(D\), on which the function \(f\) is twice continuously differentiable, and in which \(f\) has a unique staionary point \(x^*\);
The Hessian \(\nabla^2 f(x^*)\) is postive definite, and \(\nabla^2 f(x)\) is Lipschtiz continuous in a neighborhood of \(x^*\);
The sequence of matrices \(\{B_k\}\) is bounded in norm;
\(\left\lvert s_k^\top (y_k - B_ks_k) \right\rvert \geq r \lVert s_k \rVert \lVert y_k - B_ks_k \rVert\) holds at every iteration, where \(r \in (0, 1)\);
Then \(\lim_{k \to \infty}x_k = x^*\), and we have that
Also, it can be shown that