variation of Newton's method, which works even if the point of initialization does not satisfy the constraints), but can also generally be solved by eliminating the equality constraints with linear algebra or solving the dual problem. Finally, convex optimization with both linear equality constraints and convex inequality constraints can be solved by applying an unconstrained convex optimization technique to the objective function plus logarithmic barrier terms. (When the starting point is not feasible - that is, satisfying the constraints - this is preceded by so-called phase I methods, which either find a feasible point or show that none exist. Phase I methods generally consist of reducing the search in question to yet another convex optimization problem.) Convex optimization problems can also be solved by the following contemporary methods: Bundle methods (Wolfe, Lemaréchal, Kiwiel), and Subgradient projection methods (Polyak), Interior-point methods, which make use of self-concordant barrier functions and self-regular barrier functions. Cutting-plane methods Ellipsoid method Subgradient method Dual subgradients and the drift-plus-penalty method Subgradient methods can be implemented simply and so are widely used. Dual subgradient methods are subgradient methods applied to a dual problem. The drift-plus-penalty method is similar to the dual subgradient method, but takes a time average of the primal variables. Implementations Convex optimization and related algorithms have been implemented in the following software programs: Extensions Extensions of convex optimization include the optimization of biconvex, pseudo-convex, and quasiconvex functions. Extensions of the theory of convex analysis and iterative methods for approximately solving non-convex minimization problems occur in the field of generalized convexity, also known as abstract convex analysis. See also Duality Karush–Kuhn–Tucker conditions Optimization problem Proximal gradient method Notes References Hiriart-Urruty, Jean-Baptiste, and Lemaréchal, Claude. (2004). Fundamentals of Convex analysis. Berlin: Springer. Nesterov, Yurii. (2004). Introductory Lectures on Convex Optimization, Kluwer Academic Publishers Schmit, L.A.; Fleury, C. 1980: Structural synthesis by combining approximation concepts and dual methods. J. Amer. Inst. Aeronaut. Astronaut 18, 1252-1260 External links EE364a: Convex Optimization I and EE364b: Convex Optimization II, Stanford course homepages 6.253: Convex Analysis and Optimization, an MIT OCW course homepage Brian Borchers, An overview of software for convex optimization Convex Optimization Book by Lieven Vandenberghe and Stephen P. Boyd Convex analysis Mathematical optimization