table some transformations to eliminate constraints penalty functions constrained unconstrained > > exp(yy ( asin (yunfortunatelyan unconstrained minimization method used in combination with these transformations is rarely effective insteadit is more common to use penalty functions the overarching idea of penalty functions is to transform constrained problem into an unconstrained problem by adding weighted constraint-violation terms to the original objective functionwith the premise that the new problem has solution that is identical or close to the original one for exampleif there are only equality constraintsthen ai |hi ( ) ( : (xi= for some constants am and integer { }gives an exact penalty functionin the sense that the minimizer of the penalized function is equal to the minimizer of subject to the equality constraints hm with the addition of inequality constraintsone could use (xf (xai |hi ( ) max{ ( ) = = for some constants am bk alternating direction method of multipliers example (alternating direction method of multipliersthe lagrange method is designed to handle convex minimization subject to equality constraints neverthelesssome practical algorithms may still use the penalty function approach in combination with the lagrangian method an example is the alternating direction method of multipliers (admm[ the admm solves problems of the formmin xrn ,zrm (xg(zsubject toax bz ( where pxn pxm and and rn and rm are convex functions the approach is to form an augmented lagrangian (xzb: (xg(zb(ax bz ckax bz ck where is penalty parameterand are dual variables the admm then iterates through updates of the following formx( + argmin (xz(tb(txrn argmin ( ( + zb(tzrm ( + (tax( + bz( + ( +