▁useful ▁for ▁some ▁problems . ▁Ar ound ▁ 2 0 0 7 , ▁there ▁was ▁a ▁res urg ence ▁of ▁augment ed ▁L agr ang ian ▁methods ▁in ▁fields ▁such ▁as ▁total - vari ation ▁den o ising ▁and ▁com pressed ▁sens ing . ▁In ▁particular , ▁a ▁variant ▁of ▁the ▁standard ▁augment ed ▁L agr ang ian ▁method ▁that ▁uses ▁partial ▁updates ▁( similar ▁to ▁the ▁G auss - Se idel ▁method ▁for ▁solving ▁linear ▁equations ) ▁known ▁as ▁the ▁altern ating ▁direction ▁method ▁of ▁multip liers ▁or ▁AD MM ▁gained ▁some ▁attention . ▁ ▁General ▁method ▁▁ ▁Let ▁us ▁say ▁we ▁are ▁solving ▁the ▁following ▁const rained ▁problem : ▁ ▁subject ▁to ▁ ▁This ▁problem ▁can ▁be ▁solved ▁as ▁a ▁series ▁of ▁un const rained ▁minim ization ▁problems . ▁ ▁For ▁reference , ▁we ▁first ▁list ▁the ▁k th ▁step ▁of ▁the ▁penalty ▁method ▁approach : ▁ ▁The ▁penalty ▁method ▁solves ▁this ▁problem , ▁then ▁at ▁the ▁next ▁iteration ▁it ▁re - sol ves ▁the ▁problem ▁using ▁a ▁larger ▁value ▁of ▁ ▁( and ▁using ▁the ▁old ▁solution ▁as ▁the ▁initial ▁guess ▁or ▁" w arm - start "). ▁ ▁The ▁augment ed ▁L agr ang ian ▁method ▁uses ▁the ▁following ▁un const rained ▁objective : ▁ ▁and ▁after ▁each ▁iteration , ▁in ▁addition ▁to ▁updating ▁, ▁the ▁variable ▁ ▁is ▁also ▁updated ▁according ▁to ▁the ▁rule ▁ ▁where ▁ ▁is ▁the ▁solution ▁to ▁the ▁un const rained ▁problem ▁at ▁the ▁k th ▁step , ▁i . e . ▁▁ ▁The