clustering method. Types Approximate non-negative matrix factorization Usually the number of columns of and the number of rows of in NMF are selected so the product will become an approximation to . The full decomposition of then amounts to the two non-negative matrices and as well as a residual , such that: . The elements of the residual matrix can either be negative or positive. When and are smaller than they become easier to store and manipulate. Another reason for factorizing into smaller matrices and , is that if one is able to approximately represent the elements of by significantly less data, then one has to infer some latent structure in the data. Convex non-negative matrix factorization In standard NMF, matrix factor ， i.e., can be anything in that space. Convex NMF restricts the columns of to convex combinations of the input data vectors . This greatly improves the quality of data representation of . Furthermore, the resulting matrix factor becomes more sparse and orthogonal. Nonnegative rank factorization In case the nonnegative rank of is equal to its actual rank, is called a nonnegative rank factorization (NRF). The problem of finding the NRF of , if it exists, is known to be NP-hard. Different cost functions and regularizations There are different types of non-negative matrix factorizations. The different types arise from using different cost functions for measuring the divergence between and and possibly by regularization of the and/or matrices. Two simple divergence functions studied by Lee and Seung are the squared error (or Frobenius norm) and an extension of the Kullback–Leibler divergence to positive matrices (the original Kullback–Leibler divergence is defined on probability distributions). Each divergence leads to a different NMF algorithm, usually minimizing the divergence using iterative update rules. The factorization problem in the squared error version of NMF may be stated as: Given a matrix find nonnegative matrices W and H that minimize the function Another type of NMF for images is based on the total variation norm. When L1 regularization (akin to Lasso) is added to NMF with the mean squared error cost function, the resulting problem may be called non-negative sparse coding due to the similarity to the sparse coding problem, although it may also still be referred to as NMF. Online NMF Many standard NMF algorithms analyze all the data together; i.e., the whole matrix is available from the start. This may be unsatisfactory in