applications where there are too many data to fit into memory or where the data are provided in streaming fashion. One such use is for collaborative filtering in recommendation systems, where there may be many users and many items to recommend, and it would be inefficient to recalculate everything when one user or one item is added to the system. The cost function for optimization in these cases may or may not be the same as for standard NMF, but the algorithms need to be rather different. Convolutional NMF If the columns of represent data sampled over spatial or temporal dimensions, e.g. time signals, images, or video, features that are equivariant w.r.t. shifts along these dimensions can be learned by Convolutional NMF. In this case, is sparse with columns having local non-zero weight windows that are shared across shifts along the spatio-temporal dimensions of , representing convolution kernels. By spatio-temporal pooling of and repeatedly using the resulting representation as input to convolutional NMF, deep feature hierarchies can be learned. Algorithms There are several ways in which the and may be found: Lee and Seung's multiplicative update rule has been a popular method due to the simplicity of implementation. This algorithm is: initialize: and non negative. Then update the values in and by computing the following, with as an index of the iteration. and Until and are stable. Note that the updates are done on an element by element basis not matrix multiplication. We note that the multiplicative factors for and , i.e. the and terms, are matrices of ones when . More recently other algorithms have been developed. Some approaches are based on alternating non-negative least squares: in each step of such an algorithm, first is fixed and found by a non-negative least squares solver, then is fixed and is found analogously. The procedures used to solve for and may be the same or different, as some NMF variants regularize one of and . Specific approaches include the projected gradient descent methods, the active set method, the optimal gradient method, and the block principal pivoting method among several others. Current algorithms are sub-optimal in that they only guarantee finding a local minimum, rather than a global minimum of the cost function. A provably optimal algorithm is unlikely in the near future as the problem has been shown to generalize the k-means clustering problem which is known to be NP-complete. However,