Can all known ML algorithms be written as a sequence of matrix operations? I keep hearing that machine learning is just linear algebra.

Does that mean all known (and all possible?) ML algos, from random forest, to support-vector machines, to recursive neural networks, can be written as a sequence of basic matrix and vector operations, such as multiply, add, sum, etc?

Would writing the algorithm in this way be able to capture all aspects of the model, e.g. architecture, activation functions, learning rate, optimization method?

Can advanced features like convolution be expressed as basic matrix operations?

Is there any ML algorithm for which this isn't true?

Is matrix algebra Turing complete?
Linear algebra is very important for ML, but ML is not "just" linear algebra.

Any sequence of linear matrix-vector operations is linear.
Therefore any non-linear operation cannot be computed using linear algebra. 

A few examples of non-linear operations that are used in ML:


The if statement: e.g. (x &lt; 0) 
The softmax activation
The rectified linear unit



  Can advanced features like convolution be expressed as basic matrix
  operations?


This statement is true actually. A single convolution can be expressed as matrix. However, any modern CNN has non-linearities following the convolution.