Using Mean Squared Error in Gradient Descent I've recently been writing linear regression algorithms from scratch to gain an understanding of how the maths behind it works (something that was a bit of a black box beforehand), and so I got around to differentiating the cost function. Without realising it I used the Squared Error for the cost function - the MSE but without dividing by the dataset length. Is there any benefit (faster approach of the minimum or other) to using the Mean Squared Error over just summing the squares of the error?
No, it is exactly the same. Optimizing a function and the same function divided by a constant is equivalent, both in the analytical and the numerical sense. You will get exactly the same optimal parameters.