Lasso regression not getting better without random features
In answer to your first question:
The reason that your RMSE proceeded to increase as you increased the strength of your regularization (the value of $\lambda$) can be explained by reviewing the intuition behind what is happening when you increase the regularization of your model.

Why did could my RMSE have kept increasing as I increased my regularization strength?

When you have no penalty (i.e. $\lambda = 0$) the error for your model (RMSE) is inflated significantly. As you penalize the model this error begins to fall. At some point in your modeling, by increasing the value of $\lambda$ too much, you are adding too much bias to the model. At this point, the bias will become too large and your model will begin to under-fit the data. This will result in an increase in the RMSE.
For your model above, this seems to occur somewhere between $\lambda = 10^{-3}$ and $\lambda = 10^{-2}$.
In answer to your second question:
Lasso Regression can be used as a form of feature selection since it shrinks some of the values for the coefficients down to zero. If you increase the value of $\lambda$ significantly, many of the features will be shrunk to zero.

Does the fact that the coefficients of feat1 and feat3 were pushed to
0 at the same time as the random feature mean that they're not good
and I should remove them from the model? (It felt strange that they
hit 0 at the same time. I was expecting some difference.)

To decide whether you should remove any of these features, you should consider a few things. What is the degree of collinearity that exists between your features. You can check this easily since it looks like your have a small number of features. Also, it is important to consider the number of features you have before regularization. If you have a small number, perhaps you do not want to drop a large amount of them.
Most importantly, you should examine the change in your RMSE error associated with each increase in $\lambda$ and choose your $\lambda$ that minimizes the RMSE. If certain of the features are pushed to zero based on the optimal choice of $\lambda$ then maybe they should be removed. The rate at which they move towards zero (both at the same time) does not necessarily suggest that they should be removed, only that they might be candidate features to remove.