Can categorical features be linearly distributed? Still on my early days, so I cannot really say whether categorical values can be used when linear regression is a key element.

Judging from the below plot, based on my dataset, I'd say this possible (judging from this answer) and the graph below shows I can further use them, can't I?

My question is, can I use the linear regression (e.g. the one from sklearn) to calculate the RSME?
I am not sure what you mean with a linear distribution, but you can use a categorical features with a linear regression (that is, if your dependent variable is continuous). One thing to make though is to encode the categorical feature as such, e.g. use one hot encoding.