Linear Regression with Category variables
We just encode the categorical variable to some sort of numerical representation (like one-hot encoding)

The choice of representation matters, because it has to preserve the properties of a categorical variable: one-hot-encoding is a standard option, but directly encoding categorical values as integers is a mistake because it introduces order where there is none.

the use-cases of such a method: under what kind of circumstances would using categorical data to predict a numeric value be useful?

There are many applications for which regression from categorical data is useful. For example text data is often represented as &quot;bag of words&quot; (BoW), where each word is a categorical variable. Some tasks involve predicting a numerical target from some text input, for example:

Predicting the grade of an essay
Predicting a score representing the likelihood of plagiarism (e.g.Turnitin)
Predicting sentiment of a text on a scale from 1 to 5

There are many other examples, and not only from text data.

And what type of data format should I have my data in before doing the encoding? (does having two columns with one numeric results and the other the corresponding category work?)

The data can have as many independent variables (features) as needed. Since the categorical variables are typically one-hot-encoded, what matters is the number of features after encoding. It's often necessary to preprocess the data in order to avoid overfitting (typically rare values should be discarded).

I would also like to know the different ways we can visualize and analyse the results of our model (and its predictions), especially if we have a sizeable amount of categorical variables.

As far as I know there's no specific method for analyzing the result of the model, it's the same principle as with numerical features. If one wants to analyze the impact of an individual feature on the performance of the model, a simple technique is to train the model with/without this feature and compare the results.