RNN: Multiple inputs per time step with categorical variables
I would recommend using embedding in this case so that each unique category within a categorical feature is represented as a vector of floats. Then, you can concatenate all vectors coming from different categorical features, and append this long vector along with all other numerical feature values at each timestep.
Besides, if your categorical features are time-invariant (meaning not change over time, e.g. gender), then it might have some issues incorporating static feature values along with other numerical values since RNN is trying to capture the temporal information within the given sequence. But, there is a very helpful link regarding how to add non-temporal features into RNN.
Hope this helps.