How to implement sequence to sequence models?
you need to define how many time steps you want to have in each time series block.
then for each unique patient, you need to create these blocks so the training set going to be a 3D matrix, and the dimensions are:
number of blocks * number of time steps * number of features
in addition to time series data, you can also add another head to feed NN with the constant data like patient id, day of the week, and ...
for the time series part, you can use biLSTM or biGRU or and other RNN layers and then a Dense layer and then concatanate the results with the results coming from constant features.
for the output, you can also use an RNN layer to predict the sequence of future visits or simply use Dense layers.
==============Updated
consider below flow chart
if your other features (like drugs, lab tests) don't have the same time structure, consider it as groups of time series and for each one add other heads along with RNNs with different input shapes (for the number of time steps). the key is to bring all your features to the latent space and from there, do the decoding and predict the sequence of the future.