Tensorflow/Keras, How to convert tf.feature_column into input tensors?
I think my answer over there https://stackoverflow.com/questions/56693937/tensorflow-how-to-convert-feature-column-to-vector might help you.

Look at 

feature_layer = tf.keras.layers.DenseFeatures(feature_columns)