How to make model for Multivariate timeseries using tensorflow probability structural model?
You just need to ensure that the model you forecast with has a design matrix covering both the observed and forecasted timesteps. That is, you'd build a model including a component along the lines of

sts.LinearRegression(
  design_matrix=tf.concat([temperature_for_observed_timesteps,
                           temperature_for_forecast_timesteps], axis=-2),
  name='temperature_effect')


(ignoring any centering and reshaping logic) and then pass that model to the forecast method.

If you don't have access to future values of the external regressor when you first build the model, a useful pattern is to encapsulate model building in a method def build_model(observed_time_series, design_matrix) that returns a StructuralTimeSeries model object. Then you can build an initial model with just the observed time steps in order to fit parameters, and then remake the model later once you have the regressors for the forecast steps on hand.