A method for constructing time-domain models including Drude materials from finite-element data in the frequency-domain is presented. To produce a model that is compact, projection-based order-reduction techniques are applied. The time-domain model is based on a state space representation and can be proven to be passive and causal. Moreover, the suggested approach provides an efficient way for reconstructing the transient electromagnetic fields.