This contribution describes a methodology for computing compact time-domain models of electromagnetic devices containing Lorentz materials from finite-element systems in the frequency domain. The procedure starts with projection-based model-order reduction, to downsize system dimension. The resulting reduced-order model is transformed to the time-domain and leads to a state-space representation. The proposed method is mathematically proved to preserve important system properties including passivity, causality, and reciprocity, and allows for reconstructing the transient electromagnetic fields.