Smart materials and structures is an international frontier field in current development of engineering and science. Representative of soft smart materials include Electroactive polymers (EAPs) and Shape Memory Polymers (SMPs), etc.. As a new kind of smart deformation material, SMPs have a wide range of applications in the field of smart material and structures due to their controllable shape memory effects. Deformation mechanism of SMP material is the basis of its applications. This paper proposed an useful thermoviscoelastic constitutive model by considering thermal expansion, structure relaxation and viscoelastic properties of Epoxy-SMP material. To verify the applicability of the model, various experiments such as isothermal uniaxial tensile tests were carried out and then be simulated. The results showed that the constitutive model could nicely predict mechanical behavior of Epoxy-SMP, the proposed constitutive model is useful for the design of SMPs structures.