The optimal control of fractional-order active isolation system is researched based on the optimal control theory, and the effect of fractional-order derivative on passive isolation system is also analyzed. The mechanical model is established where viscoelastic features of isolation materials are described by fractional-order derivative. The viscoelastic property of the fractional-order derivative in dynamical system is studied and the fractional-order derivative could be divided into linear stiffness and linear damping. It is found that both the fractional coefficient and the fractional order could affect not only the resonance amplitude through the equivalent linear damping coefficient but also the resonance frequency by the equivalent linear stiffness. Based on optimal control theory, the feedback gain of fractional-order active isolation system under harmonic excitation is obtained, which is changed with the excitation frequency. The statistical responses of the displacement and velocity for passive and active vibration isolation systems subjected to random excitation are also presented, which further verifies the excellent performance of fractional-order derivative in vibration control engineering.