Security protocols are the key to ensure network security. In the context of the state of the art, so many methods have been developed to analyze the security properties of security protocols, such as Ban logic, theorem proving and model checking etc. This paper used model checking method to formally verify security protocols because of its high degree of automation, briefness and effectiveness. The model checker Spin with sound algorithm design has an extraordinary ability of checking and a good support for LTL. This paper studied the use of Spin on security protocols, and proposed a more effective intruder model to formally verify the security properties of security protocols, such as authentication. The method in this paper decreased the number of model states by a wide margin, and avoided the state space explosion effectively. This paper exampled NSPK protocol and DS protocol, and good experimental results were shown.