With the tremendous growth of Internet, large amounts of data are generated and create big challenges for nowadays computing technologies and systems. However, on the other hand, it also sheds new light on the areas of data analytics and mining which enables uncovering the patterns and laws beneath the big data. In recent years, big data analytics have been successfully applied to many areas, such as E-commerce, Healthcare, and Industry. As the same time, security analytics based on big data also receive great attention from both academic and industry. In this paper, we give a comprehensive sketch of techniques about the applications of big data in network security analytics. The existing research works are classified into three types: supervised, unsupervised and hybrid approaches. Then we elaborate the technical issues of the three kinds of approaches and compare their advantages and disadvantages. Finally we outlook the potentials and research directions in the future.