Internet and organizational network security is still threatened by devastating malicious activities. Given the continuous escalation of such attacks in terms of their frequency, sophistication and stealthiness, it is of paramount importance to generate effective cyber threat intelligence that aim at inferring, attributing, characterizing and mitigating such misdemeanors. Nevertheless, such imperative tasks are partially impeded by the lack of approaches that can produce prompt and accurate actionable intelligence by investigating various network traffic sources. In this paper, we propose and evaluate a big data architecture that is rooted in real-time network traffic processing, distributed messaging and scalable data storage. The key benefit of the proposed architecture is that it automates the aggregation and transformations of heterogeneous network data, allowing for greater focus on cyber threat intelligence analytics, rather than data management, aggregation, reconciliation and formatting. Empirical evaluations investigating the application of machine learning analytics by exploiting the artifacts resulting from a prototype of the proposed architecture and by using 100 GB of real network traffic, indeed demonstrate the practicality, effectiveness, and added-value of the proposed architecture.