Although there exist very accurate hardware systems for measuring traffic on the internet, their widespread use for analysis tasks is limited by their high cost. On the other hand, less expensive, software-based systems exist that are widely available and can be used to perform a number of simple analysis tasks. The caveat with using such software systems is that application of standard analysis methods cannot proceed blindly because inherent distortions exist in the measurements obtained from software systems. The goal of this paper is to analyze common Internet measurement systems to discover the effect of these distortions on common analysis tasks. Then by selecting one specific task, periodic signal detection, a more in-depth an alysis is conducted which derives a signal representation to capture the salient features of the measurement and develops a periodic detection mechanism designed for the measurement system which outperforms an existing detection method not optimized for the measurement system. Finally, through experiments the importance of understanding the relationship between the input traffic, measurement system configuration and detection method performance is em phasized.
@cite_8: Several high-bandwidth network interfaces use Interrupt Coalescence (IC), i.e., they generate a single interrupt for multiple packets sent or received in a short time interval. IC decreases the per-packet interrupt processing overhead. However, IC also introduces queueing delays and alters the “dispersion” (i.e., interarrival time spacing) of packet pairs or trains. In this work, we first explain how IC works in two popular Gigabit Ethernet controllers. Then, we identify the negative effects of IC on active and passive network measurements. Specifically, we show that IC can affect active bandwidth estimation techniques, causing erroneous measurements. It can also alter the packet interarrivals in passive monitors that use commodity network interfaces. Based on the “signature” that IC leaves on the dispersion and one-way delays of packet trains, we show how to detect IC and how to filter its effects from raw measurements. Finally, we show that IC can be detrimental to TCP self-clocking, causing bursty delivery of ACKs and subsequent bursty transmission of data segments.
@cite_10: Internet traffic contains a rich set of periodic patterns. Examples include regular packet transmissions along bottleneck links, periodic routing information exchange, and periodicities inside Denial-of-Service attack streams. Analyzing such periodic patterns has wide applications, including a better understanding of network traffic dynamics, diagnosis of network anomalies, and detection of Denial-of-Service attacks. However, current understanding of periodic behavior in aggregate traffic is quite limited. Many previous approaches often analyze traffic on a per-flow basis, and are not suited to analyze high speed aggregate traffic. In addition, a number of approaches only indicate that they may reveal periodic patterns, but fall short of proposing automatic detection algorithms and quantitatively evaluating their performance. This thesis explores the application of spectral and statistical methods to detect periodic patterns in Internet traffic. In our approach we first apply spectral techniques to obtain the traffic spectrum, and then use algorithms based on rigorous statistical methods to automatically detect periodic patterns from the traffic spectrum. One salient feature of our approach is that it operates at the aggregate traffic level and does not require flow separation. We first carry out controlled lab experiments to demonstrate the spectral characteristics of various periodic patterns. We then propose four non-parametric detection algorithms and evaluate their performance using real-world Internet traffic. Results show that one of them, the Top-Frequency Algorithm, is the best choice in terms of detection performance and algorithm simplicity. It can provide excellent accuracy (up to 95 ) for detecting the periodic pattern caused a bottleneck link even when the traffic through the bottleneck accounts for less than 10 of the aggregate traffic observed at the monitoring point. We also investigate two extensions to our algorithms. The first one is to utilize harmonics, and the second one is to have parametric detection that considers the variation of traffic spectra according to other factors, such as traffic volume. Evaluation results show that we can get significant improvement by considering harmonics for traffic similar to the training data and marginal improvement by considering traffic volume for parametric detection.
@cite_19: ■ An increase in traffic volumes means that maltraffic can easily hide in background traffic, yet still do great harm. ■ The appearance of application cloaking, both intentional (varying port allocation or actively concealing
@cite_13: This paper develops two parametric methods to detect low-rate denial-of-service attacks and other similar near-periodic traffic, without the need for flow separation. The first method, the periodic attack detector, is based on a previous approach that exploits the near-periodic nature of attack traffic in aggregate traffic by modeling the peak frequency in the traffic spectrum. The new method adopts simple statistical models for attack and background traffic in the time-domain. Both approaches use sequential probability ratio tests (SPRTs), allowing control over false alarm rate while examining the trade-off between detection time and attack strength. We evaluate these methods with real and synthetic traces, observing that the new Poisson- based scheme uniformly detects attacks more rapidly, often in less than 200 ms, and with lower complexity than the periodic attack detector. Current entropy-based detection methods provide an equivalent time to detection but require flow-separation since they utilize source destination IP addresses. We evaluate sensitivity to attack strength (compared to the rate of background traffic) with synthetic traces, finding that the new approach can detect attacks that represent only 10 of the total traffic bitrate in fractions of a second.
@cite_17: Denial of Service attacks have become a weapon for extortion and vandalism causing damages in the millions of dollars to commercial and government sites. Legal prosecution is a powerful deterrent, but requires attribution of attacks, currently a difficult task. In this paper we propose a method to automatically fingerprint and identify repeated attack scenarios—a combination of attacking hosts and attack tool. Such fingerprints not only aid in attribution for criminal and civil prosecution of attackers, but also help justify and focus response measures. Since packet contents can be easily manipulated, we base our fingerprints on the spectral characteristics of the attack stream which are hard to forge. We validate our methodology by applying it to real attacks captured at a regional ISP and comparing the outcome with header-based classification. Finally, we conduct controlled experiments to identify and isolate factors that affect the attack fingerprint.
Prasad looked at how interrupt coalescence altered standard network tomography methods, and discussed some methods to detect the presence of coalescence in measured signals in @cite_8 . Our work is different because we do not simply show the effect of the measurement system, but instead we use our knowledge of the measurement system and its inherent effect to design strategies for a specific networking problem, namely DOS attack detection. We chose DOS attack detection because recent work @cite_17 @cite_19 has shown malware and specifically DOS attacks can produce periodic traffic. Thus DOS attack detection becomes a problem of periodic signal detection, which is a common signal processing task. For comparison we use an attack detection method, called the periodic attack detector (PAD), proposed by He in @cite_10 , which is was shown to produce good detection performance in @cite_13 . The PAD is different from our method because it is not designed for a specific measurement system.