Abstract
Monitoring tasks, such as anomaly and DDoS detection, require identifying frequent flow aggregates based on common IP prefixes. These are known as hierarchical heavy hitters (HHH), where the hierarchy is determined based on the type of prefixes of interest in a given application. The per packet complexity of existing HHH algorithms is proportional to the size of the hierarchy, imposing significant overheads. In this paper, we propose a randomized constant time algorithm for HHH. We prove probabilistic precision bounds backed by an empirical evaluation. Using four real Internet packet traces, we demonstrate that our algorithm indeed obtains comparable accuracy and recall as previous works, while running up to 62 times faster. Finally, we extended Open vSwitch (OVS) with our algorithm and showed it is able to handle 13.8 million packets per second. In contrast, incorporating previous works in OVS only obtained 2.5 times lower throughput.
Original language | English |
---|---|
Title of host publication | SIGCOMM 2017 - Proceedings of the 2017 Conference of the ACM Special Interest Group on Data Communication |
Publisher | Association for Computing Machinery, Inc |
Pages | 127-140 |
Number of pages | 14 |
ISBN (Electronic) | 9781450346535 |
DOIs | |
State | Published - 7 Aug 2017 |
Externally published | Yes |
Event | 2017 Conference of the ACM Special Interest Group on Data Communication, SIGCOMM 2017 - Los Angeles, United States Duration: 21 Aug 2017 → 25 Aug 2017 |
Publication series
Name | SIGCOMM 2017 - Proceedings of the 2017 Conference of the ACM Special Interest Group on Data Communication |
---|
Conference
Conference | 2017 Conference of the ACM Special Interest Group on Data Communication, SIGCOMM 2017 |
---|---|
Country/Territory | United States |
City | Los Angeles |
Period | 21/08/17 → 25/08/17 |
Bibliographical note
Publisher Copyright:© 2017 ACM.
Funding
A downside of RHHH is that it requires some minimal number of packets in order to converge to the desired formal accuracy guarantees. In practice, this is a minor limitation as busy links deliver many millions of packets every second. For example, in the settings reported in Section 4.1, RHHH requires up to 100 millions packets to fully converge, yet even after as little as 8 millions packets, the error reduces to around 1%. With a modern switch that can serve 10 million packets per second, this translates into a 10 seconds delay for complete convergence and around 1% error after 1 second. As line rates will continue to improve, these delays would become even shorter accordingly. The code used in this work is open sourced [4] Acknowledgments. We thank Ori Rottenstreich for his insightful comments and Ohad Eytan for helping with the code release. We would also like to thank the anonymous reviewers and our shepherd, Michael Mitzenmacher, for helping us improve this work. This work was partially funded by the Israeli Science Foundation grant #1505/16 and the Technion-HPI research school. Marcelo Caggiani Luizelli is supported by the research fellowship program funded by CNPq (201798/2015-8).
Funders | Funder number |
---|---|
Conselho Nacional de Desenvolvimento Científico e Tecnológico | 201798/2015-8 |
Israel Science Foundation | 1505/16 |
Keywords
- Heavy Hitters
- Measurement
- Monitoring
- Streaming