Constant time updates in hierarchical heavy hitters

Ran Ben Basat, Gil Einziger, Roy Friedman, Marcelo C. Luizelli, Erez Waisbard

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

111 Scopus citations

Abstract

Monitoring tasks, such as anomaly and DDoS detection, require identifying frequent flow aggregates based on common IP prefixes. These are known as hierarchical heavy hitters (HHH), where the hierarchy is determined based on the type of prefixes of interest in a given application. The per packet complexity of existing HHH algorithms is proportional to the size of the hierarchy, imposing significant overheads. In this paper, we propose a randomized constant time algorithm for HHH. We prove probabilistic precision bounds backed by an empirical evaluation. Using four real Internet packet traces, we demonstrate that our algorithm indeed obtains comparable accuracy and recall as previous works, while running up to 62 times faster. Finally, we extended Open vSwitch (OVS) with our algorithm and showed it is able to handle 13.8 million packets per second. In contrast, incorporating previous works in OVS only obtained 2.5 times lower throughput.

Original languageEnglish
Title of host publicationSIGCOMM 2017 - Proceedings of the 2017 Conference of the ACM Special Interest Group on Data Communication
PublisherAssociation for Computing Machinery, Inc
Pages127-140
Number of pages14
ISBN (Electronic)9781450346535
DOIs
StatePublished - 7 Aug 2017
Externally publishedYes
Event2017 Conference of the ACM Special Interest Group on Data Communication, SIGCOMM 2017 - Los Angeles, United States
Duration: 21 Aug 201725 Aug 2017

Publication series

NameSIGCOMM 2017 - Proceedings of the 2017 Conference of the ACM Special Interest Group on Data Communication

Conference

Conference2017 Conference of the ACM Special Interest Group on Data Communication, SIGCOMM 2017
Country/TerritoryUnited States
CityLos Angeles
Period21/08/1725/08/17

Bibliographical note

Publisher Copyright:
© 2017 ACM.

Funding

A downside of RHHH is that it requires some minimal number of packets in order to converge to the desired formal accuracy guarantees. In practice, this is a minor limitation as busy links deliver many millions of packets every second. For example, in the settings reported in Section 4.1, RHHH requires up to 100 millions packets to fully converge, yet even after as little as 8 millions packets, the error reduces to around 1%. With a modern switch that can serve 10 million packets per second, this translates into a 10 seconds delay for complete convergence and around 1% error after 1 second. As line rates will continue to improve, these delays would become even shorter accordingly. The code used in this work is open sourced [4] Acknowledgments. We thank Ori Rottenstreich for his insightful comments and Ohad Eytan for helping with the code release. We would also like to thank the anonymous reviewers and our shepherd, Michael Mitzenmacher, for helping us improve this work. This work was partially funded by the Israeli Science Foundation grant #1505/16 and the Technion-HPI research school. Marcelo Caggiani Luizelli is supported by the research fellowship program funded by CNPq (201798/2015-8).

FundersFunder number
Conselho Nacional de Desenvolvimento Científico e Tecnológico201798/2015-8
Israel Science Foundation1505/16

    Keywords

    • Heavy Hitters
    • Measurement
    • Monitoring
    • Streaming

    Fingerprint

    Dive into the research topics of 'Constant time updates in hierarchical heavy hitters'. Together they form a unique fingerprint.

    Cite this