Title:
Network Data Streaming: Algorithms for Network Measurement and Monitoring
Network Data Streaming: Algorithms for Network Measurement and Monitoring
Author(s)
Kumar, Abhishek
Advisor(s)
Zegura, Ellen W.
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
With the emergence of computer networks as one of the primary modes of
communication, and with their adoption for an increasingly wide range
of applications, there is a growing need to understand and
characterize the traffic they carry. The rise of large scale
network attacks adds urgency to this need. However, the large size,
high speed and increasing complexity of these networks imply that
tracking and characterizing the traffic they carry is an increasingly
difficult problem. Dealing with higher level aggregates, such as flows
instead of packets, does not solve the problem because these
aggregates tend to be quite numerous and exhibit dynamics of their
own.
In this thesis, we investigate a novel approach to deal with the
immense amounts of data associated with problems in network
measurement and monitoring. Building upon the paradigm of Data
Streaming, which processes a large stream of data using a small
working memory to answer a class of queries, we develop an
architecture for Network Data Streaming that can accommodate
additional constraints imposed in the context of network monitoring.
Using this architecture, we design algorithms for monitoring
properties of network traffic that have traditionally been considered
too difficult to monitor at high speed network links and routers. Our
first algorithm provides the ability to accurately estimate the size
of individual flows. A second algorithm to estimate the distribution of
flow sizes enables network operators to monitor anomalies in the
traffic. Incorporating the use of packet sampling, we can extend the
latter algorithm to estimate the flow size distribution of arbitrary
subpopulations.
Finally, we apply the tools of Network Data Streaming to the operation
of packet sampling itself. Using the ability to efficiently estimate
flow-statistics such as approximate per-flow size, we design a family
of mechanisms where the sampling decision is guided by this knowledge.
The individual solutions developed in this thesis share a common
architectural theme, supporting the monitoring of highly dynamic
populations. Integrating this with the traditional sampling based
framework for network monitoring will enable a broad range of
applications for accurate and comprehensive monitoring of network
traffic.
Sponsor
Date Issued
2005-11-18
Extent
2290391 bytes
Resource Type
Text
Resource Subtype
Dissertation