The Network Metrics Project
Hypothesis
Connections have been made between generalized Renyi entropies and network topologies which suggest that entropy-based functions and spectra could serve as a set of metrics for a useful description of networks such as the internet.
Specifically, the entropy spectra of the connection matrix have been shown, under some circumstances, to detect cluster topology and other flow patterns in simulated environments. These entropy metrics distil extensive and massive microscopic information into a few variables and reflect the patterns of average topological order and disorder.
The hypothesis is put forward that abnormal variance of these entropy metric functions will reflect abnormal topological structural changes that could indicate attack, malicious processes, and system failures.
Objective
The objective of this research is to show that these functions can be useful in practical intrusion detection environments. Phase I of this work designed and tested a "toolset" of proposed entropy functions, spectra, procedures and insights for use which have been described in past DARPA reports, DARPA conferences and unpublished technical reports. The foundation was thus laid for real applications.
In addition, to better ensure that the behavioral characteristics of the intrusion detection domain are accurately represented, and that the Phase I hypothesis can be validated, a collaborative team of intrusion detection experts has been assembled for this project. Workshops have been held to design the experiments to validate the theoretical hypothesis of Phase I – that entropy based metrics can, under some circumstances, detect network intrusions, and specifically, that the total Renyi entropy, the spectra of row and column Renyi entropies, and the entropies of these spectra can detect changing network topologies.
Additionally, a corporate alliance has been organized for practical testing through deployment in a real-world application by an independent corporation using real internet data in parallel with statistical analytic techniques.
Plan
This investigation will be carried out via three intertwined focus areas — Simulation, Prototype, and Deployment — to explicitly study normal and abnormal entropy variations in networks.
Simulation: The first investigation focus will use mathematical simulations to investigate the validity and optimization of the entropy toolset over multiple critical parameters including network size, data flow rates, time sampling, invasive topologies, and various signal/noise ratios.
Prototype: The second focus is a parallel complementary effort that will concentrate on tracking the proposed entropies as they vary over time, to seek abnormal fluctuations both in real internet data and in simulations based upon real data with real intrusions.
Deployment: The last focus will concentrate on deploying the toolset on DOD based network data in a commercial contract environment. In short, we specifically seek the identification of abnormal ranges of the entropy metrics as correlates of intrusions and abnormal topologies in each of these three separate but complementary domains.
Workshops, Conferences, and Meetings
Since July, 2004, Dr. Joseph E. Johnson has organized and been invited to participate in several meetings related to network security and network metrics. In October, 2004, he hosted a meeting with approximately thirty experts in the field at Kiawah Island, South Carolina. More recently, he was invited to present a paper, Networks, Markov Lie Monoids, and Generalized Entropy, at the Mathematical Methods, Models and Architectures for Computer Network Security Workshop in St. Petersburg Russia (September 24 - 28, 2005). The St. Petersburg conference is sponsored by St. Petersburg Institute for Informatics and Automation, Binghamton University (SUNY) and US Air Force Research Laboratory/Information Directorate. Finally, Dr. Johnson has been invited to join a NATO Exploratory Team, "Complexity and Scalability in C41SR Systems," which is comprised of researchers from around the world. Their kick-off meeting will be in Canada later this fall (mid-October).
Publications and Presentations
- Networks, Markov Lie Monoids, and Generalized Entropy (PDF)
- Markov-type Lie groups in GL (PDF)
- Networks, Lie Monoids, & Generalized Entropy Metrics (PPT)
For information about this project, please contact info@asg.sc.edu.
