|
Network Analytics: Data Acquisition | Data Mining | Security | Visualization Data MiningFeature Extraction:Given the massive quantity of data created during the flow summarization process, feature extraction is absolutely necessary for efficiently characterizing flow and host behaviors. LANL's Hive system calculates the entropy of numerous measures of a host's activity based on packet densities, session durations, peer IPs, etc. These entropy measures are calculated over several striding time windows ranging in length from 10 minutes to 1 month. Strong deviations over time of entropy for a particular host tend to indicate significant changes in host behavior. Clustering:Using a K-means clustering of entropy measures from the Hive system, we automatically cluster hosts or flows from LANL's network (e.g., DNS servers, mail servers, web servers, and common desktop machines). Adaptive Resonance Theory (ART) neural networks are used to efficiently cluster network flows. Using a periodogram-like measure of overall network session densities within striding time windows, UNM has employed ART to detect periods of time showing novel levels of activity. Classification:The ultimate goal of our tools is to classify hosts and the sessions between them. Our approach is to use various automated clustering methods along with expert analysis and labeling of these clusters, for example, Emaad "normal level of activity" and "abnormal level of activity." Hive automatically clusters hosts and network analysts have assigned useful labels to various clusters or sets of clusters such as "normal user workstation" or "mail server" or "web server" or "scanning host" or "infected host." ART clustering applied to temporally correlated data recognizes "interesting intervals of activity." Anomaly DetectionNetwork operators and security analysts are faced with voluminous event logs, signature matches, flow data, and utilization info. Our Emaad system performs efficient, unsupervised anomaly detection on time-series data such as raw flow data, signature events from intrusion detection systems, and medium-term statistical profiling of hosts. A web-based application provides users a prioritized list of events of interest and the drill-down tools to examine them. |
TeamThis work is performed by the members of the Network Security Team:
|