|
|
Details Common modules
The ACARM is based on the Alcopop application that provides some correlation filters. According to the schema presented above, the Correlation Module consists of two main modules: the Correlation Engine and Collection and Storage. The Correlation Engine module is a Java application that provides all the correlation filters. The Collection and Storage module uses the Prelude framework and incorporates additional mechanisms which enable communication with other components. The Prelude framework is Open Source software, which makes it possible to collect alerts from a variety of sensors and to store them in various databases. It also makes it possible to send the data in a common format to other applications. WCNS has developed a special plug-in for Prelude which allows it to report alerts to the Correlation Engine. To process data that comes from vendor-specific sensors, the module can use a "Prelude Import" component which makes such processing possible. Additionally, the Correlation Module takes advantage of LibPrelude and BEEP interfaces for communication between components and some IDMEF to Java and Java to IODEF conversion classes needed for the Correlation Engine. The process finishes the Attack Analyzer module that analyzes data about attacks coming from the Correlation Engine, generates the configuration and sends it to the appropriate plug-ins.
Figure 1 – Detailed schema of the ACARM module Correlation engine The Correlation Engine carries out the tasks of an attack correlation process. The correlation process of alert data is complex and consists of several stages. Particular phases of the correlation process are described below. Correlation process As Figure 2 shows, data which comes from IDS sensors, is received by the BEEP server, which will later be called "Source". The BEEP server is responsible for processing the data encapsulated in the BEEP protocol. The BEEP server receives a message and converts it to a string. The data is then passed to the Pre-filter module. The common task of that module is to throw out unimportant alerts on the output of the module. This solution is especially important when there is a large number of IDS sensors and will be described later. Next, the data is passed to the IDMEF-IAS module, which is responsible for conversion of IDMEF messages to the corresponding Java classes. IDMEF messages are converted to an Internal Alert Structure (IAS), which is a Java language representation of an attack in the correlation engine. The IAS contains most of the features contained in an IODEF message. During the next stage, data stored in Java classes is analyzed by the Preprocessing module that checks if all the time stamps (detect time, start time, end time and report time) are correctly supplied. It also determines the type of the origin of the attack (NIDS or HIDS) and checks if the attacks contain source and destination addresses in the form of a number. If there is a string instead of a number, it generates a DNS request. 
Figure 2 – Structure of the Correlation Engine Prefilter The common task of this module is to throw out unimportant alerts on the output of the module. This solution is especially important when there is a large number of IDS sensors. At present, when the HIDS sensors are installed on clusters, there is a problem with redundant information that comes from these sensors. In order to resolve this problem, the pre-filter enables the user to create dedicated, trusted zones that shouldn’t be monitored. A good example is turning off the file transfer monitoring system between the computers from WCNS, ignoring ICMP packets or information about a successful login of a PAM module. The pre-filter is fully configurable and has a dedicated configuration file in XML format. In the beginning the user should choose the filtering policy. There are two modes – drop or accept. The first one causes a filter to throw out the packets that match the rules and accept the remaining. The second one forces it to accept only alerts that match the rules. We recommend using the first one due to security concerns. The pre-filter enables classification of alerts regarding the following classifiers:
The user has the possibility to construct any logical condition using these classifiers. Preprocessing algorithm Some sensors don’t provide necessary information that will be used during further correlation processes and even if alerts are normalized, additional preprocessing may be required to provide this data. The goal of this module is to supply, as accurately as possible, missing alert attributes (i.e. start-time, end-time, source, and target) that are important for further correlation processes. The IDMEF standard defines three different classes to represent time:
As a start time the preprocessing component uses DetectTime. If DetectTime is not present, it uses CreateTime and if there is no CreateTime value, it uses AnalyzerTime. When no end time (or appropriate duration information) is present, the end time is assumed to be identical to the start time. Fusion filter The alert fusion algorithm combines alerts that represent independent detection of the same attack occurrence by different intrusion detection systems. The decision about the fusion of two alerts is based on the temporal difference between these alerts and the information that they contain. The alert fusion component keeps a sliding time window of alerts. The alerts within the time window are stored in a time-ordered queue. When a new alert arrives, it is compared to the alerts in the queue, starting with the alert with the earliest timestamp. A match is found if all overlapping attributes are equal and the new alert is produced by a sensor different from the sensors already associated with the alert being matched. One-to-one algorithm This algorithm correlates alerts that are caused by an attacker who generates different exploits against a certain application or that runs a similar exploit multiple times to guess the correct values for certain parameters. This algorithm also keeps a sliding window. Two alerts are considered as a match if the source and target attributes are equivalent. Different from fusion, the sensor alerts need not be produced by different sensors. Merging is done by creating a meta-alert containing only the attributes that are equal in both sensor alerts. The timestamp of the meta-alert is assigned the earlier of the two start-times and the later of the two end-times. Network-Host algorithm The goal of the attack session reconstruction component is to link network-based alerts to related host-based alerts. Identifying relations between these two types of alerts is difficult because the information that is present in the alerts differs significantly. Network-based sensors usually provide the source and destination IP addresses and ports of packets that contain evidence of attacks. Host-based sensors, on the other hand, include information about the process that is attacked and the user on whose behalf the process is executed. One-to-many and many-to-one filters The goal of the attack focus recognition component is to identify hosts that are either the source or the target of a substantial number of attacks. More specifically, this component aggregates the alerts associated with single hosts attacking multiple victims (called a one-to-many scenario) and single victims that are targeted by multiple attackers (called a many-to-one scenario). This correlation component is effective in reducing the number of alerts caused by distributed denial-of-service (DDoS) attacks and large-scale scans. Alerts related to a DDoS attempt can be merged into a single many2one meta-alert, while alerts related to individual scans can be combined into a single one2many meta-alert. Both scenarios are based on a sliding time window with an extended timeout model. In the one-to-many scenario, the number of different targets is recorded for each attack source during the extended time window. If this number exceeds an a priori specified threshold, an appropriate meta-alert is generated. The source of the meta-alert is set to be the attacker’s host, while the target is the union of the targets of the individual attacks. The many-to-one scenario operates in a similar way, the only difference being that the roles of the attacker (source) and victim (target) are reversed. The Severity Assessment module After each merging process, the alert is checked with respect to the severity of the damage it causes by the Severity Assessment module (visualized on the schema as the additional block called “Any filter”). This mechanism was implemented in order to generate early information about the appearance of a potential hazard. If a violation appears, the module sends it immediately to the Assessment Module with the early warning flag. Then the Assessment module knows that an IODEF will appear having the same alert ID. In the opposite case, a dangerous alert would have to go through all the correlation filters, which would take about 300 seconds or longer depending on the defined time windows. In case of a detection of a serious incident, the Assessment Module is immediately informed about it and the dangerous alert is analyzed by all the correlation filters. This analysis is performed because the alert could be part of a major attack. The Severity analysis mechanism consists of a few rules. In the first step it checks if, in the set of severity assessments, the “high” value is present. If yes, the alert is treated as dangerous and sent to the assessment module. In the second step, the algorithm checks the number of failed logins, defined in the configuration file. If this number is too high for the alert and the action is finished by a successful login, we can suspect that a brute-force attack appeared. Another rule takes into consideration the amount of ancestors of a particular attack also stored in the configuration file. If it is too high, the attack can be recognized as a DoS or DDoS. Furthermore, all network-host, recon-breaking-escalation and island-hopping attacks are treated as dangerous. Severity Assessment ultimately makes the decision about sending information about the attack to the Assessment Module. Only attacks recognized as dangerous are sent to the Assessment Module. Alerts characterized by a low level of severity or alerts not evaluated as dangerous (even if they can trigger the dangerous alert) aren’t sent out. The last stage is generation of an IODEF meta-alert. The IODEF generator, also called a Sink, is a module which analyzes data contained in the Java classes and generates an IODEF message based on them. The Correlation Module provides functionality which enables the user to define communication paths between chosen modules and thus makes it possible to create various kinds of correlation engines. Additional information Queues in filters Each filter contains a queue which stores alerts that will be correlated. Alerts are stored as they come to the filter, having ascending timestamps. An incoming alert is compared with alerts in the queue until the first match. Each alert stays in the queue for a time, which is at most equal to the time window defined for each filter. If the difference between the oldest and the youngest alert in the queue exceeds the time window, the oldest is passed to the next filter. Communication between the correlation components Communication between particular components of the Correlation Engine is performed using pipes. From a technical point of view, each pipe is represented by one separate thread. The thread sends a request to the first filter and checks if there are alerts in its output “outside” the time window. If there are – the oldest one is passed to the queue of the next filter. The thread sends requests in half-second intervals and during the request, only the one, oldest alert is passed. Safety Services Services which aren’t taken into account during the correlation process are defined as Safety Services. We assume that alerts that are generated by these services are not efficient (false positives). For example, many attacks generated by the daemon “cron” are false positives. Such attacks can also take part in the correlation and influence the generation of wrong information. Collection and storage The tasks of the Collection and Storage module are performed by a Prelude-IDS framework. Prelude is a meta IDS framework. It provides functionality which enables failover storage and normalization of data which comes from sensors made by different vendors. It relies on the IDMEF format, which enables different kinds of sensors to generate events using a unified language. Simply speaking, Prelude has three main tasks as a storage element of the Correlation Module:
Prelude provides a LibPrelude interface, which is used for integration with different IDS sensors, using an IDMEF format. For integration with IDSs, which use a specific format, a special tool called “Prelude-Import” was released by the Prelude vendor. Attack analyzer The main goal of the Attack Analyzer is to analyze alerts that come from the Correlation Module, determine if an alert is severe enough and to generate and initiate deployment of an appropriate configuration, for networking devices and services, based on these alerts. This new configuration will protect the network infrastructure from further occurrences of similar events. This module is in the development phase now. Plug-ins The configuration generated by the Attack Analyzer will be passed to the particular plug-ins that will be associated with the network devices and services. The main goal of the plug-ins is to deploy the configuration to the network devices or services. These components are also in the development stage. Alert database The Correlation Engine analyzes the IODEF message and passes the alert details to an SQL database. The database stores the most important information about alerts like source and target IP address, ports, alert creation time, alert type, type of the sensor that generated the alert, alert severity level and others. Structure of the database enables the user to perform an alert search using a variety of information from the alert’s fields. ACARM database client To help the administrator analyze the alert data a database client was developed. The client is a Java application that enables an administrator to search and analyze information about alerts generated by ACARM. The client retrieves information from the database described above and provides the following functionalities:
|