This paper aims to provide a comparison between session-based and packet-based modeling techniques for building anomaly detection systems. The authors state that they will propose a methodology for performing such comparisons.
Although this research topic is interesting indeed, the paper falls short in several ways. The authors use the 1999 Defense Advanced Research Projects Agency (DARPA) evaluation dataset, which has well-known shortcomings . They do not fully describe the data preparation: why they chose to discard certain features, such as the source/destination hosts, is not explained. They also do not describe their classifier, making it difficult to replicate their results.
After creating a model of the DARPA dataset, they apply it to an unlabeled dataset drawn from their own network, which is completely different from the 1999 testbed. It is quite challenging to understand how this could possibly work.
But, in fact, we do not know whether or not the experiment succeeded. The authors do not give any meaningful results, such as a detection rate or false-positive rate. They just give a “positive rate,” leaving us to guess whether their approach is a great detector, or just a generator of white noise.
From these limited observations, no conclusions can be drawn regarding the original question of comparing session- and packet-based mechanisms for anomaly detection.