[Bro-Dev] Making scan.bro great again.

Azoff, Justin S jazoff at illinois.edu
Tue Aug 2 16:36:37 PDT 2016

> On Aug 2, 2016, at 6:03 PM, Slagell, Adam J <slagell at ILLINOIS.EDU> wrote:
> Wow. Big difference 

Indeed :-)  I realized one of the bigger issues in the sumstats based code is not really the detection of scans, but what happens AFTER the detection.  After detection it keeps accumulating data, or possibly only slightly better, keeps trying to accumulate data.

Connection events are used to generate sumstats observations
sumstats observations feed into the sumstats framework
which may cross a threshold
which generates notices that are fed into the notice framework
which are often suppressed for at least 1hr by default.

However, sumstats has no idea that the only reason it is collecting observations is to raise a notice that could be currently suppressed for an entire day.

The observations don't stop once attacker has already triggered a notice.  The whole machine keeps running even though nothing visible will ever come out of it.

I managed to fix this in the sumstats based unified scan.bro, but it is only a partial fix.

The code that does this is in this version:


Basically this part:

global known_scanners: table[addr] of interval &create_expire=10secs &expire_func=adjust_known_scanner_expiration;

# There's no way to set a key to expire at a specific time, so we
# First set the keys value to the duration we want, and then
# use expire_func to return the desired time.
event Notice::begin_suppression(n: Notice::Info)
    if (n$note == Port_Scan || n$note == Address_Scan || n$note == Random_Scan)
        known_scanners[n$src] = n$suppress_for;

function adjust_known_scanner_expiration(s: table[addr] of interval, idx: addr): interval
    local duration = s[idx];
    s[idx] = 0secs;
    return duration;

and then later, the checks are aborted early with:

    if ( scanner in known_scanners )

This works, but the reason it is incomplete is that if the notice was triggered by an intermediate update,  sumstats still contains data for this attacker.  This data will be sent over to the manager at the end of the epoch, even though by that time it isn't needed anymore since there was only one threshold and it was already crossed. The fix for that would be some more changes inside of sumstats..

if (intermediate update crossed a threshold && the number of thresholds is 1)
    instruct all worker nodes to purge any data associated with this key

my non-sumstats based scan.bro and AAshish's scan-NG both handle 'known scanner' suppression directly with a cluster event.  Because there's no middleman (sumstats) it ends up being a bit simpler.

This Notice::begin_suppression trick itself is nice though, since it lets you configure suppression intervals in one place, instead of potentially having to configure policy specific suppression and keep them in sync.

- Justin Azoff

More information about the bro-dev mailing list