[Bro] Intel + Sumstats frameworks

[Josh Liburdi]

Having some unidentifiable issues working with the intelligence and
Sumstats frameworks that I hope the development team or community can
comment on. I've written multiple Sumstats scripts that return results, but
I have an example where it doesn't when running on a cluster. Locally, the
following snip of script runs and returns the expected results:

@load base/frameworks/sumstats
@load base/frameworks/notice
@load base/frameworks/intel

module Intel;

export {

    redef enum Notice::Type += {
        Test_Indicators
    };
}

event log_intel(rec: Info)
    {
    SumStats::observe("intel.indicators",
[$str=cat(rec$uid,"`",rec$id$orig_h,"`",rec$id$resp_h)],
[$str=rec$seen$indicator]);
    }

event bro_init()
    {
    local r1: SumStats::Reducer = [$stream="intel.indicators",
$apply=set(SumStats::UNIQUE)];
    SumStats::create([$name="test-intel",
                      $epoch=2mins,
                      $reducers=set(r1),
                      $epoch_result(ts: time, key: SumStats::Key, result:
SumStats::Result) =
                        {
                        NOTICE([$note=Test_Indicators,
                             #$src=to_addr(parts[3]),
                             #$dst=to_addr(parts[5]),
                             $msg="passed sumstats",
                             #$sub=sub_msg,
                             $identifier=key$str]);
                    }]);
    }

Each time a log is sent to intel.log, this observes the indicators seen in
unique connections. For testing purposes, I have it writing a notice
whenever the Sumstats event finishes. While this works as expected locally,
when run in production (on a cluster), no notices are written-- that
suggests to me that the data is not being sent to Sumstats. (And I am
generating lines in intel.log in prod, so it cannot be the lack of
intel.log activity.)

The only identifiable difference between my local version of Bro and the
one running in production is clustering. Is this expected behavior? I have
used log_ events successfully with Sumstats in the past, so I can't think
of what is preventing this notice from firing. I also verified the syntax
of the above script by changing the event from log_intel to something more
common (http_reply) and it worked both locally and in prod; changing
log_intel to Intel::match (mimicking intel/do_notice.bro) worked locally
but not in prod.

Interested in reading thoughts on this...

Thanks for reading,
Josh Liburdi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140418/ee85f2e4/attachment.html