[Bro] Bro programming intro
tritium.cat at gmail.com
Tue Mar 19 11:18:45 PDT 2013
Thanks for your comments, good to reflect on over coffee. FWIW I am
running the cluster model with plenty of RAM and CPU to spare with a near
I do not believe what I am trying to learn and implement is premature
optimization but thanks for the reminder, I've made that mistake a few
Another thing I would like to do is tag every orig_h and resp_h with
additional identifiers relative to the prefix, sorta like BGP ASNs. I
usually use a Patricia-Trie for this. Is there a special data type and BiF
I should consider ?
>From the documentation on extending logging  it seems that is a bit
beyond the scripting layer. I read about the input framework  and it
seems it might work for this application but I'm not sure if this is best.
What do you suggest ? I would like all logs that have an orig_h and/or
resp_p to include the tags.
On Tue, Mar 19, 2013 at 9:51 AM, Clark, Gilbert <gc355804 at ohio.edu> wrote:
> One paper I can think of off-hand that appeared in RAID a few years back
> and may be relevant here:
> On principle, there isn't a hard and fast rule for exactly how much state
> you can allocate or how many cycles you can spend processing individual
> packets: it's really going to depend on load, and will likely need to be
> experimentally determined (and constantly tuned). One way to do this is to
> record snippets of the traffic you normally see at the border and run bro
> against that same set of traffic over and over (bro's —pseudo-realtime
> option can help get a realistic sample of what happens if you want to run
> this locally, or you can alternatively replay traces onto a local link to
> possibly get some more realistic results) while modifying the scripts bro
> has loaded to see what happens. This kind of testing can help you identify
> the limits of what a specific bro configuration can handle in your
> Also, I'd like to point out that some folks simply use bro for offline
> trace analysis and the like; while bro does well as a real-time tool, it
> doesn't necessarily have to be.
> "I am skeptical of how much emphasis is placed on doing things within
> Relevant cliche: Premature optimization is the root of much evil.
> The emphasis, I think, isn't placed on doing things within bro. Instead,
> I believe the emphasis is placed on doing things that work for the folks
> who are deploying bro into their environments. Most of the discussion here
> focuses on the scripting layer here because, in my humble opinion, that is
> the most straightforward interface to bro's event engine, and it's fast
> enough to do what folks need it to. In the event it isn't, there's
> always the cluster model … there's at least one piece of pretty cool
> hardware I know of that rewrites destination MAC addresses and allows you
> to load-balance across a cluster pretty effectively, and PF_RING can also
> even let you cluster on a local box. Often, investing in clustering winds
> up being cheaper (in both time and money) than trying to throw more experts
> at the problem to squeeze that last 15 Mbps out of a single node.
> Then again, trying to squeeze more out of a single node is always a fun
> programming challenge to solve :)
> Normal disclaimer applies: just my $0.02, I'm not an expert, etc.
> --Gilbert Clark
> From: Tritium Cat <tritium.cat at gmail.com>
> Date: Tue, 19 Mar 2013 11:46:03 -0400
> To: Seth Hall <seth at icir.org>
> Cc: "bro at bro-ids.org" <bro at bro-ids.org>
> Subject: Re: [Bro] Bro programming intro
> Ok. I am skeptical of how much emphasis is placed on doing things within
> BroIDS. Simply buffering uids per Notice? seems much easier and less
> resource intensive than storing additional? samples. Where is the limit
> with tracking too much state or using too many cycles within the "IDS" ? I
> am weary of inadvertently creating DoS conditions with a philosophy that
> may encompass every script I write in Bro.
> I am still interested in a list of key papers on the internals if anyone
> has a few.
> On Tue, Mar 19, 2013 at 8:13 AM, Seth Hall <seth at icir.org> wrote:
>> On Mar 18, 2013, at 8:03 PM, Tritium Cat <tritium.cat at gmail.com> wrote:
>> > I want to modify the SQL Injection detection in
>> policy/protocols/http/detect-sqli.bro to include a vector that tracks the
>> associated http request uids and includes them in an additional log field.
>> After getting it working I would like to apply it generally to other
>> Notices such as SSH Password_Guessing.
>> The upcoming release actually results in this script getting rewritten a
>> bit because of a rewrite of the metrics (now measurement) framework. The
>> new version actually keeps samples of the requests. It will be relatively
>> easy to write your own script that tracks uid's instead of urls but the
>> benefit to sampling the urls is that if you have Bro send you email for the
>> notice it will add those sample urls to the email (it's been very
>> convenient for determining if something is a false positive without even
>> searching logs).
>> Otherwise, with the metrics framework in 2.1 there isn't a good way to do
>> Seth Hall
>> International Computer Science Institute
>> (Bro) because everyone has a network
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Bro