[Bro] Speed up bro execution

[Ed Sealing]

There are two ways you may be able to address the issues of maximizing
processor use, but both would take some additional work. May be worth it if
you need to do this a lot.

1. Setup a TCPREPLAY server that forwards the traffic to the bro system and
use the normal fanout options (AF_PACKET, PF_RING, etc) to maximize
processor usage. You could theoretically do the replay and listening on the
same box, but tcpreplay would likely bog down one of the procs (2 more for
the manager/proxy, leaving 5 for Bro analysis).

2.Divide up your 500gb PCAP into smaller chunks (e.g. different pcaps based
on SRC/DST) and run "bro -r /path/to/pcap" multiple times in parallel.
You'd have to point each run to a different log-file path and combine them
later. Best to write a script for this if you often have to run large pcaps
offline.




On Fri, Apr 21, 2017 at 6:32 AM, mike anastasakis <anastasakis62 at gmail.com>
wrote:

> Hello,
>
> I am handling rather big pcap files in the size of 500gb and bro execution
> takes a few hours to complete. For this reason I am looking for ways to
> speed up the execution.
>
> I want to keep only specific logs files with the goal of making my bro
> execution faster. For my research I want to keep the following files:  *
> conn.log, ssl.log, x509.log, dns.log, http.log*
> From what I understood this command should do the trick: *bro -r
> <pcap_file_name> -b base/protocols/ssl base/protocols/dns
> base/protocols/conn base/protocols/http*
> However, with the addition of base/protocol/ssl I also get the tunnel.log
> and files.log which I do not need. Is there a way to exclude these files
> from logging?
>
>
> Moreover, I have a rather powerful machines with 8 cores and 8gb of RAM
> does anyone know a way to fully utilize that when using bro?
>
>
> Thanks all,
> Mike
>
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 

R/S

*Ed Sealing President / CEO*
*CISSP, CEH, RHCSA*

7134 Columbia Gateway Dr
Suite 160
Columbia, MD 21046
Mobile: (301) 885-6947
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170427/50d69984/attachment.html