[Bro] processing many files with bro

[Fabian Schneider]

Veronica Estrada wrote:
> Hello,
> 
> I am processing several hours of captured traffic split into pcap files
> that covers 1 minute traffic each. Actually I am having this basic
> script to do that.
> 
> #!/bin/bash
> path=("$@")
> for f in $(ls $path);do
> export BRO_LOG_SUFFIX=$f;
> /usr/local/bro/bin/bro -r  $path/$f brolite mysite
> done
> 
> But my goal is that bro recognize connections that could be split in
> several files. I am thinking that one solution is to modified some
> variables and make them "persistent". Is it correct? Which variables
> should I modified?

I would rather try to write a pcap application that removes the pcap file
headers from a set of input, and that have this application read the files one
by one and pipe the output to bro.

> The other solution. I know that split pcap files can be merged in one
> bigger file, but I will have problems with memory, and bro may crash if
> it has a limitation for processing big size pcap file. So I am not
> considering this option.

I am not aware of any problems of Bro reading huge input file. We are
operationally using Bro and have instances analyze Terabytes of traces in one
run. But of course the more data you put in the more state might be built up.


   best
   Fabian

--
Fabian Schneider, An-Institut Deutsche Telekom Laboratories
Technische Universitaet Berlin,
Fakultaet IV -- Elektrotechnik und Informatik
address: Sekr. TEL 4, FG INET, Ernst-Reuter-Platz 7, 10587 Berlin
e-mail: fabian at net.t-labs.tu-berlin.de,
WWW: http://www.net.t-labs.tu-berlin.de/~fabian
phone: +49 30 8353 - 58513, mobile: +49 160 479 43 97