[Bro] Bro terminates on its own in PCAP read mode

[Jeremy Gin]

Thank you for explaining in-depth, Jeff.

It does seem like Bro ran out of memory, but the VM I used to run Bro had 4
GB of RAM. I tried running it with 10 GB of RAM, and initially, it does
seem to finish the process. If Bro is having such a hard time, how is this
type of failure to be avoided in real life? Is taking down a Bro server
really as simple as generating millions of conversations?

Is this just a design flaw in Bro?

On Tue, Apr 4, 2017 at 11:24 AM, Jeff Barber <jbarber at computer.org> wrote:

> Possibly just out of memory? That pcap has -- according to wireshark --
> 678714 IPv6 conversations. So bro will create that many connection table
> entries. Those entries are not small and a number of related structures get
> created too, so you end up with a ton of memory used by bro. And the
> packets are all "received" within a few seconds so none of the connection
> table entries will have timed out by the time you get to the end.
>
> It's traditional on linux that the kernel allows memory to be
> "overcommitted" but then if the kernel runs out of memory for critical
> functions, it chooses a fat process to kill. References here:
>
> https://linux-mm.org/OOM_Killer
> https://unix.stackexchange.com/questions/153585/how-the-
> oom-killer-decides-which-process-to-kill-first
>
>
> So it's not that the pcap itself is too large -- bro basically reads and
> processes one packet at a time -- it's that processing it takes more memory
> than you have available.
>
>
> On Mon, Apr 3, 2017 at 11:54 PM, Jeremy Gin <jgin at utexas.edu> wrote:
>
>> ​
>>  m_atk3_set0_t0.pcap
>> <https://drive.google.com/a/utexas.edu/file/d/0B2LZm9YbN6FOVEtLeDg2RUFiUGs/view?usp=drive_web>
>> ​Hello,
>>
>> I am trying to run Bro in PCAP read mode on PCAPs that contain flooding
>> attacks created in a lab environment. I installed Bro from source and did
>> not modify the local.bro. The command I am using is:
>>
>> "bro -r <name>.pcap -C local --time"
>>
>> This returns the following output:
>> "WARNING: No Site::local_nets have been defined.  It's usually a good
>> idea to define your local networks.
>> # initialization 2.756138
>> # initialization 59M/49M
>> Killed"
>>
>> I have attached the PCAP. My initial reaction is that the PCAP is too big
>> as this happens to only PCAPs containing DOS attacks. However, the attached
>> PCAP is 69 MB and Bro successfully runs on other PCAPs that are around 73
>> MB. Can anyone explain why Bro is terminating itself?
>>
>> Any insight you can provide is much appreciated.
>>
>> Thanks,
>> Jeremy
>>
>>
>>
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170404/da190daa/attachment-0001.html