[Bro] Bro2 Random Crashes

[Chris Crawford]

Thanks for the suggestions.  I ended up using tcpdump to do the
network capture.  I wrote a cron job to regularly run bro over the the
files generated by tcpdump.  Definitely not as ideal as having bro run
continuously, but it gets the job done reliably without crashing.

This might be an interesting issue for Bro in the future, though.
Running bro continuously from a xen VM would be incredibly useful.
I'd be curious if anybody can reproduce the crash conditions, or if it
is just something weird about my setup or environment.

-Chris

On Sun, Mar 4, 2012 at 3:27 PM, G. Clark <gc355804 at ohio.edu> wrote:
> http://www.westnet.com/~gsmith/content/linux-pdflush.htm
>
> Specifically:
>
> dirty_background_ratio: Primary tunable to adjust, probably downward. If
> your goal is to reduce the amount of data Linux keeps cached in memory,
> so that it writes it more consistently to the disk rather than in a
> batch, lowering dirty_background_ratio is the most effective way to do
> that. It is more likely the default is too large in situations where the
> system has large amounts of memory and/or slow physical I/O.
>
> You might also take a look at:
>
> http://www.mjmwired.net/kernel/Documentation/cgroups/memory.txt
>
> HTH
>
> --Gilbert
>
> On 3/2/12 2:54 PM, Gregor Maier wrote:
>> On 3/1/12 6:32 , Nicholas Weaver wrote:
>>>
>>>
>>> On Mar 1, 2012, at 6:02 AM, Chris Crawford wrote:
>>>> The only thing I've been able to key in on, is that over time Bro2
>>>> eventually causes all free memory in Ubuntu to change to cached
>>>> memory.  From the Citrix Xen Console, that cached memory shows up as
>>>> used memory.  So, maybe Xen interprets cached memory as being used?
>>>> Also -- when Xen senses that most of the memory is "used" (but,
>>>> really, it's cached inside Ubuntu), the percent utilization in one of
>>>> the CPUs in the VM spikes.  After Bro2 crashes, CPU utilization
>>>> returns to normal, but memory is never freed -- it remains cached
>>>> forever.
>>>
>>> Cached memory IS used memory by the OS.  Modern operating systems are very aggressive about using all available "physical" memory as a disk cache when otherwise not being used, especially Linux variants.  But at the same time, it should also aggressively evict those entries when it think that there are limits on physical memory. [1]
>>
>> Indeed. I found in several other cases that Linux caching strategy can
>> be really annoying. I'm regularly having performance problem on with
>> high throughput workloads....
>>
>>
>>
>> cu
>> Gregor
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro