[Bro] question about intel files

philosnef philosnef at yahoo.com
Wed Aug 3 07:33:25 PDT 2016


This was about 2 hours after Bro was rebooted. Here is the output of a bro box with nearly identical throughput that has had bro up and running for the past 48 hours. As you can see, the buffer at this rate has shot up to 4g per parent. If I did not reboot the first box reported every 8 hours or so, we would see the same result there.
$ free -m             total       used       free     shared    buffers     cachedMem:        387495     375736      11758          0       1073     248216-/+ buffers/cache:     126446     261048Swap:        15999        267      15732$ /opt/bro/bin/broctl topwaiting for lock (owned by PID 71999) ...Name         Type    Host             Pid     Proc    VSize  Rss  Cpu   Cmdmanager      manager localhost        69286   parent  853M   325M 108%  bromanager      manager localhost        69313   child   381M   171M  21%  broproxy-1      proxy   localhost        69363   parent    1G   845M  34%  broproxy-1      proxy   localhost        69394   child   215M   112M   3%  broproxy-2      proxy   localhost        69391   child   210M    94M   7%  broproxy-2      proxy   localhost        69364   parent  935M   829M   3%  broworker-1-1   worker  localhost        69517   parent    4G     4G  79%  broworker-1-1   worker  localhost        70127   child   712M   627M   1%  broworker-1-10  worker  localhost        69526   parent    4G     4G  98%  broworker-1-10  worker  localhost        70123   child   712M   626M   1%  broworker-1-11  worker  localhost        69537   parent    4G     4G  83%  broworker-1-11  worker  localhost        70095   child   712M   627M   1%  broworker-1-12  worker  localhost        69545   parent    4G     4G  86%  broworker-1-12  worker  localhost        70098   child   712M   628M   1%  broworker-1-13  worker  localhost        69563   parent    4G     4G  92%  broworker-1-13  worker  localhost        70027   child   712M   628M   1%  broworker-1-14  worker  localhost        69564   parent    4G     4G  98%  broworker-1-14  worker  localhost        70140   child   712M   626M   1%  broworker-1-15  worker  localhost        69582   parent    4G     4G  98%  broworker-1-15  worker  localhost        70143   child   712M   628M   1%  broworker-1-16  worker  localhost        69577   parent    4G     4G 100%  broworker-1-16  worker  localhost        70125   child   712M   628M   0%  broworker-1-17  worker  localhost        69595   parent    4G     4G  98%  broworker-1-17  worker  localhost        70135   child   712M   629M   1%  broworker-1-18  worker  localhost        69600   parent    4G     4G  79%  broworker-1-18  worker  localhost        70141   child   712M   628M   0%  broworker-1-19  worker  localhost        69618   parent    4G     4G  77%  broworker-1-19  worker  localhost        70106   child   712M   624M   1%  broworker-1-2   worker  localhost        69615   parent    4G     4G  79%  broworker-1-2   worker  localhost        70138   child   712M   628M   0%  broworker-1-20  worker  localhost        69620   parent    4G     4G  88%  broworker-1-20  worker  localhost        70131   child   712M   628M   1%  broworker-1-3   worker  localhost        69631   parent    4G     4G  81%  broworker-1-3   worker  localhost        70025   child   712M   626M   1%  broworker-1-4   worker  localhost        69639   parent    4G     4G  86%  broworker-1-4   worker  localhost        70139   child   712M   628M   1%  broworker-1-5   worker  localhost        69636   parent    4G     4G  98%  broworker-1-5   worker  localhost        70108   child   712M   626M   1%  broworker-1-6   worker  localhost        69646   parent    4G     4G 100%  broworker-1-6   worker  localhost        70107   child   712M   625M   1%  broworker-1-7   worker  localhost        69647   parent    4G     4G  67%  broworker-1-7   worker  localhost        70097   child   712M   622M   1%  broworker-1-8   worker  localhost        69649   parent    4G     4G  84%  broworker-1-8   worker  localhost        70026   child   712M   626M   1%  broworker-1-9   worker  localhost        69651   parent    4G     4G  67%  broworker-1-9   worker  localhost        70134   child   712M   628M   1%  bro
 

    On Wednesday, August 3, 2016 10:24 AM, "Azoff, Justin S" <jazoff at illinois.edu> wrote:
 

 
> On Aug 3, 2016, at 10:03 AM, philosnef <philosnef at yahoo.com> wrote:
> 
>              total      used      free    shared    buffers    cached
> Mem:        371336    340383      30952          0        300    111823
> -/+ buffers/cache:    228259    143076 
> Swap:        15999        191      15808 
> 
> 

Ah, I think you have been looking at the wrong numbers.

You are only using 228259M, (~222G, not 355G)
111823M is unallocated and currently used for buffer/disk cache.

This amount will always grow until it ends up using almost all the 'free' memory on the machine.

The reason why the OOM killer isn't killing anything is because you still have over 128G of ram free.

I added up all the ram usage from the output of bro top, and adding some overhead for the rounded amounts measured in gigs, came to
56184M.

Minus splunk, that does still leave about 150G unaccounted for.

I believe some of that will be used by packet buffers in the kernel, depending on how you have configured pf_ring.

But even at a huge 1G buffer for each of 20 workers (which I think is much much more than it uses by default) that is only another 20G.


-- 
- Justin Azoff


  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160803/47f54425/attachment-0001.html 


More information about the Bro mailing list