[Bro] question about intel files
philosnef at yahoo.com
Wed Aug 3 06:56:11 PDT 2016
With hyperthreading that's actually 40 cores, not 20. Running 20 workers with 40 cores available should be more than sufficient. At the time brotop was run, 355 out of 390 gigs of ram are in use. The only things running on this box are bro, and a splunk forwarder. The splunk forwarder is only using about 15 gigs of ram. This excessive memory consumption is on all of our bro boxes, no matter the input stream. Even on boxes only getting 500Mb/s, we see this memory creep until it is exhausted. At no point is oomkiller called however, so it is not exceeding available memory, just consuming all of the available memory.
brotop---Name Type Host Pid Proc VSize Rss Cpu Cmdmanager manager localhost 67408 parent 884M 343M 136% bromanager manager localhost 67442 child 346M 179M 24% broproxy-1 proxy localhost 67512 parent 366M 284M 3% broproxy-1 proxy localhost 67542 child 201M 114M 3% broproxy-2 proxy localhost 67543 child 201M 107M 3% broproxy-2 proxy localhost 67513 parent 366M 284M 1% broworker-1-1 worker localhost 67683 parent 1G 1G 100% broworker-1-1 worker localhost 68236 child 716M 625M 3% broworker-1-10 worker localhost 67688 parent 1G 1G 96% broworker-1-10 worker localhost 68278 child 716M 629M 1% broworker-1-11 worker localhost 67697 parent 2G 2G 100% broworker-1-11 worker localhost 68229 child 716M 628M 0% broworker-1-12 worker localhost 67712 parent 1G 1G 83% broworker-1-12 worker localhost 68264 child 716M 629M 1% broworker-1-13 worker localhost 67717 parent 4G 4G 100% broworker-1-13 worker localhost 68233 child 716M 627M 1% broworker-1-14 worker localhost 67737 parent 1G 1G 98% broworker-1-14 worker localhost 68223 child 716M 626M 1% broworker-1-15 worker localhost 67752 parent 2G 2G 100% broworker-1-15 worker localhost 68269 child 716M 626M 0% broworker-1-16 worker localhost 67749 parent 1G 1G 72% broworker-1-16 worker localhost 68228 child 716M 630M 0% broworker-1-17 worker localhost 67758 parent 2G 2G 87% broworker-1-17 worker localhost 68263 child 716M 627M 1% broworker-1-18 worker localhost 67764 parent 1G 1G 98% broworker-1-18 worker localhost 68254 child 716M 626M 1% broworker-1-19 worker localhost 67767 parent 1G 1G 66% broworker-1-19 worker localhost 68239 child 716M 629M 0% broworker-1-2 worker localhost 67774 parent 1G 1G 98% broworker-1-2 worker localhost 68230 child 716M 625M 0% broworker-1-20 worker localhost 67794 parent 3G 3G 98% broworker-1-20 worker localhost 68245 child 716M 629M 3% broworker-1-3 worker localhost 67792 parent 1G 1G 91% broworker-1-3 worker localhost 68265 child 716M 627M 3% broworker-1-4 worker localhost 67800 parent 1G 1G 83% broworker-1-4 worker localhost 68248 child 716M 628M 1% broworker-1-5 worker localhost 67799 parent 1G 1G 98% broworker-1-5 worker localhost 68277 child 716M 626M 0% broworker-1-6 worker localhost 67801 parent 1G 1G 85% broworker-1-6 worker localhost 68279 child 716M 626M 1% broworker-1-7 worker localhost 67813 parent 1G 1G 100% broworker-1-7 worker localhost 68251 child 716M 628M 1% broworker-1-8 worker localhost 67812 parent 1G 1G 79% broworker-1-8 worker localhost 68244 child 716M 629M 0% broworker-1-9 worker localhost 67814 parent 1G 1G 96% broworker-1-9 worker localhost 68266 child 716M 626M 1% bro
On Wednesday, August 3, 2016 9:43 AM, "Azoff, Justin S" <jazoff at illinois.edu> wrote:
> On Aug 3, 2016, at 7:22 AM, philosnef <philosnef at yahoo.com> wrote:
> We have 2 10 physical core systems with 20 logical cores for a total of 40. Bro has a capture loss of sub .5% across all workers, so it seems unlikely that the box is overloaded. The capture rate of the box, per pfring is about 3.5Gb/s. We reported memory issues in the past, but those were written off as not related to the memory leak recently patched in the 24 branch and the 25 branch.
What process is using memory? Workers? Proxies? Manager? If you can include the output of 'broctl top' that would be helpful. Otherwise it is pretty hard to determine what the issue may even be.
If you have a dual 10 core system and are running 20 workers then that leaves no room for the manager or for any tasks like log rotation. For a 20 core system I would run at most 18 workers.
- Justin Azoff
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Bro