[Bro-Dev] help Reading the backtrace

Azoff, Justin S jazoff at illinois.edu
Wed Jan 18 11:27:17 PST 2017


Yeah.. lots of expires may have something to do with it, your traceback shows

TableEntryVal16ExpireAccessTimeEv

But I also wonder what you are doing that is triggering Dictionary9NextEntryERP7HashKeyRP10IterCookiei

which would be

void* Dictionary::NextEntry(HashKey*& h, IterCookie*& cookie, int return_hash) const

Tables should be fine, but I wonder what you're doing that is triggering so much iteration.

-- 
- Justin Azoff

> On Jan 18, 2017, at 1:40 PM, Aashish Sharma <asharma at lbl.gov> wrote:
> 
> Yes, I have been making heavy use of tables ( think a million entries a day and million expires a day) 
> 
> Let me figure out a way to upload the scripts on github or send them yours and justin's way otherwise. 
> 
> Strangely this code kept running fine for last month and reasonably stable. I am not sure what little thing I added/changed that has caused bro to run but all workers in uwait state with 6% CPU. (I'll be doing svn diffs to figure out) 
> 
> Seems like bro is stuck in:
> 
> 0x00000004039ccadc in _umtx_op_err () from /lib/libthr.so.3
> (gdb) bt
> #0  0x00000004039ccadc in _umtx_op_err () from /lib/libthr.so.3
> #1  0x00000004039c750b in _thr_umtx_timedwait_uint () from /lib/libthr.so.3
> #2  0x00000004039cea06 in ?? () from /lib/libthr.so.3
> #3  0x00000000009042ee in threading::Queue<threading::BasicInputMessage*>::Get (this=0x404543038) at /home/bro/install/bro-2.5/src/threading/Queue.h:173
> #4  0x0000000000902a31 in threading::MsgThread::RetrieveIn (this=0x404543000) at /home/bro/install/bro-2.5/src/threading/MsgThread.cc:349
> #5  0x0000000000902ce4 in threading::MsgThread::Run (this=0x404543000) at /home/bro/install/bro-2.5/src/threading/MsgThread.cc:366
> #6  0x00000000008fb952 in threading::BasicThread::launcher (arg=0x404543000) at /home/bro/install/bro-2.5/src/threading/BasicThread.cc:201
> #7  0x00000004039c6260 in ?? () from /lib/libthr.so.3
> #8  0x0000000000000000 in ?? ()
> Backtrace stopped: Cannot access memory at address 0x7fffffbfe000
> (gdb)
> 
> 
> 
> Aashish 
> 
> On Wed, Jan 18, 2017 at 06:37:08PM +0100, Jan Grashöfer wrote:
>> Hi Aashish,
>> 
>>> So I am running a new detection package and everything seemed right but somehow since yesterday each worker is running at 5.7% to 6.3% CPU and not generating logs.
>> 
>> my guess would be that the script makes (heavy) use of tables and table
>> expiration, right? Can you share the script?
>> 
>> Jan
>> _______________________________________________
>> bro-dev mailing list
>> bro-dev at bro.org
>> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev




More information about the bro-dev mailing list