[Bro] [BRO-ISSUE]: bro crash when so many Repoter::Error calls

[Myth Ren]

Hi, Justin
  currently i hava a bro code snippet to stable reproduce the crash.
  (no Reporter log to kafka with explicit remove default filter at
`bro_init`)

  I record the backtrace and threads info in this gist:
    https://gist.github.com/MythRen/3d77111fb810cac941c48311dc273289, and
the snippet.

  steps to reproduce:
  0. prepare an environment with bro 2.5.1 installed, with kafka plugin
  1. upload the `test.crash.bro` to your server
  2. run `$BRO_BIN/bro -i INTERFACE -C test.crash.bro`
  3. open another terminal and login to your server
  4. use `ab` to get more bandwidth: ab -n 100000000 -c 10 '
https://www.google.com' (better with local http server, make sure traffic
go out through INTERFACE in step 3)
  5. waitting for crash. usually in seconds, in my test case.

  i think the problem is the access of `head`, 'tail'
  without lock between `Event.cc#86-91` and `Event.cc#118-119` in
multi-thread.

  but i don't known the effective if we add lock here.

  Wish you and others could help.

Best regards,

Myth


2018-01-26 0:46 GMT+08:00 Azoff, Justin S <jazoff at illinois.edu>:

>
> > On Jan 25, 2018, at 11:18 AM, Myth Ren <email4myth at gmail.com> wrote:
> >
> >  then KafkaWriter call Reporter::Error to report the runtime error.
>
> This would be a problem if bro is configured to send the reporter.log to
> Kafka.
>
> Reporter::Error generates a reporter_error event which then calls
>
>     Log::write(Reporter::LOG, [$ts=t, $level=ERROR, $message=msg,
> $location=location]);
>
> So you're probably also ending up in the situation where bro is trying to
> log to Kafka the fact that Kafka is broken.
>
> If you tell bro to not send the reporter.log to Kafka, does your problem
> go away?
>
> —
> Justin Azoff
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180401/470d35fc/attachment-0001.html