[Zeek] Input framework

[Nabil Memon]

Hi Justin,

I didn't try that yet.
I got caught up in some other side effect. Bro's CPU usage goes 100% when I
use input framework(or maybe my understanding is incorrect). I have no idea
what triggered this.

I tested with and without two lines(integrating bro's input framework) in
the bro script.

*test.bro (On which I observe 100% CPU usage):*
event bro_init()
{
----------
----------
Input::add_table([$source=file_port_list_loc, $name="file_port_list",
$idx=Portsx, $destination=file_port_list]);
Input::remove("file_port_list");
}

*test.bro (On which I CPU usage is normal):*
event bro_init()
{
----------
----------
*#* Input::add_table([$source=file_port_list_loc, $name="file_port_list",
$idx=Portsx, $destination=file_port_list]);
*#* Input::remove("file_port_list");
}

As you can see, I have commented the actual IF's calls in the latter file.

I did strace of the both the PIDs and below is the output.

*strace -p <PID> (PID of Bro reaching 100% CPU usage):*
select(25, [9 13 15 18 20 22 24], [0 13 15], [0 13 15], {0, 0}) = 1 (out
[0], left {0, 0})
select(25, [9 13 15 18 20 22 24], [0 13 15], [0 13 15], {0, 0}) = 1 (out
[0], left {0, 0})
select(25, [9 13 15 18 20 22 24], [0 13 15], [0 13 15], {0, 0}) = 1 (out
[0], left {0, 0})
select(25, [9 13 15 18 20 22 24], [0 13 15], [0 13 15], {0, 0}) = 1 (out
[0], left {0, 0})

*strace -p <PID> (PID of a nice Bro)::*
select(24, [9 13 15 18 19 21 23], [0 13 15], [0 13 15], {0, 0}) = 1 (out
[0], left {0, 0})
select(0, NULL, NULL, NULL, {0, 20})    = 0 (Timeout)
select(24, [9 13 15 18 19 21 23], [0 13 15], [0 13 15], {0, 0}) = 1 (out
[0], left {0, 0})
select(0, NULL, NULL, NULL, {0, 20})    = 0 (Timeout)

As you can notice, difference between these two outputs, is that the nice
process has one more select of timeout of 20 microseconds(which is I
suppose sleep for 20 microseconds). But the same select call is not being
made with the bad process. Because of which the process takes up 100% CPU.

Am I missing anything here?? I am not able to identify what exactly is
causing this behavior.

Thanks a lot for reading !!

Regards,
Nabil


On Fri, May 22, 2020 at 7:36 PM Justin Azoff <justin at corelight.com> wrote:

> Try deleting this part:
>
> event bro_done()
> {
>    Input::remove("file_port_list");
> }
>
> On Fri, May 22, 2020 at 6:19 AM Nabil Memon <nabilmemon.ec at gmail.com>
> wrote:
> >
> > Hi Zeek,
> >
> > Hope you're all doing well.
> >
> > I am using an Input framework to provide some dynamic input to bro.
> > Here's how the infra looks like,
> >
> > cat /usr/local/bro/share/bro/base/protocols/file_port_list
> > #fields port_num
> > 60000/tcp
> > 8080/tcp
> > 49154/tcp
> > 55907/tcp
> > 49152/tcp
> > 49153/tcp
> > 8000/tcp
> > 5357/tcp
> >
> >
> > type Portsx: record {
> >    port_num: port;
> > };
> >
> > global file_port_list: set[port] = set();
> >
> > redef Communication::nodes += {
> >      ["python"] = [$host = 127.0.0.1, $events = /config_update/,
> $connect=F, $ssl=F]
> > };
> >
> > event bro_init()
> > {
> >    Input::add_table([$source=file_port_list_loc, $name="file_port_list",
> $idx=Portsx, $destination=file_port_list]);
> > }
> >
> > event bro_done()
> > {
> >    Input::remove("file_port_list");
> > }
> >
> >
> > event config_update()
> > {
> >    Input::force_update("file_port_list");
> > }
> >
> > It works really well. But when I terminate bro process, it goes into
> defunct state and throws an error on the console.
> > Error: received signal while waiting for thread
> /usr/local/bro/share/bro/base/protocols/file_port_list/Input::READER_ASCII,
> aborting all ...
> >
> > Anything am I doing wrong??
> >
> > Thanks a lot for looking..!!
> >
> >
> > Regards,
> > Nabil
> > _______________________________________________
> > Zeek mailing list
> > zeek at zeek.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
>
> --
> Justin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200526/87407ba4/attachment-0001.html