[Zeek] Input framework

[Nabil Memon]

Hi Jon,

I want to keep bro in suspended mode until a signal through broccoli
framework kicks in.
I see bro utilizing 100% CPU when in suspended state. And normal
percentage CPU(roughly 15-20%) usage when in continued state.

Why bro is taking up so much CPU when in suspended state?

*cat test.bro*
*=======================================================================*
global file_port_list: set[port] = set();
global file_port_list_loc: string = "/tmp/file_port_list";
type Portsx: record {
  port_num: port;
};
event bro_init()
{
    suspend_processing();
    Input::add_table([$source=file_port_list_loc, $name="file_port_list",
$idx=Portsx, $destination=file_port_list]);
    Input::remove("file_port_list");
}

#event Input::end_of_data(name: string, source: string)
#{
#  if ( name == "file_port_list"  ) {}
#    #continue_processing();
#}
*======================================================================= *

*bro -i eth1 test.bro -C*
CPU usage using ps and top command shows 100% usage.
>From the source I could just figure out that, when all the sources are
idle, bro takes some rest of 20 usec. But it seems sources are not getting
idle for some reason.

Below is the strace output.

*strace -p <PID of bro>*
select(17, [4 7 9 11 12 14 16], [0 7 9], [0 7 9], {0, 0}) = 1 (out [0],
left {0, 0})
poll([{fd=11, events=POLLIN}], 1, 0)    = 0 (Timeout)
select(17, [4 7 9 11 12 14 16], [0 7 9], [0 7 9], {0, 0}) = 1 (out [0],
left {0, 0})
poll([{fd=11, events=POLLIN}], 1, 0)    = 0 (Timeout)
select(17, [4 7 9 11 12 14 16], [0 7 9], [0 7 9], {0, 0}) = 1 (out [0],
left {0, 0})
poll([{fd=11, events=POLLIN}], 1, 0)    = 0 (Timeout)


- Nabil

On Thu, May 28, 2020 at 10:04 PM Jon Siwek <jsiwek at corelight.com> wrote:

> Are you missing a matching call to `continue_processing()` ?
>
> event Input::end_of_data(name: string, source: string)
>     {
>     if ( name == "file_port_list" )
>         continue_processing();
>     }
>
> - Jon
>
> On Thu, May 28, 2020 at 2:28 AM Nabil Memon <nabilmemon.ec at gmail.com>
> wrote:
> >
> > Hi Jon,
> >
> > Even in the latest zeek version which is 3.1.3, it reaches 100% when I
> integrate Input framework,
> >
> > cat  /tmp/file_port_list
> > #fields port_num
> > 60000/tcp
> > 8080/tcp
> > 49154/tcp
> > 55907/tcp
> > 49152/tcp
> > 49153/tcp
> > 8000/tcp
> > 5357/tcp
> > 80/tcp
> >
> > cat test.zeek
> > global file_port_list: set[port] = set();
> > global file_port_list_loc: string = "/tmp/file_port_list";
> > type Portsx: record {
> >   port_num: port;
> > };
> > event zeek_init() &priority=5
> > {
> >   suspend_processing();
> >   Input::add_table([$source=file_port_list_loc, $name="file_port_list",
> $idx=Portsx, $destination=file_port_list]);
> >   Input::remove("file_port_list");
> > }
> >
> > Regards,
> > Nabil
> >
> >
> > On Tue, May 26, 2020 at 11:05 PM Nabil Memon <nabilmemon.ec at gmail.com>
> wrote:
> >>
> >> I am using 2.6.x version.
> >>
> >> On Tue, 26 May, 2020, 10:53 pm Jon Siwek, <jsiwek at corelight.com> wrote:
> >>>
> >>> On Tue, May 26, 2020 at 8:47 AM Nabil Memon <nabilmemon.ec at gmail.com>
> wrote:
> >>>
> >>> > I got caught up in some other side effect. Bro's CPU usage goes 100%
> when I use input framework
> >>>
> >>> What version are you using?  You should try comparing against one of
> >>> the latest 3.0.x or 3.1.x releases since there's specifically things
> >>> addressed in them that might explain that behavior.  E.g.
> >>> https://github.com/zeek/broker/pull/97
> >>>
> >>> - Jon
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200529/ebacb819/attachment.html