[Bro-Dev] Broker's remote logging (BIT-1784)

Alan Commike commike at reservoir.com
Tue Jan 31 16:06:30 PST 2017


On Tue, Jan 31, 2017 at 3:48 PM Azoff, Justin S <jazoff at illinois.edu> wrote:

>
> 2. Having the logger node be as much of a dumb byte mover as possible is
> best for performance reasons.  Having the log events and log filters run on
> the workers lets that functionality scale out across the nodes.  Especially
> if a filter is used that would remove a large percent of the entries.
>

This.

Especially over time as we see more and more cores per processor, it's best
to distribute the processing load. By putting the filter in the logger, the
logger will then need to enter the interpreter for each log message to
determine if it needs to throw away data it just received. That's expensive
and limits scalability on multiple fronts.

   ...alan



>
> If someone really wanted the log_* events to run on the manager, they
> could redef Cluster::worker2manager_events right?
>
> > I'm wondering if there's a reason that in the Broker case things
> > *have* to be this way. Is there something that prevents the Broker
> > manager from doing the same as the RemoteSerializer?
> >
>
> Jon would know best, but I'd guess one form was more convenient to use
> than the other and it may have been assumed that they both did the same
> thing.
>
>
> --
> - Justin Azoff
>
>
>
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20170201/d832fb37/attachment.html 


More information about the bro-dev mailing list