[Bro-Dev] Broker's remote logging (BIT-1784)

[Azoff, Justin S]

> On Jan 31, 2017, at 5:41 PM, Robin Sommer <robin at icir.org> wrote:
> 
...
> The fact that RemoteSerializer and broker::Manager are calling
> different Write() functions seems to be a broader issue: we get
> different semantics that way. For RemoteSerializer, log events and log
> filters run only on the originating nodes; those guys make all
> decisions about what's getting logged exactly and they then send that
> on to the manager, which just writes out the data it receives. With
> Broker, however, both events and filters run (also) on the manager, so
> that it's making its own decisions on what to record. The filters can
> be different on the manager, and they will have access to different
> state.
> 
> I'm not sure what approach is better actually, I think the Broker
> semantics can be both helpful and harmful, depending on use case.  In
> any case, it's a change in semantics compacted to the old
> communication system, and I'm not sure we want that.

I think we want the old behavior for 2 reasons:

1. The workers only send the &log fields to the managers, so the events are raised with half the fields missing.

2. Having the logger node be as much of a dumb byte mover as possible is best for performance reasons.  Having the log events and log filters run on the workers lets that functionality scale out across the nodes.  Especially if a filter is used that would remove a large percent of the entries.

If someone really wanted the log_* events to run on the manager, they could redef Cluster::worker2manager_events right?

> I'm wondering if there's a reason that in the Broker case things
> *have* to be this way. Is there something that prevents the Broker
> manager from doing the same as the RemoteSerializer?
> 

Jon would know best, but I'd guess one form was more convenient to use than the other and it may have been assumed that they both did the same thing.  


-- 
- Justin Azoff