[Bro-Dev] Broker::publish API

[Robin Sommer]

On Fri, Aug 10, 2018 at 10:24 -0500, Jonathan Siwek wrote:

> Or is it a matter of "if a user needed it for something, then it's
> available" ?

Yeah, including matching expectations: if there's a
"bro/cluster/worker" topic, I'd expect I can publish there to reach
all the workers (from anywhere). However, I think I'm with you now
that maybe we just shouldn't do do/support any forwarding in the
cluster right now. Pools and manual relaying are a (currently better)
alternative, and we can change things later. And at least it's a clear
message: no forwarding across cluster nodes.

> However, I can see Broker::forward() could make it a bit easier for a
> user wanting to manually set up a forwarding route between clusters or
> other external applications.  Is that a clear use-case we need to
> cater to now?

Well, if it were easy to add the forward() function, that could indeed
be quite useful for external integrations still. With that, one could
selectively forward custom topics (at one's own risk), without causing
a mess for the cluster. I'm thinking osquery integration for example,
where messages might go through an intermediary Bro. One advantage
that Broker-internal forwarding has compared to manual relaying is
that messages won't be propagated back to the sender.

But it's a matter of effort at this point I'd say.

> RR via proxy is not just load-balancing either, but fault-tolerance as
> well.

Yeah, that's right.

> But here you're talking more about removing the relay() functions and
> doing the RR-via-proxy "manually", right?  That seems ok to me -- once
> "real" routing is available, you then have the option to simplify your
> script and get a minor optimization by not having to manually
> handle+forward the event on proxies.

Ok, let's make that change then, I think removing relay() will help
for sure making the API easier.

Robin

-- 
Robin Sommer * Corelight, Inc. * robin at corelight.com * www.corelight.com