[Bro-Dev] Broker::publish API

[Azoff, Justin S]

> On Jul 27, 2018, at 1:39 PM, Robin Sommer <robin at corelight.com> wrote:
> 
> I'm wondering if we should give it another try to simply this API
> while we still can (i.e., before 2.6 goes out). To me, the most
> intuitive publish operation is "send to topic T and propagate to
> everybody subscribed to that topic". I'd structure the API around
> that, making that the main publish function for that simply:
> 
>    Broker::publish(topic, args);
> 
> That would send to all neighbors, which then process locally and relay
> to their neighbors. Right now, that would propagate just across one
> hop but once we have multihop that'd start being broadcasted out
> broadly.

This would do weird things on workers, since they connect to both the manager and proxies.

Worker 1 would send to it's neighbors [manager, proxy1, proxy2] but then those 3 nodes would
relay to all of the other workers.  The TTL would stop the propagation, but you'd still end up sending
3 copies of the same message to each worker.

I do agree that there's room for a lot of simplification, for example a worker broadcasting a message efficiently to all
other workers needs to do something like this from the docs:

        Cluster::relay_rr(Cluster::proxy_pool, "example_key",
                          Cluster::worker_topic, worker_to_workers,
                          Cluster::node + " (via a proxy)");

But a lot of that could have defaults:

Most use cases would want to relay through the default proxy pool
Since round robin is in use, they key shouldn't matter.
The round robin part itself is really an implementation detail for proxy load balancing and maybe not something that
should be exposed in the API.  Now that I think of it I'm not sure why one would ever use relay_hrw over relay_rr.

Removing a lot of that gets close to what you are suggesting:

    Cluster::relay(Cluster::worker_topic, worker_to_workers, Cluster::node + " (via a proxy)");

which is I guess just

    Cluster::relay(topic, args)

like you said.

— 
Justin Azoff