[Bro-Dev] Broker::publish API

[Jon Siwek]

On Mon, Aug 13, 2018 at 8:09 AM Jan Grashöfer <jan.grashoefer at gmail.com> wrote:
>
> On 10/08/18 17:12, Robin Sommer wrote:
> > I hear you, but I think I haven't quite understood the concern yet.
> > Can you give me an example where the difference matters? What's
> > different between publishing intel events to bro/cluster/worker/intel
> > vs bro/cluster/worker if both go to all workers? Or is it so that some
> > workers can decide not to receive the intel events?
>
> The use case I had in my mind is an external application that is
> interested in interfacing with the intelligence framework. Either for
> querying it similar to workers of for managing purposes. If possible, it
> could be beneficial for such an application to receive only the relevant
> parts of cluster communication.
>
> On 10/08/18 17:52, Jon Siwek wrote:
> > (1) if the event you're publishing just facilitates scalable cluster
> > analysis: you'd tend to use the topic names which target node classes
> > within a cluster (eventually this might be "bro/<cluster-id>/worker")
> >
> > (2) if the event you're publishing is intended for external
> > consumption, then you should use a topic which describes some specific
> > qualities of the message (e.g. "jan/intel")
>
> The case described above seems to be both. On the one hand the primary
> use case is internal cluster communication. On the other hand it feels
> quite natural to dock here for external applications. Another
> (debatable) use case might be directly interfacing the configuration
> framework, skipping the configuration file layer.

I'm generally thinking there's nothing stopping one from picking a new
topic name to re-publish some set of events under.  Would that be
possible in the case you're imagining?

I don't think we're going to come up with a general (or enforce-able)
way of picking topic names such that they'll be useful for any
arbitrary, external use-case.  So we pick the topic name that is best
for the use-case we have at time of writing a script (e.g. we just
want to get it working on a cluster so use the pre-existing topics
that are available for that), and then let others re-publish a subset
of events under different topics dependent on their specific use-case.

- Jon