[Bro-Dev] design summary: porting Bro scripts to use Broker
Johanna Amann
johanna at icir.org
Tue Oct 10 12:05:14 PDT 2017
> On Fri, Oct 06, 2017 at 16:53 +0000, you wrote:
>
> > # contains topic prefixes
> > const Cluster::manager_subscriptions: set[string] &redef;
> >
> > # contains (topic string, event name) pairs
> > const Cluster::manager_publications: set[string, string] &redef;
>
> I'm wondering if we can simplify this with Broker. With the old comm
> system we needed the event names because that's what was subscribed
> to. Now that we have topics, does the cluster framework still need to
> know about the events at all? I'm thinking we could just go with a
> topic convention and then the various scripts would publish there
> directly.
>
> In the most simple version of this, the cluster framework would just
> hard-code a subscription to "bro/cluster/". And then scripts like the
> Intel framework would just publish all their events to "bro/cluster/"
> directly through Broker.
>
> To allow for distinguishing by node type we can define separate topic
> hierarchies: "bro/cluster/{manager,worker,logger}/". Each node
> subscribes to the hierarchy corresponding to its type, and each script
> publishes according to where it wants to send events to (again
> directly using the Broker API).
>
> I think we could fit in Justin's hashing here too: We add per node
> topics as well ("bro/cluster/node/worker-1/",
> "bro/cluster/node/worker-2/", etc.) and then the cluster framework can
> provide a function that maps a hash key to a topic that corresponds to
> currently active node:
>
> local topic = Cluster:topic_for_key("abcdef"); # Returns, e.g., "bro/cluster/node/worker-1"
> Broker::publish(topic, event);
>
> And that scheme may suggest that instead of hard-coding topics on the
> sender side, the Cluster framework could generally provide a set of
> functions to retrieve the right topic:
>
> # In SumStats framework:
> local topic = Cluster::topic_for_manager() # Returns "bro/cluster/manager".
> Broker::public(topic, event);
>
> Bottom-line: If we can find a way to steer information by setting up
> topics appropriately, we might not need much additional configuration
> at all.
Just to add my two cents here - I like this a whole lot better and agree
that mostly steering events through topics seems like a neat choice.
Johanna
More information about the bro-dev
mailing list