[Bro-Dev] design summary: porting Bro scripts to use Broker

Johanna Amann johanna at icir.org
Tue Oct 10 12:05:14 PDT 2017


> On Fri, Oct 06, 2017 at 16:53 +0000, you wrote:
> 
> > 	# contains topic prefixes
> > 	const Cluster::manager_subscriptions: set[string] &redef;
> > 
> > 	# contains (topic string, event name) pairs
> > 	const Cluster::manager_publications: set[string, string] &redef;
> 
> I'm wondering if we can simplify this with Broker. With the old comm
> system we needed the event names because that's what was subscribed
> to. Now that we have topics, does the cluster framework still need to
> know about the events at all? I'm thinking we could just go with a
> topic convention and then the various scripts would publish there
> directly.
> 
> In the most simple version of this, the cluster framework would just
> hard-code a subscription to "bro/cluster/". And then scripts like the
> Intel framework would just publish all their events to "bro/cluster/"
> directly through Broker.
> 
> To allow for distinguishing by node type we can define separate topic
> hierarchies: "bro/cluster/{manager,worker,logger}/". Each node
> subscribes to the hierarchy corresponding to its type, and each script
> publishes according to where it wants to send events to (again
> directly using the Broker API).
> 
> I think we could fit in Justin's hashing here too: We add per node
> topics as well ("bro/cluster/node/worker-1/",
> "bro/cluster/node/worker-2/", etc.) and then the cluster framework can
> provide a function that maps a hash key to a topic that corresponds to
> currently active node:
> 
>     local topic = Cluster:topic_for_key("abcdef"); # Returns, e.g., "bro/cluster/node/worker-1"
>     Broker::publish(topic, event);
> 
> And that scheme may suggest that instead of hard-coding topics on the
> sender side, the Cluster framework could generally provide a set of
> functions to retrieve the right topic:
> 
>     # In SumStats framework:
>     local topic = Cluster::topic_for_manager() # Returns "bro/cluster/manager".
>     Broker::public(topic, event);
> 
> Bottom-line: If we can find a way to steer information by setting up
> topics appropriately, we might not need much additional configuration
> at all.

Just to add my two cents here - I like this a whole lot better and agree
that mostly steering events through topics seems like a neat choice.

Johanna


More information about the bro-dev mailing list