[Bro-Dev] [Bro-Commits] [git/bro] topic/actor-system: First-pass broker-enabled Cluster scripting API + misc. (07ad06b)

Siwek, Jon jsiwek at illinois.edu
Thu Nov 2 19:15:38 PDT 2017

> On Nov 2, 2017, at 5:33 PM, Azoff, Justin S <jazoff at illinois.edu> wrote:
> The optimization could be built into broker though, something like
>    Broker::broadcast_magic_once_whatever(Cluster::worker_pool, key, SSL:: intermediate_add, key, value);
> That would hash the key, send it to a data node, then have the data node broadcast the 
> event

The first thing I’d suggest is a new internal broker message type that can publish a message via a topic and then on any receiving nodes re-publish via a second topic.  In this way, we could distribute to all workers via a single proxy/data node as:

	local one_proxy_topic = Cluster::rr_topic(Cluster::proxy_pool, “ssl/intermediate_add”));
	local e = Broker::make_event(SSL::intermediate_add, key, value);
	Broker::relay(one_proxy_topic, Cluster::worker_topic, e);

Or potentially compressed into one call:

	Cluster::relay(first_topic, second_topic, rr_key, event, varargs...);

> while adding key to a 'recently broadcasted keys' table that only needs to buffer for 10s or so.
> This would enable you to efficiently broadcast an event (once) across all workers with a single line of code.

I’m not sure about this part.  Is that just for throttling potential duplicates?  Is that something you want generally or just for the this particular example?  I’m thinking maybe it can wait until we know we have several places where it’s actually needed/used before compressing that pattern into a single BIF/function.

> I guess my whole point with all of this is that if the intent of a script is that an event should be seen on all workers, the script should look something like
>    Broker::broadcast(Cluster::worker_pool, SSL:: intermediate_add, key, value);
> and not have a bunch of redefs and @ifs so that the script can eventually have
>    event SSL::new_intermediate(key, value);

Yeah, minimizing the need for people to have to constantly implement various code paths that depend on cluster/node-type is a good goal and what I’m also aiming at.

>>> How would something like policy/protocols/ssl/validate-certs.bro look with intermediate_cache as a data store?
>> global intermediate_store: Cluster::StoreInfo;
>> event bro_init()
>> 	{
>> 	intermediate_store = Cluster::create_store(“ssl/validate-certs/intermediate_store");
>> 	}
>> And then port the rest of that script to use broker data store api (get/put/exists calls) to access that store.
>> - Jon
> Does that have the same performance profile as the current method?
>    if (issuer in intermediate_cache)
> vs
>    Broker::get(intermediate_cache, issuer)

Theoretically, they’re both in-memory hash-table lookups, though the implementations are obviously very different and I don’t know how they compare in reality.

I think it’s true that many scripts could be ported to use either data stores or just explicitly exchange events.  Probably the preference to use a data store would be for cases where it may be useful to keep persistent data across crashes/restarts.

- Jon

More information about the bro-dev mailing list