[Bro-Dev] [desired broker api as oppose to whats in known-hosts.bro]

Aashish Sharma asharma at lbl.gov
Fri Mar 3 13:36:06 PST 2017

SO I came across a sample of Broker-API usage:

when (local res = Broker::exists(Cluster::cluster_store, Broker::data("known_hosts")))
       		local res_bool = Broker::refine_to_bool(res$result);
		when ( local res2 = Broker::lookup(Cluster::cluster_store, Broker::data("known_hosts")) )
			local res2_bool = Broker::set_contains(res2$result, Broker::data(host));

				Broker::add_to_set(Cluster::cluster_store, Broker::data("known_hosts"), Broker::data(host));
				Log::write(Known::HOSTS_LOG, [$ts=network_time(), $host=host]);
		timeout 10sec
                	{ print "timeout"; }
             timeout 20sec
             { print "timeout"; }

Now this isn't too complicated but I find it cumbersome and one needs to understand execution flow since "when" is involved etc etc etc. 


Here is how I'd envision broker usage (and I know easier said than done...)


global known_hosts: table[addr] of blah &store ; 


when I query the table :
	if (addr in known_hosts)

1a) Given &store directive above, Bro should go and check store if value isn't already in the table and update as needed in background - if value isn't in the store, if condition would fail anyways. 

may be 	
1b) You can probably maintain a bloomfilter which builds itself from whats out there in table and works as an index or a membership check 


1c) We also need another directive something akin to "&expire_store = 1 hrs", which expires entires from the table in memory and puts into the store. 

on implementation side I am pretty sure there are complexities since broker model is different and I don't grasp it yet. 

But from current bro scripting prespective: 

(1a) will be If value isn't in the bloomfilter, call a Input::Event (which reads the data from the store) and fire end-of-data like event or similiar to input-framework give me a capability to fire events when data read from store is complete. So this would eliminate the "when" construct and give a much clearer event based code execution path (from scripters prespective) 

for (1c) Likewise, for &expire_store - I am merely using &(read|write|create)_expire functions to write to the database. 

So in-summary we need a directive &store which works with sets, tables and bro data types. 

If value is in member OK else broker in background goes and checks store and in background updates sets, tables, datastructures and then fires events when done. 

Save me from all the above "when" constructs and Broker::lookup routines. Let those be in background. 

I'd be happy to talk in person or video conference if more clearification is needed. 


More information about the bro-dev mailing list