[Bro-Dev] Plugins providing threads?

[Aashish Sharma]

Very timely question, I've be mulling over this and I would like to vote for adding thread component. 

This may allow us to do a lot more processing of data in the script land.

Now my use case may not be likely an ideal one.  

I am *experimenting* with a policy to flag very long sustained persistant connections between two hosts (irrespective of ports). For this, I am storing connection information [src, dst] in a table and based on various conditions/heuristics I am expiring these at different dynamic times (so read_expire isn't helpful). For 'spike' elimination from scanners, I iterate through the table every min. (unoptimal but stay with me). At present when table stores about 4 Million elements, iterating through it freezes entire bro (no growth of conn log etc).  

A seperate thread could allow me to delegate these kinds of tasks outside Bro's main thread/event queue. 

Aashish




On Tue, Oct 07, 2014 at 03:43:01PM -0700, Robin Sommer wrote:
> I'm wondering if we should add another type of plugin component:
> threads. This would be for functionality that's to run in parallel
> with Bro's main thread, and communicate with it via message passing.
> 
> We have the structure for that in place already, logging and reading
> are using it as already. But this would formalize the notion a bit
> more directly that a plugin can provide a new thread, with its own
> logic; and also extend the interface that it has available from inside
> that thread (e.g., being able to raise events; have bif functions
> that, when called, get passed through to the thread).
> 
> One use case is Christian's OpenFlow plugin: if we went the route of
> integrating an external library for speaking OpenFlow directly, that
> communication needs to be handled somewhere. Traditionally, that would
> be an IOSource hooked into the main loop. The plugin model could
> support that, too, but being able to fully decouple it inside its own
> thread seems appealing.
> 
> Jon, how are you planing to integrate Broker into Bro? Would this help
> there as well if you could just follow a similar structure with Broker
> running inside its own thread?
> 
> Robin
> 
> -- 
> Robin Sommer * Phone +1 (510) 722-6541 *     robin at icir.org
> ICSI/LBNL    * Fax   +1 (510) 666-2956 * www.icir.org/robin
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

-- 
Aashish Sharma	(asharma at lbl.gov) 				 
Cyber Security, 
Lawrence Berkeley National Laboratory  
http://go.lbl.gov/pgp-aashish 
Office: (510)-495-2680  Cell: (510)-612-7971
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20141007/32bf996e/attachment.bin