<div><div dir="auto">I believe that Robin meant the intel framework instead of sumstats. (Hopefully this avoids some confusion)</div><div dir="auto"><br></div><div dir="auto"> .Seth</div><br><div class="gmail_quote"><div>On Sat, Aug 26, 2017 at 11:12 AM Robin Sommer <<a href="mailto:robin@icir.org">robin@icir.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Jon, replacing send_id() may indeed work better with an extension at<br>
the C++/Broker level. I'd like to avoid introducing new dependencies<br>
on Bro's serialization code, as I'm very much hoping that once the old<br>
communication code code goes we won't need that serialization layer<br>
anymore either (I know we're using it for opaque values over Broker<br>
too, but that's quite contained and should be easy to replace).<br>
<br>
Some thoughts here:<br>
<br>
- I'm thinking the best approach may be a new Bro-specific<br>
message for Broker, similar to the log-create/write messages.<br>
We can add that to the Bro shim in Broker. It'd send the name<br>
of the ID and the new value as a broker::data instance, and the<br>
receiving Bro updates the value as the message comes in.<br>
<br>
- There's one larger problem with replacing send_id() though: the<br>
old communication system has logic to send large values<br>
incrementally, so that send_id() won't block stuff. As Seth<br>
reminded me the SumStats framework is relying on that quite<br>
extensively for sending tables around. Incremental operation is<br>
something we don't have with Broker. I think that's ok, we can<br>
replace the few existing use cases of sending large values with<br>
something else. For SumStats that should probably be data<br>
stores. I don't remember if there even are further instances of<br>
this, my guess is no (I don't think we need to consider<br>
broctl's configuration updating here; those values are small<br>
and a non-incrememtnal send_id() is fine).<br>
<br>
- Another approach for broctl's updating could be switching to<br>
the upcoming configuration framework, which takes a different<br>
approach to dynamic reconfiguration. However, it's a bit out<br>
still until we can that switch completely, so for now providing<br>
a substitute for send_id() that can cover the simple uses cases<br>
looks like the best way to go.<br>
<br>
Robin<br>
<br>
On Fri, Aug 25, 2017 at 13:15 -0500, Jonathan Siwek wrote:<br>
<br>
> commit 8dddae17db9340f5261d11382aa9b67e965d5fef<br>
> Author: Jon Siwek <<a href="mailto:jsiwek@illinois.edu" target="_blank">jsiwek@illinois.edu</a>><br>
> Date: Fri Aug 25 13:15:00 2017 -0500<br>
><br>
> Finish port of control framework to use broker.<br>
><br>
> To replace usages of the send_id BIF, I had to make 3 new BIFs<br>
><br>
> - serialize<br>
> - deserialize<br>
> - update_ID<br>
><br>
> Using those, any Bro value can be explicitly converted to a string<br>
> of bytes, sent to a peer via a Broker event, unserialized to a Bro<br>
> val on the other side and then installed into a global identifier<br>
> via its name.<br>
><br>
> I think this may be the most straightforward way to get this to<br>
> work for now, at least without changing significantly the Broker<br>
> internals or messaging format. It mostly boils down to not being<br>
> able to deserialize into a Bro value with the 'any' type, at least<br>
> not without also carrying Bro's actual type information somewhere<br>
> inside Broker's serialized message.<br>
><br>
> And I think deserializing into 'any' would be needed because it's<br>
> not possible to e.g. explicitly enumerate all possible types in a Bro<br>
> script and have a particular event signature to use for any given one.<br>
> That's not possible because there's infinite ways you can create<br>
> composite types (tables, sets, vectors, etc).<br>
<br>
--<br>
Robin Sommer * ICSI/LBNL * <a href="mailto:robin@icir.org" target="_blank">robin@icir.org</a> * <a href="http://www.icir.org/robin" rel="noreferrer" target="_blank">www.icir.org/robin</a><br>
_______________________________________________<br>
bro-dev mailing list<br>
<a href="mailto:bro-dev@bro.org" target="_blank">bro-dev@bro.org</a><br>
<a href="http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev" rel="noreferrer" target="_blank">http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev</a><br>
</blockquote></div></div><div dir="ltr">-- <br></div><div class="gmail_signature" data-smartmail="gmail_signature">Seth Hall * Corelight, Inc * <a href="mailto:seth@corelight.com" target="_blank">seth@corelight.com</a> * <a href="http://www.corelight.com" target="_blank">www.corelight.com</a></div>