[Bro-Dev] send_id (Re: [Bro-Commits] [git/bro] topic/jsiwek/actor-system: Finish port of control framework to use broker. (8dddae1))

[Seth Hall]

I believe that Robin meant the intel framework instead of sumstats.
(Hopefully this avoids some confusion)

  .Seth

On Sat, Aug 26, 2017 at 11:12 AM Robin Sommer <robin at icir.org> wrote:

> Jon, replacing send_id() may indeed work better with an extension at
> the C++/Broker level. I'd like to avoid introducing new dependencies
> on Bro's serialization code, as I'm very much hoping that once the old
> communication code code goes we won't need that serialization layer
> anymore either (I know we're using it for opaque values over Broker
> too, but that's quite contained and should be easy to replace).
>
> Some thoughts here:
>
>      - I'm thinking the best approach may be a new Bro-specific
>        message for Broker, similar to the log-create/write messages.
>        We can add that to the Bro shim in Broker. It'd send the name
>        of the ID and the new value as a broker::data instance, and the
>        receiving Bro updates the value as the message comes in.
>
>      - There's one larger problem with replacing send_id() though: the
>        old communication system has logic to send large values
>        incrementally, so that send_id() won't block stuff. As Seth
>        reminded me the SumStats framework is relying on that quite
>        extensively for sending tables around. Incremental operation is
>        something we don't have with Broker. I think that's ok, we can
>        replace the few existing use cases of sending large values with
>        something else. For SumStats that should probably be data
>        stores. I don't remember if there even are further instances of
>        this, my guess is no (I don't think we need to consider
>        broctl's configuration updating here; those values are small
>        and a non-incrememtnal send_id() is fine).
>
>      - Another approach for broctl's updating could be switching to
>        the upcoming configuration framework, which takes a different
>        approach to dynamic reconfiguration. However, it's a bit out
>        still until we can that switch completely, so for now providing
>        a substitute for send_id() that can cover the simple uses cases
>        looks like the best way to go.
>
> Robin
>
> On Fri, Aug 25, 2017 at 13:15 -0500, Jonathan Siwek wrote:
>
> > commit 8dddae17db9340f5261d11382aa9b67e965d5fef
> > Author: Jon Siwek <jsiwek at illinois.edu>
> > Date:   Fri Aug 25 13:15:00 2017 -0500
> >
> >     Finish port of control framework to use broker.
> >
> >     To replace usages of the send_id BIF, I had to make 3 new BIFs
> >
> >     - serialize
> >     - deserialize
> >     - update_ID
> >
> >     Using those, any Bro value can be explicitly converted to a string
> >     of bytes, sent to a peer via a Broker event, unserialized to a Bro
> >     val on the other side and then installed into a global identifier
> >     via its name.
> >
> >     I think this may be the most straightforward way to get this to
> >     work for now, at least without changing significantly the Broker
> >     internals or messaging format.  It mostly boils down to not being
> >     able to deserialize into a Bro value with the 'any' type, at least
> >     not without also carrying Bro's actual type information somewhere
> >     inside Broker's serialized message.
> >
> >     And I think deserializing into 'any' would be needed because it's
> >     not possible to e.g. explicitly enumerate all possible types in a Bro
> >     script and have a particular event signature to use for any given
> one.
> >     That's not possible because there's infinite ways you can create
> >     composite types (tables, sets, vectors, etc).
>
> --
> Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>
-- 
Seth Hall * Corelight, Inc * seth at corelight.com * www.corelight.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20170827/6c7a4abd/attachment.html