[Bro-Dev] libbroker status/plans

Siwek, Jon jsiwek at illinois.edu
Wed Oct 22 12:39:19 PDT 2014

If anyone has time/interest, I feel like the main components of Broker are established now and deserving feedback/critique.  Rather than try to detail how things work here, it’s probably best for people to try figuring things out from the repo (e.g. source code comments and unit test examples) and ask questions about what's unclear.

But it would be helpful to start a discussion on some of the planned features and open questions.  I’ll try literally pasting my TODO list and hope it’s readable.  The items are roughly ordered from most-certainty to least-certainty.  Feedback welcome generally, but particularly where questions are posed.

Broker TODO


- Python bindings

- Persistent storage backend

- SSL/IPv6 (dependent on actor-framework support)

- Need/want overload or flow-control mechanisms?

    E.g. a simple policy for handling overload is to let a user specify
    a threshold for how many items are allowed in a queue before new
    messages are dropped.

- In-place data store value modifications

    Plan to support increment/decrement on integral values.
    Need any other operations?

    What to do when applying an operation to invalid data type?
    Plan to just send error message back to sender and leave further
    decisions up to them.

- Data store support for optional expiry model

    What are the desired mechanims?  Options:

        (1) Inserter may specify "expire this entry at time X" ?

        (2) Inserter may specify "expire this entry based on
            create/read/modification access time" ?  Going this route,
            seems read access times would need to be shared across
            clones (contradicts goal of lightweight, local read-access)?

        (3) Other hooks to make expiry conditional?

- Data typing model

    Currently data in Broker is similar to Bro's threading::Value in
    that full type info (from Bro's perspective) isn't available, just
    the raw storage required for the types.  Broker currently differs in
    that it doesn't use any tag to distinguish between primitives that
    share the same storage (e.g. enum/string or double/interval).
    Interpretation of types is left entirely up to the receiver.

    Do we need more strict typing than this?  Options:

        (1) Data holds additional type tag to suggest how to interpret

        (2) Fully implement separate Bro-types.

    Planning to try integrating w/ Bro as it is and see what specific
    problems arise.  I think (1) may end up being helpful, but maybe not
    required and I'd like to avoid (2) if possible.

- Bro integration

    Is Broker the default in Bro 2.4 ?  That implies requiring C++11.
    Also I'm requiring CMake 2.8.12+ and may be hard to go below 2.8.
    Bro is still happy with 2.6.3.

More information about the bro-dev mailing list