[Bro-Dev] New logging architecture

Robin Sommer robin at icir.org
Mon Jul 4 15:48:51 PDT 2011


On Fri, Jul 01, 2011 at 16:38 -0700, you wrote:

> So, to change this to support threading, I was planning to turn 
> LogMgr::Stream into a self-contained object with two 0mq message-passing 
> sockets attached:

Thinking about this a bit more, I'm not sure Stream is the right piece
to turn into a thread. Two issues:

    - That would give us only one thread per type of writer (e.g., one 
    for all ASCII logs).

    - Not all the things that the Stream does are sufficiently
    decoupled from the main thread, and they wouldn't be thread-safe.
    There's in particular the filtering, where Bro goes back into
    script-land, but also raising the log event and just adding new
    filters.

So how about instead turning each LogWriter into a thread in the same
way as you propose: making it self-contained with message-passing 0mq
sockets and a simple protocol for sending the various types of 
messages (like Init, Write, etc.; just as you said).

> than encapsulating everything when passing within a single process...

By encapsulating do you mean the LogVal::{Read,Write} serializations?
I don't think we'll actually get around them. They are to make things
thread-safe by decoupling the writer's data from Bro's main data
structures.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


More information about the bro-dev mailing list