[Bro-Dev] 0MQ security considerations

[Matthias Vallentin]

(I read Robin's follow up and have no objections with keeping the
hardware thread model instead of a task-based worker thread model.
My comments below are just for the record.)

> [..] this brings up a more general problem: what if we use a library
> that doesn't offer us the ability to do this?

That's a big issue. Some research at the Berkeley ParLab tried to
address this by providing a broker library; sort of a meta scheduler
that seeks to make sure cross-library calls do not hurt concurrency.
Alas, I cannot remember the name of the project but if you are
interested I can find it out.

> Then again, if the library doesn't let us do this, maybe we just
> shouldn't be using that particular library...?

If the library is uncontrollable with respect to the number of threads
it spawns, that's tough. Though most well-written libraries (not
stand-alone applications) I came across have some way to configure or
limit the number of used threads. Even if we used 4 different libraries,
each of which can be trimmed down to two threads, I'd say we have a
tractable situation because the number of cores will probably grow
faster than the minimum number of threads in libraries we use.

> What I don't like about this is that it means any scheduling the task
> manager does would be coarse-grained at best.

Agreed, the logging framework is not a component of Bro that would reap
major benefits from the worker thread model. The reason why I proposed
the task-based architecture is that it is a strong spine for modern
concurrent applications. However, Bro has enough muscle at this point,
which is why focusing on other body parts is probably a better idea. (I
hope this metaphor is not too awkward :-)

> Rough cut of a general overview is attached.  Probably going to change
> quite a bit, but I think it might help illustrate the way the system
> works overall.

Cool, that's a nice diagram. Having similar ones for the big parts of
Bro is a valuable resource for developers. Some quick
questions/comments:

    - In the LogMgr description, what's a log stream object?

    - I'm not sure if I understand the crossed-out CONTEXT BOUNDARY
      text. Does this essentially mean that the queue interface can (in
      the future) adapt based on how the distributed system is
      configured? I.e., if reader and writer are part of the same
      program, it is some sort of IN_PROC queue, and for IPC and
      networking it will transparently serialize the messages?

    - What is the Pull FIFO for? Does the LogEmissary receive
      feedback from the LogWriter?

    - Just out of curiousity, what type of thread-safe queue is it?
      Single-writer/single-reader? Single-writer/multiple-reader? There
      are zillions of thread-safe queue implementations out there, each
      optimized for some different use case! While we're at it, is the
      LogEmissary to LogWriter multiplicity 1:1 or 1:n?

    - About BasicThread: it seems there is one channel for both control
      and data messages, i.e., the queue can include a special terminate
      event to shutdown the BasicThread. Are these control messages
      filtered out in the base class (BasicThread) code before the
      message is passed to the child (E.g. LogWriter) code?

> Feedback / corrections / "THAT'S A STUPID MODEL!" are always
> appreciated.

I like the architecture (+1 with your words ;-) and am looking forward
to see it in action. What I am really interested in are profiling
benchmarks (very easy with Google perftools) to see where the non-I/O
bottlenecks occur. 

Sorry if I have missed it, but do we use now 0mq as messaging
middle-layer or is all message passing based on custom code?

    Matthias