[Bro] bro cluster in containers

[Maerz, Stefan A.]

> On Jun 1, 2018, at 10:04 AM, Poore, Jeffrey S <jeffrey.s.poore at bankofamerica.com> wrote:
> 
>> I've been meaning to try to build this out using k8s, just haven't had time.
> 
> We do plan to migrate to k8s at some point. In fact, I'd prefer doing it that way. The reason we are using Mesos is that it was chosen by two of the original developers before I joined the project. They were not so much focused on the containers part of it as they were on hosting MapR.
> 
>> To really be useful you also need to automate the configuration of the tapagg layer.
> 
> Can you elaborate on what that means? :)

Tap aggregator is a network switch with provides a layer of indirection between your bro cluster and your network taps. It allows you to “route around" or duplicate data (maybe you also want the same data sent to Snort and full PCAP) and slice it up into smaller, more manageable, network flows. In a bro cluster, it is what load balances between bro worker nodes.

Automation of the tapagg layer means that you would use the API your Tap aggregator has to turn on and turn off the switch ports that connect to containers. The orchestration component, would spin up/tear down containers/pods and simultaneously turn on/off the corresponding switchport on the tap aggregator. The way I’m thinking of it, you wouldn’t use Kubernetes/Mesos/ect. to route network traffic around, rather your orchestration platform would control the dataplane of your tap aggregator. My experience is that handling high throughput traffic in software is a recipe for traffic loss and frustration. Do as much as you can in hardware.




Either way, I am very interested in this conversation — we run RedHat’s Openshift which is a Kubernetes distribution. Running bare metal works really well for us, but if I could get on OpenShift, I could get rid of most of our remaining physical machines. My question is whether or not broctl can be made to gracefully handle the rapid elastic characteristics of container orchestration — for example can I add/remove bro worker nodes/containers without doing a “broctl deploy” or “broctl restart”? I’m sure I can just restart the service, but that seems like a disruptive and non elegant solution.


Best Regards,
-Stefan



--
Stefan Maerz
HPC Cyber Security Engineer
Oak Ridge National Laboratory
linkedin.com/in/stefanmaerz