[Bro] High-CPU on just a single worker in the cluster

[Seth Hall]

> On Apr 14, 2016, at 8:26 AM, Dave Crawford <bro at pingtrip.com> wrote:
> 
> Below is a fresh 'netstats' from this morning (WIN_INT-9 is obviously the culprit).

You are seeing the side effects of hot spotting.  It's likely that you have a connection which is massively larger than all others that your host is seeing.  You could filter your traffic to stop seeing that flow which would cut down on the problems you are seeing.

I just pushed a new repository that might help you address your issue.  It will still require a lot of clean up and documentation but if you just load the module as-is, it might automatically help you. :)  It automatically identifies large and fast flows and places BPF filters to automatically shunt them while still watching for SYN/ACK/FIN/RST flags.  

	https://github.com/sethhall/bro-bulkflows

There are a couple of configuration options you'll want to pay attention to, so I'll just give you an example blurb you could put into your local.bro:

@load bro-bulkflows
# Set a minimum flow size before considering shunting (32MB)
redef BulkFlows::size_threshold = 33554432;
# Set a minimum flow data rate before considering shunting (50Mbps)
BulkFlows::speed_threshold = 6250000;

# If you have VLAN tags on your traffic, you will also have to add the following line because packet filtering with BPF is generally broken otherwise...
#redef PacketFilter::restricted_filter="vlan";

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/