[Bro] High packet drop rates

[Donaldson, John]

I've recently run into a problem, with Bro 2.4.1, where I have extremely high packet loss.

I'm running on a server with dual quad-core Xeon processors, with no hyperthreading, and 64GB of RAM, monitoring a few small links, averaging, in aggregate, about 100Mbps of traffic. There isn't too much else running on this system, but we're seeing drop rates that average in the 70-99% range (via the capture-loss.bro policy), even though Bro's CPU utilization sits at around 20-30%. 

Most of the traffic comes in from a bond interface, running on top of some Intel NICs, but we're seeing similarly high drop rates when directly capturing from another, non-bonded interface. Using other tools, we're not seeing any dropped packets (even with a heavily-loaded Snort instance). We've tried PF_RING and load-balancing across several workers, pinned to several CPUs, but all that we end up with, then, are multiple processes with 2-30% CPU utilization and 70-99% drop rates. PF_RING isn't showing any drops on its side, and hans't had issues with insufficient memory. We're pretty sure that we're not just seeing TCP-related chaff that's throwing off our numbers, because records of known connections are showing up malformed.

Any insights?

John Donaldson