[Bro] Bro drop packets while not using CPU at full capacity

[Mehmet LEBLEBİCİ]

Hi,

I had such a problem before, there was no cpu or other problem but bro was
dropping the packets. It was saying packets are larger than expected MTU,
so it was dropping then. It turned out that some NICs offload the
reassembly of traffic into bigger packets so that fewer packets are passed
up to stack. Actually the intention is to reduce burden on CPU, but it also
causes bro to drop packets. So it may be a similar problem in your case. I
solved the problem changing interface settings, you can try the following
settings. I hope this will solve your problem.

ethtool -K eth0 sg off
ethtool -K eth0 tso off
ethtool -K eth0 ufo off
ethtool -K eth0 gso off
ethtool -K eth0 gro off
ethtool -K eth0 lro off

Best regards,

Mehmet Leblebici


27 Haziran 2016 Pazartesi tarihinde, Neslog <neslog at gmail.com> yazdı:

> I've been troubleshooting my clusters recently. I'm seeing some drops in
> the kernel using drop watch.  Previously I've seen loss from spans when
> approaching link saturation
> On Jun 25, 2016 7:34 PM, "Hashem Alaidaros" <aidaros.dev at gmail.com
> <javascript:_e(%7B%7D,'cvml','aidaros.dev at gmail.com');>> wrote:
>
>> I started my experiments when Bro 2.3 was the latest stable version. All
>> my results are based on 2.3, I can not shift to newer version now.
>> Anyone can clarify why Packet are dropping while no fully CPU
>> utilization.?
>>
>> Best regards
>>
>> On Sun, Jun 26, 2016 at 1:50 AM, Joe Blow <blackhole.em at gmail.com
>> <javascript:_e(%7B%7D,'cvml','blackhole.em at gmail.com');>> wrote:
>>
>>> Is there any reason you aren't using 2.4.x?  Step one would be to use
>>> that I would think. 2.4.x fixed a great many bugs I believe.
>>>
>>> Cheers,
>>>
>>> JB
>>>
>>> Sent from my BlackBerry Smartphone on the Verizon 4G LTE Network
>>> *From:*aidaros.dev at gmail.com
>>> <javascript:_e(%7B%7D,'cvml','aidaros.dev at gmail.com');>
>>> *Sent:*June 25, 2016 7:15 AM
>>> *To:*bro at pingtrip.com <javascript:_e(%7B%7D,'cvml','bro at pingtrip.com');>
>>> *Cc:*bro at bro.org <javascript:_e(%7B%7D,'cvml','bro at bro.org');>
>>> *Subject:*Re: [Bro] Bro drop packets while not using CPU at full
>>> capacity
>>>
>>> Thanks Dave,
>>> I couldn't get what you mean. How stats.bro calculate CPU usage, is it
>>> per core utilization? My bro machine is quad-core with hypertheading
>>> enabled, means 8 logical cores. So, if one core is fully utilized then
>>> stats should report 12.5% (100/8), not 40% or 60% as in my case. How my Bro
>>> report 60% CPU with 20% drop packet rate reported? Is there any reason that
>>> make packet drop?
>>> Anyone could clarify please.
>>> Thanks in advance
>>>
>>> On Sat, Jun 25, 2016 at 10:50 AM, Dave Crawford <bro at pingtrip.com
>>> <javascript:_e(%7B%7D,'cvml','bro at pingtrip.com');>> wrote:
>>>
>>>> Is it possible that the CPU has two cores and Bro is consuming 100% of
>>>> one core? Some tools average the core utilization to report "CPU usage".
>>>>
>>>> > On Jun 24, 2016, at 7:45 PM, Hashem Alaidaros <aidaros.dev at gmail.com
>>>> <javascript:_e(%7B%7D,'cvml','aidaros.dev at gmail.com');>> wrote:
>>>> >
>>>> > Hi All
>>>> > I use Bro for my PhD research, I add scripts in Bro and then see the
>>>> CPU and packet drop rate, using @load stats.bro. I'm using Bro 2.3 with
>>>> standard libcap.
>>>> > I use tcpreplay from Machine A to replay the pre-captured traffic
>>>> into Bro multi-core machine B through port mirror switch. I replay the
>>>> traffic from 100 to 1000 Mbps , When reach 200 Mbps and onward, packet
>>>> start drop and increases. Surprisingly, the CPU is not fully utilized, CPU
>>>> still 40% usage. What we know is that drop packet resulted from CPU full
>>>> load, but in our case CPU still less than 50%, so  My question, what is the
>>>> cause of this packet drop? Is it normal?
>>>> >
>>>> > Best regards
>>>> > Aidaros
>>>> >
>>>> > _______________________________________________
>>>> > Bro mailing list
>>>> > bro at bro-ids.org <javascript:_e(%7B%7D,'cvml','bro at bro-ids.org');>
>>>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>>
>>>>
>>>
>>>
>>> --
>>> A friend in need Is a friend indeed
>>>
>>
>>
>>
>> --
>> A friend in need Is a friend indeed
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org <javascript:_e(%7B%7D,'cvml','bro at bro-ids.org');>
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160627/48789e7e/attachment.html