[Bro] Tap configuration

Dave Crawford bro at pingtrip.com
Sat Jan 14 11:26:30 PST 2017 

This is what I use in my sensor /etc/network/interfaces config along with a custom “post-up” script. I use Debian for my Bro clusters, so your application will differ. I’m also using af_packet (v4.8.0 kernel) so some of the performance settings may need to be adjusted for your setup. My tuning is aimed at keeping the packets in L3 cache on the CPU vid the NIC hardware, hence the reduced rings. 

auto eth6
iface eth6 inet manual
  up ip link set $IFACE promisc on arp off mtu 1500 up
  down ip link set $IFACE promisc off down
  post-up /opt/tools/post-up_settings.sh $IFACE


And the /opt/tools/post-up_settings.sh script:

#!/bin/bash

IFACE=$1

if [[ -n "$IFACE" ]]; then
    
    # Lower the NIC ring descriptor size
    /sbin/ethtool -G $IFACE rx 512

    # Disable offloading functions
    for i in rx tx sg tso ufo gso gro lro rxhash ntuple txvlan rxvlan; do ethtool -K $IFACE $i off; done
    
    # Enforce a single RX queue
    /sbin/ethtool -L $IFACE combined 1

    # Disable pause frames
    /sbin/ethtool -A $IFACE rx off tx off
    
    # Limit the maximum number of interrupts per second
    /sbin/ethtool -C $IFACE adaptive-rx on rx-usecs 100

    # Disable IPv6
    /bin/echo 1 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6

    # Pin IRQ to local CPU
    /opt/tools/set_irq_affinity local $IFACE
fi

-Dave

> On Jan 13, 2017, at 3:28 PM, Daniel Manzo <daniel.manzo at bayer.com> wrote:
> 
> Thank you for the help. I tried the settings, but I have noticed any difference in packets. The main test that I am doing is that I would open two putty sessions to the server, and have one running capstats on eth12 while my other session was downloading a 1GB file to /dev/null. Last week, I was able to see the packets increase greatly via capstats, but now they stay steady at 7 or 8 packets per second.
> 
> Best regards,
> Dan Manzo
> 
> -----Original Message-----
> From: Seth Hall [mailto:seth at icir.org] 
> Sent: Friday, January 13, 2017 9:29 AM
> To: Daniel Manzo
> Cc: Neslog; Hosom, Stephen M; Bro-IDS
> Subject: Re: [Bro] Tap configuration
> 
> I would recommend leaving checksum validation on in Bro, but disable checksum offloading on the NIC.
> 
> I typically point people to this blog post by Doug Burks (of the SecurityOnion project)...
> 	http://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html
> 
> There is one further thing I would recommend though that we discovered well after this blog post was written.  If you are using an Intel NIC with the ixgbe driver, your nic has a feature called "flow director" that you will want to disable because it will negatively impact your analysis by reordering packets.  It can be disabled like this on linux:
> 	ethtool -L eth12 combined 1
> 
> This will cause your NIC to have only a single hardware queue which will disable the flow director feature and prevent your NIC from reordering packets.  Do that along with the suggestions in the blog post above and things should be better.
> 
>  .Seth
> 
> 
>> On Jan 13, 2017, at 8:58 AM, Daniel Manzo <daniel.manzo at bayer.com> wrote:
>> 
>> I have tried disabling checksum offloading, but still no luck. Here is the ifcfg file for my eth interface:
>> 
>> DEVICE=eth12
>> ONBOOT=yes
>> BOOTPROTO=static
>> PROMISC=yes
>> USERCTL=no
>> 
>> Freundliche Grüße / Best regards,
>> 
>> Dan Manzo
>> Asst Analyst I
>> ________________________
>> 
>> Bayer: Science For A Better Life
>> 
>> Bayer U.S. LLC
>> Country Platform US
>> Scientific Computing Competence Ctr
>> Bayer Road
>> 15205 Pittsburgh (PA), United States
>> Tel:                        +1 412 7772171
>> Mobile:                +1 412 5258332
>> E-mail:  daniel.manzo at bayer.com
>> 
>> From: Neslog [mailto:neslog at gmail.com] 
>> Sent: Thursday, January 12, 2017 4:59 PM
>> To: Hosom, Stephen M
>> Cc: Bro-IDS; Daniel Manzo
>> Subject: Re: [Bro] Tap configuration
>> 
>> I've had success disabling checksum. 
>> ignore_checksums
>> 
>> 
>> On Jan 12, 2017 2:24 PM, "Hosom, Stephen M" <hosom at battelle.org> wrote:
>> Have you looked into checksum offloading? If enabled, it can result in Bro not producing many of the logs you would expect.
>> 
>> From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Daniel Manzo
>> Sent: Thursday, January 12, 2017 11:05 AM
>> To: bro at bro.org
>> Subject: [Bro] Tap configuration
>> 
>> Hi all,
>> 
>> I have Bro 2.4 configured on a RHEL 6.8 server and was wondering how to properly configure the network interfaces so that Bro can see as much of the network traffic as possible. My tap is connected in line with the network, and I believe that I was previously seeing the correct traffic, but now Bro has reporting much less information. I want to make sure that I have the interfaces configured correctly before moving on to troubleshooting other areas. Currently, I have two eth interfaces set up in PROMISC mode. Thank you for the help
>> 
>> Best regards,
>> Dan Manzo
>> 
>> 
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 
> 
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170114/15514309/attachment.html

More information about the Bro mailing list