[Bro] Tap configuration

Zeolla@GMail.com zeolla at gmail.com
Sun Jan 15 20:38:07 PST 2017


So I'm not sure I follow exactly why you'd want to specifically emphasize
keeping packets in the L3 cache.  Is there a specific hardware
configuration where this makes more sense?

As of right now, I do pretty much the same thing you posted earlier except
I map the # of RX queues to the # of physical CPU cores and maximize
the NIC ring descriptor size.

Jon

On Sat, Jan 14, 2017 at 2:35 PM Dave Crawford <bro at pingtrip.com> wrote:

> This is what I use in my sensor /etc/network/interfaces config along with
> a custom “post-up” script. I use Debian for my Bro clusters, so your
> application will differ. I’m also using af_packet (v4.8.0 kernel) so some
> of the performance settings may need to be adjusted for your setup. My
> tuning is aimed at keeping the packets in L3 cache on the CPU vid the NIC
> hardware, hence the reduced rings.
>
> auto eth6
> iface eth6 inet manual
>   up ip link set $IFACE promisc on arp off mtu 1500 up
>   down ip link set $IFACE promisc off down
>   post-up /opt/tools/post-up_settings.sh $IFACE
>
>
> And the /opt/tools/post-up_settings.sh script:
>
> #!/bin/bash
>
> IFACE=$1
>
> if [[ -n "$IFACE" ]]; then
>
>     # Lower the NIC ring descriptor size
>     /sbin/ethtool -G $IFACE rx 512
>
>     # Disable offloading functions
>     for i in rx tx sg tso ufo gso gro lro rxhash ntuple txvlan rxvlan; do
> ethtool -K $IFACE $i off; done
>
>     # Enforce a single RX queue
>     /sbin/ethtool -L $IFACE combined 1
>
>     # Disable pause frames
>     /sbin/ethtool -A $IFACE rx off tx off
>
>     # Limit the maximum number of interrupts per second
>     /sbin/ethtool -C $IFACE adaptive-rx on rx-usecs 100
>
>     # Disable IPv6
>     /bin/echo 1 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6
>
>     # Pin IRQ to local CPU
>     /opt/tools/set_irq_affinity local $IFACE
> fi
>
> -Dave
>
> On Jan 13, 2017, at 3:28 PM, Daniel Manzo <daniel.manzo at bayer.com> wrote:
>
> Thank you for the help. I tried the settings, but I have noticed any
> difference in packets. The main test that I am doing is that I would open
> two putty sessions to the server, and have one running capstats on eth12
> while my other session was downloading a 1GB file to /dev/null. Last week,
> I was able to see the packets increase greatly via capstats, but now they
> stay steady at 7 or 8 packets per second.
>
> Best regards,
> Dan Manzo
>
> -----Original Message-----
> From: Seth Hall [mailto:seth at icir.org <seth at icir.org>]
> Sent: Friday, January 13, 2017 9:29 AM
> To: Daniel Manzo
> Cc: Neslog; Hosom, Stephen M; Bro-IDS
> Subject: Re: [Bro] Tap configuration
>
> I would recommend leaving checksum validation on in Bro, but disable
> checksum offloading on the NIC.
>
> I typically point people to this blog post by Doug Burks (of the
> SecurityOnion project)...
>
> http://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html
>
> There is one further thing I would recommend though that we discovered
> well after this blog post was written.  If you are using an Intel NIC with
> the ixgbe driver, your nic has a feature called "flow director" that you
> will want to disable because it will negatively impact your analysis by
> reordering packets.  It can be disabled like this on linux:
> ethtool -L eth12 combined 1
>
> This will cause your NIC to have only a single hardware queue which will
> disable the flow director feature and prevent your NIC from reordering
> packets.  Do that along with the suggestions in the blog post above and
> things should be better.
>
>  .Seth
>
>
> On Jan 13, 2017, at 8:58 AM, Daniel Manzo <daniel.manzo at bayer.com> wrote:
>
> I have tried disabling checksum offloading, but still no luck. Here is the
> ifcfg file for my eth interface:
>
> DEVICE=eth12
> ONBOOT=yes
> BOOTPROTO=static
> PROMISC=yes
> USERCTL=no
>
> Freundliche Grüße / Best regards,
>
> Dan Manzo
> Asst Analyst I
> ________________________
>
> Bayer: Science For A Better Life
>
> Bayer U.S. LLC
> Country Platform US
> Scientific Computing Competence Ctr
> Bayer Road
> 15205 Pittsburgh (PA), United States
> Tel:                        +1 412 7772171 <(412)%20777-2171>
> Mobile:                +1 412 5258332 <(412)%20525-8332>
> E-mail:  daniel.manzo at bayer.com
>
> From: Neslog [mailto:neslog at gmail.com]
> Sent: Thursday, January 12, 2017 4:59 PM
> To: Hosom, Stephen M
> Cc: Bro-IDS; Daniel Manzo
> Subject: Re: [Bro] Tap configuration
>
> I've had success disabling checksum.
> ignore_checksums
>
>
> On Jan 12, 2017 2:24 PM, "Hosom, Stephen M" <hosom at battelle.org> wrote:
> Have you looked into checksum offloading? If enabled, it can result in Bro
> not producing many of the logs you would expect.
>
> From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of
> Daniel Manzo
> Sent: Thursday, January 12, 2017 11:05 AM
> To: bro at bro.org
> Subject: [Bro] Tap configuration
>
> Hi all,
>
> I have Bro 2.4 configured on a RHEL 6.8 server and was wondering how to
> properly configure the network interfaces so that Bro can see as much of
> the network traffic as possible. My tap is connected in line with the
> network, and I believe that I was previously seeing the correct traffic,
> but now Bro has reporting much less information. I want to make sure that I
> have the interfaces configured correctly before moving on to
> troubleshooting other areas. Currently, I have two eth interfaces set up in
> PROMISC mode. Thank you for the help
>
> Best regards,
> Dan Manzo
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
>
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-- 

Jon

Sent from my mobile device
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170116/94e0b01f/attachment.html 


More information about the Bro mailing list