[Bro] bridge interface vs. bpf bonding (patch?) on FreeBSD 6.1

[Mark Dedlow]

Matt,

We also use netgraph, but don't have the same problems John has.

Here's the netgraph config:

# load module
kldload ng_ether

# bring up the real interfaces
ifconfig em0 promisc -arp up
ifconfig em1 promisc -arp up

# create ngeth0 and bind em1 and em2 to it
ngctl mkpeer . eiface hook ether
ngctl mkpeer ngeth0: one2many lower one
ngctl connect em0: ngeth0:lower lower many0
ngctl connect em1: ngeth0:lower lower many1

# bring up ngeth0
ifconfig ngeth0 -arp up


Mark


John Ives wrote:
> Matt,
> 
> We are running a very similar configuration and are using netgraph to
> bond the two interfaces into one virtual interface  which we monitor
> (again similar to your method) and it has been working fairly well for
> us.  My understanding is that the kernel patch is no longer necessary
> because netgraph is already in the source code, it just needs to be
> compiled in by adding "options NETGRAPH" to the kernel config file and
> then running a script during startup that creates the virtual
> interface.  The one problem I have seen with two of our systems is that
> the interface periodically goes deaf and doesn't come back unless with
> ifconfig down and up all of the interfaces involved (so I wrote a script
> that tests the interface every few minutes and restarts it and notifies
> me if there is no traffic).  This only seems to happen on two or the 5
> boxes I use this on (not the bro box), and I suspect it is partially a
> function of something else I may be running (or is based upon load).
> 
> John
> 
> Matt Cuttler wrote:
>> Ennobled bro users and developers,
>>
>> I'm looking for some clarification on the use of bro and multiple
>> interfaces.
>>
>> FreeBSD 6.1 machine with two em* (Intel 1000 fibre) interfaces. Each
>> interface's RX port is connected to one of the two TX ports on a
>> regenerative tap.
>>
>> Bro.cfg was originally configured as:
>> BRO_CAPTURE_INTERFACE="em0 em1"
>>
>> Additionally, we tried enabling and disabling:
>> BRO_BPFBOND_ENABLE="YES"
>> and
>> BRO_BPFBOND_FLAGS="em0 em1"
>>
>> In all cases above, we got indications that this configuration was not
>> correct, and that bro might not be getting all of the traffic across
>> both interfaces properly (see example #1 below, with content gaps in the
>> smtp log).
>>
>> We then set up a bond interface:
>> ifconfig bridge0 create
>> ifconfig bridge0 addm em0 addm em1 up
>> ..and changed our bro.cfg to:
>> BRO_CAPTURE_INTERFACE="bond0"
>> BRO_BPFBOND_ENABLE="NO"
>>
>> This seems to work properly now; at least we no longer get content gaps
>> logged to the smtp log (see example #2 below).
>>
>> My questions are: Is this (bridge device method) the "right" way to
>> handle multiple interfaces for my hardware/software? The documentation
>> mentions kernel patches to enable bpf bonding on FreeBSD 4.1. Is this
>> not necessary on later FreeBSD releases?
>>
>> Thanks,
>> Matt Cuttler
>>
>> ===
>> example #1, using em0 and em1:
>> 1.2.3.4/1880 > 5.6.7.8/smtp start internal
>> 1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: content gap: \
>>   seq = 30, len = 33
>> 1.2.3.4/1880 < 5.6.7.8/smtp: unusual command/reply: \
>>   (UNKNOWN)() --> 250(OK)
>> 1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: unexpected \
>>   command: RCPT reply = 0 state = 12
>> 1.2.3.4/1880 < 5.6.7.8/smtp: unexpected: content gap: \
>>   seq = 139, len = 14
>> 1.2.3.4/1880 < 5.6.7.8/smtp: unexpected: content gap: \
>>   seq = 153, len = 14
>> 1.2.3.4/1880 < 5.6.7.8/smtp: unusual command/reply: \
>>   (UNKNOWN)() --> 250(Accepted)
>> 1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: unexpected \
>>   command: DATA reply = 0 state = 12
>> 1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: content gap: \
>>   seq = 149, len = 1460
>> 1.2.3.4/1880 > 5.6.7.8/smtp: unexpected: content gap: \
>>   seq = 1609, len = 1697
>> 1.2.3.4/1880 < 5.6.7.8/smtp: unexpected: content gap: \
>>   seq = 237, len = 28
>> 1.2.3.4/1880 < 5.6.7.8/smtp: unusual command/reply: \
>>   (UNKNOWN)() --> 221(mail.host.net closing connection)
>> finish
>> ===
>>
>> ===
>> Example #2, using bond0:
>>
>> 1.2.3.4/19100 > 5.6.7.8/smtp start external
>> recipient: <user at email.address>
>> finish
>>
>> ===
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>>
>>
>>   
> 
>