From goeldich at ee.ethz.ch  Mon May  2 06:55:18 2005
From: goeldich at ee.ethz.ch (Christoph Goeldi)
Date: Mon, 02 May 2005 15:55:18 +0200
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
Message-ID: <42763146.1040103@ee.ethz.ch>

Hi


I'm quite happy with Bro. Thanks to the developers!

Nevertheless, i remarked another problem today when i tried to make Bro 
listening on two interfaces.

What i did:
bro -i eth0 -i eth3 test

*** test.bro ********************************************
event connection_attempt(c: connection)
{ print "connection_attempt"; }

event connection_rejected(c: connection)
{ print "connection_rejected"; }
*********************************************************
(I think it doesn't depend on what this script does.)

I runned attacks on only one and on both interfaces on this host.

What happens:
Most of the packets are dropped by Bro:
> 8970 packets received on interface eth0, 31021 dropped
> 1580 packets received on interface eth3, 10770 dropped

I tried this on two completely different Linux'. Kernel 2.4 and 2.6.
And with the newest Bro release.

When i start Bro and set only one -i flag, it works perfect.
"tcpdump -i any" works also and doesn't drop any packets.

any ideas?


Thanx for any help
Christoph


From bchen at cs.ucf.edu  Tue May  3 19:29:24 2005
From: bchen at cs.ucf.edu (bchen at cs.ucf.edu)
Date: Tue,  3 May 2005 22:29:24 -0400
Subject: [Bro] type conversion
In-Reply-To: <200505021901.j42J1L2L019420@fruitcake.ICSI.Berkeley.EDU>
References: <200505021901.j42J1L2L019420@fruitcake.ICSI.Berkeley.EDU>
Message-ID: <20050503222924.eq1pd48i2o8gs48o@mail.cs.ucf.edu>

Hello everyone,
      I have a simple question: how to convert a string to a port? I 
extract the
port information from a string and need to assign it to a port variable. Any
suggestion will be appreciated.

Bing



From vern at icir.org  Tue May  3 19:41:23 2005
From: vern at icir.org (Vern Paxson)
Date: Tue, 03 May 2005 19:41:23 -0700
Subject: [Bro] type conversion
In-Reply-To: Your message of Tue, 03 May 2005 22:29:24 EDT.
Message-ID: <200505040241.j442fN7I038033@jaguar.icir.org>

>       I have a simple question: how to convert a string to a port? I 
> extract the
> port information from a string and need to assign it to a port variable. Any
> suggestion will be appreciated.

There's no support for that.  You'll need to write a built-in function
(see src/bro.bif).

What's the context?  Knowing that would help me in thinking about various
needs for type conversion in the Bro language.

		Vern


From bchen at cs.ucf.edu  Wed May  4 13:20:13 2005
From: bchen at cs.ucf.edu (bchen at cs.ucf.edu)
Date: Wed,  4 May 2005 16:20:13 -0400
Subject: [Bro] type conversion
In-Reply-To: <200505040241.j442fN7I038033@jaguar.icir.org>
References: <200505040241.j442fN7I038033@jaguar.icir.org>
Message-ID: <20050504162013.ddycbjdy8kwogo00@mail.cs.ucf.edu>

Hi Vern,
   I am monitoring the pm_getport event. If a suspicious remote host sends a
reqest to the monitored server and successfully get the port # of a specific
rpc service, I would like to track all incoming traffic to this 
service. I need
the port # of the service for this purpose. I can parse the "addl" 
parameter of
the portmapper request event handler and pull out the port number. I 
hope I can
store this port number into a port variable and this is why I need to 
convert a
string to a port value. If I can't do so, are there other ways to get the port
number? Thank you for your time.

Bing


Quoting Vern Paxson <vern at icir.org>:

>>       I have a simple question: how to convert a string to a port? I
>> extract the
>> port information from a string and need to assign it to a port variable. Any
>> suggestion will be appreciated.
>
> There's no support for that.  You'll need to write a built-in function
> (see src/bro.bif).
>
> What's the context?  Knowing that would help me in thinking about various
> needs for type conversion in the Bro language.
>
> 		Vern
>




From vern at icir.org  Wed May  4 13:31:43 2005
From: vern at icir.org (Vern Paxson)
Date: Wed, 04 May 2005 13:31:43 -0700
Subject: [Bro] type conversion
In-Reply-To: Your message of Wed, 04 May 2005 16:20:13 EDT.
Message-ID: <200505042031.j44KVhTZ017784@jaguar.icir.org>

> Hi Vern,
>    I am monitoring the pm_getport event. If a suspicious remote host sends a
> reqest to the monitored server and successfully get the port # of a specific
> rpc service, I would like to track all incoming traffic to this 
> service. I need
> the port # of the service for this purpose.

The way to get it is to define your own pm_request_getport event handler
(you can do this in addition to the normal one).  See portmapper.bro
for how the default one works, from which you should be able to derive
an additional handler to do what you want.

		Vern


From bchen at cs.ucf.edu  Wed May  4 17:54:11 2005
From: bchen at cs.ucf.edu (bchen at cs.ucf.edu)
Date: Wed,  4 May 2005 20:54:11 -0400
Subject: [Bro] type conversion
In-Reply-To: <200505042031.j44KVhTZ017784@jaguar.icir.org>
References: <200505042031.j44KVhTZ017784@jaguar.icir.org>
Message-ID: <20050504205411.lgji5naksg48sw8s@mail.cs.ucf.edu>

I got it. thanks.

Bing



Quoting Vern Paxson <vern at icir.org>:

>> Hi Vern,
>>    I am monitoring the pm_getport event. If a suspicious remote host sends a
>> reqest to the monitored server and successfully get the port # of a specific
>> rpc service, I would like to track all incoming traffic to this
>> service. I need
>> the port # of the service for this purpose.
>
> The way to get it is to define your own pm_request_getport event handler
> (you can do this in addition to the normal one).  See portmapper.bro
> for how the default one works, from which you should be able to derive
> an additional handler to do what you want.
>
> 		Vern
>




From christian at whoop.org  Fri May  6 15:13:31 2005
From: christian at whoop.org (Christian Kreibich)
Date: Fri, 06 May 2005 15:13:31 -0700
Subject: [Bro] Broccoli 0.7.1 released
Message-ID: <1115417611.3190.90.camel@localhost>

Hi all,

Broccoli 0.7.1 is now available in tar.gz and RPM formats from

  http://www.cl.cam.ac.uk/~cpk25/broccoli/index.html

This release fixes portability issues I overlooked in 0.7 and has
updated documentation. Details are in the ChangeLog, as usual:

  http://www.cl.cam.ac.uk/~cpk25/broccoli/ChangeLog

Thanks,
Christian.
-- 
________________________________________________________________________
                                          http://www.cl.cam.ac.uk/~cpk25
                                                    http://www.whoop.org




From bchen at cs.ucf.edu  Sat May  7 17:38:34 2005
From: bchen at cs.ucf.edu (bchen at cs.ucf.edu)
Date: Sat,  7 May 2005 20:38:34 -0400
Subject: [Bro] udp_request and icmp_sent event handlers
In-Reply-To: <200505071901.j47J112L000137@fruitcake.ICSI.Berkeley.EDU>
References: <200505071901.j47J112L000137@fruitcake.ICSI.Berkeley.EDU>
Message-ID: <20050507203834.uxqb5ngqoe8wo0wo@mail.cs.ucf.edu>

Hello Everyone,
      I met a problem that event handlers udp_request, udp_reply, icmp_sent
doesn't work (i.e., they are not processed) when Bro read tcpdump data that
consist of udp and icmp packets. I did load udp and icmp analyzers. I noticed
that icmp_request and icmp_reply event handlers worked. Has anyone met 
the same
problem?

thanks

Bing



From vern at icir.org  Sat May  7 17:51:38 2005
From: vern at icir.org (Vern Paxson)
Date: Sat, 07 May 2005 17:51:38 -0700
Subject: [Bro] udp_request and icmp_sent event handlers
In-Reply-To: Your message of Sat, 07 May 2005 20:38:34 EDT.
Message-ID: <200505080051.j480pcwM085800@jaguar.icir.org>

>       I met a problem that event handlers udp_request, udp_reply, icmp_sent
> doesn't work (i.e., they are not processed) when Bro read tcpdump data that
> consist of udp and icmp packets. I did load udp and icmp analyzers. I noticed
> that icmp_request and icmp_reply event handlers worked. Has anyone met 
> the same
> problem?

udp.bro doesn't turn on a capture-all-UDP filter.  So try adding

	redef capture_filters += { ["all udp"] = "udp" };

- Vern


From bchen at cs.ucf.edu  Sun May  8 20:12:56 2005
From: bchen at cs.ucf.edu (bchen at cs.ucf.edu)
Date: Sun,  8 May 2005 23:12:56 -0400
Subject: [Bro] udp_request and icmp_sent event handlers
In-Reply-To: <200505080051.j480pcwM085800@jaguar.icir.org>
References: <200505080051.j480pcwM085800@jaguar.icir.org>
Message-ID: <20050508231256.mvk3folgw84so4s0@mail.cs.ucf.edu>

Hi Vern,
    Thank you for your help. Yes, it works after I redef the capture_filters.
How about the icmp_sent event handler? Should the icmp_sent event handler see
all kinds of icmp packets?

Bing

>
> udp.bro doesn't turn on a capture-all-UDP filter.  So try adding
>
> 	redef capture_filters += { ["all udp"] = "udp" };
>
> - Vern
>




From vern at icir.org  Sun May  8 22:39:21 2005
From: vern at icir.org (Vern Paxson)
Date: Sun, 08 May 2005 22:39:21 -0700
Subject: [Bro] udp_request and icmp_sent event handlers
In-Reply-To: Your message of Sun, 08 May 2005 23:12:56 EDT.
Message-ID: <200505090539.j495dLZW050087@jaguar.icir.org>

>     Thank you for your help. Yes, it works after I redef the capture_filters.

Great, glad to hear it.

> How about the icmp_sent event handler? Should the icmp_sent event handler see
> all kinds of icmp packets?

If you look in icmp.bro you'll see that it already sets a capture_filters
entry, so yes, it should see all kinds of ICMP packets (at least, those
for which the event engine generates events).

		Vern


From bchen at cs.ucf.edu  Mon May  9 14:18:40 2005
From: bchen at cs.ucf.edu (bchen at cs.ucf.edu)
Date: Mon,  9 May 2005 17:18:40 -0400
Subject: [Bro] udp_request and icmp_sent event handlers
In-Reply-To: <200505090539.j495dLZW050087@jaguar.icir.org>
References: <200505090539.j495dLZW050087@jaguar.icir.org>
Message-ID: <20050509171840.gpuugwyisgws0s4c@mail.cs.ucf.edu>

Hi Vern,
   I can see that icmp_echo_request, icmp_reply, flow_summary and
icmp_unreachable event handlers work. But icmp_sent doesn't work. I just put
print statement in each of these handlers, only icmp_sent didn't print
anything. Any idea about this problem? thanks

Bing



>> How about the icmp_sent event handler? Should the icmp_sent event 
>> handler see
>> all kinds of icmp packets?
>
> If you look in icmp.bro you'll see that it already sets a capture_filters
> entry, so yes, it should see all kinds of ICMP packets (at least, those
> for which the event engine generates events).
>
> 		Vern
>




From goeldich at ee.ethz.ch  Tue May 10 03:50:15 2005
From: goeldich at ee.ethz.ch (Christoph Goeldi)
Date: Tue, 10 May 2005 12:50:15 +0200
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
Message-ID: <428091E7.5080405@ee.ethz.ch>

hi

i'm still working on my problem described before.
and i have still no idea where i have to search.

i looked at the c-code. i runned it on different machines and
on various interfaces. bro still drops most of the packets
when i force it to listen on two interfaces.

is it a libpcap problem?
a bro problem?
a linux problem?

do you need more information from me?

please help! any help is welcome.

thanx
christoph


From hhoffman at ip-solutions.net  Tue May 10 05:41:36 2005
From: hhoffman at ip-solutions.net (Harry Hoffman)
Date: Tue, 10 May 2005 08:41:36 -0400
Subject: [Bro] bro rpms
Message-ID: <4280AC00.8040608@ip-solutions.net>

Hi,

I wanted to ask if anyone has created rpms of bro before I attempt to do so?


Thanks,
Harry


From vern at icir.org  Tue May 10 07:38:45 2005
From: vern at icir.org (Vern Paxson)
Date: Tue, 10 May 2005 07:38:45 -0700
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: Your message of Tue, 10 May 2005 12:50:15 +0200.
Message-ID: <200505101438.j4AEck4t069243@jaguar.icir.org>

> i looked at the c-code. i runned it on different machines and
> on various interfaces. bro still drops most of the packets
> when i force it to listen on two interfaces.
> 
> is it a libpcap problem?
> a bro problem?
> a linux problem?

I believe it's a Linux problem.  We do this under FreeBSD in two different
ways, either merging the interfaces in the kernel into one logical interface
(via a custom patch), or at user level.  While the in-kernel version
performs better, the user-level one isn't a disaster like you describe.

I also recall hearing others mention that multiple interfaces under Linux
do not work well in general.  I don't use Linux, though, so can't comment
more directly.

		Vern


From tbrooks at ncsa.uiuc.edu  Tue May 10 08:01:29 2005
From: tbrooks at ncsa.uiuc.edu (Tim Brooks)
Date: Tue, 10 May 2005 10:01:29 -0500
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <200505101438.j4AEck4t069243@jaguar.icir.org>
References: <200505101438.j4AEck4t069243@jaguar.icir.org>
Message-ID: <4280CCC9.8040209@ncsa.uiuc.edu>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ours works fine on linux with the interfaces set in etc/bro.cfg like:

BRO_CAPTURE_INTERFACE="eth2 eth3"

Tim Brooks

Vern Paxson wrote:

>>i looked at the c-code. i runned it on different machines and
>>on various interfaces. bro still drops most of the packets
>>when i force it to listen on two interfaces.
>>
>>is it a libpcap problem?
>>a bro problem?
>>a linux problem?
>
>
>I believe it's a Linux problem. We do this under FreeBSD in two different
>ways, either merging the interfaces in the kernel into one logical interface
>(via a custom patch), or at user level. While the in-kernel version
>performs better, the user-level one isn't a disaster like you describe.
>
>I also recall hearing others mention that multiple interfaces under Linux
>do not work well in general. I don't use Linux, though, so can't comment
>more directly.
>
> Vern
>_______________________________________________
>Bro mailing list
>bro at bro-ids.org
>http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


- --
Tim Brooks
Security Engineer

National Center for Supercomputing Applications
605 East Springfield Avenue   Champaign, IL 61820


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCgMzJZsl1Kn8xw9YRAo5SAJwOnyv8SRrZd6M0j+ulW4XG1zcHXwCgwMF2
RolyqC5DpSo+7aCct02Oqms=
=in3B
-----END PGP SIGNATURE-----



From goeldich at ee.ethz.ch  Tue May 10 08:06:39 2005
From: goeldich at ee.ethz.ch (=?ISO-8859-1?Q?Christoph_G=F6ldi?=)
Date: Tue, 10 May 2005 17:06:39 +0200
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <200505101438.j4AEck4t069243@jaguar.icir.org>
References: <200505101438.j4AEck4t069243@jaguar.icir.org>
Message-ID: <D3DC33E195D30CA0A8C9DC7F@[192.168.0.2]>

hi vern

thanks for your statement.

>> i looked at the c-code. i runned it on different machines and
>> on various interfaces. bro still drops most of the packets
>> when i force it to listen on two interfaces.
>> 
>> is it a libpcap problem?
>> a bro problem?
>> a linux problem?
> 
> I believe it's a Linux problem.  We do this under FreeBSD in two different
> ways, either merging the interfaces in the kernel into one logical
> interface (via a custom patch), or at user level.  While the in-kernel
> version performs better, the user-level one isn't a disaster like you
> describe.

i also thought about bonding the interfaces and maybe it won't be a problem.
but we plan to introduce Luca Deri's PF_RING kernel patch in the future and
then interface bonding won't be possible any more.

> I also recall hearing others mention that multiple interfaces under Linux
> do not work well in general.  I don't use Linux, though, so can't comment
> more directly.

do you know other linux software which allows to listen on two interfaces?
so that i can use it for testing?
tcpdump only allows to listen on one interface or on all (any).

thanx
christoph


From sommer at in.tum.de  Tue May 10 08:47:00 2005
From: sommer at in.tum.de (Robin Sommer)
Date: Tue, 10 May 2005 17:47:00 +0200
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <428091E7.5080405@ee.ethz.ch>
References: <428091E7.5080405@ee.ethz.ch>
Message-ID: <20050510154700.GA1464@net.informatik.tu-muenchen.de>

On Tue, May 10, 2005 at 12:50 +0200, Christoph Goeldi wrote:

> on various interfaces. bro still drops most of the packets
> when i force it to listen on two interfaces.

I haven't used multiple interfaces myself yet, but in general
capturing seems to be more problematic on Linux than on FreeBSD.
With a stock Linux kernel, the performance is actually rather bad.
It works much better with Phil Wood's mmap pcap[1]. We've also
experimented with Luca Deri's RING patch, but there were some
strange problems (like not filtering correctly), and it doesn't
support SMP systems. 

That said, could you try configuring Bro with --enable-selectloop
and see if that changes anything?

Robin

[1] Although rather recently I have encountered some weird
timestamps when using it...

-- 
Robin Sommer * Room        01.08.055 * www.net.in.tum.de
TU Muenchen  * Phone (089) 289-18006 *  sommer at in.tum.de 


From hhoffman at ip-solutions.net  Tue May 10 10:39:12 2005
From: hhoffman at ip-solutions.net (Harry Hoffman)
Date: Tue, 10 May 2005 13:39:12 -0400
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <D3DC33E195D30CA0A8C9DC7F@[192.168.0.2]>
References: <200505101438.j4AEck4t069243@jaguar.icir.org> <D3DC33E195D30CA0A8C9DC7F@[192.168.0.2]>
Message-ID: <4280F1C0.5030602@ip-solutions.net>

Christoph,

If the interfaces don't have an IP address could you bridge them?

brctl addbr br0
brctl addif br0 eth0
brctl addif br0 eth1

--Harry

Christoph G?ldi wrote:
> hi vern
...
>>
>>I believe it's a Linux problem.  We do this under FreeBSD in two different
>>ways, either merging the interfaces in the kernel into one logical
>>interface (via a custom patch), or at user level.  While the in-kernel
>>version performs better, the user-level one isn't a disaster like you
>>describe.


From christian at whoop.org  Tue May 10 18:00:25 2005
From: christian at whoop.org (Christian Kreibich)
Date: Tue, 10 May 2005 18:00:25 -0700
Subject: [Bro] bro rpms
In-Reply-To: <4280AC00.8040608@ip-solutions.net>
References: <4280AC00.8040608@ip-solutions.net>
Message-ID: <1115773226.21570.151.camel@localhost>

Hi Harry,

On Tue, 2005-05-10 at 08:41 -0400, Harry Hoffman wrote:
> Hi,
> 
> I wanted to ask if anyone has created rpms of bro before I attempt to do so?

no, no-one has created any so far and I think Roger Winslow and I were
sort of shortlisted to do this but I'd *love* to see someone else taking
care of this! Bro should normally pass make distcheck, so I'd assume
creating a spec file shouldn't take much time at all. Thanks for
offering, and feel free to post your spec file here once you get it to
work!

Thanks,
Christian.
-- 
________________________________________________________________________
                                          http://www.cl.cam.ac.uk/~cpk25
                                                    http://www.whoop.org




From bchen at cs.ucf.edu  Wed May 11 19:06:39 2005
From: bchen at cs.ucf.edu (bchen at cs.ucf.edu)
Date: Wed, 11 May 2005 22:06:39 -0400
Subject: [Bro] detect Ack flooding attack
In-Reply-To: <200505111903.j4BJ34gY011604@fruitcake.ICSI.Berkeley.EDU>
References: <200505111903.j4BJ34gY011604@fruitcake.ICSI.Berkeley.EDU>
Message-ID: <20050511220639.1bb74oqmocw84cg0@mail.cs.ucf.edu>

Hello everyone,
      I am trying to use Bro to detect a DOS attack in a tcpdump file. This
attack uses Ack packets with spoofed random source IPs and random destination
ports to flood a remote server. I thought weird analyzer should catch this
attack. I searched all log files generated by Bro and found Bro didn't capture
any of these packets.
      I have thought that Bro might drop these packets because there are no SYN
packets seen by Bro, so I run the following command:

./bro -f "(tcp and ((tcp[13] & 0x7 != 0) or (tcp[13] & 0x10 == 1)) ) or udp or
icmp" -r dos.dump mt

It unfortunately didn't work. Does anyone have any suggestion?

Thanks

Bing



From bchen at cs.ucf.edu  Thu May 12 15:59:23 2005
From: bchen at cs.ucf.edu (bchen at cs.ucf.edu)
Date: Thu, 12 May 2005 18:59:23 -0400
Subject: Fwd: Re: [Bro] detect Ack flooding attack
Message-ID: <20050512185923.2y2uzsxov4gs8ssc@mail.cs.ucf.edu>

Hi Scott,
   Thank you for your reply. Bro didn't work without filter expression. this is
why I tried to use this filter expression. conn and weird analyzers did
generated some log records, but none of them captured the packets
having random source IP addresses. Weird analyzer recorded some "weird"
connections having good syn, ack and fin flags.
     This traffic data is actually the DAPAR 2000 data set of MIT LL. I believe
it was captured from a real Lan. I doubt that Bro, in default setting, doesn't
record connection information if it did see SYN packets. The problem is how to
change this default setting.

thanks again

Bing


> bchen at cs.ucf.edu wrote:
>> Hello everyone,
>>      I am trying to use Bro to detect a DOS attack in a tcpdump file. This
>> attack uses Ack packets with spoofed random source IPs and random
>> destination
>> ports to flood a remote server. I thought weird analyzer should catch this
>> attack. I searched all log files generated by Bro and found Bro didn't
>> capture
>> any of these packets.
>>      I have thought that Bro might drop these packets because there are
>> no SYN
>> packets seen by Bro, so I run the following command:
>>
>> ./bro -f "(tcp and ((tcp[13] & 0x7 != 0) or (tcp[13] & 0x10 == 1)) ) or
>> udp or
>> icmp" -r dos.dump mt
>>
>> It unfortunately didn't work. Does anyone have any suggestion?
>>
>> Thanks
>>
>> Bing
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> Few questions:
>
> Does it work without the filter expression?
> Are you seeing any connection information from the trace file at all?
> Is the traffic in a vlan?
>
> You are correct that there should be something in the weird records.
> This is a little odd.
>
> scott
>




----- End forwarded message -----


From a.greenhalgh at cs.ucl.ac.uk  Sun May 15 09:57:30 2005
From: a.greenhalgh at cs.ucl.ac.uk (Adam Greenhalgh)
Date: Sun, 15 May 2005 17:57:30 +0100
Subject: [Bro] is snort2bro missing from 0.9-current
Message-ID: <42877F7A.60701@cs.ucl.ac.uk>

Hi

I've been looking to include snort rules into bro, however in the
0.9-current release there doesn't seem to be the snort2bro convertor ?
Is it missing or is it no longer needed ?

Many thanks

Adam


From goeldich at ee.ethz.ch  Tue May 17 04:08:23 2005
From: goeldich at ee.ethz.ch (Christoph Goeldi)
Date: Tue, 17 May 2005 12:08:23 +0100
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <20050510154700.GA1464@net.informatik.tu-muenchen.de>
References: <428091E7.5080405@ee.ethz.ch>
	<20050510154700.GA1464@net.informatik.tu-muenchen.de>
Message-ID: <20050517120823.gwqsct5kkkwcsgsg@email.ee.ethz.ch>

Hi Robin

Zitat von Robin Sommer <sommer at in.tum.de>:
>
> On Tue, May 10, 2005 at 12:50 +0200, Christoph Goeldi wrote:
>
>> on various interfaces. bro still drops most of the packets
>> when i force it to listen on two interfaces.
>
> I haven't used multiple interfaces myself yet, but in general
> capturing seems to be more problematic on Linux than on FreeBSD.
> With a stock Linux kernel, the performance is actually rather bad.
> It works much better with Phil Wood's mmap pcap[1]. We've also
> experimented with Luca Deri's RING patch, but there were some
> strange problems (like not filtering correctly), and it doesn't
> support SMP systems.

Thank you for the tip, we are already evaluating Phil Wood's mmap pcap.

> That said, could you try configuring Bro with --enable-selectloop
> and see if that changes anything?

I tried the --enable-selectloop option and it doesn't seem to change
anything. Listening on two interfaces still doesn't work... :-(


Thank you anyway
Christoph





From goeldich at ee.ethz.ch  Tue May 17 04:13:16 2005
From: goeldich at ee.ethz.ch (Christoph Goeldi)
Date: Tue, 17 May 2005 12:13:16 +0100
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <4280CCC9.8040209@ncsa.uiuc.edu>
References: <200505101438.j4AEck4t069243@jaguar.icir.org>
	<4280CCC9.8040209@ncsa.uiuc.edu>
Message-ID: <20050517121316.2hqs51l6ckwo4ko0@email.ee.ethz.ch>

Hi Tim

Zitat von Tim Brooks <tbrooks at ncsa.uiuc.edu>:
>
> Ours works fine on linux with the interfaces set in etc/bro.cfg like:
>
> BRO_CAPTURE_INTERFACE="eth2 eth3"

Are you realy sure, that Bro doesn't drop the most of the captured packets?

I like to know what Linux version (distro), what Bro version and what
interfaces (100Mbit or 1Gbit / manufacturer) do you use?

Thank you for your time
Christoph



From tbrooks at ncsa.uiuc.edu  Tue May 17 12:11:14 2005
From: tbrooks at ncsa.uiuc.edu (Tim Brooks)
Date: Tue, 17 May 2005 14:11:14 -0500
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <20050517121316.2hqs51l6ckwo4ko0@email.ee.ethz.ch>
References: <200505101438.j4AEck4t069243@jaguar.icir.org>	<4280CCC9.8040209@ncsa.uiuc.edu> <20050517121316.2hqs51l6ckwo4ko0@email.ee.ethz.ch>
Message-ID: <428A41D2.6090002@ncsa.uiuc.edu>

Christoph,

I'll let my co-worker, Aashish Sharma, reply to the specific issue of
bro dropping packets.  However, there are two corrections we made to
correct for dropped packets and errors that we were receiving on our
1GB fiber interfaces after first installing bro and turning it on.

First, we set the MTU from 1500 to 9000.
Second, we set LowLatency=On (i.e. modprobe sk98lin LowLatency=On)

That second fix is specific to the fiber cards we are using.  After
these two changes, we are no longer receiving errors on the interfaces.

Aashish Sharma will follow up with bro specific packet loss answer.

Thanks,

Tim

Christoph Goeldi wrote:

> Hi Tim
>
> Zitat von Tim Brooks <tbrooks at ncsa.uiuc.edu>:
>
>>
>> Ours works fine on linux with the interfaces set in etc/bro.cfg like:
>>
>> BRO_CAPTURE_INTERFACE="eth2 eth3"
>
>
> Are you realy sure, that Bro doesn't drop the most of the captured
> packets?
>
> I like to know what Linux version (distro), what Bro version and what
> interfaces (100Mbit or 1Gbit / manufacturer) do you use?
>
> Thank you for your time
> Christoph


-- 
Tim Brooks
Security Engineer

National Center for Supercomputing Applications
605 East Springfield Avenue   Champaign, IL 61820




From vern at icir.org  Tue May 17 23:38:22 2005
From: vern at icir.org (Vern Paxson)
Date: Tue, 17 May 2005 23:38:22 -0700
Subject: [Bro] udp_request and icmp_sent event handlers
In-Reply-To: Your message of Mon, 09 May 2005 17:18:40 EDT.
Message-ID: <200505180638.j4I6cMbv087889@jaguar.icir.org>

>    I can see that icmp_echo_request, icmp_reply, flow_summary and
> icmp_unreachable event handlers work. But icmp_sent doesn't work. I just put
> print statement in each of these handlers, only icmp_sent didn't print
> anything. Any idea about this problem? thanks

icmp_sent is a generic event, only generated for ICMPs for which there's
not a more specific analyzer.  So you won't see it for echo request/reply,
time exceeded, or unreachables.

		Vern


From vern at icir.org  Tue May 17 23:43:46 2005
From: vern at icir.org (Vern Paxson)
Date: Tue, 17 May 2005 23:43:46 -0700
Subject: [Bro] detect Ack flooding attack
In-Reply-To: Your message of Wed, 11 May 2005 22:06:39 EDT.
Message-ID: <200505180643.j4I6hkqQ087985@jaguar.icir.org>

> ./bro -f "(tcp and ((tcp[13] & 0x7 != 0) or (tcp[13] & 0x10 == 1)) ) or udp or

The second test you give, for capturing ACK packets, is incorrect.
It needs to be

	(tcp[13] & 0x10 == 0x10)

- Vern


From vern at icir.org  Tue May 17 23:48:44 2005
From: vern at icir.org (Vern Paxson)
Date: Tue, 17 May 2005 23:48:44 -0700
Subject: [Bro] is snort2bro missing from 0.9-current
In-Reply-To: Your message of Sun, 15 May 2005 17:57:30 BST.
Message-ID: <200505180648.j4I6miIt088187@jaguar.icir.org>

> I've been looking to include snort rules into bro, however in the
> 0.9-current release there doesn't seem to be the snort2bro convertor ?
> Is it missing or is it no longer needed ?

Oops, sorry about that - it's missing.  I'll see about getting this fixed
for the next release.

		Vern


From bchen at cs.ucf.edu  Wed May 18 11:01:43 2005
From: bchen at cs.ucf.edu (bchen at cs.ucf.edu)
Date: Wed, 18 May 2005 14:01:43 -0400
Subject: [Bro] detect Ack flooding attack
In-Reply-To: <200505180643.j4I6hkqQ087985@jaguar.icir.org>
References: <200505180643.j4I6hkqQ087985@jaguar.icir.org>
Message-ID: <20050518140143.zopggq9jwgg0k0s0@mail.cs.ucf.edu>

Hi Vern,
     Thank you for your reply. I corrected this filter expression and run Bro,
but I got the same result. I can see these spoofed source IP packets with
Ethereal. All of them target the same host but with different destination
ports. The TCP flag of these packets is 0x0010 (ack). I found no single log
record was for such packets. Am I missing anything?
     By the way, I am using the DARPA 2000 data set (Scenario one, inside
tcpdump file). This is the link for this data:
http://www.ll.mit.edu/IST/ideval/data/2000/LLS_DDOS_1.0.html


thank you for your time.

Bing



Quoting Vern Paxson <vern at icir.org>:

>> ./bro -f "(tcp and ((tcp[13] & 0x7 != 0) or (tcp[13] & 0x10 == 1)) ) 
>> or udp or
>
> The second test you give, for capturing ACK packets, is incorrect.
> It needs to be
>
> 	(tcp[13] & 0x10 == 0x10)
>
> - Vern
>




From vern at icir.org  Wed May 18 11:09:23 2005
From: vern at icir.org (Vern Paxson)
Date: Wed, 18 May 2005 11:09:23 -0700
Subject: [Bro] detect Ack flooding attack
In-Reply-To: Your message of Wed, 18 May 2005 14:01:43 EDT.
Message-ID: <200505181809.j4II9NBG044438@jaguar.icir.org>

>      Thank you for your reply. I corrected this filter expression and run Bro,
> but I got the same result. I can see these spoofed source IP packets with
> Ethereal. All of them target the same host but with different destination
> ports. The TCP flag of these packets is 0x0010 (ack). I found no single log
> record was for such packets. Am I missing anything?
>      By the way, I am using the DARPA 2000 data set (Scenario one, inside
> tcpdump file). This is the link for this data:
> http://www.ll.mit.edu/IST/ideval/data/2000/LLS_DDOS_1.0.html

Please send a small trace that can be used to reproduce the problem.
Thanks.

		Vern


From bchen at cs.ucf.edu  Wed May 18 12:06:21 2005
From: bchen at cs.ucf.edu (bchen at cs.ucf.edu)
Date: Wed, 18 May 2005 15:06:21 -0400
Subject: [Bro] detect Ack flooding attack
In-Reply-To: <200505181809.j4II9NBG044438@jaguar.icir.org>
References: <200505181809.j4II9NBG044438@jaguar.icir.org>
Message-ID: <20050518150621.cmcahra9uogkgwsg@mail.cs.ucf.edu>

Hi Vern,
   The attachment is a small trace file. thanks

Bing


Quoting Vern Paxson <vern at icir.org>:

>>      Thank you for your reply. I corrected this filter expression 
>> and run Bro,
>> but I got the same result. I can see these spoofed source IP packets with
>> Ethereal. All of them target the same host but with different destination
>> ports. The TCP flag of these packets is 0x0010 (ack). I found no single log
>> record was for such packets. Am I missing anything?
>>      By the way, I am using the DARPA 2000 data set (Scenario one, inside
>> tcpdump file). This is the link for this data:
>> http://www.ll.mit.edu/IST/ideval/data/2000/LLS_DDOS_1.0.html
>
> Please send a small trace that can be used to reproduce the problem.
> Thanks.
>
> 		Vern
>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smalltrace
Type: application/octet-stream
Size: 7574 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20050518/78b6b5d6/attachment.obj 

From aashish at uiuc.edu  Thu May 19 07:21:52 2005
From: aashish at uiuc.edu (Aashish Sharma)
Date: Thu, 19 May 2005 09:21:52 -0500
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <428A41D2.6090002@ncsa.uiuc.edu>
Message-ID: <20050519142152.GA21065@uiuc.edu>

Hi Chistoph, Tim, All: 
(I was waiting to get some information/clarifications but nothing yet) 

Yes, we were/are seeing two very different types of dropped packets notifications :  
 
1) Initially packets were dropped at the interface but as Tim pointed out, that got fixed and current count is substentially low: 

          RX packets:3550702871 errors:25 dropped:23 overruns:2 frame:2
          RX packets:1577193887 errors:13 dropped:13 overruns:0 frame:0

2) Dropped packets notice in the notice.log files. Example: 

t=1116392591.523558 no=DroppedPackets na=NOTICE_FILE msg=4475\ packets\ dropped\ after\ filtering,\ 21924\ received

Looking at the policy file (netstats.bro) I am inclined to think that these notices are generated because of bro filter. Please correct me here. 

So in short, since we are using direct network feed, right now, I am relying on error count on interfaces which shows a very low number of packet drops.  While with bro we do get dropped packets notices notice.log file which are due to bro filter.

What I cannot answer/understand right now is:

Is there any way I can find out is bro actually dropping packets, if at all ?

Aashish 
 
> I'll let my co-worker, Aashish Sharma, reply to the specific issue of
> bro dropping packets.  However, there are two corrections we made to
> correct for dropped packets and errors that we were receiving on our
> 1GB fiber interfaces after first installing bro and turning it on.
> 
> First, we set the MTU from 1500 to 9000.
> Second, we set LowLatency=On (i.e. modprobe sk98lin LowLatency=On)
> 
> That second fix is specific to the fiber cards we are using.  After
> these two changes, we are no longer receiving errors on the interfaces.
> 
> Aashish Sharma will follow up with bro specific packet loss answer.
> 
> Thanks,
> 
> Tim
> 
> Christoph Goeldi wrote:
> 
> > Hi Tim
> >
> > Zitat von Tim Brooks <tbrooks at ncsa.uiuc.edu>:
> >
> >>
> >> Ours works fine on linux with the interfaces set in etc/bro.cfg like:
> >>
> >> BRO_CAPTURE_INTERFACE="eth2 eth3"
> >
> >
> > Are you realy sure, that Bro doesn't drop the most of the captured
> > packets?
> >
> > I like to know what Linux version (distro), what Bro version and what
> > interfaces (100Mbit or 1Gbit / manufacturer) do you use?
> >
> > Thank you for your time
> > Christoph
> 
> 
> -- 
> Tim Brooks
> Security Engineer
> 
> National Center for Supercomputing Applications
> 605 East Springfield Avenue   Champaign, IL 61820
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20050519/05eb712f/attachment.bin 

From vern at icir.org  Thu May 19 12:08:36 2005
From: vern at icir.org (Vern Paxson)
Date: Thu, 19 May 2005 12:08:36 -0700
Subject: [Bro] Problem: Bro listening on two ethernet interfaces 
In-Reply-To: Your message of Thu, 19 May 2005 09:21:52 CDT.
Message-ID: <200505191908.j4JJ8aH5000215@jaguar.icir.org>

> t=3D1116392591.523558 no=3DDroppedPackets na=3DNOTICE_FILE msg=3D4475\ pack=
> ets\ dropped\ after\ filtering,\ 21924\ received
> 
> Looking at the policy file (netstats.bro) I am inclined to think that these=
>  notices are generated because of bro filter. Please correct me here.=20

I'm not sure what you mean by "because of bro filter", but those reports
are generated based on retrieving statistics from libpcap, and "after
filtering" means after applying the filter Bro specified to pcap, which
you can see using print-filter.bro.

> What I cannot answer/understand right now is:
> 
> Is there any way I can find out is bro actually dropping packets, if at all=
>  ?

Look for AckAboveHole and ContentGap notices.  These can both happen for
other reasons, but primarily happen due to dropped packets.

		Vern


From aashish at uiuc.edu  Thu May 19 14:20:35 2005
From: aashish at uiuc.edu (Aashish Sharma)
Date: Thu, 19 May 2005 16:20:35 -0500
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <200505191908.j4JJ8aH5000215@jaguar.icir.org>
Message-ID: <20050519212035.GA29453@uiuc.edu>

> Look for AckAboveHole and ContentGap notices.  These can both happen for
> other reasons, but primarily happen due to dropped packets.

I looked in previous logs and found no alerts what so ever for AckAboveHole and ContentGap to realize that brolite had notice_ignore for them. 

# Remove these notices from logging since they can be too noisy.
redef notice_action_filters += {
        [[ContentGap, AckAboveHole]] = notice_ignore,
};

Which I changed to file_notice. 
        [[ContentGap, AckAboveHole]] = file_notice,


So. in last 1h15m I have 165065 ContentGap and 36596 AckAboveHole notices. Is this normal (noisy?) or is this the traffic getting dropped ? 

And by getting dropped - do we mean that bro is not processing this traffic ? 

2) Also Can I put it this way : 

Of all traffic we get on interfaces we filter a part using bro filter (packetdrop notices) and then there are dropped packets (AckAboveHole and ContentGap notices)  and then rest of the traffic is processed by bro ? 


> I'm not sure what you mean by "because of bro filter", but those reports
Here, I meant is since we have applied bro-filter, all these packet drops could be accounted for. (they are dropped by the bro-filter) 


Thanks a lot for all the help. We have been constantly tuning bro and results are great. 


Aashish 



On Thu, May 19, 2005 at 12:08:36PM -0700, Vern Paxson wrote:
> > t=3D1116392591.523558 no=3DDroppedPackets na=3DNOTICE_FILE msg=3D4475\ pack=
> > ets\ dropped\ after\ filtering,\ 21924\ received
> > 
> > Looking at the policy file (netstats.bro) I am inclined to think that these=
> >  notices are generated because of bro filter. Please correct me here.=20
> 
> I'm not sure what you mean by "because of bro filter", but those reports
> are generated based on retrieving statistics from libpcap, and "after
> filtering" means after applying the filter Bro specified to pcap, which
> you can see using print-filter.bro.
> 
> > What I cannot answer/understand right now is:
> > 
> > Is there any way I can find out is bro actually dropping packets, if at all=
> >  ?
> 
> Look for AckAboveHole and ContentGap notices.  These can both happen for
> other reasons, but primarily happen due to dropped packets.
> 
> 		Vern
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20050519/aac8e7b0/attachment.bin 

From vern at icir.org  Fri May 20 00:16:14 2005
From: vern at icir.org (Vern Paxson)
Date: Fri, 20 May 2005 00:16:14 -0700
Subject: [Bro] new Bro CURRENT release (0.9a9)
Message-ID: <200505200716.j4K7GEGY048078@jaguar.icir.org>

A new CURRENT release, 0.9a9, is now available from:

	ftp://ftp.ee.lbl.gov/bro-0.9-current.tar.gz

This release includes a significant number of changes and bug fixes, per
the appended.  It has one known glitch, which is some bogus alarms generated
when using the DNS analyzer.  We hope to have those fixed soon.

		Vern


-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


0.9a9 Thu May 19 23:31:33 PDT 2005

- First cut at analyzer for NFS (Vern Paxson).  It generates the following
  events:
	event nfs_request_null(n: connection)
	event nfs_request_getattr(n: connection, fh: string, attrs: nfs3_attrs)
	event nfs_request_lookup(n: connection, req: nfs3_lookup_args,
				rep: nfs3_lookup_reply)
	event nfs_request_fsstat(n: connection, root_fh: string,
				stat: nfs3_fsstat)
	event nfs_attempt_null(n: connection, status: count)
	event nfs_attempt_getattr(n: connection, status: count, fh: string)
	event nfs_attempt_lookup(n: connection, status: count,
				req: nfs3_lookup_args,
				dir_attrs: nfs3_opt_attrs)
	event nfs_attempt_fsstat(n: connection, status: count,
				root_fh: string, obj_attrs: nfs3_opt_attrs)

- The new script OS-fingerprint.bro integrates Bro's new passive OS
  fingerprinting mechanism with the software.bro framework (Vern Paxson).

- You can now operate on patterns using && and || (Vern Paxson).
  If p1 and p2 are patterns, then p1 && p2 yields a pattern that matches
  their concatenation and p1 || p2 yields a pattern that matches either.
  Note that the syntax for this may change in the future to a single '&'
  or '|', which would be more consistent with the use of '|' in
  constructing pattern constants.

- An experimental "connection compressor" tracks not-yet-established
  connections using much less memory than Bro normally does (Robin Sommer).
  This is potentially a major win during flooding attacks and high-speed
  scans.  You activate it by setting use_connection_compressor to T.  You
  can then control the granularity of its processing using the variables
  cc_handle_resets, cc_handle_only_syns, and cc_instantiate_on_data.  See
  bro.init for brief discussion of these.

- The experimental new script firewall.bro supports firewall-rule-like
  processing of connections in terms of allow/deny (Robin Sommer).  It is
  not particularly efficient.

- sensor-sshd.bro provides an experimental interface for receiving
  events from instrumented SSH servers that communicate with Bro via
  the Broccoli client library (Christian Kreibich and Robin Sommer).
  Supporting this also entailed extensions to login.bro so it can
  process the events even though they don't correspond to a connection
  known to Bro's event engine.

- The new built-in function match_signatures() can be used in a policy
  script to send text directly into the signature engine (Robin Sommer).

- Correction: the 0.9a8 CHANGES states that the mail_script variable used
  for NOTICE_EMAIL defaults to mail_script.sh.  The correct value is instead
  "mail_notice.sh".

- The scripts rsh.bro and passwords.bro, and the passive-fingerprinting
  signatures policy/sigs/p0fsyn.osf were inadvertantly left out of the
  0.9a8 distribution.

- Added s2b (snort to bro) files into the distribution. (Jason Lee)

- Non-blocking packet capture under Linux has been fixed (Robin Sommer).

- Fixed printing of DNS replies, which used to work but was broken
  a number of months ago (Vern Paxson).

- The new script brolite-sigs separates out how signatures are configured
  in Bro Lite so the functionality can be enabled/disabled with a simple
  load statement (Roger Winslow).  That is, to use signatures with Bro
  lite, simply add "@load brolite-sigs".

- The new script variable enable_syslog (default T) controls whether
  alarm's are syslog'd (Robin Sommer).  As before, syslogs can only happen
  when Bro is reading from live network traffic (this should be changed
  at some point, to accommodate real-time Bro's that don't read the network
  but collect events from other sensors).  Previously, in that case syslog's
  always happened; now, you can turn them off using this variable.

- The new script variable expensive_profiling_multiple controls how
  often, when doing profiling, to perform more expensive forms of
  profiling, in particular, memory consumption profiling (Robin Sommer).
  If profiling_interval is set to 15 sec and expensive_profiling_multiple
  is set to 20, then expensive profiling will be done every 5 minutes
  (these are the defaults now in profiling.bro).  Also, the profiling_update
  event now includes a second argument, expensive: bool, which indicates
  whether the update corresponds to one of these expensive profiling
  intervals.

- First cut at parsing DNS AAAA replies (Scott Campbell).  This is quite
  incomplete - currently, the replies are turned into fake A record replies,
  due to the difficulty of dealing with IPv6 addresses if Bro wasn't built
  to analyze IPv6 traffic.

- software.bro has been tweaked to have a new control variable,
  "only_report_local" (default F).  If true, then only software versions
  for local addresses (as determined by is_local_addr()) will be
  reported.

- synflood.bro now has a script variable max_sources (default 100) that
  specifies the maximum number of sources to track for a given victim
  (Robin Sommer).

- Remote peers now negotiate their versions of the serialization format
  (Robin Sommer). If they don't agree then the connection is terminated.

- Generic UDP request/response processing has been moved into the new
  policy script udp-common.bro, which, unlike udp.bro, does *not* set the
  packet filter to capture all UDP traffic (Robin Sommer).  A number
  of UDP-based policy scripts have been modified to use udp-common.bro
  rather than udp.bro.

- When printing serialized/independent state, access times are now
  again included (Robin Sommer).

- Bro's implementation of timers has been switched (reverted) to using
  priority queues (Vern Paxson).

- The http-request.bro script variables skip_remote_sensitive_URIs and
  const sensitive_post_URIs are now exported so they can be accessed
  externally (Robin Sommer).

- Some new rootkit filenames have been added to ftp.bro and
  http-request.bro (Brian Tierney).  The plan is to eventually
  merge these lists so there's only one main list.

- trw.bro is now scoped as a module "TRW" (Brian Tierney).

- Better support of the '--disable-localpcap' flag to configure, and
  consolidated all the pcap checks in configure.in (Jason Lee).

- A bug in processing bare carriage-returns in Telnet input/output
  has been fixed (Vern Paxson).

- The Bro Lite bro.rc script has been tweaked to use the 'ax' flags
  instead of '-ax' (Jason Lee).

- A bug with reporting ICMP "ports" (i.e., type + code) has been fixed
  (Vern Paxson).

- Bug fix for excessively large RPC messages (Ruoming Pang).

- A bug with /0 subnet prefixes has been fixed (Robin Sommer).

- The function record_connection() now takes the file to write to
  as its first argument (Robin Sommer).

- remote.bro now tracks whether a given Destination is connected
  (Robin Sommer).

- mail_notice.sh is now installed as part of installing a distribution
  (Jason Lee).

- Fixed bug where the sort order for the test suite changed depending
  on locale. (Jason Lee)

- Bug fix for email_notice() when notice_action_filters not defined for
  given notice (Vern Paxson).

- The test suite test for rare-events fixed to not give false positives
  (Jason Lee).

- Date added for 0.9a8 release.


From goeldich at ee.ethz.ch  Fri May 20 01:57:41 2005
From: goeldich at ee.ethz.ch (Christoph Goeldi)
Date: Fri, 20 May 2005 09:57:41 +0100
Subject: [Bro] new Bro CURRENT release (0.9a9)
In-Reply-To: <200505200716.j4K7GEGY048078@jaguar.icir.org>
References: <200505200716.j4K7GEGY048078@jaguar.icir.org>
Message-ID: <20050520095741.ghiyaytigoccok8g@email.ee.ethz.ch>

YEAH!!! Very nice. I will test this release soon.

Thanx to all you developer.

Maybe you can fix the links on the download page of the homepage:
http://www.bro-ids.org/download.html

Cheers
Christoph


Zitat von Vern Paxson <vern at icir.org>:

> A new CURRENT release, 0.9a9, is now available from:
>
> 	ftp://ftp.ee.lbl.gov/bro-0.9-current.tar.gz
>
> This release includes a significant number of changes and bug fixes, per
> the appended.  It has one known glitch, which is some bogus alarms generated
> when using the DNS analyzer.  We hope to have those fixed soon.
>
> 		Vern
>
>
> -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
>
> 0.9a9 Thu May 19 23:31:33 PDT 2005
>
> - First cut at analyzer for NFS (Vern Paxson).  It generates the following
>  events:
> 	event nfs_request_null(n: connection)
> 	event nfs_request_getattr(n: connection, fh: string, attrs: nfs3_attrs)
> 	event nfs_request_lookup(n: connection, req: nfs3_lookup_args,
> 				rep: nfs3_lookup_reply)
> 	event nfs_request_fsstat(n: connection, root_fh: string,
> 				stat: nfs3_fsstat)
> 	event nfs_attempt_null(n: connection, status: count)
> 	event nfs_attempt_getattr(n: connection, status: count, fh: string)
> 	event nfs_attempt_lookup(n: connection, status: count,
> 				req: nfs3_lookup_args,
> 				dir_attrs: nfs3_opt_attrs)
> 	event nfs_attempt_fsstat(n: connection, status: count,
> 				root_fh: string, obj_attrs: nfs3_opt_attrs)
>
> - The new script OS-fingerprint.bro integrates Bro's new passive OS
>  fingerprinting mechanism with the software.bro framework (Vern Paxson).
>
> - You can now operate on patterns using && and || (Vern Paxson).
>  If p1 and p2 are patterns, then p1 && p2 yields a pattern that matches
>  their concatenation and p1 || p2 yields a pattern that matches either.
>  Note that the syntax for this may change in the future to a single '&'
>  or '|', which would be more consistent with the use of '|' in
>  constructing pattern constants.
>
> - An experimental "connection compressor" tracks not-yet-established
>  connections using much less memory than Bro normally does (Robin Sommer).
>  This is potentially a major win during flooding attacks and high-speed
>  scans.  You activate it by setting use_connection_compressor to T.  You
>  can then control the granularity of its processing using the variables
>  cc_handle_resets, cc_handle_only_syns, and cc_instantiate_on_data.  See
>  bro.init for brief discussion of these.
>
> - The experimental new script firewall.bro supports firewall-rule-like
>  processing of connections in terms of allow/deny (Robin Sommer).  It is
>  not particularly efficient.
>
> - sensor-sshd.bro provides an experimental interface for receiving
>  events from instrumented SSH servers that communicate with Bro via
>  the Broccoli client library (Christian Kreibich and Robin Sommer).
>  Supporting this also entailed extensions to login.bro so it can
>  process the events even though they don't correspond to a connection
>  known to Bro's event engine.
>
> - The new built-in function match_signatures() can be used in a policy
>  script to send text directly into the signature engine (Robin Sommer).
>
> - Correction: the 0.9a8 CHANGES states that the mail_script variable used
>  for NOTICE_EMAIL defaults to mail_script.sh.  The correct value is instead
>  "mail_notice.sh".
>
> - The scripts rsh.bro and passwords.bro, and the passive-fingerprinting
>  signatures policy/sigs/p0fsyn.osf were inadvertantly left out of the
>  0.9a8 distribution.
>
> - Added s2b (snort to bro) files into the distribution. (Jason Lee)
>
> - Non-blocking packet capture under Linux has been fixed (Robin Sommer).
>
> - Fixed printing of DNS replies, which used to work but was broken
>  a number of months ago (Vern Paxson).
>
> - The new script brolite-sigs separates out how signatures are configured
>  in Bro Lite so the functionality can be enabled/disabled with a simple
>  load statement (Roger Winslow).  That is, to use signatures with Bro
>  lite, simply add "@load brolite-sigs".
>
> - The new script variable enable_syslog (default T) controls whether
>  alarm's are syslog'd (Robin Sommer).  As before, syslogs can only happen
>  when Bro is reading from live network traffic (this should be changed
>  at some point, to accommodate real-time Bro's that don't read the network
>  but collect events from other sensors).  Previously, in that case syslog's
>  always happened; now, you can turn them off using this variable.
>
> - The new script variable expensive_profiling_multiple controls how
>  often, when doing profiling, to perform more expensive forms of
>  profiling, in particular, memory consumption profiling (Robin Sommer).
>  If profiling_interval is set to 15 sec and expensive_profiling_multiple
>  is set to 20, then expensive profiling will be done every 5 minutes
>  (these are the defaults now in profiling.bro).  Also, the profiling_update
>  event now includes a second argument, expensive: bool, which indicates
>  whether the update corresponds to one of these expensive profiling
>  intervals.
>
> - First cut at parsing DNS AAAA replies (Scott Campbell).  This is quite
>  incomplete - currently, the replies are turned into fake A record replies,
>  due to the difficulty of dealing with IPv6 addresses if Bro wasn't built
>  to analyze IPv6 traffic.
>
> - software.bro has been tweaked to have a new control variable,
>  "only_report_local" (default F).  If true, then only software versions
>  for local addresses (as determined by is_local_addr()) will be
>  reported.
>
> - synflood.bro now has a script variable max_sources (default 100) that
>  specifies the maximum number of sources to track for a given victim
>  (Robin Sommer).
>
> - Remote peers now negotiate their versions of the serialization format
>  (Robin Sommer). If they don't agree then the connection is terminated.
>
> - Generic UDP request/response processing has been moved into the new
>  policy script udp-common.bro, which, unlike udp.bro, does *not* set the
>  packet filter to capture all UDP traffic (Robin Sommer).  A number
>  of UDP-based policy scripts have been modified to use udp-common.bro
>  rather than udp.bro.
>
> - When printing serialized/independent state, access times are now
>  again included (Robin Sommer).
>
> - Bro's implementation of timers has been switched (reverted) to using
>  priority queues (Vern Paxson).
>
> - The http-request.bro script variables skip_remote_sensitive_URIs and
>  const sensitive_post_URIs are now exported so they can be accessed
>  externally (Robin Sommer).
>
> - Some new rootkit filenames have been added to ftp.bro and
>  http-request.bro (Brian Tierney).  The plan is to eventually
>  merge these lists so there's only one main list.
>
> - trw.bro is now scoped as a module "TRW" (Brian Tierney).
>
> - Better support of the '--disable-localpcap' flag to configure, and
>  consolidated all the pcap checks in configure.in (Jason Lee).
>
> - A bug in processing bare carriage-returns in Telnet input/output
>  has been fixed (Vern Paxson).
>
> - The Bro Lite bro.rc script has been tweaked to use the 'ax' flags
>  instead of '-ax' (Jason Lee).
>
> - A bug with reporting ICMP "ports" (i.e., type + code) has been fixed
>  (Vern Paxson).
>
> - Bug fix for excessively large RPC messages (Ruoming Pang).
>
> - A bug with /0 subnet prefixes has been fixed (Robin Sommer).
>
> - The function record_connection() now takes the file to write to
>  as its first argument (Robin Sommer).
>
> - remote.bro now tracks whether a given Destination is connected
>  (Robin Sommer).
>
> - mail_notice.sh is now installed as part of installing a distribution
>  (Jason Lee).
>
> - Fixed bug where the sort order for the test suite changed depending
>  on locale. (Jason Lee)
>
> - Bug fix for email_notice() when notice_action_filters not defined for
>  given notice (Vern Paxson).
>
> - The test suite test for rare-events fixed to not give false positives
>  (Jason Lee).
>
> - Date added for 0.9a8 release.
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>





From goeldich at ee.ethz.ch  Fri May 20 01:30:14 2005
From: goeldich at ee.ethz.ch (Christoph Goeldi)
Date: Fri, 20 May 2005 10:30:14 +0200
Subject: [Bro] Still outstanding, scheduled events when Bro terminates
Message-ID: <428DA016.3020807@ee.ethz.ch>

Hi

I write regularly (5min interval) some information to a file
which I use externally in a webpage by parsing the content.
I want to overwrite the old file everytime and don't like to
use log rotation because of disk space consumption.

I use the following code:

******** log.bro **********
# writes a summary of the currently seen hosts
# and its states to the file "current_states.log"
# do this regularly all Xmins (log_interval)
event print_current_states_file() {
         # opens the file "current_states.log"
         local current_states_file = open("current_states.log");

         # write file header
         print "...";
	...

         # close the file "current_states.log"
         close(current_states_file);

         # do it again in Xmins
         schedule log_interval {print_current_states_file()};
}

event bro_init() {
         # start log
         schedule log_interval {print_current_states_file()};
}
***************************

Now my Problem:
When I terminate Bro it writes the scheduled print_current_states_file 
event a last time. Unfortunately, the data that i write to the file is
already incomplete because Bro already started to delete these tables.
Therefore, the file which remains after the termination of Bro is
unusable.

Any ideas, how to solve this problem?

Cheers
Christoph



From sommer at in.tum.de  Fri May 20 02:52:34 2005
From: sommer at in.tum.de (Robin Sommer)
Date: Fri, 20 May 2005 11:52:34 +0200
Subject: [Bro] Still outstanding, scheduled events when Bro terminates
In-Reply-To: <428DA016.3020807@ee.ethz.ch>
References: <428DA016.3020807@ee.ethz.ch>
Message-ID: <20050520095234.GA7125@net.informatik.tu-muenchen.de>

On Fri, May 20, 2005 at 10:30 +0200, Christoph Goeldi wrote:

> When I terminate Bro it writes the scheduled print_current_states_file 
> event a last time. Unfortunately, the data that i write to the file is
> already incomplete because Bro already started to delete these tables.

The function bro_is_terminating returns true if Bro is cleaning up.
You could call that in print_current_states_file to avoid writing
the file.

Robin

-- 
Robin Sommer * Room        01.08.055 * www.net.in.tum.de
TU Muenchen  * Phone (089) 289-18006 *  sommer at in.tum.de 


From goeldich at ee.ethz.ch  Fri May 20 04:03:53 2005
From: goeldich at ee.ethz.ch (Christoph Goeldi)
Date: Fri, 20 May 2005 12:03:53 +0100
Subject: [Bro] Still outstanding, scheduled events when Bro terminates
In-Reply-To: <20050520095234.GA7125@net.informatik.tu-muenchen.de>
References: <428DA016.3020807@ee.ethz.ch>
	<20050520095234.GA7125@net.informatik.tu-muenchen.de>
Message-ID: <20050520120353.bywropr54os0k08w@email.ee.ethz.ch>

Hi Robin

>> When I terminate Bro it writes the scheduled print_current_states_file
>> event a last time. Unfortunately, the data that i write to the file is
>> already incomplete because Bro already started to delete these tables.
>
> The function bro_is_terminating returns true if Bro is cleaning up.
> You could call that in print_current_states_file to avoid writing
> the file.

Very nice. Thank you. Works perfect.

Cheers
Christoph




From vern at icir.org  Fri May 20 08:55:57 2005
From: vern at icir.org (Vern Paxson)
Date: Fri, 20 May 2005 08:55:57 -0700
Subject: [Bro] new Bro CURRENT release (0.9a9)
In-Reply-To: Your message of Fri, 20 May 2005 09:57:41 BST.
Message-ID: <200505201556.j4KFtvXB092844@jaguar.icir.org>

> Maybe you can fix the links on the download page of the homepage:
> http://www.bro-ids.org/download.html

Oops - fixed.

		Vern


From goeldich at ee.ethz.ch  Mon May 23 09:54:00 2005
From: goeldich at ee.ethz.ch (Christoph Goeldi)
Date: Mon, 23 May 2005 17:54:00 +0100
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <200505101438.j4AEck4t069243@jaguar.icir.org>
References: <200505101438.j4AEck4t069243@jaguar.icir.org>
Message-ID: <20050523175400.9bxnnly9co840swc@email.ee.ethz.ch>

Zitat von Vern Paxson <vern at icir.org>:

>> i looked at the c-code. i runned it on different machines and
>> on various interfaces. bro still drops most of the packets
>> when i force it to listen on two interfaces.
>>
>> is it a libpcap problem?
>> a bro problem?
>> a linux problem?
>
> I believe it's a Linux problem.  We do this under FreeBSD in two different
> ways, either merging the interfaces in the kernel into one logical interface
> (via a custom patch), or at user level.  While the in-kernel version
> performs better, the user-level one isn't a disaster like you describe.
>
> I also recall hearing others mention that multiple interfaces under Linux
> do not work well in general.  I don't use Linux, though, so can't comment
> more directly.

I found a small C-program that allows to listen on multiple interfaces and to
write the captured packets to a file:
http://www.isi.edu/~hussain/software/snoop.c

And it works!!!
I'm really not (yet) the pcap-crack. Does somebody know what's the difference
between this program and the bro implementation?

I really appreciate any help.

Cheers
Christoph




From chema at cs.berkeley.edu  Mon May 23 11:49:44 2005
From: chema at cs.berkeley.edu (Jose M. Gonzalez)
Date: Mon, 23 May 2005 11:49:44 -0700
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <20050523175400.9bxnnly9co840swc@email.ee.ethz.ch>
References: <200505101438.j4AEck4t069243@jaguar.icir.org> <20050523175400.9bxnnly9co840swc@email.ee.ethz.ch>
Message-ID: <20050523184944.GA9960@cs.berkeley.edu>

Christoph, 

The C-program you mention opens several interfaces and select()'s its 
descriptors. A per-packet call to select() can be too expensive in 
high-volume environments. Moreover, it's not clear select() is the 
cheapest way to attend several descriptors. If you want to play with 
this, Kohler's click FromDevice element permits selecting between 
select(), poll(), and FreeBSD's kevent() (though the latter may be buggy 
when used with BPF devices). 

http://pdos.csail.mit.edu/click/

FYI, Bro tries to limit the calls to select() to just those instants 
when all the sources are dry (or every often; check IOSource.cc and 
PktSrc.cc, where all the pcap stuff is located). Also, Bro orders 
packets received from different sources by their timestamp (the 
C-program is biased to processing packets from the first interface). 

BTW, you can't compare this program with Bro. The former just dumps 
packets to a file. Bro is a stateful intrusion detection system. 

-Chema



Christoph Goeldi wrote:
> Zitat von Vern Paxson <vern at icir.org>:
> 
> >>i looked at the c-code. i runned it on different machines and
> >>on various interfaces. bro still drops most of the packets
> >>when i force it to listen on two interfaces.
> >>
> >>is it a libpcap problem?
> >>a bro problem?
> >>a linux problem?
> >
> >I believe it's a Linux problem.  We do this under FreeBSD in two different
> >ways, either merging the interfaces in the kernel into one logical 
> >interface
> >(via a custom patch), or at user level.  While the in-kernel version
> >performs better, the user-level one isn't a disaster like you describe.
> >
> >I also recall hearing others mention that multiple interfaces under Linux
> >do not work well in general.  I don't use Linux, though, so can't comment
> >more directly.
> 
> I found a small C-program that allows to listen on multiple interfaces and 
> to
> write the captured packets to a file:
> http://www.isi.edu/~hussain/software/snoop.c
> 
> And it works!!!
> I'm really not (yet) the pcap-crack. Does somebody know what's the 
> difference
> between this program and the bro implementation?
> 
> I really appreciate any help.
> 
> Cheers
> Christoph
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From christian at whoop.org  Mon May 23 12:19:12 2005
From: christian at whoop.org (Christian Kreibich)
Date: Mon, 23 May 2005 12:19:12 -0700
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <20050523175400.9bxnnly9co840swc@email.ee.ethz.ch>
References: <200505101438.j4AEck4t069243@jaguar.icir.org>
	 <20050523175400.9bxnnly9co840swc@email.ee.ethz.ch>
Message-ID: <1116875953.7332.3.camel@wifi167.icsi.berkeley.edu>

Hi Christoph,

On Mon, 2005-05-23 at 17:54 +0100, Christoph Goeldi wrote:
> 
> I found a small C-program that allows to listen on multiple interfaces and to
> write the captured packets to a file:
> http://www.isi.edu/~hussain/software/snoop.c
> 
> And it works!!!
> I'm really not (yet) the pcap-crack. Does somebody know what's the difference
> between this program and the bro implementation?

I had a quick look at snoop.c and it basically does the most
straightforward thing for the task: a select() on the file descriptors
associated with the pcap handles of the interfaces.

Bro's approach is somewhat more involved because you cannot afford a
per-packet select() call on a busy link (see Robin's comments in
IOSource.cc). Maybe IOSourceRegistry::FindSoonest() would be a good
place to start digging.

> I really appreciate any help.

I'm sorry I can't help any further regarding this -- if you're on Linux,
have you tried letting the kernel sort this out and just use the "any"
interface (I forget whether this has been proposed in this thread
before)?

Cheers,
Christian.
-- 
________________________________________________________________________
                                          http://www.cl.cam.ac.uk/~cpk25
                                                    http://www.whoop.org




From goeldich at ee.ethz.ch  Mon May 23 13:28:10 2005
From: goeldich at ee.ethz.ch (=?ISO-8859-1?Q?Christoph_G=F6ldi?=)
Date: Mon, 23 May 2005 22:28:10 +0200
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <20050523184944.GA9960@cs.berkeley.edu>
References: <200505101438.j4AEck4t069243@jaguar.icir.org>
 <20050523175400.9bxnnly9co840swc@email.ee.ethz.ch>
 <20050523184944.GA9960@cs.berkeley.edu>
Message-ID: <9E9FADAB07AA98FD3FEB5E8D@[192.168.0.2]>

Hi Chema


> The C-program you mention opens several interfaces and select()'s its 
> descriptors. A per-packet call to select() can be too expensive in 
> high-volume environments. Moreover, it's not clear select() is the 
> cheapest way to attend several descriptors. If you want to play with 
> this, Kohler's click FromDevice element permits selecting between 
> select(), poll(), and FreeBSD's kevent() (though the latter may be buggy 
> when used with BPF devices). 
> 
> http://pdos.csail.mit.edu/click/
> 
> FYI, Bro tries to limit the calls to select() to just those instants 
> when all the sources are dry (or every often; check IOSource.cc and 
> PktSrc.cc, where all the pcap stuff is located). Also, Bro orders 
> packets received from different sources by their timestamp (the 
> C-program is biased to processing packets from the first interface). 

Thank you for your explanations. I don't understand the details of
these possibilities when capturing packets. But I will try to learn
these things to find the problem which appears when bro is listening 
on multiple interfaces.

> BTW, you can't compare this program with Bro. The former just dumps 
> packets to a file. Bro is a stateful intrusion detection system. 

I know that this 300-lines-program has not the same functionality
like bro! ;-)
I just try to understand why the capturing of traffic on multiple
interfaces doesn't work with Linux.


Thank you for your time
Christoph


> Christoph Goeldi wrote:
>> Zitat von Vern Paxson <vern at icir.org>:
>> 
>> >> i looked at the c-code. i runned it on different machines and
>> >> on various interfaces. bro still drops most of the packets
>> >> when i force it to listen on two interfaces.
>> >> 
>> >> is it a libpcap problem?
>> >> a bro problem?
>> >> a linux problem?
>> > 
>> > I believe it's a Linux problem.  We do this under FreeBSD in two
>> > different ways, either merging the interfaces in the kernel into one
>> > logical  interface
>> > (via a custom patch), or at user level.  While the in-kernel version
>> > performs better, the user-level one isn't a disaster like you describe.
>> > 
>> > I also recall hearing others mention that multiple interfaces under
>> > Linux do not work well in general.  I don't use Linux, though, so
>> > can't comment more directly.
>> 
>> I found a small C-program that allows to listen on multiple interfaces
>> and  to
>> write the captured packets to a file:
>> http://www.isi.edu/~hussain/software/snoop.c
>> 
>> And it works!!!
>> I'm really not (yet) the pcap-crack. Does somebody know what's the 
>> difference
>> between this program and the bro implementation?
>> 
>> I really appreciate any help.
>> 
>> Cheers
>> Christoph
>> 
>> 
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro






From goeldich at ee.ethz.ch  Mon May 23 13:41:21 2005
From: goeldich at ee.ethz.ch (=?ISO-8859-1?Q?Christoph_G=F6ldi?=)
Date: Mon, 23 May 2005 22:41:21 +0200
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <1116875953.7332.3.camel@wifi167.icsi.berkeley.edu>
References: <200505101438.j4AEck4t069243@jaguar.icir.org>	
 <20050523175400.9bxnnly9co840swc@email.ee.ethz.ch>
 <1116875953.7332.3.camel@wifi167.icsi.berkeley.edu>
Message-ID: <53EC33A266C6897545E3A228@[192.168.0.2]>

Hi Christian


>> I found a small C-program that allows to listen on multiple interfaces
>> and to write the captured packets to a file:
>> http://www.isi.edu/~hussain/software/snoop.c
>> 
>> And it works!!!
>> I'm really not (yet) the pcap-crack. Does somebody know what's the
>> difference between this program and the bro implementation?
> 
> I had a quick look at snoop.c and it basically does the most
> straightforward thing for the task: a select() on the file descriptors
> associated with the pcap handles of the interfaces.
> 
> Bro's approach is somewhat more involved because you cannot afford a
> per-packet select() call on a busy link (see Robin's comments in
> IOSource.cc). Maybe IOSourceRegistry::FindSoonest() would be a good
> place to start digging.

Okay. I'll try to figure out more about this ominous select().

>> I really appreciate any help.
> 
> I'm sorry I can't help any further regarding this -- if you're on Linux,
> have you tried letting the kernel sort this out and just use the "any"
> interface (I forget whether this has been proposed in this thread
> before)?

I'll try the any interface tomorrow. But it wouldn't solve my problems
anyway because I want to specifically select the observed interfaces and
not capture the packets of all interfaces of this host.


Thanks for your help.
Cheers
Christoph 


From chema at cs.berkeley.edu  Mon May 23 17:45:12 2005
From: chema at cs.berkeley.edu (Jose M. Gonzalez)
Date: Mon, 23 May 2005 17:45:12 -0700
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <9E9FADAB07AA98FD3FEB5E8D@[192.168.0.2]>
References: <200505101438.j4AEck4t069243@jaguar.icir.org> <20050523175400.9bxnnly9co840swc@email.ee.ethz.ch> <20050523184944.GA9960@cs.berkeley.edu> <9E9FADAB07AA98FD3FEB5E8D@[192.168.0.2]>
Message-ID: <20050524004512.GA19464@cs.berkeley.edu>

Christoph G?ldi wrote:
> > BTW, you can't compare this program with Bro. The former just dumps 
> > packets to a file. Bro is a stateful intrusion detection system. 
> 
> I know that this 300-lines-program has not the same functionality
> like bro! ;-)
> I just try to understand why the capturing of traffic on multiple
> interfaces doesn't work with Linux.
Slightly offtopic, I had a related problem capturing traffic in Linux 
(RHEL WS 4 running 2.6.9-5.ELsmp), in just one interface at the same 
time. 

I tried injecting packets into a network device, and capturing them in 
the same machine (tcpreplay/tcpdump). In both cases, I used a 230 Mbps, 
real-traffic trace that lasts 3.4 seconds (~250000 packets). 

When I used a real device (Intel PRO/100 Ethernet Driver), tcpreplay got 
slowed down to 90 Mbps, and tcpdump got duplicated packets (not that many, 
~262 out of 250000). 

When I used a virtual device (VMWare's vmnet), tcpdump lost ~66 consecutive 
packets out of the 250000, which were reported as "dropped by kernel." 

Maybe somebody has some ideas. 
-Chema



From bchen at cs.ucf.edu  Mon May 23 19:52:52 2005
From: bchen at cs.ucf.edu (bchen at cs.ucf.edu)
Date: Mon, 23 May 2005 22:52:52 -0400
Subject: [Bro] detect Ack flooding attack
In-Reply-To: <200505181809.j4II9NBG044438@jaguar.icir.org>
References: <200505181809.j4II9NBG044438@jaguar.icir.org>
Message-ID: <20050523225252.5hqy409jc40og00k@mail.cs.ucf.edu>

Hi Vern,
    I have an interesting finding about the problem I met. It was the backdoor
analyzer that prevented those ack flooding packets from logging. If I load the
backdoor.bro into mt.bro and run bro to read tcpdump file (command line: ./bro
-r 2000.dump mt), those ack flooding entries are missing in conn.log and
weird.log. If I unload the backdoor.bro from mt.bro and run bro, those ack
flooding packets are logged in conn.log and weird.log. The interesting 
thing is
these ack flooding packets are sent by a backdoor program (Mstream DDOS 
tool). I
don't understand why the backdoor analyzer blocks the logging of these 
packets.
   By the way, the latest version just released looks much slower than previous
version in my machine (Linux).

Bing


Quoting Vern Paxson <vern at icir.org>:

>>      Thank you for your reply. I corrected this filter expression 
>> and run Bro,
>> but I got the same result. I can see these spoofed source IP packets with
>> Ethereal. All of them target the same host but with different destination
>> ports. The TCP flag of these packets is 0x0010 (ack). I found no single log
>> record was for such packets. Am I missing anything?
>>      By the way, I am using the DARPA 2000 data set (Scenario one, inside
>> tcpdump file). This is the link for this data:
>> http://www.ll.mit.edu/IST/ideval/data/2000/LLS_DDOS_1.0.html
>
> Please send a small trace that can be used to reproduce the problem.
> Thanks.
>
> 		Vern
>




From sommer at in.tum.de  Tue May 24 00:31:48 2005
From: sommer at in.tum.de (Robin Sommer)
Date: Tue, 24 May 2005 09:31:48 +0200
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <1116875953.7332.3.camel@wifi167.icsi.berkeley.edu>
References: <200505101438.j4AEck4t069243@jaguar.icir.org> <20050523175400.9bxnnly9co840swc@email.ee.ethz.ch> <1116875953.7332.3.camel@wifi167.icsi.berkeley.edu>
Message-ID: <20050524073148.GA24348@net.informatik.tu-muenchen.de>

On Mon, May 23, 2005 at 12:19 -0700, Christian wrote:

> Maybe IOSourceRegistry::FindSoonest() would be a good place to
> start digging.

As a general note, the IOSource code is still rather new and may not
yet perform optimal in all cases. The major problem is that the
precise semantics of the libpcap interface varies depending on OS
version and libpcap version (and patches to either of them). I would
not be suprised if the problem w/ two Linux interfaces is related to
this.

I'd be happy to help sorting this out. However, I don't have a Linux
machine at hand which has two interfaces seeing a high-volume stream
(and I am a bit tight on time right now...).

Robin

-- 
Robin Sommer * Room        01.08.055 * www.net.in.tum.de
TU Muenchen  * Phone (089) 289-18006 *  sommer at in.tum.de 


From christian at whoop.org  Tue May 24 11:19:30 2005
From: christian at whoop.org (Christian Kreibich)
Date: Tue, 24 May 2005 11:19:30 -0700
Subject: [Bro] detect Ack flooding attack
In-Reply-To: <20050523225252.5hqy409jc40og00k@mail.cs.ucf.edu>
References: <200505181809.j4II9NBG044438@jaguar.icir.org>
	 <20050523225252.5hqy409jc40og00k@mail.cs.ucf.edu>
Message-ID: <1116958771.7332.101.camel@wifi167.icsi.berkeley.edu>

Hi Bing,

On Mon, 2005-05-23 at 22:52 -0400, bchen at cs.ucf.edu wrote:
>    By the way, the latest version just released looks much slower than previous
> version in my machine (Linux).

can you qualify this statement a bit? How does it seem much slower?

Cheers,
Christian.
-- 
________________________________________________________________________
                                          http://www.cl.cam.ac.uk/~cpk25
                                                    http://www.whoop.org




From bchen at cs.ucf.edu  Wed May 25 13:45:42 2005
From: bchen at cs.ucf.edu (bchen at cs.ucf.edu)
Date: Wed, 25 May 2005 16:45:42 -0400
Subject: [Bro] Re: detect Ack flooding attack
Message-ID: <20050525164542.chixaida80g80w40@mail.cs.ucf.edu>

Hi Christian,
   I am sorry about my subjective comments. When I ran the new version at first
time, I felt it took longer than the previous one, it might be because my
computer was heavy at that time. After reading your feedback, I ran time
command on these two versions with the same trace. It turns out they took
almost the same time. So please forget my careless whisper.

Thank you for your previous helps.

Bing



> Subject: Re: [Bro] detect Ack flooding attack
> From: Christian Kreibich <christian at whoop.org>
> To: Bro List <bro at bro-ids.org>
> Date: Tue, 24 May 2005 11:19:30 -0700
>
> Hi Bing,
>
> On Mon, 2005-05-23 at 22:52 -0400, bchen at cs.ucf.edu wrote:
>>    By the way, the latest version just released looks much slower 
>> than previous
>> version in my machine (Linux).
>
> can you qualify this statement a bit? How does it seem much slower?
>
> Cheers,
> Christian.



From christian at whoop.org  Wed May 25 14:49:32 2005
From: christian at whoop.org (Christian Kreibich)
Date: Wed, 25 May 2005 14:49:32 -0700
Subject: [Bro] Re: detect Ack flooding attack
In-Reply-To: <20050525164542.chixaida80g80w40@mail.cs.ucf.edu>
References: <20050525164542.chixaida80g80w40@mail.cs.ucf.edu>
Message-ID: <1117057773.7332.282.camel@wifi167.icsi.berkeley.edu>

On Wed, 2005-05-25 at 16:45 -0400, bchen at cs.ucf.edu wrote:
> Hi Christian,
>    I am sorry about my subjective comments. When I ran the new version at first
> time, I felt it took longer than the previous one, it might be because my
> computer was heavy at that time. After reading your feedback, I ran time
> command on these two versions with the same trace.

Cool.

> It turns out they took
> almost the same time. So please forget my careless whisper.

No worries, just wanted to make sure no major hiccups found their way
into the code :)

Cheers,
Christian.
-- 
________________________________________________________________________
                                          http://www.cl.cam.ac.uk/~cpk25
                                                    http://www.whoop.org




From aashish at uiuc.edu  Tue May 31 11:30:13 2005
From: aashish at uiuc.edu (Aashish Sharma)
Date: Tue, 31 May 2005 13:30:13 -0500
Subject: [Bro] Problem: Bro listening on two ethernet interfaces
In-Reply-To: <429387B3.3080805@lbl.gov>
Message-ID: <20050531183013.GA11522@uiuc.edu>

Jason : 

Thanks for all the help.

Here is the output you'd ask for: 

[/tmp]# bro -i eth2 -i eth3 mt.bro print-filter.bro
listening on eth2
listening on eth3
Reading .state/state.bst ...
((((((((port 111) or (port telnet or tcp port 513)) or (port finger)) or ((ip[6:2] & 0x3fff != 0) and tcp)) or (tcp[13] & 7 != 0)) or (udp port 69)) or (por
t ftp)) or (tcp port 113)) or (udp port 123)


[/tmp]# /usr/local/bro/bin/bro -i eth2 -i eth3 mt.bro
listening on eth2
listening on eth3
Reading .state/state.bst ...
1117554540.647671 received termination signal
3374 packets received on interface eth2, 479 dropped
3602 packets received on interface eth3, 610 dropped

[ Re-ran the cammond ] 

[/tmp]# /usr/local/bro/bin/bro -i eth2 -i eth3 mt.bro
listening on eth2
listening on eth3
1117554872.292885 received termination signal
2297 packets received on interface eth2, 50 dropped
2435 packets received on interface eth3, 48 dropped



[/tmp]#  /usr/bin/time tcpdump -c 25000 -i eth2 -n -w /dev/null
tcpdump: listening on eth2
26277 packets received by filter
1270 packets dropped by kernel
0.04user 0.25system 0:03.88elapsed 7%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (189major+123minor)pagefaults 0swaps


[/tmp]#  /usr/bin/time tcpdump -c 25000 -i eth3 -n -w /dev/null
tcpdump: listening on eth3
25188 packets received by filter
184 packets dropped by kernel
0.04user 0.12system 0:04.17elapsed 3%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (189major+123minor)pagefaults 0swaps


Also, Here is bro capture filter which is seen in the info.log once I start bro. Since, we are adding some some capture filter in site policy file, I thought I should paste the capture filter too.

Bro Version: 0.9a8
Started with the following command line options:  -i eth2 -i eth3 bro
listening on eth2
listening on eth3
Reading .state/state.bst ...
Capture filter: (((((((((((((((((((((port 53) or (port smtp)) or (port 111)) or (port 111)) or (port 143)) or (tcp src port 3128 or tcp src port 3120)) or (port smtp)) or (tcp[2:2] > 32770 and tcp[2:2] < 32901 and tcp[0:2] != 80 and tcp[0:2] != 22 and tcp[0:2] != 139)) or (port ftp)) or (port 161 or port 162)) or ( icmp)) or ((ip[6:2] & 0x3fff != 0) and tcp)) or (tcp dst port 3128 or tcp dst port 3120)) or (tcp port 80 or tcp port 8080 or tcp port 8000 or tcp port 8001)) or (port telnet or tcp port 513)) or (port ftp)) or (tcp[13] & 7 != 0)) or (tcp port 514)) or (port 512 or port 513 or port 515)) or (udp port 69)) or (port telnet)) or ((src net 141.142.0.0/16) and (dst port 135 or dst port 137 or dst port 139 or dst port 445))

Thanks a lot for looking into this. 

Aashish 


On Tue, May 24, 2005 at 12:59:47PM -0700, Jason Lee wrote:
> 
> Aashish,
> 
> Maybe I can help.
> 
>  Can you give me a feel for how much traffic you are looking at?
>  Number of pkts per second per interface.
>  What is the filter that you are running bro with?
>  (ie. bro -i eth1 -i eth2 mt.bro print-filter.bro)
>  should print the filter (sub in your filter for where I have mt.bro above)
> 
>   What kinds of numbers you get if you control-c it after
>   a 30 seconds or so:
> 
> [/tmp]# /usr/local/bro/bin/bro -i eth2 -i eth3 mt.bro
> pcap bufsize = 8192
> listening on eth2
> pcap bufsize = 8192
> listening on eth3
> 1116964082.421280 received termination signal
> 27771 packets received on interface eth2, 0 dropped
> 54353 packets received on interface eth3, 0 dropped
> 
> 
> And maybe  snaps of what packet rates you are seeing
> on the interfaces:
> 
> [/tmp]# /usr/bin/time tcpdump -c 25000 -i eth3 -n -w /dev/null
> tcpdump: listening on eth3
> 25012 packets received by filter
> 0 packets dropped by kernel
> 0.02user 0.13system 0:03.56elapsed 4%CPU
> 
> /tmp]# /usr/bin/time tcpdump -c 25000 -i eth2 -n -w /dev/null
> tcpdump: listening on eth2
> 25006 packets received by filter
> 0 packets dropped by kernel
> 0.00user 0.07system 0:04.44elapsed 1%CPU
> 
> 
> Cheers,
> jason
> 
>  
> 
> 
> Aashish Sharma wrote:
> 
> >>Look for AckAboveHole and ContentGap notices.  These can both happen for
> >>other reasons, but primarily happen due to dropped packets.
> >>    
> >>
> >
> >I looked in previous logs and found no alerts what so ever for AckAboveHole and ContentGap to realize that brolite had notice_ignore for them. 
> >
> ># Remove these notices from logging since they can be too noisy.
> >redef notice_action_filters += {
> >        [[ContentGap, AckAboveHole]] = notice_ignore,
> >};
> >
> >Which I changed to file_notice. 
> >        [[ContentGap, AckAboveHole]] = file_notice,
> >
> >
> >So. in last 1h15m I have 165065 ContentGap and 36596 AckAboveHole notices. Is this normal (noisy?) or is this the traffic getting dropped ? 
> >
> >And by getting dropped - do we mean that bro is not processing this traffic ? 
> >
> >2) Also Can I put it this way : 
> >
> >Of all traffic we get on interfaces we filter a part using bro filter (packetdrop notices) and then there are dropped packets (AckAboveHole and ContentGap notices)  and then rest of the traffic is processed by bro ? 
> >
> >
> >  
> >
> >>I'm not sure what you mean by "because of bro filter", but those reports
> >>    
> >>
> >Here, I meant is since we have applied bro-filter, all these packet drops could be accounted for. (they are dropped by the bro-filter) 
> >
> >
> >Thanks a lot for all the help. We have been constantly tuning bro and results are great. 
> >
> >
> >Aashish 
> >
> >
> >
> >On Thu, May 19, 2005 at 12:08:36PM -0700, Vern Paxson wrote:
> >  
> >
> >>>t=3D1116392591.523558 no=3DDroppedPackets na=3DNOTICE_FILE msg=3D4475\ pack=
> >>>ets\ dropped\ after\ filtering,\ 21924\ received
> >>>
> >>>Looking at the policy file (netstats.bro) I am inclined to think that these=
> >>> notices are generated because of bro filter. Please correct me here.=20
> >>>      
> >>>
> >>I'm not sure what you mean by "because of bro filter", but those reports
> >>are generated based on retrieving statistics from libpcap, and "after
> >>filtering" means after applying the filter Bro specified to pcap, which
> >>you can see using print-filter.bro.
> >>
> >>    
> >>
> >>>What I cannot answer/understand right now is:
> >>>
> >>>Is there any way I can find out is bro actually dropping packets, if at all=
> >>> ?
> >>>      
> >>>
> >>Look for AckAboveHole and ContentGap notices.  These can both happen for
> >>other reasons, but primarily happen due to dropped packets.
> >>
> >>		Vern
> >>_______________________________________________
> >>Bro mailing list
> >>bro at bro-ids.org
> >>http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >>    
> >>