From mangel12321 at hotmail.com  Mon Aug  4 01:33:18 2008
From: mangel12321 at hotmail.com (Miguel Angel Calvo Moya)
Date: Mon, 4 Aug 2008 10:33:18 +0200
Subject: [Bro] signarture dst-port issue
Message-ID: <BLU130-W798FDC80C818195E29E43F0780@phx.gbl>


Hello,
 
I am having trouble using signatures on bro. Lets say we have the following signature
 
signature s2b-99999-9 {
  ip-proto == tcp
  #dst-port == 80
  #dst-port == http_ports
  tcp-state established
  event "Sample Signature"
} 
 
when I using "dst-port == 80" everything goes seems to work. By 'work' I mean that bro generates plenty of warnings placing them on signatures.log, however, if we replace "st-port == 80" for "dst-port == http_ports" and throw it again against the same trace, we no longer get any warnings.
 
http_ports is declared on $BROPATH/policy/snort.bro as:
const http_ports = { 80/tcp, 8000/tcp, 8001/tcp, 8080/tcp };
 
I assume snort..bro is loaded correctly since otherwise throws non-declaration errors. It is loaded it on my hostname.bro file which I use when running bro. 

Also, does anybody know whether next release will implement other regular expression matching library? I am trying to update snort2bro to support the snort pcre, however it is not a trivial task.
 
Any suggestions?
Thank you!
Miguel


From robin at icir.org  Mon Aug  4 10:52:46 2008
From: robin at icir.org (Robin Sommer)
Date: Mon, 4 Aug 2008 10:52:46 -0700
Subject: [Bro] signarture dst-port issue
In-Reply-To: <BLU130-W798FDC80C818195E29E43F0780@phx.gbl>
References: <BLU130-W798FDC80C818195E29E43F0780@phx.gbl>
Message-ID: <20080804175246.GA51343@icir.org>


On Mon, Aug 04, 2008 at 10:33 +0200, Miguel Angel Calvo Moya wrote:

> I am having trouble using signatures on bro. Lets say we have the following signature

Can you send me a small trace with which you see the problem and the
exect command line you're using to start Bro? 

> Also, does anybody know whether next release will implement other
> regular expression matching library?

No, we don't have any plans to switch to another regexp library.

>  I am trying to update snort2bro to support the snort pcre, however
>  it is not a trivial task.

Cool! but yeah, not exactly trivial. :)

Robin

-- 
Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org 
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From geek00l at gmail.com  Mon Aug  4 18:02:00 2008
From: geek00l at gmail.com (CS Lee)
Date: Tue, 5 Aug 2008 09:02:00 +0800
Subject: [Bro] Bro Digest, Vol 28, Issue 2
In-Reply-To: <mailman.1.1217876403.1965.bro@ICSI.Berkeley.EDU>
References: <mailman.1.1217876403.1965.bro@ICSI.Berkeley.EDU>
Message-ID: <1bb5dd90808041802r1361445eq2706787023a353d8@mail.gmail.com>

hi miguel,

If u are using hostname.bro which may load brolite.bro(it will load
http.bro), and http_ports is defined in http.bro as well, you may find the
following lines -

# DPM configuration.
# global http_ports = {
#       80/tcp, 81/tcp, 631/tcp, 3138/tcp,
#       8000/tcp, 8080/tcp, 8888/tcp,
# } &redef;

I comment them out, instead in snort.bro I comment out one line and add the
similar config in http.bro

#const http_ports = { 80/tcp, 8000/tcp, 8001/tcp, 8080/tcp };
global http_ports = {
        80/tcp, 81/tcp, 631/tcp, 3128/tcp,
        8000/tcp, 8080/tcp, 8888/tcp,
} &redef;

Then try to run it again and see if it works.

Cheers


On Tue, Aug 5, 2008 at 3:00 AM, <bro-request at icsi.berkeley.edu> wrote:

> Send Bro mailing list submissions to
>        bro at ICSI.Berkeley.EDU
>
> To subscribe or unsubscribe via the World Wide Web, visit
>        http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> or, via email, send a message with subject or body 'help' to
>        bro-request at ICSI.Berkeley.EDU
>
> You can reach the person managing the list at
>        bro-owner at ICSI.Berkeley.EDU
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Bro digest..."
>
>
> Today's Topics:
>
>   1. signarture dst-port issue (Miguel Angel Calvo Moya)
>   2. Re: signarture dst-port issue (Robin Sommer)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 4 Aug 2008 10:33:18 +0200
> From: Miguel Angel Calvo Moya <mangel12321 at hotmail.com>
> Subject: [Bro] signarture dst-port issue
> To: <bro at ICSI.Berkeley.EDU>
> Message-ID: <BLU130-W798FDC80C818195E29E43F0780 at phx.gbl>
> Content-Type: text/plain; charset="iso-8859-1"
>
>
> Hello,
>
> I am having trouble using signatures on bro. Lets say we have the following
> signature
>
> signature s2b-99999-9 {
>  ip-proto == tcp
>  #dst-port == 80
>  #dst-port == http_ports
>  tcp-state established
>  event "Sample Signature"
> }
>
> when I using "dst-port == 80" everything goes seems to work. By 'work' I
> mean that bro generates plenty of warnings placing them on signatures.log,
> however, if we replace "st-port == 80" for "dst-port == http_ports" and
> throw it again against the same trace, we no longer get any warnings.
>
> http_ports is declared on $BROPATH/policy/snort.bro as:
> const http_ports = { 80/tcp, 8000/tcp, 8001/tcp, 8080/tcp };
>
> I assume snort..bro is loaded correctly since otherwise throws
> non-declaration errors. It is loaded it on my hostname.bro file which I use
> when running bro.
>
> Also, does anybody know whether next release will implement other regular
> expression matching library? I am trying to update snort2bro to support the
> snort pcre, however it is not a trivial task.
>
> Any suggestions?
> Thank you!
> Miguel
>
>
> ------------------------------
>
> Message: 2
> Date: Mon, 4 Aug 2008 10:52:46 -0700
> From: Robin Sommer <robin at icir.org>
> Subject: Re: [Bro] signarture dst-port issue
> To: bro at ICSI.Berkeley.EDU, Miguel Angel Calvo Moya
>        <mangel12321 at hotmail.com>
> Message-ID: <20080804175246.GA51343 at icir.org>
> Content-Type: text/plain; charset=us-ascii
>
>
> On Mon, Aug 04, 2008 at 10:33 +0200, Miguel Angel Calvo Moya wrote:
>
> > I am having trouble using signatures on bro. Lets say we have the
> following signature
>
> Can you send me a small trace with which you see the problem and the
> exect command line you're using to start Bro?
>
> > Also, does anybody know whether next release will implement other
> > regular expression matching library?
>
> No, we don't have any plans to switch to another regexp library.
>
> >  I am trying to update snort2bro to support the snort pcre, however
> >  it is not a trivial task.
>
> Cool! but yeah, not exactly trivial. :)
>
> Robin
>
> --
> Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org
> ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
>
>
> ------------------------------
>
> _______________________________________________
> Bro mailing list
> Bro at ICSI.Berkeley.EDU
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> End of Bro Digest, Vol 28, Issue 2
> **********************************
>



-- 
Best Regards,

CS Lee<geek00L[at]gmail.com>

http://geek00l.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080805/aecdc449/attachment.html 

From mangel12321 at hotmail.com  Tue Aug  5 01:42:18 2008
From: mangel12321 at hotmail.com (Miguel Angel Calvo Moya)
Date: Tue, 5 Aug 2008 10:42:18 +0200
Subject: [Bro] Bro Digest, Vol 28, Issue 2
In-Reply-To: <1bb5dd90808041802r1361445eq2706787023a353d8@mail.gmail.com>
References: <mailman.1.1217876403.1965.bro@ICSI.Berkeley.EDU>
	<1bb5dd90808041802r1361445eq2706787023a353d8@mail.gmail.com>
Message-ID: <BLU130-W25DDEC08BF9987C50E5A2AF07B0@phx.gbl>


> If u are using hostname.bro which may load brolite.bro(it will load http.bro), and http_ports is defined in http.bro as well, you may find the following lines -
> 
> # DPM configuration.
> # global http_ports = {
> #       80/tcp, 81/tcp, 631/tcp, 3138/tcp,
> #       8000/tcp, 8080/tcp, 8888/tcp,
> # } &redef;
> 
> I comment them out, instead in snort.bro I comment out one line and add the similar config in http.bro
> 
> #const http_ports = { 80/tcp, 8000/tcp, 8001/tcp, 8080/tcp };
> global http_ports = {
>         80/tcp, 81/tcp, 631/tcp, 3128/tcp,
>         8000/tcp, 8080/tcp, 8888/tcp,
> } &redef;
> 
> Then try to run it again and see if it works.

As you suggested I declared http_posts at snort.bro as 'global' instead of 'const'. It did not work. Fortunately, when declaring http_ports as

global http_ports = { 
         80, 81, 631, 3128,
         8000, 8080, 8888
} &redef

without '/tcp' the signature works. Since 'ip-proto == tcp' is already set, the results will not differ, am I right?

Thank you
Miguel


From robin at icir.org  Tue Aug  5 17:07:47 2008
From: robin at icir.org (Robin Sommer)
Date: Tue, 5 Aug 2008 17:07:47 -0700
Subject: [Bro] Bro Digest, Vol 28, Issue 2
In-Reply-To: <BLU130-W25DDEC08BF9987C50E5A2AF07B0@phx.gbl>
References: <mailman.1.1217876403.1965.bro@ICSI.Berkeley.EDU>
	<1bb5dd90808041802r1361445eq2706787023a353d8@mail.gmail.com>
	<BLU130-W25DDEC08BF9987C50E5A2AF07B0@phx.gbl>
Message-ID: <20080806000747.GJ58005@icir.org>


On Tue, Aug 05, 2008 at 10:42 +0200, you wrote:

> without '/tcp' the signature works.

Ah, I see. That's a bug. Can you try the attached patch and see if
that solves the problem?

Robin

-- 
Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org 
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
-------------- next part --------------
Index: RuleMatcher.cc
===================================================================
--- RuleMatcher.cc	(revision 6040)
+++ RuleMatcher.cc	(working copy)
@@ -1048,11 +1048,15 @@
 	MaskedValue* mval = new MaskedValue;
 
 	switch ( v->Type()->Tag() ) {
+		case TYPE_PORT:
+			mval->val = v->AsPortVal()->Port();
+			mval->mask = 0xffffffff;
+			break;
+
 		case TYPE_BOOL:
 		case TYPE_COUNT:
 		case TYPE_ENUM:
 		case TYPE_INT:
-		case TYPE_PORT:
 			mval->val = v->CoerceToUnsigned();
 			mval->mask = 0xffffffff;
 			break;

From mangel12321 at hotmail.com  Wed Aug  6 02:59:07 2008
From: mangel12321 at hotmail.com (Miguel Angel Calvo Moya)
Date: Wed, 6 Aug 2008 11:59:07 +0200
Subject: [Bro] Bro Digest, Vol 28, Issue 2
In-Reply-To: <20080806000747.GJ58005@icir.org>
References: <mailman.1.1217876403.1965.bro@ICSI.Berkeley.EDU>
	<1bb5dd90808041802r1361445eq2706787023a353d8@mail.gmail.com>
	<BLU130-W25DDEC08BF9987C50E5A2AF07B0@phx.gbl>
	<20080806000747.GJ58005@icir.org>
Message-ID: <BLU130-W294F405640EE9B50EB4B28F07A0@phx.gbl>

> > without '/tcp' the signature works.
> 
> Ah, I see. That's a bug. Can you try the attached patch and see if
> that solves the problem?

Thank you Robin,



The patch did work.



Thank you

Miguel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080806/59655c3e/attachment.html 

From geek00l at gmail.com  Wed Aug  6 18:46:16 2008
From: geek00l at gmail.com (CS Lee)
Date: Thu, 7 Aug 2008 09:46:16 +0800
Subject: [Bro] bro 1.4 pre release
Message-ID: <1bb5dd90808061846k17ab1b99s4b19c0446f2488d0@mail.gmail.com>

hi robin,

I encounter problem while installing pybroccoli

shell>python setup.py install
..........
building '_broccoli_intern' extension
creating build/temp.freebsd-7.0-RELEASE-p2-i386-2.5
cc -fno-strict-aliasing -DNDEBUG -O2 -fno-strict-aliasing -pipe
-D__wchar_t=wchar_t -DTHREAD_STACK_SIZE=0x20000 -fPIC
-I/usr/local/include/python2.5 -c broccoli_intern_wrap.c -o
build/temp.freebsd-7.0-RELEASE-p2-i386-2.5/broccoli_intern_wrap.o
broccoli_intern_wrap.c:2514: warning: useless storage class specifier in
empty declaration
broccoli_intern_wrap.c: In function '_wrap_bro_event_add_val':
broccoli_intern_wrap.c:4976: warning: assignment discards qualifiers from
pointer target type
broccoli_intern_wrap.c: In function '_wrap_bro_event_set_val':
broccoli_intern_wrap.c:5046: warning: assignment discards qualifiers from
pointer target type
broccoli_intern_wrap.c: In function '_wrap_bro_record_add_val':
broccoli_intern_wrap.c:6131: warning: assignment discards qualifiers from
pointer target type
broccoli_intern_wrap.c: In function '_wrap_bro_record_set_nth_val':
broccoli_intern_wrap.c:6286: warning: assignment discards qualifiers from
pointer target type
broccoli_intern_wrap.c: In function '_wrap_bro_record_set_named_val':
broccoli_intern_wrap.c:6357: warning: assignment discards qualifiers from
pointer target type
cc -shared -pthread
build/temp.freebsd-7.0-RELEASE-p2-i386-2.5/broccoli_intern_wrap.o
-L../../src/.libs -lbroccoli -o
build/lib.freebsd-7.0-RELEASE-p2-i386-2.5/_broccoli_intern.so
running install_lib
copying build/lib.freebsd-7.0-RELEASE-p2-i386-2.5/broccoli.py ->
/usr/local/lib/python2.5/site-packages
copying build/lib.freebsd-7.0-RELEASE-p2-i386-2.5/_broccoli_intern.so ->
/usr/local/lib/python2.5/site-packages
byte-compiling /usr/local/lib/python2.5/site-packages/broccoli.py to
broccoli.pyc
running install_egg_info
Writing /usr/local/lib/python2.5/site-packages/pybroccoli-0.1-py2.5.egg-info

shell>python -c "import broccoli"

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "broccoli.py", line 6, in <module>
    from _broccoli_intern import *
ImportError: /usr/local/lib/python2.5/site-packages/_broccoli_intern.so:
Undefined symbol "BIO_ctrl"

Thanks.

-- 
Best Regards,

CS Lee<geek00L[at]gmail.com>

http://geek00l.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080807/ac6548ad/attachment.html 

From robin at icir.org  Thu Aug  7 10:14:39 2008
From: robin at icir.org (Robin Sommer)
Date: Thu, 7 Aug 2008 10:14:39 -0700
Subject: [Bro] bro 1.4 pre release
In-Reply-To: <1bb5dd90808061846k17ab1b99s4b19c0446f2488d0@mail.gmail.com>
References: <1bb5dd90808061846k17ab1b99s4b19c0446f2488d0@mail.gmail.com>
Message-ID: <20080807171439.GC337@icir.org>


On Thu, Aug 07, 2008 at 09:46 +0800, you wrote:

> I encounter problem while installing pybroccoli

Thanks for the note, that looks like a problem with linking in
OpenSSL. I'll take a look and then might get back to you for more
details. 

Robin

-- 
Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org 
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From dkopecek at redhat.com  Sat Aug  9 11:23:55 2008
From: dkopecek at redhat.com (Daniel Kopecek)
Date: Sat, 9 Aug 2008 20:23:55 +0200
Subject: [Bro] License
Message-ID: <20080809202355.4a9b5ec4@dhcp-lab-189.englab.brq.redhat.com>

Hi,
 I'm a little confused about the license under which is Bro distributed.
Is it "BSD" or "BSD with advertising"? It looks like that in the
COPYING file is "BSD with advertising" and on
http://bro-ids.org/license.html "BSD". Could you please tell me
which one is the right one?

Thanks,
Dan


From robin at icir.org  Mon Aug 11 09:44:35 2008
From: robin at icir.org (Robin Sommer)
Date: Mon, 11 Aug 2008 09:44:35 -0700
Subject: [Bro] License
In-Reply-To: <20080809202355.4a9b5ec4@dhcp-lab-189.englab.brq.redhat.com>
References: <20080809202355.4a9b5ec4@dhcp-lab-189.englab.brq.redhat.com>
Message-ID: <20080811164435.GD69565@icir.org>


On Sat, Aug 09, 2008 at 20:23 +0200, you wrote:

> I'm a little confused about the license under which is Bro distributed.
> Is it "BSD" or "BSD with advertising"? It looks like that in the
> COPYING file is "BSD with advertising" and on
> http://bro-ids.org/license.html "BSD".

Good catch, I never noticed that that's inconsistent. As "BSD with
advertising" comes with the source, and the Web site came later, I
suppose that's the one which applies. But that's for Vern to answer
authoritatively (he's out at so that might take a bit). 

Robin

-- 
Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org 
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From adayadil.thomas at gmail.com  Mon Aug 11 12:11:15 2008
From: adayadil.thomas at gmail.com (Adayadil Thomas)
Date: Mon, 11 Aug 2008 15:11:15 -0400
Subject: [Bro] Overlaps Question
Message-ID: <fb7befa20808111211q37b7b6a8k4cce1207ac730476@mail.gmail.com>

Hello and Greetings !

I have a question on overlaps - TCP segment overlaps and IP fragments
overlap - how common they are
and how legitimate?

AFAIK, TCP segmentation overlaps can be seen in normal traffic and by
themselves cannot be deemed
malicious.

Is IP fragmentation overlap abnormal always? What is a scenario when
this can happen in a normal scenario?

Thanks in advance for the reply !


From robin at icir.org  Mon Aug 11 16:10:16 2008
From: robin at icir.org (Robin Sommer)
Date: Mon, 11 Aug 2008 16:10:16 -0700
Subject: [Bro] Bro 1.4, another pre-release
In-Reply-To: <200808010619.m716JHId001843@pork.ICSI.Berkeley.EDU>
References: <200808010619.m716JHId001843@pork.ICSI.Berkeley.EDU>
Message-ID: <20080811231016.GC5066@icir.org>


We have made another Bro 1.4 pre-release available at 

   http://www.icir.org/robin/tmp/bro-1.4.prerelease.1.tar.gz

Let us know if you encounter anything unexpected.

Since the earlier pre-release, some minor issues have been fixed
(mainly platform incompatibilities).  

Robin

-- 
Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org 
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From mangel12321 at hotmail.com  Tue Aug 12 09:13:22 2008
From: mangel12321 at hotmail.com (Miguel Angel Calvo Moya)
Date: Tue, 12 Aug 2008 18:13:22 +0200
Subject: [Bro] Bro 1.4, sinatures.bro policy warning
Message-ID: <BLU130-W326BFA4EB6FD0AF19AE41F0700@phx.gbl>


Hello Robin,

When using bro on offline mode and loading signatures.bro policy I get the following warning

suse:/usr/local/bro # export BROPATH="/usr/local/bro/site:/usr/local/bro/share/bro"
suse:/usr/local/bro # export BROHOME="/usr/local/bro"
suse:/usr/local/bro # bro -r /tmp/trace.pcap signatures
line 1: warning: event handlers never invoked:
line 1: warning:         Drop::restore_dropped_address

BROPATH to points at the new policy folder.
BROPATH="/usr/local/bro/site:/usr/local/bro/share/bro"

It happens with every trace I've tried, but just in case here I'm attaching the sample used on the example.

Also I noticed there is no longer a 'site' folder. Where would be the right place to place our  host-specific Bro policy file?

Best Regards

Miguel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: trace.pcap
Type: application/octet-stream
Size: 920 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080812/fd8a8c63/attachment.obj 

From robin at icir.org  Tue Aug 12 14:17:50 2008
From: robin at icir.org (Robin Sommer)
Date: Tue, 12 Aug 2008 14:17:50 -0700
Subject: [Bro] [Fwd] Re:  Bro 1.4, another pre-release
Message-ID: <20080812211750.GA53517@icir.org>

(Below a reply I sent earlier in private mail. Miguel, please don't
send a question separately both in private mail and on the list,
thanks.)

Robin

----- Forwarded message from Robin Sommer <robin at icir.org> -----

From: Robin Sommer <robin at icir.org>
Date: Tue, 12 Aug 2008 10:09:25 -0700
Subject: Re: [Bro] Bro 1.4, another pre-release
To: Miguel Angel Calvo Moya <mangel12321 at hotmail.com>
Message-ID: <20080812170925.GC5416 at icir.org>


On Tue, Aug 12, 2008 at 18:03 +0200, you wrote:

> line 1: warning: event handlers never invoked:
> line 1: warning:         Drop::restore_dropped_address

This is actually expected and harmless. It says that Bro sees an
handler for an event yet based on the scripts you load can tell that
the event will never actually be raised. In this case, that's
because your configuration never calls the function
Drop::drop_address (which is a rare thing to do anyway).

There's a switch to suppress these warnings:

        # If true, warns about unused event handlers at startup.
        const check_for_unused_event_handlers = T &redef;

I've already been wondering whether we should change this to off by
default. 

> Also I noticed there is no longer a 'site' folder. Where would be
> the right place to place our host-specific Bro policy file?

Hmmm... Good question, didn't think about that when redoing the
default installation. How about $prefix/share/bro/site? Looks like 
"make install" should then create that directory if it doesn't
exist.

Robin

-- 
Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org 
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

----- End forwarded message -----





-- 
Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org 
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From geek00l at gmail.com  Thu Aug 14 21:02:36 2008
From: geek00l at gmail.com (CS Lee)
Date: Fri, 15 Aug 2008 12:02:36 +0800
Subject: [Bro] time machine
Message-ID: <1bb5dd90808142102q1c84e69cg8a8851028fd6968d@mail.gmail.com>

hi robin,

I just notice the new version of tm is out, the download link is correct but
the name to click for download is wrong

20060814

I think it should be 20080814?

Cheers ;]

-- 
Best Regards,

CS Lee<geek00L[at]gmail.com>

http://geek00l.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080815/2ef3976c/attachment.html 

From Stephan at rheoli.net  Mon Aug 18 04:50:01 2008
From: Stephan at rheoli.net (Stephan)
Date: Mon, 18 Aug 2008 13:50:01 +0200
Subject: [Bro] Bro 1.4, another pre-release (Solaris Patch)
In-Reply-To: <20080811231016.GC5066@icir.org>
References: <200808010619.m716JHId001843@pork.ICSI.Berkeley.EDU>
	<20080811231016.GC5066@icir.org>
Message-ID: <20080818115000.GA32317@rheoli.net>

Hi,

I the attachment you find some patches for Solaris.
It runs then on Solaris 8 + 10.

The first definition in any bro rule should be:
redef use_connection_compressor = F;

Patch description:
 - patch-aux-nftools-nfcommin.h:
    include the main config.h for some u_int_(8|16|32)
    definitions who are not defined in Solaris.
 - patch-src-ARP.h:
    without including net/if.h there are no compile errors
 - patch-src-Conn.h:
    some changes (from Robin) to switch on compressor
 - patch-src-Desc.cc + patch-src-FlowSrc.cc:
    include errno.h otherwise the compiler could not find
    errno
 - patch-src-Makefile.in:
    the configure in libedit miss the termcap.h in the include path
    so it will be faked

After this patches it runs on Solaris 8 with the default rules or 
to extract SSL certificates.


Best regards,
Stephan


On Mon, Aug 11, 2008 at 04:10:16PM -0700, Robin Sommer wrote:
> 
> We have made another Bro 1.4 pre-release available at 
> 
>    http://www.icir.org/robin/tmp/bro-1.4.prerelease.1.tar.gz
> 
> Let us know if you encounter anything unexpected.
> 
> Since the earlier pre-release, some minor issues have been fixed
> (mainly platform incompatibilities).  
> 
> Robin
> 
> -- 
> Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org 
> ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bro-1.4.prerelease.1.tar.gz
Type: application/x-tar-gz
Size: 1347 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080818/65c6a290/attachment.bin 

From msz at seclab.tuwien.ac.at  Tue Aug 19 13:07:04 2008
From: msz at seclab.tuwien.ac.at (Martin Szydlowski)
Date: Tue, 19 Aug 2008 22:07:04 +0200
Subject: [Bro] weird behavior in binpac-generated analyzer
Message-ID: <48AB27E8.2020509@seclab.tuwien.ac.at>

Hi all,

I am extending the BitTorrent analyzer (kudos to Nadi Sarrar) that has
finally made it into the 1.4.prerelease to handle messages beyond the
standard protocol. In particular, I am including the Azureus Messaging
protocol and there's where the trouble begins. When "switching" from
standard to azureus protocol, the analyzer consistently swallows the
first 4 bytes in the upflow direction. The downflow is unaffected and
the analysis works fine there and it does not matter if the first
message after the switch was upflow or downflow, it is always the first
upflow message that is affected.

The effect of the swallow is that the analyzer misses the 4-byte length
field and takes the first four bytes of the following string instead.
That produces a huge number that causes an out_of_bound exception and
stops the anaylsis for the upflow.

I did some debugging with gdb and found that FlowBuffer::NewMessage
in aux/binpac/lib/binpac_buffer.cc increases the orig_data_begin_
pointer by 4 when it is actually pointing at the correct spot. Either
the increment is wrong, or the data pointer is 4 bytes to far already, I
cannot tell.

I have only made changes to ,pac (and .bro) files, therefore all the
code is binpac-generated. As long as the network dumps do not contain
azureus traffic, my code does not get invoked, and the analysis runs
fine, therefore the fault must lie in my changes. However, since the
code for upflow and downflow is obviously identical and only upflow is
affected by this error, maybe my code triggers some obscure bug in
binpac or bro. Oh and BTW, I also tried disabling the connection
compressor, since in the past that caused trouble with localhost
connections (but that bug should be fixed by now).

I am attaching a diff of my changes (against SVN revision 6054).
The dumps I have been testing against can be found at
http://hades.seclab.tuwien.ac.at/msz/azu.dump and
http://hades.seclab.tuwien.ac.at/msz/azu2.dump (both ~30MB).
When using this dumps, you must use the -C flag since the tcp checksums
are invalid (as always when capturing traffic on the originating host).
(My invocation of bro is: $ bro -C -r xxx.dump bittorrent bt-tracker weird)

I hope someone can look at this and tell me what is going wrong.

greetz Martin


-------------- next part --------------
A non-text attachment was scrubbed...
Name: r6054v4.diff.zip
Type: application/x-zip-compressed
Size: 3270 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080819/e7ee5e7b/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 258 bytes
Desc: OpenPGP digital signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080819/e7ee5e7b/attachment-0001.bin 

From msz at seclab.tuwien.ac.at  Wed Aug 20 08:02:34 2008
From: msz at seclab.tuwien.ac.at (Martin Szydlowski)
Date: Wed, 20 Aug 2008 17:02:34 +0200
Subject: [Bro]  weird behavior in binpac-generated analyzer
Message-ID: <48AC320A.8030204@seclab.tuwien.ac.at>

Hello again,

Half a day more of GDB have unearthed the root cause of this problem. I
am posting this, since this is behavior that potential developers of Bro
analyzers will want to know about. Or possibly, it is a bug.

In my binpac grammar, at one point I need to decide which rule to take
based on the state of both upflow and downflow. The first payload packet
(handshake) comes from upflow and, while processing, I check if a bit is
set. The second packet processed is the handshake from downflow where
the bit is also checked. For any subseqent packets/messages a different
rule is taken depending on if both bits are set or not.

However, Bro does something (at least for me) unexpected. After
processing the upflow handshake, it starts processing the next message
in this flow, although the data is not there yet. Since the downflow
message has not been processed yet the flag is not set, and Bro decides
(somehow correctly) that the next message is going to be of the standard
type. It creates a message object, and sets the FlowBuffer frame size to
4, since this is how long the next message is at least supposed to be.
Then it stops processing since the data isn't actually there, and starts
working on the downflow.

When the processing comes back to the upflow, the downflow handshake has
been parsed and the flag set. Therefore, Bro does not reuse the message
object, but creates a new one for the other type of message. However,
the state in FlowBuffer (namely the frame size) is not reset, and the
first step in parsing the new message is to skip the 4 bytes from the
(actually nonexistent) previous message which effectively swallows the
first 4 bytes of the current message - and that causes the parsing to
use the wrong fields and in the end causes an out_of_bounds exception
that stops the processing.

I my grammar, I did not expect processing of messages before they arrive
and it should work perfectly fine when the messages are processed in the
same order they appear in the dump file. I'm curious if the observed
behavior is some sort of optimization that you just need to consider
when writing binpac grammars (and even more wondering if for some cases
it's possible to write the grammar in the required way) or if protocols
where both sides contribute to the state have not been considered when
designing binpac. I have noticed that the Bro wiki mentions the
$context.flow macro but not the $context.connection macro which is also
there. For now, I am rewriting my grammar in the hope that I can
circumvent my problem, but I would welcome any comments or
clarifications from the authors or savvy developers on that issue.

greetz Martin


From rpang at cs.princeton.edu  Wed Aug 20 08:43:37 2008
From: rpang at cs.princeton.edu (Ruoming Pang)
Date: Wed, 20 Aug 2008 11:43:37 -0400
Subject: [Bro] weird behavior in binpac-generated analyzer
In-Reply-To: <48AC320A.8030204@seclab.tuwien.ac.at>
References: <48AC320A.8030204@seclab.tuwien.ac.at>
Message-ID: <e82e81920808200843n5c6ba620k150fa95e874f4a4c@mail.gmail.com>

Hi Martin,

Can you post your binpac script?

Thanks,
Ruoming

On Wed, Aug 20, 2008 at 11:02 AM, Martin Szydlowski <msz at seclab.tuwien.ac.at
> wrote:

> Hello again,
>
> Half a day more of GDB have unearthed the root cause of this problem. I
> am posting this, since this is behavior that potential developers of Bro
> analyzers will want to know about. Or possibly, it is a bug.
>
> In my binpac grammar, at one point I need to decide which rule to take
> based on the state of both upflow and downflow. The first payload packet
> (handshake) comes from upflow and, while processing, I check if a bit is
> set. The second packet processed is the handshake from downflow where
> the bit is also checked. For any subseqent packets/messages a different
> rule is taken depending on if both bits are set or not.
>
> However, Bro does something (at least for me) unexpected. After
> processing the upflow handshake, it starts processing the next message
> in this flow, although the data is not there yet. Since the downflow
> message has not been processed yet the flag is not set, and Bro decides
> (somehow correctly) that the next message is going to be of the standard
> type. It creates a message object, and sets the FlowBuffer frame size to
> 4, since this is how long the next message is at least supposed to be.
> Then it stops processing since the data isn't actually there, and starts
> working on the downflow.
>
> When the processing comes back to the upflow, the downflow handshake has
> been parsed and the flag set. Therefore, Bro does not reuse the message
> object, but creates a new one for the other type of message. However,
> the state in FlowBuffer (namely the frame size) is not reset, and the
> first step in parsing the new message is to skip the 4 bytes from the
> (actually nonexistent) previous message which effectively swallows the
> first 4 bytes of the current message - and that causes the parsing to
> use the wrong fields and in the end causes an out_of_bounds exception
> that stops the processing.
>
> I my grammar, I did not expect processing of messages before they arrive
> and it should work perfectly fine when the messages are processed in the
> same order they appear in the dump file. I'm curious if the observed
> behavior is some sort of optimization that you just need to consider
> when writing binpac grammars (and even more wondering if for some cases
> it's possible to write the grammar in the required way) or if protocols
> where both sides contribute to the state have not been considered when
> designing binpac. I have noticed that the Bro wiki mentions the
> $context.flow macro but not the $context.connection macro which is also
> there. For now, I am rewriting my grammar in the hope that I can
> circumvent my problem, but I would welcome any comments or
> clarifications from the authors or savvy developers on that issue.
>
> greetz Martin
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080820/44286d35/attachment.html 

From msz at seclab.tuwien.ac.at  Wed Aug 20 11:31:07 2008
From: msz at seclab.tuwien.ac.at (Martin Szydlowski)
Date: Wed, 20 Aug 2008 20:31:07 +0200
Subject: [Bro] weird behavior in binpac-generated analyzer
In-Reply-To: <e82e81920808200843n5c6ba620k150fa95e874f4a4c@mail.gmail.com>
References: <48AC320A.8030204@seclab.tuwien.ac.at>
	<e82e81920808200843n5c6ba620k150fa95e874f4a4c@mail.gmail.com>
Message-ID: <A01DE1D7-2597-4C10-B33E-39FABB8131F7@seclab.tuwien.ac.at>

Hello,

I have attached a zipped patch in my previous post. Since I have  
modified 4 files, a patch seems better suited than posting all the  
files. I am attaching a patch with the fixed grammar to this post. All  
patches are against revision 6054 from SVN trunk.

greetz Martin

On 20.08.2008, at 17:43, Ruoming Pang wrote:

> Hi Martin,
>
> Can you post your binpac script?
>
> Thanks,
> Ruoming

-------------- next part --------------
A non-text attachment was scrubbed...
Name: r6054v5.diff.zip
Type: application/zip
Size: 3858 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080820/7dc0bbad/attachment.zip 
-------------- next part --------------


From rpang at cs.princeton.edu  Wed Aug 20 06:40:33 2008
From: rpang at cs.princeton.edu (Ruoming Pang)
Date: Wed, 20 Aug 2008 09:40:33 -0400
Subject: [Bro] weird behavior in binpac-generated analyzer
In-Reply-To: <48AB27E8.2020509@seclab.tuwien.ac.at>
References: <48AB27E8.2020509@seclab.tuwien.ac.at>
Message-ID: <e82e81920808200640w33b845f1r1629beefba512449@mail.gmail.com>

Hi Martin,

I can help you look at the issue. The first thing to do is to extract the
exact connection that causes the problem and use tcpdump to extract a dump
that only contains the connection. I assume that there's no cross-connection
state that could affect the analysis.

Ruoming

2008/8/19 Martin Szydlowski <msz at seclab.tuwien.ac.at>

> Hi all,
>
> I am extending the BitTorrent analyzer (kudos to Nadi Sarrar) that has
> finally made it into the 1.4.prerelease to handle messages beyond the
> standard protocol. In particular, I am including the Azureus Messaging
> protocol and there's where the trouble begins. When "switching" from
> standard to azureus protocol, the analyzer consistently swallows the
> first 4 bytes in the upflow direction. The downflow is unaffected and
> the analysis works fine there and it does not matter if the first
> message after the switch was upflow or downflow, it is always the first
> upflow message that is affected.
>
> The effect of the swallow is that the analyzer misses the 4-byte length
> field and takes the first four bytes of the following string instead.
> That produces a huge number that causes an out_of_bound exception and
> stops the anaylsis for the upflow.
>
> I did some debugging with gdb and found that FlowBuffer::NewMessage
> in aux/binpac/lib/binpac_buffer.cc increases the orig_data_begin_
> pointer by 4 when it is actually pointing at the correct spot. Either
> the increment is wrong, or the data pointer is 4 bytes to far already, I
> cannot tell.
>
> I have only made changes to ,pac (and .bro) files, therefore all the
> code is binpac-generated. As long as the network dumps do not contain
> azureus traffic, my code does not get invoked, and the analysis runs
> fine, therefore the fault must lie in my changes. However, since the
> code for upflow and downflow is obviously identical and only upflow is
> affected by this error, maybe my code triggers some obscure bug in
> binpac or bro. Oh and BTW, I also tried disabling the connection
> compressor, since in the past that caused trouble with localhost
> connections (but that bug should be fixed by now).
>
> I am attaching a diff of my changes (against SVN revision 6054).
> The dumps I have been testing against can be found at
> http://hades.seclab.tuwien.ac.at/msz/azu.dump and
> http://hades.seclab.tuwien.ac.at/msz/azu2.dump (both ~30MB).
> When using this dumps, you must use the -C flag since the tcp checksums
> are invalid (as always when capturing traffic on the originating host).
> (My invocation of bro is: $ bro -C -r xxx.dump bittorrent bt-tracker weird)
>
> I hope someone can look at this and tell me what is going wrong.
>
> greetz Martin
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080820/3a6c4704/attachment.html 

From vern at icir.org  Thu Aug 21 20:12:22 2008
From: vern at icir.org (Vern Paxson)
Date: Thu, 21 Aug 2008 20:12:22 -0700
Subject: [Bro] Overlaps Question
In-Reply-To: <fb7befa20808111211q37b7b6a8k4cce1207ac730476@mail.gmail.com>
	(Mon, 11 Aug 2008 15:11:15 EDT).
Message-ID: <200808220312.m7M3CN7v002006@pork.ICSI.Berkeley.EDU>

> I have a question on overlaps - TCP segment overlaps and IP fragments
> overlap - how common they are
> and how legitimate?

TCP segment overlaps are, surprisingly, quite common.  We discuss this
in a recent paper of ours:

	Efficient and Robust TCP Stream Normalization
	M. Vutukuru, H. Balakrishnan and V. Paxson
	Proc. IEEE Symposium on Security and Privacy, May 2008
	http://www.icir.org/vern/papers/tcpnorm-oak08.pdf

Fragment overlaps definitely occur too, though the ones I've tracked down
(not many) have been due to holding fragments for a long time and the IP
ID counter rolling over (producing a new set of fragments with the same ID).
I don't know how often they occur within the fragment reassembly time window.

		Vern


From christoph.leuzinger at cs.uni-dortmund.de  Sat Aug 23 07:02:33 2008
From: christoph.leuzinger at cs.uni-dortmund.de (Christoph Leuzinger)
Date: Sat, 23 Aug 2008 16:02:33 +0200
Subject: [Bro] Compiling Broccoli on NetBSD 4
Message-ID: <20080823160233.9483a267.christoph.leuzinger@cs.uni-dortmund.de>

Hello,

I ran into a problem while building the Broccoli part of the Bro
development version (1.3.2) on NetBSD 4.0. On NetBSD, <sys/sem.h> does
not define the union semun. The attached patch fixes this build problem
for me.

Best regards,
  Christoph Leuzinger

-------------- next part --------------
A non-text attachment was scrubbed...
Name: netbsd4_broccoli_sem_sysv.patch
Type: text/x-diff
Size: 461 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080823/7da9dc36/attachment.bin 

From christoph.leuzinger at cs.uni-dortmund.de  Sat Aug 23 07:52:34 2008
From: christoph.leuzinger at cs.uni-dortmund.de (Christoph Leuzinger)
Date: Sat, 23 Aug 2008 16:52:34 +0200
Subject: [Bro] Compiling Broccoli on NetBSD 4
In-Reply-To: <20080823160233.9483a267.christoph.leuzinger@cs.uni-dortmund.de>
References: <20080823160233.9483a267.christoph.leuzinger@cs.uni-dortmund.de>
Message-ID: <20080823165234.224b2d15.christoph.leuzinger@cs.uni-dortmund.de>

Christoph Leuzinger <christoph.leuzinger at cs.uni-dortmund.de> wrote:

> The attached patch fixes this build problem for me.

I just noticed the Bro 1.4 pre-release posted on this list. The patch
applies to that release, too.

Regards,
  Christoph Leuzinger


From vern at icir.org  Sat Aug 23 17:26:47 2008
From: vern at icir.org (Vern Paxson)
Date: Sat, 23 Aug 2008 17:26:47 -0700
Subject: [Bro] Compiling Broccoli on NetBSD 4
In-Reply-To: <20080823160233.9483a267.christoph.leuzinger@cs.uni-dortmund.de>
	(Sat, 23 Aug 2008 16:02:33 +0200).
Message-ID: <200808240026.m7O0Qmio013732@pork.ICSI.Berkeley.EDU>

> I ran into a problem while building the Broccoli part of the Bro
> development version (1.3.2) on NetBSD 4.0. On NetBSD, <sys/sem.h> does
> not define the union semun. The attached patch fixes this build problem
> for me.

Thanks, I've integrated this for the forthcoming 1.4 release.

		Vern


From dkopecek at redhat.com  Wed Aug 27 07:53:51 2008
From: dkopecek at redhat.com (Daniel =?UTF-8?B?S29wZcSNZWs=?=)
Date: Wed, 27 Aug 2008 14:53:51 +0000
Subject: [Bro] Bro & Autoconf 2.62
Message-ID: <20080827145351.385a9d66@dhcp-lab-189.englab.brq.redhat.com>

Hi,
 I encountered a problem with Autoconf during packaging Bro (svn rev.
6043) for Fedora 10 and after consulting it with Stepan Kasal he
created the attached patch that fixes this problem. It was caused by a
collision with the internal variable in Autoconf 2.62.

Dan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bro-20080804-configure-opt-check.patch
Type: text/x-patch
Size: 580 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080827/6075e5f0/attachment.bin 

From christian at whoop.org  Wed Aug 27 06:50:37 2008
From: christian at whoop.org (Christian Kreibich)
Date: Wed, 27 Aug 2008 15:50:37 +0200
Subject: [Bro] Bro & Autoconf 2.62
In-Reply-To: <20080827145351.385a9d66@dhcp-lab-189.englab.brq.redhat.com>
References: <20080827145351.385a9d66@dhcp-lab-189.englab.brq.redhat.com>
Message-ID: <1219845037.7877.288.camel@strangepork>

Hi Dan,

On Wed, 2008-08-27 at 14:53 +0000, Daniel Kope?ek wrote:
> Hi,
>  I encountered a problem with Autoconf during packaging Bro (svn rev.
> 6043) for Fedora 10

Cool. (The packaging, not the encounter. ;) May I ask what is your
strategy for the packaging? Ideally there'd be three separate RPMs for
Bro, Broccoli, and BinPAC, since they all serve different purposes and
just happen to live in a single tree (this has caused config headaches
more than once). Broccoli has a spec file, but I haven't touched it in a
while. The BinPAC tree at least has its own configure setup.

> and after consulting it with Stepan Kasal he
> created the attached patch that fixes this problem. It was caused by a
> collision with the internal variable in Autoconf 2.62.

Thanks -- the patch seems to do a little more than just rename a
colliding variable (which I presume was "simplified"). I like the use of
m4_translit, assuming it is provided by a wide range of m4 versions.
Also, I presume the purpose of option_checking is to keep 
--enable-option-checking working?

-- 
Cheers,
Christian



From dkopecek at redhat.com  Wed Aug 27 10:11:34 2008
From: dkopecek at redhat.com (Daniel =?UTF-8?B?S29wZcSNZWs=?=)
Date: Wed, 27 Aug 2008 17:11:34 +0000
Subject: [Bro] Bro & Autoconf 2.62
In-Reply-To: <1219845037.7877.288.camel@strangepork>
References: <20080827145351.385a9d66@dhcp-lab-189.englab.brq.redhat.com>
	<1219845037.7877.288.camel@strangepork>
Message-ID: <20080827171134.0961cb8c@dhcp-lab-189.englab.brq.redhat.com>

On Wed, 27 Aug 2008 15:50:37 +0200
Christian Kreibich <christian at whoop.org> wrote:

> Hi Dan,
> 
> On Wed, 2008-08-27 at 14:53 +0000, Daniel Kope?ek wrote:
> > Hi,
> >  I encountered a problem with Autoconf during packaging Bro (svn
> > rev. 6043) for Fedora 10
> 
> Cool. (The packaging, not the encounter. ;) May I ask what is your
> strategy for the packaging? Ideally there'd be three separate RPMs for
> Bro, Broccoli, and BinPAC, since they all serve different purposes and
> just happen to live in a single tree (this has caused config headaches
> more than once). Broccoli has a spec file, but I haven't touched it
> in a while. The BinPAC tree at least has its own configure setup.

At the moment, only one package exists that installs only Bro.
Here is the CVS url: http://cvs.fedoraproject.org/viewvc/rpms/bro/
RPMs are here:
http://koji.fedoraproject.org/koji/buildinfo?buildID=60298

And I forgot to mention that there was an another problem with "make
install" to a different DESTDIR. I wrote a patch for this which is
available in CVS
(http://cvs.fedoraproject.org/viewvc/rpms/bro/devel/bro-20080804-installpolicy.patch?revision=1.1).

Dan




From christian at whoop.org  Fri Aug 29 04:06:32 2008
From: christian at whoop.org (Christian Kreibich)
Date: Fri, 29 Aug 2008 13:06:32 +0200
Subject: [Bro] Bro & Autoconf 2.62
In-Reply-To: <20080829103738.GA18798@camelia.ucw.cz>
References: <20080827145351.385a9d66@dhcp-lab-189.englab.brq.redhat.com>
	<1219845037.7877.288.camel@strangepork>
	<20080829103738.GA18798@camelia.ucw.cz>
Message-ID: <1220007993.11158.79.camel@strangepork>

Hi Stepan,

On Fri, 2008-08-29 at 12:37 +0200, Stepan Kasal wrote:
> Hello Christian and Vern,
>
> > Thanks -- the patch seems to do a little more than just rename a
> > colliding variable (which I presume was "simplified").
> 
> Actually, it's not the case.  I have bundled two changes into the
> patch, without proper explanation; I'm sorry for that.
> 
> The problem was that configure built with Autoconf 2.62 died
> conpaining that unknown option "option-checking" was used.
>
> The problem is that with Autoconf 2.62, shell variable
> enable_option_checking is set near the top of the script, no matter
> whether --enable-option-checking was given or not.
> 
> When your macro AC_LBL_ENABLE_CHECK finds this variable in the
> environment, it bails out compaining about unknown option named
> "option_checking".  That makes the configuire script unusable.

Ah! That rings a bell, but I can't seem to remember the circumstances
under which this was pointed out to us in the past. I'm sure though that
it was, and in fact we were considering ditching the (dated)
AC_LBL_ENABLE_CHECK in favor of the new built-in option checking once
2.62 is in widespread use.

> The issue can be fixed by this change:
> 
> --- bro-20080804/acinclude.m4.orig	2006-04-23 07:55:10.000000000 +0200
> +++ bro-20080804/acinclude.m4	2008-08-26 16:19:45.000000000 +0200
> @@ -192,7 +192,7 @@
>  	    ok=0
>          dnl change '-' to '_' 
>          simplified=`echo $1| sed -e 's/-/_/g'`
> -        for o in $simplified; do
> +        for o in option_checking $simplified; do
>  		    if test "${o}" = "${var}" ; then
>  			    ok=1
>  			    break
> 
> This way, the built in enable_option_checking option is added to the
> list of options supplied as a parameter to AC_LBL_ENABLE_CHECK.

Understood, and agreed with.

> > I like the use of
> > m4_translit, assuming it is provided by a wide range of m4 versions.
> 
> That was the other change; I thought the substitution should rather
> be done when configure is created, not during its run. And yes,
> m4_translit is available for very long time, no problem.

Okay, likewise agreed with then.

> Yes, Autoconf 2.62 introduces option checking (for both --enable-* and
> --with-* options).  Unfortunately, this feature is not generally
> compatible with recursive configure calls, at least in its present
> form, so it's switched on by default as soon as AC_CONFIG_SUBDIRS is
> used.  You could run configure with --enable-option-checking, but you
> would get many false warnings from the sub configures.

Got it.

> Let me remind: AC_CONFIG_SUBDIRS is not primarily meant to define a
> structure within a huge project, it is meant for incorporating
> tarballs, which are not under your control.  The tarballs are
> generally supposed to have different maintainers, the configure
> scripts might be generated by different version of Autoconf, etc.

We're aware of that. We needed a way to combine individual subtrees into
a single tree so that the overall tree would "just build", while
preserving the ability to produce tarballs of the subtrees when needed.

> You might file a feature request to enhance the
> --enable-option-checking so that it supports sub-configures, to
> bug-autoconf at gnu.org.
> 
> Or, if the sub-projects are maintained more or less together with the
> main project, you might integrate them so that they use a tree of
> Makefiles, but one common configure script.  I would be glad to help
> you investigate this possibility, but for general good, we should
> move to autoconf at gnu.org with that.

Regarding this, see my point above -- we primarily want a tree that
builds from scratch, but can provide subtree tarballs when needed.
Advice on how to do this, assuming our current approach is horribly
wrong, would be appreciated, but the real problem for us is lack of time
for a much-needed overhaul of our build scripts.

> Hope this mail helps.  I hope I managed to keep the offense rate low.
> ;-)

I'll be the last person to claim that our autotools setup is in good
shape at this point, so don't worry about offenses. :^)

-- 
Cheers,
Christian