From mcholste at gmail.com  Sat Oct  1 09:11:44 2011
From: mcholste at gmail.com (Martin Holste)
Date: Sat, 1 Oct 2011 11:11:44 -0500
Subject: [Bro-Dev] #615: Cluster manager crash
In-Reply-To: <061.f0f3e83d97f4df07e36e61675595e3e7@tracker.bro-ids.org>
References: <046.aca24e97ce3580f474951460ead41b9e@tracker.bro-ids.org>
	<061.f0f3e83d97f4df07e36e61675595e3e7@tracker.bro-ids.org>
Message-ID: <CANpnLHi-o22xeQd3kOY93eUGmT0zuvzbpZC=AWxFu=Tb-9P2XQ@mail.gmail.com>

Yep :(  Current workaround is to execute broctl start via cron every
minute.  The manager and sometimes the proxy crash every hour or so
(but not on the hour).

On Fri, Sep 30, 2011 at 8:32 PM, Bro Tracker <bro at tracker.bro-ids.org> wrote:
> #615: Cluster manager crash
> ----------------------+----------------------
> ?Reporter: ?seth ? ? | ? ? ?Owner:
> ? ? ?Type: ?Problem ?| ? ? Status: ?reopened
> ?Priority: ?High ? ? | ?Milestone: ?Bro1.6
> ?Component: ?Bro ? ? ?| ? ?Version:
> Resolution: ? ? ? ? ? | ? Keywords: ?beta
> ----------------------+----------------------
> Changes (by seth):
>
> ?* status: ?closed => reopened
> ?* resolution: ?Solved/Applied =>
>
>
> Comment:
>
> ?Martin has reported still seeing crash on his manager. ?Reopening.
>
> --
> Ticket URL: <http://tracker.bro-ids.org/bro/ticket/615#comment:6>
> Bro Tracker <http://tracker.bro-ids.org/bro>
> Bro Issue Tracker
>
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>


From robin at icir.org  Sat Oct  1 09:27:43 2011
From: robin at icir.org (Robin Sommer)
Date: Sat, 1 Oct 2011 09:27:43 -0700
Subject: [Bro-Dev] #615: Cluster manager crash
In-Reply-To: <CANpnLHi-o22xeQd3kOY93eUGmT0zuvzbpZC=AWxFu=Tb-9P2XQ@mail.gmail.com>
References: <046.aca24e97ce3580f474951460ead41b9e@tracker.bro-ids.org>
	<061.f0f3e83d97f4df07e36e61675595e3e7@tracker.bro-ids.org>
	<CANpnLHi-o22xeQd3kOY93eUGmT0zuvzbpZC=AWxFu=Tb-9P2XQ@mail.gmail.com>
Message-ID: <20111001162743.GC41117@icir.org>


On Sat, Oct 01, 2011 at 11:11 -0500, Martin wrote:

> Yep :(  Current workaround is to execute broctl start via cron every
> minute.  The manager and sometimes the proxy crash every hour or so
> (but not on the hour).

... and the stack backtraces you get are still looking like the one
shown in this ticket? (I.e., it's not the 101 error in particular?)

My cluster is running stable now (with all processes running on
different boxes).

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From mcholste at gmail.com  Sat Oct  1 09:50:31 2011
From: mcholste at gmail.com (Martin Holste)
Date: Sat, 1 Oct 2011 11:50:31 -0500
Subject: [Bro-Dev] #615: Cluster manager crash
In-Reply-To: <20111001162743.GC41117@icir.org>
References: <046.aca24e97ce3580f474951460ead41b9e@tracker.bro-ids.org>
	<061.f0f3e83d97f4df07e36e61675595e3e7@tracker.bro-ids.org>
	<CANpnLHi-o22xeQd3kOY93eUGmT0zuvzbpZC=AWxFu=Tb-9P2XQ@mail.gmail.com>
	<20111001162743.GC41117@icir.org>
Message-ID: <CANpnLHiaB5AJUbwsMrx6iDMEW-VK4iz93xOr6bmiKrTbd-xN_g@mail.gmail.com>

Ah, new error:

1317470292.374975 internal error in
/usr/local/bro-git/share/bro/base/protocols/http/./main.bro, line 269:
field value missing (HTTP::c$http$status_code)
/usr/local/bro-git/share/broctl/scripts/run-bro: line 60: 17933
Aborted                 (core dumped) nohup $mybro $@

Looks like another easy one involving the nested data structures not
checking for intermediate structs.

On Sat, Oct 1, 2011 at 11:27 AM, Robin Sommer <robin at icir.org> wrote:
>
> On Sat, Oct 01, 2011 at 11:11 -0500, Martin wrote:
>
>> Yep :( ?Current workaround is to execute broctl start via cron every
>> minute. ?The manager and sometimes the proxy crash every hour or so
>> (but not on the hour).
>
> ... and the stack backtraces you get are still looking like the one
> shown in this ticket? (I.e., it's not the 101 error in particular?)
>
> My cluster is running stable now (with all processes running on
> different boxes).
>
> Robin
>
> --
> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
> ICSI/LBNL ? ?* Fax ? +1 (510) 666-2956 * ? www.icir.org
>


From seth at icir.org  Sat Oct  1 13:43:35 2011
From: seth at icir.org (Seth Hall)
Date: Sat, 1 Oct 2011 16:43:35 -0400
Subject: [Bro-Dev] #615: Cluster manager crash
In-Reply-To: <CANpnLHiaB5AJUbwsMrx6iDMEW-VK4iz93xOr6bmiKrTbd-xN_g@mail.gmail.com>
References: <046.aca24e97ce3580f474951460ead41b9e@tracker.bro-ids.org>
	<061.f0f3e83d97f4df07e36e61675595e3e7@tracker.bro-ids.org>
	<CANpnLHi-o22xeQd3kOY93eUGmT0zuvzbpZC=AWxFu=Tb-9P2XQ@mail.gmail.com>
	<20111001162743.GC41117@icir.org>
	<CANpnLHiaB5AJUbwsMrx6iDMEW-VK4iz93xOr6bmiKrTbd-xN_g@mail.gmail.com>
Message-ID: <0B51AC7E-D9B6-4C20-9FE9-895D4F1EE5B1@icir.org>


On Oct 1, 2011, at 12:50 PM, Martin Holste wrote:

> 1317470292.374975 internal error in
> /usr/local/bro-git/share/bro/base/protocols/http/./main.bro, line 269:
> field value missing (HTTP::c$http$status_code)
> /usr/local/bro-git/share/broctl/scripts/run-bro: line 60: 17933
> Aborted                 (core dumped) nohup $mybro $@
> 
> Looks like another easy one involving the nested data structures not
> checking for intermediate structs.

Yep, it was fixed yesterday in the repository.

 .Seth

From bro at tracker.bro-ids.org  Sun Oct  2 08:42:16 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sun, 02 Oct 2011 15:42:16 -0000
Subject: [Bro-Dev] #633: Redefable globals
Message-ID: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>

#633: Redefable globals
------------------------+---------------------
 Reporter:  robin       |       Type:  Problem
   Status:  new         |   Priority:  Normal
Milestone:  Bro1.6      |  Component:  Bro
  Version:  git/master  |   Keywords:  beta
------------------------+---------------------
 We still have a number of globals that are redefable and therefore
 should be constants instead:

     > grep -R ^global  scripts | grep redef

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/633>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Sun Oct  2 08:57:58 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sun, 02 Oct 2011 15:57:58 -0000
Subject: [Bro-Dev] #633: Globals cleanup (was: Redefable globals)
In-Reply-To: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
References: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
Message-ID: <062.b147dc94940b6b0a6a06257e8cce5d8c@tracker.bro-ids.org>

#633: Globals cleanup
----------------------+------------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------
Description changed by robin:

Old description:

> We still have a number of globals that are redefable and therefore
> should be constants instead:
>
>     > grep -R ^global  scripts | grep redef

New description:

 There're a few little things regarding script globals that we should
 cleanup:

 - We still have a number of globals that are redefable and therefore
 should be constants instead:

     {{{> grep -R ^global  scripts | grep redef}}}

 - There are some globals that look like they should be {{{const ...
 &redef}}} instead. Example:
    {{{ global local_nets_table: table[subnet] of subnet = {}; }}} in
 {{{./base/utils/site.bro}}}

 - Not everything that should be {{{&synchronized}}} yet is already,
 Roughly speaking, once the two tasks above are done, all remaining
 ``globals`` that store state not limited to a specific host pair, should
 be {{{&synchronized}}}. Example {{{global did_notice_orig}}} in
 {{{./base/frameworks/notice/weird.bro}}}. Exception: there are some global
 tables that are static after initialization (like ``filters`` in
 {{{./base/frameworks/logging/main.bro}}}; these are fine (ideally, we
 would have a way to mark this usage pattern and catch it if somebody
 changes stuff later)).

--

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/633#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From vern at icir.org  Sun Oct  2 13:47:04 2011
From: vern at icir.org (Vern Paxson)
Date: Sun, 02 Oct 2011 13:47:04 -0700
Subject: [Bro-Dev] SSL paper using Bro
In-Reply-To: <033AF1D4-1E28-4FAF-9487-D3F91D9F6838@icir.org> (Fri,
	30 Sep 2011 07:30:19 EDT).
Message-ID: <20111002204704.48A3F2C4002@rock.ICSI.Berkeley.EDU>

> The authors are planning to present the paper at IMC but most importantly, they used Bro. :)

Yeah, but the bummer is that for a while now we've been kicking around
doing a similar study, and now we've been scooped, sigh.

		Vern

From bro at tracker.bro-ids.org  Sun Oct  2 17:19:20 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 03 Oct 2011 00:19:20 -0000
Subject: [Bro-Dev] #633: Globals cleanup
In-Reply-To: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
References: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
Message-ID: <062.ed4f5efece23ded4197c858acfc06586@tracker.bro-ids.org>

#633: Globals cleanup
----------------------+------------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by seth):

 > - We still have a number of globals that are redefable and therefore
 > should be constants instead:
 >
 >     {{{> grep -R ^global  scripts | grep redef}}}

 It's possible that I've left those with &redef because someone may want to
 modify an expiration attribute.  I think this was what we missed when we
 were talking about getting rid of &redef.  There are cases where you want
 to &redef globals to modify attributes.  Due to how attributes are set I
 can't turn the attribute arguments into variables either because you can't
 set them at init time.  We may want to discuss this further soon and try
 to address it in an upcoming release.

 > - There are some globals that look like they should be {{{const ...
 > &redef}}} instead. Example:
 >    {{{ global local_nets_table: table[subnet] of subnet = {}; }}} in
 > {{{./base/utils/site.bro}}}

 That's actually deliberate.  local_nets_table is generated at runtime
 (well, init time and possible runtime if I ever integrate it with the
 control framework which I've been meaning to do).

 > - Not everything that should be {{{&synchronized}}} yet is already,
 > Roughly speaking, once the two tasks above are done, all remaining
 > ``globals`` that store state not limited to a specific host pair, should
 > be {{{&synchronized}}}. Example {{{global did_notice_orig}}} in
 > {{{./base/frameworks/notice/weird.bro}}}.

 Hm, that probably just needs to be refactored.  Notice suppression is
 built into the notice framework at a low level now and it is already
 integrated with the cluster framework so &synchronized won't be needed in
 that case.

 > Exception: there are some global
 > tables that are static after initialization (like ``filters`` in
 > {{{./base/frameworks/logging/main.bro}}}; these are fine (ideally, we
 > would have a way to mark this usage pattern and catch it if somebody
 > changes stuff later)).

 Ah, yes.  This is the same as the local_nets_table issue.  All of these
 need to work transparently with the control framework (modified remotely
 or locally at runtime but remain mostly static) as well which needs to be
 wired together manually now but there be a better abstraction we could
 come up with.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/633#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Sun Oct  2 20:14:03 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 03 Oct 2011 03:14:03 -0000
Subject: [Bro-Dev] #633: Globals cleanup
In-Reply-To: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
References: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
Message-ID: <062.da2cef7cd782c3c86650fbf2ec7336ea@tracker.bro-ids.org>

#633: Globals cleanup
----------------------+------------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by robin):

 Note, the examples were just some I randomly picked. Let me phrase the
 task differently: we should quickly go through all the globals to make
 sure they are correct in regards to constness, &redef, and
 &synchronized. That's probably just a few greps. If you tell me they
 all are right already, I'm happy. :)

 Robin

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/633#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Sun Oct  2 20:21:20 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 03 Oct 2011 03:21:20 -0000
Subject: [Bro-Dev] #633: Globals cleanup
In-Reply-To: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
References: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
Message-ID: <062.12a9244f8de80783e912c19c7ca2e4cb@tracker.bro-ids.org>

#633: Globals cleanup
----------------------+------------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by seth):

 > If you tell me they
 > all are right already, I'm happy. :)

 I'll check over them again tomorrow but I expect to only add &redef to a
 few globals since I've been paying relatively close attention to them
 during development. :)

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/633#comment:5>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct  3 08:39:38 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 03 Oct 2011 15:39:38 -0000
Subject: [Bro-Dev] #634: CouchDB writer
Message-ID: <053.06c318cee769c34ae468c38f0621a66a@tracker.bro-ids.org>

#634: CouchDB writer
-------------------------+-----------------------------
 Reporter:  jeff.baumes  |       Type:  Feature Request
   Status:  new          |   Priority:  Normal
Milestone:               |  Component:  Bro
  Version:               |   Keywords:
-------------------------+-----------------------------
 Attached is a git patch for logging information to CouchDB. It has a new
 dependence on libcurl which it searches for with a find_package CMake
 command, and JsonCpp (MIT license), whose code is included directly in the
 source tree.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/634>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From mcholste at gmail.com  Mon Oct  3 11:57:13 2011
From: mcholste at gmail.com (Martin Holste)
Date: Mon, 3 Oct 2011 13:57:13 -0500
Subject: [Bro-Dev] #634: CouchDB writer
In-Reply-To: <053.06c318cee769c34ae468c38f0621a66a@tracker.bro-ids.org>
References: <053.06c318cee769c34ae468c38f0621a66a@tracker.bro-ids.org>
Message-ID: <CANpnLHjTEaRm6Z_AA8XJgUepyx=wKsByGoKvVdfX6hcsaOiRSw@mail.gmail.com>

CouchDB will be far too slow for any large-scale setup.  Inserts at
higher than 1k/sec will start to back up.  Even MongoDB, which is
much, much faster than Couch, can only handle 2-3k/sec sustained.  You
may want to include a warning in your documentation to only send
notice or a few other types of logs into CouchDB or risk loss.

On Mon, Oct 3, 2011 at 10:39 AM, Bro Tracker <bro at tracker.bro-ids.org> wrote:
> #634: CouchDB writer
> -------------------------+-----------------------------
> ?Reporter: ?jeff.baumes ?| ? ? ? Type: ?Feature Request
> ? Status: ?new ? ? ? ? ?| ? Priority: ?Normal
> Milestone: ? ? ? ? ? ? ? | ?Component: ?Bro
> ?Version: ? ? ? ? ? ? ? | ? Keywords:
> -------------------------+-----------------------------
> ?Attached is a git patch for logging information to CouchDB. It has a new
> ?dependence on libcurl which it searches for with a find_package CMake
> ?command, and JsonCpp (MIT license), whose code is included directly in the
> ?source tree.
>
> --
> Ticket URL: <http://tracker.bro-ids.org/bro/ticket/634>
> Bro Tracker <http://tracker.bro-ids.org/bro>
> Bro Issue Tracker
>
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>


From mcholste at gmail.com  Mon Oct  3 12:48:54 2011
From: mcholste at gmail.com (Martin Holste)
Date: Mon, 3 Oct 2011 14:48:54 -0500
Subject: [Bro-Dev] #615: Cluster manager crash
In-Reply-To: <0B51AC7E-D9B6-4C20-9FE9-895D4F1EE5B1@icir.org>
References: <046.aca24e97ce3580f474951460ead41b9e@tracker.bro-ids.org>
	<061.f0f3e83d97f4df07e36e61675595e3e7@tracker.bro-ids.org>
	<CANpnLHi-o22xeQd3kOY93eUGmT0zuvzbpZC=AWxFu=Tb-9P2XQ@mail.gmail.com>
	<20111001162743.GC41117@icir.org>
	<CANpnLHiaB5AJUbwsMrx6iDMEW-VK4iz93xOr6bmiKrTbd-xN_g@mail.gmail.com>
	<0B51AC7E-D9B6-4C20-9FE9-895D4F1EE5B1@icir.org>
Message-ID: <CANpnLHg-Np-B9-=-5PP+re-Sjn+6f0t-DRy0bR-CHoi1C3f3LQ@mail.gmail.com>

Sorry, still not fixed as of yesterday:
internal error: unknown msg type 101 in Poll()
/usr/local/bro-git/share/broctl/scripts/run-bro: line 60:  6329
Aborted                 (core dumped) nohup $mybro $@

On Sat, Oct 1, 2011 at 3:43 PM, Seth Hall <seth at icir.org> wrote:
>
> On Oct 1, 2011, at 12:50 PM, Martin Holste wrote:
>
>> 1317470292.374975 internal error in
>> /usr/local/bro-git/share/bro/base/protocols/http/./main.bro, line 269:
>> field value missing (HTTP::c$http$status_code)
>> /usr/local/bro-git/share/broctl/scripts/run-bro: line 60: 17933
>> Aborted ? ? ? ? ? ? ? ? (core dumped) nohup $mybro $@
>>
>> Looks like another easy one involving the nested data structures not
>> checking for intermediate structs.
>
> Yep, it was fixed yesterday in the repository.
>
> ?.Seth
>


From bro at tracker.bro-ids.org  Mon Oct  3 12:55:49 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 03 Oct 2011 19:55:49 -0000
Subject: [Bro-Dev] #615: Cluster manager crash
In-Reply-To: <046.aca24e97ce3580f474951460ead41b9e@tracker.bro-ids.org>
References: <046.aca24e97ce3580f474951460ead41b9e@tracker.bro-ids.org>
Message-ID: <061.471368c40f8967fa3733a33ce216e96b@tracker.bro-ids.org>

#615: Cluster manager crash
----------------------+----------------------
  Reporter:  seth     |      Owner:
      Type:  Problem  |     Status:  reopened
  Priority:  High     |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:           |   Keywords:  beta
----------------------+----------------------

Comment (by seth):

 > Sorry, still not fixed as of yesterday:
 > internal error: unknown msg type 101 in Poll()

 This really looks like a communications system overload issue.  Is the
 backtrace you are getting the same as the one in the description?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/615#comment:11>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From mcholste at gmail.com  Mon Oct  3 12:57:12 2011
From: mcholste at gmail.com (Martin Holste)
Date: Mon, 3 Oct 2011 14:57:12 -0500
Subject: [Bro-Dev] #615: Cluster manager crash
In-Reply-To: <061.471368c40f8967fa3733a33ce216e96b@tracker.bro-ids.org>
References: <046.aca24e97ce3580f474951460ead41b9e@tracker.bro-ids.org>
	<061.471368c40f8967fa3733a33ce216e96b@tracker.bro-ids.org>
Message-ID: <CANpnLHiTKoBsWBT+bJfbsXE2h+YZUCoHRvEOBAjpiXg2LXM5WA@mail.gmail.com>

Not sure, had to re-enable core dumps.  I can tell you that it's the
proxy, not the manager, which is problematic because you have to issue
a restart (not a start) for the workers to sync back up.  This means
my babysitter script of broctl start doesn't help.

On Mon, Oct 3, 2011 at 2:55 PM, Bro Tracker <bro at tracker.bro-ids.org> wrote:
> #615: Cluster manager crash
> ----------------------+----------------------
> ?Reporter: ?seth ? ? | ? ? ?Owner:
> ? ? ?Type: ?Problem ?| ? ? Status: ?reopened
> ?Priority: ?High ? ? | ?Milestone: ?Bro1.6
> ?Component: ?Bro ? ? ?| ? ?Version:
> Resolution: ? ? ? ? ? | ? Keywords: ?beta
> ----------------------+----------------------
>
> Comment (by seth):
>
> ?> Sorry, still not fixed as of yesterday:
> ?> internal error: unknown msg type 101 in Poll()
>
> ?This really looks like a communications system overload issue. ?Is the
> ?backtrace you are getting the same as the one in the description?
>
> --
> Ticket URL: <http://tracker.bro-ids.org/bro/ticket/615#comment:11>
> Bro Tracker <http://tracker.bro-ids.org/bro>
> Bro Issue Tracker
>
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>


From bro at tracker.bro-ids.org  Mon Oct  3 13:09:38 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 03 Oct 2011 20:09:38 -0000
Subject: [Bro-Dev] #615: Cluster manager crash
In-Reply-To: <046.aca24e97ce3580f474951460ead41b9e@tracker.bro-ids.org>
References: <046.aca24e97ce3580f474951460ead41b9e@tracker.bro-ids.org>
Message-ID: <061.27d32492da7e2802542938d0751dd0de@tracker.bro-ids.org>

#615: Cluster manager crash
----------------------+----------------------
  Reporter:  seth     |      Owner:
      Type:  Problem  |     Status:  reopened
  Priority:  High     |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:           |   Keywords:  beta
----------------------+----------------------

Comment (by seth):

 > Not sure, had to re-enable core dumps.  I can tell you that it's the
 > proxy, not the manager, which is problematic because you have to issue
 > a restart (not a start) for the workers to sync back up.  This means
 > my babysitter script of broctl start doesn't help.


 Hm.  A back trace would definitely be helpful.  We've been searching for
 this bug for quite some time.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/615#comment:12>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From mcholste at gmail.com  Tue Oct  4 06:52:52 2011
From: mcholste at gmail.com (Martin Holste)
Date: Tue, 4 Oct 2011 08:52:52 -0500
Subject: [Bro-Dev] Naive struct check bug back in HEAD
Message-ID: <CANpnLHgpug-oPxnX1RZp7pbh_2eaLkLoY_Pmgkfhf_i5a8rtnw@mail.gmail.com>

Line 269 of scripts/base/protocols/http/main.bro seems to have
reverted back to the segfault-able version in git.

From bro at tracker.bro-ids.org  Tue Oct  4 08:28:40 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 04 Oct 2011 15:28:40 -0000
Subject: [Bro-Dev] #635: Test to crash proxies
Message-ID: <046.062c5fd9c726ce6c82e3706cd0c61267@tracker.bro-ids.org>

#635: Test to crash proxies
---------------------+-----------------
 Reporter:  seth     |      Owner:
     Type:  Problem  |     Status:  new
 Priority:  Normal   |  Milestone:
Component:  Bro      |    Version:
 Keywords:           |
---------------------+-----------------
 I have a btest test that can reliably crash proxies (on my laptop at
 least).  I'm not assigning this to the next release though because it just
 seems to be a case of filling the pipe between the parent and child and
 not the weird "poll 101" crash that is occasionally seen in some cases,
 I'm still trying to create a test to cause that crash.

 Here's the error you get from btest from each proxy since each proxy in
 the test crashes with the same message::

 {{{
   error: parent: 1317739862.781639 fatal error, shutting down
 communication: Resource temporarily unavailable [35]
   error: parent: 1317739862.781754 fatal error, shutting down
 communication: Resource temporarily unavailable [35]
   error: parent: 1317739862.781775 fatal error, shutting down
 communication: Resource temporarily unavailable [35]
   Assertion failed: (peer->log_buffer), function SendLogWrite, file
 ./src/RemoteSerializer.cc, line 2556.
 }}}

 Here's the test::

 {{{
 # @TEST-EXEC: btest-bg-run manager-1 BROPATH=$BROPATH:..
 CLUSTER_NODE=manager-1 bro %INPUT
 # @TEST-EXEC: btest-bg-run proxy-1   BROPATH=$BROPATH:..
 CLUSTER_NODE=proxy-1 bro %INPUT
 # @TEST-EXEC: btest-bg-run proxy-2   BROPATH=$BROPATH:..
 CLUSTER_NODE=proxy-2 bro %INPUT
 # @TEST-EXEC: btest-bg-run worker-1  BROPATH=$BROPATH:..
 CLUSTER_NODE=worker-1 bro %INPUT
 # @TEST-EXEC: btest-bg-run worker-2  BROPATH=$BROPATH:..
 CLUSTER_NODE=worker-2 bro %INPUT
 # @TEST-EXEC: btest-bg-run worker-3  BROPATH=$BROPATH:..
 CLUSTER_NODE=worker-3 bro %INPUT
 # @TEST-EXEC: btest-bg-run worker-4  BROPATH=$BROPATH:..
 CLUSTER_NODE=worker-4 bro %INPUT

 # @TEST-EXEC: btest-bg-wait -k 10
 # @TEST-EXEC: btest-diff proxy-1/.stdout

 @TEST-START-FILE cluster-layout.bro
 redef Cluster::nodes = {
         ["manager-1"] = [$node_type=Cluster::MANAGER, $ip=127.0.0.1,
 $p=37757/tcp, $workers=set("worker-1")],
         ["proxy-1"] = [$node_type=Cluster::PROXY,     $ip=127.0.0.1,
 $p=37758/tcp, $manager="manager-1", $workers=set("worker-1", "worker-3")],
         ["proxy-2"] = [$node_type=Cluster::PROXY,     $ip=127.0.0.1,
 $p=37759/tcp, $manager="manager-1", $workers=set("worker-2", "worker-4")],
         ["worker-1"] = [$node_type=Cluster::WORKER,   $ip=127.0.0.1,
 $p=37760/tcp, $manager="manager-1", $proxy="proxy-1", $interface="eth0"],
         ["worker-2"] = [$node_type=Cluster::WORKER,   $ip=127.0.0.1,
 $p=37761/tcp, $manager="manager-1", $proxy="proxy-2", $interface="eth1"],
         ["worker-3"] = [$node_type=Cluster::WORKER,   $ip=127.0.0.1,
 $p=37762/tcp, $manager="manager-1", $proxy="proxy-1", $interface="eth0"],
         ["worker-4"] = [$node_type=Cluster::WORKER,   $ip=127.0.0.1,
 $p=37763/tcp, $manager="manager-1", $proxy="proxy-2", $interface="eth1"],

 };
 @TEST-END-FILE

 global all_data: set[string] = set() &synchronized;
 global blah = 0;

 @if ( Cluster::local_node_type() == Cluster::WORKER )
 event slam_proxy()
         {
         add all_data[unique_id(peer_description)];
         ++blah;
         if ( blah < 10000 )
                 event slam_proxy();
         }

 event bro_init()
         {
         event slam_proxy();
         }
 @endif
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/635>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From mcholste at gmail.com  Tue Oct  4 10:07:39 2011
From: mcholste at gmail.com (Martin Holste)
Date: Tue, 4 Oct 2011 12:07:39 -0500
Subject: [Bro-Dev] Naive struct check bug back in HEAD
In-Reply-To: <CANpnLHgpug-oPxnX1RZp7pbh_2eaLkLoY_Pmgkfhf_i5a8rtnw@mail.gmail.com>
References: <CANpnLHgpug-oPxnX1RZp7pbh_2eaLkLoY_Pmgkfhf_i5a8rtnw@mail.gmail.com>
Message-ID: <CANpnLHiRGrAOweqAM0gu=HfcghT1KchZsGKZ945dz5cgQAtLbg@mail.gmail.com>

I'm also seeing this cause worker crashes:

1317736351.925801 internal error in
/usr/local/bro-870bdf796d8ae1428389ed0f3302c8a818b7e78a/share/bro/policy/protocols/ssl/cert-hash.bro,
line 20: field value missing (SSL::c$ssl)
/usr/local/bro-870bdf796d8ae1428389ed0f3302c8a818b7e78a/share/broctl/scripts/run-bro:
line 60: 16497 Aborted                 (core dumped) nohup $mybro $@


On Tue, Oct 4, 2011 at 8:52 AM, Martin Holste <mcholste at gmail.com> wrote:
> Line 269 of scripts/base/protocols/http/main.bro seems to have
> reverted back to the segfault-able version in git.
>

From bro at tracker.bro-ids.org  Tue Oct  4 11:58:26 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 04 Oct 2011 18:58:26 -0000
Subject: [Bro-Dev] #619: Log rotations seem broken
In-Reply-To: <047.81859bf695ec52d8dd6278a9176c26bd@tracker.bro-ids.org>
References: <047.81859bf695ec52d8dd6278a9176c26bd@tracker.bro-ids.org>
Message-ID: <062.4d5f424e00eb339fd47fbea73b15a6b5@tracker.bro-ids.org>

#619: Log rotations seem broken
-----------------------------+------------------------
  Reporter:  robin           |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  High            |  Milestone:  Bro1.6
 Component:  BroControl      |    Version:  git/master
Resolution:  Solved/Applied  |   Keywords:  beta
-----------------------------+------------------------
Changes (by jsiwek):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 Looks like this just wasn't closed when the fix got merged, reopen if
 there's still a problem.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/619#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct  4 12:41:55 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 04 Oct 2011 19:41:55 -0000
Subject: [Bro-Dev] #636: topic/jsiwek/comphash-func-determinism
Message-ID: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>

#636: topic/jsiwek/comphash-func-determinism
---------------------------+------------------------
 Reporter:  jsiwek         |      Owner:
     Type:  Merge Request  |     Status:  new
 Priority:  Normal         |  Milestone:
Component:  Bro            |    Version:  git/master
 Keywords:                 |
---------------------------+------------------------
 This branch makes CompositeHash key computation and Val recovery for
 functions deterministic by basing it off the function identifier name
 instead of memory address

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/636>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct  4 17:56:06 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 05 Oct 2011 00:56:06 -0000
Subject: [Bro-Dev] #634: CouchDB writer
In-Reply-To: <053.06c318cee769c34ae468c38f0621a66a@tracker.bro-ids.org>
References: <053.06c318cee769c34ae468c38f0621a66a@tracker.bro-ids.org>
Message-ID: <068.0113673e38d517e86547409895c15b92@tracker.bro-ids.org>

#634: CouchDB writer
--------------------------+--------------------
  Reporter:  jeff.baumes  |      Owner:
      Type:  patch        |     Status:  new
  Priority:  Normal       |  Milestone:  Bro1.7
 Component:  Bro          |    Version:
Resolution:               |   Keywords:
--------------------------+--------------------
Changes (by robin):

 * type:  Feature Request => patch
 * milestone:   => Bro1.7


Comment:

 This is very cool.

 I'm marking this ticket for the release after the next when we'll
 start including new log writers and also do internal threading of the
 logging code to decouple it from the main process. Please update the
 ticket in the meantime if there updates to the code.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/634#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From rakesh.illini at gmail.com  Wed Oct  5 05:31:13 2011
From: rakesh.illini at gmail.com (Rakesh Gopchandani)
Date: Wed, 5 Oct 2011 17:31:13 +0500
Subject: [Bro-Dev] Changes in entropy computation code.
Message-ID: <CACNk0dZGJSAwBtQKGh7Q4P9BLudVbwqsu=ADHVU44qXSMd3V_w@mail.gmail.com>

Hi,

These changes went in a while ago: http://tracker.bro-ids.org/bro/ticket/265

However, there is an issue with how the entropy is being computed. Also, I
needed a special bif to compute entropy for strings of a specified length,
so added that too.


-
rakesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111005/34229f06/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Fixing-entopy-computation-code-and-adding-bif-for-co.patch
Type: text/x-patch
Size: 690 bytes
Desc: not available
Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111005/34229f06/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-Adding-a-new-bif-for-computing-entropy-for-specified.patch
Type: text/x-patch
Size: 1374 bytes
Desc: not available
Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111005/34229f06/attachment-0001.bin 

From seth at icir.org  Wed Oct  5 06:06:43 2011
From: seth at icir.org (Seth Hall)
Date: Wed, 5 Oct 2011 09:06:43 -0400
Subject: [Bro-Dev] Changes in entropy computation code.
In-Reply-To: <CACNk0dZGJSAwBtQKGh7Q4P9BLudVbwqsu=ADHVU44qXSMd3V_w@mail.gmail.com>
References: <CACNk0dZGJSAwBtQKGh7Q4P9BLudVbwqsu=ADHVU44qXSMd3V_w@mail.gmail.com>
Message-ID: <4DD1EC8F-CA64-4740-B8DC-15C5019FA5F9@icir.org>


On Oct 5, 2011, at 8:31 AM, Rakesh Gopchandani wrote:

> However, there is an issue with how the entropy is being computed.

This is well outside of my expertise, but your change is in opposition to how ENT[1] does it.  

-			ent += prob[i] * rt_log2(1 / prob[i]);
+			ent += prob[i] * rt_log2(prob[i]);

I just went back and verified and it looks like the original line is how it's done.

> Also, I needed a special bif to compute entropy for strings of a specified length, so added that too. 

I would rather not integrate this.  My suggestion would be to trim the string with the sub_bytes BiF before passing it to the find_entropy function.

  .Seth

1. http://www.fourmilab.ch/random/

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From jsiwek at ncsa.illinois.edu  Wed Oct  5 07:28:59 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Wed, 5 Oct 2011 09:28:59 -0500
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: HTTP bug fix reported
	by Martin. (0e4fecd)
In-Reply-To: <201110051335.p95DZths007939@bro-ids.icir.org>
References: <201110051335.p95DZths007939@bro-ids.icir.org>
Message-ID: <681557D8-E3CC-42C5-B978-A80BD5112941@ncsa.illinois.edu>

The way I thought to fix it is in fastpath commit c9a540b9 -- only delay logging when it's known for sure to be 1xx. This way would delay logging if also the status code is unknown, right?  Not sure if it matters much, but I thought my way would be closer to the original behavior.

- Jon

On Oct 5, 2011, at 8:35 AM, Seth Hall wrote:

> Repository : ssh://git at bro-ids.icir.org/bro
> 
> On branch  : master
> Link       : http://tracker.bro-ids.org/bro/changeset/0e4fecdfe4e31200ce00353a633b289fd4724f15/bro
> 
>> ---------------------------------------------------------------
> 
> commit 0e4fecdfe4e31200ce00353a633b289fd4724f15
> Author: Seth Hall <seth at icir.org>
> Date:   Wed Oct 5 09:35:19 2011 -0400
> 
>    HTTP bug fix reported by Martin.
> 
> 
>> ---------------------------------------------------------------
> 
> 0e4fecdfe4e31200ce00353a633b289fd4724f15
> scripts/base/protocols/http/main.bro |    3 ++-
> 1 files changed, 2 insertions(+), 1 deletions(-)
> 
> diff --git a/scripts/base/protocols/http/main.bro b/scripts/base/protocols/http/main.bro
> index 0faefc4..cbf1224 100644
> --- a/scripts/base/protocols/http/main.bro
> +++ b/scripts/base/protocols/http/main.bro
> @@ -266,7 +266,8 @@ event http_message_done(c: connection, is_orig: bool, stat: http_message_stat) &
> 		{
> 		# If the response was an informational 1xx, we're still expecting
> 		# the real response later, so we'll continue using the same record.
> -		if ( ! code_in_range(c$http$status_code, 100, 199) )
> +		if ( c$http?$status_code && 
> +			 ! code_in_range(c$http$status_code, 100, 199) )
> 			{
> 			Log::write(HTTP::LOG, c$http);
> 			delete c$http_state$pending[c$http_state$current_response];
> 
> _______________________________________________
> bro-commits mailing list
> bro-commits at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits
> 



From rakesh.illini at gmail.com  Wed Oct  5 07:54:08 2011
From: rakesh.illini at gmail.com (Rakesh Gopchandani)
Date: Wed, 5 Oct 2011 19:54:08 +0500
Subject: [Bro-Dev] Changes in entropy computation code.
In-Reply-To: <CACNk0dZuETEgFffnz4tJc471KWJMFb+oF2hnXyzkc-SCP32jig@mail.gmail.com>
References: <CACNk0dZGJSAwBtQKGh7Q4P9BLudVbwqsu=ADHVU44qXSMd3V_w@mail.gmail.com>
	<4DD1EC8F-CA64-4740-B8DC-15C5019FA5F9@icir.org>
	<CACNk0dZuETEgFffnz4tJc471KWJMFb+oF2hnXyzkc-SCP32jig@mail.gmail.com>
Message-ID: <CACNk0dYYspAvzmDLVfgGJQ99ysjMC36HW2CT_-Z1kXbLXi8BcA@mail.gmail.com>

Hi,


> This is well outside of my expertise, but your change is in opposition to
> how ENT[1] does it.
>
> -                       ent += prob[i] * rt_log2(1 / prob[i]);
> +                       ent += prob[i] * rt_log2(prob[i]);
>
> I just went back and verified and it looks like the original line is how
> it's done.
>
>
I checked it out. I think rt_log(prob[i]) is the correct way to do this. It
is the sum over entire alphabat, probability multiplied by log of
probablity:
http://en.wikipedia.org/wiki/Entropy_%28information_theory%29#Definition



> I would rather not integrate this.  My suggestion would be to trim the
> string with the sub_bytes BiF before passing it to the find_entropy
> function.
>

I see, thanks for pointing that out, just started scripting. :)

-

rakesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111005/10be75e5/attachment.html 

From gc355804 at ohio.edu  Wed Oct  5 11:09:05 2011
From: gc355804 at ohio.edu (Gilbert Clark)
Date: Wed, 05 Oct 2011 14:09:05 -0400
Subject: [Bro-Dev] Changes in entropy computation code.
In-Reply-To: <CACNk0dYYspAvzmDLVfgGJQ99ysjMC36HW2CT_-Z1kXbLXi8BcA@mail.gmail.com>
References: <CACNk0dZGJSAwBtQKGh7Q4P9BLudVbwqsu=ADHVU44qXSMd3V_w@mail.gmail.com>	<4DD1EC8F-CA64-4740-B8DC-15C5019FA5F9@icir.org>	<CACNk0dZuETEgFffnz4tJc471KWJMFb+oF2hnXyzkc-SCP32jig@mail.gmail.com>
	<CACNk0dYYspAvzmDLVfgGJQ99ysjMC36HW2CT_-Z1kXbLXi8BcA@mail.gmail.com>
Message-ID: <4E8C9D41.6000103@ohio.edu>

On 10/5/2011 10:54 AM, Rakesh Gopchandani wrote:
> Hi,
>
>
>     This is well outside of my expertise, but your change is in
>     opposition to how ENT[1] does it.
>
>     -                       ent += prob[i] * rt_log2(1 / prob[i]);
>     +                       ent += prob[i] * rt_log2(prob[i]);
>
>     I just went back and verified and it looks like the original line
>     is how it's done.
>
>
> I checked it out. I think rt_log(prob[i]) is the correct way to do 
> this. It is the sum over entire alphabat, probability multiplied by 
> log of probablity: 
> http://en.wikipedia.org/wiki/Entropy_%28information_theory%29#Definition
>

Unfortunately, e-mail is not the best medium for arguing stuff like 
this.  Forgive the terrible formatting.

Let SUM(i, f(i)) be the sum of f(i) for all values of i that exist in 
some data set (in our case, the array prob).

Base Definition: H(X) = -1 * SUM(i, prob[i] * rt_log2(prob[i]))

What we want to prove:

(1) H(X) = -1 * SUM(i, prob[i] * rt_log2(prob[i]))

is equivalent to

(2) H(X) = SUM(i, prob[i] * rt_log2(1 / prob[i]))

Assuming I remember how to do math, I believe we can get from (2) to (1) 
with a little bit of algebraic manipulation (as follows):

SUM(i, prob[i] * rt_log2(1 / prob[i]))  // we're starting at (2)

SUM(i, prob[i] * (rt_log2(1) - rt_log2(prob[i])) )       // rule of 
logarithms: log(1 / x) = log(1) - log(x)

SUM(i, prob[i] * -rt_log2(prob[i]) )          // log(1) = 0

SUM(i, -1 * prob[i] * rt_log2(prob[i]) )                         // pull 
the -1 to the front to make the next step more obvious

-1 * SUM(i, prob[i] * rt_log2(prob[i]) )                   // Pull the 
constant -1 out of the SUM

... and we should be good.

--Gilbert

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111005/86b3d3a4/attachment-0001.html 

From bro at tracker.bro-ids.org  Wed Oct  5 11:48:50 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 05 Oct 2011 18:48:50 -0000
Subject: [Bro-Dev] #498: Efficiency problem with remote log flushing
In-Reply-To: <047.837129432a53b3b6b387597dd321a7c9@tracker.bro-ids.org>
References: <047.837129432a53b3b6b387597dd321a7c9@tracker.bro-ids.org>
Message-ID: <062.31d9f1b4f958730c6ba1532c764f15d5@tracker.bro-ids.org>

#498: Efficiency problem with remote log flushing
----------------------+------------------------
  Reporter:  robin    |      Owner:  robin
      Type:  Problem  |     Status:  reopened
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------
Changes (by jsiwek):

 * status:  closed => reopened
 * resolution:  fixed =>


Comment:

 This seems to make it so that log buffers may be flushed at a frequency
 lower than that of the rotation interval of a remote peer, dependent on
 traffic volume & amount of writes that happen to a given stream.  So for a
 given rotation interval, the remote peer's logs might have none of the
 content actually logged in that interval because it wasn't flushed yet or
 content from previous intervals gets mixed in because the flush was
 delayed too long.

 Does there need to be a negotiation of log flushing around rotation to
 make low volume remote logs always rotate properly?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/498#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From vern at icir.org  Wed Oct  5 13:51:27 2011
From: vern at icir.org (Vern Paxson)
Date: Wed, 05 Oct 2011 13:51:27 -0700
Subject: [Bro-Dev] Bro logo in-the-small
Message-ID: <20111005205127.05A882C4009@rock.ICSI.Berkeley.EDU>

Viewing Gilbert's poster, I was struck at how confusing the Bro logo is
when it appears small.  You can't make out the network plug, it just looks
like there's a blotch on the eye.  Seems worth at some point getting the
artist to do a logo specifically meant for use when the image is small.

		Vern

From slagell at ncsa.illinois.edu  Wed Oct  5 14:08:51 2011
From: slagell at ncsa.illinois.edu (Adam J. Slagell)
Date: Wed, 5 Oct 2011 16:08:51 -0500
Subject: [Bro-Dev] Bro logo in-the-small
In-Reply-To: <20111005205127.05A882C4009@rock.ICSI.Berkeley.EDU>
References: <20111005205127.05A882C4009@rock.ICSI.Berkeley.EDU>
Message-ID: <CC76C80E-E79A-4507-83D0-8CBFA174C875@ncsa.illinois.edu>


On Oct 5, 2011, at 3:51 PM, Vern Paxson wrote:

> Viewing Gilbert's poster, I was struck at how confusing the Bro logo is
> when it appears small.  You can't make out the network plug, it just looks
> like there's a blotch on the eye.  Seems worth at some point getting the
> artist to do a logo specifically meant for use when the image is small.
> 
> 		Vern

We have a simple logo, a simple color logo, and a logo suitable for printing.I think one of those others would work better for the poster.

From bro at tracker.bro-ids.org  Wed Oct  5 14:35:26 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 05 Oct 2011 21:35:26 -0000
Subject: [Bro-Dev] #629: Cluster script load order
In-Reply-To: <047.3262250a91e9b4a9f1aae8f971e72bcd@tracker.bro-ids.org>
References: <047.3262250a91e9b4a9f1aae8f971e72bcd@tracker.bro-ids.org>
Message-ID: <062.44df88dee478de8cf911972873c8a3be@tracker.bro-ids.org>

#629: Cluster script load order
-------------------------+--------------------
  Reporter:  robin       |      Owner:  jsiwek
      Type:  Problem     |     Status:  closed
  Priority:  Normal      |  Milestone:  Bro1.6
 Component:  BroControl  |    Version:
Resolution:  fixed       |   Keywords:  beta
-------------------------+--------------------
Changes (by jsiwek):

 * owner:   => jsiwek
 * status:  new => closed
 * resolution:   => fixed


Comment:

 In [4cccd7d76553d003328b210d0c0f8a9725d16b46/broctl]:
 {{{
 #!CommitTicketReference repository="broctl"
 revision="4cccd7d76553d003328b210d0c0f8a9725d16b46"
 Various broctl changes

 - new broctl.cfg option for log rotation interval, addresses #630

 - removed some of the broct/nodes/* scripts and instead consolidated
   their functionality into the node-specific scripts that come with
   bro's cluster framework

 - within the cluster framework, local-<node>.bro scripts should now
   be loaded after the distributions <node>.bro script so things
   can be overrided

 - auto-generated broctl scripts are loaded after all node-specific
   scripts and can override their options

 All the above should also fix #629.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/629#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct  5 14:35:26 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 05 Oct 2011 21:35:26 -0000
Subject: [Bro-Dev] #630: Adding log rotation interval option to
	broctl.cfg
In-Reply-To: <047.c5fa14817d6b5013bdabf7e5c3698edc@tracker.bro-ids.org>
References: <047.c5fa14817d6b5013bdabf7e5c3698edc@tracker.bro-ids.org>
Message-ID: <062.c9e805dfea9f94519619bdd1f6570804@tracker.bro-ids.org>

#630: Adding log rotation interval option to broctl.cfg
------------------------------+--------------------
  Reporter:  robin            |      Owner:
      Type:  Feature Request  |     Status:  new
  Priority:  Normal           |  Milestone:  Bro1.6
 Component:  BroControl       |    Version:
Resolution:                   |   Keywords:  beta
------------------------------+--------------------

Comment (by jsiwek):

 In [4cccd7d76553d003328b210d0c0f8a9725d16b46/broctl]:
 {{{
 #!CommitTicketReference repository="broctl"
 revision="4cccd7d76553d003328b210d0c0f8a9725d16b46"
 Various broctl changes

 - new broctl.cfg option for log rotation interval, addresses #630

 - removed some of the broct/nodes/* scripts and instead consolidated
   their functionality into the node-specific scripts that come with
   bro's cluster framework

 - within the cluster framework, local-<node>.bro scripts should now
   be loaded after the distributions <node>.bro script so things
   can be overrided

 - auto-generated broctl scripts are loaded after all node-specific
   scripts and can override their options

 All the above should also fix #629.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/630#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct  5 14:53:20 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 05 Oct 2011 21:53:20 -0000
Subject: [Bro-Dev] #630: Adding log rotation interval option to
	broctl.cfg
In-Reply-To: <047.c5fa14817d6b5013bdabf7e5c3698edc@tracker.bro-ids.org>
References: <047.c5fa14817d6b5013bdabf7e5c3698edc@tracker.bro-ids.org>
Message-ID: <062.b46dc361c30578c216feacf5c3de589b@tracker.bro-ids.org>

#630: Adding log rotation interval option to broctl.cfg
----------------------------+-----------------------
  Reporter:  robin          |      Owner:
      Type:  Merge Request  |     Status:  new
  Priority:  Normal         |  Milestone:  Bro1.6
 Component:  BroControl     |    Version:  git/topic
Resolution:                 |   Keywords:  beta
----------------------------+-----------------------
Changes (by jsiwek):

 * version:   => git/topic
 * type:  Feature Request => Merge Request


Comment:

 In `topic/jsiwek/broctl-tweaks` of both the `bro` and `broctl` repo.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/630#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Wed Oct  5 15:20:24 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 5 Oct 2011 15:20:24 -0700
Subject: [Bro-Dev] [Bro-Commits] [git/broctl]
	topic/jsiwek/broctl-tweaks: Various	broctl changes (4cccd7d)
In-Reply-To: <201110052135.p95LZSOg006596@bro-ids.icir.org>
References: <201110052135.p95LZSOg006596@bro-ids.icir.org>
Message-ID: <20111005222024.GC6058@icir.org>


On Wed, Oct 05, 2011 at 14:35 -0700, Jonathan Siwek wrote:

> +LogRotationInterval = 1hrs

Quick question: should we just always assume a unit of hours here and
skip the suffix in broctl.cfg? The reason is that otherwise people
could write something here that Bro might later not be able to parse,
with probably odd error messages coming out of it.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Wed Oct  5 16:46:41 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 05 Oct 2011 23:46:41 -0000
Subject: [Bro-Dev] #636: topic/jsiwek/comphash-func-determinism
In-Reply-To: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
References: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
Message-ID: <063.044f3c3b4285e733db8b93e5708fef93@tracker.bro-ids.org>

#636: topic/jsiwek/comphash-func-determinism
----------------------------+------------------------
  Reporter:  jsiwek         |      Owner:  jsiwek
      Type:  Merge Request  |     Status:  assigned
  Priority:  Normal         |  Milestone:
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:
----------------------------+------------------------
Changes (by robin):

 * owner:   => jsiwek
 * status:  new => assigned


Comment:

 Don't really like this patch I have to say.

 - I don't like assigning the unique names to anonymous functions, with
 them then printing like

          {{{anonymous-function3 { return ((Notice::n$note in
 Notice::alarmed_types)); }}}}

 - More generally,assigning dummy names seems like overkill to me just to
 achieve consistent output for testing. Doesn't the earlier suggestion work
 to hash the functions Describe() output when in seeded-hash mode?

 -  Is the is_singelton case correct in the following? I don't see the
 distinction
   made when the hash is created?

  {{{

                         if ( is_singleton )
                                 {
                                 kp1 = kp0;
                                 n = k->Size();
                                 }
                         else
                                 {
                                 const int* const kp = AlignType<int>(kp0);
                                 n = *kp;
                                 kp1 = reinterpret_cast<const char*>(kp+1);
                                 }

                         string name(kp1, n);

 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/636#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct  5 17:21:38 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 06 Oct 2011 00:21:38 -0000
Subject: [Bro-Dev] #630: Adding log rotation interval option to
	broctl.cfg
In-Reply-To: <047.c5fa14817d6b5013bdabf7e5c3698edc@tracker.bro-ids.org>
References: <047.c5fa14817d6b5013bdabf7e5c3698edc@tracker.bro-ids.org>
Message-ID: <062.955ba90a76d75610c86ea0a35968869e@tracker.bro-ids.org>

#630: Adding log rotation interval option to broctl.cfg
----------------------------+-----------------------
  Reporter:  robin          |      Owner:  jsiwek
      Type:  Merge Request  |     Status:  assigned
  Priority:  Normal         |  Milestone:  Bro1.6
 Component:  BroControl     |    Version:  git/topic
Resolution:                 |   Keywords:  beta
----------------------------+-----------------------
Changes (by robin):

 * owner:   => jsiwek
 * status:  new => assigned


Comment:

 Merged, but I'm wondering whether `@load site/local-*` should be `@load
 load-*`? Isn't `site` already in BROPATH?

 And we may want to use hours as unit in `broctl.cfg`, per my earlier mail.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/630#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Wed Oct  5 18:27:40 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 5 Oct 2011 18:27:40 -0700
Subject: [Bro-Dev] Linking problem on Linux
Message-ID: <20111006012740.GI6058@icir.org>

With current master, I get this on an x86_84 Linux box:

    Linking CXX shared module _SubnetTree.so
    /usr/bin/ld: /local/lib/python2.6/config/libpython2.6.a(abstract.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
    /local/lib/python2.6/config/libpython2.6.a: could not read symbols: Bad value

Any ideas?

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Wed Oct  5 21:50:54 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 06 Oct 2011 04:50:54 -0000
Subject: [Bro-Dev] #498: Efficiency problem with remote log flushing
In-Reply-To: <047.837129432a53b3b6b387597dd321a7c9@tracker.bro-ids.org>
References: <047.837129432a53b3b6b387597dd321a7c9@tracker.bro-ids.org>
Message-ID: <062.8e67a65b443a7162a54e6ec9855778ac@tracker.bro-ids.org>

#498: Efficiency problem with remote log flushing
----------------------+------------------------
  Reporter:  robin    |      Owner:  robin
      Type:  Problem  |     Status:  reopened
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by robin):

 On Wed, Oct 05, 2011 at 18:48 -0000, you wrote:

 >  content from previous intervals gets mixed in because the flush was
 >  delayed too long.
 >
 >  Does there need to be a negotiation of log flushing around rotation to
 >  make low volume remote logs always rotate properly?

 This would be tricky. The manager would need to delay rotation until
 it has received the ack from everybody. That in turn might then cause
 other data to be in the wrong log, as the other nodes will keep
 sending. Not sure this is really solvable. What we perhaps could do is
 forcing a flush like once a second. That would at least limit the
 problematic time interval.

 Robin

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/498#comment:10>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Wed Oct  5 21:53:38 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 5 Oct 2011 21:53:38 -0700
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: HTTP bug fix	reported
	by Martin. (0e4fecd)
In-Reply-To: <681557D8-E3CC-42C5-B978-A80BD5112941@ncsa.illinois.edu>
References: <201110051335.p95DZths007939@bro-ids.icir.org>
	<681557D8-E3CC-42C5-B978-A80BD5112941@ncsa.illinois.edu>
Message-ID: <20111006045338.GI28778@icir.org>


On Wed, Oct 05, 2011 at 09:28 -0500, you wrote:

> The way I thought to fix it is in fastpath commit c9a540b9 -- only
> delay logging when it's known for sure to be 1xx.

I've applied this version now.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From mcholste at gmail.com  Thu Oct  6 07:03:36 2011
From: mcholste at gmail.com (Martin Holste)
Date: Thu, 6 Oct 2011 09:03:36 -0500
Subject: [Bro-Dev] [Bro-Commits] [git/broctl]
 topic/jsiwek/broctl-tweaks: Various broctl changes (4cccd7d)
In-Reply-To: <20111005222024.GC6058@icir.org>
References: <201110052135.p95LZSOg006596@bro-ids.icir.org>
	<20111005222024.GC6058@icir.org>
Message-ID: <CANpnLHjfhxb24ksKwQFCMr-Y3Yw=RMF-=VxBeMnDn31G5h_qOw@mail.gmail.com>

I think the usual way to do this is to express the time as an integer
of seconds, and only accept that.

On Wed, Oct 5, 2011 at 5:20 PM, Robin Sommer <robin at icir.org> wrote:
>
> On Wed, Oct 05, 2011 at 14:35 -0700, Jonathan Siwek wrote:
>
>> +LogRotationInterval = 1hrs
>
> Quick question: should we just always assume a unit of hours here and
> skip the suffix in broctl.cfg? The reason is that otherwise people
> could write something here that Bro might later not be able to parse,
> with probably odd error messages coming out of it.
>
> Robin
>
> --
> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
> ICSI/LBNL ? ?* Fax ? +1 (510) 666-2956 * ? www.icir.org
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>


From mcholste at gmail.com  Thu Oct  6 07:06:18 2011
From: mcholste at gmail.com (Martin Holste)
Date: Thu, 6 Oct 2011 09:06:18 -0500
Subject: [Bro-Dev] Linking problem on Linux
In-Reply-To: <20111006012740.GI6058@icir.org>
References: <20111006012740.GI6058@icir.org>
Message-ID: <CANpnLHjtxxLY8g=efQy=P1xBDk+JmCqJxgd=EHEarw89+3-rwg@mail.gmail.com>

That must be some python-specific problem.  I haven't had any issues
(yesterday anyway) on Ubuntu 10.04 LTS.

On Wed, Oct 5, 2011 at 8:27 PM, Robin Sommer <robin at icir.org> wrote:
> With current master, I get this on an x86_84 Linux box:
>
> ? ?Linking CXX shared module _SubnetTree.so
> ? ?/usr/bin/ld: /local/lib/python2.6/config/libpython2.6.a(abstract.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
> ? ?/local/lib/python2.6/config/libpython2.6.a: could not read symbols: Bad value
>
> Any ideas?
>
> Robin
>
> --
> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
> ICSI/LBNL ? ?* Fax ? +1 (510) 666-2956 * ? www.icir.org
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>


From bro at tracker.bro-ids.org  Thu Oct  6 08:04:03 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 06 Oct 2011 15:04:03 -0000
Subject: [Bro-Dev] #636: topic/jsiwek/comphash-func-determinism
In-Reply-To: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
References: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
Message-ID: <063.e5490b2f1dfd37b5c27b6b8dd858acef@tracker.bro-ids.org>

#636: topic/jsiwek/comphash-func-determinism
----------------------------+------------------------
  Reporter:  jsiwek         |      Owner:  jsiwek
      Type:  Merge Request  |     Status:  assigned
  Priority:  Normal         |  Milestone:
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:
----------------------------+------------------------

Comment (by jsiwek):

 > - I don't like assigning the unique names to anonymous functions, with
 them then printing like
 >
 >          {{{anonymous-function3 { return ((Notice::n$note in
 Notice::alarmed_types)); }}}}
 >
 > - More generally,assigning dummy names seems like overkill to me just to
 achieve consistent output for testing.

 The other goal was to achieve consistent ordering during the application
 of notice policy items in normal operation.  I think when I was looking
 for alternatives, not using the same priority in multiple `PolicyItem`s
 would meet that goal, but I'm not sure if the underlying issue would crop
 up again someplace else in the future.

 > Doesn't the earlier suggestion work to hash the functions Describe()
 output when in seeded-hash mode?

 Per the goal above I thought we said it has to work even outside of
 seeded-mode?

 As for hashing the function's `Describe()` output, how would the recovery
 process work for that?  If you wanted the original func Val that was
 hashed, that would still involve some sort of lookup against a global
 mapping of function bodies, otherwise we'd need to create a new func Val
 from the function body?  It seemed easiest to lookup by a unique name.

 > -  Is the is_singelton case correct in the following?

 I think so -- it should be almost equivalent to how strings are done.

 > I don't see the distinction made when the hash is created?

 I thought the distinction came from `ComputeHash()` branching to either
 `ComputeSingletonHash()` or `SingleValHash()` depending on `is_singleton`.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/636#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From seth at icir.org  Thu Oct  6 08:27:43 2011
From: seth at icir.org (Seth Hall)
Date: Thu, 6 Oct 2011 11:27:43 -0400
Subject: [Bro-Dev] identifier not exported
Message-ID: <EA1297BF-9BAE-467E-AC8E-D565DB0791EF@icir.org>

Does anyone know why were aren't getting the identifier name in these messages in report.log?

1317914544.693438	Reporter::ERROR	identifier is not exported:	-

There doesn't seem to be any location data associated with the message either but this message is fairly common and would be good to get fixed.

  .Seth


--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From dnthayer at ncsa.illinois.edu  Thu Oct  6 08:30:28 2011
From: dnthayer at ncsa.illinois.edu (Daniel Thayer)
Date: Thu, 6 Oct 2011 10:30:28 -0500 (CDT)
Subject: [Bro-Dev] Linking problem on Linux
In-Reply-To: <20111006012740.GI6058@icir.org>
Message-ID: <123368970.2305.1317915028602.JavaMail.root@zimbra-1.ncsa.uiuc.edu>

I do not see this error.  I'm using Ubuntu 10.04.3 (x86_64).

-Daniel


----- Original Message -----
From: "Robin Sommer" <robin at icir.org>
To: bro-dev at bro-ids.org
Sent: Wednesday, October 5, 2011 8:27:40 PM
Subject: [Bro-Dev] Linking problem on Linux

With current master, I get this on an x86_84 Linux box:

    Linking CXX shared module _SubnetTree.so
    /usr/bin/ld: /local/lib/python2.6/config/libpython2.6.a(abstract.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
    /local/lib/python2.6/config/libpython2.6.a: could not read symbols: Bad value

Any ideas?

Robin

From bro at tracker.bro-ids.org  Thu Oct  6 08:36:13 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 06 Oct 2011 15:36:13 -0000
Subject: [Bro-Dev] #636: topic/jsiwek/comphash-func-determinism
In-Reply-To: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
References: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
Message-ID: <063.f018b681ad1fa5e262f6c7f42ce4939f@tracker.bro-ids.org>

#636: topic/jsiwek/comphash-func-determinism
----------------------------+------------------------
  Reporter:  jsiwek         |      Owner:  jsiwek
      Type:  Merge Request  |     Status:  assigned
  Priority:  Normal         |  Milestone:
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:
----------------------------+------------------------

Comment (by robin):

 On Thu, Oct 06, 2011 at 15:04 -0000, you wrote:

 >  Per the goal above I thought we said it has to work even outside of
 >  seeded-mode?

 Hmm, I must have missed that part of the discussion. But makes sense.

 Here's another thought: rather than assigning IDs to all functions
 that don't have one yet, what about just enumerating them internally
 with an integer? Each function gets a new "int unique_func_id" and we
 just keep incrementing it. Then we use that for hashing. As we don't
 have many functions, having the additional field won't matter; but
 different than the IDs, it doesn't have any user-visible effects and
 doesn't need support in the parser (the Function ctor can do that
 internally and also keep a static vector of id->ptr).

 Robin

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/636#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Thu Oct  6 08:37:12 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 6 Oct 2011 08:37:12 -0700
Subject: [Bro-Dev] Linking problem on Linux
In-Reply-To: <123368970.2305.1317915028602.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
References: <20111006012740.GI6058@icir.org>
	<123368970.2305.1317915028602.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
Message-ID: <20111006153712.GE66685@icir.org>


On Thu, Oct 06, 2011 at 10:30 -0500, you wrote:

> I do not see this error.  I'm using Ubuntu 10.04.3 (x86_64).

Ok, then I'll need to figure out what's wrong with the Python
installation on that machine.

Thanks (also to Martin),

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Thu Oct  6 08:39:21 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 06 Oct 2011 15:39:21 -0000
Subject: [Bro-Dev] #630: Adding log rotation interval option to
	broctl.cfg
In-Reply-To: <047.c5fa14817d6b5013bdabf7e5c3698edc@tracker.bro-ids.org>
References: <047.c5fa14817d6b5013bdabf7e5c3698edc@tracker.bro-ids.org>
Message-ID: <062.0e1d963626e7b6e6fd9e6652a652fd2e@tracker.bro-ids.org>

#630: Adding log rotation interval option to broctl.cfg
----------------------------+-----------------------
  Reporter:  robin          |      Owner:  jsiwek
      Type:  Merge Request  |     Status:  assigned
  Priority:  Normal         |  Milestone:  Bro1.6
 Component:  BroControl     |    Version:  git/topic
Resolution:                 |   Keywords:  beta
----------------------------+-----------------------

Comment (by jsiwek):

 > Merged, but I'm wondering whether `@load site/local-*` should be `@load
 local-*`? Isn't `site` already in BROPATH?

 They both work, but you're right that `@load site/...` can be simplified.
 Fixed if you merge `topic/jsiwek/broctl-tweaks` in `bro`.

 > And we may want to use hours as unit in `broctl.cfg`, per my earlier
 mail.

 Changed to always be seconds since that's most flexible if you merge
 `topic/jsiwek/broctl-tweaks` in `broctl`.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/630#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From jsiwek at ncsa.illinois.edu  Thu Oct  6 08:45:07 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Thu, 6 Oct 2011 10:45:07 -0500
Subject: [Bro-Dev] Linking problem on Linux
In-Reply-To: <20111006012740.GI6058@icir.org>
References: <20111006012740.GI6058@icir.org>
Message-ID: <11F064A5-EAE0-4175-98D6-7D8A6398AB7C@ncsa.illinois.edu>

> With current master, I get this on an x86_84 Linux box:
> 
>    Linking CXX shared module _SubnetTree.so
>    /usr/bin/ld: /local/lib/python2.6/config/libpython2.6.a(abstract.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
>    /local/lib/python2.6/config/libpython2.6.a: could not read symbols: Bad value
> 
> Any ideas?

I think the situation you're in is that you're building a shared library on an AMD64 architecture which are required to be PIC-enabled, but you're linking to a static archive which was not built with the -fPIC flag.

Was this a manual build of python?  You might need to configure it to enable shared libraries (or if you really want to just use the static one, you have to configure it with the -fPIC flag).

- Jon

From gc355804 at ohio.edu  Thu Oct  6 08:55:13 2011
From: gc355804 at ohio.edu (Gilbert Clark)
Date: Thu, 06 Oct 2011 11:55:13 -0400
Subject: [Bro-Dev] Linking problem on Linux
In-Reply-To: <11F064A5-EAE0-4175-98D6-7D8A6398AB7C@ncsa.illinois.edu>
References: <20111006012740.GI6058@icir.org>
	<11F064A5-EAE0-4175-98D6-7D8A6398AB7C@ncsa.illinois.edu>
Message-ID: <4E8DCF61.6000100@ohio.edu>

+1 this.

Reference: 
http://stackoverflow.com/questions/629961/how-can-i-set-ccshared-fpic-while-executing-configure

--Gilbert

On 10/6/2011 11:45 AM, Jonathan Siwek wrote:
>> With current master, I get this on an x86_84 Linux box:
>>
>>     Linking CXX shared module _SubnetTree.so
>>     /usr/bin/ld: /local/lib/python2.6/config/libpython2.6.a(abstract.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
>>     /local/lib/python2.6/config/libpython2.6.a: could not read symbols: Bad value
>>
>> Any ideas?
> I think the situation you're in is that you're building a shared library on an AMD64 architecture which are required to be PIC-enabled, but you're linking to a static archive which was not built with the -fPIC flag.
>
> Was this a manual build of python?  You might need to configure it to enable shared libraries (or if you really want to just use the static one, you have to configure it with the -fPIC flag).
>
> - Jon
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev


From gc355804 at ohio.edu  Thu Oct  6 09:01:51 2011
From: gc355804 at ohio.edu (Gilbert Clark)
Date: Thu, 06 Oct 2011 12:01:51 -0400
Subject: [Bro-Dev] Linking problem on Linux
In-Reply-To: <4E8DCF61.6000100@ohio.edu>
References: <20111006012740.GI6058@icir.org>	<11F064A5-EAE0-4175-98D6-7D8A6398AB7C@ncsa.illinois.edu>
	<4E8DCF61.6000100@ohio.edu>
Message-ID: <4E8DD0EF.4090807@ohio.edu>

Oh, actually, this may be relevant as well...

http://code.google.com/p/modwsgi/wiki/InstallationIssues

Under "Mixing 32 Bit And 64 Bit Packages"

[quote] This error is believed to be result of the version of Python 
being used having been originally compiled for the generic X86 32 bit 
architecture whereas mod_wsgi is being compiled for X86 64 bit 
architecture. The actual error arises in this case because 'libtool' 
would appear to be unable to generate a dynamically loadable module for 
the X86 64 bit architecture from a X86 32 bit static library. 
Alternatively, the problem is due to 'libtool' on this platform not 
being able to create a loadable module from a X86 64 bit static library 
in all cases.

If the first issue, the only solution to this problem is to recompile 
Python for the X86 64 bit architecture. When doing this, it is 
preferable, and may actually be necessary, to ensure that the 
'--enable-shared' option is provided to the 'configure' script for 
Python when it is being compiled and installed.

If rebuilding Python to generate a shared library, do make sure that the 
Python shared library, or a symlink to it appears in the Python 'config' 
directory of your Python installation. If the shared library doesn't 
appear here next to the static version of the library, 'libtool' will 
not be able to find it and will still use the static version of the 
library. It is understood that the Python build process may not actually 
do this, so you may have to do it by hand.

If the version of Python being used was compiled for X86 64 bit 
architecture and a shared library does exist, but not in the 'config' 
directory, then adding the missing symlink may be all that is required.  
[/quote]

--Gilbert

On 10/6/2011 11:55 AM, Gilbert Clark wrote:
> +1 this.
>
> Reference:
> http://stackoverflow.com/questions/629961/how-can-i-set-ccshared-fpic-while-executing-configure
>
> --Gilbert
>
> On 10/6/2011 11:45 AM, Jonathan Siwek wrote:
>>> With current master, I get this on an x86_84 Linux box:
>>>
>>>      Linking CXX shared module _SubnetTree.so
>>>      /usr/bin/ld: /local/lib/python2.6/config/libpython2.6.a(abstract.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
>>>      /local/lib/python2.6/config/libpython2.6.a: could not read symbols: Bad value
>>>
>>> Any ideas?
>> I think the situation you're in is that you're building a shared library on an AMD64 architecture which are required to be PIC-enabled, but you're linking to a static archive which was not built with the -fPIC flag.
>>
>> Was this a manual build of python?  You might need to configure it to enable shared libraries (or if you really want to just use the static one, you have to configure it with the -fPIC flag).
>>
>> - Jon
>> _______________________________________________
>> bro-dev mailing list
>> bro-dev at bro-ids.org
>> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev


From bro at tracker.bro-ids.org  Thu Oct  6 09:05:13 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 06 Oct 2011 16:05:13 -0000
Subject: [Bro-Dev] #636: topic/jsiwek/comphash-func-determinism
In-Reply-To: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
References: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
Message-ID: <063.93644a208ee21dab44a8b7031efeb213@tracker.bro-ids.org>

#636: topic/jsiwek/comphash-func-determinism
----------------------------+------------------------
  Reporter:  jsiwek         |      Owner:  jsiwek
      Type:  Merge Request  |     Status:  assigned
  Priority:  Normal         |  Milestone:
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:
----------------------------+------------------------

Comment (by jsiwek):

 >  what about just enumerating them internally with an integer?

 I like that; will try it out.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/636#comment:5>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct  6 09:20:23 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 06 Oct 2011 16:20:23 -0000
Subject: [Bro-Dev] #632: process command does not produce logs in
	cluster mode
In-Reply-To: <047.ea938b17f19ea4f32708e6af5c0b3a5d@tracker.bro-ids.org>
References: <047.ea938b17f19ea4f32708e6af5c0b3a5d@tracker.bro-ids.org>
Message-ID: <062.e60bb17dd0406a35dcb471fd70111070@tracker.bro-ids.org>

#632: process command does not produce logs in cluster mode
-------------------------+--------------------
  Reporter:  robin       |      Owner:
      Type:  Problem     |     Status:  new
  Priority:  Normal      |  Milestone:  Bro1.6
 Component:  BroControl  |    Version:
Resolution:              |   Keywords:  beta
-------------------------+--------------------

Comment (by jsiwek):

 In [db0343ea6fe3bb1915df89232c877908a1c51721/broctl]:
 {{{
 #!CommitTicketReference repository="broctl"
 revision="db0343ea6fe3bb1915df89232c877908a1c51721"
 Force broctl 'process' command to enable local logging.

 Addresses #632
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/632#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Thu Oct  6 11:48:25 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 6 Oct 2011 11:48:25 -0700
Subject: [Bro-Dev] identifier not exported
In-Reply-To: <EA1297BF-9BAE-467E-AC8E-D565DB0791EF@icir.org>
References: <EA1297BF-9BAE-467E-AC8E-D565DB0791EF@icir.org>
Message-ID: <20111006184825.GJ66685@icir.org>


On Thu, Oct 06, 2011 at 11:27 -0400, you wrote:

> 1317914544.693438	Reporter::ERROR	identifier is not exported:	-

This should fix it:

diff --git a/src/Scope.cc b/src/Scope.cc
index 4ed8def..6491ca7 100644
--- a/src/Scope.cc
+++ b/src/Scope.cc
@@ -138,7 +138,7 @@ ID* lookup_ID(const char* name, const char* curr_module, bool no_global,
                if ( id )
                        {
                        if ( need_export && ! id->IsExport() && ! in_debug )
-                               reporter->Error("identifier is not exported:",
+                               reporter->Error("identifier is not exported: %s",
                                      fullname.c_str());
 
                        Ref(id);




-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From robin at icir.org  Thu Oct  6 12:08:34 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 6 Oct 2011 12:08:34 -0700
Subject: [Bro-Dev] Changes in entropy computation code.
In-Reply-To: <4E8C9D41.6000103@ohio.edu>
References: <CACNk0dZGJSAwBtQKGh7Q4P9BLudVbwqsu=ADHVU44qXSMd3V_w@mail.gmail.com>
	<4DD1EC8F-CA64-4740-B8DC-15C5019FA5F9@icir.org>
	<CACNk0dZuETEgFffnz4tJc471KWJMFb+oF2hnXyzkc-SCP32jig@mail.gmail.com>
	<CACNk0dYYspAvzmDLVfgGJQ99ysjMC36HW2CT_-Z1kXbLXi8BcA@mail.gmail.com>
	<4E8C9D41.6000103@ohio.edu>
Message-ID: <20111006190834.GN66685@icir.org>


On Wed, Oct 05, 2011 at 14:09 -0400, you wrote:

> ... and we should be good.

Gilbert, I'm not sure what your conclusion is?

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Thu Oct  6 12:38:24 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 06 Oct 2011 19:38:24 -0000
Subject: [Bro-Dev] #636: topic/jsiwek/comphash-func-determinism2 (was:
 topic/jsiwek/comphash-func-determinism)
In-Reply-To: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
References: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
Message-ID: <063.dc7f2b776b07713a0e8a5f3ccc5aa1f7@tracker.bro-ids.org>

#636: topic/jsiwek/comphash-func-determinism2
----------------------------+------------------------
  Reporter:  jsiwek         |      Owner:  jsiwek
      Type:  Merge Request  |     Status:  assigned
  Priority:  Normal         |  Milestone:
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:
----------------------------+------------------------

Comment (by jsiwek):

 Started over in `topic/jsiwek/comphash-func-determinism2`, look better?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/636#comment:6>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From gc355804 at ohio.edu  Thu Oct  6 13:19:16 2011
From: gc355804 at ohio.edu (Clark, Gilbert)
Date: Thu, 6 Oct 2011 16:19:16 -0400
Subject: [Bro-Dev] Changes in entropy computation code.
In-Reply-To: <20111006190834.GN66685@icir.org>
References: <CACNk0dZGJSAwBtQKGh7Q4P9BLudVbwqsu=ADHVU44qXSMd3V_w@mail.gmail.com>
	<4DD1EC8F-CA64-4740-B8DC-15C5019FA5F9@icir.org>
	<CACNk0dZuETEgFffnz4tJc471KWJMFb+oF2hnXyzkc-SCP32jig@mail.gmail.com>
	<CACNk0dYYspAvzmDLVfgGJQ99ysjMC36HW2CT_-Z1kXbLXi8BcA@mail.gmail.com>
	<4E8C9D41.6000103@ohio.edu>,<20111006190834.GN66685@icir.org>
Message-ID: <BD011092DE668F47B6BCA79D95195FAB3AF6CC51D0@EXMAIL1.ohio.edu>

Two conclusions here (referencing the following two lines of the patch):

-                       ent += prob[i] * rt_log2(1 / prob[i]);
+                       ent += prob[i] * rt_log2(prob[i]);

(1)  That the patched code would need to be:

ent -= prob[i] * rt_log2(prob[i]);

to be technically correct (unless the sign is flipped later and I just missed it :).  That said, I don't really know that the negative sign in front of the entropy matters much.

(2)  If the sign were correct, the submitted patch would not change the result of the code (e.g. that the code in there at the moment is technically correct).  [1]

That said, the patch *might* make the code's function a little more obvious, though; it took me a minute to recognize what was going on there, since the form for entropy that I was taught matches what's described in the patch.  

Related: there is an interesting effect described in 2.1 of [2] (called "cancellation"; see the footnote), though: as the individual probabilities decrease, I'd imagine that the accuracy of our calculation would fall (since the entropy is calculated incrementally, and individual contributions can be vanishingly small).  It might be good to try to find a similarly alternative definition here, as well [3]

--Gilbert

[1] I've since found a reference here: http://astarte.csustan.edu/~tom/SFI-CSSS/info-theory/info-lec.pdf that explicitly defines entropy with the prob[i] * rt_log2(1 / prob[i]) formula.
[2] http://www.eecs.berkeley.edu/~mhoemmen/cs194/Tutorials/variance.pdf
[3] Assuming I'm understanding this effect correctly...

________________________________________
From: Robin Sommer [robin at icir.org]
Sent: Thursday, October 06, 2011 3:08 PM
To: Clark, Gilbert
Cc: bro-dev at bro-ids.org
Subject: Re: [Bro-Dev] Changes in entropy computation code.

On Wed, Oct 05, 2011 at 14:09 -0400, you wrote:

> ... and we should be good.

Gilbert, I'm not sure what your conclusion is?

Robin

--
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From seth at icir.org  Thu Oct  6 17:00:29 2011
From: seth at icir.org (Seth Hall)
Date: Thu, 6 Oct 2011 20:00:29 -0400
Subject: [Bro-Dev] Changes in entropy computation code.
In-Reply-To: <BD011092DE668F47B6BCA79D95195FAB3AF6CC51D0@EXMAIL1.ohio.edu>
References: <CACNk0dZGJSAwBtQKGh7Q4P9BLudVbwqsu=ADHVU44qXSMd3V_w@mail.gmail.com>
	<4DD1EC8F-CA64-4740-B8DC-15C5019FA5F9@icir.org>
	<CACNk0dZuETEgFffnz4tJc471KWJMFb+oF2hnXyzkc-SCP32jig@mail.gmail.com>
	<CACNk0dYYspAvzmDLVfgGJQ99ysjMC36HW2CT_-Z1kXbLXi8BcA@mail.gmail.com>
	<4E8C9D41.6000103@ohio.edu>, <20111006190834.GN66685@icir.org>
	<BD011092DE668F47B6BCA79D95195FAB3AF6CC51D0@EXMAIL1.ohio.edu>
Message-ID: <A5910062-AD76-4A89-B0B5-8E67A54101DF@icir.org>


On Oct 6, 2011, at 4:19 PM, Clark, Gilbert wrote:

> Two conclusions here (referencing the following two lines of the patch):
> 
> -                       ent += prob[i] * rt_log2(1 / prob[i]);
> +                       ent += prob[i] * rt_log2(prob[i]);
> 
> (1)  That the patched code would need to be:
> 
> ent -= prob[i] * rt_log2(prob[i]);


So, this is what we should do with this line?  

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/


From bro at tracker.bro-ids.org  Thu Oct  6 17:03:23 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 00:03:23 -0000
Subject: [Bro-Dev] #636: topic/jsiwek/comphash-func-determinism2
In-Reply-To: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
References: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
Message-ID: <063.02c9cb16b24932a1e157f33962e425b0@tracker.bro-ids.org>

#636: topic/jsiwek/comphash-func-determinism2
----------------------------+------------------------
  Reporter:  jsiwek         |      Owner:  jsiwek
      Type:  Merge Request  |     Status:  assigned
  Priority:  Normal         |  Milestone:
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:
----------------------------+------------------------

Comment (by robin):

 Yeah, I like this much better. Thanks!

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/636#comment:7>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct  6 17:24:25 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 00:24:25 -0000
Subject: [Bro-Dev] #636: topic/jsiwek/comphash-func-determinism2
In-Reply-To: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
References: <048.a96aa09767e05ad182e1c655036fa346@tracker.bro-ids.org>
Message-ID: <063.119d5f74fe892899eaed68442fdf92b3@tracker.bro-ids.org>

#636: topic/jsiwek/comphash-func-determinism2
----------------------------+------------------------
  Reporter:  jsiwek         |      Owner:  jsiwek
      Type:  Merge Request  |     Status:  closed
  Priority:  Normal         |  Milestone:
 Component:  Bro            |    Version:  git/master
Resolution:  fixed          |   Keywords:
----------------------------+------------------------
Changes (by robin):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [7e5254ee2f2874e46d2e0f061d75a657bb9df111/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="7e5254ee2f2874e46d2e0f061d75a657bb9df111"
 Merge remote-tracking branch 'origin/topic/jsiwek/comphash-func-
 determinism2'

 Closes #636.

 * origin/topic/jsiwek/comphash-func-determinism2:
   Make CompHash computation/recovery for functions deterministic
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/636#comment:8>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct  6 17:25:01 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 00:25:01 -0000
Subject: [Bro-Dev] #630: Adding log rotation interval option to
	broctl.cfg
In-Reply-To: <047.c5fa14817d6b5013bdabf7e5c3698edc@tracker.bro-ids.org>
References: <047.c5fa14817d6b5013bdabf7e5c3698edc@tracker.bro-ids.org>
Message-ID: <062.d65655dfadc0eb6ac7ccfa03784658b0@tracker.bro-ids.org>

#630: Adding log rotation interval option to broctl.cfg
----------------------------+-----------------------
  Reporter:  robin          |      Owner:  jsiwek
      Type:  Merge Request  |     Status:  closed
  Priority:  Normal         |  Milestone:  Bro1.6
 Component:  BroControl     |    Version:  git/topic
Resolution:  fixed          |   Keywords:  beta
----------------------------+-----------------------
Changes (by robin):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [cf75e03def0f9d6196d2b4d0d2d3a9160a23b112/broctl]:
 {{{
 #!CommitTicketReference repository="broctl"
 revision="cf75e03def0f9d6196d2b4d0d2d3a9160a23b112"
 Merge remote-tracking branch 'origin/topic/jsiwek/broctl-tweaks'

 * origin/topic/jsiwek/broctl-tweaks:
   Change broctl.cfg LogRotationInterval to be specificed in seconds.

 Closes #630.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/630#comment:5>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct  6 17:39:33 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 00:39:33 -0000
Subject: [Bro-Dev] #612: Segfault in identify_data BiF
In-Reply-To: <048.2145c0c1e4de755bc9fa18b2c7b0ac68@tracker.bro-ids.org>
References: <048.2145c0c1e4de755bc9fa18b2c7b0ac68@tracker.bro-ids.org>
Message-ID: <063.b29dffe1f778863c76e97ba572562ab9@tracker.bro-ids.org>

#612: Segfault in identify_data BiF
----------------------+------------------------
  Reporter:  gregor   |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------
Changes (by robin):

 * priority:  High => Normal


Comment:

 I'm not seeing this anymore.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/612#comment:7>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct  6 17:41:07 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 00:41:07 -0000
Subject: [Bro-Dev] #498: Efficiency problem with remote log flushing
In-Reply-To: <047.837129432a53b3b6b387597dd321a7c9@tracker.bro-ids.org>
References: <047.837129432a53b3b6b387597dd321a7c9@tracker.bro-ids.org>
Message-ID: <062.07e6f7e447fce8f850f2cacad4c86991@tracker.bro-ids.org>

#498: Efficiency problem with remote log flushing
----------------------+------------------------
  Reporter:  robin    |      Owner:  robin
      Type:  Problem  |     Status:  reopened
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by robin):

 I'm adding this, let me know if that helps.q

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/498#comment:11>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct  6 17:52:33 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 00:52:33 -0000
Subject: [Bro-Dev] #577: DNS analyzer doesn't use ProtocolViolation /
 ProtocolConfirmation
In-Reply-To: <048.29ecdb2d548b5d4be16bd03c4228b3f7@tracker.bro-ids.org>
References: <048.29ecdb2d548b5d4be16bd03c4228b3f7@tracker.bro-ids.org>
Message-ID: <063.c5d9c46b26b44c6f7e887e704c3cfa64@tracker.bro-ids.org>

#577: DNS analyzer doesn't use ProtocolViolation / ProtocolConfirmation
----------------------+------------------------
  Reporter:  gregor   |      Owner:  robin
      Type:  Problem  |     Status:  accepted
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  BETA
----------------------+------------------------
Changes (by robin):

 * owner:   => robin
 * status:  new => accepted


Comment:

 Adding this.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/577#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct  6 18:17:00 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 01:17:00 -0000
Subject: [Bro-Dev] #615: Cluster manager crash
In-Reply-To: <046.aca24e97ce3580f474951460ead41b9e@tracker.bro-ids.org>
References: <046.aca24e97ce3580f474951460ead41b9e@tracker.bro-ids.org>
Message-ID: <061.da6946200f907f23e4f144b1c06cb277@tracker.bro-ids.org>

#615: Cluster manager crash
-----------------------------+--------------------
  Reporter:  seth            |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  High            |  Milestone:  Bro1.6
 Component:  Bro             |    Version:
Resolution:  Solved/Applied  |   Keywords:  beta
-----------------------------+--------------------
Changes (by robin):

 * status:  reopened => closed
 * resolution:   => Solved/Applied


Comment:

 So it sound like this *this* crash isn't appearing anymore, so I'm once
 again going to close it.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/615#comment:13>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct  6 19:07:36 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 02:07:36 -0000
Subject: [Bro-Dev] #577: DNS analyzer doesn't use ProtocolViolation /
 ProtocolConfirmation
In-Reply-To: <048.29ecdb2d548b5d4be16bd03c4228b3f7@tracker.bro-ids.org>
References: <048.29ecdb2d548b5d4be16bd03c4228b3f7@tracker.bro-ids.org>
Message-ID: <063.f8aa5a605f0d95a0b17acca307b5fa7d@tracker.bro-ids.org>

#577: DNS analyzer doesn't use ProtocolViolation / ProtocolConfirmation
----------------------+------------------------
  Reporter:  gregor   |      Owner:  robin
      Type:  Problem  |     Status:  closed
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:  fixed    |   Keywords:  BETA
----------------------+------------------------
Changes (by robin):

 * status:  accepted => closed
 * resolution:   => fixed


Comment:

 In [6fe2b2c0f3d0bb16d21652e01c87b8d4fe5d97f5/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="6fe2b2c0f3d0bb16d21652e01c87b8d4fe5d97f5"
 DNS now raises DPD events.

 Closes #577.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/577#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct  6 19:07:36 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 02:07:36 -0000
Subject: [Bro-Dev] #498: Efficiency problem with remote log flushing
In-Reply-To: <047.837129432a53b3b6b387597dd321a7c9@tracker.bro-ids.org>
References: <047.837129432a53b3b6b387597dd321a7c9@tracker.bro-ids.org>
Message-ID: <062.9ac75886157a9b97e790e4a06f07b20a@tracker.bro-ids.org>

#498: Efficiency problem with remote log flushing
----------------------+------------------------
  Reporter:  robin    |      Owner:  robin
      Type:  Problem  |     Status:  reopened
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by robin):

 In [f1ae48ea53e76d50784302bab32e41e78fbad11c/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="f1ae48ea53e76d50784302bab32e41e78fbad11c"
 Remote logs are auto-flushed if the last write was longer than a
 second ago. Addresses #498.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/498#comment:12>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct  6 21:08:29 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 04:08:29 -0000
Subject: [Bro-Dev] #559: Remove conn compressor
In-Reply-To: <047.44f48ecee506dc9ee47f21a82ca03af5@tracker.bro-ids.org>
References: <047.44f48ecee506dc9ee47f21a82ca03af5@tracker.bro-ids.org>
Message-ID: <062.1bcea987fd56c2860bacfc8625620235@tracker.bro-ids.org>

#559: Remove conn compressor
---------------------+------------------------
  Reporter:  robin   |      Owner:  robin
      Type:  Task    |     Status:  new
  Priority:  Normal  |  Milestone:  Bro1.7
 Component:  Bro     |    Version:  git/master
Resolution:          |   Keywords:  beta
---------------------+------------------------
Changes (by robin):

 * milestone:  Bro1.6 Bro1.7 => Bro1.7


Comment:

 Seth and I discussed that rather removing the compressor for 2.0, we'll
 just disable it. If that goes well, we'll remove for the next release.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/559#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct  6 21:12:24 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 04:12:24 -0000
Subject: [Bro-Dev] #609: Unit tests need to pass
In-Reply-To: <047.430e076354ee4907df9d41917a8ed78b@tracker.bro-ids.org>
References: <047.430e076354ee4907df9d41917a8ed78b@tracker.bro-ids.org>
Message-ID: <062.6abe2256e3392f976e2ccaa3d612e2c4@tracker.bro-ids.org>

#609: Unit tests need to pass
-----------------------------+--------------------
  Reporter:  robin           |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro1.6
 Component:  Bro             |    Version:
Resolution:  Solved/Applied  |   Keywords:  beta
-----------------------------+--------------------
Changes (by robin):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 All unit tests pass for me now. Yippie!

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/609#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct  6 21:26:20 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 04:26:20 -0000
Subject: [Bro-Dev] #559: Remove conn compressor
In-Reply-To: <047.44f48ecee506dc9ee47f21a82ca03af5@tracker.bro-ids.org>
References: <047.44f48ecee506dc9ee47f21a82ca03af5@tracker.bro-ids.org>
Message-ID: <062.3cc7f77fe8cb9ef2ba10a0d9eb2b4a28@tracker.bro-ids.org>

#559: Remove conn compressor
---------------------+------------------------
  Reporter:  robin   |      Owner:  robin
      Type:  Task    |     Status:  new
  Priority:  Normal  |  Milestone:  Bro1.7
 Component:  Bro     |    Version:  git/master
Resolution:          |   Keywords:  beta
---------------------+------------------------

Comment (by robin):

 In [517a9acfb9650e496baef83bbf49944fe6c2beb8/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="517a9acfb9650e496baef83bbf49944fe6c2beb8"
 Connection compressor now disabled by default.

 Addresses #559.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/559#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From seth at icir.org  Fri Oct  7 01:18:51 2011
From: seth at icir.org (Seth Hall)
Date: Fri, 7 Oct 2011 04:18:51 -0400
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Connection compressor
	now disabled by default. (517a9ac)
In-Reply-To: <201110070426.p974QKDJ024783@bro-ids.icir.org>
References: <201110070426.p974QKDJ024783@bro-ids.icir.org>
Message-ID: <AEA2AE08-B610-46DA-BC1A-E6F13B7C5692@icir.org>


On Oct 7, 2011, at 12:26 AM, Robin Sommer wrote:

>    Connection compressor now disabled by default.


It looks like this has slightly changed the output of two unit tests...

	core.vlan-mpls
	core.print-bpf-filters-ipv4

The tests are actually finding some differences between having the connection compressor enabled and disabled!

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From bro at tracker.bro-ids.org  Fri Oct  7 03:13:18 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 10:13:18 -0000
Subject: [Bro-Dev] #637: Integrate ssl alerts into the ssl log
Message-ID: <046.76b9bd1e6633db885efa09dbe80c5cd8@tracker.bro-ids.org>

#637: Integrate ssl alerts into the ssl log
---------------------+--------------------
 Reporter:  seth     |      Owner:  seth
     Type:  Problem  |     Status:  new
 Priority:  Normal   |  Milestone:  Bro1.6
Component:  Bro      |    Version:
 Keywords:  beta     |
---------------------+--------------------
 Right now it's not uncommon to get lines in the ssl.log that are confusing
 because most (and sometimes) all of the actual SSL related fields will be
 empty.  I think this is typically happening because of SSL alerts.  We
 should work SSL alerts into the log message somehow so those lines make
 sense.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/637>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct  7 03:17:27 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 10:17:27 -0000
Subject: [Bro-Dev] #633: Globals cleanup
In-Reply-To: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
References: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
Message-ID: <062.2b73527218249cacb967cab2b24925fd@tracker.bro-ids.org>

#633: Globals cleanup
----------------------+------------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by seth):

 Actually, thinking about this more, I think that anything in the export
 section should be implicitly redef-able (are we back to not needing the
 &redef attribute again?).  Globals need to remain redef-able so that
 people can redef attributes attached to them (expiration attrs, etc).

 What do you think about making everything in export sections implicitly
 redef-able?  I don't know how that will affect things like events where
 the event prototype is in an export section though.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/633#comment:6>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct  7 03:21:21 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 10:21:21 -0000
Subject: [Bro-Dev] #493: RemoteSerializer::Log method should give peer
 to remote_log event
In-Reply-To: <046.92af99e6494fd1fafea993dd0397bf7b@tracker.bro-ids.org>
References: <046.92af99e6494fd1fafea993dd0397bf7b@tracker.bro-ids.org>
Message-ID: <061.9e7ad9d6491a01d0dfcac53f21237004@tracker.bro-ids.org>

#493: RemoteSerializer::Log method should give peer to remote_log event
------------------------------+--------------------
  Reporter:  seth             |      Owner:
      Type:  Feature Request  |     Status:  new
  Priority:  Normal           |  Milestone:  Bro1.6
 Component:  Bro              |    Version:
Resolution:                   |   Keywords:
------------------------------+--------------------

Comment (by seth):

 This is line 2930 in RemoteSerializer.cc.  2930-2933 need to be removed
 and an instance of the event_peer record needs to be created and included
 with the remote_log event.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/493#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct  7 08:34:11 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 15:34:11 -0000
Subject: [Bro-Dev] #633: Globals cleanup
In-Reply-To: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
References: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
Message-ID: <062.de353cdf3a02281710105b490a77caaa@tracker.bro-ids.org>

#633: Globals cleanup
----------------------+------------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by robin):

 On Fri, Oct 07, 2011 at 10:17 -0000, you wrote:

 >  &redef attribute again?).  Globals need to remain redef-able so that
 >  people can redef attributes attached to them (expiration attrs, etc).

 Good point.

 >  What do you think about making everything in export sections implicitly
 >  redef-able?

 I can generally see doing that, but didn't we have a reason at some
 point that we didn't do that already?

 >   I don't know how that will affect things like events where the event
 >   prototype is in an export section though.

 We can always hardcode a check that prevent redef from being used with
 certain types.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/633#comment:7>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Fri Oct  7 08:40:41 2011
From: robin at icir.org (Robin Sommer)
Date: Fri, 7 Oct 2011 08:40:41 -0700
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Connection compressor
	now	disabled by default. (517a9ac)
In-Reply-To: <AEA2AE08-B610-46DA-BC1A-E6F13B7C5692@icir.org>
References: <201110070426.p974QKDJ024783@bro-ids.icir.org>
	<AEA2AE08-B610-46DA-BC1A-E6F13B7C5692@icir.org>
Message-ID: <20111007154041.GD16791@icir.org>


On Fri, Oct 07, 2011 at 04:18 -0400, you wrote:

> The tests are actually finding some differences between having the connection compressor enabled and disabled!

Yeah, it's the kind of change that's expected: slight changes in the
new IP-level volume counters. I'll update the baseline.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Fri Oct  7 09:52:53 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 16:52:53 -0000
Subject: [Bro-Dev] #632: process command does not produce logs in
	cluster mode
In-Reply-To: <047.ea938b17f19ea4f32708e6af5c0b3a5d@tracker.bro-ids.org>
References: <047.ea938b17f19ea4f32708e6af5c0b3a5d@tracker.bro-ids.org>
Message-ID: <062.282788f6d371cb0ec23a9dd6610ea798@tracker.bro-ids.org>

#632: process command does not produce logs in cluster mode
-----------------------------+--------------------
  Reporter:  robin           |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro1.6
 Component:  BroControl      |    Version:
Resolution:  Solved/Applied  |   Keywords:  beta
-----------------------------+--------------------
Changes (by jsiwek):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 Fix was merged to master, and still works for me with a fresh install.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/632#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct  7 09:54:17 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 16:54:17 -0000
Subject: [Bro-Dev] #498: Efficiency problem with remote log flushing
In-Reply-To: <047.837129432a53b3b6b387597dd321a7c9@tracker.bro-ids.org>
References: <047.837129432a53b3b6b387597dd321a7c9@tracker.bro-ids.org>
Message-ID: <062.9c7e4108131c1653ee41c0f196e635b4@tracker.bro-ids.org>

#498: Efficiency problem with remote log flushing
-----------------------------+------------------------
  Reporter:  robin           |      Owner:  robin
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro1.6
 Component:  Bro             |    Version:  git/master
Resolution:  Solved/Applied  |   Keywords:  beta
-----------------------------+------------------------
Changes (by jsiwek):

 * status:  reopened => closed
 * resolution:   => Solved/Applied


Comment:

 Works better for me now.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/498#comment:13>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct  7 14:40:14 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 07 Oct 2011 21:40:14 -0000
Subject: [Bro-Dev] #577: DNS analyzer doesn't use ProtocolViolation /
 ProtocolConfirmation
In-Reply-To: <048.29ecdb2d548b5d4be16bd03c4228b3f7@tracker.bro-ids.org>
References: <048.29ecdb2d548b5d4be16bd03c4228b3f7@tracker.bro-ids.org>
Message-ID: <063.f49ac881b6aec9d12d2104b5fe89ec48@tracker.bro-ids.org>

#577: DNS analyzer doesn't use ProtocolViolation / ProtocolConfirmation
----------------------+------------------------
  Reporter:  gregor   |      Owner:  robin
      Type:  Problem  |     Status:  reopened
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  BETA
----------------------+------------------------
Changes (by jsiwek):

 * status:  closed => reopened
 * resolution:  fixed =>


Comment:

 > In [6fe2b2c0f3d0bb16d21652e01c87b8d4fe5d97f5/bro]:
 > {{{
 > #!CommitTicketReference repository="bro"
 revision="6fe2b2c0f3d0bb16d21652e01c87b8d4fe5d97f5"
 > DNS now raises DPD events.
 >
 > Closes #577.
 > }}}

 With this change an assertion is hit when running the long external trace
 test:

 {{{
 Program received signal SIGABRT, Aborted.
 0x00007fff8aa5a82a in __kill ()
 (gdb) where
 #0  0x00007fff8aa5a82a in __kill ()
 #1  0x00007fff939d9a9c in abort ()
 #2  0x00007fff93a0c5de in __assert_rtn ()
 #3  0x00000001000dffd1 in Analyzer::~Analyzer (this=0x102650230) at
 Analyzer.cc:254
 #4  0x00000001002bcbc6 in
 TCP_ApplicationAnalyzer::~TCP_ApplicationAnalyzer (this=0x102650230) at
 TCP.h:266
 #5  0x00000001001393dc in DNS_Analyzer::~DNS_Analyzer (this=0x102650230)
 at DNS.cc:1132
 #6  0x00000001000daa40 in Analyzer::ForwardPacket (this=0x104526ab0,
 len=119, data=0x10126c62a "", is_orig=true, seq=-1, ip=0x7fff5fbfe448,
 caplen=119) at Analyzer.cc:440
 #7  0x0000000100384807 in UDP_Analyzer::DeliverPacket (this=0x104526ab0,
 len=119, data=0x10126c62a "", is_orig=true, seq=-1, ip=0x7fff5fbfe448,
 caplen=119) at UDP.cc:163
 #8  0x00000001000dfcec in Analyzer::NextPacket (this=0x104526ab0, len=127,
 data=0x10126c622 "\024?\024?", is_orig=true, seq=-1, ip=0x7fff5fbfe448,
 caplen=127) at Analyzer.cc:335
 #9  0x000000010012011a in Connection::NextPacket (this=0x1027e8580,
 t=1258615443.2526419, is_orig=1, ip=0x7fff5fbfe448, len=127, caplen=127,
 data=@0x7fff5fbfe310, record_packet=@0x7fff5fbfe2cc,
 record_content=@0x7fff5fbfe2c8, hdr=0x1020eee20, pkt=0x10126c600 "\001",
 hdr_size=14) at Conn.cc:246
 #10 0x000000010032aa47 in NetSessions::DoNextPacket (this=0x10218bf40,
 t=1258615443.2526419, hdr=0x1020eee20, ip_hdr=0x7fff5fbfe448,
 pkt=0x10126c600 "\001", hdr_size=14) at Sessions.cc:608
 #11 0x000000010032b104 in NetSessions::NextPacket (this=0x10218bf40,
 t=1258615443.2526419, hdr=0x1020eee20, pkt=0x10126c600 "\001",
 hdr_size=14, pkt_elem=0x0) at Sessions.cc:279
 #12 0x000000010032b593 in NetSessions::DispatchPacket (this=0x10218bf40,
 t=1258615443.2526419, hdr=0x1020eee20, pkt=0x10126c600 "\001",
 hdr_size=14, src_ps=0x1020eede0, pkt_elem=0x0) at Sessions.cc:228
 #13 0x000000010029a99b in net_packet_dispatch (t=1258615443.2526419,
 hdr=0x1020eee20, pkt=0x10126c600 "\001", hdr_size=14, src_ps=0x1020eede0,
 pkt_elem=0x0) at Net.cc:351
 #14 0x000000010029b0fe in net_packet_arrival (t=1258615443.2526419,
 hdr=0x1020eee20, pkt=0x10126c600 "\001", hdr_size=14, src_ps=0x1020eede0)
 at Net.cc:414
 #15 0x00000001002b8936 in PktSrc::Process (this=0x1020eede0) at
 PktSrc.cc:273
 #16 0x000000010029ad46 in net_run () at Net.cc:444
 #17 0x00000001000ce4c2 in main (argc=4, argv=0x7fff5fbff4d0) at
 main.cc:1009
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/577#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct  7 17:17:29 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sat, 08 Oct 2011 00:17:29 -0000
Subject: [Bro-Dev] #638: Doc mode crashes
Message-ID: <047.7b045acd377faeccdbb32687966ab133@tracker.bro-ids.org>

#638: Doc mode crashes
---------------------+--------------------
 Reporter:  robin    |      Owner:
     Type:  Problem  |     Status:  new
 Priority:  Normal   |  Milestone:  Bro1.6
Component:  Bro      |    Version:
 Keywords:  beta     |
---------------------+--------------------
 Something in recent script changes triggers all the `doc/` tests to crash
 with `  internal error: bad reference count [2]
 `.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/638>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct  7 17:18:25 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sat, 08 Oct 2011 00:18:25 -0000
Subject: [Bro-Dev] #638: Doc mode crashes
In-Reply-To: <047.7b045acd377faeccdbb32687966ab133@tracker.bro-ids.org>
References: <047.7b045acd377faeccdbb32687966ab133@tracker.bro-ids.org>
Message-ID: <062.8e1c5c966f9bc3cdbdf54a9929b8c802@tracker.bro-ids.org>

#638: Doc mode crashes
----------------------+--------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:           |   Keywords:  beta
----------------------+--------------------
Description changed by robin:

Old description:

> Something in recent script changes triggers all the `doc/` tests to crash
> with `  internal error: bad reference count [2]
> `.

New description:

 Something in recent script changes triggers all the `doc/` tests to crash
 with `  internal error: bad reference count [2]`.

 It works with commit `eca3e4db4eb48ad7648ce0f947b0470fc30464e7`, but not
 HEAD.

--

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/638#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct  7 17:40:30 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sat, 08 Oct 2011 00:40:30 -0000
Subject: [Bro-Dev] #577: DNS analyzer doesn't use ProtocolViolation /
 ProtocolConfirmation
In-Reply-To: <048.29ecdb2d548b5d4be16bd03c4228b3f7@tracker.bro-ids.org>
References: <048.29ecdb2d548b5d4be16bd03c4228b3f7@tracker.bro-ids.org>
Message-ID: <063.918d7478dd9db9607dfd3ed12e0f899d@tracker.bro-ids.org>

#577: DNS analyzer doesn't use ProtocolViolation / ProtocolConfirmation
----------------------+------------------------
  Reporter:  gregor   |      Owner:  robin
      Type:  Problem  |     Status:  closed
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:  fixed    |   Keywords:  BETA
----------------------+------------------------
Changes (by robin):

 * status:  reopened => closed
 * resolution:   => fixed


Comment:

 In [8fa059fb1054266c26898a37ad41a4b76371a081/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="8fa059fb1054266c26898a37ad41a4b76371a081"
 Fix in code for disabling analyzers.

 Plus some refactoring.

 Closes #577.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/577#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Fri Oct  7 18:47:09 2011
From: robin at icir.org (Robin Sommer)
Date: Fri, 7 Oct 2011 18:47:09 -0700
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Modification to
	the	Communication framework API. (da9b8cc)
In-Reply-To: <201110071902.p97J2wwN002611@bro-ids.icir.org>
References: <201110071902.p97J2wwN002611@bro-ids.icir.org>
Message-ID: <20111008014709.GI83146@icir.org>


On Fri, Oct 07, 2011 at 12:02 -0700, Seth Hall wrote:

>     	redef Communication::listen_encrypted=T;

What would you think about renaming thus to listen_ssl? It's not only
encrypted but also authenticated, and "ssl" generally seems to say
better what it's about.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From robin at icir.org  Fri Oct  7 18:47:57 2011
From: robin at icir.org (Robin Sommer)
Date: Fri, 7 Oct 2011 18:47:57 -0700
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Fixed a TODO in the
	DNS	analysis script. (8600b67)
In-Reply-To: <201110071903.p97J371V002647@bro-ids.icir.org>
References: <201110071903.p97J371V002647@bro-ids.icir.org>
Message-ID: <20111008014757.GJ83146@icir.org>


On Fri, Oct 07, 2011 at 12:03 -0700, Seth Hall wrote:

> -event do_reply(c: connection, msg: dns_msg, ans: dns_answer, reply: string) &priority=5
> +event DNS::do_reply(c: connection, msg: dns_msg, ans: dns_answer, reply: string) &priority=5

Why's this necessary (and why not for all events then)?

I believe there are still some inconsistencies when using namespaces
with events; not sure I like using that at this time.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From seth at icir.org  Fri Oct  7 19:45:34 2011
From: seth at icir.org (Seth Hall)
Date: Fri, 7 Oct 2011 22:45:34 -0400
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Fixed a TODO in the
	DNS	analysis script. (8600b67)
In-Reply-To: <20111008014757.GJ83146@icir.org>
References: <201110071903.p97J371V002647@bro-ids.icir.org>
	<20111008014757.GJ83146@icir.org>
Message-ID: <684C99B4-7884-495E-86C4-4AF9ECCC78F2@icir.org>


On Oct 7, 2011, at 9:47 PM, Robin Sommer wrote:

> Why's this necessary (and why not for all events then)?

99% of events are in the GLOBAL namespace.

> I believe there are still some inconsistencies when using namespaces
> with events; not sure I like using that at this time.


I believe that Bro won't check the current namespace for event handlers so if you have an event handler in a namespace you always have to specify the namespace even if you are in the correct namespace.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Fri Oct  7 19:46:23 2011
From: seth at icir.org (Seth Hall)
Date: Fri, 7 Oct 2011 22:46:23 -0400
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Modification to
	the	Communication framework API. (da9b8cc)
In-Reply-To: <20111008014709.GI83146@icir.org>
References: <201110071902.p97J2wwN002611@bro-ids.icir.org>
	<20111008014709.GI83146@icir.org>
Message-ID: <A63987E8-E12E-44CC-AB52-06897219C20B@icir.org>


On Oct 7, 2011, at 9:47 PM, Robin Sommer wrote:

> On Fri, Oct 07, 2011 at 12:02 -0700, Seth Hall wrote:
> 
>>    	redef Communication::listen_encrypted=T;
> 
> What would you think about renaming thus to listen_ssl? It's not only
> encrypted but also authenticated, and "ssl" generally seems to say
> better what it's about.


Ah, sure.  Good point.  I'll make all of the requisite changes now.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/


From seth at icir.org  Fri Oct  7 21:01:51 2011
From: seth at icir.org (Seth Hall)
Date: Sat, 8 Oct 2011 00:01:51 -0400
Subject: [Bro-Dev] Weird stuff in weird.log
Message-ID: <A8AC8850-225B-4C48-859E-9D261DC6261F@icir.org>

I'm seeing some odd stuff in my weird.log on a cluster.  It looks an extra tab character is getting put in after null values and then it's escaped.  Any clue what happened?

1318046447.381516	-\x09	-\x09	-\x09	-\x09	-\x09	non_IPv4_packet	-\x09	T
1318046447.960574	-\x09	-\x09	-\x09	-\x09	-\x09	non_IPv4_packet	-\x09	T
1318046447.960867	-\x09	-\x09	-\x09	-\x09	-\x09	non_IPv4_packet	-\x09	T
1318046513.637314	-\x09	-\x09	-\x09	-\x09	-\x09	non_IPv4_packet	-\x09	T
1318046447.932012	-\x09	-\x09	-\x09	-\x09	-\x09	non_IPv4_packet	-\x09	T
1318046447.935414	-\x09	-\x09	-\x09	-\x09	-\x09	non_IPv4_packet	-\x09	T

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From bro at tracker.bro-ids.org  Sun Oct  9 14:19:03 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sun, 09 Oct 2011 21:19:03 -0000
Subject: [Bro-Dev] #639: Bro 1.5.3 Compile error on BT5R1
Message-ID: <052.cde16d88eb7d39ef5e5bfa8788a4cc36@tracker.bro-ids.org>

#639: Bro 1.5.3 Compile error on BT5R1
------------------------+---------------------
 Reporter:  maxfeldman  |       Type:  Problem
   Status:  new         |   Priority:  Normal
Milestone:              |  Component:  Bro
  Version:  1.5.3       |   Keywords:
------------------------+---------------------
 I am running Backtrack Linux 5, R1.

 I was unable to compile due to FlowSrc.cc missing an include statement.
 I added "#include <errno.h>" and was then able to compile.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/639>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Sun Oct  9 20:46:48 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 10 Oct 2011 03:46:48 -0000
Subject: [Bro-Dev] #639: Bro 1.5.3 Compile error on BT5R1
In-Reply-To: <052.cde16d88eb7d39ef5e5bfa8788a4cc36@tracker.bro-ids.org>
References: <052.cde16d88eb7d39ef5e5bfa8788a4cc36@tracker.bro-ids.org>
Message-ID: <067.3b2e8f9d5a6178c456b14f5e6dc784c6@tracker.bro-ids.org>

#639: Bro 1.5.3 Compile error on BT5R1
-----------------------------+--------------------
  Reporter:  maxfeldman      |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:
 Component:  Bro             |    Version:  1.5.3
Resolution:  Solved/Applied  |   Keywords:
-----------------------------+--------------------
Changes (by robin):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 Thanks, this is already fixed in current git master.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/639#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Sun Oct  9 21:15:27 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 10 Oct 2011 04:15:27 -0000
Subject: [Bro-Dev] #534: Leak in DNS resolver
In-Reply-To: <047.a6cec0e639c569fb09138a1ce7c3bfb4@tracker.bro-ids.org>
References: <047.a6cec0e639c569fb09138a1ce7c3bfb4@tracker.bro-ids.org>
Message-ID: <062.91548e8280a2c5d43077e78d1939b7fb@tracker.bro-ids.org>

#534: Leak in DNS resolver
-----------------------------+------------------------
  Reporter:  robin           |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  High            |  Milestone:  Bro1.6
 Component:  Bro             |    Version:  git/master
Resolution:  Solved/Applied  |   Keywords:  BETA
-----------------------------+------------------------
Changes (by robin):

 * status:  new => closed
 * resolution:   => Solved/Applied


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/534#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 10 07:48:44 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 10 Oct 2011 14:48:44 -0000
Subject: [Bro-Dev] #638: Doc mode crashes
In-Reply-To: <047.7b045acd377faeccdbb32687966ab133@tracker.bro-ids.org>
References: <047.7b045acd377faeccdbb32687966ab133@tracker.bro-ids.org>
Message-ID: <062.e6cf70ef73521926b957ba5751479f89@tracker.bro-ids.org>

#638: Doc mode crashes
-----------------------------+--------------------
  Reporter:  robin           |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro1.6
 Component:  Bro             |    Version:
Resolution:  Solved/Applied  |   Keywords:  beta
-----------------------------+--------------------
Changes (by jsiwek):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 Should be fixed when [58692ef9e0dc456b8028f686a09df655a7cf135b/bro] is
 merged from fastpath.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/638#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From dnthayer at ncsa.illinois.edu  Mon Oct 10 15:06:04 2011
From: dnthayer at ncsa.illinois.edu (Daniel Thayer)
Date: Mon, 10 Oct 2011 17:06:04 -0500 (CDT)
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Modification to the
 Communication	framework API. (da9b8cc)
In-Reply-To: <201110071902.p97J2wwN002611@bro-ids.icir.org>
Message-ID: <406826675.4114.1318284364861.JavaMail.root@zimbra-1.ncsa.uiuc.edu>

It seems that this change causes Bro to fail to start (due to 'listen_port_clear'
in aux/broctl/BroControl/install.py)

-Daniel


----- Original Message -----
From: "Seth Hall" <seth at icir.org>
To: bro-commits at bro-ids.org
Sent: Friday, October 7, 2011 2:02:58 PM
Subject: [Bro-Commits] [git/bro] master: Modification to the Communication	framework API. (da9b8cc)

Repository : ssh://git at bro-ids.icir.org/bro

On branch  : master
Link       : http://tracker.bro-ids.org/bro/changeset/da9b8cc2832a79d4ee2597e2b7515ad8a703ca61/bro

>---------------------------------------------------------------

commit da9b8cc2832a79d4ee2597e2b7515ad8a703ca61
Author: Seth Hall <seth at icir.org>
Date:   Fri Oct 7 13:29:26 2011 -0400

    Modification to the Communication framework API.
    
    - Simplified the communication API and made it easier to change
      to encrypted connections by not having separate variables to
      define encrypted and unencrypted ports.
    
    - Now, to enable listening without configuring nodes just
      load the frameworks/communication/listen script.
    
    - If encrypted listening is desired set the following:
    	redef Communication::listen_encrypted=T;
    
    - Accompanying test updates.


>---------------------------------------------------------------

da9b8cc2832a79d4ee2597e2b7515ad8a703ca61
 scripts/base/frameworks/cluster/__load__.bro       |    4 +-
 scripts/base/frameworks/communication/main.bro     |   21 +++++++++++--------
 .../frameworks/communication/listen-clear.bro      |   20 -------------------
 .../policy/frameworks/communication/listen-ssl.bro |   21 --------------------
 scripts/policy/frameworks/communication/listen.bro |   12 +++++++++++
 scripts/policy/frameworks/control/controllee.bro   |    2 +-
 scripts/test-all-policy.bro                        |    3 +-
 testing/btest/coverage/bare-mode-errors.test       |    2 +-
 testing/btest/istate/events-ssl.bro                |    3 +-
 testing/btest/istate/events.bro                    |    2 +-
 testing/btest/istate/sync.bro                      |    2 +-
 .../frameworks/control/configuration_update.bro    |    2 +-
 .../scripts/base/frameworks/control/id_value.bro   |    2 +-
 .../scripts/base/frameworks/control/shutdown.bro   |    2 +-
 .../base/frameworks/logging/remote-types.bro       |    2 +-
 .../scripts/base/frameworks/logging/remote.bro     |    2 +-
 16 files changed, 38 insertions(+), 64 deletions(-)

diff --git a/scripts/base/frameworks/cluster/__load__.bro b/scripts/base/frameworks/cluster/__load__.bro
index 5aa39f9..3334164 100644
--- a/scripts/base/frameworks/cluster/__load__.bro
+++ b/scripts/base/frameworks/cluster/__load__.bro
@@ -21,10 +21,10 @@ redef peer_description = Cluster::node;
 
 # Don't load the listening script until we're a bit more sure that the
 # cluster framework is actually being enabled.
- at load frameworks/communication/listen-clear
+ at load frameworks/communication/listen
 
 ## Set the port that this node is supposed to listen on.
-redef Communication::listen_port_clear = Cluster::nodes[Cluster::node]$p;
+redef Communication::listen_port = Cluster::nodes[Cluster::node]$p;
 
 @if ( Cluster::local_node_type() == Cluster::MANAGER )
 @load ./nodes/manager
diff --git a/scripts/base/frameworks/communication/main.bro b/scripts/base/frameworks/communication/main.bro
index 2e7c948..44d6ace 100644
--- a/scripts/base/frameworks/communication/main.bro
+++ b/scripts/base/frameworks/communication/main.bro
@@ -8,12 +8,18 @@ module Communication;
 export {
 	redef enum Log::ID += { LOG };
 	
-	const default_port_ssl = 47756/tcp &redef;
-	const default_port_clear = 47757/tcp &redef;
+	## Which interface to listen on (0.0.0.0 for any interface).
+	const listen_interface = 0.0.0.0 &redef;
+	
+	## Which port to listen on.
+	const listen_port = 47757/tcp &redef;
+	
+	## This defines if a listening socket should use encryption.
+	const listen_encrypted = F &redef;
 
 	## Default compression level.  Compression level is 0-9, with 0 = no 
 	## compression.
-	global default_compression = 0 &redef;
+	global compression_level = 0 &redef;
 
 	type Info: record {
 		ts:                  time   &log;
@@ -77,11 +83,8 @@ export {
 		## Whether to use SSL-based communication.
 		ssl: bool &default = F;
 
-		## Take-over state from this host (activated by loading hand-over.bro)
-		hand_over: bool &default = F;
-
 		## Compression level is 0-9, with 0 = no compression.
-		compression: count &default = default_compression;
+		compression: count &default = compression_level;
 
 		## The remote peer.
 		peer: event_peer &optional;
@@ -135,7 +138,7 @@ function do_script_log(p: event_peer, msg: string)
 function connect_peer(peer: string)
 	{
 	local node = nodes[peer];
-	local p = node$ssl ? default_port_ssl : default_port_clear;
+	local p = listen_port;
 
 	if ( node?$p )
 		p = node$p;
@@ -238,7 +241,7 @@ event remote_connection_established(p: event_peer)
 			}
 
 		if ( ! found )
-			set_compression_level(p, default_compression);
+			set_compression_level(p, compression_level);
 		}
 
 	complete_handshake(p);
diff --git a/scripts/policy/frameworks/communication/listen-clear.bro b/scripts/policy/frameworks/communication/listen-clear.bro
deleted file mode 100644
index ea94fe2..0000000
--- a/scripts/policy/frameworks/communication/listen-clear.bro
+++ /dev/null
@@ -1,20 +0,0 @@
-##! Listen for other Bro instances to make unencrypted connections.
-
- at load base/frameworks/communication
-
-module Communication;
-
-export {
-	## Which port to listen on for clear connections.
-	const listen_port_clear = Communication::default_port_clear &redef;
-	
-	## Which IP address to bind to (0.0.0.0 for any interface).
-	const listen_if_clear = 0.0.0.0 &redef;
-	
-}
-
-event bro_init() &priority=-10
-	{
-	enable_communication();
-	listen(listen_if_clear, listen_port_clear, F);
-	}
diff --git a/scripts/policy/frameworks/communication/listen-ssl.bro b/scripts/policy/frameworks/communication/listen-ssl.bro
deleted file mode 100644
index b228289..0000000
--- a/scripts/policy/frameworks/communication/listen-ssl.bro
+++ /dev/null
@@ -1,21 +0,0 @@
-##! Listen for other Bro instances and encrypt the connection with SSL.
-
- at load base/frameworks/communication
-
-module Communication;
-
-export {
-	## Which port to listen on for SSL encrypted connections.
-	const listen_port_ssl = Communication::default_port_ssl &redef;
-	
-	## Which IP address to bind to for SSL encrypted connections
-	## (0.0.0.0 for any interface).
-	const listen_if_ssl = 0.0.0.0 &redef;
-	
-}
-
-event bro_init() &priority=-10
-	{
-	enable_communication();
-	listen(listen_if_ssl, listen_port_ssl, T);
-	}
diff --git a/scripts/policy/frameworks/communication/listen.bro b/scripts/policy/frameworks/communication/listen.bro
new file mode 100644
index 0000000..b42271b
--- /dev/null
+++ b/scripts/policy/frameworks/communication/listen.bro
@@ -0,0 +1,12 @@
+##! Loading this script will make the Bro instance listen for remote 
+##! Bro instances to connect.
+
+ at load base/frameworks/communication
+
+module Communication;
+
+event bro_init() &priority=-10
+	{
+	enable_communication();
+	listen(listen_interface, listen_port, listen_encrypted);
+	}
diff --git a/scripts/policy/frameworks/control/controllee.bro b/scripts/policy/frameworks/control/controllee.bro
index e055b8c..798ab88 100644
--- a/scripts/policy/frameworks/control/controllee.bro
+++ b/scripts/policy/frameworks/control/controllee.bro
@@ -1,7 +1,7 @@
 @load base/frameworks/control
 # If an instance is a controllee, it implicitly needs to listen for remote
 # connections.
- at load frameworks/communication/listen-clear
+ at load frameworks/communication/listen
 
 module Control;
 
diff --git a/scripts/test-all-policy.bro b/scripts/test-all-policy.bro
index 37b1679..75f7b1e 100644
--- a/scripts/test-all-policy.bro
+++ b/scripts/test-all-policy.bro
@@ -9,8 +9,7 @@
 
 # The base/ scripts are all loaded by default and not included here.
 
-# @load frameworks/communication/listen-clear.bro
-# @load frameworks/communication/listen-ssl.bro
+# @load frameworks/communication/listen.bro
 # @load frameworks/control/controllee.bro
 # @load frameworks/control/controller.bro
 @load frameworks/dpd/detect-protocols.bro
diff --git a/testing/btest/coverage/bare-mode-errors.test b/testing/btest/coverage/bare-mode-errors.test
index 3e23a73..9fd1830 100644
--- a/testing/btest/coverage/bare-mode-errors.test
+++ b/testing/btest/coverage/bare-mode-errors.test
@@ -6,6 +6,6 @@
 # when writing a new bro scripts.
 #
 # @TEST-EXEC: test -d $DIST/scripts
-# @TEST-EXEC: for script in `find $DIST/scripts -name \*\.bro -not -path '*/site/*'`; do echo $script; if echo "$script" | egrep -q 'listen-clear|listen-ssl|controllee'; then rm -rf load_attempt .bgprocs; btest-bg-run load_attempt bro -b $script; btest-bg-wait -k 2; cat load_attempt/.stderr >>allerrors; else bro -b $script 2>>allerrors; fi done || exit 0
+# @TEST-EXEC: for script in `find $DIST/scripts -name \*\.bro -not -path '*/site/*'`; do echo $script; if echo "$script" | egrep -q 'communication/listen|controllee'; then rm -rf load_attempt .bgprocs; btest-bg-run load_attempt bro -b $script; btest-bg-wait -k 2; cat load_attempt/.stderr >>allerrors; else bro -b $script 2>>allerrors; fi done || exit 0
 # @TEST-EXEC: cat allerrors | grep -v "received termination signal" | sort | uniq > unique_errors
 # @TEST-EXEC: btest-diff unique_errors
diff --git a/testing/btest/istate/events-ssl.bro b/testing/btest/istate/events-ssl.bro
index cfacae9..9110648 100644
--- a/testing/btest/istate/events-ssl.bro
+++ b/testing/btest/istate/events-ssl.bro
@@ -16,7 +16,8 @@
 
 @TEST-START-FILE sender.bro
 
- at load frameworks/communication/listen-ssl
+ at load frameworks/communication/listen
+redef Communication::listen_encrypted=T;
 
 event bro_init()
     {
diff --git a/testing/btest/istate/events.bro b/testing/btest/istate/events.bro
index ecf2f2e..a0dc494 100644
--- a/testing/btest/istate/events.bro
+++ b/testing/btest/istate/events.bro
@@ -16,7 +16,7 @@
 
 @TEST-START-FILE sender.bro
 
- at load frameworks/communication/listen-clear
+ at load frameworks/communication/listen
 
 event bro_init()
     {
diff --git a/testing/btest/istate/sync.bro b/testing/btest/istate/sync.bro
index 567bbf2..1ccdc55 100644
--- a/testing/btest/istate/sync.bro
+++ b/testing/btest/istate/sync.bro
@@ -129,7 +129,7 @@ function modify()
 	foo2 = 1234567;
 }
 
- at load frameworks/communication/listen-clear
+ at load frameworks/communication/listen
 
 event remote_connection_handshake_done(p: event_peer)
 	{
diff --git a/testing/btest/scripts/base/frameworks/control/configuration_update.bro b/testing/btest/scripts/base/frameworks/control/configuration_update.bro
index 23b4998..eb86ec5 100644
--- a/testing/btest/scripts/base/frameworks/control/configuration_update.bro
+++ b/testing/btest/scripts/base/frameworks/control/configuration_update.bro
@@ -1,4 +1,4 @@
-# @TEST-EXEC: btest-bg-run controllee  BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controllee Communication::listen_port_clear=65531/tcp 
+# @TEST-EXEC: btest-bg-run controllee  BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controllee Communication::listen_port=65531/tcp 
 # @TEST-EXEC: btest-bg-run controller  BROPATH=$BROPATH:.. bro %INPUT test-redef frameworks/control/controller Control::host=127.0.0.1 Control::host_port=65531/tcp Control::cmd=configuration_update
 # @TEST-EXEC: btest-bg-run controller2 BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controller Control::host=127.0.0.1 Control::host_port=65531/tcp Control::cmd=shutdown
 # @TEST-EXEC: btest-bg-wait 1
diff --git a/testing/btest/scripts/base/frameworks/control/id_value.bro b/testing/btest/scripts/base/frameworks/control/id_value.bro
index 9f0cb76..90a5367 100644
--- a/testing/btest/scripts/base/frameworks/control/id_value.bro
+++ b/testing/btest/scripts/base/frameworks/control/id_value.bro
@@ -1,4 +1,4 @@
-# @TEST-EXEC: btest-bg-run controllee  BROPATH=$BROPATH:.. bro %INPUT only-for-controllee frameworks/control/controllee Communication::listen_port_clear=65532/tcp 
+# @TEST-EXEC: btest-bg-run controllee  BROPATH=$BROPATH:.. bro %INPUT only-for-controllee frameworks/control/controllee Communication::listen_port=65532/tcp 
 # @TEST-EXEC: btest-bg-run controller  BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controller Control::host=127.0.0.1 Control::host_port=65532/tcp Control::cmd=id_value Control::arg=test_var
 # @TEST-EXEC: btest-bg-wait -k 1
 # @TEST-EXEC: btest-diff controller/.stdout
diff --git a/testing/btest/scripts/base/frameworks/control/shutdown.bro b/testing/btest/scripts/base/frameworks/control/shutdown.bro
index 55af973..73319a7 100644
--- a/testing/btest/scripts/base/frameworks/control/shutdown.bro
+++ b/testing/btest/scripts/base/frameworks/control/shutdown.bro
@@ -1,4 +1,4 @@
-# @TEST-EXEC: btest-bg-run controllee BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controllee Communication::listen_port_clear=65530/tcp 
+# @TEST-EXEC: btest-bg-run controllee BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controllee Communication::listen_port=65530/tcp 
 # @TEST-EXEC: btest-bg-run controller BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controller Control::host=127.0.0.1 Control::host_port=65530/tcp Control::cmd=shutdown
 # @TEST-EXEC: btest-bg-wait 1
 
diff --git a/testing/btest/scripts/base/frameworks/logging/remote-types.bro b/testing/btest/scripts/base/frameworks/logging/remote-types.bro
index 60c00e5..9af45cf 100644
--- a/testing/btest/scripts/base/frameworks/logging/remote-types.bro
+++ b/testing/btest/scripts/base/frameworks/logging/remote-types.bro
@@ -48,7 +48,7 @@ event bro_init()
 
 module Test;
 
- at load frameworks/communication/listen-clear
+ at load frameworks/communication/listen
 
 event remote_connection_handshake_done(p: event_peer)
 	{
diff --git a/testing/btest/scripts/base/frameworks/logging/remote.bro b/testing/btest/scripts/base/frameworks/logging/remote.bro
index 0b31153..b244c72 100644
--- a/testing/btest/scripts/base/frameworks/logging/remote.bro
+++ b/testing/btest/scripts/base/frameworks/logging/remote.bro
@@ -40,7 +40,7 @@ event bro_init()
 
 module Test;
 
- at load frameworks/communication/listen-clear
+ at load frameworks/communication/listen
 
 function fail(rec: Log): bool
 	{

_______________________________________________
bro-commits mailing list
bro-commits at bro-ids.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits


From robin at icir.org  Mon Oct 10 20:39:15 2011
From: robin at icir.org (Robin Sommer)
Date: Mon, 10 Oct 2011 20:39:15 -0700
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Modification to
	the	Communication framework API. (da9b8cc)
In-Reply-To: <406826675.4114.1318284364861.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
References: <201110071902.p97J2wwN002611@bro-ids.icir.org>
	<406826675.4114.1318284364861.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
Message-ID: <20111011033915.GB52721@icir.org>


On Mon, Oct 10, 2011 at 17:06 -0500, you wrote:

> It seems that this change causes Bro to fail to start (due to 'listen_port_clear'
> in aux/broctl/BroControl/install.py)

Did you make sure the submodules are all at the correct state? Try

    git submodule update --recursive --init

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From dnthayer at ncsa.illinois.edu  Tue Oct 11 08:13:59 2011
From: dnthayer at ncsa.illinois.edu (Daniel Thayer)
Date: Tue, 11 Oct 2011 10:13:59 -0500 (CDT)
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Modification to
	the	Communication framework API. (da9b8cc)
In-Reply-To: <20111011033915.GB52721@icir.org>
Message-ID: <1484769806.1493.1318346039440.JavaMail.root@zimbra-1.ncsa.uiuc.edu>

Yes, I tried that (didn't seem to make any difference).  
I also started with a fresh copy of the sources:

$ git clone --recursive ssh://git at git.bro-ids.org/bro
$ grep -R listen_port_clear bro 
bro/aux/broctl/BroControl/install.py:        print >>out, "redef Communication::listen_port_clear = %s/tcp;" % nextPort(manager)


After building and installing Bro, I did this:

$ broctl
[BroControl] > install
[BroControl] > start
starting bro ...
bro terminated immediately after starting; check output with "diag"
[BroControl] > diag

==== stderr.log
error in /myprefix/spool/policy/auto/standalone-layout.bro, line 2: "redef" used but not previously defined (Communication::listen_port_clear)


----- Original Message -----
From: "Robin Sommer" <robin at icir.org>
To: "Daniel Thayer" <dnthayer at ncsa.illinois.edu>
Cc: bro-dev at bro-ids.org
Sent: Monday, October 10, 2011 10:39:15 PM
Subject: Re: [Bro-Dev] [Bro-Commits] [git/bro] master: Modification to the	Communication framework API. (da9b8cc)


On Mon, Oct 10, 2011 at 17:06 -0500, you wrote:

> It seems that this change causes Bro to fail to start (due to 'listen_port_clear'
> in aux/broctl/BroControl/install.py)

Did you make sure the submodules are all at the correct state? Try

    git submodule update --recursive --init

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From seth at icir.org  Tue Oct 11 08:58:23 2011
From: seth at icir.org (Seth Hall)
Date: Tue, 11 Oct 2011 11:58:23 -0400
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Modification to
	the	Communication framework API. (da9b8cc)
In-Reply-To: <1484769806.1493.1318346039440.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
References: <1484769806.1493.1318346039440.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
Message-ID: <E09C2743-E621-4243-8B87-3DF95A6AAFEF@icir.org>


On Oct 11, 2011, at 11:13 AM, Daniel Thayer wrote:

> $ git clone --recursive ssh://git at git.bro-ids.org/bro
> $ grep -R listen_port_clear bro 
> bro/aux/broctl/BroControl/install.py:        print >>out, "redef Communication::listen_port_clear = %s/tcp;" % nextPort(manager)


Ugh, sorry about that.  It'll be fixed when you pull and update your submodules.  Thanks for catching that.
  
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From robin at icir.org  Tue Oct 11 10:35:46 2011
From: robin at icir.org (Robin Sommer)
Date: Tue, 11 Oct 2011 10:35:46 -0700
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Modification to
	the	Communication framework API. (da9b8cc)
In-Reply-To: <E09C2743-E621-4243-8B87-3DF95A6AAFEF@icir.org>
References: <1484769806.1493.1318346039440.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
	<E09C2743-E621-4243-8B87-3DF95A6AAFEF@icir.org>
Message-ID: <20111011173546.GC81098@icir.org>


On Tue, Oct 11, 2011 at 11:58 -0400, you wrote:

> Ugh, sorry about that.  It'll be fixed when you pull and update your submodules.  Thanks for catching that.

Now I'm wondering why it was working fine for me ... Odd.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From jsiwek at ncsa.illinois.edu  Tue Oct 11 10:42:59 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Tue, 11 Oct 2011 12:42:59 -0500
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Modification to
	the	Communication framework API. (da9b8cc)
In-Reply-To: <20111011173546.GC81098@icir.org>
References: <1484769806.1493.1318346039440.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
	<E09C2743-E621-4243-8B87-3DF95A6AAFEF@icir.org>
	<20111011173546.GC81098@icir.org>
Message-ID: <FB0A33C3-686C-45B2-AF5E-19131115EFE5@ncsa.illinois.edu>

>> Ugh, sorry about that.  It'll be fixed when you pull and update your submodules.  Thanks for catching that.
> 
> Now I'm wondering why it was working fine for me ... Odd.

Maybe you hadn't done a `broctl install` yet and so everything still consistently used listen_port_clear?

- Jon

From robin at icir.org  Tue Oct 11 10:47:35 2011
From: robin at icir.org (Robin Sommer)
Date: Tue, 11 Oct 2011 10:47:35 -0700
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Modification to
	the	Communication framework API. (da9b8cc)
In-Reply-To: <FB0A33C3-686C-45B2-AF5E-19131115EFE5@ncsa.illinois.edu>
References: <1484769806.1493.1318346039440.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
	<E09C2743-E621-4243-8B87-3DF95A6AAFEF@icir.org>
	<20111011173546.GC81098@icir.org>
	<FB0A33C3-686C-45B2-AF5E-19131115EFE5@ncsa.illinois.edu>
Message-ID: <20111011174734.GF81098@icir.org>


On Tue, Oct 11, 2011 at 12:42 -0500, you wrote:

> Maybe you hadn't done a `broctl install` yet and so everything still consistently used listen_port_clear?

Actually I was pretty sure I did, but well, perhaps not.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From seth at icir.org  Tue Oct 11 11:35:47 2011
From: seth at icir.org (Seth Hall)
Date: Tue, 11 Oct 2011 14:35:47 -0400
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Modification to the
	Communication framework API. (da9b8cc)
In-Reply-To: <20111011174734.GF81098@icir.org>
References: <1484769806.1493.1318346039440.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
	<E09C2743-E621-4243-8B87-3DF95A6AAFEF@icir.org>
	<20111011173546.GC81098@icir.org>
	<FB0A33C3-686C-45B2-AF5E-19131115EFE5@ncsa.illinois.edu>
	<20111011174734.GF81098@icir.org>
Message-ID: <B154866E-3055-4204-A984-DB02E77C6414@icir.org>


On Oct 11, 2011, at 1:47 PM, Robin Sommer wrote:

> On Tue, Oct 11, 2011 at 12:42 -0500, you wrote:
> 
>> Maybe you hadn't done a `broctl install` yet and so everything still consistently used listen_port_clear?
> 
> Actually I was pretty sure I did, but well, perhaps not.


I thought I did too.  It's fixed now at least.

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From dnthayer at ncsa.illinois.edu  Tue Oct 11 13:12:29 2011
From: dnthayer at ncsa.illinois.edu (Daniel Thayer)
Date: Tue, 11 Oct 2011 15:12:29 -0500 (CDT)
Subject: [Bro-Dev] minimum required version of python?
In-Reply-To: <1403710164.3864.1318362553806.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
Message-ID: <1767338535.4406.1318363949331.JavaMail.root@zimbra-1.ncsa.uiuc.edu>

Do we require a minimum version of Python in order to use Bro?

For example, I've noticed various uses of conditional expressions
 (true_value if condition else false_value) in the code (this
feature was introduced in Python 2.5), and since RHEL5 has Python 2.4,
then Bro (or at least the parts that rely on Python) won't work on RHEL5.

I was able to build Bro on a RHEL5 machine without (apparently)
any errors, but when I try to run broctl, it fails with this message:

SyntaxError: invalid syntax

The only place where I can see any mention of a minimum required
version of Python is in the broctl README, where it says Python >= 2.4
is required.
 

-Daniel

From bro at tracker.bro-ids.org  Wed Oct 12 06:55:12 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 12 Oct 2011 13:55:12 -0000
Subject: [Bro-Dev] #640: BiFs to enable or disable events.
Message-ID: <046.1f1e3ac9563bd29d452598383f9647bd@tracker.bro-ids.org>

#640: BiFs to enable or disable events.
-----------------------------+--------------------
 Reporter:  seth             |      Owner:
     Type:  Feature Request  |     Status:  new
 Priority:  Normal           |  Milestone:  Bro1.7
Component:  Bro              |    Version:
 Keywords:  language         |
-----------------------------+--------------------
 We need BiFs to enable/disable event handlers.  The existing
 enable_event_group and disable_event_group functions push too much into
 the core and are too rigid.

 Even better would be if we had some way to place limited preconditions on
 event handlers.  I would really like to be able to do this::

 {{{
 redef Event::policy += {
     ["prevent-port-53-dns-requests"] = [$if="port 53", $ev=dns_request,
 $action=Event::DISABLE],
     ["no-dns-responses"] = [$ev=dns_response, $action=Event::DISABLE],
     ["
 };
 }}}

 I'm trying to follow the general API style that we've been following with
 other frameworks but i'm using that a quasi-bpf filter in place of a
 predicate since this would need to be extremely fast if it were to offer
 any benefit but there is probably lots of room for further discussion
 here.  The other thing I don't like is that the way I defined it,
 Event::policy would be a const and only definable at startup.  It would be
 very helpful to be able to write Bro scripts that can tune this at
 runtime.

 I think ultimately this is two tickets.  One for creating the correct BiFs
 after figuring out all of the requirements and then creating a framework
 overtop of the BiFs to make it easier to use.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/640>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct 12 07:00:21 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 12 Oct 2011 14:00:21 -0000
Subject: [Bro-Dev] #640: BiFs to enable or disable events.
In-Reply-To: <046.1f1e3ac9563bd29d452598383f9647bd@tracker.bro-ids.org>
References: <046.1f1e3ac9563bd29d452598383f9647bd@tracker.bro-ids.org>
Message-ID: <061.7da15c72d80db8ab8c56912bedbe2b15@tracker.bro-ids.org>

#640: BiFs to enable or disable events.
------------------------------+----------------------
  Reporter:  seth             |      Owner:
      Type:  Feature Request  |     Status:  new
  Priority:  Normal           |  Milestone:  Bro1.7
 Component:  Bro              |    Version:
Resolution:                   |   Keywords:  language
------------------------------+----------------------
Description changed by seth:

Old description:

> We need BiFs to enable/disable event handlers.  The existing
> enable_event_group and disable_event_group functions push too much into
> the core and are too rigid.
>
> Even better would be if we had some way to place limited preconditions on
> event handlers.  I would really like to be able to do this::
>
> {{{
> redef Event::policy += {
>     ["prevent-port-53-dns-requests"] = [$if="port 53", $ev=dns_request,
> $action=Event::DISABLE],
>     ["no-dns-responses"] = [$ev=dns_response, $action=Event::DISABLE],
>     ["
> };
> }}}
>
> I'm trying to follow the general API style that we've been following with
> other frameworks but i'm using that a quasi-bpf filter in place of a
> predicate since this would need to be extremely fast if it were to offer
> any benefit but there is probably lots of room for further discussion
> here.  The other thing I don't like is that the way I defined it,
> Event::policy would be a const and only definable at startup.  It would
> be very helpful to be able to write Bro scripts that can tune this at
> runtime.
>
> I think ultimately this is two tickets.  One for creating the correct
> BiFs after figuring out all of the requirements and then creating a
> framework overtop of the BiFs to make it easier to use.

New description:

 We need BiFs to enable/disable event handlers.  The existing
 enable_event_group and disable_event_group functions push too much into
 the core and are too rigid.

 Even better would be if we had some way to place limited preconditions on
 event handlers.  I would really like to be able to do this::

 {{{
 redef Event::policy += {
     ["prevent-port-53-dns-requests"] = [$if="port 53", $ev=dns_request,
 $action=Event::DISABLE],
     ["no-dns-responses"] = [$ev=dns_response, $action=Event::DISABLE],
 };
 }}}

 I'm trying to follow the general API style that we've been following with
 other frameworks but i'm using that a quasi-bpf filter in place of a
 predicate since this would need to be extremely fast if it were to offer
 any benefit but there is probably lots of room for further discussion
 here.  The other thing I don't like is that the way I defined it,
 Event::policy would be a const and only definable at startup.  It would be
 very helpful to be able to write Bro scripts that can tune this at
 runtime.

 I think ultimately this is two tickets.  One for creating the correct BiFs
 after figuring out all of the requirements and then creating a framework
 overtop of the BiFs to make it easier to use.

--

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/640#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Wed Oct 12 09:09:56 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 12 Oct 2011 09:09:56 -0700
Subject: [Bro-Dev] minimum required version of python?
In-Reply-To: <1767338535.4406.1318363949331.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
References: <1403710164.3864.1318362553806.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
	<1767338535.4406.1318363949331.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
Message-ID: <20111012160956.GC29013@icir.org>


On Tue, Oct 11, 2011 at 15:12 -0500, you wrote:

> Do we require a minimum version of Python in order to use Bro?

For BroControl it used to be 2.4 but that's apparently not the case
anymore if there are 2.5 constructs in there now. If it's easy to
eliminate things that prevent it from running with 2.4, we could do
that; but on the other hand I'm tempted to just say we require at
least 2.6. That's not exactly new anymore, and if people can't find
packages, it's actually straight-forward to compile from source (I've
done it many times :-)

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From robin at icir.org  Wed Oct 12 09:15:20 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 12 Oct 2011 09:15:20 -0700
Subject: [Bro-Dev] Beta schedule
Message-ID: <20111012161520.GE29013@icir.org>

Sounds like everybody feels we're getting ready to do our beta
release. Let's aim for early next week. Here's what I think is left to
do:

    - #511: Cleanup distribution a bit more (Robin)
    - #601: Write docs for notice framework (Seth)
    - #601: Cleanup and organize the documentation we have (Robin[1])   
    - #544: Get scan.bro into shape (Seth)
    - #622: Some of these seem to addressed, others not. (John, can
            you take the lead closing this ticket? Some might just need a
            decision/reminder if what's we're currenty doing is the right
            thing.)

I'm fine pushing the rest to the final release. Let me know if anybody
see any other showstopper.

Robin

[1] I'll make sure things get online and make a pass over the content
if time permits. Anybody else, please also look through and edit what
you find.

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From seth at icir.org  Wed Oct 12 09:19:19 2011
From: seth at icir.org (Seth Hall)
Date: Wed, 12 Oct 2011 12:19:19 -0400
Subject: [Bro-Dev] minimum required version of python?
In-Reply-To: <20111012160956.GC29013@icir.org>
References: <1403710164.3864.1318362553806.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
	<1767338535.4406.1318363949331.JavaMail.root@zimbra-1.ncsa.uiuc.edu>
	<20111012160956.GC29013@icir.org>
Message-ID: <94921F94-2EA1-4901-AFE4-8D99763FCC7C@icir.org>


On Oct 12, 2011, at 12:09 PM, Robin Sommer wrote:

> but on the other hand I'm tempted to just say we require at least 2.6.


That sounds best to me.

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/


From seth at icir.org  Wed Oct 12 09:22:31 2011
From: seth at icir.org (Seth Hall)
Date: Wed, 12 Oct 2011 12:22:31 -0400
Subject: [Bro-Dev] Beta schedule
In-Reply-To: <20111012161520.GE29013@icir.org>
References: <20111012161520.GE29013@icir.org>
Message-ID: <DEF61527-D956-421A-8F3B-EFF1F9344638@icir.org>


On Oct 12, 2011, at 12:15 PM, Robin Sommer wrote:

> I'm fine pushing the rest to the final release. Let me know if anybody
> see any other showstopper.


Sounds great to me.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/


From bro at tracker.bro-ids.org  Wed Oct 12 10:03:25 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 12 Oct 2011 17:03:25 -0000
Subject: [Bro-Dev] #622: Installation issues
In-Reply-To: <047.9f791003db697f134f198a006352d1c5@tracker.bro-ids.org>
References: <047.9f791003db697f134f198a006352d1c5@tracker.bro-ids.org>
Message-ID: <062.fdc03168951f66fb2a93cd57c1d13877@tracker.bro-ids.org>

#622: Installation issues
----------------------+------------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by jsiwek):

 > - It creates "$prefix/logs". I don't remember whether we discussed
 >   this already, but is that a good default location for logs?

 It's slightly more complicated, generally it's `$prefix/logs` when
 compiling from source, `/var/opt/bro/logs` for the current binary
 packages, and `/var/log/bro` in the special case of `$prefix == /usr`.
 They're all reasonable IMO, plus they're configurable afterwards in
 `broctl.cfg` via `LogDir`, but do you think it would be improved by

 1) consistently use "log" versus "logs"
 2) in the general case, use `$prefix/log/bro`

 > - Was there a reason we install the spool pieces directly into
 >   $prefix/spool/ instead of into $prefix/spool/bro/?

 There's again minor variation similar to the way the log dir is explained
 above, but I don't think there's a real reason why the later isn't used;
 I'll try it out and see.

 > - test-all-policy.bro ends up in share/bro, which doesn't seem right.

 Yeah, I'll take care of it.

 > - Do we really need to install share/bro/site/local-proxy.bro? It's
 >   empty and unlikley that many people will want to edit it.

 Seems pretty harmless to me.  Seth, thoughts?

 > - Thoughts about local.bro:
 >
 >     * Why is "@load protocols/http/detect-webapps" commented out? We
 >     should add a comment explaining when one would want to include it.
 >
 >     * "Requires that the Site::local_zones variable". We should add
 >     where/how to do that. Perhaps an empty definition right in
 >     local.bro?

 Seth, can you get this?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/622#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct 12 10:10:56 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 12 Oct 2011 17:10:56 -0000
Subject: [Bro-Dev] #622: Installation issues
In-Reply-To: <047.9f791003db697f134f198a006352d1c5@tracker.bro-ids.org>
References: <047.9f791003db697f134f198a006352d1c5@tracker.bro-ids.org>
Message-ID: <062.1c81f802d64f6c591bfc17fa4c514af3@tracker.bro-ids.org>

#622: Installation issues
----------------------+------------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by robin):

 On Wed, Oct 12, 2011 at 17:03 -0000, you wrote:

 >  > - It creates "$prefix/logs". I don't remember whether we discussed
 >  >   this already, but is that a good default location for logs?

 >  > - Was there a reason we install the spool pieces directly into
 >  >   $prefix/spool/ instead of into $prefix/spool/bro/?

 Thinking about this again, I believe my problem comes from the
 difference between using a prefix of (for example) (1) /usr/local vs
 (2) /usr/local/bro.

 The current paths make sense for (2), but not for (1). I know we
 settled on going with the model (2), but that still doesn't feel right
 to me as it's different to what any other software does.

 Anyway, if I'm the only one feeling this way, I'm fine leaving things
 as they are at this point; changing would have implications for the
 quickstart guide etc.

 (I'm only talking about from-source installation here, the rest is
 fine).

 Robin

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/622#comment:5>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct 12 10:38:12 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 12 Oct 2011 17:38:12 -0000
Subject: [Bro-Dev] #622: Installation issues
In-Reply-To: <047.9f791003db697f134f198a006352d1c5@tracker.bro-ids.org>
References: <047.9f791003db697f134f198a006352d1c5@tracker.bro-ids.org>
Message-ID: <062.9d4165307c2c031ae21d90c7f02c62d6@tracker.bro-ids.org>

#622: Installation issues
----------------------+------------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by jsiwek):

 > Thinking about this again, I believe my problem comes from the
 > difference between using a prefix of (for example) (1) /usr/local vs
 > (2) /usr/local/bro.
 >
 > The current paths make sense for (2), but not for (1). I know we
 > settled on going with the model (2), but that still doesn't feel right
 > to me as it's different to what any other software does.

 Can you give an example of what's specifically different?  Is it that the
 default prefix for most other software is `/usr/local` ?

 > Anyway, if I'm the only one feeling this way, I'm fine leaving things
 > as they are at this point; changing would have implications for the
 > quickstart guide etc.

 My perspective right now is that the FHS says: "Local placement of local
 files is a local issue, so FHS does not attempt to usurp system
 administrators."

 Given that we've provided a way for the installer to change the root
 install prefix, and the log/spool dirs independently of it, the current
 setup seems adequate.  So I won't touch anything right now (unless someone
 jumps into the discussion to help us reason out a better approach).

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/622#comment:6>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct 12 10:48:02 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 12 Oct 2011 17:48:02 -0000
Subject: [Bro-Dev] #622: Installation issues
In-Reply-To: <047.9f791003db697f134f198a006352d1c5@tracker.bro-ids.org>
References: <047.9f791003db697f134f198a006352d1c5@tracker.bro-ids.org>
Message-ID: <062.89cfb666a022527bc3f2d2fbba06bad0@tracker.bro-ids.org>

#622: Installation issues
----------------------+------------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by jsiwek):

 In [d84de52ee000d6529b1a20bebd7618dd0767f227/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="d84de52ee000d6529b1a20bebd7618dd0767f227"
 Don't install test-all-policy.bro script as it's for testing only.

 Addresses #622
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/622#comment:7>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From gregor at icir.org  Wed Oct 12 15:12:38 2011
From: gregor at icir.org (Gregor Maier)
Date: Wed, 12 Oct 2011 15:12:38 -0700
Subject: [Bro-Dev] Beta schedule
In-Reply-To: <20111012161520.GE29013@icir.org>
References: <20111012161520.GE29013@icir.org>
Message-ID: <4E9610D6.5020701@icir.org>

Maybe I missed them but what's with (all from #601):

  * Upgrade guide
  * How-to for new logging framework (how to filter, log file format,
    maybe how to add columns to the log file (i.e., how to use
    c$PROTOCOL))
  * Complete script reference (though that can wait to final release)


cu
gregor



-- 
Gregor Maier
<gregor at icir.org>  <gregor at icsi.berkeley.edu>
Int. Computer Science Institute (ICSI)
1947 Center St., Ste. 600
Berkeley, CA 94704, USA
http://www.icir.org/gregor/

From christian at icir.org  Wed Oct 12 15:53:18 2011
From: christian at icir.org (Christian Kreibich)
Date: Wed, 12 Oct 2011 15:53:18 -0700
Subject: [Bro-Dev] A few questions around Broccoli and its Ruby bindings
Message-ID: <4E961A5E.3050700@icir.org>

I guess it's a sign of the times, but it seems I'm unable to install
Broccoli these days. :) Perhaps somebody could help me out. I'm trying
to work on http://tracker.bro-ids.org/bro/ticket/606 and am trying to
reproduce the scenario Seth mentions in the ticket.

For starters, I've configured a fresh recursive git checkout of the bro
repository via

  $ ./configure --enable-debug --prefix=$HOME/inst/

This built fine. I'd now like to install just Broccoli (into my home),
so I went into build/aux/broccoli and did a "make install". Is that the
right thing to do? It seems so, because it gets close:

  $ make install
  [ 40%] Built target broccoli
  [ 80%] Built target broccoliStatic
  [ 86%] Built target broconftest
  [ 86%] Built target broconn
  [ 86%] Built target broenum
  [ 86%] Built target brohose
  [ 93%] Built target broping
  [ 93%] Built target brotable
  [100%] Built target _broccoli_intern
  [100%] Built target broccoli_ext
  Install the project...
  -- Install configuration: "Debug"
  -- Installing: /home/kreibich/inst/etc/broccoli.conf
  -- Installing: /home/kreibich/inst/bin/broccoli-config
  -- Installing: /home/kreibich/inst/lib/libbroccoli.so.3.0.0
  -- Installing: /home/kreibich/inst/lib/libbroccoli.so.3
  -- Installing: /home/kreibich/inst/lib/libbroccoli.so
  -- Set runtime path of "/home/kreibich/inst/lib/libbroccoli.so.3.0.0"
to "/home/kreibich/inst/lib"
  -- Installing: /home/kreibich/inst/lib/libbroccoli.a
  -- Installing: /home/kreibich/inst/include/broccoli.h
  -- Installing: /home/kreibich/inst/lib/broctl/broccoli.py
  -- Installing: /home/kreibich/inst/lib/broctl/_broccoli_intern.so
  -- Set runtime path of
"/home/kreibich/inst/lib/broctl/_broccoli_intern.so" to
"/home/kreibich/inst/lib"

  -- Installing: /usr/lib/ruby/site_ruby/1.8/broccoli.rb
  CMake Error at bindings/broccoli-ruby/cmake_install.cmake:38 (FILE):
    file INSTALL cannot copy file

"/home/kreibich/devel/bro/aux/broccoli/bindings/broccoli-ruby/lib/broccoli.rb"
    to "/usr/lib/ruby/site_ruby/1.8/broccoli.rb".
  Call Stack (most recent call first):
    cmake_install.cmake:65 (INCLUDE)


  make: *** [install] Error 1

So it looks as though the Broccoli library installed fine but some Ruby
stuff still wanted to go into the system directory. Is that a deficiency
or am I not doing this right?

Fwiw, I tried to move on and checked out the broccoli-ruby repository
(which differs from bro/aux/broccoli/bindings/broccoli-ruby, correct? At
least the former seems to have way more content). This, when running the
same configure command, tells me it cannot find the Broccoli package.
broccoli-config is now installed, but I presume the package is referring
to something cmake-related that didn't yet get installed.

Help, thanks,
-C.

From seth at icir.org  Wed Oct 12 17:07:41 2011
From: seth at icir.org (Seth Hall)
Date: Wed, 12 Oct 2011 20:07:41 -0400
Subject: [Bro-Dev] A few questions around Broccoli and its Ruby bindings
In-Reply-To: <4E961A5E.3050700@icir.org>
References: <4E961A5E.3050700@icir.org>
Message-ID: <19989E62-83B6-4A02-AEBD-E12F08266728@icir.org>


On Oct 12, 2011, at 6:53 PM, Christian Kreibich wrote:

> -- Installing: /usr/lib/ruby/site_ruby/1.8/broccoli.rb
> CMake Error at bindings/broccoli-ruby/cmake_install.cmake:38 (FILE):
>   file INSTALL cannot copy file
<snip>
> So it looks as though the Broccoli library installed fine but some Ruby
> stuff still wanted to go into the system directory. Is that a deficiency
> or am I not doing this right?

Yeah, I don't like how this works but I wasn't sure how to handle it better.  The ruby bindings always want to install to the system's ruby directory now (--prefix is currently ignored) because if I don't install them there it's a mess to use the library.  It's no different from any other language binding in that respect.  Perhaps there's a better way to handle this situation?

> Fwiw, I tried to move on and checked out the broccoli-ruby repository
> (which differs from bro/aux/broccoli/bindings/broccoli-ruby, correct? At
> least the former seems to have way more content). This, when running the
> same configure command, tells me it cannot find the Broccoli package.
> broccoli-config is now installed, but I presume the package is referring
> to something cmake-related that didn't yet get installed.


I guess that depends on if you're using the setup.rb script to install or using the cmake stuff to install (two way to install the same thing unfortunately).  I haven't tested the old setup.rb layout, but I think it should still work.  I do think that you will need to have broccoli-config in your path for either method though.  

The repository shouldn't have anything different either, they're the same thing since the broccoli-ruby repository is just a submodule of broccoli.

(thanks for looking into the ticket btw!)

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From vallentin at icir.org  Wed Oct 12 17:31:58 2011
From: vallentin at icir.org (Matthias Vallentin)
Date: Wed, 12 Oct 2011 17:31:58 -0700
Subject: [Bro-Dev] A few questions around Broccoli and its Ruby bindings
In-Reply-To: <19989E62-83B6-4A02-AEBD-E12F08266728@icir.org>
References: <4E961A5E.3050700@icir.org>
	<19989E62-83B6-4A02-AEBD-E12F08266728@icir.org>
Message-ID: <1318465603-sup-8514@samurai.local>

> The ruby bindings always want to install to the system's ruby
> directory now (--prefix is currently ignored) because if I don't
> install them there it's a mess to use the library.  

Would you mind elaborating on the pain point a little bit? I do remember
that is an issue with libraries installed as gems, but can't recall that
it caused a lot of agony.

> It's no different from any other language binding in that respect.
> Perhaps there's a better way to handle this situation?

What comes to mind is installation in ~/.gem of the current user? This
is still not respecting PREFIX, yet at least less invasive than a
system-wide installation.

    Matthias

From jsiwek at ncsa.illinois.edu  Wed Oct 12 19:50:10 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Wed, 12 Oct 2011 21:50:10 -0500
Subject: [Bro-Dev] Beta schedule
In-Reply-To: <4E9610D6.5020701@icir.org>
References: <20111012161520.GE29013@icir.org> <4E9610D6.5020701@icir.org>
Message-ID: <E2398D93-D150-4C41-BB79-8132D1C0FAA2@ncsa.illinois.edu>

> Maybe I missed them but what's with (all from #601):
> 
>  * Upgrade guide

tracked in more detail here: http://tracker.bro-ids.org/bro/ticket/510

and put up on the site here: http://www.bro-ids.org/documentation/upgrade.html

>  * How-to for new logging framework (how to filter, log file format,
>    maybe how to add columns to the log file (i.e., how to use
>    c$PROTOCOL))

Think that's mostly done: http://www.bro-ids.org/development/logging.html

>  * Complete script reference (though that can wait to final release)

Does that mean the generated bro script docs (think I'm going to start calling them "Broxygen docs" for short) ?

If so, I didn't look at how far Robin got on the www new-git branch, but part of that will help incorporate those docs into the website.

Another general task might be to cleanup the scattered TODO/XXX's.

- Jon

From bro at tracker.bro-ids.org  Fri Oct 14 08:24:28 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 14 Oct 2011 15:24:28 -0000
Subject: [Bro-Dev] #641: Multiple nested VLAN tags don't work
Message-ID: <046.1068b8313acf350f78e8f8b5df577789@tracker.bro-ids.org>

#641: Multiple nested VLAN tags don't work
---------------------+------------------------
 Reporter:  seth     |      Owner:
     Type:  Problem  |     Status:  new
 Priority:  Normal   |  Milestone:
Component:  Bro      |    Version:  git/master
 Keywords:           |
---------------------+------------------------
 If traffic is seen with two vlan tags, Bro is currently failing.  The
 assumption in the code now is that if there is a vlan tag after the
 ethernet header there is a single vlan header and then IP is within that.
 We need to be checking the "type" field of the vlan tag to see what is
 contained within it and pass it back to the input again after stripping
 that header.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/641>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 14 09:00:33 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 14 Oct 2011 16:00:33 -0000
Subject: [Bro-Dev] #641: Multiple nested VLAN tags don't work
In-Reply-To: <046.1068b8313acf350f78e8f8b5df577789@tracker.bro-ids.org>
References: <046.1068b8313acf350f78e8f8b5df577789@tracker.bro-ids.org>
Message-ID: <061.79dd52a3f9126528f7026c74c947c9ce@tracker.bro-ids.org>

#641: Multiple nested VLAN tags don't work
----------------------+------------------------
  Reporter:  seth     |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------

Comment (by robin):

 Do you have a trace?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/641#comment:0>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 17 07:40:17 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 17 Oct 2011 14:40:17 -0000
Subject: [Bro-Dev] #514: NMI scripts under version control
In-Reply-To: <050.07eb55afaed5089f5ed7b9ac51224efd@tracker.bro-ids.org>
References: <050.07eb55afaed5089f5ed7b9ac51224efd@tracker.bro-ids.org>
Message-ID: <065.7349062f9bd85b3f1bd28f892175cdfd@tracker.bro-ids.org>

#514: NMI scripts under version control
-----------------------+----------------------
  Reporter:  appleman  |      Owner:  dnthayer
      Type:  Task      |     Status:  accepted
  Priority:  Low       |  Milestone:
 Component:  Bro       |    Version:
Resolution:            |   Keywords:  nmi
-----------------------+----------------------
Changes (by dnthayer):

 * owner:   => dnthayer
 * status:  new => accepted


Comment:

 I will create a new repository (called "nmi") for these scripts.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/514#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From christian at icir.org  Mon Oct 17 18:12:03 2011
From: christian at icir.org (Christian Kreibich)
Date: Mon, 17 Oct 2011 18:12:03 -0700
Subject: [Bro-Dev] A few questions around Broccoli and its Ruby bindings
In-Reply-To: <19989E62-83B6-4A02-AEBD-E12F08266728@icir.org>
References: <4E961A5E.3050700@icir.org>
	<19989E62-83B6-4A02-AEBD-E12F08266728@icir.org>
Message-ID: <4E9CD263.7020402@icir.org>

On 10/12/2011 05:07 PM, Seth Hall wrote:
> Yeah, I don't like how this works but I wasn't sure how to handle it
> better.  The ruby bindings always want to install to the system's
> ruby directory now (--prefix is currently ignored) because if I don't
> install them there it's a mess to use the library.  It's no different
> from any other language binding in that respect.  Perhaps there's a
> better way to handle this situation?

I just noticed you already offer --disable-{python,ruby}, so at least
normally there is a way around the problem when one doesn't care about
the bindings.

Anyway, no biggie -- I'll install for now and move on to looking at the
actual bug.

> The repository shouldn't have anything different either, they're the
> same thing since the broccoli-ruby repository is just a submodule of
> broccoli.

That is so weird! I just wiped the two checkouts and repeated them, and
now the directory contents are indeed the same. I could swear I did the
same before, but apparently I didn't.

-C.

From robin at icir.org  Mon Oct 17 21:27:46 2011
From: robin at icir.org (Robin Sommer)
Date: Mon, 17 Oct 2011 21:27:46 -0700
Subject: [Bro-Dev] Beta schedule
In-Reply-To: <20111012161520.GE29013@icir.org>
References: <20111012161520.GE29013@icir.org>
Message-ID: <20111018042746.GA45062@icir.org>


On Wed, Oct 12, 2011 at 09:15 -0700, I wrote:

> Sounds like everybody feels we're getting ready to do our beta
> release. Let's aim for early next week.

Updates:

- I have spent some time redoing how git files are pulled in by
  www.bro-ids.org. That now allows us to have separate sets of docs
  for release, beta, and master; and generally makes usage easier.

- I have also cleaned up the submodules to unify README structure,
  licenses, etc.

- I'm making releases (with tarballs) for all the submodules, except
  broctl. There shouldn't be changing much for them during the Bro
  beta period, and even if, we can always release new versions. Jus
  trying to get them out of the way.

I'm planing to push all this out tomorrow.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Mon Oct 17 22:14:02 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 18 Oct 2011 05:14:02 -0000
Subject: [Bro-Dev] #606: broccoli and connection records
In-Reply-To: <046.83cc834c3a71aa4f378a2b7bec3e66a4@tracker.bro-ids.org>
References: <046.83cc834c3a71aa4f378a2b7bec3e66a4@tracker.bro-ids.org>
Message-ID: <061.616033804d3dae8897eecf245182e9d9@tracker.bro-ids.org>

#606: broccoli and connection records
-----------------------+-----------------
  Reporter:  seth      |      Owner:
      Type:  Problem   |     Status:  new
  Priority:  Normal    |  Milestone:
 Component:  Broccoli  |    Version:
Resolution:            |   Keywords:
-----------------------+-----------------

Comment (by kreibich):

 I've looked at this for a while now. Unfortunately I don't yet have a
 solution, but I have a suspicion as to what's going on. Both you and
 Matthias (in his email from Sep 26) mentioned c$conn. This confused me at
 first because I didn't see it in the connection record definition, and I
 haven't used this stuff in a while. I then saw that
 base/protocols/conn/main.bro [*] redefs the connection record to include
 conn, which is, uhm, a big record. In particular, it looks like it
 includes fields of types Broccoli does not yet support. This line in the
 output is probably key:

 {{{
   59970 1315678636.044779 /tmp/tmp/bro/aux/broccoli/src/bro_sobject.c/109
 Creation of object type 0x8a0a failed.
 }}}

 0x8a0a = 0x8000 | 0x0a00 | 0x000a = a serialized object | that is a type |
 that is an '''enum'''.

 After that it all goes down the tubes. I'm not sure why things aren't
 recovering better, but the problem very likely isn't that Broccoli cannot
 handle an optional, still-null '''value''' that Bro sends, it's that
 Broccoli needs to understand the corresponding '''type''', sent first, in
 its entirety. I could be wrong and Bro isn't in fact sending the optional
 part of the type if the corresponding value doesn't actually need that
 type -- I need to dig further to figure it out. Alas, I don't think I can
 get this done before tomorrow's release.

 [*] OMG you killed bro.init! I am in awe!

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/606#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 17 22:14:51 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 18 Oct 2011 05:14:51 -0000
Subject: [Bro-Dev] #606: broccoli and connection records
In-Reply-To: <046.83cc834c3a71aa4f378a2b7bec3e66a4@tracker.bro-ids.org>
References: <046.83cc834c3a71aa4f378a2b7bec3e66a4@tracker.bro-ids.org>
Message-ID: <061.7b5cfbb5c41d2b9c62dc7ac28ddfb0f8@tracker.bro-ids.org>

#606: broccoli and connection records
-----------------------+----------------------
  Reporter:  seth      |      Owner:  kreibich
      Type:  Problem   |     Status:  accepted
  Priority:  Normal    |  Milestone:
 Component:  Broccoli  |    Version:
Resolution:            |   Keywords:
-----------------------+----------------------
Changes (by kreibich):

 * owner:   => kreibich
 * status:  new => accepted


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/606#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Tue Oct 18 12:36:34 2011
From: robin at icir.org (Robin Sommer)
Date: Tue, 18 Oct 2011 12:36:34 -0700
Subject: [Bro-Dev] Packaging (Re:  Beta schedule)
In-Reply-To: <20111018042746.GA45062@icir.org>
References: <20111012161520.GE29013@icir.org> <20111018042746.GA45062@icir.org>
Message-ID: <20111018193634.GB16748@icir.org>

> - I'm making releases (with tarballs) for all the submodules.

I'm trying to build tar-files for all our submodules but I'm running
into a number of things.

First, afaics, not all modules yet provide the capability to create
the tarballs (or at least I can't directly figure out how).
Specifically:

    - capstats, binpac, and bro-aux: they have "dist" Makefile
      targets, but running thise gives me:

        make[1]: *** No rule to make target `package_source'.  Stop.

    - broccoli-ruby: No "dist" Makefile target and I have no idea how
    to do that for Ruby packages (I've just added an empty dummy one.)

Then, the top-level Bro "make-src-packages" also builts the Broccoli
and BroControl tgz. I think it would be better to leave that to the
submodules themselves, otherwise things get inconsistent (because we
have all these further submodules as well, which do it themselves). 
So I've removed those lines from make-src-packages.

I also noticed that Bro's make-src-packages essentially just tars
everything that's in the source directory (except for some tmp files
etc.). There's a problem with that: I'm sure that sometimes I will
have additional files lying around there which will then end up in the
distribution. Is there a way to make sure that only stuff under
version control gets actually into the tgz? (And please remind me:
what was the reason that we can't just let CMake do the tarballs?)


Here's how I'd like to do the tarballs: I've written a script that
recursively goes through all submodules and runs "make dist" there.
It's in aux/devel-tools now. The script then collects all the
resulting tarballs at the top-level with the right naming/path scheme
for copying them over to the web server. For that to work, we need to
ensure that (1) each Makefile has indeed a working "dist" target, and
(2) that that targets outputs the path to the generated tarball so
that the top-level script can pick it up. Most CMake-builds do (2)
already and for the Python modules I've added output in the form of
"Package: relative/path/to/tgz" (which is then grepped for).[1]

Robin

[1] It works to write out multiple lines like that if more than one
tgz is built (like bro vs bro-all). Btw, I'm wondering if we should
name them the opposite way: bro-2.0.tgz would include all submodules,
and bro-2.0-minimal.tgz would not. Same scheme for BroControl and
Broccoli (where we don't do two tarballs yet).

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From jsiwek at ncsa.illinois.edu  Tue Oct 18 13:41:14 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Tue, 18 Oct 2011 15:41:14 -0500
Subject: [Bro-Dev] Packaging (Re:  Beta schedule)
In-Reply-To: <20111018193634.GB16748@icir.org>
References: <20111012161520.GE29013@icir.org> <20111018042746.GA45062@icir.org>
	<20111018193634.GB16748@icir.org>
Message-ID: <A1A829E9-BFC3-41A6-B76D-E4A8DF4E07D4@ncsa.illinois.edu>


>> - I'm making releases (with tarballs) for all the submodules.
> 
> I'm trying to build tar-files for all our submodules but I'm running
> into a number of things.
> 
> First, afaics, not all modules yet provide the capability to create
> the tarballs (or at least I can't directly figure out how).
> Specifically:
> 
>    - capstats, binpac, and bro-aux: they have "dist" Makefile
>      targets, but running thise gives me:
> 
>        make[1]: *** No rule to make target `package_source'.  Stop.
> 
>    - broccoli-ruby: No "dist" Makefile target and I have no idea how
>    to do that for Ruby packages (I've just added an empty dummy one.)

The package_source target is something provided by CMake, and adding it would require changing top-level CMakeLists.txt to include the cmake/ConfigurePackaging.cmake script, but I don't think that's ultimately the way to go...

> 
> Then, the top-level Bro "make-src-packages" also builts the Broccoli
> and BroControl tgz. I think it would be better to leave that to the
> submodules themselves, otherwise things get inconsistent (because we
> have all these further submodules as well, which do it themselves). 
> So I've removed those lines from make-src-packages.

Yeah, sounds good -- repos for which we we want to create recursive packages should handle that themselves...

> 
> I also noticed that Bro's make-src-packages essentially just tars
> everything that's in the source directory (except for some tmp files
> etc.). There's a problem with that: I'm sure that sometimes I will
> have additional files lying around there which will then end up in the
> distribution. Is there a way to make sure that only stuff under
> version control gets actually into the tgz?

I think we need to use a combination of `git ls-files` to generate a package manifest as input to `tar -T`.

The trick is that all non-recursive source packages need to include the contents of their cmake/ submodule if it exists.

For the recursive package bundles (Bro, BroControl, Broccoli), the output of a recursive `git ls-files` needs to be altered to prepend the right relative path of submodules.

> (And please remind me:
> what was the reason that we can't just let CMake do the tar balls?

So that the packager doesn't need to have CMake as a dependency.
And actually it's worse than that: the packaging configuration provided by CMake is also currently tied to the full suite of ./configure dependency checks as if one were going to actually build the project.
(This was originally pointed out as a pretty bad inconvenience by Craig Leres)

> Here's how I'd like to do the tarballs: I've written a script that
> recursively goes through all submodules and runs "make dist" there.
> It's in aux/devel-tools now. The script then collects all the
> resulting tarballs at the top-level with the right naming/path scheme
> for copying them over to the web server. For that to work, we need to
> ensure that (1) each Makefile has indeed a working "dist" target, and
> (2) that that targets outputs the path to the generated tarball so
> that the top-level script can pick it up. Most CMake-builds do (2)
> already and for the Python modules I've added output in the form of
> "Package: relative/path/to/tgz" (which is then grepped for).[1]
> 
> Robin
> 
> [1] It works to write out multiple lines like that if more than one
> tgz is built (like bro vs bro-all). Btw, I'm wondering if we should
> name them the opposite way: bro-2.0.tgz would include all submodules,
> and bro-2.0-minimal.tgz would not. Same scheme for BroControl and
> Broccoli (where we don't do two tarballs yet).

Sounds good, I'll work on cleaning things up.

- Jon

From robin at icir.org  Tue Oct 18 14:13:14 2011
From: robin at icir.org (Robin Sommer)
Date: Tue, 18 Oct 2011 14:13:14 -0700
Subject: [Bro-Dev] Packaging (Re:  Beta schedule)
In-Reply-To: <A1A829E9-BFC3-41A6-B76D-E4A8DF4E07D4@ncsa.illinois.edu>
References: <20111012161520.GE29013@icir.org> <20111018042746.GA45062@icir.org>
	<20111018193634.GB16748@icir.org>
	<A1A829E9-BFC3-41A6-B76D-E4A8DF4E07D4@ncsa.illinois.edu>
Message-ID: <20111018211314.GA39271@icir.org>


On Tue, Oct 18, 2011 at 15:41 -0500, you wrote:

> Yeah, sounds good -- repos for which we we want to create recursive
> packages should handle that themselves...

Right.

> I think we need to use a combination of `git ls-files` to generate a package manifest as input to `tar -T`.

I just remembered "git clean". That could delete everything we don't
want?

> So that the packager doesn't need to have CMake as a dependency. And
> actually it's worse than that: the packaging configuration provided by
> CMake is also currently tied to the full suite of ./configure
> dependency checks as if one were going to actually build the project.
> (This was originally pointed out as a pretty bad inconvenience by
> Craig Leres)

Hmm, ok. Not sure I'd see that as a showstopper, but I don't recall
the full discussion right now.

> Sounds good, I'll work on cleaning things up.

Ok, thanks!

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Tue Oct 18 15:10:22 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 18 Oct 2011 22:10:22 -0000
Subject: [Bro-Dev] #511: Misc distribution cleanup
In-Reply-To: <047.7fad0a85cd281981d5bd58a78ff6a111@tracker.bro-ids.org>
References: <047.7fad0a85cd281981d5bd58a78ff6a111@tracker.bro-ids.org>
Message-ID: <062.d99c8669545d002e04fdf2bb8cda0f31@tracker.bro-ids.org>

#511: Misc distribution cleanup
-----------------------------+--------------------
  Reporter:  robin           |      Owner:
      Type:  Task            |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro1.6
 Component:  Bro             |    Version:
Resolution:  Solved/Applied  |   Keywords:  beta
-----------------------------+--------------------
Changes (by robin):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 Done with this from my perspective.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/511#comment:9>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 18 15:34:29 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 18 Oct 2011 22:34:29 -0000
Subject: [Bro-Dev] #519: policy/protocols/http/headers.bro only logs
	client headers
In-Reply-To: <046.5777f3c379efa3c38e8a08a6979a91db@tracker.bro-ids.org>
References: <046.5777f3c379efa3c38e8a08a6979a91db@tracker.bro-ids.org>
Message-ID: <061.9453a631126016bdb54ca389a5431457@tracker.bro-ids.org>

#519: policy/protocols/http/headers.bro only logs client headers
----------------------+----------------------
  Reporter:  vern     |      Owner:  seth
      Type:  Problem  |     Status:  reopened
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:           |   Keywords:  beta
----------------------+----------------------

Comment (by robin):

 I lost track what the state of this one is?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/519#comment:5>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 18 15:39:07 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 18 Oct 2011 22:39:07 -0000
Subject: [Bro-Dev] #576: Conn.log does not use well known ports for
 service field anymore
In-Reply-To: <048.db8d9c5b5dbcaeb9188fac26e3c86ea5@tracker.bro-ids.org>
References: <048.db8d9c5b5dbcaeb9188fac26e3c86ea5@tracker.bro-ids.org>
Message-ID: <063.f0c5347ac8171c5ecf42eaf9abe6cf3f@tracker.bro-ids.org>

#576: Conn.log does not use well known ports for service field anymore
------------------------------+--------------------
  Reporter:  gregor           |      Owner:
      Type:  Feature Request  |     Status:  new
  Priority:  Normal           |  Milestone:  Bro1.7
 Component:  Bro              |    Version:
Resolution:                   |   Keywords:  BETA
------------------------------+--------------------
Changes (by robin):

 * milestone:  Bro1.6 => Bro1.7


Comment:

 I think we decided to postpone this and make it an optional script later.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/576#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 18 15:40:38 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 18 Oct 2011 22:40:38 -0000
Subject: [Bro-Dev] #610: topic/seth/syslog-analyzer-updates - Updates
 for syslog analyzer
In-Reply-To: <046.cb3dee1a160d6374e0f15924076c0111@tracker.bro-ids.org>
References: <046.cb3dee1a160d6374e0f15924076c0111@tracker.bro-ids.org>
Message-ID: <061.3a85c82d743a2df059703ed105e9a4a9@tracker.bro-ids.org>

#610: topic/seth/syslog-analyzer-updates - Updates for syslog analyzer
---------------------+--------------------
  Reporter:  seth    |      Owner:
      Type:  Task    |     Status:  new
  Priority:  Normal  |  Milestone:  Bro1.7
 Component:  Bro     |    Version:
Resolution:          |   Keywords:  beta
---------------------+--------------------
Changes (by robin):

 * milestone:  Bro1.6 => Bro1.7


Comment:

 Postponing, will need more testing.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/610#comment:5>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 18 15:41:04 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 18 Oct 2011 22:41:04 -0000
Subject: [Bro-Dev] #612: Segfault in identify_data BiF
In-Reply-To: <048.2145c0c1e4de755bc9fa18b2c7b0ac68@tracker.bro-ids.org>
References: <048.2145c0c1e4de755bc9fa18b2c7b0ac68@tracker.bro-ids.org>
Message-ID: <063.58dd33d526efb39a540cc6215975652f@tracker.bro-ids.org>

#612: Segfault in identify_data BiF
----------------------+------------------------
  Reporter:  gregor   |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------
Changes (by robin):

 * keywords:  beta =>


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/612#comment:8>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 18 16:06:02 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 18 Oct 2011 23:06:02 -0000
Subject: [Bro-Dev] #622: Installation issues
In-Reply-To: <047.9f791003db697f134f198a006352d1c5@tracker.bro-ids.org>
References: <047.9f791003db697f134f198a006352d1c5@tracker.bro-ids.org>
Message-ID: <062.f719793ee27193ec21fc073f74e69280@tracker.bro-ids.org>

#622: Installation issues
----------------------+------------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  beta
----------------------+------------------------

Comment (by robin):

 > Can you give an example of what's specifically different? Is it that the
 default prefix for most other software is /usr/local ?

 Yes, with the main conceptual difference that that is a path shared across
 apps. `/usr/local/bro` is more like `/opt/bro` in the sense that there's
 nothing else below that hierarchy. As such, however, it's unusual to go
 into `/usr/local/*`.

 Anyway, closing this ticket.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/622#comment:8>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 18 16:06:11 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 18 Oct 2011 23:06:11 -0000
Subject: [Bro-Dev] #622: Installation issues
In-Reply-To: <047.9f791003db697f134f198a006352d1c5@tracker.bro-ids.org>
References: <047.9f791003db697f134f198a006352d1c5@tracker.bro-ids.org>
Message-ID: <062.6dab6ca30b6d317b63fee58f097eaab4@tracker.bro-ids.org>

#622: Installation issues
-----------------------------+------------------------
  Reporter:  robin           |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro1.6
 Component:  Bro             |    Version:  git/master
Resolution:  Solved/Applied  |   Keywords:  beta
-----------------------------+------------------------
Changes (by robin):

 * status:  new => closed
 * resolution:   => Solved/Applied


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/622#comment:9>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 18 16:06:43 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 18 Oct 2011 23:06:43 -0000
Subject: [Bro-Dev] #624: Bro won't shutdown if not seeing packets
In-Reply-To: <046.06582d3d3a1ee7e1010ccc163ac53a7f@tracker.bro-ids.org>
References: <046.06582d3d3a1ee7e1010ccc163ac53a7f@tracker.bro-ids.org>
Message-ID: <061.4124e8342f099fa768f92df3c9f6dc7c@tracker.bro-ids.org>

#624: Bro won't shutdown if not seeing packets
----------------------+--------------------
  Reporter:  seth     |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------
Changes (by robin):

 * keywords:  beta =>


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/624#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 18 16:07:07 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 18 Oct 2011 23:07:07 -0000
Subject: [Bro-Dev] #633: Globals cleanup
In-Reply-To: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
References: <047.cf4580f5113530897d7e17a5b9de40dc@tracker.bro-ids.org>
Message-ID: <062.c5ccc83ff1c5555c2b295bf22e747d33@tracker.bro-ids.org>

#633: Globals cleanup
----------------------+------------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------
Changes (by robin):

 * keywords:  beta =>


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/633#comment:8>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 18 16:07:25 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 18 Oct 2011 23:07:25 -0000
Subject: [Bro-Dev] #637: Integrate ssl alerts into the ssl log
In-Reply-To: <046.76b9bd1e6633db885efa09dbe80c5cd8@tracker.bro-ids.org>
References: <046.76b9bd1e6633db885efa09dbe80c5cd8@tracker.bro-ids.org>
Message-ID: <061.85d962f9d19e7ec07d8173dcc5d1a92d@tracker.bro-ids.org>

#637: Integrate ssl alerts into the ssl log
----------------------+--------------------
  Reporter:  seth     |      Owner:  seth
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------
Changes (by robin):

 * keywords:  beta =>


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/637#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Tue Oct 18 18:24:05 2011
From: robin at icir.org (Robin Sommer)
Date: Tue, 18 Oct 2011 18:24:05 -0700
Subject: [Bro-Dev] Beta schedule
In-Reply-To: <20111012161520.GE29013@icir.org>
References: <20111012161520.GE29013@icir.org> <4E9610D6.5020701@icir.org>
	<20111012161520.GE29013@icir.org>
Message-ID: <20111019012405.GA92944@icir.org>

We are getting really close!

On Wed, Oct 12, 2011 at 09:15 -0700, I wrote:

>     - #511: Cleanup distribution a bit more (Robin)
>     - #601: Cleanup and organize the documentation we have (Robin[1])   

Done with these, except that I'm still planing to make a pass over the
upgrade docs.

>     - #622: Some of these seem to addressed, others not. (John, can
>             you take the lead closing this ticket? Some might just need a
>             decision/reminder if what's we're currenty doing is the right
>             thing.)

Closed.

>     - #601: Write docs for notice framework (Seth)
>     - #544: Get scan.bro into shape (Seth)

I think Seth is working on these.

Seth, one more: Almost all tests pass for me now (just updated a few
more baselines). The one exception is bro-testing:tests.m57-long,
which still gives me a long list of diffs. That's waiting for you I'm
afraid ...


Once these are done, I'll be happy to push out the beta.


On Wed, Oct 12, 2011 at 15:12 -0700, Gregor wrote:

>  * Complete script reference (though that can wait to final release)

Yeah, I was hoping to have the "Broxygen docs" in shape for the beta,
but there's still quite a bit to do there (both in terms of polishing
up comments in scripts, and also with structure and optics). Let's
postpone that to the release (which however I won't let go out without
this!)

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Tue Oct 18 19:46:12 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 19 Oct 2011 02:46:12 -0000
Subject: [Bro-Dev] #567: Protect/secure format strings in Reporter.cc
In-Reply-To: <048.2977795f08466e96df0bed12ea39f590@tracker.bro-ids.org>
References: <048.2977795f08466e96df0bed12ea39f590@tracker.bro-ids.org>
Message-ID: <063.a5337aa25f271ea352cfbda767ab3832@tracker.bro-ids.org>

#567: Protect/secure format strings in Reporter.cc
----------------------+------------------------
  Reporter:  gregor   |      Owner:  robin
      Type:  Problem  |     Status:  closed
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:  fixed    |   Keywords:  beta
----------------------+------------------------
Changes (by robin):

 * owner:   => robin
 * status:  new => closed
 * resolution:   => fixed


Comment:

 In [63b46a0ae2486c7058a8557bdb51285b348f2a12/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="63b46a0ae2486c7058a8557bdb51285b348f2a12"
 Fixing a bunch of format strings.

 Also leveraging GCC if available to check format specificier.

 Closes #567.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/567#comment:7>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 18 20:06:59 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 19 Oct 2011 03:06:59 -0000
Subject: [Bro-Dev] #642: Configure only checks presence of swig and not of
	swig-python
Message-ID: <048.54bb35f2e689a4c53862890a6fd4bf42@tracker.bro-ids.org>

#642: Configure only checks presence of swig and not of swig-python
--------------------+---------------------
 Reporter:  amannb  |       Type:  Problem
   Status:  new     |   Priority:  Normal
Milestone:          |  Component:  Bro
  Version:          |   Keywords:
--------------------+---------------------
 On Mac OS X, macports usually installs swig without the python bindings
 (they are in a separate package).

 When only swig and not swig-python is installed, the configure script runs
 ok, but the build fails with

 {{{
 ...
 [ 78%] Building C object src/CMakeFiles/bro.dir/strsep.c.o
 [ 78%] Building C object src/CMakeFiles/bro.dir/modp_numtoa.c.o
 [ 78%] Building C object src/CMakeFiles/bro.dir/nb_dns.c.o
 Linking CXX executable bro
 [ 78%] Built target bro
 [ 78%] Swig source
 :3: Error: Unable to find 'python.swg'
 make[3]: *** [aux/broctl/aux/pysubnettree/SubnetTreePYTHON_wrap.cxx] Error
 1
 make[2]: *** [aux/broctl/aux/pysubnettree/CMakeFiles/_SubnetTree.dir/all]
 Error 2
 make[1]: *** [all] Error 2
 make: *** [all] Error 2
 }}}

 the same is true for ruby:

 {{{
 [ 97%] Building C object
 aux/broccoli/test/CMakeFiles/brotable.dir/brotable.c.o
 Linking C executable brotable
 [ 97%] Built target brotable
 [ 97%] Swig source
 Scanning dependencies of target _broccoli_intern
 [ 98%] Building C object aux/broccoli/bindings/broccoli-
 python/CMakeFiles/_broccoli_intern.dir/broccoli_internPYTHON_wrap.c.o
 /Users/bernhard/bro/bro/build/aux/broccoli/bindings/broccoli-
 python/broccoli_internPYTHON_wrap.c: In function ?valToPyObj?:
 /Users/bernhard/bro/bro/build/aux/broccoli/bindings/broccoli-
 python/broccoli_internPYTHON_wrap.c:3057: warning: pointer targets in
 passing argument 1 of ?PyString_FromStringAndSize? differ in signedness
 /Users/bernhard/bro/bro/build/aux/broccoli/bindings/broccoli-
 python/broccoli_internPYTHON_wrap.c: In function ?pyObjToVal?:
 /Users/bernhard/bro/bro/build/aux/broccoli/bindings/broccoli-
 python/broccoli_internPYTHON_wrap.c:3154: warning: pointer targets in
 assignment differ in signedness
 /Users/bernhard/bro/bro/build/aux/broccoli/bindings/broccoli-
 python/broccoli_internPYTHON_wrap.c: In function
 ?_wrap_bro_event_add_val?:
 /Users/bernhard/bro/bro/build/aux/broccoli/bindings/broccoli-
 python/broccoli_internPYTHON_wrap.c:3734: warning: assignment discards
 qualifiers from pointer target type
 Linking C shared module _broccoli_intern.so
 [ 98%] Built target _broccoli_intern
 [100%] Swig source
 :3: Error: Unable to find 'ruby.swg'
 /Users/bernhard/bro/bro/aux/broccoli/bindings/broccoli-
 ruby/ext/broccoli_ext/broccoli_intern.i:4: Error: Unable to find
 'typemaps.i'
 make[3]: *** [aux/broccoli/bindings/broccoli-
 ruby/ext/broccoli_ext/broccoli_internRUBY_wrap.c] Error 1
 make[2]: *** [aux/broccoli/bindings/broccoli-
 ruby/CMakeFiles/broccoli_ext.dir/all] Error 2
 make[1]: *** [all] Error 2
 make: *** [all] Error 2
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/642>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 18 20:23:24 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 19 Oct 2011 03:23:24 -0000
Subject: [Bro-Dev] #642: Configure only checks presence of swig and not
 of swig-python
In-Reply-To: <048.54bb35f2e689a4c53862890a6fd4bf42@tracker.bro-ids.org>
References: <048.54bb35f2e689a4c53862890a6fd4bf42@tracker.bro-ids.org>
Message-ID: <063.f99a8c3f83a973887fa44c72489ebc1a@tracker.bro-ids.org>

#642: Configure only checks presence of swig and not of swig-python
----------------------+--------------------
  Reporter:  amannb   |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------
Changes (by robin):

 * milestone:   => Bro1.6


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/642#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Tue Oct 18 20:51:04 2011
From: robin at icir.org (Robin Sommer)
Date: Tue, 18 Oct 2011 20:51:04 -0700
Subject: [Bro-Dev] OpenSSL and Lion
Message-ID: <20111019035104.GD918@icir.org>

On Mac OS Lion, OpenSSL has been deprecated, which results in tons of
warnings when compiling. Seems the compiler reports about every single
OpenSSL function it sees being called.

I don't think we want/need to fix that immediately, but it's something
to keep on the radar. The good news: Apple has a replacement library,
Common Crypto[1].Perhaps that's something to use in the future instead
of OpenSSL (yeah! :) ...

Robin

[1] http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man3/CC_crypto.3cc.html

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From jsiwek at ncsa.illinois.edu  Tue Oct 18 21:13:17 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Tue, 18 Oct 2011 23:13:17 -0500
Subject: [Bro-Dev] Packaging (Re:  Beta schedule)
In-Reply-To: <20111018211314.GA39271@icir.org>
References: <20111012161520.GE29013@icir.org> <20111018042746.GA45062@icir.org>
	<20111018193634.GB16748@icir.org>
	<A1A829E9-BFC3-41A6-B76D-E4A8DF4E07D4@ncsa.illinois.edu>
	<20111018211314.GA39271@icir.org>
Message-ID: <5AB57258-5D28-4021-A16D-9C84B8ACC528@ncsa.illinois.edu>

> I just remembered "git clean". That could delete everything we don't
> want?

Added that part to the `make-release` script.

And all the `make dist` targets should all work better I think; try it out?

- Jon

From gregor at icir.org  Wed Oct 19 06:40:13 2011
From: gregor at icir.org (Gregor Maier)
Date: Wed, 19 Oct 2011 06:40:13 -0700
Subject: [Bro-Dev] OpenSSL and Lion
In-Reply-To: <20111019035104.GD918@icir.org>
References: <20111019035104.GD918@icir.org>
Message-ID: <4E9ED33D.2090701@icir.org>

Yeah. I guess more and more OSes try do ditch OpenSSL. Ubunutu is trying 
hard to use GnuTLS instead of OpenSSL......


On 10/18/11 20:51 , Robin Sommer wrote:
> On Mac OS Lion, OpenSSL has been deprecated, which results in tons of
> warnings when compiling. Seems the compiler reports about every single
> OpenSSL function it sees being called.
>
> I don't think we want/need to fix that immediately, but it's something
> to keep on the radar. The good news: Apple has a replacement library,
> Common Crypto[1].Perhaps that's something to use in the future instead
> of OpenSSL (yeah! :) ...
>
> Robin
>
> [1] http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man3/CC_crypto.3cc.html
>


-- 
Gregor Maier
<gregor at icir.org>  <gregor at icsi.berkeley.edu>
Int. Computer Science Institute (ICSI)
1947 Center St., Ste. 600
Berkeley, CA 94704, USA
http://www.icir.org/gregor/

From slagell at illinois.edu  Wed Oct 19 06:47:15 2011
From: slagell at illinois.edu (Slagell, Adam J)
Date: Wed, 19 Oct 2011 13:47:15 +0000
Subject: [Bro-Dev] OpenSSL and Lion
In-Reply-To: <4E9ED33D.2090701@icir.org>
References: <20111019035104.GD918@icir.org> <4E9ED33D.2090701@icir.org>
Message-ID: <54218D25-E3E5-4D39-9050-26858507BF6A@illinois.edu>

Which may make porting across multiple OS'es even more "fun".

On Oct 19, 2011, at 8:40 AM, Gregor Maier wrote:

> Yeah. I guess more and more OSes try do ditch OpenSSL. Ubunutu is trying 
> hard to use GnuTLS instead of OpenSSL......
> 
> 
> On 10/18/11 20:51 , Robin Sommer wrote:
>> On Mac OS Lion, OpenSSL has been deprecated, which results in tons of
>> warnings when compiling. Seems the compiler reports about every single
>> OpenSSL function it sees being called.
>> 
>> I don't think we want/need to fix that immediately, but it's something
>> to keep on the radar. The good news: Apple has a replacement library,
>> Common Crypto[1].Perhaps that's something to use in the future instead
>> of OpenSSL (yeah! :) ...
>> 
>> Robin
>> 
>> [1] http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man3/CC_crypto.3cc.html
>> 
> 
> 
> -- 
> Gregor Maier
> <gregor at icir.org>  <gregor at icsi.berkeley.edu>
> Int. Computer Science Institute (ICSI)
> 1947 Center St., Ste. 600
> Berkeley, CA 94704, USA
> http://www.icir.org/gregor/
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
> 

------

Adam J. Slagell, CISSP
Sr. Security Engineer
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
www.slagell.info
217.244.8965










From bro at tracker.bro-ids.org  Wed Oct 19 07:12:09 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 19 Oct 2011 14:12:09 -0000
Subject: [Bro-Dev] #519: policy/protocols/http/headers.bro only logs
	client headers
In-Reply-To: <046.5777f3c379efa3c38e8a08a6979a91db@tracker.bro-ids.org>
References: <046.5777f3c379efa3c38e8a08a6979a91db@tracker.bro-ids.org>
Message-ID: <061.d18f465c7073ecd567b3817d8eb3758b@tracker.bro-ids.org>

#519: policy/protocols/http/headers.bro only logs client headers
----------------------+----------------------
  Reporter:  vern     |      Owner:  seth
      Type:  Problem  |     Status:  reopened
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:           |   Keywords:  beta
----------------------+----------------------

Comment (by seth):

 > I lost track what the state of this one is?

 I have some changes queued up that should address this (among other
 things).  I'll see if I can get to it today.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/519#comment:11>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From hlin33 at illinois.edu  Wed Oct 19 08:29:31 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Wed, 19 Oct 2011 10:29:31 -0500
Subject: [Bro-Dev] Hui Lin_Binpac Question_Can several "case" in binpac
 share the same parsing method
Message-ID: <CAKq214kQr+gg_Qsz_rUqtm1=jv+AxUz38bYFYAZQTRL2a9wDMA@mail.gmail.com>

HI,

Just try to save space and time when writing case in Binpac for example,

type SMB_string(case_index: int) = case case_index of {
   1    -> a: type1;
   2    -> b: type1;
   3    -> c: type1;
   default -> d: type2;
 };

is there any method that I can avoid of repeated writing case 1, 2, 3 as
they actually correspond to same type1?

Best


-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111019/fc7e5968/attachment.html 

From bro at tracker.bro-ids.org  Wed Oct 19 08:38:34 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 19 Oct 2011 15:38:34 -0000
Subject: [Bro-Dev] #514: NMI scripts under version control
In-Reply-To: <050.07eb55afaed5089f5ed7b9ac51224efd@tracker.bro-ids.org>
References: <050.07eb55afaed5089f5ed7b9ac51224efd@tracker.bro-ids.org>
Message-ID: <065.8e34854ed6a9b6877c098f3c5c22215a@tracker.bro-ids.org>

#514: NMI scripts under version control
-----------------------------+----------------------
  Reporter:  appleman        |      Owner:  dnthayer
      Type:  Task            |     Status:  closed
  Priority:  Low             |  Milestone:
 Component:  Bro             |    Version:
Resolution:  Solved/Applied  |   Keywords:  nmi
-----------------------------+----------------------
Changes (by dnthayer):

 * status:  accepted => closed
 * resolution:   => Solved/Applied


Comment:

 Created the "nmi" repository which contains all of the files in
 the "~broteam/nmi" directory on the NMI login/submit host.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/514#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct 19 09:34:24 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 19 Oct 2011 16:34:24 -0000
Subject: [Bro-Dev] #642: Configure only checks presence of swig and not
 of swig-python
In-Reply-To: <048.54bb35f2e689a4c53862890a6fd4bf42@tracker.bro-ids.org>
References: <048.54bb35f2e689a4c53862890a6fd4bf42@tracker.bro-ids.org>
Message-ID: <063.ffe5edeabcbe514618108944d8820f7d@tracker.bro-ids.org>

#642: Configure only checks presence of swig and not of swig-python
----------------------+--------------------
  Reporter:  amannb   |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by jsiwek):

 In [01b71b3f2027398ae59bff65d5d93a46ebda6ee4/broccoli-python]:
 {{{
 #!CommitTicketReference repository="broccoli-python"
 revision="01b71b3f2027398ae59bff65d5d93a46ebda6ee4"
 Add configure-time check that swig can generate python wrappers.

 Addresses #642.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/642#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct 19 09:35:08 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 19 Oct 2011 16:35:08 -0000
Subject: [Bro-Dev] #642: Configure only checks presence of swig and not
 of swig-python
In-Reply-To: <048.54bb35f2e689a4c53862890a6fd4bf42@tracker.bro-ids.org>
References: <048.54bb35f2e689a4c53862890a6fd4bf42@tracker.bro-ids.org>
Message-ID: <063.1088b820bf4f10ae4111eee206344ecd@tracker.bro-ids.org>

#642: Configure only checks presence of swig and not of swig-python
----------------------+--------------------
  Reporter:  amannb   |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by jsiwek):

 In [181dd0995c88cb25fc40679a2f00414099829a0d/broccoli-ruby]:
 {{{
 #!CommitTicketReference repository="broccoli-ruby"
 revision="181dd0995c88cb25fc40679a2f00414099829a0d"
 Add configure-time check that swig can generate Ruby wrappers.

 Addresses #642.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/642#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct 19 09:45:37 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 19 Oct 2011 16:45:37 -0000
Subject: [Bro-Dev] #642: Configure only checks presence of swig and not
 of swig-python
In-Reply-To: <048.54bb35f2e689a4c53862890a6fd4bf42@tracker.bro-ids.org>
References: <048.54bb35f2e689a4c53862890a6fd4bf42@tracker.bro-ids.org>
Message-ID: <063.869f64408151c5766dca4bf1374cfe8a@tracker.bro-ids.org>

#642: Configure only checks presence of swig and not of swig-python
-----------------------------+--------------------
  Reporter:  amannb          |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro1.6
 Component:  Bro             |    Version:
Resolution:  Solved/Applied  |   Keywords:
-----------------------------+--------------------
Changes (by jsiwek):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 Checking for ability to generate ruby/python wrappers now happens at
 configure time and I added some documentation to the quickstart guide
 regarding `MacPorts` having separating packages per language.  Let me know
 if you had other suggestions or if it's still not behaving well for you.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/642#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From christian at icir.org  Wed Oct 19 11:01:23 2011
From: christian at icir.org (Christian Kreibich)
Date: Wed, 19 Oct 2011 11:01:23 -0700
Subject: [Bro-Dev] OpenSSL and Lion
In-Reply-To: <54218D25-E3E5-4D39-9050-26858507BF6A@illinois.edu>
References: <20111019035104.GD918@icir.org> <4E9ED33D.2090701@icir.org>
	<54218D25-E3E5-4D39-9050-26858507BF6A@illinois.edu>
Message-ID: <4E9F1073.8000403@icir.org>

Whatever's next, it won't be worse! :)
-C.

From robin at icir.org  Wed Oct 19 12:40:07 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 19 Oct 2011 12:40:07 -0700
Subject: [Bro-Dev] Packaging (Re:  Beta schedule)
In-Reply-To: <5AB57258-5D28-4021-A16D-9C84B8ACC528@ncsa.illinois.edu>
References: <20111012161520.GE29013@icir.org> <20111018042746.GA45062@icir.org>
	<20111018193634.GB16748@icir.org>
	<A1A829E9-BFC3-41A6-B76D-E4A8DF4E07D4@ncsa.illinois.edu>
	<20111018211314.GA39271@icir.org>
	<5AB57258-5D28-4021-A16D-9C84B8ACC528@ncsa.illinois.edu>
Message-ID: <20111019194007.GO88391@icir.org>

Thanks, looks good, but one more request:

On Tue, Oct 18, 2011 at 23:13 -0500, you wrote:

> Added that part to the `make-release` script.

The disadadvantage of this is that is erases all local changes right
inside the repository I'm working in. Could "make dist" instead run
the "git clean" inside the temporary copy that it's creating? (Which
would probably mean that it needs to copy .git over first, then run
git clean, and then rm .git). That way nothing I have lying around
locally would be touched. I'm sure otherwise I will at some point
accidentally delete something.

Also, the "git submodule foreach ..." descends into all subdirectories
but doesn't execute inside the top-levle parent module. I think we
need another "clean ..." at the top-level as well, right?

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From jsiwek at ncsa.illinois.edu  Wed Oct 19 13:04:24 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Wed, 19 Oct 2011 15:04:24 -0500
Subject: [Bro-Dev] Packaging (Re:  Beta schedule)
In-Reply-To: <20111019194007.GO88391@icir.org>
References: <20111012161520.GE29013@icir.org> <20111018042746.GA45062@icir.org>
	<20111018193634.GB16748@icir.org>
	<A1A829E9-BFC3-41A6-B76D-E4A8DF4E07D4@ncsa.illinois.edu>
	<20111018211314.GA39271@icir.org>
	<5AB57258-5D28-4021-A16D-9C84B8ACC528@ncsa.illinois.edu>
	<20111019194007.GO88391@icir.org>
Message-ID: <56C00EFF-AC9E-4A1B-AF96-AED23EF5DA1B@ncsa.illinois.edu>

> The disadadvantage of this is that is erases all local changes right
> inside the repository I'm working in. Could "make dist" instead run
> the "git clean" inside the temporary copy that it's creating? (Which
> would probably mean that it needs to copy .git over first, then run
> git clean, and then rm .git). That way nothing I have lying around
> locally would be touched. I'm sure otherwise I will at some point
> accidentally delete something.

Ok, I'll change that.

> 
> Also, the "git submodule foreach ..." descends into all subdirectories
> but doesn't execute inside the top-levle parent module. I think we
> need another "clean ..." at the top-level as well, right?

yeah, forgot about that, but I'll remove that entirely since it will be done in  the `make dists`

- Jon

From bro at tracker.bro-ids.org  Wed Oct 19 13:36:43 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 19 Oct 2011 20:36:43 -0000
Subject: [Bro-Dev] #643: Bro segfaults when giving invalid option to -B
Message-ID: <048.40d0e497c567f465432333bc7678afab@tracker.bro-ids.org>

#643: Bro segfaults when giving invalid option to -B
--------------------+---------------------
 Reporter:  amannb  |       Type:  Problem
   Status:  new     |   Priority:  Normal
Milestone:          |  Component:  Bro
  Version:          |   Keywords:
--------------------+---------------------
 Bro segfaults, when the -B flag is used with an invalid identifier

 {{{

 wifi188:la bernhard$ gdb --args bro -B evilString broping-record.bro

 ...
 Starting program: /Users/bernhard/sw/bin/bro -B evilString broping-
 record.bro
 Reading symbols for shared libraries +++++++......................... done

 Program received signal EXC_BAD_ACCESS, Could not access memory.
 Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000010
 0x000000010026798c in std::list<std::pair<Location const*, Location
 const*>, std::allocator<std::pair<Location const*, Location const*> >
 >::begin (this=0x10) at stl_list.h:592
 592           { return const_iterator(this->_M_impl._M_node._M_next); }
 (gdb) bt
 #0  0x000000010026798c in std::list<std::pair<Location const*, Location
 const*>, std::allocator<std::pair<Location const*, Location const*> >
 >::begin (this=0x10) at stl_list.h:592
 #1  0x0000000100267b8a in std::list<std::pair<Location const*, Location
 const*>, std::allocator<std::pair<Location const*, Location const*> >
 >::size (this=0x10) at stl_list.h:660
 #2  0x000000010026429c in Reporter::DoLog (this=0x0, prefix=0x100415c82
 "internal error", event=@0x7fff5fbfeea0, out=0x7fff7a8ae580, conn=0x0,
 addl=0x0, location=true, time=false, fmt=0x10040c9b3 "unknown debug stream
 %s\n", ap=0x7fff5fbfee88) at Reporter.cc:191
 #3  0x0000000100266ac1 in Reporter::InternalError (this=0x0,
 fmt=0x10040c9b3 "unknown debug stream %s\n") at Reporter.cc:97
 #4  0x00000001001683b0 in DebugLogger::EnableStreams (this=0x10052bb20,
 s=0x7fff5fbffcce "evilString") at DebugLogger.cc:75
 #5  0x00000001000cc220 in main (argc=4, argv=0x7fff5fbffb98) at
 main.cc:642
 (gdb) q
 The program is running.  Exit anyway? (y or n) y
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/643>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct 19 17:01:47 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 20 Oct 2011 00:01:47 -0000
Subject: [Bro-Dev] #644: Compile SWIG bindings with no-strict-aliasing
Message-ID: <047.9cd2c206c9e71636a6e1fe53bf980b59@tracker.bro-ids.org>

#644: Compile SWIG bindings with no-strict-aliasing
----------------------+--------------------
 Reporter:  robin     |      Owner:
     Type:  Problem   |     Status:  new
 Priority:  Normal    |  Milestone:  Bro1.6
Component:  Broccoli  |    Version:
 Keywords:            |
----------------------+--------------------
 The SWIG bindings generate a lot of ` warning: dereferencing type-punned
 pointer will break strict-aliasing rules` warnings.

 The right thing to do seems to compiling them with `-f no-strict-
 aliasing`. Let's add that to CFLAGS when compiling the bindings.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/644>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct 19 17:04:11 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 20 Oct 2011 00:04:11 -0000
Subject: [Bro-Dev] #645: Figure out where sendmail is
Message-ID: <047.30bfd70cef14df8f513f2d5595dceaae@tracker.bro-ids.org>

#645: Figure out where sendmail is
------------------------+--------------------
 Reporter:  robin       |      Owner:
     Type:  Problem     |     Status:  new
 Priority:  Normal      |  Milestone:  Bro1.6
Component:  BroControl  |    Version:
 Keywords:              |
------------------------+--------------------
 I keep thinking that this entry shouldn't be in the default `broctl.cfg`:

   {{{
   SendMail = /usr/sbin/sendmail
 }}}

 Can we figure out at configure time where sendmail is and preset the
 option accordingly?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/645>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Wed Oct 19 17:50:41 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 19 Oct 2011 17:50:41 -0700
Subject: [Bro-Dev] Packaging (Re:  Beta schedule)
In-Reply-To: <20111019194007.GO88391@icir.org>
References: <20111012161520.GE29013@icir.org> <20111018042746.GA45062@icir.org>
	<20111018193634.GB16748@icir.org>
	<A1A829E9-BFC3-41A6-B76D-E4A8DF4E07D4@ncsa.illinois.edu>
	<20111018211314.GA39271@icir.org>
	<5AB57258-5D28-4021-A16D-9C84B8ACC528@ncsa.illinois.edu>
	<20111019194007.GO88391@icir.org>
Message-ID: <20111020005041.GA39419@icir.org>


On Wed, Oct 19, 2011 at 12:40 -0700, I wrote:

> I'm sure otherwise I will at some point accidentally delete something.

It just happened ... Typing "make distclean" is deleting the repos in
testing/external/*. So let's please switch that back to the previous
"rm -rf build".

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Wed Oct 19 18:42:50 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 20 Oct 2011 01:42:50 -0000
Subject: [Bro-Dev] #643: Bro segfaults when giving invalid option to -B
In-Reply-To: <048.40d0e497c567f465432333bc7678afab@tracker.bro-ids.org>
References: <048.40d0e497c567f465432333bc7678afab@tracker.bro-ids.org>
Message-ID: <063.3fcfab5c3babfe2fb41917880aad3564@tracker.bro-ids.org>

#643: Bro segfaults when giving invalid option to -B
----------------------+--------------------
  Reporter:  amannb   |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------
Changes (by robin):

 * milestone:   => Bro1.6


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/643#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Wed Oct 19 18:44:41 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 19 Oct 2011 18:44:41 -0700
Subject: [Bro-Dev] Hui Lin_Binpac Question_Can several "case" in
	binpac	share the same parsing method
In-Reply-To: <CAKq214kQr+gg_Qsz_rUqtm1=jv+AxUz38bYFYAZQTRL2a9wDMA@mail.gmail.com>
References: <CAKq214kQr+gg_Qsz_rUqtm1=jv+AxUz38bYFYAZQTRL2a9wDMA@mail.gmail.com>
Message-ID: <20111020014441.GE21245@icir.org>


On Wed, Oct 19, 2011 at 10:29 -0500, you wrote:

>    1    -> a: type1;
>    2    -> b: type1;
>    3    -> c: type1;

As far as I recall, there's no shortcut version unfortuantely.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From jsiwek at ncsa.illinois.edu  Fri Oct 21 09:28:38 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Fri, 21 Oct 2011 11:28:38 -0500
Subject: [Bro-Dev] Packaging (Re:  Beta schedule)
In-Reply-To: <20111020005041.GA39419@icir.org>
References: <20111012161520.GE29013@icir.org> <20111018042746.GA45062@icir.org>
	<20111018193634.GB16748@icir.org>
	<A1A829E9-BFC3-41A6-B76D-E4A8DF4E07D4@ncsa.illinois.edu>
	<20111018211314.GA39271@icir.org>
	<5AB57258-5D28-4021-A16D-9C84B8ACC528@ncsa.illinois.edu>
	<20111019194007.GO88391@icir.org> <20111020005041.GA39419@icir.org>
Message-ID: <2D2230E9-55BA-4F24-BFD5-D4FCE91260E9@ncsa.illinois.edu>

> It just happened ... Typing "make distclean" is deleting the repos in
> testing/external/*. So let's please switch that back to the previous
> "rm -rf build".

Sorry.  I will fix it.

- Jon

From hlin33 at illinois.edu  Fri Oct 21 09:28:36 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Fri, 21 Oct 2011 11:28:36 -0500
Subject: [Bro-Dev] Hui Lin_External Communication with Bro
Message-ID: <CAKq214mS7C+TQ368VO1jXEvsLWHXRwSZdnPezSF=-TZbU0AwEw@mail.gmail.com>

Hi,

I am currently writing a policy to use Bro process event from external
source, e.g. auth.log (under /var/log). I just want to catch some sudo
operations from system. The effect that I want is any runtime changes in
"auth.log" will be caught by Bro's event handler.

So I review 2009 workshop exercise related to this topic. I understand how
Broccoli and Bro-pipe works. But I just confusion on the run-time usage of
it. Use Bro-Pipe for example, it uses "bro-pipe" text file with specific
format as the input to Bro. Such as

ssh_fail_login double=1184518203 addr=85.14.95.10 addr=131.243.2.11
string=aggie string=password
ssh_fail_login double=1184529743 addr=81.68.198.23 addr=131.243.2.11
string=ailsa string=password
ssh_fail_login double=1184529745 addr=81.68.198.23 addr=131.243.2.11
string=aim string=password

So there should be a script to transform the original log file into this
"bro-pipe" text file. My question is that is that possible to dynamically
update this "bro-pipe" text file when the log file is updated during the
runtime? if possible, what script is used and how to do that?

Best,

Hui


-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111021/28504d33/attachment.html 

From bro at tracker.bro-ids.org  Fri Oct 21 10:03:00 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 21 Oct 2011 17:03:00 -0000
Subject: [Bro-Dev] #646: Cleanup interpreter error handling.
Message-ID: <047.f66ed674d262f7ebac52f1c4e6df3a3e@tracker.bro-ids.org>

#646: Cleanup interpreter error handling.
--------------------+--------------------
 Reporter:  robin   |      Owner:
     Type:  Task    |     Status:  new
 Priority:  Normal  |  Milestone:  Bro1.7
Component:  Bro     |    Version:
 Keywords:          |
--------------------+--------------------
 From 15ab2874369b5d7a3e6a14df24b141fa759999bb (which has been merged into
 master):

 {{{
  Currently, a lot of interpreter runtime errors, such as an access to
     an unset optional record field, cause Bro to abort with an internal
     error. This is an experimental branch that turns such errors into
     non-fatal runtime errors by internally raising exceptions. These are
     caught upstream and processing continues afterwards.

     For now, not many errors actually raise exceptions (the example above
     does though). We'll need to go through them eventually and adapt the
     current Internal() calls (and potentially others). More generally, at
     some point we should cleanup the interpreter error handling (unifying
     errors reported at parse- and runtime; and switching to exceptions for
     all Expr/Stmt/Vals). But that's a larger change and left for later.

 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/646>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From seth at icir.org  Fri Oct 21 11:13:20 2011
From: seth at icir.org (Seth Hall)
Date: Fri, 21 Oct 2011 14:13:20 -0400
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Small script
	refinements and documentation updates. (8661abe)
In-Reply-To: <201110211811.p9LIBQ0Q020925@bro-ids.icir.org>
References: <201110211811.p9LIBQ0Q020925@bro-ids.icir.org>
Message-ID: <A14E3032-9DDB-4A57-A85B-B0A91AAAF678@icir.org>


On Oct 21, 2011, at 2:11 PM, Seth Hall wrote:

> Repository : ssh://git at bro-ids.icir.org/bro
> 
> On branches: master,topic/seth/weird-updates

Whoops, I forgot to push out master first.  Almost all of these commits were to master except the one relating to the weird changes.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From bro at tracker.bro-ids.org  Fri Oct 21 11:43:32 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 21 Oct 2011 18:43:32 -0000
Subject: [Bro-Dev] #643: Bro segfaults when giving invalid option to -B
In-Reply-To: <048.40d0e497c567f465432333bc7678afab@tracker.bro-ids.org>
References: <048.40d0e497c567f465432333bc7678afab@tracker.bro-ids.org>
Message-ID: <063.b7b950c41e8dd939d38ba822b5297509@tracker.bro-ids.org>

#643: Bro segfaults when giving invalid option to -B
----------------------+--------------------
  Reporter:  amannb   |      Owner:  robin
      Type:  Problem  |     Status:  closed
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:  fixed    |   Keywords:
----------------------+--------------------
Changes (by robin):

 * owner:   => robin
 * status:  new => closed
 * resolution:   => fixed


Comment:

 In [5e5e29f345ec3bdc2c0be38642b5a76520991bb0/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="5e5e29f345ec3bdc2c0be38642b5a76520991bb0"
 Fixing crash with unknown debug streams.

 Closes #643.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/643#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 24 10:09:11 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 24 Oct 2011 17:09:11 -0000
Subject: [Bro-Dev] #647: SDCH support
Message-ID: <046.95060f0c54e55e518c6e6817c50c1af5@tracker.bro-ids.org>

#647: SDCH support
---------------------+--------------------
 Reporter:  seth     |      Owner:
     Type:  Problem  |     Status:  new
 Priority:  Normal   |  Milestone:  Bro1.6
Component:  Bro      |    Version:
 Keywords:           |
---------------------+--------------------
 When Chrome and other SDCH supporting http clients request content from
 SDCH compatible HTTP servers the response includes a header that looks
 like this:

 {{{
 Content-Encoding: sdch,gzip
 }}}

 Bro's HTTP analyzer doesn't currently do substring matches on the content-
 encoding header so the resulting sdch/gzip content is identified as gzip
 only.  Two things need to happen here:

     1. Support substring matches on the content-encoding header to
 identify that the content is gzip encoded.
     2. Support some notion of the SDCH protocol.

 I think that point 1 should be done for the 2.0 release  but point 2 can
 wait until later when we have a better notion of what SDCH support would
 entail.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/647>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 24 12:54:21 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 24 Oct 2011 19:54:21 -0000
Subject: [Bro-Dev] #519: policy/protocols/http/headers.bro only logs
	client headers
In-Reply-To: <046.5777f3c379efa3c38e8a08a6979a91db@tracker.bro-ids.org>
References: <046.5777f3c379efa3c38e8a08a6979a91db@tracker.bro-ids.org>
Message-ID: <061.cc2fa4ea4d37f7990e43773fd0add32b@tracker.bro-ids.org>

#519: policy/protocols/http/headers.bro only logs client headers
----------------------+----------------------
  Reporter:  vern     |      Owner:  seth
      Type:  Problem  |     Status:  reopened
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+----------------------
Changes (by robin):

 * keywords:  beta =>


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/519#comment:12>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 24 13:18:26 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 24 Oct 2011 20:18:26 -0000
Subject: [Bro-Dev] #510: Upgrade How-To
In-Reply-To: <047.e793bc8325e1f7ba35cd0d2c4f23760e@tracker.bro-ids.org>
References: <047.e793bc8325e1f7ba35cd0d2c4f23760e@tracker.bro-ids.org>
Message-ID: <062.2372da7674cd00985e98eb3e50ee2b45@tracker.bro-ids.org>

#510: Upgrade How-To
-----------------------------+------------------------
  Reporter:  robin           |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro1.6
 Component:  Bro             |    Version:  git/master
Resolution:  Solved/Applied  |   Keywords:  beta
-----------------------------+------------------------
Changes (by robin):

 * status:  new => closed
 * resolution:   => Solved/Applied


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/510#comment:5>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From jsiwek at ncsa.illinois.edu  Tue Oct 25 11:02:24 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Tue, 25 Oct 2011 13:02:24 -0500
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: First commit of
	notice framework docs. (88e988f)
In-Reply-To: <201110251547.p9PFloFf010049@bro-ids.icir.org>
References: <201110251547.p9PFloFf010049@bro-ids.icir.org>
Message-ID: <4FBFD900-13EE-4F3B-B410-92AD8FEF2239@ncsa.illinois.edu>


>    First commit of notice framework docs.

This currently causes a bunch of errors in generating the www docs (I think it ultimately still updates things, though) because the :bro:enum:, :bro:id:, etc. reST roles are only defined in the Broxygen-sphinx setup.  One option would be to move static documentation under the sphinx source dir so that you can easily link to the Bro API docs.  Another option could be just to do that later and remove those roles from the notice documentation for now.

- Jon

From hlin33 at illinois.edu  Tue Oct 25 11:24:42 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Tue, 25 Oct 2011 13:24:42 -0500
Subject: [Bro-Dev] Hui Lin_Problem to run simple Bro-pipe
Message-ID: <CAKq214kVNxQ34ha93z-iePSQtGYX8mLC3-yvUZpFcO15N2DeVw@mail.gmail.com>

Hi,

I think Bro-pipe is a special Broccoli client. So I try to test to run
Bro-pipe to see its effect. I can run it in older version of Bro (1.5) based
on 2009 workshop exercise. But when I follow the same step and run it in Bro
(1.6), nothing show up. I observe two situations:

1. directly run Bro binary without indicating interface
if I run Bro through command /usr/local/bro/bin/bro *.bro (without
indicating interface),
In Bro 1.5, after executing this command, Bro will continue execution.
But in Bro 1.6, Bro will terminate immediately. Is that the right
phenomenon?

2. run bro-pipe to send event to Bro instance
In Bro 1.5, I just first run command
/usr/local/bro/bin/bro *.bro  (let Bro run)
and run bro-pipe
/usr/local/bro/bin/bropipe host=127.0.0.1:47757 <http://127.0.0.1:47758> -f
*.bro-pipe
And Bro can detect event

But in Bro 1.6, I need to run command
/usr/local/bro/bin/bro -i eth0 *.bro  (let Bro run)
and run bro-pipe
/usr/local/bro/bin/bropipe host=127.0.0.1:47757 <http://127.0.0.1:47758/> -f
*.bro-pipe (I also try port 47758)
But Bro-pipe just stick there and there is even no warning such as "could
not connect Bro at ...". Bro does not detect any event

So how can I run Bro-Pipe in Bro 1.6


-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111025/8939d11d/attachment.html 

From hlin33 at illinois.edu  Tue Oct 25 12:19:19 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Tue, 25 Oct 2011 14:19:19 -0500
Subject: [Bro-Dev] Hui Lin_Install Broccoli-python
Message-ID: <CAKq214mGrCkcBqmm6=wYJArMbwxHpqE=BqoEfHA4A4rD3Oq-5w@mail.gmail.com>

Hi,
I try to install Broccoli-python and I get the following error:

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -I../../src -I/usr/include/python2.6 -c
broccoli_intern_wrap.c -o build/temp.linux-i686-2.6/broccoli_intern_wrap.o
broccoli_intern_wrap.c:2698:22: error: broccoli.h: No such file or directory

I search that broccoli.h is installed in
/bro/build/aux/broccoli/src/broccoli.h. not in ../../src
(/bro/aux/broccoli/src). So I guess that the script is kind of out-dated?
In ../../src (//bro/aux/broccoli/src), there is a file broccoli.h.in.

Best,

Hui




-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111025/638312bf/attachment.html 

From hlin33 at illinois.edu  Tue Oct 25 12:38:53 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Tue, 25 Oct 2011 14:38:53 -0500
Subject: [Bro-Dev] Hui Lin_Install Broccoli-python
In-Reply-To: <CAKq214mGrCkcBqmm6=wYJArMbwxHpqE=BqoEfHA4A4rD3Oq-5w@mail.gmail.com>
References: <CAKq214mGrCkcBqmm6=wYJArMbwxHpqE=BqoEfHA4A4rD3Oq-5w@mail.gmail.com>
Message-ID: <CAKq214=Rt4kE1RDdti4pqnnjzXybRs8eODDfP7HgUL8KLMCt_w@mail.gmail.com>

Continue trying to install broccoli.

I think the reason about my issue is that I did not install Broccoli C API
first.

Sorry about that

Best,

Hui

On Tue, Oct 25, 2011 at 2:19 PM, Hui Lin (Hugo) <hlin33 at illinois.edu> wrote:

> Hi,
> I try to install Broccoli-python and I get the following error:
>
> gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
> -Wstrict-prototypes -fPIC -I../../src -I/usr/include/python2.6 -c
> broccoli_intern_wrap.c -o build/temp.linux-i686-2.6/broccoli_intern_wrap.o
> broccoli_intern_wrap.c:2698:22: error: broccoli.h: No such file or
> directory
>
> I search that broccoli.h is installed in
> /bro/build/aux/broccoli/src/broccoli.h. not in ../../src
> (/bro/aux/broccoli/src). So I guess that the script is kind of out-dated?
> In ../../src (//bro/aux/broccoli/src), there is a file broccoli.h.in.
>
> Best,
>
> Hui
>
>
>
>
> --
> Hui Lin
> Research Assistant
> DEPEND Research Group, ECE Department
> University of Illinois at Urbana-Champaign
> hlin33 at illinois.edu
>



-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111025/63669aef/attachment.html 

From jsiwek at ncsa.illinois.edu  Tue Oct 25 12:47:04 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Tue, 25 Oct 2011 14:47:04 -0500
Subject: [Bro-Dev] Hui Lin_Problem to run simple Bro-pipe
In-Reply-To: <CAKq214kVNxQ34ha93z-iePSQtGYX8mLC3-yvUZpFcO15N2DeVw@mail.gmail.com>
References: <CAKq214kVNxQ34ha93z-iePSQtGYX8mLC3-yvUZpFcO15N2DeVw@mail.gmail.com>
Message-ID: <FF53CBD5-D05C-4F8B-A3E4-E93A3A47E0AB@ncsa.illinois.edu>

> I think Bro-pipe is a special Broccoli client. So I try to test to run Bro-pipe to see its effect. I can run it in older version of Bro (1.5) based on 2009 workshop exercise. But when I follow the same step and run it in Bro (1.6), nothing show up. I observe two situations:

I haven't tried to build bro-pipe lately, but in your tests, did you compile separate versions to use against the Bro 1.6 and Bro 1.5 binaries?  The communication protocol or at least serialization scheme I think maybe changed between versions and might cause trouble?

- Jon

From jsiwek at ncsa.illinois.edu  Tue Oct 25 12:53:49 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Tue, 25 Oct 2011 14:53:49 -0500
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Experimental code to
	better handle interpreter errors. (15ab287)
In-Reply-To: <201110211843.p9LIhfd8007228@bro-ids.icir.org>
References: <201110211843.p9LIhfd8007228@bro-ids.icir.org>
Message-ID: <2C206CF9-01B7-4AB9-8AE9-8E13B2DBA372@ncsa.illinois.edu>

I think this commit caused the failures of

testing/btest/scripts/base/frameworks/logging/pred.bro
testing/btest/scripts/base/frameworks/logging/remote.bro

Looks like something to do with log filtering got broken, not sure how.

- Jon

On Oct 21, 2011, at 1:43 PM, Robin Sommer wrote:

> Repository : ssh://git at bro-ids.icir.org/bro
> 
> On branches: master,topic/robin/interpreter-exceptions
> Link       : http://tracker.bro-ids.org/bro/changeset/15ab2874369b5d7a3e6a14df24b141fa759999bb/bro
> 
>> ---------------------------------------------------------------
> 
> commit 15ab2874369b5d7a3e6a14df24b141fa759999bb
> Author: Robin Sommer <robin at icir.org>
> Date:   Sun Oct 9 20:28:06 2011 -0700
> 
>    Experimental code to better handle interpreter errors.
> 
>    Currently, a lot of interpreter runtime errors, such as an access to
>    an unset optional record field, cause Bro to abort with an internal
>    error. This is an experimental branch that turns such errors into
>    non-fatal runtime errors by internally raising exceptions. These are
>    caught upstream and processing continues afterwards.
> 
>    For now, not many errors actually raise exceptions (the example above
>    does though). We'll need to go through them eventually and adapt the
>    current Internal() calls (and potentially others). More generally, at
>    some point we should cleanup the interpreter error handling (unifying
>    errors reported at parse- and runtime; and switching to exceptions for
>    all Expr/Stmt/Vals). But that's a larger change and left for later.
> 
>    The main question for now is if this code is already helpful enough to
>    go into 2.0. It will quite likely prevent a number of crashes due to
>    script errors.
> 
> 
>> ---------------------------------------------------------------
> 
> 15ab2874369b5d7a3e6a14df24b141fa759999bb
> src/Discard.cc               |   48 ++++++++++++++++++++++++++++++++++++++---
> src/Event.h                  |   11 ++++++++-
> src/EventHandler.cc          |    1 +
> src/Expr.cc                  |   10 +++++---
> src/LogMgr.cc                |   43 +++++++++++++++++++++++++++++++------
> src/PersistenceSerializer.cc |    8 ++++++-
> src/RemoteSerializer.cc      |    8 ++++++-
> src/Reporter.cc              |   44 +++++++++++++++++++++++++++++--------
> src/Reporter.h               |   24 ++++++++++++++++++++-
> src/RuleCondition.cc         |   16 +++++++++++--
> src/Sessions.cc              |    9 +++++++-
> src/Val.cc                   |   29 +++++++++++++++++++++----
> 12 files changed, 213 insertions(+), 38 deletions(-)
> 
> diff --git a/src/Discard.cc b/src/Discard.cc
> index 2705aa5..a71b810 100644
> --- a/src/Discard.cc
> +++ b/src/Discard.cc
> @@ -42,7 +42,17 @@ int Discarder::NextPacket(const IP_Hdr* ip, int len, int caplen)
> 		{
> 		val_list* args = new val_list;
> 		args->append(BuildHeader(ip4));
> -		discard_packet = check_ip->Call(args)->AsBool();
> +
> +		try
> +			{
> +			discard_packet = check_ip->Call(args)->AsBool();
> +			}
> +
> +		catch ( InterpreterException& e )
> +			{
> +			discard_packet = false;
> +			}
> +
> 		delete args;
> 
> 		if ( discard_packet )
> @@ -90,7 +100,17 @@ int Discarder::NextPacket(const IP_Hdr* ip, int len, int caplen)
> 			args->append(BuildHeader(ip4));
> 			args->append(BuildHeader(tp, len));
> 			args->append(BuildData(data, th_len, len, caplen));
> -			discard_packet = check_tcp->Call(args)->AsBool();
> +
> +			try
> +				{
> +				discard_packet = check_tcp->Call(args)->AsBool();
> +				}
> +
> +			catch ( InterpreterException& e )
> +				{
> +				discard_packet = false;
> +				}
> +
> 			delete args;
> 			}
> 		}
> @@ -106,7 +126,17 @@ int Discarder::NextPacket(const IP_Hdr* ip, int len, int caplen)
> 			args->append(BuildHeader(ip4));
> 			args->append(BuildHeader(up));
> 			args->append(BuildData(data, uh_len, len, caplen));
> -			discard_packet = check_udp->Call(args)->AsBool();
> +
> +			try
> +				{
> +				discard_packet = check_udp->Call(args)->AsBool();
> +				}
> +
> +			catch ( InterpreterException& e )
> +				{
> +				discard_packet = false;
> +				}
> +
> 			delete args;
> 			}
> 		}
> @@ -120,7 +150,17 @@ int Discarder::NextPacket(const IP_Hdr* ip, int len, int caplen)
> 			val_list* args = new val_list;
> 			args->append(BuildHeader(ip4));
> 			args->append(BuildHeader(ih));
> -			discard_packet = check_icmp->Call(args)->AsBool();
> +
> +			try
> +				{
> +				discard_packet = check_icmp->Call(args)->AsBool();
> +				}
> +
> +			catch ( InterpreterException& e )
> +				{
> +				discard_packet = false;
> +				}
> +
> 			delete args;
> 			}
> 		}
> diff --git a/src/Event.h b/src/Event.h
> index 805396a..1740372 100644
> --- a/src/Event.h
> +++ b/src/Event.h
> @@ -41,7 +41,16 @@ protected:
> 		if ( handler->ErrorHandler() )
> 			reporter->BeginErrorHandler();
> 
> -		handler->Call(args, no_remote);
> +		try
> +			{
> +			handler->Call(args, no_remote);
> +			}
> +			
> +		catch ( InterpreterException& e )
> +			{
> +			// Already reported.
> +			}
> +
> 		if ( obj )
> 			// obj->EventDone();
> 			Unref(obj);
> diff --git a/src/EventHandler.cc b/src/EventHandler.cc
> index 55f9902..2867b63 100644
> --- a/src/EventHandler.cc
> +++ b/src/EventHandler.cc
> @@ -68,6 +68,7 @@ void EventHandler::Call(val_list* vl, bool no_remote)
> 		}
> 
> 	if ( local )
> +		// No try/catch here; we pass exceptions upstream.
> 		Unref(local->Call(vl));
> 	else
> 		{
> diff --git a/src/Expr.cc b/src/Expr.cc
> index 9fecab4..f6d1fc5 100644
> --- a/src/Expr.cc
> +++ b/src/Expr.cc
> @@ -3118,8 +3118,9 @@ Val* FieldExpr::Fold(Val* v) const
> 		return def_attr->AttrExpr()->Eval(0);
> 	else
> 		{
> -		Internal("field value missing");
> -		return 0;
> +		reporter->ExprRuntimeError(this, "field value missing");
> +		assert(false);
> +		return 0; // Will never get here, but compiler can't tell.
> 		}
> 	}
> 
> @@ -3730,6 +3731,7 @@ Val* RecordMatchExpr::Fold(Val* v1, Val* v2) const
> 				}
> 			}
> 
> +		// No try/catch here; we pass exceptions upstream.
> 		Val* pred_val =
> 			match_rec->Lookup(pred_field_index)->AsFunc()->Call(&args);
> 		bool is_zero = pred_val->IsZero();
> @@ -4220,7 +4222,7 @@ Val* FlattenExpr::Fold(Val* v) const
> 			l->Append(fa->AttrExpr()->Eval(0));
> 
> 		else
> -			Internal("missing field value");
> +			reporter->ExprRuntimeError(this, "missing field value");
> 		}
> 
> 	return l;
> @@ -4646,7 +4648,7 @@ Val* CallExpr::Eval(Frame* f) const
> 
> 		if ( f )
> 			f->SetCall(this);
> -		ret = func->Call(v, f);
> +		ret = func->Call(v, f); // No try/catch here; we pass exceptions upstream.
> 		if ( f )
> 			f->ClearCall();
> 		// Don't Unref() the arguments, as Func::Call already did that.
> diff --git a/src/LogMgr.cc b/src/LogMgr.cc
> index 3f088fd..473f480 100644
> --- a/src/LogMgr.cc
> +++ b/src/LogMgr.cc
> @@ -888,9 +888,18 @@ bool LogMgr::Write(EnumVal* id, RecordVal* columns)
> 			// to log this record.
> 			val_list vl(1);
> 			vl.append(columns->Ref());
> -			Val* v = filter->pred->Call(&vl);
> -			int result = v->AsBool();
> -			Unref(v);
> +
> +			int result = 1;
> +
> +			try
> +				{
> +				Val* v = filter->pred->Call(&vl);
> +				int result = v->AsBool();
> +				Unref(v);
> +				}
> +
> +			catch ( InterpreterException& e )
> +				{ /* Already reported. */ }
> 
> 			if ( ! result )
> 				continue;
> @@ -920,7 +929,17 @@ bool LogMgr::Write(EnumVal* id, RecordVal* columns)
> 
> 			vl.append(rec_arg);
> 
> -			Val* v = filter->path_func->Call(&vl);
> +			Val* v = 0;
> +
> +			try
> +				{
> +				v = filter->path_func->Call(&vl);
> +				}
> +
> +			catch ( InterpreterException& e )
> +				{
> +				return false;
> +				}
> 
> 			if ( ! v->Type()->Tag() == TYPE_STRING )
> 				{
> @@ -1569,9 +1588,19 @@ bool LogMgr::FinishedRotation(LogWriter* writer, string new_name, string old_nam
> 	// Call the postprocessor function.
> 	val_list vl(1);
> 	vl.append(info);
> -	Val* v = func->Call(&vl);
> -	int result = v->AsBool();
> -	Unref(v);
> +
> +	int result = 0;
> +
> +	try
> +		{
> +		Val* v = func->Call(&vl);
> +		result = v->AsBool();
> +		Unref(v);
> +		}
> +
> +	catch ( InterpreterException& e )
> +		{ /* Already reported. */ }
> +
> 	return result;
> 	}
> 
> diff --git a/src/PersistenceSerializer.cc b/src/PersistenceSerializer.cc
> index c72f59c..c757467 100644
> --- a/src/PersistenceSerializer.cc
> +++ b/src/PersistenceSerializer.cc
> @@ -200,7 +200,13 @@ void PersistenceSerializer::GotEvent(const char* name, double time,
> void PersistenceSerializer::GotFunctionCall(const char* name, double time,
> 					Func* func, val_list* args)
> 	{
> -	func->Call(args);
> +	try
> +		{
> +		func->Call(args);
> +		}
> +
> +	catch ( InterpreterException& e )
> +		{ /* Already reported. */ }
> 	}
> 
> void PersistenceSerializer::GotStateAccess(StateAccess* s)
> diff --git a/src/RemoteSerializer.cc b/src/RemoteSerializer.cc
> index e359a4a..7926894 100644
> --- a/src/RemoteSerializer.cc
> +++ b/src/RemoteSerializer.cc
> @@ -2809,7 +2809,13 @@ void RemoteSerializer::GotFunctionCall(const char* name, double time,
> 		return;
> 		}
> 
> -	function->Call(args);
> +	try
> +		{
> +		function->Call(args);
> +		}
> +
> +	catch ( InterpreterException& e )
> +		{ /* Already reported. */ }
> 	}
> 
> void RemoteSerializer::GotID(ID* id, Val* val)
> diff --git a/src/Reporter.cc b/src/Reporter.cc
> index b2bffd3..fca9a6a 100644
> --- a/src/Reporter.cc
> +++ b/src/Reporter.cc
> @@ -39,7 +39,7 @@ void Reporter::Info(const char* fmt, ...)
> 	{
> 	va_list ap;
> 	va_start(ap, fmt);
> -	DoLog("", reporter_info, stderr, 0, 0, true, true, fmt, ap);
> +	DoLog("", reporter_info, stderr, 0, 0, true, true, 0, fmt, ap);
> 	va_end(ap);
> 	}
> 
> @@ -47,7 +47,7 @@ void Reporter::Warning(const char* fmt, ...)
> 	{
> 	va_list ap;
> 	va_start(ap, fmt);
> -	DoLog("warning", reporter_warning, stderr, 0, 0, true, true, fmt, ap);
> +	DoLog("warning", reporter_warning, stderr, 0, 0, true, true, 0, fmt, ap);
> 	va_end(ap);
> 	}
> 
> @@ -56,7 +56,7 @@ void Reporter::Error(const char* fmt, ...)
> 	++errors;
> 	va_list ap;
> 	va_start(ap, fmt);
> -	DoLog("error", reporter_error, stderr, 0, 0, true, true, fmt, ap);
> +	DoLog("error", reporter_error, stderr, 0, 0, true, true, 0, fmt, ap);
> 	va_end(ap);
> 	}
> 
> @@ -66,7 +66,7 @@ void Reporter::FatalError(const char* fmt, ...)
> 	va_start(ap, fmt);
> 
> 	// Always log to stderr.
> -	DoLog("fatal error", 0, stderr, 0, 0, true, false, fmt, ap);
> +	DoLog("fatal error", 0, stderr, 0, 0, true, false, 0, fmt, ap);
> 
> 	va_end(ap);
> 
> @@ -80,7 +80,7 @@ void Reporter::FatalErrorWithCore(const char* fmt, ...)
> 	va_start(ap, fmt);
> 
> 	// Always log to stderr.
> -	DoLog("fatal error", 0, stderr, 0, 0, true, false, fmt, ap);
> +	DoLog("fatal error", 0, stderr, 0, 0, true, false, 0, fmt, ap);
> 
> 	va_end(ap);
> 
> @@ -88,13 +88,29 @@ void Reporter::FatalErrorWithCore(const char* fmt, ...)
> 	abort();
> 	}
> 
> +void Reporter::ExprRuntimeError(const Expr* expr, const char* fmt, ...)
> +	{
> +	++errors;
> +
> +	ODesc d;
> +	expr->Describe(&d);
> +
> +	PushLocation(expr->GetLocationInfo());
> +	va_list ap;
> +	va_start(ap, fmt);
> +	DoLog("expression error", reporter_error, stderr, 0, 0, true, true, d.Description(), fmt, ap);
> +	va_end(ap);
> +	PopLocation();
> +	throw InterpreterException();
> +	}
> +
> void Reporter::InternalError(const char* fmt, ...)
> 	{
> 	va_list ap;
> 	va_start(ap, fmt);
> 
> 	// Always log to stderr.
> -	DoLog("internal error", 0, stderr, 0, 0, true, false, fmt, ap);
> +	DoLog("internal error", 0, stderr, 0, 0, true, false, 0, fmt, ap);
> 
> 	va_end(ap);
> 
> @@ -106,7 +122,7 @@ void Reporter::InternalWarning(const char* fmt, ...)
> 	{
> 	va_list ap;
> 	va_start(ap, fmt);
> -	DoLog("internal warning", reporter_warning, stderr, 0, 0, true, true, fmt, ap);
> +	DoLog("internal warning", reporter_warning, stderr, 0, 0, true, true, 0, fmt, ap);
> 	va_end(ap);
> 	}
> 
> @@ -133,7 +149,7 @@ void Reporter::WeirdHelper(EventHandlerPtr event, Val* conn_val, const char* add
> 
> 	va_list ap;
> 	va_start(ap, fmt_name);
> -	DoLog("weird", event, stderr, 0, vl, false, false, fmt_name, ap);
> +	DoLog("weird", event, stderr, 0, vl, false, false, 0, fmt_name, ap);
> 	va_end(ap);
> 
> 	delete vl;
> @@ -147,7 +163,7 @@ void Reporter::WeirdFlowHelper(const uint32* orig, const uint32* resp, const cha
> 
> 	va_list ap;
> 	va_start(ap, fmt_name);
> -	DoLog("weird", flow_weird, stderr, 0, vl, false, false, fmt_name, ap);
> +	DoLog("weird", flow_weird, stderr, 0, vl, false, false, 0, fmt_name, ap);
> 	va_end(ap);
> 
> 	delete vl;
> @@ -173,7 +189,7 @@ void Reporter::Weird(const uint32* orig, const uint32* resp, const char* name)
> 	WeirdFlowHelper(orig, resp, "%s", name);
> 	}
> 
> -void Reporter::DoLog(const char* prefix, EventHandlerPtr event, FILE* out, Connection* conn, val_list* addl, bool location, bool time, const char* fmt, va_list ap)
> +void Reporter::DoLog(const char* prefix, EventHandlerPtr event, FILE* out, Connection* conn, val_list* addl, bool location, bool time, const char* postfix, const char* fmt, va_list ap)
> 	{
> 	static char tmp[512];
> 
> @@ -235,6 +251,9 @@ void Reporter::DoLog(const char* prefix, EventHandlerPtr event, FILE* out, Conne
> 		int n = vsnprintf(buffer, size, fmt, aq);
> 		va_end(aq);
> 
> +		if ( postfix )
> +			n += strlen(postfix) + 10; // Add a bit of slack.
> +
> 		if ( n > -1 && n < size )
> 			// We had enough space;
> 			break;
> @@ -247,6 +266,11 @@ void Reporter::DoLog(const char* prefix, EventHandlerPtr event, FILE* out, Conne
> 			FatalError("out of memory in Reporter");
> 		}
> 
> +	if ( postfix )
> +		// Note, if you change this fmt string, adjust the additional
> +		// buffer size above.
> +		sprintf(buffer + strlen(buffer), " [%s]", postfix);
> +
> 	if ( event && via_events && ! in_error_handler )
> 		{
> 		val_list* vl = new val_list;
> diff --git a/src/Reporter.h b/src/Reporter.h
> index d0fc662..52a2aff 100644
> --- a/src/Reporter.h
> +++ b/src/Reporter.h
> @@ -14,6 +14,22 @@
> 
> class Connection;
> class Location;
> +class Reporter;
> +
> +// One cannot raise this exception directly, go through the
> +// Reporter's methods instead.
> +
> +class ReporterException {
> +protected:
> +	friend class Reporter;
> +	ReporterException()	{}
> +};
> +
> +class InterpreterException : public ReporterException {
> +protected:
> +	friend class Reporter;
> +	InterpreterException()	{}
> +};
> 
> class Reporter {
> public:
> @@ -42,6 +58,10 @@ public:
> 	// reported and always generate a core dump.
> 	void FatalErrorWithCore(const char* fmt, ...);
> 
> +	// Report a runtime error in evaluating a Bro script expression. This
> +	// function will not return but raise an InterpreterException.
> +	void ExprRuntimeError(const Expr* expr, const char* fmt, ...);
> +
> 	// Report a traffic weirdness, i.e., an unexpected protocol situation
> 	// that may lead to incorrectly processing a connnection.
> 	void Weird(const char* name);	// Raises net_weird().
> @@ -87,7 +107,9 @@ public:
> 	void EndErrorHandler()	{ --in_error_handler; }
> 
> private:
> -	void DoLog(const char* prefix, EventHandlerPtr event, FILE* out, Connection* conn, val_list* addl, bool location, bool time, const char* fmt, va_list ap);
> +	void DoLog(const char* prefix, EventHandlerPtr event, FILE* out,
> +		   Connection* conn, val_list* addl, bool location, bool time,
> +		   const char* postfix, const char* fmt, va_list ap);
> 
> 	// The order if addl, name needs to be like that since fmt_name can
> 	// contain format specifiers
> diff --git a/src/RuleCondition.cc b/src/RuleCondition.cc
> index 1b94fcf..8852747 100644
> --- a/src/RuleCondition.cc
> +++ b/src/RuleCondition.cc
> @@ -149,9 +149,19 @@ bool RuleConditionEval::DoMatch(Rule* rule, RuleEndpointState* state,
> 	else
> 		args.append(new StringVal(""));
> 
> -	Val* val = id->ID_Val()->AsFunc()->Call(&args);
> -	bool result = val->AsBool();
> -	Unref(val);
> +	bool result = 0;
> +
> +	try
> +		{
> +		Val* val = id->ID_Val()->AsFunc()->Call(&args);
> +		result = val->AsBool();
> +		Unref(val);
> +		}
> +
> +	catch ( InterpreterException& e )
> +		{
> +		result = false;
> +		}
> 
> 	return result;
> 	}
> diff --git a/src/Sessions.cc b/src/Sessions.cc
> index 1d2604b..572e8de 100644
> --- a/src/Sessions.cc
> +++ b/src/Sessions.cc
> @@ -331,7 +331,14 @@ void NetSessions::NextPacketSecondary(double /* t */, const struct pcap_pkthdr*
> 			args->append(cmd_val);
> 			args->append(BuildHeader(ip));
> 			// ### Need to queue event here.
> -			sp->Event()->Event()->Call(args);
> +			try
> +				{
> +				sp->Event()->Event()->Call(args);
> +				}
> +
> +			catch ( InterpreterException& e )
> +				{ /* Already reported. */ }
> +
> 			delete args;
> 			}
> 		}
> diff --git a/src/Val.cc b/src/Val.cc
> index 164994b..ddf04bb 100644
> --- a/src/Val.cc
> +++ b/src/Val.cc
> @@ -2007,7 +2007,16 @@ Val* TableVal::Default(Val* index)
> 	else
> 		vl->append(index->Ref());
> 
> -	Val* result = f->Call(vl);
> +	Val* result = 0;
> +
> +	try
> +		{
> +		result = f->Call(vl);
> +		}
> +		
> +	catch ( InterpreterException& e )
> +		{ /* Already reported. */ }
> +
> 	delete vl;
> 
> 	if ( ! result )
> @@ -2466,10 +2475,20 @@ double TableVal::CallExpireFunc(Val* idx)
> 
> 	vl->append(idx);
> 
> -	Val* vs = expire_expr->Eval(0)->AsFunc()->Call(vl);
> -	double secs = vs->AsInterval();
> -	Unref(vs);
> -	delete vl;
> +	double secs;
> +
> +	try
> +		{
> +		Val* vs = expire_expr->Eval(0)->AsFunc()->Call(vl);
> +		secs = vs->AsInterval();
> +		Unref(vs);
> +		delete vl;
> +		}
> +
> +	catch ( InterpreterException& e )
> +		{
> +		secs = 0;
> +		}
> 
> 	return secs;
> 	}
> 
> _______________________________________________
> bro-commits mailing list
> bro-commits at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits
> 



From hlin33 at illinois.edu  Tue Oct 25 12:55:54 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Tue, 25 Oct 2011 14:55:54 -0500
Subject: [Bro-Dev] Hui Lin_Problem to run simple Bro-pipe
In-Reply-To: <96b1587ccc124a3797bab5198443e44b@CITESHT4.ad.uillinois.edu>
References: <CAKq214kVNxQ34ha93z-iePSQtGYX8mLC3-yvUZpFcO15N2DeVw@mail.gmail.com>
	<96b1587ccc124a3797bab5198443e44b@CITESHT4.ad.uillinois.edu>
Message-ID: <CAKq214nQWyYWv0_YWN_NLvS-PUTUTZcLEbCjKb7QsQh=qKGi4g@mail.gmail.com>

Bro 1.5 is actually installed in Ubuntu Hardy (8.04) a kind of old version.

Bro 1.6 is from git and installed on Ubuntu Lucid (10.04).

For current version  of Bro, how can I make it continue running either in
the foreground or background? And Bro in this version listen to 47785 port?

I am taking look at Broccoli C API right now, hope Broccoli manual can solve
some of my confusion.



On Tue, Oct 25, 2011 at 2:47 PM, Jonathan Siwek <jsiwek at ncsa.illinois.edu>wrote:

> > I think Bro-pipe is a special Broccoli client. So I try to test to run
> Bro-pipe to see its effect. I can run it in older version of Bro (1.5) based
> on 2009 workshop exercise. But when I follow the same step and run it in Bro
> (1.6), nothing show up. I observe two situations:
>
> I haven't tried to build bro-pipe lately, but in your tests, did you
> compile separate versions to use against the Bro 1.6 and Bro 1.5 binaries?
>  The communication protocol or at least serialization scheme I think maybe
> changed between versions and might cause trouble?
>
> - Jon




-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111025/31c8d6e2/attachment.html 

From jsiwek at ncsa.illinois.edu  Tue Oct 25 13:05:08 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Tue, 25 Oct 2011 15:05:08 -0500
Subject: [Bro-Dev] Hui Lin_Problem to run simple Bro-pipe
In-Reply-To: <CAKq214nQWyYWv0_YWN_NLvS-PUTUTZcLEbCjKb7QsQh=qKGi4g@mail.gmail.com>
References: <CAKq214kVNxQ34ha93z-iePSQtGYX8mLC3-yvUZpFcO15N2DeVw@mail.gmail.com>
	<96b1587ccc124a3797bab5198443e44b@CITESHT4.ad.uillinois.edu>
	<CAKq214nQWyYWv0_YWN_NLvS-PUTUTZcLEbCjKb7QsQh=qKGi4g@mail.gmail.com>
Message-ID: <CF5BDFB0-D9EE-4046-B8EB-E121D7D04407@ncsa.illinois.edu>

> Bro 1.5 is actually installed in Ubuntu Hardy (8.04) a kind of old version.
> 
> Bro 1.6 is from git and installed on Ubuntu Lucid (10.04). 

I meant make sure that you compile different bro-pipes when you try to talk to the different bros?

> 
> For current version  of Bro, how can I make it continue running either in the foreground or background? And Bro in this version listen to 47785 port?

Try `bro frameworks/communication/listen` and it should listen on 47757.

- Jon

From hlin33 at illinois.edu  Tue Oct 25 13:20:21 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Tue, 25 Oct 2011 15:20:21 -0500
Subject: [Bro-Dev] Hui Lin_Problem to run simple Bro-pipe
In-Reply-To: <d9eef4020b7944059e3e1d15ba44e567@CITESHT4.ad.uillinois.edu>
References: <CAKq214kVNxQ34ha93z-iePSQtGYX8mLC3-yvUZpFcO15N2DeVw@mail.gmail.com>
	<96b1587ccc124a3797bab5198443e44b@CITESHT4.ad.uillinois.edu>
	<CAKq214nQWyYWv0_YWN_NLvS-PUTUTZcLEbCjKb7QsQh=qKGi4g@mail.gmail.com>
	<d9eef4020b7944059e3e1d15ba44e567@CITESHT4.ad.uillinois.edu>
Message-ID: <CAKq214kGJn9XBntx-TPufEbt=Y69KX9Khozb3KqS4c-X2KTUNA@mail.gmail.com>

On Tue, Oct 25, 2011 at 3:05 PM, Jonathan Siwek <jsiwek at ncsa.illinois.edu>wrote:

> > Bro 1.5 is actually installed in Ubuntu Hardy (8.04) a kind of old
> version.
> >
> > Bro 1.6 is from git and installed on Ubuntu Lucid (10.04).
>
> I meant make sure that you compile different bro-pipes when you try to talk
> to the different bros?
>
> >
> > For current version  of Bro, how can I make it continue running either in
> the foreground or background? And Bro in this version listen to 47785 port?
>
> Try `bro frameworks/communication/listen` and it should listen on 47757.
>

Directly running this command in the shell? where is this
?frameworks/communication/listen??

>
> - Jon




-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111025/c3fa9b85/attachment.html 

From jsiwek at ncsa.illinois.edu  Tue Oct 25 13:23:08 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Tue, 25 Oct 2011 15:23:08 -0500
Subject: [Bro-Dev] Hui Lin_Problem to run simple Bro-pipe
In-Reply-To: <CAKq214kGJn9XBntx-TPufEbt=Y69KX9Khozb3KqS4c-X2KTUNA@mail.gmail.com>
References: <CAKq214kVNxQ34ha93z-iePSQtGYX8mLC3-yvUZpFcO15N2DeVw@mail.gmail.com>
	<96b1587ccc124a3797bab5198443e44b@CITESHT4.ad.uillinois.edu>
	<CAKq214nQWyYWv0_YWN_NLvS-PUTUTZcLEbCjKb7QsQh=qKGi4g@mail.gmail.com>
	<d9eef4020b7944059e3e1d15ba44e567@CITESHT4.ad.uillinois.edu>
	<CAKq214kGJn9XBntx-TPufEbt=Y69KX9Khozb3KqS4c-X2KTUNA@mail.gmail.com>
Message-ID: <11939326-429E-47B0-BE37-F1B28D6941CD@ncsa.illinois.edu>


> Directly running this command in the shell? 

Yes.

> where is this ?frameworks/communication/listen?? 

That's shorthand, but the actually script is in scripts/policy/frameworks/communication/listen.bro (within the Bro source tree.

- Jon

From robin at icir.org  Tue Oct 25 14:09:45 2011
From: robin at icir.org (Robin Sommer)
Date: Tue, 25 Oct 2011 14:09:45 -0700
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Experimental code
	to	better handle interpreter errors. (15ab287)
In-Reply-To: <2C206CF9-01B7-4AB9-8AE9-8E13B2DBA372@ncsa.illinois.edu>
References: <201110211843.p9LIhfd8007228@bro-ids.icir.org>
	<2C206CF9-01B7-4AB9-8AE9-8E13B2DBA372@ncsa.illinois.edu>
Message-ID: <20111025210945.GQ40905@icir.org>


On Tue, Oct 25, 2011 at 14:53 -0500, you wrote:

> testing/btest/scripts/base/frameworks/logging/pred.bro
> testing/btest/scripts/base/frameworks/logging/remote.bro

Don't think I saw them failing for me, will check.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From robin at icir.org  Tue Oct 25 14:59:12 2011
From: robin at icir.org (Robin Sommer)
Date: Tue, 25 Oct 2011 14:59:12 -0700
Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: First commit of
	notice	framework docs. (88e988f)
In-Reply-To: <4FBFD900-13EE-4F3B-B410-92AD8FEF2239@ncsa.illinois.edu>
References: <201110251547.p9PFloFf010049@bro-ids.icir.org>
	<4FBFD900-13EE-4F3B-B410-92AD8FEF2239@ncsa.illinois.edu>
Message-ID: <20111025215912.GY40905@icir.org>


On Tue, Oct 25, 2011 at 13:02 -0500, you wrote:

> think it ultimately still updates things, though) because the
> :bro:enum:, :bro:id:, etc. reST roles are only defined in the
> Broxygen-sphinx setup.  One option would be to move static
> documentation under the sphinx source dir so that you can easily link
> to the Bro API docs.  Another option could be just to do that later
> and remove those roles from the notice documentation for now.

Ah, didn't think about that. Yes, moving under the Sphinx source would
be good eventually. Let's do that when we have better integrated the
Sphinx pages with how the rest of the web docs (which I started at
some point already, but never got very far with it).

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From hlin33 at illinois.edu  Tue Oct 25 18:40:07 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Tue, 25 Oct 2011 20:40:07 -0500
Subject: [Bro-Dev] Hui Lin_Problem to run simple Bro-pipe
In-Reply-To: <335c5d761ad04429b80aa57eef90562b@CITESHT4.ad.uillinois.edu>
References: <CAKq214kVNxQ34ha93z-iePSQtGYX8mLC3-yvUZpFcO15N2DeVw@mail.gmail.com>
	<96b1587ccc124a3797bab5198443e44b@CITESHT4.ad.uillinois.edu>
	<CAKq214nQWyYWv0_YWN_NLvS-PUTUTZcLEbCjKb7QsQh=qKGi4g@mail.gmail.com>
	<d9eef4020b7944059e3e1d15ba44e567@CITESHT4.ad.uillinois.edu>
	<CAKq214kGJn9XBntx-TPufEbt=Y69KX9Khozb3KqS4c-X2KTUNA@mail.gmail.com>
	<335c5d761ad04429b80aa57eef90562b@CITESHT4.ad.uillinois.edu>
Message-ID: <CAKq214kFDB=2y5ak-L8nO83HNfAPTeOtk24oN+9pM_HdvpQMDQ@mail.gmail.com>

On Tue, Oct 25, 2011 at 3:23 PM, Jonathan Siwek <jsiwek at ncsa.illinois.edu>wrote:

>
> > Directly running this command in the shell?
>
> Yes.
>
> > where is this ?frameworks/communication/listen??
>
> That's shorthand, but the actually script is in
> scripts/policy/frameworks/communication/listen.bro (within the Bro source
> tree.
>

that is weird, I don't have scripts directory in my bro source package. And
I also could find "listen.bro" both in Bro source package as well Bro's
installation directory.

>
> - Jon




-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111025/1cf86917/attachment.html 

From robin at icir.org  Tue Oct 25 19:25:47 2011
From: robin at icir.org (Robin Sommer)
Date: Tue, 25 Oct 2011 19:25:47 -0700
Subject: [Bro-Dev] bro-cut
Message-ID: <20111026022547.GB8301@icir.org>

I'm thinking bro-cut is something worth installing by default into
$prefix/bin/, even though it's living in bro-aux. Assuming you guys
agree, Jon, could you add the CMake magic for that? Thanks,

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Tue Oct 25 20:30:34 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 03:30:34 -0000
Subject: [Bro-Dev] #601: Beta documentation
In-Reply-To: <047.b53a75afaafe0544bba44907e863138d@tracker.bro-ids.org>
References: <047.b53a75afaafe0544bba44907e863138d@tracker.bro-ids.org>
Message-ID: <062.025c6123d6da3957f10e012507168a61@tracker.bro-ids.org>

#601: Beta documentation
-----------------------------+------------------------
  Reporter:  robin           |      Owner:
      Type:  Task            |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro1.6
 Component:  Bro             |    Version:  git/master
Resolution:  Solved/Applied  |   Keywords:  beta
-----------------------------+------------------------
Changes (by robin):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 Closing the for the beta, most of it is in place now.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/601#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From hlin33 at illinois.edu  Tue Oct 25 20:30:21 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Tue, 25 Oct 2011 22:30:21 -0500
Subject: [Bro-Dev] Hui Lin_Need Help on Git Merge Bro's master
Message-ID: <CAKq214nL=7n2AkBfERh4Hf3L54tqa4uaj_AHhPs02vcijrtzBA@mail.gmail.com>

Hi,

After two hours of trying and testing, I am totally messed up with Bro's Git
repository.

Today I discuss something with Jonathan about using broccoli and I find that
for the long time, I just work on my branch and I did not merge and update
from master branch into my branch. But I need to use some .bro policy in the
current master branch.

So I try, 'git fetch', 'git merge origin/master' in my own development
branch or in my local master branch. All I got is the error when I
./configure Bro's source code. Here are the errors:

**********************************************
CMake Error at CMakeLists.txt:3 (include):
  include could not find load file:

    cmake/CommonCMakeConfig.cmake


CMake Error at CMakeLists.txt:38 (include):
  include could not find load file:

    FindRequiredPackage
**************************

any comment?

How can I get the scripts developed in Master branch into my branch without
error?

Best,

Hui

-- Found sed: /bin/sed
CMake Error at CMakeLists.txt:50 (FindRequiredPackage):
  Unknown CMake command "FindRequiredPackage".




-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111025/d8082503/attachment.html 

From hlin33 at illinois.edu  Tue Oct 25 20:48:09 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Tue, 25 Oct 2011 22:48:09 -0500
Subject: [Bro-Dev] Hui Lin_default path to load .bro policy file
Message-ID: <CAKq214k=XRQ5yc9uxRpwhKnpSmZCSJ-bLscrK30Dmf==Y09qww@mail.gmail.com>

Hi,

Since I could not merge master branch into my own developing branch, I
manually copy ./script into separate location.
I try to run Bro with "listen.bro" policy. In this policy, it contains the
load command
@load base/frameworks/communication

When I run it, Bro complains that 'can't open
base/frameworks/communication'. I guess there should be some configuration
that Bro can know where to start find the .bro policy. I am just wondering
how to configure such path?

Best,

Hui



-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111025/65ce23bf/attachment.html 

From hlin33 at illinois.edu  Tue Oct 25 20:58:22 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Tue, 25 Oct 2011 22:58:22 -0500
Subject: [Bro-Dev] Hui Lin_broping can not connect bro
Message-ID: <CAKq214nTbz-0ThzvqyZF-fjEQRsaZymZ=bDFnvjvXzMQCEcb7A@mail.gmail.com>

Hi,

I compile broccoli c api and run the test program broping and I get the
following error:

"Could not connect to Bro at 127.0.0.1:47758."

I am wondering how to run Bro such that it listens on 47758 port?

Best,



-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111025/c2ea9b70/attachment.html 

From robin at icir.org  Tue Oct 25 21:25:45 2011
From: robin at icir.org (Robin Sommer)
Date: Tue, 25 Oct 2011 21:25:45 -0700
Subject: [Bro-Dev] Hui Lin_Need Help on Git Merge Bro's master
In-Reply-To: <CAKq214nL=7n2AkBfERh4Hf3L54tqa4uaj_AHhPs02vcijrtzBA@mail.gmail.com>
References: <CAKq214nL=7n2AkBfERh4Hf3L54tqa4uaj_AHhPs02vcijrtzBA@mail.gmail.com>
Message-ID: <20111026042545.GH40905@icir.org>


On Tue, Oct 25, 2011 at 22:30 -0500, you wrote:

>     cmake/CommonCMakeConfig.cmake

This sounds like the new cmake submodule isnt set up yet. Try:

    git submodule update --recursive --init

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From hlin33 at illinois.edu  Tue Oct 25 21:28:56 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Tue, 25 Oct 2011 23:28:56 -0500
Subject: [Bro-Dev] Hui Lin_Need Help on Git Merge Bro's master
In-Reply-To: <35f7149eceea4952922f6dbf5e5950e1@CITESHT4.ad.uillinois.edu>
References: <CAKq214nL=7n2AkBfERh4Hf3L54tqa4uaj_AHhPs02vcijrtzBA@mail.gmail.com>
	<35f7149eceea4952922f6dbf5e5950e1@CITESHT4.ad.uillinois.edu>
Message-ID: <CAKq214mah03kKqAQLd4LZsktDWZvktw5xO2ApxKn4KQ9g+CnfA@mail.gmail.com>

On Tue, Oct 25, 2011 at 11:25 PM, Robin Sommer <robin at icir.org> wrote:

>
> On Tue, Oct 25, 2011 at 22:30 -0500, you wrote:
>
> >     cmake/CommonCMakeConfig.cmake
>
> This sounds like the new cmake submodule isnt set up yet. Try:
>
>    git submodule update --recursive --init
>

Nope, this same error still happens for configure command.

>
> Robin
>
> --
> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
> ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
>



-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111025/412fb38c/attachment.html 

From hlin33 at illinois.edu  Tue Oct 25 21:30:19 2011
From: hlin33 at illinois.edu (Hui Lin (Hugo) )
Date: Tue, 25 Oct 2011 23:30:19 -0500
Subject: [Bro-Dev] Hui Lin_Need Help on Git Merge Bro's master
In-Reply-To: <CAKq214mah03kKqAQLd4LZsktDWZvktw5xO2ApxKn4KQ9g+CnfA@mail.gmail.com>
References: <CAKq214nL=7n2AkBfERh4Hf3L54tqa4uaj_AHhPs02vcijrtzBA@mail.gmail.com>
	<35f7149eceea4952922f6dbf5e5950e1@CITESHT4.ad.uillinois.edu>
	<CAKq214mah03kKqAQLd4LZsktDWZvktw5xO2ApxKn4KQ9g+CnfA@mail.gmail.com>
Message-ID: <CAKq214=wcAKq50dQu0Q2bVzeq3iYv5b1NAUwmEkKsvemgFGVNA@mail.gmail.com>

It seems that my master is too old. I have not updated since June.
I git clone separate one on different location and the master branch works
fine. But I don't know how to pull out my branch remotely into this separate
location.

On Tue, Oct 25, 2011 at 11:28 PM, Hui Lin (Hugo) <hlin33 at illinois.edu>wrote:

>
>
> On Tue, Oct 25, 2011 at 11:25 PM, Robin Sommer <robin at icir.org> wrote:
>
>>
>> On Tue, Oct 25, 2011 at 22:30 -0500, you wrote:
>>
>> >     cmake/CommonCMakeConfig.cmake
>>
>> This sounds like the new cmake submodule isnt set up yet. Try:
>>
>>    git submodule update --recursive --init
>>
>
> Nope, this same error still happens for configure command.
>
>>
>> Robin
>>
>> --
>> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
>> ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
>>
>
>
>
> --
> Hui Lin
> Research Assistant
> DEPEND Research Group, ECE Department
> University of Illinois at Urbana-Champaign
> hlin33 at illinois.edu
>



-- 
Hui Lin
Research Assistant
DEPEND Research Group, ECE Department
University of Illinois at Urbana-Champaign
hlin33 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111025/d397abfc/attachment-0001.html 

From bro at tracker.bro-ids.org  Tue Oct 25 21:35:30 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 04:35:30 -0000
Subject: [Bro-Dev] #620: SSL analyzer memory usage -- find root cause
In-Reply-To: <048.be0914f46337bb36376aa0f26a41a1d6@tracker.bro-ids.org>
References: <048.be0914f46337bb36376aa0f26a41a1d6@tracker.bro-ids.org>
Message-ID: <063.12d4a6394924d2dc6d67d9ff0037a574@tracker.bro-ids.org>

#620: SSL analyzer memory usage -- find root cause
-----------------------------+------------------------
  Reporter:  gregor          |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro1.7
 Component:  Bro             |    Version:  git/master
Resolution:  Solved/Applied  |   Keywords:
-----------------------------+------------------------
Changes (by seth):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 This was fixed by adding a &transient attribute to arrays in binpac.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/620#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 25 21:52:14 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 04:52:14 -0000
Subject: [Bro-Dev] #271: conn.log lost when cluster is restarted
In-Reply-To: <056.ef72654a3ac799a7ce9a658d554c14a8@tracker.bro-ids.org>
References: <056.ef72654a3ac799a7ce9a658d554c14a8@tracker.bro-ids.org>
Message-ID: <071.6d2ca863ad92c462fb1603fd0f388bdf@tracker.bro-ids.org>

#271: conn.log lost when cluster is restarted
-----------------------------+--------------------
  Reporter:  Tyler.Schoenke  |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:
 Component:  Bro             |    Version:  1.5.2
Resolution:  Solved/Applied  |   Keywords:
-----------------------------+--------------------
Changes (by seth):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 I believe this is fixed.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/271#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 25 21:54:35 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 04:54:35 -0000
Subject: [Bro-Dev] #246: (IRC::active_channels[IRC::my_c]): run-time
 error, no such index
In-Reply-To: <056.f18cc83b71a31929337c75a54c4aa645@tracker.bro-ids.org>
References: <056.f18cc83b71a31929337c75a54c4aa645@tracker.bro-ids.org>
Message-ID: <071.933a2b98d3a525e1298b79b1e9b5f128@tracker.bro-ids.org>

#246: (IRC::active_channels[IRC::my_c]): run-time error, no such index
-----------------------------+--------------------
  Reporter:  Tyler.Schoenke  |      Owner:
      Type:  defect          |     Status:  closed
  Priority:  Normal          |  Milestone:
 Component:  Bro             |    Version:  1.5.2
Resolution:  Solved/Applied  |   Keywords:
-----------------------------+--------------------
Changes (by seth):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 This code has been rewritten so this ticket is obsoleted.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/246#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 25 22:07:17 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 05:07:17 -0000
Subject: [Bro-Dev] #237: SSH connection states not correct
In-Reply-To: <047.aa22f7b6ea44599cd97222bae31793f3@tracker.bro-ids.org>
References: <047.aa22f7b6ea44599cd97222bae31793f3@tracker.bro-ids.org>
Message-ID: <062.8d720d29bbd41a974ca127b6873b51cf@tracker.bro-ids.org>

#237: SSH connection states not correct
----------------------+--------------------
  Reporter:  robin    |      Owner:  robin
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.7
 Component:  Bro      |    Version:  1.5.1
Resolution:           |   Keywords:
----------------------+--------------------
Changes (by seth):

 * milestone:   => Bro1.7


Comment:

 Assigning this to the next release.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/237#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 25 22:08:48 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 05:08:48 -0000
Subject: [Bro-Dev] #245: Recommendations for default cluster config
In-Reply-To: <046.f48eb6faaae2a0ab0722676852e1ded8@tracker.bro-ids.org>
References: <046.f48eb6faaae2a0ab0722676852e1ded8@tracker.bro-ids.org>
Message-ID: <061.cf259e52e06c5be05f427cc460d6c648@tracker.bro-ids.org>

#245: Recommendations for default cluster config
-----------------------------+--------------------
  Reporter:  seth            |      Owner:  robin
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:
 Component:  Bro             |    Version:  1.5.1
Resolution:  Solved/Applied  |   Keywords:
-----------------------------+--------------------
Changes (by seth):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 This is no longer relevant, the new scripts have been rewritten in ways
 that cope with overload in most situations better.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/245#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 25 22:13:58 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 05:13:58 -0000
Subject: [Bro-Dev] #145: Bro linker error with newer OpenSSL versions
In-Reply-To: <050.facdc95497ea903bc2a9d7f7d6d0d952@tracker.bro-ids.org>
References: <050.facdc95497ea903bc2a9d7f7d6d0d952@tracker.bro-ids.org>
Message-ID: <065.b55f5f3e236516809b0ae7c6fd0fa24d@tracker.bro-ids.org>

#145: Bro linker error with newer OpenSSL versions
-----------------------------+-----------------------------
  Reporter:  vallenti        |      Owner:  kreibich
      Type:  Problem         |     Status:  closed
  Priority:  Low             |  Milestone:
 Component:  Broccoli        |    Version:  1.5.2
Resolution:  Solved/Applied  |   Keywords:  openssl, linker
-----------------------------+-----------------------------
Changes (by seth):

 * status:  seen => closed
 * resolution:   => Solved/Applied


Comment:

 I haven't heard of any trouble with this in a long time so I'm closing
 this ticket.  We may have fixed the problem.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/145#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 25 22:15:16 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 05:15:16 -0000
Subject: [Bro-Dev] #76: Low port trolling vs. backscatter
In-Reply-To: <046.1fb244e6bfad5814adea9acd046e799b@tracker.bro-ids.org>
References: <046.1fb244e6bfad5814adea9acd046e799b@tracker.bro-ids.org>
Message-ID: <061.2611c96d2f4a54ecbc6ae27cac82b902@tracker.bro-ids.org>

#76: Low port trolling vs. backscatter
----------------------+------------------
  Reporter:  vern     |      Owner:
      Type:  Problem  |     Status:  seen
  Priority:  Low      |  Milestone:
 Component:  Bro      |    Version:
Resolution:           |   Keywords:  scan
----------------------+------------------
Changes (by seth):

 * keywords:   => scan


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/76#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 25 22:29:58 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 05:29:58 -0000
Subject: [Bro-Dev] #644: Compile SWIG bindings with no-strict-aliasing
In-Reply-To: <047.9cd2c206c9e71636a6e1fe53bf980b59@tracker.bro-ids.org>
References: <047.9cd2c206c9e71636a6e1fe53bf980b59@tracker.bro-ids.org>
Message-ID: <062.a971352a46a0daf0e11fd412bdbbe0e6@tracker.bro-ids.org>

#644: Compile SWIG bindings with no-strict-aliasing
-----------------------+--------------------
  Reporter:  robin     |      Owner:
      Type:  Problem   |     Status:  new
  Priority:  Normal    |  Milestone:  Bro1.6
 Component:  Broccoli  |    Version:
Resolution:            |   Keywords:
-----------------------+--------------------

Comment (by seth):

 Did this end up being done?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/644#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Oct 25 22:33:28 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 05:33:28 -0000
Subject: [Bro-Dev] #417: Bro Crash: internal_error () at
	SSLInterpreter.cc:30
In-Reply-To: <049.3deb9e3d518529d36f8f234ad1435b1b@tracker.bro-ids.org>
References: <049.3deb9e3d518529d36f8f234ad1435b1b@tracker.bro-ids.org>
Message-ID: <064.28fc1c20cae4d5b8e90622033b143b6e@tracker.bro-ids.org>

#417: Bro Crash: internal_error () at SSLInterpreter.cc:30
-------------------------------+--------------------
  Reporter:  aashish           |      Owner:
      Type:  Problem           |     Status:  closed
  Priority:  Normal            |  Milestone:
 Component:  Bro               |    Version:  1.5.2
Resolution:  Feedback Missing  |   Keywords:
-------------------------------+--------------------
Changes (by seth):

 * status:  new => closed
 * resolution:   => Feedback Missing


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/417#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From jsiwek at ncsa.illinois.edu  Wed Oct 26 06:21:12 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Wed, 26 Oct 2011 08:21:12 -0500
Subject: [Bro-Dev] Hui Lin_Problem to run simple Bro-pipe
In-Reply-To: <CAKq214kFDB=2y5ak-L8nO83HNfAPTeOtk24oN+9pM_HdvpQMDQ@mail.gmail.com>
References: <CAKq214kVNxQ34ha93z-iePSQtGYX8mLC3-yvUZpFcO15N2DeVw@mail.gmail.com>
	<96b1587ccc124a3797bab5198443e44b@CITESHT4.ad.uillinois.edu>
	<CAKq214nQWyYWv0_YWN_NLvS-PUTUTZcLEbCjKb7QsQh=qKGi4g@mail.gmail.com>
	<d9eef4020b7944059e3e1d15ba44e567@CITESHT4.ad.uillinois.edu>
	<CAKq214kGJn9XBntx-TPufEbt=Y69KX9Khozb3KqS4c-X2KTUNA@mail.gmail.com>
	<335c5d761ad04429b80aa57eef90562b@CITESHT4.ad.uillinois.edu>
	<CAKq214kFDB=2y5ak-L8nO83HNfAPTeOtk24oN+9pM_HdvpQMDQ@mail.gmail.com>
Message-ID: <C084715F-7F69-415E-9667-BB9FD2299953@ncsa.illinois.edu>

> that is weird, I don't have scripts directory in my bro source package. And I also could find "listen.bro" both in Bro source package as well Bro's installation directory.    

You're  working from a local clone of the git repository?  Have you updated it lately with the commands:

`git checkout master && git pull && git submodule update --recursive --init` ?

And if you were working from a branch, you can check it back out and merge the newly updated master branch into it.  Remember to also rebuild afterwards if anything needed updating.

- Jon

From jsiwek at ncsa.illinois.edu  Wed Oct 26 06:27:44 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Wed, 26 Oct 2011 08:27:44 -0500
Subject: [Bro-Dev] Hui Lin_default path to load .bro policy file
In-Reply-To: <CAKq214k=XRQ5yc9uxRpwhKnpSmZCSJ-bLscrK30Dmf==Y09qww@mail.gmail.com>
References: <CAKq214k=XRQ5yc9uxRpwhKnpSmZCSJ-bLscrK30Dmf==Y09qww@mail.gmail.com>
Message-ID: <CD8A1BBF-E9B0-4ED1-8529-2BEFAFD4144D@ncsa.illinois.edu>

> 
> Since I could not merge master branch into my own developing branch, I manually copy ./script into separate location.
> I try to run Bro with "listen.bro" policy. In this policy, it contains the load command 
> @load base/frameworks/communication
> 
> When I run it, Bro complains that 'can't open base/frameworks/communication'. I guess there should be some configuration that Bro can know where to start find the .bro policy. I am just wondering how to configure such path?

Probably the easiest way is modifying your BROPATH environment variable, but I'd recommend working out merging master with the changes in your topic branch to make sure things are sync'd up right.  I see you had another thread about that; will take a look a respond there if I see what's wrong.

- Jon

From jsiwek at ncsa.illinois.edu  Wed Oct 26 06:31:31 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Wed, 26 Oct 2011 08:31:31 -0500
Subject: [Bro-Dev] Hui Lin_broping can not connect bro
In-Reply-To: <CAKq214nTbz-0ThzvqyZF-fjEQRsaZymZ=bDFnvjvXzMQCEcb7A@mail.gmail.com>
References: <CAKq214nTbz-0ThzvqyZF-fjEQRsaZymZ=bDFnvjvXzMQCEcb7A@mail.gmail.com>
Message-ID: <1F997F90-CCA5-42C8-823B-FA88D08523BF@ncsa.illinois.edu>

> Hi, 
> 
> I compile broccoli c api and run the test program broping and I get the following error:
> 
> "Could not connect to Bro at 127.0.0.1:47758."
> 
> I am wondering how to run Bro such that it listens on 47758 port?

Yeah, the default port broping tries to connect to looks like 47758, not sure if that needs to change right now, but you can try either giving the "-p 47757" switch to broping to explicitly tell it which port to use, or redef the variable in Bro that sets the default listen port (e.g. "redef Communication::listen_port=47758/tcp")

- Jon

From jsiwek at ncsa.illinois.edu  Wed Oct 26 06:42:56 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Wed, 26 Oct 2011 08:42:56 -0500
Subject: [Bro-Dev] Hui Lin_Need Help on Git Merge Bro's master
In-Reply-To: <CAKq214=wcAKq50dQu0Q2bVzeq3iYv5b1NAUwmEkKsvemgFGVNA@mail.gmail.com>
References: <CAKq214nL=7n2AkBfERh4Hf3L54tqa4uaj_AHhPs02vcijrtzBA@mail.gmail.com>
	<35f7149eceea4952922f6dbf5e5950e1@CITESHT4.ad.uillinois.edu>
	<CAKq214mah03kKqAQLd4LZsktDWZvktw5xO2ApxKn4KQ9g+CnfA@mail.gmail.com>
	<CAKq214=wcAKq50dQu0Q2bVzeq3iYv5b1NAUwmEkKsvemgFGVNA@mail.gmail.com>
Message-ID: <0EB53F8A-7965-4ECB-BACC-8CA8DBE3433E@ncsa.illinois.edu>

> It seems that my master is too old. I have not updated since June. 
> I git clone separate one on different location and the master branch works fine. But I don't know how to pull out my branch remotely into this separate location.  
> 

>From the new clone, I think the command sequence will be:

git checkout <your topic branch>
git merge master

and if everything looks fine:

git push origin HEAD

If you had uncommitted/unpushed changes in your old clone that you wanted to recover and you had time to just come by my office, I can try to help -- I would have expected the recursive submodule update to fix it for you.

- Jon

From bro at tracker.bro-ids.org  Wed Oct 26 06:57:25 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 13:57:25 -0000
Subject: [Bro-Dev] #612: Segfault in identify_data BiF
In-Reply-To: <048.2145c0c1e4de755bc9fa18b2c7b0ac68@tracker.bro-ids.org>
References: <048.2145c0c1e4de755bc9fa18b2c7b0ac68@tracker.bro-ids.org>
Message-ID: <063.7b41e5b428390cedf7e77f51f0ea4d43@tracker.bro-ids.org>

#612: Segfault in identify_data BiF
----------------------+------------------------
  Reporter:  gregor   |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro1.6
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------

Comment (by seth):

 The seems to be gone, can we close the ticket?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/612#comment:9>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From jsiwek at ncsa.illinois.edu  Wed Oct 26 07:49:00 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Wed, 26 Oct 2011 09:49:00 -0500
Subject: [Bro-Dev] bro-cut
In-Reply-To: <20111026022547.GB8301@icir.org>
References: <20111026022547.GB8301@icir.org>
Message-ID: <E77672A0-D6AB-49A5-AA3B-B956B68EFC46@ncsa.illinois.edu>


> I'm thinking bro-cut is something worth installing by default into
> $prefix/bin/, even though it's living in bro-aux. Assuming you guys
> agree, Jon, could you add the CMake magic for that? Thanks,

Done.

From bro at tracker.bro-ids.org  Wed Oct 26 08:16:28 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 15:16:28 -0000
Subject: [Bro-Dev] #644: Compile SWIG bindings with no-strict-aliasing
In-Reply-To: <047.9cd2c206c9e71636a6e1fe53bf980b59@tracker.bro-ids.org>
References: <047.9cd2c206c9e71636a6e1fe53bf980b59@tracker.bro-ids.org>
Message-ID: <062.66944a54b3cb93c8162d1d0fb4fdbfa0@tracker.bro-ids.org>

#644: Compile SWIG bindings with no-strict-aliasing
-----------------------+--------------------
  Reporter:  robin     |      Owner:
      Type:  Problem   |     Status:  new
  Priority:  Normal    |  Milestone:  Bro1.6
 Component:  Broccoli  |    Version:
Resolution:            |   Keywords:
-----------------------+--------------------

Comment (by jsiwek):

 In [49be0367658ec9a3cd268a98970900ed32d1cd06/broccoli-python]:
 {{{
 #!CommitTicketReference repository="broccoli-python"
 revision="49be0367658ec9a3cd268a98970900ed32d1cd06"
 Compile SWIG bindings with no-strict-aliasing (addresses #644)
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/644#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct 26 08:16:40 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 15:16:40 -0000
Subject: [Bro-Dev] #644: Compile SWIG bindings with no-strict-aliasing
In-Reply-To: <047.9cd2c206c9e71636a6e1fe53bf980b59@tracker.bro-ids.org>
References: <047.9cd2c206c9e71636a6e1fe53bf980b59@tracker.bro-ids.org>
Message-ID: <062.2d356a775d317dd7aeada4bbd1d21cb8@tracker.bro-ids.org>

#644: Compile SWIG bindings with no-strict-aliasing
-----------------------+--------------------
  Reporter:  robin     |      Owner:
      Type:  Problem   |     Status:  new
  Priority:  Normal    |  Milestone:  Bro1.6
 Component:  Broccoli  |    Version:
Resolution:            |   Keywords:
-----------------------+--------------------

Comment (by jsiwek):

 In [fb869608178eb3c21878ab4054ac12ac6a5f1a4b/broccoli-ruby]:
 {{{
 #!CommitTicketReference repository="broccoli-ruby"
 revision="fb869608178eb3c21878ab4054ac12ac6a5f1a4b"
 Compile SWIG bindings with no-strict-aliasing (addresses #644)
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/644#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Oct 26 08:17:20 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 26 Oct 2011 15:17:20 -0000
Subject: [Bro-Dev] #644: Compile SWIG bindings with no-strict-aliasing
In-Reply-To: <047.9cd2c206c9e71636a6e1fe53bf980b59@tracker.bro-ids.org>
References: <047.9cd2c206c9e71636a6e1fe53bf980b59@tracker.bro-ids.org>
Message-ID: <062.65803565ffe1b3aff6eb6b00160cf341@tracker.bro-ids.org>

#644: Compile SWIG bindings with no-strict-aliasing
-----------------------+--------------------
  Reporter:  robin     |      Owner:  jsiwek
      Type:  Problem   |     Status:  closed
  Priority:  Normal    |  Milestone:  Bro1.6
 Component:  Broccoli  |    Version:
Resolution:  fixed     |   Keywords:
-----------------------+--------------------
Changes (by jsiwek):

 * owner:   => jsiwek
 * status:  new => closed
 * resolution:   => fixed


Comment:

 In [78b897fd86072758c79524b5efadd61690ed9170/pysubnettree]:
 {{{
 #!CommitTicketReference repository="pysubnettree"
 revision="78b897fd86072758c79524b5efadd61690ed9170"
 Compile SWIG bindings with no-strict-aliasing (closes #644)
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/644#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Wed Oct 26 10:43:43 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 26 Oct 2011 10:43:43 -0700
Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/pp-alarms: A new
	notice	script that pretty-prints alarms in the summary email.
	(73d5643)
In-Reply-To: <201110261741.p9QHfEjP030476@bro-ids.icir.org>
References: <201110261741.p9QHfEjP030476@bro-ids.icir.org>
Message-ID: <20111026174343.GM40905@icir.org>

Seth, what do you think of this approach? If loaded (which I'd like to
do by default), it will replace the standard raw notice alarms with
pretty printed versions.

Note, it's not finished yet; in particular, the actual pretty printing
is missing. But is this a good way to hook into the processing?

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From seth at icir.org  Wed Oct 26 11:09:36 2011
From: seth at icir.org (Seth Hall)
Date: Wed, 26 Oct 2011 14:09:36 -0400
Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/pp-alarms: A new
	notice script that pretty-prints alarms in the summary email.
	(73d5643)
In-Reply-To: <20111026174343.GM40905@icir.org>
References: <201110261741.p9QHfEjP030476@bro-ids.icir.org>
	<20111026174343.GM40905@icir.org>
Message-ID: <666FC09F-9583-421C-9693-1810805A2AD2@icir.org>


On Oct 26, 2011, at 1:43 PM, Robin Sommer wrote:

> Seth, what do you think of this approach? If loaded (which I'd like to
> do by default), it will replace the standard raw notice alarms with
> pretty printed versions.
> 
> Note, it's not finished yet; in particular, the actual pretty printing
> is missing. But is this a good way to hook into the processing?


Yep.  Handling the Notice::notice event is definitely the right way to hook in.

As I'm reading through it I'm having a few thoughts.  Maybe we should just make an extensions/ directory in the notice framework instead of the current extend-email/ and actions/ directories?  They're all basically just extensions they're just using different extension mechanisms.  I wouldn't say this exactly fits into actions and I didn't abstract ACTION_ALARM into alarms (which possibly should have) and then you could have just directly implemented this there.

And I'm going to back up on what I said before.  I can see this script taking a shell script or something to process the log file through as an alternate way of converting the log into the email body.  That might even make more sense as the primary approach.  I guess it would be like a prerotation filter or something?  We may be able to use something like that in more ways than we are even thinking of right now.  That way someone could provide a script that can turn Bro logs into webpages (as a sort of obvious example) if people want to receive their alarm emails in a "pretty" format.  Or, it could even attach it to the email as an attachment if people prefer that way.

I'm pretty sure you were right on that one, I just hadn't fully thought through it yet. ;)

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From robin at icir.org  Wed Oct 26 11:20:27 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 26 Oct 2011 11:20:27 -0700
Subject: [Bro-Dev] Notices and alarms (Re: [Bro-Commits]
	[git/bro]	topic/robin/pp-alarms: A new notice script that
	pretty-prints	alarms in the summary email. (73d5643))
In-Reply-To: <666FC09F-9583-421C-9693-1810805A2AD2@icir.org>
References: <201110261741.p9QHfEjP030476@bro-ids.icir.org>
	<20111026174343.GM40905@icir.org>
	<666FC09F-9583-421C-9693-1810805A2AD2@icir.org>
Message-ID: <20111026182027.GN40905@icir.org>


On Wed, Oct 26, 2011 at 14:09 -0400, you wrote:

> As I'm reading through it I'm having a few thoughts.  Maybe we should
> just make an extensions/ directory in the notice framework instead of
> the current extend-email/ and actions/ directories?

Yeah, sounds good, I was wonderng where to put this.

> And I'm going to back up on what I said before.  I can see this script
> taking a shell script or something to process the log file through as
> an alternate way of converting the log into the email body.

That's how I wanted to start but then one needs to parse the log in
shell. Thinking about it more, I agreed with what you said earlier and
did it in pure Bro, which is easier (or will be once I actually add
the code for the formatting). 

>  That might even make more sense as the primary approach.  I guess it
>  would be like a prerotation filter or something?

Maybe, but not sure. That may be better to postpone until we have an
better idea how it might look like.


Ok, I'll finish this up then. Would like to get that into the beta as
it would be step backwards from 1.5 otherwise. Are you ok turning it
on by default?


One more thing that bothers me a bit: by default, there are no
alarms... Everything just goes into notice.log.  To get any mails, one
needs to configure things appropiately, which is hard at the beginning
because one won't have a good idea what can be generated.

I'm sure you have thought about this: what's the reason for not using
ACTION_ALARM as the default action, now that there aren't actually
that many notices generated anymore?

Robin


-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From seth at icir.org  Wed Oct 26 13:30:03 2011
From: seth at icir.org (Seth Hall)
Date: Wed, 26 Oct 2011 16:30:03 -0400
Subject: [Bro-Dev] Notices and alarms (Re: [Bro-Commits] [git/bro]
	topic/robin/pp-alarms: A new notice script that pretty-prints
	alarms in the summary email. (73d5643))
In-Reply-To: <20111026182027.GN40905@icir.org>
References: <201110261741.p9QHfEjP030476@bro-ids.icir.org>
	<20111026174343.GM40905@icir.org>
	<666FC09F-9583-421C-9693-1810805A2AD2@icir.org>
	<20111026182027.GN40905@icir.org>
Message-ID: <991062F4-BC3E-4114-99DA-28BF1568946C@icir.org>


On Oct 26, 2011, at 2:20 PM, Robin Sommer wrote:

> Ok, I'll finish this up then. Would like to get that into the beta as
> it would be step backwards from 1.5 otherwise. Are you ok turning it
> on by default?

Sure, it seems reasonable to me.

> One more thing that bothers me a bit: by default, there are no
> alarms... Everything just goes into notice.log.  To get any mails, one
> needs to configure things appropiately, which is hard at the beginning
> because one won't have a good idea what can be generated.

Eventually (maybe before the final?) I'd like to have a page in the auto-generated docs with all of the Notice::Type values and the associated documentation so users could browse that single location and see what notices they have available for emailing, alarming, etc along with the script that needs to be loaded in order for that notice to be raised.  

> I'm sure you have thought about this: what's the reason for not using
> ACTION_ALARM as the default action, now that there aren't actually
> that many notices generated anymore?


In my mind that violates the "Bro is policy neutral" rule (or whatever I've seen in slides).  Bro generates the notices and it's a site local decision if those notices are applicable to their environment.  Making ACTION_ALARM the default would essentially be saying that all notices are applicable to their environment.  It always sort of bothered me that anything was turned into an alarm out of the box after hearing about that.  That said, I think that shipping with some notices added to the Notice::emailed_types and Notice::alarmed_type variables would in local.bro would be cool since I see local.bro as our chance to give a suggested configuration.

  .Seth


--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From robin at icir.org  Wed Oct 26 13:41:31 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 26 Oct 2011 13:41:31 -0700
Subject: [Bro-Dev] Notices and alarms (Re: [Bro-Commits]
	[git/bro]	topic/robin/pp-alarms: A new notice script that
	pretty-prints	alarms in the summary email. (73d5643))
In-Reply-To: <991062F4-BC3E-4114-99DA-28BF1568946C@icir.org>
References: <201110261741.p9QHfEjP030476@bro-ids.icir.org>
	<20111026174343.GM40905@icir.org>
	<666FC09F-9583-421C-9693-1810805A2AD2@icir.org>
	<20111026182027.GN40905@icir.org>
	<991062F4-BC3E-4114-99DA-28BF1568946C@icir.org>
Message-ID: <20111026204131.GB43260@icir.org>


On Wed, Oct 26, 2011 at 16:30 -0400, you wrote:

> Eventually (maybe before the final?) I'd like to have a page in the
> auto-generated docs with all of the Notice::Type values and the

That would be nice. Can Broxygen already extract that information?

> > I'm sure you have thought about this: what's the reason for not using
> > ACTION_ALARM as the default action, now that there aren't actually
> > that many notices generated anymore?

> In my mind that violates the "Bro is policy neutral" rule (or whatever
> I've seen in slides).

That argument usually applies to the event engine. For scripts,
considering policy is hard to avoid (and as you say, by default, Bro
1.x did turn everything into an alarm). 

>  That said, I think that shipping with some notices added to the
>  Notice::emailed_types and Notice::alarmed_type variables would in
>  local.bro would be cool since I see local.bro as our chance to give a
>  suggested configuration.

Or we could provide an option "default_alarm" or so that enable
ACTION_ALARM as the default, and then put that option into local.bro
for everybody to tune. The question would then be what to default to.
I'm leaning towards alarming by default, in the spirit of showing off
what Bro can do (and showing that *is* doing something).

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From jsiwek at ncsa.illinois.edu  Wed Oct 26 13:51:27 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Wed, 26 Oct 2011 15:51:27 -0500
Subject: [Bro-Dev] Notices and alarms (Re: [Bro-Commits]
	[git/bro]	topic/robin/pp-alarms: A new notice script that
	pretty-prints	alarms in the summary email. (73d5643))
In-Reply-To: <20111026204131.GB43260@icir.org>
References: <201110261741.p9QHfEjP030476@bro-ids.icir.org>
	<20111026174343.GM40905@icir.org>
	<666FC09F-9583-421C-9693-1810805A2AD2@icir.org>
	<20111026182027.GN40905@icir.org>
	<991062F4-BC3E-4114-99DA-28BF1568946C@icir.org>
	<20111026204131.GB43260@icir.org>
Message-ID: <A7882B26-0C4E-4776-8044-63682223D3B9@ncsa.illinois.edu>


>> Eventually (maybe before the final?) I'd like to have a page in the
>> auto-generated docs with all of the Notice::Type values and the
> 
> That would be nice. Can Broxygen already extract that information?


Yeah, it already has somewhat, but finding it in the sphinx-generated docs might not be so obvious right now.

All that info can be found by looking in the index for everything under "Notice::Type (enum values)",
and then the link leads to the documentation for it and the bro script in which it's declared.

- Jon

From seth at icir.org  Wed Oct 26 14:02:30 2011
From: seth at icir.org (Seth Hall)
Date: Wed, 26 Oct 2011 17:02:30 -0400
Subject: [Bro-Dev] Notices and alarms (Re: [Bro-Commits] [git/bro]
	topic/robin/pp-alarms: A new notice script that pretty-prints
	alarms in the summary email. (73d5643))
In-Reply-To: <20111026204131.GB43260@icir.org>
References: <201110261741.p9QHfEjP030476@bro-ids.icir.org>
	<20111026174343.GM40905@icir.org>
	<666FC09F-9583-421C-9693-1810805A2AD2@icir.org>
	<20111026182027.GN40905@icir.org>
	<991062F4-BC3E-4114-99DA-28BF1568946C@icir.org>
	<20111026204131.GB43260@icir.org>
Message-ID: <8ED58F54-A17E-49FD-8FCB-CA03E671FEE5@icir.org>


On Oct 26, 2011, at 4:41 PM, Robin Sommer wrote:

> That argument usually applies to the event engine. For scripts,
> considering policy is hard to avoid (and as you say, by default, Bro
> 1.x did turn everything into an alarm). 

I don't see any reason to not apply it all the way up to the scripts. :)  

I think it just comes down to the correct mix of documentation and shortcuts.

> Or we could provide an option "default_alarm" or so that enable
> ACTION_ALARM as the default, and then put that option into local.bro
> for everybody to tune. The question would then be what to default to.
> I'm leaning towards alarming by default, in the spirit of showing off
> what Bro can do (and showing that *is* doing something).


Yuck, I *really* don't want to have that option.  We could actually just implement that as an item in Notice::policy anyway.  It's a big mental change to adjust initially to multiple actions being applied to a notice.  You think that people would want to start receiving all of their notices in email prior to getting a chance to look through the notices to see what they want?

Personally, I can see adding the following to local.bro to cover the example you are trying to accomplish...

redef Notice::alarmed_types += { 
	HTTP::SQL_Injection_Attack_Against,
	HTTP::SQL_Injection_Attacker,
	HTTP::Malware_Hash_Registry_Match,
	HTTP::Incorrect_File_Type,
	FTP::Site_Exec_Success,
	SSH::Interesting_Hostname_Login,
	SSH::Password_Guessing,
	SSH::Watched_Country_Login,
	Software::Vulnerable_Version,
};

Well crap, I guess that actually includes many of the notices that we're shipping right now (there are some higher volume ones filtered out still).  Maybe this?

# Uncomment the following line to begin receiving hourly emails containing all of your notices.
#redef Notice::policy += { [$action = Notice::ACTION_ALARM, $priority = 0] };

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Wed Oct 26 14:03:30 2011
From: seth at icir.org (Seth Hall)
Date: Wed, 26 Oct 2011 17:03:30 -0400
Subject: [Bro-Dev] Notices and alarms (Re: [Bro-Commits]
	[git/bro]	topic/robin/pp-alarms: A new notice script that
	pretty-prints	alarms in the summary email. (73d5643))
In-Reply-To: <A7882B26-0C4E-4776-8044-63682223D3B9@ncsa.illinois.edu>
References: <201110261741.p9QHfEjP030476@bro-ids.icir.org>
	<20111026174343.GM40905@icir.org>
	<666FC09F-9583-421C-9693-1810805A2AD2@icir.org>
	<20111026182027.GN40905@icir.org>
	<991062F4-BC3E-4114-99DA-28BF1568946C@icir.org>
	<20111026204131.GB43260@icir.org>
	<A7882B26-0C4E-4776-8044-63682223D3B9@ncsa.illinois.edu>
Message-ID: <15BE8CB9-4C91-4F47-ADF8-3F65C9431DB7@icir.org>


On Oct 26, 2011, at 4:51 PM, Jonathan Siwek wrote:

> All that info can be found by looking in the index for everything under "Notice::Type (enum values)",
> and then the link leads to the documentation for it and the bro script in which it's declared.


Way too much work. :P  Would it be a lot of effort to pull all of that into a single page? (i know i already asked you this, but i don't remember exactly what you said)

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From jsiwek at ncsa.illinois.edu  Wed Oct 26 14:15:46 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Wed, 26 Oct 2011 16:15:46 -0500
Subject: [Bro-Dev] Notices and alarms (Re: [Bro-Commits]
	[git/bro]	topic/robin/pp-alarms: A new notice script that
	pretty-prints	alarms in the summary email. (73d5643))
In-Reply-To: <15BE8CB9-4C91-4F47-ADF8-3F65C9431DB7@icir.org>
References: <201110261741.p9QHfEjP030476@bro-ids.icir.org>
	<20111026174343.GM40905@icir.org>
	<666FC09F-9583-421C-9693-1810805A2AD2@icir.org>
	<20111026182027.GN40905@icir.org>
	<991062F4-BC3E-4114-99DA-28BF1568946C@icir.org>
	<20111026204131.GB43260@icir.org>
	<A7882B26-0C4E-4776-8044-63682223D3B9@ncsa.illinois.edu>
	<15BE8CB9-4C91-4F47-ADF8-3F65C9431DB7@icir.org>
Message-ID: <97AE93F0-08FA-4978-BD4A-9DC7C62A068B@ncsa.illinois.edu>


> Way too much work. :P  Would it be a lot of effort to pull all of that into a single page? (i know i already asked you this, but i don't remember exactly what you said)

Probably not difficult, the effort is just in looking into the best way to do that (which I'm not sure of right now, i.e. dunno if I want to do it inside Sphinx or inside Bro).

- Jon

From robin at icir.org  Wed Oct 26 14:26:58 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 26 Oct 2011 14:26:58 -0700
Subject: [Bro-Dev] Notices and alarms (Re: [Bro-Commits]
	[git/bro]	topic/robin/pp-alarms: A new notice script that
	pretty-prints	alarms in the summary email. (73d5643))
In-Reply-To: <8ED58F54-A17E-49FD-8FCB-CA03E671FEE5@icir.org>
References: <201110261741.p9QHfEjP030476@bro-ids.icir.org>
	<20111026174343.GM40905@icir.org>
	<666FC09F-9583-421C-9693-1810805A2AD2@icir.org>
	<20111026182027.GN40905@icir.org>
	<991062F4-BC3E-4114-99DA-28BF1568946C@icir.org>
	<20111026204131.GB43260@icir.org>
	<8ED58F54-A17E-49FD-8FCB-CA03E671FEE5@icir.org>
Message-ID: <20111026212658.GD43260@icir.org>


On Wed, Oct 26, 2011 at 17:02 -0400, you wrote:

> You think that people would want to start receiving all of their
> notices in email prior to getting a chance to look through the notices
> to see what they want?

Yes, I do actually. It's the push vs. pull model; seeing what I want
and adjusting the config takes effort. Also, at least for smaller
networks, it's actually not a problem; and I'm hoping we'll be getting
plenty of those in the future as well.

> # Uncomment the following line to begin receiving hourly emails containing all of your notices.
> #redef Notice::policy += { [$action = Notice::ACTION_ALARM, $priority = 0] };

I like that! It's short and even better than the option I suggested
because one can start tweaking right there. And I'm fine leaving it
commented out. I'll add that.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From robin at icir.org  Wed Oct 26 15:30:33 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 26 Oct 2011 15:30:33 -0700
Subject: [Bro-Dev] problem with test (Re: [Bro-Commits] [git/bro] master:
	Merge	remote-tracking branch 'origin/topic/robin/pp-alarms'
	(ec2a8d7))
In-Reply-To: <201110262228.p9QMSadN031958@bro-ids.icir.org>
References: <201110262228.p9QMSadN031958@bro-ids.icir.org>
Message-ID: <20111026223033.GE43260@icir.org>

With my recent alarm mail changes, I now see one test failing with
master and I have no clue why. Any idea?

Robin



scripts.base.frameworks.control.configuration_update ...  failed
  % 'btest-diff controllee/.stdout' failed unexpectedly (exit code 1)
  % cat .diag
  == File ===============================
  ORIGINAL VALUE (this should be printed out first)
  ORIGINAL VALUE (this should be printed out first)
  == Diff ===============================
  --- /tmp/test-diff.22007.controllee..stdout.baseline.tmp      2011-10-26 22:27:52.570262720 +0000
  +++ /tmp/test-diff.22007.controllee..stdout.tmp       2011-10-26 22:27:52.580263105 +0000
  @@ -1,2 +1,2 @@
  ORIGINAL VALUE (this should be printed out first)
  -NEW VALUE (this should be printed out second)
  +ORIGINAL VALUE (this should be printed out first)
  =======================================

  % cat .stderr
  <<< [21952] BROPATH=.:/home/robin/bro/master/scripts:/home/robin/bro/master/scripts/policy:/home/robin/bro/master/scripts/site:/home/robin/bro/master/build/src:.. bro /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.control.configuration_update/configuration_update.bro frameworks/control/controllee Communication::listen_port=65531/tcp
  >>>
  <<< [21964] BROPATH=.:/home/robin/bro/master/scripts:/home/robin/bro/master/scripts/policy:/home/robin/bro/master/scripts/site:/home/robin/bro/master/build/src:.. bro /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.control.configuration_update/configuration_update.bro test-redef frameworks/control/controller Control::host=127.0.0.1 Control::host_port=65531/tcp Control::cmd=configuration_update
  error in <params>, line 1: fatal error, shutting down communication: Resource temporarily unavailable [11]
  >>>
  <<< [21976] BROPATH=.:/home/robin/bro/master/scripts:/home/robin/bro/master/scripts/policy:/home/robin/bro/master/scripts/site:/home/robin/bro/master/build/src:.. bro /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.control.configuration_update/configuration_update.bro frameworks/control/controller Control::host=127.0.0.1 Control::host_port=65531/tcp Control::cmd=shutdown
  >>>

1 test failed


-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From robin at icir.org  Wed Oct 26 21:24:58 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 26 Oct 2011 21:24:58 -0700
Subject: [Bro-Dev] snaplen and drops
Message-ID: <20111027042458.GA87273@icir.org>

On a reasonable fast Linux box seeing (currently) <10M/bps, I'm
getting lots packet drops with current master, even though CPU is very
low. I did the usual sysctl tuning, but that didn't help. Then I
reduced the snaplen (which now defaults to 65K) down to 8K, and the
drops disappeared.

That seems is quite an extreme effect of the new default value. Should
we reconsider and (1) use a smaller default, and/or (2) make the
snaplen accesible from the scripting layer (right now, there's only
-s; which doens't work well with BroControl).

Is there other tuning to get around the problem (with standard kernel,
not PF_RING etc.)? 

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From seth at icir.org  Thu Oct 27 05:15:22 2011
From: seth at icir.org (Seth Hall)
Date: Thu, 27 Oct 2011 08:15:22 -0400
Subject: [Bro-Dev] problem with test (Re: [Bro-Commits] [git/bro]
	master: Merge	remote-tracking branch
	'origin/topic/robin/pp-alarms' (ec2a8d7))
In-Reply-To: <20111026223033.GE43260@icir.org>
References: <201110262228.p9QMSadN031958@bro-ids.icir.org>
	<20111026223033.GE43260@icir.org>
Message-ID: <622194BE-FA6F-4221-B2CC-7E6DF714635B@icir.org>


On Oct 26, 2011, at 6:30 PM, Robin Sommer wrote:

> With my recent alarm mail changes, I now see one test failing with
> master and I have no clue why. Any idea?

> error in <params>, line 1: fatal error, shutting down communication: Resource temporarily unavailable [11]

It looks like you probably have a left over Bro process hanging around.  Bro that btest started couldn't open it's port.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Thu Oct 27 05:30:10 2011
From: seth at icir.org (Seth Hall)
Date: Thu, 27 Oct 2011 08:30:10 -0400
Subject: [Bro-Dev] problem with test (Re: [Bro-Commits] [git/bro]
	master: Merge	remote-tracking branch
	'origin/topic/robin/pp-alarms' (ec2a8d7))
In-Reply-To: <622194BE-FA6F-4221-B2CC-7E6DF714635B@icir.org>
References: <201110262228.p9QMSadN031958@bro-ids.icir.org>
	<20111026223033.GE43260@icir.org>
	<622194BE-FA6F-4221-B2CC-7E6DF714635B@icir.org>
Message-ID: <2CB1E2F5-3BAF-4A17-BCB8-E67A361F7001@icir.org>


On Oct 27, 2011, at 8:15 AM, Seth Hall wrote:

> 
> On Oct 26, 2011, at 6:30 PM, Robin Sommer wrote:
>> error in <params>, line 1: fatal error, shutting down communication: Resource temporarily unavailable [11]
> 
> It looks like you probably have a left over Bro process hanging around.  Bro that btest started couldn't open it's port.


Nevermind, I'm seeing a different error for this test.  I'll take a look right now.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From robin at icir.org  Thu Oct 27 07:14:41 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 27 Oct 2011 07:14:41 -0700
Subject: [Bro-Dev] problem with test (Re: [Bro-Commits]
	[git/bro]	master: Merge remote-tracking branch
	'origin/topic/robin/pp-alarms'	(ec2a8d7))
In-Reply-To: <2CB1E2F5-3BAF-4A17-BCB8-E67A361F7001@icir.org>
References: <201110262228.p9QMSadN031958@bro-ids.icir.org>
	<20111026223033.GE43260@icir.org>
	<622194BE-FA6F-4221-B2CC-7E6DF714635B@icir.org>
	<2CB1E2F5-3BAF-4A17-BCB8-E67A361F7001@icir.org>
Message-ID: <20111027141441.GA36458@icir.org>


On Thu, Oct 27, 2011 at 08:30 -0400, you wrote:

> Nevermind, I'm seeing a different error for this test.  I'll take a look right now.

Thanks, once we have that fixed, I'll roll up a test package.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From mcholste at gmail.com  Thu Oct 27 09:15:31 2011
From: mcholste at gmail.com (Martin Holste)
Date: Thu, 27 Oct 2011 11:15:31 -0500
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <20111027042458.GA87273@icir.org>
References: <20111027042458.GA87273@icir.org>
Message-ID: <CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>

That's very weird.  It's abnormal to get packets > 1514, right?  Are
you monitoring a link with a lot of jumbo packets?  Something is
wrong, as that small bandwidth shouldn't matter no matter what the
packet sizes are anyway.

On Wed, Oct 26, 2011 at 11:24 PM, Robin Sommer <robin at icir.org> wrote:
> On a reasonable fast Linux box seeing (currently) <10M/bps, I'm
> getting lots packet drops with current master, even though CPU is very
> low. I did the usual sysctl tuning, but that didn't help. Then I
> reduced the snaplen (which now defaults to 65K) down to 8K, and the
> drops disappeared.
>
> That seems is quite an extreme effect of the new default value. Should
> we reconsider and (1) use a smaller default, and/or (2) make the
> snaplen accesible from the scripting layer (right now, there's only
> -s; which doens't work well with BroControl).
>
> Is there other tuning to get around the problem (with standard kernel,
> not PF_RING etc.)?
>
> Robin
>
> --
> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
> ICSI/LBNL ? ?* Fax ? +1 (510) 666-2956 * ? www.icir.org
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>


From jones at tacc.utexas.edu  Thu Oct 27 09:19:29 2011
From: jones at tacc.utexas.edu (William Jones)
Date: Thu, 27 Oct 2011 16:19:29 +0000
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <20111027042458.GA87273@icir.org>
References: <20111027042458.GA87273@icir.org>
Message-ID: <D0BD4A5C03717E40AC104854AF37B8C215FCE5@EXMBX01.austin.utexas.edu>

On linux there set the two parameters that need to be tuned for bro:

net.core.rmem_max
net.core.rmem_default

This controls the buffer used by the raw socket interface. I would start at 20 Megabytes if the interface is a 1 GigE Ethernet 200 Megabyte if it a 10 GigE Ethernet.  Keep increasing it as till it longer has an effect on the drop rate.  I use "tcpdump -I <interface> -w <file>" to check the drop rate.   Let tcpdump run about 10 to 20 seconds and hit ctrl-c.   Tcpdump will report the packets dropped by the system and the total packets recived.  

Increase net.core.netdev_max_backlog to 100000 

There are some  1 GigE Ethernet nicks that just can't be tuned.    



-----Original Message-----
From: bro-dev-bounces at bro-ids.org [mailto:bro-dev-bounces at bro-ids.org] On Behalf Of Robin Sommer
Sent: Wednesday, October 26, 2011 11:25 PM
To: bro-dev at bro-ids.org
Subject: [Bro-Dev] snaplen and drops

On a reasonable fast Linux box seeing (currently) <10M/bps, I'm
getting lots packet drops with current master, even though CPU is very
low. I did the usual sysctl tuning, but that didn't help. Then I
reduced the snaplen (which now defaults to 65K) down to 8K, and the
drops disappeared.

That seems is quite an extreme effect of the new default value. Should
we reconsider and (1) use a smaller default, and/or (2) make the
snaplen accesible from the scripting layer (right now, there's only
-s; which doens't work well with BroControl).

Is there other tuning to get around the problem (with standard kernel,
not PF_RING etc.)? 

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
_______________________________________________
bro-dev mailing list
bro-dev at bro-ids.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev


From seth at icir.org  Thu Oct 27 10:03:13 2011
From: seth at icir.org (Seth Hall)
Date: Thu, 27 Oct 2011 13:03:13 -0400
Subject: [Bro-Dev] paper on protocol identification
Message-ID: <2B6F309B-B052-44DA-97EB-DDF48E562DBA@icir.org>

Seems weird that it doesn't reference the DPD paper..

	http://unina.academia.edu/AntonioPescape/Papers/808031/PortLoad_taking_the_best_of_two_worlds_in_traffic_classification

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Thu Oct 27 10:53:49 2011
From: seth at icir.org (Seth Hall)
Date: Thu, 27 Oct 2011 13:53:49 -0400
Subject: [Bro-Dev] problem with test (Re: [Bro-Commits] [git/bro]
	master: Merge	remote-tracking branch
	'origin/topic/robin/pp-alarms' (ec2a8d7))
In-Reply-To: <20111026223033.GE43260@icir.org>
References: <201110262228.p9QMSadN031958@bro-ids.icir.org>
	<20111026223033.GE43260@icir.org>
Message-ID: <B35142BF-F8A6-4E81-9976-37D5261236A0@icir.org>


On Oct 26, 2011, at 6:30 PM, Robin Sommer wrote:

> With my recent alarm mail changes, I now see one test failing with
> master and I have no clue why. Any idea?


I can't get it to work.  Is there some reason the send_id BiF could have broken?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Thu Oct 27 10:56:11 2011
From: seth at icir.org (Seth Hall)
Date: Thu, 27 Oct 2011 13:56:11 -0400
Subject: [Bro-Dev] problem with test (Re: [Bro-Commits] [git/bro]
	master: Merge	remote-tracking branch
	'origin/topic/robin/pp-alarms' (ec2a8d7))
In-Reply-To: <B35142BF-F8A6-4E81-9976-37D5261236A0@icir.org>
References: <201110262228.p9QMSadN031958@bro-ids.icir.org>
	<20111026223033.GE43260@icir.org>
	<B35142BF-F8A6-4E81-9976-37D5261236A0@icir.org>
Message-ID: <257EC0E3-82C0-425A-BF51-3B43F7D2777B@icir.org>


On Oct 27, 2011, at 1:53 PM, Seth Hall wrote:

> I can't get it to work.  Is there some reason the send_id BiF could have broken?


One more thing, the communication.log file for the controller has these error messages:

  1319738043.456203	bro	parent	-	-	-	error	[#1/127.0.0.1:65531] serializing ID: failed
  1319738043.456203	bro	parent	-	-	-	error	fatal error, shutting down communication: Resource temporarily unavailable [35]
  1319738043.456203	bro	parent	-	-	-	error	fatal error, shutting down communication: Operation timed out [60]

It definitely looks like there is something wrong with the send_id BiF.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From robin at icir.org  Thu Oct 27 10:59:38 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 27 Oct 2011 10:59:38 -0700
Subject: [Bro-Dev] problem with test (Re: [Bro-Commits]
	[git/bro]	master: Merge remote-tracking branch
	'origin/topic/robin/pp-alarms'	(ec2a8d7))
In-Reply-To: <257EC0E3-82C0-425A-BF51-3B43F7D2777B@icir.org>
References: <201110262228.p9QMSadN031958@bro-ids.icir.org>
	<20111026223033.GE43260@icir.org>
	<B35142BF-F8A6-4E81-9976-37D5261236A0@icir.org>
	<257EC0E3-82C0-425A-BF51-3B43F7D2777B@icir.org>
Message-ID: <20111027175938.GO36458@icir.org>


On Thu, Oct 27, 2011 at 13:56 -0400, you wrote:

> It definitely looks like there is something wrong with the send_id BiF.

I didn't touch anything even remotely related to that I believe. But
perhaps some change triggered something we haven't before. I'll dig
into it.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From robin at icir.org  Thu Oct 27 11:02:43 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 27 Oct 2011 11:02:43 -0700
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
References: <20111027042458.GA87273@icir.org>
	<CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
Message-ID: <20111027180243.GP36458@icir.org>


On Thu, Oct 27, 2011 at 11:15 -0500, you wrote:

> That's very weird.  It's abnormal to get packets > 1514, right?  Are
> you monitoring a link with a lot of jumbo packets?

Not at all, but iirc, the kernel reserves spaces for packets of size
snaplen, which means that with larger snaplen, less will fit into its
buffers.

>  Something is wrong, as that small bandwidth shouldn't matter no
>  matter what the packet sizes are anyway.

Yeah, I'm thinking so too. Something's odd going on, need to look into
it more closely.

As long as other's aren't seeing similar problems with the default 65K
snaplen, I'm fine.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From robin at icir.org  Thu Oct 27 11:03:45 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 27 Oct 2011 11:03:45 -0700
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <D0BD4A5C03717E40AC104854AF37B8C215FCE5@EXMBX01.austin.utexas.edu>
References: <20111027042458.GA87273@icir.org>
	<D0BD4A5C03717E40AC104854AF37B8C215FCE5@EXMBX01.austin.utexas.edu>
Message-ID: <20111027180345.GQ36458@icir.org>


On Thu, Oct 27, 2011 at 16:19 +0000, you wrote:

> net.core.rmem_max
> net.core.rmem_default
> Increase net.core.netdev_max_backlog to 100000 

Yeah, I had tuned these already, didn't help. But I'll take another
look.

Thanks,

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From robin at icir.org  Thu Oct 27 11:31:12 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 27 Oct 2011 11:31:12 -0700
Subject: [Bro-Dev] problem with test (Re: [Bro-Commits]
	[git/bro]	master: Merge remote-tracking branch
	'origin/topic/robin/pp-alarms'	(ec2a8d7))
In-Reply-To: <20111027175938.GO36458@icir.org>
References: <201110262228.p9QMSadN031958@bro-ids.icir.org>
	<20111026223033.GE43260@icir.org>
	<B35142BF-F8A6-4E81-9976-37D5261236A0@icir.org>
	<257EC0E3-82C0-425A-BF51-3B43F7D2777B@icir.org>
	<20111027175938.GO36458@icir.org>
Message-ID: <20111027183112.GR36458@icir.org>


On Thu, Oct 27, 2011 at 10:59 -0700, I wrote:

> perhaps some change triggered something we haven't before. I'll dig
> into it.

Found it: it's the redefable global function. Something's in there
doesn't seem to (un-)serialize well. But sending complete functions
around isn't really supported anyway, so I think we can just stop
trying to update functions on the fly.

Seth, is there anything that relies on that? (It would still send the
notice_policy because the top-level ID for that is a table; in
principle, the function in there could cause similar trouble, but that
seems unlikely). 

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From seth at icir.org  Thu Oct 27 11:37:40 2011
From: seth at icir.org (Seth Hall)
Date: Thu, 27 Oct 2011 14:37:40 -0400
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <20111027180345.GQ36458@icir.org>
References: <20111027042458.GA87273@icir.org>
	<D0BD4A5C03717E40AC104854AF37B8C215FCE5@EXMBX01.austin.utexas.edu>
	<20111027180345.GQ36458@icir.org>
Message-ID: <BF959A2C-CCFD-4819-B820-B8F5C3AEF5F3@icir.org>


On Oct 27, 2011, at 2:03 PM, Robin Sommer wrote:

> Yeah, I had tuned these already, didn't help. But I'll take another
> look.


We were seeing high loss with relatively low traffic but that was with myricom cards.  I suppose it could have been due to the snap length though, I never tried changing it.
 
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Thu Oct 27 11:41:19 2011
From: seth at icir.org (Seth Hall)
Date: Thu, 27 Oct 2011 14:41:19 -0400
Subject: [Bro-Dev] problem with test (Re: [Bro-Commits] [git/bro]
	master: Merge remote-tracking branch
	'origin/topic/robin/pp-alarms' (ec2a8d7))
In-Reply-To: <20111027183112.GR36458@icir.org>
References: <201110262228.p9QMSadN031958@bro-ids.icir.org>
	<20111026223033.GE43260@icir.org>
	<B35142BF-F8A6-4E81-9976-37D5261236A0@icir.org>
	<257EC0E3-82C0-425A-BF51-3B43F7D2777B@icir.org>
	<20111027175938.GO36458@icir.org> <20111027183112.GR36458@icir.org>
Message-ID: <D9841EFB-D686-437E-8251-7AAC371BBCD2@icir.org>


On Oct 27, 2011, at 2:31 PM, Robin Sommer wrote:

> Found it: it's the redefable global function.

Hm, the control framework only tries to update consts.  I'm surprised it's even trying to send it unless functions are actually const internally even though they're declared global.

> Seth, is there anything that relies on that? (It would still send the
> notice_policy because the top-level ID for that is a table; in
> principle, the function in there could cause similar trouble, but that
> seems unlikely). 


Not that I can think of.  I've had the notion of sending functions in mind for a while and I think it would be great to be able to do that (primarily at first for live updates to the notice policy at first as you indicated) but for now we can just cut out functions.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From robin at icir.org  Thu Oct 27 11:43:47 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 27 Oct 2011 11:43:47 -0700
Subject: [Bro-Dev] problem with test (Re: [Bro-Commits]
	[git/bro]	master: Merge remote-tracking branch
	'origin/topic/robin/pp-alarms'	(ec2a8d7))
In-Reply-To: <D9841EFB-D686-437E-8251-7AAC371BBCD2@icir.org>
References: <201110262228.p9QMSadN031958@bro-ids.icir.org>
	<20111026223033.GE43260@icir.org>
	<B35142BF-F8A6-4E81-9976-37D5261236A0@icir.org>
	<257EC0E3-82C0-425A-BF51-3B43F7D2777B@icir.org>
	<20111027175938.GO36458@icir.org> <20111027183112.GR36458@icir.org>
	<D9841EFB-D686-437E-8251-7AAC371BBCD2@icir.org>
Message-ID: <20111027184346.GS36458@icir.org>


On Thu, Oct 27, 2011 at 14:41 -0400, you wrote:

> Hm, the control framework only tries to update consts.  I'm surprised
> it's even trying to send it unless functions are actually const
> internally even though they're declared global.

Yes, I think that's the case. The global_ids() call indicates it's
const (and technically that's right in the sense that the global does
have a constant value because there's no way to change it).

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From vallentin at icir.org  Thu Oct 27 12:37:45 2011
From: vallentin at icir.org (Matthias Vallentin)
Date: Thu, 27 Oct 2011 12:37:45 -0700
Subject: [Bro-Dev] bro-cut
In-Reply-To: <20111026022547.GB8301@icir.org>
References: <20111026022547.GB8301@icir.org>
Message-ID: <1319743958-sup-9574@samurai.local>

> I'm thinking bro-cut is something worth installing by default into
> $prefix/bin/, even though it's living in bro-aux.

Yup, that tool is really handy and I am using it right away for the
workshop exercises. A few comments:

    - Neither -h nor --help seem to be a valid switch, although they
      "correctly" display the usage.

    - The usage currently ends with

        For time conversion, the def

      Something seems to miss.

    - Would you mind adding a way to specify an output separator (OFS in
      awk)? For example, when concatenating URLs from http.log, I would
      like to do something along the lines of:

        bro-cut -F '' host uri < http.log

      to get output in this form:

        mt1.google.com/vt/lyrs=m at 162254822&hl=en&x=657&y=1581&z=12&s=
        mt0.google.com/vt/lyrs=m at 162249697&hl=en&x=656&y=1581&z=12&s=Galil

    Matthias

From robin at icir.org  Thu Oct 27 12:55:12 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 27 Oct 2011 12:55:12 -0700
Subject: [Bro-Dev] bro-cut
In-Reply-To: <1319743958-sup-9574@samurai.local>
References: <20111026022547.GB8301@icir.org>
	<1319743958-sup-9574@samurai.local>
Message-ID: <20111027195512.GC64398@icir.org>


On Thu, Oct 27, 2011 at 12:37 -0700, you wrote:

>     - Neither -h nor --help seem to be a valid switch, although they
>       "correctly" display the usage.
> 
>     - The usage currently ends with
> 
>         For time conversion, the def
> 
>       Something seems to miss.

Thanks, I've fixed those. I also added a check that if no columns are
given, the help message gets printed as well (rather than printing
lots of empty lines).

>     - Would you mind adding a way to specify an output separator (OFS in
>       awk)?

I can see that, but that's something for later. Please file a ticket.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Thu Oct 27 12:59:19 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 27 Oct 2011 19:59:19 -0000
Subject: [Bro-Dev] #648: SpoolDir setting isn't used everywhere
Message-ID: <046.ea9ac6eac3f6ed983568e802720fbd53@tracker.bro-ids.org>

#648: SpoolDir setting isn't used everywhere
------------------------+--------------------
 Reporter:  seth        |      Owner:
     Type:  Problem     |     Status:  new
 Priority:  Normal      |  Milestone:  Bro1.6
Component:  BroControl  |    Version:
 Keywords:              |
------------------------+--------------------
 If you change the SpoolDir setting in broctl.conf from the default, there
 is something that isn't picking it up.  I don't recall which it is, but
 either when you do "broctl install" it installs to the default (even if
 the setting is changed) or it sets BROPATH to the default value.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/648>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Oct 27 13:14:20 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 27 Oct 2011 20:14:20 -0000
Subject: [Bro-Dev] #649: Output field separator to bro-cut
Message-ID: <050.7df31ebdf5c434589e1f80977417c79d@tracker.bro-ids.org>

#649: Output field separator to bro-cut
-----------------------------+------------------------
 Reporter:  vallenti         |      Owner:  robin
     Type:  Feature Request  |     Status:  new
 Priority:  Normal           |  Milestone:  Bro1.6
Component:  Bro              |    Version:  git/master
 Keywords:  bro-cut          |
-----------------------------+------------------------
 It would be nice to have a way to specify an output separator (like OFS in
 awk). This comes in handy, for example, when concatenating URLs from
 `http.log`:
 {{{
 bro-cut -F '' host uri < http.log
 }}}
 The out would then be along the lines of:
 {{{
 host1/index.html
 host2/foo/bar/baz.html
 }}}

 A more verbose way of achieving the same output right now would be:
 {{{
 bro-cut host uri < http.log | awk '{ print $1$2 }'
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/649>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From vallentin at icir.org  Thu Oct 27 13:15:01 2011
From: vallentin at icir.org (Matthias Vallentin)
Date: Thu, 27 Oct 2011 13:15:01 -0700
Subject: [Bro-Dev] bro-cut
In-Reply-To: <20111027195512.GC64398@icir.org>
References: <20111026022547.GB8301@icir.org>
	<1319743958-sup-9574@samurai.local>
	<20111027195512.GC64398@icir.org>
Message-ID: <1319746477-sup-5420@samurai.local>

> I can see that, but that's something for later. Please file a ticket.

Okay, done (#649).

    Matthias

From bro at tracker.bro-ids.org  Thu Oct 27 13:20:00 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 27 Oct 2011 20:20:00 -0000
Subject: [Bro-Dev] #650: Finish Broxygen docs
Message-ID: <047.9abf8a0851b3b85991457973560c772f@tracker.bro-ids.org>

#650: Finish Broxygen docs
--------------------+--------------------
 Reporter:  robin   |      Owner:
     Type:  Task    |     Status:  new
 Priority:  Normal  |  Milestone:  Bro1.6
Component:  Bro     |    Version:
 Keywords:          |
--------------------+--------------------
 This task includes:

 - Polishing content  structure and optics.

 - Finish embedded script documentation.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/650>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From jones at tacc.utexas.edu  Thu Oct 27 13:30:28 2011
From: jones at tacc.utexas.edu (William Jones)
Date: Thu, 27 Oct 2011 20:30:28 +0000
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <20111027180243.GP36458@icir.org>
References: <20111027042458.GA87273@icir.org>
	<CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
	<20111027180243.GP36458@icir.org>
Message-ID: <D0BD4A5C03717E40AC104854AF37B8C2160221@EXMBX01.austin.utexas.edu>

The linux kernel will enable Large Receive Offload on nic's that support it.   This allows the nic to present multiple contiguous tcp packets as one large packet to the kernel.  I hqve seen tcpdump report packet size of 24K on interfaces with MTU sizes of 1500 bytes when LRO is on.  

The only was to turn of this feature in linux righ now is to turn route forwarding on  and reboot.. 

-----Original Message-----
From: bro-dev-bounces at bro-ids.org [mailto:bro-dev-bounces at bro-ids.org] On Behalf Of Robin Sommer
Sent: Thursday, October 27, 2011 1:03 PM
To: Martin Holste
Cc: bro-dev at bro-ids.org
Subject: Re: [Bro-Dev] snaplen and drops


On Thu, Oct 27, 2011 at 11:15 -0500, you wrote:

> That's very weird.  It's abnormal to get packets > 1514, right?  Are
> you monitoring a link with a lot of jumbo packets?

Not at all, but iirc, the kernel reserves spaces for packets of size
snaplen, which means that with larger snaplen, less will fit into its
buffers.

>  Something is wrong, as that small bandwidth shouldn't matter no
>  matter what the packet sizes are anyway.

Yeah, I'm thinking so too. Something's odd going on, need to look into
it more closely.

As long as other's aren't seeing similar problems with the default 65K
snaplen, I'm fine.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
_______________________________________________
bro-dev mailing list
bro-dev at bro-ids.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev


From braun at net.in.tum.de  Thu Oct 27 13:31:09 2011
From: braun at net.in.tum.de (Lothar Braun)
Date: Thu, 27 Oct 2011 22:31:09 +0200
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
References: <20111027042458.GA87273@icir.org>
	<CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
Message-ID: <535469B6-200D-4940-A2E5-704C89457850@net.in.tum.de>

Hi,

On Oct 27, 2011, at 6:15 PM, Martin Holste wrote:

> That's very weird.  It's abnormal to get packets > 1514, right?  Are
> you monitoring a link with a lot of jumbo packets?

On Linux you can observe packets > 1514 bytes, even if you monitor a link that does not carry a single jumbo frame. 

You can have large packets if your NIC supports RSC (Receive Side Coalescing). RSC is implemented in some network cards (e.g. Intel 10GE with a 82599 chipset), and reassembles subsequent TCP segments into larger packets in order to reduce the number of packets that need to be handled by the kernel.

Even if your network card does not implement RSC, you might also see large packets due to LRO/GRO (Large Receive Offload / Generic Receive Offload) done in software (more information: http://lwn.net/Articles/358910/).

However, this needs to be supported by your NIC driver and enabled via ethtool.

ethtool -k <dev>

will show you if you have LRO or GRO enabled. 

>> On a reasonable fast Linux box seeing (currently) <10M/bps, I'm
>> getting lots packet drops with current master, even though CPU is very
>> low. I did the usual sysctl tuning, but that didn't help. Then I
>> reduced the snaplen (which now defaults to 65K) down to 8K, and the
>> drops disappeared.

Which kernel and libpcap version do you use? 

Todays' Linux kernels support memory mapping for packet exchange between userland and kernel. Since version 1.0.0, libpcap uses this feature by default:

libpcap requests a two megabyte sized shared buffer (default size) from the kernel. The snaplen is passed to the kernel which will align captured packets according to the snaplen. The snaplen is therefore the only parameter that decides how many  packets fit into the buffer between kernel and application.

If you configure a snaplen of 64 KB, you will have space for only 32 packets in your buffer (2 MB / 64 KB).

You could try to make libpcap allocate a bigger buffer with pcap_set_buffer_size(). However, this must be called before pcap_activate(), which means that you cannot use pcap_open_live() but have to call pcap_create, pcap_set_snaplen, pcap_set_timeout, pcap_activate yourself..

Best regards,
  Lothar  

--
Lothar Braun
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universit?t M?nchen
Boltzmannstr. 3, 85748 Garching bei M?nchen, Germany
Phone:  +49 89 289-18010       Fax: +49 89 289-18033
E-mail: braun at net.in.tum.de 








From braun at net.in.tum.de  Thu Oct 27 13:41:18 2011
From: braun at net.in.tum.de (Lothar Braun)
Date: Thu, 27 Oct 2011 22:41:18 +0200
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <D0BD4A5C03717E40AC104854AF37B8C2160221@EXMBX01.austin.utexas.edu>
References: <20111027042458.GA87273@icir.org>
	<CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
	<20111027180243.GP36458@icir.org>
	<D0BD4A5C03717E40AC104854AF37B8C2160221@EXMBX01.austin.utexas.edu>
Message-ID: <5641E2E1-3247-4330-9CD2-2B74CAA98A01@net.in.tum.de>

Hi,

On Oct 27, 2011, at 10:30 PM, William Jones wrote:

> The linux kernel will enable Large Receive Offload on nic's that support it.   This allows the nic to present multiple contiguous tcp packets as one large packet to the kernel.  I hqve seen tcpdump report packet size of 24K on interfaces with MTU sizes of 1500 bytes when LRO is on.  
> 
> The only was to turn of this feature in linux righ now is to turn route forwarding on  and reboot.. 


You should be able to turn it off using ethtool:

ethtool -K <dev> lro off

Make sure you turn off GRO as well, if you don't want to have these large packets:

ethtool -K <dev> gro off

Best regards,
  Lothar

--
Lothar Braun
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universit?t M?nchen
Boltzmannstr. 3, 85748 Garching bei M?nchen, Germany
Phone:  +49 89 289-18010       Fax: +49 89 289-18033
E-mail: braun at net.in.tum.de 








From jones at tacc.utexas.edu  Thu Oct 27 14:21:25 2011
From: jones at tacc.utexas.edu (William Jones)
Date: Thu, 27 Oct 2011 21:21:25 +0000
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <5641E2E1-3247-4330-9CD2-2B74CAA98A01@net.in.tum.de>
References: <20111027042458.GA87273@icir.org>
	<CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
	<20111027180243.GP36458@icir.org>
	<D0BD4A5C03717E40AC104854AF37B8C2160221@EXMBX01.austin.utexas.edu>
	<5641E2E1-3247-4330-9CD2-2B74CAA98A01@net.in.tum.de>
Message-ID: <D0BD4A5C03717E40AC104854AF37B8C2160320@EXMBX01.austin.utexas.edu>

What version of linux are you running.   Redhat 5 doesn't seem to have a lro option just the gro option which does not seem to affect the LRO behavior.

Bill Jones

-----Original Message-----
From: Lothar Braun [mailto:braun at net.in.tum.de] 
Sent: Thursday, October 27, 2011 3:41 PM
To: William Jones
Cc: 'Robin Sommer'; Martin Holste; bro-dev at bro-ids.org
Subject: Re: [Bro-Dev] snaplen and drops

Hi,

On Oct 27, 2011, at 10:30 PM, William Jones wrote:

> The linux kernel will enable Large Receive Offload on nic's that support it.   This allows the nic to present multiple contiguous tcp packets as one large packet to the kernel.  I hqve seen tcpdump report packet size of 24K on interfaces with MTU sizes of 1500 bytes when LRO is on.  
> 
> The only was to turn of this feature in linux righ now is to turn route forwarding on  and reboot.. 


You should be able to turn it off using ethtool:

ethtool -K <dev> lro off

Make sure you turn off GRO as well, if you don't want to have these large packets:

ethtool -K <dev> gro off

Best regards,
  Lothar

--
Lothar Braun
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universit?t M?nchen
Boltzmannstr. 3, 85748 Garching bei M?nchen, Germany
Phone:  +49 89 289-18010       Fax: +49 89 289-18033
E-mail: braun at net.in.tum.de 








From bro at tracker.bro-ids.org  Thu Oct 27 14:29:27 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 27 Oct 2011 21:29:27 -0000
Subject: [Bro-Dev] #651: Use "referer" rather than "referrer" in http.log
Message-ID: <050.5c1aa53d04ba928c31b4ba2c89fdbc9e@tracker.bro-ids.org>

#651: Use "referer" rather than "referrer" in http.log
----------------------+------------------------
 Reporter:  vallenti  |      Owner:  seth
     Type:  Problem   |     Status:  new
 Priority:  Normal    |  Milestone:  Bro1.6
Component:  Bro       |    Version:  git/master
 Keywords:            |
----------------------+------------------------
 Folks that have worked with HTTP in that past might intuitively expect to
 extract the value of the `Referer` header by using the same field in the
 bro log, namely `referer`. The current naming scheme departs from this
 intuition and uses `referrer` as the field name in `http.log`. What about
 renaming this field to `referer`?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/651>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From braun at net.in.tum.de  Thu Oct 27 14:59:30 2011
From: braun at net.in.tum.de (Lothar Braun)
Date: Thu, 27 Oct 2011 23:59:30 +0200
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <D0BD4A5C03717E40AC104854AF37B8C2160320@EXMBX01.austin.utexas.edu>
References: <20111027042458.GA87273@icir.org>
	<CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
	<20111027180243.GP36458@icir.org>
	<D0BD4A5C03717E40AC104854AF37B8C2160221@EXMBX01.austin.utexas.edu>
	<5641E2E1-3247-4330-9CD2-2B74CAA98A01@net.in.tum.de>
	<D0BD4A5C03717E40AC104854AF37B8C2160320@EXMBX01.austin.utexas.edu>
Message-ID: <39A0C941-D981-446B-A2AF-D924C29A9A2F@net.in.tum.de>

Hi,

On Oct 27, 2011, at 11:21 PM, William Jones wrote:

> What version of linux are you running.   Redhat 5 doesn't seem to have a lro option just the gro option which does not seem to affect the LRO behavior.


Indeed, gro will only affect the software-based receive offload.

Old ethtool versions do not allow lro manipulations on the command line. We are using ethtool 2.6.34, which offers this option.

Best regards,
  Lothar

--
Lothar Braun
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universit?t M?nchen
Boltzmannstr. 3, 85748 Garching bei M?nchen, Germany
Phone:  +49 89 289-18010       Fax: +49 89 289-18033
E-mail: braun at net.in.tum.de 








From jones at tacc.utexas.edu  Thu Oct 27 15:21:11 2011
From: jones at tacc.utexas.edu (William Jones)
Date: Thu, 27 Oct 2011 22:21:11 +0000
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <39A0C941-D981-446B-A2AF-D924C29A9A2F@net.in.tum.de>
References: <20111027042458.GA87273@icir.org>
	<CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
	<20111027180243.GP36458@icir.org>
	<D0BD4A5C03717E40AC104854AF37B8C2160221@EXMBX01.austin.utexas.edu>
	<5641E2E1-3247-4330-9CD2-2B74CAA98A01@net.in.tum.de>
	<D0BD4A5C03717E40AC104854AF37B8C2160320@EXMBX01.austin.utexas.edu>
	<39A0C941-D981-446B-A2AF-D924C29A9A2F@net.in.tum.de>
Message-ID: <D0BD4A5C03717E40AC104854AF37B8C2160445@EXMBX01.austin.utexas.edu>

Thanks!

-----Original Message-----
From: Lothar Braun [mailto:braun at net.in.tum.de] 
Sent: Thursday, October 27, 2011 5:00 PM
To: William Jones
Cc: 'Robin Sommer'; Martin Holste; bro-dev at bro-ids.org
Subject: Re: [Bro-Dev] snaplen and drops

Hi,

On Oct 27, 2011, at 11:21 PM, William Jones wrote:

> What version of linux are you running.   Redhat 5 doesn't seem to have a lro option just the gro option which does not seem to affect the LRO behavior.


Indeed, gro will only affect the software-based receive offload.

Old ethtool versions do not allow lro manipulations on the command line. We are using ethtool 2.6.34, which offers this option.

Best regards,
  Lothar

--
Lothar Braun
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universit?t M?nchen
Boltzmannstr. 3, 85748 Garching bei M?nchen, Germany
Phone:  +49 89 289-18010       Fax: +49 89 289-18033
E-mail: braun at net.in.tum.de 








From bro at tracker.bro-ids.org  Thu Oct 27 17:18:58 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 00:18:58 -0000
Subject: [Bro-Dev] #608: broctl print times out if the table is too big
In-Reply-To: <049.8e88348ff12d18f23c9143d4a84c4f7d@tracker.bro-ids.org>
References: <049.8e88348ff12d18f23c9143d4a84c4f7d@tracker.bro-ids.org>
Message-ID: <064.582b1c8ba9f8996c2e7d03c604305a99@tracker.bro-ids.org>

#608: broctl print times out if the table is too big
----------------------+------------------------
  Reporter:  aashish  |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------
Changes (by robin):

 * version:  1.5.4 => git/master
 * milestone:   => Bro2.0


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/608#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Thu Oct 27 20:36:46 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 27 Oct 2011 20:36:46 -0700
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <535469B6-200D-4940-A2E5-704C89457850@net.in.tum.de>
References: <20111027042458.GA87273@icir.org>
	<CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
	<535469B6-200D-4940-A2E5-704C89457850@net.in.tum.de>
Message-ID: <20111028033646.GA90146@icir.org>


On Thu, Oct 27, 2011 at 22:31 +0200, you wrote:

> Which kernel and libpcap version do you use? 

Fedora 15, kernel 2.6.40, libpcap 1.1.1, Intel NIC.

> libpcap requests a two megabyte sized shared buffer (default size)
> from the kernel.

That's it! I just patched libcap to request a much larger buffer, and
right now it's looking like the drops are indeed gone even at the 65K
snaplen.

> You could try to make libpcap allocate a bigger buffer with
> pcap_set_buffer_size(). However, this must be called before
> pcap_activate(), which means that you cannot use pcap_open_live() but
> have to call pcap_create, pcap_set_snaplen, pcap_set_timeout,
> pcap_activate yourself...

Argh. Are they serious? There's essentially no way to control the
buffer size? My patch now looks for an environment variable ...

Thanks for pointing this out,

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Fri Oct 28 00:36:15 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 07:36:15 -0000
Subject: [Bro-Dev] #651: Use "referer" rather than "referrer" in http.log
In-Reply-To: <050.5c1aa53d04ba928c31b4ba2c89fdbc9e@tracker.bro-ids.org>
References: <050.5c1aa53d04ba928c31b4ba2c89fdbc9e@tracker.bro-ids.org>
Message-ID: <065.da6dedea983a471f0bd392bae3e27f63@tracker.bro-ids.org>

#651: Use "referer" rather than "referrer" in http.log
-----------------------+------------------------
  Reporter:  vallenti  |      Owner:  seth
      Type:  Problem   |     Status:  new
  Priority:  Normal    |  Milestone:  Bro2.0
 Component:  Bro       |    Version:  git/master
Resolution:            |   Keywords:
-----------------------+------------------------

Comment (by vern):

 How about simply renaming it to "ref"?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/651#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From vern at icir.org  Fri Oct 28 00:36:38 2011
From: vern at icir.org (Vern Paxson)
Date: Fri, 28 Oct 2011 00:36:38 -0700
Subject: [Bro-Dev] paper on protocol identification
In-Reply-To: <2B6F309B-B052-44DA-97EB-DDF48E562DBA@icir.org> (Thu,
	27 Oct 2011 13:03:13 EDT).
Message-ID: <20111028073638.8CD762C4002@rock.ICSI.Berkeley.EDU>

> Seems weird that it doesn't reference the DPD paper..

As you work (crawl) your way up the ivory tower, you learn to read tea
leaves associated with lame work.  In this case, (1) INFOCOM is an
unimpressive venue to appear in (crap-shoot regarding the significance of
accepted papers), but in particular (2) once you finally read between the
lines, this is a 5-page *workshop* paper co-located with INFOCOM (expected
quality level drops precipitously).

So yeah, the lack of citation winds up not being surprising.

That said, they appear to be pushing on looking at port numbers plus
payload of first packet, which is certainly more light-weight than DPD.
OTOH, I'm pretty sure previous schemes have looked at this, too.

		Vern

From braun at net.in.tum.de  Fri Oct 28 01:49:19 2011
From: braun at net.in.tum.de (Lothar Braun)
Date: Fri, 28 Oct 2011 10:49:19 +0200
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <20111028033646.GA90146@icir.org>
References: <20111027042458.GA87273@icir.org>
	<CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
	<535469B6-200D-4940-A2E5-704C89457850@net.in.tum.de>
	<20111028033646.GA90146@icir.org>
Message-ID: <A74413A4-28D1-40B3-8356-314239E1174C@net.in.tum.de>

Hi,

On Oct 28, 2011, at 5:36 AM, Robin Sommer wrote:
>> You could try to make libpcap allocate a bigger buffer with
>> pcap_set_buffer_size(). However, this must be called before
>> pcap_activate(), which means that you cannot use pcap_open_live() but
>> have to call pcap_create, pcap_set_snaplen, pcap_set_timeout,
>> pcap_activate yourself...
> 
> Argh. Are they serious? There's essentially no way to control the
> buffer size? My patch now looks for an environment variable ...


You can change the buffer size in Bro if you use the new API for opening an interface. However, this will not work with libpcap versions < 1.0.0 

But that's not a real problem (at leat for  Linux) because these versions do not support memory mapping, anyways. 

If you want to use the new API and do not want to drop support for libpcap < 1.0.0, you have to check the pcap version in cmake and set some define for old versions (e.g. -DOLD_PCAP). Then you can have something like the following in PktSrc.cc:

#ifdef OLD_PCAP
	pd = pcap_open_live(...);
	if (!pd) 
		do_some_complaining();
		return;
#else
	int status;
        pd = pcap_create(device, errorbuf);
        if (!pd)
            do_some_complaining();
        status = pcap_set_snaplen(pd, snaplen);
        if (status < 0)
                goto fail;
        status = pcap_set_promisc(pd, promisc);
        if (status < 0)
                goto fail;
        status = pcap_set_timeout(pd, to_ms);
        if (status < 0)
                goto fail;

	/* increase the buffer size */
	status = pcap_set_buffer_size(pd, new_bigger_buffer_size)
        if (status < 0)
                goto fail;

        status = pcap_activate(p);
        if (status < 0)
                goto fail;
#endif

	do_some_more_useful_stuff_if_necessary();

#ifndef OLD_PCAP
fail:
	do_some_complaining();
	pcap_close(pd);
#endif


--
Lothar Braun
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universit?t M?nchen
Boltzmannstr. 3, 85748 Garching bei M?nchen, Germany
Phone:  +49 89 289-18010       Fax: +49 89 289-18033
E-mail: braun at net.in.tum.de 








From bro at tracker.bro-ids.org  Fri Oct 28 08:53:49 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 15:53:49 -0000
Subject: [Bro-Dev] #648: SpoolDir setting isn't used everywhere
In-Reply-To: <046.ea9ac6eac3f6ed983568e802720fbd53@tracker.bro-ids.org>
References: <046.ea9ac6eac3f6ed983568e802720fbd53@tracker.bro-ids.org>
Message-ID: <061.0d73cc4e73f9abc4e0fdd253a1a47266@tracker.bro-ids.org>

#648: SpoolDir setting isn't used everywhere
-------------------------+--------------------
  Reporter:  seth        |      Owner:
      Type:  Problem     |     Status:  new
  Priority:  Normal      |  Milestone:  Bro2.0
 Component:  BroControl  |    Version:
Resolution:              |   Keywords:
-------------------------+--------------------

Comment (by jsiwek):

 The way I'm seeing this break is:

 1) install bro with default settings
 2) change `broctl.cfg` to use a `SpoolDir = /usr/local/bro/spool2`
 3) `mkdir /usr/local/bro/spool2`
 4) `broctl` then `install` and `start` will hang
 5) In `spool2/bro/stderr.log` I see `nohup: cannot run command '/run-bro':
 No such file or directory`

 Will look into it further...

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/648#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Fri Oct 28 08:56:35 2011
From: robin at icir.org (Robin Sommer)
Date: Fri, 28 Oct 2011 08:56:35 -0700
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <A74413A4-28D1-40B3-8356-314239E1174C@net.in.tum.de>
References: <20111027042458.GA87273@icir.org>
	<CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
	<535469B6-200D-4940-A2E5-704C89457850@net.in.tum.de>
	<20111028033646.GA90146@icir.org>
	<A74413A4-28D1-40B3-8356-314239E1174C@net.in.tum.de>
Message-ID: <20111028155635.GB12238@icir.org>


On Fri, Oct 28, 2011 at 10:49 +0200, you wrote:

> If you want to use the new API and do not want to drop support for
> libpcap < 1.0.0, you have to check the pcap version in cmake and set
> some define for old versions (e.g. -DOLD_PCAP). Then you can have
> something like the following in PktSrc.cc:

Thanks for the code example, I hadn't really looked at the new API
yet. I'm not that concerned about dropping support for libpcap < 1.
The part I don't like is how the new parameter "buffer size" impacts
behaviour of existing programs without given the user a hook to change
the default. That doesn't seem right to me.

Anyways, for Bro is probably makes most sense to address this as a
part of a larger piece we already have on our to-do list: overhauling
Bro's code for packet aquisition. It's in pretty bad shape right now:
(1) the main packet loop still works around problems with non-blocking
mode in older libpcap/OS versions; I would hope that's not necessary
anymore. (2), we don't have a nice interface for using other packet
sources than libpcap; we need an abstraction there. And finally (3),
if we got an interface in to exploit further NIC-level features, like
load-balancing, that would be pretty cool. 

Not sure when we somebody will start working on all this though.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Fri Oct 28 09:25:12 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 16:25:12 -0000
Subject: [Bro-Dev] #648: SpoolDir setting isn't used everywhere
In-Reply-To: <046.ea9ac6eac3f6ed983568e802720fbd53@tracker.bro-ids.org>
References: <046.ea9ac6eac3f6ed983568e802720fbd53@tracker.bro-ids.org>
Message-ID: <061.f78fbe52132a2ce8c9c4780a270211b0@tracker.bro-ids.org>

#648: SpoolDir setting isn't used everywhere
-------------------------+--------------------
  Reporter:  seth        |      Owner:
      Type:  Problem     |     Status:  new
  Priority:  Normal      |  Milestone:  Bro2.0
 Component:  BroControl  |    Version:
Resolution:              |   Keywords:
-------------------------+--------------------

Comment (by jsiwek):

 Looks to be because `$prefix/share/broctl/scripts/broctl-config.sh` is
 installed as a symlink to whatever the spool directory was at configure-
 time and `broctl install` does not relink to the current value of the
 spool dir.  I think I will have `broctl install` attempt to refresh the
 link to point to the current spool dir and give an error asking the user
 to do it if it fails.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/648#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 10:09:07 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 17:09:07 -0000
Subject: [Bro-Dev] #388: Fix more compiler warnings
In-Reply-To: <047.179fc98dc41c3ce74c9f5d347bf34b3d@tracker.bro-ids.org>
References: <047.179fc98dc41c3ce74c9f5d347bf34b3d@tracker.bro-ids.org>
Message-ID: <062.f0829ec81e7fdfdb54990eb50489579e@tracker.bro-ids.org>

#388: Fix more compiler warnings
---------------------+--------------------
  Reporter:  robin   |      Owner:
      Type:  Task    |     Status:  new
  Priority:  Normal  |  Milestone:  Bro2.0
 Component:  Bro     |    Version:
Resolution:          |   Keywords:
---------------------+--------------------

Comment (by robin):

 More warnings in Bro as well. OS X Lion's compiler now reports a number
 of:

 {{{
 warning: format not a string literal and no format arguments
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/388#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 10:13:12 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 17:13:12 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
Message-ID: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
---------------------+--------------------
 Reporter:  robin    |      Owner:
     Type:  Problem  |     Status:  new
 Priority:  Normal   |  Milestone:  Bro2.0
Component:  Bro      |    Version:
 Keywords:           |
---------------------+--------------------
 (I think this was reported before but not recorded in a ticket).

 Installing as  non-root with a custom ``--prefix`` runs into trouble when
 trying into install Ruby (and probably Python) code system-wide. See
 below.

 Not sure what the right answer is, but at the minimum such errors should
 be ignored and installation proceed for the rest.

 {{{
 -- Installing: /Users/robin/tmp/bro/include/broccoli.h
 -- Installing: /Users/robin/tmp/bro/lib/broctl/broccoli.py
 -- Installing: /Users/robin/tmp/bro/lib/broctl/_broccoli_intern.so
 -- Installing: /Library/Ruby/Site/1.8/broccoli.rb
 CMake Error at aux/broccoli/bindings/broccoli-ruby/cmake_install.cmake:33
 (FILE):
   file INSTALL cannot copy file
   "/Users/robin/tmp/bro-2.0-beta/aux/broccoli/bindings/broccoli-
 ruby/lib/broccoli.rb"
   to "/Library/Ruby/Site/1.8/broccoli.rb".
 Call Stack (most recent call first):
   aux/broccoli/cmake_install.cmake:60 (INCLUDE)
   cmake_install.cmake:51 (INCLUDE)

 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 10:35:55 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 17:35:55 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
In-Reply-To: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
References: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
Message-ID: <062.c26eb8ee372a720b3d883c48b2ac2b70@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
----------------------+--------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by dnthayer):

 I'm seeing this problem only when my build system (ubuntu 10.04 x86_64)
 has both "ruby" and "ruby-dev" installed.  When I run "./configure
 --prefix=/my/prefix",
 in the output under "Broccoli-Ruby Build Summary", I see that it doesn't
 use
 my install prefix (it uses /usr/local/lib instead).

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 10:41:00 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 17:41:00 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
In-Reply-To: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
References: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
Message-ID: <062.89bb8f055037f538f430d65e66357866@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
----------------------+--------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by robin):

 On Fri, Oct 28, 2011 at 17:35 -0000, you wrote:

 >  I'm seeing this problem only when my build system (ubuntu 10.04 x86_64)
 >  has both "ruby" and "ruby-dev" installed.

 That's interesting, I was already wondering why I hadn't seen this
 earlier; seems it depends on circumstances (this was Mac OS X, without
 any custom Ruby packages installed I believe).

 Robin

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 10:51:01 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 17:51:01 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
In-Reply-To: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
References: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
Message-ID: <062.cbfbad5dfb6cec64d89c3462311b7dd4@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
----------------------+--------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by dnthayer):

 Replying to [comment:1 robin]:
 > On Fri, Oct 28, 2011 at 17:35 -0000, you wrote:
 >
 > >  I'm seeing this problem only when my build system (ubuntu 10.04
 x86_64)
 > >  has both "ruby" and "ruby-dev" installed.
 >
 > That's interesting, I was already wondering why I hadn't seen this
 > earlier; seems it depends on circumstances (this was Mac OS X, without
 > any custom Ruby packages installed I believe).
 >
 > Robin

 Forgot to mention that I saw this issue also on the NMI BaTLab
 Mac OS X 10.6 system when I was trying to fix a build problem
 on that platform.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 10:55:03 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 17:55:03 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
In-Reply-To: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
References: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
Message-ID: <062.f694d6d42bad73c959fbfa223009d548@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
----------------------+--------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by jsiwek):

 > >  I'm seeing this problem only when my build system (ubuntu 10.04
 x86_64)
 > >  has both "ruby" and "ruby-dev" installed.
 >
 > That's interesting, I was already wondering why I hadn't seen this
 > earlier; seems it depends on circumstances (this was Mac OS X, without
 > any custom Ruby packages installed I believe).

 Yeah, when `broccoli-ruby` is included as a sub-project and dependencies
 are not satisfied, it just gives a warning that the ruby bindings will not
 be built (i.e. it's an optional component).  Which means in some cases it
 might never get the chance to ignore your `--prefix` option.

 I still need to review the issues behind the decision to ignore `--prefix`
 (I'm not an advanced Ruby user), but it seems not good to me as it is.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 10:58:07 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 17:58:07 -0000
Subject: [Bro-Dev] #648: SpoolDir setting isn't used everywhere
In-Reply-To: <046.ea9ac6eac3f6ed983568e802720fbd53@tracker.bro-ids.org>
References: <046.ea9ac6eac3f6ed983568e802720fbd53@tracker.bro-ids.org>
Message-ID: <061.4f99db644be3fb9e1094a4f88986cc45@tracker.bro-ids.org>

#648: SpoolDir setting isn't used everywhere
-------------------------+--------------------
  Reporter:  seth        |      Owner:
      Type:  Problem     |     Status:  new
  Priority:  Normal      |  Milestone:  Bro2.0
 Component:  BroControl  |    Version:
Resolution:              |   Keywords:
-------------------------+--------------------

Comment (by jsiwek):

 In [c1ba44b1594e6aa37587f44a1bffe4bb23e9e0cf/broctl]:
 {{{
 #!CommitTicketReference repository="broctl"
 revision="c1ba44b1594e6aa37587f44a1bffe4bb23e9e0cf"
 Make symlink to broctl-config.sh update with `broctl install`

 Addresses #648
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/648#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 10:59:08 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 17:59:08 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
In-Reply-To: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
References: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
Message-ID: <062.da9d34f0e991e3fd28ea39de49f87055@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
----------------------+--------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by robin):

 But also sounds like this is a Ruby-only issue (I.e., there isn't a
 similar effect for Python)?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 11:07:07 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 18:07:07 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
In-Reply-To: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
References: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
Message-ID: <062.396a3c7c0166e59e8263f7e5e0982a5e@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
----------------------+--------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by dnthayer):

 Replying to [comment:3 jsiwek]:
 > Yeah, when `broccoli-ruby` is included as a sub-project and dependencies
 are not satisfied, it just gives a warning that the ruby bindings will not
 be built (i.e. it's an optional component).  Which means in some cases it
 might never get the chance to ignore your `--prefix` option.

 On my ubuntu build machine, if I don't have both "ruby" and "ruby-dev"
 packages
 installed, then the configure just outputs a warning message that the
 ruby bindings will not be built (and doing a "make install" succeeds
 as a non-root user with a custom --prefix, because it doesn't try
 to install the ruby bindings).

 However, if I have both "ruby" and "ruby-dev" packages installed,
 then my "make install" (non-root user with custom --prefix) fails
 because my custom --prefix is being ignored by the ruby bindings.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652#comment:5>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 11:09:39 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 18:09:39 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
In-Reply-To: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
References: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
Message-ID: <062.06ca390a2b6e4be2af43453d3ae54ab1@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
----------------------+--------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by dnthayer):

 Replying to [comment:4 robin]:
 > But also sounds like this is a Ruby-only issue (I.e., there isn't a
 > similar effect for Python)?

 I have seen this bug only when my build machine has both
 "ruby" and "ruby-dev" installed.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652#comment:6>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 11:16:13 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 18:16:13 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
In-Reply-To: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
References: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
Message-ID: <062.b3b04997223cfb045f86cf132997b0ba@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
----------------------+--------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by seth):

 > But also sounds like this is a Ruby-only issue (I.e., there isn't a
 > similar effect for Python)?

 I expect that it's similar for python.  I just made the decision to do it
 the it is because then the ruby bindings get installed in the ruby system
 path so you don't need to modify your load path to load them.

 I agree with what everyone else seems to be coming to though, the
 python/ruby/perl bindings should all probably have similar build and
 install behavior when they're built as submodules of Bro.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652#comment:7>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 11:24:51 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 18:24:51 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
In-Reply-To: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
References: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
Message-ID: <062.5798389c4dead8b0742f07b8266879d9@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
----------------------+--------------------
  Reporter:  robin    |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by dnthayer):

 Replying to [comment:7 seth]:
 > > But also sounds like this is a Ruby-only issue (I.e., there isn't a
 > > similar effect for Python)?
 >
 > I expect that it's similar for python.  I just made the decision to do
 it the it is because then the ruby bindings get installed in the ruby
 system path so you don't need to modify your load path to load them.
 >
 > I agree with what everyone else seems to be coming to though, the
 python/ruby/perl bindings should all probably have similar build and
 install behavior when they're built as submodules of Bro.

 On my system, when I do a "./configure --prefix=/my/prefix", in the
 output under "PyBroccoli Build Summary", it shows "Install dir:
 /my/prefix/lib/broctl".

 However, under "Broccoli-Ruby Build Summary", it shows "Lib install dir:
 /usr/local/lib/site_ruby/1.8"

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652#comment:8>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 12:03:33 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 19:03:33 -0000
Subject: [Bro-Dev] #605: cmake warning when disabling python bindings
In-Reply-To: <046.8b41a6db267ec257b10840da1b679fb9@tracker.bro-ids.org>
References: <046.8b41a6db267ec257b10840da1b679fb9@tracker.bro-ids.org>
Message-ID: <061.420b3891da7256bae80580e56f04c70d@tracker.bro-ids.org>

#605: cmake warning when disabling python bindings
-----------------------+--------------------
  Reporter:  seth      |      Owner:  jsiwek
      Type:  Problem   |     Status:  closed
  Priority:  Normal    |  Milestone:
 Component:  Broccoli  |    Version:
Resolution:  fixed     |   Keywords:
-----------------------+--------------------
Changes (by jsiwek):

 * status:  new => closed
 * resolution:   => fixed


Comment:

 In [7527ae2973e1f93f1d7fccb869901626f351fe74/broccoli]:
 {{{
 #!CommitTicketReference repository="broccoli"
 revision="7527ae2973e1f93f1d7fccb869901626f351fe74"
 Fix CMake warning when python bindings are disabled (fixes #605).
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/605#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 12:58:46 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 19:58:46 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
In-Reply-To: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
References: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
Message-ID: <062.000866d03b913114ee51e6ec0ff7856b@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
----------------------+--------------------
  Reporter:  robin    |      Owner:  jsiwek
      Type:  Problem  |     Status:  closed
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:  fixed    |   Keywords:
----------------------+--------------------
Changes (by jsiwek):

 * owner:   => jsiwek
 * status:  new => closed
 * resolution:   => fixed


Comment:

 In [a1ad4185fa243453e1f17532581f3ef1b52bcbe8/broccoli-ruby]:
 {{{
 #!CommitTicketReference repository="broccoli-ruby"
 revision="a1ad4185fa243453e1f17532581f3ef1b52bcbe8"
 Changes to broccoli-ruby installation scheme. (fixes #652)

 - `--home` and `--prefix` configure options are now respected when
   installing as the main CMake project.  If not given, the Ruby
   system path is automatically detected and used.

 - when being installed as a CMake sub-project, then the "home"-style
   installation is performed.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652#comment:9>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 13:02:50 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 20:02:50 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
In-Reply-To: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
References: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
Message-ID: <062.4d41260fad4f8d28d1d4a0f74ee8f9eb@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
----------------------+--------------------
  Reporter:  robin    |      Owner:  jsiwek
      Type:  Problem  |     Status:  closed
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:  fixed    |   Keywords:
----------------------+--------------------

Comment (by jsiwek):

 > But also sounds like this is a Ruby-only issue (I.e., there isn't a
 > similar effect for Python)?

 Right.  Let me know if you guys still see anything preventing non-root
 installs (works for me now).

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652#comment:10>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 13:28:05 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 20:28:05 -0000
Subject: [Bro-Dev] #645: Figure out where sendmail is
In-Reply-To: <047.30bfd70cef14df8f513f2d5595dceaae@tracker.bro-ids.org>
References: <047.30bfd70cef14df8f513f2d5595dceaae@tracker.bro-ids.org>
Message-ID: <062.f80df28516a319959a8ea7f209686d8e@tracker.bro-ids.org>

#645: Figure out where sendmail is
-------------------------+--------------------
  Reporter:  robin       |      Owner:
      Type:  Problem     |     Status:  new
  Priority:  Normal      |  Milestone:  Bro2.0
 Component:  BroControl  |    Version:
Resolution:              |   Keywords:
-------------------------+--------------------

Comment (by jsiwek):

 Replying to [ticket:645 robin]:
 >
 > I keep thinking that this entry shouldn't be in the default
 `broctl.cfg`:
 >
 >   {{{
 >   SendMail = /usr/sbin/sendmail
 > }}}
 >
 > Can we figure out at configure time where sendmail is and preset the
 option accordingly?

 Yes, but do you have a preference for what the behavior should be if no
 sendmail is found? e.g. make it required and abort the configuration, go
 back to default "/usr/sbin/sendmail", or leave it blank?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/645#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Oct 28 14:15:13 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 28 Oct 2011 21:15:13 -0000
Subject: [Bro-Dev] #652: Installation permissions for Broccoli bindings
In-Reply-To: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
References: <047.57c1a60515758adb0fc1e287349e57db@tracker.bro-ids.org>
Message-ID: <062.749f270aaa07aa96e587da89eceb7438@tracker.bro-ids.org>

#652: Installation permissions for Broccoli bindings
----------------------+--------------------
  Reporter:  robin    |      Owner:  jsiwek
      Type:  Problem  |     Status:  closed
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:  fixed    |   Keywords:
----------------------+--------------------

Comment (by dnthayer):

 Replying to [comment:10 jsiwek]:
 > > But also sounds like this is a Ruby-only issue (I.e., there isn't a
 > > similar effect for Python)?
 >
 > Right.  Let me know if you guys still see anything preventing non-root
 installs (works for me now).

 Tested on ubuntu 10.04, with "--prefix" and without.  It now
 works as expected.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/652#comment:11>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From mcholste at gmail.com  Fri Oct 28 15:21:03 2011
From: mcholste at gmail.com (Martin Holste)
Date: Fri, 28 Oct 2011 17:21:03 -0500
Subject: [Bro-Dev] snaplen and drops
In-Reply-To: <20111028155635.GB12238@icir.org>
References: <20111027042458.GA87273@icir.org>
	<CANpnLHiYrPYRJntdH5yHQe19hf86N2V5DUy0=tcX5y0WovJK8w@mail.gmail.com>
	<535469B6-200D-4940-A2E5-704C89457850@net.in.tum.de>
	<20111028033646.GA90146@icir.org>
	<A74413A4-28D1-40B3-8356-314239E1174C@net.in.tum.de>
	<20111028155635.GB12238@icir.org>
Message-ID: <CANpnLHh8jy+ymGpGOD0RvjdzXt-X4ukDygVb=_FPiaUo_qGOAw@mail.gmail.com>

Glad you were able to sort this out.  I use PF_RING exclusively for
packet capture, so I've not run into this before.

In the future, AF_PACKET support would be a great addition to Bro and
would bring it closer to Snort and Suricata as far as acquisition.
It's got performance reasonably close to PF_RING without having to
download anything extra.  However, you need to be running a 3.0 Linux
kernel to do software load-balancing, which is one of the reasons I
use PF_RING.

On Fri, Oct 28, 2011 at 10:56 AM, Robin Sommer <robin at icir.org> wrote:
>
> On Fri, Oct 28, 2011 at 10:49 +0200, you wrote:
>
>> If you want to use the new API and do not want to drop support for
>> libpcap < 1.0.0, you have to check the pcap version in cmake and set
>> some define for old versions (e.g. -DOLD_PCAP). Then you can have
>> something like the following in PktSrc.cc:
>
> Thanks for the code example, I hadn't really looked at the new API
> yet. I'm not that concerned about dropping support for libpcap < 1.
> The part I don't like is how the new parameter "buffer size" impacts
> behaviour of existing programs without given the user a hook to change
> the default. That doesn't seem right to me.
>
> Anyways, for Bro is probably makes most sense to address this as a
> part of a larger piece we already have on our to-do list: overhauling
> Bro's code for packet aquisition. It's in pretty bad shape right now:
> (1) the main packet loop still works around problems with non-blocking
> mode in older libpcap/OS versions; I would hope that's not necessary
> anymore. (2), we don't have a nice interface for using other packet
> sources than libpcap; we need an abstraction there. And finally (3),
> if we got an interface in to exploit further NIC-level features, like
> load-balancing, that would be pretty cool.
>
> Not sure when we somebody will start working on all this though.
>
> Robin
>
> --
> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
> ICSI/LBNL ? ?* Fax ? +1 (510) 666-2956 * ? www.icir.org
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>


From bro at tracker.bro-ids.org  Mon Oct 31 10:30:06 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 17:30:06 -0000
Subject: [Bro-Dev] #608: broctl print times out if the table is too big
In-Reply-To: <049.8e88348ff12d18f23c9143d4a84c4f7d@tracker.bro-ids.org>
References: <049.8e88348ff12d18f23c9143d4a84c4f7d@tracker.bro-ids.org>
Message-ID: <064.6b04af0c2c704de24b2a87ac49317c55@tracker.bro-ids.org>

#608: broctl print times out if the table is too big
----------------------+------------------------
  Reporter:  aashish  |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------

Comment (by seth):

 One small comment.  This ticket doesn't *completely* apply to git/master
 since the variable bring printed is no longer a variable in git/master.
 The response you should be getting now is something about the variable not
 being defined.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/608#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 10:35:25 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 17:35:25 -0000
Subject: [Bro-Dev] #646: Cleanup interpreter error handling.
In-Reply-To: <047.f66ed674d262f7ebac52f1c4e6df3a3e@tracker.bro-ids.org>
References: <047.f66ed674d262f7ebac52f1c4e6df3a3e@tracker.bro-ids.org>
Message-ID: <062.8bfeb4280237c567d08d3bbcea82218e@tracker.bro-ids.org>

#646: Cleanup interpreter error handling.
---------------------+----------------------
  Reporter:  robin   |      Owner:
      Type:  Task    |     Status:  new
  Priority:  Normal  |  Milestone:  Bro2.1
 Component:  Bro     |    Version:
Resolution:          |   Keywords:  language
---------------------+----------------------
Changes (by robin):

 * keywords:   => language


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/646#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 10:47:09 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 17:47:09 -0000
Subject: [Bro-Dev] #608: broctl print times out if the table is too big
In-Reply-To: <049.8e88348ff12d18f23c9143d4a84c4f7d@tracker.bro-ids.org>
References: <049.8e88348ff12d18f23c9143d4a84c4f7d@tracker.bro-ids.org>
Message-ID: <064.86d31b4c8b4167f7cb96e880815280d1@tracker.bro-ids.org>

#608: broctl print times out if the table is too big
----------------------+------------------------
  Reporter:  aashish  |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------

Comment (by robin):

 On Mon, Oct 31, 2011 at 17:30 -0000, you wrote:

 >  One small comment.  This ticket doesn't *completely* apply to
 git/master
 >  since the variable bring printed is no longer a variable in git/master.

 That's right, my thinking was to generally fix printing large tables,
 which likely will all have the same problem.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/608#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From asharma at lbl.gov  Mon Oct 31 10:54:46 2011
From: asharma at lbl.gov (Aashish Sharma)
Date: Mon, 31 Oct 2011 10:54:46 -0700
Subject: [Bro-Dev] #608: broctl print times out if the table is too big
In-Reply-To: <064.6b04af0c2c704de24b2a87ac49317c55@tracker.bro-ids.org>
References: <049.8e88348ff12d18f23c9143d4a84c4f7d@tracker.bro-ids.org>
	<064.6b04af0c2c704de24b2a87ac49317c55@tracker.bro-ids.org>
Message-ID: <20111031175445.GK14607@f0-4d-a2-28-3c-de.dhcp.lbl.gov>

I think irrespective of the existence of the policy (and the variable),
timeout and not printing the vaule of a variable is still an issue which
needs to be addressed. 

Also, for bro-2.0-beta released code, I tried printing a few variables
eg:

bro> ~bro/bro-20/bin/broctl print Scan::scan_triples 
       bro   Scan::scan_triples = <unknown id>

and saw the <unknown id> error. 

Possible, I am missing something..? 

Aashish 


On Mon, Oct 31, 2011 at 05:30:06PM -0000, Bro Tracker wrote:
> #608: broctl print times out if the table is too big
> ----------------------+------------------------
>   Reporter:  aashish  |      Owner:
>       Type:  Problem  |     Status:  new
>   Priority:  Normal   |  Milestone:  Bro2.0
>  Component:  Bro      |    Version:  git/master
> Resolution:           |   Keywords:
> ----------------------+------------------------
> 
> Comment (by seth):
> 
>  One small comment.  This ticket doesn't *completely* apply to git/master
>  since the variable bring printed is no longer a variable in git/master.
>  The response you should be getting now is something about the variable not
>  being defined.
> 
> -- 
> Ticket URL: <http://tracker.bro-ids.org/bro/ticket/608#comment:2>
> Bro Tracker <http://tracker.bro-ids.org/bro>
> Bro Issue Tracker

-- 
Aashish Sharma	(asharma at lbl.gov) 				 
Cyber Security, Information Technology Division  
Lawrence Berkeley National Laboratory  
http://www.lbl.gov/cyber/pgp-aashish.txt 
Office: (510)-495-2680  Cell: (510)-457-1525
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20111031/c4a692e8/attachment.bin 

From bro at tracker.bro-ids.org  Mon Oct 31 10:54:50 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 17:54:50 -0000
Subject: [Bro-Dev] #608: broctl print times out if the table is too big
In-Reply-To: <049.8e88348ff12d18f23c9143d4a84c4f7d@tracker.bro-ids.org>
References: <049.8e88348ff12d18f23c9143d4a84c4f7d@tracker.bro-ids.org>
Message-ID: <064.ab61744f5367ab8a2554036322122db4@tracker.bro-ids.org>

#608: broctl print times out if the table is too big
----------------------+------------------------
  Reporter:  aashish  |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------

Comment (by aashish):

 None
 Filename None could not be saved, problem: [Errno 13] Permission denied:
 '/da/trac/projects/bro/attachments/ticket/608'\I think irrespective of the
 existence of the policy (and the variable),
 timeout and not printing the vaule of a variable is still an issue which
 needs to be addressed.

 Also, for bro-2.0-beta released code, I tried printing a few variables
 eg:

 bro> ~bro/bro-20/bin/broctl print Scan::scan_triples
        bro   Scan::scan_triples = <unknown id>

 and saw the <unknown id> error.

 Possible, I am missing something..?

 Aashish


 On Mon, Oct 31, 2011 at 05:30:06PM -0000, Bro Tracker wrote:
 > #608: broctl print times out if the table is too big
 > ----------------------+------------------------
 >   Reporter:  aashish  |      Owner:
 >       Type:  Problem  |     Status:  new
 >   Priority:  Normal   |  Milestone:  Bro2.0
 >  Component:  Bro      |    Version:  git/master
 > Resolution:           |   Keywords:
 > ----------------------+------------------------
 >
 > Comment (by seth):
 >
 >  One small comment.  This ticket doesn't *completely* apply to
 git/master
 >  since the variable bring printed is no longer a variable in git/master.
 >  The response you should be getting now is something about the variable
 not
 >  being defined.
 >
 > --
 > Ticket URL: <http://tracker.bro-ids.org/bro/ticket/608#comment:2>
 > Bro Tracker <http://tracker.bro-ids.org/bro>
 > Bro Issue Tracker

 [attachment:"None"]

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/608#comment:5>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 11:02:51 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 18:02:51 -0000
Subject: [Bro-Dev] #608: broctl print times out if the table is too big
In-Reply-To: <049.8e88348ff12d18f23c9143d4a84c4f7d@tracker.bro-ids.org>
References: <049.8e88348ff12d18f23c9143d4a84c4f7d@tracker.bro-ids.org>
Message-ID: <064.b9ea7c501add289c29ec93e8fa44c150@tracker.bro-ids.org>

#608: broctl print times out if the table is too big
----------------------+------------------------
  Reporter:  aashish  |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------

Comment (by seth):

 On Oct 31, 2011, at 1:54 PM, Aashish Sharma wrote:

 > I think irrespective of the existence of the policy (and the variable),
 > timeout and not printing the vaule of a variable is still an issue which
 > needs to be addressed.

 Yep, we're going to be looking into it.

 > bro> ~bro/bro-20/bin/broctl print Scan::scan_triples
 >       bro   Scan::scan_triples = <unknown id>
 >
 > and saw the <unknown id> error.


 That variable doesn't exist right now.  The scan.bro script will be coming
 back in the contributed scripts repository soon, but it's not there yet.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/608#comment:6>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 12:38:49 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 19:38:49 -0000
Subject: [Bro-Dev] #613: Problem with LogExpireInterval setting in
	BroControl
In-Reply-To: <046.6b2381d7e3479123e85b6eccb468740e@tracker.bro-ids.org>
References: <046.6b2381d7e3479123e85b6eccb468740e@tracker.bro-ids.org>
Message-ID: <061.0f180b9e8933e90cd3944fe6d9353e0c@tracker.bro-ids.org>

#613: Problem with LogExpireInterval setting in BroControl
-------------------------+--------------------
  Reporter:  seth        |      Owner:
      Type:  Problem     |     Status:  new
  Priority:  Normal      |  Milestone:  Bro2.0
 Component:  BroControl  |    Version:
Resolution:              |   Keywords:
-------------------------+--------------------

Comment (by jsiwek):

 Replying to [ticket:613 seth]:
 > It doesn't seem to work.  It should be deleting logs, but the logs are
 still there.

 This works for me.  It's dependent on `broctl cron`, though, did you have
 that running?  And it also checks actual file modification times so if you
 tried to simply cheat it by renaming directories that doesn't work, you
 have to `touch -m -t` stuff.

 > I also wonder if we should have this turned off by default?  I'd rather
 not have people surprised when their logs start disappearing after a month
 (the default setting is 30 days).

 Yeah, I think just commenting it out in `broctl.cfg` might be good.  I'll
 do that and put it on fastpath.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/613#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 13:17:48 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 20:17:48 -0000
Subject: [Bro-Dev] #613: Problem with LogExpireInterval setting in
	BroControl
In-Reply-To: <046.6b2381d7e3479123e85b6eccb468740e@tracker.bro-ids.org>
References: <046.6b2381d7e3479123e85b6eccb468740e@tracker.bro-ids.org>
Message-ID: <061.ddf1eb11e97f0d512f5ff1c3f718ca46@tracker.bro-ids.org>

#613: Problem with LogExpireInterval setting in BroControl
-------------------------+--------------------
  Reporter:  seth        |      Owner:
      Type:  Problem     |     Status:  new
  Priority:  Normal      |  Milestone:  Bro2.0
 Component:  BroControl  |    Version:
Resolution:              |   Keywords:
-------------------------+--------------------

Comment (by jsiwek):

 In [5e96d6b38072b13a63ed0f73010d8bea18ec2287/broctl]:
 {{{
 #!CommitTicketReference repository="broctl"
 revision="5e96d6b38072b13a63ed0f73010d8bea18ec2287"
 Disable log expiration by default. (Addresses #613)
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/613#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 14:32:54 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 21:32:54 -0000
Subject: [Bro-Dev] #612: Segfault in identify_data BiF
In-Reply-To: <048.2145c0c1e4de755bc9fa18b2c7b0ac68@tracker.bro-ids.org>
References: <048.2145c0c1e4de755bc9fa18b2c7b0ac68@tracker.bro-ids.org>
Message-ID: <063.f08ec34a7eb8630ec347d61c6e5ecf95@tracker.bro-ids.org>

#612: Segfault in identify_data BiF
-----------------------------+------------------------
  Reporter:  gregor          |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro2.0
 Component:  Bro             |    Version:  git/master
Resolution:  Solved/Applied  |   Keywords:
-----------------------------+------------------------
Changes (by jsiwek):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 Closing since the segfault hasn't been encountered recently anymore.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/612#comment:10>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 14:40:10 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 21:40:10 -0000
Subject: [Bro-Dev] #624: Bro won't shutdown if not seeing packets
In-Reply-To: <046.06582d3d3a1ee7e1010ccc163ac53a7f@tracker.bro-ids.org>
References: <046.06582d3d3a1ee7e1010ccc163ac53a7f@tracker.bro-ids.org>
Message-ID: <061.064fd949b9be87691f097a56ed681006@tracker.bro-ids.org>

#624: Bro won't shutdown if not seeing packets
----------------------+--------------------
  Reporter:  seth     |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by jsiwek):

 My previous attempts to reproduce this weren't successful, it may have
 been something to do with PF_RING-enabled Bro like #599.  I'll try once
 more to reproduce it, else I'm going to close it if you still aren't
 seeing it.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/624#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 14:41:16 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 21:41:16 -0000
Subject: [Bro-Dev] #624: Bro won't shutdown if not seeing packets
In-Reply-To: <046.06582d3d3a1ee7e1010ccc163ac53a7f@tracker.bro-ids.org>
References: <046.06582d3d3a1ee7e1010ccc163ac53a7f@tracker.bro-ids.org>
Message-ID: <061.2af93401a875df3df4d01b5a9490be07@tracker.bro-ids.org>

#624: Bro won't shutdown if not seeing packets
----------------------+----------------------
  Reporter:  seth     |      Owner:  jsiwek
      Type:  Problem  |     Status:  assigned
  Priority:  Normal   |  Milestone:  Bro2.0
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+----------------------
Changes (by jsiwek):

 * owner:   => jsiwek
 * status:  new => assigned


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/624#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 14:41:40 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 21:41:40 -0000
Subject: [Bro-Dev] #645: Figure out where sendmail is
In-Reply-To: <047.30bfd70cef14df8f513f2d5595dceaae@tracker.bro-ids.org>
References: <047.30bfd70cef14df8f513f2d5595dceaae@tracker.bro-ids.org>
Message-ID: <062.8a502833a257eaac6360d80acfb80a59@tracker.bro-ids.org>

#645: Figure out where sendmail is
-------------------------+----------------------
  Reporter:  robin       |      Owner:  jsiwek
      Type:  Problem     |     Status:  assigned
  Priority:  Normal      |  Milestone:  Bro2.0
 Component:  BroControl  |    Version:
Resolution:              |   Keywords:
-------------------------+----------------------
Changes (by jsiwek):

 * owner:   => jsiwek
 * status:  new => assigned


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/645#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 14:42:31 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 21:42:31 -0000
Subject: [Bro-Dev] #650: Finish Broxygen docs
In-Reply-To: <047.9abf8a0851b3b85991457973560c772f@tracker.bro-ids.org>
References: <047.9abf8a0851b3b85991457973560c772f@tracker.bro-ids.org>
Message-ID: <062.328bbdc277c2b7b06991fcce293efd12@tracker.bro-ids.org>

#650: Finish Broxygen docs
---------------------+----------------------
  Reporter:  robin   |      Owner:  jsiwek
      Type:  Task    |     Status:  assigned
  Priority:  Normal  |  Milestone:  Bro2.0
 Component:  Bro     |    Version:
Resolution:          |   Keywords:
---------------------+----------------------
Changes (by jsiwek):

 * owner:   => jsiwek
 * status:  new => assigned


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/650#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 15:30:31 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 31 Oct 2011 22:30:31 -0000
Subject: [Bro-Dev] #653: bro-cut does not work with mawk
Message-ID: <050.b131409afa45c564960b689e75d671f3@tracker.bro-ids.org>

#653: bro-cut does not work with mawk
----------------------+------------------------
 Reporter:  dnthayer  |      Owner:
     Type:  Problem   |     Status:  new
 Priority:  Normal    |  Milestone:
Component:  bro-aux   |    Version:  git/master
 Keywords:            |
----------------------+------------------------
 The "bro-cut" script does not seem to work with the implementation
 of awk (mawk 1.3.3) on my system (ubuntu 11.04).  When I try to
 use "bro-cut" it fails with this not-so-helpful message:

 awk: line 52: illegal reference to array f
 awk: line 65: illegal reference to array columns
 awk: line 88: illegal reference to array columns

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/653>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 17:29:10 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 01 Nov 2011 00:29:10 -0000
Subject: [Bro-Dev] #654: Off_Port_Protocol_Found not used
Message-ID: <047.15c2d0cfc177e12da263861fac931591@tracker.bro-ids.org>

#654: Off_Port_Protocol_Found not used
---------------------+--------------------
 Reporter:  robin    |      Owner:
     Type:  Problem  |     Status:  new
 Priority:  Normal   |  Milestone:  Bro2.0
Component:  Bro      |    Version:
 Keywords:           |
---------------------+--------------------
 In `dpd/detect-protocols` there are defined:

 {{{
                  Off_Port_Protocol_Found, # raised for each connection
 found
                 Protocol_Found,
 }}}

 But only the 2nd one is raised in that script. Is there an intended
 difference between the two or is it just a left over?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/654>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 20:30:42 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 01 Nov 2011 03:30:42 -0000
Subject: [Bro-Dev] #653: bro-cut does not work with mawk
In-Reply-To: <050.b131409afa45c564960b689e75d671f3@tracker.bro-ids.org>
References: <050.b131409afa45c564960b689e75d671f3@tracker.bro-ids.org>
Message-ID: <065.b500ba9cc313acd3371c517b2dab6749@tracker.bro-ids.org>

#653: bro-cut does not work with mawk
-----------------------+------------------------
  Reporter:  dnthayer  |      Owner:
      Type:  Problem   |     Status:  new
  Priority:  Normal    |  Milestone:  Bro2.0
 Component:  bro-aux   |    Version:  git/master
Resolution:            |   Keywords:
-----------------------+------------------------
Changes (by robin):

 * milestone:   => Bro2.0


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/653#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Oct 31 21:20:51 2011
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 01 Nov 2011 04:20:51 -0000
Subject: [Bro-Dev] #606: broccoli and connection records
In-Reply-To: <046.83cc834c3a71aa4f378a2b7bec3e66a4@tracker.bro-ids.org>
References: <046.83cc834c3a71aa4f378a2b7bec3e66a4@tracker.bro-ids.org>
Message-ID: <061.7999f28339cb0afad1081c211fbbb622@tracker.bro-ids.org>

#606: broccoli and connection records
-----------------------+----------------------
  Reporter:  seth      |      Owner:  kreibich
      Type:  Problem   |     Status:  accepted
  Priority:  Normal    |  Milestone:
 Component:  Broccoli  |    Version:
Resolution:            |   Keywords:
-----------------------+----------------------

Comment (by seth):

 Any progress to report on this?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/606#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker