From noreply at bro-ids.org  Thu Nov  1 00:00:09 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Thu, 1 Nov 2012 00:00:09 -0700
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211010700.qA1709ku008380@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |   915 [1] | seth       | robin      | Normal | topic/seth/modbus-merge [2]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | e020e03    | Seth Hall       | 2012-11-01 | Script in base for detecting cases of checksum offloading. [3]



 [1]       #915: http://tracker.bro-ids.org/bro/ticket/915
 [2] modbus-merge: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/modbus-merge
 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/e020e034ae4ebabf3c84f6ca8ea9cf538197ca1c/bro


From vladg at cmu.edu  Thu Nov  1 08:47:53 2012
From: vladg at cmu.edu (Vlad Grigorescu)
Date: Thu, 1 Nov 2012 15:47:53 +0000
Subject: [Bro-Dev] [Bro-Commits-Internal] CompileDebug - Build # 235
	-	Failure!
In-Reply-To: <7167_1351732625_qA11H43G019249_20121101011701.GA37734@icir.org>
References: <657613901.1.1351730282106.JavaMail.jenkins@brotestbed.ncsa.illinois.edu>
	<7167_1351732625_qA11H43G019249_20121101011701.GA37734@icir.org>
Message-ID: <1202BE242E080642B0CD0AD0A03E855259F610@PGH-MSGMB-03.andrew.ad.cmu.edu>

Sorry about that - I should've done more digging to see if that would work on BSD-type systems.

>From what I can tell, you can just use bind to bind a socket to an address, but there's no way to bind directly to an interface. If we want to continue with specifying interfaces instead of addresses, we'd have to call getifaddrs, and iterate through the interfaces to get the address associated with a specific interface name.

This would introduce the limitation that you can only send RSTs out of an interface that has an address associated with it. I don't think that's a particularly troublesome limitation, but just something to keep in mind.

Does this sound like a reasonable way to proceed?

  --Vlad

On Oct 31, 2012, at 9:17 PM, Robin Sommer <robin at icir.org> wrote:

> 
> On Wed, Oct 31, 2012 at 19:38 -0500, jenkins at brotestbed.ncsa.illinois.edu wrote:
> 
>> /home/jenkins/workspace/CompileDebug/bro/aux/bro-aux/rst/rst.c:364: error: 'SO_BINDTODEVICE' undeclared (first use in this function)
> 
> Yeah, I was afraid that this could cause trouble. Any way to make that
> portable?
> 
> Robin
> 
> -- 
> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
> ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev



From jsiwek at illinois.edu  Thu Nov  1 09:04:47 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Thu, 1 Nov 2012 16:04:47 +0000
Subject: [Bro-Dev] [Bro-Commits-Internal] CompileDebug - Build #
	235	-	Failure!
In-Reply-To: <1202BE242E080642B0CD0AD0A03E855259F610@PGH-MSGMB-03.andrew.ad.cmu.edu>
References: <657613901.1.1351730282106.JavaMail.jenkins@brotestbed.ncsa.illinois.edu>
	<7167_1351732625_qA11H43G019249_20121101011701.GA37734@icir.org>
	<1202BE242E080642B0CD0AD0A03E855259F610@PGH-MSGMB-03.andrew.ad.cmu.edu>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED339B@CITESMBX2.ad.uillinois.edu>

>> From what I can tell, you can just use bind to bind a socket to an address, but there's no way to bind directly to an interface. If we want to continue with specifying interfaces instead of addresses, we'd have to call getifaddrs, and iterate through the interfaces to get the address associated with a specific interface name.

I think this excerpt from [1] has another reason why that might not be the most reliable solution:

"The bind() system call is frequently misunderstood. It is used to bind to a particular IP address. Only packets destined to that IP address will be received, and any transmitted packets will carry that IP address as their source. bind() does not control anything about the routing of transmitted packets. So for example, if you bound to the IP address of eth0 but you send a packet to a destination where the kernel's best route goes out eth1, it will happily send the packet out eth1 with the source IP address of eth0."

And I didn't see alternatives to SO_BINDTODEVICE in my brief search.  There was mentions of IP_SENDIF for BSD, but that looked like it might be available only as a patch right now.

So maybe the thing to do for now (if know one else knows how to make it portable) is just make it a Linux-only feature -- add some configure-time checks for platform (if they aren't there already), and then wrap the code in preprocessor directives.

    Jon

[1] http://codingrelic.geekhold.com/2009/10/code-snippet-sobindtodevice.html

From robin at icir.org  Thu Nov  1 09:22:35 2012
From: robin at icir.org (Robin Sommer)
Date: Thu, 1 Nov 2012 09:22:35 -0700
Subject: [Bro-Dev] [Bro-Commits-Internal] CompileDebug - Build # 235
	-	Failure!
In-Reply-To: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED339B@CITESMBX2.ad.uillinois.edu>
References: <657613901.1.1351730282106.JavaMail.jenkins@brotestbed.ncsa.illinois.edu>
	<7167_1351732625_qA11H43G019249_20121101011701.GA37734@icir.org>
	<1202BE242E080642B0CD0AD0A03E855259F610@PGH-MSGMB-03.andrew.ad.cmu.edu>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED339B@CITESMBX2.ad.uillinois.edu>
Message-ID: <20121101162235.GJ48672@icir.org>


On Thu, Nov 01, 2012 at 16:04 +0000, you wrote:

> So maybe the thing to do for now (if know one else knows how to make
> it portable) is just make it a Linux-only feature -- add some
> configure-time checks for platform (if they aren't there already), and
> then wrap the code in preprocessor directives.

yeah, I had the same thought.

Robin


-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Thu Nov  1 09:37:19 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 01 Nov 2012 16:37:19 -0000
Subject: [Bro-Dev] #914: topic/seth/intel-framework
In-Reply-To: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org>
References: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org>
Message-ID: <061.dcc242537d0d6fa845422738e716ed27@tracker.bro-ids.org>

#914: topic/seth/intel-framework
----------------------+------------------------
  Reporter:  seth     |      Owner:  seth
      Type:  Problem  |     Status:  assigned
  Priority:  Normal   |  Milestone:  Bro2.2
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------

Comment (by robin):

 Ok, so I had updated (I had checked that actually) but there was indeed
 some mixup. Redone the merge now, and this looks better now.

 But http/detect-intel.bro is still there. Removing that.

 coverage tests still failing. I've fixed all of those except coverage
 .bare-mode-errors.

 Pushing my changes to topic/robin/intel-framework-merge for now.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/914#comment:8>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Nov  1 09:37:49 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 01 Nov 2012 16:37:49 -0000
Subject: [Bro-Dev] #914: topic/seth/intel-framework
In-Reply-To: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org>
References: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org>
Message-ID: <061.ef577184cb3c09006527acd554627581@tracker.bro-ids.org>

#914: topic/seth/intel-framework
----------------------+------------------------
  Reporter:  seth     |      Owner:  seth
      Type:  Problem  |     Status:  closed
  Priority:  Normal   |  Milestone:  Bro2.2
 Component:  Bro      |    Version:  git/master
Resolution:  fixed    |   Keywords:
----------------------+------------------------
Changes (by robin):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [fb7ba82babe25ebbb983d974b1c353e2affc5235/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="fb7ba82babe25ebbb983d974b1c353e2affc5235"
 Merge remote-tracking branch 'origin/topic/seth/intel-framework'

 * origin/topic/seth/intel-framework: (21 commits)
   Extracting URLs from message bodies over SMTP and sending them to Intel
 framework.
   Small comment updates in the Intel framework CIF support.
   Intelligence framework documentation first draft.
   Only the manager tries to read files with the input framework now.
   Initial support for Bro's Intel framework with the Collective
 Intelligence Framework.
   Initial API for Intel framework is complete.
   Fixed an issue with cluster data distribution.
   Updating some intel framework test baselines.
   Reworked cluster intelligence data distribution mechanism and fixed
 tests.
   Lots more intelligence checking in SMTP traffic.
   Added intelligence check for "Received" path checking and a bit of
 reshuffling.
   Added sources to the intel log.
   Fixing a problem with intel distribution on clusters.
   Updated intel framework test to include matching.
   Restructuring the scripts that feed data into the intel framework
 slightly.
   One test for cluster transparency of the intel framework.
   Fixed a cluster support bug.
   Intelligence framework checkpoint
   Major updates to fix the Intel framework API.
   Checkpoint commit.  This is all a huge mess right now. :)
   ...

 Closes #914.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/914#comment:9>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Nov  1 10:05:35 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 01 Nov 2012 17:05:35 -0000
Subject: [Bro-Dev] #914: topic/seth/intel-framework
In-Reply-To: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org>
References: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org>
Message-ID: <061.565f892a20002cca362e1636aaa96b5b@tracker.bro-ids.org>

#914: topic/seth/intel-framework
----------------------+------------------------
  Reporter:  seth     |      Owner:  seth
      Type:  Problem  |     Status:  reopened
  Priority:  Normal   |  Milestone:  Bro2.2
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------
Changes (by robin):

 * status:  closed => reopened
 * resolution:  fixed =>


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/914#comment:10>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Nov  1 10:54:01 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 01 Nov 2012 17:54:01 -0000
Subject: [Bro-Dev] #914: topic/seth/intel-framework
In-Reply-To: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org>
References: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org>
Message-ID: <061.2e6c09d8bf411b802cc314ca1190e105@tracker.bro-ids.org>

#914: topic/seth/intel-framework
----------------------+------------------------
  Reporter:  seth     |      Owner:  seth
      Type:  Problem  |     Status:  reopened
  Priority:  Normal   |  Milestone:  Bro2.2
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------

Comment (by seth):

 > But http/detect-intel.bro is still there. Removing that.


 That was indeed a mistake.  I forgot to remove it.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/914#comment:15>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From dnthayer at illinois.edu  Thu Nov  1 12:02:48 2012
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Thu, 1 Nov 2012 14:02:48 -0500
Subject: [Bro-Dev] [Bro-Commits-Internal] CompileDebug - Build # 235	-
 Failure!
In-Reply-To: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED339B@CITESMBX2.ad.uillinois.edu>
References: <657613901.1.1351730282106.JavaMail.jenkins@brotestbed.ncsa.illinois.edu>
	<7167_1351732625_qA11H43G019249_20121101011701.GA37734@icir.org>
	<1202BE242E080642B0CD0AD0A03E855259F610@PGH-MSGMB-03.andrew.ad.cmu.edu>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED339B@CITESMBX2.ad.uillinois.edu>
Message-ID: <5092C758.6050605@illinois.edu>

On 11/01/2012 11:04 AM, Siwek, Jonathan Luke wrote:
>>>  From what I can tell, you can just use bind to bind a socket to an address, but there's no way to bind directly to an interface. If we want to continue with specifying interfaces instead of addresses, we'd have to call getifaddrs, and iterate through the interfaces to get the address associated with a specific interface name.
>
> I think this excerpt from [1] has another reason why that might not be the most reliable solution:
>
> "The bind() system call is frequently misunderstood. It is used to bind to a particular IP address. Only packets destined to that IP address will be received, and any transmitted packets will carry that IP address as their source. bind() does not control anything about the routing of transmitted packets. So for example, if you bound to the IP address of eth0 but you send a packet to a destination where the kernel's best route goes out eth1, it will happily send the packet out eth1 with the source IP address of eth0."
>
> And I didn't see alternatives to SO_BINDTODEVICE in my brief search.  There was mentions of IP_SENDIF for BSD, but that looked like it might be available only as a patch right now.
>
> So maybe the thing to do for now (if know one else knows how to make it portable) is just make it a Linux-only feature -- add some configure-time checks for platform (if they aren't there already), and then wrap the code in preprocessor directives.
>
>      Jon
>
> [1] http://codingrelic.geekhold.com/2009/10/code-snippet-sobindtodevice.html

I also couldn't find an easy non-Linux solution for this, so I
just #ifdef'd the code to make it Linux-only.

-Daniel



From seth at icir.org  Thu Nov  1 13:35:17 2012
From: seth at icir.org (Seth Hall)
Date: Thu, 1 Nov 2012 16:35:17 -0400
Subject: [Bro-Dev] changing Notice::policy mechanism
Message-ID: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>

I've discussed changing the Notice::policy notice handling mechanism with a few people and I think that generally everyone agrees that the current mechanism, while very powerful, sucks from a usability perspective.  This is a good thing to address now since we're either on the precipice of repeating a mistake to more frameworks or adapting to a style that is easier to understand.  I'm going to give an example of how I think it could work in an evented model now.  Please shoot holes. :)

These implement the decidedly silly case of sending a notice to an email if the SSH connection originated locally.

event Notice::policy(n: Notice::Info) 
	{
	if ( n$note == SSH::Login &&
	     Site::is_local_host(n$id$orig_h) )
		{
		add n$actions[Notice::ACTION_EMAIL];
		}
	}

That would be replacement to the current model of this:

redef Notice::policy += {
	[$pred(n: Notice::Info) = { 
		return ( n$note == SSH::Login && Site::is_local_host(n$id$orig_h) );
	 },
	 $action = Notice::ACTION_EMAIL],
};

Here are some random thoughts about these two approaches in no particular order:

 - The evented model (top one) is more Bro-y and easier which is a BIG plus.

 - The PolicyItem model (bottom one) has the ability to halt further processing with the $halt attribute of PolicyItems.  I don't think I'm convinced that this is a huge issue.

 - The evented model has latency from the event queue, but I don't think this is a huge issue.  The latency is normally ok.  Jon, is it an issue for the file analysis framework?  I don't remember.  The actions being applied would be processed through an event queue too so they will be processed after the policy events anyway.

 - Code block prioritization is built into the evented model using the &priority attribute.  It's specifically implemented for PolicyItem model.

Any thoughts?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From robin at icir.org  Thu Nov  1 13:39:32 2012
From: robin at icir.org (Robin Sommer)
Date: Thu, 1 Nov 2012 13:39:32 -0700
Subject: [Bro-Dev] DNS TXT Queries and the Cache File
In-Reply-To: <1202BE242E080642B0CD0AD0A03E8552593265@PGH-MSGMB-03.andrew.ad.cmu.edu>
References: <1202BE242E080642B0CD0AD0A03E855234C4F1@PGH-MSGMB-03.andrew.ad.cmu.edu>
	<503F3AB2.4040900@illinois.edu>
	<352_1346341138_q7UFcvg7017598_20120830153851.GA89651@icir.org>
	<1202BE242E080642B0CD0AD0A03E8552593265@PGH-MSGMB-03.andrew.ad.cmu.edu>
Message-ID: <20121101203932.GE52215@icir.org>

Thanks, I've merged this in, can you try master and see if it works
for you?

> e let me know if anyone sees any issues. There is a save TXT
> function, but there is no capability to read the data back from a
> file, as I mentioned. If someone wants to take a stab to getting that
> working properly, please feel free. Otherwise, let me know and I'll
> remove the save function.

I've removed it, don't think we need it. If we wanted to bring it
back, that's easy to do.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From dnthayer at illinois.edu  Thu Nov  1 14:01:40 2012
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Thu, 1 Nov 2012 16:01:40 -0500
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
Message-ID: <5092E334.7070906@illinois.edu>

On 11/01/2012 03:35 PM, Seth Hall wrote:
> I've discussed changing the Notice::policy notice handling mechanism with a few people and I think that generally everyone agrees that the current mechanism, while very powerful, sucks from a usability perspective.  This is a good thing to address now since we're either on the precipice of repeating a mistake to more frameworks or adapting to a style that is easier to understand.  I'm going to give an example of how I think it could work in an evented model now.  Please shoot holes. :)
>
> These implement the decidedly silly case of sending a notice to an email if the SSH connection originated locally.
>
> event Notice::policy(n: Notice::Info)
> 	{
> 	if ( n$note == SSH::Login &&
> 	     Site::is_local_host(n$id$orig_h) )
> 		{
> 		add n$actions[Notice::ACTION_EMAIL];
> 		}
> 	}
>
> That would be replacement to the current model of this:
>
> redef Notice::policy += {
> 	[$pred(n: Notice::Info) = {
> 		return ( n$note == SSH::Login && Site::is_local_host(n$id$orig_h) );
> 	 },
> 	 $action = Notice::ACTION_EMAIL],
> };
>
> Here are some random thoughts about these two approaches in no particular order:
>
>   - The evented model (top one) is more Bro-y and easier which is a BIG plus.
>
>   - The PolicyItem model (bottom one) has the ability to halt further processing with the $halt attribute of PolicyItems.  I don't think I'm convinced that this is a huge issue.
>
>   - The evented model has latency from the event queue, but I don't think this is a huge issue.  The latency is normally ok.  Jon, is it an issue for the file analysis framework?  I don't remember.  The actions being applied would be processed through an event queue too so they will be processed after the policy events anyway.
>
>   - Code block prioritization is built into the evented model using the &priority attribute.  It's specifically implemented for PolicyItem model.
>
> Any thoughts?
>
>    .Seth
>

I like this idea (it seems MUCH easier to use).  How would
"suppress_for" be implemented in the evented model?


From jsiwek at illinois.edu  Thu Nov  1 14:47:39 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Thu, 1 Nov 2012 21:47:39 +0000
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>

> - The PolicyItem model (bottom one) has the ability to halt further processing with the $halt attribute of PolicyItems.  I don't think I'm convinced that this is a huge issue.

I think the same thing could be done with the event-model -- if Notice::Info just had a boolean field that each event handler body could check and then bail out early if necessary.  And that field could even be something left up to a user to define in a redef.

> - The evented model has latency from the event queue, but I don't think this is a huge issue.  The latency is normally ok.  Jon, is it an issue for the file analysis framework?

I think the latency is a problem in that case since immediate feedback is helpful if certain actions are to work without buffering incoming file data.  E.g. if the only thing I want to happen is for files to be extracted to disk, then I wouldn't expect that to require buffering.

It may be workable, but I think the latency does make things more complicated so it might not make sense to re-use this policy model for the file analysis framework, but that shouldn't be a showstopper for using it for notices.  I think it's fine to have the policy models be inconsistent, as long as both are easy to understand and use for their purpose.

    Jon

From seth at icir.org  Thu Nov  1 19:07:13 2012
From: seth at icir.org (Seth Hall)
Date: Thu, 1 Nov 2012 22:07:13 -0400
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <5092E334.7070906@illinois.edu>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
	<5092E334.7070906@illinois.edu>
Message-ID: <6FECB4D7-3666-45D4-9C2B-8A4E568C2962@icir.org>


On Nov 1, 2012, at 5:01 PM, Daniel Thayer <dnthayer at illinois.edu> wrote:

> I like this idea (it seems MUCH easier to use).  How would
> "suppress_for" be implemented in the evented model?


Suppression is implemented at a pretty low level in the notice framework.  That Notice::policy event would never be generated for suppressed notices (just like the notice policy variable currently isn't evaluated for suppressed notices).

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Thu Nov  1 19:28:38 2012
From: seth at icir.org (Seth Hall)
Date: Thu, 1 Nov 2012 22:28:38 -0400
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>
Message-ID: <69CAE6B4-81E9-49B8-8E38-F7AC30A6FBD6@icir.org>


On Nov 1, 2012, at 5:47 PM, "Siwek, Jonathan Luke" <jsiwek at illinois.edu> wrote:

> It may be workable, but I think the latency does make things more complicated so it might not make sense to re-use this policy model for the file analysis framework, but that shouldn't be a showstopper for using it for notices.  I think it's fine to have the policy models be inconsistent, as long as both are easy to understand and use for their purpose.


As we discussed earlier and I never brought up in my overview, would it be better if we had some mechanism sitting between events and functions?  "Synchronous events"? "Multi-bodied functions"?  Here's an example of how a synchronous event might look at the script land?

global awesome: event(a: count) &synchronous;
global foobar: event(a: count);

event bro_init()
	{
	event foobar(1);
	event awesome(5);
	}

event awesome(a: count) &priority=5
	{
	print fmt("awesome: %d", a+5);
	}

event awesome(a: count) &priority=2
	{
	print fmt("awesome: %d", a+2);
	}

event foobar(a: count) 
	{
	print "foobar handler!";
	}


Assuming synchronous events existed, the output would be this:

awesome: 10
awesome: 7
foobar handler!

I would really kind of like to move away from this policy model.  The more I think about it, the less Bro-y it feels and I have trouble remembering how exactly to use them too (which I don't think bodes well for anyone!).  Focusing on simplicity and consistency at the script layer is something that we'll probably have to continue to struggle and fight for forever.  Although, I'm generally ok with complexity anywhere in Bro as long as we find a clear and obvious way to expose it to users.

  .Seth


--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Thu Nov  1 19:44:45 2012
From: seth at icir.org (Seth Hall)
Date: Thu, 1 Nov 2012 22:44:45 -0400
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <69CAE6B4-81E9-49B8-8E38-F7AC30A6FBD6@icir.org>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>
	<69CAE6B4-81E9-49B8-8E38-F7AC30A6FBD6@icir.org>
Message-ID: <DAF7C752-1FFF-4134-BB18-D73635EA0EF9@icir.org>


On Nov 1, 2012, at 10:28 PM, Seth Hall <seth at icir.org> wrote:

> As we discussed earlier and I never brought up in my overview, would it be better if we had some mechanism sitting between events and functions?  "Synchronous events"? "Multi-bodied functions"?  Here's an example of how a synchronous event might look at the script land?


Aaand, to reply to myself?  Now that I implemented that sample code, I don't like it.  Events presume the event queue.  If we bypass that, it just seems bad and inconsistent.  Maybe we just need a mechanism for multi-bodied functions?  It would basically be functions that could be defined over and over, could have a &priority attribute applied to them, and would be enforced to have no return value.

global foobar: function(a: count) &multibody;

function foobar(a: count) &priority=5
	{
	print "foobar at priority 5";
	}

function foobar(a: count) &priority=8
	{
	print "foobar at priority 8";
	}

print foobar(1); // <-- Maybe it could return a count to indicate number of bodies executed?



Sample output
=============
foobar at priority 8
foobar at priority 5
2

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Thu Nov  1 20:11:29 2012
From: seth at icir.org (Seth Hall)
Date: Thu, 1 Nov 2012 23:11:29 -0400
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <DAF7C752-1FFF-4134-BB18-D73635EA0EF9@icir.org>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>
	<69CAE6B4-81E9-49B8-8E38-F7AC30A6FBD6@icir.org>
	<DAF7C752-1FFF-4134-BB18-D73635EA0EF9@icir.org>
Message-ID: <9957A0E9-A872-4EEE-9A8A-EA37B3FD0E70@icir.org>

One more reply to myself. :)

On Nov 1, 2012, at 10:44 PM, Seth Hall <seth at icir.org> wrote:

> global foobar: function(a: count) &multibody;
> 
> function foobar(a: count) &priority=5
> 	{
> 	print "foobar at priority 5";
> 	}
> 
> function foobar(a: count) &priority=8
> 	{
> 	print "foobar at priority 8";
> 	}


I was thinking about how this would look internally and I suspect that the easiest thing to do would be to "inline" the bodies of all of the functions into a single function body.  There would be some work that would need to be done to deal with edge cases brought about by smashing scopes together like this, but I think it should be possible to watch for locals being created and making sure that there aren't any conflicting names (Bro should probably not start up if there are conflicting names).

  .Seth


--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Thu Nov  1 20:19:12 2012
From: seth at icir.org (Seth Hall)
Date: Thu, 1 Nov 2012 23:19:12 -0400
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <9957A0E9-A872-4EEE-9A8A-EA37B3FD0E70@icir.org>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>
	<69CAE6B4-81E9-49B8-8E38-F7AC30A6FBD6@icir.org>
	<DAF7C752-1FFF-4134-BB18-D73635EA0EF9@icir.org>
	<9957A0E9-A872-4EEE-9A8A-EA37B3FD0E70@icir.org>
Message-ID: <D9172BB4-B485-4972-A337-68E52262E5E9@icir.org>

Dammit.

On Nov 1, 2012, at 11:11 PM, Seth Hall <seth at icir.org> wrote:

>  There would be some work that would need to be done to deal with edge cases brought about by smashing scopes together like this, but I think it should be possible to watch for locals being created and making sure that there aren't any conflicting names (Bro should probably not start up if there are conflicting names).


Actually, nevermind.  I'm not fully thinking through the scope smashing.  We could just create a scope for each of the function implementations?

>> function foobar(a: count) &priority=5
>> 	{
>> 	local test = "nope";
>> 	print test;
>> 	}
>> 
>> function foobar(a: count) &priority=8
>> 	{
>> 	local test = "yup";
>> 	print test;
>> 	}

Those would inline into this?

function foobar(a: count)
        {
       	{
	local test = "nope";
	print test;
	}
	{
	local test = "yup";
	print test;
	}
	}

The only problem is that this doesn't currently work, but it begs the question? should it?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From noreply at bro-ids.org  Fri Nov  2 00:00:10 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Fri, 2 Nov 2012 00:00:10 -0700
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211020700.qA270AJG003775@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |   915 [1] | seth       | robin      | Normal | topic/seth/modbus-merge [2]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
        bro-aux | 75a841f    | Daniel Thayer   | 2012-11-01 | Use #ifdef for Linux-specific code [3]
            bro | e020e03    | Seth Hall       | 2012-11-01 | Script in base for detecting cases of checksum offloading. [4]



 [1]       #915: http://tracker.bro-ids.org/bro/ticket/915
 [2] modbus-merge: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/modbus-merge
 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/75a841fd7cfa32916ad5f7e80511fec2f119e058/bro-aux
 [4] fastpath: http://tracker.bro-ids.org/bro/changeset/e020e034ae4ebabf3c84f6ca8ea9cf538197ca1c/bro


From jsiwek at illinois.edu  Fri Nov  2 09:06:14 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Fri, 2 Nov 2012 16:06:14 +0000
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <D9172BB4-B485-4972-A337-68E52262E5E9@icir.org>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>
	<69CAE6B4-81E9-49B8-8E38-F7AC30A6FBD6@icir.org>
	<DAF7C752-1FFF-4134-BB18-D73635EA0EF9@icir.org>
	<9957A0E9-A872-4EEE-9A8A-EA37B3FD0E70@icir.org>
	<D9172BB4-B485-4972-A337-68E52262E5E9@icir.org>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED42C3@CITESMBX2.ad.uillinois.edu>


>>> function foobar(a: count) &priority=5
>>> 	{
>>> 	local test = "nope";
>>> 	print test;
>>> 	}
>>> 
>>> function foobar(a: count) &priority=8
>>> 	{
>>> 	local test = "yup";
>>> 	print test;
>>> 	}

Adding the &multibody attribute to do this I think might work alright.  And I think it's not much to implement it -- I just did some quick hacks as a proof of concept and so far it didn't look like a problem.

> Those would inline into this?
> 
> function foobar(a: count)
>        {
>       	{
> 	local test = "nope";
> 	print test;
> 	}
> 	{
> 	local test = "yup";
> 	print test;
> 	}
> 	}
> 
> The only problem is that this doesn't currently work, but it begs the question? should it?

That probably should work, but it's also not related to implementing the proposed &multibody attribute for functions since the bodies are already parsed separately with different scopes and can then be internally stored as separate bodies/scopes, there's no need to do the mashing together under a common scope -- just keep them separate and iterate over the bodies according to priority.

This code already exists because of the similarity of functions and events, it's just that functions are only allowed multiple body definitions if they have &redef, and in that case, the later definition *replaces* earlier ones instead of being pushed on to the vector of bodies.

    Jon

From seth at icir.org  Fri Nov  2 09:20:10 2012
From: seth at icir.org (Seth Hall)
Date: Fri, 2 Nov 2012 12:20:10 -0400
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED42C3@CITESMBX2.ad.uillinois.edu>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>
	<69CAE6B4-81E9-49B8-8E38-F7AC30A6FBD6@icir.org>
	<DAF7C752-1FFF-4134-BB18-D73635EA0EF9@icir.org>
	<9957A0E9-A872-4EEE-9A8A-EA37B3FD0E70@icir.org>
	<D9172BB4-B485-4972-A337-68E52262E5E9@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED42C3@CITESMBX2.ad.uillinois.edu>
Message-ID: <276DEB36-B114-45DD-8239-D2F48DBECF73@icir.org>


On Nov 2, 2012, at 12:06 PM, "Siwek, Jonathan Luke" <jsiwek at illinois.edu> wrote:

> This code already exists because of the similarity of functions and events, it's just that functions are only allowed multiple body definitions if they have &redef, and in that case, the later definition *replaces* earlier ones instead of being pushed on to the vector of bodies.

Ahh, that makes sense.

I actually thought about it a bit more and I'm still not completely attached to the idea.  I sort of don't like how the documentation for that would look where we have to say that normally you shouldn't implement a function over and over, but in special cases it's just how something works.

I almost wonder if we should have another type of code block in addition to events and functions.  It's really just a difference in semantics at this point, but it would be much easier to document if there was a completely different name.  We could document the use and behavior of each of them very clearly with the third name.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From robin at icir.org  Fri Nov  2 09:38:59 2012
From: robin at icir.org (Robin Sommer)
Date: Fri, 2 Nov 2012 09:38:59 -0700
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <D9172BB4-B485-4972-A337-68E52262E5E9@icir.org>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>
	<69CAE6B4-81E9-49B8-8E38-F7AC30A6FBD6@icir.org>
	<DAF7C752-1FFF-4134-BB18-D73635EA0EF9@icir.org>
	<9957A0E9-A872-4EEE-9A8A-EA37B3FD0E70@icir.org>
	<D9172BB4-B485-4972-A337-68E52262E5E9@icir.org>
Message-ID: <20121102163859.GK67486@icir.org>


Can't say I like the multi-body functions; it's an unusual concept and
they'd come with untuitive limitations (like the no return value
constraint). Or put differently, I believe what you call multi-body
functions here are actually just the synchronous events you proposed
first. :-)

Let me try to structure this a bit. The current policy approach gives
us four things I believe (let me know if I'm missing something):

    (1) Immediate execution to completion (that's the synchronous).

    (2) The ability for an item to prevent others from being considered.

    (3) Returning values back to the code running executing the policy
        (actions, suppress_for, etc.)

    (4) Items sorted by priority.

Events currently only provide (4), but I can see changing them to
do (1) and (2):

    - (1): That's the "synchronous" part. There's actually nothing
    internally that would prevent us from executing event handlers
    immediately, rather than queuing them first. In some sense, they
    would jump to the head of the queue and be processed immediately.
    We could indeed introduce a new attribute for such events, though
    I'm not sure I like &synchronous (but no better idea right now).

    This would vioalate the current FIFO order of events, but with
    explicit marking, I think that'd be ok.

    - (2): Jon suggested a boolean flag but I don't like that because
    now each handler would need check for it, which is easy to forget
    and generally repetetive. Another idea for this has been floating
    around already: allow event handlers to stop execution of further
    handlers of the same kind for an event. Something like:

        event foo()
        {
           [...]
           event stop; # No further handlers for this event.
        }


(3) is tricky though, not sure how to do that without return values?
Question is: can we do without? We don't need it for actions (they can
just turn into function calls), but what about suppress-for style
stuff. Seth, you said that suppress_for is not a problem either?

Robin


-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From seth at icir.org  Fri Nov  2 09:51:24 2012
From: seth at icir.org (Seth Hall)
Date: Fri, 2 Nov 2012 12:51:24 -0400
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <20121102163859.GK67486@icir.org>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>
	<69CAE6B4-81E9-49B8-8E38-F7AC30A6FBD6@icir.org>
	<DAF7C752-1FFF-4134-BB18-D73635EA0EF9@icir.org>
	<9957A0E9-A872-4EEE-9A8A-EA37B3FD0E70@icir.org>
	<D9172BB4-B485-4972-A337-68E52262E5E9@icir.org>
	<20121102163859.GK67486@icir.org>
Message-ID: <6801F4E8-1094-4191-9BAB-77C78C1ABA91@icir.org>


On Nov 2, 2012, at 12:38 PM, Robin Sommer <robin at icir.org> wrote:

>   - (1): That's the "synchronous" part. There's actually nothing
>    internally that would prevent us from executing event handlers
>    immediately, rather than queuing them first. In some sense, they
>    would jump to the head of the queue and be processed immediately.
>    We could indeed introduce a new attribute for such events, though
>    I'm not sure I like &synchronous (but no better idea right now).

I don't like that attribute either.

>        event foo()
>        {
>           [...]
>           event stop; # No further handlers for this event.
>        }

I actually don't like this.  I like to be able to assume that all event handlers will execute and I can see users causing some major debugging headaches for us with it.

> (3) is tricky though, not sure how to do that without return values?
> Question is: can we do without? We don't need it for actions (they can
> just turn into function calls), but what about suppress-for style
> stuff. Seth, you said that suppress_for is not a problem either?


Correct, suppress_for isn't a problem and I don't think we need (3) anyway.  

What about my proposal I sent out a little bit ago?  Make a third type of code block with all of the characteristics that we want?  We don't have to compromise on existing and good designs for functions or events that way.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From jsiwek at illinois.edu  Fri Nov  2 09:59:19 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Fri, 2 Nov 2012 16:59:19 +0000
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <20121102163859.GK67486@icir.org>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>
	<69CAE6B4-81E9-49B8-8E38-F7AC30A6FBD6@icir.org>
	<DAF7C752-1FFF-4134-BB18-D73635EA0EF9@icir.org>
	<9957A0E9-A872-4EEE-9A8A-EA37B3FD0E70@icir.org>
	<D9172BB4-B485-4972-A337-68E52262E5E9@icir.org>
	<20121102163859.GK67486@icir.org>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED44B7@CITESMBX2.ad.uillinois.edu>

> (3) is tricky though, not sure how to do that without return values?
> Question is: can we do without? 

I think so.  People could design the "return value" as something expected as a side effect of the call ?  e.g. the value of a record field could get modified in the bodies and then inspected after.

    Jon

From seth at icir.org  Fri Nov  2 12:09:17 2012
From: seth at icir.org (Seth Hall)
Date: Fri, 2 Nov 2012 15:09:17 -0400
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <20121102163859.GK67486@icir.org>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>
	<69CAE6B4-81E9-49B8-8E38-F7AC30A6FBD6@icir.org>
	<DAF7C752-1FFF-4134-BB18-D73635EA0EF9@icir.org>
	<9957A0E9-A872-4EEE-9A8A-EA37B3FD0E70@icir.org>
	<D9172BB4-B485-4972-A337-68E52262E5E9@icir.org>
	<20121102163859.GK67486@icir.org>
Message-ID: <C5948622-E8DD-49AA-8280-CE51312A9F8B@icir.org>


On Nov 2, 2012, at 12:38 PM, Robin Sommer <robin at icir.org> wrote:

> Can't say I like the multi-body functions;


Continuing the discussion from the group call, I think that we're starting to converge on adding a "policy" keyword to the language to represent something between a function and an event.  Jon and I chatted about it some more and we couldn't come to a more abstracted and still clear keyword than "policy". (I really like the policy keyword here)

As a concrete example of the existing policy mechanism in the notice framework and the reworked mechanism I'll give an equivalent implementation of both.

# New style under discussion.
policy Notice::trigger(n: Notice::Info) &priority=5
	{
	if (n$note == Whatever && Site::is_local_host(n$conn$id$orig_h)
		add n$actions[ACTION_BLAH];
	}

# Existing style
redef Notice::policy += {
	[$pred(n: Notice::Info) = {
	 	return (n$note == Whatever && Site::is_local_host(n$conn$id$orig_h);
	 },
	 $priority = 5,
	 $action = ACTION_BLAH]
};


To address your various points about how this needs to work:

  1. Immediate policy handler execution.  Not even by jumping to the beginning of the
     event queue and waiting for the queue to be flushed, just direct execution. 

  2. Adding a "break"-like keyword to halt policy execution (only valid in policy handlers).

  3. No return value(s) when calling policy handlers.  We'll reuse the mutability mechanism
     that I subversively began using all over the place by modifying records in place.
     Calling policy handlers would look like this:
         policy Notice::trigger(n);

  4. Policy handlers sorted by priority.  Equivalent priority has undefined execution order,
     but I don't see that as a problem since it's already a concern for event handlers.


 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From bro at tracker.bro-ids.org  Fri Nov  2 18:05:17 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sat, 03 Nov 2012 01:05:17 -0000
Subject: [Bro-Dev] #584: DNS TXT record lookup bif
In-Reply-To: <046.2077446f247be33289c55cd0258699d3@tracker.bro-ids.org>
References: <046.2077446f247be33289c55cd0258699d3@tracker.bro-ids.org>
Message-ID: <061.34e0e179a1e485aae36fad5980d51589@tracker.bro-ids.org>

#584: DNS TXT record lookup bif
------------------------------+--------------------
  Reporter:  seth             |      Owner:
      Type:  Feature Request  |     Status:  closed
  Priority:  Normal           |  Milestone:  Bro2.2
 Component:  Bro              |    Version:
Resolution:  Solved/Applied   |   Keywords:
------------------------------+--------------------
Changes (by seth):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 Vlad picked this up and it was committed recently.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/584#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Nov  2 18:22:10 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sat, 03 Nov 2012 01:22:10 -0000
Subject: [Bro-Dev] #857: Change capture port in HTTP analyzer from
 3138/tcp instead of 3128/tcp
In-Reply-To: <049.2545d71b81bdfb740467af0719e58c1f@tracker.bro-ids.org>
References: <049.2545d71b81bdfb740467af0719e58c1f@tracker.bro-ids.org>
Message-ID: <064.7ec6a1dec9012ba115c9bf5c3df53523@tracker.bro-ids.org>

#857: Change capture port in HTTP analyzer from 3138/tcp instead of 3128/tcp
-----------------------------+------------------------
  Reporter:  aashish         |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  High            |  Milestone:
 Component:  Bro             |    Version:  git/master
Resolution:  Solved/Applied  |   Keywords:
-----------------------------+------------------------
Changes (by seth):

 * status:  new => closed
 * resolution:   => Solved/Applied


Comment:

 Forgot to close it.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/857#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Nov  2 19:35:03 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sat, 03 Nov 2012 02:35:03 -0000
Subject: [Bro-Dev] #579: "Raw" logging writer (was: Syslog logging
	writer)
In-Reply-To: <046.4e6efad585e65a2ccb68427348524651@tracker.bro-ids.org>
References: <046.4e6efad585e65a2ccb68427348524651@tracker.bro-ids.org>
Message-ID: <061.8b0d677a1102f895599fbade796951c0@tracker.bro-ids.org>

#579: "Raw" logging writer
----------------------+------------------------
  Reporter:  seth     |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  High     |  Milestone:  Bro2.2
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:
----------------------+------------------------
Description changed by seth:

Old description:

> Martin has completely convinced me of the need for this.  I don't know
> about timeline we should put on it though.  The one thought I have about
> it is that it needs to use TCP due to extremely long lines that Bro logs
> tend to have.  I think it would be ok for it to have the same output
> rendering that the LogAscii writer has.

New description:

 This was formerly a ticket about creating syslog logging writer, but I
 think we found a better and more general approach in a "raw" writer.  The
 raw writer would abandon the normal tab separated output from the Ascii
 writer and instead would be based on a templating format passed through
 the config filter field.  There should also be options for sending the
 formatted data to files, sockets, and syslog.

 This writer would open several doors for us:

   * Direct integration from script-land with ELSA.
   * Functional replacement for PRADS in script-land with integration into
 Sguil.
   * Direct script-land integration with the metrics framework and
 Graphite.

 Here is a made up example of creating a metrics filter for sending data to
 Graphite:

 {{{
 Log::add_filter(Metrics::LOG, [$name="graphite",
                                $writer=Log::WRITER_RAW,
                                $path="tcp://1.2.3.4:2003/",
                                $config = table(["fmt"] = "{{metric}}
 {{value}} {{ts}}")]);
 }}}

--

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/579#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From noreply at bro-ids.org  Sat Nov  3 00:00:05 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Sat, 3 Nov 2012 00:00:05 -0700
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211030700.qA3705ks009448@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |   915 [1] | seth       | robin      | Normal | topic/seth/modbus-merge [2]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
        bro-aux | 75a841f    | Daniel Thayer   | 2012-11-01 | Use #ifdef for Linux-specific code [3]
            bro | e020e03    | Seth Hall       | 2012-11-01 | Script in base for detecting cases of checksum offloading. [4]



 [1]       #915: http://tracker.bro-ids.org/bro/ticket/915
 [2] modbus-merge: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/modbus-merge
 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/75a841fd7cfa32916ad5f7e80511fec2f119e058/bro-aux
 [4] fastpath: http://tracker.bro-ids.org/bro/changeset/e020e034ae4ebabf3c84f6ca8ea9cf538197ca1c/bro


From vladg at cmu.edu  Sat Nov  3 15:01:33 2012
From: vladg at cmu.edu (Vlad Grigorescu)
Date: Sat, 3 Nov 2012 22:01:33 +0000
Subject: [Bro-Dev] DNS TXT Queries and the Cache File
In-Reply-To: <2218_1351802376_qA1KdY5W005470_20121101203932.GE52215@icir.org>
References: <1202BE242E080642B0CD0AD0A03E855234C4F1@PGH-MSGMB-03.andrew.ad.cmu.edu>
	<503F3AB2.4040900@illinois.edu>
	<352_1346341138_q7UFcvg7017598_20120830153851.GA89651@icir.org>
	<1202BE242E080642B0CD0AD0A03E8552593265@PGH-MSGMB-03.andrew.ad.cmu.edu>
	<2218_1351802376_qA1KdY5W005470_20121101203932.GE52215@icir.org>
Message-ID: <1202BE242E080642B0CD0AD0A03E85525B73DA@PGH-MSGMB-03.andrew.ad.cmu.edu>

Thanks, Robin. 

I tested master, and it wasn't working properly for me. I traced the problem to the simplification of DNS_Mgr::AsyncLookupName. For both lookups, it was using AsyncRequestNameMap. I split off the processing of TXT queries so that it'd use AsyncRequestTextMap. That was committed in: <https://github.com/grigorescu/bro/commit/a1c0b853fe180ee0dc97880653d2d274b7e08e21>.

I also have a couple of other small commits, which are ready in <https://github.com/grigorescu/bro/commits/topic/vladg/dns_txt_queries>. One has some fixes for the BIF, and one switches the MHR script to use TXT queries, with a newly introduced threshold.

I tested these manually, and they work for me. Unfortunately, I don't have any good ideas on how we'd go about writing some tests for the TXT queries - this was the one case where being able to load the cache from a file did seem useful.

  --Vlad

On Nov 1, 2012, at 4:39 PM, Robin Sommer <robin at icir.org> wrote:

> Thanks, I've merged this in, can you try master and see if it works
> for you?
> 
>> e let me know if anyone sees any issues. There is a save TXT
>> function, but there is no capability to read the data back from a
>> file, as I mentioned. If someone wants to take a stab to getting that
>> working properly, please feel free. Otherwise, let me know and I'll
>> remove the save function.
> 
> I've removed it, don't think we need it. If we wanted to bring it
> back, that's easy to do.
> 
> Robin
> 
> -- 
> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
> ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev



From noreply at bro-ids.org  Sun Nov  4 00:00:03 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Sun, 4 Nov 2012 00:00:03 -0700
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211040700.qA4703MG032723@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |   915 [1] | seth       | robin      | Normal | topic/seth/modbus-merge [2]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
        bro-aux | 75a841f    | Daniel Thayer   | 2012-11-01 | Use #ifdef for Linux-specific code [3]
            bro | e020e03    | Seth Hall       | 2012-11-01 | Script in base for detecting cases of checksum offloading. [4]



 [1]       #915: http://tracker.bro-ids.org/bro/ticket/915
 [2] modbus-merge: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/modbus-merge
 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/75a841fd7cfa32916ad5f7e80511fec2f119e058/bro-aux
 [4] fastpath: http://tracker.bro-ids.org/bro/changeset/e020e034ae4ebabf3c84f6ca8ea9cf538197ca1c/bro


From vern at icir.org  Sun Nov  4 23:13:06 2012
From: vern at icir.org (Vern Paxson)
Date: Sun, 04 Nov 2012 23:13:06 -0800
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <C5948622-E8DD-49AA-8280-CE51312A9F8B@icir.org> (Fri,
	02 Nov 2012 15:09:17 EDT).
Message-ID: <20121105071306.58F902C4002@rock.ICSI.Berkeley.EDU>

> Continuing the discussion from the group call, I think that we're starting
> to converge on adding a "policy" keyword to the language to represent
> something between a function and an event.

Hmmmm.  This is sounding weird.  I think it's telling that you're heading
towards introducing a concept that isn't (in terms of the name that comes
to mind for it) about anything relating to language semantics, but rather
to intended use, i.e., that this mechanism is for honing policy.  That's
often a sign that it's time to stop & rethink things.

Let's start with what's deficient about the current style.  The overarching
problem is it's hard to understand/explain, correct?  How much of this is
the funky [$pred...] syntax, vs. that we've wound up adding a bunch of
separate hooks into the framework so it's hard to know which hook to use,
and/or it's hard to understand what a given collection of hooks does?

I feel like we need to better determine just what problem we want/need to
solve in order to then think about how to coherently provide better support
for it.

		Vern

From noreply at bro-ids.org  Mon Nov  5 00:00:13 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Mon, 5 Nov 2012 00:00:13 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211050800.qA580DDH024020@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |   915 [1] | seth       | robin      | Normal | topic/seth/modbus-merge [2]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
        bro-aux | 75a841f    | Daniel Thayer   | 2012-11-01 | Use #ifdef for Linux-specific code [3]
            bro | e020e03    | Seth Hall       | 2012-11-01 | Script in base for detecting cases of checksum offloading. [4]



 [1]       #915: http://tracker.bro-ids.org/bro/ticket/915
 [2] modbus-merge: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/modbus-merge
 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/75a841fd7cfa32916ad5f7e80511fec2f119e058/bro-aux
 [4] fastpath: http://tracker.bro-ids.org/bro/changeset/e020e034ae4ebabf3c84f6ca8ea9cf538197ca1c/bro


From vladg at cmu.edu  Mon Nov  5 07:46:29 2012
From: vladg at cmu.edu (Vlad Grigorescu)
Date: Mon, 5 Nov 2012 15:46:29 +0000
Subject: [Bro-Dev] Question about a BroString Constructor
Message-ID: <1202BE242E080642B0CD0AD0A03E85525D3692@PGH-MSGMB-03.andrew.ad.cmu.edu>

Hi,

I recently shot myself in the foot using the following BroString constructor:

> BroString::BroString(int arg_final_NUL, byte_vec str, int arg_n)
>         {
>         b = str;
>         n = arg_n;
>         final_NUL = arg_final_NUL;
>         use_free_to_delete = 0;
>         }

I didn't realize that when creating a new BroString, the byte_vec I pass in doesn't get copied. I was using the byte_vec I was passing in further down, and couldn't understand why BroString wasn't acting properly. (byte_vec is defined as u_char*, not as a constant).

BroString::Set uses memcpy to copy the string. The other constructors all seem to use BroString::Set.

> BroString::BroString(const u_char* str, int arg_n, int add_NUL)
>       ...
>        Set(str, arg_n, add_NUL);
>
> BroString::BroString(const char* str)
>       ...
>         Set(str);
>
> BroString::BroString(const string &str)
>       ...
>        Set(str);
>

The behavior is inconsistent at best. Personally, when I create a new string object, I'd expect the data to be copied, but I could see a few special cases where you might want to avoid that for performance reasons.

Any thoughts on this? Is this intended behavior? I suspect that any changes to BroString would require a lot of changes downstream.

  --Vlad

From seth at icir.org  Mon Nov  5 08:23:52 2012
From: seth at icir.org (Seth Hall)
Date: Mon, 5 Nov 2012 11:23:52 -0500
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <20121105071306.58F902C4002@rock.ICSI.Berkeley.EDU>
References: <20121105071306.58F902C4002@rock.ICSI.Berkeley.EDU>
Message-ID: <2561128D-EC3E-4716-866A-DAD3732BB0FC@icir.org>


On Nov 5, 2012, at 2:13 AM, Vern Paxson <vern at icir.org> wrote:

> Let's start with what's deficient about the current style.  The overarching
> problem is it's hard to understand/explain, correct?

Yep.  

>  How much of this is
> the funky [$pred...] syntax, vs. that we've wound up adding a bunch of
> separate hooks into the framework so it's hard to know which hook to use,
> and/or it's hard to understand what a given collection of hooks does?

Personally, it's mostly the syntax.  There are also some cases where it's hard to express what you want in a single policy item too.  Some of this could probably be fixed within the notice policy now, but then we'd essentially be turning the notice policy into a set of functions which starts to look suspiciously similar to events being executed immediately without the familiar syntax of events.

> I feel like we need to better determine just what problem we want/need to
> solve in order to then think about how to coherently provide better support
> for it.


I suspect that the reason the notice policy mechanism was first created was due to need for immediately evaluated events.  The event handling mechanism feels very natural in Bro after using it for a while and the existing notice policy ends up feeling weird because you have to do a lot of uncommon and unfamiliar syntax.  I'm not completely tied to the "policy" keyword, but does it seem unreasonable to have something like this immediately executed event that we've been discussing?

As

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Mon Nov  5 08:28:09 2012
From: seth at icir.org (Seth Hall)
Date: Mon, 5 Nov 2012 11:28:09 -0500
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <2561128D-EC3E-4716-866A-DAD3732BB0FC@icir.org>
References: <20121105071306.58F902C4002@rock.ICSI.Berkeley.EDU>
	<2561128D-EC3E-4716-866A-DAD3732BB0FC@icir.org>
Message-ID: <43CFE561-1F51-410B-868F-F12912B53EAB@icir.org>


On Nov 5, 2012, at 11:23 AM, Seth Hall <seth at icir.org> wrote:

> As


I think I was done.  I'm not sure what the stray "As" is there for. :)

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/


From vern at icir.org  Mon Nov  5 08:41:10 2012
From: vern at icir.org (Vern Paxson)
Date: Mon, 05 Nov 2012 08:41:10 -0800
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <2561128D-EC3E-4716-866A-DAD3732BB0FC@icir.org> (Mon,
	05 Nov 2012 11:23:52 EST).
Message-ID: <20121105164110.294232C4002@rock.ICSI.Berkeley.EDU>

> I'm not completely tied to the "policy" keyword, but does it seem
> unreasonable to have something like this immediately executed event that
> we've been discussing?

Yeah, it seems somewhat strange and confusing.

If we back up, it seems to me there are several things the framework
tries to achieve.

	(1) Associate predicates with different possible cases.  This
	    is the "n$note == Whatever" clause that's currently included
	    in basically every policy element.

	(2) A predicate that applies logic when that case holds to yield
	    a classification decision.

	(3) Priorities to decide which predicate "wins" when multiple ones
	    of them match the case.

	(4) The ability for an early winner to short-circuit processing by
	    others that match the case, either for efficiency or to prevent
	    their side-effects.

What I wonder about is having a starting point of dealing with (1) differently.
Conceptually, one would like to say:

	switch n$note {

	case Notice::Sneakiness:
		if ( pred1() ) ...
		if ( pred2() ) ...

	}

but the problem is that we want refinement of the code inside each
case, without having to edit the overall code block.

I know that languages going as far back as CLOS had mechanisms for
doing this sort of refinement.  Not being a CLOS hacker, this is just
from ancient memory, but I believe the CLOS equivalent for what we
want would be:

	function policy_action(n: Notice::Info &case n$note==Notice::Sneakiness,
				other_arguments_here ...)
		{
		...
		}

That seems like too much verbiage.  But it has me wondering whether there
might be some sort of prioritized case construct that works better than
"match ... using".  Ideally this would be a language construct that nicely
applies in other switch-like contexts, too.

		Vern

From jsiwek at illinois.edu  Mon Nov  5 08:43:05 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Mon, 5 Nov 2012 16:43:05 +0000
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <C5948622-E8DD-49AA-8280-CE51312A9F8B@icir.org>
References: <4E789D88-ECE3-492A-AD74-9234C10B5A4F@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED3EBC@CITESMBX2.ad.uillinois.edu>
	<69CAE6B4-81E9-49B8-8E38-F7AC30A6FBD6@icir.org>
	<DAF7C752-1FFF-4134-BB18-D73635EA0EF9@icir.org>
	<9957A0E9-A872-4EEE-9A8A-EA37B3FD0E70@icir.org>
	<D9172BB4-B485-4972-A337-68E52262E5E9@icir.org>
	<20121102163859.GK67486@icir.org>
	<C5948622-E8DD-49AA-8280-CE51312A9F8B@icir.org>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED4B7D@CITESMBX2.ad.uillinois.edu>

>  2. Adding a "break"-like keyword to halt policy execution (only valid in policy handlers).
> 
>  3. No return value(s) when calling policy handlers.

Just an alternative idea I had -- maybe they could be required to return a boolean value that's used internally to determine whether to keep traversing the vector of handler bodies.  Then you don't need to add a new keyword for the halt/break functionality.

    Jon

From jsiwek at illinois.edu  Mon Nov  5 08:57:55 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Mon, 5 Nov 2012 16:57:55 +0000
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <20121105071306.58F902C4002@rock.ICSI.Berkeley.EDU>
References: <20121105071306.58F902C4002@rock.ICSI.Berkeley.EDU>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED4C75@CITESMBX2.ad.uillinois.edu>

> I feel like we need to better determine just what problem we want/need to
> solve in order to then think about how to coherently provide better support
> for it.

I think the construct we need to solve the problem already exists purely in the script-layer:

    type TriggerFunc: function(n: Notice::Info): bool;
    global Notice::policy: vector of set[TriggerFunc];

I think that the issue is it being cumbersome to manipulate/redef?  So we're working around that with the new proposal, but maybe we can do more to make redefining something like that syntactically clean and easy?

    Jon

From vern at icir.org  Mon Nov  5 09:13:08 2012
From: vern at icir.org (Vern Paxson)
Date: Mon, 05 Nov 2012 09:13:08 -0800
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED4C75@CITESMBX2.ad.uillinois.edu>
	(Mon, 05 Nov 2012 16:57:55 GMT).
Message-ID: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>

> I think the construct we need to solve the problem already exists purely in the script-layer:
> 
>     type TriggerFunc: function(n: Notice::Info): bool;
>     global Notice::policy: vector of set[TriggerFunc];

That doesn't have the attribute that I think we likely want, namely an
explicit "switch" on the particular notice type.

Sepaking of which, maybe we don't want an exact split-case-by-case but
rather something a bit broader.  Here I'm thinking of policies that I'd
like to apply to any Notice of n different types; it's important to not
have to replicate that code, but instead just list the n types and the
associated predicate.

		Vern

From jsiwek at illinois.edu  Mon Nov  5 09:24:39 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Mon, 5 Nov 2012 17:24:39 +0000
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>
References: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED4D85@CITESMBX2.ad.uillinois.edu>

>> I think the construct we need to solve the problem already exists purely in the script-layer:
>> 
>>    type TriggerFunc: function(n: Notice::Info): bool;
>>    global Notice::policy: vector of set[TriggerFunc];
> 
> That doesn't have the attribute that I think we likely want, namely an
> explicit "switch" on the particular notice type.

Right, your switch/case idea reminded me of that optimization.  Would a table work for that?  So revising the example:

    type TriggerType: enum;
    type TriggerFunc: function(n: Notice::Info): bool;
    type PolicyHandlers: vector of set[TriggerFunc];
    global Notice::policy: table[TriggerType] of PolicyHandlers &redef;

Again, the issue probably being the cumbersome syntax of manipulation/redefinitions (if it's even possible right now) that's expected to be done by the user.

> Sepaking of which, maybe we don't want an exact split-case-by-case but
> rather something a bit broader.  Here I'm thinking of policies that I'd
> like to apply to any Notice of n different types; it's important to not
> have to replicate that code, but instead just list the n types and the
> associated predicate.

Think the above example supports that if the common code is just implemented as a TriggerFunc and then inserted in to the PolicyHandlers for the desired TriggerTypes.

    Jon

From seth at icir.org  Mon Nov  5 10:35:56 2012
From: seth at icir.org (Seth Hall)
Date: Mon, 5 Nov 2012 13:35:56 -0500
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>
References: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>
Message-ID: <A7FDD5F2-1303-4854-A0DE-EC580A0ED7BD@icir.org>


On Nov 5, 2012, at 12:13 PM, Vern Paxson <vern at icir.org> wrote:

>> I think the construct we need to solve the problem already exists purely in the script-layer:
>> 
>>    type TriggerFunc: function(n: Notice::Info): bool;
>>    global Notice::policy: vector of set[TriggerFunc];
> 
> That doesn't have the attribute that I think we likely want, namely an
> explicit "switch" on the particular notice type.
> 
> Sepaking of which, maybe we don't want an exact split-case-by-case but
> rather something a bit broader.  Here I'm thinking of policies that I'd
> like to apply to any Notice of n different types; it's important to not
> have to replicate that code, but instead just list the n types and the
> associated predicate.

This is getting at the heart of what I think I want.  Being able to define a notice type or notice types and apply an action is the end result, but how you implement the support for that in the script layer is currently too complicated.  Optimally, I think it would be nice if people never had to use the action notice policy and that was why I wrote those "cheater" variables? Notice::ignored_types, Notice::emailed_types, Notice::alarmed_types, etc.  The problem is, is that I think that no matter what "cheater" variables we create people will always need to be more expressive (that's been my experience with the few variables I already created).  

What I'm pressing for is that make that next step more obvious because it would be a familiar Bro experience.  Also, the implementation of that support in the script layer would be a lot clearer.  Right now, there is a lot of code just to support the policy mechanism in the notice framework.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From robin at icir.org  Mon Nov  5 13:41:03 2012
From: robin at icir.org (Robin Sommer)
Date: Mon, 5 Nov 2012 13:41:03 -0800
Subject: [Bro-Dev] DNS TXT Queries and the Cache File
In-Reply-To: <1202BE242E080642B0CD0AD0A03E85525B73DA@PGH-MSGMB-03.andrew.ad.cmu.edu>
References: <1202BE242E080642B0CD0AD0A03E855234C4F1@PGH-MSGMB-03.andrew.ad.cmu.edu>
	<503F3AB2.4040900@illinois.edu>
	<352_1346341138_q7UFcvg7017598_20120830153851.GA89651@icir.org>
	<1202BE242E080642B0CD0AD0A03E8552593265@PGH-MSGMB-03.andrew.ad.cmu.edu>
	<2218_1351802376_qA1KdY5W005470_20121101203932.GE52215@icir.org>
	<1202BE242E080642B0CD0AD0A03E85525B73DA@PGH-MSGMB-03.andrew.ad.cmu.edu>
Message-ID: <20121105214103.GV47020@icir.org>


On Sat, Nov 03, 2012 at 22:01 +0000, you wrote:

> I tested master, and it wasn't working properly for me. I traced the
> problem to the simplification of DNS_Mgr::AsyncLookupName.

Ah, I wasn't looking right it seems. With that, joining the functions
actually doesn't look like a good change, so I'm going to move this
back to your original implementation. Sorry about the confusion.

> on how we'd go about writing some tests for the TXT queries - this was
> the one case where being able to load the cache from a file did seem
> useful.

Yeah, but even that would only test a part of the chain, not the
requests themselves.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From robin at icir.org  Mon Nov  5 13:52:30 2012
From: robin at icir.org (Robin Sommer)
Date: Mon, 5 Nov 2012 13:52:30 -0800
Subject: [Bro-Dev] Question about a BroString Constructor
In-Reply-To: <1202BE242E080642B0CD0AD0A03E85525D3692@PGH-MSGMB-03.andrew.ad.cmu.edu>
References: <1202BE242E080642B0CD0AD0A03E85525D3692@PGH-MSGMB-03.andrew.ad.cmu.edu>
Message-ID: <20121105215230.GW47020@icir.org>


On Mon, Nov 05, 2012 at 15:46 +0000, you wrote:

> Any thoughts on this? Is this intended behavior? I suspect that any
> changes to BroString would require a lot of changes downstream.

Agreed, it's inconsistent and error-prone. Worse, we have more such
cases in other classes as well; sometimes methods take ownership and
sometimes they don't. Unfortunately, as you say, changing these things
is hard because of the ripple effect it can have downstream, including
the potential to introduce very subtle bugs.

What we can do easily though is document these cases: just comments 
about ownership would already help a lot. I'm going ahead and will add
these for the BroString ctors. Feel free to submit patches for other
cases you run across.

(Longer-term, perhaps there's somethign to do with automatic code
analysis/refactoring tools; here, for example, a tool could at least
identify all uses of that particular method, and maybe even report
what an ownership change might break down the road).

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Mon Nov  5 17:11:08 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 06 Nov 2012 01:11:08 -0000
Subject: [Bro-Dev] #915: topic/seth/modbus-merge
In-Reply-To: <046.7fd5131d44e94e3cc695acf8bde185db@tracker.bro-ids.org>
References: <046.7fd5131d44e94e3cc695acf8bde185db@tracker.bro-ids.org>
Message-ID: <061.3787ade094d51afdb1221a866fa2163b@tracker.bro-ids.org>

#915: topic/seth/modbus-merge
----------------------------+------------------------
  Reporter:  seth           |      Owner:  robin
      Type:  Merge Request  |     Status:  closed
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:  fixed          |   Keywords:
----------------------------+------------------------
Changes (by robin):

 * status:  new => closed
 * resolution:   => fixed


Comment:

 In [86ce564107f3536abb866e2d1934b67c9f5839c7/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="86ce564107f3536abb866e2d1934b67c9f5839c7"
 Merge remote-tracking branch 'remotes/origin/topic/seth/modbus-merge'

 * remotes/origin/topic/seth/modbus-merge:
   Small modbus documentation update and tiny refactoring.
   Final touches to modbus analyzer for now.
   Major revisions to Modbus analyzer support (not quite done yet).
   put some make-up on Modbus analyser
   Modbus analyser, added support: FC=20,21
   Modbus analyzer,added support: FC=1,2,15,24
   Modbus analyzer, current support: FC=3,4,5,6,7,16,22,23

 Closes #915.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/915#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Nov  5 17:11:08 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 06 Nov 2012 01:11:08 -0000
Subject: [Bro-Dev] #914: topic/seth/intel-framework
In-Reply-To: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org>
References: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org>
Message-ID: <061.6841816ef2c8fde8aabc365737e1932c@tracker.bro-ids.org>

#914: topic/seth/intel-framework
----------------------+------------------------
  Reporter:  seth     |      Owner:  seth
      Type:  Problem  |     Status:  closed
  Priority:  Normal   |  Milestone:  Bro2.2
 Component:  Bro      |    Version:  git/master
Resolution:  fixed    |   Keywords:
----------------------+------------------------
Changes (by robin):

 * status:  reopened => closed
 * resolution:   => fixed


Comment:

 In [a40b00d4abd67ea92708c828965df10aee177e23/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="a40b00d4abd67ea92708c828965df10aee177e23"
 Merge branch 'topic/robin/intel-framework-merge'

 * topic/robin/intel-framework-merge: (22 commits)
   Fixing tests after intel-framework merge.
   Extracting URLs from message bodies over SMTP and sending them to Intel
 framework.
   Small comment updates in the Intel framework CIF support.
   Intelligence framework documentation first draft.
   Only the manager tries to read files with the input framework now.
   Initial support for Bro's Intel framework with the Collective
 Intelligence Framework.
   Initial API for Intel framework is complete.
   Fixed an issue with cluster data distribution.
   Updating some intel framework test baselines.
   Reworked cluster intelligence data distribution mechanism and fixed
 tests.
   Lots more intelligence checking in SMTP traffic.
   Added intelligence check for "Received" path checking and a bit of
 reshuffling.
   Added sources to the intel log.
   Fixing a problem with intel distribution on clusters.
   Updated intel framework test to include matching.
   Restructuring the scripts that feed data into the intel framework
 slightly.
   One test for cluster transparency of the intel framework.
   Fixed a cluster support bug.
   Intelligence framework checkpoint
   Major updates to fix the Intel framework API.
   ...

 Closes #914.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/914#comment:16>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Tue Nov  6 07:44:10 2012
From: robin at icir.org (Robin Sommer)
Date: Tue, 6 Nov 2012 07:44:10 -0800
Subject: [Bro-Dev] [Fwd] [Bro-Commits-Internal] UnitTests - Build # 454 -
	Still	Failing!
Message-ID: <20121106154410.GH57057@icir.org>

Must be the recent DNS changes. From a quick look I don't see where
it's leaking but my local perftools unfortunately doesn't report it.

Can we get the output of that pprof command?

Robin

----- Forwarded message from jenkins at brotestbed.ncsa.illinois.edu -----

Date: Mon, 5 Nov 2012 20:14:32 -0600 (CST)
From: jenkins at brotestbed.ncsa.illinois.edu
To: bro-commits-internal at bro-ids.org
Subject: [Bro-Commits-Internal] UnitTests - Build # 454 - Still Failing!
Message-ID: <1781954848.7.1352168072776.JavaMail.jenkins at brotestbed.ncsa.illinois.edu>
Reply-To: bro-commits-internal at bro-ids.org

[...]

core.leaks.dns ... failed
  % 'HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local bro -m -r $TRACES/wikipedia.trace /home/jenkins/workspace/UnitTests/bro/testing/btest/.tmp/core.leaks.dns/dns.bro' failed unexpectedly (exit code -6)
  % cat .stderr
  WARNING: Perftools heap leak checker is active -- Performance may suffer
  Leak check net_run detected leaks of 233 bytes in 4 objects
  The 2 largest leaks:
  Leak of 208 bytes in 2 objects allocated from:
  @ 6ae0b9
  @ 6b0f76
  @ 6b0ba3
  @ 77770f
  @ 6675c0
  @ 39d8e1ecdd
  Leak of 25 bytes in 2 objects allocated from:
  @ 669df9
  @ 6ab75b
  @ 6ae0d1
  @ 6b0f76
  @ 6b0ba3
  @ 77770f
  @ 6675c0
  @ 39d8e1ecdd
  
  
  If the preceding stack traces are not enough to find the leaks, try running THIS shell command:
  
  pprof bro "./bro.31993.net_run-end.heap" --inuse_objects --lines --heapcheck  --edgefraction=1e-10 --nodefraction=1e-10 --gv
  
  If you are still puzzled about why the leaks are there, try rerunning this program with HEAP_CHECK_TEST_POINTER_ALIGNMENT=1 and/or with HEAP_CHECK_MAX_POINTER_OFFSET=-1
  If the leak report occurs in a small fraction of runs, try running with TCMALLOC_MAX_FREE_QUEUE_SIZE of few hundred MB or with TCMALLOC_RECLAIM_MEMORY=false, it might help find leaks more repeatably
  Memory leaks - aborting.

core.leaks.remote ... ok
core.leaks.ayiya ... ok
1 of 13 tests failed
Build step 'Execute shell' marked build as failure
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

_______________________________________________
bro-commits-internal mailing list
bro-commits-internal at bro-ids.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits-internal


----- End forwarded message -----


-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From jsiwek at illinois.edu  Tue Nov  6 13:01:23 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Tue, 6 Nov 2012 21:01:23 +0000
Subject: [Bro-Dev] [Fwd] [Bro-Commits-Internal] UnitTests - Build # 454
 -	Still	Failing!
In-Reply-To: <20121106154410.GH57057@icir.org>
References: <20121106154410.GH57057@icir.org>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED6016@CITESMBX2.ad.uillinois.edu>


> Must be the recent DNS changes. From a quick look I don't see where
> it's leaking but my local perftools unfortunately doesn't report it.
> 
> Can we get the output of that pprof command?

That particular one is gone now, but here's another:

(pprof) top
Total: 4 objects
       2  50.0%  50.0%        4 100.0% DNS_Mgr::AddResult /home/jsiwek/bro/src/DNS_Mgr.cc:708
       2  50.0% 100.0%        2  50.0% copy_string /home/jsiwek/bro/src/util.cc:154
       0   0.0% 100.0%        2  50.0% DNS_Mapping::DNS_Mapping /home/jsiwek/bro/src/DNS_Mgr.cc:174
       0   0.0% 100.0%        4 100.0% DNS_Mgr::DoProcess /home/jsiwek/bro/src/DNS_Mgr.cc:1319
       0   0.0% 100.0%        4 100.0% DNS_Mgr::Process /home/jsiwek/bro/src/DNS_Mgr.cc:1269
       0   0.0% 100.0%        4 100.0% __libc_start_main ??:0
       0   0.0% 100.0%        4 100.0% main /home/jsiwek/bro/src/main.cc:1073
       0   0.0% 100.0%        4 100.0% net_run /home/jsiwek/bro/src/Net.cc:446

It's reproducible for me if need to interactively run it yourself, ssh in to brotestbed and you can compile your own version to test with.

    Jon 

> 
> Robin
> 
> ----- Forwarded message from jenkins at brotestbed.ncsa.illinois.edu -----
> 
> Date: Mon, 5 Nov 2012 20:14:32 -0600 (CST)
> From: jenkins at brotestbed.ncsa.illinois.edu
> To: bro-commits-internal at bro-ids.org
> Subject: [Bro-Commits-Internal] UnitTests - Build # 454 - Still Failing!
> Message-ID: <1781954848.7.1352168072776.JavaMail.jenkins at brotestbed.ncsa.illinois.edu>
> Reply-To: bro-commits-internal at bro-ids.org
> 
> [...]
> 
> core.leaks.dns ... failed
>  % 'HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local bro -m -r $TRACES/wikipedia.trace /home/jenkins/workspace/UnitTests/bro/testing/btest/.tmp/core.leaks.dns/dns.bro' failed unexpectedly (exit code -6)
>  % cat .stderr
>  WARNING: Perftools heap leak checker is active -- Performance may suffer
>  Leak check net_run detected leaks of 233 bytes in 4 objects
>  The 2 largest leaks:
>  Leak of 208 bytes in 2 objects allocated from:
>  @ 6ae0b9
>  @ 6b0f76
>  @ 6b0ba3
>  @ 77770f
>  @ 6675c0
>  @ 39d8e1ecdd
>  Leak of 25 bytes in 2 objects allocated from:
>  @ 669df9
>  @ 6ab75b
>  @ 6ae0d1
>  @ 6b0f76
>  @ 6b0ba3
>  @ 77770f
>  @ 6675c0
>  @ 39d8e1ecdd
> 
> 
>  If the preceding stack traces are not enough to find the leaks, try running THIS shell command:
> 
>  pprof bro "./bro.31993.net_run-end.heap" --inuse_objects --lines --heapcheck  --edgefraction=1e-10 --nodefraction=1e-10 --gv
> 
>  If you are still puzzled about why the leaks are there, try rerunning this program with HEAP_CHECK_TEST_POINTER_ALIGNMENT=1 and/or with HEAP_CHECK_MAX_POINTER_OFFSET=-1
>  If the leak report occurs in a small fraction of runs, try running with TCMALLOC_MAX_FREE_QUEUE_SIZE of few hundred MB or with TCMALLOC_RECLAIM_MEMORY=false, it might help find leaks more repeatably
>  Memory leaks - aborting.
> 
> core.leaks.remote ... ok
> core.leaks.ayiya ... ok
> 1 of 13 tests failed
> Build step 'Execute shell' marked build as failure
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
> 
> _______________________________________________
> bro-commits-internal mailing list
> bro-commits-internal at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits-internal
> 
> 
> ----- End forwarded message -----
> 
> 
> -- 
> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
> ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
> 



From seth at icir.org  Wed Nov  7 06:24:01 2012
From: seth at icir.org (Seth Hall)
Date: Wed, 7 Nov 2012 09:24:01 -0500
Subject: [Bro-Dev] Events and communication
Message-ID: <70E5F8F8-C9BC-4054-9E3B-E2273CA8E0A5@icir.org>

This question is probably mostly for Robin. :)

Is there a particular reason that only exported events can be sent to remote Bro's?  I would like to recommend that we change that to any event is available for being sent remotely since I have some events in the cluster support for the metrics framework that need to be sent between Bro's, but are definitely not something that should be in the public API for the metrics framework.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From jsiwek at illinois.edu  Wed Nov  7 10:34:44 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Wed, 7 Nov 2012 18:34:44 +0000
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <A7FDD5F2-1303-4854-A0DE-EC580A0ED7BD@icir.org>
References: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>
	<A7FDD5F2-1303-4854-A0DE-EC580A0ED7BD@icir.org>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED6949@CITESMBX2.ad.uillinois.edu>

Here's an idea that combines a lot of the previous discussion -- how about we implement the capabilities of a "switch" statement, except we use a new type of function to do it.  So there would then be function types designated by keywords "function", "event", and "switch".  The "switch" would be like a "function" in that it can be called synchronously and doesn't have to be scheduled like an "event", but it's also like an "event" in that you can implement multiple bodies for the same "switch" identifier and designate &priority for each.  Additionally, a new attribute called &case can be used with "switch" functions, and the argument to it is evaluated each time the function is called to determine whether or not to actually execute a given body.  Also, if the body of a "switch" function returns as a result of a "break" statement, then that aborts the execution of the "switch" function and no more bodies get processed.

So an example of how it works in code form would be...

type TriggerType: enum {
    Good,
    Bad,
    Ugly,
};

type MyInfo: record {
    a: count;
    trigger: TriggerType;
};

# this declaration of the switch function would be optional (same as how functions work now)
global trigger: switch(n: MyInfo);

switch trigger(n: MyInfo) &case=(n$trigger==Bad) &priority=-5
    {
    this_stuff_never_happens();
    }

switch trigger(n: MyInfo) &case=(n$trigger==Bad) &priority=10
    {
    do_stuff();
    break;
    }

switch trigger(n: MyInfo) &case=(n$trigger==Ugly) &priority=1
    {
    do_other_stuff();
    }

event bro_init()
    {
    local rec: MyInfo = [$a=13, $trigger=Bad]);
    trigger(rec);
    # rec can be inspected here for meaningful modifications
    }

That should give everything we want:

(1) case expressions allow conditional application of logic
(2) logic can be prioritized
(3) logic can be short-circuited
(4) easy to understand how a user can refine/insert new logic for a priority/condition of their choosing
(5) its use should be generalizable and not specific to "policy" evaluation

One downside might be that it's not the typical "switch" people could be familiar with, but I don't think it's that confusing to take previous knowledge of how "switch" statements work when trying to understand how to use "switch" functions.  I also don't think this would preclude the later implementation of "switch" in the usual statement form, it's just that implementing it using functions satisfies the "refinement" requirement more easily.

    Jon

From jsiwek at illinois.edu  Wed Nov  7 13:11:30 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Wed, 7 Nov 2012 21:11:30 +0000
Subject: [Bro-Dev] [Fwd] [Bro-Commits-Internal] UnitTests - Build # 454
 -	Still	Failing!
In-Reply-To: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED6016@CITESMBX2.ad.uillinois.edu>
References: <20121106154410.GH57057@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED6016@CITESMBX2.ad.uillinois.edu>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED6D68@CITESMBX2.ad.uillinois.edu>

I can look in to fixing this now unless you, or someone, is already.

    Jon

On Nov 6, 2012, at 3:01 PM, "Siwek, Jonathan Luke" <jsiwek at illinois.edu> wrote:

> 
>> Must be the recent DNS changes. From a quick look I don't see where
>> it's leaking but my local perftools unfortunately doesn't report it.
>> 
>> Can we get the output of that pprof command?
> 
> That particular one is gone now, but here's another:
> 
> (pprof) top
> Total: 4 objects
>       2  50.0%  50.0%        4 100.0% DNS_Mgr::AddResult /home/jsiwek/bro/src/DNS_Mgr.cc:708
>       2  50.0% 100.0%        2  50.0% copy_string /home/jsiwek/bro/src/util.cc:154
>       0   0.0% 100.0%        2  50.0% DNS_Mapping::DNS_Mapping /home/jsiwek/bro/src/DNS_Mgr.cc:174
>       0   0.0% 100.0%        4 100.0% DNS_Mgr::DoProcess /home/jsiwek/bro/src/DNS_Mgr.cc:1319
>       0   0.0% 100.0%        4 100.0% DNS_Mgr::Process /home/jsiwek/bro/src/DNS_Mgr.cc:1269
>       0   0.0% 100.0%        4 100.0% __libc_start_main ??:0
>       0   0.0% 100.0%        4 100.0% main /home/jsiwek/bro/src/main.cc:1073
>       0   0.0% 100.0%        4 100.0% net_run /home/jsiwek/bro/src/Net.cc:446
> 
> It's reproducible for me if need to interactively run it yourself, ssh in to brotestbed and you can compile your own version to test with.
> 
>    Jon 
> 
>> 
>> Robin
>> 
>> ----- Forwarded message from jenkins at brotestbed.ncsa.illinois.edu -----
>> 
>> Date: Mon, 5 Nov 2012 20:14:32 -0600 (CST)
>> From: jenkins at brotestbed.ncsa.illinois.edu
>> To: bro-commits-internal at bro-ids.org
>> Subject: [Bro-Commits-Internal] UnitTests - Build # 454 - Still Failing!
>> Message-ID: <1781954848.7.1352168072776.JavaMail.jenkins at brotestbed.ncsa.illinois.edu>
>> Reply-To: bro-commits-internal at bro-ids.org
>> 
>> [...]
>> 
>> core.leaks.dns ... failed
>> % 'HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local bro -m -r $TRACES/wikipedia.trace /home/jenkins/workspace/UnitTests/bro/testing/btest/.tmp/core.leaks.dns/dns.bro' failed unexpectedly (exit code -6)
>> % cat .stderr
>> WARNING: Perftools heap leak checker is active -- Performance may suffer
>> Leak check net_run detected leaks of 233 bytes in 4 objects
>> The 2 largest leaks:
>> Leak of 208 bytes in 2 objects allocated from:
>> @ 6ae0b9
>> @ 6b0f76
>> @ 6b0ba3
>> @ 77770f
>> @ 6675c0
>> @ 39d8e1ecdd
>> Leak of 25 bytes in 2 objects allocated from:
>> @ 669df9
>> @ 6ab75b
>> @ 6ae0d1
>> @ 6b0f76
>> @ 6b0ba3
>> @ 77770f
>> @ 6675c0
>> @ 39d8e1ecdd
>> 
>> 
>> If the preceding stack traces are not enough to find the leaks, try running THIS shell command:
>> 
>> pprof bro "./bro.31993.net_run-end.heap" --inuse_objects --lines --heapcheck  --edgefraction=1e-10 --nodefraction=1e-10 --gv
>> 
>> If you are still puzzled about why the leaks are there, try rerunning this program with HEAP_CHECK_TEST_POINTER_ALIGNMENT=1 and/or with HEAP_CHECK_MAX_POINTER_OFFSET=-1
>> If the leak report occurs in a small fraction of runs, try running with TCMALLOC_MAX_FREE_QUEUE_SIZE of few hundred MB or with TCMALLOC_RECLAIM_MEMORY=false, it might help find leaks more repeatably
>> Memory leaks - aborting.
>> 
>> core.leaks.remote ... ok
>> core.leaks.ayiya ... ok
>> 1 of 13 tests failed
>> Build step 'Execute shell' marked build as failure
>> Recording test results
>> Email was triggered for: Failure
>> Sending email for trigger: Failure
>> 
>> _______________________________________________
>> bro-commits-internal mailing list
>> bro-commits-internal at bro-ids.org
>> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits-internal
>> 
>> 
>> ----- End forwarded message -----
>> 
>> 
>> -- 
>> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
>> ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
>> _______________________________________________
>> bro-dev mailing list
>> bro-dev at bro-ids.org
>> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>> 
> 
> 
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
> 



From robin at icir.org  Wed Nov  7 13:14:45 2012
From: robin at icir.org (Robin Sommer)
Date: Wed, 7 Nov 2012 13:14:45 -0800
Subject: [Bro-Dev] [Fwd] [Bro-Commits-Internal] UnitTests - Build #
	454	- Still Failing!
In-Reply-To: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED6D68@CITESMBX2.ad.uillinois.edu>
References: <20121106154410.GH57057@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED6016@CITESMBX2.ad.uillinois.edu>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED6D68@CITESMBX2.ad.uillinois.edu>
Message-ID: <20121107211444.GF90927@icir.org>


On Wed, Nov 07, 2012 at 21:11 +0000, you wrote:

> I can look in to fixing this now unless you, or someone, is already.

I'm not yet, if you're up for it, appreciated!

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From seth at icir.org  Wed Nov  7 14:04:28 2012
From: seth at icir.org (Seth Hall)
Date: Wed, 7 Nov 2012 17:04:28 -0500
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED6949@CITESMBX2.ad.uillinois.edu>
References: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>
	<A7FDD5F2-1303-4854-A0DE-EC580A0ED7BD@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED6949@CITESMBX2.ad.uillinois.edu>
Message-ID: <16C508BA-1006-4EAD-9EFC-EE92A5C638B2@icir.org>


On Nov 7, 2012, at 1:34 PM, "Siwek, Jonathan Luke" <jsiwek at illinois.edu> wrote:

> switch trigger(n: MyInfo) &case=(n$trigger==Bad) &priority=-5
>    {
>    this_stuff_never_happens();
>    }


Jon and I chatted about this for a while and I *really* don't like the &case attribute and I don't like the switch keyword much either.  I think that making this switch-like is confusing because it's not really that much like a switch, it's more like a ladder of function bodies (I don't know of a comparison to make with a feature of another language).  I think that the &case attribute would be confusing too because if you had larger conditions it would start to get unmanageable quickly and then you fall back to doing a block condition at the top of the code block which is already a pretty well established convention in Bro scripts, like this?

if ( n$note != Some_Notice )
	return;

Instead of using the "policy" keyword, how does everyone feel about "rule"?  It would probably be confusing to people coming from Snort and Suricata, but I'm not overly concerned about that.  To try and make my case, I got notice policies from a couple of sites.  These are real world, actively used notice policies.

Current style at site 1?

redef Notice::policy += {
      [$action = Notice::ACTION_BLOCK,
       $pred(n: Notice::Info) = { return n$note == SSH::Password_Guessing && n?$src && !Site::is_local_addr(n$src) && !Site::is_private_addr(n$src) && !(n$src in external_ssh_scanners_whitelist || n$src in planetlab_hosts); } ]
};

redef Notice::policy += {
      [$action = Notice::ACTION_EMAIL,
       $pred(n: Notice::Info) = { return n$note == SSH::Password_Guessing && n?$src && ( Site::is_local_addr(n$src) || Site::is_private_addr(n$src) ) && !(n$src in internal_ssh_scanners_whitelist || n$src in planetlab_hosts); } ]
};

redef Notice::policy += {
      [$action = Notice::ACTION_EMAIL,
       $pred(n: Notice::Info) = { return n$note == HTTP::Malware_Hash_Registry_Match; } ]
};

redef Notice::policy += {
      [$action = Notice::ACTION_EMAIL,
       $pred(n: Notice::Info) = { return n$note == FTP::Site_Exec_Success; } ]
};

redef Notice::policy += {
      [$action = Notice::ACTION_EMAIL,
       $pred(n: Notice::Info) = { return n$note == SSH::Watched_Country_Login; } ]
};

redef Notice::policy += {
      [$action = Notice::ACTION_EMAIL, $suppress_for = 1 day,
       $pred(n: Notice::Info) = { return n$note == Software::Vulnerable_Java; } ]
};	


New proposed style at site 1?

rule Notice::policy(n: Notice::Info)
	{
	if ( n$note == SSH::Password_Guessing && 
	     n?$src && 
	     !Site::is_local_addr(n$src) &&
	     !Site::is_private_addr(n$src) && 
	     !(n$src in external_ssh_scanners_whitelist || n$src in planetlab_hosts) )
		{
		add n$actions[Notice::ACTION_BLOCK];
		add n$actions[Notice::ACTION_EMAIL];
		}
	}

rule Notice::policy(n: Notice::Info)
	{
	if ( n$note == HTTP::Malware_Hash_Registry_Match )
		add n$actions[Notice::ACTION_EMAIL];

	else if ( n$note == FTP::Site_Exec_Success )
		add n$actions[Notice::ACTION_EMAIL];
	
	else if ( n$note == SSH::Watched_Country_Login )
		add n$actions[Notice::ACTION_EMAIL];

	else if ( n$note == Software::Vulnerable_Java )
		{
		n$suppress_for = 1day;
		add n$actions[Notice::ACTION_EMAIL];
		}
	}


Current style at site 2?

const dont_email_types: set[Notice::Type] = {
    SSL::Invalid_Server_Cert,
    SSH::Login,
    HTTP::MD5,
};

redef Notice::policy += {
        [$pred(n: Notice::Info) = { return (n$note in Notice::block_types); },
         $action = ACTION_BLOCK,
         $priority = 10],
};

redef Notice::policy += { [$action = Notice::ACTION_ALARM, $priority = 0,
                           $pred(n: Notice::Info) = { return (n$note !in dont_email_types);}
                          ] };
    
redef Notice::policy += { [$action = Notice::ACTION_LOG, $priority = 10, $halt=T,
                           $pred(n: Notice::Info) = { return (n$note == HTTP::SQL_Injection_Victim && !Site::is_local_addr(n$src));}
                          ] };

redef Notice::policy += { [$action = Notice::ACTION_LOG, $priority = 10, $halt=T,
                           $pred(n: Notice::Info) = { return (n$note == HTTP::SQL_Injection_Attacker && Site::is_local_addr(n$src));}
                          ] };

New proposed style at site 2...

const dont_email_types: set[Notice::Type] = {
    SSL::Invalid_Server_Cert,
    SSH::Login,
    HTTP::MD5,
};

redef Notice::emailed_types += {
    #DNS::EXTERNAL_FOREIGN_DNS,
    HTTP::IncorrectFileTypeBadHost,
    HTTP::SQL_Injection_Victim,
};


rule Notice::policy(n: Notice::Info) &priority=10
	{
	if ( n$note in Notice::ipblocker_types )
		add n$actions[ACTION_BLOCK];
	}

rule Notice::policy(n: Notice::Info) &priority=0
	{
	if ( n$note !in dont_email_types)
		{
		add n$actions[ACTION_ALARM];
		}
	}

rule Notice::policy(n: Notice::Info) &priority=10
	{
	if ( (n$note == HTTP::SQL_Injection_Victim && !Site::is_local_addr(n$src)) ||
	     (n$note == HTTP::SQL_Injection_Attacker && Site::is_local_addr(n$src)) )
	     {
	     n$action = ACTION_LOG;
	     return F;
	     }
	return T;
	}


I think the new style is a lot clearer and easier to work with in both cases, but I assume this will still require more discussion. :)

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From robin at icir.org  Wed Nov  7 14:19:31 2012
From: robin at icir.org (Robin Sommer)
Date: Wed, 7 Nov 2012 14:19:31 -0800
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <16C508BA-1006-4EAD-9EFC-EE92A5C638B2@icir.org>
References: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>
	<A7FDD5F2-1303-4854-A0DE-EC580A0ED7BD@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED6949@CITESMBX2.ad.uillinois.edu>
	<16C508BA-1006-4EAD-9EFC-EE92A5C638B2@icir.org>
Message-ID: <20121107221931.GG95864@icir.org>


On Wed, Nov 07, 2012 at 17:04 -0500, you wrote:

> Instead of using the "policy" keyword, how does everyone feel about
> "rule"?

I still haven't fully caught up with this thread but I veto this
suggestion. :) Rule is already overloaded (the Snort association, and
also Bro already has "rules" (indeed in the Snort sense).

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From robin at icir.org  Wed Nov  7 20:35:25 2012
From: robin at icir.org (Robin Sommer)
Date: Wed, 7 Nov 2012 20:35:25 -0800
Subject: [Bro-Dev] Events and communication
In-Reply-To: <70E5F8F8-C9BC-4054-9E3B-E2273CA8E0A5@icir.org>
References: <70E5F8F8-C9BC-4054-9E3B-E2273CA8E0A5@icir.org>
Message-ID: <20121108043525.GF6355@icir.org>


On Wed, Nov 07, 2012 at 09:24 -0500, you wrote:

> Is there a particular reason that only exported events can be sent to
> remote Bro's?

I believe the reasoning was that a remote subscriber should be
restricted to the exported interface because it's just another user of
the script. In your case I guess it's the *same* script that wants the
remote events, so it's more an internal usage.

Seems fine to change.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From robin at icir.org  Wed Nov  7 21:27:12 2012
From: robin at icir.org (Robin Sommer)
Date: Wed, 7 Nov 2012 21:27:12 -0800
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <16C508BA-1006-4EAD-9EFC-EE92A5C638B2@icir.org>
References: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>
	<A7FDD5F2-1303-4854-A0DE-EC580A0ED7BD@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED6949@CITESMBX2.ad.uillinois.edu>
	<16C508BA-1006-4EAD-9EFC-EE92A5C638B2@icir.org>
Message-ID: <20121108052712.GH6355@icir.org>


Here's another variant. Don't know yet if I like it but we could make
use of type overloading for finding the right cases:

    with (n: Notice::Info)
    {
        if ( xxx )
            break; # short circuit
    }

    with (n: Notice::Info)
    {
        if ( yyy )
            do_someting();
    }

    [...]
    local n: Notice::Info;
    switch on n;
    [...]


"switch on" runs all "with"s that have a matching argument type.
Predicated need to be written explictly as if() stmts.

There might be nicer syntax than "with" and "switch on".

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From jsiwek at illinois.edu  Thu Nov  8 08:36:18 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Thu, 8 Nov 2012 16:36:18 +0000
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <20121108052712.GH6355@icir.org>
References: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>
	<A7FDD5F2-1303-4854-A0DE-EC580A0ED7BD@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED6949@CITESMBX2.ad.uillinois.edu>
	<16C508BA-1006-4EAD-9EFC-EE92A5C638B2@icir.org>
	<20121108052712.GH6355@icir.org>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED7433@CITESMBX2.ad.uillinois.edu>

More variants?

It could be like synchronous signal handling with "raise" and "trap" keywords.

   trap trigger(n: Notice::Info) &priority=5
   {
   if ( xxx )
      break; # short circuit
   }

   trap trigger(n: Notice::Info) &priority=-5
   {
   if ( yyy )
      do_someting();
   }

   [...]
   local n: Notice::Info;
   raise trigger(n);
   [?]

It could be like exception handling with "throw", and "catch" keywords.

   catch trigger(n: Notice::Info) &priority=5
   {
   if ( xxx )
      throw; # rethrow to continue handling
   }

   catch trigger(n: Notice::Info) &priority=-5
   {
   if ( yyy )
      do_someting();
   # handling stops here because it's not rethrown
   }

   [...]
   local n: Notice::Info;
   throw trigger(n);
   [?]

    Jon

From seth at icir.org  Thu Nov  8 09:03:00 2012
From: seth at icir.org (Seth Hall)
Date: Thu, 8 Nov 2012 12:03:00 -0500
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED7433@CITESMBX2.ad.uillinois.edu>
References: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>
	<A7FDD5F2-1303-4854-A0DE-EC580A0ED7BD@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED6949@CITESMBX2.ad.uillinois.edu>
	<16C508BA-1006-4EAD-9EFC-EE92A5C638B2@icir.org>
	<20121108052712.GH6355@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED7433@CITESMBX2.ad.uillinois.edu>
Message-ID: <D20C9DE2-0208-43A9-8B4B-19CE637D61DC@icir.org>


On Nov 8, 2012, at 11:36 AM, "Siwek, Jonathan Luke" <jsiwek at illinois.edu> wrote:

> More variants?
> 
> It could be like synchronous signal handling with "raise" and "trap" keywords.


Oh, I kind of like this.  To me it even makes the break-ing more obvious since there are a series of traps being executed.  I've been thinking of these more and more like an easy to use hooking mechanism and I think raise and trap fit in with that.

Actually, what about just "hook"?

  hook trigger(n: Notice::Info) &priority=5
  {
  if ( xxx )
     break; # short circuit
  }

  hook trigger(n: Notice::Info) &priority=-5
  {
  if ( yyy )
     do_someting();
  }

  [...]
  local n: Notice::Info;
  hook trigger(n);
  [?]


  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From robin at icir.org  Fri Nov  9 08:12:19 2012
From: robin at icir.org (Robin Sommer)
Date: Fri, 9 Nov 2012 08:12:19 -0800
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <D20C9DE2-0208-43A9-8B4B-19CE637D61DC@icir.org>
References: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>
	<A7FDD5F2-1303-4854-A0DE-EC580A0ED7BD@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED6949@CITESMBX2.ad.uillinois.edu>
	<16C508BA-1006-4EAD-9EFC-EE92A5C638B2@icir.org>
	<20121108052712.GH6355@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED7433@CITESMBX2.ad.uillinois.edu>
	<D20C9DE2-0208-43A9-8B4B-19CE637D61DC@icir.org>
Message-ID: <20121109161219.GG44254@icir.org>


On Thu, Nov 08, 2012 at 12:03 -0500, you wrote:

> Actually, what about just "hook"?

Now it looks like HILTI. :-)

Actually I like this best of all yet.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Fri Nov  9 13:23:41 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 09 Nov 2012 21:23:41 -0000
Subject: [Bro-Dev] #916: stftime crash
Message-ID: <046.e7d8804857c190cba17b00b98b04e597@tracker.bro-ids.org>

#916: stftime crash
---------------------+------------------------
 Reporter:  seth     |      Owner:
     Type:  Problem  |     Status:  new
 Priority:  Normal   |  Milestone:  Bro2.2
Component:  Bro      |    Version:  git/master
 Keywords:           |
---------------------+------------------------
 From file...

 {{{
   filename = 0x2777820
 "/usr/local/bro/share/bro/base/frameworks/notice/./actions/pp-alarms.bro",
 first_line = 195, last_line = 195, first_column = 0, last_column = 0,
 }}}

 Here's the backtrace...

 {{{
 #0  0x00007f9929298ca5 in ?? () from /lib/libc.so.6
 #1  0x00007f992929ad56 in strftime_l () from /lib/libc.so.6
 #2  0x00000000005b0f7c in do_fmt (fmt=@0x7fff87c77388, v=0x1c84a7c0,
 d=0x7fff87c77330) at bro.bif:148
 #3  0x00000000005b11c6 in next_fmt (frame=<value optimized out>,
 BiF_ARGS=0x26772da0) at bro.bif:292
 #4  BifFunc::bro_fmt (frame=<value optimized out>, BiF_ARGS=0x26772da0) at
 bro.bif:1694
 #5  0x00000000005bad0f in BuiltinFunc::Call (this=0x23ea480,
 args=0x26772da0, parent=0x15eccaf0) at /root/src/bro-2.1/src/Func.cc:485
 #6  0x00000000005a55ae in CallExpr::Eval (this=0x279a3c0, f=0x15eccaf0) at
 /root/src/bro-2.1/src/Expr.cc:4512
 #7  0x000000000059496a in AssignExpr::Eval (this=0x2798600, f=0x15eccaf0)
 at /root/src/bro-2.1/src/Expr.cc:2598
 #8  0x0000000000660ba0 in ExprStmt::Exec (this=0x279a320, f=0x0,
 flow=@0x6e92c5) at /root/src/bro-2.1/src/Stmt.cc:369
 #9  0x00000000006599b1 in StmtList::Exec (this=0x2792ba0, f=0x15eccaf0,
 flow=@0x7fff87c7771c) at /root/src/bro-2.1/src/Stmt.cc:1404
 #10 0x00000000005bba09 in BroFunc::Call (this=0x279e620, args=<value
 optimized out>, parent=<value optimized out>) at
 /root/src/bro-2.1/src/Func.cc:335
 #11 0x00000000005a55ae in CallExpr::Eval (this=0x27894b0, f=0x21b64af0) at
 /root/src/bro-2.1/src/Expr.cc:4512
 #12 0x0000000000660ba0 in ExprStmt::Exec (this=0x2789460, f=0x0,
 flow=@0x6e92c5) at /root/src/bro-2.1/src/Stmt.cc:369
 #13 0x00000000006599b1 in StmtList::Exec (this=0x26a4ae0, f=0x21b64af0,
 flow=@0x7fff87c7794c) at /root/src/bro-2.1/src/Stmt.cc:1404
 #14 0x00000000005bba09 in BroFunc::Call (this=0x25a1540, args=<value
 optimized out>, parent=<value optimized out>) at
 /root/src/bro-2.1/src/Func.cc:335
 #15 0x000000000056fb8b in EventHandler::Call (this=0x25a4c40,
 vl=0x18517a00, no_remote=<value optimized out>) at
 /root/src/bro-2.1/src/EventHandler.cc:72
 #16 0x000000000056f2ba in Event::Dispatch (this=0x981640) at
 /root/src/bro-2.1/src/Event.h:46
 #17 EventMgr::Dispatch (this=0x981640) at
 /root/src/bro-2.1/src/Event.cc:105
 #18 0x000000000056f428 in EventMgr::Drain (this=0x981640) at
 /root/src/bro-2.1/src/Event.cc:117
 #19 0x0000000000604ebf in net_packet_dispatch (t=<value optimized out>,
 hdr=0x22ed440, pkt=0x7f99247c494e <Address 0x7f99247c494e out of bounds>,
 hdr_size=14, src_ps=0x22ed400,
     pkt_elem=0x0) at /root/src/bro-2.1/src/Net.cc:354
 #20 0x00000000006158a8 in PktSrc::Process (this=0x22ed400) at
 /root/src/bro-2.1/src/PktSrc.cc:284
 #21 0x000000000060521b in net_run () at /root/src/bro-2.1/src/Net.cc:446
 #22 0x000000000052662f in main (argc=53607040, argv=<value optimized out>)
 at /root/src/bro-2.1/src/main.cc:1073
 Justin Azoff @ 4:12
 (gdb) print fmt
 $4 = (const char *&) @0x7fff87c77388: 0x277cdb5 "D %s %s"
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/916>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From seth at icir.org  Fri Nov  9 13:57:30 2012
From: seth at icir.org (Seth Hall)
Date: Fri, 9 Nov 2012 16:57:30 -0500
Subject: [Bro-Dev] changing Notice::policy mechanism
In-Reply-To: <20121109161219.GG44254@icir.org>
References: <20121105171308.DD5932C4002@rock.ICSI.Berkeley.EDU>
	<A7FDD5F2-1303-4854-A0DE-EC580A0ED7BD@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED6949@CITESMBX2.ad.uillinois.edu>
	<16C508BA-1006-4EAD-9EFC-EE92A5C638B2@icir.org>
	<20121108052712.GH6355@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED7433@CITESMBX2.ad.uillinois.edu>
	<D20C9DE2-0208-43A9-8B4B-19CE637D61DC@icir.org>
	<20121109161219.GG44254@icir.org>
Message-ID: <C4786430-6DC9-4F89-8B7E-BDC8BA14B1FB@icir.org>


On Nov 9, 2012, at 11:12 AM, Robin Sommer <robin at icir.org> wrote:

> Actually I like this best of all yet.

Is everyone that cares to speak up ok with "hook"?  

Seems to make sense to me since I think what we're talking about here will be used as a way to synchronously hook into processing in various places (in a less syntactically verbose manner than the current approach in the notice framework).  Notice and file analysis hooking is essentially trying to hook into those processing pipelines to change how they are dealt with.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From scampbell at lbl.gov  Sun Nov 11 09:36:17 2012
From: scampbell at lbl.gov (Scott Campbell)
Date: Sun, 11 Nov 2012 11:36:17 -0600
Subject: [Bro-Dev] IPv6 and software.log records
Message-ID: <509FE211.50403@lbl.gov>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

We are running (git current) at SC this year and have a good mix of v4
and v6 traffic.  I am seeing excellent coverage of both protocols for
everything but software.log - in that case there are no v6 records.
Is this a known issue?  If not, I will open a ticket.

thanks!
scott
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iD8DBQFQn+IRK2Plq8B7ZBwRAquUAJ9yu/eAURcQwQz59auoOxWod+aQYACfXo3d
Axiwr0PJ11HaKFvd21DDJco=
=eQAs
-----END PGP SIGNATURE-----

From seth at icir.org  Sun Nov 11 14:55:06 2012
From: seth at icir.org (Seth Hall)
Date: Sun, 11 Nov 2012 17:55:06 -0500
Subject: [Bro-Dev] IPv6 and software.log records
In-Reply-To: <509FE211.50403@lbl.gov>
References: <509FE211.50403@lbl.gov>
Message-ID: <384CBB7F-F8FD-4825-A495-91B6B09BE593@icir.org>


On Nov 11, 2012, at 12:36 PM, Scott Campbell <scampbell at lbl.gov> wrote:

> We are running (git current) at SC this year and have a good mix of v4
> and v6 traffic.  I am seeing excellent coverage of both protocols for
> everything but software.log - in that case there are no v6 records.
> Is this a known issue?  If not, I will open a ticket.


Do you have the "local" IPv6 address space configured in Site::local_nets or networks.cfg (if running BroControl)?

By default, the software framework only tracks local software.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Mon Nov 12 18:14:31 2012
From: seth at icir.org (Seth Hall)
Date: Mon, 12 Nov 2012 21:14:31 -0500
Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/modbus-fixes:
	Adjust modbus register array parsing. (c911d03)
In-Reply-To: <201211122253.qACMrsLc028976@bro-ids.icir.org>
References: <201211122253.qACMrsLc028976@bro-ids.icir.org>
Message-ID: <8BA547AA-6901-488F-B94D-DE4AE6483BAE@icir.org>


On Nov 12, 2012, at 5:53 PM, Jonathan Siwek <jsiwek at ncsa.illinois.edu> wrote:

>   Added another parameter to modbus events that carry register arrays to
>    the script-layer which indicates the associated byte count from the
>    message (allowing for invalid values to be detected):


I actually don't like this.  If an invalid value is found, I'd rather that a protocol violation or weird gets generated instead since it actually indicates a protocol error (I believe it does in this case at least).

What do you think about that?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From bro at tracker.bro-ids.org  Tue Nov 13 07:51:39 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 13 Nov 2012 15:51:39 -0000
Subject: [Bro-Dev] #323: State of passive fingerprinting signatures?
In-Reply-To: <047.9bd93768b3e416ec419d19ee657027c9@tracker.bro-ids.org>
References: <047.9bd93768b3e416ec419d19ee657027c9@tracker.bro-ids.org>
Message-ID: <062.6b08574a028bdbbe4173001eba8b552b@tracker.bro-ids.org>

#323: State of passive fingerprinting signatures?
-----------------------+------------------------
  Reporter:  robin     |      Owner:
      Type:  Problem   |     Status:  closed
  Priority:  Normal    |  Milestone:
 Component:  Bro       |    Version:  git/master
Resolution:  Rejected  |   Keywords:
-----------------------+------------------------
Changes (by seth):

 * status:  new => closed
 * resolution:   => Rejected


Comment:

 I'm just going to close this ticket.  If anyone comes back around to p0f
 in Bro, this ticket won't help.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/323#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From jsiwek at illinois.edu  Tue Nov 13 08:09:45 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Tue, 13 Nov 2012 16:09:45 +0000
Subject: [Bro-Dev] [Bro-Commits] [git/bro]
	topic/jsiwek/modbus-fixes:	Adjust modbus register array
	parsing. (c911d03)
In-Reply-To: <8BA547AA-6901-488F-B94D-DE4AE6483BAE@icir.org>
References: <201211122253.qACMrsLc028976@bro-ids.icir.org>
	<8BA547AA-6901-488F-B94D-DE4AE6483BAE@icir.org>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED981A@CITESMBX2.ad.uillinois.edu>

>>  Added another parameter to modbus events that carry register arrays to
>>   the script-layer which indicates the associated byte count from the
>>   message (allowing for invalid values to be detected):
> 
> I actually don't like this.  If an invalid value is found, I'd rather that a protocol violation or weird gets generated instead since it actually indicates a protocol error (I believe it does in this case at least).

I can add a protocol violation and not generate the event in these cases? but do you care whether the extra parameter is provided to the events in the valid cases (I don't think it's hurting anything) ?

And really I'm just trying fix stuff that results in a crash... there's also a bunch of &check attributes in modbus-protocol.pac that seem to indicate protocol violations, but they don't do anything since that attribute is a no-op in binpac.  I didn't intend to go through and fix all those, but I suppose I could if you're concerned about that kind of thing.

    Jon

From seth at icir.org  Tue Nov 13 08:16:03 2012
From: seth at icir.org (Seth Hall)
Date: Tue, 13 Nov 2012 11:16:03 -0500
Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/modbus-fixes:
	Adjust modbus register array parsing. (c911d03)
In-Reply-To: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED981A@CITESMBX2.ad.uillinois.edu>
References: <201211122253.qACMrsLc028976@bro-ids.icir.org>
	<8BA547AA-6901-488F-B94D-DE4AE6483BAE@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116ED981A@CITESMBX2.ad.uillinois.edu>
Message-ID: <F1C6DEBB-C304-40E9-B424-11D9E97AAE72@icir.org>


On Nov 13, 2012, at 11:09 AM, "Siwek, Jonathan Luke" <jsiwek at illinois.edu> wrote:

> I can add a protocol violation and not generate the event in these cases? but do you care whether the extra parameter is provided to the events in the valid cases (I don't think it's hurting anything) ?

You're right, it's not hurting anything, but I think the events are redundant if they include those lengths since it's implicitly included in the number of registers included.  As I've changed events and added new ones, I try not include length fields that are implicit in other fields.  This case is a little weird because the protocol itself is a little weird, but I still think it makes sense to leave the length out.

> And really I'm just trying fix stuff that results in a crash... there's also a bunch of &check attributes in modbus-protocol.pac that seem to indicate protocol violations, but they don't do anything since that attribute is a no-op in binpac.


Yeah, I left those there as place holders so that we didn't need to refer back to the standard for those values when we port this to binpac++.  Feel free to ignore those.

  .Seth


--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From bro at tracker.bro-ids.org  Tue Nov 13 08:21:05 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 13 Nov 2012 16:21:05 -0000
Subject: [Bro-Dev] #908: Need for function bytestring_to_double
In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org>
References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org>
Message-ID: <064.36c23711005f02f298da6a648ff5dcd9@tracker.bro-ids.org>

#908: Need for function bytestring_to_double
------------------------------+-------------------------------
  Reporter:  carsten          |      Owner:
      Type:  Feature Request  |     Status:  needs information
  Priority:  Normal           |  Milestone:  Bro2.2
 Component:  Bro              |    Version:  git/master
Resolution:                   |   Keywords:
------------------------------+-------------------------------

Comment (by carsten):

 The status of this ticket is still "needs information". I gave feedback 2
 weeks ago (comments 10-12). Does the status just need updating, or is
 still some information needed from me? If so, which information?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/908#comment:13>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Tue Nov 13 10:17:41 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 13 Nov 2012 18:17:41 -0000
Subject: [Bro-Dev] #917: topic/jsiwek/modbus-fixes
Message-ID: <048.9242150ea938281a198c127edc4ba389@tracker.bro-ids.org>

#917: topic/jsiwek/modbus-fixes
---------------------------+------------------------
 Reporter:  jsiwek         |      Owner:
     Type:  Merge Request  |     Status:  new
 Priority:  Normal         |  Milestone:  Bro2.2
Component:  Bro            |    Version:  git/master
 Keywords:                 |
---------------------------+------------------------
 This branch is in the `bro` and `binpac` repos and fixes a couple crashes
 with modbus that I found through fuzzing (maybe there's more, but this is
 all I've been able to find for now).

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/917>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From vladg at cmu.edu  Tue Nov 13 11:11:50 2012
From: vladg at cmu.edu (Vlad Grigorescu)
Date: Tue, 13 Nov 2012 19:11:50 +0000
Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/modbus-fixes:
	Remove byte	count parameter from modbus events carrying
	register arrays	(fd5eb23)
In-Reply-To: <2391_1352830357_qADICZJB014281_201211131812.qADICS0j011771@bro-ids.icir.org>
References: <2391_1352830357_qADICZJB014281_201211131812.qADICS0j011771@bro-ids.icir.org>
Message-ID: <1202BE242E080642B0CD0AD0A03E855261E206@PGH-MSGMB-03.andrew.ad.cmu.edu>

Thanks for looking into this, Jon.

I'm still seeing crashes with this commit (trace included below). Should I open a new ticket for this? I don't want to latch onto the merge request at #917 unnecessarily. Thanks,

  --Vlad

[New LWP 3282]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/bro/bin/bro -i eth4 -U .status -p broctl -p broctl-live -p local -p'.
Program terminated with signal 6, Aborted.
#0  0x00007f46f893f425 in raise () from /lib/x86_64-linux-gnu/libc.so.6

Thread 1 (Thread 0x7f46fac7e780 (LWP 3282)):
#0  0x00007f46f893f425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f46f8942b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007f46f89380ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00007f46f8938192 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x000000000068cd9f in ClearPreviousData (this=<optimized out>) at /home/bro/src/bro/aux/binpac/lib/binpac_buffer.cc:213
#5  binpac::FlowBuffer::ClearPreviousData (this=<optimized out>) at /home/bro/src/bro/aux/binpac/lib/binpac_buffer.cc:208
#6  0x000000000068d246 in binpac::FlowBuffer::NewData (this=0xb1b08c0, begin=0xbfcf7a0 "\300\205", end=0xbfcf7ad "") at /home/bro/src/bro/aux/binpac/lib/binpac_buffer.cc:176
#7  0x0000000000503ce0 in binpac::ModbusTCP::ModbusTCP_Flow::NewData (this=0x46c1f90, t_begin_of_data=<optimized out>, t_end_of_data=<optimized out>) at /home/bro/src/bro/build/src/modbus_pac.cc:2867
#8  0x000000000052169d in Analyzer::NextStream (this=0xb1a3640, len=<optimized out>, data=<optimized out>, is_orig=<optimized out>) at /home/bro/src/bro/src/Analyzer.cc:369
#9  0x00000000005222b6 in Analyzer::ForwardStream (this=0xb19b240, len=13, data=0xbfcf7a0 "\300\205", is_orig=false) at /home/bro/src/bro/src/Analyzer.cc:456
#10 0x0000000000646e80 in TCP_Reassembler::DeliverBlock (this=0x5fdf8c0, seq=15, len=13, data=0xbfcf7a0 "\300\205") at /home/bro/src/bro/src/TCP_Reassembler.cc:618
#11 0x00000000006471aa in BlockInserted (start_block=<optimized out>, this=<optimized out>) at /home/bro/src/bro/src/TCP_Reassembler.cc:359
#12 TCP_Reassembler::BlockInserted (this=0x5fdf8c0, start_block=<optimized out>) at /home/bro/src/bro/src/TCP_Reassembler.cc:334
#13 0x0000000000646d28 in TCP_Reassembler::DataSent (this=0x5fdf8c0, t=<optimized out>, seq=<optimized out>, len=<optimized out>, data=<optimized out>, replaying=<optimized out>) at /home/bro/src/bro/src/TCP_Reassembler.cc:458
#14 0x0000000000645cc6 in TCP_Endpoint::DataSent (this=0xb1a8790, t=<optimized out>, seq=15, len=13, caplen=13, data=0x7f46e2eefffe <Address 0x7f46e2eefffe out of bounds>, ip=<optimized out>, tp=0x7f46e2eeffea) at /home/bro/src/bro/src/TCP_Endpoint.cc:183
#15 0x00000000006446f8 in TCP_Analyzer::DeliverPacket (this=0xb19b240, len=13, data=0x7f46e2eefffe <Address 0x7f46e2eefffe out of bounds>, is_orig=false, seq=<optimized out>, ip=0x7fff69511bf0, caplen=13) at /home/bro/src/bro/src/TCP.cc:1039
#16 0x0000000000521571 in Analyzer::NextPacket (this=0xb19b240, len=<optimized out>, data=<optimized out>, is_orig=<optimized out>, seq=<optimized out>, ip=<optimized out>, caplen=33) at /home/bro/src/bro/src/Analyzer.cc:341
#17 0x000000000053aa70 in Connection::NextPacket (this=<optimized out>, t=<optimized out>, is_orig=<optimized out>, ip=<optimized out>, len=<optimized out>, caplen=<optimized out>, data=<optimized out>, record_packet=@0x7fff69511868: 1, record_content=@0x7fff6951186c: 1, hdr=0x1abd040, pkt=0x7f46e2eeffc8 <Address 0x7f46e2eeffc8 out of bounds>, hdr_size=14) at /home/bro/src/bro/src/Conn.cc:259
#18 0x000000000062e2f0 in NetSessions::DoNextPacket (this=0x2bd0c00, t=1352833032.1424849, hdr=0x1abd040, ip_hdr=0x7fff69511bf0, pkt=0x7f46e2eeffc8 <Address 0x7f46e2eeffc8 out of bounds>, hdr_size=14, encapsulation=0x0) at /home/bro/src/bro/src/Sessions.cc:700
#19 0x000000000062f8c5 in NetSessions::NextPacket (this=0x2bd0c00, t=1352833032.1424849, hdr=0x1abd040, pkt=0x7f46e2eeffc8 <Address 0x7f46e2eeffc8 out of bounds>, hdr_size=14, pkt_elem=<optimized out>) at /home/bro/src/bro/src/Sessions.cc:238
#20 0x00000000005ec14b in net_packet_dispatch (t=1352833032.1424849, hdr=0x1abd040, pkt=0x7f46e2eeffc8 <Address 0x7f46e2eeffc8 out of bounds>, hdr_size=14, src_ps=0x1abd000, pkt_elem=0x0) at /home/bro/src/bro/src/Net.cc:353
#21 0x00000000005fb0cf in Process (this=0x1abd000) at /home/bro/src/bro/src/PktSrc.cc:303
#22 PktSrc::Process (this=0x1abd000) at /home/bro/src/bro/src/PktSrc.cc:175
#23 0x00000000005ec547 in net_run () at /home/bro/src/bro/src/Net.cc:446
#24 0x00000000004c06ea in main (argc=<optimized out>, argv=<optimized out>) at /home/bro/src/bro/src/main.cc:1073

==== No reporter.log

==== stderr.log
listening on eth4, capture length 8192 bytes

bro: /home/bro/src/bro/aux/binpac/lib/binpac_buffer.cc:213: void binpac::FlowBuffer::ClearPreviousData(): Assertion `buffer_n_ == 0' failed.
/usr/local/bro/share/broctl/scripts/run-bro: line 60:  3282 Aborted                 (core dumped) nohup $mybro $@



On Nov 13, 2012, at 1:12 PM, Jonathan Siwek <jsiwek at ncsa.illinois.edu> wrote:

> Repository : ssh://git at bro-ids.icir.org/bro
> 
> On branch  : topic/jsiwek/modbus-fixes
> Link       : http://tracker.bro-ids.org/bro/changeset/fd5eb23fa6ac654471d71645eb37dacc6d45896b/bro
> 
>> ---------------------------------------------------------------
> 
> commit fd5eb23fa6ac654471d71645eb37dacc6d45896b
> Author: Jon Siwek <jsiwek at ncsa.illinois.edu>
> Date:   Tue Nov 13 12:09:14 2012 -0600
> 
>    Remove byte count parameter from modbus events carrying register arrays
> 
>    Instead of these events being generated for invalid byte count values
>    (they should always be even, not odd), a protocol_violation is raised.
> 
>        modbus_read_holding_registers_response
>        modbus_read_input_registers_response
>        modbus_write_multiple_registers_request
>        modbus_read_write_multiple_registers_request
>        modbus_read_write_multiple_registers_response
>        modbus_read_fifo_queue_respons
> 
> 
>> ---------------------------------------------------------------
> 
> fd5eb23fa6ac654471d71645eb37dacc6d45896b
> scripts/policy/protocols/modbus/track-memmap.bro   |    2 +-
> src/event.bif                                      |   24 ++-------
> src/modbus-analyzer.pac                            |   50 +++++++++++++++++--
> .../output                                         |    5 +-
> .../btest/scripts/base/protocols/modbus/events.bro |   12 ++--
> .../base/protocols/modbus/register_parsing.bro     |    9 ++--
> 6 files changed, 64 insertions(+), 38 deletions(-)
> 
> diff --git a/scripts/policy/protocols/modbus/track-memmap.bro b/scripts/policy/protocols/modbus/track-memmap.bro
> index cc02ce9..fc02d9b 100644
> --- a/scripts/policy/protocols/modbus/track-memmap.bro
> +++ b/scripts/policy/protocols/modbus/track-memmap.bro
> @@ -60,7 +60,7 @@ event modbus_read_holding_registers_request(c: connection, headers: ModbusHeader
> 	c$modbus$track_address = start_address+1;
> 	}
> 
> -event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters)
> +event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
> 	{
> 	local slave = c$id$resp_h;
> 
> diff --git a/src/event.bif b/src/event.bif
> index cc8acb1..b965c26 100644
> --- a/src/event.bif
> +++ b/src/event.bif
> @@ -6623,10 +6623,8 @@ event modbus_read_holding_registers_request%(c: connection, headers: ModbusHeade
> ##
> ## headers: The headers for the modbus function.
> ##
> -## byte_count: The number of bytes in the message that comprise register values.
> -##
> ## registers: The register values returned from the device.
> -event modbus_read_holding_registers_response%(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters%);
> +event modbus_read_holding_registers_response%(c: connection, headers: ModbusHeaders, registers: ModbusRegisters%);
> 
> ## Generated for a Modbus read input registers request.
> ##
> @@ -6645,10 +6643,8 @@ event modbus_read_input_registers_request%(c: connection, headers: ModbusHeaders
> ##
> ## headers: The headers for the modbus function.
> ##
> -## byte_count: The number of bytes in the message that comprise register values.
> -##
> ## registers: The register values returned from the device.
> -event modbus_read_input_registers_response%(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters%);
> +event modbus_read_input_registers_response%(c: connection, headers: ModbusHeaders, registers: ModbusRegisters%);
> 
> ## Generated for a Modbus write single coil request.
> ##
> @@ -6724,10 +6720,8 @@ event modbus_write_multiple_coils_response%(c: connection, headers: ModbusHeader
> ##
> ## start_address: The memory address of the first register to be written.
> ##
> -## byte_count: The number of bytes in the message that comprise register values.
> -##
> ## registers: The values to be written to the registers.
> -event modbus_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, start_address: count, byte_count: count, registers: ModbusRegisters%);
> +event modbus_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, start_address: count, registers: ModbusRegisters%);
> 
> ## Generated for a Modbus write multiple registers response.
> ##
> @@ -6818,10 +6812,8 @@ event modbus_mask_write_register_response%(c: connection, headers: ModbusHeaders
> ##
> ## write_start_address: The memory address of the first register to be written.
> ##
> -## write_byte_count: Number of bytes in message that comprise register values.
> -##
> ## write_registers: The values to be written to the registers.
> -event modbus_read_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_byte_count: count, write_registers: ModbusRegisters%);
> +event modbus_read_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_registers: ModbusRegisters%);
> 
> ## Generated for a Modbus read/write multiple registers response.
> ##
> @@ -6829,10 +6821,8 @@ event modbus_read_write_multiple_registers_request%(c: connection, headers: Modb
> ##
> ## headers: The headers for the modbus function.
> ##
> -## byte_count: The number of bytes in the message that comprise register values.
> -##
> ## written_registers: The register values read from the registers specified in the request.
> -event modbus_read_write_multiple_registers_response%(c: connection, headers: ModbusHeaders, byte_count: count, written_registers: ModbusRegisters%);
> +event modbus_read_write_multiple_registers_response%(c: connection, headers: ModbusHeaders, written_registers: ModbusRegisters%);
> 
> ## Generated for a Modbus read FIFO queue request.
> ##
> @@ -6849,10 +6839,8 @@ event modbus_read_fifo_queue_request%(c: connection, headers: ModbusHeaders, sta
> ##
> ## headers: The headers for the modbus function.
> ##
> -## byte_count: The number of bytes in the message that comprise register values.
> -##
> ## fifos: The register values read from the FIFO queue on the device.
> -event modbus_read_fifo_queue_response%(c: connection, headers: ModbusHeaders, byte_count: count, fifos: ModbusRegisters%);
> +event modbus_read_fifo_queue_response%(c: connection, headers: ModbusHeaders, fifos: ModbusRegisters%);
> 
> ## Raised for informational messages reported via Bro's reporter framework. Such
> ## messages may be generated internally by the event engine and also by other
> diff --git a/src/modbus-analyzer.pac b/src/modbus-analyzer.pac
> index 155da96..b03df9d 100644
> --- a/src/modbus-analyzer.pac
> +++ b/src/modbus-analyzer.pac
> @@ -135,8 +135,16 @@ refine flow ModbusTCP_Flow += {
> 	# RESPONSE FC=3
> 	function deliver_ReadHoldingRegistersResponse(header: ModbusTCP_TransportHeader, message: ReadHoldingRegistersResponse): bool
> 		%{
> +		if ( ${message.byte_count} % 2 != 0 )
> +			{
> +			connection()->bro_analyzer()->ProtocolViolation(
> +			    fmt("invalid value for modbus read holding register response byte count %d", ${message.byte_count}));
> +			return false;
> +			}
> +
> 		if ( ::modbus_read_holding_registers_response )
> 			{
> +
> 			VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
> 			for ( unsigned int i=0; i < ${message.registers}->size(); ++i )
> 				{
> @@ -147,7 +155,6 @@ refine flow ModbusTCP_Flow += {
> 			BifEvent::generate_modbus_read_holding_registers_response(connection()->bro_analyzer(),
> 			                                                          connection()->bro_analyzer()->Conn(),
> 			                                                          HeaderToBro(header),
> -			                                                          ${message.byte_count},
> 			                                                          t);
> 			}
> 
> @@ -172,6 +179,13 @@ refine flow ModbusTCP_Flow += {
> 	# RESPONSE FC=4
> 	function deliver_ReadInputRegistersResponse(header: ModbusTCP_TransportHeader, message: ReadInputRegistersResponse): bool
> 		%{
> +		if ( ${message.byte_count} % 2 != 0 )
> +			{
> +			connection()->bro_analyzer()->ProtocolViolation(
> +			    fmt("invalid value for modbus read input register response byte count %d", ${message.byte_count}));
> +			return false;
> +			}
> +
> 		if ( ::modbus_read_input_registers_response )
> 			{
> 			VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
> @@ -184,7 +198,6 @@ refine flow ModbusTCP_Flow += {
> 			BifEvent::generate_modbus_read_input_registers_response(connection()->bro_analyzer(),
> 			                                                        connection()->bro_analyzer()->Conn(),
> 			                                                        HeaderToBro(header),
> -			                                                        ${message.byte_count},
> 			                                                        t);
> 			}
> 
> @@ -309,6 +322,13 @@ refine flow ModbusTCP_Flow += {
> 	# REQUEST FC=16
> 	function deliver_WriteMultipleRegistersRequest(header: ModbusTCP_TransportHeader, message: WriteMultipleRegistersRequest): bool
> 		%{
> +		if ( ${message.byte_count} % 2 != 0 )
> +			{
> +			connection()->bro_analyzer()->ProtocolViolation(
> +			    fmt("invalid value for modbus write multiple registers request byte count %d", ${message.byte_count}));
> +			return false;
> +			}
> +
> 		if ( ::modbus_write_multiple_registers_request )
> 			{
> 			VectorVal * t = new VectorVal(BifType::Vector::ModbusRegisters);
> @@ -321,7 +341,7 @@ refine flow ModbusTCP_Flow += {
> 				BifEvent::generate_modbus_write_multiple_registers_request(connection()->bro_analyzer(),
> 				                                                           connection()->bro_analyzer()->Conn(),
> 				                                                           HeaderToBro(header),
> -				                                                           ${message.start_address}, ${message.byte_count}, t);
> +				                                                           ${message.start_address}, t);
> 			}
> 
> 		return true;
> @@ -486,6 +506,13 @@ refine flow ModbusTCP_Flow += {
> 	# REQUEST FC=23
> 	function deliver_ReadWriteMultipleRegistersRequest(header: ModbusTCP_TransportHeader, message: ReadWriteMultipleRegistersRequest): bool
> 		%{
> +		if ( ${message.write_byte_count} % 2 != 0 )
> +			{
> +			connection()->bro_analyzer()->ProtocolViolation(
> +			    fmt("invalid value for modbus read write multiple registers request write byte count %d", ${message.write_byte_count}));
> +			return false;
> +			}
> +
> 		if ( ::modbus_read_write_multiple_registers_request )
> 			{
> 			VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
> @@ -501,7 +528,6 @@ refine flow ModbusTCP_Flow += {
> 			                                                                ${message.read_start_address},
> 			                                                                ${message.read_quantity},
> 			                                                                ${message.write_start_address},
> -			                                                                ${message.write_byte_count},
> 			                                                                t);
> 			}
> 
> @@ -511,6 +537,13 @@ refine flow ModbusTCP_Flow += {
> 	# RESPONSE FC=23
> 	function deliver_ReadWriteMultipleRegistersResponse(header: ModbusTCP_TransportHeader, message: ReadWriteMultipleRegistersResponse): bool
> 		%{
> +		if ( ${message.byte_count} % 2 != 0 )
> +			{
> +			connection()->bro_analyzer()->ProtocolViolation(
> +			    fmt("invalid value for modbus read write multiple registers response byte count %d", ${message.byte_count}));
> +			return false;
> +			}
> +
> 		if ( ::modbus_read_write_multiple_registers_response )
> 			{
> 			VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
> @@ -523,7 +556,6 @@ refine flow ModbusTCP_Flow += {
> 			BifEvent::generate_modbus_read_write_multiple_registers_response(connection()->bro_analyzer(),
> 			                                                                 connection()->bro_analyzer()->Conn(),
> 			                                                                 HeaderToBro(header),
> -			                                                                 ${message.byte_count},
> 			                                                                 t);
> 			}
> 
> @@ -548,6 +580,13 @@ refine flow ModbusTCP_Flow += {
> 	# RESPONSE FC=24
> 	function deliver_ReadFIFOQueueResponse(header: ModbusTCP_TransportHeader, message: ReadFIFOQueueResponse): bool
> 		%{
> +		if ( ${message.byte_count} % 2 != 0 )
> +			{
> +			connection()->bro_analyzer()->ProtocolViolation(
> +			    fmt("invalid value for modbus read FIFO queue response byte count %d", ${message.byte_count}));
> +			return false;
> +			}
> +
> 		if ( ::modbus_read_fifo_queue_response )
> 			{
> 			VectorVal* t = new VectorVal(new VectorType(base_type(TYPE_COUNT)));
> @@ -560,7 +599,6 @@ refine flow ModbusTCP_Flow += {
> 			BifEvent::generate_modbus_read_fifo_queue_response(connection()->bro_analyzer(),
> 			                                                   connection()->bro_analyzer()->Conn(),
> 			                                                   HeaderToBro(header),
> -			                                                   ${message.byte_count},
> 			                                                   t);
> 			}
> 
> diff --git a/testing/btest/Baseline/scripts.base.protocols.modbus.register_parsing/output b/testing/btest/Baseline/scripts.base.protocols.modbus.register_parsing/output
> index 353f85d..5bb5f1b 100644
> --- a/testing/btest/Baseline/scripts.base.protocols.modbus.register_parsing/output
> +++ b/testing/btest/Baseline/scripts.base.protocols.modbus.register_parsing/output
> @@ -1,5 +1,4 @@
> modbus_read_input_registers_request, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=1119, pid=0, uid=255, function_code=4], 900, 147
> -modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=2606, pid=0, uid=255, function_code=4], [0, 0, 0, 0, 0, 0, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690], 100, 200
> -modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=6714, pid=0, uid=255, function_code=4], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3840, 0, 0, 31, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 37, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 64, 129
> +modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=2606, pid=0, uid=255, function_code=4], [0, 0, 0, 0, 0, 0, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690], 100
> modbus_read_input_registers_request, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=12993, pid=0, uid=255, function_code=4], 400, 100
> -modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=17667, pid=0, uid=255, function_code=4], [49, 18012, 51, 42, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 49, 50, 51, 54324, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 69, 63, 64, 65, 66, 67, 68, 49, 189, 51, 52, 53, 54, 4151, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 136, 49, 50, 51, 212, 53, 54, 170, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690], 100, 200
> +modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=17667, pid=0, uid=255, function_code=4], [49, 18012, 51, 42, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 49, 50, 51, 54324, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 69, 63, 64, 65, 66, 67, 68, 49, 189, 51, 52, 53, 54, 4151, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 136, 49, 50, 51, 212, 53, 54, 170, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690], 100
> diff --git a/testing/btest/scripts/base/protocols/modbus/events.bro b/testing/btest/scripts/base/protocols/modbus/events.bro
> index 6c47dc6..f648a0a 100644
> --- a/testing/btest/scripts/base/protocols/modbus/events.bro
> +++ b/testing/btest/scripts/base/protocols/modbus/events.bro
> @@ -41,7 +41,7 @@ event modbus_read_holding_registers_request(c: connection, headers: ModbusHeader
>     print "modbus_read_holding_registers_request", c, headers, start_address, quantity;
> }
> 
> -event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters)
> +event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
> {
>     print "modbus_read_holding_registers_response", c, headers, registers;
> }
> @@ -51,7 +51,7 @@ event modbus_read_input_registers_request(c: connection, headers: ModbusHeaders,
>     print "modbus_read_input_registers_request", c, headers, start_address, quantity;
> }
> 
> -event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters)
> +event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
> {
>     print "modbus_read_input_registers_response", c, headers, registers;
> }
> @@ -86,7 +86,7 @@ event modbus_write_multiple_coils_response(c: connection, headers: ModbusHeaders
>     print "modbus_write_multiple_coils_response", c, headers, start_address, quantity;
> }
> 
> -event modbus_write_multiple_registers_request(c: connection, headers: ModbusHeaders, start_address: count, byte_count: count, registers: ModbusRegisters)
> +event modbus_write_multiple_registers_request(c: connection, headers: ModbusHeaders, start_address: count, registers: ModbusRegisters)
> {
>     print "modbus_write_multiple_registers_request", c, headers, start_address, registers;
> }
> @@ -126,12 +126,12 @@ event modbus_mask_write_register_response(c: connection, headers: ModbusHeaders,
>     print "modbus_mask_write_register_response", c, headers, address, and_mask, or_mask;
> }
> 
> -event modbus_read_write_multiple_registers_request(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_byte_count: count, write_registers: ModbusRegisters)
> +event modbus_read_write_multiple_registers_request(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_registers: ModbusRegisters)
> {
>     print "modbus_read_write_multiple_registers_request", c, headers, read_start_address, read_quantity, write_start_address, write_registers;
> }
> 
> -event modbus_read_write_multiple_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, written_registers: ModbusRegisters)
> +event modbus_read_write_multiple_registers_response(c: connection, headers: ModbusHeaders, written_registers: ModbusRegisters)
> {
>     print "modbus_read_write_multiple_registers_response", c, headers, written_registers;
> }
> @@ -141,7 +141,7 @@ event modbus_read_fifo_queue_request(c: connection, headers: ModbusHeaders, star
>     print "modbus_read_fifo_queue_request", c, headers, start_address;
> }
> 
> -event modbus_read_fifo_queue_response(c: connection, headers: ModbusHeaders, byte_count: count, fifos: ModbusRegisters)
> +event modbus_read_fifo_queue_response(c: connection, headers: ModbusHeaders, fifos: ModbusRegisters)
> {
>     print "modbus_read_fifo_queue_response", c, headers, fifos;
> }
> diff --git a/testing/btest/scripts/base/protocols/modbus/register_parsing.bro b/testing/btest/scripts/base/protocols/modbus/register_parsing.bro
> index 300dd75..1641860 100644
> --- a/testing/btest/scripts/base/protocols/modbus/register_parsing.bro
> +++ b/testing/btest/scripts/base/protocols/modbus/register_parsing.bro
> @@ -6,15 +6,16 @@
> # of register values, with the quantity being derived from a byte count value
> # that is also sent.  If the byte count value is invalid (e.g. an odd value
> # might not be valid since registers must be 2-byte values), then the parser
> -# should not trigger any asserts, but the resulting event could indicate
> -# the strangeness (i.e. byte_count != 2*|registers|).
> +# should not trigger any asserts, but generate a protocol_violation (in this
> +# case TCP_ApplicationAnalyzer::ProtocolViolation asserts its behavior for
> +# incomplete connections).
> 
> event modbus_read_input_registers_request(c: connection, headers: ModbusHeaders, start_address: count, quantity: count)
> 	{
> 	print "modbus_read_input_registers_request", c$id, headers, start_address, quantity;
> 	}
> 
> -event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters)
> +event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
> 	{
> -	print "modbus_read_input_registers_response", c$id, headers, registers, |registers|, byte_count;
> +	print "modbus_read_input_registers_response", c$id, headers, registers, |registers|;
> 	}
> 
> _______________________________________________
> bro-commits mailing list
> bro-commits at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits



From bro at tracker.bro-ids.org  Tue Nov 13 11:42:34 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Tue, 13 Nov 2012 19:42:34 -0000
Subject: [Bro-Dev] #908: Need for function bytestring_to_double
In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org>
References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org>
Message-ID: <064.5a4309a704a84d1208721fd44547d30a@tracker.bro-ids.org>

#908: Need for function bytestring_to_double
----------------------------+-------------------------------
  Reporter:  carsten        |      Owner:
      Type:  Merge Request  |     Status:  needs information
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:
----------------------------+-------------------------------
Changes (by robin):

 * type:  Feature Request => Merge Request


Comment:

 I still need to digest the thread but I'll turn it into a merge request
 and aim to do that soon.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/908#comment:14>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From jsiwek at illinois.edu  Tue Nov 13 12:35:21 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Tue, 13 Nov 2012 20:35:21 +0000
Subject: [Bro-Dev] [Bro-Commits] [git/bro]
	topic/jsiwek/modbus-fixes:	Remove byte	count parameter from
	modbus events carrying	register arrays	(fd5eb23)
In-Reply-To: <1202BE242E080642B0CD0AD0A03E855261E206@PGH-MSGMB-03.andrew.ad.cmu.edu>
References: <2391_1352830357_qADICZJB014281_201211131812.qADICS0j011771@bro-ids.icir.org>
	<1202BE242E080642B0CD0AD0A03E855261E206@PGH-MSGMB-03.andrew.ad.cmu.edu>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116ED9D64@CITESMBX2.ad.uillinois.edu>


On Nov 13, 2012, at 1:11 PM, Vlad Grigorescu <vladg at cmu.edu> wrote:

> I'm still seeing crashes with this commit (trace included below). Should I open a new ticket for this? I don't want to latch onto the merge request at #917 unnecessarily. Thanks,

Did you also checkout the changes I did in the aux/binpac repo (I did changes in branch "topic/jsiwek/modbus-fixes" in both bro and aux/binpac) ?

That stack trace looks similar to something I encountered that needed a change in binpac.

    Jon

> [New LWP 3282]
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> Core was generated by `/usr/local/bro/bin/bro -i eth4 -U .status -p broctl -p broctl-live -p local -p'.
> Program terminated with signal 6, Aborted.
> #0  0x00007f46f893f425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> 
> Thread 1 (Thread 0x7f46fac7e780 (LWP 3282)):
> #0  0x00007f46f893f425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1  0x00007f46f8942b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2  0x00007f46f89380ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
> #3  0x00007f46f8938192 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
> #4  0x000000000068cd9f in ClearPreviousData (this=<optimized out>) at /home/bro/src/bro/aux/binpac/lib/binpac_buffer.cc:213
> #5  binpac::FlowBuffer::ClearPreviousData (this=<optimized out>) at /home/bro/src/bro/aux/binpac/lib/binpac_buffer.cc:208
> #6  0x000000000068d246 in binpac::FlowBuffer::NewData (this=0xb1b08c0, begin=0xbfcf7a0 "\300\205", end=0xbfcf7ad "") at /home/bro/src/bro/aux/binpac/lib/binpac_buffer.cc:176
> #7  0x0000000000503ce0 in binpac::ModbusTCP::ModbusTCP_Flow::NewData (this=0x46c1f90, t_begin_of_data=<optimized out>, t_end_of_data=<optimized out>) at /home/bro/src/bro/build/src/modbus_pac.cc:2867
> #8  0x000000000052169d in Analyzer::NextStream (this=0xb1a3640, len=<optimized out>, data=<optimized out>, is_orig=<optimized out>) at /home/bro/src/bro/src/Analyzer.cc:369
> #9  0x00000000005222b6 in Analyzer::ForwardStream (this=0xb19b240, len=13, data=0xbfcf7a0 "\300\205", is_orig=false) at /home/bro/src/bro/src/Analyzer.cc:456
> #10 0x0000000000646e80 in TCP_Reassembler::DeliverBlock (this=0x5fdf8c0, seq=15, len=13, data=0xbfcf7a0 "\300\205") at /home/bro/src/bro/src/TCP_Reassembler.cc:618
> #11 0x00000000006471aa in BlockInserted (start_block=<optimized out>, this=<optimized out>) at /home/bro/src/bro/src/TCP_Reassembler.cc:359
> #12 TCP_Reassembler::BlockInserted (this=0x5fdf8c0, start_block=<optimized out>) at /home/bro/src/bro/src/TCP_Reassembler.cc:334
> #13 0x0000000000646d28 in TCP_Reassembler::DataSent (this=0x5fdf8c0, t=<optimized out>, seq=<optimized out>, len=<optimized out>, data=<optimized out>, replaying=<optimized out>) at /home/bro/src/bro/src/TCP_Reassembler.cc:458
> #14 0x0000000000645cc6 in TCP_Endpoint::DataSent (this=0xb1a8790, t=<optimized out>, seq=15, len=13, caplen=13, data=0x7f46e2eefffe <Address 0x7f46e2eefffe out of bounds>, ip=<optimized out>, tp=0x7f46e2eeffea) at /home/bro/src/bro/src/TCP_Endpoint.cc:183
> #15 0x00000000006446f8 in TCP_Analyzer::DeliverPacket (this=0xb19b240, len=13, data=0x7f46e2eefffe <Address 0x7f46e2eefffe out of bounds>, is_orig=false, seq=<optimized out>, ip=0x7fff69511bf0, caplen=13) at /home/bro/src/bro/src/TCP.cc:1039
> #16 0x0000000000521571 in Analyzer::NextPacket (this=0xb19b240, len=<optimized out>, data=<optimized out>, is_orig=<optimized out>, seq=<optimized out>, ip=<optimized out>, caplen=33) at /home/bro/src/bro/src/Analyzer.cc:341
> #17 0x000000000053aa70 in Connection::NextPacket (this=<optimized out>, t=<optimized out>, is_orig=<optimized out>, ip=<optimized out>, len=<optimized out>, caplen=<optimized out>, data=<optimized out>, record_packet=@0x7fff69511868: 1, record_content=@0x7fff6951186c: 1, hdr=0x1abd040, pkt=0x7f46e2eeffc8 <Address 0x7f46e2eeffc8 out of bounds>, hdr_size=14) at /home/bro/src/bro/src/Conn.cc:259
> #18 0x000000000062e2f0 in NetSessions::DoNextPacket (this=0x2bd0c00, t=1352833032.1424849, hdr=0x1abd040, ip_hdr=0x7fff69511bf0, pkt=0x7f46e2eeffc8 <Address 0x7f46e2eeffc8 out of bounds>, hdr_size=14, encapsulation=0x0) at /home/bro/src/bro/src/Sessions.cc:700
> #19 0x000000000062f8c5 in NetSessions::NextPacket (this=0x2bd0c00, t=1352833032.1424849, hdr=0x1abd040, pkt=0x7f46e2eeffc8 <Address 0x7f46e2eeffc8 out of bounds>, hdr_size=14, pkt_elem=<optimized out>) at /home/bro/src/bro/src/Sessions.cc:238
> #20 0x00000000005ec14b in net_packet_dispatch (t=1352833032.1424849, hdr=0x1abd040, pkt=0x7f46e2eeffc8 <Address 0x7f46e2eeffc8 out of bounds>, hdr_size=14, src_ps=0x1abd000, pkt_elem=0x0) at /home/bro/src/bro/src/Net.cc:353
> #21 0x00000000005fb0cf in Process (this=0x1abd000) at /home/bro/src/bro/src/PktSrc.cc:303
> #22 PktSrc::Process (this=0x1abd000) at /home/bro/src/bro/src/PktSrc.cc:175
> #23 0x00000000005ec547 in net_run () at /home/bro/src/bro/src/Net.cc:446
> #24 0x00000000004c06ea in main (argc=<optimized out>, argv=<optimized out>) at /home/bro/src/bro/src/main.cc:1073
> 
> ==== No reporter.log
> 
> ==== stderr.log
> listening on eth4, capture length 8192 bytes
> 
> bro: /home/bro/src/bro/aux/binpac/lib/binpac_buffer.cc:213: void binpac::FlowBuffer::ClearPreviousData(): Assertion `buffer_n_ == 0' failed.
> /usr/local/bro/share/broctl/scripts/run-bro: line 60:  3282 Aborted                 (core dumped) nohup $mybro $@
> 
> 
> 
> On Nov 13, 2012, at 1:12 PM, Jonathan Siwek <jsiwek at ncsa.illinois.edu> wrote:
> 
>> Repository : ssh://git at bro-ids.icir.org/bro
>> 
>> On branch  : topic/jsiwek/modbus-fixes
>> Link       : http://tracker.bro-ids.org/bro/changeset/fd5eb23fa6ac654471d71645eb37dacc6d45896b/bro
>> 
>>> ---------------------------------------------------------------
>> 
>> commit fd5eb23fa6ac654471d71645eb37dacc6d45896b
>> Author: Jon Siwek <jsiwek at ncsa.illinois.edu>
>> Date:   Tue Nov 13 12:09:14 2012 -0600
>> 
>>   Remove byte count parameter from modbus events carrying register arrays
>> 
>>   Instead of these events being generated for invalid byte count values
>>   (they should always be even, not odd), a protocol_violation is raised.
>> 
>>       modbus_read_holding_registers_response
>>       modbus_read_input_registers_response
>>       modbus_write_multiple_registers_request
>>       modbus_read_write_multiple_registers_request
>>       modbus_read_write_multiple_registers_response
>>       modbus_read_fifo_queue_respons
>> 
>> 
>>> ---------------------------------------------------------------
>> 
>> fd5eb23fa6ac654471d71645eb37dacc6d45896b
>> scripts/policy/protocols/modbus/track-memmap.bro   |    2 +-
>> src/event.bif                                      |   24 ++-------
>> src/modbus-analyzer.pac                            |   50 +++++++++++++++++--
>> .../output                                         |    5 +-
>> .../btest/scripts/base/protocols/modbus/events.bro |   12 ++--
>> .../base/protocols/modbus/register_parsing.bro     |    9 ++--
>> 6 files changed, 64 insertions(+), 38 deletions(-)
>> 
>> diff --git a/scripts/policy/protocols/modbus/track-memmap.bro b/scripts/policy/protocols/modbus/track-memmap.bro
>> index cc02ce9..fc02d9b 100644
>> --- a/scripts/policy/protocols/modbus/track-memmap.bro
>> +++ b/scripts/policy/protocols/modbus/track-memmap.bro
>> @@ -60,7 +60,7 @@ event modbus_read_holding_registers_request(c: connection, headers: ModbusHeader
>> 	c$modbus$track_address = start_address+1;
>> 	}
>> 
>> -event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters)
>> +event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
>> 	{
>> 	local slave = c$id$resp_h;
>> 
>> diff --git a/src/event.bif b/src/event.bif
>> index cc8acb1..b965c26 100644
>> --- a/src/event.bif
>> +++ b/src/event.bif
>> @@ -6623,10 +6623,8 @@ event modbus_read_holding_registers_request%(c: connection, headers: ModbusHeade
>> ##
>> ## headers: The headers for the modbus function.
>> ##
>> -## byte_count: The number of bytes in the message that comprise register values.
>> -##
>> ## registers: The register values returned from the device.
>> -event modbus_read_holding_registers_response%(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters%);
>> +event modbus_read_holding_registers_response%(c: connection, headers: ModbusHeaders, registers: ModbusRegisters%);
>> 
>> ## Generated for a Modbus read input registers request.
>> ##
>> @@ -6645,10 +6643,8 @@ event modbus_read_input_registers_request%(c: connection, headers: ModbusHeaders
>> ##
>> ## headers: The headers for the modbus function.
>> ##
>> -## byte_count: The number of bytes in the message that comprise register values.
>> -##
>> ## registers: The register values returned from the device.
>> -event modbus_read_input_registers_response%(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters%);
>> +event modbus_read_input_registers_response%(c: connection, headers: ModbusHeaders, registers: ModbusRegisters%);
>> 
>> ## Generated for a Modbus write single coil request.
>> ##
>> @@ -6724,10 +6720,8 @@ event modbus_write_multiple_coils_response%(c: connection, headers: ModbusHeader
>> ##
>> ## start_address: The memory address of the first register to be written.
>> ##
>> -## byte_count: The number of bytes in the message that comprise register values.
>> -##
>> ## registers: The values to be written to the registers.
>> -event modbus_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, start_address: count, byte_count: count, registers: ModbusRegisters%);
>> +event modbus_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, start_address: count, registers: ModbusRegisters%);
>> 
>> ## Generated for a Modbus write multiple registers response.
>> ##
>> @@ -6818,10 +6812,8 @@ event modbus_mask_write_register_response%(c: connection, headers: ModbusHeaders
>> ##
>> ## write_start_address: The memory address of the first register to be written.
>> ##
>> -## write_byte_count: Number of bytes in message that comprise register values.
>> -##
>> ## write_registers: The values to be written to the registers.
>> -event modbus_read_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_byte_count: count, write_registers: ModbusRegisters%);
>> +event modbus_read_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_registers: ModbusRegisters%);
>> 
>> ## Generated for a Modbus read/write multiple registers response.
>> ##
>> @@ -6829,10 +6821,8 @@ event modbus_read_write_multiple_registers_request%(c: connection, headers: Modb
>> ##
>> ## headers: The headers for the modbus function.
>> ##
>> -## byte_count: The number of bytes in the message that comprise register values.
>> -##
>> ## written_registers: The register values read from the registers specified in the request.
>> -event modbus_read_write_multiple_registers_response%(c: connection, headers: ModbusHeaders, byte_count: count, written_registers: ModbusRegisters%);
>> +event modbus_read_write_multiple_registers_response%(c: connection, headers: ModbusHeaders, written_registers: ModbusRegisters%);
>> 
>> ## Generated for a Modbus read FIFO queue request.
>> ##
>> @@ -6849,10 +6839,8 @@ event modbus_read_fifo_queue_request%(c: connection, headers: ModbusHeaders, sta
>> ##
>> ## headers: The headers for the modbus function.
>> ##
>> -## byte_count: The number of bytes in the message that comprise register values.
>> -##
>> ## fifos: The register values read from the FIFO queue on the device.
>> -event modbus_read_fifo_queue_response%(c: connection, headers: ModbusHeaders, byte_count: count, fifos: ModbusRegisters%);
>> +event modbus_read_fifo_queue_response%(c: connection, headers: ModbusHeaders, fifos: ModbusRegisters%);
>> 
>> ## Raised for informational messages reported via Bro's reporter framework. Such
>> ## messages may be generated internally by the event engine and also by other
>> diff --git a/src/modbus-analyzer.pac b/src/modbus-analyzer.pac
>> index 155da96..b03df9d 100644
>> --- a/src/modbus-analyzer.pac
>> +++ b/src/modbus-analyzer.pac
>> @@ -135,8 +135,16 @@ refine flow ModbusTCP_Flow += {
>> 	# RESPONSE FC=3
>> 	function deliver_ReadHoldingRegistersResponse(header: ModbusTCP_TransportHeader, message: ReadHoldingRegistersResponse): bool
>> 		%{
>> +		if ( ${message.byte_count} % 2 != 0 )
>> +			{
>> +			connection()->bro_analyzer()->ProtocolViolation(
>> +			    fmt("invalid value for modbus read holding register response byte count %d", ${message.byte_count}));
>> +			return false;
>> +			}
>> +
>> 		if ( ::modbus_read_holding_registers_response )
>> 			{
>> +
>> 			VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
>> 			for ( unsigned int i=0; i < ${message.registers}->size(); ++i )
>> 				{
>> @@ -147,7 +155,6 @@ refine flow ModbusTCP_Flow += {
>> 			BifEvent::generate_modbus_read_holding_registers_response(connection()->bro_analyzer(),
>> 			                                                          connection()->bro_analyzer()->Conn(),
>> 			                                                          HeaderToBro(header),
>> -			                                                          ${message.byte_count},
>> 			                                                          t);
>> 			}
>> 
>> @@ -172,6 +179,13 @@ refine flow ModbusTCP_Flow += {
>> 	# RESPONSE FC=4
>> 	function deliver_ReadInputRegistersResponse(header: ModbusTCP_TransportHeader, message: ReadInputRegistersResponse): bool
>> 		%{
>> +		if ( ${message.byte_count} % 2 != 0 )
>> +			{
>> +			connection()->bro_analyzer()->ProtocolViolation(
>> +			    fmt("invalid value for modbus read input register response byte count %d", ${message.byte_count}));
>> +			return false;
>> +			}
>> +
>> 		if ( ::modbus_read_input_registers_response )
>> 			{
>> 			VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
>> @@ -184,7 +198,6 @@ refine flow ModbusTCP_Flow += {
>> 			BifEvent::generate_modbus_read_input_registers_response(connection()->bro_analyzer(),
>> 			                                                        connection()->bro_analyzer()->Conn(),
>> 			                                                        HeaderToBro(header),
>> -			                                                        ${message.byte_count},
>> 			                                                        t);
>> 			}
>> 
>> @@ -309,6 +322,13 @@ refine flow ModbusTCP_Flow += {
>> 	# REQUEST FC=16
>> 	function deliver_WriteMultipleRegistersRequest(header: ModbusTCP_TransportHeader, message: WriteMultipleRegistersRequest): bool
>> 		%{
>> +		if ( ${message.byte_count} % 2 != 0 )
>> +			{
>> +			connection()->bro_analyzer()->ProtocolViolation(
>> +			    fmt("invalid value for modbus write multiple registers request byte count %d", ${message.byte_count}));
>> +			return false;
>> +			}
>> +
>> 		if ( ::modbus_write_multiple_registers_request )
>> 			{
>> 			VectorVal * t = new VectorVal(BifType::Vector::ModbusRegisters);
>> @@ -321,7 +341,7 @@ refine flow ModbusTCP_Flow += {
>> 				BifEvent::generate_modbus_write_multiple_registers_request(connection()->bro_analyzer(),
>> 				                                                           connection()->bro_analyzer()->Conn(),
>> 				                                                           HeaderToBro(header),
>> -				                                                           ${message.start_address}, ${message.byte_count}, t);
>> +				                                                           ${message.start_address}, t);
>> 			}
>> 
>> 		return true;
>> @@ -486,6 +506,13 @@ refine flow ModbusTCP_Flow += {
>> 	# REQUEST FC=23
>> 	function deliver_ReadWriteMultipleRegistersRequest(header: ModbusTCP_TransportHeader, message: ReadWriteMultipleRegistersRequest): bool
>> 		%{
>> +		if ( ${message.write_byte_count} % 2 != 0 )
>> +			{
>> +			connection()->bro_analyzer()->ProtocolViolation(
>> +			    fmt("invalid value for modbus read write multiple registers request write byte count %d", ${message.write_byte_count}));
>> +			return false;
>> +			}
>> +
>> 		if ( ::modbus_read_write_multiple_registers_request )
>> 			{
>> 			VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
>> @@ -501,7 +528,6 @@ refine flow ModbusTCP_Flow += {
>> 			                                                                ${message.read_start_address},
>> 			                                                                ${message.read_quantity},
>> 			                                                                ${message.write_start_address},
>> -			                                                                ${message.write_byte_count},
>> 			                                                                t);
>> 			}
>> 
>> @@ -511,6 +537,13 @@ refine flow ModbusTCP_Flow += {
>> 	# RESPONSE FC=23
>> 	function deliver_ReadWriteMultipleRegistersResponse(header: ModbusTCP_TransportHeader, message: ReadWriteMultipleRegistersResponse): bool
>> 		%{
>> +		if ( ${message.byte_count} % 2 != 0 )
>> +			{
>> +			connection()->bro_analyzer()->ProtocolViolation(
>> +			    fmt("invalid value for modbus read write multiple registers response byte count %d", ${message.byte_count}));
>> +			return false;
>> +			}
>> +
>> 		if ( ::modbus_read_write_multiple_registers_response )
>> 			{
>> 			VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
>> @@ -523,7 +556,6 @@ refine flow ModbusTCP_Flow += {
>> 			BifEvent::generate_modbus_read_write_multiple_registers_response(connection()->bro_analyzer(),
>> 			                                                                 connection()->bro_analyzer()->Conn(),
>> 			                                                                 HeaderToBro(header),
>> -			                                                                 ${message.byte_count},
>> 			                                                                 t);
>> 			}
>> 
>> @@ -548,6 +580,13 @@ refine flow ModbusTCP_Flow += {
>> 	# RESPONSE FC=24
>> 	function deliver_ReadFIFOQueueResponse(header: ModbusTCP_TransportHeader, message: ReadFIFOQueueResponse): bool
>> 		%{
>> +		if ( ${message.byte_count} % 2 != 0 )
>> +			{
>> +			connection()->bro_analyzer()->ProtocolViolation(
>> +			    fmt("invalid value for modbus read FIFO queue response byte count %d", ${message.byte_count}));
>> +			return false;
>> +			}
>> +
>> 		if ( ::modbus_read_fifo_queue_response )
>> 			{
>> 			VectorVal* t = new VectorVal(new VectorType(base_type(TYPE_COUNT)));
>> @@ -560,7 +599,6 @@ refine flow ModbusTCP_Flow += {
>> 			BifEvent::generate_modbus_read_fifo_queue_response(connection()->bro_analyzer(),
>> 			                                                   connection()->bro_analyzer()->Conn(),
>> 			                                                   HeaderToBro(header),
>> -			                                                   ${message.byte_count},
>> 			                                                   t);
>> 			}
>> 
>> diff --git a/testing/btest/Baseline/scripts.base.protocols.modbus.register_parsing/output b/testing/btest/Baseline/scripts.base.protocols.modbus.register_parsing/output
>> index 353f85d..5bb5f1b 100644
>> --- a/testing/btest/Baseline/scripts.base.protocols.modbus.register_parsing/output
>> +++ b/testing/btest/Baseline/scripts.base.protocols.modbus.register_parsing/output
>> @@ -1,5 +1,4 @@
>> modbus_read_input_registers_request, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=1119, pid=0, uid=255, function_code=4], 900, 147
>> -modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=2606, pid=0, uid=255, function_code=4], [0, 0, 0, 0, 0, 0, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690], 100, 200
>> -modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=6714, pid=0, uid=255, function_code=4], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3840, 0, 0, 31, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 37, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 64, 129
>> +modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=2606, pid=0, uid=255, function_code=4], [0, 0, 0, 0, 0, 0, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690], 100
>> modbus_read_input_registers_request, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=12993, pid=0, uid=255, function_code=4], 400, 100
>> -modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=17667, pid=0, uid=255, function_code=4], [49, 18012, 51, 42, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 49, 50, 51, 54324, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 69, 63, 64, 65, 66, 67, 68, 49, 189, 51, 52, 53, 54, 4151, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 136, 49, 50, 51, 212, 53, 54, 170, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690], 100, 200
>> +modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=17667, pid=0, uid=255, function_code=4], [49, 18012, 51, 42, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 49, 50, 51, 54324, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 69, 63, 64, 65, 66, 67, 68, 49, 189, 51, 52, 53, 54, 4151, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 136, 49, 50, 51, 212, 53, 54, 170, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690], 100
>> diff --git a/testing/btest/scripts/base/protocols/modbus/events.bro b/testing/btest/scripts/base/protocols/modbus/events.bro
>> index 6c47dc6..f648a0a 100644
>> --- a/testing/btest/scripts/base/protocols/modbus/events.bro
>> +++ b/testing/btest/scripts/base/protocols/modbus/events.bro
>> @@ -41,7 +41,7 @@ event modbus_read_holding_registers_request(c: connection, headers: ModbusHeader
>>    print "modbus_read_holding_registers_request", c, headers, start_address, quantity;
>> }
>> 
>> -event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters)
>> +event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
>> {
>>    print "modbus_read_holding_registers_response", c, headers, registers;
>> }
>> @@ -51,7 +51,7 @@ event modbus_read_input_registers_request(c: connection, headers: ModbusHeaders,
>>    print "modbus_read_input_registers_request", c, headers, start_address, quantity;
>> }
>> 
>> -event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters)
>> +event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
>> {
>>    print "modbus_read_input_registers_response", c, headers, registers;
>> }
>> @@ -86,7 +86,7 @@ event modbus_write_multiple_coils_response(c: connection, headers: ModbusHeaders
>>    print "modbus_write_multiple_coils_response", c, headers, start_address, quantity;
>> }
>> 
>> -event modbus_write_multiple_registers_request(c: connection, headers: ModbusHeaders, start_address: count, byte_count: count, registers: ModbusRegisters)
>> +event modbus_write_multiple_registers_request(c: connection, headers: ModbusHeaders, start_address: count, registers: ModbusRegisters)
>> {
>>    print "modbus_write_multiple_registers_request", c, headers, start_address, registers;
>> }
>> @@ -126,12 +126,12 @@ event modbus_mask_write_register_response(c: connection, headers: ModbusHeaders,
>>    print "modbus_mask_write_register_response", c, headers, address, and_mask, or_mask;
>> }
>> 
>> -event modbus_read_write_multiple_registers_request(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_byte_count: count, write_registers: ModbusRegisters)
>> +event modbus_read_write_multiple_registers_request(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_registers: ModbusRegisters)
>> {
>>    print "modbus_read_write_multiple_registers_request", c, headers, read_start_address, read_quantity, write_start_address, write_registers;
>> }
>> 
>> -event modbus_read_write_multiple_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, written_registers: ModbusRegisters)
>> +event modbus_read_write_multiple_registers_response(c: connection, headers: ModbusHeaders, written_registers: ModbusRegisters)
>> {
>>    print "modbus_read_write_multiple_registers_response", c, headers, written_registers;
>> }
>> @@ -141,7 +141,7 @@ event modbus_read_fifo_queue_request(c: connection, headers: ModbusHeaders, star
>>    print "modbus_read_fifo_queue_request", c, headers, start_address;
>> }
>> 
>> -event modbus_read_fifo_queue_response(c: connection, headers: ModbusHeaders, byte_count: count, fifos: ModbusRegisters)
>> +event modbus_read_fifo_queue_response(c: connection, headers: ModbusHeaders, fifos: ModbusRegisters)
>> {
>>    print "modbus_read_fifo_queue_response", c, headers, fifos;
>> }
>> diff --git a/testing/btest/scripts/base/protocols/modbus/register_parsing.bro b/testing/btest/scripts/base/protocols/modbus/register_parsing.bro
>> index 300dd75..1641860 100644
>> --- a/testing/btest/scripts/base/protocols/modbus/register_parsing.bro
>> +++ b/testing/btest/scripts/base/protocols/modbus/register_parsing.bro
>> @@ -6,15 +6,16 @@
>> # of register values, with the quantity being derived from a byte count value
>> # that is also sent.  If the byte count value is invalid (e.g. an odd value
>> # might not be valid since registers must be 2-byte values), then the parser
>> -# should not trigger any asserts, but the resulting event could indicate
>> -# the strangeness (i.e. byte_count != 2*|registers|).
>> +# should not trigger any asserts, but generate a protocol_violation (in this
>> +# case TCP_ApplicationAnalyzer::ProtocolViolation asserts its behavior for
>> +# incomplete connections).
>> 
>> event modbus_read_input_registers_request(c: connection, headers: ModbusHeaders, start_address: count, quantity: count)
>> 	{
>> 	print "modbus_read_input_registers_request", c$id, headers, start_address, quantity;
>> 	}
>> 
>> -event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters)
>> +event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
>> 	{
>> -	print "modbus_read_input_registers_response", c$id, headers, registers, |registers|, byte_count;
>> +	print "modbus_read_input_registers_response", c$id, headers, registers, |registers|;
>> 	}
>> 
>> _______________________________________________
>> bro-commits mailing list
>> bro-commits at bro-ids.org
>> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits
> 
> 
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
> 



From vladg at cmu.edu  Tue Nov 13 13:04:05 2012
From: vladg at cmu.edu (Vlad Grigorescu)
Date: Tue, 13 Nov 2012 21:04:05 +0000
Subject: [Bro-Dev] [Bro-Commits]
	[git/bro]	topic/jsiwek/modbus-fixes:	Remove byte	count
	parameter from	modbus events carrying	register arrays	(fd5eb23)
In-Reply-To: <21645_1352838929_qADKZR1b001072_B3E27C90B7D2DD4CAEF09215A29EB0D116ED9D64@CITESMBX2.ad.uillinois.edu>
References: <2391_1352830357_qADICZJB014281_201211131812.qADICS0j011771@bro-ids.icir.org>
	<1202BE242E080642B0CD0AD0A03E855261E206@PGH-MSGMB-03.andrew.ad.cmu.edu>
	<21645_1352838929_qADKZR1b001072_B3E27C90B7D2DD4CAEF09215A29EB0D116ED9D64@CITESMBX2.ad.uillinois.edu>
Message-ID: <1202BE242E080642B0CD0AD0A03E855261E735@PGH-MSGMB-03.andrew.ad.cmu.edu>

Ah, I didn't catch those commits. Updated to use both, and so far (*knock on wood*) it hasn't crashed. Statistically speaking, the buggy code should've crashed by now. I'll continue to monitor for any issues.

Thanks,

  --Vlad


On Nov 13, 2012, at 3:35 PM, "Siwek, Jonathan Luke" <jsiwek at illinois.edu>
 wrote:

> 
> On Nov 13, 2012, at 1:11 PM, Vlad Grigorescu <vladg at cmu.edu> wrote:
> 
>> I'm still seeing crashes with this commit (trace included below). Should I open a new ticket for this? I don't want to latch onto the merge request at #917 unnecessarily. Thanks,
> 
> Did you also checkout the changes I did in the aux/binpac repo (I did changes in branch "topic/jsiwek/modbus-fixes" in both bro and aux/binpac) ?
> 
> That stack trace looks similar to something I encountered that needed a change in binpac.
> 
>    Jon
> 
>> [New LWP 3282]
>> [Thread debugging using libthread_db enabled]
>> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
>> Core was generated by `/usr/local/bro/bin/bro -i eth4 -U .status -p broctl -p broctl-live -p local -p'.
>> Program terminated with signal 6, Aborted.
>> #0  0x00007f46f893f425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
>> 
>> Thread 1 (Thread 0x7f46fac7e780 (LWP 3282)):
>> #0  0x00007f46f893f425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
>> #1  0x00007f46f8942b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
>> #2  0x00007f46f89380ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
>> #3  0x00007f46f8938192 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
>> #4  0x000000000068cd9f in ClearPreviousData (this=<optimized out>) at /home/bro/src/bro/aux/binpac/lib/binpac_buffer.cc:213
>> #5  binpac::FlowBuffer::ClearPreviousData (this=<optimized out>) at /home/bro/src/bro/aux/binpac/lib/binpac_buffer.cc:208
>> #6  0x000000000068d246 in binpac::FlowBuffer::NewData (this=0xb1b08c0, begin=0xbfcf7a0 "\300\205", end=0xbfcf7ad "") at /home/bro/src/bro/aux/binpac/lib/binpac_buffer.cc:176
>> #7  0x0000000000503ce0 in binpac::ModbusTCP::ModbusTCP_Flow::NewData (this=0x46c1f90, t_begin_of_data=<optimized out>, t_end_of_data=<optimized out>) at /home/bro/src/bro/build/src/modbus_pac.cc:2867
>> #8  0x000000000052169d in Analyzer::NextStream (this=0xb1a3640, len=<optimized out>, data=<optimized out>, is_orig=<optimized out>) at /home/bro/src/bro/src/Analyzer.cc:369
>> #9  0x00000000005222b6 in Analyzer::ForwardStream (this=0xb19b240, len=13, data=0xbfcf7a0 "\300\205", is_orig=false) at /home/bro/src/bro/src/Analyzer.cc:456
>> #10 0x0000000000646e80 in TCP_Reassembler::DeliverBlock (this=0x5fdf8c0, seq=15, len=13, data=0xbfcf7a0 "\300\205") at /home/bro/src/bro/src/TCP_Reassembler.cc:618
>> #11 0x00000000006471aa in BlockInserted (start_block=<optimized out>, this=<optimized out>) at /home/bro/src/bro/src/TCP_Reassembler.cc:359
>> #12 TCP_Reassembler::BlockInserted (this=0x5fdf8c0, start_block=<optimized out>) at /home/bro/src/bro/src/TCP_Reassembler.cc:334
>> #13 0x0000000000646d28 in TCP_Reassembler::DataSent (this=0x5fdf8c0, t=<optimized out>, seq=<optimized out>, len=<optimized out>, data=<optimized out>, replaying=<optimized out>) at /home/bro/src/bro/src/TCP_Reassembler.cc:458
>> #14 0x0000000000645cc6 in TCP_Endpoint::DataSent (this=0xb1a8790, t=<optimized out>, seq=15, len=13, caplen=13, data=0x7f46e2eefffe <Address 0x7f46e2eefffe out of bounds>, ip=<optimized out>, tp=0x7f46e2eeffea) at /home/bro/src/bro/src/TCP_Endpoint.cc:183
>> #15 0x00000000006446f8 in TCP_Analyzer::DeliverPacket (this=0xb19b240, len=13, data=0x7f46e2eefffe <Address 0x7f46e2eefffe out of bounds>, is_orig=false, seq=<optimized out>, ip=0x7fff69511bf0, caplen=13) at /home/bro/src/bro/src/TCP.cc:1039
>> #16 0x0000000000521571 in Analyzer::NextPacket (this=0xb19b240, len=<optimized out>, data=<optimized out>, is_orig=<optimized out>, seq=<optimized out>, ip=<optimized out>, caplen=33) at /home/bro/src/bro/src/Analyzer.cc:341
>> #17 0x000000000053aa70 in Connection::NextPacket (this=<optimized out>, t=<optimized out>, is_orig=<optimized out>, ip=<optimized out>, len=<optimized out>, caplen=<optimized out>, data=<optimized out>, record_packet=@0x7fff69511868: 1, record_content=@0x7fff6951186c: 1, hdr=0x1abd040, pkt=0x7f46e2eeffc8 <Address 0x7f46e2eeffc8 out of bounds>, hdr_size=14) at /home/bro/src/bro/src/Conn.cc:259
>> #18 0x000000000062e2f0 in NetSessions::DoNextPacket (this=0x2bd0c00, t=1352833032.1424849, hdr=0x1abd040, ip_hdr=0x7fff69511bf0, pkt=0x7f46e2eeffc8 <Address 0x7f46e2eeffc8 out of bounds>, hdr_size=14, encapsulation=0x0) at /home/bro/src/bro/src/Sessions.cc:700
>> #19 0x000000000062f8c5 in NetSessions::NextPacket (this=0x2bd0c00, t=1352833032.1424849, hdr=0x1abd040, pkt=0x7f46e2eeffc8 <Address 0x7f46e2eeffc8 out of bounds>, hdr_size=14, pkt_elem=<optimized out>) at /home/bro/src/bro/src/Sessions.cc:238
>> #20 0x00000000005ec14b in net_packet_dispatch (t=1352833032.1424849, hdr=0x1abd040, pkt=0x7f46e2eeffc8 <Address 0x7f46e2eeffc8 out of bounds>, hdr_size=14, src_ps=0x1abd000, pkt_elem=0x0) at /home/bro/src/bro/src/Net.cc:353
>> #21 0x00000000005fb0cf in Process (this=0x1abd000) at /home/bro/src/bro/src/PktSrc.cc:303
>> #22 PktSrc::Process (this=0x1abd000) at /home/bro/src/bro/src/PktSrc.cc:175
>> #23 0x00000000005ec547 in net_run () at /home/bro/src/bro/src/Net.cc:446
>> #24 0x00000000004c06ea in main (argc=<optimized out>, argv=<optimized out>) at /home/bro/src/bro/src/main.cc:1073
>> 
>> ==== No reporter.log
>> 
>> ==== stderr.log
>> listening on eth4, capture length 8192 bytes
>> 
>> bro: /home/bro/src/bro/aux/binpac/lib/binpac_buffer.cc:213: void binpac::FlowBuffer::ClearPreviousData(): Assertion `buffer_n_ == 0' failed.
>> /usr/local/bro/share/broctl/scripts/run-bro: line 60:  3282 Aborted                 (core dumped) nohup $mybro $@
>> 
>> 
>> 
>> On Nov 13, 2012, at 1:12 PM, Jonathan Siwek <jsiwek at ncsa.illinois.edu> wrote:
>> 
>>> Repository : ssh://git at bro-ids.icir.org/bro
>>> 
>>> On branch  : topic/jsiwek/modbus-fixes
>>> Link       : http://tracker.bro-ids.org/bro/changeset/fd5eb23fa6ac654471d71645eb37dacc6d45896b/bro
>>> 
>>>> ---------------------------------------------------------------
>>> 
>>> commit fd5eb23fa6ac654471d71645eb37dacc6d45896b
>>> Author: Jon Siwek <jsiwek at ncsa.illinois.edu>
>>> Date:   Tue Nov 13 12:09:14 2012 -0600
>>> 
>>>  Remove byte count parameter from modbus events carrying register arrays
>>> 
>>>  Instead of these events being generated for invalid byte count values
>>>  (they should always be even, not odd), a protocol_violation is raised.
>>> 
>>>      modbus_read_holding_registers_response
>>>      modbus_read_input_registers_response
>>>      modbus_write_multiple_registers_request
>>>      modbus_read_write_multiple_registers_request
>>>      modbus_read_write_multiple_registers_response
>>>      modbus_read_fifo_queue_respons
>>> 
>>> 
>>>> ---------------------------------------------------------------
>>> 
>>> fd5eb23fa6ac654471d71645eb37dacc6d45896b
>>> scripts/policy/protocols/modbus/track-memmap.bro   |    2 +-
>>> src/event.bif                                      |   24 ++-------
>>> src/modbus-analyzer.pac                            |   50 +++++++++++++++++--
>>> .../output                                         |    5 +-
>>> .../btest/scripts/base/protocols/modbus/events.bro |   12 ++--
>>> .../base/protocols/modbus/register_parsing.bro     |    9 ++--
>>> 6 files changed, 64 insertions(+), 38 deletions(-)
>>> 
>>> diff --git a/scripts/policy/protocols/modbus/track-memmap.bro b/scripts/policy/protocols/modbus/track-memmap.bro
>>> index cc02ce9..fc02d9b 100644
>>> --- a/scripts/policy/protocols/modbus/track-memmap.bro
>>> +++ b/scripts/policy/protocols/modbus/track-memmap.bro
>>> @@ -60,7 +60,7 @@ event modbus_read_holding_registers_request(c: connection, headers: ModbusHeader
>>> 	c$modbus$track_address = start_address+1;
>>> 	}
>>> 
>>> -event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters)
>>> +event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
>>> 	{
>>> 	local slave = c$id$resp_h;
>>> 
>>> diff --git a/src/event.bif b/src/event.bif
>>> index cc8acb1..b965c26 100644
>>> --- a/src/event.bif
>>> +++ b/src/event.bif
>>> @@ -6623,10 +6623,8 @@ event modbus_read_holding_registers_request%(c: connection, headers: ModbusHeade
>>> ##
>>> ## headers: The headers for the modbus function.
>>> ##
>>> -## byte_count: The number of bytes in the message that comprise register values.
>>> -##
>>> ## registers: The register values returned from the device.
>>> -event modbus_read_holding_registers_response%(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters%);
>>> +event modbus_read_holding_registers_response%(c: connection, headers: ModbusHeaders, registers: ModbusRegisters%);
>>> 
>>> ## Generated for a Modbus read input registers request.
>>> ##
>>> @@ -6645,10 +6643,8 @@ event modbus_read_input_registers_request%(c: connection, headers: ModbusHeaders
>>> ##
>>> ## headers: The headers for the modbus function.
>>> ##
>>> -## byte_count: The number of bytes in the message that comprise register values.
>>> -##
>>> ## registers: The register values returned from the device.
>>> -event modbus_read_input_registers_response%(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters%);
>>> +event modbus_read_input_registers_response%(c: connection, headers: ModbusHeaders, registers: ModbusRegisters%);
>>> 
>>> ## Generated for a Modbus write single coil request.
>>> ##
>>> @@ -6724,10 +6720,8 @@ event modbus_write_multiple_coils_response%(c: connection, headers: ModbusHeader
>>> ##
>>> ## start_address: The memory address of the first register to be written.
>>> ##
>>> -## byte_count: The number of bytes in the message that comprise register values.
>>> -##
>>> ## registers: The values to be written to the registers.
>>> -event modbus_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, start_address: count, byte_count: count, registers: ModbusRegisters%);
>>> +event modbus_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, start_address: count, registers: ModbusRegisters%);
>>> 
>>> ## Generated for a Modbus write multiple registers response.
>>> ##
>>> @@ -6818,10 +6812,8 @@ event modbus_mask_write_register_response%(c: connection, headers: ModbusHeaders
>>> ##
>>> ## write_start_address: The memory address of the first register to be written.
>>> ##
>>> -## write_byte_count: Number of bytes in message that comprise register values.
>>> -##
>>> ## write_registers: The values to be written to the registers.
>>> -event modbus_read_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_byte_count: count, write_registers: ModbusRegisters%);
>>> +event modbus_read_write_multiple_registers_request%(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_registers: ModbusRegisters%);
>>> 
>>> ## Generated for a Modbus read/write multiple registers response.
>>> ##
>>> @@ -6829,10 +6821,8 @@ event modbus_read_write_multiple_registers_request%(c: connection, headers: Modb
>>> ##
>>> ## headers: The headers for the modbus function.
>>> ##
>>> -## byte_count: The number of bytes in the message that comprise register values.
>>> -##
>>> ## written_registers: The register values read from the registers specified in the request.
>>> -event modbus_read_write_multiple_registers_response%(c: connection, headers: ModbusHeaders, byte_count: count, written_registers: ModbusRegisters%);
>>> +event modbus_read_write_multiple_registers_response%(c: connection, headers: ModbusHeaders, written_registers: ModbusRegisters%);
>>> 
>>> ## Generated for a Modbus read FIFO queue request.
>>> ##
>>> @@ -6849,10 +6839,8 @@ event modbus_read_fifo_queue_request%(c: connection, headers: ModbusHeaders, sta
>>> ##
>>> ## headers: The headers for the modbus function.
>>> ##
>>> -## byte_count: The number of bytes in the message that comprise register values.
>>> -##
>>> ## fifos: The register values read from the FIFO queue on the device.
>>> -event modbus_read_fifo_queue_response%(c: connection, headers: ModbusHeaders, byte_count: count, fifos: ModbusRegisters%);
>>> +event modbus_read_fifo_queue_response%(c: connection, headers: ModbusHeaders, fifos: ModbusRegisters%);
>>> 
>>> ## Raised for informational messages reported via Bro's reporter framework. Such
>>> ## messages may be generated internally by the event engine and also by other
>>> diff --git a/src/modbus-analyzer.pac b/src/modbus-analyzer.pac
>>> index 155da96..b03df9d 100644
>>> --- a/src/modbus-analyzer.pac
>>> +++ b/src/modbus-analyzer.pac
>>> @@ -135,8 +135,16 @@ refine flow ModbusTCP_Flow += {
>>> 	# RESPONSE FC=3
>>> 	function deliver_ReadHoldingRegistersResponse(header: ModbusTCP_TransportHeader, message: ReadHoldingRegistersResponse): bool
>>> 		%{
>>> +		if ( ${message.byte_count} % 2 != 0 )
>>> +			{
>>> +			connection()->bro_analyzer()->ProtocolViolation(
>>> +			    fmt("invalid value for modbus read holding register response byte count %d", ${message.byte_count}));
>>> +			return false;
>>> +			}
>>> +
>>> 		if ( ::modbus_read_holding_registers_response )
>>> 			{
>>> +
>>> 			VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
>>> 			for ( unsigned int i=0; i < ${message.registers}->size(); ++i )
>>> 				{
>>> @@ -147,7 +155,6 @@ refine flow ModbusTCP_Flow += {
>>> 			BifEvent::generate_modbus_read_holding_registers_response(connection()->bro_analyzer(),
>>> 			                                                          connection()->bro_analyzer()->Conn(),
>>> 			                                                          HeaderToBro(header),
>>> -			                                                          ${message.byte_count},
>>> 			                                                          t);
>>> 			}
>>> 
>>> @@ -172,6 +179,13 @@ refine flow ModbusTCP_Flow += {
>>> 	# RESPONSE FC=4
>>> 	function deliver_ReadInputRegistersResponse(header: ModbusTCP_TransportHeader, message: ReadInputRegistersResponse): bool
>>> 		%{
>>> +		if ( ${message.byte_count} % 2 != 0 )
>>> +			{
>>> +			connection()->bro_analyzer()->ProtocolViolation(
>>> +			    fmt("invalid value for modbus read input register response byte count %d", ${message.byte_count}));
>>> +			return false;
>>> +			}
>>> +
>>> 		if ( ::modbus_read_input_registers_response )
>>> 			{
>>> 			VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
>>> @@ -184,7 +198,6 @@ refine flow ModbusTCP_Flow += {
>>> 			BifEvent::generate_modbus_read_input_registers_response(connection()->bro_analyzer(),
>>> 			                                                        connection()->bro_analyzer()->Conn(),
>>> 			                                                        HeaderToBro(header),
>>> -			                                                        ${message.byte_count},
>>> 			                                                        t);
>>> 			}
>>> 
>>> @@ -309,6 +322,13 @@ refine flow ModbusTCP_Flow += {
>>> 	# REQUEST FC=16
>>> 	function deliver_WriteMultipleRegistersRequest(header: ModbusTCP_TransportHeader, message: WriteMultipleRegistersRequest): bool
>>> 		%{
>>> +		if ( ${message.byte_count} % 2 != 0 )
>>> +			{
>>> +			connection()->bro_analyzer()->ProtocolViolation(
>>> +			    fmt("invalid value for modbus write multiple registers request byte count %d", ${message.byte_count}));
>>> +			return false;
>>> +			}
>>> +
>>> 		if ( ::modbus_write_multiple_registers_request )
>>> 			{
>>> 			VectorVal * t = new VectorVal(BifType::Vector::ModbusRegisters);
>>> @@ -321,7 +341,7 @@ refine flow ModbusTCP_Flow += {
>>> 				BifEvent::generate_modbus_write_multiple_registers_request(connection()->bro_analyzer(),
>>> 				                                                           connection()->bro_analyzer()->Conn(),
>>> 				                                                           HeaderToBro(header),
>>> -				                                                           ${message.start_address}, ${message.byte_count}, t);
>>> +				                                                           ${message.start_address}, t);
>>> 			}
>>> 
>>> 		return true;
>>> @@ -486,6 +506,13 @@ refine flow ModbusTCP_Flow += {
>>> 	# REQUEST FC=23
>>> 	function deliver_ReadWriteMultipleRegistersRequest(header: ModbusTCP_TransportHeader, message: ReadWriteMultipleRegistersRequest): bool
>>> 		%{
>>> +		if ( ${message.write_byte_count} % 2 != 0 )
>>> +			{
>>> +			connection()->bro_analyzer()->ProtocolViolation(
>>> +			    fmt("invalid value for modbus read write multiple registers request write byte count %d", ${message.write_byte_count}));
>>> +			return false;
>>> +			}
>>> +
>>> 		if ( ::modbus_read_write_multiple_registers_request )
>>> 			{
>>> 			VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
>>> @@ -501,7 +528,6 @@ refine flow ModbusTCP_Flow += {
>>> 			                                                                ${message.read_start_address},
>>> 			                                                                ${message.read_quantity},
>>> 			                                                                ${message.write_start_address},
>>> -			                                                                ${message.write_byte_count},
>>> 			                                                                t);
>>> 			}
>>> 
>>> @@ -511,6 +537,13 @@ refine flow ModbusTCP_Flow += {
>>> 	# RESPONSE FC=23
>>> 	function deliver_ReadWriteMultipleRegistersResponse(header: ModbusTCP_TransportHeader, message: ReadWriteMultipleRegistersResponse): bool
>>> 		%{
>>> +		if ( ${message.byte_count} % 2 != 0 )
>>> +			{
>>> +			connection()->bro_analyzer()->ProtocolViolation(
>>> +			    fmt("invalid value for modbus read write multiple registers response byte count %d", ${message.byte_count}));
>>> +			return false;
>>> +			}
>>> +
>>> 		if ( ::modbus_read_write_multiple_registers_response )
>>> 			{
>>> 			VectorVal* t = new VectorVal(BifType::Vector::ModbusRegisters);
>>> @@ -523,7 +556,6 @@ refine flow ModbusTCP_Flow += {
>>> 			BifEvent::generate_modbus_read_write_multiple_registers_response(connection()->bro_analyzer(),
>>> 			                                                                 connection()->bro_analyzer()->Conn(),
>>> 			                                                                 HeaderToBro(header),
>>> -			                                                                 ${message.byte_count},
>>> 			                                                                 t);
>>> 			}
>>> 
>>> @@ -548,6 +580,13 @@ refine flow ModbusTCP_Flow += {
>>> 	# RESPONSE FC=24
>>> 	function deliver_ReadFIFOQueueResponse(header: ModbusTCP_TransportHeader, message: ReadFIFOQueueResponse): bool
>>> 		%{
>>> +		if ( ${message.byte_count} % 2 != 0 )
>>> +			{
>>> +			connection()->bro_analyzer()->ProtocolViolation(
>>> +			    fmt("invalid value for modbus read FIFO queue response byte count %d", ${message.byte_count}));
>>> +			return false;
>>> +			}
>>> +
>>> 		if ( ::modbus_read_fifo_queue_response )
>>> 			{
>>> 			VectorVal* t = new VectorVal(new VectorType(base_type(TYPE_COUNT)));
>>> @@ -560,7 +599,6 @@ refine flow ModbusTCP_Flow += {
>>> 			BifEvent::generate_modbus_read_fifo_queue_response(connection()->bro_analyzer(),
>>> 			                                                   connection()->bro_analyzer()->Conn(),
>>> 			                                                   HeaderToBro(header),
>>> -			                                                   ${message.byte_count},
>>> 			                                                   t);
>>> 			}
>>> 
>>> diff --git a/testing/btest/Baseline/scripts.base.protocols.modbus.register_parsing/output b/testing/btest/Baseline/scripts.base.protocols.modbus.register_parsing/output
>>> index 353f85d..5bb5f1b 100644
>>> --- a/testing/btest/Baseline/scripts.base.protocols.modbus.register_parsing/output
>>> +++ b/testing/btest/Baseline/scripts.base.protocols.modbus.register_parsing/output
>>> @@ -1,5 +1,4 @@
>>> modbus_read_input_registers_request, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=1119, pid=0, uid=255, function_code=4], 900, 147
>>> -modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=2606, pid=0, uid=255, function_code=4], [0, 0, 0, 0, 0, 0, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690], 100, 200
>>> -modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=6714, pid=0, uid=255, function_code=4], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3840, 0, 0, 31, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 37, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 64, 129
>>> +modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=2606, pid=0, uid=255, function_code=4], [0, 0, 0, 0, 0, 0, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690], 100
>>> modbus_read_input_registers_request, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=12993, pid=0, uid=255, function_code=4], 400, 100
>>> -modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=17667, pid=0, uid=255, function_code=4], [49, 18012, 51, 42, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 49, 50, 51, 54324, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 69, 63, 64, 65, 66, 67, 68, 49, 189, 51, 52, 53, 54, 4151, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 136, 49, 50, 51, 212, 53, 54, 170, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690], 100, 200
>>> +modbus_read_input_registers_response, [orig_h=10.1.1.234, orig_p=51411/tcp, resp_h=10.10.5.104, resp_p=502/tcp], [tid=17667, pid=0, uid=255, function_code=4], [49, 18012, 51, 42, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 49, 50, 51, 54324, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 69, 63, 64, 65, 66, 67, 68, 49, 189, 51, 52, 53, 54, 4151, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 136, 49, 50, 51, 212, 53, 54, 170, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690, 43690], 100
>>> diff --git a/testing/btest/scripts/base/protocols/modbus/events.bro b/testing/btest/scripts/base/protocols/modbus/events.bro
>>> index 6c47dc6..f648a0a 100644
>>> --- a/testing/btest/scripts/base/protocols/modbus/events.bro
>>> +++ b/testing/btest/scripts/base/protocols/modbus/events.bro
>>> @@ -41,7 +41,7 @@ event modbus_read_holding_registers_request(c: connection, headers: ModbusHeader
>>>   print "modbus_read_holding_registers_request", c, headers, start_address, quantity;
>>> }
>>> 
>>> -event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters)
>>> +event modbus_read_holding_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
>>> {
>>>   print "modbus_read_holding_registers_response", c, headers, registers;
>>> }
>>> @@ -51,7 +51,7 @@ event modbus_read_input_registers_request(c: connection, headers: ModbusHeaders,
>>>   print "modbus_read_input_registers_request", c, headers, start_address, quantity;
>>> }
>>> 
>>> -event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters)
>>> +event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
>>> {
>>>   print "modbus_read_input_registers_response", c, headers, registers;
>>> }
>>> @@ -86,7 +86,7 @@ event modbus_write_multiple_coils_response(c: connection, headers: ModbusHeaders
>>>   print "modbus_write_multiple_coils_response", c, headers, start_address, quantity;
>>> }
>>> 
>>> -event modbus_write_multiple_registers_request(c: connection, headers: ModbusHeaders, start_address: count, byte_count: count, registers: ModbusRegisters)
>>> +event modbus_write_multiple_registers_request(c: connection, headers: ModbusHeaders, start_address: count, registers: ModbusRegisters)
>>> {
>>>   print "modbus_write_multiple_registers_request", c, headers, start_address, registers;
>>> }
>>> @@ -126,12 +126,12 @@ event modbus_mask_write_register_response(c: connection, headers: ModbusHeaders,
>>>   print "modbus_mask_write_register_response", c, headers, address, and_mask, or_mask;
>>> }
>>> 
>>> -event modbus_read_write_multiple_registers_request(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_byte_count: count, write_registers: ModbusRegisters)
>>> +event modbus_read_write_multiple_registers_request(c: connection, headers: ModbusHeaders, read_start_address: count, read_quantity: count, write_start_address: count, write_registers: ModbusRegisters)
>>> {
>>>   print "modbus_read_write_multiple_registers_request", c, headers, read_start_address, read_quantity, write_start_address, write_registers;
>>> }
>>> 
>>> -event modbus_read_write_multiple_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, written_registers: ModbusRegisters)
>>> +event modbus_read_write_multiple_registers_response(c: connection, headers: ModbusHeaders, written_registers: ModbusRegisters)
>>> {
>>>   print "modbus_read_write_multiple_registers_response", c, headers, written_registers;
>>> }
>>> @@ -141,7 +141,7 @@ event modbus_read_fifo_queue_request(c: connection, headers: ModbusHeaders, star
>>>   print "modbus_read_fifo_queue_request", c, headers, start_address;
>>> }
>>> 
>>> -event modbus_read_fifo_queue_response(c: connection, headers: ModbusHeaders, byte_count: count, fifos: ModbusRegisters)
>>> +event modbus_read_fifo_queue_response(c: connection, headers: ModbusHeaders, fifos: ModbusRegisters)
>>> {
>>>   print "modbus_read_fifo_queue_response", c, headers, fifos;
>>> }
>>> diff --git a/testing/btest/scripts/base/protocols/modbus/register_parsing.bro b/testing/btest/scripts/base/protocols/modbus/register_parsing.bro
>>> index 300dd75..1641860 100644
>>> --- a/testing/btest/scripts/base/protocols/modbus/register_parsing.bro
>>> +++ b/testing/btest/scripts/base/protocols/modbus/register_parsing.bro
>>> @@ -6,15 +6,16 @@
>>> # of register values, with the quantity being derived from a byte count value
>>> # that is also sent.  If the byte count value is invalid (e.g. an odd value
>>> # might not be valid since registers must be 2-byte values), then the parser
>>> -# should not trigger any asserts, but the resulting event could indicate
>>> -# the strangeness (i.e. byte_count != 2*|registers|).
>>> +# should not trigger any asserts, but generate a protocol_violation (in this
>>> +# case TCP_ApplicationAnalyzer::ProtocolViolation asserts its behavior for
>>> +# incomplete connections).
>>> 
>>> event modbus_read_input_registers_request(c: connection, headers: ModbusHeaders, start_address: count, quantity: count)
>>> 	{
>>> 	print "modbus_read_input_registers_request", c$id, headers, start_address, quantity;
>>> 	}
>>> 
>>> -event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, byte_count: count, registers: ModbusRegisters)
>>> +event modbus_read_input_registers_response(c: connection, headers: ModbusHeaders, registers: ModbusRegisters)
>>> 	{
>>> -	print "modbus_read_input_registers_response", c$id, headers, registers, |registers|, byte_count;
>>> +	print "modbus_read_input_registers_response", c$id, headers, registers, |registers|;
>>> 	}
>>> 
>>> _______________________________________________
>>> bro-commits mailing list
>>> bro-commits at bro-ids.org
>>> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits
>> 
>> 
>> _______________________________________________
>> bro-dev mailing list
>> bro-dev at bro-ids.org
>> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>> 
> 
> 
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev



From bro at tracker.bro-ids.org  Tue Nov 13 20:02:03 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 14 Nov 2012 04:02:03 -0000
Subject: [Bro-Dev] #917: topic/jsiwek/modbus-fixes
In-Reply-To: <048.9242150ea938281a198c127edc4ba389@tracker.bro-ids.org>
References: <048.9242150ea938281a198c127edc4ba389@tracker.bro-ids.org>
Message-ID: <063.6f0ac67e90a60175cb0446c8c89cfee4@tracker.bro-ids.org>

#917: topic/jsiwek/modbus-fixes
----------------------------+------------------------
  Reporter:  jsiwek         |      Owner:  robin
      Type:  Merge Request  |     Status:  closed
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:  fixed          |   Keywords:
----------------------------+------------------------
Changes (by robin):

 * owner:   => robin
 * status:  new => closed
 * resolution:   => fixed


Comment:

 In [759faa255c676f7bec61f4173d70dc9cb8b37791/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="759faa255c676f7bec61f4173d70dc9cb8b37791"
 Merge remote-tracking branch 'origin/topic/jsiwek/modbus-fixes'

 * origin/topic/jsiwek/modbus-fixes:
   Remove byte count parameter from modbus events carrying register arrays
   Adjust modbus register array parsing.
   Adjustments to modbus test cases.

 Closes #917.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/917#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From noreply at bro-ids.org  Wed Nov 14 00:00:02 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Wed, 14 Nov 2012 00:00:02 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211140800.qAE802IN016195@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |   908 [1] | carsten    |            | Normal | Need for function bytestring_to_double



 [1]       #908: http://tracker.bro-ids.org/bro/ticket/908


From robin at icir.org  Wed Nov 14 07:56:28 2012
From: robin at icir.org (Robin Sommer)
Date: Wed, 14 Nov 2012 07:56:28 -0800
Subject: [Bro-Dev] Modbus fixes merged
Message-ID: <20121114155628.GH75267@icir.org>

For those seeing Modbus problems, we have now fixes in master (thanks,
Jon!). Please try and let us know if you still see trouble.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From bro at tracker.bro-ids.org  Wed Nov 14 14:35:57 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 14 Nov 2012 22:35:57 -0000
Subject: [Bro-Dev] #908: Need for function bytestring_to_double
In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org>
References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org>
Message-ID: <064.6d53a737e36f60d9418b513a6fae3bc8@tracker.bro-ids.org>

#908: Need for function bytestring_to_double
----------------------------+------------------------
  Reporter:  carsten        |      Owner:  robin
      Type:  Merge Request  |     Status:  closed
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:  fixed          |   Keywords:
----------------------------+------------------------
Changes (by robin):

 * owner:   => robin
 * status:  needs information => closed
 * resolution:   => fixed


Comment:

 In [5508a5bb80c144fd41b1a0fb760a7e2840355f65/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="5508a5bb80c144fd41b1a0fb760a7e2840355f65"
 Merge remote-tracking branch 'origin/topic/dnthayer/bytes-to-double'

 * origin/topic/dnthayer/bytes-to-double:
   Portability improvement (data alignment issues)
   Add test cases for the bytestring_to_double BIF
   Add a new BIF "bytestring_to_double"

 Closes #908.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/908#comment:15>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Nov 15 01:52:57 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 15 Nov 2012 09:52:57 -0000
Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only
 print CSV header line
In-Reply-To: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
References: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
Message-ID: <064.e2e25b905215e6ba5b3614e1d2f60e45@tracker.bro-ids.org>

#912: Need option for ASCII logging writer to only print CSV header line
------------------------------+-------------------------------
  Reporter:  carsten          |      Owner:
      Type:  Feature Request  |     Status:  new
  Priority:  Normal           |  Milestone:  Bro2.2
 Component:  Bro              |    Version:  git/master
Resolution:                   |   Keywords:  logging csv ascii
------------------------------+-------------------------------

Comment (by carsten):

 When writing the previous comment, I had problems with the Trac GUI
 (looked like server overload causing rejections) and did not receive an
 e-mail confirmation of my changes, so I don't know if adding the comment
 has triggered all processes as it should. Therefore I just add today this
 comment to retrigger the normal ticket update process.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/912#comment:8>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Nov 15 08:09:42 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 15 Nov 2012 16:09:42 -0000
Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only
 print CSV header line
In-Reply-To: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
References: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
Message-ID: <064.429e4f37febc774ceab2b0c3b5ee8deb@tracker.bro-ids.org>

#912: Need option for ASCII logging writer to only print CSV header line
----------------------------+-------------------------------
  Reporter:  carsten        |      Owner:
      Type:  Merge Request  |     Status:  new
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:  logging csv ascii
----------------------------+-------------------------------
Changes (by robin):

 * type:  Feature Request => Merge Request


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/912#comment:9>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Nov 15 12:06:59 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 15 Nov 2012 20:06:59 -0000
Subject: [Bro-Dev] #918: topic/jsiwek/hook
Message-ID: <048.8bbad75d47a7d6400cc12123e7951748@tracker.bro-ids.org>

#918: topic/jsiwek/hook
---------------------------+------------------------
 Reporter:  jsiwek         |      Owner:
     Type:  Merge Request  |     Status:  new
 Priority:  Normal         |  Milestone:  Bro2.2
Component:  Bro            |    Version:  git/master
 Keywords:                 |
---------------------------+------------------------
 This branch is in `bro` and `broccoli` repos, and adds a new flavor of
 function called a "hook" that behaves like a "synchronous event".  See the
 documentation I added for more info on usage.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/918>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Nov 15 15:04:05 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 15 Nov 2012 23:04:05 -0000
Subject: [Bro-Dev] #919: topic/jsiwek/improve-enum-doc
Message-ID: <048.6394710ed0c343484e88f8806361c682@tracker.bro-ids.org>

#919: topic/jsiwek/improve-enum-doc
---------------------------+------------------------
 Reporter:  jsiwek         |      Owner:
     Type:  Merge Request  |     Status:  new
 Priority:  Normal         |  Milestone:  Bro2.2
Component:  Bro            |    Version:  git/master
 Keywords:                 |
---------------------------+------------------------
 This branch adjusts the auto-generated enum documentation to actually
 allow things that hold enum values to reference the specific enum type
 instead of just showing a generic "enum" type.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/919>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From noreply at bro-ids.org  Fri Nov 16 00:00:02 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Fri, 16 Nov 2012 00:00:02 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211160800.qAG802Ve026198@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |   912 [1] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [2] | jsiwek     |            | Normal | topic/jsiwek/hook [3]
            Bro |   919 [4] | jsiwek     |            | Normal | topic/jsiwek/improve-enum-doc [5]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | e080549    | Jon Siwek       | 2012-11-15 | Fix some warnings from sphinx when building docs. [6]



 [1]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [2]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [3]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook
 [4]       #919: http://tracker.bro-ids.org/bro/ticket/919
 [5] improve-enum-doc: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/improve-enum-doc
 [6] fastpath: http://tracker.bro-ids.org/bro/changeset/e0805498c6fd491bf56d51a4b3533c51c607d060/bro


From bro at tracker.bro-ids.org  Fri Nov 16 10:57:20 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 16 Nov 2012 18:57:20 -0000
Subject: [Bro-Dev] #80: Checking for the existance keys in
 multi-dimensional tables gives error message
In-Reply-To: <050.c01d8a094407695f72fd3d3d599c4f48@tracker.bro-ids.org>
References: <050.c01d8a094407695f72fd3d3d599c4f48@tracker.bro-ids.org>
Message-ID: <065.520deddebab8202f8a9cbc75aab4f498@tracker.bro-ids.org>

#80: Checking for the existance keys in multi-dimensional tables gives error
message
-----------------------+------------------------
  Reporter:  bernhard  |      Owner:
      Type:  Problem   |     Status:  seen
  Priority:  Low       |  Milestone:  Bro2.2
 Component:  Bro       |    Version:  git/master
Resolution:            |   Keywords:
-----------------------+------------------------

Comment (by jsiwek):

 In [56e359ca9d7e54b61ed8806324f6b0bb89e96315/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="56e359ca9d7e54b61ed8806324f6b0bb89e96315"
 Fix ambiguity between composite table index and record ctor expressions.

 For tables of a composite index type with the first type being a
 record, membership checks with an inline index key could be
 misinterpreted as a record constructor instead of an expression list.
 E.g, if the table type is "global t = table[conn_id, bool] of count",
 then checking membership like "[c$id, is_orig] in t" now works.

 Addresses #80.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/80#comment:11>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Nov 16 11:31:18 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 16 Nov 2012 19:31:18 -0000
Subject: [Bro-Dev] #80: Checking for the existance keys in
 multi-dimensional tables gives error message
In-Reply-To: <050.c01d8a094407695f72fd3d3d599c4f48@tracker.bro-ids.org>
References: <050.c01d8a094407695f72fd3d3d599c4f48@tracker.bro-ids.org>
Message-ID: <065.35ca6e51702b003ecbe975dd28fe99c0@tracker.bro-ids.org>

#80: Checking for the existance keys in multi-dimensional tables gives error
message
----------------------------+------------------------
  Reporter:  bernhard       |      Owner:
      Type:  Merge Request  |     Status:  seen
  Priority:  Low            |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:
----------------------------+------------------------
Changes (by jsiwek):

 * type:  Problem => Merge Request


Comment:

 Fix is in `topic/jsiwek/composite-table-index-fix`.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/80#comment:12>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From seth at icir.org  Fri Nov 16 11:54:20 2012
From: seth at icir.org (Seth Hall)
Date: Fri, 16 Nov 2012 14:54:20 -0500
Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/seth/metrics-merge:
	Metrics framework checkpoint. (d919507)
In-Reply-To: <201211160738.qAG7cEH6010497@bro-ids.icir.org>
References: <201211160738.qAG7cEH6010497@bro-ids.icir.org>
Message-ID: <5AB93D58-D626-4F65-A5AB-3BE47CE8B7E8@icir.org>


On Nov 16, 2012, at 2:38 AM, Seth Hall <seth at icir.org> wrote:

>    - New scan.bro merged in and reworked a bit.


Now would be a great time for the probabilistic cardinality measurement.  This has quite a bit of memory overhead right now. :)

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From bro at tracker.bro-ids.org  Fri Nov 16 12:33:58 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 16 Nov 2012 20:33:58 -0000
Subject: [Bro-Dev] #870: Merge Modbus analyzer
In-Reply-To: <047.8a1df8bdf67712135b5746491e183bd3@tracker.bro-ids.org>
References: <047.8a1df8bdf67712135b5746491e183bd3@tracker.bro-ids.org>
Message-ID: <062.a168cf8123ee8f3547fde19c3437eb39@tracker.bro-ids.org>

#870: Merge Modbus analyzer
-----------------------------+------------------------
  Reporter:  robin           |      Owner:
      Type:  Task            |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro2.2
 Component:  Bro             |    Version:  git/master
Resolution:  Solved/Applied  |   Keywords:
-----------------------------+------------------------
Changes (by seth):

 * status:  new => closed
 * resolution:   => Solved/Applied


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/870#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From robin at icir.org  Fri Nov 16 18:56:00 2012
From: robin at icir.org (Robin Sommer)
Date: Fri, 16 Nov 2012 18:56:00 -0800
Subject: [Bro-Dev] Hooks (Re: [Bro-Commits] [git/bro] topic/jsiwek/hook: Add
	new	function flavor called a "hook". (e0fb9eb))
In-Reply-To: <201211151948.qAFJmElK005666@bro-ids.icir.org>
References: <201211151948.qAFJmElK005666@bro-ids.icir.org>
Message-ID: <20121117025600.GA54980@icir.org>

That was fast! :)

> +        hook myhook(s: string) &priority=10
> +            {
> +            print "priority 10 myhook handler", s;
> +            s = "bye";
> +            }
> +
> +        hook myhook(s: string)
> +            {
> +            print "break out of myhook handling", s;
> +            break;
> +            }
> +
> +        hook myhook(s: string) &priority=-5
> +            {
> +            print "not going to happen", s;
> +            }

[...]
> +        hook myhook("hi");

I haven't looked too closely yet but I want to mention one thing Seth
and I discussed earlier this week: we could give hooks a mandatory
boolean return value and use that instead of the "break" command:
"return T" would continue with the next hook, and "return F" stops. In
addition, the hook statement becomes an expression that returns the
boolean (i.e., T if all ran through, and F is one stopped execution).

Thinking now about it, I believe either way is fine. The boolean hook
would convey a bit more information back to the caller and allow
simple yes/no decisions that way. But I'm not sure we need that. On
the other hand, Seth didn't like the "break" so that would go away.

Wanted to put that up for discussion, not leaning strnonly either way
right now.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From seth at icir.org  Fri Nov 16 19:41:20 2012
From: seth at icir.org (Seth Hall)
Date: Fri, 16 Nov 2012 22:41:20 -0500
Subject: [Bro-Dev] Hooks (Re: [Bro-Commits] [git/bro] topic/jsiwek/hook:
	Add new	function flavor called a "hook". (e0fb9eb))
In-Reply-To: <20121117025600.GA54980@icir.org>
References: <201211151948.qAFJmElK005666@bro-ids.icir.org>
	<20121117025600.GA54980@icir.org>
Message-ID: <6A57852B-CF05-473F-95E3-CFD9D6F049A2@icir.org>


On Nov 16, 2012, at 9:56 PM, Robin Sommer <robin at icir.org> wrote:

> Thinking now about it, I believe either way is fine. The boolean hook
> would convey a bit more information back to the caller and allow
> simple yes/no decisions that way. But I'm not sure we need that. On
> the other hand, Seth didn't like the "break" so that would go away.

Jon actually convinced me in favor of "break" earlier today.  Something about needing to return from these hooks feels wrong.

I was pushing for the change to making this an expression earlier today too and having a return value (I was actually pushing for a return record that had more than just if the execution was broken) and Jon made a good point about the partial inconsistency with the hook prototype.

If we had the simple boolean return from hook as an expression?

   global my_hook: hook(foobar: string): bool;

It's weird because that's not the prototype for each hook handler (not sure what to call those?) since each hook handler doesn't have a return value.

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From noreply at bro-ids.org  Sat Nov 17 00:00:02 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Sat, 17 Nov 2012 00:00:02 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211170800.qAH802qO030116@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |    80 [1] | bernhard   |            | Low    | Checking for the existance keys in multi-dimensional tables gives error message
            Bro |   912 [2] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [3] | jsiwek     |            | Normal | topic/jsiwek/hook [4]
            Bro |   919 [5] | jsiwek     |            | Normal | topic/jsiwek/improve-enum-doc [6]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | e080549    | Jon Siwek       | 2012-11-15 | Fix some warnings from sphinx when building docs. [7]



 [1]        #80: http://tracker.bro-ids.org/bro/ticket/80
 [2]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [3]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [4]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook
 [5]       #919: http://tracker.bro-ids.org/bro/ticket/919
 [6] improve-enum-doc: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/improve-enum-doc
 [7] fastpath: http://tracker.bro-ids.org/bro/changeset/e0805498c6fd491bf56d51a4b3533c51c607d060/bro


From noreply at bro-ids.org  Sun Nov 18 00:00:02 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Sun, 18 Nov 2012 00:00:02 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211180800.qAI802Um019198@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |    80 [1] | bernhard   |            | Low    | Checking for the existance keys in multi-dimensional tables gives error message
            Bro |   912 [2] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [3] | jsiwek     |            | Normal | topic/jsiwek/hook [4]
            Bro |   919 [5] | jsiwek     |            | Normal | topic/jsiwek/improve-enum-doc [6]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | e080549    | Jon Siwek       | 2012-11-15 | Fix some warnings from sphinx when building docs. [7]



 [1]        #80: http://tracker.bro-ids.org/bro/ticket/80
 [2]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [3]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [4]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook
 [5]       #919: http://tracker.bro-ids.org/bro/ticket/919
 [6] improve-enum-doc: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/improve-enum-doc
 [7] fastpath: http://tracker.bro-ids.org/bro/changeset/e0805498c6fd491bf56d51a4b3533c51c607d060/bro


From vallentin at icir.org  Sun Nov 18 19:26:59 2012
From: vallentin at icir.org (Matthias Vallentin)
Date: Sun, 18 Nov 2012 19:26:59 -0800
Subject: [Bro-Dev] Proposing a new type: opaque
Message-ID: <20121119032659.GY24247@icir.org>

Bro currently misuses the any type to describe "handle" or "descriptor"
types. Consider for example the MD5 function family, which take a state
object as first argument. At the scripting layer, it has no other
purpose than mapping to the corresponding MD5 state at the core. Users
don't use it otherwise. There exists no dedicated handle type in Bro,
although it wouldn't be complicated to add one.

I am encountering this during the review of Soumya's cardinality counter
code. Before continuing with the merge, I propose to add a dedicated
opaque type that has the form:

    opaque of T

where T is an identifier from the parsers perspective. For example, this
would entail a signature change from

    md5_hash_init(idx: any): bool

to
    md5_hash_init(idx: opaque of MD5): bool

At script load time, Bro can ensure type safety by performing string
comparisons that come with the identifier. Such a type would finally add
structure to the process of adding new BiFs and related extensions. 

   Matthias

From noreply at bro-ids.org  Mon Nov 19 00:00:02 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Mon, 19 Nov 2012 00:00:02 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211190800.qAJ802IV020470@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |    80 [1] | bernhard   |            | Low    | Checking for the existance keys in multi-dimensional tables gives error message
            Bro |   912 [2] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [3] | jsiwek     |            | Normal | topic/jsiwek/hook [4]
            Bro |   919 [5] | jsiwek     |            | Normal | topic/jsiwek/improve-enum-doc [6]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | e080549    | Jon Siwek       | 2012-11-15 | Fix some warnings from sphinx when building docs. [7]



 [1]        #80: http://tracker.bro-ids.org/bro/ticket/80
 [2]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [3]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [4]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook
 [5]       #919: http://tracker.bro-ids.org/bro/ticket/919
 [6] improve-enum-doc: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/improve-enum-doc
 [7] fastpath: http://tracker.bro-ids.org/bro/changeset/e0805498c6fd491bf56d51a4b3533c51c607d060/bro


From seth at icir.org  Mon Nov 19 07:30:32 2012
From: seth at icir.org (Seth Hall)
Date: Mon, 19 Nov 2012 10:30:32 -0500
Subject: [Bro-Dev] Proposing a new type: opaque
In-Reply-To: <20121119032659.GY24247@icir.org>
References: <20121119032659.GY24247@icir.org>
Message-ID: <C643B124-15E1-4BC0-A1C5-9861A92C21E8@icir.org>


On Nov 18, 2012, at 10:26 PM, Matthias Vallentin <vallentin at icir.org> wrote:

> There exists no dedicated handle type in Bro,
> although it wouldn't be complicated to add one.

Yes, having a handle type in Bro would be incredibly handy.

> I am encountering this during the review of Soumya's cardinality counter
> code. Before continuing with the merge, I propose to add a dedicated
> opaque type that has the form:
> 
>    opaque of T

Not sure I like this.  It seems too similar to the current syntax of tables and vectors (I do want to see the vector syntax eventually though).  It looks like T would be a yield value.

In the spirit of keeping the discussion going I'll try and propose an alternate syntax. :)

>    md5_hash_init(idx: opaque of MD5): bool


opaque<MD5>

I'm not sure I like this either because it would be adding new tokens to the parser, but it is a pretty new concept to add to Bro too.

  .Seth


--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From vallentin at icir.org  Mon Nov 19 08:57:51 2012
From: vallentin at icir.org (Matthias Vallentin)
Date: Mon, 19 Nov 2012 08:57:51 -0800
Subject: [Bro-Dev] Proposing a new type: opaque
In-Reply-To: <C643B124-15E1-4BC0-A1C5-9861A92C21E8@icir.org>
References: <20121119032659.GY24247@icir.org>
	<C643B124-15E1-4BC0-A1C5-9861A92C21E8@icir.org>
Message-ID: <20121119165751.GC24247@icir.org>

> >    opaque of T
> 
> Not sure I like this.  It seems too similar to the current syntax of
> tables and vectors (I do want to see the vector syntax eventually
> though).  It looks like T would be a yield value.

I see the similarity to other language elements rather as a consistent
form of integration. Why would the "T of Y" syntax only pertain to
container types? Admittedly, we have currently no other uses, but looks
like a reasonable one to me.

> opaque<MD5>

While this looks good to folks familiar with C++ and templates, it looks
like an exceptional one-off to the Bro language. As you said, it would
involve tweaking both lexer and parser, and when is a about to happen,
we should rethink the added value.

   Matthias

From lethilieuctk32 at gmail.com  Mon Nov 19 10:21:47 2012
From: lethilieuctk32 at gmail.com (=?UTF-8?B?TGnDqnUgTMOqIFRo4buL?=)
Date: Tue, 20 Nov 2012 01:21:47 +0700
Subject: [Bro-Dev] wireshark word with BRO
Message-ID: <CAG8JgP61D+1wYLYiabUnKsiDPXiuEQmoDiNQUAG86CtVKROOxA@mail.gmail.com>

Hi, everyone.
Now, I'm having a problem with Bro and wireshark. I don't know how to bro
work with wireshak to analysis a trace captured by wireshark. Could have
 someone help me?
Thanks
Lieu
-- 
=========================================
*L? Th? Li?u*
Email: lethilieuctk32 at gmail.com
Mobile:01248262338; 0974437270
Life Guidelines:" ??ng ?? ??n ng?y mai nh?ng vi?c g? c? th? l?m h?m nay."
=========================================

???ng tuy ng?n kh?ng ?i kh?ng ??n
Vi?c tuy d? kh?ng l?m kh?ng xong
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20121120/bc8e3515/attachment.html 

From seth at icir.org  Mon Nov 19 10:36:54 2012
From: seth at icir.org (Seth Hall)
Date: Mon, 19 Nov 2012 13:36:54 -0500
Subject: [Bro-Dev] wireshark word with BRO
In-Reply-To: <CAG8JgP61D+1wYLYiabUnKsiDPXiuEQmoDiNQUAG86CtVKROOxA@mail.gmail.com>
References: <CAG8JgP61D+1wYLYiabUnKsiDPXiuEQmoDiNQUAG86CtVKROOxA@mail.gmail.com>
Message-ID: <3B833FE1-1D58-4BBC-934A-5561C3430599@icir.org>


On Nov 19, 2012, at 1:21 PM, Li?u L? Th? <lethilieuctk32 at gmail.com> wrote:

> Now, I'm having a problem with Bro and wireshark. I don't know how to bro work with wireshak to analysis a trace captured by wireshark. Could have  someone help me?

This is definitely the wrong list for this question.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From noreply at bro-ids.org  Tue Nov 20 00:00:02 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Tue, 20 Nov 2012 00:00:02 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211200800.qAK802jS031731@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |    80 [1] | bernhard   |            | Low    | Checking for the existance keys in multi-dimensional tables gives error message
            Bro |   912 [2] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [3] | jsiwek     |            | Normal | topic/jsiwek/hook [4]
            Bro |   919 [5] | jsiwek     |            | Normal | topic/jsiwek/improve-enum-doc [6]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | e080549    | Jon Siwek       | 2012-11-15 | Fix some warnings from sphinx when building docs. [7]



 [1]        #80: http://tracker.bro-ids.org/bro/ticket/80
 [2]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [3]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [4]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook
 [5]       #919: http://tracker.bro-ids.org/bro/ticket/919
 [6] improve-enum-doc: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/improve-enum-doc
 [7] fastpath: http://tracker.bro-ids.org/bro/changeset/e0805498c6fd491bf56d51a4b3533c51c607d060/bro


From seth at icir.org  Tue Nov 20 08:15:51 2012
From: seth at icir.org (Seth Hall)
Date: Tue, 20 Nov 2012 11:15:51 -0500
Subject: [Bro-Dev] test-all-policy.bro
Message-ID: <171CA866-CE7E-4BB6-B945-BF11BB600166@icir.org>

Am I supposed to maintain this file automatically? (i know that i should already know the answer to this question, but I don't).

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Tue Nov 20 08:21:33 2012
From: seth at icir.org (Seth Hall)
Date: Tue, 20 Nov 2012 11:21:33 -0500
Subject: [Bro-Dev] topic/seth/metrics-merge crash
Message-ID: <56BBF866-CE0A-4F8F-AC1C-60AE4221C22B@icir.org>

I'm seeing Bro actually crash for the scripts.base.frameworks.metrics.cluster-intermediate-update test and I'm having a hard time figuring out why it's crashing.  Would someone mind taking a look at that soon?

Thanks!
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From seth at icir.org  Tue Nov 20 08:29:14 2012
From: seth at icir.org (Seth Hall)
Date: Tue, 20 Nov 2012 11:29:14 -0500
Subject: [Bro-Dev] test-all-policy.bro
In-Reply-To: <171CA866-CE7E-4BB6-B945-BF11BB600166@icir.org>
References: <171CA866-CE7E-4BB6-B945-BF11BB600166@icir.org>
Message-ID: <82D960CE-3A6F-4FE5-93E4-53FC59B0A58D@icir.org>


On Nov 20, 2012, at 11:15 AM, Seth Hall <seth at icir.org> wrote:

> Am I supposed to maintain this file automatically?

Ok, that came out wrong.  Am I supposed to maintain this file or is there something to help me maintain it?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From jsiwek at illinois.edu  Tue Nov 20 08:33:59 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Tue, 20 Nov 2012 16:33:59 +0000
Subject: [Bro-Dev] Hooks (Re: [Bro-Commits] [git/bro] topic/jsiwek/hook:
	Add new	function flavor called a "hook". (e0fb9eb))
In-Reply-To: <20121117025600.GA54980@icir.org>
References: <201211151948.qAFJmElK005666@bro-ids.icir.org>
	<20121117025600.GA54980@icir.org>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116EED3D7@CITESMBX2.ad.uillinois.edu>


> I haven't looked too closely yet but I want to mention one thing Seth
> and I discussed earlier this week: we could give hooks a mandatory
> boolean return value and use that instead of the "break" command:
> "return T" would continue with the next hook, and "return F" stops. In
> addition, the hook statement becomes an expression that returns the
> boolean (i.e., T if all ran through, and F is one stopped execution).
> 
> Thinking now about it, I believe either way is fine. The boolean hook
> would convey a bit more information back to the caller and allow
> simple yes/no decisions that way. But I'm not sure we need that. On
> the other hand, Seth didn't like the "break" so that would go away.
> 
> Wanted to put that up for discussion, not leaning strnonly either way
> right now.

I don't have a strong opinion, either, but I'm leaning toward allowing a boolean return value since it's the more flexible and most similar/familiar to how functions work.

But I do think if it had a return value, then it shouldn't mix in "break" to allow implicitly returning values, but require "return" like normal.

If you want, you can hold off on merge until this is decided, it's probably not just a trivial change to do (but not likely too difficult either).

    Jon

From jsiwek at illinois.edu  Tue Nov 20 08:36:23 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Tue, 20 Nov 2012 16:36:23 +0000
Subject: [Bro-Dev] test-all-policy.bro
In-Reply-To: <171CA866-CE7E-4BB6-B945-BF11BB600166@icir.org>
References: <171CA866-CE7E-4BB6-B945-BF11BB600166@icir.org>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116EED3F8@CITESMBX2.ad.uillinois.edu>

> Am I supposed to maintain this file automatically? (i know that i should already know the answer to this question, but I don't).

Manually.

    Jon

From robin at icir.org  Tue Nov 20 08:51:33 2012
From: robin at icir.org (Robin Sommer)
Date: Tue, 20 Nov 2012 08:51:33 -0800
Subject: [Bro-Dev] Hooks (Re: [Bro-Commits] [git/bro] topic/jsiwek/hook:
	Add new	function flavor called a "hook". (e0fb9eb))
In-Reply-To: <B3E27C90B7D2DD4CAEF09215A29EB0D116EED3D7@CITESMBX2.ad.uillinois.edu>
References: <201211151948.qAFJmElK005666@bro-ids.icir.org>
	<20121117025600.GA54980@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116EED3D7@CITESMBX2.ad.uillinois.edu>
Message-ID: <20121120165133.GH19960@icir.org>


On Tue, Nov 20, 2012 at 16:33 +0000, you wrote:

> But I do think if it had a return value, then it shouldn't mix in
> "break" to allow implicitly returning values, but require "return"
> like normal.

Hmmm, good thought. Can we even get the best of both worlds by making
it all implicit: the hook definition doesn't define any return value,
but executing the hook still returns a boolean indicating whether one 
of the hook implementations short-circuuted with a "break" or not.

And, maybe, we could use a different syntax instead of "break"; like
"stop" or "hook stop", to make it more explicit what happens (not that
I like either particularlly).

> If you want, you can hold off on merge until this is decided, it's
> probably not just a trivial change to do (but not likely too difficult
> either).

Either way is fine for me, we can tweak further after the merge. I
would have merged it already but didn't get to it yet (and the rest of
the week doesn't look much better right now unfortunately). If
somebody depends on this for moving forward, please work from the
branch in the meantime.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org

From noreply at bro-ids.org  Wed Nov 21 00:00:02 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Wed, 21 Nov 2012 00:00:02 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211210800.qAL802GG023504@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |    80 [1] | bernhard   |            | Low    | Checking for the existance keys in multi-dimensional tables gives error message
            Bro |   912 [2] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [3] | jsiwek     |            | Normal | topic/jsiwek/hook [4]
            Bro |   919 [5] | jsiwek     |            | Normal | topic/jsiwek/improve-enum-doc [6]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | e080549    | Jon Siwek       | 2012-11-15 | Fix some warnings from sphinx when building docs. [7]



 [1]        #80: http://tracker.bro-ids.org/bro/ticket/80
 [2]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [3]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [4]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook
 [5]       #919: http://tracker.bro-ids.org/bro/ticket/919
 [6] improve-enum-doc: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/improve-enum-doc
 [7] fastpath: http://tracker.bro-ids.org/bro/changeset/e0805498c6fd491bf56d51a4b3533c51c607d060/bro


From seth at icir.org  Wed Nov 21 09:27:07 2012
From: seth at icir.org (Seth Hall)
Date: Wed, 21 Nov 2012 12:27:07 -0500
Subject: [Bro-Dev] [Bro-Commits] [git/bro] branch
	'topic/seth/analyzer-framework' created
In-Reply-To: <201211211720.qALHKmkw023688@bro-ids.icir.org>
References: <201211211720.qALHKmkw023688@bro-ids.icir.org>
Message-ID: <3A3A87C2-CB10-489B-A93C-60FF67FC3E5A@icir.org>


On Nov 21, 2012, at 12:20 PM, Seth Hall <seth at icir.org> wrote:

> Repository : ssh://git at bro-ids.icir.org/bro
> 
> New branch : topic/seth/analyzer-framework

This is a rename of the topic/seth/scripts-for-2.1 branch which never was merged into 2.1 and was really only the packet filter framework.  The analyzer framework and the packet filter framework will be benefitted from being developed together so I'm going to do both in this branch.

For the record, we'll be gaining an analyzers framework, but the dpd framework will merge into the analyzer framework so it should be break even in terms of number of frameworks (at the moment i have a "protocols" framework in this branch but it will merge into the analyzer framework too).

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From noreply at bro-ids.org  Thu Nov 22 00:00:02 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Thu, 22 Nov 2012 00:00:02 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211220800.qAM802YZ005738@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |    80 [1] | bernhard   |            | Low    | Checking for the existance keys in multi-dimensional tables gives error message
            Bro |   912 [2] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [3] | jsiwek     |            | Normal | topic/jsiwek/hook [4]
            Bro |   919 [5] | jsiwek     |            | Normal | topic/jsiwek/improve-enum-doc [6]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | e080549    | Jon Siwek       | 2012-11-15 | Fix some warnings from sphinx when building docs. [7]



 [1]        #80: http://tracker.bro-ids.org/bro/ticket/80
 [2]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [3]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [4]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook
 [5]       #919: http://tracker.bro-ids.org/bro/ticket/919
 [6] improve-enum-doc: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/improve-enum-doc
 [7] fastpath: http://tracker.bro-ids.org/bro/changeset/e0805498c6fd491bf56d51a4b3533c51c607d060/bro


From lethilieuctk32 at gmail.com  Thu Nov 22 17:12:29 2012
From: lethilieuctk32 at gmail.com (=?UTF-8?B?TGnDqnUgTMOqIFRo4buL?=)
Date: Fri, 23 Nov 2012 08:12:29 +0700
Subject: [Bro-Dev] Bro failed
Message-ID: <CAG8JgP6M8T-fDpGfSH1YjdAqXO0-O0U7H17b=0bBRP8gQCf6vA@mail.gmail.com>

Hi everyone.
I have a problem when start Bro IDS,
When I enter command check, it return:
[BroControl] > check
bro failed.
   /usr/local/bro/share/broctl/scripts/check-config: line 28:
/usr/local/bro/bin/bro: cannot execute binary file

Who tell me how to fix it?
thanks and Regard.
Lieu
-- 
=========================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20121123/d1f54067/attachment.html 

From noreply at bro-ids.org  Fri Nov 23 00:00:03 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Fri, 23 Nov 2012 00:00:03 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211230800.qAN803vS017939@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |    80 [1] | bernhard   |            | Low    | Checking for the existance keys in multi-dimensional tables gives error message
            Bro |   912 [2] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [3] | jsiwek     |            | Normal | topic/jsiwek/hook [4]
            Bro |   919 [5] | jsiwek     |            | Normal | topic/jsiwek/improve-enum-doc [6]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | e080549    | Jon Siwek       | 2012-11-15 | Fix some warnings from sphinx when building docs. [7]



 [1]        #80: http://tracker.bro-ids.org/bro/ticket/80
 [2]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [3]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [4]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook
 [5]       #919: http://tracker.bro-ids.org/bro/ticket/919
 [6] improve-enum-doc: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/improve-enum-doc
 [7] fastpath: http://tracker.bro-ids.org/bro/changeset/e0805498c6fd491bf56d51a4b3533c51c607d060/bro


From seth at icir.org  Fri Nov 23 06:26:00 2012
From: seth at icir.org (Seth Hall)
Date: Fri, 23 Nov 2012 09:26:00 -0500
Subject: [Bro-Dev] Bro failed
In-Reply-To: <CAG8JgP6M8T-fDpGfSH1YjdAqXO0-O0U7H17b=0bBRP8gQCf6vA@mail.gmail.com>
References: <CAG8JgP6M8T-fDpGfSH1YjdAqXO0-O0U7H17b=0bBRP8gQCf6vA@mail.gmail.com>
Message-ID: <D1AD1788-A111-4C27-B5F8-AE9E53E6F6B5@icir.org>


On Nov 22, 2012, at 8:12 PM, Li?u L? Th? <lethilieuctk32 at gmail.com> wrote:

> bro failed.
>    /usr/local/bro/share/broctl/scripts/check-config: line 28: /usr/local/bro/bin/bro: cannot execute binary file

Did you build Bro from source?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From ford.516 at osu.edu  Fri Nov 23 19:07:42 2012
From: ford.516 at osu.edu (Solomon Ford)
Date: Fri, 23 Nov 2012 22:07:42 -0500
Subject: [Bro-Dev] Bro failed
In-Reply-To: <CAG8JgP6M8T-fDpGfSH1YjdAqXO0-O0U7H17b=0bBRP8gQCf6vA@mail.gmail.com>
References: <CAG8JgP6M8T-fDpGfSH1YjdAqXO0-O0U7H17b=0bBRP8gQCf6vA@mail.gmail.com>
Message-ID: <482F96C2-5717-48E8-A0B7-79436CE6DDA6@osu.edu>

Sounds like you were trying to run a bro binary that was compiled for a different system architecture.  I've run into this problem before when I've mistakenly tried to copy & use the bro folder compiled for a 64-bit machine on a 32-bit machine instead.

As Seth suggests, to be safe, ensure you have compiled bro on the target machine in question before trying to run it.

--
Solomon Ford
OCIO Enterprise Security
The Ohio State University
ford.516 at osu.edu



On Nov 22, 2012, at 8:12 PM, Li?u L? Th? <lethilieuctk32 at gmail.com>
 wrote:

> Hi everyone.
> I have a problem when start Bro IDS, 
> When I enter command check, it return:
> [BroControl] > check
> bro failed.
>    /usr/local/bro/share/broctl/scripts/check-config: line 28: /usr/local/bro/bin/bro: cannot execute binary file
> 
> Who tell me how to fix it?
> thanks and Regard.
> Lieu
> -- 
> =========================================
> 
> 
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev




From bro at tracker.bro-ids.org  Fri Nov 23 19:35:48 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sat, 24 Nov 2012 03:35:48 -0000
Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only
 print CSV header line
In-Reply-To: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
References: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
Message-ID: <064.73fd14ad1806e3c7088aecd29341bb2b@tracker.bro-ids.org>

#912: Need option for ASCII logging writer to only print CSV header line
----------------------------+-------------------------------
  Reporter:  carsten        |      Owner:
      Type:  Merge Request  |     Status:  new
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:  logging csv ascii
----------------------------+-------------------------------

Comment (by robin):

 Merged (with a bit of adjustments). Also added a test. Notes:

  - I've removed short-cut table from the script, that doesn't look like a
 great idiom to me (e.g., what if we had options, then we could use the
 short-cut only if we set only one. We don't have a great place to document
 the new config options so I moved a (shortened) version into the file
 header.

 - I'm wondering if we should simply rename the option to "csv" instead of
 "only_single_header_row" ?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/912#comment:10>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Nov 23 19:42:10 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sat, 24 Nov 2012 03:42:10 -0000
Subject: [Bro-Dev] #918: topic/jsiwek/hook
In-Reply-To: <048.8bbad75d47a7d6400cc12123e7951748@tracker.bro-ids.org>
References: <048.8bbad75d47a7d6400cc12123e7951748@tracker.bro-ids.org>
Message-ID: <063.0496e637639ed0fdfb45264eb8a4b0bc@tracker.bro-ids.org>

#918: topic/jsiwek/hook
----------------------------+------------------------
  Reporter:  jsiwek         |      Owner:
      Type:  Merge Request  |     Status:  new
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:
----------------------------+------------------------

Comment (by robin):

 Merged, good stuff.

 What do you thing of the idea I floated by mail to still have the hook
 execution return a boolean indicating whether a ```break``` occurred? (But
 without otherwise having to declare a hook return type or use
 ```return```)?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/918#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Nov 23 20:05:17 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sat, 24 Nov 2012 04:05:17 -0000
Subject: [Bro-Dev] #80: Checking for the existance keys in
 multi-dimensional tables gives error message
In-Reply-To: <050.c01d8a094407695f72fd3d3d599c4f48@tracker.bro-ids.org>
References: <050.c01d8a094407695f72fd3d3d599c4f48@tracker.bro-ids.org>
Message-ID: <065.f0c5e8a419cf218406c5d112e0a8b676@tracker.bro-ids.org>

#80: Checking for the existance keys in multi-dimensional tables gives error
message
----------------------------+------------------------
  Reporter:  bernhard       |      Owner:  robin
      Type:  Merge Request  |     Status:  closed
  Priority:  Low            |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:  fixed          |   Keywords:
----------------------------+------------------------
Changes (by robin):

 * owner:   => robin
 * status:  seen => closed
 * resolution:   => fixed


Comment:

 In [3357746857fcef2339e772a6af04fc216058ed67/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="3357746857fcef2339e772a6af04fc216058ed67"
 Merge remote-tracking branch 'origin/topic/jsiwek/composite-table-index-
 fix'

 * origin/topic/jsiwek/composite-table-index-fix:
   Fix ambiguity between composite table index and record ctor expressions.

 Cool.

 Closes #80.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/80#comment:13>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Nov 23 20:05:17 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sat, 24 Nov 2012 04:05:17 -0000
Subject: [Bro-Dev] #919: topic/jsiwek/improve-enum-doc
In-Reply-To: <048.6394710ed0c343484e88f8806361c682@tracker.bro-ids.org>
References: <048.6394710ed0c343484e88f8806361c682@tracker.bro-ids.org>
Message-ID: <063.3156b77f3ee98c80c4da522d7db6e160@tracker.bro-ids.org>

#919: topic/jsiwek/improve-enum-doc
----------------------------+------------------------
  Reporter:  jsiwek         |      Owner:  robin
      Type:  Merge Request  |     Status:  closed
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:  fixed          |   Keywords:
----------------------------+------------------------
Changes (by robin):

 * owner:   => robin
 * status:  new => closed
 * resolution:   => fixed


Comment:

 In [96ce99590dcab1b27a4d3217c28936c81199bb4c/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="96ce99590dcab1b27a4d3217c28936c81199bb4c"
 Merge remote-tracking branch 'origin/topic/jsiwek/improve-enum-doc'

 * origin/topic/jsiwek/improve-enum-doc:
   Improve auto-generated enum documentation.

 Closes #919.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/919#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From noreply at bro-ids.org  Sat Nov 24 00:00:02 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Sat, 24 Nov 2012 00:00:02 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211240800.qAO802ho011931@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |   912 [1] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [2] | jsiwek     |            | Normal | topic/jsiwek/hook [3]



 [1]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [2]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [3]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook


From lethilieuctk32 at gmail.com  Sat Nov 24 16:22:24 2012
From: lethilieuctk32 at gmail.com (=?UTF-8?B?TGnDqnUgTMOqIFRo4buL?=)
Date: Sun, 25 Nov 2012 07:22:24 +0700
Subject: [Bro-Dev] bro-dev Digest, Vol 27, Issue 31
In-Reply-To: <mailman.3.1353787202.23807.bro-dev@bro-ids.org>
References: <mailman.3.1353787202.23807.bro-dev@bro-ids.org>
Message-ID: <CAG8JgP4Qexe+R5K99Sf7RKwUz=tLoA0MZV5Q=PcgW6y9DQMA7Q@mail.gmail.com>

Hello  ,

i am searching to find the Bro signature about :

MS08-067, ms05-017-msmq and some exploit vulnerability of
Microsoft.but unfortunately i can not find that .

any link where i can find?

thank you in advance


On 25 November 2012 03:00, <bro-dev-request at bro-ids.org> wrote:

> --------------+-------------------------------
>   Reporter:  carsten        |      Owner:
>       Type:  Merge Request  |     Status:  new
>   Priority:  Normal         |  Milestone:  Bro2.2
>  Component:  Bro            |    Version:  git/master
> Resolution:                 |   Keywords:  logging
>



-- 
=========================================
*L? Th? Li?u*
Email: lethilieuctk32 at gmail.com
Mobile:01248262338; 0974437270
Life Guidelines:" ??ng ?? ??n ng?y mai nh?ng vi?c g? c? th? l?m h?m nay."
=========================================

???ng tuy ng?n kh?ng ?i kh?ng ??n
Vi?c tuy d? kh?ng l?m kh?ng xong
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20121125/edd66e11/attachment.html 

From noreply at bro-ids.org  Sun Nov 25 00:00:02 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Sun, 25 Nov 2012 00:00:02 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211250800.qAP802Te000700@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |   912 [1] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [2] | jsiwek     |            | Normal | topic/jsiwek/hook [3]



 [1]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [2]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [3]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook


From lethilieuctk32 at gmail.com  Sun Nov 25 01:16:31 2012
From: lethilieuctk32 at gmail.com (=?UTF-8?B?TGnDqnUgTMOqIFRo4buL?=)
Date: Sun, 25 Nov 2012 16:16:31 +0700
Subject: [Bro-Dev] Bro cfg
Message-ID: <CAG8JgP6jRAtxgTfOw0jKVNSDWGQi6xyV+fR0wye44ipcqGqTvQ@mail.gmail.com>

Hi everyone.
Somebody send me a example of bro.cfg, please
Thanks and regards.
Lieu

--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20121125/23363f72/attachment.html 

From mthompson at hexwave.com  Sun Nov 25 12:27:49 2012
From: mthompson at hexwave.com (Matt Thompson)
Date: Sun, 25 Nov 2012 14:27:49 -0600
Subject: [Bro-Dev] DNS script bug?
Message-ID: <50B27F45.4080706@hexwave.com>

I'm working from latest git. It seems that dns.bro is only logging to 
the DNS log on connection_state_remove().

dns$ready is never set to T .. what is the following if supposed to be 
doing? dns$total answers == 0 here.

     if ( c$dns?$answers && c$dns?$total_answers &&
          |c$dns$answers| == c$dns$total_answers )
       {
       add c$dns_state$finished_answers[c$dns$trans_id];
       # Indicate this request/reply pair is ready to be logged.
       c$dns$ready = T;
       }
     }

Cheers,
Matt

From mthompson at hexwave.com  Sun Nov 25 12:49:52 2012
From: mthompson at hexwave.com (Matt Thompson)
Date: Sun, 25 Nov 2012 14:49:52 -0600
Subject: [Bro-Dev] DNS script bug?
In-Reply-To: <50B27F45.4080706@hexwave.com>
References: <50B27F45.4080706@hexwave.com>
Message-ID: <50B28470.2010708@hexwave.com>

I think I figured it out.

     if ( ! c$dns?$total_answers )
       c$dns$total_answers = msg$num_answers;

That sets total_answers if the total_answers field does not exist (?$). 
It does exist with a default of 0, so the field was never set. Removing 
this if statement results in c$dns$ready becoming true.

I noticed this because the following wasn't working in my local scripts

event DNS::do_reply(c: connection, msg: dns_msg, ans: dns_answer, reply: 
string) &priority=4
   {
     if ( c$dns$ready )
       {
         print "DNS!";
       }
   }


Cheers,
Matt

On 11/25/2012 02:27 PM, Matt Thompson wrote:
> I'm working from latest git. It seems that dns.bro is only logging to
> the DNS log on connection_state_remove().
>
> dns$ready is never set to T .. what is the following if supposed to be
> doing? dns$total answers == 0 here.
>
>       if ( c$dns?$answers&&  c$dns?$total_answers&&
>            |c$dns$answers| == c$dns$total_answers )
>         {
>         add c$dns_state$finished_answers[c$dns$trans_id];
>         # Indicate this request/reply pair is ready to be logged.
>         c$dns$ready = T;
>         }
>       }
>
> Cheers,
> Matt
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
>    


From noreply at bro-ids.org  Mon Nov 26 00:00:03 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Mon, 26 Nov 2012 00:00:03 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211260800.qAQ803aX024524@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |   912 [1] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [2] | jsiwek     |            | Normal | topic/jsiwek/hook [3]



 [1]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [2]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [3]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook


From bro at tracker.bro-ids.org  Mon Nov 26 07:50:34 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 26 Nov 2012 15:50:34 -0000
Subject: [Bro-Dev] #918: topic/jsiwek/hook
In-Reply-To: <048.8bbad75d47a7d6400cc12123e7951748@tracker.bro-ids.org>
References: <048.8bbad75d47a7d6400cc12123e7951748@tracker.bro-ids.org>
Message-ID: <063.4d7a0c25a54acbf538757b87d9d2c78d@tracker.bro-ids.org>

#918: topic/jsiwek/hook
---------------------+------------------------
  Reporter:  jsiwek  |      Owner:  jsiwek
      Type:  Task    |     Status:  assigned
  Priority:  Normal  |  Milestone:  Bro2.2
 Component:  Bro     |    Version:  git/master
Resolution:          |   Keywords:
---------------------+------------------------
Changes (by jsiwek):

 * owner:   => jsiwek
 * status:  new => assigned
 * type:  Merge Request => Task


Comment:

 Replying to [comment:1 robin]:
 > Merged, good stuff.
 >
 > What do you thing of the idea I floated by mail to still have the hook
 execution return a boolean indicating whether a ```break``` occurred? (But
 without otherwise having to declare a hook return type or use
 ```return```)?

 Other than it looking a bit weird/mysterious, I don't have much of a
 problem with that.  I'll see if I can make it happen.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/918#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From seth at icir.org  Mon Nov 26 07:40:06 2012
From: seth at icir.org (Seth Hall)
Date: Mon, 26 Nov 2012 10:40:06 -0500
Subject: [Bro-Dev] DNS script bug?
In-Reply-To: <50B28470.2010708@hexwave.com>
References: <50B27F45.4080706@hexwave.com> <50B28470.2010708@hexwave.com>
Message-ID: <F47AE8DC-6882-47A0-9C20-C31B7DD4BD63@icir.org>


On Nov 25, 2012, at 3:49 PM, Matt Thompson <mthompson at hexwave.com> wrote:

> I think I figured it out.
> 
>     if ( ! c$dns?$total_answers )
>       c$dns$total_answers = msg$num_answers;


Good catch, thanks!  I'll fix it in fastpath.

I'm actually going to fix it slightly differently though by setting the field to be &optional which seems to be what I had in mind in the first place.

Thanks again,
  .Seth




From bro at tracker.bro-ids.org  Mon Nov 26 08:46:43 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 26 Nov 2012 16:46:43 -0000
Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only
 print CSV header line
In-Reply-To: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
References: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
Message-ID: <064.392d13fc00977b4fd83a564d97744d1b@tracker.bro-ids.org>

#912: Need option for ASCII logging writer to only print CSV header line
----------------------------+-------------------------------
  Reporter:  carsten        |      Owner:
      Type:  Merge Request  |     Status:  new
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:  logging csv ascii
----------------------------+-------------------------------

Comment (by seth):

 > - I'm wondering if we should simply rename the option to "csv" instead
 of
 > "only_single_header_row" ?


 That would imply comma separation though.  I haven't looked yet, but is
 there a way to make this a global setting? (I know, I'm playing both sides
 of this argument)

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/912#comment:14>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From jsiwek at illinois.edu  Mon Nov 26 10:11:20 2012
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Mon, 26 Nov 2012 18:11:20 +0000
Subject: [Bro-Dev] topic/seth/metrics-merge crash
In-Reply-To: <56BBF866-CE0A-4F8F-AC1C-60AE4221C22B@icir.org>
References: <56BBF866-CE0A-4F8F-AC1C-60AE4221C22B@icir.org>
Message-ID: <B3E27C90B7D2DD4CAEF09215A29EB0D116EEDC5C@CITESMBX2.ad.uillinois.edu>


On Nov 20, 2012, at 10:21 AM, Seth Hall <seth at icir.org> wrote:

> I'm seeing Bro actually crash for the scripts.base.frameworks.metrics.cluster-intermediate-update test and I'm having a hard time figuring out why it's crashing.  Would someone mind taking a look at that soon?

See if the patch below makes sense.  Also not sure if that index_requests table is missing some initialization somewhere.

diff --git a/scripts/base/frameworks/metrics/cluster.bro b/scripts/base/frameworks/metrics/cluster.bro
index 9650b80..d23b9cd 100644
--- a/scripts/base/frameworks/metrics/cluster.bro
+++ b/scripts/base/frameworks/metrics/cluster.bro
@@ -190,7 +190,12 @@ event Metrics::cluster_index_response(uid: string, id: string, filter_name: stri
    {
    #print fmt("%0.6f MANAGER: receiving index data from %s - %s=%s", network_time(), get_event_peer()$descr, index2str(index), val);

-   local merged_val = merge_result_vals(index_requests[uid, id, filter_name, index], val);
+   local merged_val: ResultVal;
+
+   if ( [uid, id, filter_name, index] in index_requests )
+       merged_val = merge_result_vals(index_requests[uid, id, filter_name, index], val);
+   else
+       merged_val = val;
    index_requests[uid, id, filter_name, index] = merged_val;
    local ir = index_requests[uid, id, filter_name, index];

And as an optimization, maybe that "local ir" isn't needed since that index was just assigned in the line above?

    Jon

From seth at icir.org  Mon Nov 26 13:11:33 2012
From: seth at icir.org (Seth Hall)
Date: Mon, 26 Nov 2012 16:11:33 -0500
Subject: [Bro-Dev] topic/seth/metrics-merge crash
In-Reply-To: <B3E27C90B7D2DD4CAEF09215A29EB0D116EEDC5C@CITESMBX2.ad.uillinois.edu>
References: <56BBF866-CE0A-4F8F-AC1C-60AE4221C22B@icir.org>
	<B3E27C90B7D2DD4CAEF09215A29EB0D116EEDC5C@CITESMBX2.ad.uillinois.edu>
Message-ID: <03A23A88-0413-437C-8D1E-2ABACE2C3689@icir.org>


On Nov 26, 2012, at 1:11 PM, "Siwek, Jonathan Luke" <jsiwek at illinois.edu> wrote:

> On Nov 20, 2012, at 10:21 AM, Seth Hall <seth at icir.org> wrote:
> 
>> I'm seeing Bro actually crash for the scripts.base.frameworks.metrics.cluster-intermediate-update test and I'm having a hard time figuring out why it's crashing.  Would someone mind taking a look at that soon?
> 
> See if the patch below makes sense.  Also not sure if that index_requests table is missing some initialization somewhere.

Ah!  Thanks.  I bet you're right about this.  I'll test it tonight.

>    local ir = index_requests[uid, id, filter_name, index];
> 
> And as an optimization, maybe that "local ir" isn't needed since that index was just assigned in the line above?


The ir is actually an optimization so that I don't have to keep indexing into that table over and over later in the event handler.  It's even possible that I only need to do that assignment when the first value comes in anyway.  I'll play around with that more tonight too.

Thanks for helping me out. :)

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From bro at tracker.bro-ids.org  Mon Nov 26 14:03:46 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 26 Nov 2012 22:03:46 -0000
Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only
 print CSV header line
In-Reply-To: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
References: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
Message-ID: <064.a454b2496605293bd36417177a88b69e@tracker.bro-ids.org>

#912: Need option for ASCII logging writer to only print CSV header line
----------------------------+-------------------------------
  Reporter:  carsten        |      Owner:
      Type:  Merge Request  |     Status:  new
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:  logging csv ascii
----------------------------+-------------------------------

Comment (by carsten):

 Replying to [comment:10 robin]:
 >  - I've removed short-cut table from the script, that doesn't look like
 a great idiom to me (e.g., what if we had options, then we could use the
 short-cut only if we set only one. We don't have a great place to document
 the new config options so I moved a (shortened) version into the file
 header.

 My main goal was to help people use it. To me the general usage of options
 for writers was not obvious, so I though an example and a predefined
 short-cut would help. At least you have kept the example, so that is fine
 with me.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/912#comment:15>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Nov 26 14:11:10 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 26 Nov 2012 22:11:10 -0000
Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only
 print CSV header line
In-Reply-To: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
References: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
Message-ID: <064.0565c785adef6e4d22f614af576526e3@tracker.bro-ids.org>

#912: Need option for ASCII logging writer to only print CSV header line
----------------------------+-------------------------------
  Reporter:  carsten        |      Owner:
      Type:  Merge Request  |     Status:  new
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:  logging csv ascii
----------------------------+-------------------------------

Comment (by carsten):

 Replying to [comment:14 seth]:
 > > - I'm wondering if we should simply rename the option to "csv" instead
 of
 > > "only_single_header_row" ?
 >
 >
 > That would imply comma separation though.

 Personally I'm used to that "CSV" only means 0 or 1 header rows and
 separated by "something", no guarantee for comma or semicolon or tab or
 whatever. So I have no problem renaming it to CSV, as it does not imply
 for me that a comma must be used. Originally I wanted to avoid this
 discussion and named the option by what it actually does: printing only 1
 header row and all other options like field separator have to be specified
 separately. Bottom line: I don't mind either way.

 > I haven't looked yet, but is there a way to make this a global setting?
 (I know, I'm playing both sides of this argument)

 Well, it *was* a global setting in my first patch, and then *someone* told
 me it should not be a global but a per-writer option :-)
 I rather think now that all the other options could be per-writer options
 as well, rather than global.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/912#comment:16>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Nov 26 14:17:58 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 26 Nov 2012 22:17:58 -0000
Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only
 print CSV header line
In-Reply-To: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
References: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org>
Message-ID: <064.6c3209e150e63a241c952352c6e376c0@tracker.bro-ids.org>

#912: Need option for ASCII logging writer to only print CSV header line
----------------------------+-------------------------------
  Reporter:  carsten        |      Owner:
      Type:  Merge Request  |     Status:  new
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:  logging csv ascii
----------------------------+-------------------------------

Comment (by carsten):

 Replying to [comment:10 robin]:
 > Merged (with a bit of adjustments).

 I read the merged source code and I wonder about Ascii.cc, line 124, where
 it says: "invalid value for 'only_single_header_row', must be boolean
 (T/F)"
 Actually, it must not be a boolean, but a string of value "T" or "F". Can
 you change that, please?

 In ascii.bro I'm now missing the following part from my patch, which
 explains what the option really does.
 "The header row is not prepended with "fields", but starts with the field
 name of the first column. Also the field name is not prepended with the
 meta_prefix, i.e. the option meta_prefix is ignored. No opening nor close
 times are written."
 Can you re-add it, please?

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/912#comment:17>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Nov 26 15:10:22 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 26 Nov 2012 23:10:22 -0000
Subject: [Bro-Dev] #918: topic/jsiwek/hook
In-Reply-To: <048.8bbad75d47a7d6400cc12123e7951748@tracker.bro-ids.org>
References: <048.8bbad75d47a7d6400cc12123e7951748@tracker.bro-ids.org>
Message-ID: <063.93a15139e07f3861be5818fff732f782@tracker.bro-ids.org>

#918: topic/jsiwek/hook
---------------------+------------------------
  Reporter:  jsiwek  |      Owner:  jsiwek
      Type:  Task    |     Status:  assigned
  Priority:  Normal  |  Milestone:  Bro2.2
 Component:  Bro     |    Version:  git/master
Resolution:          |   Keywords:
---------------------+------------------------

Comment (by jsiwek):

 In [378ee699ffc058f368a06691e4237c7f1e791462/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="378ee699ffc058f368a06691e4237c7f1e791462"
 Hook functions now directly callable instead of w/ "hook" statements.

 The return value of the call is an implicit boolean value of T if all
 hook handlers ran, or F if one hook handler exited as a result of a
 break statement and potentially prevented other handlers from running.

 Scripts don't need to declare hooks with an explicit return type of bool
 (internally, that's assumed), and any values given to (optional) return
 statements in handler definitions are just ignored.

 Addresses #918.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/918#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Mon Nov 26 15:13:03 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Mon, 26 Nov 2012 23:13:03 -0000
Subject: [Bro-Dev] #918: topic/jsiwek/hook
In-Reply-To: <048.8bbad75d47a7d6400cc12123e7951748@tracker.bro-ids.org>
References: <048.8bbad75d47a7d6400cc12123e7951748@tracker.bro-ids.org>
Message-ID: <063.abcecadf25a4a13952b4d3b5442ac4e6@tracker.bro-ids.org>

#918: topic/jsiwek/hook
----------------------------+------------------------
  Reporter:  jsiwek         |      Owner:  jsiwek
      Type:  Merge Request  |     Status:  assigned
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:
----------------------------+------------------------
Changes (by jsiwek):

 * type:  Task => Merge Request


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/918#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From noreply at bro-ids.org  Tue Nov 27 00:00:03 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Tue, 27 Nov 2012 00:00:03 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211270800.qAR80358015943@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |   912 [1] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [2] | jsiwek     | jsiwek     | Normal | topic/jsiwek/hook [3]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | c98301e    | Seth Hall       | 2012-11-26 | Fixed a DNS attribute issue (reported by Matt Thompson). [4]



 [1]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [2]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [3]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook
 [4] fastpath: http://tracker.bro-ids.org/bro/changeset/c98301e51f6c4cfd48d299e7cfe59e03565cd2d8/bro


From vern at icir.org  Tue Nov 27 22:03:14 2012
From: vern at icir.org (Vern Paxson)
Date: Tue, 27 Nov 2012 22:03:14 -0800
Subject: [Bro-Dev] Proposing a new type: opaque
In-Reply-To: <20121119165751.GC24247@icir.org> (Mon, 19 Nov 2012 08:57:51 PST).
Message-ID: <20121128060314.DEE8B2C4005@rock.ICSI.Berkeley.EDU>

> > ... It looks like T would be a yield value.
> 
> I see the similarity to other language elements rather as a consistent
> form of integration. Why would the "T of Y" syntax only pertain to
> container types? Admittedly, we have currently no other uses, but looks
> like a reasonable one to me.
> 
> > opaque<MD5>
> 
> While this looks good to folks familiar with C++ and templates, it looks
> like an exceptional one-off to the Bro language. As you said, it would
> involve tweaking both lexer and parser, and when is a about to happen,
> we should rethink the added value.

Yeah, I agree with Matthias here on both of these points.  "opaque of T"
strikes me as reasonable syntax.

		Vern

From noreply at bro-ids.org  Wed Nov 28 00:00:03 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Wed, 28 Nov 2012 00:00:03 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211280800.qAS8036K022882@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |   912 [1] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [2] | jsiwek     | jsiwek     | Normal | topic/jsiwek/hook [3]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | c98301e    | Seth Hall       | 2012-11-26 | Fixed a DNS attribute issue (reported by Matt Thompson). [4]



 [1]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [2]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [3]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook
 [4] fastpath: http://tracker.bro-ids.org/bro/changeset/c98301e51f6c4cfd48d299e7cfe59e03565cd2d8/bro


From seth at icir.org  Wed Nov 28 06:47:21 2012
From: seth at icir.org (Seth Hall)
Date: Wed, 28 Nov 2012 09:47:21 -0500
Subject: [Bro-Dev] Proposing a new type: opaque
In-Reply-To: <20121128060314.DEE8B2C4005@rock.ICSI.Berkeley.EDU>
References: <20121128060314.DEE8B2C4005@rock.ICSI.Berkeley.EDU>
Message-ID: <1E45883F-0255-4B68-A7FD-DBA6CD574F41@icir.org>


On Nov 28, 2012, at 1:03 AM, Vern Paxson <vern at icir.org> wrote:

> Yeah, I agree with Matthias here on both of these points.  "opaque of T"
> strikes me as reasonable syntax.


Works for me then.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/


From bro at tracker.bro-ids.org  Wed Nov 28 12:29:36 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 28 Nov 2012 20:29:36 -0000
Subject: [Bro-Dev] #5: Bug with table[] of set[] initializers
In-Reply-To: <051.c52aaf28444d3cd950eb1cb7e9501fab@tracker.bro-ids.org>
References: <051.c52aaf28444d3cd950eb1cb7e9501fab@tracker.bro-ids.org>
Message-ID: <066.5f3cf8a90d9de9d6d661641b26426c1e@tracker.bro-ids.org>

#5: Bug with table[] of set[] initializers
------------------------+------------------
  Reporter:  Christian  |      Owner:
      Type:  Problem    |     Status:  seen
  Priority:  Normal     |  Milestone:
 Component:  Bro        |    Version:
Resolution:             |   Keywords:
------------------------+------------------

Comment (by jsiwek):

 In [a0590b2140adae5b9cc4ae5520bd566d12a4331d/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="a0590b2140adae5b9cc4ae5520bd566d12a4331d"
 Fix table(), set(), vector() constructors in table initializer lists.

 Also adds type checking of yield values to table() constructor and
 fixes the type checking of yield values in vector() constructor.

 Addresses #5.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/5#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Wed Nov 28 12:57:58 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Wed, 28 Nov 2012 20:57:58 -0000
Subject: [Bro-Dev] #5: Bug with table[] of set[] initializers
In-Reply-To: <051.c52aaf28444d3cd950eb1cb7e9501fab@tracker.bro-ids.org>
References: <051.c52aaf28444d3cd950eb1cb7e9501fab@tracker.bro-ids.org>
Message-ID: <066.9f01afdc7c8eaf548416bcbde307a0c1@tracker.bro-ids.org>

#5: Bug with table[] of set[] initializers
----------------------------+--------------------
  Reporter:  Christian      |      Owner:
      Type:  Merge Request  |     Status:  seen
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:
Resolution:                 |   Keywords:
----------------------------+--------------------
Changes (by jsiwek):

 * type:  Problem => Merge Request
 * milestone:   => Bro2.2


Comment:

 In `topic/jsiwek/table-init-container-ctors`.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/5#comment:4>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From noreply at bro-ids.org  Thu Nov 29 00:00:03 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Thu, 29 Nov 2012 00:00:03 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211290800.qAT803nc027722@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |     5 [1] | Christian  |            | Normal | Bug with table[] of set[] initializers
            Bro |   912 [2] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [3] | jsiwek     | jsiwek     | Normal | topic/jsiwek/hook [4]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | c98301e    | Seth Hall       | 2012-11-26 | Fixed a DNS attribute issue (reported by Matt Thompson). [5]



 [1]         #5: http://tracker.bro-ids.org/bro/ticket/5
 [2]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [3]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [4]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook
 [5] fastpath: http://tracker.bro-ids.org/bro/changeset/c98301e51f6c4cfd48d299e7cfe59e03565cd2d8/bro


From bro at tracker.bro-ids.org  Thu Nov 29 14:01:27 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 29 Nov 2012 22:01:27 -0000
Subject: [Bro-Dev] #866: Problem with set initializers
In-Reply-To: <046.979bb585920072a482373f58f7afe6e9@tracker.bro-ids.org>
References: <046.979bb585920072a482373f58f7afe6e9@tracker.bro-ids.org>
Message-ID: <061.009c752c5b5ca6d40f06f2efb6c8ed40@tracker.bro-ids.org>

#866: Problem with set initializers
----------------------+------------------------
  Reporter:  seth     |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:  Bro2.2
 Component:  Bro      |    Version:  git/master
Resolution:           |   Keywords:  language
----------------------+------------------------

Comment (by jsiwek):

 In [f7e07f5f097036bf80252b03e5e79408f3f243be/bro]:
 {{{
 #!CommitTicketReference repository="bro"
 revision="f7e07f5f097036bf80252b03e5e79408f3f243be"
 Fix various bugs with table/set attributes.

 - Identifiers that are initialized with set()/table() constructor
   expressions now inherit attributes from the expression.  Before,
   statements like

      const i: set[string] = set() &redef;

   associated the attribute with the set() constructor, but not the
   "i" identifier, preventing redefinition.  Addresses #866.

 - Allow &default attribute to apply to tables initialized as empty
   (via either "{ }" or "table()") or if the expression supplied to it
   can evaluate to a type that's promotable to the same yield type as
   the table.
 }}}

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/866#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Nov 29 14:09:45 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 29 Nov 2012 22:09:45 -0000
Subject: [Bro-Dev] #866: Problem with set initializers
In-Reply-To: <046.979bb585920072a482373f58f7afe6e9@tracker.bro-ids.org>
References: <046.979bb585920072a482373f58f7afe6e9@tracker.bro-ids.org>
Message-ID: <061.8b119e5c413882ccb81d7c69d05f81ad@tracker.bro-ids.org>

#866: Problem with set initializers
----------------------------+------------------------
  Reporter:  seth           |      Owner:
      Type:  Merge Request  |     Status:  new
  Priority:  Normal         |  Milestone:  Bro2.2
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:  language
----------------------------+------------------------
Changes (by jsiwek):

 * type:  Problem => Merge Request


Comment:

 Fixed in `topic/jsiwek/table-attribute-fixes`

 (note: I branched that from `topic/jsiwek/table-init-container-ctors`
 which has a merge request in #5, just to make sure the changes were
 compatible).

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/866#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Nov 29 14:30:35 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 29 Nov 2012 22:30:35 -0000
Subject: [Bro-Dev] #690: GTP de-tunneling
In-Reply-To: <047.bf6216f5c5ffabe44fd63ab9aa939d04@tracker.bro-ids.org>
References: <047.bf6216f5c5ffabe44fd63ab9aa939d04@tracker.bro-ids.org>
Message-ID: <062.42a1b1899839f7d1a26861b7c2db3d19@tracker.bro-ids.org>

#690: GTP de-tunneling
----------------------------+------------------------
  Reporter:  robin          |      Owner:  jsiwek
      Type:  Merge Request  |     Status:  assigned
  Priority:  Normal         |  Milestone:  2.1
 Component:  Bro            |    Version:  git/master
Resolution:                 |   Keywords:
----------------------------+------------------------
Changes (by jsiwek):

 * type:  Task => Merge Request


Comment:

 > Carsten reported a problem with this, so going to resolve that first.

 The problem seemed to be due to the "uniqueness" of the testing
 environment/method, not the code itself, which he also checked over and
 thinks is ok.  It's in `topic/jsiwek/gtp`.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/690#comment:5>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Nov 29 14:59:14 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 29 Nov 2012 22:59:14 -0000
Subject: [Bro-Dev] #16: Core dump with duplicates in subnets sets
In-Reply-To: <047.456b075f5c5e5485a55432f3104c0257@tracker.bro-ids.org>
References: <047.456b075f5c5e5485a55432f3104c0257@tracker.bro-ids.org>
Message-ID: <062.006a8d3514e105a09bd285d1352b1cf9@tracker.bro-ids.org>

#16: Core dump with duplicates in subnets sets
-----------------------------+--------------------
  Reporter:  robin           |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro2.2
 Component:  Bro             |    Version:  1.5.2
Resolution:  Solved/Applied  |   Keywords:
-----------------------------+--------------------
Changes (by jsiwek):

 * status:  seen => closed
 * resolution:   => Solved/Applied
 * milestone:   => Bro2.2


Comment:

 > while this...
 > {{{
 > const data: set[subnet] &redef;
 > redef data += { 127.0.0.1/32 };
 > redef data += { 127.0.0.1/32 };
 > }}}
 >
 > crashes with the reported error
 > {{{
 > ./crasher.bro, line 3: error: bad address mask value - 3223313923
 > ./patricia.c:683: failed assertion `prefix->bitlen <= patricia->maxbits'
 > Abort trap
 > }}}

 This works since 2.1.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/16#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Nov 29 15:55:15 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Thu, 29 Nov 2012 23:55:15 -0000
Subject: [Bro-Dev] #146: IPv6 addresses not support as values in
	signatures.
In-Reply-To: <046.1b1cfdd38eb13db9271012bd8f14fcb8@tracker.bro-ids.org>
References: <046.1b1cfdd38eb13db9271012bd8f14fcb8@tracker.bro-ids.org>
Message-ID: <061.70dfe7420a1e7320a3d51ababb68a527@tracker.bro-ids.org>

#146: IPv6 addresses not support as values in signatures.
----------------------+-------------------
  Reporter:  seth     |      Owner:
      Type:  Problem  |     Status:  seen
  Priority:  Normal   |  Milestone:
 Component:  Bro      |    Version:  1.5.2
Resolution:           |   Keywords:
----------------------+-------------------

Comment (by grigorescu):

 I believe this was fixed when #901 was closed.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/146#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Thu Nov 29 16:12:03 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 30 Nov 2012 00:12:03 -0000
Subject: [Bro-Dev] #696: broctl dumps python traceback information when
 bro is communicating with other worker nodes and quits
In-Reply-To: <049.98d6f78884b685b89df7ef541cd92394@tracker.bro-ids.org>
References: <049.98d6f78884b685b89df7ef541cd92394@tracker.bro-ids.org>
Message-ID: <064.3f7db1916f59ce7e3bc28636a1229993@tracker.bro-ids.org>

#696: broctl dumps python traceback information when bro is communicating with
other worker nodes and quits
----------------------+-------------------
  Reporter:  aashish  |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  Normal   |  Milestone:
 Component:  Bro      |    Version:  1.5.3
Resolution:           |   Keywords:
----------------------+-------------------

Comment (by grigorescu):

 This was fixed a while ago, and the ticket can be closed.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/696#comment:1>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From noreply at bro-ids.org  Fri Nov 30 00:00:02 2012
From: noreply at bro-ids.org (Merge Tracker)
Date: Fri, 30 Nov 2012 00:00:02 -0800
Subject: [Bro-Dev] [Auto] Merge Status
Message-ID: <201211300800.qAU802Lq002913@bro-ids.icir.org>

> Open Merge Requests for Bro2.2
> ==============================

      Component |    Id     | Reporter   | Owner      | Prio   | Summary
  ------------------------------------------------------------------------------------------------------------------
            Bro |     5 [1] | Christian  |            | Normal | Bug with table[] of set[] initializers
            Bro |   866 [2] | seth       |            | Normal | Problem with set initializers
            Bro |   912 [3] | carsten    |            | Normal | Need option for ASCII logging writer to only print CSV header line
            Bro |   918 [4] | jsiwek     | jsiwek     | Normal | topic/jsiwek/hook [5]


> Unmerged Fastpath Commits
> =========================

      Component | Revision   | Committer       | Date       | Summary
  ------------------------------------------------------------------------------------------------------------------
            bro | c98301e    | Seth Hall       | 2012-11-26 | Fixed a DNS attribute issue (reported by Matt Thompson). [6]



 [1]         #5: http://tracker.bro-ids.org/bro/ticket/5
 [2]       #866: http://tracker.bro-ids.org/bro/ticket/866
 [3]       #912: http://tracker.bro-ids.org/bro/ticket/912
 [4]       #918: http://tracker.bro-ids.org/bro/ticket/918
 [5]       hook: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/hook
 [6] fastpath: http://tracker.bro-ids.org/bro/changeset/c98301e51f6c4cfd48d299e7cfe59e03565cd2d8/bro


From bro at tracker.bro-ids.org  Fri Nov 30 07:45:06 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 30 Nov 2012 15:45:06 -0000
Subject: [Bro-Dev] #146: IPv6 addresses not support as values in
	signatures.
In-Reply-To: <046.1b1cfdd38eb13db9271012bd8f14fcb8@tracker.bro-ids.org>
References: <046.1b1cfdd38eb13db9271012bd8f14fcb8@tracker.bro-ids.org>
Message-ID: <061.2e1ed349a7c791f38920e6045f33921b@tracker.bro-ids.org>

#146: IPv6 addresses not support as values in signatures.
-----------------------------+--------------------
  Reporter:  seth            |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:  Bro2.2
 Component:  Bro             |    Version:  1.5.2
Resolution:  Solved/Applied  |   Keywords:
-----------------------------+--------------------
Changes (by jsiwek):

 * status:  seen => closed
 * resolution:   => Solved/Applied
 * milestone:   => Bro2.2


Comment:

 Replying to [comment:2 grigorescu]:
 > I believe this was fixed when #901 was closed.

 Yes, should be.  Thanks.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/146#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Nov 30 07:47:10 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Fri, 30 Nov 2012 15:47:10 -0000
Subject: [Bro-Dev] #696: broctl dumps python traceback information when
 bro is communicating with other worker nodes and quits
In-Reply-To: <049.98d6f78884b685b89df7ef541cd92394@tracker.bro-ids.org>
References: <049.98d6f78884b685b89df7ef541cd92394@tracker.bro-ids.org>
Message-ID: <064.bd84e500bf476ef8d9a2ddeac3340182@tracker.bro-ids.org>

#696: broctl dumps python traceback information when bro is communicating with
other worker nodes and quits
-----------------------------+--------------------
  Reporter:  aashish         |      Owner:
      Type:  Problem         |     Status:  closed
  Priority:  Normal          |  Milestone:
 Component:  Bro             |    Version:  1.5.3
Resolution:  Solved/Applied  |   Keywords:
-----------------------------+--------------------
Changes (by jsiwek):

 * status:  new => closed
 * resolution:   => Solved/Applied


-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/696#comment:2>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker


From bro at tracker.bro-ids.org  Fri Nov 30 19:29:19 2012
From: bro at tracker.bro-ids.org (Bro Tracker)
Date: Sat, 01 Dec 2012 03:29:19 -0000
Subject: [Bro-Dev] #741: Remove HTTP verbs from HTTP analyzer
In-Reply-To: <046.33191738c5e4da46b75e4c33d1e6e42a@tracker.bro-ids.org>
References: <046.33191738c5e4da46b75e4c33d1e6e42a@tracker.bro-ids.org>
Message-ID: <061.fbc75855029079b4c1e92aedfb49d471@tracker.bro-ids.org>

#741: Remove HTTP verbs from HTTP analyzer
----------------------+--------------------
  Reporter:  seth     |      Owner:
      Type:  Problem  |     Status:  new
  Priority:  High     |  Milestone:  Bro2.2
 Component:  Bro      |    Version:
Resolution:           |   Keywords:
----------------------+--------------------

Comment (by grigorescu):

 This is fixed in: https://github.com/grigorescu/bro/tree/topic/vladg/http-
 verbs

 I just grab the method with get_word, and follow the same workflow as
 before. Instead of getting rid of the verbs entirely, I moved the HTTP
 verbs into script-land, and generate a weird when I see a method not in
 that list.

-- 
Ticket URL: <http://tracker.bro-ids.org/bro/ticket/741#comment:3>
Bro Tracker <http://tracker.bro-ids.org/bro>
Bro Issue Tracker