From xydarcher at 163.com  Sat Dec  3 23:09:54 2011
From: xydarcher at 163.com (readon shaw)
Date: Sun, 4 Dec 2011 15:09:54 +0800 (CST)
Subject: [Bro] using Bro as traffic analyzer.
Message-ID: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>

I was searching for a long time to find a framework can support fast & custom network traffic analysis.
some specific features of traffic data from monitor, such as interval of SYN and SYN-ACK, should be extracted and grouped by host.
i find Bro is so widely used, which seems can fulfill the requirement.
Can i disable other functions embedded in Bro, and add a plugin myself?
What is the point to archieve this, modify the core .cpp source file or add a .bro file?

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111204/d9f7a99d/attachment.html 

From vallentin at icir.org  Sat Dec  3 23:44:45 2011
From: vallentin at icir.org (Matthias Vallentin)
Date: Sat, 3 Dec 2011 23:44:45 -0800
Subject: [Bro] using Bro as traffic analyzer.
In-Reply-To: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
Message-ID: <CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>

> i find Bro is so widely used, which seems can fulfill the requirement.

Good to hear it works for you and welcome aboard!

> Can i disable other functions embedded in Bro, and add a plugin myself?

With the upcoming release of 2.0, Bro enables policy-neutral protocol
analysis by default, meaning it gives you a neutral picture of what's
going on in your network. For additional analyses and detectors, you
need to load the corresponding scripts in the policy directory;
local.bro is a good starting point. That said, you only pay for basic
protocol decoding by default.

> What is the point to archieve this, modify the core .cpp source file or add
> a .bro file?

This depends on the functionality you would like to add. Would you
mind elaborating a bit so that we can give you more helpful advice?
Changing the format of the log output or modifying analyzer behavior
generally works at the scripting layer. Bro features a Turing-complete
scripting language. You can write your own new functions and events.
If you would like to haul C/C++ functionality up to the scripting
layer, you might want to consider writing your own built-in function
(BiF). See src/bro.bif for examples. If you would like to add a new
protocol analyzer, then BinPAC is the right tool for you.

    Matthias


From gmhoward at gmail.com  Sun Dec  4 11:49:40 2011
From: gmhoward at gmail.com (Gaspar Modelo-Howard)
Date: Sun, 04 Dec 2011 14:49:40 -0500
Subject: [Bro] trace-summary script and number of local networks defined
In-Reply-To: <CANpnLHifgx=N8kaSoKJUQCga+ohCY=o2SMW1av4HQ=qGd0ra1Q@mail.gmail.com>
References: <CANpnLHifgx=N8kaSoKJUQCga+ohCY=o2SMW1av4HQ=qGd0ra1Q@mail.gmail.com>
Message-ID: <1323028180.3391.30.camel@kareem>

Set up Bro (version 2.0-beta-47) and have a couple of questions
regarding the etc/networks.cfg file:

(1) When defining more than 24 local networks in etc/networks.cfg, the
trace-summary script throws the following error message and does not
email the regular connections stats summary:

Traceback (most recent call last):
  File "/usr/local/bro/bin/trace-summary", line 816, in <module>
    LocalNetsIntervals[net] = i
  File "/usr/local/bro/lib/broctl/SubnetTree.py", line 86, in
__setitem__
    def __setitem__(self, *args): return
_SubnetTree.SubnetTree___setitem__(self, *args)
IndexError: cannot insert network
Command exited with non-zero status 1
0:00.04 real, 0.05 user, 0.00 sys, 0K total memory

I tested for different number of local network declarations (all /24)
and 24 seems to reach a limit. When defining 25 or more nets, get the
error message from above. Have been looking at trace-summary and
subnetTree.py but have not yet detected the cause of the problem.

(2) Can you 'aggregate' local networks on the etc/networks.cfg file so
the stats for several defined networks can be reported by trace-summary
as just one network? For example, say I declare the local networks in
networks.cfg as:
1.1.1.0/24  Network A
1.1.2.0/24  Network B
1.1.3.0/24  Network C

Then I want trace-summary to give the aggregated stats for both Networks
A and C, in a single table. Feasible? Tried using same tags for
different networks but did not work.

Wonder if anybody has run into any of these two issues.
Thanks,


Gaspar



From xydarcher at 163.com  Sun Dec  4 18:47:48 2011
From: xydarcher at 163.com (Readon Shaw)
Date: Mon, 5 Dec 2011 10:47:48 +0800
Subject: [Bro] using Bro as traffic analyzer.
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>, 
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
Message-ID: <2011120510474838157151@163.com>

>> i find Bro is so widely used, which seems can fulfill the requirement.
>
>Good to hear it works for you and welcome aboard!
It is a greate platform, thank you for your works.
>
>> Can i disable other functions embedded in Bro, and add a plugin myself?
>
>With the upcoming release of 2.0, Bro enables policy-neutral protocol
>analysis by default, meaning it gives you a neutral picture of what's
>going on in your network. For additional analyses and detectors, you
>need to load the corresponding scripts in the policy directory;
>local.bro is a good starting point. That said, you only pay for basic
>protocol decoding by default.
>
>> What is the point to archieve this, modify the core .cpp source file or add
>> a .bro file?
>
>This depends on the functionality you would like to add. Would you
>mind elaborating a bit so that we can give you more helpful advice?
>Changing the format of the log output or modifying analyzer behavior
>generally works at the scripting layer. Bro features a Turing-complete
>scripting language. You can write your own new functions and events.
>If you would like to haul C/C++ functionality up to the scripting
>layer, you might want to consider writing your own built-in function
>(BiF). See src/bro.bif for examples. If you would like to add a new
>protocol analyzer, then BinPAC is the right tool for you.
>
I want to match tcp handshake pairs and record the intervals between 
each SYN and SYN-ACK pairs with their arrival time. At the same time, 
roughly packet loss rate (vs different timescales) should be calculated 
by tcp retransmission rate. It is a statistical analysis on network traffic 
that would be processed by .bro files i think. some of them are similar 
with functions already existed. Would you please give me some notes 
on which files i should start with?

btw: I read the document and find that all C/C++ code is designed 
for decoding packets. bro files take charge in statistal or general processing.
Is it right? Any general pictures were provided in bro?
>    Matthias

--------------
Readon Shaw




From vallentin at icir.org  Sun Dec  4 19:19:58 2011
From: vallentin at icir.org (Matthias Vallentin)
Date: Sun, 4 Dec 2011 19:19:58 -0800
Subject: [Bro] using Bro as traffic analyzer.
In-Reply-To: <2011120510474838157151@163.com>
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>
Message-ID: <CADTpMNaD5FXD5tjoO9Az-MtnHT1k7=6aQ7sR6pPG8SNTWALFFA@mail.gmail.com>

> I want to match tcp handshake pairs and record the intervals between
> each SYN and SYN-ACK pairs with their arrival time. At the same time,
> roughly packet loss rate (vs different timescales) should be calculated
> by tcp retransmission rate. It is a statistical analysis on network
> traffic that would be processed by .bro files i think. some of them
> are similar with functions already existed. Would you please give me
> some notes on which files i should start with?

Yes, you're right. This sort of analysis can be entirely done at the
scripting layer, i.e., it only involves *.bro scripts.

Bro reassembles the full TCP byte stream. When you deal with connection
data, duplicates/retransmission are already removed. If you want to
compute round-trip times at the packet level, you can write a handler
for the events new_connection (which is generated for every new
SYN packet) and connection_established (which is generated for a
successful TCP handshake after a SYN-ACK).

For retransmissions, have a look at the events: rexmit_inconsistency,
content_gap, and gap_report. Unfortunately I cannot provide more
detailed information other than pointing you to our ongoing
documentation effort in the git repository:

    git clone git://git.bro-ids.org/bro.git
    git checkout topic/script-reference
    less src/event.bif

Maybe others can chime in and give you further guidance.

(Also, to measure system/NIC capture loss, there is
policy/misc/capture-loss.bro.)

> btw: I read the document and find that all C/C++ code is designed
> for decoding packets. bro files take charge in statistal or general
> processing. Is it right? Any general pictures were provided in bro?

That's correct. Packet "decoding" is done at the Bro core. Bro
reassembles the TCP byte stream and presents it as a connection to the
user. You may find our workshop materials helpful to better understand
the architecture of Bro: http://www.bro-ids.org/bro-workshop-2011

    Matthias


From james.swaro at gmail.com  Sun Dec  4 22:54:06 2011
From: james.swaro at gmail.com (James Swaro)
Date: Mon, 5 Dec 2011 01:54:06 -0500
Subject: [Bro] using Bro as traffic analyzer.
In-Reply-To: <2011120510474838157151@163.com>
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>
Message-ID: <CAAG4iq=uszsfRR2ymThPMM5StKVTYjsSpVyBSXffriSsCxUYxg@mail.gmail.com>

>
>  At the same time, roughly packet loss rate (vs different timescales)
> should be calculated
> by tcp retransmission rate.
>

Are you interested in the packet loss rate for the three-way handshake
only, or for all retransmissions in each connection?

What level of detail are you looking for? (IE. Just that the segment was
retransmitted or perhaps more specific data about how it was retransmitted
such as in response to a retransmission timeout or other recovery method?)

-- 
*- *James Swaro*
*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111205/d208be96/attachment.html 

From xydarcher at 163.com  Mon Dec  5 03:03:13 2011
From: xydarcher at 163.com (Readon Shaw)
Date: Mon, 5 Dec 2011 19:03:13 +0800
Subject: [Bro] using Bro as traffic analyzer.
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>, 
	<CAAG4iq=uszsfRR2ymThPMM5StKVTYjsSpVyBSXffriSsCxUYxg@mail.gmail.com>
Message-ID: <201112051903131426517@163.com>

>>
>>  At the same time, roughly packet loss rate (vs different timescales)
>> should be calculated
>> by tcp retransmission rate.
>>
>
>Are you interested in the packet loss rate for the three-way handshake
>only, or for all retransmissions in each connection?
>
all retransmissions should be collected.
>What level of detail are you looking for? (IE. Just that the segment was
>retransmitted or perhaps more specific data about how it was retransmitted
>such as in response to a retransmission timeout or other recovery method?)
it would be better that retrans mode were detected.
>
>-- 
>*- *James Swaro*
>*
>

--------------
Readon Shaw




From xydarcher at 163.com  Mon Dec  5 03:31:17 2011
From: xydarcher at 163.com (Readon Shaw)
Date: Mon, 5 Dec 2011 19:31:17 +0800
Subject: [Bro] using Bro as traffic analyzer.
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>, 
	<CADTpMNaD5FXD5tjoO9Az-MtnHT1k7=6aQ7sR6pPG8SNTWALFFA@mail.gmail.com>
Message-ID: <201112051911272309048@163.com>

>> I want to match tcp handshake pairs and record the intervals between
>> each SYN and SYN-ACK pairs with their arrival time. At the same time,
>> roughly packet loss rate (vs different timescales) should be calculated
>> by tcp retransmission rate. It is a statistical analysis on network
>> traffic that would be processed by .bro files i think. some of them
>> are similar with functions already existed. Would you please give me
>> some notes on which files i should start with?
>
>Yes, you're right. This sort of analysis can be entirely done at the
>scripting layer, i.e., it only involves *.bro scripts.
>
>Bro reassembles the full TCP byte stream. When you deal with connection
>data, duplicates/retransmission are already removed. If you want to
>compute round-trip times at the packet level, you can write a handler
>for the events new_connection (which is generated for every new
>SYN packet) and connection_established (which is generated for a
>successful TCP handshake after a SYN-ACK).
>
>For retransmissions, have a look at the events: rexmit_inconsistency,
>content_gap, and gap_report. Unfortunately I cannot provide more
>detailed information other than pointing you to our ongoing
>documentation effort in the git repository:
>
>    git clone git://git.bro-ids.org/bro.git
>    git checkout topic/script-reference
>    less src/event.bif
>
>Maybe others can chime in and give you further guidance.
>
>(Also, to measure system/NIC capture loss, there is
>policy/misc/capture-loss.bro.)
>
>> btw: I read the document and find that all C/C++ code is designed
>> for decoding packets. bro files take charge in statistal or general
>> processing. Is it right? Any general pictures were provided in bro?
>
>That's correct. Packet "decoding" is done at the Bro core. Bro
>reassembles the TCP byte stream and presents it as a connection to the
>user. You may find our workshop materials helpful to better understand
>the architecture of Bro: http://www.bro-ids.org/bro-workshop-2011
>
>    Matthias
Thank you very much. It is very useful!
--------------
Readon Shaw




From xydarcher at 163.com  Mon Dec  5 03:33:17 2011
From: xydarcher at 163.com (Readon Shaw)
Date: Mon, 5 Dec 2011 19:33:17 +0800
Subject: [Bro] using Bro as traffic analyzer.
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>, 
	<CADTpMNaD5FXD5tjoO9Az-MtnHT1k7=6aQ7sR6pPG8SNTWALFFA@mail.gmail.com>
Message-ID: <201112051911272309048@163.com>

>> I want to match tcp handshake pairs and record the intervals between
>> each SYN and SYN-ACK pairs with their arrival time. At the same time,
>> roughly packet loss rate (vs different timescales) should be calculated
>> by tcp retransmission rate. It is a statistical analysis on network
>> traffic that would be processed by .bro files i think. some of them
>> are similar with functions already existed. Would you please give me
>> some notes on which files i should start with?
>
>Yes, you're right. This sort of analysis can be entirely done at the
>scripting layer, i.e., it only involves *.bro scripts.
>
>Bro reassembles the full TCP byte stream. When you deal with connection
>data, duplicates/retransmission are already removed. If you want to
>compute round-trip times at the packet level, you can write a handler
>for the events new_connection (which is generated for every new
>SYN packet) and connection_established (which is generated for a
>successful TCP handshake after a SYN-ACK).
>
>For retransmissions, have a look at the events: rexmit_inconsistency,
>content_gap, and gap_report. Unfortunately I cannot provide more
>detailed information other than pointing you to our ongoing
>documentation effort in the git repository:
>
>    git clone git://git.bro-ids.org/bro.git
>    git checkout topic/script-reference
>    less src/event.bif
>
>Maybe others can chime in and give you further guidance.
>
>(Also, to measure system/NIC capture loss, there is
>policy/misc/capture-loss.bro.)
>
>> btw: I read the document and find that all C/C++ code is designed
>> for decoding packets. bro files take charge in statistal or general
>> processing. Is it right? Any general pictures were provided in bro?
>
>That's correct. Packet "decoding" is done at the Bro core. Bro
>reassembles the TCP byte stream and presents it as a connection to the
>user. You may find our workshop materials helpful to better understand
>the architecture of Bro: http://www.bro-ids.org/bro-workshop-2011
>
>    Matthias
Thank you very much. It is very useful!
--------------
Readon Shaw




From james.swaro at gmail.com  Mon Dec  5 11:31:00 2011
From: james.swaro at gmail.com (James Swaro)
Date: Mon, 5 Dec 2011 14:31:00 -0500
Subject: [Bro] using Bro as traffic analyzer.
In-Reply-To: <201112051911272309048@163.com>
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>
	<CADTpMNaD5FXD5tjoO9Az-MtnHT1k7=6aQ7sR6pPG8SNTWALFFA@mail.gmail.com>
	<201112051911272309048@163.com>
Message-ID: <CAAG4iqn3LYWf0iEOsg77UnLgm8NXDbTiL1wY6L-eLmYON+jevg@mail.gmail.com>

Correct me if I am wrong Matthias.

Bro can do most of what you are looking for out of the box. Sampling the
round-trip time of the three-way handshake is doable at scripting level.
You can get bro to output the retransmission data through the tcp_rexmit
event. It does not give detailed information about how the data was
retransmitted but will tell you how many bytes were retransmitted and how
much data was outstanding when the retransmission occurred.

That should be sufficient for what you are looking for.

If you need more detailed information, I am currently working on an
analyzer for Bro that attempts to give more detailed information about the
retransmission behavior of a TCP connection as part of on-going research.
However, It is not in a state that is ready for release.

-- 
James Swaro*
*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111205/42b2d505/attachment.html 

From robin at icir.org  Mon Dec  5 09:14:37 2011
From: robin at icir.org (Robin Sommer)
Date: Mon, 5 Dec 2011 09:14:37 -0800
Subject: [Bro] trace-summary script and number of local networks defined
In-Reply-To: <1323028180.3391.30.camel@kareem>
References: <CANpnLHifgx=N8kaSoKJUQCga+ohCY=o2SMW1av4HQ=qGd0ra1Q@mail.gmail.com>
	<1323028180.3391.30.camel@kareem>
Message-ID: <20111205171437.GA81071@icir.org>


On Sun, Dec 04, 2011 at 14:49 -0500, you wrote:

> IndexError: cannot insert network
> Command exited with non-zero status 1

That is odd, no immediate idea why that would happen. Can you file a
bug report with the tracker please, ideally with a networks.cfg
attached that triggers the error along with the trace-summary command
line. Thanks.

> (2) Can you 'aggregate' local networks on the etc/networks.cfg file so
> the stats for several defined networks can be reported by trace-summary
> as just one network?

No, that's not supported currently. The aggregation is done by prefix,
not tag.

(I suppose you can't find a larger network mask that covers what you'd
like to aggregate?)

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From jmellander at lbl.gov  Mon Dec  5 16:20:21 2011
From: jmellander at lbl.gov (Jim Mellander)
Date: Mon, 5 Dec 2011 16:20:21 -0800
Subject: [Bro] Reassembly of IPv6 TCP streams
Message-ID: <CADju=b5jjK=wT3jZhKR5LqG2HCkhAbvP8f_AnCNH0sfvqC1rFQ@mail.gmail.com>

Hi all:

contents.bro performs session reconstruction of IPv4 traffic, but when
running Bro 1.5 contents.bro against an IPv6 packet trace, it creates
0-length files, but doesn't extract the session contents to those
files.  Is this in the works?

Thanks in advance


From seth at icir.org  Mon Dec  5 19:50:37 2011
From: seth at icir.org (Seth Hall)
Date: Mon, 5 Dec 2011 22:50:37 -0500
Subject: [Bro] Reassembly of IPv6 TCP streams
In-Reply-To: <CADju=b5jjK=wT3jZhKR5LqG2HCkhAbvP8f_AnCNH0sfvqC1rFQ@mail.gmail.com>
References: <CADju=b5jjK=wT3jZhKR5LqG2HCkhAbvP8f_AnCNH0sfvqC1rFQ@mail.gmail.com>
Message-ID: <83EF06F4-96C2-45D4-B19C-021D695FDF55@icir.org>


On Dec 5, 2011, at 7:20 PM, Jim Mellander wrote:

> contents.bro performs session reconstruction of IPv4 traffic, but when
> running Bro 1.5 contents.bro against an IPv6 packet trace, it creates
> 0-length files, but doesn't extract the session contents to those
> files.  Is this in the works?


That will be part of the work for 2.1.  I'll file a ticket for that to make sure we look into it.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/




From vern at icir.org  Mon Dec  5 19:51:15 2011
From: vern at icir.org (Vern Paxson)
Date: Mon, 05 Dec 2011 19:51:15 -0800
Subject: [Bro] using Bro as traffic analyzer.
In-Reply-To: <CAAG4iqn3LYWf0iEOsg77UnLgm8NXDbTiL1wY6L-eLmYON+jevg@mail.gmail.com>
	(Mon, 05 Dec 2011 14:31:00 EST).
Message-ID: <20111206035115.E85A72C4004@rock.ICSI.Berkeley.EDU>

> If you need more detailed information, I am currently working on an
> analyzer for Bro that attempts to give more detailed information about the
> retransmission behavior of a TCP connection as part of on-going research.

You should for sure contact Katrina LaCurts <katrina at csail.mit.edu>,
who did an internship with us working on integrating this sort of analysis
into Bro.

		Vern


From james.swaro at gmail.com  Mon Dec  5 22:37:58 2011
From: james.swaro at gmail.com (James Swaro)
Date: Tue, 6 Dec 2011 01:37:58 -0500
Subject: [Bro] using Bro as traffic analyzer.
In-Reply-To: <20111206035115.E85A72C4004@rock.ICSI.Berkeley.EDU>
References: <CAAG4iqn3LYWf0iEOsg77UnLgm8NXDbTiL1wY6L-eLmYON+jevg@mail.gmail.com>
	<20111206035115.E85A72C4004@rock.ICSI.Berkeley.EDU>
Message-ID: <CAAG4iq=zvywAQ=gvAtgVyiRFMWD7_s1VkHySVvogTqO4cTV2TQ@mail.gmail.com>

On Mon, Dec 5, 2011 at 10:51 PM, Vern Paxson <vern at icir.org> wrote:

> > If you need more detailed information, I am currently working on an
> > analyzer for Bro that attempts to give more detailed information about
> the
> > retransmission behavior of a TCP connection as part of on-going research.
>
> You should for sure contact Katrina LaCurts <katrina at csail.mit.edu>,
> who did an internship with us working on integrating this sort of analysis
> into Bro.
>
>                Vern
>

I've actually seen quite a bit of her work that she emailed to me last
year. It was a phenomenal base for what I've tried to expand upon. I'd be
eager to see if there had been further developments with it aside from what
I've seen in the development branch.

Katrina could give a more accurate description of the stats analyzer than
perhaps I could.
-- 
James Swaro*
*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111206/82c05f8b/attachment.html 

From xydarcher at 163.com  Tue Dec  6 01:01:40 2011
From: xydarcher at 163.com (Readon Shaw)
Date: Tue, 6 Dec 2011 17:01:40 +0800
Subject: [Bro] Is it applicapable to specific target ip using command line
	in bro?
Message-ID: <2011120617014057329041@163.com>

I want to analysis traffic in/out specific host (identified by ip) in trace file, 
where processing for in/out streams are different. So i would be a problem to
notify the script what is my target host. A python script was used to generate 
the command lines, such as 
bro -r xxx.pcap yyyy.bro. 
But here the bro script can't get the target ip through this kind of command. 
Is there any mechanism in bro to fulfull this requirement?

There is a way to config ip in files, but i think that would meet its limited 
on multi-thread processing.

or broccoli-python suit for me? how would it communicate with a trace file based bro server?




Readon Shaw
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111206/16a3050a/attachment.html 

From sri at basam.org  Tue Dec  6 07:23:23 2011
From: sri at basam.org (sridhar basam)
Date: Tue, 6 Dec 2011 10:23:23 -0500
Subject: [Bro] Is it applicapable to specific target ip using command
 line in bro?
In-Reply-To: <2011120617014057329041@163.com>
References: <2011120617014057329041@163.com>
Message-ID: <CAGnVnBmnDvmxddmdW_T+yXLQwLnPHPfQL9kwSgkg4qFYR+d-MQ@mail.gmail.com>

On Tue, Dec 6, 2011 at 4:01 AM, Readon Shaw <xydarcher at 163.com> wrote:

> **
> I want to analysis traffic in/out specific host (identified by ip)
> in trace file,
> where processing for in/out streams are different. So i would be a problem
> to
> notify the script what is my target host. A python script was used to
> generate
> the command lines, such as
> bro -r xxx.pcap yyyy.bro.
> But here the bro script can't get the target ip through this kind
> of command.
> Is there any mechanism in bro to fulfull this requirement?
>

Could you just script it to pass the ip as a filter to bro?

bro -r <file.pcap> -f "host a.b.c.d" myscript.bro?

 Sridhar




>
> There is a way to config ip in files, but i think that would meet its
> limited
> on multi-thread processing.
>
> or broccoli-python suit for me? how would it communicate with a trace file
> based bro server?
>
>  ------------------------------
>  Readon Shaw
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111206/1dd0f15f/attachment.html 

From jsiwek at illinois.edu  Tue Dec  6 07:24:04 2011
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Tue, 6 Dec 2011 15:24:04 +0000
Subject: [Bro] Is it applicapable to specific target ip using command
 line	in bro?
In-Reply-To: <2011120617014057329041@163.com>
References: <2011120617014057329041@163.com>
Message-ID: <ED6CD411-1259-4202-AF95-7DA29334A54B@illinois.edu>

> bro -r xxx.pcap yyyy.bro.
> But here the bro script can't get the target ip through this kind of command.
> Is there any mechanism in bro to fulfull this requirement?

You can set any &redef variable from the command line.  e.g. if yyyy.bro contains:

    const target_ip: addr &redef;

    event bro_init()
        {
        print target_ip;
        }

Then you can do the following:

    $ bro yyyy.bro  target_ip=1.2.3.4
    1.2.3.4

Does that help?

+Jon


From vallentin at icir.org  Tue Dec  6 08:58:04 2011
From: vallentin at icir.org (Matthias Vallentin)
Date: Tue, 6 Dec 2011 08:58:04 -0800
Subject: [Bro] Is it applicapable to specific target ip using command
 line in bro?
In-Reply-To: <ED6CD411-1259-4202-AF95-7DA29334A54B@illinois.edu>
References: <2011120617014057329041@163.com>
	<ED6CD411-1259-4202-AF95-7DA29334A54B@illinois.edu>
Message-ID: <CADTpMNYGNYYVcTkv8Gugwd8RWFH7sDxheE5+vfMj3UjGDCn++g@mail.gmail.com>

> ? ?$ bro yyyy.bro ?target_ip=1.2.3.4

Hah, I didn't know about that. Is that new with 2.x?

    Matthias



From jsiwek at illinois.edu  Tue Dec  6 09:11:30 2011
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Tue, 6 Dec 2011 17:11:30 +0000
Subject: [Bro] Is it applicapable to specific target ip using command
 line in bro?
In-Reply-To: <CADTpMNYGNYYVcTkv8Gugwd8RWFH7sDxheE5+vfMj3UjGDCn++g@mail.gmail.com>
References: <2011120617014057329041@163.com>
	<ED6CD411-1259-4202-AF95-7DA29334A54B@illinois.edu>
	<CADTpMNYGNYYVcTkv8Gugwd8RWFH7sDxheE5+vfMj3UjGDCn++g@mail.gmail.com>
Message-ID: <E67995C8-6445-4AD4-A066-51FF80B245D7@illinois.edu>


>>    $ bro yyyy.bro  target_ip=1.2.3.4
> 
> Hah, I didn't know about that. Is that new with 2.x?

Shouldn't be, I remember seeing a chunk of code in the lexer when I first had to hack on it that adds "redef" statements for any X=Y command line parameters.

+Jon


From katrina at csail.mit.edu  Tue Dec  6 09:53:55 2011
From: katrina at csail.mit.edu (Katrina LaCurts)
Date: Tue, 6 Dec 2011 12:53:55 -0500
Subject: [Bro] using Bro as traffic analyzer.
In-Reply-To: <CAAG4iq=zvywAQ=gvAtgVyiRFMWD7_s1VkHySVvogTqO4cTV2TQ@mail.gmail.com>
References: <CAAG4iqn3LYWf0iEOsg77UnLgm8NXDbTiL1wY6L-eLmYON+jevg@mail.gmail.com>
	<20111206035115.E85A72C4004@rock.ICSI.Berkeley.EDU>
	<CAAG4iq=zvywAQ=gvAtgVyiRFMWD7_s1VkHySVvogTqO4cTV2TQ@mail.gmail.com>
Message-ID: <814CF7BF-E93C-439D-801F-5EB768E7826C@csail.mit.edu>

> On Mon, Dec 5, 2011 at 10:51 PM, Vern Paxson <vern at icir.org> wrote:
> > If you need more detailed information, I am currently working on an
> > analyzer for Bro that attempts to give more detailed information about the
> > retransmission behavior of a TCP connection as part of on-going research.
> 
> You should for sure contact Katrina LaCurts <katrina at csail.mit.edu>,
> who did an internship with us working on integrating this sort of analysis
> into Bro.
> 
>                Vern
> 
> I've actually seen quite a bit of her work that she emailed to me last year. It was a phenomenal base for what I've tried to expand upon. I'd be eager to see if there had been further developments with it aside from what I've seen in the development branch. 

Thanks James :)

Readon, my code can give you the RTT based on the 3-way handshake, as well as some additional RTT estimates as a connection continues, without any scripting work on your part.  Email me and I'll be happy to point you towards the development branch and explain the events to you.

Katrina


From seth at icir.org  Tue Dec  6 13:14:45 2011
From: seth at icir.org (Seth Hall)
Date: Tue, 6 Dec 2011 16:14:45 -0500
Subject: [Bro] Is it applicapable to specific target ip using command
	line in bro?
In-Reply-To: <CADTpMNYGNYYVcTkv8Gugwd8RWFH7sDxheE5+vfMj3UjGDCn++g@mail.gmail.com>
References: <2011120617014057329041@163.com>
	<ED6CD411-1259-4202-AF95-7DA29334A54B@illinois.edu>
	<CADTpMNYGNYYVcTkv8Gugwd8RWFH7sDxheE5+vfMj3UjGDCn++g@mail.gmail.com>
Message-ID: <1DEE234F-7513-44DB-8459-D5298C4EF97A@icir.org>


On Dec 6, 2011, at 11:58 AM, Matthias Vallentin wrote:

>>    $ bro yyyy.bro  target_ip=1.2.3.4
> 
> Hah, I didn't know about that. Is that new with 2.x?


Nope, it's been there for a long time.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/



From jmellander at lbl.gov  Tue Dec  6 14:56:53 2011
From: jmellander at lbl.gov (Jim Mellander)
Date: Tue, 6 Dec 2011 14:56:53 -0800
Subject: [Bro] Reassembly of IPv6 TCP streams
In-Reply-To: <83EF06F4-96C2-45D4-B19C-021D695FDF55@icir.org>
References: <CADju=b5jjK=wT3jZhKR5LqG2HCkhAbvP8f_AnCNH0sfvqC1rFQ@mail.gmail.com>
	<83EF06F4-96C2-45D4-B19C-021D695FDF55@icir.org>
Message-ID: <CADju=b5o+boZAms3NMCmaSnoqYnKstXQDMR1yLWtQjSdxkLEmQ@mail.gmail.com>

As a workaround, the latest version of tcpflow 1.0.2 (
http://freecode.com/projects/tcpflow ) performs IPv6 session
reassembly into files.  I had to hack it a bit to get it to work on
freebsd, but will be submitting the patches upstream.


On Mon, Dec 5, 2011 at 7:50 PM, Seth Hall <seth at icir.org> wrote:
>
> On Dec 5, 2011, at 7:20 PM, Jim Mellander wrote:
>
>> contents.bro performs session reconstruction of IPv4 traffic, but when
>> running Bro 1.5 contents.bro against an IPv6 packet trace, it creates
>> 0-length files, but doesn't extract the session contents to those
>> files. ?Is this in the works?
>
>
> That will be part of the work for 2.1. ?I'll file a ticket for that to make sure we look into it.
>
> ?.Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro-ids.org/
>



From xydarcher at 163.com  Tue Dec  6 17:19:36 2011
From: xydarcher at 163.com (Readon Shaw)
Date: Wed, 7 Dec 2011 09:19:36 +0800
Subject: [Bro] Is it applicapable to specific target ip using command
	line	in bro?
References: <2011120617014057329041@163.com>, 
	<ED6CD411-1259-4202-AF95-7DA29334A54B@illinois.edu>
Message-ID: <2011120709193667508411@163.com>

>> bro -r xxx.pcap yyyy.bro.
>> But here the bro script can't get the target ip through this kind of command.
>> Is there any mechanism in bro to fulfull this requirement?
>
>You can set any &redef variable from the command line.  e.g. if yyyy.bro contains:
>
>    const target_ip: addr &redef;
>
>    event bro_init()
>        {
>        print target_ip;
>        }
>
>Then you can do the following:
>
>    $ bro yyyy.bro  target_ip=1.2.3.4
>    1.2.3.4
>
>Does that help?
>
>+Jon
This is wonderful.
It's fit for me very well.
--------------
Readon Shaw




From xydarcher at 163.com  Fri Dec  9 02:36:21 2011
From: xydarcher at 163.com (Readon Shaw)
Date: Fri, 9 Dec 2011 18:36:21 +0800
Subject: [Bro] using Bro as traffic analyzer.
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>, 
	<CADTpMNaD5FXD5tjoO9Az-MtnHT1k7=6aQ7sR6pPG8SNTWALFFA@mail.gmail.com>
Message-ID: <2011120918362118592417@163.com>

>> I want to match tcp handshake pairs and record the intervals between
>> each SYN and SYN-ACK pairs with their arrival time. At the same time,
>> roughly packet loss rate (vs different timescales) should be calculated
>> by tcp retransmission rate. It is a statistical analysis on network
>> traffic that would be processed by .bro files i think. some of them
>> are similar with functions already existed. Would you please give me
>> some notes on which files i should start with?
>
>Yes, you're right. This sort of analysis can be entirely done at the
>scripting layer, i.e., it only involves *.bro scripts.
>
>Bro reassembles the full TCP byte stream. When you deal with connection
>data, duplicates/retransmission are already removed. If you want to
>compute round-trip times at the packet level, you can write a handler
>for the events new_connection (which is generated for every new
>SYN packet) and connection_established (which is generated for a
>successful TCP handshake after a SYN-ACK).

I have wrtie a script called local.bro which was applied to connect event connection_established & connection_first_ACK
but it seems that the event have not triggered. I tested the script with network trace "http.pcap" provided in Bro website.


the script is followed as attachment.

>
>For retransmissions, have a look at the events: rexmit_inconsistency,
>content_gap, and gap_report. Unfortunately I cannot provide more
>detailed information other than pointing you to our ongoing
>documentation effort in the git repository:
>
>    git clone git://git.bro-ids.org/bro.git
>    git checkout topic/script-reference
>    less src/event.bif
>
>Maybe others can chime in and give you further guidance.
>
>(Also, to measure system/NIC capture loss, there is
>policy/misc/capture-loss.bro.)
>
>> btw: I read the document and find that all C/C++ code is designed
>> for decoding packets. bro files take charge in statistal or general
>> processing. Is it right? Any general pictures were provided in bro?
>
>That's correct. Packet "decoding" is done at the Bro core. Bro
>reassembles the TCP byte stream and presents it as a connection to the
>user. You may find our workshop materials helpful to better understand
>the architecture of Bro: http://www.bro-ids.org/bro-workshop-2011
>
>    Matthias

--------------
Readon Shaw

-------------- next part --------------
A non-text attachment was scrubbed...
Name: local.bro
Type: application/octet-stream
Size: 666 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111209/e6f34199/attachment.obj 

From xydarcher at 163.com  Fri Dec  9 02:49:01 2011
From: xydarcher at 163.com (Readon Shaw)
Date: Fri, 9 Dec 2011 18:49:01 +0800
Subject: [Bro] using Bro as traffic analyzer.
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>, 
	<CADTpMNaD5FXD5tjoO9Az-MtnHT1k7=6aQ7sR6pPG8SNTWALFFA@mail.gmail.com>
Message-ID: <2011120918435096986326@163.com>

>> I want to match tcp handshake pairs and record the intervals between
>> each SYN and SYN-ACK pairs with their arrival time. At the same time,
>> roughly packet loss rate (vs different timescales) should be calculated
>> by tcp retransmission rate. It is a statistical analysis on network
>> traffic that would be processed by .bro files i think. some of them
>> are similar with functions already existed. Would you please give me
>> some notes on which files i should start with?
>
>Yes, you're right. This sort of analysis can be entirely done at the
>scripting layer, i.e., it only involves *.bro scripts.
>
>Bro reassembles the full TCP byte stream. When you deal with connection
>data, duplicates/retransmission are already removed. If you want to
>compute round-trip times at the packet level, you can write a handler
>for the events new_connection (which is generated for every new
>SYN packet) and connection_established (which is generated for a
>successful TCP handshake after a SYN-ACK).

I have wrtie a script called local.bro which was applied to connect event connection_established & connection_first_ACK
but it seems that the event have not triggered. I tested the script with network trace "http.pcap" provided in Bro website.

There are some points need to be noticed.
1.Bro 2.0 beta was installed correctly. I have tested some example.
2.bro_script_loaded event (in the script) works as well as log.

The script file is attached. Thank you.

>
>For retransmissions, have a look at the events: rexmit_inconsistency,
>content_gap, and gap_report. Unfortunately I cannot provide more
>detailed information other than pointing you to our ongoing
>documentation effort in the git repository:
>
>    git clone git://git.bro-ids.org/bro.git
>    git checkout topic/script-reference
>    less src/event.bif
>
>Maybe others can chime in and give you further guidance.
>
>(Also, to measure system/NIC capture loss, there is
>policy/misc/capture-loss.bro.)
>
>> btw: I read the document and find that all C/C++ code is designed
>> for decoding packets. bro files take charge in statistal or general
>> processing. Is it right? Any general pictures were provided in bro?
>
>That's correct. Packet "decoding" is done at the Bro core. Bro
>reassembles the TCP byte stream and presents it as a connection to the
>user. You may find our workshop materials helpful to better understand
>the architecture of Bro: http://www.bro-ids.org/bro-workshop-2011
>
>    Matthias

--------------
Readon Shaw
-------------- next part --------------
A non-text attachment was scrubbed...
Name: local.bro
Type: application/octet-stream
Size: 666 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111209/85c14906/attachment.obj 

From jsiwek at illinois.edu  Fri Dec  9 08:16:23 2011
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Fri, 9 Dec 2011 16:16:23 +0000
Subject: [Bro] using Bro as traffic analyzer.
In-Reply-To: <2011120918435096986326@163.com>
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>,
	<CADTpMNaD5FXD5tjoO9Az-MtnHT1k7=6aQ7sR6pPG8SNTWALFFA@mail.gmail.com>
	<2011120918435096986326@163.com>
Message-ID: <26B9D9C8-11E5-4905-9BDA-A0E547E33B0E@illinois.edu>


> I have wrtie a script called local.bro which was applied to connect event connection_established & connection_first_ACK
> but it seems that the event have not triggered. I tested the script with network trace "http.pcap" provided in Bro website.

If you check reporter.log, there's some hints indicating that your c$loc optional field value is missing at the times when you try to write to the log (meaning the event handlers are actually invoked, but don't do anything because of the error).  To fix it you should first check that c$loc is initialized in the handlers and also fill in any of its fields that you can.  Have a look at the alterations I made in the attached file to see if it makes sense for what you were trying to do.

+Jon
-------------- next part --------------
A non-text attachment was scrubbed...
Name: local.bro
Type: application/octet-stream
Size: 899 bytes
Desc: local.bro
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111209/297602dc/attachment.obj 

From xydarcher at 163.com  Fri Dec  9 18:10:56 2011
From: xydarcher at 163.com (Readon Shaw)
Date: Sat, 10 Dec 2011 10:10:56 +0800
Subject: [Bro] using Bro as traffic analyzer.
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>, 
	<CADTpMNaD5FXD5tjoO9Az-MtnHT1k7=6aQ7sR6pPG8SNTWALFFA@mail.gmail.com>
	<2011120918435096986326@163.com>, 
	<26B9D9C8-11E5-4905-9BDA-A0E547E33B0E@illinois.edu>
Message-ID: <201112101010568429237@163.com>

>
>> I have wrtie a script called local.bro which was applied to connect event connection_established & connection_first_ACK
>> but it seems that the event have not triggered. I tested the script with network trace "http.pcap" provided in Bro website.
>
>If you check reporter.log, there's some hints indicating that your c$loc optional field value is missing at the times when you try to write to the log (meaning the event handlers are actually invoked, but don't do anything because of the error).  To fix it you should first check that c$loc is initialized in the handlers and also fill in any of its fields that you can.  Have a look at the alterations I made in the attached file to see if it makes sense for what you were trying to do.
That is the point.
It works now, but i didn't find reporter.log in the directory. Is there some thing important i have missing in command?
>
>+Jon
>

--------------
Readon Shaw




From xydarcher at 163.com  Fri Dec  9 23:29:03 2011
From: xydarcher at 163.com (Readon Shaw)
Date: Sat, 10 Dec 2011 15:29:03 +0800
Subject: [Bro] using Bro as traffic analyzer.
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>
	<CADTpMNaD5FXD5tjoO9Az-MtnHT1k7=6aQ7sR6pPG8SNTWALFFA@mail.gmail.com>
	<201112051911272309048@163.com>, 
	<CAAG4iqn3LYWf0iEOsg77UnLgm8NXDbTiL1wY6L-eLmYON+jevg@mail.gmail.com>
Message-ID: <2011121015055752306939@163.com>

>Correct me if I am wrong Matthias.
>
>Bro can do most of what you are looking for out of the box. Sampling the
>round-trip time of the three-way handshake is doable at scripting level.
>You can get bro to output the retransmission data through the tcp_rexmit
>event. It does not give detailed information about how the data was
>retransmitted but will tell you how many bytes were retransmitted and how
>much data was outstanding when the retransmission occurred.
>
>That should be sufficient for what you are looking for.
I checked the code of tcp_rexmit event in TCP.cc. 
It seems that the event was processed with max_top_seq.
There are two issues should be considered.
1. how can i distinguish tcp_retrasmission caused by packet loss & out of order?
2. if the retransmission occurs when handshaking, would it be correctly triggered?

>
>If you need more detailed information, I am currently working on an
>analyzer for Bro that attempts to give more detailed information about the
>retransmission behavior of a TCP connection as part of on-going research.
>However, It is not in a state that is ready for release.
>
>-- 
>James Swaro*
>*
>

--------------
Readon Shaw




From james.swaro at gmail.com  Sat Dec 10 00:28:23 2011
From: james.swaro at gmail.com (James Swaro)
Date: Sat, 10 Dec 2011 03:28:23 -0500
Subject: [Bro] using Bro as traffic analyzer.
In-Reply-To: <2011121015055752306939@163.com>
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>
	<CADTpMNaD5FXD5tjoO9Az-MtnHT1k7=6aQ7sR6pPG8SNTWALFFA@mail.gmail.com>
	<201112051911272309048@163.com>
	<CAAG4iqn3LYWf0iEOsg77UnLgm8NXDbTiL1wY6L-eLmYON+jevg@mail.gmail.com>
	<2011121015055752306939@163.com>
Message-ID: <CAAG4iq=kHbw4=fzWObYHX8MLcD05CNOhw52MzdBBZSMhBZc5Ew@mail.gmail.com>

>
> I checked the code of tcp_rexmit event in TCP.cc.
> It seems that the event was processed with max_top_seq.
> There are two issues should be considered.
> 1. how can i distinguish tcp_retrasmission caused by packet loss & out of
> order?
>

It would be entirely possible to make certain assumptions during
post-processing of your logs. It is possible to determine which
retransmissions are legitimately out-of-order and not actual
retransmissions, if you have some sense of the round trip time of the
connection or other methods. Perhaps Katrina or someone else could chime in
and explain this in more detail. I am curious to know as well.


> 2. if the retransmission occurs when handshaking, would it be correctly
> triggered?
>
>
The sequence number to be acknowledged remains the same during handshaking.
The event should be correctly triggered.

int seq_delta = top_seq - max_top_seq;
> if ( seq_delta <= 0 )


Consider that top_seq is always set to the sequence number in the TCP
header plus the length of the packet. Given that the syn/syn-ack packets
never carry data, you will always trigger the retransmit event upon the
second transmission of a syn/syn-ack segment with a sequence number less
than or equal to the maximum sequence number observed. It should always
trigger.

-- James Swaro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111210/950608c6/attachment.html 

From katrina at csail.mit.edu  Sat Dec 10 19:26:09 2011
From: katrina at csail.mit.edu (Katrina LaCurts)
Date: Sat, 10 Dec 2011 22:26:09 -0500
Subject: [Bro] using Bro as traffic analyzer.
In-Reply-To: <CAAG4iq=kHbw4=fzWObYHX8MLcD05CNOhw52MzdBBZSMhBZc5Ew@mail.gmail.com>
References: <58db867a.8798.13407e8589f.Coremail.xydarcher@163.com>
	<CADTpMNa1iiTLA-rYeuj1CBr-RkO2X6MOpYsmDye_jf0oUFWaBw@mail.gmail.com>
	<2011120510474838157151@163.com>
	<CADTpMNaD5FXD5tjoO9Az-MtnHT1k7=6aQ7sR6pPG8SNTWALFFA@mail.gmail.com>
	<201112051911272309048@163.com>
	<CAAG4iqn3LYWf0iEOsg77UnLgm8NXDbTiL1wY6L-eLmYON+jevg@mail.gmail.com>
	<2011121015055752306939@163.com>
	<CAAG4iq=kHbw4=fzWObYHX8MLcD05CNOhw52MzdBBZSMhBZc5Ew@mail.gmail.com>
Message-ID: <9442AC64-0904-4E91-B45D-B74C7CD2933E@csail.mit.edu>

> It is possible to determine which retransmissions are legitimately out-of-order and not actual retransmissions, if you have some sense of the round trip time of the connection or other methods. Perhaps Katrina or someone else could chime in and explain this in more detail. I am curious to know as well.

That's the general idea.  You can check retransmissions vs. out-of-order (vs. replay packets) by examining the obvious things such as IP IDs and sequence numbers, and then checking the inter-arrival time between the packet in question and the previous packet.  If that IAT is less than the minimum RTT you've observed on the connection, then you're likely dealing with either a replay packet or an out-of-order packet (and that distinction can be resolved with sequence numbers and IP IDs).

It is a bit of a pain (one has to keep track of RTTs, what sequence numbers we've seen, etc.), but that's how my analyzer handles it.

Katrina


From xydarcher at 163.com  Sun Dec 11 01:27:28 2011
From: xydarcher at 163.com (Readon Shaw)
Date: Sun, 11 Dec 2011 17:27:28 +0800
Subject: [Bro] Does Bro support pcap file with id 9?
Message-ID: <2011121117272820214111@163.com>

When i was processing traces downloaded from WAND, it was reported that "Unknown data link type 0x9".
I am using Bro 2.0 Beta.
If not, is there any solution?




Readon Shaw
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111211/9d2d8c44/attachment.html 

From jdmfontz at yahoo.com  Sun Dec 11 05:16:42 2011
From: jdmfontz at yahoo.com (Martin Fontanez)
Date: Sun, 11 Dec 2011 05:16:42 -0800 (PST)
Subject: [Bro] broccoli/swig issue with 2.0-beta
In-Reply-To: <20111108160558.GC6004@datacomm.albany.edu>
Message-ID: <1323609402.5285.YahooMailClassic@web120602.mail.ne1.yahoo.com>

Pls. remove me from this distro.
?
?
Thank you,
?
Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111211/e87e51c1/attachment.html 

From jdmfontz at yahoo.com  Sun Dec 11 05:18:12 2011
From: jdmfontz at yahoo.com (Martin Fontanez)
Date: Sun, 11 Dec 2011 05:18:12 -0800 (PST)
Subject: [Bro] Remove
In-Reply-To: <1323609402.5285.YahooMailClassic@web120602.mail.ne1.yahoo.com>
Message-ID: <1323609492.73856.YahooMailClassic@web120604.mail.ne1.yahoo.com>





Pls. remove me from this distro.? Pardon the spam.

--- On Sun, 12/11/11, Martin Fontanez <jdmfontz at yahoo.com> wrote:


From: Martin Fontanez <jdmfontz at yahoo.com>
Subject: Re: [Bro] broccoli/swig issue with 2.0-beta
To: bro at bro-ids.org
Date: Sunday, December 11, 2011, 8:16 AM







Pls. remove me from this distro.
?
?
Thank you,
?
Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111211/477051f3/attachment.html 

From seth at icir.org  Sun Dec 11 16:59:46 2011
From: seth at icir.org (Seth Hall)
Date: Sun, 11 Dec 2011 19:59:46 -0500
Subject: [Bro] Does Bro support pcap file with id 9?
In-Reply-To: <2011121117272820214111@163.com>
References: <2011121117272820214111@163.com>
Message-ID: <9A2E5BA5-BD96-4DBF-B4C5-35C3394E454F@icir.org>


On Dec 11, 2011, at 4:27 AM, Readon Shaw wrote:

> When i was processing traces downloaded from WAND, it was reported that "Unknown data link type 0x9".
> I am using Bro 2.0 Beta.
> If not, is there any solution?

Unfortunately we don't support PPP packets (link type 9) yet.  If you point us to the traces you're are using though, we can make sure that we support PPP when we get around to reworking our packet input process.

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/




From xydarcher at 163.com  Sun Dec 11 18:07:16 2011
From: xydarcher at 163.com (Readon Shaw)
Date: Mon, 12 Dec 2011 10:07:16 +0800
Subject: [Bro] Does Bro support pcap file with id 9?
References: <2011121117272820214111@163.com>, 
	<9A2E5BA5-BD96-4DBF-B4C5-35C3394E454F@icir.org>
Message-ID: <2011121210071608832420@163.com>

I have attached a pcap file in previous mail.
>
>On Dec 11, 2011, at 4:27 AM, Readon Shaw wrote:
>
>> When i was processing traces downloaded from WAND, it was reported that "Unknown data link type 0x9".
>> I am using Bro 2.0 Beta.
>> If not, is there any solution?
>
>Unfortunately we don't support PPP packets (link type 9) yet.  If you point us to the traces you're are using though, we can make sure that we support PPP when we get around to reworking our packet input process.
>
> .Seth
>
>--
>Seth Hall
>International Computer Science Institute
>(Bro) because everyone has a network
>http://www.bro-ids.org/
>

--------------
Readon Shaw




From panos.sakkos at gmail.com  Mon Dec 12 01:29:26 2011
From: panos.sakkos at gmail.com (Panos Sakkos)
Date: Mon, 12 Dec 2011 10:29:26 +0100
Subject: [Bro] nprobe, ngrep, tcpdump and tcpflow -like behavior of BRO ids?
Message-ID: <CAERNvERfdELi7yemi278=jbeF-YTDbC-06bCwK8oT+XetvdF3w@mail.gmail.com>

Hello!

I want to ask you if BRO ids can totally replace the following software:


   - nprobe
   - ngrep
   - tcpdump
   - and tcpflow

Thank you in advance!
panos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111212/d94f2b8c/attachment.html 

From seth at icir.org  Mon Dec 12 07:19:05 2011
From: seth at icir.org (Seth Hall)
Date: Mon, 12 Dec 2011 10:19:05 -0500
Subject: [Bro] nprobe, ngrep,
	tcpdump and tcpflow -like behavior of BRO ids?
In-Reply-To: <CAERNvERfdELi7yemi278=jbeF-YTDbC-06bCwK8oT+XetvdF3w@mail.gmail.com>
References: <CAERNvERfdELi7yemi278=jbeF-YTDbC-06bCwK8oT+XetvdF3w@mail.gmail.com>
Message-ID: <A80BE606-8A85-4E27-9973-3DD3937E1BAF@icir.org>


On Dec 12, 2011, at 4:29 AM, Panos Sakkos wrote:

> I want to ask you if BRO ids can totally replace the following software:
> 
> 	? nprobe
> 	? ngrep
> 	? tcpdump
> 	? and tcpflow 

Instead of pointing to tools and asking if Bro can replace them, could you explain tasks you need to accomplish with a network monitoring tool?  All of those tools have a lot of functionality and Bro certainly doesn't implement every bit of functionality they have. :)
 
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/




From panos.sakkos at gmail.com  Mon Dec 12 08:08:43 2011
From: panos.sakkos at gmail.com (Panos Sakkos)
Date: Mon, 12 Dec 2011 17:08:43 +0100
Subject: [Bro] nprobe, ngrep,
	tcpdump and tcpflow -like behavior of BRO ids?
In-Reply-To: <A80BE606-8A85-4E27-9973-3DD3937E1BAF@icir.org>
References: <CAERNvERfdELi7yemi278=jbeF-YTDbC-06bCwK8oT+XetvdF3w@mail.gmail.com>
	<A80BE606-8A85-4E27-9973-3DD3937E1BAF@icir.org>
Message-ID: <799555C9-B164-4F96-9E2D-F709817C1BCC@gmail.com>

I am sorry for my incomplete question.
Here are the functionalities we need:

nprobe => convert raw network traffic to netflow format
ngrep => extract fields from incoming and outgoing HTTP traffic (url, referer, ...)
tcpdump => store size-limited TCP session (for an incoming SSH connection for example)
tcpflow => reconstruct TCP flows for given sessions (given source ip for example)

Thank you,
Panos

On Dec 12, 2011, at 4:19 PM, Seth Hall wrote:

> 
> On Dec 12, 2011, at 4:29 AM, Panos Sakkos wrote:
> 
>> I want to ask you if BRO ids can totally replace the following software:
>> 
>> 	? nprobe
>> 	? ngrep
>> 	? tcpdump
>> 	? and tcpflow 
> 
> Instead of pointing to tools and asking if Bro can replace them, could you explain tasks you need to accomplish with a network monitoring tool?  All of those tools have a lot of functionality and Bro certainly doesn't implement every bit of functionality they have. :)
> 
>  .Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro-ids.org/
> 




From seth at icir.org  Mon Dec 12 10:15:51 2011
From: seth at icir.org (Seth Hall)
Date: Mon, 12 Dec 2011 13:15:51 -0500
Subject: [Bro] nprobe, ngrep,
	tcpdump and tcpflow -like behavior of BRO ids?
In-Reply-To: <799555C9-B164-4F96-9E2D-F709817C1BCC@gmail.com>
References: <CAERNvERfdELi7yemi278=jbeF-YTDbC-06bCwK8oT+XetvdF3w@mail.gmail.com>
	<A80BE606-8A85-4E27-9973-3DD3937E1BAF@icir.org>
	<799555C9-B164-4F96-9E2D-F709817C1BCC@gmail.com>
Message-ID: <3D74E8FE-E7A3-41C3-914B-9D3761ECEF95@icir.org>


On Dec 12, 2011, at 11:08 AM, Panos Sakkos wrote:

> nprobe => convert raw network traffic to netflow format

Bro doesn't output netflow, but we have a connection analyzer and scripts that output a file named conn.log which is similar but with more information.

> ngrep => extract fields from incoming and outgoing HTTP traffic (url, referer, ?)

I don't know if I would say that is a capability of ngrep.  I guess in some cases it works for that, but the Bro 2.0 beta does a much better job.

> tcpdump => store size-limited TCP session (for an incoming SSH connection for example)

tcpdump doesn't even do this (that I know of).  We have a tool named Time Machine that can do this and more though.  It should be getting more attention and work done on it soon too.

> tcpflow => reconstruct TCP flows for given sessions (given source ip for example)

Yes

Try the 2.0 beta from our site.  It's much easier to begin using that the current 1.5 release.  You should be able to have some output in just a few minutes.  Our quick start guide is available here:
	http://www.bro-ids.org/documentation-beta/quickstart.bro.html

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/




From james.swaro at gmail.com  Fri Dec 16 08:21:50 2011
From: james.swaro at gmail.com (James Swaro)
Date: Fri, 16 Dec 2011 11:21:50 -0500
Subject: [Bro] Optional Internal Values - Bro 2.0
Message-ID: <CAAG4iqn8fOJhFFG+jxU=1EOJw_mxjO-ZVgOY7KsnFsvYVODXcw@mail.gmail.com>

Just to clarify something that I couldn't find in the documentation.

Optional Internal values can be set as arguments,

Ex. bro -Cr somefile.trace some/policy/here myvar=somevalue


as well as at the script level,

Ex. redef myvar = somevalue;


Correct?

It seems I cannot get the script level value to set, but the command line
argument is correctly set.

Thoughts?
-- 
James Swaro*
*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111216/1e54e3eb/attachment.html 

From david.bianco at ge.com  Fri Dec 16 08:42:08 2011
From: david.bianco at ge.com (Bianco, David (GE Corporate))
Date: Fri, 16 Dec 2011 11:42:08 -0500
Subject: [Bro] Optional Internal Values - Bro 2.0
In-Reply-To: <CAAG4iqn8fOJhFFG+jxU=1EOJw_mxjO-ZVgOY7KsnFsvYVODXcw@mail.gmail.com>
References: <CAAG4iqn8fOJhFFG+jxU=1EOJw_mxjO-ZVgOY7KsnFsvYVODXcw@mail.gmail.com>
Message-ID: <2983190F3328F54DAF2294B11BA9CA9B0E09DD62@CINMLVEM19.e2k.ad.ge.com>

In your broscript file, you need to qualify the variable name.  For
example:

 

    redef Module::myvar = somevalue;

 

                David

 

 

From: bro-bounces at bro-ids.org [mailto:bro-bounces at bro-ids.org] On Behalf
Of James Swaro
Sent: Friday, December 16, 2011 11:22 AM
To: Bro
Subject: [Bro] Optional Internal Values - Bro 2.0

 

Just to clarify something that I couldn't find in the documentation.

 

Optional Internal values can be set as arguments, 

	 

	Ex. bro -Cr somefile.trace some/policy/here myvar=somevalue 

 

as well as at the script level,  

 

	Ex. redef myvar = somevalue; 

 

Correct?  


 

It seems I cannot get the script level value to set, but the command
line argument is correctly set. 

 

Thoughts? 

-- 

James Swaro

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111216/c4f1a38a/attachment.html 

From seth at icir.org  Fri Dec 16 08:42:15 2011
From: seth at icir.org (Seth Hall)
Date: Fri, 16 Dec 2011 11:42:15 -0500
Subject: [Bro] Optional Internal Values - Bro 2.0
In-Reply-To: <CAAG4iqn8fOJhFFG+jxU=1EOJw_mxjO-ZVgOY7KsnFsvYVODXcw@mail.gmail.com>
References: <CAAG4iqn8fOJhFFG+jxU=1EOJw_mxjO-ZVgOY7KsnFsvYVODXcw@mail.gmail.com>
Message-ID: <CCF9B28B-6F89-4ECA-BB29-47B91ACAAA29@icir.org>


On Dec 16, 2011, at 11:21 AM, James Swaro wrote:

> It seems I cannot get the script level value to set, but the command line argument is correctly set. 

Could you give a complete example?  That feature does work, I use it regularly and it's even used in the tests.

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/




From jsiwek at illinois.edu  Fri Dec 16 08:43:49 2011
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Fri, 16 Dec 2011 16:43:49 +0000
Subject: [Bro] Optional Internal Values - Bro 2.0
In-Reply-To: <CAAG4iqn8fOJhFFG+jxU=1EOJw_mxjO-ZVgOY7KsnFsvYVODXcw@mail.gmail.com>
References: <CAAG4iqn8fOJhFFG+jxU=1EOJw_mxjO-ZVgOY7KsnFsvYVODXcw@mail.gmail.com>
Message-ID: <4A66A961-D78A-4DE5-BD33-0FF317696BCC@illinois.edu>

> Optional Internal values can be set as arguments, 
> 
> Ex. bro -Cr somefile.trace some/policy/here myvar=somevalue 
> 
> as well as at the script level,  
> 
> Ex. redef myvar = somevalue; 
> 
> Correct?  

For variables declared with the &redef attribute, yes.

> It seems I cannot get the script level value to set, but the command line argument is correctly set. 

It should work both ways.  Is there a specific error you're getting or a specific/contrived example you can give?

+Jon


From james.swaro at gmail.com  Fri Dec 16 12:41:34 2011
From: james.swaro at gmail.com (James Swaro)
Date: Fri, 16 Dec 2011 15:41:34 -0500
Subject: [Bro] Optional Internal Values - Bro 2.0
In-Reply-To: <4A66A961-D78A-4DE5-BD33-0FF317696BCC@illinois.edu>
References: <CAAG4iqn8fOJhFFG+jxU=1EOJw_mxjO-ZVgOY7KsnFsvYVODXcw@mail.gmail.com>
	<4A66A961-D78A-4DE5-BD33-0FF317696BCC@illinois.edu>
Message-ID: <CAAG4iq=Du6bnFZXWzPsQLREUvMTyQa1h=hzqvxqVycb1Qmbtog@mail.gmail.com>

Sorry about the late reply. The error is actually 'pebcac' in nature. The
scripts that I had been editing weren't placed back into the default policy
path, which is why no matter the change , the same result occurred. Perhaps
something that could be attributed to a lack of coffee in the early
morning. Sorry guys :)

All is still well in the world of bro scripts.

On Fri, Dec 16, 2011 at 11:43 AM, Siwek, Jonathan Luke
<jsiwek at illinois.edu>wrote:

> > Optional Internal values can be set as arguments,
> >
> > Ex. bro -Cr somefile.trace some/policy/here myvar=somevalue
> >
> > as well as at the script level,
> >
> > Ex. redef myvar = somevalue;
> >
> > Correct?
>
> For variables declared with the &redef attribute, yes.
>
> > It seems I cannot get the script level value to set, but the command
> line argument is correctly set.
>
> It should work both ways.  Is there a specific error you're getting or a
> specific/contrived example you can give?
>
> +Jon




-- 
James Swaro*
*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111216/03ecd582/attachment.html 

From ke17 at aub.edu.lb  Fri Dec 30 07:13:34 2011
From: ke17 at aub.edu.lb (Khaled El Dassouki)
Date: Fri, 30 Dec 2011 17:13:34 +0200
Subject: [Bro] tcp delay events!?
Message-ID: <20111230171334.20886h7ac69bw2se@imail.aub.edu.lb>

Hello,
I am using Bro in my research work. My problem is that I am trying to  
write a Bro script that fires alarms based on TCP packet delays. I  
didn?t find any Bro event that could be handled at every received  
packet. I tried the tcp_packet and new_packet events but it seems that  
they are not fired at every received packet. Even I tried to write a  
signature that could be hit at every tcp packet but I found that  
unfortunately tcp signatures could be hit only once at the receiving  
of the first tcp packet.
Please help I am really tired?




From vern at icir.org  Sat Dec 31 12:17:44 2011
From: vern at icir.org (Vern Paxson)
Date: Sat, 31 Dec 2011 12:17:44 -0800
Subject: [Bro] tcp delay events!?
In-Reply-To: <20111230171334.20886h7ac69bw2se@imail.aub.edu.lb> (Fri,
	30 Dec 2011 17:13:34 +0200).
Message-ID: <20111231201744.9D04F2C4114@rock.ICSI.Berkeley.EDU>

> I tried the tcp_packet and new_packet events but it seems that  
> they are not fired at every received packet.

They pretty much should indeed be generated for every received packet,
other than corner-case exceptions such as bad packet headers, or fragments
(there are a number of these).  What I suspect is happening is that
the traffic you're interested in isn't matching the packet-capture filter,
so it's not being looked at in the first place.  The way to check this
is to invoke bro using "-f tcp" to set the capture filter to all TCP packets.

		Vern


From vallentin at icir.org  Sat Dec 31 16:43:29 2011
From: vallentin at icir.org (Matthias Vallentin)
Date: Sun, 1 Jan 2012 01:43:29 +0100
Subject: [Bro] tcp delay events!?
In-Reply-To: <20111231201744.9D04F2C4114@rock.ICSI.Berkeley.EDU>
References: <20111230171334.20886h7ac69bw2se@imail.aub.edu.lb>
	<20111231201744.9D04F2C4114@rock.ICSI.Berkeley.EDU>
Message-ID: <CADTpMNYczXnLt-Ea+pjCfTS=UzD-jmXugkcEws-bfBRfdgiVrg@mail.gmail.com>

> (there are a number of these)

When you're running from a trace, you may want to use -C to also
process packets with invalid checksums.

    Matthias