From j.sentier206 at laposte.net  Mon May  2 07:23:17 2011
From: j.sentier206 at laposte.net (j.sentier206)
Date: Mon,  2 May 2011 16:23:17 +0200 (CEST)
Subject: [Bro] ICMP Redirect patch
Message-ID: <17164168.69581.1304346197310.JavaMail.www@wwinf8215>

Hello,

Here is a small and quick patch I made to Bro to get icmp_redirect events with the redirection address.
Hope it will be useful to you.

Best regards,
Julien


Une messagerie gratuite, garantie ? vie et des services en plus, ?a vous tente ?
Je cr?e ma bo?te mail www.laposte.net
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: redirect.diff
Url: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110502/b150e4e0/attachment.ksh 

From misha2fun at gmail.com  Mon May  2 08:20:00 2011
From: misha2fun at gmail.com (misha singhal)
Date: Mon, 2 May 2011 20:50:00 +0530
Subject: [Bro] Script against DOS Attack
Message-ID: <BANLkTi=in_Jftehpm9h5SXZZ6KA5g+X4Pg@mail.gmail.com>

Can anybody help in writing script against DOS Attack.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110502/344fa0cd/attachment.html 

From baxterw3232 at gmail.com  Mon May  2 09:36:03 2011
From: baxterw3232 at gmail.com (Will)
Date: Mon, 2 May 2011 12:36:03 -0400
Subject: [Bro] Internal hostname in mail.log
Message-ID: <BANLkTim_6wW358V=vuD4UBbRAtPHxORnyQ@mail.gmail.com>

Hi All,

Can anyone help me determine which broctl script/postprocessor (I'm
assuming) is adding an internal hostname (user.host.com) to mail.log.

cat mail.log:
>  2011-05-02-11:43:05 HTTP_IncorrectFileType (L) 10.0.0.1 = user.host.com
  <bro> application/x-dosexec http://www[.]bad[.]com/PrintPeer[.]dat

I would like to add internal hostnames to some other alerts and would
like to avoid re-creating the wheel.

Thanks in advance for any help!

Will


From robin at icir.org  Mon May  2 11:13:25 2011
From: robin at icir.org (Robin Sommer)
Date: Mon, 2 May 2011 11:13:25 -0700
Subject: [Bro] Internal hostname in mail.log
In-Reply-To: <BANLkTim_6wW358V=vuD4UBbRAtPHxORnyQ@mail.gmail.com>
References: <BANLkTim_6wW358V=vuD4UBbRAtPHxORnyQ@mail.gmail.com>
Message-ID: <20110502181325.GB9443@icir.org>


On Mon, May 02, 2011 at 12:36 -0400, you wrote:

> Can anyone help me determine which broctl script/postprocessor (I'm
> assuming) is adding an internal hostname (user.host.com) to mail.log.

It's done inside Bro actually, see the
aux/broctl/policy/mail-alarms.bro script.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From baxterw3232 at gmail.com  Mon May  2 11:18:54 2011
From: baxterw3232 at gmail.com (Will)
Date: Mon, 2 May 2011 14:18:54 -0400
Subject: [Bro] Internal hostname in mail.log
In-Reply-To: <20110502181325.GB9443@icir.org>
References: <BANLkTim_6wW358V=vuD4UBbRAtPHxORnyQ@mail.gmail.com>
	<20110502181325.GB9443@icir.org>
Message-ID: <BANLkTikOgsMvjSs=8isOQnbpNBzehLgG0g@mail.gmail.com>

Perfect! Thanks Robin!

Will

On Mon, May 2, 2011 at 2:13 PM, Robin Sommer <robin at icir.org> wrote:
>
> On Mon, May 02, 2011 at 12:36 -0400, you wrote:
>
>> Can anyone help me determine which broctl script/postprocessor (I'm
>> assuming) is adding an internal hostname (user.host.com) to mail.log.
>
> It's done inside Bro actually, see the
> aux/broctl/policy/mail-alarms.bro script.
>
> Robin
>
> --
> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
> ICSI/LBNL ? ?* Fax ? +1 (510) 666-2956 * ? www.icir.org
>



From jmellander at lbl.gov  Tue May  3 16:40:15 2011
From: jmellander at lbl.gov (Jim Mellander)
Date: Tue, 3 May 2011 16:40:15 -0700
Subject: [Bro] Experiences with Bro and FreeBSD 8.2 Zerocopy BPF
Message-ID: <BANLkTinjuzcA0xOAS8PfSTNueJcz7oG1rQ@mail.gmail.com>

Greetings:

We were eager to explore Zerocopy BPF and after making sure bro was fully
functional, we changed to 0-copy via:

sysctl net.bpf.zerocopy_enable=1

We should have known we were in for trouble when tcpdump then immediately
began coredumping on exit.  We installed the latest and greatest tcpdump and
libpcap (v 1.1.1) via FreeBSD ports, and had the same user-experience.  The
following is offered in the hope that others may avoid the special type of
fun that we enjoyed - keep in mind this fun is only to be had when 0 copy is
enabled:


   1. As previously mentioned, tcpdump coredumps, gdb indicates that it
   tried to call free() upon exit, presumably trying to free a kernel-owned
   buffer.  Didn't debug it any more, but it was a portent of things to come.
    Later found a patch for this issue at
   http://sourceforge.net/tracker/?func=detail&aid=3290385&group_id=53067&atid=469579
   2. Bro failed to run with 0-copy - quite a bit of dithering indicated
   that it was freezing at pcap_next(), which reads the next packet from the
   interface.
   3. Wrote a test program using pcap_next() - it fails under 0-copy after
   several hundred packets.  Well, since tcpdump does work (except for the
   coredump), lets see what its doing:
   4. tcpdump is working using pcap_next_ex() instead of pcap_next(), so I
   wrote a replacement pcap_next() in terms of pcap_next_ex(), and it correctly
   grabs packets.
   5. grafted replacement pcap_next() into bro, and the user experience was
   the same :-(
   6. Lots of debugging using various cutlery on bro, eventually libpcap
   came into focus as a potential culprit
   7. sliced and diced the 0-copy code inside of libpcap - found a few
   places where improvements could be made (but that's a different story),
   which gave quite a bit of insight into its innerds - here's a presentation
   on 0-copy:
   http://www.seccuris.com/documents/whitepapers/20070517-devsummit-zerocopybpf.pdf
   8. Ran bro with a simplified policy of just conn, tcp & vlan - (our
   packets at this point in our network are vlan tagged) - it worked!
   9. Ran again with our policy, it freezes!
   10. After a somewhat binary search of policy, discovered that remote.bro
   causes zero-copy to freeze.  So, after all that, it turns out that bro works
   with libpcap-1.1.1 on 0-copy, but it took a lot to figure that out.

So turning off the remote communication fixes the issue in the short term,
but doesn't solve it for us, since broctl uses the same mechanism :-(

Haven't finished debugging yet, but it appears that broccoli may be causing
the issue on 0-copy - when it becomes clearer, I will send more.

This is written in the hopes that folks won't be tearing their hair out,
like us, as they go forward in this direction.  If anyone has any
suggestions, etc. (particularly in going forward with solving this problem),
I would appreciate it.

Hope this helps,

Jim Mellander
NERSC CyberSecurity

BTW - it appears on zero copy that net.bpf.maxbufsize & net.bpf.bufsize are
limited to 2 megs in size - they can be bigger but apparently it won't be
used, per netstat -B, which is your friend when debugging these issues.

BTW #2: zerocopy seems to be worth doing, especially at high bandwidth's
that we're moving up to, so its important to us to solve this.

BTW #3: the problem doesn't just manifest on a hi-speed link - I pointed Bro
towards our management port (100M), and it failed in the same way, so its
not a capacity issue.

BTW #4: There's no special config other than setting the sysctl to turn on
0-copy - libpcap detects that it is running 0-copy and follows a different
code path, but the API is the same - except that the issue we've been having
(and the coredump of tcpdump) indicates that 0-copy is not quite fully
baked.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110503/de103115/attachment.html 

From robin at icir.org  Wed May  4 07:45:03 2011
From: robin at icir.org (Robin Sommer)
Date: Wed, 4 May 2011 07:45:03 -0700
Subject: [Bro] Experiences with Bro and FreeBSD 8.2 Zerocopy BPF
In-Reply-To: <BANLkTinjuzcA0xOAS8PfSTNueJcz7oG1rQ@mail.gmail.com>
References: <BANLkTinjuzcA0xOAS8PfSTNueJcz7oG1rQ@mail.gmail.com>
Message-ID: <20110504144503.GK93325@icir.org>


On Tue, May 03, 2011 at 16:40 -0700, you wrote:

> like us, as they go forward in this direction.  If anyone has any
> suggestions, etc. (particularly in going forward with solving this problem),
> I would appreciate it.

Just a quick thought: you could see if configuring with
--disable-select-loop makes a difference. That changes the pcap code
in Bro a bit (on FreeBSD actually not that much, but still worth a
try).

If it did help, not sure what that would mean though: we have just
ripped out the alternative code path for Bro 1.6 ...

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From jmellander at lbl.gov  Wed May  4 08:46:20 2011
From: jmellander at lbl.gov (Jim Mellander)
Date: Wed, 4 May 2011 08:46:20 -0700
Subject: [Bro] Experiences with Bro and FreeBSD 8.2 Zerocopy BPF
In-Reply-To: <20110504144503.GK93325@icir.org>
References: <BANLkTinjuzcA0xOAS8PfSTNueJcz7oG1rQ@mail.gmail.com>
	<20110504144503.GK93325@icir.org>
Message-ID: <BANLkTi=YFG0YDCJVwu-W5hknVKmp5TMFsw@mail.gmail.com>

We tried configuring with and without the select loop option, as well as
quite a bit of custom hacking.  I'm wondering if there is something that
broccoli does that tinkers with socket settings that affects in some weird
way the bpf socket.

At some point in the next couple of weeks I'm gonna pick this up again.

On Wed, May 4, 2011 at 7:45 AM, Robin Sommer <robin at icir.org> wrote:

>
> On Tue, May 03, 2011 at 16:40 -0700, you wrote:
>
> > like us, as they go forward in this direction.  If anyone has any
> > suggestions, etc. (particularly in going forward with solving this
> problem),
> > I would appreciate it.
>
> Just a quick thought: you could see if configuring with
> --disable-select-loop makes a difference. That changes the pcap code
> in Bro a bit (on FreeBSD actually not that much, but still worth a
> try).
>
> If it did help, not sure what that would mean though: we have just
> ripped out the alternative code path for Bro 1.6 ...
>
> Robin
>
> --
> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
> ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110504/06f48e09/attachment.html 

From robin at icir.org  Thu May  5 17:58:47 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 5 May 2011 17:58:47 -0700
Subject: [Bro] Open Position at NCSA
Message-ID: <20110506005847.GB63816@icir.org>

Have you always wanted to work on Bro and be paid for it? :-)

NCSA has an opening for a Senior Research Programmer:

    https://jobs.illinois.edu/default.cfm?page=job&jobID=8847&returnPage=sear

This is a full-time position that focuses on Bro development in close
collaboration with our group here at ICSI.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From seth at icir.org  Wed May 11 06:18:28 2011
From: seth at icir.org (Seth Hall)
Date: Wed, 11 May 2011 09:18:28 -0400
Subject: [Bro] http-ext-identified-files
In-Reply-To: <BANLkTi=SNY8TXq6xO4WnZMPqLmhQs1b+Jw@mail.gmail.com>
References: <BANLkTi=SNY8TXq6xO4WnZMPqLmhQs1b+Jw@mail.gmail.com>
Message-ID: <32435DF4-2528-4822-A58B-A0F9D8A56D04@icir.org>

Sorry for not reply earlier.  I started a response to your email and never finished it. :)

On Apr 1, 2011, at 2:20 PM, Will wrote:

> 1. The old way of flagging via 'HTTP_WatchedMIMEType' appears to have gone away

Hm, I wonder why I removed that?  There will be a solution for this problem in the next release.  

Did you end up figuring out what was wrong with this?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/




From init.conf at gmail.com  Wed May 11 10:00:56 2011
From: init.conf at gmail.com (Aashish Sharma)
Date: Wed, 11 May 2011 10:00:56 -0700
Subject: [Bro] http-ext-identified-files
In-Reply-To: <6DDEEC38-07CD-4C86-8C26-1322F83172F1@gmail.com>
References: <BANLkTi=SNY8TXq6xO4WnZMPqLmhQs1b+Jw@mail.gmail.com>
	<32435DF4-2528-4822-A58B-A0F9D8A56D04@icir.org>
	<6DDEEC38-07CD-4C86-8C26-1322F83172F1@gmail.com>
Message-ID: <BANLkTikBuQG7-0e40mnH5cAosvxwCFLbGA@mail.gmail.com>

>
> Hello:
>
> HTTP_WatchedMIMEType is declared in
> bro/share/bro/http-identified-files.bro.
>
> I think you can make the code work by doing the following changes in the
> http-ext-identified-files.bro
>
> 1) Load http-identified-files
> 2) change "const" to "redef" for the following variables:
> watched_mime_types, ignored_urls, mime_types_extensions, ignored_signatures
> 3) Comment out declaration of HTTP_IncorrectFileType from
> http-ext-identified-files.bro
>
>
> + @load http-identified-files
>
> -       redef enum Notice += {
> -               # This notice is thrown when the file extension doesn't
> -               # seem to match the file contents.
> -               HTTP_IncorrectFileType,
> -       };
>
> -       const watched_mime_types = /application\/x-dosexec/
> +       redef watched_mime_types = /application\/x-dosexec/
>
>
> -       const ignored_urls =
> /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/
> &redef;
> +       redef ignored_urls =
> /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/ ;
>
>
> -       redef mime_types_extensions: table[string] of pattern = {
> +       const mime_types_extensions: table[string] of pattern = {
>
>
> - const ignored_signatures += /^matchfile-/ &redef;
> + redef ignored_signatures += /^matchfile-/;
>
> Aashish
>
> On May 11, 2011, at 6:18 AM, Seth Hall wrote:
>
> > Sorry for not reply earlier.  I started a response to your email and
> never finished it. :)
> >
> > On Apr 1, 2011, at 2:20 PM, Will wrote:
> >
> >> 1. The old way of flagging via 'HTTP_WatchedMIMEType' appears to have
> gone away
> >
> > Hm, I wonder why I removed that?  There will be a solution for this
> problem in the next release.
> >
> > Did you end up figuring out what was wrong with this?
> >
> >  .Seth
> >
> > --
> > Seth Hall
> > International Computer Science Institute
> > (Bro) because everyone has a network
> > http://www.bro-ids.org/
> >
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110511/f7ddd9b9/attachment.html 

From baxterw3232 at gmail.com  Thu May 12 10:19:57 2011
From: baxterw3232 at gmail.com (Will)
Date: Thu, 12 May 2011 13:19:57 -0400
Subject: [Bro] http-ext-identified-files
In-Reply-To: <6DDEEC38-07CD-4C86-8C26-1322F83172F1@gmail.com>
References: <BANLkTi=SNY8TXq6xO4WnZMPqLmhQs1b+Jw@mail.gmail.com>
	<32435DF4-2528-4822-A58B-A0F9D8A56D04@icir.org>
	<6DDEEC38-07CD-4C86-8C26-1322F83172F1@gmail.com>
Message-ID: <BANLkTim+YR2y2KrY5iDXz9qHGgcK2CirPA@mail.gmail.com>

On Wed, May 11, 2011 at 12:54 PM, Aashish SHARMA <aashish043 at gmail.com> wrote:
> Hello:
>
> HTTP_WatchedMIMEType is declared in bro/share/bro/http-identified-files.bro.
>
> I think you can make the code work by doing the following changes in the http-ext-identified-files.bro
>
> 1) Load http-identified-files
> 2) change "const" to "redef" for the following variables: watched_mime_types, ignored_urls, mime_types_extensions, ignored_signatures
> 3) Comment out declaration of HTTP_IncorrectFileType from http-ext-identified-files.bro
>
>
> + @load http-identified-files
>
> - ? ? ? redef enum Notice += {
> - ? ? ? ? ? ? ? # This notice is thrown when the file extension doesn't
> - ? ? ? ? ? ? ? # seem to match the file contents.
> - ? ? ? ? ? ? ? HTTP_IncorrectFileType,
> - ? ? ? };
>
> - ? ? ? const watched_mime_types = /application\/x-dosexec/
> + ? ? ? redef watched_mime_types = /application\/x-dosexec/
>
>
> - ? ? ? const ignored_urls = /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/ &redef;
> + ? ? ? redef ignored_urls = /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/ ;
>
>
> - ? ? ? redef mime_types_extensions: table[string] of pattern = {
> + ? ? ? const mime_types_extensions: table[string] of pattern = {
>
>
> - const ignored_signatures += /^matchfile-/ &redef;
> + redef ignored_signatures += /^matchfile-/;
>
> Aashish
>
> On May 11, 2011, at 6:18 AM, Seth Hall wrote:
>
>> Sorry for not reply earlier. ?I started a response to your email and never finished it. :)
>>
>> On Apr 1, 2011, at 2:20 PM, Will wrote:
>>
>>> 1. The old way of flagging via 'HTTP_WatchedMIMEType' appears to have gone away
>>
>> Hm, I wonder why I removed that? ?There will be a solution for this problem in the next release.
>>
>> Did you end up figuring out what was wrong with this?
>>

Yes, pretty close to what Aashish describes to do above. Though I
don't see what changing the ignored_signatures file does, because it
already looks redef'd. Our "whitelist" is larger and slightly more
custom to our environment, but otherwise just as below. The
mis-matched file type is great for when a file is down loaded with a
random string and doesn't have a "watched" mime type, i.e. a php file
named "WJ4JR874".

Here is what we are using and seems to be working seemlessly:

@load global-ext
@load http-ext
@load http-reply
@load http-body
@load signatures
redef signature_files += "http-ext-identified-files.sig";

module HTTP;

export {
        redef enum Notice += {
                # This notice is thrown when the file extension doesn't
                # seem to match the file contents.
                HTTP_IncorrectFileType,

                # Generated when we see a MIME type we flagged for watching.
                HTTP_WatchedMIMEType,
        };

        # MIME types that you'd like this script to identify and log.
        const watched_mime_types = /application\/x-dosexec/
                                | /application\/x-executable/
                                | /application\/octet-stream/
                                | /application\/x-compressed/
                                | /application\/x-msdownload/ &redef;

        # URLs included here are not logged and notices are not thrown.
        # Take care when defining regexes to not be overly broad.
        const ignored_urls =
/^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/
                                | /^http:\/\/.*\.adobe\.com\//
                                | /^http:\/\/.*\.cisco\.com\//
                                | /^http:\/\/.*\.hp\.com\//
                                | /^http:\/\/.*\.macromedia\.com\//
                                | /^http:\/\/.*\.microsoft\.com\//
                                | /^http:\/\/.*\.sun\.com\// &redef;

        # Create regexes that *should* in be in the urls for specifics
mime types.
        # Notices are thrown if the pattern doesn't match the url for
the file type.
        const mime_types_extensions: table[string] of pattern = {
                ["application/x-dosexec"] = /\.([eE][xX][eE]|[dD][lL][lL])/,
        } &redef;
}

# Don't delete the http sessions at the end of the request!
redef watch_reply=T;

# Ignore the signatures used to match files
redef ignored_signatures += /^matchfile-/;

# This script uses the file tagging method to create a separate file.
event bro_init()
        {
        # Add the tag for log file splitting.
        LOG::define_tag("http-ext", "identified-files");
        }

event signature_match(state: signature_state, msg: string, data: string)
        {
        # Only signatures matching file types are dealt with here.
        if ( /^matchfile/ !in state$id ) return;

        # Not much point in any of this if we don't know about the
        # HTTP-ness of the connection.
        if ( state$conn$id !in conn_info ) return;

        local si = conn_info[state$conn$id];
        # Set the mime type seen.
        si$mime_type = msg;
        local defanged_url = gsub(si$url, /\./, "[.]");
        local message = fmt("%s %s", msg, defanged_url);
        if ( ignored_urls !in si$url )
                {
                if ( watched_mime_types in msg )
                        {
                        NOTICE([$note=HTTP_WatchedMIMEType,
$msg=message, $conn=state$conn, $method=si$method, $URL=si$url]);
                        # Add a tag for logging purposes.
                        add si$tags["identified-files"];
                        }

                if ( msg in mime_types_extensions &&
                     mime_types_extensions[msg] !in si$url )
                        {
                        NOTICE([$note=HTTP_IncorrectFileType,
$msg=message, $conn=state$conn, $method=si$method, $URL=si$url]);
                        }

                event file_transferred(state$conn, data, "", msg);
                }
        }

Thanks to both!

-Will

>> ?.Seth
>>
>> --
>> Seth Hall
>> International Computer Science Institute
>> (Bro) because everyone has a network
>> http://www.bro-ids.org/
>>
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>



From baxterw3232 at gmail.com  Thu May 12 10:34:39 2011
From: baxterw3232 at gmail.com (Will)
Date: Thu, 12 May 2011 13:34:39 -0400
Subject: [Bro]  6to4 Tunnelling
Message-ID: <BANLkTi=sqApkQEB+2aoMeEhC0jzEZzgjwg@mail.gmail.com>

On another note, I know there is alot of progress being made on bro
compatibility with IPv6. Are there any groups using bro to detect 6to4
tunnelling or "Teredo"?

So, if your network has some devices that are configured to run IPv6
through Toredo (or "need" to for some reason or another?!?), then
blocking 3544 isn't acceptable and isn't a great solution regardless.
I am wondering if it would be possible to inspect IPv4 UDP traffic for
wrapped IPv6 packets. Has anyone looked into this already or doing it?
If so, whitelisting known hosts that are allowed to send tunnelled
traffic would be trivial.

Thanks in advance.

-Will

Side note:
Is "tunnelling" spelled with one "L" or two? Or optional?

http://www.merriam-webster.com/dictionary/tunnelling


On Thu, May 12, 2011 at 1:19 PM, Will <baxterw3232 at gmail.com> wrote:
> On Wed, May 11, 2011 at 12:54 PM, Aashish SHARMA <aashish043 at gmail.com> wrote:
>> Hello:
>>
>> HTTP_WatchedMIMEType is declared in bro/share/bro/http-identified-files.bro.
>>
>> I think you can make the code work by doing the following changes in the http-ext-identified-files.bro
>>
>> 1) Load http-identified-files
>> 2) change "const" to "redef" for the following variables: watched_mime_types, ignored_urls, mime_types_extensions, ignored_signatures
>> 3) Comment out declaration of HTTP_IncorrectFileType from http-ext-identified-files.bro
>>
>>
>> + @load http-identified-files
>>
>> - ? ? ? redef enum Notice += {
>> - ? ? ? ? ? ? ? # This notice is thrown when the file extension doesn't
>> - ? ? ? ? ? ? ? # seem to match the file contents.
>> - ? ? ? ? ? ? ? HTTP_IncorrectFileType,
>> - ? ? ? };
>>
>> - ? ? ? const watched_mime_types = /application\/x-dosexec/
>> + ? ? ? redef watched_mime_types = /application\/x-dosexec/
>>
>>
>> - ? ? ? const ignored_urls = /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/ &redef;
>> + ? ? ? redef ignored_urls = /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/ ;
>>
>>
>> - ? ? ? redef mime_types_extensions: table[string] of pattern = {
>> + ? ? ? const mime_types_extensions: table[string] of pattern = {
>>
>>
>> - const ignored_signatures += /^matchfile-/ &redef;
>> + redef ignored_signatures += /^matchfile-/;
>>
>> Aashish
>>
>> On May 11, 2011, at 6:18 AM, Seth Hall wrote:
>>
>>> Sorry for not reply earlier. ?I started a response to your email and never finished it. :)
>>>
>>> On Apr 1, 2011, at 2:20 PM, Will wrote:
>>>
>>>> 1. The old way of flagging via 'HTTP_WatchedMIMEType' appears to have gone away
>>>
>>> Hm, I wonder why I removed that? ?There will be a solution for this problem in the next release.
>>>
>>> Did you end up figuring out what was wrong with this?
>>>
>
> Yes, pretty close to what Aashish describes to do above. Though I
> don't see what changing the ignored_signatures file does, because it
> already looks redef'd. Our "whitelist" is larger and slightly more
> custom to our environment, but otherwise just as below. The
> mis-matched file type is great for when a file is down loaded with a
> random string and doesn't have a "watched" mime type, i.e. a php file
> named "WJ4JR874".
>
> Here is what we are using and seems to be working seemlessly:
>
> @load global-ext
> @load http-ext
> @load http-reply
> @load http-body
> @load signatures
> redef signature_files += "http-ext-identified-files.sig";
>
> module HTTP;
>
> export {
> ? ? ? ?redef enum Notice += {
> ? ? ? ? ? ? ? ?# This notice is thrown when the file extension doesn't
> ? ? ? ? ? ? ? ?# seem to match the file contents.
> ? ? ? ? ? ? ? ?HTTP_IncorrectFileType,
>
> ? ? ? ? ? ? ? ?# Generated when we see a MIME type we flagged for watching.
> ? ? ? ? ? ? ? ?HTTP_WatchedMIMEType,
> ? ? ? ?};
>
> ? ? ? ?# MIME types that you'd like this script to identify and log.
> ? ? ? ?const watched_mime_types = /application\/x-dosexec/
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| /application\/x-executable/
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| /application\/octet-stream/
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| /application\/x-compressed/
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| /application\/x-msdownload/ &redef;
>
> ? ? ? ?# URLs included here are not logged and notices are not thrown.
> ? ? ? ?# Take care when defining regexes to not be overly broad.
> ? ? ? ?const ignored_urls =
> /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| /^http:\/\/.*\.adobe\.com\//
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| /^http:\/\/.*\.cisco\.com\//
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| /^http:\/\/.*\.hp\.com\//
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| /^http:\/\/.*\.macromedia\.com\//
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| /^http:\/\/.*\.microsoft\.com\//
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?| /^http:\/\/.*\.sun\.com\// &redef;
>
> ? ? ? ?# Create regexes that *should* in be in the urls for specifics
> mime types.
> ? ? ? ?# Notices are thrown if the pattern doesn't match the url for
> the file type.
> ? ? ? ?const mime_types_extensions: table[string] of pattern = {
> ? ? ? ? ? ? ? ?["application/x-dosexec"] = /\.([eE][xX][eE]|[dD][lL][lL])/,
> ? ? ? ?} &redef;
> }
>
> # Don't delete the http sessions at the end of the request!
> redef watch_reply=T;
>
> # Ignore the signatures used to match files
> redef ignored_signatures += /^matchfile-/;
>
> # This script uses the file tagging method to create a separate file.
> event bro_init()
> ? ? ? ?{
> ? ? ? ?# Add the tag for log file splitting.
> ? ? ? ?LOG::define_tag("http-ext", "identified-files");
> ? ? ? ?}
>
> event signature_match(state: signature_state, msg: string, data: string)
> ? ? ? ?{
> ? ? ? ?# Only signatures matching file types are dealt with here.
> ? ? ? ?if ( /^matchfile/ !in state$id ) return;
>
> ? ? ? ?# Not much point in any of this if we don't know about the
> ? ? ? ?# HTTP-ness of the connection.
> ? ? ? ?if ( state$conn$id !in conn_info ) return;
>
> ? ? ? ?local si = conn_info[state$conn$id];
> ? ? ? ?# Set the mime type seen.
> ? ? ? ?si$mime_type = msg;
> ? ? ? ?local defanged_url = gsub(si$url, /\./, "[.]");
> ? ? ? ?local message = fmt("%s %s", msg, defanged_url);
> ? ? ? ?if ( ignored_urls !in si$url )
> ? ? ? ? ? ? ? ?{
> ? ? ? ? ? ? ? ?if ( watched_mime_types in msg )
> ? ? ? ? ? ? ? ? ? ? ? ?{
> ? ? ? ? ? ? ? ? ? ? ? ?NOTICE([$note=HTTP_WatchedMIMEType,
> $msg=message, $conn=state$conn, $method=si$method, $URL=si$url]);
> ? ? ? ? ? ? ? ? ? ? ? ?# Add a tag for logging purposes.
> ? ? ? ? ? ? ? ? ? ? ? ?add si$tags["identified-files"];
> ? ? ? ? ? ? ? ? ? ? ? ?}
>
> ? ? ? ? ? ? ? ?if ( msg in mime_types_extensions &&
> ? ? ? ? ? ? ? ? ? ? mime_types_extensions[msg] !in si$url )
> ? ? ? ? ? ? ? ? ? ? ? ?{
> ? ? ? ? ? ? ? ? ? ? ? ?NOTICE([$note=HTTP_IncorrectFileType,
> $msg=message, $conn=state$conn, $method=si$method, $URL=si$url]);
> ? ? ? ? ? ? ? ? ? ? ? ?}
>
> ? ? ? ? ? ? ? ?event file_transferred(state$conn, data, "", msg);
> ? ? ? ? ? ? ? ?}
> ? ? ? ?}
>
> Thanks to both!
>
> -Will
>
>>> ?.Seth
>>>
>>> --
>>> Seth Hall
>>> International Computer Science Institute
>>> (Bro) because everyone has a network
>>> http://www.bro-ids.org/
>>>
>>>
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>>
>



From gregor at icir.org  Thu May 12 13:15:43 2011
From: gregor at icir.org (Gregor Maier)
Date: Thu, 12 May 2011 13:15:43 -0700
Subject: [Bro] 6to4 Tunnelling
In-Reply-To: <BANLkTi=sqApkQEB+2aoMeEhC0jzEZzgjwg@mail.gmail.com>
References: <BANLkTi=sqApkQEB+2aoMeEhC0jzEZzgjwg@mail.gmail.com>
Message-ID: <4DCC3FEF.804@icir.org>

I haven't done it myself but 6to4 and 6-in-4 tunnels are fairly easy to
detect, since they use a specific IP protocol number (41).

There's no immediate way for Toredo and other UDP encapsulated traffic
though (other than checking which hosts have traffic on port 3544 and
build whitelists/blacklists based on that). However, it should be fairly
easy to write an analyzer that parses UDP packets and checks if it is
Toredo or some other tunneling technique by checking
  a) whether there's an IPv6 header in the payload that makes sense
     (e.g., next header)
  b) the IPv6 address prefix in the header makes sense (e.g., is
     2001::/31 for Toldedo, etc.)
  c) (maybe) check that the IPv4 address is encoded in the IPv6 address
     according to the tunneling scheme.

Such an analyzer should be very lightweight and it doesn't need state.
It can either be run on only port 3544 traffic, all UDP traffic. Or, I
guess one could make DPD signatures for it by looking for the prefix
(e.g., 2001::/32, ip-version, an next-header value that makes sense)


Actually extracting and parsing the encapsulated v6 traffic is more
difficult, since the extracted packets would have be injected back into
Bro's processing above the analyzer trees.


cu
Gregor
-- 
Gregor Maier
<gregor at icir.org>  <gregor at icsi.berkeley.edu>
Int. Computer Science Institute (ICSI)
1947 Center St., Ste. 600
Berkeley, CA 94704, USA
http://www.icir.org/gregor/


From robin at icir.org  Thu May 12 14:16:52 2011
From: robin at icir.org (Robin Sommer)
Date: Thu, 12 May 2011 14:16:52 -0700
Subject: [Bro] 6to4 Tunnelling
In-Reply-To: <4DCC3FEF.804@icir.org>
References: <BANLkTi=sqApkQEB+2aoMeEhC0jzEZzgjwg@mail.gmail.com>
	<4DCC3FEF.804@icir.org>
Message-ID: <20110512211652.GB59697@icir.org>


On Thu, May 12, 2011 at 13:15 -0700, Gregor Maier wrote:

> Actually extracting and parsing the encapsulated v6 traffic is more
> difficult, since the extracted packets would have be injected back into
> Bro's processing above the analyzer trees.

Yeah, the right solution would be to extend the tree structure to
cover IP as well. We should do that at some point anyway.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From sanmeetkbhatia at gmail.com  Thu May 12 23:29:14 2011
From: sanmeetkbhatia at gmail.com (Sanmeet Bhatia)
Date: Fri, 13 May 2011 11:59:14 +0530
Subject: [Bro] regarding installation of Bro
Message-ID: <BANLkTimdEm3SHZhoZJTK1vddCoUcFeiU8g@mail.gmail.com>

Dear sir,

I am trying to install Bro 1.5.3 on Ubuntu 9.04. But I m unable to make that
after configure. The following is the error code. I have used bro 1.2.1
earlier. I have python2.4 and python2.4-dev installed. The configuration
summary is:

 Bro Configuration Summary
==========================================================

  - Debugging enabled:      no
  - OpenSSL support:        yes
  - Non-blocking main loop: yes
  - Non-blocking resolver:  yes
  - Installation prefix:    /usr/local/bro
  - Perl interpreter:       /usr/bin/perl
  - Using basic_string:     yes
  - Using libmagic:         yes
  - Using perftools:        no
  - Binpac used:            shipped with Bro
  - Using libGeoIP:         yes
  - Enabled broctl:         yes
  - Enabled cluster:        no
  - Pcap used:              system-provided
The make stops here:

broccoli_intern_wrap.c:112:20: error: Python.h: No such file or directory
broccoli_intern_wrap.c:717: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:772: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:793: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c: In function ?SWIG_Python_AddErrorMsg?:
broccoli_intern_wrap.c:840: error: ?PyObject? undeclared (first use in this
function)
broccoli_intern_wrap.c:840: error: (Each undeclared identifier is reported
only once
broccoli_intern_wrap.c:840: error: for each function it appears in.)
broccoli_intern_wrap.c:840: error: ?type? undeclared (first use in this
function)
broccoli_intern_wrap.c:841: error: ?value? undeclared (first use in this
function)
broccoli_intern_wrap.c:842: error: ?traceback? undeclared (first use in this
function)
broccoli_intern_wrap.c:844: warning: implicit declaration of function
?PyErr_Occurred?
broccoli_intern_wrap.c:844: warning: implicit declaration of function
?PyErr_Fetch?
broccoli_intern_wrap.c:846: error: ?old_str? undeclared (first use in this
function)
broccoli_intern_wrap.c:846: warning: implicit declaration of function
?PyObject_Str?
broccoli_intern_wrap.c:847: warning: implicit declaration of function
?PyErr_Clear?
broccoli_intern_wrap.c:848: warning: implicit declaration of function
?Py_XINCREF?
broccoli_intern_wrap.c:849: warning: implicit declaration of function
?PyErr_Format?
broccoli_intern_wrap.c:849: warning: implicit declaration of function
?PyString_AsString?
broccoli_intern_wrap.c:850: warning: implicit declaration of function
?Py_DECREF?
broccoli_intern_wrap.c:853: error: ?PyExc_RuntimeError? undeclared (first
use in this function)
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:1021: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:1029: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:1040: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:1047: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1093: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:1195: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1206: error: expected specifier-qualifier-list before
?PyObject?
broccoli_intern_wrap.c: In function ?SWIG_Python_CheckImplicit?:
broccoli_intern_wrap.c:1218: error: ?PySwigClientData? has no member named
?implicitconv?
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:1221: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1230: error: expected ?)? before ?*? token
broccoli_intern_wrap.c: In function ?PySwigClientData_Del?:
broccoli_intern_wrap.c:1285: warning: implicit declaration of function
?Py_XDECREF?
broccoli_intern_wrap.c:1285: error: ?PySwigClientData? has no member named
?newraw?
broccoli_intern_wrap.c:1286: error: ?PySwigClientData? has no member named
?newargs?
broccoli_intern_wrap.c:1287: error: ?PySwigClientData? has no member named
?destroy?
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:1293: error: expected specifier-qualifier-list before
?PyObject_HEAD?
broccoli_intern_wrap.c:1300: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1306: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1324: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1330: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1336: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1359: error: expected declaration specifiers or ?...?
before ?FILE?
broccoli_intern_wrap.c: In function ?PySwigObject_print?:
broccoli_intern_wrap.c:1364: error: ?PyObject? undeclared (first use in this
function)
broccoli_intern_wrap.c:1364: error: ?repr? undeclared (first use in this
function)
broccoli_intern_wrap.c:1364: warning: implicit declaration of function
?PySwigObject_repr?
broccoli_intern_wrap.c:1367: warning: implicit declaration of function
?fputs?
broccoli_intern_wrap.c:1367: error: ?fp? undeclared (first use in this
function)
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:1375: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c: In function ?PySwigObject_compare?:
broccoli_intern_wrap.c:1386: error: ?PySwigObject? has no member named ?ptr?
broccoli_intern_wrap.c:1387: error: ?PySwigObject? has no member named ?ptr?
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:1391: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1393: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1400: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:1405: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1409: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:1442: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1459: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1475: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1487: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1499: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1547: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?swigobject_methods?
broccoli_intern_wrap.c:1559: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1566: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1672: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1690: error: expected specifier-qualifier-list before
?PyObject_HEAD?
broccoli_intern_wrap.c:1697: error: expected declaration specifiers or ?...?
before ?FILE?
broccoli_intern_wrap.c: In function ?PySwigPacked_print?:
broccoli_intern_wrap.c:1700: error: ?fp? undeclared (first use in this
function)
broccoli_intern_wrap.c:1701: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1701: error: ?PySwigPacked? has no member named
?size?
broccoli_intern_wrap.c:1705: error: ?PySwigPacked? has no member named ?ty?
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:1710: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1721: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c: In function ?PySwigPacked_compare?:
broccoli_intern_wrap.c:1735: error: ?PySwigPacked? has no member named
?size?
broccoli_intern_wrap.c:1736: error: ?PySwigPacked? has no member named
?size?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?size?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?size?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?size?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?pack?
broccoli_intern_wrap.c:1738: error: ?PySwigPacked? has no member named
?size?
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:1741: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1743: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1750: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:1756: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:1765: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1835: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1855: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:1871: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1877: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:1887: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:1938: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:1953: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:2033: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:2060: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:2082: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:2146: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:2167: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:2185: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:2206: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c: In function ?SWIG_Python_GetModule?:
broccoli_intern_wrap.c:2227: warning: implicit declaration of function
?PyCObject_Import?
broccoli_intern_wrap.c:2228: warning: assignment makes pointer from integer
without a cast
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:2242: error: expected ?)? before ?*? token
broccoli_intern_wrap.c: In function ?SWIG_Python_DestroyModule?:
broccoli_intern_wrap.c:2283: warning: implicit declaration of function
?SWIG_This?
broccoli_intern_wrap.c: In function ?SWIG_Python_SetModule?:
broccoli_intern_wrap.c:2288: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?swig_empty_runtime_method_table?
broccoli_intern_wrap.c:2288: error: ?swig_empty_runtime_method_table?
undeclared (first use in this function)
broccoli_intern_wrap.c:2288: error: expected expression before ?]? token
broccoli_intern_wrap.c:2290: error: ?PyObject? undeclared (first use in this
function)
broccoli_intern_wrap.c:2290: error: ?module? undeclared (first use in this
function)
broccoli_intern_wrap.c:2290: warning: implicit declaration of function
?Py_InitModule?
broccoli_intern_wrap.c:2292: error: ?pointer? undeclared (first use in this
function)
broccoli_intern_wrap.c:2292: warning: implicit declaration of function
?PyCObject_FromVoidPtr?
broccoli_intern_wrap.c:2294: warning: implicit declaration of function
?PyModule_AddObject?
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:2301: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c: In function ?SWIG_Python_TypeQuery?:
broccoli_intern_wrap.c:2310: error: ?PyObject? undeclared (first use in this
function)
broccoli_intern_wrap.c:2310: error: ?cache? undeclared (first use in this
function)
broccoli_intern_wrap.c:2310: warning: implicit declaration of function
?SWIG_Python_TypeCache?
broccoli_intern_wrap.c:2311: error: ?key? undeclared (first use in this
function)
broccoli_intern_wrap.c:2311: warning: implicit declaration of function
?PyString_FromString?
broccoli_intern_wrap.c:2312: error: ?obj? undeclared (first use in this
function)
broccoli_intern_wrap.c:2312: warning: implicit declaration of function
?PyDict_GetItem?
broccoli_intern_wrap.c:2315: warning: implicit declaration of function
?PyCObject_AsVoidPtr?
broccoli_intern_wrap.c:2321: warning: implicit declaration of function
?PyDict_SetItem?
broccoli_intern_wrap.c: In function ?SWIG_Python_AddErrMesg?:
broccoli_intern_wrap.c:2340: error: ?PyObject? undeclared (first use in this
function)
broccoli_intern_wrap.c:2340: error: ?type? undeclared (first use in this
function)
broccoli_intern_wrap.c:2341: error: ?value? undeclared (first use in this
function)
broccoli_intern_wrap.c:2342: error: ?traceback? undeclared (first use in
this function)
broccoli_intern_wrap.c:2345: error: ?old_str? undeclared (first use in this
function)
broccoli_intern_wrap.c: In function ?SWIG_Python_ArgFail?:
broccoli_intern_wrap.c:2367: warning: implicit declaration of function
?snprintf?
broccoli_intern_wrap.c:2367: warning: incompatible implicit declaration of
built-in function ?snprintf?
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:2375: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:2383: error: expected declaration specifiers or ?...?
before ?PyObject?
broccoli_intern_wrap.c: In function ?SWIG_Python_TypeError?:
broccoli_intern_wrap.c:2397: error: ?obj? undeclared (first use in this
function)
broccoli_intern_wrap.c:2399: error: ?PyObject? undeclared (first use in this
function)
broccoli_intern_wrap.c:2399: error: ?str? undeclared (first use in this
function)
broccoli_intern_wrap.c:2402: error: ?PyExc_TypeError? undeclared (first use
in this function)
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:2421: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:2463:4: error: #error "This python version requires
swig to be run with the '-classic' option"
broccoli_intern_wrap.c:2495: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:2505: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:2554: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:2632: error: expected ?)? before ?*? token
broccoli_intern_wrap.c: In function ?event_callback?:
broccoli_intern_wrap.c:2750: error: ?PyObject? undeclared (first use in this
function)
broccoli_intern_wrap.c:2750: error: ?func? undeclared (first use in this
function)
broccoli_intern_wrap.c:2750: error: expected expression before ?)? token
broccoli_intern_wrap.c:2753: error: ?pyargs? undeclared (first use in this
function)
broccoli_intern_wrap.c:2753: warning: implicit declaration of function
?PyTuple_New?
broccoli_intern_wrap.c:2755: warning: implicit declaration of function
?PyTuple_SetItem?
broccoli_intern_wrap.c:2755: warning: implicit declaration of function
?valToPyObj?
broccoli_intern_wrap.c:2757: error: ?result? undeclared (first use in this
function)
broccoli_intern_wrap.c:2757: warning: implicit declaration of function
?PyObject_Call?
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:2770: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:2791: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:2858: error: expected ?)? before ?*? token
broccoli_intern_wrap.c: In function ?SWIG_CanCastAsInteger?:
broccoli_intern_wrap.c:2914: error: ?errno? undeclared (first use in this
function)
broccoli_intern_wrap.c:2914: error: ?EDOM? undeclared (first use in this
function)
broccoli_intern_wrap.c:2914: error: ?ERANGE? undeclared (first use in this
function)
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:2938: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:2977: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:2997: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3017: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3051: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3082: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3102: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3122: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3142: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3167: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3186: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3245: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3272: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3316: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3329: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?SwigMethods?
broccoli_intern_wrap.c:3622: error: expected specifier-qualifier-list before
?PyObject?
broccoli_intern_wrap.c:3628: error: expected specifier-qualifier-list before
?PyObject_HEAD?
broccoli_intern_wrap.c:3632: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3637: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c: In function ?swig_varlink_print?:
broccoli_intern_wrap.c:3651: error: ?PyObject? undeclared (first use in this
function)
broccoli_intern_wrap.c:3651: error: ?str? undeclared (first use in this
function)
broccoli_intern_wrap.c:3651: warning: implicit declaration of function
?swig_varlink_str?
broccoli_intern_wrap.c:3653: warning: format ?%s? expects type ?char *?, but
argument 3 has type ?int?
broccoli_intern_wrap.c: In function ?swig_varlink_dealloc?:
broccoli_intern_wrap.c:3660: error: ?swig_varlinkobject? has no member named
?vars?
broccoli_intern_wrap.c:3662: error: ?swig_globalvar? has no member named
?next?
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:3669: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3687: error: expected declaration specifiers or ?...?
before ?PyObject?
broccoli_intern_wrap.c: In function ?swig_varlink_setattr?:
broccoli_intern_wrap.c:3689: error: ?swig_varlinkobject? has no member named
?vars?
broccoli_intern_wrap.c:3692: error: ?swig_globalvar? has no member named
?set_attr?
broccoli_intern_wrap.c:3692: error: ?p? undeclared (first use in this
function)
broccoli_intern_wrap.c:3695: error: ?swig_globalvar? has no member named
?next?
broccoli_intern_wrap.c:3698: warning: implicit declaration of function
?PyErr_SetString?
broccoli_intern_wrap.c:3698: error: ?PyExc_NameError? undeclared (first use
in this function)
broccoli_intern_wrap.c: At top level:
broccoli_intern_wrap.c:3703: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3755: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3765: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:3781: error: expected ?=?, ?,?, ?;?, ?asm? or
?__attribute__? before ?*? token
broccoli_intern_wrap.c:3794: error: expected ?)? before ?*? token
broccoli_intern_wrap.c:3821: error: expected ?)? before ?*? token
broccoli_intern_wrap.c: In function ?init_broccoli_intern?:
broccoli_intern_wrap.c:3874: error: ?PyObject? undeclared (first use in this
function)
broccoli_intern_wrap.c:3874: error: ?m? undeclared (first use in this
function)
broccoli_intern_wrap.c:3874: error: ?d? undeclared (first use in this
function)
broccoli_intern_wrap.c:3874: warning: left-hand operand of comma expression
has no effect
broccoli_intern_wrap.c:3877: warning: implicit declaration of function
?SWIG_Python_FixMethods?
broccoli_intern_wrap.c:3877: error: ?SwigMethods? undeclared (first use in
this function)
broccoli_intern_wrap.c:3880: warning: implicit declaration of function
?PyModule_GetDict?
broccoli_intern_wrap.c:3883: warning: implicit declaration of function
?SWIG_Python_InstallConstants?
broccoli_intern_wrap.c:3886: warning: implicit declaration of function
?SWIG_Python_SetConstant?
broccoli_intern_wrap.c:3886: warning: implicit declaration of function
?SWIG_From_int?
error: command 'gcc' failed with exit status 1
make[4]: *** [pybroccoli] Error 1
make[4]: Leaving directory `/home/sanmeet/bro-1.5.3/aux/broctl'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory `/home/sanmeet/bro-1.5.3/aux'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/sanmeet/bro-1.5.3/aux'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/sanmeet/bro-1.5.3'
make: *** [all] Error 2



Kindly help.
Regards,
Sanmeet Bhatia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110513/5cb73e98/attachment.html 

From jsiwek at ncsa.illinois.edu  Fri May 13 07:36:46 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Fri, 13 May 2011 09:36:46 -0500 (CDT)
Subject: [Bro] regarding installation of Bro
In-Reply-To: <BANLkTimdEm3SHZhoZJTK1vddCoUcFeiU8g@mail.gmail.com>
Message-ID: <460507683.1250.1305297406663.JavaMail.root@zimbra-1.ncsa.uiuc.edu>

> I am trying to install Bro 1.5.3 on Ubuntu 9.04. But I m unable to
> make that after configure. The following is the error code. I have
> used bro 1.2.1 earlier. I have python2.4 and python2.4-dev installed.
> ...
> broccoli_intern_wrap.c:112:20: error: Python.h: No such file or directory

This failure is happening when building a sub-package of broccoli, which is a sub-package of bro.

If you do need to build the broccoli sub-package with python bindings, then you need to figure out whether python2.4-dev is actually providing Python.h for you or whether you might need to use a different Python version (maybe self-built/installed).

If you don't need the broccoli sub-package, you could try just disabling it (e.g. `./configure --disable-broccoli`).  There's also another sub-package, broctl, that depends on Python that you can disable if you're not going to use it (use the "--disable-broctl" option to ./configure).

That said, I don't think any Ubuntu 9.x versions are even supported by Canonical anymore, so if you run into more problems and you have the option of just upgrading the OS, that might be a good thing.

- Jon


From baxterw3232 at gmail.com  Fri May 13 08:26:25 2011
From: baxterw3232 at gmail.com (Will)
Date: Fri, 13 May 2011 11:26:25 -0400
Subject: [Bro] 6to4 Tunnelling
In-Reply-To: <4DCC3FEF.804@icir.org>
References: <BANLkTi=sqApkQEB+2aoMeEhC0jzEZzgjwg@mail.gmail.com>
	<4DCC3FEF.804@icir.org>
Message-ID: <BANLkTinGa4qr8PjFnzsqhQw0j=7WEbPmeQ@mail.gmail.com>

On Thu, May 12, 2011 at 4:15 PM, Gregor Maier <gregor at icir.org> wrote:
> I haven't done it myself but 6to4 and 6-in-4 tunnels are fairly easy to
> detect, since they use a specific IP protocol number (41).
>
> There's no immediate way for Toredo and other UDP encapsulated traffic
> though (other than checking which hosts have traffic on port 3544 and
> build whitelists/blacklists based on that). However, it should be fairly
> easy to write an analyzer that parses UDP packets and checks if it is
> Toredo or some other tunneling technique by checking
> ?a) whether there's an IPv6 header in the payload that makes sense
> ? ? (e.g., next header)
> ?b) the IPv6 address prefix in the header makes sense (e.g., is
> ? ? 2001::/31 for Toldedo, etc.)
> ?c) (maybe) check that the IPv4 address is encoded in the IPv6 address
> ? ? according to the tunneling scheme.
>
> Such an analyzer should be very lightweight and it doesn't need state.
> It can either be run on only port 3544 traffic, all UDP traffic. Or, I
> guess one could make DPD signatures for it by looking for the prefix
> (e.g., 2001::/32, ip-version, an next-header value that makes sense)
>
Yes, I think this sounds like the way to do it. I appreciate the feedback.
>
> Actually extracting and parsing the encapsulated v6 traffic is more
> difficult, since the extracted packets would have be injected back into
> Bro's processing above the analyzer trees.
>
Extracting the encapsulated traffic would be great and probably should
be the end goal, but for now, I would settle with knowing which hosts
are using the protocol then ensuring they should be based on policy.

Thanks again for the info!

Will

>
> cu
> Gregor
> --
> Gregor Maier
> <gregor at icir.org> ?<gregor at icsi.berkeley.edu>
> Int. Computer Science Institute (ICSI)
> 1947 Center St., Ste. 600
> Berkeley, CA 94704, USA
> http://www.icir.org/gregor/
>



From lruppert at syr.edu  Fri May 13 10:33:40 2011
From: lruppert at syr.edu (Louis F Ruppert)
Date: Fri, 13 May 2011 13:33:40 -0400
Subject: [Bro] Debian pkg and packet loss metrics
Message-ID: <32E412533D42EF41ACBCFA96B6C230F4018617B69AFB@SUEX07-MBX-07.ad.syr.edu>

Hey,

Two questions, too lazy to write two messages:

1. There was some talk about bro 1.6 having the means to easily generate
Debian packages with cmake.  I see scripts for generating rpm packages
and scripts for generating mac packages, but nothing for Debian.  Does
anyone have pointers on how to do this, or should I plan on generating
something in house?

2. How are people non-intrusively measuring packet loss in their
clusters?  I can get a vague idea of what bro is losing via netstats,
but I'd hate to interfere with cluster operations by running it every
few minutes.  We also split our taps with Click router, so that still
wouldn't tell me what, if anything, Click or the kernel itself are
losing.  (For the record, the kernel and Click both claim to be losing
close to nothing, while bro loses ~3-4%, based on a few tests I've
done.)


Thanks,


-Lou




From jsiwek at ncsa.illinois.edu  Fri May 13 12:25:56 2011
From: jsiwek at ncsa.illinois.edu (Jonathan Siwek)
Date: Fri, 13 May 2011 14:25:56 -0500 (CDT)
Subject: [Bro] Debian pkg
In-Reply-To: <32E412533D42EF41ACBCFA96B6C230F4018617B69AFB@SUEX07-MBX-07.ad.syr.edu>
Message-ID: <837cb3ef-36d3-4696-b75d-9e4cf4dc9239@tangent.ncsa.illinois.edu>

> 1. There was some talk about bro 1.6 having the means to easily
> generate Debian packages with cmake.  I see scripts for generating rpm
> packages and scripts for generating mac packages, but nothing for Debian.
> Does anyone have pointers on how to do this, or should I plan on
> generating something in house?

DEB packages as generated by CMake/CPack weren't initially targeted because they lacked the ability to automatically determine the package's dependency meta-info, which was a big turn-off.  However, it looks as though this capability works in more recent CMake/CPack versions, so I'm going to look into it again:

http://tracker.bro-ids.org/bro/ticket/458

If it works out, these will probably be of a "working" quality, but not of a quality that would be accepted into a public package repository.

- Jon


From robin at icir.org  Fri May 13 16:19:48 2011
From: robin at icir.org (Robin Sommer)
Date: Fri, 13 May 2011 16:19:48 -0700
Subject: [Bro] Debian pkg and packet loss metrics
In-Reply-To: <32E412533D42EF41ACBCFA96B6C230F4018617B69AFB@SUEX07-MBX-07.ad.syr.edu>
References: <32E412533D42EF41ACBCFA96B6C230F4018617B69AFB@SUEX07-MBX-07.ad.syr.edu>
Message-ID: <20110513231948.GC8623@icir.org>


On Fri, May 13, 2011 at 13:33 -0400, you wrote:

> 2. How are people non-intrusively measuring packet loss in their
> clusters?

Take a look at policy/capture-loss.bro. That count ACKs for sequence
numbers above a gap, which gives quite a good indicator of loss.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From robin at icir.org  Fri May 13 18:53:51 2011
From: robin at icir.org (Robin Sommer)
Date: Fri, 13 May 2011 18:53:51 -0700
Subject: [Bro] ICMP Redirect patch
In-Reply-To: <17164168.69581.1304346197310.JavaMail.www@wwinf8215>
References: <17164168.69581.1304346197310.JavaMail.www@wwinf8215>
Message-ID: <20110514015351.GE12671@icir.org>


On Mon, May 02, 2011 at 16:23 +0200, you wrote:

> Here is a small and quick patch I made to Bro to get icmp_redirect events with the redirection address.

Thanks, Julien. I've applied this now.

(Btw, the prefered way to submit patches is via the tracker.)

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org


From sxz135 at case.edu  Sun May 15 21:05:43 2011
From: sxz135 at case.edu (Song Zhao)
Date: Mon, 16 May 2011 00:05:43 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline trace
Message-ID: <BANLkTi=_4LO8e8fSCgJHt0NON8mWW2jWdg@mail.gmail.com>

Hello, All

I am trying to use the policy script http-rewriter.bro in Bro-1.5.1 to
anonymize the HTTP message-body of all HTTP packets in a big dumped trace
larger than 100GB ( http-rewriter.bro actually deletes all HTTP message-body
and add one new header field named X-Actual-Data-Length, right?) .

I am not sure if  Bro itself and http-rewriter.bro  has the ability of
reordering all  tcp packets and deleting tcp retransmitted packets in every
connection of the dumped trace?

If they cannot do that, whether I can reorder all packets and delete the
retransmitted packets in every connection first by using some tools and then
use http-rewriter.bro ?  Is this way reasonable? What's your suggestion
about the tools I can use?

Besides, I want to test if special HTTP packets exist. Special packet here
means there are more than one HTTP construct(headers + message body) in one
packet. When using http-rewriter.bro on several special pakcets I created,
it seems that it can delete the message-body correctly for almost all of
cases as long as the packets in the connection are in order and complete.
Can http-rewriter.bro handle the special cases correctly as what I found?

Expect your answer and thank you very much.

Song Zhao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110516/1b62da4b/attachment.html 

From jones at tacc.utexas.edu  Mon May 16 11:25:02 2011
From: jones at tacc.utexas.edu (William Jones)
Date: Mon, 16 May 2011 13:25:02 -0500
Subject: [Bro] Large flows
In-Reply-To: <821f2ea41002061136j440870daod4d3eeb73583ab3a@mail.gmail.com>
References: <821f2ea41002060721g6984e948v9cba13ed22d1b51b@mail.gmail.com>
	<9D3E489884294646B1627EFEACA86436025F05DD@exchange.corp.epnet.com>
	<821f2ea41002061012w652595a9o215b1c451b49bbf1@mail.gmail.com>
	<20100206185103.GA2827@datacomm.albany.edu>
	<821f2ea41002061104i656f937bk36435abc9d9a111f@mail.gmail.com>
	<20100206191851.GB2827@datacomm.albany.edu>
	<821f2ea41002061136j440870daod4d3eeb73583ab3a@mail.gmail.com>
Message-ID: <CF3466DD76A94945B5723C073FE74B0A6FA09A463D@MAIL02.austin.utexas.edu>


I have a few machines that can produce large flows in access 100Mbytes over NLR.  This cause bro to drop packets.  Right now I am using pcap filters and extra workers on the sensor machine to insures that only packets dropped are from the file transfers machines.   This is not that big down side but it make we wonder if just what is the biggest flow that bro can handle on a single cpu.   










From jones at tacc.utexas.edu  Mon May 16 11:25:49 2011
From: jones at tacc.utexas.edu (William Jones)
Date: Mon, 16 May 2011 13:25:49 -0500
Subject: [Bro] Large flows
In-Reply-To: <CF3466DD76A94945B5723C073FE74B0A6FA09A463D@MAIL02.austin.utexas.edu>
References: <821f2ea41002060721g6984e948v9cba13ed22d1b51b@mail.gmail.com>
	<9D3E489884294646B1627EFEACA86436025F05DD@exchange.corp.epnet.com>
	<821f2ea41002061012w652595a9o215b1c451b49bbf1@mail.gmail.com>
	<20100206185103.GA2827@datacomm.albany.edu>
	<821f2ea41002061104i656f937bk36435abc9d9a111f@mail.gmail.com>
	<20100206191851.GB2827@datacomm.albany.edu>
	<821f2ea41002061136j440870daod4d3eeb73583ab3a@mail.gmail.com>
	<CF3466DD76A94945B5723C073FE74B0A6FA09A463D@MAIL02.austin.utexas.edu>
Message-ID: <CF3466DD76A94945B5723C073FE74B0A6FA09A463E@MAIL02.austin.utexas.edu>

Woops!

-----Original Message-----
From: bro-bounces at bro-ids.org [mailto:bro-bounces at bro-ids.org] On Behalf Of William Jones
Sent: Monday, May 16, 2011 1:25 PM
To: bro at ICSI.Berkeley.EDU
Subject: [Bro] Large flows


I have a few machines that can produce large flows in access 100Mbytes over NLR.  This cause bro to drop packets.  Right now I am using pcap filters and extra workers on the sensor machine to insures that only packets dropped are from the file transfers machines.   This is not that big down side but it make we wonder if just what is the biggest flow that bro can handle on a single cpu.   








_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From rpang at cs.princeton.edu  Mon May 16 13:24:54 2011
From: rpang at cs.princeton.edu (Ruoming Pang)
Date: Mon, 16 May 2011 16:24:54 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTi=_4LO8e8fSCgJHt0NON8mWW2jWdg@mail.gmail.com>
References: <BANLkTi=_4LO8e8fSCgJHt0NON8mWW2jWdg@mail.gmail.com>
Message-ID: <BANLkTim-aGLbLywD+x0iSFf1yB73H5f9bQ@mail.gmail.com>

Hi,

I forgot about the details, but here is the basic idea. The rewritten
packets will not reproduce the original TCP segment ordering and
retransmission, however, the timestamps will be preserved by creating one
output packet that correspond to every input packet timestamp. So if you
remove a big chunk of body, you will see a bunch of empty packets (which
compress quite well).

Ruoming

On Mon, May 16, 2011 at 12:05 AM, Song Zhao <sxz135 at case.edu> wrote:

> Hello, All
>
> I am trying to use the policy script http-rewriter.bro in Bro-1.5.1 to
> anonymize the HTTP message-body of all HTTP packets in a big dumped trace
> larger than 100GB ( http-rewriter.bro actually deletes all HTTP message-body
> and add one new header field named X-Actual-Data-Length, right?) .
>
> I am not sure if  Bro itself and http-rewriter.bro  has the ability of
> reordering all  tcp packets and deleting tcp retransmitted packets in every
> connection of the dumped trace?
>
> If they cannot do that, whether I can reorder all packets and delete the
> retransmitted packets in every connection first by using some tools and then
> use http-rewriter.bro ?  Is this way reasonable? What's your suggestion
> about the tools I can use?
>
> Besides, I want to test if special HTTP packets exist. Special packet here
> means there are more than one HTTP construct(headers + message body) in one
> packet. When using http-rewriter.bro on several special pakcets I created,
> it seems that it can delete the message-body correctly for almost all of
> cases as long as the packets in the connection are in order and complete.
> Can http-rewriter.bro handle the special cases correctly as what I found?
>
> Expect your answer and thank you very much.
>
> Song Zhao
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110516/3de493ce/attachment.html 

From sxz135 at case.edu  Mon May 16 13:51:28 2011
From: sxz135 at case.edu (Song Zhao)
Date: Mon, 16 May 2011 16:51:28 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTim-aGLbLywD+x0iSFf1yB73H5f9bQ@mail.gmail.com>
References: <BANLkTi=_4LO8e8fSCgJHt0NON8mWW2jWdg@mail.gmail.com>
	<BANLkTim-aGLbLywD+x0iSFf1yB73H5f9bQ@mail.gmail.com>
Message-ID: <BANLkTikeFuBs8jcM7ieu0YC4mbLdtU-f8A@mail.gmail.com>

Hi,

 Thanks for your explanation.But I am still a little confused. Can
http-rewriter.bro rewriter all HTTP packets in a TCP connection where out of
order and retransmitted packets exist?

Song Zhao

On Mon, May 16, 2011 at 4:24 PM, Ruoming Pang <rpang at cs.princeton.edu>wrote:

> Hi,
>
> I forgot about the details, but here is the basic idea. The rewritten
> packets will not reproduce the original TCP segment ordering and
> retransmission, however, the timestamps will be preserved by creating one
> output packet that correspond to every input packet timestamp. So if you
> remove a big chunk of body, you will see a bunch of empty packets (which
> compress quite well).
>
> Ruoming
>
> On Mon, May 16, 2011 at 12:05 AM, Song Zhao <sxz135 at case.edu> wrote:
>
>> Hello, All
>>
>> I am trying to use the policy script http-rewriter.bro in Bro-1.5.1 to
>> anonymize the HTTP message-body of all HTTP packets in a big dumped trace
>> larger than 100GB ( http-rewriter.bro actually deletes all HTTP message-body
>> and add one new header field named X-Actual-Data-Length, right?) .
>>
>> I am not sure if  Bro itself and http-rewriter.bro  has the ability of
>> reordering all  tcp packets and deleting tcp retransmitted packets in every
>> connection of the dumped trace?
>>
>> If they cannot do that, whether I can reorder all packets and delete the
>> retransmitted packets in every connection first by using some tools and then
>> use http-rewriter.bro ?  Is this way reasonable? What's your suggestion
>> about the tools I can use?
>>
>> Besides, I want to test if special HTTP packets exist. Special packet here
>> means there are more than one HTTP construct(headers + message body) in one
>> packet. When using http-rewriter.bro on several special pakcets I created,
>> it seems that it can delete the message-body correctly for almost all of
>> cases as long as the packets in the connection are in order and complete.
>> Can http-rewriter.bro handle the special cases correctly as what I found?
>>
>> Expect your answer and thank you very much.
>>
>> Song Zhao
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110516/43db9ced/attachment.html 

From rpang at cs.princeton.edu  Mon May 16 13:54:34 2011
From: rpang at cs.princeton.edu (Ruoming Pang)
Date: Mon, 16 May 2011 16:54:34 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTikeFuBs8jcM7ieu0YC4mbLdtU-f8A@mail.gmail.com>
References: <BANLkTi=_4LO8e8fSCgJHt0NON8mWW2jWdg@mail.gmail.com>
	<BANLkTim-aGLbLywD+x0iSFf1yB73H5f9bQ@mail.gmail.com>
	<BANLkTikeFuBs8jcM7ieu0YC4mbLdtU-f8A@mail.gmail.com>
Message-ID: <BANLkTim0rbxrOZ68ddAqe6h3WQV4gSNoUQ@mail.gmail.com>

http-rewriter.bro sits above TCP layer and does not see TCP retransmission
or out of order packets.

On Mon, May 16, 2011 at 4:51 PM, Song Zhao <sxz135 at case.edu> wrote:

> Hi,
>
>  Thanks for your explanation.But I am still a little confused. Can
> http-rewriter.bro rewriter all HTTP packets in a TCP connection where out of
> order and retransmitted packets exist?
>
> Song Zhao
>
>
> On Mon, May 16, 2011 at 4:24 PM, Ruoming Pang <rpang at cs.princeton.edu>wrote:
>
>> Hi,
>>
>> I forgot about the details, but here is the basic idea. The rewritten
>> packets will not reproduce the original TCP segment ordering and
>> retransmission, however, the timestamps will be preserved by creating one
>> output packet that correspond to every input packet timestamp. So if you
>> remove a big chunk of body, you will see a bunch of empty packets (which
>> compress quite well).
>>
>> Ruoming
>>
>> On Mon, May 16, 2011 at 12:05 AM, Song Zhao <sxz135 at case.edu> wrote:
>>
>>> Hello, All
>>>
>>> I am trying to use the policy script http-rewriter.bro in Bro-1.5.1 to
>>> anonymize the HTTP message-body of all HTTP packets in a big dumped trace
>>> larger than 100GB ( http-rewriter.bro actually deletes all HTTP message-body
>>> and add one new header field named X-Actual-Data-Length, right?) .
>>>
>>> I am not sure if  Bro itself and http-rewriter.bro  has the ability of
>>> reordering all  tcp packets and deleting tcp retransmitted packets in every
>>> connection of the dumped trace?
>>>
>>> If they cannot do that, whether I can reorder all packets and delete the
>>> retransmitted packets in every connection first by using some tools and then
>>> use http-rewriter.bro ?  Is this way reasonable? What's your suggestion
>>> about the tools I can use?
>>>
>>> Besides, I want to test if special HTTP packets exist. Special packet
>>> here means there are more than one HTTP construct(headers + message body) in
>>> one packet. When using http-rewriter.bro on several special pakcets I
>>> created, it seems that it can delete the message-body correctly for almost
>>> all of cases as long as the packets in the connection are in order and
>>> complete. Can http-rewriter.bro handle the special cases correctly as what I
>>> found?
>>>
>>> Expect your answer and thank you very much.
>>>
>>> Song Zhao
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110516/2ca025d8/attachment.html 

From sxz135 at case.edu  Mon May 16 14:04:29 2011
From: sxz135 at case.edu (Song Zhao)
Date: Mon, 16 May 2011 17:04:29 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTim0rbxrOZ68ddAqe6h3WQV4gSNoUQ@mail.gmail.com>
References: <BANLkTi=_4LO8e8fSCgJHt0NON8mWW2jWdg@mail.gmail.com>
	<BANLkTim-aGLbLywD+x0iSFf1yB73H5f9bQ@mail.gmail.com>
	<BANLkTikeFuBs8jcM7ieu0YC4mbLdtU-f8A@mail.gmail.com>
	<BANLkTim0rbxrOZ68ddAqe6h3WQV4gSNoUQ@mail.gmail.com>
Message-ID: <BANLkTim5YVsepwfeP=Ti9jU-ChB3LJ1_MA@mail.gmail.com>

Can Bro itself differentiate these retransmitted and out of order packets?
If yes, does http-rewriter.bro use such a Bro's method?

Besides, can http-rewriter.bro handle the special HTTP packet which, for
example, includes 2 or more requests or response or even one and half
requests or responses?

On Mon, May 16, 2011 at 4:54 PM, Ruoming Pang <rpang at cs.princeton.edu>wrote:

> http-rewriter.bro sits above TCP layer and does not see TCP retransmission
> or out of order packets.
>
>
> On Mon, May 16, 2011 at 4:51 PM, Song Zhao <sxz135 at case.edu> wrote:
>
>> Hi,
>>
>>  Thanks for your explanation.But I am still a little confused. Can
>> http-rewriter.bro rewriter all HTTP packets in a TCP connection where out of
>> order and retransmitted packets exist?
>>
>> Song Zhao
>>
>>
>> On Mon, May 16, 2011 at 4:24 PM, Ruoming Pang <rpang at cs.princeton.edu>wrote:
>>
>>> Hi,
>>>
>>> I forgot about the details, but here is the basic idea. The rewritten
>>> packets will not reproduce the original TCP segment ordering and
>>> retransmission, however, the timestamps will be preserved by creating one
>>> output packet that correspond to every input packet timestamp. So if you
>>> remove a big chunk of body, you will see a bunch of empty packets (which
>>> compress quite well).
>>>
>>> Ruoming
>>>
>>> On Mon, May 16, 2011 at 12:05 AM, Song Zhao <sxz135 at case.edu> wrote:
>>>
>>>> Hello, All
>>>>
>>>> I am trying to use the policy script http-rewriter.bro in Bro-1.5.1 to
>>>> anonymize the HTTP message-body of all HTTP packets in a big dumped trace
>>>> larger than 100GB ( http-rewriter.bro actually deletes all HTTP message-body
>>>> and add one new header field named X-Actual-Data-Length, right?) .
>>>>
>>>> I am not sure if  Bro itself and http-rewriter.bro  has the ability of
>>>> reordering all  tcp packets and deleting tcp retransmitted packets in every
>>>> connection of the dumped trace?
>>>>
>>>> If they cannot do that, whether I can reorder all packets and delete the
>>>> retransmitted packets in every connection first by using some tools and then
>>>> use http-rewriter.bro ?  Is this way reasonable? What's your suggestion
>>>> about the tools I can use?
>>>>
>>>> Besides, I want to test if special HTTP packets exist. Special packet
>>>> here means there are more than one HTTP construct(headers + message body) in
>>>> one packet. When using http-rewriter.bro on several special pakcets I
>>>> created, it seems that it can delete the message-body correctly for almost
>>>> all of cases as long as the packets in the connection are in order and
>>>> complete. Can http-rewriter.bro handle the special cases correctly as what I
>>>> found?
>>>>
>>>> Expect your answer and thank you very much.
>>>>
>>>> Song Zhao
>>>> _______________________________________________
>>>> Bro mailing list
>>>> bro at bro-ids.org
>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110516/05371788/attachment.html 

From vern at icir.org  Mon May 16 16:25:35 2011
From: vern at icir.org (Vern Paxson)
Date: Mon, 16 May 2011 16:25:35 -0700
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTim5YVsepwfeP=Ti9jU-ChB3LJ1_MA@mail.gmail.com> (Mon,
	16 May 2011 17:04:29 EDT).
Message-ID: <20110516232535.33A8B36A37C@taffy.ICSI.Berkeley.EDU>

> Can Bro itself differentiate these retransmitted and out of order packets?

It's not clear what you mean by differentiate.  Bro reassembles the
TCP bytestream, correctly acounting for retransmitted and out-of-order
packets.

> Besides, can http-rewriter.bro handle the special HTTP packet which, for
> example, includes 2 or more requests or response or even one and half
> requests or responses?

Per Ruoming's earlier comment, http-rewriter.bro does *not* operate on
individual packets, it operates on the reassembled bytestream.  It then
constructs new packets from that bytestream.  The timing of these packets
reflects the timing of the original packets, but the *sequencing* of the
packets does not.

		Vern


From sxz135 at case.edu  Tue May 17 17:24:50 2011
From: sxz135 at case.edu (Song Zhao)
Date: Tue, 17 May 2011 20:24:50 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <20110516232535.33A8B36A37C@taffy.ICSI.Berkeley.EDU>
References: <BANLkTim5YVsepwfeP=Ti9jU-ChB3LJ1_MA@mail.gmail.com>
	<20110516232535.33A8B36A37C@taffy.ICSI.Berkeley.EDU>
Message-ID: <BANLkTi=kfmTVEf5jaSc7zQ5+k3E7YUJ7pg@mail.gmail.com>

Is that function of reassembling TCP bytestream embedded in event engine and
enabled by default when using http-rewriter.bro, or there is a policy
script  we need to call to sort out the tcp packets? Thanks.

On Mon, May 16, 2011 at 7:25 PM, Vern Paxson <vern at icir.org> wrote:

> > Can Bro itself differentiate these retransmitted and out of order
> packets?
>
> It's not clear what you mean by differentiate.  Bro reassembles the
> TCP bytestream, correctly acounting for retransmitted and out-of-order
> packets.
>
> > Besides, can http-rewriter.bro handle the special HTTP packet which, for
> > example, includes 2 or more requests or response or even one and half
> > requests or responses?
>
> Per Ruoming's earlier comment, http-rewriter.bro does *not* operate on
> individual packets, it operates on the reassembled bytestream.  It then
> constructs new packets from that bytestream.  The timing of these packets
> reflects the timing of the original packets, but the *sequencing* of the
> packets does not.
>
>                Vern
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110517/711d6ace/attachment.html 

From vern at icir.org  Wed May 18 10:43:20 2011
From: vern at icir.org (Vern Paxson)
Date: Wed, 18 May 2011 10:43:20 -0700
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTi=kfmTVEf5jaSc7zQ5+k3E7YUJ7pg@mail.gmail.com> (Tue,
	17 May 2011 20:24:50 EDT).
Message-ID: <20110518174320.31E58369FC8@taffy.ICSI.Berkeley.EDU>

> Is that function of reassembling TCP bytestream embedded in event engine and
> enabled by default when using http-rewriter.bro

It's fundamental to how the event engine works.

		Vern


From sxz135 at case.edu  Wed May 25 14:48:00 2011
From: sxz135 at case.edu (Song Zhao)
Date: Wed, 25 May 2011 17:48:00 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <20110518174320.31E58369FC8@taffy.ICSI.Berkeley.EDU>
References: <BANLkTi=kfmTVEf5jaSc7zQ5+k3E7YUJ7pg@mail.gmail.com>
	<20110518174320.31E58369FC8@taffy.ICSI.Berkeley.EDU>
Message-ID: <BANLkTi=ExNPy9ADUgw=XDPYaOsttU3b1nQ@mail.gmail.com>

Hi, all

Sorry to bother you guys again. I still have some very basic questions about
Bro and http-rewriter.bro.

1. Is the command to use http-rewriter.bro on captured offline trace is as
follows?
     ./bro -r 'the name of tracefile we want to deal with' http-rewriter.bro
- w 'the name of tracefile where we want to write the resulting packets

2. If question 1 is yes, will this command call event engine to reassemble
the TCP bytestrem (reorder out of order packets and delete retranmitted
packets) in the captured trace and then event engine will provide the
ressambled byte stream to the upper level where http-rewriter.bro can
rewrite them?

3. Whether http-rewriter.bro and event engine can deal with a big
trace(about 400GB) correctly which is merged by several traces?

Expect your answer and thank you very much.

Song


On Wed, May 18, 2011 at 1:43 PM, Vern Paxson <vern at icir.org> wrote:

> > Is that function of reassembling TCP bytestream embedded in event engine
> and
> > enabled by default when using http-rewriter.bro
>
> It's fundamental to how the event engine works.
>
>                Vern
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110525/129fd858/attachment.html 

From hsbedi at memphis.edu  Wed May 25 16:25:34 2011
From: hsbedi at memphis.edu (Harkeerat Bedi)
Date: Wed, 25 May 2011 18:25:34 -0500
Subject: [Bro] How to throttle (or limit) the bitrate of a UDP connection
	using BRO?
Message-ID: <BANLkTinqHxsJPtKynKFrC=uCVTK7-j2sFw@mail.gmail.com>

Hello All,

I am using BRO for a part of my project. Following is what I intend to do:

1. Monitor UDP connections.
2. Compute their bitrates
3. Throttle the bitrates of these UDP connections based on some
calculations.

I was able to complete tasks 1 and 2. However I don't know how I can
accomplish task 3.

My current setup includes a Client (Node1) sending UDP data to a Server
(Node3). The traffic has to pass through a Gateway (Node2) which is in
between the Client and Server and is running BRO.

Node1 (Client) <------>   Node2 (running BRO) < ------ > Node3 (Server)

If I have a UDP connection (between the Client and the Server) with a bit
rate of 2Mb/s. How can I reduce its bitrate to a user set value - say:
1Mb/s, using BRO?

I am assuming one way may be to drop packets of a connection with a certain
frequency such that the overall bitrate of that connection reduces? Is there
a way we can accomplish this using BRO?

Or, is there any other way?

Kindly suggest.

Thank you,
Harkeerat Bedi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110525/a0d1fdf0/attachment.html 

From vern at icir.org  Wed May 25 18:27:33 2011
From: vern at icir.org (Vern Paxson)
Date: Wed, 25 May 2011 18:27:33 -0700
Subject: [Bro] How to throttle (or limit) the bitrate of a UDP
	connection using BRO?
In-Reply-To: <BANLkTinqHxsJPtKynKFrC=uCVTK7-j2sFw@mail.gmail.com> (Wed,
	25 May 2011 18:25:34 CDT).
Message-ID: <20110526012733.2C0E636A039@taffy.ICSI.Berkeley.EDU>

> Node1 (Client) <------>   Node2 (running BRO) < ------ > Node3 (Server)

Bro does not support inline operation.

		Vern


From seth at icir.org  Wed May 25 18:42:13 2011
From: seth at icir.org (Seth Hall)
Date: Wed, 25 May 2011 21:42:13 -0400
Subject: [Bro] How to throttle (or limit) the bitrate of a UDP
	connection using BRO?
In-Reply-To: <BANLkTinqHxsJPtKynKFrC=uCVTK7-j2sFw@mail.gmail.com>
References: <BANLkTinqHxsJPtKynKFrC=uCVTK7-j2sFw@mail.gmail.com>
Message-ID: <2EF5BFFE-3CB3-4255-BED2-95ED2CA94468@icir.org>


On May 25, 2011, at 7:25 PM, Harkeerat Bedi wrote:

> Node1 (Client) <------>   Node2 (running BRO) < ------ > Node3 (Server)
> 
> If I have a UDP connection (between the Client and the Server) with a bit rate of 2Mb/s. How can I reduce its bitrate to a user set value - say: 1Mb/s, using BRO?

I suppose you could write a script that would install a firewall rule on the box to do the packet dropping (assuming whatever firewall you're using supports that).  You can use the system() function in Bro to call your external script that would put the firewall rule in place.  But generally Vern's point still applies that Bro doesn't ship with any consideration toward this deployment scenario.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/




From sxz135 at case.edu  Wed May 25 22:05:32 2011
From: sxz135 at case.edu (Song Zhao)
Date: Thu, 26 May 2011 01:05:32 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTi=ExNPy9ADUgw=XDPYaOsttU3b1nQ@mail.gmail.com>
References: <BANLkTi=kfmTVEf5jaSc7zQ5+k3E7YUJ7pg@mail.gmail.com>
	<20110518174320.31E58369FC8@taffy.ICSI.Berkeley.EDU>
	<BANLkTi=ExNPy9ADUgw=XDPYaOsttU3b1nQ@mail.gmail.com>
Message-ID: <BANLkTi=QGRjSsKxQn38FMwfwu5eaeV-omg@mail.gmail.com>

Add one more question:

4. If the command I use is as the one in question 1, which kind of packets
would be filtered? Only TCP packets, right? If so, which ports the packets
use  would be filtered?
According to codes of http.bro, global http_ports are
80,81,631,1080,3138,8000,8080 and 8888.
However, when checking the big trace rewritten by the command in question 1,
majority of them are using 20480. Is port 20480 also an http port? Besides,
there are still a small portion with port numbers diffrent from all above.
So I am confused with the filteration of http-rewriter.bro.

Thanks for your help.

Song Zhao
On Wed, May 25, 2011 at 5:48 PM, Song Zhao <sxz135 at case.edu> wrote:

> Hi, all
>
> Sorry to bother you guys again. I still have some very basic questions
> about Bro and http-rewriter.bro.
>
>
> 1. Is the command to use http-rewriter.bro on captured offline trace is as
> follows?
>
> ./bro -r 'the name of tracefile we want to deal with' http-rewriter.bro - w
> 'the name of tracefile where we want to write the resulting packets
>
> 2. If question 1 is yes, will this command call event engine to reassemble
> the TCP bytestrem (reorder out of order packets and delete retranmitted
> packets) in the captured trace and then event engine will provide the
> ressambled byte stream to the upper level where http-rewriter.bro can
> rewrite them?
>
> 3. Whether http-rewriter.bro and event engine can deal with a big
> trace(about 400GB) correctly which is merged by several traces?
>
> Expect your answer and thank you very much.
>
> Song
>
>
> On Wed, May 18, 2011 at 1:43 PM, Vern Paxson <vern at icir.org> wrote:
>
>> > Is that function of reassembling TCP bytestream embedded in event engine
>> and
>> > enabled by default when using http-rewriter.bro
>>
>> It's fundamental to how the event engine works.
>>
>>                Vern
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110526/7f32963d/attachment.html 

From vern at icir.org  Thu May 26 21:23:51 2011
From: vern at icir.org (Vern Paxson)
Date: Thu, 26 May 2011 21:23:51 -0700
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTi=ExNPy9ADUgw=XDPYaOsttU3b1nQ@mail.gmail.com> (Wed,
	25 May 2011 17:48:00 EDT).
Message-ID: <20110527042351.7859236A39D@taffy.ICSI.Berkeley.EDU>

> 1. Is the command to use http-rewriter.bro on captured offline trace is as
> follows?
>      ./bro -r 'the name of tracefile we want to deal with' http-rewriter.bro
> - w 'the name of tracefile where we want to write the resulting packets

It's -A, not -w.

> 2. If question 1 is yes, will this command call event engine to reassemble
> the TCP bytestrem (reorder out of order packets and delete retranmitted
> packets) in the captured trace and then event engine will provide the
> ressambled byte stream to the upper level where http-rewriter.bro can
> rewrite them?

Yep.

> 3. Whether http-rewriter.bro and event engine can deal with a big
> trace(about 400GB) correctly which is merged by several traces?

It should be able to, though that code hasn't been stressed all that
much and might wind up having a memory leak (or simply memory that
doesn't get reclaimed), which could cause it to blow up on a really
big input.

> 4. If the command I use is as the one in question 1, which kind of packets
> would be filtered? Only TCP packets, right? If so, which ports the packets
> use  would be filtered?

http-rewriter loads http-reply.bro, which specifies the filter as:

	tcp src port 80 or tcp src port 8080 or tcp src port 8000

> According to codes of http.bro, global http_ports are
> 80,81,631,1080,3138,8000,8080 and 8888.

Note, that list is used only if you turn on DPD.

> However, when checking the big trace rewritten by the command in question 1,
> majority of them are using 20480. Is port 20480 also an http port?

Well, other than 80, none of them is a standardized HTTP port.  But you
can add 20480 to the list in http-reply.bro to ensure it's captured.

> Besides,
> there are still a small portion with port numbers diffrent from all above.
> So I am confused with the filteration of http-rewriter.bro.

Then in principle you should use DPD.  However, I don't know whether
it's integrated with the rewriting framework.

		Vern


From hsbedi at memphis.edu  Thu May 26 21:39:14 2011
From: hsbedi at memphis.edu (Harkeerat Bedi)
Date: Thu, 26 May 2011 23:39:14 -0500
Subject: [Bro] How to throttle (or limit) the bitrate of a UDP
 connection using BRO?
In-Reply-To: <2EF5BFFE-3CB3-4255-BED2-95ED2CA94468@icir.org>
References: <BANLkTinqHxsJPtKynKFrC=uCVTK7-j2sFw@mail.gmail.com>
	<2EF5BFFE-3CB3-4255-BED2-95ED2CA94468@icir.org>
Message-ID: <BANLkTikcQ+UM0x25iiWM+x1NZsjWAW1etA@mail.gmail.com>

OK. Thank you Seth and Vern for your feedback. I will follow the suggestions
provided by Seth.

One of the reasons I thought about this was because I came across a function
in BRO called "terminate_connection(c: connection)". This function, as per
the wiki: attempts to terminate a given connection using a rst utility.
However, now I understand that BRO does not support inline operation as this
rst utility is not a part of BRO.

Thanks again,
Harkeerat Bedi


On Wed, May 25, 2011 at 8:42 PM, Seth Hall <seth at icir.org> wrote:

>
> On May 25, 2011, at 7:25 PM, Harkeerat Bedi wrote:
>
> > Node1 (Client) <------>   Node2 (running BRO) < ------ > Node3 (Server)
> >
> > If I have a UDP connection (between the Client and the Server) with a bit
> rate of 2Mb/s. How can I reduce its bitrate to a user set value - say:
> 1Mb/s, using BRO?
>
> I suppose you could write a script that would install a firewall rule on
> the box to do the packet dropping (assuming whatever firewall you're using
> supports that).  You can use the system() function in Bro to call your
> external script that would put the firewall rule in place.  But generally
> Vern's point still applies that Bro doesn't ship with any consideration
> toward this deployment scenario.
>
>  .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro-ids.org/
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110526/f7f797f5/attachment.html 

From hsbedi at memphis.edu  Thu May 26 21:47:09 2011
From: hsbedi at memphis.edu (Harkeerat Bedi)
Date: Thu, 26 May 2011 23:47:09 -0500
Subject: [Bro] How to throttle (or limit) the bitrate of a UDP
 connection using BRO?
In-Reply-To: <BANLkTi=E5L0b6pkhGoKk51BUarzuOqiV-A@mail.gmail.com>
References: <BANLkTinqHxsJPtKynKFrC=uCVTK7-j2sFw@mail.gmail.com>
	<BANLkTi=E5L0b6pkhGoKk51BUarzuOqiV-A@mail.gmail.com>
Message-ID: <BANLkTinHn3k+2K0B_xPPk137tVProZMpXg@mail.gmail.com>

Thank you Aaron for your suggestions :) I will look into them.

Regards,
Harkeerat Bedi


On Thu, May 26, 2011 at 4:29 AM, Aaron Glenn <aaron.glenn at gmail.com> wrote:

> On Wed, May 25, 2011 at 11:25 PM, Harkeerat Bedi <hsbedi at memphis.edu>
> wrote:
> > a way we can accomplish this using BRO?
>
> BRO is not what you want in this scenario
>
> > Or, is there any other way?
> > Kindly suggest.
>
> netgraph in freebsd
> pf in openbsd
> npf in netbsd
> I urge you try them all (:
>
> > Thank you,
> > Harkeerat Bedi
>
> best of luck,
> aaron glenn
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110526/999096ea/attachment.html 

From sxz135 at case.edu  Fri May 27 08:22:48 2011
From: sxz135 at case.edu (Song Zhao)
Date: Fri, 27 May 2011 11:22:48 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <20110527042351.7859236A39D@taffy.ICSI.Berkeley.EDU>
References: <BANLkTi=ExNPy9ADUgw=XDPYaOsttU3b1nQ@mail.gmail.com>
	<20110527042351.7859236A39D@taffy.ICSI.Berkeley.EDU>
Message-ID: <BANLkTinOrYqbUtVRdWB8jj12sfCKtoDq8g@mail.gmail.com>

> 1. Is the command to use http-rewriter.bro on captured offline trace is as
> follows?
>      ./bro -r 'the name of tracefile we want to deal with'
http-rewriter.bro
> - w 'the name of tracefile where we want to write the resulting packets

   >>It's -A, not -w.

Will there be any difference between  -A and -w for the use of
http-rewriter.bro?  I just used -A to rewrite some examples and it seems
that the resulting files are the same as those one using - w.

> According to codes of http.bro, global http_ports are
 > 80,81,631,1080,3138,8000,8080 and 8888.

Note, that list is used only if you turn on DPD.

> Besides,
> there are still a small portion with port numbers diffrent from all above.
> So I am confused with the filteration of http-rewriter.bro.

Then in principle you should use DPD.  However, I don't know whether
it's integrated with the rewriting framework.

The command I used is only " ./bro -r readfile http-rewriter.bro -w
writerfile. I don't know if DPD is turned on. Actually, http.bro is loaded
by http-request.bro, which is also loaded by http-reply.bro. In http.bro, I
think there are codes about DPD as follows:
# DPM configuration.
global http_ports = {
    80/tcp, 81/tcp, 631/tcp, 1080/tcp, 3138/tcp,
    8000/tcp, 8080/tcp, 8888/tcp,
};
redef dpd_config += { [ANALYZER_HTTP] = [$ports = http_ports] };
redef dpd_config += { [ANALYZER_HTTP_BINPAC] = [$ports = http_ports] };
Dose it mean DPD has been integrated within the rewriting framework? And
whether  it is reason why the majority of rewritten trace I got is from port
20480 and also from some ports other than 80,8000,8080?

Thanks a lot.

Song
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110527/d2c44716/attachment.html 

From rpang at cs.princeton.edu  Fri May 27 10:27:58 2011
From: rpang at cs.princeton.edu (Ruoming Pang)
Date: Fri, 27 May 2011 13:27:58 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTinOrYqbUtVRdWB8jj12sfCKtoDq8g@mail.gmail.com>
References: <BANLkTi=ExNPy9ADUgw=XDPYaOsttU3b1nQ@mail.gmail.com>
	<20110527042351.7859236A39D@taffy.ICSI.Berkeley.EDU>
	<BANLkTinOrYqbUtVRdWB8jj12sfCKtoDq8g@mail.gmail.com>
Message-ID: <BANLkTi=9usONecTRXKhWi55SbU5zBJnzQg@mail.gmail.com>

On Fri, May 27, 2011 at 11:22 AM, Song Zhao <sxz135 at case.edu> wrote:

> > 1. Is the command to use http-rewriter.bro on captured offline trace is
> as
> > follows?
> >      ./bro -r 'the name of tracefile we want to deal with'
> http-rewriter.bro
> > - w 'the name of tracefile where we want to write the resulting packets
>
>    >>It's -A, not -w.
>
> Will there be any difference between  -A and -w for the use of
> http-rewriter.bro?  I just used -A to rewrite some examples and it seems
> that the resulting files are the same as those one using - w.
>
> > According to codes of http.bro, global http_ports are
>  > 80,81,631,1080,3138,8000,8080 and 8888.
>
> Note, that list is used only if you turn on DPD.
>
> > Besides,
> > there are still a small portion with port numbers diffrent from all
> above.
> > So I am confused with the filteration of http-rewriter.bro.
>
> Then in principle you should use DPD.  However, I don't know whether
> it's integrated with the rewriting framework.
>
> The command I used is only " ./bro -r readfile http-rewriter.bro -w
> writerfile.


I'm not sure if it still matters, but one used to need to special all
options before arguments, so try:

./bro -r readfile -A writerfile http-rewriter.bro


> I don't know if DPD is turned on. Actually, http.bro is loaded by
> http-request.bro, which is also loaded by http-reply.bro. In http.bro, I
> think there are codes about DPD as follows:
> # DPM configuration.
> global http_ports = {
>     80/tcp, 81/tcp, 631/tcp, 1080/tcp, 3138/tcp,
>     8000/tcp, 8080/tcp, 8888/tcp,
> };
> redef dpd_config += { [ANALYZER_HTTP] = [$ports = http_ports] };
> redef dpd_config += { [ANALYZER_HTTP_BINPAC] = [$ports = http_ports] };
> Dose it mean DPD has been integrated within the rewriting framework? And
> whether  it is reason why the majority of rewritten trace I got is from port
> 20480 and also from some ports other than 80,8000,8080?
>
> Thanks a lot.
>
> Song
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110527/167083e0/attachment.html 

From sxz135 at case.edu  Fri May 27 11:45:41 2011
From: sxz135 at case.edu (Song Zhao)
Date: Fri, 27 May 2011 14:45:41 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTi=9usONecTRXKhWi55SbU5zBJnzQg@mail.gmail.com>
References: <BANLkTi=ExNPy9ADUgw=XDPYaOsttU3b1nQ@mail.gmail.com>
	<20110527042351.7859236A39D@taffy.ICSI.Berkeley.EDU>
	<BANLkTinOrYqbUtVRdWB8jj12sfCKtoDq8g@mail.gmail.com>
	<BANLkTi=9usONecTRXKhWi55SbU5zBJnzQg@mail.gmail.com>
Message-ID: <BANLkTim=_s8eBsA51S5rFX4HSk3Zj77D+g@mail.gmail.com>

Hi ,Ruoming

1.
I also tried ./bro -r readfile -A writerfile http-rewriter.bro, whose
results seem to be the same as those of ./bro -r readfile http-rewriter.bro
-A writefile. And is there any difference of the resulting trace between
using -A and - w for http-rewriter.bro? I tried some examples and their
results seem the same.

2.
Does http-rewriter.bro by default use DPD to find http streams intead of
port numbers?
After rewriting a big trace which insists of all kind of streams(TCP and
UDP) using http-rewriter.bro, the ports of the resulting trace range widely,
including 80,8000,8080,631,1080 and so forth. Interestingly, majority of
them are port 20480. Is it because of use of DPD?

Thanks.

Song Zhao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110527/0307fe25/attachment.html 

From vern at icir.org  Fri May 27 13:43:51 2011
From: vern at icir.org (Vern Paxson)
Date: Fri, 27 May 2011 13:43:51 -0700
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTim=_s8eBsA51S5rFX4HSk3Zj77D+g@mail.gmail.com> (Fri,
	27 May 2011 14:45:41 EDT).
Message-ID: <20110527204351.E200C36A3A3@taffy.ICSI.Berkeley.EDU>

> I also tried ./bro -r readfile -A writerfile http-rewriter.bro, whose
> results seem to be the same as those of ./bro -r readfile http-rewriter.bro
> -A writefile. And is there any difference of the resulting trace between
> using -A and - w for http-rewriter.bro?

If you specify both, then you get the untransformed trace in the -w file
and the transformed one in -A.  If you specify just one, then that's the
transformed file.

> Does http-rewriter.bro by default use DPD to find http streams intead of
> port numbers?

I don't know.  But you can avoid this question by just wiring in the
ports of interest into the initialization of capture_filters in http-reply.bro.

> Interestingly, majority of
> them are port 20480.

Note, 20480 = 80 but little endian.  This suggests either a bug in how
you're viewing the port numbers, or in how Bro is displaying (or possibly
processing them).

		Vern


From suman.nandi at chitkara.edu.in  Sun May 29 23:23:57 2011
From: suman.nandi at chitkara.edu.in (Suman Nandi)
Date: Mon, 30 May 2011 11:53:57 +0530
Subject: [Bro] Current IDS and Data Mining research
Message-ID: <BANLkTi=pfrxkgD6SzOmN5yrejS3G5MDJxg@mail.gmail.com>

Dear Bro Developer and contributer
I have been working on IDS and Data Mining .I would like to know the current
research in this area that IDS using Data Mining and what are the current
reseach areas and objectives where Data Mining can provide solutions to IDS?

-- 
Regards
SUMAN KUMAR NANDI
HOD-Computer Applications
Chitkara University, Punjab
India
 Mobile:+919501105658
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110530/8ae49156/attachment.html 

From sxz135 at case.edu  Mon May 30 00:22:08 2011
From: sxz135 at case.edu (Song Zhao)
Date: Mon, 30 May 2011 03:22:08 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <20110527204351.E200C36A3A3@taffy.ICSI.Berkeley.EDU>
References: <BANLkTim=_s8eBsA51S5rFX4HSk3Zj77D+g@mail.gmail.com>
	<20110527204351.E200C36A3A3@taffy.ICSI.Berkeley.EDU>
Message-ID: <BANLkTi=3WDaWZ2sT8TcPvOM=ifG8zJPC=w@mail.gmail.com>

Hi,

In the 12G rewritten trace, the port numbers range widely. http-rewriter.bro
loads http-reply.bro,which loads http-request.bro,which loads http.bro. The
codes about filteration in these policy scripts are as follows:

In http-request.bro:
redef capture_filters +=  {
    ["http-request"] = "tcp dst port 80 or tcp dst port 8080 or tcp dst port
8000"
};
In http-reply.bro:
redef capture_filters += {
    ["http-reply"] = "tcp src port 80 or tcp src port 8080 or tcp src port
8000"
};
In http.bro:
# DPM configuration.
global http_ports = {
    80/tcp, 81/tcp, 631/tcp, 1080/tcp, 3138/tcp,
    8000/tcp, 8080/tcp, 8888/tcp,
};
redef dpd_config += { [ANALYZER_HTTP] = [$ports = http_ports] };
redef dpd_config += { [ANALYZER_HTTP_BINPAC] = [$ports = http_ports] };

Any of them sets DPD on? If not, why the port numbers in the rewritten trace
range so widely, which range much more widely than the range of global
http_ports?
I didn't load dpd.bro anywhere. After checking the payloads roughly, as far
as I found, they all contain HTTP requests or responses. I mean they are
really "HTTP streams" whatever the port number is.

Thanks
Song Zhao

On Fri, May 27, 2011 at 4:43 PM, Vern Paxson <vern at icir.org> wrote:

> > I also tried ./bro -r readfile -A writerfile http-rewriter.bro, whose
> > results seem to be the same as those of ./bro -r readfile
> http-rewriter.bro
> > -A writefile. And is there any difference of the resulting trace between
> > using -A and - w for http-rewriter.bro?
>
> If you specify both, then you get the untransformed trace in the -w file
> and the transformed one in -A.  If you specify just one, then that's the
> transformed file.
>
> > Does http-rewriter.bro by default use DPD to find http streams intead of
> > port numbers?
>
> I don't know.  But you can avoid this question by just wiring in the
> ports of interest into the initialization of capture_filters in
> http-reply.bro.
>
> > Interestingly, majority of
> > them are port 20480.
>
> Note, 20480 = 80 but little endian.  This suggests either a bug in how
> you're viewing the port numbers, or in how Bro is displaying (or possibly
> processing them).
>
>                Vern
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110530/005188f5/attachment.html 

From ssm_as at yahoo.com  Mon May 30 12:40:35 2011
From: ssm_as at yahoo.com (ssm_as)
Date: Mon, 30 May 2011 12:40:35 -0700 (PDT)
Subject: [Bro] Current IDS and Data Mining research
In-Reply-To: <mailman.1.1306782001.14653.bro@bro-ids.org>
Message-ID: <959507.99336.qm@web161403.mail.bf1.yahoo.com>

Hi,
Well, it depend on what are you trying to do?
For example, do you want to use data mining in IDS alerts analysis (e.g. alerts verification, alerts aggregation, alerts correlation)? In this case you will find? a lot of research work submitted in that area.

Do you want to use data mining in building IDS to detect intrusions? Then you probably taking about anomaly detection based IDS? In my opinion data mining is not the best approach to do that. Probably, you will need to think about soft computing approaches (neural network, artificial immune system, swarm intelligence, etc).

The issue with your question is that you are using very abstract keywords "data mining" and "IDS" . you should be more specific.

Thanks,
Sherif Saad 
Ph.D Candidate, University of Victoria 

--- On Mon, 5/30/11, bro-request at bro-ids.org <bro-request at bro-ids.org> wrote:

From: bro-request at bro-ids.org <bro-request at bro-ids.org>
Subject: Bro Digest, Vol 61, Issue 16
To: bro at bro-ids.org
Received: Monday, May 30, 2011, 10:00 PM

Send Bro mailing list submissions to
??? bro at bro-ids.org

To subscribe or unsubscribe via the World Wide Web, visit
??? http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
or, via email, send a message with subject or body 'help' to
??? bro-request at bro-ids.org

You can reach the person managing the list at
??? bro-owner at bro-ids.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Bro digest..."


Today's Topics:

???1. Current IDS and Data Mining research (Suman Nandi)
???2. Re: handle out of order and retransmitted packets in offline
? ? ? trace (Song Zhao)


----------------------------------------------------------------------

Message: 1
Date: Mon, 30 May 2011 11:53:57 +0530
From: Suman Nandi <suman.nandi at chitkara.edu.in>
Subject: [Bro] Current IDS and Data Mining research
To: bro at bro-ids.org
Message-ID: <BANLkTi=pfrxkgD6SzOmN5yrejS3G5MDJxg at mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Dear Bro Developer and contributer
I have been working on IDS and Data Mining .I would like to know the current
research in this area that IDS using Data Mining and what are the current
reseach areas and objectives where Data Mining can provide solutions to IDS?

-- 
Regards
SUMAN KUMAR NANDI
HOD-Computer Applications
Chitkara University, Punjab
India
 Mobile:+919501105658
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110530/8ae49156/attachment-0001.html 

------------------------------

Message: 2
Date: Mon, 30 May 2011 03:22:08 -0400
From: Song Zhao <sxz135 at case.edu>
Subject: Re: [Bro] handle out of order and retransmitted packets in
??? offline??? trace
To: Vern Paxson <vern at icir.org>
Cc: bro at bro-ids.org, Ruoming Pang <rpang at cs.princeton.edu>
Message-ID: <BANLkTi=3WDaWZ2sT8TcPvOM=ifG8zJPC=w at mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hi,

In the 12G rewritten trace, the port numbers range widely. http-rewriter.bro
loads http-reply.bro,which loads http-request.bro,which loads http.bro. The
codes about filteration in these policy scripts are as follows:

In http-request.bro:
redef capture_filters +=? {
? ? ["http-request"] = "tcp dst port 80 or tcp dst port 8080 or tcp dst port
8000"
};
In http-reply.bro:
redef capture_filters += {
? ? ["http-reply"] = "tcp src port 80 or tcp src port 8080 or tcp src port
8000"
};
In http.bro:
# DPM configuration.
global http_ports = {
? ? 80/tcp, 81/tcp, 631/tcp, 1080/tcp, 3138/tcp,
? ? 8000/tcp, 8080/tcp, 8888/tcp,
};
redef dpd_config += { [ANALYZER_HTTP] = [$ports = http_ports] };
redef dpd_config += { [ANALYZER_HTTP_BINPAC] = [$ports = http_ports] };

Any of them sets DPD on? If not, why the port numbers in the rewritten trace
range so widely, which range much more widely than the range of global
http_ports?
I didn't load dpd.bro anywhere. After checking the payloads roughly, as far
as I found, they all contain HTTP requests or responses. I mean they are
really "HTTP streams" whatever the port number is.

Thanks
Song Zhao

On Fri, May 27, 2011 at 4:43 PM, Vern Paxson <vern at icir.org> wrote:

> > I also tried ./bro -r readfile -A writerfile http-rewriter.bro, whose
> > results seem to be the same as those of ./bro -r readfile
> http-rewriter.bro
> > -A writefile. And is there any difference of the resulting trace between
> > using -A and - w for http-rewriter.bro?
>
> If you specify both, then you get the untransformed trace in the -w file
> and the transformed one in -A.? If you specify just one, then that's the
> transformed file.
>
> > Does http-rewriter.bro by default use DPD to find http streams intead of
> > port numbers?
>
> I don't know.? But you can avoid this question by just wiring in the
> ports of interest into the initialization of capture_filters in
> http-reply.bro.
>
> > Interestingly, majority of
> > them are port 20480.
>
> Note, 20480 = 80 but little endian.? This suggests either a bug in how
> you're viewing the port numbers, or in how Bro is displaying (or possibly
> processing them).
>
>? ? ? ? ? ? ? ? Vern
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110530/005188f5/attachment-0001.html 

------------------------------

_______________________________________________
Bro mailing list
Bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


End of Bro Digest, Vol 61, Issue 16
***********************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110530/a26140bc/attachment.html 

From sxz135 at case.edu  Mon May 30 23:24:57 2011
From: sxz135 at case.edu (Song Zhao)
Date: Tue, 31 May 2011 02:24:57 -0400
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTi=3WDaWZ2sT8TcPvOM=ifG8zJPC=w@mail.gmail.com>
References: <BANLkTim=_s8eBsA51S5rFX4HSk3Zj77D+g@mail.gmail.com>
	<20110527204351.E200C36A3A3@taffy.ICSI.Berkeley.EDU>
	<BANLkTi=3WDaWZ2sT8TcPvOM=ifG8zJPC=w@mail.gmail.com>
Message-ID: <BANLkTi=p3dPvfHor+MmNL4AD02ySn21PuQ@mail.gmail.com>

Hi, all

I found the reason why the majority of the port numbers in the rewritten
trace is 20480 instead of 80 is that in the library <netinet/tcp.h> the
variables representing source port and destination port(th_sport and
th_dport) don't store the real port numbers as I think. Actually, it stores
port 80 as 20480 and it stores other port numbers differently from they are
supposed to be. Anyone knows the reason? Is it a kind of one to one mapping?
Or I made a mistake on using it?

Thanks
Song
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110531/875e459a/attachment.html 

From vern at icir.org  Tue May 31 09:17:08 2011
From: vern at icir.org (Vern Paxson)
Date: Tue, 31 May 2011 09:17:08 -0700
Subject: [Bro] handle out of order and retransmitted packets in offline
	trace
In-Reply-To: <BANLkTi=p3dPvfHor+MmNL4AD02ySn21PuQ@mail.gmail.com> (Tue,
	31 May 2011 02:24:57 EDT).
Message-ID: <20110531161708.BF07536A38B@taffy.ICSI.Berkeley.EDU>

> Actually, it stores
> port 80 as 20480 and it stores other port numbers differently from they are
> supposed to be. Anyone knows the reason? Is it a kind of one to one mapping?

As I already told you via private email, you are looking at the little-endian
version of 80 rather than the big-endian.

		Vern


From robin at icir.org  Tue May 31 20:51:09 2011
From: robin at icir.org (Robin Sommer)
Date: Tue, 31 May 2011 20:51:09 -0700
Subject: [Bro] Open Position at NCSA
In-Reply-To: <20110506005847.GB63816@icir.org>
References: <20110506005847.GB63816@icir.org>
Message-ID: <20110601035109.GE70777@icir.org>

Just a reminder: the application deadline for this position is Jun 14,
so there's still time to apply. :)

Robin

On Thu, May 05, 2011 at 17:58 -0700, I wrote:

> Have you always wanted to work on Bro and be paid for it? :-)
> 
> NCSA has an opening for a Senior Research Programmer:
> 
>     https://jobs.illinois.edu/default.cfm?page=job&jobID=8847&returnPage=sear
> 
> This is a full-time position that focuses on Bro development in close
> collaboration with our group here at ICSI.

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org