From drummachina at tdhack.com  Wed Jan  1 04:39:16 2014
From: drummachina at tdhack.com (drum)
Date: Wed, 1 Jan 2014 13:39:16 +0100
Subject: [Bro] extract jar files from HTTP stream
In-Reply-To: <52C17674.6040500@psc.edu>
References: <CAAzamUcuU=faUCsXHDEXgDsg-c6Ni4Seh+q1Uv9uZcU-5v32Yg@mail.gmail.com>
	<52C17674.6040500@psc.edu>
Message-ID: <CAAzamUfPQxtqq_Ns36Eb0yX9PT2-PcSmcUwZZuGjC8E0Te-EWA@mail.gmail.com>

Thanks Shane, could you please write step-by-step instruction where should
I put your code. I have no idea how to port it into my installation. Let's
assume I use 'plain' bro (no security onion) installed in /opt/bro. What is
the next step? BTW - I have these both files (extract.bro and main.bro):

root at onion:~# ls -al /opt/bro/share/bro/base/files/extract/main.bro
/opt/bro/share/bro/file-extraction/extract.bro
-rw-r--r-- 1 root root 2126 Nov  7 18:27
/opt/bro/share/bro/base/files/extract/main.bro
-rw-r--r-- 1 root root  572 Jan  1 12:26
/opt/bro/share/bro/file-extraction/extract.bro



Pozdrawiam,
drumm


2013/12/30 Shane Filus <filus at psc.edu>

> On 12/29/13 2:25 PM, drum wrote:
> > Hello,
> >
> >
> > Is there a tutorial for version 2.2 of BRO? I'd like to understand how
> > can I write my own scripts to support extraction of verious files,
> > like jar. So far I tried adding "application/jar" (it was logged to
> > /nsm/bro/logs/current/files.
> > log) as mime type to /opt/bro/share/bro/file-extraction/extract.bro
> > file but it seems I have to do something else too as this change is
> > not capturing files to /nsm/bro/extracted/ directory.
> Hello,
>
>
> Check the second example under 'Adding Analysis'  for a start in file
> extraction.
> http://www.bro.org/sphinx/frameworks/file-analysis.html#adding-analysis
>
> Also, not sure how it maps to Sec Onion, but there is
> $PREFX/share/bro/base/files/extract/main.bro from a source install.
> Might be your 'extract.bro'? I don't that file name in either 2.1 or 2.2
> source trees.
>
> Used the code below to do something similar. There's probably a more
> elegant, or efficient solution, but it seems to working as expected,
> given the limited testing I've done.
>
>
> # define file extraction filters
> const match_file_source = /HTTP/ |
>               /IRC/ |
>               /IRC_DATA/ |
>               /FTP/ |
>               /FTP_DATA/ &redef;
>
> const match_file_mime =   /text\/x-perl/ |
>               /text\/x-msdos-batch/ |
>               /text\/x-java/ |
>               /application\/x-gzip/ |
>               /application\/x-bzip2/ |
>               /application\/x-dosexec/ |
>               /application\/zip/ |
>               /application\/jar/ |
>               /application\/x-tar/ |
>               /application\/x-archive/ |
>               /application\/mac-binhex40/ |
>               /application\/x-java-keystore/ |
>               /application\/x-java-jce-keystore/ |
>               /application\/x-executable/ |
>               /application\/javascript/ &redef;
>
> # add analyer to file_new event
> event file_new(f: fa_file)
>     {
>     if ( f?$mime_type &&
>         match_file_source in f$source &&
>         match_file_mime in f$mime_type )
>             Files::add_analyzer(f, Files::ANALYZER_EXTRACT);
>        }
>
>
> Thanks,
>
>
> Shane
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140101/76d8a2a5/attachment.html 

From doug.burks at gmail.com  Wed Jan  1 06:16:08 2014
From: doug.burks at gmail.com (Doug Burks)
Date: Wed, 1 Jan 2014 09:16:08 -0500
Subject: [Bro] extract jar files from HTTP stream
In-Reply-To: <CAAzamUfPQxtqq_Ns36Eb0yX9PT2-PcSmcUwZZuGjC8E0Te-EWA@mail.gmail.com>
References: <CAAzamUcuU=faUCsXHDEXgDsg-c6Ni4Seh+q1Uv9uZcU-5v32Yg@mail.gmail.com>
	<52C17674.6040500@psc.edu>
	<CAAzamUfPQxtqq_Ns36Eb0yX9PT2-PcSmcUwZZuGjC8E0Te-EWA@mail.gmail.com>
Message-ID: <CAK8kjrB9SR8JhXw3OS3Ym4ncaOoXF7+mJC2kSqMQ3+PN6aELNQ@mail.gmail.com>

Hi drum,

Start off with the following:

- edit /opt/bro/share/bro/file-extraction/extract.bro

- change the following line:
    if ( ! f?$mime_type || f$mime_type != "application/x-dosexec" )
to:
    if ( ! f?$mime_type || f$mime_type != "application/jar" )

- run the following:
sudo broctl install
sudo broctl restart

Bro should now be extracting jar files to /nsm/bro/extracted/.

Once you have that working, then you should be able to add in Shane's
match_file_mime to the same script to allow you to extract multiple
file types.

On Wed, Jan 1, 2014 at 7:39 AM, drum <drummachina at tdhack.com> wrote:
> Thanks Shane, could you please write step-by-step instruction where should I
> put your code. I have no idea how to port it into my installation. Let's
> assume I use 'plain' bro (no security onion) installed in /opt/bro. What is
> the next step? BTW - I have these both files (extract.bro and main.bro):
>
> root at onion:~# ls -al /opt/bro/share/bro/base/files/extract/main.bro
> /opt/bro/share/bro/file-extraction/extract.bro
> -rw-r--r-- 1 root root 2126 Nov  7 18:27
> /opt/bro/share/bro/base/files/extract/main.bro
> -rw-r--r-- 1 root root  572 Jan  1 12:26
> /opt/bro/share/bro/file-extraction/extract.bro
>
>
>
> Pozdrawiam,
> drumm
>
>
> 2013/12/30 Shane Filus <filus at psc.edu>
>>
>> On 12/29/13 2:25 PM, drum wrote:
>> > Hello,
>> >
>> >
>> > Is there a tutorial for version 2.2 of BRO? I'd like to understand how
>> > can I write my own scripts to support extraction of verious files,
>> > like jar. So far I tried adding "application/jar" (it was logged to
>> > /nsm/bro/logs/current/files.
>> > log) as mime type to /opt/bro/share/bro/file-extraction/extract.bro
>> > file but it seems I have to do something else too as this change is
>> > not capturing files to /nsm/bro/extracted/ directory.
>> Hello,
>>
>>
>> Check the second example under 'Adding Analysis'  for a start in file
>> extraction.
>> http://www.bro.org/sphinx/frameworks/file-analysis.html#adding-analysis
>>
>> Also, not sure how it maps to Sec Onion, but there is
>> $PREFX/share/bro/base/files/extract/main.bro from a source install.
>> Might be your 'extract.bro'? I don't that file name in either 2.1 or 2.2
>> source trees.
>>
>> Used the code below to do something similar. There's probably a more
>> elegant, or efficient solution, but it seems to working as expected,
>> given the limited testing I've done.
>>
>>
>> # define file extraction filters
>> const match_file_source = /HTTP/ |
>>               /IRC/ |
>>               /IRC_DATA/ |
>>               /FTP/ |
>>               /FTP_DATA/ &redef;
>>
>> const match_file_mime =   /text\/x-perl/ |
>>               /text\/x-msdos-batch/ |
>>               /text\/x-java/ |
>>               /application\/x-gzip/ |
>>               /application\/x-bzip2/ |
>>               /application\/x-dosexec/ |
>>               /application\/zip/ |
>>               /application\/jar/ |
>>               /application\/x-tar/ |
>>               /application\/x-archive/ |
>>               /application\/mac-binhex40/ |
>>               /application\/x-java-keystore/ |
>>               /application\/x-java-jce-keystore/ |
>>               /application\/x-executable/ |
>>               /application\/javascript/ &redef;
>>
>> # add analyer to file_new event
>> event file_new(f: fa_file)
>>     {
>>     if ( f?$mime_type &&
>>         match_file_source in f$source &&
>>         match_file_mime in f$mime_type )
>>             Files::add_analyzer(f, Files::ANALYZER_EXTRACT);
>>        }
>>
>>
>> Thanks,
>>
>>
>> Shane
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



-- 
Doug Burks


From drummachina at tdhack.com  Wed Jan  1 07:26:15 2014
From: drummachina at tdhack.com (drum)
Date: Wed, 1 Jan 2014 16:26:15 +0100
Subject: [Bro] extract jar files from HTTP stream
In-Reply-To: <CAK8kjrB9SR8JhXw3OS3Ym4ncaOoXF7+mJC2kSqMQ3+PN6aELNQ@mail.gmail.com>
References: <CAAzamUcuU=faUCsXHDEXgDsg-c6Ni4Seh+q1Uv9uZcU-5v32Yg@mail.gmail.com>
	<52C17674.6040500@psc.edu>
	<CAAzamUfPQxtqq_Ns36Eb0yX9PT2-PcSmcUwZZuGjC8E0Te-EWA@mail.gmail.com>
	<CAK8kjrB9SR8JhXw3OS3Ym4ncaOoXF7+mJC2kSqMQ3+PN6aELNQ@mail.gmail.com>
Message-ID: <CAAzamUebCk0o-WWE=6Qi5JiG9fnKhT6FTiDHWWOxn6iu6100mA@mail.gmail.com>

Thank you Doug, that worked. Actually I ended up with following (ugly)
syntax:

root at onion:~# cat /opt/bro/share/bro/file-extraction/extract.bro
global ext_map: table[string] of string = {
    ["application/x-dosexec"] = "exe",
    ["text/plain"] = "txt",
    ["image/jpeg"] = "jpg",
    ["image/png"] = "png",
    ["text/html"] = "html",
} &default ="";

event file_new(f: fa_file)
    {
    #if ( ! f?$mime_type || f$mime_type != "application/x-dosexec" )
    if ( ! f?$mime_type || f$mime_type != "application/jar" )
        return;

    local ext = "";

    if ( f?$mime_type )
        ext = ext_map[f$mime_type];

    local fname = fmt("/nsm/bro/extracted/%s-%s.%s", f$source, f$id, ext);
    Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
[$extract_filename=fname]);
    }

# define file extraction filters
const match_file_source = /HTTP/ |
              /IRC/ |
              /IRC_DATA/ |
              /FTP/ |
              /FTP_DATA/ &redef;

const match_file_mime =   /text\/x-perl/ |
              /text\/x-msdos-batch/ |
              /text\/x-java/ |
              /application\/x-gzip/ |
              /application\/x-bzip2/ |
              /application\/x-dosexec/ |
              /application\/zip/ |
              /application\/jar/ |
              /application\/x-tar/ |
              /application\/x-archive/ |
              /application\/mac-binhex40/ |
              /application\/x-java-keystore/ |
              /application\/x-java-jce-keystore/ |
              /application\/x-executable/ |
              /application\/javascript/ &redef;

# add analyer to file_new event
event file_new(f: fa_file)
    {
    if ( f?$mime_type &&
        match_file_source in f$source &&
        match_file_mime in f$mime_type )
            Files::add_analyzer(f, Files::ANALYZER_EXTRACT);
       }

and I bet it can be written better.

During this excersise I noticed that /nsm/bro/logs/current/files.log was
not present. Found this in google:
https://groups.google.com/forum/#!topic/security-onion/r4eZWOegvsY and
followed suggestions. Indeed, /nsm/bro/logs/current/communication.log file
contained:

1388589086.005591       manager child   -       -       -       error
can't bind to 0.0.0.0:47761, Address already in use

I had to use lsof command to check which process was it and:

root at onion:/nsm/bro/logs/current# lsof -i:47761
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
bro     12253 root    0u  IPv4 300348      0t0  TCP *:47761 (LISTEN)
bro     12253 root    1u  IPv6 300349      0t0  TCP *:47761 (LISTEN)

so I killed it. After doing service nsm restart everything seems to be
working again (logs + file extraction. BTW - jar files are stored without
extension "jar"). But I still can see errors in communication.log:

root at onion:~# cat /nsm/bro/logs/current/communication.log |grep Address
1388589202.005024       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589204.006373       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589235.000845       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589233.001513       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589264.004692       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589266.005739       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589297.004983       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589295.005424       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589328.004598       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589326.005488       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589359.004987       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589357.004749       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589390.004760       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589388.004887       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589419.005759       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589421.005335       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589450.004988       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589452.005818       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589481.001524       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589483.001843       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589512.004547       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589514.004785       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589543.005441       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589545.004584       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589574.005125       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589576.005318       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589605.005628       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589607.004816       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589636.005317       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589638.005756       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589667.005455       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589669.005977       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589700.006115       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589698.004967       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589729.000811       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589731.012333       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589760.005435       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589762.005389       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589791.004834       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589793.005790       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589824.005289       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589822.004770       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use
1388589855.005452       onion-eth1-1    child   -       -       -
error   can't bind to 0.0.0.0:47763, Address already in use
1388589853.006436       proxy   child   -       -       -       error
can't bind to 0.0.0.0:47762, Address already in use


Should I be worried about these errors? I mean, file extraction is working
fine, but maybe other service is not?


PS> Am I mixing two project here? BRO and OnionSecurity?


Pozdrawiam,
drumm


2014/1/1 Doug Burks <doug.burks at gmail.com>

> Hi drum,
>
> Start off with the following:
>
> - edit /opt/bro/share/bro/file-extraction/extract.bro
>
> - change the following line:
>     if ( ! f?$mime_type || f$mime_type != "application/x-dosexec" )
> to:
>     if ( ! f?$mime_type || f$mime_type != "application/jar" )
>
> - run the following:
> sudo broctl install
> sudo broctl restart
>
> Bro should now be extracting jar files to /nsm/bro/extracted/.
>
> Once you have that working, then you should be able to add in Shane's
> match_file_mime to the same script to allow you to extract multiple
> file types.
>
> On Wed, Jan 1, 2014 at 7:39 AM, drum <drummachina at tdhack.com> wrote:
> > Thanks Shane, could you please write step-by-step instruction where
> should I
> > put your code. I have no idea how to port it into my installation. Let's
> > assume I use 'plain' bro (no security onion) installed in /opt/bro. What
> is
> > the next step? BTW - I have these both files (extract.bro and main.bro):
> >
> > root at onion:~# ls -al /opt/bro/share/bro/base/files/extract/main.bro
> > /opt/bro/share/bro/file-extraction/extract.bro
> > -rw-r--r-- 1 root root 2126 Nov  7 18:27
> > /opt/bro/share/bro/base/files/extract/main.bro
> > -rw-r--r-- 1 root root  572 Jan  1 12:26
> > /opt/bro/share/bro/file-extraction/extract.bro
> >
> >
> >
> > Pozdrawiam,
> > drumm
> >
> >
> > 2013/12/30 Shane Filus <filus at psc.edu>
> >>
> >> On 12/29/13 2:25 PM, drum wrote:
> >> > Hello,
> >> >
> >> >
> >> > Is there a tutorial for version 2.2 of BRO? I'd like to understand how
> >> > can I write my own scripts to support extraction of verious files,
> >> > like jar. So far I tried adding "application/jar" (it was logged to
> >> > /nsm/bro/logs/current/files.
> >> > log) as mime type to /opt/bro/share/bro/file-extraction/extract.bro
> >> > file but it seems I have to do something else too as this change is
> >> > not capturing files to /nsm/bro/extracted/ directory.
> >> Hello,
> >>
> >>
> >> Check the second example under 'Adding Analysis'  for a start in file
> >> extraction.
> >> http://www.bro.org/sphinx/frameworks/file-analysis.html#adding-analysis
> >>
> >> Also, not sure how it maps to Sec Onion, but there is
> >> $PREFX/share/bro/base/files/extract/main.bro from a source install.
> >> Might be your 'extract.bro'? I don't that file name in either 2.1 or 2.2
> >> source trees.
> >>
> >> Used the code below to do something similar. There's probably a more
> >> elegant, or efficient solution, but it seems to working as expected,
> >> given the limited testing I've done.
> >>
> >>
> >> # define file extraction filters
> >> const match_file_source = /HTTP/ |
> >>               /IRC/ |
> >>               /IRC_DATA/ |
> >>               /FTP/ |
> >>               /FTP_DATA/ &redef;
> >>
> >> const match_file_mime =   /text\/x-perl/ |
> >>               /text\/x-msdos-batch/ |
> >>               /text\/x-java/ |
> >>               /application\/x-gzip/ |
> >>               /application\/x-bzip2/ |
> >>               /application\/x-dosexec/ |
> >>               /application\/zip/ |
> >>               /application\/jar/ |
> >>               /application\/x-tar/ |
> >>               /application\/x-archive/ |
> >>               /application\/mac-binhex40/ |
> >>               /application\/x-java-keystore/ |
> >>               /application\/x-java-jce-keystore/ |
> >>               /application\/x-executable/ |
> >>               /application\/javascript/ &redef;
> >>
> >> # add analyer to file_new event
> >> event file_new(f: fa_file)
> >>     {
> >>     if ( f?$mime_type &&
> >>         match_file_source in f$source &&
> >>         match_file_mime in f$mime_type )
> >>             Files::add_analyzer(f, Files::ANALYZER_EXTRACT);
> >>        }
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Shane
> >
> >
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
>
> --
> Doug Burks
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140101/754ca399/attachment.html 

From doug.burks at gmail.com  Wed Jan  1 13:20:33 2014
From: doug.burks at gmail.com (Doug Burks)
Date: Wed, 1 Jan 2014 16:20:33 -0500
Subject: [Bro] extract jar files from HTTP stream
In-Reply-To: <CAAzamUebCk0o-WWE=6Qi5JiG9fnKhT6FTiDHWWOxn6iu6100mA@mail.gmail.com>
References: <CAAzamUcuU=faUCsXHDEXgDsg-c6Ni4Seh+q1Uv9uZcU-5v32Yg@mail.gmail.com>
	<52C17674.6040500@psc.edu>
	<CAAzamUfPQxtqq_Ns36Eb0yX9PT2-PcSmcUwZZuGjC8E0Te-EWA@mail.gmail.com>
	<CAK8kjrB9SR8JhXw3OS3Ym4ncaOoXF7+mJC2kSqMQ3+PN6aELNQ@mail.gmail.com>
	<CAAzamUebCk0o-WWE=6Qi5JiG9fnKhT6FTiDHWWOxn6iu6100mA@mail.gmail.com>
Message-ID: <CAK8kjrB9s_01Qeo7H8V+TxanOmZreFzkXbbRAtsqPAmnDNjNKw@mail.gmail.com>

Check that your IP address is correct in /opt/bro/etc/node.cfg and
then run the following:
sudo broctl install
sudo reboot

If you continue to have issues, please start a new thread on the
Security Onion mailing list and we can troubleshoot further there.

Thanks,
Doug

On Wed, Jan 1, 2014 at 10:26 AM, drum <drummachina at tdhack.com> wrote:
> Thank you Doug, that worked. Actually I ended up with following (ugly)
> syntax:
>
> root at onion:~# cat /opt/bro/share/bro/file-extraction/extract.bro
> global ext_map: table[string] of string = {
>     ["application/x-dosexec"] = "exe",
>     ["text/plain"] = "txt",
>     ["image/jpeg"] = "jpg",
>     ["image/png"] = "png",
>     ["text/html"] = "html",
> } &default ="";
>
> event file_new(f: fa_file)
>     {
>     #if ( ! f?$mime_type || f$mime_type != "application/x-dosexec" )
>
>     if ( ! f?$mime_type || f$mime_type != "application/jar" )
>         return;
>
>     local ext = "";
>
>     if ( f?$mime_type )
>         ext = ext_map[f$mime_type];
>
>     local fname = fmt("/nsm/bro/extracted/%s-%s.%s", f$source, f$id, ext);
>     Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
> [$extract_filename=fname]);
>
>     }
>
> # define file extraction filters
> const match_file_source = /HTTP/ |
>               /IRC/ |
>               /IRC_DATA/ |
>               /FTP/ |
>               /FTP_DATA/ &redef;
>
> const match_file_mime =   /text\/x-perl/ |
>               /text\/x-msdos-batch/ |
>               /text\/x-java/ |
>               /application\/x-gzip/ |
>               /application\/x-bzip2/ |
>               /application\/x-dosexec/ |
>               /application\/zip/ |
>               /application\/jar/ |
>               /application\/x-tar/ |
>               /application\/x-archive/ |
>               /application\/mac-binhex40/ |
>               /application\/x-java-keystore/ |
>               /application\/x-java-jce-keystore/ |
>               /application\/x-executable/ |
>               /application\/javascript/ &redef;
>
> # add analyer to file_new event
> event file_new(f: fa_file)
>     {
>     if ( f?$mime_type &&
>         match_file_source in f$source &&
>         match_file_mime in f$mime_type )
>             Files::add_analyzer(f, Files::ANALYZER_EXTRACT);
>        }
>
> and I bet it can be written better.
>
> During this excersise I noticed that /nsm/bro/logs/current/files.log was not
> present. Found this in google:
> https://groups.google.com/forum/#!topic/security-onion/r4eZWOegvsY and
> followed suggestions. Indeed, /nsm/bro/logs/current/communication.log file
> contained:
>
> 1388589086.005591       manager child   -       -       -       error
> can't bind to 0.0.0.0:47761, Address already in use
>
> I had to use lsof command to check which process was it and:
>
> root at onion:/nsm/bro/logs/current# lsof -i:47761
> COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
> bro     12253 root    0u  IPv4 300348      0t0  TCP *:47761 (LISTEN)
> bro     12253 root    1u  IPv6 300349      0t0  TCP *:47761 (LISTEN)
>
> so I killed it. After doing service nsm restart everything seems to be
> working again (logs + file extraction. BTW - jar files are stored without
> extension "jar"). But I still can see errors in communication.log:
>
> root at onion:~# cat /nsm/bro/logs/current/communication.log |grep Address
> 1388589202.005024       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589204.006373       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589235.000845       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589233.001513       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589264.004692       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589266.005739       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589297.004983       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589295.005424       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589328.004598       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589326.005488       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589359.004987       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589357.004749       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589390.004760       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589388.004887       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589419.005759       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589421.005335       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589450.004988       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589452.005818       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589481.001524       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589483.001843       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589512.004547       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589514.004785       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589543.005441       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589545.004584       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589574.005125       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589576.005318       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589605.005628       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589607.004816       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589636.005317       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589638.005756       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589667.005455       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589669.005977       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589700.006115       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589698.004967       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589729.000811       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589731.012333       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589760.005435       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589762.005389       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589791.004834       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589793.005790       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589824.005289       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589822.004770       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
> 1388589855.005452       onion-eth1-1    child   -       -       -
> error   can't bind to 0.0.0.0:47763, Address already in use
> 1388589853.006436       proxy   child   -       -       -       error
> can't bind to 0.0.0.0:47762, Address already in use
>
>
> Should I be worried about these errors? I mean, file extraction is working
> fine, but maybe other service is not?
>
>
> PS> Am I mixing two project here? BRO and OnionSecurity?
>
>
> Pozdrawiam,
> drumm
>
>
> 2014/1/1 Doug Burks <doug.burks at gmail.com>
>>
>> Hi drum,
>>
>> Start off with the following:
>>
>> - edit /opt/bro/share/bro/file-extraction/extract.bro
>>
>> - change the following line:
>>     if ( ! f?$mime_type || f$mime_type != "application/x-dosexec" )
>> to:
>>     if ( ! f?$mime_type || f$mime_type != "application/jar" )
>>
>> - run the following:
>> sudo broctl install
>> sudo broctl restart
>>
>> Bro should now be extracting jar files to /nsm/bro/extracted/.
>>
>> Once you have that working, then you should be able to add in Shane's
>> match_file_mime to the same script to allow you to extract multiple
>> file types.
>>
>> On Wed, Jan 1, 2014 at 7:39 AM, drum <drummachina at tdhack.com> wrote:
>> > Thanks Shane, could you please write step-by-step instruction where
>> > should I
>> > put your code. I have no idea how to port it into my installation. Let's
>> > assume I use 'plain' bro (no security onion) installed in /opt/bro. What
>> > is
>> > the next step? BTW - I have these both files (extract.bro and main.bro):
>> >
>> > root at onion:~# ls -al /opt/bro/share/bro/base/files/extract/main.bro
>> > /opt/bro/share/bro/file-extraction/extract.bro
>> > -rw-r--r-- 1 root root 2126 Nov  7 18:27
>> > /opt/bro/share/bro/base/files/extract/main.bro
>> > -rw-r--r-- 1 root root  572 Jan  1 12:26
>> > /opt/bro/share/bro/file-extraction/extract.bro
>> >
>> >
>> >
>> > Pozdrawiam,
>> > drumm
>> >
>> >
>> > 2013/12/30 Shane Filus <filus at psc.edu>
>> >>
>> >> On 12/29/13 2:25 PM, drum wrote:
>> >> > Hello,
>> >> >
>> >> >
>> >> > Is there a tutorial for version 2.2 of BRO? I'd like to understand
>> >> > how
>> >> > can I write my own scripts to support extraction of verious files,
>> >> > like jar. So far I tried adding "application/jar" (it was logged to
>> >> > /nsm/bro/logs/current/files.
>> >> > log) as mime type to /opt/bro/share/bro/file-extraction/extract.bro
>> >> > file but it seems I have to do something else too as this change is
>> >> > not capturing files to /nsm/bro/extracted/ directory.
>> >> Hello,
>> >>
>> >>
>> >> Check the second example under 'Adding Analysis'  for a start in file
>> >> extraction.
>> >> http://www.bro.org/sphinx/frameworks/file-analysis.html#adding-analysis
>> >>
>> >> Also, not sure how it maps to Sec Onion, but there is
>> >> $PREFX/share/bro/base/files/extract/main.bro from a source install.
>> >> Might be your 'extract.bro'? I don't that file name in either 2.1 or
>> >> 2.2
>> >> source trees.
>> >>
>> >> Used the code below to do something similar. There's probably a more
>> >> elegant, or efficient solution, but it seems to working as expected,
>> >> given the limited testing I've done.
>> >>
>> >>
>> >> # define file extraction filters
>> >> const match_file_source = /HTTP/ |
>> >>               /IRC/ |
>> >>               /IRC_DATA/ |
>> >>               /FTP/ |
>> >>               /FTP_DATA/ &redef;
>> >>
>> >> const match_file_mime =   /text\/x-perl/ |
>> >>               /text\/x-msdos-batch/ |
>> >>               /text\/x-java/ |
>> >>               /application\/x-gzip/ |
>> >>               /application\/x-bzip2/ |
>> >>               /application\/x-dosexec/ |
>> >>               /application\/zip/ |
>> >>               /application\/jar/ |
>> >>               /application\/x-tar/ |
>> >>               /application\/x-archive/ |
>> >>               /application\/mac-binhex40/ |
>> >>               /application\/x-java-keystore/ |
>> >>               /application\/x-java-jce-keystore/ |
>> >>               /application\/x-executable/ |
>> >>               /application\/javascript/ &redef;
>> >>
>> >> # add analyer to file_new event
>> >> event file_new(f: fa_file)
>> >>     {
>> >>     if ( f?$mime_type &&
>> >>         match_file_source in f$source &&
>> >>         match_file_mime in f$mime_type )
>> >>             Files::add_analyzer(f, Files::ANALYZER_EXTRACT);
>> >>        }
>> >>
>> >>
>> >> Thanks,
>> >>
>> >>
>> >> Shane
>> >
>> >
>> >
>> > _______________________________________________
>> > Bro mailing list
>> > bro at bro-ids.org
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>>
>>
>> --
>> Doug Burks
>
>



-- 
Doug Burks


From logoyda at gmail.com  Thu Jan  2 07:02:37 2014
From: logoyda at gmail.com (Michael Logoyda)
Date: Thu, 2 Jan 2014 08:02:37 -0700
Subject: [Bro] extract jar files from HTTP stream
In-Reply-To: <CAK8kjrB9s_01Qeo7H8V+TxanOmZreFzkXbbRAtsqPAmnDNjNKw@mail.gmail.com>
References: <CAAzamUcuU=faUCsXHDEXgDsg-c6Ni4Seh+q1Uv9uZcU-5v32Yg@mail.gmail.com>
	<52C17674.6040500@psc.edu>
	<CAAzamUfPQxtqq_Ns36Eb0yX9PT2-PcSmcUwZZuGjC8E0Te-EWA@mail.gmail.com>
	<CAK8kjrB9SR8JhXw3OS3Ym4ncaOoXF7+mJC2kSqMQ3+PN6aELNQ@mail.gmail.com>
	<CAAzamUebCk0o-WWE=6Qi5JiG9fnKhT6FTiDHWWOxn6iu6100mA@mail.gmail.com>
	<CAK8kjrB9s_01Qeo7H8V+TxanOmZreFzkXbbRAtsqPAmnDNjNKw@mail.gmail.com>
Message-ID: <CAJwbkSaEKGrXR_WLbF9Jtv6wp+KgOTsZQ8S_uVntmJkEZcKMCw@mail.gmail.com>

Is there a way to name the extracted files based on what IP or domain
name they originated from? In the event file_new(f: fa_file) section, I'm
not able to access anything from f$conns to use for such naming. That would
make the extracted files much more useful.



On Wed, Jan 1, 2014 at 2:20 PM, Doug Burks <doug.burks at gmail.com> wrote:

> Check that your IP address is correct in /opt/bro/etc/node.cfg and
> then run the following:
> sudo broctl install
> sudo reboot
>
> If you continue to have issues, please start a new thread on the
> Security Onion mailing list and we can troubleshoot further there.
>
> Thanks,
> Doug
>
> On Wed, Jan 1, 2014 at 10:26 AM, drum <drummachina at tdhack.com> wrote:
> > Thank you Doug, that worked. Actually I ended up with following (ugly)
> > syntax:
> >
> > root at onion:~# cat /opt/bro/share/bro/file-extraction/extract.bro
> > global ext_map: table[string] of string = {
> >     ["application/x-dosexec"] = "exe",
> >     ["text/plain"] = "txt",
> >     ["image/jpeg"] = "jpg",
> >     ["image/png"] = "png",
> >     ["text/html"] = "html",
> > } &default ="";
> >
> > event file_new(f: fa_file)
> >     {
> >     #if ( ! f?$mime_type || f$mime_type != "application/x-dosexec" )
> >
> >     if ( ! f?$mime_type || f$mime_type != "application/jar" )
> >         return;
> >
> >     local ext = "";
> >
> >     if ( f?$mime_type )
> >         ext = ext_map[f$mime_type];
> >
> >     local fname = fmt("/nsm/bro/extracted/%s-%s.%s", f$source, f$id,
> ext);
> >     Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
> > [$extract_filename=fname]);
> >
> >     }
> >
> > # define file extraction filters
> > const match_file_source = /HTTP/ |
> >               /IRC/ |
> >               /IRC_DATA/ |
> >               /FTP/ |
> >               /FTP_DATA/ &redef;
> >
> > const match_file_mime =   /text\/x-perl/ |
> >               /text\/x-msdos-batch/ |
> >               /text\/x-java/ |
> >               /application\/x-gzip/ |
> >               /application\/x-bzip2/ |
> >               /application\/x-dosexec/ |
> >               /application\/zip/ |
> >               /application\/jar/ |
> >               /application\/x-tar/ |
> >               /application\/x-archive/ |
> >               /application\/mac-binhex40/ |
> >               /application\/x-java-keystore/ |
> >               /application\/x-java-jce-keystore/ |
> >               /application\/x-executable/ |
> >               /application\/javascript/ &redef;
> >
> > # add analyer to file_new event
> > event file_new(f: fa_file)
> >     {
> >     if ( f?$mime_type &&
> >         match_file_source in f$source &&
> >         match_file_mime in f$mime_type )
> >             Files::add_analyzer(f, Files::ANALYZER_EXTRACT);
> >        }
> >
> > and I bet it can be written better.
> >
> > During this excersise I noticed that /nsm/bro/logs/current/files.log was
> not
> > present. Found this in google:
> > https://groups.google.com/forum/#!topic/security-onion/r4eZWOegvsY and
> > followed suggestions. Indeed, /nsm/bro/logs/current/communication.log
> file
> > contained:
> >
> > 1388589086.005591       manager child   -       -       -       error
> > can't bind to 0.0.0.0:47761, Address already in use
> >
> > I had to use lsof command to check which process was it and:
> >
> > root at onion:/nsm/bro/logs/current# lsof -i:47761
> > COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
> > bro     12253 root    0u  IPv4 300348      0t0  TCP *:47761 (LISTEN)
> > bro     12253 root    1u  IPv6 300349      0t0  TCP *:47761 (LISTEN)
> >
> > so I killed it. After doing service nsm restart everything seems to be
> > working again (logs + file extraction. BTW - jar files are stored without
> > extension "jar"). But I still can see errors in communication.log:
> >
> > root at onion:~# cat /nsm/bro/logs/current/communication.log |grep Address
> > 1388589202.005024       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589204.006373       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589235.000845       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589233.001513       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589264.004692       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589266.005739       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589297.004983       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589295.005424       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589328.004598       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589326.005488       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589359.004987       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589357.004749       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589390.004760       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589388.004887       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589419.005759       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589421.005335       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589450.004988       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589452.005818       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589481.001524       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589483.001843       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589512.004547       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589514.004785       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589543.005441       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589545.004584       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589574.005125       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589576.005318       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589605.005628       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589607.004816       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589636.005317       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589638.005756       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589667.005455       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589669.005977       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589700.006115       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589698.004967       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589729.000811       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589731.012333       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589760.005435       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589762.005389       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589791.004834       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589793.005790       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589824.005289       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589822.004770       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> > 1388589855.005452       onion-eth1-1    child   -       -       -
> > error   can't bind to 0.0.0.0:47763, Address already in use
> > 1388589853.006436       proxy   child   -       -       -       error
> > can't bind to 0.0.0.0:47762, Address already in use
> >
> >
> > Should I be worried about these errors? I mean, file extraction is
> working
> > fine, but maybe other service is not?
> >
> >
> > PS> Am I mixing two project here? BRO and OnionSecurity?
> >
> >
> > Pozdrawiam,
> > drumm
> >
> >
> > 2014/1/1 Doug Burks <doug.burks at gmail.com>
> >>
> >> Hi drum,
> >>
> >> Start off with the following:
> >>
> >> - edit /opt/bro/share/bro/file-extraction/extract.bro
> >>
> >> - change the following line:
> >>     if ( ! f?$mime_type || f$mime_type != "application/x-dosexec" )
> >> to:
> >>     if ( ! f?$mime_type || f$mime_type != "application/jar" )
> >>
> >> - run the following:
> >> sudo broctl install
> >> sudo broctl restart
> >>
> >> Bro should now be extracting jar files to /nsm/bro/extracted/.
> >>
> >> Once you have that working, then you should be able to add in Shane's
> >> match_file_mime to the same script to allow you to extract multiple
> >> file types.
> >>
> >> On Wed, Jan 1, 2014 at 7:39 AM, drum <drummachina at tdhack.com> wrote:
> >> > Thanks Shane, could you please write step-by-step instruction where
> >> > should I
> >> > put your code. I have no idea how to port it into my installation.
> Let's
> >> > assume I use 'plain' bro (no security onion) installed in /opt/bro.
> What
> >> > is
> >> > the next step? BTW - I have these both files (extract.bro and
> main.bro):
> >> >
> >> > root at onion:~# ls -al /opt/bro/share/bro/base/files/extract/main.bro
> >> > /opt/bro/share/bro/file-extraction/extract.bro
> >> > -rw-r--r-- 1 root root 2126 Nov  7 18:27
> >> > /opt/bro/share/bro/base/files/extract/main.bro
> >> > -rw-r--r-- 1 root root  572 Jan  1 12:26
> >> > /opt/bro/share/bro/file-extraction/extract.bro
> >> >
> >> >
> >> >
> >> > Pozdrawiam,
> >> > drumm
> >> >
> >> >
> >> > 2013/12/30 Shane Filus <filus at psc.edu>
> >> >>
> >> >> On 12/29/13 2:25 PM, drum wrote:
> >> >> > Hello,
> >> >> >
> >> >> >
> >> >> > Is there a tutorial for version 2.2 of BRO? I'd like to understand
> >> >> > how
> >> >> > can I write my own scripts to support extraction of verious files,
> >> >> > like jar. So far I tried adding "application/jar" (it was logged to
> >> >> > /nsm/bro/logs/current/files.
> >> >> > log) as mime type to /opt/bro/share/bro/file-extraction/extract.bro
> >> >> > file but it seems I have to do something else too as this change is
> >> >> > not capturing files to /nsm/bro/extracted/ directory.
> >> >> Hello,
> >> >>
> >> >>
> >> >> Check the second example under 'Adding Analysis'  for a start in file
> >> >> extraction.
> >> >>
> http://www.bro.org/sphinx/frameworks/file-analysis.html#adding-analysis
> >> >>
> >> >> Also, not sure how it maps to Sec Onion, but there is
> >> >> $PREFX/share/bro/base/files/extract/main.bro from a source install.
> >> >> Might be your 'extract.bro'? I don't that file name in either 2.1 or
> >> >> 2.2
> >> >> source trees.
> >> >>
> >> >> Used the code below to do something similar. There's probably a more
> >> >> elegant, or efficient solution, but it seems to working as expected,
> >> >> given the limited testing I've done.
> >> >>
> >> >>
> >> >> # define file extraction filters
> >> >> const match_file_source = /HTTP/ |
> >> >>               /IRC/ |
> >> >>               /IRC_DATA/ |
> >> >>               /FTP/ |
> >> >>               /FTP_DATA/ &redef;
> >> >>
> >> >> const match_file_mime =   /text\/x-perl/ |
> >> >>               /text\/x-msdos-batch/ |
> >> >>               /text\/x-java/ |
> >> >>               /application\/x-gzip/ |
> >> >>               /application\/x-bzip2/ |
> >> >>               /application\/x-dosexec/ |
> >> >>               /application\/zip/ |
> >> >>               /application\/jar/ |
> >> >>               /application\/x-tar/ |
> >> >>               /application\/x-archive/ |
> >> >>               /application\/mac-binhex40/ |
> >> >>               /application\/x-java-keystore/ |
> >> >>               /application\/x-java-jce-keystore/ |
> >> >>               /application\/x-executable/ |
> >> >>               /application\/javascript/ &redef;
> >> >>
> >> >> # add analyer to file_new event
> >> >> event file_new(f: fa_file)
> >> >>     {
> >> >>     if ( f?$mime_type &&
> >> >>         match_file_source in f$source &&
> >> >>         match_file_mime in f$mime_type )
> >> >>             Files::add_analyzer(f, Files::ANALYZER_EXTRACT);
> >> >>        }
> >> >>
> >> >>
> >> >> Thanks,
> >> >>
> >> >>
> >> >> Shane
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > Bro mailing list
> >> > bro at bro-ids.org
> >> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >>
> >>
> >>
> >> --
> >> Doug Burks
> >
> >
>
>
>
> --
> Doug Burks
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140102/df095165/attachment.html 

From seth at icir.org  Thu Jan  2 08:22:29 2014
From: seth at icir.org (Seth Hall)
Date: Thu, 2 Jan 2014 11:22:29 -0500
Subject: [Bro] extract jar files from HTTP stream
In-Reply-To: <CAJwbkSaEKGrXR_WLbF9Jtv6wp+KgOTsZQ8S_uVntmJkEZcKMCw@mail.gmail.com>
References: <CAAzamUcuU=faUCsXHDEXgDsg-c6Ni4Seh+q1Uv9uZcU-5v32Yg@mail.gmail.com>
	<52C17674.6040500@psc.edu>
	<CAAzamUfPQxtqq_Ns36Eb0yX9PT2-PcSmcUwZZuGjC8E0Te-EWA@mail.gmail.com>
	<CAK8kjrB9SR8JhXw3OS3Ym4ncaOoXF7+mJC2kSqMQ3+PN6aELNQ@mail.gmail.com>
	<CAAzamUebCk0o-WWE=6Qi5JiG9fnKhT6FTiDHWWOxn6iu6100mA@mail.gmail.com>
	<CAK8kjrB9s_01Qeo7H8V+TxanOmZreFzkXbbRAtsqPAmnDNjNKw@mail.gmail.com>
	<CAJwbkSaEKGrXR_WLbF9Jtv6wp+KgOTsZQ8S_uVntmJkEZcKMCw@mail.gmail.com>
Message-ID: <7CDD1AFD-FE3B-4A31-BE78-98824203074E@icir.org>


On Jan 2, 2014, at 10:02 AM, Michael Logoyda <logoyda at gmail.com> wrote:

> Is there a way to name the extracted files based on what IP or domain name they originated from? In the event file_new(f: fa_file) section, I'm not able to access anything from f$conns to use for such naming. That would make the extracted files much more useful. 

What do you mean that you aren't able to access anything from f$conns?

Generally, giving extracted files names like that is complicated because the file handling in Bro is separated from everything else.  There are some strategic points where they tie together, but generally you have to be careful.

Since the best way is probably through concrete examples, I'll give one and we'll see if it sticks.  If you want to name extracted files like HTTP_1.2.3.4:12345-5.6.7.8:80.resp.dat you can do this?

	https://gist.github.com/sethhall/8221401

This will only extract files over HTTP with these special file names.  You can modify that script if you want it to behave differently.  One thing people ask a lot is if you can extract files and name them by their SHA1 or MD5 hash.  Generally this is possible but it's something that would need to be done when the file is completely extracted because you don't know the file hash at the beginning of the file but you need to give a filename to start writing the file into.  In the normal case you would extract the file and then move it into it's new filename (hopefully on the same file system).

Actually, I'll do one more small modification to the script to show you how to add the domain?

	https://gist.github.com/sethhall/8221692

One final thing to notice is that I've made both of these scripts only use the "special" filename for cases where a file is being received over HTTP.  If the client sends data over HTTP or another protocol things will revert to the default filename.

There are also some small considerations being ignored in this example like single files transferred over multiple connections (which is possible in Bro).

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140102/082edf41/attachment.bin 

From seth at icir.org  Thu Jan  2 09:21:10 2014
From: seth at icir.org (Seth Hall)
Date: Thu, 2 Jan 2014 12:21:10 -0500
Subject: [Bro] count connection bytes
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF82041801@DRTEXW238VM.na.d-rco.com>
References: <BDF3793B557F624699D4C68A3B8FCF82041801@DRTEXW238VM.na.d-rco.com>
Message-ID: <93E564A7-DA18-45C9-B0EE-F7DEFBBB4C18@icir.org>


On Dec 23, 2013, at 4:26 PM, "Kellogg, Brian D (OLN)" <bkellogg at dresser-rand.com> wrote:

> event connection_finished(c:connection)
>         {
>         print c$orig$num_bytes_ip;
>         print c$resp$num_bytes_ip;
>         }
> 
> I'm probably missing something obvious but it is escaping me.  thanks

You probably want to use the connection_state_remove event instead as it indicates when a connection is expunged from memory.  connection_finished has some extra context to it that you may not care about.

Also, the num_bytes_ip field is a per-packet field and includes the size of the IP header on down (tcp/udp + payload typically).  If you are looking for content bytes you will want c$orig$size which will show you the size of the reassembled TCP contents in the case of TCP.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140102/297a1ab3/attachment.bin 

From bkellogg at dresser-rand.com  Thu Jan  2 15:13:49 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Thu, 2 Jan 2014 23:13:49 +0000
Subject: [Bro] script working from cmd line but not from local.bro
Message-ID: <BDF3793B557F624699D4C68A3B8FCF82044E47@DRTEXW238VM.na.d-rco.com>

I have a script I've been writing for a couple weeks that looks at every connection's total bytes.  If the total bytes when the connection is removed from memory is over X bytes then raise a Bro notice.  I have a global variable structure defined to keep track of internal hosts that have uploaded more than X bytes in a connection.

The script works find when running it from the command line.  The notice.log is created with the custom notice entry.  However when I run load this into Bro via the local.bro file the notice is never raised although Bro seems to start and run as it should.  I hope I'm missing something really simply as that has been the case in most of the roadblocks I've run into so far.  I'm using Elsa in SO to query for the custom Notice entry by simply querying for all notices.

I'm not exporting anything in the script as I do not need to reference anything in it from any other script; I think I'm understanding the export directive correctly.

Thanks for any help.

Thank you,
Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140102/45836bab/attachment.html 

From seth at icir.org  Fri Jan  3 06:07:40 2014
From: seth at icir.org (Seth Hall)
Date: Fri, 3 Jan 2014 09:07:40 -0500
Subject: [Bro] script working from cmd line but not from local.bro
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF82044E47@DRTEXW238VM.na.d-rco.com>
References: <BDF3793B557F624699D4C68A3B8FCF82044E47@DRTEXW238VM.na.d-rco.com>
Message-ID: <1AD53FB1-A426-4D90-AA28-1BABF36DBDDE@icir.org>


On Jan 2, 2014, at 6:13 PM, "Kellogg, Brian D (OLN)" <bkellogg at dresser-rand.com> wrote:

> I have a script I've been writing for a couple weeks that looks at every connection's total bytes.  If the total bytes when the connection is removed from memory is over X bytes then raise a Bro notice.  I have a global variable structure defined to keep track of internal hosts that have uploaded more than X bytes in a connection.

Please post the script so we can review it.

Thanks,
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140103/5968e6a8/attachment.bin 

From bkellogg at dresser-rand.com  Fri Jan  3 06:29:34 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Fri, 3 Jan 2014 14:29:34 +0000
Subject: [Bro] script working from cmd line but not from local.bro
In-Reply-To: <1AD53FB1-A426-4D90-AA28-1BABF36DBDDE@icir.org>
References: <BDF3793B557F624699D4C68A3B8FCF82044E47@DRTEXW238VM.na.d-rco.com>
	<1AD53FB1-A426-4D90-AA28-1BABF36DBDDE@icir.org>
Message-ID: <BDF3793B557F624699D4C68A3B8FCF820451C2@DRTEXW238VM.na.d-rco.com>

susTx.bro is the simplified version of the script that works.

trackOutTx.bro is the one that doesn't.

Another thing I'm seeing is I cannot find these notices in Elsa once the notice.log has been rotated by SO.  I'm sure I'm just not understanding something as I'm quite new to SO, Bro, and Elsa.  Is there something else I have to do to ensure these notices show up in the Elsa archive?  Is there a delay of a several hours before they show up in Elsa?


Thank you,
Brian Kellogg
Security Analyst; IT Governance, Risk, and Compliance
500 Paul Clark Drive, Olean,  NY 14760
T: (716) 375-3186 | F: (716) 375-3557

-----Original Message-----
From: Seth Hall [mailto:seth at icir.org] 
Sent: Friday, January 03, 2014 9:08 AM
To: Kellogg, Brian D (OLN)
Cc: bro at bro.org
Subject: Re: [Bro] script working from cmd line but not from local.bro


On Jan 2, 2014, at 6:13 PM, "Kellogg, Brian D (OLN)" <bkellogg at dresser-rand.com> wrote:

> I have a script I've been writing for a couple weeks that looks at every connection's total bytes.  If the total bytes when the connection is removed from memory is over X bytes then raise a Bro notice.  I have a global variable structure defined to keep track of internal hosts that have uploaded more than X bytes in a connection.

Please post the script so we can review it.

Thanks,
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: susTx.bro
Type: application/octet-stream
Size: 1253 bytes
Desc: susTx.bro
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140103/19877b54/attachment.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: trackOutTx.bro
Type: application/octet-stream
Size: 4869 bytes
Desc: trackOutTx.bro
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140103/19877b54/attachment-0001.obj 

From doug.burks at gmail.com  Fri Jan  3 07:25:03 2014
From: doug.burks at gmail.com (Doug Burks)
Date: Fri, 3 Jan 2014 10:25:03 -0500
Subject: [Bro] script working from cmd line but not from local.bro
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF820451C2@DRTEXW238VM.na.d-rco.com>
References: <BDF3793B557F624699D4C68A3B8FCF82044E47@DRTEXW238VM.na.d-rco.com>
	<1AD53FB1-A426-4D90-AA28-1BABF36DBDDE@icir.org>
	<BDF3793B557F624699D4C68A3B8FCF820451C2@DRTEXW238VM.na.d-rco.com>
Message-ID: <CAK8kjrAcvXjLiMg5Gqe82gQ7AO7iCs3x1rQBXkWRNgSbXBrF4Q@mail.gmail.com>

Hi Brian,

While troubleshooting the Bro scripts with Seth, take ELSA out of the
picture and just look at the raw Bro notice.log in
/nsm/bro/logs/current/.  Once you have the scripts working correctly,
then we can help you with any ELSA issues over on the Security Onion
mailing list.

On Fri, Jan 3, 2014 at 9:29 AM, Kellogg, Brian D (OLN)
<bkellogg at dresser-rand.com> wrote:
> susTx.bro is the simplified version of the script that works.
>
> trackOutTx.bro is the one that doesn't.
>
> Another thing I'm seeing is I cannot find these notices in Elsa once the notice.log has been rotated by SO.  I'm sure I'm just not understanding something as I'm quite new to SO, Bro, and Elsa.  Is there something else I have to do to ensure these notices show up in the Elsa archive?  Is there a delay of a several hours before they show up in Elsa?
>
>
> Thank you,
> Brian Kellogg
> Security Analyst; IT Governance, Risk, and Compliance
> 500 Paul Clark Drive, Olean,  NY 14760
> T: (716) 375-3186 | F: (716) 375-3557
>
> -----Original Message-----
> From: Seth Hall [mailto:seth at icir.org]
> Sent: Friday, January 03, 2014 9:08 AM
> To: Kellogg, Brian D (OLN)
> Cc: bro at bro.org
> Subject: Re: [Bro] script working from cmd line but not from local.bro
>
>
> On Jan 2, 2014, at 6:13 PM, "Kellogg, Brian D (OLN)" <bkellogg at dresser-rand.com> wrote:
>
>> I have a script I've been writing for a couple weeks that looks at every connection's total bytes.  If the total bytes when the connection is removed from memory is over X bytes then raise a Bro notice.  I have a global variable structure defined to keep track of internal hosts that have uploaded more than X bytes in a connection.
>
> Please post the script so we can review it.
>
> Thanks,
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



-- 
Doug Burks



From JAzoff at albany.edu  Fri Jan  3 09:40:27 2014
From: JAzoff at albany.edu (Justin Azoff)
Date: Fri, 3 Jan 2014 12:40:27 -0500
Subject: [Bro] script working from cmd line but not from local.bro
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF820451C2@DRTEXW238VM.na.d-rco.com>
References: <BDF3793B557F624699D4C68A3B8FCF82044E47@DRTEXW238VM.na.d-rco.com>
	<1AD53FB1-A426-4D90-AA28-1BABF36DBDDE@icir.org>
	<BDF3793B557F624699D4C68A3B8FCF820451C2@DRTEXW238VM.na.d-rco.com>
Message-ID: <20140103174027.GW12701@datacomm.albany.edu>

On Fri, Jan 03, 2014 at 02:29:34PM +0000, Kellogg, Brian D (OLN) wrote:
> susTx.bro is the simplified version of the script that works.
> 
> trackOutTx.bro is the one that doesn't.

First thing to do would be to check the reporter.log for errors.

For performance/clarity reasons you should change this:

    for (x in psubs) {  
            if (c$id$orig_h in psubs[x]) {  
                    # check to see if dest ip is not an internal IP  
                    for (y in psubs) {  
                            if (c$id$resp_h in psubs[y]) {  
                                    notFound = F;  
                                    break;  
                            }  
                    }  
                    # dest IP was not an internal IP so lets do this  


to something like

    # check to see if orig IP is an internal IP
    if(!Site::is_local_addr(c$id$orig_h))
        return;

    # check to see if dest ip is not an internal IP  
    if(Site::is_local_addr(c$id$resp_h))
        return;

or Site::is_private_addr, but you probably want is_local_addr
Bro natively supports sets of addresses so you don't need to loop.

-- 
-- Justin Azoff
-- Network Security & Performance Analyst


From bkellogg at dresser-rand.com  Fri Jan  3 12:04:49 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Fri, 3 Jan 2014 20:04:49 +0000
Subject: [Bro] script working from cmd line but not from local.bro
In-Reply-To: <1AD53FB1-A426-4D90-AA28-1BABF36DBDDE@icir.org>
References: <BDF3793B557F624699D4C68A3B8FCF82044E47@DRTEXW238VM.na.d-rco.com>
	<1AD53FB1-A426-4D90-AA28-1BABF36DBDDE@icir.org>
Message-ID: <BDF3793B557F624699D4C68A3B8FCF8204548D@DRTEXW238VM.na.d-rco.com>

I believe some of the problem is my formatting.  I added some stuff back into the simpler script along with the changes that Justin recommended and had the same problem.  I then went in and moved {} for the "if else" statements to their own lines like I see in the scripts that ship with Bro and then the script works.  I haven't tried the entire more complex script yet, but I'll let this run and add to it over the weekend.  Thanks for all the help and input.

Thank you,
Brian Kellogg
Security Analyst; IT Governance, Risk, and Compliance
500 Paul Clark Drive, Olean,  NY 14760
T: (716) 375-3186 | F: (716) 375-3557


-----Original Message-----
From: Seth Hall [mailto:seth at icir.org] 
Sent: Friday, January 03, 2014 9:08 AM
To: Kellogg, Brian D (OLN)
Cc: bro at bro.org
Subject: Re: [Bro] script working from cmd line but not from local.bro


On Jan 2, 2014, at 6:13 PM, "Kellogg, Brian D (OLN)" <bkellogg at dresser-rand.com> wrote:

> I have a script I've been writing for a couple weeks that looks at every connection's total bytes.  If the total bytes when the connection is removed from memory is over X bytes then raise a Bro notice.  I have a global variable structure defined to keep track of internal hosts that have uploaded more than X bytes in a connection.

Please post the script so we can review it.

Thanks,
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/




From david.gugelmann at tik.ee.ethz.ch  Sun Jan  5 11:37:03 2014
From: david.gugelmann at tik.ee.ethz.ch (David Gugelmann)
Date: Sun, 05 Jan 2014 20:37:03 +0100
Subject: [Bro]  unreported packet loss
Message-ID: <52C9B45F.3010505@tik.ee.ethz.ch>

Hello everybody,

I am quite new to bro, so I am not sure whether I am missing something.
It seems to me that bro (v2.1 and v2.2) does in some cases not report
packet loss.

I discovered this by comparing resp_bytes, resp_ip_bytes and
missed_bytes from conn.log. I found several TCP streams, for which
resp_ip_bytes < resp_bytes but missed_bytes is 0, that is, there are
more TCP-bytes than IP-bytes but at the same time no packet losses,
which seemed strange.
Analyzing the corresponding TCP sequence numbers more in detail, I found
that this seems to be caused by packet loss that is not reflected in
bro's missing_bytes field. Also capture_loss.log did not show any loss.

You can find two example TCP streams, bro's output and Wireshark screen
shots here:
http://people.ee.ethz.ch/~gugdavid/bro_missed_bytes.zip
(Note: This is no real user traffic, these traffic samples have been
automatically generated in a testbed using mechanized Firefox instances.)

Am I missing something or did anybody encounter something similar?

Thank you,
David


From jsiwek at illinois.edu  Mon Jan  6 11:29:01 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Mon, 6 Jan 2014 19:29:01 +0000
Subject: [Bro] Disable Base script
In-Reply-To: <BLU172-W41C532D64AF2CA07A60F8AB4C00@phx.gbl>
References: <BLU172-W41C532D64AF2CA07A60F8AB4C00@phx.gbl>
Message-ID: <852FA86F-C751-4F4D-859F-4A1E3BB16193@illinois.edu>


On Dec 24, 2013, at 12:50 PM, Ward Sladek <wsladekjr at hotmail.com> wrote:

> What is the best practice for disabling a Base script?  For example, I would like to disable syslog monitoring all together.  I have included the following in my local.bro:
> 
> event bro_init()
>     {
>     Log::disable_stream(Syslog::LOG);
>     }
> 
> 
> This disables the logging of syslog messages, but does it prevent Bro from loading the base/protocols/syslog scripts?  If not, what is the best practice for doing so?  I'm trying to tune/tweak bro for best performance.

For command-line usage, `bro -b` or `bro ?bare-mode` prevents base/init-default.bro and the scripts it references from being loaded by default.  From there, you can pick and choose freely.

For BroControl usage, I presume that setting ?BroArgs = -b? in broctl.cfg and pruning site/local.bro as desired would work.

- Jon


From schworer at gmail.com  Thu Jan  9 14:36:24 2014
From: schworer at gmail.com (Andy Schworer)
Date: Thu, 9 Jan 2014 14:36:24 -0800
Subject: [Bro] Example node.cfg for Multi-Interface collection on one box
Message-ID: <CANQy5G2n8cc=w=VeeuO8O9GCC99WavkqJtFv5C6xT7GyPO79VQ@mail.gmail.com>

I'm having trouble finding complete documentation on how to configure a bro
instance for monitoring multiple interfaces on the same host.

[bro]

type=standalone

host=localhost

interface=eth1


What needs to change to monitor eth1 & eth2?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140109/dcb4218b/attachment.html 

From jlay at slave-tothe-box.net  Thu Jan  9 14:51:09 2014
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 09 Jan 2014 15:51:09 -0700
Subject: [Bro] Example node.cfg for Multi-Interface collection on one box
In-Reply-To: <CANQy5G2n8cc=w=VeeuO8O9GCC99WavkqJtFv5C6xT7GyPO79VQ@mail.gmail.com>
References: <CANQy5G2n8cc=w=VeeuO8O9GCC99WavkqJtFv5C6xT7GyPO79VQ@mail.gmail.com>
Message-ID: <9df45100819a901214c3729b77162a84@localhost>

On 2014-01-09 15:36, Andy Schworer wrote:
> Im having trouble finding complete documentation on how to configure 
> a
> bro instance for monitoring multiple interfaces on the same host.
>
> [bro]
>
> type=standalone
>
> host=localhost
>
> interface=eth1
>
> What needs to change to monitor eth1 & eth2?

Assuming you're using broctl, add:

broargs = -i eth2

to your broctl.cfg file

James


From charles.fair at mac.com  Fri Jan 10 05:32:54 2014
From: charles.fair at mac.com (Charles A. Fair)
Date: Fri, 10 Jan 2014 07:32:54 -0600
Subject: [Bro] Example node.cfg for Multi-Interface collection on one box
In-Reply-To: <CANQy5G2n8cc=w=VeeuO8O9GCC99WavkqJtFv5C6xT7GyPO79VQ@mail.gmail.com>
References: <CANQy5G2n8cc=w=VeeuO8O9GCC99WavkqJtFv5C6xT7GyPO79VQ@mail.gmail.com>
Message-ID: <58E07AB6-FFDB-4C96-B642-D7E6F23A06A9@mac.com>

On Jan 9, 2014, at 4:36 PM, Andy Schworer <schworer at gmail.com> wrote:

> I'm having trouble finding complete documentation on how to configure a bro instance for monitoring multiple interfaces on the same host.
> [bro]
> 
> type=standalone
> 
> host=localhost
> 
> interface=eth1
> 
> What needs to change to monitor eth1 & eth2?
> 
Advanced Setup will prompt you to configure as may interfaces as you wish.  

Chuck

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140110/2269671d/attachment.html 

From charles.fair at mac.com  Fri Jan 10 05:34:41 2014
From: charles.fair at mac.com (Charles A. Fair)
Date: Fri, 10 Jan 2014 07:34:41 -0600
Subject: [Bro] Example node.cfg for Multi-Interface collection on one box
In-Reply-To: <CANQy5G2n8cc=w=VeeuO8O9GCC99WavkqJtFv5C6xT7GyPO79VQ@mail.gmail.com>
References: <CANQy5G2n8cc=w=VeeuO8O9GCC99WavkqJtFv5C6xT7GyPO79VQ@mail.gmail.com>
Message-ID: <1B9ABB68-80A4-4874-A7F7-786B83C64964@mac.com>

On Jan 9, 2014, at 4:36 PM, Andy Schworer <schworer at gmail.com> wrote:

> I'm having trouble finding complete documentation on how to configure a bro instance for monitoring multiple interfaces on the same host.
> [bro]
> type=standalone
> 
> host=localhost
> 
> interface=eth1
> 
> What needs to change to monitor eth1 & eth2?
> 
> 
Disregard my advice... that was specifically for Security Onion.  

Chuck

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140110/c5855ee2/attachment.html 

From kingzyycn at gmail.com  Tue Jan 14 09:46:13 2014
From: kingzyycn at gmail.com (John Zhang)
Date: Wed, 15 Jan 2014 01:46:13 +0800
Subject: [Bro] complete analysis system for detecting malware
Message-ID: <CALD8BLRUdUGHMsC0RXy9XCtnPbLFv_1sF3R+FNohR2zeeJANTA@mail.gmail.com>

Hi all,

Actually I am planning one complete analysis system(long term) for
detecting and tracing malware and other threats, it can do:
1, live capture full-content network(up to several GBs)
2, and extract files and contents from traffic, specially these contents in
http, ftp, email traffic
3, and send these contents to local sandbox, or to remote sandbox service,
for checking them; or check them against external threat intelligence.

Could you help recommend some tools for the above jobs?

I do need the experience, suggestion and comment from you all.

Thank you !

Regards,
John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140115/ddeecf8d/attachment.html 

From taosecurity at gmail.com  Tue Jan 14 09:56:49 2014
From: taosecurity at gmail.com (Richard Bejtlich)
Date: Tue, 14 Jan 2014 12:56:49 -0500
Subject: [Bro] complete analysis system for detecting malware
In-Reply-To: <CALD8BLRUdUGHMsC0RXy9XCtnPbLFv_1sF3R+FNohR2zeeJANTA@mail.gmail.com>
References: <CALD8BLRUdUGHMsC0RXy9XCtnPbLFv_1sF3R+FNohR2zeeJANTA@mail.gmail.com>
Message-ID: <CA+oHO0DiAGPLkBQXwt7eu5mj4LTF=iJcHE_vAW0xcvMDyMD4kw@mail.gmail.com>

FireEye?

Sorry, I couldn't resist...

Richard

On Tue, Jan 14, 2014 at 12:46 PM, John Zhang <kingzyycn at gmail.com> wrote:
> Hi all,
>
> Actually I am planning one complete analysis system(long term) for detecting
> and tracing malware and other threats, it can do:
> 1, live capture full-content network(up to several GBs)
> 2, and extract files and contents from traffic, specially these contents in
> http, ftp, email traffic
> 3, and send these contents to local sandbox, or to remote sandbox service,
> for checking them; or check them against external threat intelligence.
>
> Could you help recommend some tools for the above jobs?
>
> I do need the experience, suggestion and comment from you all.
>
> Thank you !
>
> Regards,
> John
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From kingzyycn at gmail.com  Tue Jan 14 10:03:32 2014
From: kingzyycn at gmail.com (John Zhang)
Date: Wed, 15 Jan 2014 02:03:32 +0800
Subject: [Bro] complete analysis system for detecting malware
In-Reply-To: <CA+oHO0DiAGPLkBQXwt7eu5mj4LTF=iJcHE_vAW0xcvMDyMD4kw@mail.gmail.com>
References: <CALD8BLRUdUGHMsC0RXy9XCtnPbLFv_1sF3R+FNohR2zeeJANTA@mail.gmail.com>
	<CA+oHO0DiAGPLkBQXwt7eu5mj4LTF=iJcHE_vAW0xcvMDyMD4kw@mail.gmail.com>
Message-ID: <CALD8BLRd19n2-fvtuKsvHowCytBVNUMkM8pHSpryTgsb8CdFdQ@mail.gmail.com>

Hi Richard,

similar, but hope using open source tools. Any comment?

BTW, I love your APT1 report.

Thanks!

John


2014/1/15 Richard Bejtlich <taosecurity at gmail.com>

> FireEye?
>
> Sorry, I couldn't resist...
>
> Richard
>
> On Tue, Jan 14, 2014 at 12:46 PM, John Zhang <kingzyycn at gmail.com> wrote:
> > Hi all,
> >
> > Actually I am planning one complete analysis system(long term) for
> detecting
> > and tracing malware and other threats, it can do:
> > 1, live capture full-content network(up to several GBs)
> > 2, and extract files and contents from traffic, specially these contents
> in
> > http, ftp, email traffic
> > 3, and send these contents to local sandbox, or to remote sandbox
> service,
> > for checking them; or check them against external threat intelligence.
> >
> > Could you help recommend some tools for the above jobs?
> >
> > I do need the experience, suggestion and comment from you all.
> >
> > Thank you !
> >
> > Regards,
> > John
> >
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140115/60d8e2b9/attachment.html 

From dpearso at sandia.gov  Wed Jan 15 13:10:28 2014
From: dpearso at sandia.gov (Pearson, David)
Date: Wed, 15 Jan 2014 21:10:28 +0000
Subject: [Bro] SMB Event Prototype Issue
Message-ID: <C41581D1AA533F48AB7F9E17CE26D6DF529392E9@EXMB02.srn.sandia.gov>

Hello all,

I've been porting the SMB script over to Bro 2.x, but I seem to have run into a problem with one of the event prototypes. When trying to work with the event smb_com_tree_connect_andx (with prototype event (c: connection, hdr: smb_hdr, path: string, service: string)), I am unable to correctly parse the path argument. Using the SANS 2013 Holiday PCAP as an example:

david at david-sec-onion:~/Desktop/sans_analysis$ bro -C -r sansholidayhack2013.pcap smb.bro | more
[flags=0, password=\0, path=\\10.25.22.58\IPC$, service=?????]

To me, it looks like the path field might actually be a set, though I'm relatively new to Bro. Does anybody have thoughts regarding this?

Thanks,
David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140115/f84edaa4/attachment.html 

From dpearso at sandia.gov  Wed Jan 15 15:08:03 2014
From: dpearso at sandia.gov (Pearson, David)
Date: Wed, 15 Jan 2014 23:08:03 +0000
Subject: [Bro] SMB Event Prototype Issue
Message-ID: <C41581D1AA533F48AB7F9E17CE26D6DF529393A9@EXMB02.srn.sandia.gov>

Hello again,

You can ignore my last question; I found Seth's topic github branch on SMB.

Thanks,
David


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140115/fa282430/attachment.html 

From ginsko3 at gmail.com  Thu Jan 16 07:12:20 2014
From: ginsko3 at gmail.com (George Insko)
Date: Thu, 16 Jan 2014 10:12:20 -0500
Subject: [Bro] BPF?
Message-ID: <CAEk_EiJBL8YJaMdNFvYuo2-m7KEuz9Mum3zmJb33DbJNesYJQw@mail.gmail.com>

Hi all,

I need to block all SSL traffic going to my Bro box. I was going to
use BPF  to accomplish this
task using a zero source address and a port number. So something like this:

#Nothing from src host to dst port
!(src host 0.0.0.0/0 && dst port 443) &&

Does that make sense and will it work? Do you all have any other ways to
permanently filter traffic?
-- 
*George Insko*
Email:    ginsko3 at gmail.com
Twitter: @ginsko3
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140116/b46b5966/attachment.html 

From seth at icir.org  Thu Jan 16 07:33:26 2014
From: seth at icir.org (Seth Hall)
Date: Thu, 16 Jan 2014 10:33:26 -0500
Subject: [Bro] BPF?
In-Reply-To: <CAEk_EiJBL8YJaMdNFvYuo2-m7KEuz9Mum3zmJb33DbJNesYJQw@mail.gmail.com>
References: <CAEk_EiJBL8YJaMdNFvYuo2-m7KEuz9Mum3zmJb33DbJNesYJQw@mail.gmail.com>
Message-ID: <37B434B4-BBDA-4B43-A389-7D7FB2A9D69C@icir.org>


On Jan 16, 2014, at 10:12 AM, George Insko <ginsko3 at gmail.com> wrote:

> #Nothing from src host to dst port
> !(src host 0.0.0.0/0 && dst port 443) &&
> Does that make sense and will it work? Do you all have any other ways to permanently filter traffic? 

I think you meant to do?

(not src port 443 and not dst port 443)

  .Seth


--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140116/2449e59d/attachment.bin 

From ginsko3 at gmail.com  Thu Jan 16 07:39:23 2014
From: ginsko3 at gmail.com (George Insko)
Date: Thu, 16 Jan 2014 10:39:23 -0500
Subject: [Bro] BPF?
In-Reply-To: <37B434B4-BBDA-4B43-A389-7D7FB2A9D69C@icir.org>
References: <CAEk_EiJBL8YJaMdNFvYuo2-m7KEuz9Mum3zmJb33DbJNesYJQw@mail.gmail.com>
	<37B434B4-BBDA-4B43-A389-7D7FB2A9D69C@icir.org>
Message-ID: <CAEk_Ei+wkaiYPiX+SroN8oAPWozgQeCS8PRkXE9BcyZw0bOV6g@mail.gmail.com>

Good call. Thanks.


On Thu, Jan 16, 2014 at 10:33 AM, Seth Hall <seth at icir.org> wrote:

>
> On Jan 16, 2014, at 10:12 AM, George Insko <ginsko3 at gmail.com> wrote:
>
> > #Nothing from src host to dst port
> > !(src host 0.0.0.0/0 && dst port 443) &&
> > Does that make sense and will it work? Do you all have any other ways to
> permanently filter traffic?
>
> I think you meant to do?
>
> (not src port 443 and not dst port 443)
>
>   .Seth
>
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>


-- 
*George Insko*
Email:    ginsko3 at gmail.com
Twitter: @ginsko3
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140116/f79456be/attachment.html 

From mike.patterson at uwaterloo.ca  Thu Jan 16 07:40:05 2014
From: mike.patterson at uwaterloo.ca (Mike Patterson)
Date: Thu, 16 Jan 2014 10:40:05 -0500
Subject: [Bro] BPF?
In-Reply-To: <CAEk_Ei+wkaiYPiX+SroN8oAPWozgQeCS8PRkXE9BcyZw0bOV6g@mail.gmail.com>
References: <CAEk_EiJBL8YJaMdNFvYuo2-m7KEuz9Mum3zmJb33DbJNesYJQw@mail.gmail.com>
	<37B434B4-BBDA-4B43-A389-7D7FB2A9D69C@icir.org>
	<CAEk_Ei+wkaiYPiX+SroN8oAPWozgQeCS8PRkXE9BcyZw0bOV6g@mail.gmail.com>
Message-ID: <429B7867-42A9-425A-A157-6E739014F56F@uwaterloo.ca>

If you?re anxious to avoid any SSL traffic based on port exclusions, you might consider other well-known ports - 587, 465, etc.

Mike

On Jan 16, 2014, at 10:39 AM, George Insko <ginsko3 at gmail.com> wrote:

> Good call. Thanks. 
> 
> 
> On Thu, Jan 16, 2014 at 10:33 AM, Seth Hall <seth at icir.org> wrote:
> 
> On Jan 16, 2014, at 10:12 AM, George Insko <ginsko3 at gmail.com> wrote:
> 
> > #Nothing from src host to dst port
> > !(src host 0.0.0.0/0 && dst port 443) &&
> > Does that make sense and will it work? Do you all have any other ways to permanently filter traffic?
> 
> I think you meant to do?
> 
> (not src port 443 and not dst port 443)
> 
>   .Seth
> 
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
> 
> 
> 
> 
> -- 
> George Insko
> Email:    ginsko3 at gmail.com
> Twitter: @ginsko3
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro




From vladg at cmu.edu  Thu Jan 16 07:53:43 2014
From: vladg at cmu.edu (Vlad Grigorescu)
Date: Thu, 16 Jan 2014 15:53:43 +0000
Subject: [Bro] BPF?
In-Reply-To: <27901_1389886916_s0GFftMV005295_429B7867-42A9-425A-A157-6E739014F56F@uwaterloo.ca>
References: <CAEk_EiJBL8YJaMdNFvYuo2-m7KEuz9Mum3zmJb33DbJNesYJQw@mail.gmail.com>
	<37B434B4-BBDA-4B43-A389-7D7FB2A9D69C@icir.org>
	<CAEk_Ei+wkaiYPiX+SroN8oAPWozgQeCS8PRkXE9BcyZw0bOV6g@mail.gmail.com>
	<27901_1389886916_s0GFftMV005295_429B7867-42A9-425A-A157-6E739014F56F@uwaterloo.ca>
Message-ID: <59C5FF3B-6540-4582-86BA-FF8A2966B242@andrew.cmu.edu>

Would it be sufficient to disable the SSL analyzer? That way you don't have to play the port shell game, but you don't get an ssl.log or any SSL-related notices, if that's your concern.

  --Vlad

On Jan 16, 2014, at 10:40 AM, Mike Patterson <mike.patterson at uwaterloo.ca> wrote:

> If you?re anxious to avoid any SSL traffic based on port exclusions, you might consider other well-known ports - 587, 465, etc.
> 
> Mike
> 
> On Jan 16, 2014, at 10:39 AM, George Insko <ginsko3 at gmail.com> wrote:
> 
>> Good call. Thanks. 
>> 
>> 
>> On Thu, Jan 16, 2014 at 10:33 AM, Seth Hall <seth at icir.org> wrote:
>> 
>> On Jan 16, 2014, at 10:12 AM, George Insko <ginsko3 at gmail.com> wrote:
>> 
>>> #Nothing from src host to dst port
>>> !(src host 0.0.0.0/0 && dst port 443) &&
>>> Does that make sense and will it work? Do you all have any other ways to permanently filter traffic?
>> 
>> I think you meant to do?
>> 
>> (not src port 443 and not dst port 443)
>> 
>>  .Seth
>> 
>> 
>> --
>> Seth Hall
>> International Computer Science Institute
>> (Bro) because everyone has a network
>> http://www.bro.org/
>> 
>> 
>> 
>> 
>> -- 
>> George Insko
>> Email:    ginsko3 at gmail.com
>> Twitter: @ginsko3
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140116/bb3aa952/attachment.bin 

From ginsko3 at gmail.com  Thu Jan 16 08:58:06 2014
From: ginsko3 at gmail.com (George Insko)
Date: Thu, 16 Jan 2014 11:58:06 -0500
Subject: [Bro] BPF?
In-Reply-To: <59C5FF3B-6540-4582-86BA-FF8A2966B242@andrew.cmu.edu>
References: <CAEk_EiJBL8YJaMdNFvYuo2-m7KEuz9Mum3zmJb33DbJNesYJQw@mail.gmail.com>
	<37B434B4-BBDA-4B43-A389-7D7FB2A9D69C@icir.org>
	<CAEk_Ei+wkaiYPiX+SroN8oAPWozgQeCS8PRkXE9BcyZw0bOV6g@mail.gmail.com>
	<27901_1389886916_s0GFftMV005295_429B7867-42A9-425A-A157-6E739014F56F@uwaterloo.ca>
	<59C5FF3B-6540-4582-86BA-FF8A2966B242@andrew.cmu.edu>
Message-ID: <CAEk_EiKutnDkRGbk+mxKg0xLPwtPVRoHkAAipFuEgczk3Y9NFA@mail.gmail.com>

I think just the amount of traffic coming in is my issue. So, I am trying
to block things I can do little about. Hoping in vain that this will let me
focus on the sloppy hackers and the low hanging fruit.

This is not my primary IDS so missing somethings is ok.


On Thu, Jan 16, 2014 at 10:53 AM, Vlad Grigorescu <vladg at cmu.edu> wrote:

> Would it be sufficient to disable the SSL analyzer? That way you don't
> have to play the port shell game, but you don't get an ssl.log or any
> SSL-related notices, if that's your concern.
>
>   --Vlad
>
> On Jan 16, 2014, at 10:40 AM, Mike Patterson <mike.patterson at uwaterloo.ca>
> wrote:
>
> > If you?re anxious to avoid any SSL traffic based on port exclusions, you
> might consider other well-known ports - 587, 465, etc.
> >
> > Mike
> >
> > On Jan 16, 2014, at 10:39 AM, George Insko <ginsko3 at gmail.com> wrote:
> >
> >> Good call. Thanks.
> >>
> >>
> >> On Thu, Jan 16, 2014 at 10:33 AM, Seth Hall <seth at icir.org> wrote:
> >>
> >> On Jan 16, 2014, at 10:12 AM, George Insko <ginsko3 at gmail.com> wrote:
> >>
> >>> #Nothing from src host to dst port
> >>> !(src host 0.0.0.0/0 && dst port 443) &&
> >>> Does that make sense and will it work? Do you all have any other ways
> to permanently filter traffic?
> >>
> >> I think you meant to do?
> >>
> >> (not src port 443 and not dst port 443)
> >>
> >>  .Seth
> >>
> >>
> >> --
> >> Seth Hall
> >> International Computer Science Institute
> >> (Bro) because everyone has a network
> >> http://www.bro.org/
> >>
> >>
> >>
> >>
> >> --
> >> George Insko
> >> Email:    ginsko3 at gmail.com
> >> Twitter: @ginsko3
> >> _______________________________________________
> >> Bro mailing list
> >> bro at bro-ids.org
> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>


-- 
*George Insko*
Email:    ginsko3 at gmail.com
Twitter: @ginsko3
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140116/fef2c252/attachment.html 

From jthoel at gmail.com  Thu Jan 16 09:37:13 2014
From: jthoel at gmail.com (Jeremy Hoel)
Date: Thu, 16 Jan 2014 17:37:13 +0000
Subject: [Bro] BPF?
In-Reply-To: <CAEk_EiKutnDkRGbk+mxKg0xLPwtPVRoHkAAipFuEgczk3Y9NFA@mail.gmail.com>
References: <CAEk_EiJBL8YJaMdNFvYuo2-m7KEuz9Mum3zmJb33DbJNesYJQw@mail.gmail.com>
	<37B434B4-BBDA-4B43-A389-7D7FB2A9D69C@icir.org>
	<CAEk_Ei+wkaiYPiX+SroN8oAPWozgQeCS8PRkXE9BcyZw0bOV6g@mail.gmail.com>
	<27901_1389886916_s0GFftMV005295_429B7867-42A9-425A-A157-6E739014F56F@uwaterloo.ca>
	<59C5FF3B-6540-4582-86BA-FF8A2966B242@andrew.cmu.edu>
	<CAEk_EiKutnDkRGbk+mxKg0xLPwtPVRoHkAAipFuEgczk3Y9NFA@mail.gmail.com>
Message-ID: <CAH_p-VP5JTs6htccgtMqtDxtkWvWkAdJOrKTVBHs8n++A=GjGg@mail.gmail.com>

Just as a side note, there are some apps that will send clear text
over 443.. so you also won't be looking at those if you limit that
port.

On Thu, Jan 16, 2014 at 4:58 PM, George Insko <ginsko3 at gmail.com> wrote:
> I think just the amount of traffic coming in is my issue. So, I am trying to
> block things I can do little about. Hoping in vain that this will let me
> focus on the sloppy hackers and the low hanging fruit.
>
> This is not my primary IDS so missing somethings is ok.
>
>
> On Thu, Jan 16, 2014 at 10:53 AM, Vlad Grigorescu <vladg at cmu.edu> wrote:
>>
>> Would it be sufficient to disable the SSL analyzer? That way you don't
>> have to play the port shell game, but you don't get an ssl.log or any
>> SSL-related notices, if that's your concern.
>>
>>   --Vlad
>>
>> On Jan 16, 2014, at 10:40 AM, Mike Patterson <mike.patterson at uwaterloo.ca>
>> wrote:
>>
>> > If you?re anxious to avoid any SSL traffic based on port exclusions, you
>> > might consider other well-known ports - 587, 465, etc.
>> >
>> > Mike
>> >
>> > On Jan 16, 2014, at 10:39 AM, George Insko <ginsko3 at gmail.com> wrote:
>> >
>> >> Good call. Thanks.
>> >>
>> >>
>> >> On Thu, Jan 16, 2014 at 10:33 AM, Seth Hall <seth at icir.org> wrote:
>> >>
>> >> On Jan 16, 2014, at 10:12 AM, George Insko <ginsko3 at gmail.com> wrote:
>> >>
>> >>> #Nothing from src host to dst port
>> >>> !(src host 0.0.0.0/0 && dst port 443) &&
>> >>> Does that make sense and will it work? Do you all have any other ways
>> >>> to permanently filter traffic?
>> >>
>> >> I think you meant to do?
>> >>
>> >> (not src port 443 and not dst port 443)
>> >>
>> >>  .Seth
>> >>
>> >>
>> >> --
>> >> Seth Hall
>> >> International Computer Science Institute
>> >> (Bro) because everyone has a network
>> >> http://www.bro.org/
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> George Insko
>> >> Email:    ginsko3 at gmail.com
>> >> Twitter: @ginsko3
>> >> _______________________________________________
>> >> Bro mailing list
>> >> bro at bro-ids.org
>> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> >
>> >
>> > _______________________________________________
>> > Bro mailing list
>> > bro at bro-ids.org
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
>
>
> --
> George Insko
> Email:    ginsko3 at gmail.com
> Twitter: @ginsko3
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From jbabio at po-box.esu.edu  Thu Jan 16 18:45:18 2014
From: jbabio at po-box.esu.edu (John Babio)
Date: Fri, 17 Jan 2014 02:45:18 +0000
Subject: [Bro] time base notice
Message-ID: <6E7D7EC4661DB5438A60388F3ED1BBAA088502@msxmb2.admin.esu.edu>

Is there a way to setup notice based on time of access via ssh? In other words, create a notice if access to a server happens at 11pm?



From seth at icir.org  Thu Jan 16 19:28:39 2014
From: seth at icir.org (Seth Hall)
Date: Thu, 16 Jan 2014 22:28:39 -0500
Subject: [Bro] time base notice
In-Reply-To: <6E7D7EC4661DB5438A60388F3ED1BBAA088502@msxmb2.admin.esu.edu>
References: <6E7D7EC4661DB5438A60388F3ED1BBAA088502@msxmb2.admin.esu.edu>
Message-ID: <1F8969D3-5862-4EC9-8B53-BBFB4D608F6B@icir.org>


On Jan 16, 2014, at 9:45 PM, John Babio <jbabio at po-box.esu.edu> wrote:

> Is there a way to setup notice based on time of access via ssh? In other words, create a notice if access to a server happens at 11pm?


Unfortunately that's something I've been wanting to address but we haven't had the cycles to approach it quite yet.  I'm starting to think that maybe I should finally hack a solution together soon (it's possible now, but *really* nasty and hacky).  If I get a chance soon I'll hack together something that would let you define time in that way.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140116/495fb39c/attachment.bin 

From vern at icir.org  Thu Jan 16 21:06:37 2014
From: vern at icir.org (Vern Paxson)
Date: Thu, 16 Jan 2014 21:06:37 -0800
Subject: [Bro] pcap syntax (Re:  BPF?)
In-Reply-To: <37B434B4-BBDA-4B43-A389-7D7FB2A9D69C@icir.org> (Thu,
	16 Jan 2014 10:33:26 EST).
Message-ID: <20140117050637.0FA862C4008@rock.ICSI.Berkeley.EDU>

> I think you meant to do
> 
> (not src port 443 and not dst port 443)

(A nit: that's equivalent to "not port 443".  Nit #2: more correct would
be "not tcp port 443".  The above will weed out UDP traffic that happens
to use 443.)


From jbabio at po-box.esu.edu  Fri Jan 17 04:22:03 2014
From: jbabio at po-box.esu.edu (John Babio)
Date: Fri, 17 Jan 2014 12:22:03 +0000
Subject: [Bro] time base notice
In-Reply-To: <1F8969D3-5862-4EC9-8B53-BBFB4D608F6B@icir.org>
References: <6E7D7EC4661DB5438A60388F3ED1BBAA088502@msxmb2.admin.esu.edu>
	<1F8969D3-5862-4EC9-8B53-BBFB4D608F6B@icir.org>
Message-ID: <CEFE89A2.CB06%jbabio@po-box.esu.edu>

Sounds awesome. I look forward to it. :)
I didn?t seen anything in snort with this ability either. Keep up the good
work Seth!

On 1/16/14, 10:28 PM, "Seth Hall" <seth at icir.org> wrote:

>
>On Jan 16, 2014, at 9:45 PM, John Babio <jbabio at po-box.esu.edu> wrote:
>
>> Is there a way to setup notice based on time of access via ssh? In
>>other words, create a notice if access to a server happens at 11pm?
>
>
>Unfortunately that's something I've been wanting to address but we
>haven't had the cycles to approach it quite yet.  I'm starting to think
>that maybe I should finally hack a solution together soon (it's possible
>now, but *really* nasty and hacky).  If I get a chance soon I'll hack
>together something that would let you define time in that way.
>
>  .Seth
>
>--
>Seth Hall
>International Computer Science Institute
>(Bro) because everyone has a network
>http://www.bro.org/
>




From prateekgupta.3991 at gmail.com  Sun Jan 19 00:29:30 2014
From: prateekgupta.3991 at gmail.com (Prateek Gupta)
Date: Sun, 19 Jan 2014 13:59:30 +0530
Subject: [Bro] control flow
Message-ID: <CAL6GwxPuJPiSVNdavUNtA9u5gUSaqo37eT7EsMJr0aw53=RFFA@mail.gmail.com>

Hello,
Can someone help me to understand the work flow of Bro and tell me how to
explore all the info related to MIME files in HTTP?

Regards,
Prateek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140119/2f192273/attachment.html 

From bkellogg at dresser-rand.com  Sun Jan 19 08:45:11 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Sun, 19 Jan 2014 16:45:11 +0000
Subject: [Bro] Bro bug?
Message-ID: <BDF3793B557F624699D4C68A3B8FCF820B09F4@DRTEXW238VM.na.d-rco.com>

largeTx.bro alerts on any outgoing Txs over X bytes.  If of sufficient size it sends an email alert.

get.bash uses tshark to extract captured Security Onion FPC packets from /nsm/sensor_data/so-OLN-eth0/dailylogs/2014-01-19.

I received an email alert saying that 1.1.1.1 transmitted over 1GB of information to 2.2.2.2.  Therefore I went to the FPC directory above to extract this communication to see what it was.  The extracted content was ~3.5MB in size. 

I've tested this with several large file uploads and have gotten consistent and accurate results with all tests.  Therefore I'm confused as to how this alert was generated.

Is this an intermittent bug possibly or am I not understanding something?

The email alert I received from Bro 2.2 on Security Onion with all the latest patches is included as well.  The duration is odd as well.  I've received a handful of similar alerts for large transfers and very short durations.


Thank you,
Brian Kellogg


-------------- next part --------------
A non-text attachment was scrubbed...
Name: largeTx.bro
Type: application/octet-stream
Size: 1812 bytes
Desc: largeTx.bro
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140119/7ceb6811/attachment.obj 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: alert-email.txt
Url: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140119/7ceb6811/attachment.txt 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: get.bash
Type: application/octet-stream
Size: 307 bytes
Desc: get.bash
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140119/7ceb6811/attachment-0001.obj 

From bkellogg at dresser-rand.com  Sun Jan 19 09:06:03 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Sun, 19 Jan 2014 17:06:03 +0000
Subject: [Bro] Bro bug?
In-Reply-To: <CAFVce3TD92y7bOCG3tH2DiB1RPFbAC0H6oS5r7vTjPGK_-xauw@mail.gmail.com>
References: <BDF3793B557F624699D4C68A3B8FCF820B09F4@DRTEXW238VM.na.d-rco.com>
	<CAFVce3TD92y7bOCG3tH2DiB1RPFbAC0H6oS5r7vTjPGK_-xauw@mail.gmail.com>
Message-ID: <BDF3793B557F624699D4C68A3B8FCF820B223B@DRTEXW238VM.na.d-rco.com>

Yes and that is the intention.  If you look at the email alert in the email you will see a report size of over 1GB and the Bro script only send emails on any Tx over 50MB.  

Thank you,
Brian Kellogg


-----Original Message-----
From: John Green [mailto:john at giggled.org] 
Sent: Sunday, January 19, 2014 12:01 PM
To: Kellogg, Brian D (OLN)
Subject: Re: [Bro] Bro bug?

Hi Brian,
Doesn't

        const recordTx = 1024000;
                 # destination hosts to record if over this many bytes

alert on any transfer approximately > 1MB rather than 1GB?

Cheers
John


On 19 January 2014 16:45, Kellogg, Brian D (OLN) <bkellogg at dresser-rand.com> wrote:
> largeTx.bro alerts on any outgoing Txs over X bytes.  If of sufficient size it sends an email alert.
>
> get.bash uses tshark to extract captured Security Onion FPC packets from /nsm/sensor_data/so-OLN-eth0/dailylogs/2014-01-19.
>
> I received an email alert saying that 1.1.1.1 transmitted over 1GB of information to 2.2.2.2.  Therefore I went to the FPC directory above to extract this communication to see what it was.  The extracted content was ~3.5MB in size.
>
> I've tested this with several large file uploads and have gotten consistent and accurate results with all tests.  Therefore I'm confused as to how this alert was generated.
>
> Is this an intermittent bug possibly or am I not understanding something?
>
> The email alert I received from Bro 2.2 on Security Onion with all the latest patches is included as well.  The duration is odd as well.  I've received a handful of similar alerts for large transfers and very short durations.
>
>
> Thank you,
> Brian Kellogg
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From JAzoff at albany.edu  Sun Jan 19 09:09:01 2014
From: JAzoff at albany.edu (Justin Azoff)
Date: Sun, 19 Jan 2014 12:09:01 -0500
Subject: [Bro] Bro bug?
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF820B09F4@DRTEXW238VM.na.d-rco.com>
References: <BDF3793B557F624699D4C68A3B8FCF820B09F4@DRTEXW238VM.na.d-rco.com>
Message-ID: <20140119170901.GC23482@datacomm.albany.edu>

On Sun, Jan 19, 2014 at 04:45:11PM +0000, Kellogg, Brian D (OLN) wrote:
> largeTx.bro alerts on any outgoing Txs over X bytes.  If of sufficient size it sends an email alert.
..
> I received an email alert saying that 1.1.1.1 transmitted over 1GB of information to 2.2.2.2.  Therefore I went to the FPC directory above to extract this communication to see what it was.  The extracted content was ~3.5MB in size. 


> Message: Orig transmitted 1056737769 bytes to resp.  Duration 0.092641 sec.  Connection UID Cma6473thsxripFj9k.

Can you post the full conn.log entry for this connection? That might help
explain what is going on.

    grep Cma6473thsxripFj9k conn.log

should find the exact entry.


-- 
-- Justin Azoff


From bkellogg at dresser-rand.com  Sun Jan 19 09:22:10 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Sun, 19 Jan 2014 17:22:10 +0000
Subject: [Bro] Bro bug?
In-Reply-To: <20140119170901.GC23482@datacomm.albany.edu>
References: <BDF3793B557F624699D4C68A3B8FCF820B09F4@DRTEXW238VM.na.d-rco.com>
	<20140119170901.GC23482@datacomm.albany.edu>
Message-ID: <BDF3793B557F624699D4C68A3B8FCF820B228E@DRTEXW238VM.na.d-rco.com>

1390143300.845103	Cma6473thsxripFj9k	1.1.1.1	3326	2.2.2.2	80	tcp	-	0.092641	1056737769	0	RSTOS0	T	0	SaR	2	88	1	40	(empty)	-	US	so-eth0

Thank you,
Brian Kellogg


-----Original Message-----
From: Justin Azoff [mailto:JAzoff at albany.edu] 
Sent: Sunday, January 19, 2014 12:09 PM
To: Kellogg, Brian D (OLN)
Cc: bro at bro.org
Subject: Re: [Bro] Bro bug?

On Sun, Jan 19, 2014 at 04:45:11PM +0000, Kellogg, Brian D (OLN) wrote:
> largeTx.bro alerts on any outgoing Txs over X bytes.  If of sufficient size it sends an email alert.
..
> I received an email alert saying that 1.1.1.1 transmitted over 1GB of information to 2.2.2.2.  Therefore I went to the FPC directory above to extract this communication to see what it was.  The extracted content was ~3.5MB in size. 


> Message: Orig transmitted 1056737769 bytes to resp.  Duration 0.092641 sec.  Connection UID Cma6473thsxripFj9k.

Can you post the full conn.log entry for this connection? That might help explain what is going on.

    grep Cma6473thsxripFj9k conn.log

should find the exact entry.


--
-- Justin Azoff



From bkellogg at dresser-rand.com  Sun Jan 19 09:33:32 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Sun, 19 Jan 2014 17:33:32 +0000
Subject: [Bro] Bro bug?
References: <BDF3793B557F624699D4C68A3B8FCF820B09F4@DRTEXW238VM.na.d-rco.com>
	<20140119170901.GC23482@datacomm.albany.edu> 
Message-ID: <BDF3793B557F624699D4C68A3B8FCF820B22BE@DRTEXW238VM.na.d-rco.com>

SaR in the history field is in common for both of the erroneous email alerts I received today.


Thank you,
Brian Kellogg


-----Original Message-----
From: Kellogg, Brian D (OLN) 
Sent: Sunday, January 19, 2014 12:22 PM
To: 'Justin Azoff'
Cc: bro at bro.org
Subject: RE: [Bro] Bro bug?

1390143300.845103	Cma6473thsxripFj9k	1.1.1.1	3326	2.2.2.2	80	tcp	-	0.092641	1056737769	0	RSTOS0	T	0	SaR	2	88	1	40	(empty)	-	US	so-eth0

Thank you,
Brian Kellogg


-----Original Message-----
From: Justin Azoff [mailto:JAzoff at albany.edu] 
Sent: Sunday, January 19, 2014 12:09 PM
To: Kellogg, Brian D (OLN)
Cc: bro at bro.org
Subject: Re: [Bro] Bro bug?

On Sun, Jan 19, 2014 at 04:45:11PM +0000, Kellogg, Brian D (OLN) wrote:
> largeTx.bro alerts on any outgoing Txs over X bytes.  If of sufficient size it sends an email alert.
..
> I received an email alert saying that 1.1.1.1 transmitted over 1GB of information to 2.2.2.2.  Therefore I went to the FPC directory above to extract this communication to see what it was.  The extracted content was ~3.5MB in size. 


> Message: Orig transmitted 1056737769 bytes to resp.  Duration 0.092641 sec.  Connection UID Cma6473thsxripFj9k.

Can you post the full conn.log entry for this connection? That might help explain what is going on.

    grep Cma6473thsxripFj9k conn.log

should find the exact entry.


--
-- Justin Azoff



From bkellogg at dresser-rand.com  Sun Jan 19 09:38:40 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Sun, 19 Jan 2014 17:38:40 +0000
Subject: [Bro] Bro bug?
References: <BDF3793B557F624699D4C68A3B8FCF820B09F4@DRTEXW238VM.na.d-rco.com>
	<20140119170901.GC23482@datacomm.albany.edu>  
Message-ID: <BDF3793B557F624699D4C68A3B8FCF820B22D9@DRTEXW238VM.na.d-rco.com>

Here is the conn log entry for the other one:

1390143593.839386	C3miqNnYs6FBw74c5	3.3.3.3	4235	4.4.4.4	80	tcp	-	0.086147	1594489093	0	RSTOS0	T	0	SaR	2	88	1	40	(empty)	-	US	so -eth0

Thank you,
Brian Kellogg
Security Analyst; IT Governance, Risk, and Compliance
500 Paul Clark Drive, Olean,  NY 14760
T: (716) 375-3186 | F: (716) 375-3557


-----Original Message-----
From: Kellogg, Brian D (OLN) 
Sent: Sunday, January 19, 2014 12:34 PM
To: 'Justin Azoff'
Cc: 'bro at bro.org'
Subject: RE: [Bro] Bro bug?

SaR in the history field is in common for both of the erroneous email alerts I received today.


Thank you,
Brian Kellogg


-----Original Message-----
From: Kellogg, Brian D (OLN) 
Sent: Sunday, January 19, 2014 12:22 PM
To: 'Justin Azoff'
Cc: bro at bro.org
Subject: RE: [Bro] Bro bug?

1390143300.845103	Cma6473thsxripFj9k	1.1.1.1	3326	2.2.2.2	80	tcp	-	0.092641	1056737769	0	RSTOS0	T	0	SaR	2	88	1	40	(empty)	-	US	so-eth0

Thank you,
Brian Kellogg


-----Original Message-----
From: Justin Azoff [mailto:JAzoff at albany.edu] 
Sent: Sunday, January 19, 2014 12:09 PM
To: Kellogg, Brian D (OLN)
Cc: bro at bro.org
Subject: Re: [Bro] Bro bug?

On Sun, Jan 19, 2014 at 04:45:11PM +0000, Kellogg, Brian D (OLN) wrote:
> largeTx.bro alerts on any outgoing Txs over X bytes.  If of sufficient size it sends an email alert.
..
> I received an email alert saying that 1.1.1.1 transmitted over 1GB of information to 2.2.2.2.  Therefore I went to the FPC directory above to extract this communication to see what it was.  The extracted content was ~3.5MB in size. 


> Message: Orig transmitted 1056737769 bytes to resp.  Duration 0.092641 sec.  Connection UID Cma6473thsxripFj9k.

Can you post the full conn.log entry for this connection? That might help explain what is going on.

    grep Cma6473thsxripFj9k conn.log

should find the exact entry.


--
-- Justin Azoff



From JAzoff at albany.edu  Sun Jan 19 09:46:31 2014
From: JAzoff at albany.edu (Justin Azoff)
Date: Sun, 19 Jan 2014 12:46:31 -0500
Subject: [Bro] Bro bug?
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF820B228E@DRTEXW238VM.na.d-rco.com>
References: <BDF3793B557F624699D4C68A3B8FCF820B09F4@DRTEXW238VM.na.d-rco.com>
	<20140119170901.GC23482@datacomm.albany.edu>
	<BDF3793B557F624699D4C68A3B8FCF820B228E@DRTEXW238VM.na.d-rco.com>
Message-ID: <20140119174631.GD23482@datacomm.albany.edu>

On Sun, Jan 19, 2014 at 05:22:10PM +0000, Kellogg, Brian D (OLN) wrote:
> 1390143300.845103	Cma6473thsxripFj9k	1.1.1.1	3326	2.2.2.2	80	tcp	-	0.092641	1056737769	0	RSTOS0	T	0	SaR	2	88	1	40	(empty)	-	US	so-eth0

So, with the field names, that is:

ts 1390143300.845103
uid Cma6473thsxripFj9k
id.orig_h 1.1.1.1
id.orig_p 3326
id.resp_h 2.2.2.2
id.resp_p 80
proto tcp
service -
duration 0.092641
orig_bytes 1056737769
resp_bytes 0
conn_state RSTOS0
local_orig T
missed_bytes 0
history SaR
orig_pkts 2
orig_ip_bytes 88
resp_pkts 1
resp_ip_bytes 40

Which shows that bro calculated that there were 1056737769 bytes based
on sequence numbers, but only actually saw 88 bytes.

I think simply changing $size to $num_bytes_ip will fix your problems.

-- 
-- Justin Azoff
-- Network Security & Performance Analyst


From bkellogg at dresser-rand.com  Sun Jan 19 10:25:48 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Sun, 19 Jan 2014 18:25:48 +0000
Subject: [Bro] Bro bug?
In-Reply-To: <20140119174631.GD23482@datacomm.albany.edu>
References: <BDF3793B557F624699D4C68A3B8FCF820B09F4@DRTEXW238VM.na.d-rco.com>
	<20140119170901.GC23482@datacomm.albany.edu>
	<BDF3793B557F624699D4C68A3B8FCF820B228E@DRTEXW238VM.na.d-rco.com>
	<20140119174631.GD23482@datacomm.albany.edu>
Message-ID: <BDF3793B557F624699D4C68A3B8FCF820B230D@DRTEXW238VM.na.d-rco.com>

Ah, thank you for the clarification.  I was obvsiously misunderstanding how that field was calculated.

Thank you,
Brian Kellogg

-----Original Message-----
From: Justin Azoff [mailto:JAzoff at albany.edu] 
Sent: Sunday, January 19, 2014 12:47 PM
To: Kellogg, Brian D (OLN)
Cc: bro at bro.org
Subject: Re: [Bro] Bro bug?

On Sun, Jan 19, 2014 at 05:22:10PM +0000, Kellogg, Brian D (OLN) wrote:
> 1390143300.845103	Cma6473thsxripFj9k	1.1.1.1	3326	2.2.2.2	80	tcp	-	0.092641	1056737769	0	RSTOS0	T	0	SaR	2	88	1	40	(empty)	-	US	so-eth0

So, with the field names, that is:

ts 1390143300.845103
uid Cma6473thsxripFj9k
id.orig_h 1.1.1.1
id.orig_p 3326
id.resp_h 2.2.2.2
id.resp_p 80
proto tcp
service -
duration 0.092641
orig_bytes 1056737769
resp_bytes 0
conn_state RSTOS0
local_orig T
missed_bytes 0
history SaR
orig_pkts 2
orig_ip_bytes 88
resp_pkts 1
resp_ip_bytes 40

Which shows that bro calculated that there were 1056737769 bytes based on sequence numbers, but only actually saw 88 bytes.

I think simply changing $size to $num_bytes_ip will fix your problems.

--
-- Justin Azoff
-- Network Security & Performance Analyst



From hlein at korelogic.com  Sun Jan 19 10:38:40 2014
From: hlein at korelogic.com (Hank Leininger)
Date: Sun, 19 Jan 2014 13:38:40 -0500
Subject: [Bro] Bro bug?
In-Reply-To: <20140119174631.GD23482@datacomm.albany.edu>
References: <BDF3793B557F624699D4C68A3B8FCF820B09F4@DRTEXW238VM.na.d-rco.com>
	<20140119170901.GC23482@datacomm.albany.edu>
	<BDF3793B557F624699D4C68A3B8FCF820B228E@DRTEXW238VM.na.d-rco.com>
	<20140119174631.GD23482@datacomm.albany.edu>
Message-ID: <20140119183840.GI2693@marklar.spinoli.org>

On Sun, Jan 19, 2014 at 12:46:31PM -0500, Justin Azoff wrote:
> On Sun, Jan 19, 2014 at 05:22:10PM +0000, Kellogg, Brian D (OLN) wrote:
> > 1390143300.845103	Cma6473thsxripFj9k	1.1.1.1	3326	2.2.2.2	80	tcp	-	0.092641	1056737769	0	RSTOS0	T	0	SaR	2	88	1	40	(empty)	-	US	so-eth0
> 
> So, with the field names, that is:
[snip] 
> duration 0.092641
> orig_bytes 1056737769
> resp_bytes 0
> conn_state RSTOS0
> local_orig T
> missed_bytes 0
> history SaR
[snip]

> Which shows that bro calculated that there were 1056737769 bytes based
> on sequence numbers, but only actually saw 88 bytes.
> 
> I think simply changing $size to $num_bytes_ip will fix your problems.

That's probably a decent workaround.

Another option would be to only trigger on connections that were fully
established - with conn_state of SF (possibly also RST0 and RSTR) - as
those should have the "most reliable" byte counts.

I think the problem in Brian's case comes from this not being a normally
established / torn down connection.  conn_state RSTOS0 implies Bro saw
an incomplete connection attempt be torn down by an RST (by the orignal
sender, which is odd, and history SaR means the receiver sent not a
SYNACK, and not nothing, but a bare ACK, which is odder still... but
that is a story for another day*).

It could be argued that Bro should "know" that for certain failed /
incomplete connection types, ACK/SEQ math can be unreliable (maybe it
already does for some), and that SaR is a candidate for that (maybe that
has already been argued; there's probably good cases to be made both
ways).

[ *Snip esoteric ramble about the 4 way handshake, etc. ]

Thanks,

Hank Leininger <hlein at korelogic.com>
D24D 2C2A F3AC B9AE CD03  B506 2D57 32E1 686B 6DB3
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 447 bytes
Desc: Digital signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140119/42a2296c/attachment.bin 

From seth at icir.org  Sun Jan 19 11:39:18 2014
From: seth at icir.org (Seth Hall)
Date: Sun, 19 Jan 2014 14:39:18 -0500
Subject: [Bro] Bro bug?
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF820B22D9@DRTEXW238VM.na.d-rco.com>
References: <BDF3793B557F624699D4C68A3B8FCF820B09F4@DRTEXW238VM.na.d-rco.com>
	<20140119170901.GC23482@datacomm.albany.edu>
	<BDF3793B557F624699D4C68A3B8FCF820B22D9@DRTEXW238VM.na.d-rco.com>
Message-ID: <50455D97-BC8F-423C-999D-907C4D69B243@icir.org>


On Jan 19, 2014, at 12:38 PM, "Kellogg, Brian D (OLN)" <bkellogg at dresser-rand.com> wrote:

> 1390143593.839386	C3miqNnYs6FBw74c5	3.3.3.3	4235	4.4.4.4	80	tcp	-	0.086147	1594489093	0	RSTOS0	T	0	SaR	2	88	1	40	(empty)	-	US	so -eth0


This is actually a bug in Bro's handling of odd tcp sessions and it would be great to get a packet capture so we could fix this and work it into our test suite.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140119/99fe8655/attachment.bin 

From scott at 0x4c.com  Sun Jan 19 15:54:17 2014
From: scott at 0x4c.com (scott mcallester)
Date: Mon, 20 Jan 2014 10:54:17 +1100
Subject: [Bro] Bro bug?
Message-ID: <52DC65A9.40609@0x4c.com>

On Jan 19, 2014, at 12:38 PM, "Kellogg, Brian D (OLN)" <bkellogg at dresser-rand.com> wrote:

> 1390143593.839386	C3miqNnYs6FBw74c5	3.3.3.3	4235	4.4.4.4	80	tcp	-	0.086147	1594489093	0	RSTOS0	T	0	SaR	2	88	1	40	(empty)	-	US	so -eth0

This is actually a bug in Bro's handling of odd tcp sessions and it would be great to get a packet capture so we could fix this and work it into our test suite.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/


I was also having this issue last week when trying to find internal hosts that had sent > 1gb to external hosts, see screenshot. http://0x4c.com/bro.png
Also here's a pcap of the first entry from the screenshot http://0x4c.com/bro4.29.pcap

Scott.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140120/ddd1eeed/attachment.html 

From david.gugelmann at tik.ee.ethz.ch  Mon Jan 20 13:23:31 2014
From: david.gugelmann at tik.ee.ethz.ch (David Gugelmann)
Date: Mon, 20 Jan 2014 22:23:31 +0100
Subject: [Bro]  bro bug? - unreported packet loss
In-Reply-To: <52C9B45F.3010505@tik.ee.ethz.ch>
References: <52C9B45F.3010505@tik.ee.ethz.ch>
Message-ID: <52DD93D3.5020108@tik.ee.ethz.ch>

Hello everybody,

sorry for bothering you again. As I posted two weeks ago (see below), it
seems to me that bro sometimes does not report packet loss in TCP
connections (missed_bytes in conn.log is 0 even though there are packets
missing).

These are two examples for such connections:
1331764471.664131   CJukZE3ew98dfK4qAd  192.168.122.230 60648
77.238.160.184  80  tcp -   10.048360   538 2902    SF  -   0   ShADafF
5   750 4   172 (empty)
1331765540.407398   CpZaKz4sihR23hO2a6  192.168.122.203 64860
94.245.68.169   80  tcp -   6.424619    1270    10052   SF  -   0
ShAaDdfF    8   1602    9   6198    (empty)

You can find the corresponding pcap files and additional information here:
http://people.ee.ethz.ch/~gugdavid/bro_missed_bytes.zip

Am I missing something or did anybody encounter something similar?

Thank you,
David


-------- Original Message --------
Subject: [Bro] unreported packet loss
Date: Sun, 05 Jan 2014 20:37:03 +0100
From: David Gugelmann <david.gugelmann at tik.ee.ethz.ch>
To: bro at bro.org

Hello everybody,

I am quite new to bro, so I am not sure whether I am missing something.
It seems to me that bro (v2.1 and v2.2) does in some cases not report
packet loss.

I discovered this by comparing resp_bytes, resp_ip_bytes and
missed_bytes from conn.log. I found several TCP streams, for which
resp_ip_bytes < resp_bytes but missed_bytes is 0, that is, there are
more TCP-bytes than IP-bytes but at the same time no packet losses,
which seemed strange.
Analyzing the corresponding TCP sequence numbers more in detail, I found
that this seems to be caused by packet loss that is not reflected in
bro's missing_bytes field. Also capture_loss.log did not show any loss.

You can find two example TCP streams, bro's output and Wireshark screen
shots here:
http://people.ee.ethz.ch/~gugdavid/bro_missed_bytes.zip
(Note: This is no real user traffic, these traffic samples have been
automatically generated in a testbed using mechanized Firefox instances.)

Am I missing something or did anybody encounter something similar?

Thank you,
David



From awong5 at mail.ccsf.edu  Tue Jan 21 09:59:46 2014
From: awong5 at mail.ccsf.edu (Arthur Wong)
Date: Tue, 21 Jan 2014 09:59:46 -0800
Subject: [Bro] Running Bro on Mac OS X 10.8.5
Message-ID: <CAMa2HwxTToQAMsTPZjuWqXHjv7Dv4F+a4QxKLv619ej-dGXq=Q@mail.gmail.com>

Hello All,

I downloaded Bro ( .dmg, full ) from Bro.org and installed it.

I do not recall having any issue with the install.

I wanted to have Bro read in a pcap file and the resulted in:

/opt/bro/bin/bro -r myidftraffic.pcap
internal error: can't load magic file <default>: no magic files loaded
Abort trap: 6

How do I correct this error?

Thanks in advance.

Art
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140121/236f9f61/attachment.html 

From jsiwek at illinois.edu  Tue Jan 21 10:21:33 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Tue, 21 Jan 2014 18:21:33 +0000
Subject: [Bro] Running Bro on Mac OS X 10.8.5
In-Reply-To: <CAMa2HwxTToQAMsTPZjuWqXHjv7Dv4F+a4QxKLv619ej-dGXq=Q@mail.gmail.com>
References: <CAMa2HwxTToQAMsTPZjuWqXHjv7Dv4F+a4QxKLv619ej-dGXq=Q@mail.gmail.com>
Message-ID: <3B1B8361-60E0-42B5-BEC7-4E8057CC8054@illinois.edu>


On Jan 21, 2014, at 11:59 AM, Arthur Wong <awong5 at mail.ccsf.edu> wrote:

> I downloaded Bro ( .dmg, full ) from Bro.org and installed it.
> 
> /opt/bro/bin/bro -r myidftraffic.pcap
> internal error: can't load magic file <default>: no magic files loaded
> Abort trap: 6
> 
> How do I correct this error?

I suggest building from sources instead.

>From [1]:

"Mac OS X packages are currently for Mavericks (10.9) only and the Bro packages also require libmagic to be installed ? using the version from MacPorts should work, while other versions require setting the MAGIC environment variable to the path of the installed libmagic?s magic database file before running Bro."

- Jon

[1] http://bro.org/download/index.html#binarypackages


From jsiwek at illinois.edu  Tue Jan 21 10:39:43 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Tue, 21 Jan 2014 18:39:43 +0000
Subject: [Bro] Running Bro on Mac OS X 10.8.5
In-Reply-To: <3B1B8361-60E0-42B5-BEC7-4E8057CC8054@illinois.edu>
References: <CAMa2HwxTToQAMsTPZjuWqXHjv7Dv4F+a4QxKLv619ej-dGXq=Q@mail.gmail.com>
	<3B1B8361-60E0-42B5-BEC7-4E8057CC8054@illinois.edu>
Message-ID: <0406B4C4-B122-4498-BFB3-1F6593EAB7DB@illinois.edu>


On Jan 21, 2014, at 12:21 PM, Siwek, Jonathan Luke <jsiwek at illinois.edu> wrote:

> I suggest building from sources instead.

Or if you prefer, 2.2 is available via MacPorts or Homebrew.

- Jon


From jeremy.cox at washk12.org  Tue Jan 21 13:11:27 2014
From: jeremy.cox at washk12.org (Jeremy Cox)
Date: Tue, 21 Jan 2014 14:11:27 -0700
Subject: [Bro] Attach Barnyard2 to Bro2
Message-ID: <CAKdZko9YEwuN1X1MmFNuOv+xcg4OKhtiAiQ8M9rUHbwxCW4rdQ@mail.gmail.com>

I am attempting to use Barnyard2 to feed events from Suricata to Bro2.  It
looks like Barnyard2 wants to access Bro on 47757/tcp.  Bro is not
currently listening to that port.  And Im not sure how to get it listening,
or if thats just the old port it used to listen to.  I attempted to have
barnyard connect to 47760 in standalone mode, and 47761 or 47762 or 47763
in clustered mode.  In all instances Barnyard seems to connect to Bro, and
then unexpectedly crashes.  Am I missing something?  Should bro be
listening to 47757/tcp?

Any help would be great!

Thanks,

Jeremy


*Jeremy Cox*
Senior Network Engineer, ISO

*Washington County School District*121 W Tabernacle - St. George - UT
435-634-4315
www.washk12.org
687474703a2f2f7777772e7375706572746563686775792e636f6d

IMPORTANT NOTICE REGARDING THIS ELECTRONIC COMMUNICATION:

This e-mail, including any attachments thereto, contains information that
may be confidential or privileged, and is intended solely for the
individual or entity to whom it is addressed.  Recipient is hereby notified
that any disclosure, copying or distribution of this message is strictly
prohibited.  IF YOU ARE NOT THE INTENDED RECIPIENT, please notify the
originator of this e-mail immediately and destroy all information
received.  Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140121/66b0e68f/attachment.html 

From prateekgupta.3991 at gmail.com  Wed Jan 22 09:25:18 2014
From: prateekgupta.3991 at gmail.com (Prateek Gupta)
Date: Wed, 22 Jan 2014 22:55:18 +0530
Subject: [Bro] Control flow
Message-ID: <CAL6GwxOOG9ui2yW2CJpJrGFB=G1CPbO-8ObdgHSAd2hhjr6Sbw@mail.gmail.com>

Hello developers,
I am a bachelors in engineering student from India and doing a project in
bro-IDS for network analysis and scripts for interesting data.
I need to understand the work flow of bro from packet capture stage to the
final logging stage with reference to the order in which the activities
occur in bro for HTTP protocol.
This is the understanding that I have developed and please correct me if I
am wrong.
As far as I have understood, bro first takes in a pcap file and the
init-bare.bro extracts the information from it using the framework for
protocol independent data. This data is supplied to the corresponding
protocol which acts on it for the relevant data and generates events which
are handled by the event handlers and these handlers take the actions of
notice or logging etc.
My question is what is the mechanism for analysing the packet and the order
in which the bro code is sequenced?
Hope to get a reply soon!

Regards,
Prateek Gupta
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140122/62fa0b51/attachment.html 

From jsiwek at illinois.edu  Wed Jan 22 09:56:36 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Wed, 22 Jan 2014 17:56:36 +0000
Subject: [Bro] Attach Barnyard2 to Bro2
In-Reply-To: <CAKdZko9YEwuN1X1MmFNuOv+xcg4OKhtiAiQ8M9rUHbwxCW4rdQ@mail.gmail.com>
References: <CAKdZko9YEwuN1X1MmFNuOv+xcg4OKhtiAiQ8M9rUHbwxCW4rdQ@mail.gmail.com>
Message-ID: <586A8968-8CA0-410E-8864-F714AEB51CE4@illinois.edu>


On Jan 21, 2014, at 3:11 PM, Jeremy Cox <jeremy.cox at washk12.org> wrote:

> I am attempting to use Barnyard2 to feed events from Suricata to Bro2.  It looks like Barnyard2 wants to access Bro on 47757/tcp.  Bro is not currently listening to that port.

> Should bro be listening to 47757/tcp?

Only if you?re running a command-line bro.  47757/tcp is the default listen port for running bro command-line style.

> And Im not sure how to get it listening, or if thats just the old port it used to listen to.  I attempted to have barnyard connect to 47760 in standalone mode

47760/tcp is the default listen port of the bro instance of BroControl?s standalone node.

> and 47761 or 47762 or 47763 in clustered mode.

47761/tcp is the listen port of the bro manager node when using BroControl to manage a closer.  It probably makes most sense to configure Barnyard2 to connect to this port if you?re running a Bro cluster.

> In all instances Barnyard seems to connect to Bro, and then unexpectedly crashes.  Am I missing something?

What versions of Bro and Barnyard2 are you using (and any particular configure flags for Barnyard2, like ?enable-ipv6) ?  IIRC I think you will need at least Bro 2.1 and Barnyard 2.1.10.

I was able to get something simple working w/ latest Bro and Barnyard git master branches.  The only extra thing to do in Bro that?s not obvious is make sure the manager node has code to accept Barnyard alerts (putting it in site/local.bro is the simple way):

   @load policy/integration/barnyard2
   redef Communication::nodes += {
       ["local"] = [$host=127.0.0.1, $class="barnyard", $events=/Barnyard2::barnyard_alert/, $connect = F]
   };

Another limitation of the Barnaryd2 Bro output plugin is that if it tries to send events regarding non-TCP/UDP/ICMP ports, it may crash.  See [1] for a potential patch for that, which doesn't seem to have been reviewed or merged in to Barnyard2 yet.

- Jon

[1] http://mailman.icsi.berkeley.edu/pipermail/bro-dev/2013-January/005607.html


From jsiwek at illinois.edu  Wed Jan 22 11:46:24 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Wed, 22 Jan 2014 19:46:24 +0000
Subject: [Bro] bro bug? - unreported packet loss
In-Reply-To: <52DD93D3.5020108@tik.ee.ethz.ch>
References: <52C9B45F.3010505@tik.ee.ethz.ch> <52DD93D3.5020108@tik.ee.ethz.ch>
Message-ID: <86CE3251-CF40-43B2-851B-190942BF5B27@illinois.edu>


On Jan 20, 2014, at 3:23 PM, David Gugelmann <david.gugelmann at tik.ee.ethz.ch> wrote:

> sorry for bothering you again. As I posted two weeks ago (see below), it
> seems to me that bro sometimes does not report packet loss in TCP
> connections (missed_bytes in conn.log is 0 even though there are packets
> missing).
> 
> You can find the corresponding pcap files and additional information here:
> http://people.ee.ethz.ch/~gugdavid/bro_missed_bytes.zip
> 
> Am I missing something or did anybody encounter something similar?

You did seem to find situations where Bro doesn?t accurately report missed content.  Generally, I think it?s situations where packets are missed towards the end of connections, but specifically your example pcaps show slightly different special cases in the TCP reassembler?

example1.pcap: a gap is seen at the same time as a FIN from an endpoint and no content has been reassembled yet.  Code in TCP_Reassembler.cc skips reporting that on the chance the pcap trace if pre-filtered.  Associated comments:

 		// We could be running on a SYN/FIN/RST-filtered trace - don't
		// complain about data missing at the end of the connection.
		//
		// ### However, note that the preceding test is not a precise
		// one for filtered traces, and may fail, for example, when
		// the SYN packet carries data.
		//
		// Skip the undelivered part without reporting to the endpoint.

example2.pcap: a gap is seen, but the connection state is no longer established at both ends (a FIN has been seen from one).  Associated comments:

 			// Only report on content gaps for connections that
			// are in a cleanly established state.  In other
			// states, these can arise falsely due to things
			// like sequence number mismatches in RSTs, or
			// unseen previous packets in partial connections.
			// The one opportunity we lose here is on clean FIN
			// handshakes, but Oh Well.

The second case has a workaround by redefining the ?report_gaps_for_partial? [1] option to true at risk of unreliable gap reporting for partial connections.  There?s no workaround for the first case, but I think adding a similar redef-able option for explicitly indicating whether a filtered trace is used would be better than Bro just making a guess.

Would you create a bug report for this at bro-tracker.atlassian.net ?

- Jon

[1] http://www.bro.org/sphinx/scripts/base/init-bare.html#id-report_gaps_for_partial


From jeremy.cox at washk12.org  Wed Jan 22 11:48:18 2014
From: jeremy.cox at washk12.org (Jeremy Cox)
Date: Wed, 22 Jan 2014 12:48:18 -0700
Subject: [Bro] Attach Barnyard2 to Bro2
In-Reply-To: <586A8968-8CA0-410E-8864-F714AEB51CE4@illinois.edu>
References: <CAKdZko9YEwuN1X1MmFNuOv+xcg4OKhtiAiQ8M9rUHbwxCW4rdQ@mail.gmail.com>
	<586A8968-8CA0-410E-8864-F714AEB51CE4@illinois.edu>
Message-ID: <CAKdZko_nkOrWObMmPBP4TK=fCfK=s+v_BaAgT5efHF9JEi-qEA@mail.gmail.com>

Thanks Jon,  I'll give that a try.

Jeremy

*Jeremy Cox*
Senior Network Engineer, ISO

*Washington County School District*121 W Tabernacle - St. George - UT
435-634-4315
www.washk12.org
687474703a2f2f7777772e7375706572746563686775792e636f6d

IMPORTANT NOTICE REGARDING THIS ELECTRONIC COMMUNICATION:

This e-mail, including any attachments thereto, contains information that
may be confidential or privileged, and is intended solely for the
individual or entity to whom it is addressed.  Recipient is hereby notified
that any disclosure, copying or distribution of this message is strictly
prohibited.  IF YOU ARE NOT THE INTENDED RECIPIENT, please notify the
originator of this e-mail immediately and destroy all information
received.  Thank you.



On Wed, Jan 22, 2014 at 10:56 AM, Siwek, Jonathan Luke
<jsiwek at illinois.edu>wrote:

>
> On Jan 21, 2014, at 3:11 PM, Jeremy Cox <jeremy.cox at washk12.org> wrote:
>
> > I am attempting to use Barnyard2 to feed events from Suricata to Bro2.
>  It looks like Barnyard2 wants to access Bro on 47757/tcp.  Bro is not
> currently listening to that port.
>
> > Should bro be listening to 47757/tcp?
>
> Only if you?re running a command-line bro.  47757/tcp is the default
> listen port for running bro command-line style.
>
> > And Im not sure how to get it listening, or if thats just the old port
> it used to listen to.  I attempted to have barnyard connect to 47760 in
> standalone mode
>
> 47760/tcp is the default listen port of the bro instance of BroControl?s
> standalone node.
>
> > and 47761 or 47762 or 47763 in clustered mode.
>
> 47761/tcp is the listen port of the bro manager node when using BroControl
> to manage a closer.  It probably makes most sense to configure Barnyard2 to
> connect to this port if you?re running a Bro cluster.
>
> > In all instances Barnyard seems to connect to Bro, and then unexpectedly
> crashes.  Am I missing something?
>
> What versions of Bro and Barnyard2 are you using (and any particular
> configure flags for Barnyard2, like ?enable-ipv6) ?  IIRC I think you will
> need at least Bro 2.1 and Barnyard 2.1.10.
>
> I was able to get something simple working w/ latest Bro and Barnyard git
> master branches.  The only extra thing to do in Bro that?s not obvious is
> make sure the manager node has code to accept Barnyard alerts (putting it
> in site/local.bro is the simple way):
>
>    @load policy/integration/barnyard2
>    redef Communication::nodes += {
>        ["local"] = [$host=127.0.0.1, $class="barnyard",
> $events=/Barnyard2::barnyard_alert/, $connect = F]
>    };
>
> Another limitation of the Barnaryd2 Bro output plugin is that if it tries
> to send events regarding non-TCP/UDP/ICMP ports, it may crash.  See [1] for
> a potential patch for that, which doesn't seem to have been reviewed or
> merged in to Barnyard2 yet.
>
> - Jon
>
> [1]
> http://mailman.icsi.berkeley.edu/pipermail/bro-dev/2013-January/005607.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140122/add82124/attachment.html 

From gary at doit.wisc.edu  Thu Jan 23 14:25:47 2014
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Thu, 23 Jan 2014 16:25:47 -0600
Subject: [Bro] p0f OS fingerprinting question
Message-ID: <52E196EB.40100@doit.wisc.edu>

I was asked recently if Bro could do passive OS fingerprinting (in 
relation to keeping tabs on XP usage on our networks), and it seems that 
there was a dedicated mechanism for this using p0f, but that the Bro 
tie-ins may be deprecated per BIT 323 
<https://bro-tracker.atlassian.net/browse/BIT-323> or at least very out 
of date due to a 6 year p0f development hiatus. With p0f having been 
rewritten in 2012 are there there any plans for updating Bro to support 
the newer version? The user agent strings in software.log are useful, 
but it seems like there were even more fine grained events and records 
that came with the p0f tie-ins in regards to parsing out the OS.

Regards,
Gary

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140123/e90d8fba/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6257 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140123/e90d8fba/attachment.bin 

From mariusph at ifi.uio.no  Fri Jan 24 04:44:17 2014
From: mariusph at ifi.uio.no (Marius Portaas Haugen)
Date: Fri, 24 Jan 2014 13:44:17 +0100
Subject: [Bro] Extract files based on magic number using Bro 2.2
Message-ID: <23d46904ba85737947c876ae1b992af9@ulrik.uio.no>

Hi all!

I'm just wondering; Is it possible to extract files based solely on 
their magic number using Bro 2.2
In Bro 2.1, it was possible to extract files just by comparing the 
magic number
with the first X bytes. I used the script provided here, with great 
success:
http://scrapbook.zscaler.com/2012/05/bro-script-to-extract-artifacts-from.html

However, in Bro 2.2, thigs seem to have changed. Most examples and docs 
now only
seem to use the MIME-type to determine if a file will be extracted or 
not, e.g. here:
http://www.bro.org/sphinx-git/frameworks/file-analysis.html

I also see that there har been included some sort of "magic number 
database"(/bro/share/bro/magic/), but I find little
documentation on what its role is in regards of file extraction, as 
well as the formatting that is being used.

Have I missed something essential here?
If anyone could help me better understand how file extraction works now 
in Bro 2.2, it is most appreciated! :)

Best regards,
Marius P. Haugen.


From jsiwek at illinois.edu  Fri Jan 24 07:21:22 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Fri, 24 Jan 2014 15:21:22 +0000
Subject: [Bro] Extract files based on magic number using Bro 2.2
In-Reply-To: <23d46904ba85737947c876ae1b992af9@ulrik.uio.no>
References: <23d46904ba85737947c876ae1b992af9@ulrik.uio.no>
Message-ID: <353A35B2-F955-4560-922E-AA83A9830A74@illinois.edu>


On Jan 24, 2014, at 6:44 AM, Marius Portaas Haugen <mariusph at ifi.uio.no> wrote:

> I'm just wondering; Is it possible to extract files based solely on 
> their magic number using Bro 2.2
> In Bro 2.1, it was possible to extract files just by comparing the 
> magic number
> with the first X bytes. I used the script provided here, with great 
> success:
> http://scrapbook.zscaler.com/2012/05/bro-script-to-extract-artifacts-from.html
> 
> However, in Bro 2.2, thigs seem to have changed. Most examples and docs 
> now only
> seem to use the MIME-type to determine if a file will be extracted or 
> not, e.g. here:
> http://www.bro.org/sphinx-git/frameworks/file-analysis.html

You can try handling the ?file_new? event, compare f$bof_buffer (Beginning Of File Buffer) to whatever magic you want, and then add the file extraction analyzer to f if it matches (similar to the examples in that webpage you cite, except using f$bof_buffer as the condition instead of f$mime_type).

The ?file_new? event is network protocol agnostic so if it?s important to only extract stuff over HTTP, check the value of f$source to find the protocol over which it?s transferred.

> I also see that there har been included some sort of "magic number 
> database"(/bro/share/bro/magic/), but I find little
> documentation on what its role is in regards of file extraction, as 
> well as the formatting that is being used.

The magic database is used by libmagic (the library which implements [1]) to determine the value of f$mime_type.  See the magic(5) man page [2] for how magic files are formatted.

- Jon

[1] http://www.darwinsys.com/file/
[2] http://linux.die.net/man/5/magic


From jlay at slave-tothe-box.net  Fri Jan 24 08:24:11 2014
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 24 Jan 2014 09:24:11 -0700
Subject: [Bro] Quick Notice question
Message-ID: <b45eacec3b5fbcbe9eedaf1c3de2672e@localhost>

All,

So I was intrigued with the largeTx script.  As I'm not advanced with 
bro's scripting, I'm trying to change this to just notice instead of 
email:

#
# Define constants
# Define script and global variables
#
         redef enum Notice::Type += {            # add custom notice 
types
                 Large_Outgoing_Tx,
                 Very_Large_Outgoing_Tx,
         };

         const maxTx = 52428800;                 # single conn Tx bytes 
over which we want to alert on immediately
         const recordTx = 1024000;               # destination hosts to 
record if over this many bytes


#
# Send email if Very_Large_Outgoing_Tx
#
hook Notice::policy(n: Notice::Info)
     {
     if ( n$note == Very_Large_Outgoing_Tx )
          add n$actions[Notice::ACTION_EMAIL];
     }


#
# Alert on potential compromised internal hosts
#
event connection_state_remove(c: connection)
         {
         if (c$orig$size > recordTx)
                 {
                 # check to see if orig IP is an internal IP
                 if(!Site::is_local_addr(c$id$orig_h))
                         return;

                 # check to see if dest ip is not an internal IP
                 if(Site::is_local_addr(c$id$resp_h))
                         return;

                 if (c$orig$size > maxTx)
                         {
                         NOTICE([$note=Very_Large_Outgoing_Tx, 
$msg=fmt("Orig transmitted %d bytes to resp.  Duration %s sec.  
Connection UID %s.", c$orig$num_bytes_ip, c$duration, c$uid), $conn=c]);
                         }
                 else
                         {
                         NOTICE([$note=Large_Outgoing_Tx, $msg=fmt("Orig 
transmitted %d bytes to resp.  Duration %s sec.  Connection UID %s.", 
c$orig$num_bytes_ip, c$duration, c$uid), $conn=c]);
                         }
                 }
         }

Any assistance would be great..and thanks to the OP for this..it's a 
great idea.

James


From connar.rosebraugh at egov.com  Fri Jan 24 08:48:27 2014
From: connar.rosebraugh at egov.com (Rosebraugh, Connar)
Date: Fri, 24 Jan 2014 16:48:27 +0000
Subject: [Bro] Logstash vs. Bro ElasticSearch Writer
Message-ID: <088ED11BA811374BACE1259396F485E30151FFC1@VADC-MBX02.ad.cdc.nicusa.com>

Hi all!

I am trying to decide if I should use the Bro ElasticSearch writer or Logstash for generating ElasticSearch logs. I found this<http://www.appliednsm.com/parsing-bro-logs-with-logstash/> great write-up on using Logstash for creating ElasticSearch logs. What are the differences between the two? I know that Kibana has difficulty making sense of the ElasticSearch logs that Bro writes, especially with respect to the time that events occur.

Thanks,
Connar Rosebraugh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140124/c0d7e668/attachment.html 

From bkellogg at dresser-rand.com  Fri Jan 24 08:48:40 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Fri, 24 Jan 2014 16:48:40 +0000
Subject: [Bro] Quick Notice question
In-Reply-To: <b45eacec3b5fbcbe9eedaf1c3de2672e@localhost>
References: <b45eacec3b5fbcbe9eedaf1c3de2672e@localhost>
Message-ID: <BDF3793B557F624699D4C68A3B8FCF820BA3CB@DRTEXW238VM.na.d-rco.com>

I've added a little more smarts to the script as I become more familiar with bro scripting.  I'm simply amazed at the possibilities of Bro; thank you to those who have and continue to develop this awesome tool.  I wish I had run across it five years ago.  Attached is the current iteration.  I'm trying to keep track of and alert on hosts that have multiple large upload events in a given time and any destination hosts that have seen multiple uploads over a given time.  To disable the mail alerts just comment out the below.  If any of my inline comments are unclear yell at me.

#
# Send email if Very_Large_Outgoing_Tx notice type is generated
#
hook Notice::policy(n: Notice::Info)
    {
    if (n$note == Very_Large_Outgoing_Tx || n$note == Multiple_Large_Outgoing_Tx)
		add n$actions[Notice::ACTION_EMAIL];
    }

Thank you,
Brian Kellogg


-----Original Message-----
From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of James Lay
Sent: Friday, January 24, 2014 11:24 AM
To: Bro
Subject: [Bro] Quick Notice question

All,

So I was intrigued with the largeTx script.  As I'm not advanced with bro's scripting, I'm trying to change this to just notice instead of
email:

#
# Define constants
# Define script and global variables
#
         redef enum Notice::Type += {            # add custom notice 
types
                 Large_Outgoing_Tx,
                 Very_Large_Outgoing_Tx,
         };

         const maxTx = 52428800;                 # single conn Tx bytes 
over which we want to alert on immediately
         const recordTx = 1024000;               # destination hosts to 
record if over this many bytes


#
# Send email if Very_Large_Outgoing_Tx
#
hook Notice::policy(n: Notice::Info)
     {
     if ( n$note == Very_Large_Outgoing_Tx )
          add n$actions[Notice::ACTION_EMAIL];
     }


#
# Alert on potential compromised internal hosts
#
event connection_state_remove(c: connection)
         {
         if (c$orig$size > recordTx)
                 {
                 # check to see if orig IP is an internal IP
                 if(!Site::is_local_addr(c$id$orig_h))
                         return;

                 # check to see if dest ip is not an internal IP
                 if(Site::is_local_addr(c$id$resp_h))
                         return;

                 if (c$orig$size > maxTx)
                         {
                         NOTICE([$note=Very_Large_Outgoing_Tx, 
$msg=fmt("Orig transmitted %d bytes to resp.  Duration %s sec.  
Connection UID %s.", c$orig$num_bytes_ip, c$duration, c$uid), $conn=c]);
                         }
                 else
                         {
                         NOTICE([$note=Large_Outgoing_Tx, $msg=fmt("Orig 
transmitted %d bytes to resp.  Duration %s sec.  Connection UID %s.", 
c$orig$num_bytes_ip, c$duration, c$uid), $conn=c]);
                         }
                 }
         }

Any assistance would be great..and thanks to the OP for this..it's a 
great idea.

James
_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: largeTx.bro
Type: application/octet-stream
Size: 4268 bytes
Desc: largeTx.bro
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140124/93e36818/attachment.obj 

From jlay at slave-tothe-box.net  Fri Jan 24 09:05:12 2014
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 24 Jan 2014 10:05:12 -0700
Subject: [Bro] Quick Notice question
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF820BA3CB@DRTEXW238VM.na.d-rco.com>
References: <b45eacec3b5fbcbe9eedaf1c3de2672e@localhost>
	<BDF3793B557F624699D4C68A3B8FCF820BA3CB@DRTEXW238VM.na.d-rco.com>
Message-ID: <0b9230fef2d114c6db92d6be855a01a0@localhost>

On 2014-01-24 09:48, Kellogg, Brian D (OLN) wrote:
> I've added a little more smarts to the script as I become more
> familiar with bro scripting.  I'm simply amazed at the possibilities
> of Bro; thank you to those who have and continue to develop this
> awesome tool.  I wish I had run across it five years ago.  Attached 
> is
> the current iteration.  I'm trying to keep track of and alert on 
> hosts
> that have multiple large upload events in a given time and any
> destination hosts that have seen multiple uploads over a given time.
> To disable the mail alerts just comment out the below.  If any of my
> inline comments are unclear yell at me.
>
> #
> # Send email if Very_Large_Outgoing_Tx notice type is generated
> #
> hook Notice::policy(n: Notice::Info)
>     {
>     if (n$note == Very_Large_Outgoing_Tx || n$note ==
> Multiple_Large_Outgoing_Tx)
> 		add n$actions[Notice::ACTION_EMAIL];
>     }
>
> Thank you,
> Brian Kellogg

Thanks for this Brian...working with it now.

James


From jlay at slave-tothe-box.net  Fri Jan 24 10:21:39 2014
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 24 Jan 2014 11:21:39 -0700
Subject: [Bro] Quick Notice question
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF820BA3CB@DRTEXW238VM.na.d-rco.com>
References: <b45eacec3b5fbcbe9eedaf1c3de2672e@localhost>
	<BDF3793B557F624699D4C68A3B8FCF820BA3CB@DRTEXW238VM.na.d-rco.com>
Message-ID: <d4e05821a8dea315563cb94a8bb0588b@localhost>

On 2014-01-24 09:48, Kellogg, Brian D (OLN) wrote:
> I've added a little more smarts to the script as I become more
> familiar with bro scripting.  I'm simply amazed at the possibilities
> of Bro; thank you to those who have and continue to develop this
> awesome tool.  I wish I had run across it five years ago.  Attached 
> is
> the current iteration.  I'm trying to keep track of and alert on 
> hosts
> that have multiple large upload events in a given time and any
> destination hosts that have seen multiple uploads over a given time.
> To disable the mail alerts just comment out the below.  If any of my
> inline comments are unclear yell at me.

So I've completely removed and re-installed bro-2.2.  Here's what I get 
when I try and test the script:

[11:11:47 @analysis:~/brostuff/testbrofiles$] bro largeTx.bro
error in ./largeTx.bro, line 7: unrecognized character -
error in ./largeTx.bro, line 8: unrecognized character -
<redacted>
error in ./largeTx.bro, line 96: unrecognized character -
error in ./largeTx.bro, line 97: unrecognized character -

I've tried just downloading the file from email, copying and pasting as 
text, and even getting rid of the tab control characters.  The below 
snippet works though:

event bro_init()
{
         print "Hello World!";
}

[11:15:15 @analysis:~/brostuff/testbrofiles$] bro helloworld.bro
Hello World!


If I copy the script to /usr/local/bro/share/bro/site, add it to 
local.bro with "@load largeTx.bro", start broctl, install, then start, I 
get a fail and diag shows the same as above.
Am I missing something obvious?  Thank you.

James


From bkellogg at dresser-rand.com  Fri Jan 24 10:44:30 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Fri, 24 Jan 2014 18:44:30 +0000
Subject: [Bro] Quick Notice question
In-Reply-To: <d4e05821a8dea315563cb94a8bb0588b@localhost>
References: <b45eacec3b5fbcbe9eedaf1c3de2672e@localhost>
	<BDF3793B557F624699D4C68A3B8FCF820BA3CB@DRTEXW238VM.na.d-rco.com>
	<d4e05821a8dea315563cb94a8bb0588b@localhost>
Message-ID: <BDF3793B557F624699D4C68A3B8FCF820BA630@DRTEXW238VM.na.d-rco.com>

Hmm, not sure.

I usually edit the script with Notepad++ and then copy it into a Vi session on the SecurityOnion sensor.  Line 8 is a comment; at least on mine.

Try using "broctl check" and "broctl diag" after stopping bro and see if that turns up anything.  I'm pretty new to Bro so hopefully I'm not leading you down the wrong path.


Thank you,
Brian Kellogg
Security Analyst; IT Governance, Risk, and Compliance
500 Paul Clark Drive, Olean,  NY 14760
T: (716) 375-3186 | F: (716) 375-3557


-----Original Message-----
From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of James Lay
Sent: Friday, January 24, 2014 1:22 PM
To: Bro
Subject: Re: [Bro] Quick Notice question

On 2014-01-24 09:48, Kellogg, Brian D (OLN) wrote:
> I've added a little more smarts to the script as I become more 
> familiar with bro scripting.  I'm simply amazed at the possibilities 
> of Bro; thank you to those who have and continue to develop this 
> awesome tool.  I wish I had run across it five years ago.  Attached is 
> the current iteration.  I'm trying to keep track of and alert on hosts 
> that have multiple large upload events in a given time and any 
> destination hosts that have seen multiple uploads over a given time.
> To disable the mail alerts just comment out the below.  If any of my 
> inline comments are unclear yell at me.

So I've completely removed and re-installed bro-2.2.  Here's what I get when I try and test the script:

[11:11:47 @analysis:~/brostuff/testbrofiles$] bro largeTx.bro error in ./largeTx.bro, line 7: unrecognized character - error in ./largeTx.bro, line 8: unrecognized character - <redacted> error in ./largeTx.bro, line 96: unrecognized character - error in ./largeTx.bro, line 97: unrecognized character -

I've tried just downloading the file from email, copying and pasting as text, and even getting rid of the tab control characters.  The below snippet works though:

event bro_init()
{
         print "Hello World!";
}

[11:15:15 @analysis:~/brostuff/testbrofiles$] bro helloworld.bro Hello World!


If I copy the script to /usr/local/bro/share/bro/site, add it to local.bro with "@load largeTx.bro", start broctl, install, then start, I get a fail and diag shows the same as above.
Am I missing something obvious?  Thank you.

James
_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From jlay at slave-tothe-box.net  Fri Jan 24 13:13:25 2014
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 24 Jan 2014 14:13:25 -0700
Subject: [Bro] Quick Notice question
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF820BA630@DRTEXW238VM.na.d-rco.com>
References: <b45eacec3b5fbcbe9eedaf1c3de2672e@localhost>
	<BDF3793B557F624699D4C68A3B8FCF820BA3CB@DRTEXW238VM.na.d-rco.com>
	<d4e05821a8dea315563cb94a8bb0588b@localhost>
	<BDF3793B557F624699D4C68A3B8FCF820BA630@DRTEXW238VM.na.d-rco.com>
Message-ID: <3b04459d101ca35e9d0af41c31778478@localhost>

On 2014-01-24 11:44, Kellogg, Brian D (OLN) wrote:
> Hmm, not sure.
>
> I usually edit the script with Notepad++ and then copy it into a Vi
> session on the SecurityOnion sensor.  Line 8 is a comment; at least 
> on
> mine.
>
> Try using "broctl check" and "broctl diag" after stopping bro and see
> if that turns up anything.  I'm pretty new to Bro so hopefully I'm 
> not
> leading you down the wrong path.
>
>
> Thank you,
> Brian Kellogg

Bah...turns out something funky with copy/paste 8-|  Opened the file in 
Notepad++ then pasted into a new file on the linux box and away it 
went..thanks again Brian.

James



From jes.smith.bro at aol.com  Sun Jan 26 10:27:42 2014
From: jes.smith.bro at aol.com (Jessica Smith)
Date: Sun, 26 Jan 2014 13:27:42 -0500 (EST)
Subject: [Bro] why x509_extensions event never called?
Message-ID: <8D0E8DB6FAA348C-2740-14137@webmail-m133.sysops.aol.com>

Hi all,
I'm analyzing X509 certificates with extensions, I defined a handler for x509_extensions event, but it is never called!!! Why doesn't it work?



event x509_extension(c: connection, is_orig: bool, data: string) 
{
	print "THERE'S AN EXTENSION!";
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140126/5c7294fd/attachment.html 

From mariusph at ifi.uio.no  Sun Jan 26 13:23:39 2014
From: mariusph at ifi.uio.no (Marius Portaas Haugen)
Date: Sun, 26 Jan 2014 22:23:39 +0100
Subject: [Bro] Extract files based on magic number using Bro 2.2
In-Reply-To: <353A35B2-F955-4560-922E-AA83A9830A74@illinois.edu>
References: <23d46904ba85737947c876ae1b992af9@ulrik.uio.no>
	<353A35B2-F955-4560-922E-AA83A9830A74@illinois.edu>
Message-ID: <9f9db7dd27863fc29056b9d3f0cd2fc6@ulrik.uio.no>

Hi Jon,

thanks a lot for taking the time to answer my question!
Comparing the the bof_buffer works like a charm! Again, thanks!

- Marius

On 2014-01-24 16:21, Siwek, Jonathan Luke wrote:
> On Jan 24, 2014, at 6:44 AM, Marius Portaas Haugen 
> <mariusph at ifi.uio.no> wrote:
> 
>> I'm just wondering; Is it possible to extract files based solely on
>> their magic number using Bro 2.2
>> In Bro 2.1, it was possible to extract files just by comparing the
>> magic number
>> with the first X bytes. I used the script provided here, with great
>> success:
>> http://scrapbook.zscaler.com/2012/05/bro-script-to-extract-artifacts-from.html
>> 
>> However, in Bro 2.2, thigs seem to have changed. Most examples and 
>> docs
>> now only
>> seem to use the MIME-type to determine if a file will be extracted or
>> not, e.g. here:
>> http://www.bro.org/sphinx-git/frameworks/file-analysis.html
> 
> You can try handling the ?file_new? event, compare f$bof_buffer
> (Beginning Of File Buffer) to whatever magic you want, and then add
> the file extraction analyzer to f if it matches (similar to the
> examples in that webpage you cite, except using f$bof_buffer as the
> condition instead of f$mime_type).
> 
> The ?file_new? event is network protocol agnostic so if it?s
> important to only extract stuff over HTTP, check the value of f$source
> to find the protocol over which it?s transferred.
> 
>> I also see that there har been included some sort of "magic number
>> database"(/bro/share/bro/magic/), but I find little
>> documentation on what its role is in regards of file extraction, as
>> well as the formatting that is being used.
> 
> The magic database is used by libmagic (the library which implements
> [1]) to determine the value of f$mime_type.  See the magic(5) man page
> [2] for how magic files are formatted.
> 
> - Jon
> 
> [1] http://www.darwinsys.com/file/
> [2] http://linux.die.net/man/5/magic


From seth at icir.org  Mon Jan 27 07:02:39 2014
From: seth at icir.org (Seth Hall)
Date: Mon, 27 Jan 2014 10:02:39 -0500
Subject: [Bro] why x509_extensions event never called?
In-Reply-To: <8D0E8DB6FAA348C-2740-14137@webmail-m133.sysops.aol.com>
References: <8D0E8DB6FAA348C-2740-14137@webmail-m133.sysops.aol.com>
Message-ID: <F831675F-3648-45BD-95F8-3C385F98774E@icir.org>


On Jan 26, 2014, at 1:27 PM, Jessica Smith <jes.smith.bro at aol.com> wrote:

> event x509_extension(c: connection, is_orig: bool, data: string) 
> {
> 	print "THERE'S AN EXTENSION!";
> }

That looks correct.  You're going to have to give us more information and ideally a trace file.  Also, consider that if you aren't seeing certificates that have extensions you won't see this event fire.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140127/1b937d78/attachment.bin 

From seth at icir.org  Mon Jan 27 07:06:25 2014
From: seth at icir.org (Seth Hall)
Date: Mon, 27 Jan 2014 10:06:25 -0500
Subject: [Bro] Logstash vs. Bro ElasticSearch Writer
In-Reply-To: <088ED11BA811374BACE1259396F485E30151FFC1@VADC-MBX02.ad.cdc.nicusa.com>
References: <088ED11BA811374BACE1259396F485E30151FFC1@VADC-MBX02.ad.cdc.nicusa.com>
Message-ID: <E6AC71AC-11BA-48C6-9DA8-2CFAC2ED740E@icir.org>


On Jan 24, 2014, at 11:48 AM, "Rosebraugh, Connar" <connar.rosebraugh at egov.com> wrote:

> Hi all!
>  
> I am trying to decide if I should use the Bro ElasticSearch writer or Logstash for generating ElasticSearch logs. I found this great write-up on using Logstash for creating ElasticSearch logs. What are the differences between the two? I know that Kibana has difficulty making sense of the ElasticSearch logs that Bro writes, especially with respect to the time that events occur.

Kibana is actually fine with it you just have to make sure and configure your dashboard to use the "ts" field as time instead of @timestamp.  I do believe there are some other minor issues too because I know when I was playing with Kibana I made some small patches to Bro (I don't recall how necessary they were though).  We are hoping soon to come back around to Bro+Elasticsearch and make that much higher performance and more reliable though.  Right now there are some issues with it under extremely high load and we still don't feel completely comfortable marking it as production ready.

I will say that I don't particularly like the way that people use logstash to push logs into elasticsearch either.  Hopefully we'll have better guidance and support for this soon.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140127/93b43919/attachment.bin 

From seth at icir.org  Mon Jan 27 07:15:45 2014
From: seth at icir.org (Seth Hall)
Date: Mon, 27 Jan 2014 10:15:45 -0500
Subject: [Bro] p0f OS fingerprinting question
In-Reply-To: <52E196EB.40100@doit.wisc.edu>
References: <52E196EB.40100@doit.wisc.edu>
Message-ID: <CDD8B876-AA78-41F0-BB00-A26F14649209@icir.org>


On Jan 23, 2014, at 5:25 PM, Gary Faulkner <gary at doit.wisc.edu> wrote:

> at least very out of date due to a 6 year p0f development hiatus.

Yep.  It's in Bro still though.  If you write a script that handles this event:

	event OS_version_found(c: connection, host: addr, OS: OS_version)
		{
		print cat("p0f reported - ", OS);
		}

you will be getting the output from p0f.

>  With p0f having been rewritten in 2012 are there there any plans for updating Bro to support the newer version?

Nope, I don't think anyone plans on updating it.  The author of p0f stopped trying to fingerprint TCP stacks (mostly) and started using other deeper packet sniffing. :)  Basically the new version of p0f is something you could implement as a Bro script because he's just grabbing user-agent strings and stuff.  The problem is that it's really hard to blindly trust user-agent strings because of NAT'ed addressed and people giving fake user-agent strings.  

I'm hoping eventually in Bro to write a script that takes lots of measurements (p0f, user-agents, software update mechanisms touches, exposed services) to get a profile for a machine to decide if it's a particular type of host.  For instance, imagine that one of your windows xp machines gets identified as such by p0f (in Bro), then identifies that it's windows xp in a browser user-agent, then reaches out for windows updates and identifies that it's version of windows is some version of XP (i can't remember if this is visible for Windows system updates or not).

Anyway, this approach is replicable for many other operating systems too, it just takes time and the Bro scripts to support it.

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140127/dcbfac31/attachment.bin 

From jes.smith.bro at aol.com  Mon Jan 27 08:50:03 2014
From: jes.smith.bro at aol.com (Jessica Smith)
Date: Mon, 27 Jan 2014 11:50:03 -0500 (EST)
Subject: [Bro] why x509_extensions event never called?
In-Reply-To: <F831675F-3648-45BD-95F8-3C385F98774E@icir.org>
References: <8D0E8DB6FAA348C-2740-14137@webmail-m133.sysops.aol.com>
	<F831675F-3648-45BD-95F8-3C385F98774E@icir.org>
Message-ID: <8D0E996F5A41045-2740-1967C@webmail-m133.sysops.aol.com>

Hi Seth,
thanks for reply, but there is no more information to give you. I just visited the site www.paypal.com and all SSL events (ssl_client_hello, ssl_server_hello, ssl_established, x509_certificate) are fired except x509_extension. I cannot understand why, but the Paypal's certificate contains many extensions.





module MYMODULE;


export {
}


event bro_init() {
# fires
}


event ssl_client_hello(c: connection, version: count, possible_ts: time, client_random: string, session_id: string, ciphers: index_vec)
{
# fires
}


event ssl_server_hello(c: connection, version: count, possible_ts: time, server_random: string, session_id: string, cipher: count, comp_method: count)
{
# fires
}


event ssl_established(c: connection) 
{
# fires
}


event x509_extension(c: connection, is_orig: bool, is_critical: bool, name: string, value: string) 
{
print "THERE'S AN EXTENSION!";  # no fire
}


event x509_certificate(c: connection , is_orig: bool , cert: X509 , chain_idx: count , chain_len: count , der_cert: string )
{

# fires
}




Jessica


-----Original Message-----
From: Seth Hall <seth at icir.org>
To: Jessica Smith <jes.smith.bro at aol.com>
Cc: bro <bro at bro.org>
Sent: Mon, Jan 27, 2014 4:02 pm
Subject: Re: [Bro] why x509_extensions event never called?



On Jan 26, 2014, at 1:27 PM, Jessica Smith <jes.smith.bro at aol.com> wrote:

> event x509_extension(c: connection, is_orig: bool, data: string) 
> {
> 	print "THERE'S AN EXTENSION!";
> }

That looks correct.  You're going to have to give us more information and 
ideally a trace file.  Also, consider that if you aren't seeing certificates 
that have extensions you won't see this event fire.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/


 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140127/a1b43991/attachment.html 

From bernhard at ICSI.Berkeley.EDU  Mon Jan 27 09:00:59 2014
From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann)
Date: Mon, 27 Jan 2014 09:00:59 -0800
Subject: [Bro] why x509_extensions event never called?
In-Reply-To: <F831675F-3648-45BD-95F8-3C385F98774E@icir.org>
References: <8D0E8DB6FAA348C-2740-14137@webmail-m133.sysops.aol.com>
	<F831675F-3648-45BD-95F8-3C385F98774E@icir.org>
Message-ID: <68EDA54C-0020-40EE-BBD9-68C5924FF912@icsi.berkeley.edu>


On Jan 27, 2014, at 7:02 AM, Seth Hall <seth at icir.org> wrote:
> On Jan 26, 2014, at 1:27 PM, Jessica Smith <jes.smith.bro at aol.com> wrote:
> 
>> event x509_extension(c: connection, is_orig: bool, data: string) 
>> {
>> 	print "THERE'S AN EXTENSION!";
>> }
> 
> That looks correct.  You're going to have to give us more information and ideally a trace file.  Also, consider that if you aren't seeing certificates that have extensions you won't see this event fire.

Sorry, this one is our fault. At the moment, the x509_extension event never fires due to
a bug in the SSL analyzer.

I have this fixed in a branch ? however the fix never made it back into the Bro master
(because the branch also changes a number of other things about x509 certificate 
handling; some of them are not quite working yet).

I will backport that little part and post a patch in a bit.

Bernhard




From jeremy.cox at washk12.org  Mon Jan 27 09:54:16 2014
From: jeremy.cox at washk12.org (Jeremy Cox)
Date: Mon, 27 Jan 2014 10:54:16 -0700
Subject: [Bro] Attach Barnyard2 to Bro2
In-Reply-To: <586A8968-8CA0-410E-8864-F714AEB51CE4@illinois.edu>
References: <CAKdZko9YEwuN1X1MmFNuOv+xcg4OKhtiAiQ8M9rUHbwxCW4rdQ@mail.gmail.com>
	<586A8968-8CA0-410E-8864-F714AEB51CE4@illinois.edu>
Message-ID: <CAKdZko_XLSQ9z7pQ=JKJpuzYAiCRAGbzE_prC639S8XPYSSO+Q@mail.gmail.com>

That seems to be working well.  I had to upgrade barnyard, and I also
patched barnyard2 with the patch listed.  Barnyard appears to be pushing
the alerts into Bro.  At least it has not crashed :).

But I'm not sure where those alerts end up being logged in bro.

/mnt/iscsi/bro/logs/current# ls

communication.log  dhcp.log  dpd.log    ftp.log   known_hosts.log
smtp.log   software.log  stderr.log  syslog.log  weird.log

conn.log           dns.log   files.log  http.log  known_services.log
socks.log  ssl.log       stdout.log  tunnel.log


I haven't seen any new files show up.

I am hopeful that I can tie the two IDSs together in the same set of logs
and rotations.  So that when I analyze the logs, they are both from the
same time period.

Jeremy

*Jeremy Cox*
Senior Network Engineer, ISO

*Washington County School District*121 W Tabernacle - St. George - UT
435-634-4315
www.washk12.org
687474703a2f2f7777772e7375706572746563686775792e636f6d

IMPORTANT NOTICE REGARDING THIS ELECTRONIC COMMUNICATION:

This e-mail, including any attachments thereto, contains information that
may be confidential or privileged, and is intended solely for the
individual or entity to whom it is addressed.  Recipient is hereby notified
that any disclosure, copying or distribution of this message is strictly
prohibited.  IF YOU ARE NOT THE INTENDED RECIPIENT, please notify the
originator of this e-mail immediately and destroy all information
received.  Thank you.



On Wed, Jan 22, 2014 at 10:56 AM, Siwek, Jonathan Luke
<jsiwek at illinois.edu>wrote:

>
> On Jan 21, 2014, at 3:11 PM, Jeremy Cox <jeremy.cox at washk12.org> wrote:
>
> > I am attempting to use Barnyard2 to feed events from Suricata to Bro2.
>  It looks like Barnyard2 wants to access Bro on 47757/tcp.  Bro is not
> currently listening to that port.
>
> > Should bro be listening to 47757/tcp?
>
> Only if you?re running a command-line bro.  47757/tcp is the default
> listen port for running bro command-line style.
>
> > And Im not sure how to get it listening, or if thats just the old port
> it used to listen to.  I attempted to have barnyard connect to 47760 in
> standalone mode
>
> 47760/tcp is the default listen port of the bro instance of BroControl?s
> standalone node.
>
> > and 47761 or 47762 or 47763 in clustered mode.
>
> 47761/tcp is the listen port of the bro manager node when using BroControl
> to manage a closer.  It probably makes most sense to configure Barnyard2 to
> connect to this port if you?re running a Bro cluster.
>
> > In all instances Barnyard seems to connect to Bro, and then unexpectedly
> crashes.  Am I missing something?
>
> What versions of Bro and Barnyard2 are you using (and any particular
> configure flags for Barnyard2, like ?enable-ipv6) ?  IIRC I think you will
> need at least Bro 2.1 and Barnyard 2.1.10.
>
> I was able to get something simple working w/ latest Bro and Barnyard git
> master branches.  The only extra thing to do in Bro that?s not obvious is
> make sure the manager node has code to accept Barnyard alerts (putting it
> in site/local.bro is the simple way):
>
>    @load policy/integration/barnyard2
>    redef Communication::nodes += {
>        ["local"] = [$host=127.0.0.1, $class="barnyard",
> $events=/Barnyard2::barnyard_alert/, $connect = F]
>    };
>
> Another limitation of the Barnaryd2 Bro output plugin is that if it tries
> to send events regarding non-TCP/UDP/ICMP ports, it may crash.  See [1] for
> a potential patch for that, which doesn't seem to have been reviewed or
> merged in to Barnyard2 yet.
>
> - Jon
>
> [1]
> http://mailman.icsi.berkeley.edu/pipermail/bro-dev/2013-January/005607.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140127/f95f7457/attachment.html 

From jsiwek at illinois.edu  Mon Jan 27 10:14:31 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Mon, 27 Jan 2014 18:14:31 +0000
Subject: [Bro] Attach Barnyard2 to Bro2
In-Reply-To: <CAKdZko_XLSQ9z7pQ=JKJpuzYAiCRAGbzE_prC639S8XPYSSO+Q@mail.gmail.com>
References: <CAKdZko9YEwuN1X1MmFNuOv+xcg4OKhtiAiQ8M9rUHbwxCW4rdQ@mail.gmail.com>
	<586A8968-8CA0-410E-8864-F714AEB51CE4@illinois.edu>
	<CAKdZko_XLSQ9z7pQ=JKJpuzYAiCRAGbzE_prC639S8XPYSSO+Q@mail.gmail.com>
Message-ID: <D3B40A9E-F305-4B23-ADA4-E354CE870384@illinois.edu>


On Jan 27, 2014, at 11:54 AM, Jeremy Cox <jeremy.cox at washk12.org> wrote:

> But I'm not sure where those alerts end up being logged in bro.

It produces a barnyard2.log.

Did you redef Communication::nodes to register Bro to receive the barnyard events?  E.g. in site/local.bro put code like the following:

 @load policy/integration/barnyard2
  redef Communication::nodes += {
      ["local"] = [$host=127.0.0.1, $class="barnyard", $events=/Barnyard2::barnyard_alert/, $connect = F]
  };

- Jon


From bernhard at ICSI.Berkeley.EDU  Mon Jan 27 10:36:40 2014
From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann)
Date: Mon, 27 Jan 2014 10:36:40 -0800
Subject: [Bro] why x509_extensions event never called?
In-Reply-To: <8D0E996F5A41045-2740-1967C@webmail-m133.sysops.aol.com>
References: <8D0E8DB6FAA348C-2740-14137@webmail-m133.sysops.aol.com>
	<F831675F-3648-45BD-95F8-3C385F98774E@icir.org>
	<8D0E996F5A41045-2740-1967C@webmail-m133.sysops.aol.com>
Message-ID: <6A8B39E2-2351-4B99-A472-081A4905A00C@icsi.berkeley.edu>


On Jan 27, 2014, at 8:50 AM, Jessica Smith <jes.smith.bro at aol.com> wrote:

> Hi Seth,
> thanks for reply, but there is no more information to give you. I just visited the site www.paypal.com and all SSL events (ssl_client_hello, ssl_server_hello, ssl_established, x509_certificate) are fired except x509_extension. I cannot understand why, but the Paypal's certificate contains many extensions.

Hello Jessica,

a patch for the x509_extension event is in the topic/bernhard/fix-x509-extensions git branch. 

The event syntax slightly changed - using

event x509_extension(c: connection, is_orig: bool, cert:X509, extension: X509_extension_info) 
{
	print extension;
}

should work now.

Bernhard


From jeremy.cox at washk12.org  Mon Jan 27 10:57:05 2014
From: jeremy.cox at washk12.org (Jeremy Cox)
Date: Mon, 27 Jan 2014 11:57:05 -0700
Subject: [Bro] Attach Barnyard2 to Bro2
In-Reply-To: <D3B40A9E-F305-4B23-ADA4-E354CE870384@illinois.edu>
References: <CAKdZko9YEwuN1X1MmFNuOv+xcg4OKhtiAiQ8M9rUHbwxCW4rdQ@mail.gmail.com>
	<586A8968-8CA0-410E-8864-F714AEB51CE4@illinois.edu>
	<CAKdZko_XLSQ9z7pQ=JKJpuzYAiCRAGbzE_prC639S8XPYSSO+Q@mail.gmail.com>
	<D3B40A9E-F305-4B23-ADA4-E354CE870384@illinois.edu>
Message-ID: <CAKdZko-q8eXR7VgXWuSrhAFwmr0Q85bXKKf1A0vktsn_BT+AtA@mail.gmail.com>

I did.  I have also put it back into standalone mode to see if that did
it.... No luck.

/opt/bro2/share/bro/site# cat local.bro
##! Local site policy. Customize as appropriate.
##!
##! This file will not be overwritten when upgrading or reinstalling!

# This script logs which scripts were loaded during each run.
@load misc/loaded-scripts

# Apply the default tuning scripts for common tuning settings.
@load tuning/defaults

# Load the scan detection script.
@load misc/scan

# Log some information about web applications being used by users
# on your network.
@load misc/app-stats

# Detect traceroute being run on the network.
@load misc/detect-traceroute

# Generate notices when vulnerable versions of software are discovered.
# The default is to only monitor software found in the address space defined
# as "local".  Refer to the software framework's documentation for more
# information.
@load frameworks/software/vulnerable

# Detect software changing (e.g. attacker installing hacked SSHD).
@load frameworks/software/version-changes

# This adds signatures to detect cleartext forward and reverse windows
shells.
@load-sigs frameworks/signatures/detect-windows-shells

# Uncomment the following line to begin receiving (by default hourly) emails
# containing all of your notices.
# redef Notice::policy += { [$action = Notice::ACTION_ALARM, $priority = 0]
};

# Load all of the scripts that detect software in various protocols.
@load protocols/ftp/software
@load protocols/smtp/software
@load protocols/ssh/software
@load protocols/http/software
# The detect-webapps script could possibly cause performance trouble when
# running on live traffic.  Enable it cautiously.
#@load protocols/http/detect-webapps

# This script detects DNS results pointing toward your Site::local_nets
# where the name is not part of your local DNS zone and is being hosted
# externally.  Requires that the Site::local_zones variable is defined.
@load protocols/dns/detect-external-names

# Script to detect various activity in FTP sessions.
@load protocols/ftp/detect

# Scripts that do asset tracking.
@load protocols/conn/known-hosts
@load protocols/conn/known-services
@load protocols/ssl/known-certs

# This script enables SSL/TLS certificate validation.
@load protocols/ssl/validate-certs

# Uncomment the following line to check each SSL certificate hash against
the ICSI
# certificate notary service; see http://notary.icsi.berkeley.edu .
# @load protocols/ssl/notary

# If you have libGeoIP support built in, do some geographic detections and
# logging for SSH traffic.
@load protocols/ssh/geo-data
# Detect hosts doing SSH bruteforce attacks.
@load protocols/ssh/detect-bruteforcing
# Detect logins using "interesting" hostnames.
@load protocols/ssh/interesting-hostnames

# Detect SQL injection attacks.
@load protocols/http/detect-sqli

#### Network File Handling ####

# Enable MD5 and SHA1 hashing for all files.
@load frameworks/files/hash-all-files

# Detect SHA1 sums in Team Cymru's Malware Hash Registry.
@load frameworks/files/detect-MHR

@load policy/integration/barnyard2
   redef Communication::nodes += {
       ["local"] = [$host=127.0.0.1, $class="barnyard",
$events=/Barnyard2::barnyard_alert/, $connect = F]
   };

*Jeremy Cox*
Senior Network Engineer, ISO

*Washington County School District*121 W Tabernacle - St. George - UT
435-634-4315
www.washk12.org
687474703a2f2f7777772e7375706572746563686775792e636f6d

IMPORTANT NOTICE REGARDING THIS ELECTRONIC COMMUNICATION:

This e-mail, including any attachments thereto, contains information that
may be confidential or privileged, and is intended solely for the
individual or entity to whom it is addressed.  Recipient is hereby notified
that any disclosure, copying or distribution of this message is strictly
prohibited.  IF YOU ARE NOT THE INTENDED RECIPIENT, please notify the
originator of this e-mail immediately and destroy all information
received.  Thank you.



On Mon, Jan 27, 2014 at 11:14 AM, Siwek, Jonathan Luke
<jsiwek at illinois.edu>wrote:

>
> On Jan 27, 2014, at 11:54 AM, Jeremy Cox <jeremy.cox at washk12.org> wrote:
>
> > But I'm not sure where those alerts end up being logged in bro.
>
> It produces a barnyard2.log.
>
> Did you redef Communication::nodes to register Bro to receive the barnyard
> events?  E.g. in site/local.bro put code like the following:
>
>  @load policy/integration/barnyard2
>   redef Communication::nodes += {
>       ["local"] = [$host=127.0.0.1, $class="barnyard",
> $events=/Barnyard2::barnyard_alert/, $connect = F]
>   };
>
> - Jon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140127/751d71af/attachment.html 

From jes.smith.bro at aol.com  Mon Jan 27 11:21:48 2014
From: jes.smith.bro at aol.com (Jessica Smith)
Date: Mon, 27 Jan 2014 14:21:48 -0500 (EST)
Subject: [Bro] why x509_extensions event never called?
In-Reply-To: <6A8B39E2-2351-4B99-A472-081A4905A00C@icsi.berkeley.edu>
References: <8D0E8DB6FAA348C-2740-14137@webmail-m133.sysops.aol.com>
	<F831675F-3648-45BD-95F8-3C385F98774E@icir.org>
	<8D0E996F5A41045-2740-1967C@webmail-m133.sysops.aol.com>
	<6A8B39E2-2351-4B99-A472-081A4905A00C@icsi.berkeley.edu>
Message-ID: <8D0E9AC28BFE4F4-2740-1AB98@webmail-m133.sysops.aol.com>

Thank you so much Bernhard :-)


Jessica.



-----Original Message-----
From: Bernhard Amann <bernhard at ICSI.Berkeley.EDU>
To: Jessica Smith <jes.smith.bro at aol.com>
Cc: Seth Hall <seth at icir.org>; Bro <bro at bro.org>
Sent: Mon, Jan 27, 2014 7:36 pm
Subject: Re: [Bro] why x509_extensions event never called?

Hello Jessica,

a patch for the x509_extension event is in the topic/bernhard/fix-x509-extensions 
git branch. 

The event syntax slightly changed - using

event x509_extension(c: connection, is_orig: bool, cert:X509, extension: 
X509_extension_info) 
{
	print extension;
}

should work now.

Bernhard

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140127/1f0b6dee/attachment.html 

From jsiwek at illinois.edu  Mon Jan 27 11:25:18 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Mon, 27 Jan 2014 19:25:18 +0000
Subject: [Bro] Attach Barnyard2 to Bro2
In-Reply-To: <CAKdZko-q8eXR7VgXWuSrhAFwmr0Q85bXKKf1A0vktsn_BT+AtA@mail.gmail.com>
References: <CAKdZko9YEwuN1X1MmFNuOv+xcg4OKhtiAiQ8M9rUHbwxCW4rdQ@mail.gmail.com>
	<586A8968-8CA0-410E-8864-F714AEB51CE4@illinois.edu>
	<CAKdZko_XLSQ9z7pQ=JKJpuzYAiCRAGbzE_prC639S8XPYSSO+Q@mail.gmail.com>
	<D3B40A9E-F305-4B23-ADA4-E354CE870384@illinois.edu>
	<CAKdZko-q8eXR7VgXWuSrhAFwmr0Q85bXKKf1A0vktsn_BT+AtA@mail.gmail.com>
Message-ID: <D710165C-799A-41DA-A226-115D8ED80105@illinois.edu>


On Jan 27, 2014, at 12:57 PM, Jeremy Cox <jeremy.cox at washk12.org> wrote:

> I did.  I have also put it back into standalone mode to see if that did it.... No luck.

I?m not sure then.  You can check Bro?s communication log to see if it looks like a connection is actually established; Barnyard2?s log should also indicate whether it connected.  The other thing I can think of would be if there?s no alerts actually being generated.  How are you running Barnyard2 ?  Is it pointed at the right place w/ unified2 alert files that are being populated?

- Jon


From jeremy.cox at washk12.org  Mon Jan 27 11:36:13 2014
From: jeremy.cox at washk12.org (Jeremy Cox)
Date: Mon, 27 Jan 2014 12:36:13 -0700
Subject: [Bro] Attach Barnyard2 to Bro2
In-Reply-To: <D710165C-799A-41DA-A226-115D8ED80105@illinois.edu>
References: <CAKdZko9YEwuN1X1MmFNuOv+xcg4OKhtiAiQ8M9rUHbwxCW4rdQ@mail.gmail.com>
	<586A8968-8CA0-410E-8864-F714AEB51CE4@illinois.edu>
	<CAKdZko_XLSQ9z7pQ=JKJpuzYAiCRAGbzE_prC639S8XPYSSO+Q@mail.gmail.com>
	<D3B40A9E-F305-4B23-ADA4-E354CE870384@illinois.edu>
	<CAKdZko-q8eXR7VgXWuSrhAFwmr0Q85bXKKf1A0vktsn_BT+AtA@mail.gmail.com>
	<D710165C-799A-41DA-A226-115D8ED80105@illinois.edu>
Message-ID: <CAKdZko-wTGLLcPq3JP_Np_AfKsx6CwWtbwn=aN9n+x+B2th-=A@mail.gmail.com>

I found it.  Barnyard was pointing at the server's public IP address.  As
soon as I told Barnyard to point at 127.0.0.1, Bro started logging the
Barnyard alerts.  Thanks for all your help!!

Jeremy

*Jeremy Cox*
Senior Network Engineer, ISO

*Washington County School District*121 W Tabernacle - St. George - UT
435-634-4315
www.washk12.org
687474703a2f2f7777772e7375706572746563686775792e636f6d

IMPORTANT NOTICE REGARDING THIS ELECTRONIC COMMUNICATION:

This e-mail, including any attachments thereto, contains information that
may be confidential or privileged, and is intended solely for the
individual or entity to whom it is addressed.  Recipient is hereby notified
that any disclosure, copying or distribution of this message is strictly
prohibited.  IF YOU ARE NOT THE INTENDED RECIPIENT, please notify the
originator of this e-mail immediately and destroy all information
received.  Thank you.



On Mon, Jan 27, 2014 at 12:25 PM, Siwek, Jonathan Luke
<jsiwek at illinois.edu>wrote:

>
> On Jan 27, 2014, at 12:57 PM, Jeremy Cox <jeremy.cox at washk12.org> wrote:
>
> > I did.  I have also put it back into standalone mode to see if that did
> it.... No luck.
>
> I?m not sure then.  You can check Bro?s communication log to see if it
> looks like a connection is actually established; Barnyard2?s log should
> also indicate whether it connected.  The other thing I can think of would
> be if there?s no alerts actually being generated.  How are you running
> Barnyard2 ?  Is it pointed at the right place w/ unified2 alert files that
> are being populated?
>
> - Jon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140127/7f11236e/attachment.html 

From awong5 at mail.ccsf.edu  Mon Jan 27 11:43:10 2014
From: awong5 at mail.ccsf.edu (Arthur Wong)
Date: Mon, 27 Jan 2014 11:43:10 -0800
Subject: [Bro] Running Bro on Mac OS X 10.8.5
In-Reply-To: <0406B4C4-B122-4498-BFB3-1F6593EAB7DB@illinois.edu>
References: <CAMa2HwxTToQAMsTPZjuWqXHjv7Dv4F+a4QxKLv619ej-dGXq=Q@mail.gmail.com>
	<3B1B8361-60E0-42B5-BEC7-4E8057CC8054@illinois.edu>
	<0406B4C4-B122-4498-BFB3-1F6593EAB7DB@illinois.edu>
Message-ID: <CAMa2HwyaNq5qoRaLDYZFDQFKsQM+PJ4SWknLUiXWFXDwo6b08A@mail.gmail.com>

Thanks Jon.

Since the .dmg are for 10.9, I attempted install Bro on 10.9 and still got
same error.

I installed MacPorts, then install libmagic, and then install the Bro Mac
package.

It is possible that I am not installing one of these correctly.

To run Bro, I ended up setting the MAGIC environment variable:
export MAGIC=/opt/local/share/misc/magic.mgc

I did this from the terminal, but is there a config file that this can be
put into so I don't have to remember to set MAGIC variable every time use
Bro?

Thanks - Art



On Tue, Jan 21, 2014 at 10:39 AM, Siwek, Jonathan Luke
<jsiwek at illinois.edu>wrote:

>
> On Jan 21, 2014, at 12:21 PM, Siwek, Jonathan Luke <jsiwek at illinois.edu>
> wrote:
>
> > I suggest building from sources instead.
>
> Or if you prefer, 2.2 is available via MacPorts or Homebrew.
>
> - Jon




-- 
Art Wong
City College of San Francisco
awong5 at mail.ccsf.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140127/927e6e14/attachment.html 

From jsiwek at illinois.edu  Mon Jan 27 12:14:39 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Mon, 27 Jan 2014 20:14:39 +0000
Subject: [Bro] Running Bro on Mac OS X 10.8.5
In-Reply-To: <CAMa2HwyaNq5qoRaLDYZFDQFKsQM+PJ4SWknLUiXWFXDwo6b08A@mail.gmail.com>
References: <CAMa2HwxTToQAMsTPZjuWqXHjv7Dv4F+a4QxKLv619ej-dGXq=Q@mail.gmail.com>
	<3B1B8361-60E0-42B5-BEC7-4E8057CC8054@illinois.edu>
	<0406B4C4-B122-4498-BFB3-1F6593EAB7DB@illinois.edu>
	<CAMa2HwyaNq5qoRaLDYZFDQFKsQM+PJ4SWknLUiXWFXDwo6b08A@mail.gmail.com>
Message-ID: <525B2765-75CF-43FD-B818-9227D085862B@illinois.edu>


On Jan 27, 2014, at 1:43 PM, Arthur Wong <awong5 at mail.ccsf.edu> wrote:

> Since the .dmg are for 10.9, I attempted install Bro on 10.9 and still got same error.

Yeah, it?s a problem with Bro's binary OS X packaging, sorry about that.

> To run Bro, I ended up setting the MAGIC environment variable: 
> export MAGIC=/opt/local/share/misc/magic.mgc
> 
> I did this from the terminal, but is there a config file that this can be put into so I don't have to remember to set MAGIC variable every time use Bro?

There?s not a Bro-specific config file you can put that in, but you could probably put it in your ~/.profile.

Since you?ve got MacPorts already, you might try to install bro through that instead of use the binary package from bro.org.  The MacPorts Bro package shouldn?t require setting any extra environment vars.

- Jon


From jes.smith.bro at aol.com  Mon Jan 27 15:44:54 2014
From: jes.smith.bro at aol.com (Jessica Smith)
Date: Mon, 27 Jan 2014 18:44:54 -0500 (EST)
Subject: [Bro] how can I get the hostname in a SSL connection?
Message-ID: <8D0E9D0E9E8DA92-2740-1CD29@webmail-m133.sysops.aol.com>

Hi,



I'm analyzing X.509 certificates and I have to check that the CN/SAN matches the hostname to which I am connecting. I was using the hostname value derived from the ssl extension "server_name", but in some cases this extension is not set. 
I then tried to use the lookup_addr() function but it returns nothing.




event ssl_established(c: connection) 
{
local hostname: string;


if( c$ssl?$server_name )
hostname = c$ssl$server_name;
	
when(local host = lookup_addr(c$id$resp_h)) { print host; } 
}



It does not print the hostname. why?




Jessica.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140127/406a0b76/attachment.html 

From jes.smith.bro at aol.com  Tue Jan 28 02:18:02 2014
From: jes.smith.bro at aol.com (Jessica Smith)
Date: Tue, 28 Jan 2014 05:18:02 -0500 (EST)
Subject: [Bro] get more information from a SSL certificate?
Message-ID: <8D0EA295C577749-22F0-1F549@webmail-d140.sysops.aol.com>

Hi,


is it possible to get the "Certificate Signature Algorithm" and "Subject Public Key Info" from a X.509 certificate?


Jessica
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140128/4ce28b49/attachment.html 

From seth at icir.org  Tue Jan 28 06:01:50 2014
From: seth at icir.org (Seth Hall)
Date: Tue, 28 Jan 2014 09:01:50 -0500
Subject: [Bro] get more information from a SSL certificate?
In-Reply-To: <8D0EA295C577749-22F0-1F549@webmail-d140.sysops.aol.com>
References: <8D0EA295C577749-22F0-1F549@webmail-d140.sysops.aol.com>
Message-ID: <7D5B6304-74AC-409D-8C76-720AEA21CED6@icir.org>


On Jan 28, 2014, at 5:18 AM, Jessica Smith <jes.smith.bro at aol.com> wrote:

> is it possible to get the "Certificate Signature Algorithm" and "Subject Public Key Info" from a X.509 certificate?

Unfortunately our current analyzer doesn't extract much information from certificates.  Bernhard has been working on some changes though that ultimately will split the X.509 code out into a file analyzer which the SSL analyzer will pass certificates to.  He's also greatly expanding the amount of information extracted from certs.

So, no, that information isn't available quite yet.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140128/36260000/attachment.bin 

From seth at icir.org  Tue Jan 28 06:10:00 2014
From: seth at icir.org (Seth Hall)
Date: Tue, 28 Jan 2014 09:10:00 -0500
Subject: [Bro] how can I get the hostname in a SSL connection?
In-Reply-To: <8D0E9D0E9E8DA92-2740-1CD29@webmail-m133.sysops.aol.com>
References: <8D0E9D0E9E8DA92-2740-1CD29@webmail-m133.sysops.aol.com>
Message-ID: <037582DD-50BB-4D02-AEBB-9A777A8A331E@icir.org>


On Jan 27, 2014, at 6:44 PM, Jessica Smith <jes.smith.bro at aol.com> wrote:

> I'm analyzing X.509 certificates and I have to check that the CN/SAN matches the hostname to which I am connecting. I was using the hostname value derived from the ssl extension "server_name", but in some cases this extension is not set. 

Yep, there is no requirement that certificates have the server name indicator extension.  You will also see this in cases where an SSL session is resumed and there is no certificate exchange.

> event ssl_established(c: connection) 
> {
> local hostname: string;
> 
> if( c$ssl?$server_name )
> hostname = c$ssl$server_name;
> 	
> when(local host = lookup_addr(c$id$resp_h)) { print host; } 
> }
> 
> It does not print the hostname. why?

Are you running this on live traffic or on a trace file?  If you are running on a tracefile, it could be that Bro is terminating before the DNS reply has a chance to get back into Bro and run that code.  When statements work like closures so they aren't executed immediately.  You can think of it like the body of the when statement is stored in the background until the condition for the when statement becomes true or completes, it's only then that the body is executed.

Also, you may want to print something just before the when statement just to make sure your code is actually making it to the when statement.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140128/293bb81c/attachment.bin 

From bernhard at ICSI.Berkeley.EDU  Tue Jan 28 06:16:09 2014
From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann)
Date: Tue, 28 Jan 2014 06:16:09 -0800
Subject: [Bro] get more information from a SSL certificate?
In-Reply-To: <7D5B6304-74AC-409D-8C76-720AEA21CED6@icir.org>
References: <8D0EA295C577749-22F0-1F549@webmail-d140.sysops.aol.com>
	<7D5B6304-74AC-409D-8C76-720AEA21CED6@icir.org>
Message-ID: <9EB38256-D91C-4F23-8AE9-16144C59901C@icsi.berkeley.edu>


On Jan 28, 2014, at 6:01 AM, Seth Hall <seth at icir.org> wrote:

> 
> On Jan 28, 2014, at 5:18 AM, Jessica Smith <jes.smith.bro at aol.com> wrote:
> 
>> is it possible to get the "Certificate Signature Algorithm" and "Subject Public Key Info" from a X.509 certificate?
> 
> Unfortunately our current analyzer doesn't extract much information from certificates.  Bernhard has been working on some changes though that ultimately will split the X.509 code out into a file analyzer which the SSL analyzer will pass certificates to.  He's also greatly expanding the amount of information extracted from certs.

In case you want to be notified when the work on the new certificate analyzer is done, 
you can subscribe to the ticket https://bro-tracker.atlassian.net/browse/BIT-760 which will
be updated when it is done.

Bernhard


From mike.patterson at uwaterloo.ca  Tue Jan 28 07:08:40 2014
From: mike.patterson at uwaterloo.ca (Mike Patterson)
Date: Tue, 28 Jan 2014 10:08:40 -0500
Subject: [Bro] Stupid scripting question: SSH password detection limited to
	local networks
Message-ID: <D08CB106-EAF4-4E9C-8B73-42BA97EABD90@uwaterloo.ca>

Following directions, I did:

----
hook Notice::policy(n: Notice::Info)
        {
        if ( n$note == SSH::Password_Guessing )
        add n$actions[Notice::ACTION_EMAIL];
        }
----

and since we?ve got a very great many SSH servers exposed, I got a very great number of emails. Since I care more about outbound traffic, I did:

----
hook Notice::policy(n: Notice::Info)
        {
        if ( n$note == SSH::Password_Guessing && Site::is_local_addr(n$id$orig_h) )
        add n$actions[Notice::ACTION_EMAIL];
        }
----

and now I get no mails at all (despite the fact I know we had at least a couple outbound attacks since I set this).

I'd set a few networks in $PREFIX/bro/etc/networks.cfg, and ?config? from broctl says:
localnetscfg = /usr/local/bro-2.2/etc/networks.cfg

However, if I write a test script:
----
@load base/utils/site

print Site::local_nets;
----

and run ?bro test.bro? output is empty unless I first redef Site::local_nets += {}; with my ranges.
Obviously I?m doing something incorrect, but which is the most correct way to fix it? redef local_nets in my local.bro, or have I mistaken the format of the networks.cfg file? It contains:

----
129.97.0.0/16	Waterloo Public IPv4
2620:101:f000::/47	IPv6
----

I?m thinking my problem with the test script is simply that I?m not running it with broctl, although I do run my production instance with same so I?m still back to ?why does my revised hook not work as expected??

Output of the production broctl?s ?scripts? command includes, among other things:
        /usr/local/bro-2.2/share/bro/base/utils/site.bro
    /usr/local/bro-2.2/spool/tmp/check-config-worker-8/local-networks.bro
    /usr/local/bro-2.2/spool/tmp/check-config-worker-8/broctl-config.bro
so I believe it?s loading everything it needs.

Mike




From robin at icir.org  Tue Jan 28 08:37:46 2014
From: robin at icir.org (Robin Sommer)
Date: Tue, 28 Jan 2014 08:37:46 -0800
Subject: [Bro] Stupid scripting question: SSH password detection	limited
	to local networks
In-Reply-To: <D08CB106-EAF4-4E9C-8B73-42BA97EABD90@uwaterloo.ca>
References: <D08CB106-EAF4-4E9C-8B73-42BA97EABD90@uwaterloo.ca>
Message-ID: <20140128163746.GA77620@icir.org>



On Tue, Jan 28, 2014 at 10:08 -0500, Mike Patterson wrote:

> I?m thinking my problem with the test script is simply that I?m not
> running it with broctl

Yeah, networks.cfg will be used only when running from broctl. For
testing, broctl has a command "process" that processes a trace with
(almost) the same configuration that it's using when running live See
the corresponding entry in
http://www.bro.org/sphinx-git/components/broctl/README.html#command-reference

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 *     robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 * www.icir.org/robin


From jes.smith.bro at aol.com  Tue Jan 28 08:53:14 2014
From: jes.smith.bro at aol.com (Jessica Smith)
Date: Tue, 28 Jan 2014 11:53:14 -0500 (EST)
Subject: [Bro] how can I get the hostname in a SSL connection?
In-Reply-To: <037582DD-50BB-4D02-AEBB-9A777A8A331E@icir.org>
References: <8D0E9D0E9E8DA92-2740-1CD29@webmail-m133.sysops.aol.com>
	<037582DD-50BB-4D02-AEBB-9A777A8A331E@icir.org>
Message-ID: <8D0EA60919EDA70-22F0-21ACB@webmail-d140.sysops.aol.com>

yes, i'm working on trace files, so, there's noy way to wait the DNS reply? in that case I cannot check the validity of CN/SAN field, right?



-----Original Message-----
From: Seth Hall <seth at icir.org>
To: Jessica Smith <jes.smith.bro at aol.com>
Cc: bro <bro at bro.org>
Sent: Tue, Jan 28, 2014 3:10 pm
Subject: Re: [Bro] how can I get the hostname in a SSL connection?
Are you running this on live traffic or on a trace file?  If you are running on 
a tracefile, it could be that Bro is terminating before the DNS reply has a 
chance to get back into Bro and run that code.  When statements work like 
closures so they aren't executed immediately.  You can think of it like the body 
of the when statement is stored in the background until the condition for the 
when statement becomes true or completes, it's only then that the body is 
executed.

Also, you may want to print something just before the when statement just to 
make sure your code is actually making it to the when statement.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/


 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140128/b8de3b8f/attachment.html 

From mike.patterson at uwaterloo.ca  Tue Jan 28 09:29:47 2014
From: mike.patterson at uwaterloo.ca (Mike Patterson)
Date: Tue, 28 Jan 2014 12:29:47 -0500
Subject: [Bro] Stupid scripting question: SSH password detection limited
	to local networks
In-Reply-To: <20140128163746.GA77620@icir.org>
References: <D08CB106-EAF4-4E9C-8B73-42BA97EABD90@uwaterloo.ca>
	<20140128163746.GA77620@icir.org>
Message-ID: <88351C84-C3F6-4F58-BDE3-20ABF4177562@uwaterloo.ca>

On Jan 28, 2014, at 11:37 AM, Robin Sommer <robin at icir.org> wrote:

> 
> 
> On Tue, Jan 28, 2014 at 10:08 -0500, Mike Patterson wrote:
> 
>> I?m thinking my problem with the test script is simply that I?m not
>> running it with broctl
> 
> Yeah, networks.cfg will be used only when running from broctl. For
> testing, broctl has a command "process" that processes a trace with
> (almost) the same configuration that it's using when running live See
> the corresponding entry in
> http://www.bro.org/sphinx-git/components/broctl/README.html#command-reference
> 

Well, now I have a different problem, but the issue is somewhat tangential - unless that?s the only way for me to get Bro/broctl to tell me what it thinks my local networks are.

Issue with process: It bombs out with messages like:
error in /usr/local/bro-2.2/share/bro/policy/misc/loaded-scripts.bro, line 4: syntax error, at or near ?module"

I?m running with
[BroControl] > process /path/to/pcap /path/to/test.bro
and I?ve also tried
process ? /path/to/test.bro
process /path/to/pcap ? /path/to/test.bro

Poor bernhard was trying to help me, and his install works just fine.

So now I don?t *know* if I?ve got one issue, or two issues. ;)

Any ideas, Robin?

Mike




From gary at doit.wisc.edu  Tue Jan 28 15:11:25 2014
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Tue, 28 Jan 2014 17:11:25 -0600
Subject: [Bro] p0f OS fingerprinting question
In-Reply-To: <CDD8B876-AA78-41F0-BB00-A26F14649209@icir.org>
References: <52E196EB.40100@doit.wisc.edu>
	<CDD8B876-AA78-41F0-BB00-A26F14649209@icir.org>
Message-ID: <52E8391D.4060703@doit.wisc.edu>

On 1/27/2014 9:15 AM, Seth Hall wrote:
> Basically the new version of p0f is something you could implement as a 
> Bro script because he's just grabbing user-agent strings and stuff. 
I've never really written a Bro script outside of some of the examples 
from Bro Exchange, but would something like the below event be a valid 
starting point? For reference I started by looking at 
/bro/share/bro/policy/protocols/http/software.bro which already appeared 
to be looking at browser user agents and just started experimenting. 
Also, for learning purposes I'm OK with this not being the most reliable 
data. I also tried the p0f example on some pcaps to see what it could 
find after updating to a more recent fingerprint file. My next steps 
might be to try figuring out how to send the output to a custom log 
file, say "alleged_os.log".

event http_header(c: connection, is_orig: bool, name: string, value: 
string) &priority=2
         {
         if ( is_orig )
                 {
                 if ( name == "USER-AGENT" && /Windows NT 6.1/ in value )
                         print cat("Windows 7 detected on - ", 
$host=c$id$orig_h);
         else if ( name == "USER-AGENT" && /Windows NT 6.0/ in value )
                         print cat("Windows Vista detected on - ", 
$host=c$id$orig_h);
         else if ( name == "USER-AGENT" && /Windows NT 5.1/ in value )
             print cat("Windows XP detected on - ", $host=c$id$orig_h);
                 }
     }

Regards,

Gary Faulkner

On 1/27/2014 9:15 AM, Seth Hall wrote:
> On Jan 23, 2014, at 5:25 PM, Gary Faulkner <gary at doit.wisc.edu> wrote:
>
>> at least very out of date due to a 6 year p0f development hiatus.
> Yep.  It's in Bro still though.  If you write a script that handles this event:
>
> 	event OS_version_found(c: connection, host: addr, OS: OS_version)
> 		{
> 		print cat("p0f reported - ", OS);
> 		}
>
> you will be getting the output from p0f.
>
>>   With p0f having been rewritten in 2012 are there there any plans for updating Bro to support the newer version?
> Nope, I don't think anyone plans on updating it.  The author of p0f stopped trying to fingerprint TCP stacks (mostly) and started using other deeper packet sniffing. :)  Basically the new version of p0f is something you could implement as a Bro script because he's just grabbing user-agent strings and stuff.  The problem is that it's really hard to blindly trust user-agent strings because of NAT'ed addressed and people giving fake user-agent strings.
>
> I'm hoping eventually in Bro to write a script that takes lots of measurements (p0f, user-agents, software update mechanisms touches, exposed services) to get a profile for a machine to decide if it's a particular type of host.  For instance, imagine that one of your windows xp machines gets identified as such by p0f (in Bro), then identifies that it's windows xp in a browser user-agent, then reaches out for windows updates and identifies that it's version of windows is some version of XP (i can't remember if this is visible for Windows system updates or not).
>
> Anyway, this approach is replicable for many other operating systems too, it just takes time and the Bro scripts to support it.
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6257 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140128/c7a03245/attachment.bin 

From benson.mathews at gmail.com  Wed Jan 29 11:14:05 2014
From: benson.mathews at gmail.com (Benson Mathews)
Date: Wed, 29 Jan 2014 14:14:05 -0500
Subject: [Bro]  Bro and flood protection
Message-ID: <CAND-VUchaa-+COMczbtCj+Qhgw7qQqFunqtthSDhvGgTATCneg@mail.gmail.com>

Hi Folks,

I'm trying to learn if Bro can detect UDP flood attacks. And found the
synflood script that was provided on previous versions of Bro. Wondering if
there's something similar on Bro 2.2 or if the 1.5.x version would still
work?

Noticed that this question was raised in the past by Alex (below). Does
anyone know how this could be done on 2.2 or have a working script to
detect flood attacks already?

Thanks,
Benson

Date: Mon, 11 Nov 2013 22:19:08 -0800 (PST)
From: Laleh Arshadi <la_arshadi at yahoo.com>
Subject: [Bro] Bro and flood protection - revisited
To: "bro at bro.org" <bro at bro.org>
Message-ID:
        <1384237148.40495.YahooMailNeo at web140606.mail.bf1.yahoo.com>
Content-Type: text/plain; charset="iso-8859-1"

Dear All,
?
This message was sent a while ago but I see no one has replied to it. As I
have almost a similar question myself, I would be thankful if someone took
another look at the email and responded.
?
Regards
Laleh


______________________________
__
From: Alexander Frolkin <avf at eldamar.org.uk>
To: bro at bro.org
Sent: Thursday, November 7, 2013 3:05 PM
Subject: [Bro] Bro and flood protection


Hi,

I'm currently looking around for open-source IDSes.? What we'd like
is to have an IDS machine which monitors our Internet traffic and
responds to events by blocking the traffic using Flowspec.? This is easy
to do with Bro and ExaBGP using custom event handlers and/or hooks, and
piped_exec.

I'm currently trying to understand Bro's ability to detect floods, e.g.,
SYN flood, ACK flood, or any other kind of flood, for that matter.

The feeling I have so far is that Bro wasn't really designed for this
sort of thing, and that it's designed more for L7 stuff.

I'm playing with 2.2 beta, and I can't see anything built-in to detect
floods (although maybe I haven't looked hard enough).? In older
versions, though, there was a script called synflood.bro, but it seems
to have disappeared at some point.? Does anyone know what the history of
this is, and whether there is equivalent funtionality in the latest
version?

More generally, if I want to detect network floods, is Bro the right
thing to be using, or should I be looking elsewhere?

Thanks!


Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140129/3ba789a3/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: synflood.bro
Type: application/octet-stream
Size: 3876 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140129/3ba789a3/attachment.obj 

From gary at doit.wisc.edu  Wed Jan 29 11:41:47 2014
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Wed, 29 Jan 2014 13:41:47 -0600
Subject: [Bro] p0f OS fingerprinting question
In-Reply-To: <52E8391D.4060703@doit.wisc.edu>
References: <52E196EB.40100@doit.wisc.edu>
	<CDD8B876-AA78-41F0-BB00-A26F14649209@icir.org>
	<52E8391D.4060703@doit.wisc.edu>
Message-ID: <52E9597B.4020103@doit.wisc.edu>

You can disregard my last post. After re-reading some of the examples on 
the ryesecurity blog, stepping through some of the pre-packaged bro 
scripts, and lots of experimentation I'm most of the way to finishing my 
first bro script and logging the results where I want. Thanks again for 
the help.

On 1/28/2014 5:11 PM, Gary Faulkner wrote:
> On 1/27/2014 9:15 AM, Seth Hall wrote:
>> Basically the new version of p0f is something you could implement as 
>> a Bro script because he's just grabbing user-agent strings and stuff. 
> I've never really written a Bro script outside of some of the examples 
> from Bro Exchange, but would something like the below event be a valid 
> starting point? For reference I started by looking at 
> /bro/share/bro/policy/protocols/http/software.bro which already 
> appeared to be looking at browser user agents and just started 
> experimenting. Also, for learning purposes I'm OK with this not being 
> the most reliable data. I also tried the p0f example on some pcaps to 
> see what it could find after updating to a more recent fingerprint 
> file. My next steps might be to try figuring out how to send the 
> output to a custom log file, say "alleged_os.log".


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6257 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140129/901cfd87/attachment.bin 

From robin at icir.org  Wed Jan 29 15:20:08 2014
From: robin at icir.org (Robin Sommer)
Date: Wed, 29 Jan 2014 15:20:08 -0800
Subject: [Bro] Stupid scripting question: SSH password detection	limited
	to local networks
In-Reply-To: <88351C84-C3F6-4F58-BDE3-20ABF4177562@uwaterloo.ca>
References: <D08CB106-EAF4-4E9C-8B73-42BA97EABD90@uwaterloo.ca>
	<20140128163746.GA77620@icir.org>
	<88351C84-C3F6-4F58-BDE3-20ABF4177562@uwaterloo.ca>
Message-ID: <20140129232008.GA80589@icir.org>



On Tue, Jan 28, 2014 at 12:29 -0500, you wrote:

> error in /usr/local/bro-2.2/share/bro/policy/misc/loaded-scripts.bro, line 4: syntax error, at or near ?module"

Actually I get this too, just tried it. That looks like a bug. Here's
a work-around that works for me: when you run the "process" commnand,
it prints out a long command line. Copy that, leave broctl, paste it
into your shell and then move your test script from its position
somewhere in the middle to the very end.

I don't know if this helps solving the original problem but it should
tell you if broctl sets the local networks correctly.

I'm also filing a ticket for this, I believe broctl should always move
the the custom scripts to the end of the command line.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 *     robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 * www.icir.org/robin


From robin at icir.org  Wed Jan 29 15:29:01 2014
From: robin at icir.org (Robin Sommer)
Date: Wed, 29 Jan 2014 15:29:01 -0800
Subject: [Bro] Bro and flood protection
In-Reply-To: <CAND-VUchaa-+COMczbtCj+Qhgw7qQqFunqtthSDhvGgTATCneg@mail.gmail.com>
References: <CAND-VUchaa-+COMczbtCj+Qhgw7qQqFunqtthSDhvGgTATCneg@mail.gmail.com>
Message-ID: <20140129232901.GD92282@icir.org>



On Wed, Jan 29, 2014 at 14:14 -0500, you wrote:

> synflood script that was provided on previous versions of Bro. Wondering if
> there's something similar on Bro 2.2 or if the 1.5.x version would still
> work?

As far as I recall, the 1.5 script should still work. 

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 *     robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 * www.icir.org/robin


From gary at doit.wisc.edu  Wed Jan 29 15:35:46 2014
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Wed, 29 Jan 2014 17:35:46 -0600
Subject: [Bro] Sanity check - Grabbing platform tokens from browser user
 agents (was p0f)
Message-ID: <52E99052.3060807@doit.wisc.edu>

After being asked if Bro could be used to gather passive intelligence on 
OS usage I started investigating places to try to identify OS. I 
initially was looking into p0f and Seth showed me a way to invoke the 
existing p0f fingerprinting functionality within Bro, but also suggested 
a slew of other data sources to look at. I wasn't terribly excited with 
the p0f fingerprint output, and while browser user agents may not be the 
best data source, I decided to start by looking at platform tokens and 
reporting on those instead of the p0f data. This is my first-ish bro 
script and it is by no means a complete script (it only matches a 
handful of Windows OS). I'm wondering if folks see anything in the 
attached that would misbehave badly if used on live traffic instead of 
pcaps?

Regards,

-- 
Gary Faulkner

-------------- next part --------------
module BrowserPlatform;

export
{
    redef enum Log::ID += { LOG };

    type Info: record {
        ts:                 time    &log &optional;
        uid:                string  &log &optional;
        host:               addr    &log &optional;
        platform_token:     string  &log &optional;
        unparsed_version:   string  &log &optional;
    };

    # A set of seen IP + OS combinations. Used to prevent logging the same combo repeatedly.   
    global seen_browser_platforms: set[string] &create_expire=1day &synchronized &redef;	
}

event bro_init() &priority=5
    {
    Log::create_stream(BrowserPlatform::LOG,[$columns=Info]);
    }

event http_header(c: connection, is_orig: bool, name: string, value: string)
{
    local platform = "Unknown OS";	
    if ( is_orig )
        {
	if ( name == "USER-AGENT" && /Windows NT 5.1/ in value )
		{
		platform = "Windows XP";
		}
        else if ( name == "USER-AGENT" && /Windows NT 6.0/ in value )
                {
		platform = "Windows Vista";
                }
        else if ( name == "USER-AGENT" && /Windows NT 6.1/ in value )
                {
                platform = "Windows 7";
                }
        else if ( name == "USER-AGENT" && /Windows NT 6.2/ in value )
                {
                platform = "Windows 8";
                }
        else if ( name == "USER-AGENT" && /Windows NT 6.3/ in value )
                {
                platform = "Windows 8.1";
                }
	}
    local saw = cat(c$id$orig_h,platform); #There is probably a less ugly way to do this than cat, but it seems to work
    if ( platform != "Unknown OS" && saw !in seen_browser_platforms )
	{	
	local rec: BrowserPlatform::Info = [$ts=network_time(), $uid=c$uid, $host=c$id$orig_h, $platform_token=platform, $unparsed_version=value];
	Log::write(BrowserPlatform::LOG, rec);
	add seen_browser_platforms[saw];
	}
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6257 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140129/6a5b0762/attachment.bin 

From mike.patterson at uwaterloo.ca  Wed Jan 29 15:37:38 2014
From: mike.patterson at uwaterloo.ca (Mike Patterson)
Date: Wed, 29 Jan 2014 18:37:38 -0500
Subject: [Bro] Stupid scripting question: SSH password detection limited
	to local networks
In-Reply-To: <20140129232008.GA80589@icir.org>
References: <D08CB106-EAF4-4E9C-8B73-42BA97EABD90@uwaterloo.ca>
	<20140128163746.GA77620@icir.org>
	<88351C84-C3F6-4F58-BDE3-20ABF4177562@uwaterloo.ca>
	<20140129232008.GA80589@icir.org>
Message-ID: <DE082174-7C99-46D5-97BC-1CF3EFC2344F@uwaterloo.ca>

On Jan 29, 2014, at 6:20 PM, Robin Sommer <robin at icir.org> wrote:
> On Tue, Jan 28, 2014 at 12:29 -0500, you wrote:
> 
>> error in /usr/local/bro-2.2/share/bro/policy/misc/loaded-scripts.bro, line 4: syntax error, at or near ?module"
> 
> Actually I get this too, just tried it. That looks like a bug.

It worked for bernhard as I?d originally described it. Maybe we should steal his test machine. :-)

> Here's
> a work-around that works for me: when you run the "process" commnand,
> it prints out a long command line. Copy that, leave broctl, paste it
> into your shell and then move your test script from its position
> somewhere in the middle to the very end.
> 
> I don't know if this helps solving the original problem but it should
> tell you if broctl sets the local networks correctly.

Your workaround worked for me, and did print out what I thought local networks should be set to.

Justin had pointed out to me that I could use
bro $PREFIX/bro/spool/installed-scripts-do-not-touch/auto/local-networks.bro test.bro
which worked equally well.

He also suggested a slight change to my hook:
Site::is_local_addr(n$id$orig_h)
to
Site::is_local_addr(n$src)

and now I?m working on trying to trigger the flipping SSH password guessing logic so I can test, and not having a lot of luck. :) I have a remote host I can ssh scan with impunity, so I?ve fired hydra and a simple ?connect to port 22 and disconnect? script at it with no joy. I should have picked an easier notice to start with, I guess. :)

Mike




From mike.patterson at uwaterloo.ca  Wed Jan 29 16:27:40 2014
From: mike.patterson at uwaterloo.ca (Mike Patterson)
Date: Wed, 29 Jan 2014 19:27:40 -0500
Subject: [Bro] Stupid scripting question: SSH password detection limited
	to local networks
In-Reply-To: <D08CB106-EAF4-4E9C-8B73-42BA97EABD90@uwaterloo.ca>
References: <D08CB106-EAF4-4E9C-8B73-42BA97EABD90@uwaterloo.ca>
Message-ID: <8648B007-C6F3-4C29-B6C9-B976249AB4D3@uwaterloo.ca>

> However, if I write a test script:
> ----
> @load base/utils/site
> 
> print Site::local_nets;
> ----
> 
> and run ?bro test.bro? output is empty unless I first redef Site::local_nets += {}; with my ranges.
> Obviously I?m doing something incorrect, but which is the most correct way to fix it? redef local_nets in my local.bro, or have I mistaken the format of the networks.cfg file? It contains:

Bernhard points out that if the script is:
----
event bro_init()
	{
print Site::local_nets;
	}
----
everything works just ducky that way too (with process).

. . . and I?m still trying to trigger the heuristics. :)

Mike




From r.fulton at auckland.ac.nz  Thu Jan 30 15:10:23 2014
From: r.fulton at auckland.ac.nz (Russell Fulton)
Date: Thu, 30 Jan 2014 23:10:23 +0000
Subject: [Bro] log files not getting deleted ??
Message-ID: <043D03B5-1012-46BB-A144-E9EE356EF77D@auckland.ac.nz>

Hi Folks

I have this in my broctl.cfg file:

# Location of the log directory.  This is longer term storage for rotated logs.
LogDir = /home/sensors/data/test1/bro-logs

# Rotation interval in seconds for log files on manager/standalone node.
LogRotationInterval = 3600

# Expiration interval for log files in LogDir. Files older than this many days
# will be deleted upon running "broctl cron".
LogExpireInterval = 3

# Lower threshold for space available on the disk that holds SpoolDir. If less
# space is available, BroControl starts sending out warning emails.
MinDiskSpace = 5

but I have weeks of logs in the directory ??

rful011 at secmontst01:~$ ls /home/sensors/data/test1/bro-logs
2014-01-01  2014-01-04  2014-01-07  2014-01-10  2014-01-13  2014-01-16  2014-01-19  2014-01-22  2014-01-25  2014-01-28  2014-01-31
2014-01-02  2014-01-05  2014-01-08  2014-01-11  2014-01-14  2014-01-17  2014-01-20  2014-01-23  2014-01-26  2014-01-29  current
2014-01-03  2014-01-06  2014-01-09  2014-01-12  2014-01-15  2014-01-18  2014-01-21  2014-01-24  2014-01-27  2014-01-30  stats

and I have to keep pruning it by hand when the system runs out of disk.  

And yes I have checked the con logs to make sure broctl cron is getting run ;)

Any ideas how to figure out what is wrong?

Russell





From dnthayer at illinois.edu  Thu Jan 30 16:47:24 2014
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Thu, 30 Jan 2014 18:47:24 -0600
Subject: [Bro] log files not getting deleted ??
In-Reply-To: <043D03B5-1012-46BB-A144-E9EE356EF77D@auckland.ac.nz>
References: <043D03B5-1012-46BB-A144-E9EE356EF77D@auckland.ac.nz>
Message-ID: <52EAF29C.4040300@illinois.edu>

Did you remember to do "broctl install" after you changed the
value of LogExpireInterval ?


On 01/30/2014 05:10 PM, Russell Fulton wrote:
> Hi Folks
>
> I have this in my broctl.cfg file:
>
> # Location of the log directory.  This is longer term storage for rotated logs.
> LogDir = /home/sensors/data/test1/bro-logs
>
> # Rotation interval in seconds for log files on manager/standalone node.
> LogRotationInterval = 3600
>
> # Expiration interval for log files in LogDir. Files older than this many days
> # will be deleted upon running "broctl cron".
> LogExpireInterval = 3
>
> # Lower threshold for space available on the disk that holds SpoolDir. If less
> # space is available, BroControl starts sending out warning emails.
> MinDiskSpace = 5
>
> but I have weeks of logs in the directory ??
>
> rful011 at secmontst01:~$ ls /home/sensors/data/test1/bro-logs
> 2014-01-01  2014-01-04  2014-01-07  2014-01-10  2014-01-13  2014-01-16  2014-01-19  2014-01-22  2014-01-25  2014-01-28  2014-01-31
> 2014-01-02  2014-01-05  2014-01-08  2014-01-11  2014-01-14  2014-01-17  2014-01-20  2014-01-23  2014-01-26  2014-01-29  current
> 2014-01-03  2014-01-06  2014-01-09  2014-01-12  2014-01-15  2014-01-18  2014-01-21  2014-01-24  2014-01-27  2014-01-30  stats
>
> and I have to keep pruning it by hand when the system runs out of disk.
>
> And yes I have checked the con logs to make sure broctl cron is getting run ;)
>
> Any ideas how to figure out what is wrong?
>
> Russell



From r.fulton at auckland.ac.nz  Thu Jan 30 19:22:03 2014
From: r.fulton at auckland.ac.nz (Russell Fulton)
Date: Fri, 31 Jan 2014 03:22:03 +0000
Subject: [Bro]  log files not getting deleted ??
References: <58027041-48DD-439C-91F0-47C1E747A42B@auckland.ac.nz>
Message-ID: <8CC15B72-5A47-4F13-9811-97017C2D7CBE@auckland.ac.nz>


On 31/01/2014, at 1:47 pm, Daniel Thayer <dnthayer at illinois.edu> wrote:

> Did you remember to do "broctl install" after you changed the
> value of LogExpireInterval ?

How did you guess ;)  Looking at the command history I see I restarted rather than ?install?ed

>From now on I will do the config changes via puppet the class has now been changed to do an ?install? if the bro.cfg ever changes.

Thanks! 

R






From gary at doit.wisc.edu  Fri Jan 31 11:01:04 2014
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Fri, 31 Jan 2014 13:01:04 -0600
Subject: [Bro] Sanity check - Grabbing platform tokens from browser user
 agents (was p0f)
In-Reply-To: <52E99052.3060807@doit.wisc.edu>
References: <52E99052.3060807@doit.wisc.edu>
Message-ID: <52EBF2F0.8020208@doit.wisc.edu>

I've tried the below script now on a Bro 2.2 (release version) cluster 
watching about 8Gbps of traffic and it seems to do what I intended which 
is to create a separate log file that keeps track of IP to OS mappings, 
for a handful of Windows Desktop OS versions, as pulled from user agent 
strings sent over HTTP. In the first full day in production I 
successfully logged around 227,000 unique IP to OS mappings. I have 
since slightly modified it to only log IPs represented in local nets to 
reduce log volume as I'm mostly interested in my own networks.

One issue I'm running into is in keeping track of IP to OS mappings and 
only logging them once per day. I've set an expire timer for 1 day, but 
in production it seems to only keep track of those IPs for the duration 
of a log rotation interval which is set to 20 minutes. I have observed 
that without the expire timer each mapping will log continuously, so it 
appears to get used in some way, but just seems to be tied to log 
rotation instead of the explicit value in the script. I'm guessing I 
need to do something different, but not sure what. Thought? Script is 
pasted below.

================= Begin Script ==============
@load base/utils/site

module BrowserPlatform;

export
{
     # The fully resolved name for this log will be BrowserPlatform::LOG
     redef enum Log::ID += { LOG };

     type Info: record {
         ts:                 time    &log &optional;
         uid:                string  &log &optional;
         host:               addr    &log &optional;
         platform_token:     string  &log &optional;
         unparsed_version:   string  &log &optional;
     };

     # A set of seen IP + OS combinations. Used to prevent logging the 
same combo repeatedly.
     global seen_browser_platforms: set[string] &create_expire = 1.0 day 
&synchronized &redef;
}

event bro_init() &priority=5
     {
     Log::create_stream(BrowserPlatform::LOG,[$columns=Info]);
     }

event http_header(c: connection, is_orig: bool, name: string, value: string)
{
     local platform = "Unknown OS";
     if ( is_orig && Site::is_local_addr(c$id$orig_h) )
         {
         if ( name == "USER-AGENT" && /Windows NT 5.1/ in value )
                 {
                 platform = "Windows XP";
                 }
         else if ( name == "USER-AGENT" && /Windows NT 6.0/ in value )
                 {
                 platform = "Windows Vista";
                 }
         else if ( name == "USER-AGENT" && /Windows NT 6.1/ in value )
                 {
                 platform = "Windows 7";
                 }
         else if ( name == "USER-AGENT" && /Windows NT 6.2/ in value )
                 {
                 platform = "Windows 8";
                 }
         else if ( name == "USER-AGENT" && /Windows NT 6.3/ in value )
                 {
                 platform = "Windows 8.1";
                 }
         }
     local saw = cat(c$id$orig_h,platform); #There is probably a less 
ugly way to do this than cat, but it seems to work
     if ( platform != "Unknown OS" && saw !in seen_browser_platforms )
         {
         local rec: BrowserPlatform::Info = [$ts=network_time(), 
$uid=c$uid, $host=c$id$orig_h, $platform_token=platform, 
$unparsed_version=value];
         Log::write(BrowserPlatform::LOG, rec);
         add seen_browser_platforms[saw];
         }
}

================ End script ==================

Thanks,

Gary Faulkner
UW Madison
Office of Campus Information Security

On 1/29/2014 5:35 PM, Gary Faulkner wrote:
> After being asked if Bro could be used to gather passive intelligence 
> on OS usage I started investigating places to try to identify OS. I 
> initially was looking into p0f and Seth showed me a way to invoke the 
> existing p0f fingerprinting functionality within Bro, but also 
> suggested a slew of other data sources to look at. I wasn't terribly 
> excited with the p0f fingerprint output, and while browser user agents 
> may not be the best data source, I decided to start by looking at 
> platform tokens and reporting on those instead of the p0f data. This 
> is my first-ish bro script and it is by no means a complete script (it 
> only matches a handful of Windows OS). I'm wondering if folks see 
> anything in the attached that would misbehave badly if used on live 
> traffic instead of pcaps?


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6257 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140131/595bf545/attachment.bin 

From JAzoff at albany.edu  Fri Jan 31 16:12:42 2014
From: JAzoff at albany.edu (Justin Azoff)
Date: Fri, 31 Jan 2014 19:12:42 -0500
Subject: [Bro] Sanity check - Grabbing platform tokens from browser user
 agents (was p0f)
In-Reply-To: <52E99052.3060807@doit.wisc.edu>
References: <52E99052.3060807@doit.wisc.edu>
Message-ID: <20140201001242.GE8640@datacomm.albany.edu>


On Wed, Jan 29, 2014 at 05:35:46PM -0600, Gary Faulkner wrote:
> event http_header(c: connection, is_orig: bool, name: string, value: string)
> {
>     local platform = "Unknown OS";	
>     if ( is_orig )
>         {
> 	if ( name == "USER-AGENT" && /Windows NT 5.1/ in value )
> 		{
> 		platform = "Windows XP";
> 		}
>         else if ( name == "USER-AGENT" && /Windows NT 6.0/ in value )
>                 {
> 		platform = "Windows Vista";
>                 }
>         else if ( name == "USER-AGENT" && /Windows NT 6.1/ in value )
>                 {
>                 platform = "Windows 7";
>                 }
..

Modifying the http_header event handler as follows will increase performance:

event http_header(c: connection, is_orig: bool, name: string, value: string)
{
    if(!is_orig || name != "USER-AGENT")
        return;
    if(/Windows NT 5.1/ in value)
        platform = "Windows XP";
    else if ...

FWIW, I used to do this kind of thing outside of bro using splunk:

https://github.com/JustinAzoff/splunk-scripts/blob/master/ua2os.py

One thing you may want to do is rather than use the http_header event
use

event log_software(rec: Info)
{
    ...
}

which will be raised every time a new software version is seen.  The
software framework is already pulling most of the info out that you
might need, so you can piggy back on the work that it is doing.

-- 
-- Justin Azoff


From gary at doit.wisc.edu  Fri Jan 31 20:56:37 2014
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Fri, 31 Jan 2014 22:56:37 -0600
Subject: [Bro] Sanity check - Grabbing platform tokens from browser user
 agents (was p0f)
In-Reply-To: <20140201001242.GE8640@datacomm.albany.edu>
References: <52E99052.3060807@doit.wisc.edu>
	<20140201001242.GE8640@datacomm.albany.edu>
Message-ID: <52EC7E85.9020401@doit.wisc.edu>

Thanks for the suggestions, that cleans that bit up quite nicely. I 
actually started by trying to deconstruct the various software.bro 
scripts and work my way backwards through the framework to see what was 
doing what. I'm still trying to navigate my way through that code, but I 
agree that it would make more sense to leverage it directly than create 
a derivative just to pull out a specific bit of the data. I'm not 
currently running Splunk in any production sense, but that is pretty 
much what I'm trying to do in Bro. Thanks for sharing it!

Regards,
Gary

On 1/31/2014 6:12 PM, Justin Azoff wrote:
>
> On Wed, Jan 29, 2014 at 05:35:46PM -0600, Gary Faulkner wrote:
>> event http_header(c: connection, is_orig: bool, name: string, value: string)
>> {
>>      local platform = "Unknown OS";	
>>      if ( is_orig )
>>          {
>> 	if ( name == "USER-AGENT" && /Windows NT 5.1/ in value )
>> 		{
>> 		platform = "Windows XP";
>> 		}
>>          else if ( name == "USER-AGENT" && /Windows NT 6.0/ in value )
>>                  {
>> 		platform = "Windows Vista";
>>                  }
>>          else if ( name == "USER-AGENT" && /Windows NT 6.1/ in value )
>>                  {
>>                  platform = "Windows 7";
>>                  }
> ..
>
> Modifying the http_header event handler as follows will increase performance:
>
> event http_header(c: connection, is_orig: bool, name: string, value: string)
> {
>      if(!is_orig || name != "USER-AGENT")
>          return;
>      if(/Windows NT 5.1/ in value)
>          platform = "Windows XP";
>      else if ...
>
> FWIW, I used to do this kind of thing outside of bro using splunk:
>
> https://github.com/JustinAzoff/splunk-scripts/blob/master/ua2os.py
>
> One thing you may want to do is rather than use the http_header event
> use
>
> event log_software(rec: Info)
> {
>      ...
> }
>
> which will be raised every time a new software version is seen.  The
> software framework is already pulling most of the info out that you
> might need, so you can piggy back on the work that it is doing.
>

-- 
Gary Faulkner
UW Madison
Office of Campus Information Security