From jes.smith.bro at aol.com  Sat Feb  1 09:12:33 2014
From: jes.smith.bro at aol.com (Jessica Smith)
Date: Sat, 1 Feb 2014 12:12:33 -0500 (EST)
Subject: [Bro] How to create a CSV logging writer
Message-ID: <8D0ED87EE1AF16D-290C-EF24@webmail-m133.sysops.aol.com>


Hi,


how can I add a new writer-specifi filter option via "config" in Log::Filter? I need to store log entries in CSV format where only a single header row with the column names is printed out as meta information, with no ?# fields? prepended and no other meta data gets included in that mode.


Jessica
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140201/28f9650c/attachment.html 

From anthony.kasza at gmail.com  Sat Feb  1 11:54:55 2014
From: anthony.kasza at gmail.com (anthony kasza)
Date: Sat, 1 Feb 2014 11:54:55 -0800
Subject: [Bro] How to create a CSV logging writer
In-Reply-To: <8D0ED87EE1AF16D-290C-EF24@webmail-m133.sysops.aol.com>
References: <8D0ED87EE1AF16D-290C-EF24@webmail-m133.sysops.aol.com>
Message-ID: <CAEZw2bzYy0fCF4JVD0YPO4DPOGfCoWZ0_oj77VNKz0zbx+V0+g@mail.gmail.com>

Have you considered post processing the log files with things like
sed/grep? You should also check out the values exported by
%INSTALL_PATH%/bro/share/bro/base/frameworks/logging/main.bro.

-Anthony

On Sat, Feb 1, 2014 at 9:12 AM, Jessica Smith <jes.smith.bro at aol.com> wrote:
> Hi,
>
> how can I add a new writer-specifi filter option via "config" in
> Log::Filter? I need to store log entries in CSV format where only a single
> header row with the column names is printed out as meta information, with no
> "# fields" prepended and no other meta data gets included in that mode.
>
> Jessica
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From mthompson at hexwave.com  Sat Feb  1 13:29:12 2014
From: mthompson at hexwave.com (Matt Thompson)
Date: Sat, 01 Feb 2014 15:29:12 -0600
Subject: [Bro] How to create a CSV logging writer
In-Reply-To: <8D0ED87EE1AF16D-290C-EF24@webmail-m133.sysops.aol.com>
References: <8D0ED87EE1AF16D-290C-EF24@webmail-m133.sysops.aol.com>
Message-ID: <52ED6728.8050500@hexwave.com>

Hi Jessica,

I don't think you need a filter for CSV output.

See:
http://www.bro.org/sphinx/scripts/base/frameworks/logging/writers/ascii.html

You can get CSV behavior without meta headers with:

redef LogAscii::include_meta = F;
redef LogAscii::separator = ",";
redef LogAscii::set_separator = ";";

Cheers,
Matt Thompson

On 2/1/14, 11:12 AM, Jessica Smith wrote:
> Hi,
>
>
> how can I add a new writer-specifi filter option via "config" in Log::Filter? I need to store log entries in CSV format where only a single header row with the column names is printed out as meta information, with no "# fields" prepended and no other meta data gets included in that mode.
>
>
> Jessica
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140201/085862ca/attachment.html 

From jes.smith.bro at aol.com  Sat Feb  1 14:17:06 2014
From: jes.smith.bro at aol.com (Jessica Smith)
Date: Sat, 1 Feb 2014 17:17:06 -0500 (EST)
Subject: [Bro] How to create a CSV logging writer
In-Reply-To: <52ED6728.8050500@hexwave.com>
References: <8D0ED87EE1AF16D-290C-EF24@webmail-m133.sysops.aol.com>
	<52ED6728.8050500@hexwave.com>
Message-ID: <8D0EDB279BEBE0C-290C-1079E@webmail-m133.sysops.aol.com>

I know, but I need to print out a single header row with the column names and in this way I cannot do that.



-----Original Message-----
From: Matt Thompson <mthompson at hexwave.com>
To: bro <bro at bro.org>
Sent: Sat, Feb 1, 2014 10:37 pm
Subject: Re: [Bro] How to create a CSV logging writer


          
    
Hi Jessica,
      
      I don't think you need a filter for CSV output.
      
      See:http://www.bro.org/sphinx/scripts/base/frameworks/logging/writers/ascii.html
      
      You can get CSV behavior without meta headers with:
      
      redef LogAscii::include_meta = F;
      redef LogAscii::separator = ",";
      redef LogAscii::set_separator = ";";
      
      Cheers,
      Matt Thompson
      
      On 2/1/14, 11:12 AM, Jessica Smith wrote:
    
    
      
Hi,


how can I add a new writer-specifi filter option via "config" in Log::Filter? I need to store log entries in CSV format where only a single header row with the column names is printed out as meta information, with no ?# fields? prepended and no other meta data gets included in that mode.


Jessica


      
      
      
      
_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
    
    
  

_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140201/14b91316/attachment.html 

From mkolkebeck at gmail.com  Sat Feb  1 14:53:50 2014
From: mkolkebeck at gmail.com (Mike Kolkebeck)
Date: Sat, 1 Feb 2014 16:53:50 -0600
Subject: [Bro] File extraction exclude local sites
Message-ID: <CAKUZtURaAXM_CH4CBr-M=jjiZ+x1JUNiR4raA03S4hr5i=ZYVQ@mail.gmail.com>

Does anyone have a solution to exclude extracting files transmitted from
internal hosts?

I modified the script below from the bro exchange exercise, but I continue
to get executable files from internal hosts.  I am using broctl and
networks.cfg is populated.  I haven't tried using the
file_over_new_connection event yet, but I would prefer to use file_new
event, if I can.  Any ideas on what I'm doing wrong?

Thanks!
Mike



global ext_map: table[string] of string = {
    ["application/x-dosexec"] = "exe",
} &default ="";

event file_new(f: fa_file)
    {
    if ( ! f?$mime_type || f$mime_type != "application/x-dosexec" )
        return;

    if ( ! f?$info || ! f$info?$tx_hosts )
        return;

    for ( txhost in f$info$tx_hosts )
        {
        if ( Site::is_local_addr(txhost) )
            return;
        }

    local ext = "";

    if ( f?$mime_type )
        ext = ext_map[f$mime_type];

    local fname = fmt("%s-%s.%s", f$source, f$id, ext);
    Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
[$extract_filename=fname]);
    }
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140201/c3226448/attachment.html 

From mariusph at ifi.uio.no  Sun Feb  2 05:26:23 2014
From: mariusph at ifi.uio.no (Marius Portaas Haugen)
Date: Sun, 02 Feb 2014 14:26:23 +0100
Subject: [Bro] File extraction exclude local sites
In-Reply-To: <CAKUZtURaAXM_CH4CBr-M=jjiZ+x1JUNiR4raA03S4hr5i=ZYVQ@mail.gmail.com>
References: <CAKUZtURaAXM_CH4CBr-M=jjiZ+x1JUNiR4raA03S4hr5i=ZYVQ@mail.gmail.com>
Message-ID: <c2dd67fd9a0ce2d1a2a2b3b75c1c37d7@ulrik.uio.no>

Hi Mike,

I have planned to do something similar myself, and I'm planning on 
using the
local_orig variable and test for something like
if(f$info?$local_org) return;

but I haven't tried it out yet.
 From the documentation 
(http://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html#type-Files::Info):

local_orig: bool &log &optional
If the source of this file is a network connection, this field 
indicates if the data originated from the local network or not as 
determined by the configured Site::local_nets.

Maybe this could work?

Best regards,
Marius P. Haugen




On 2014-02-01 23:53, Mike Kolkebeck wrote:
> Does anyone have a solution to exclude extracting files transmitted
> from internal hosts?
> 
> I modified the script below from the bro exchange exercise, but I
> continue to get executable files from internal hosts. ?I am using
> broctl and networks.cfg is populated. ?I haven't tried using the
> file_over_new_connection event yet, but I would prefer to use file_new
> event, if I can. ?Any ideas on what I'm doing wrong?
> 
> Thanks!
> Mike
> 
> global ext_map: table[string] of string = {
>  ??? ["application/x-dosexec"] = "exe",
>  } &default ="";
> 
> event file_new(f: fa_file)
>  ??? {
>  ??? if ( ! f?$mime_type || f$mime_type != "application/x-dosexec" )
>  ??????? return;
>  ????????
>  ??? if ( ! f?$info || ! f$info?$tx_hosts )
>  ??????? return;
>  ????
>  ??? for ( txhost in f$info$tx_hosts )
>  ??????? {
>  ??????? if ( Site::is_local_addr(txhost) )
>  ??????????? return;
>  ??????? }
>  ????
>  ??? local ext = "";
> 
> ??? if ( f?$mime_type )
>  ??????? ext = ext_map[f$mime_type];
> 
> ??? local fname = fmt("%s-%s.%s", f$source, f$id, ext);
>  ??? Files::add_analyzer(f, Files::ANALYZER_EXTRACT, 
> [$extract_filename=fname]);
>  ??? }
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From stanhope at gmail.com  Sun Feb  2 13:14:19 2014
From: stanhope at gmail.com (Phil Stanhope)
Date: Sun, 2 Feb 2014 16:14:19 -0500
Subject: [Bro] How to create a CSV logging writer
In-Reply-To: <8D0EDB279BEBE0C-290C-1079E@webmail-m133.sysops.aol.com>
References: <8D0ED87EE1AF16D-290C-EF24@webmail-m133.sysops.aol.com>
	<52ED6728.8050500@hexwave.com>
	<8D0EDB279BEBE0C-290C-1079E@webmail-m133.sysops.aol.com>
Message-ID: <CADK3dNTxyAVH-Qd0ifJ2=W1NCRjbyauckqCE3mzQRSjdcSVeeQ@mail.gmail.com>

In the call to add_filter, you can specify the $config option like this:

$config=table(["tsv"] = "T")

I've tested this and that coupled with the following in our .bro script
file:

redef LogAscii::include_meta T;
redef LogAscii::separator ",";

Results in the minimal CSV output.

I found an example of this in
the testing/btest/scripts/base/frameworks/logging/ascii-tsv.bro
example/test file.

-phil



On Sat, Feb 1, 2014 at 5:17 PM, Jessica Smith <jes.smith.bro at aol.com> wrote:

> I know, but I need to print out a single header row with the column names
> and in this way I cannot do that.
>
>
>
> -----Original Message-----
> From: Matt Thompson <mthompson at hexwave.com>
> To: bro <bro at bro.org>
> Sent: Sat, Feb 1, 2014 10:37 pm
> Subject: Re: [Bro] How to create a CSV logging writer
>
>  Hi Jessica,
>
> I don't think you need a filter for CSV output.
>
> See:
> http://www.bro.org/sphinx/scripts/base/frameworks/logging/writers/ascii.html
>
> You can get CSV behavior without meta headers with:
>
> redef LogAscii::include_meta = F;
> redef LogAscii::separator = ",";
> redef LogAscii::set_separator = ";";
>
> Cheers,
> Matt Thompson
>
> On 2/1/14, 11:12 AM, Jessica Smith wrote:
>
> Hi,
>
>
> how can I add a new writer-specifi filter option via "config" in Log::Filter? I need to store log entries in CSV format where only a single header row with the column names is printed out as meta information, with no "# fields" prepended and no other meta data gets included in that mode.
>
>
> Jessica
>
>
>
>
> _______________________________________________
> Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
>   _______________________________________________
> Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140202/61788742/attachment.html 

From sangdrax8 at gmail.com  Mon Feb  3 09:37:01 2014
From: sangdrax8 at gmail.com (sangdrax8)
Date: Mon, 3 Feb 2014 12:37:01 -0500
Subject: [Bro] Problem changing restrict_filters
Message-ID: <CAL8ieHo0kjfCQwywSd52rT=xV7oCkk28wR1GsEWMVD=4+eV3nA@mail.gmail.com>

I am having an issue with changing my restrict_filters that I setup a while
back.  I don't know if I am just forgetting how this works, but if someone
can help me out here it would be much appreciated.

Previously I had added a section to my local.bro file to restrict the
traffic some of my nodes are seeing.  I used the following syntax with some
dummy IP's for an example:

const idsvm4_hosts = "192.168.0.1 or 192.168.0.2";

redef PacketFilter::enable_auto_protocol_capture_filters = T;
redef capture_filters = { ["all"] = "ip or not ip" };
redef restrict_filters = { ["local-src"] = "src host ("+idsvm4_hosts+")"  };
redef restrict_filters += { ["local-dst"] = "dst host ("+idsvm4_hosts+")"
 };

When I did this, I could use the print command in broctl to see that it was
in fact working as expected. (print restrict_filters idsvm4)

Now I am trying to change this list, and so I have edited the const I
declared previously.  I added a few hosts to idsvm4_hosts, and I did an
install and restart.  When I run the same print, I get back the original
restrict_filters.  It looks like the node keeps the old one.

while troubleshooting this I have gone as far as to completely remove all
my code about packet filters.  I issued an install, and restarted the
entire cluster.  Still the print statement returns with the ORIGINAL
restrict_filters I set a few months ago.  I feel like I must be missing
something here, but I just can't remember what I did.  I know I made this
variable so that in the future I could easily update it, but here I am
trying to update it with no success.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140203/c00ade03/attachment.html 

From sangdrax8 at gmail.com  Mon Feb  3 11:14:30 2014
From: sangdrax8 at gmail.com (sangdrax8)
Date: Mon, 3 Feb 2014 14:14:30 -0500
Subject: [Bro] Problem changing restrict_filters
In-Reply-To: <CAL8ieHo0kjfCQwywSd52rT=xV7oCkk28wR1GsEWMVD=4+eV3nA@mail.gmail.com>
References: <CAL8ieHo0kjfCQwywSd52rT=xV7oCkk28wR1GsEWMVD=4+eV3nA@mail.gmail.com>
Message-ID: <CAL8ieHqMPj9K4U8yOZ44aDq27eS1yVSdTnEyWpuiAg-Kz6sizA@mail.gmail.com>

I think I have the issue resolved, but I can't give more than a guess as to
what was wrong.  After not being able to update this I decided to delete
one worker node and just completely re-install it.  After doing this, it
still returned the same restrict_filter (which it shouldn't have ever seen
before).

I noticed that even after stopping the process through broctl, there was
still multiple processes running on the node.  I rebooted the machine, and
installed a second time. This time everything seems to be working as
expected.

My only guess is that it might have something to do with trying to do an
update rather than a restart.  The first time I attempted this, I did an
install to push the change and then an update.  That didn't appear to work
so I tried restarting.  After I rebooted my node I have always just been
doing install followed by a restart to pull in the changes.  Since doing
this my changes seem to apply correctly.

Wish I had something more definitive, but maybe it will save someone some
time.


On Mon, Feb 3, 2014 at 12:37 PM, sangdrax8 <sangdrax8 at gmail.com> wrote:

> I am having an issue with changing my restrict_filters that I setup a
> while back.  I don't know if I am just forgetting how this works, but if
> someone can help me out here it would be much appreciated.
>
> Previously I had added a section to my local.bro file to restrict the
> traffic some of my nodes are seeing.  I used the following syntax with some
> dummy IP's for an example:
>
> const idsvm4_hosts = "192.168.0.1 or 192.168.0.2";
>
> redef PacketFilter::enable_auto_protocol_capture_filters = T;
> redef capture_filters = { ["all"] = "ip or not ip" };
> redef restrict_filters = { ["local-src"] = "src host ("+idsvm4_hosts+")"
>  };
> redef restrict_filters += { ["local-dst"] = "dst host ("+idsvm4_hosts+")"
>  };
>
> When I did this, I could use the print command in broctl to see that it
> was in fact working as expected. (print restrict_filters idsvm4)
>
> Now I am trying to change this list, and so I have edited the const I
> declared previously.  I added a few hosts to idsvm4_hosts, and I did an
> install and restart.  When I run the same print, I get back the original
> restrict_filters.  It looks like the node keeps the old one.
>
> while troubleshooting this I have gone as far as to completely remove all
> my code about packet filters.  I issued an install, and restarted the
> entire cluster.  Still the print statement returns with the ORIGINAL
> restrict_filters I set a few months ago.  I feel like I must be missing
> something here, but I just can't remember what I did.  I know I made this
> variable so that in the future I could easily update it, but here I am
> trying to update it with no success.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140203/78a7734e/attachment.html 

From jlay at slave-tothe-box.net  Mon Feb  3 15:01:00 2014
From: jlay at slave-tothe-box.net (James Lay)
Date: Mon, 03 Feb 2014 16:01:00 -0700
Subject: [Bro] FakeAV video
Message-ID: <10210294806ad6b691d0fe48f455dc5a@localhost>

You folks deal with this a fair amount...here's a good look at what the 
users sees:

http://www.invincea.com/2014/01/k-i-a-dailymotion-part-2-fakeav-threat/

James


From gary at doit.wisc.edu  Mon Feb  3 15:06:17 2014
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Mon, 03 Feb 2014 17:06:17 -0600
Subject: [Bro] OOM-killer & Bro
Message-ID: <52F020E9.7050700@doit.wisc.edu>

Quick question for those of you running Bro clusters. I often run into 
situations where OOM-killer invokes and kills some Bro process. Do any 
of you do anything to tune OOM-killer on Linux or otherwise tune memory 
management, such as disabling OOM-killer, turning off swap etc?

Background : I've had various success tracking down the events that 
cause me to suddenly run out of memory and ultimately crash. Sometimes 
it seems to be the result of log rotation getting stuck on a really big 
file (8Gig http or dns log), or a sudden 10G traffic spike overwhelming 
the cluster. I've pursued various avenues to mitigate the issue such as 
shorter log rotation intervals, pruning known high throughput compute 
traffic, scheduling daily restarts etc. Ultimately I'm also looking to 
increase RAM, but I'm concerned even with more RAM, I'm just a traffic 
spike away from OOM-killer, especially since we are unlikely to be able 
to buy cluster hardware fast enough to keep up with traffic volumes.

Regards,

-- 
Gary Faulkner
UW Madison
Office of Campus Information Security
608-262-8591


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6257 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140203/c495dedd/attachment.bin 

From jlay at slave-tothe-box.net  Mon Feb  3 15:11:28 2014
From: jlay at slave-tothe-box.net (James Lay)
Date: Mon, 03 Feb 2014 16:11:28 -0700
Subject: [Bro] FakeAV video
In-Reply-To: <10210294806ad6b691d0fe48f455dc5a@localhost>
References: <10210294806ad6b691d0fe48f455dc5a@localhost>
Message-ID: <6a49aee031fb5146d54c191b23b38dbf@localhost>

On 2014-02-03 16:01, James Lay wrote:
> You folks deal with this a fair amount...here's a good look at what 
> the
> users sees:
>
> 
> http://www.invincea.com/2014/01/k-i-a-dailymotion-part-2-fakeav-threat/
>
> James
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

Kindly disregard....fast typing :(

James


From alexwis at gmail.com  Mon Feb  3 17:06:00 2014
From: alexwis at gmail.com (Alex Waher)
Date: Mon, 3 Feb 2014 17:06:00 -0800
Subject: [Bro] OOM-killer & Bro
In-Reply-To: <52F020E9.7050700@doit.wisc.edu>
References: <52F020E9.7050700@doit.wisc.edu>
Message-ID: <CAGURB2eKC8m3MXV_7yncpPWnzCu+=oXOOx4nAOQ76v3knr93kg@mail.gmail.com>

Are you chasing a memory leak?  `broctl top` will generally report >500MB
of reserved memory (90% of the time even >256M) per worker in a 40 worker
cluster capable of handling spikes of 10Gb?. Each worker has ~3GB RAM to it.

I recall the log rotation process is a separate cron-style job that
shouldn't really be bring down the cluster workers.

-Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140203/31313998/attachment.html 

From gary at doit.wisc.edu  Mon Feb  3 18:31:45 2014
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Mon, 03 Feb 2014 20:31:45 -0600
Subject: [Bro] OOM-killer & Bro
In-Reply-To: <CAGURB2eKC8m3MXV_7yncpPWnzCu+=oXOOx4nAOQ76v3knr93kg@mail.gmail.com>
References: <52F020E9.7050700@doit.wisc.edu>
	<CAGURB2eKC8m3MXV_7yncpPWnzCu+=oXOOx4nAOQ76v3knr93kg@mail.gmail.com>
Message-ID: <52F05111.1000908@doit.wisc.edu>

I'm not sure. I'm running Bro 2.2 (release) with the default scripts and 
most of the known memory leak issues don't seem to apply to me. Other 
than some limited testing I haven't been using any custom scripts, and 
none that depend on the input framework.

I've been thinking I may need more than 64G of RAM per node (16 core / 
3-5G traffic, & 12 workers each). I seem to run with 100% of the RAM 
allocated, but 20-30% of my RAM cached before something happens to cause 
a sudden drop in cached memory (as seen on Orca graphs) resulting in 
OOM-killer dropping one or more Bro processes.

I've been reading a bit about OOM-killer and some high performance 
situations seem to call for disabling it, so I'm investigating whether 
it makes sense to tweak vm-overcommit settings to disallow allocating 
more than the total physical RAM + SWAP, but I don't know if this is 
advisable for Bro or not, hence the query.

~ Gary

On 2/3/2014 7:06 PM, Alex Waher wrote:
> Are you chasing a memory leak?  `broctl top` will generally report >500MB
> of reserved memory (90% of the time even >256M) per worker in a 40 worker
> cluster capable of handling spikes of 10Gb?. Each worker has ~3GB RAM to it.
>
> I recall the log rotation process is a separate cron-style job that
> shouldn't really be bring down the cluster workers.
>
> -Alex
>



From carlopmart at gmail.com  Tue Feb  4 02:28:39 2014
From: carlopmart at gmail.com (C. L. Martinez)
Date: Tue, 4 Feb 2014 10:28:39 +0000
Subject: [Bro] Questions about Bro clusters deployments
Message-ID: <CAEjQA5LFc4Z4pCKcZCAqd57wih49JMhLjC89YpmBWDb8MRovWQ@mail.gmail.com>

 Hi all,

 I am thinking to install some bro sensors in our infrastructure under
CentOS and FreeBSD hosts using the new release 2.2. My idea is to use
bro cluster features to setup centralized configs and logs. But after
reading doc section about this type of deployment I have some doubts:

 a) Policy rules: Do they need to be stored in the manager or can I
deploy different rules for every bro worker? For example, if I setup
worker A and worker B and I will to deploy only 10 rules for worker A
and 20 for worker B, how can I do?

 b) About *.cfg files: Do I need to configure these files on every
worker or only on the manager? But if it is only on the manger side
and workers needs to monitor different networks as a internal
networks, how can I segregate this?

 c) About bpf filters: In this new release (2.2), Is it possible to
add bpf filters out-of-the-box or do I need to implement customized
scripts, like for example securityonion does?

 Thanks


From JAzoff at albany.edu  Tue Feb  4 07:01:52 2014
From: JAzoff at albany.edu (Justin Azoff)
Date: Tue, 4 Feb 2014 10:01:52 -0500
Subject: [Bro] OOM-killer & Bro
In-Reply-To: <52F05111.1000908@doit.wisc.edu>
References: <52F020E9.7050700@doit.wisc.edu>
	<CAGURB2eKC8m3MXV_7yncpPWnzCu+=oXOOx4nAOQ76v3knr93kg@mail.gmail.com>
	<52F05111.1000908@doit.wisc.edu>
Message-ID: <20140204150152.GM8640@datacomm.albany.edu>

On Mon, Feb 03, 2014 at 08:31:45PM -0600, Gary Faulkner wrote:
> I've been thinking I may need more than 64G of RAM per node (16 core / 
> 3-5G traffic, & 12 workers each). I seem to run with 100% of the RAM 
> allocated, but 20-30% of my RAM cached before something happens to cause 
> a sudden drop in cached memory (as seen on Orca graphs) resulting in 
> OOM-killer dropping one or more Bro processes.

You should be fine with those specs..  12 workers should be using closer
to 12G of ram, not anywhere near 64G.

Can you post the output of

    free -m     # on one of the worker nodes
    broctl top  # on the manager

and to get an idea of your msg log rate:

    cat bro/logs/current/* | wc -l ; sleep 1m ; cat bro/logs/current/* | wc -l

Can you also share the memory graph from this system over time,
particularly after a fresh restart of bro?

-- 
-- Justin Azoff
-- Network Security & Performance Analyst


From gary at doit.wisc.edu  Tue Feb  4 10:43:14 2014
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Tue, 04 Feb 2014 12:43:14 -0600
Subject: [Bro] OOM-killer & Bro
In-Reply-To: <20140204150152.GM8640@datacomm.albany.edu>
References: <52F020E9.7050700@doit.wisc.edu>
	<CAGURB2eKC8m3MXV_7yncpPWnzCu+=oXOOx4nAOQ76v3knr93kg@mail.gmail.com>
	<52F05111.1000908@doit.wisc.edu>
	<20140204150152.GM8640@datacomm.albany.edu>
Message-ID: <52F134C2.8090401@doit.wisc.edu>

The output is below. I've been running the host longer than I have Orca 
graphs for. When looking at the graphs, you can identify the restarts 
based on the sudden spike in free memory (in blue). There is a series of 
restarts toward the end of last week and beginning of this week where I 
was experimenting with a script and making changes. That script was only 
tested between last Thursday and this Monday. Traffic and log rates are 
taken between / 11AM-1PM. In some cases I tried to collect multiple samples.

The history of OOM events predates the graphs, so I thought it would be 
useful to know them as well. I have been reducing workload & workers 
over time as part of troubleshooting. I also sometimes have an issue 
where bro log rotations fails, and I need to rotate logs manually and 
restart. This usually happens when available/cached  memory drops below 
about 8G.

Machines each have 2 of e5-2670s (8-cores, 2.6Ghz) & 64G RAM. So 16 
cores / 32 HT per machine.

OOM-Killer (host 1, manager + 2 proxies + x workers):
Nov 21 - 24 workers
Nov 22 - 24 workers
Nov 25 - 24 workers
Dec  4 - 20 workers
Dec  5 - 20 workers
Dec  6 - 20 workers
Dec 27 - 12 workers
Jan 26 - 12 workers
Jan 30 - 12 workers - might be related to script testing
Feb  1 - 12 workers - might be related to script testing

OOM-Killer (host2, 2 proxies + x workers):
Nov 20 - 24 workers
Nov 21 - 24 workers
Nov 22 - 24 workers
Nov 25 - 24 workers
Dec  4 - 20 workers
Dec  5 - 20 workers
Dec  6 - 20 workers
Jan 26 - 12 workers

broctl top 11:15AM:
==================

Name       Type       Node       Pid      Proc     VSize Rss      
Cpu      Cmd
manager    manager    host1 27037    parent    14G      12G 99%      bro
manager    manager    host1 27038    child    183M     106M 32%      bro
proxy-1    proxy      host1 27089    child     94M      17M 12%      bro
proxy-1    proxy      host1 27086    parent     1G       1G 5%       bro
proxy-2    proxy      host1 27088    child     94M      19M 14%      bro
proxy-2    proxy      host1 27087    parent     1G       1G 9%       bro
proxy-3    proxy      host2 8848     child     94M      37M 12%      bro
proxy-3    proxy      host2 8846     parent     1G       1G 10%      bro
proxy-4    proxy      host2 8849     child     94M      37M 14%      bro
proxy-4    proxy      host2 8847     parent     1G       1G 5%       bro
worker-1-1 worker     host1 27320    parent     2G       2G 96%      bro
worker-1-1 worker     host1 27421    child    377M      18M 10%      bro
worker-1-10 worker     host1 27323    parent     2G       2G 97%      bro
worker-1-10 worker     host1 27414    child    373M      13M 8%       bro
worker-1-11 worker     host1 27324    parent     2G       2G 100%     bro
worker-1-11 worker     host1 27399    child    369M      10M 7%       bro
worker-1-12 worker     host1 27325    parent     2G       2G 85%      bro
worker-1-12 worker     host1 27410    child    369M       7M 10%      bro
worker-1-2 worker     host1 27328    parent     2G       2G 99%      bro
worker-1-2 worker     host1 27446    child    369M       8M 7%       bro
worker-1-3 worker     host1 27330    parent     2G       2G 97%      bro
worker-1-3 worker     host1 27427    child    369M       9M 8%       bro
worker-1-4 worker     host1 27331    parent     2G       2G 100%     bro
worker-1-4 worker     host1 27411    child    369M      10M 7%       bro
worker-1-5 worker     host1 27332    parent     2G       2G 97%      bro
worker-1-5 worker     host1 27430    child    369M       7M 10%      bro
worker-1-6 worker     host1 27333    parent     3G       3G 98%      bro
worker-1-6 worker     host1 27413    child    369M       9M 10%      bro
worker-1-7 worker     host1 27335    parent     2G       2G 100%     bro
worker-1-7 worker     host1 27426    child    369M       8M 8%       bro
worker-1-8 worker     host1 27334    parent     2G       2G 100%     bro
worker-1-8 worker     host1 27433    child    369M       9M 7%       bro
worker-1-9 worker     host1 27336    parent     2G       2G 100%     bro
worker-1-9 worker     host1 27425    child    369M      10M 9%       bro
worker-2-1 worker     host2 9495     parent     2G       2G 98%      bro
worker-2-1 worker     host2 9533     child    369M      30M 9%       bro
worker-2-10 worker     host2 9494     parent     2G       2G 100%     bro
worker-2-10 worker     host2 9582     child    369M      30M 7%       bro
worker-2-11 worker     host2 9496     parent     3G       3G 97%      bro
worker-2-11 worker     host2 9586     child    369M      30M 9%       bro
worker-2-12 worker     host2 9492     parent     2G       2G 97%      bro
worker-2-12 worker     host2 9585     child    369M      30M 9%       bro
worker-2-2 worker     host2 9502     parent     2G       2G 98%      bro
worker-2-2 worker     host2 9512     child    369M      29M 10%      bro
worker-2-3 worker     host2 9493     parent     2G       2G 98%      bro
worker-2-3 worker     host2 9511     child    370M      31M 9%       bro
worker-2-4 worker     host2 9498     parent     2G       2G 90%      bro
worker-2-4 worker     host2 9576     child    369M      30M 10%      bro
worker-2-5 worker     host2 9503     parent     2G       2G 98%      bro
worker-2-5 worker     host2 9517     child    369M      30M 9%       bro
worker-2-6 worker     host2 9500     parent     2G       2G 99%      bro
worker-2-6 worker     host2 9506     child    369M      31M 7%       bro
worker-2-7 worker     host2 9499     parent     2G       2G 100%     bro
worker-2-7 worker     host2 9538     child    369M      29M 9%       bro
worker-2-8 worker     host2 9497     parent     2G       2G 99%      bro
worker-2-8 worker     host2 9587     child    369M      29M 5%       bro
worker-2-9 worker     host2 9501     parent     2G       2G 98%      bro
worker-2-9 worker     host2 9519     child    369M      30M 7%       bro

broctl top 12:21PM:
==================

Name       Type       Node       Pid      Proc     VSize Rss      
Cpu      Cmd
manager    manager    host1 27037    parent    14G      12G 158%     bro
manager    manager    host1 27038    child    143M      66M 37%      bro
proxy-1    proxy      host1 27089    child     94M      17M 10%      bro
proxy-1    proxy      host1 27086    parent     1G       1G 5%       bro
proxy-2    proxy      host1 27088    child     94M      18M 16%      bro
proxy-2    proxy      host1 27087    parent     1G       1G 8%       bro
proxy-3    proxy      host2 8848     child     94M      37M 14%      bro
proxy-3    proxy      host2 8846     parent     1G       1G 12%      bro
proxy-4    proxy      host2 8849     child     94M      37M 16%      bro
proxy-4    proxy      host2 8847     parent     1G       1G 5%       bro
worker-1-1 worker     host1 27320    parent     3G       3G 97%      bro
worker-1-1 worker     host1 27421    child    377M      18M 9%       bro
worker-1-10 worker     host1 27323    parent     3G       3G 99%      bro
worker-1-10 worker     host1 27414    child    373M      13M 10%      bro
worker-1-11 worker     host1 27324    parent     3G       3G 96%      bro
worker-1-11 worker     host1 27399    child    369M      10M 10%      bro
worker-1-12 worker     host1 27325    parent     3G       3G 100%     bro
worker-1-12 worker     host1 27410    child    369M       7M 12%      bro
worker-1-2 worker     host1 27328    parent     3G       3G 98%      bro
worker-1-2 worker     host1 27446    child    369M       7M 10%      bro
worker-1-3 worker     host1 27330    parent     3G       3G 99%      bro
worker-1-3 worker     host1 27427    child    369M       9M 10%      bro
worker-1-4 worker     host1 27331    parent     3G       3G 100%     bro
worker-1-4 worker     host1 27411    child    369M       9M 7%       bro
worker-1-5 worker     host1 27332    parent     3G       3G 95%      bro
worker-1-5 worker     host1 27430    child    369M       7M 10%      bro
worker-1-6 worker     host1 27333    parent     3G       3G 97%      bro
worker-1-6 worker     host1 27413    child    369M       8M 10%      bro
worker-1-7 worker     host1 27335    parent     3G       3G 97%      bro
worker-1-7 worker     host1 27426    child    369M       8M 8%       bro
worker-1-8 worker     host1 27334    parent     3G       3G 99%      bro
worker-1-8 worker     host1 27433    child    369M       9M 9%       bro
worker-1-9 worker     host1 27336    parent     3G       3G 100%     bro
worker-1-9 worker     host1 27425    child    369M      10M 9%       bro
worker-2-1 worker     host2 9495     parent     3G       3G 95%      bro
worker-2-1 worker     host2 9533     child    369M      30M 5%       bro
worker-2-10 worker     host2 9494     parent     3G       3G 98%      bro
worker-2-10 worker     host2 9582     child    369M      30M 7%       bro
worker-2-11 worker     host2 9496     parent     3G       3G 99%      bro
worker-2-11 worker     host2 9586     child    369M      30M 10%      bro
worker-2-12 worker     host2 9492     parent     3G       3G 99%      bro
worker-2-12 worker     host2 9585     child    369M      30M 10%      bro
worker-2-2 worker     host2 9502     parent     3G       3G 98%      bro
worker-2-2 worker     host2 9512     child    369M      29M 10%      bro
worker-2-3 worker     host2 9493     parent     3G       3G 99%      bro
worker-2-3 worker     host2 9511     child    370M      31M 7%       bro
worker-2-4 worker     host2 9498     parent     3G       3G 97%      bro
worker-2-4 worker     host2 9576     child    369M      30M 10%      bro
worker-2-5 worker     host2 9503     parent     3G       3G 98%      bro
worker-2-5 worker     host2 9517     child    369M      30M 9%       bro
worker-2-6 worker     host2 9500     parent     3G       3G 100%     bro
worker-2-6 worker     host2 9506     child    369M      31M 9%       bro
worker-2-7 worker     host2 9499     parent     3G       3G 99%      bro
worker-2-7 worker     host2 9538     child    369M      29M 9%       bro
worker-2-8 worker     host2 9497     parent     3G       3G 98%      bro
worker-2-8 worker     host2 9587     child    369M      29M 7%       bro
worker-2-9 worker     host2 9501     parent     3G       3G 100%     bro
worker-2-9 worker     host2 9519     child    369M      30M 7%       bro
broctl capstats:
=============
11:15AM:
Interface            kpps       mbps       (10s average)
------------------------------
host1/dnacluster:21 460.3      2993.6
host2/dnacluster:22 497.4      3291.3

Total                957.7      6284.9

11:30AM:
Interface            kpps       mbps       (10s average)
------------------------------
host1/dnacluster:21 509.0      3301.1
host2/dnacluster:22 469.3      2933.3

Total                978.3      6234.4

12:15PM:

Interface            kpps       mbps       (10s average)
------------------------------
host1/dnacluster:21 565.3      3741.6
host2/dnacluster:22 522.6      3358.8

Total                1087.9     7100.4

free -m on host 1 (manager + 2 proxies + 12 workers) 11:15AM:
=======================================================
              total       used       free     shared    buffers cached
Mem:         64377      63670        707          0 71      19091
-/+ buffers/cache:      44506      19871
Swap:         1023        650        373

free -m on host 1 (manager + 2 proxies + 12 workers) 12:15PM:
=======================================================
              total       used       free     shared    buffers cached
Mem:         64377      64108        269          0 0       8245
-/+ buffers/cache:      55862       8515
Swap:         1023       1023          0

free -m on host 2 (2 proxies + 12 workers) 11:15AM:
==============================================
              total       used       free     shared    buffers cached
Mem:         64377      34827      29550          0 104       2184
-/+ buffers/cache:      32538      31839
Swap:         1023         17       1006

free -m on host 2 (2 proxies + 12 workers) 12:15PM:
==============================================
              total       used       free     shared    buffers cached
Mem:         64377      46118      18259          0 104       2186
-/+ buffers/cache:      43827      20550
Swap:         1023         17       1006

Log rate: (/current)
=================
11:15AM
cat * | wc -l ; sleep 1m ; cat * | wc -l
22006062
23762376
diff=1,756,314/min

11:30AM
cat * | wc -l ; sleep 1m ; cat * | wc -l
7618833
9873332
diff=2,254,499/min

Bro failed log rotation at 11:40AM, so I had to manually rotate logs and 
restart.

12:28PM:
  cat * | wc -l ; sleep 1m ; cat * | wc -l
14526373
16633887
diff=2,107,514/min

Regards,

Gary Faulkner
UW Madison
Office of Campus Information Security
608-262-8591

On 2/4/2014 9:01 AM, Justin Azoff wrote:
> On Mon, Feb 03, 2014 at 08:31:45PM -0600, Gary Faulkner wrote:
>> I've been thinking I may need more than 64G of RAM per node (16 core /
>> 3-5G traffic, & 12 workers each). I seem to run with 100% of the RAM
>> allocated, but 20-30% of my RAM cached before something happens to cause
>> a sudden drop in cached memory (as seen on Orca graphs) resulting in
>> OOM-killer dropping one or more Bro processes.
> You should be fine with those specs..  12 workers should be using closer
> to 12G of ram, not anywhere near 64G.
>
> Can you post the output of
>
>      free -m     # on one of the worker nodes
>      broctl top  # on the manager
>
> and to get an idea of your msg log rate:
>
>      cat bro/logs/current/* | wc -l ; sleep 1m ; cat bro/logs/current/* | wc -l
>
> Can you also share the memory graph from this system over time,
> particularly after a fresh restart of bro?
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: host1hourlymemuse4FEB2014.png
Type: image/png
Size: 6604 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140204/3736c6e3/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: host1monthlymemuse4FEB2014.png
Type: image/png
Size: 7972 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140204/3736c6e3/attachment-0001.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: host1weeklymemuse4FEB2014.png
Type: image/png
Size: 9472 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140204/3736c6e3/attachment-0002.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: host1dailymemuse4FEB2014.png
Type: image/png
Size: 10299 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140204/3736c6e3/attachment-0003.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6257 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140204/3736c6e3/attachment-0004.bin 

From JAzoff at albany.edu  Tue Feb  4 10:59:02 2014
From: JAzoff at albany.edu (Justin Azoff)
Date: Tue, 4 Feb 2014 13:59:02 -0500
Subject: [Bro] OOM-killer & Bro
In-Reply-To: <52F134C2.8090401@doit.wisc.edu>
References: <52F020E9.7050700@doit.wisc.edu>
	<CAGURB2eKC8m3MXV_7yncpPWnzCu+=oXOOx4nAOQ76v3knr93kg@mail.gmail.com>
	<52F05111.1000908@doit.wisc.edu>
	<20140204150152.GM8640@datacomm.albany.edu>
	<52F134C2.8090401@doit.wisc.edu>
Message-ID: <20140204185902.GO8640@datacomm.albany.edu>

On Tue, Feb 04, 2014 at 12:43:14PM -0600, Gary Faulkner wrote:
> 11:30AM
> cat * | wc -l ; sleep 1m ; cat * | wc -l
> 7618833
> 9873332
> diff=2,254,499/min

That is quite a lot of logs... Can you do just a `wc -l *` a minute
apart and diff that?  I'm particularly wondering what the rate of
notices/sec you are getting.  I recently ran into and fixed an issue
with notice supression using a lot of memory:

https://bro-tracker.atlassian.net/browse/BIT-1115
https://github.com/bro/bro/commit/ec3f684c610f084fdea8ed5cf85f9c4390eb58e6

I wonder if that could be the issue you are running into..

-- 
-- Justin Azoff


From gary at doit.wisc.edu  Tue Feb  4 11:19:15 2014
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Tue, 04 Feb 2014 13:19:15 -0600
Subject: [Bro] OOM-killer & Bro
In-Reply-To: <20140204185902.GO8640@datacomm.albany.edu>
References: <52F020E9.7050700@doit.wisc.edu>
	<CAGURB2eKC8m3MXV_7yncpPWnzCu+=oXOOx4nAOQ76v3knr93kg@mail.gmail.com>
	<52F05111.1000908@doit.wisc.edu>
	<20140204150152.GM8640@datacomm.albany.edu>
	<52F134C2.8090401@doit.wisc.edu>
	<20140204185902.GO8640@datacomm.albany.edu>
Message-ID: <52F13D33.9040209@doit.wisc.edu>

Here it is just after a log rotation:
         14 app_stats.log
         32 capture_loss.log
       3075 communication.log
   10515588 conn.log
    1463723 dns.log
      13760 dpd.log
    1562035 files.log
       1527 ftp.log
    1771968 http.log
         74 irc.log
        127 known_certs.log
      21540 known_hosts.log
       2696 known_services.log
        325 notice.log
        242 reporter.log
      37892 smtp.log
         13 socks.log
      78387 software.log
       3247 ssh.log
     552563 ssl.log
          4 stderr.log
          3 stdout.log
     672817 syslog.log
        556 traceroute.log
       5790 tunnel.log
     472964 weird.log
   17180962 total

1 min later:
         14 app_stats.log
         32 capture_loss.log
       3470 communication.log
   11859982 conn.log
    1619893 dns.log
      15468 dpd.log
    1760513 files.log
       1679 ftp.log
    1993477 http.log
         86 irc.log
        139 known_certs.log
      23839 known_hosts.log
       2881 known_services.log
        352 notice.log
        259 reporter.log
      42941 smtp.log
         13 socks.log
      88544 software.log
       3581 ssh.log
     622256 ssl.log
          4 stderr.log
          3 stdout.log
     750444 syslog.log
        561 traceroute.log
       6567 tunnel.log
     530259 weird.log
   19327257 total

And the diff:

0 app_stats.log
0 capture_loss.log
395 communication.log
1344394 conn.log
156170 dns.log
1708 dpd.log
198478 files.log
152 ftp.log
221509 http.log
12 irc.log
12 known_certs.log
2299 known_hosts.log
185 known_services.log
27 notice.log
17 reporter.log
5049 smtp.log
0 socks.log
10157 software.log
334 ssh.log
69693 ssl.log
0 stderr.log
0 stdout.log
77627 syslog.log
5 traceroute.log
777 tunnel.log
57295 weird.log
2146295 total

Regards,

Gary Faulkner
UW Madison
Office of Campus Information Security
608-262-8591

On 2/4/2014 12:59 PM, Justin Azoff wrote:
> On Tue, Feb 04, 2014 at 12:43:14PM -0600, Gary Faulkner wrote:
>> 11:30AM
>> cat * | wc -l ; sleep 1m ; cat * | wc -l
>> 7618833
>> 9873332
>> diff=2,254,499/min
> That is quite a lot of logs... Can you do just a `wc -l *` a minute
> apart and diff that?  I'm particularly wondering what the rate of
> notices/sec you are getting.  I recently ran into and fixed an issue
> with notice supression using a lot of memory:
>
> https://bro-tracker.atlassian.net/browse/BIT-1115
> https://github.com/bro/bro/commit/ec3f684c610f084fdea8ed5cf85f9c4390eb58e6
>
> I wonder if that could be the issue you are running into..
>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6257 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140204/8368fef8/attachment.bin 

From jes.smith.bro at aol.com  Tue Feb  4 15:59:49 2014
From: jes.smith.bro at aol.com (Jessica Smith)
Date: Tue, 4 Feb 2014 18:59:49 -0500 (EST)
Subject: [Bro] bug in ssl_alert event?
Message-ID: <8D0F01C52A53AE6-C78-5A88@webmail-m133.sysops.aol.com>


Hi,


why when I print the "level" of the alert message I get numbers different from 1 (warning) or 2 (fatal) ?


event ssl_alert(c: connection, is_orig: bool, level: count, desc: count)
{
	print fmt("%d    %d", level, desc);


}


#out

61    173
123    165
13    61
200    80
8    121
187    31
144    218
82    243
6    224
237    72
115    121
92    152
196    44
2    255
141    216
42    88







-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140204/8d38e842/attachment.html 

From jsiwek at illinois.edu  Wed Feb  5 10:11:51 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Wed, 5 Feb 2014 18:11:51 +0000
Subject: [Bro] bug in ssl_alert event?
In-Reply-To: <8D0F01C52A53AE6-C78-5A88@webmail-m133.sysops.aol.com>
References: <8D0F01C52A53AE6-C78-5A88@webmail-m133.sysops.aol.com>
Message-ID: <9B6E6622-4500-4432-B04F-DF27C3023B0D@illinois.edu>


On Feb 4, 2014, at 5:59 PM, Jessica Smith <jes.smith.bro at aol.com> wrote:

> why when I print the "level" of the alert message I get numbers different from 1 (warning) or 2 (fatal) ?

The explanations I can think of are 1) bug in the ssl parser 2) ssl parser got attached to a connection that?s not actually ssl 3) the ssl alert records actually contain those odd level values.

If you can provide a small example pcap, that could be helpful.  Else you might get more clues by checking whether the values in c$ssl end up looking sane for the connections in question.

- Jon


From carlopmart at gmail.com  Thu Feb  6 02:50:12 2014
From: carlopmart at gmail.com (C. L. Martinez)
Date: Thu, 6 Feb 2014 10:50:12 +0000
Subject: [Bro] Questions about Bro clusters deployments
In-Reply-To: <CAEjQA5LFc4Z4pCKcZCAqd57wih49JMhLjC89YpmBWDb8MRovWQ@mail.gmail.com>
References: <CAEjQA5LFc4Z4pCKcZCAqd57wih49JMhLjC89YpmBWDb8MRovWQ@mail.gmail.com>
Message-ID: <CAEjQA5JuOv3XA1mBmhko=wXu6RWuumoko-7YuZN3UM2Q4g4Y+w@mail.gmail.com>

On Tue, Feb 4, 2014 at 10:28 AM, C. L. Martinez <carlopmart at gmail.com> wrote:
>  Hi all,
>
>  I am thinking to install some bro sensors in our infrastructure under
> CentOS and FreeBSD hosts using the new release 2.2. My idea is to use
> bro cluster features to setup centralized configs and logs. But after
> reading doc section about this type of deployment I have some doubts:
>
>  a) Policy rules: Do they need to be stored in the manager or can I
> deploy different rules for every bro worker? For example, if I setup
> worker A and worker B and I will to deploy only 10 rules for worker A
> and 20 for worker B, how can I do?
>
>  b) About *.cfg files: Do I need to configure these files on every
> worker or only on the manager? But if it is only on the manger side
> and workers needs to monitor different networks as a internal
> networks, how can I segregate this?
>
>  c) About bpf filters: In this new release (2.2), Is it possible to
> add bpf filters out-of-the-box or do I need to implement customized
> scripts, like for example securityonion does?
>
>  Thanks

Please, any input?

Thanks.


From JAzoff at albany.edu  Thu Feb  6 11:29:54 2014
From: JAzoff at albany.edu (Justin Azoff)
Date: Thu, 6 Feb 2014 14:29:54 -0500
Subject: [Bro] OOM-killer & Bro
In-Reply-To: <52F13D33.9040209@doit.wisc.edu>
References: <52F020E9.7050700@doit.wisc.edu>
	<CAGURB2eKC8m3MXV_7yncpPWnzCu+=oXOOx4nAOQ76v3knr93kg@mail.gmail.com>
	<52F05111.1000908@doit.wisc.edu>
	<20140204150152.GM8640@datacomm.albany.edu>
	<52F134C2.8090401@doit.wisc.edu>
	<20140204185902.GO8640@datacomm.albany.edu>
	<52F13D33.9040209@doit.wisc.edu>
Message-ID: <20140206192954.GQ8640@datacomm.albany.edu>

On Tue, Feb 04, 2014 at 01:19:15PM -0600, Gary Faulkner wrote:
> And the diff:
> 
> 0 app_stats.log
> 0 capture_loss.log
> 395 communication.log
> 1344394 conn.log
> 156170 dns.log
> 1708 dpd.log
> 198478 files.log
> 152 ftp.log
> 221509 http.log
> 12 irc.log
> 12 known_certs.log
> 2299 known_hosts.log
> 185 known_services.log
> 27 notice.log
> 17 reporter.log
> 5049 smtp.log
> 0 socks.log
> 10157 software.log
> 334 ssh.log
> 69693 ssl.log
> 0 stderr.log
> 0 stdout.log
> 77627 syslog.log
> 5 traceroute.log
> 777 tunnel.log
> 57295 weird.log
> 2146295 total


You only had 27 notices, so it wasn't that problem..

I think @load'ing misc/profiling would be a good next troubleshooting
step.  I believe the resulting prof.log can indicate which tables in
memory are growing too large.

-- 
-- Justin Azoff


From anthony.kasza at gmail.com  Thu Feb  6 18:42:33 2014
From: anthony.kasza at gmail.com (anthony kasza)
Date: Thu, 6 Feb 2014 18:42:33 -0800
Subject: [Bro] Bro Process Initialization and Stages
Message-ID: <CAEZw2bydnwgpwieZN34PVBA7rHc=qfZiLG6d7YDbgaohQ5t_5Q@mail.gmail.com>

Hi All,

I'm trying to determine the "stages" of a Bro process when invoked
from bro/bin/bro. I live mainly in scriptland and could be very wrong,
but in my mind things occur in the following order:

1. The Bro process is spawned.
2. The core does some initialization things (I'm guessing).
3. @ statements, including @load / @if / @idef / etc, are handled (in
order of appearance until all scripts are loaded).
4. Script land is initialized (all bro_init() events are handled by priority).
5. Export statements are handled making things globally available.
6. Packet processing in the core begins.
7. Events are raised by the core and handled by scriptland
(asynchronously and, again, by priority).
8. Scriptland terminates (all bro_done() events are handled by priority).
9. The core terminates (I'm assuming some garbage collection or clean
up happens here).
10. The Bro process dies.

In my mind I compare these stages to runlevels. I assume I'm missing
quite a few details. Perhaps someone here can add some insight into
each of these stages or stages I've overlooked. Thanks,

-Anthony


From robin at icir.org  Fri Feb  7 07:42:09 2014
From: robin at icir.org (Robin Sommer)
Date: Fri, 7 Feb 2014 07:42:09 -0800
Subject: [Bro] Bro Process Initialization and Stages
In-Reply-To: <CAEZw2bydnwgpwieZN34PVBA7rHc=qfZiLG6d7YDbgaohQ5t_5Q@mail.gmail.com>
References: <CAEZw2bydnwgpwieZN34PVBA7rHc=qfZiLG6d7YDbgaohQ5t_5Q@mail.gmail.com>
Message-ID: <20140207154209.GN19640@icir.org>

Anthony,

nice summary. One one correction: Step 5 happens earlier as the
scripts are parsed, i.e., as part of step 3. That's the reason for
having redef: in step 3 we can still change the value of constants,
afterwards we can't anymore.

Step 2 has internally become quite complex these days but that doesn't
really matter much from script-land perspective; it basically
preparares all the infrastructure that will later be needed. Step 9 is
the same in reverse (i.e., shutting down all the infrastructure
pieces).

Robin

On Thu, Feb 06, 2014 at 18:42 -0800, you wrote:

> Hi All,
> 
> I'm trying to determine the "stages" of a Bro process when invoked
> from bro/bin/bro. I live mainly in scriptland and could be very wrong,
> but in my mind things occur in the following order:
> 
> 1. The Bro process is spawned.
> 2. The core does some initialization things (I'm guessing).
> 3. @ statements, including @load / @if / @idef / etc, are handled (in
> order of appearance until all scripts are loaded).
> 4. Script land is initialized (all bro_init() events are handled by priority).
> 5. Export statements are handled making things globally available.
> 6. Packet processing in the core begins.
> 7. Events are raised by the core and handled by scriptland
> (asynchronously and, again, by priority).
> 8. Scriptland terminates (all bro_done() events are handled by priority).
> 9. The core terminates (I'm assuming some garbage collection or clean
> up happens here).
> 10. The Bro process dies.
> 
> In my mind I compare these stages to runlevels. I assume I'm missing
> quite a few details. Perhaps someone here can add some insight into
> each of these stages or stages I've overlooked. Thanks,
> 
> -Anthony
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 


-- 
Robin Sommer * Phone +1 (510) 722-6541 *     robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 * www.icir.org/robin


From tray at 21ct.com  Fri Feb  7 11:04:58 2014
From: tray at 21ct.com (Tim Ray)
Date: Fri, 7 Feb 2014 13:04:58 -0600
Subject: [Bro] Question about tuning
Message-ID: <CF1A8A7A.A580%tray@21ct.com>

Getting lots of this in dpd:
unexpected Handshake message SERVER HELLO from responder in state INITIAL

Looks like in the SSL analyzer. By far the bulk of the messages we?re
seeing. Anyone seen this and tuned it? Or is it indicative of a serious
misconfiguration?


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140207/f7b1fc97/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5038 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140207/f7b1fc97/attachment.bin 

From bernhard at ICSI.Berkeley.EDU  Fri Feb  7 12:24:05 2014
From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann)
Date: Fri, 7 Feb 2014 12:24:05 -0800
Subject: [Bro] Question about tuning
In-Reply-To: <CF1A8A7A.A580%tray@21ct.com>
References: <CF1A8A7A.A580%tray@21ct.com>
Message-ID: <666EA811-9EFA-4B53-AE26-377C1BBE81D3@icsi.berkeley.edu>

Hello Tim,

without actually looking into the analyzer source - if I am not mistaken what the message is saying is that 
bro saw a server hello message being sent without the client hello being sent first (which
is required by the protocol). 

I have not seen heard of this happening anywhere consistently, and cannot really
see how that usually should happen on a regular basis. Would it perhaps be possible to get a
trace of one connection that triggers this message?

Bernhard

On Feb 7, 2014, at 11:04 AM, Tim Ray <tray at 21ct.com> wrote:

> Getting lots of this in dpd:
> unexpected Handshake message SERVER HELLO from responder in state INITIAL
> 
> Looks like in the SSL analyzer. By far the bulk of the messages we?re seeing. Anyone seen this and tuned it? Or is it indicative of a serious misconfiguration?
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro




From jlanders at paymetric.com  Fri Feb  7 12:56:55 2014
From: jlanders at paymetric.com (John Landers)
Date: Fri, 7 Feb 2014 14:56:55 -0600
Subject: [Bro] Question about tuning
In-Reply-To: <666EA811-9EFA-4B53-AE26-377C1BBE81D3@icsi.berkeley.edu>
References: <CF1A8A7A.A580%tray@21ct.com>
	<666EA811-9EFA-4B53-AE26-377C1BBE81D3@icsi.berkeley.edu>
Message-ID: <199F5CD38D5E984990F92D2CDE955F664C65E15474@34093-MBX-C12.mex07a.mlsrvr.com>

This happens a lot in my environment as well. From some research I've done in the past, it's largely an issue of timing where a client does initiate the conversation but the server waits too long (for a variety of reasons) and the connection attempt was already reset.  

I generally ignore it as network garbage and I, too, would be interested in tuning this out of Bro.

John Landers

-----Original Message-----
From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Bernhard Amann
Sent: Friday, February 7, 2014 2:24 PM
To: Tim Ray
Cc: Bro
Subject: Re: [Bro] Question about tuning

Hello Tim,

without actually looking into the analyzer source - if I am not mistaken what the message is saying is that bro saw a server hello message being sent without the client hello being sent first (which is required by the protocol). 

I have not seen heard of this happening anywhere consistently, and cannot really see how that usually should happen on a regular basis. Would it perhaps be possible to get a trace of one connection that triggers this message?

Bernhard

On Feb 7, 2014, at 11:04 AM, Tim Ray <tray at 21ct.com> wrote:

> Getting lots of this in dpd:
> unexpected Handshake message SERVER HELLO from responder in state 
> INITIAL
> 
> Looks like in the SSL analyzer. By far the bulk of the messages we're seeing. Anyone seen this and tuned it? Or is it indicative of a serious misconfiguration?
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From gary at doit.wisc.edu  Mon Feb 10 09:50:53 2014
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Mon, 10 Feb 2014 11:50:53 -0600
Subject: [Bro] Sanity check - Grabbing platform tokens from browser user
 agents (was p0f)
In-Reply-To: <52EC7E85.9020401@doit.wisc.edu>
References: <52E99052.3060807@doit.wisc.edu>
	<20140201001242.GE8640@datacomm.albany.edu>
	<52EC7E85.9020401@doit.wisc.edu>
Message-ID: <52F9117D.4070609@doit.wisc.edu>

After running various iterations of the original script against several 
pcaps of our local traffic (and a couple days of live traffic) I ended 
up finding a lot of user agents that would match against the 
desktop/server OS rules, but were not necessarily desktops or servers. I 
ended up adding to the matching rules in part to parse out these things 
and also to detect other things we were interested in. Checking for more 
things seems to incur a performance penalty, so I also made some effort 
to move some of the more common matches sooner in the if/else statements 
to avoid having to check all of the less likely items first. The 
create_expire statement still doesn't behave as I expected, as each 
match is logged once per log rotation as opposed to once per day, but 
the matching seems to work with the exception that it doesn't check for 
every possible user agent case. I may also be missing explicitly 
including scripts that are already commonly loaded.

======================== Begin Script ========================
@load base/utils/site

module BrowserPlatform;

export
{
     # The fully resolved name for this log will be BrowserPlatform::LOG
     redef enum Log::ID += { LOG };

     type Info: record {
         ts:                 time    &log &optional;
         uid:                string  &log &optional;
         host:               addr    &log &optional;
         platform_token:     string  &log &optional;
         unparsed_version:   string  &log &optional;
     };

     # A set of seen IP + OS combinations. Used to prevent logging the 
same combo repeatedly.
     global seen_browser_platforms: set[string] &create_expire = 1.0 day 
&synchronized &redef;
}

event bro_init() &priority=5
     {
     Log::create_stream(BrowserPlatform::LOG,[$columns=Info]);
     }

event http_header(c: connection, is_orig: bool, name: string, value: string)
{
     local platform = "Unknown OS";
     if (!is_orig || name != "USER-AGENT" || 
!Site::is_local_addr(c$id$orig_h))
         return;

# Parse out Apple IOS and Android variants first as some apps will 
dispay as compatible with a desktop OS version

     if ( /iPhone/ in value )
     platform = "iPhone";
     else if ( /iPad/ in value )
         platform = "iPad";
     else if ( /iPod/ in value )
         platform = "iPod";
     else if ( /Android/ in value )
         platform = "Android";

# Once we've parsed out mobiles move onto desktop/server OS
# User agents listed in order of expected use or to pre-parse 
user-agents that might otherwise match multiple rules.

     else if ( /Windows/ in value )
         {
     if ( /Xbox/ in value ) # often includes a Windows OS version or 
identifies as a Mobile browser
         platform = "Xbox";
         else if ( /Phone/ in value || /Mobile/ in value ) # often 
includes Windows OS version
             platform = "Windows Phone";
         else if ( /Windows NT 6.1/ in value )
              platform = "Windows 7";
         else if ( /Windows NT 5.1/ in value )
              platform = "Windows XP";
         else if ( /Windows NT 5.2/ in value && /WOW64/ in value )
              platform = "Windows XP x64";
         else if ( /Windows NT 6.0/ in value )
              platform = "Windows Vista";
         else if ( /Windows NT 6.2/ in value )
              platform = "Windows 8";
         else if ( /Windows NT 6.3/ in value )
              platform = "Windows 8.1";
        else if ( /Windows 95/ in value )
              platform = "Windows 95";
         else if ( /Windows 98/ in value && /4.90/ !in value )
              platform = "Windows 98";
         else if ( /Win 9x 4.90/ in value )
              platform = "Windows Me";
         else if ( /Windows NT 4.0/ in value )
              platform = "Windows NT 4.0";
         else if ( /Windows NT 5.0/ in value || /Windows 2000/ in value )
              platform = "Windows 2000";
#    Catch-all for identifying less common user-agents. Can be noisy.
#       else
#            platform = "Windows Other";
         }
     else if ( /Mac OS X/ in value )
         {
     if ( /Mac OS X 10_9/ in value || /Mac OS X 10.9/ in value )
             platform = "Mac OS X 10.9";
         else if ( /Mac OS X 10_8/ in value || /Mac OS X 10.8/ in value )
             platform = "Mac OS X 10.8";
         else if ( /Mac OS X 10_7/ in value || /Mac OS X 10.7/ in value )
             platform = "Mac OS X 10.7";
         else if ( /Mac OS X 10_6/ in value || /Mac OS X 10.6/ in value )
             platform = "Mac OS X 10.6";
         else if ( /Mac OS X 10_5/ in value || /Mac OS X 10.5/ in value )
             platform = "Mac OS X 10.5";
         else if ( /Mac OS X 10_4/ in value || /Mac OS X 10.4/ in value )
             platform = "Mac OS X 10.4";
#       Catch-all for identifying less common user-agents. Can be noisy.
#       else
#           platform = "Mac OS X Other";
         }
     else if ( /Linux/ in value )
         platform = "Linux";

# Check to see if IP+OS combo already logged and if not log it and add 
it to the list of tracked combos.

     local saw = cat(c$id$orig_h,platform); #There is probably a less 
ugly way to do this than cat, but it seems to work
     if ( platform != "Unknown OS" && saw !in seen_browser_platforms )
         {
         local rec: BrowserPlatform::Info = [$ts=network_time(), 
$uid=c$uid, $host=c$id$orig_h, $platform_token=platform, 
$unparsed_version=value];
         Log::write(BrowserPlatform::LOG, rec);
         add seen_browser_platforms[saw];
         }
}

======================== End Script ========================
On 1/31/2014 10:56 PM, Gary Faulkner wrote:
> Thanks for the suggestions, that cleans that bit up quite nicely. I
> actually started by trying to deconstruct the various software.bro
> scripts and work my way backwards through the framework to see what was
> doing what. I'm still trying to navigate my way through that code, but I
> agree that it would make more sense to leverage it directly than create
> a derivative just to pull out a specific bit of the data. I'm not
> currently running Splunk in any production sense, but that is pretty
> much what I'm trying to do in Bro. Thanks for sharing it!
>
> Regards,
> Gary
>
> On 1/31/2014 6:12 PM, Justin Azoff wrote:
>> On Wed, Jan 29, 2014 at 05:35:46PM -0600, Gary Faulkner wrote:
>>> event http_header(c: connection, is_orig: bool, name: string, value: string)
>>> {
>>>       local platform = "Unknown OS";	
>>>       if ( is_orig )
>>>           {
>>> 	if ( name == "USER-AGENT" && /Windows NT 5.1/ in value )
>>> 		{
>>> 		platform = "Windows XP";
>>> 		}
>>>           else if ( name == "USER-AGENT" && /Windows NT 6.0/ in value )
>>>                   {
>>> 		platform = "Windows Vista";
>>>                   }
>>>           else if ( name == "USER-AGENT" && /Windows NT 6.1/ in value )
>>>                   {
>>>                   platform = "Windows 7";
>>>                   }
>> ..
>>
>> Modifying the http_header event handler as follows will increase performance:
>>
>> event http_header(c: connection, is_orig: bool, name: string, value: string)
>> {
>>       if(!is_orig || name != "USER-AGENT")
>>           return;
>>       if(/Windows NT 5.1/ in value)
>>           platform = "Windows XP";
>>       else if ...
>>
>> FWIW, I used to do this kind of thing outside of bro using splunk:
>>
>> https://github.com/JustinAzoff/splunk-scripts/blob/master/ua2os.py
>>
>> One thing you may want to do is rather than use the http_header event
>> use
>>
>> event log_software(rec: Info)
>> {
>>       ...
>> }
>>
>> which will be raised every time a new software version is seen.  The
>> software framework is already pulling most of the info out that you
>> might need, so you can piggy back on the work that it is doing.
>>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6257 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140210/f1da0222/attachment.bin 

From antonio.nappa at imdea.org  Tue Feb 11 10:15:10 2014
From: antonio.nappa at imdea.org (Antonio Nappa)
Date: Tue, 11 Feb 2014 19:15:10 +0100
Subject: [Bro] Filename in bro script
Message-ID: <CAOJuYzViG4FXx5njs2ddeOapFmDAOABi0L-144MPBkyJnxTRqA@mail.gmail.com>

Hi guys,

just a quick question, when I call bro with: bro myscript.bro -r file.pcap
is there a way to have the file.pcap string passed as a variable or
parameter inside the script?


Thank you


Antonio Nappa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140211/0c0e0a30/attachment.html 

From shaleta.bennett at gmail.com  Tue Feb 11 10:39:41 2014
From: shaleta.bennett at gmail.com (Shaleta Bennett)
Date: Tue, 11 Feb 2014 13:39:41 -0500
Subject: [Bro] How to detect if a DNS request occurred without corresponding
	HTTP request
Message-ID: <CAOQCEU_6_d5WVQo-ypU08bczJ3883rE8KLYcafkbPxtqcxGLhQ@mail.gmail.com>

Hello,

I am trying to write a script to figure out how to detect if a DNS request
occurred without a corresponding HTTP request? Can you explain to me how
this can be done?

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140211/dafc892a/attachment.html 

From anthony.kasza at gmail.com  Tue Feb 11 15:12:34 2014
From: anthony.kasza at gmail.com (anthony kasza)
Date: Tue, 11 Feb 2014 15:12:34 -0800
Subject: [Bro] Filename in bro script
In-Reply-To: <CAOJuYzViG4FXx5njs2ddeOapFmDAOABi0L-144MPBkyJnxTRqA@mail.gmail.com>
References: <CAOJuYzViG4FXx5njs2ddeOapFmDAOABi0L-144MPBkyJnxTRqA@mail.gmail.com>
Message-ID: <CAEZw2byxJQHD=GxsT726FPpSLOVbOLtw2DdApr1Ei9Ln_1=1mg@mail.gmail.com>

I've looked for this feature as well and have not been able to locate it.
It would be nice to have a @PKTSOURCE variable similar to the @FILENAME and
@DIR variables.
On Feb 11, 2014 10:19 AM, "Antonio Nappa" <antonio.nappa at imdea.org> wrote:

> Hi guys,
>
> just a quick question, when I call bro with: bro myscript.bro -r file.pcap
> is there a way to have the file.pcap string passed as a variable or
> parameter inside the script?
>
>
> Thank you
>
>
> Antonio Nappa
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140211/df70ff29/attachment.html 

From robin at icir.org  Tue Feb 11 16:32:16 2014
From: robin at icir.org (Robin Sommer)
Date: Tue, 11 Feb 2014 16:32:16 -0800
Subject: [Bro] Filename in bro script
In-Reply-To: <CAEZw2byxJQHD=GxsT726FPpSLOVbOLtw2DdApr1Ei9Ln_1=1mg@mail.gmail.com>
References: <CAOJuYzViG4FXx5njs2ddeOapFmDAOABi0L-144MPBkyJnxTRqA@mail.gmail.com>
	<CAEZw2byxJQHD=GxsT726FPpSLOVbOLtw2DdApr1Ei9Ln_1=1mg@mail.gmail.com>
Message-ID: <20140212003216.GH55513@icir.org>

Yeah, that information is not currently available. Can you guys say a
bit more about the use case(s) here?

I'm asking because we could either make it available directly (that
should be pretty straight-forward) or cover it implicitly by exposing
all command line parameters to script-land, along with some library
functionality to parse them easily (which Seth is rooting for :-)

In any case, please file a feature request ticket.

Robin

On Tue, Feb 11, 2014 at 15:12 -0800, anthony kasza wrote:

> I've looked for this feature as well and have not been able to locate it.
> It would be nice to have a @PKTSOURCE variable similar to the @FILENAME and
> @DIR variables.
> On Feb 11, 2014 10:19 AM, "Antonio Nappa" <antonio.nappa at imdea.org> wrote:
> 
> > Hi guys,
> >
> > just a quick question, when I call bro with: bro myscript.bro -r file.pcap
> > is there a way to have the file.pcap string passed as a variable or
> > parameter inside the script?
> >
> >
> > Thank you
> >
> >
> > Antonio Nappa
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


-- 
Robin Sommer * Phone +1 (510) 722-6541 *     robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 * www.icir.org/robin


From shaleta.bennett at gmail.com  Tue Feb 11 17:31:46 2014
From: shaleta.bennett at gmail.com (Shaleta Bennett)
Date: Tue, 11 Feb 2014 20:31:46 -0500
Subject: [Bro] BRO: DNS TTL
Message-ID: <CAOQCEU_v-9uP6RyrHz4v8HhH5sPuOZacfgbEtFnDwuPxHGdHYA@mail.gmail.com>

Hi, I am trying to detect if a dns request is made before its TTL has
expired. For example, if I make a request to www.example.com and I
immediately make another request towww.example.com before the TTL is up, I
would like to see a notice for this. The code below compiles without errors
but I am not getting any notice for the example explained above. I think
there may be an issue with the TTL vector. I would like to store it in a
vector the same way I did for dnsTime and dnsQuery. However, TTLs is
already a vector of interval. Do you have any suggestions after viewing the
code below? Thanks.


redef enum Notice:: Type+= {DetectDNSTTL}

global dnsTime: time;
global dnsQuery: string;
global dsnTTL: vector of interval;

global dnsTimeVector: vector of time;
global dnsQueryVector: vector of string;
global dnsTTLVector: vector of interval;

event dns_request (c:connection, msg: dns_msg, query: string, qtype: count,
qclass: count)
{

dnsTime = c$dns$ts;
dnsQuery = c$dns$query;
dnsTTL = c$dns$TTLs;

dnsTimeVector = vector(dnsTime);
dnsQueryVector = vector(dnsQuery);

#save vector TTLs in dnsTTLVector
for (j in dnsTTL)
{
      dnsTTLVector = vector(dnsTTL[j]);
}

#check if query is already in vector
for (i in dnsQueryVector)
{
   if (dnsQuery == dnsQueryVector[i])
   {
       #Calculate the TTL expiration by adding the dns request TTL and time
        local ttlExpiration = dnsTTLVector[i] + dnsTimeVector[i];

         #Send a notice if dns request time is less than TTL expiration time
          if (dnsTime <= ttlExpiration )
           {
                 NOTICE([$note = DetectDNSTTL,
                 $msg = "DNS Request occurred before TTL expired",
                 $conn = c] );
           }
     }
  }

}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140211/714ce550/attachment.html 

From gary at doit.wisc.edu  Tue Feb 11 18:12:37 2014
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Tue, 11 Feb 2014 20:12:37 -0600
Subject: [Bro] BRO: DNS TTL
In-Reply-To: <CAOQCEU_v-9uP6RyrHz4v8HhH5sPuOZacfgbEtFnDwuPxHGdHYA@mail.gmail.com>
References: <CAOQCEU_v-9uP6RyrHz4v8HhH5sPuOZacfgbEtFnDwuPxHGdHYA@mail.gmail.com>
Message-ID: <52FAD895.6000406@doit.wisc.edu>

I haven't run the script, but on first glance there appears to be a typo 
in the below line (dsn instead of dns):

On 2/11/2014 7:31 PM, Shaleta Bennett wrote:
> global dsnTTL: vector of interval;

Regards,
Gary


From shaleta.bennett at gmail.com  Tue Feb 11 18:24:06 2014
From: shaleta.bennett at gmail.com (Shaleta Bennett)
Date: Tue, 11 Feb 2014 21:24:06 -0500
Subject: [Bro] BRO: DNS TTL
In-Reply-To: <52FAD895.6000406@doit.wisc.edu>
References: <CAOQCEU_v-9uP6RyrHz4v8HhH5sPuOZacfgbEtFnDwuPxHGdHYA@mail.gmail.com>
	<52FAD895.6000406@doit.wisc.edu>
Message-ID: <CAOQCEU83afWE9T+_C7myNrrLeXDr7-1T0QizmePKD4FfNaxxdA@mail.gmail.com>

Thanks. I fixed the typo. I'm still not getting anything in notice.log.


On Tue, Feb 11, 2014 at 9:12 PM, Gary Faulkner <gary at doit.wisc.edu> wrote:

> I haven't run the script, but on first glance there appears to be a typo
> in the below line (dsn instead of dns):
>
>
> On 2/11/2014 7:31 PM, Shaleta Bennett wrote:
>
>> global dsnTTL: vector of interval;
>>
>
> Regards,
> Gary
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140211/3c6073f8/attachment.html 

From anthony.kasza at gmail.com  Tue Feb 11 20:06:45 2014
From: anthony.kasza at gmail.com (anthony kasza)
Date: Tue, 11 Feb 2014 20:06:45 -0800
Subject: [Bro] Filename in bro script
In-Reply-To: <20140212003216.GH55513@icir.org>
References: <CAOJuYzViG4FXx5njs2ddeOapFmDAOABi0L-144MPBkyJnxTRqA@mail.gmail.com>
	<CAEZw2byxJQHD=GxsT726FPpSLOVbOLtw2DdApr1Ei9Ln_1=1mg@mail.gmail.com>
	<20140212003216.GH55513@icir.org>
Message-ID: <CAEZw2bxeymNV2ybS1198Tr2+P5oyek1bBWdUnr==yOmr92JA8w@mail.gmail.com>

Submitted here <https://bro-tracker.atlassian.net/browse/BIT-1131>. Is
this the preferred method for submitting feature requests instead of
posting to this list?

-AK

On Tue, Feb 11, 2014 at 4:32 PM, Robin Sommer <robin at icir.org> wrote:
> Yeah, that information is not currently available. Can you guys say a
> bit more about the use case(s) here?
>
> I'm asking because we could either make it available directly (that
> should be pretty straight-forward) or cover it implicitly by exposing
> all command line parameters to script-land, along with some library
> functionality to parse them easily (which Seth is rooting for :-)
>
> In any case, please file a feature request ticket.
>
> Robin
>
> On Tue, Feb 11, 2014 at 15:12 -0800, anthony kasza wrote:
>
>> I've looked for this feature as well and have not been able to locate it.
>> It would be nice to have a @PKTSOURCE variable similar to the @FILENAME and
>> @DIR variables.
>> On Feb 11, 2014 10:19 AM, "Antonio Nappa" <antonio.nappa at imdea.org> wrote:
>>
>> > Hi guys,
>> >
>> > just a quick question, when I call bro with: bro myscript.bro -r file.pcap
>> > is there a way to have the file.pcap string passed as a variable or
>> > parameter inside the script?
>> >
>> >
>> > Thank you
>> >
>> >
>> > Antonio Nappa
>> >
>> > _______________________________________________
>> > Bro mailing list
>> > bro at bro-ids.org
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> >
>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> --
> Robin Sommer * Phone +1 (510) 722-6541 *     robin at icir.org
> ICSI/LBNL    * Fax   +1 (510) 666-2956 * www.icir.org/robin


From robin at icir.org  Tue Feb 11 20:17:27 2014
From: robin at icir.org (Robin Sommer)
Date: Tue, 11 Feb 2014 20:17:27 -0800
Subject: [Bro] Filename in bro script
In-Reply-To: <CAEZw2bxeymNV2ybS1198Tr2+P5oyek1bBWdUnr==yOmr92JA8w@mail.gmail.com>
References: <CAOJuYzViG4FXx5njs2ddeOapFmDAOABi0L-144MPBkyJnxTRqA@mail.gmail.com>
	<CAEZw2byxJQHD=GxsT726FPpSLOVbOLtw2DdApr1Ei9Ln_1=1mg@mail.gmail.com>
	<20140212003216.GH55513@icir.org>
	<CAEZw2bxeymNV2ybS1198Tr2+P5oyek1bBWdUnr==yOmr92JA8w@mail.gmail.com>
Message-ID: <20140212041727.GK55513@icir.org>



On Tue, Feb 11, 2014 at 20:06 -0800, anthony kasza wrote:

> Is this the preferred method for submitting feature requests instead
> of posting to this list?

Yes, for a specific feature that's the best way to make sure it gets
recorded. But it's fine to discuss on the list first if you are
looking for further input/ideas. 

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 *     robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 * www.icir.org/robin


From shaleta.bennett at gmail.com  Wed Feb 12 11:09:59 2014
From: shaleta.bennett at gmail.com (Shaleta Bennett)
Date: Wed, 12 Feb 2014 14:09:59 -0500
Subject: [Bro] Fwd: BRO: DNS TTL
In-Reply-To: <CAOQCEU_v-9uP6RyrHz4v8HhH5sPuOZacfgbEtFnDwuPxHGdHYA@mail.gmail.com>
References: <CAOQCEU_v-9uP6RyrHz4v8HhH5sPuOZacfgbEtFnDwuPxHGdHYA@mail.gmail.com>
Message-ID: <CAOQCEU9qAYKrKMOxkkn_nNaDuTPZg-ypZWAHk_nAteXhQDoSiA@mail.gmail.com>

Hi, I would like to detect if DNS Request for a host occurred before the
TTL expired for that particular host. I gave it try but it doesn't work. My
code is below. Can anyone help me with this or tell me what I am
doing wrong?

Thanks,
Shaleta

---------- Forwarded message ----------
From: Shaleta Bennett <shaleta.bennett at gmail.com>
Date: Tue, Feb 11, 2014 at 8:31 PM
Subject: BRO: DNS TTL
To: bro at bro.org


 Hi, I am trying to detect if a dns request is made before its TTL has
expired. For example, if I make a request to www.example.com and I
immediately make another request towww.example.com before the TTL is up, I
would like to see a notice for this. The code below compiles without errors
but I am not getting any notice for the example explained above. I think
there may be an issue with the TTL vector. I would like to store it in a
vector the same way I did for dnsTime and dnsQuery. However, TTLs is
already a vector of interval. Do you have any suggestions after viewing the
code below? Thanks.


redef enum Notice:: Type+= {DetectDNSTTL}

global dnsTime: time;
global dnsQuery: string;
global dsnTTL: vector of interval;

global dnsTimeVector: vector of time;
global dnsQueryVector: vector of string;
global dnsTTLVector: vector of interval;

event dns_request (c:connection, msg: dns_msg, query: string, qtype: count,
qclass: count)
{

dnsTime = c$dns$ts;
dnsQuery = c$dns$query;
dnsTTL = c$dns$TTLs;

dnsTimeVector = vector(dnsTime);
dnsQueryVector = vector(dnsQuery);

#save vector TTLs in dnsTTLVector
for (j in dnsTTL)
{
      dnsTTLVector = vector(dnsTTL[j]);
}

#check if query is already in vector
for (i in dnsQueryVector)
{
   if (dnsQuery == dnsQueryVector[i])
   {
       #Calculate the TTL expiration by adding the dns request TTL and time
        local ttlExpiration = dnsTTLVector[i] + dnsTimeVector[i];

         #Send a notice if dns request time is less than TTL expiration time
          if (dnsTime <= ttlExpiration )
           {
                 NOTICE([$note = DetectDNSTTL,
                 $msg = "DNS Request occurred before TTL expired",
                 $conn = c] );
           }
     }
  }

}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140212/aa0118af/attachment.html 

From christopher.p.crawford at gmail.com  Wed Feb 12 13:44:22 2014
From: christopher.p.crawford at gmail.com (Chris Crawford)
Date: Wed, 12 Feb 2014 16:44:22 -0500
Subject: [Bro] Additional Records in DNS
In-Reply-To: <CFD20297-E929-4504-989F-D2DFEB9CEF3B@icir.org>
References: <CAG1VgObKzCKtdF183-p+GkKrv9f=wiQaY7jZysAW5HCXrHU4ZQ@mail.gmail.com>
	<BACA0BD9-5409-4EF7-A6BF-F35602879491@icir.org>
	<CAG1VgOZfunFwBkA8pcbsLXJ+H-9+O2A_r-DmFGGzbW0+KDOK+A@mail.gmail.com>
	<CFD20297-E929-4504-989F-D2DFEB9CEF3B@icir.org>
Message-ID: <CAG1VgOaLprodutFzF49hc1O-BK=di22pbKJAd05o7S_UvNkjbw@mail.gmail.com>

I finally got a round to giving this a try on bro 2.2, but it looks like
dns_EDNS_addl is still unimplemented.

Nothing happens if I try the old script from earlier in this thread:
event dns_EDNS_addl(c: connection, msg: dns_msg, ans: dns_edns_additional){
        print c$uid;
}

To make things simpler, I tried this:
event dns_EDNS_addl(c: connection, msg: dns_msg, ans: dns_edns_additional){
        print "Additional";
}


Nothing happens with either script.

I would be happy to take a stab at implementing this myself.  I'm OK with
writing some C or some C++, but I'm not really familiar with how something
in Bro scriptland ultimately ends up calling code in the compiled part of
bro.

I ran a grep over the bro 2.2 code base for dns_EDNS_addl, and these files
seem to be the most promising candidates for where I'd need to add some of
my own code to get this function implemented:

src/analyzer/protocol/dns/events.bif
src/analyzer/protocol/dns/DNS.cc
build/src/analyzer/protocol/dns/events.bif.init.cc
build/src/analyzer/protocol/dns/events.bif.h
build/src/analyzer/protocol/dns/events.bif.cc


Am I on the right track?

If anybody has implemented one of these functions before, would you be
willing to share any tips on what your process was? (i.e. which files did
you have to modify, where did you put your logic, did you use any special
compiler flags to recompile your changes, etc)

Also - does anybody have any pointers on how data flows between the
compiled part of bro and scriptland?

-Chris


On Fri, Jul 12, 2013 at 11:09 AM, Seth Hall <seth at icir.org> wrote:

>
> On Jul 12, 2013, at 11:05 AM, Chris Crawford <
> christopher.p.crawford at gmail.com> wrote:
>
> > Did the core analyzer part get implemented in an update on git after v
> 2.1?
>
> I don't know.  I thought those events were implemented, but perhaps not.
>  It might be worthwhile looking through the analyzer itself to see if they
> are.
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140212/6a4394b2/attachment.html 

From shaleta.bennett at gmail.com  Thu Feb 13 08:21:24 2014
From: shaleta.bennett at gmail.com (Shaleta Bennett)
Date: Thu, 13 Feb 2014 11:21:24 -0500
Subject: [Bro] Is DNS Query equal to HTTP Host?
Message-ID: <CAOQCEU_hkzyOyx2Zh2uBSeJMW9LAWXw-Q5VXu7y-XSTtx7Syvw@mail.gmail.com>

Hi can anyone help me figure out if the dns query is the same as the http
host?

I've tried doing the following but did not get any output.

if(c$dns$query == c$http$host)
{

    #send notice to notice.log
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140213/ac71345e/attachment.html 

From anthony.kasza at gmail.com  Thu Feb 13 08:55:34 2014
From: anthony.kasza at gmail.com (anthony kasza)
Date: Thu, 13 Feb 2014 08:55:34 -0800
Subject: [Bro] Is DNS Query equal to HTTP Host?
In-Reply-To: <CAOQCEU_hkzyOyx2Zh2uBSeJMW9LAWXw-Q5VXu7y-XSTtx7Syvw@mail.gmail.com>
References: <CAOQCEU_hkzyOyx2Zh2uBSeJMW9LAWXw-Q5VXu7y-XSTtx7Syvw@mail.gmail.com>
Message-ID: <CAEZw2bxZjPace6QKYk=VOUbpPbDuN4674iDEedVtF3=7iLc-xA@mail.gmail.com>

A connection object is created for a DNS query and a DNS response.
Subsequent connections made utilizing the results of that DNS query have
their own connection objects. You'll have to keep a DNS cache in userland
and watch for connections to the resolved IP address with HTTP host fields
differing from the domain that was resolved in the cache.

-AK
On Feb 13, 2014 8:28 AM, "Shaleta Bennett" <shaleta.bennett at gmail.com>
wrote:

> Hi can anyone help me figure out if the dns query is the same as the http
> host?
>
> I've tried doing the following but did not get any output.
>
> if(c$dns$query == c$http$host)
> {
>
>     #send notice to notice.log
> }
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140213/d430ce82/attachment.html 

From jsiwek at illinois.edu  Thu Feb 13 13:45:54 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Thu, 13 Feb 2014 21:45:54 +0000
Subject: [Bro] Additional Records in DNS
In-Reply-To: <CAG1VgOaLprodutFzF49hc1O-BK=di22pbKJAd05o7S_UvNkjbw@mail.gmail.com>
References: <CAG1VgObKzCKtdF183-p+GkKrv9f=wiQaY7jZysAW5HCXrHU4ZQ@mail.gmail.com>
	<BACA0BD9-5409-4EF7-A6BF-F35602879491@icir.org>
	<CAG1VgOZfunFwBkA8pcbsLXJ+H-9+O2A_r-DmFGGzbW0+KDOK+A@mail.gmail.com>
	<CFD20297-E929-4504-989F-D2DFEB9CEF3B@icir.org>
	<CAG1VgOaLprodutFzF49hc1O-BK=di22pbKJAd05o7S_UvNkjbw@mail.gmail.com>
Message-ID: <112786D8-8E6A-49DE-908A-25EC787083E5@illinois.edu>


On Feb 12, 2014, at 3:44 PM, Chris Crawford <christopher.p.crawford at gmail.com> wrote:

> I finally got a round to giving this a try on bro 2.2, but it looks like dns_EDNS_addl is still unimplemented.

It?s not integrated in the default DNS script, but the DNS parser does seem like it can generate that event.

> Am I on the right track?

It?s not clear from your original email if you actually need EDNS support (a particular type of resource record) or just to get the stuff from the Authority and Additional sections of a DNS reply?  

If it?s the later, looking at scripts/policy/protocols/dns/auth-addl.bro may help (if not already do exactly what you want).  You?ll see the trick about that script are the redefs of ?dns_skip_all_auth? and ?dns_skip_all_addl? ? by default Bro will skip parsing Authority/Additional sections (for ?performance reasons? I suppose) unless explicitly told not to.

- Jon



From engineer.demo2020 at gmail.com  Thu Feb 13 21:57:19 2014
From: engineer.demo2020 at gmail.com (Mr Smith)
Date: Fri, 14 Feb 2014 09:27:19 +0330
Subject: [Bro] Bro Anomaly Detection
Message-ID: <CAEL_4HJ8dAzxNrQ0+1n0NTYYCU99Kef97xq_Ez=JvhJSvdh_Yw@mail.gmail.com>

 Hi, I have two questions regarding the Bro anomaly detection capability.
1.How does the Bro detect anomalies? Using writing rules(anomaly rules) or
using a separate module ?
2.Is it possible to run the signature-based and anomaly-based parts of Bro
separately?
I mean, can the Bro be used only for the detection of anomalies.If it is
possible, how?
Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140214/9f59d175/attachment.html 

From engineer.demo2020 at gmail.com  Thu Feb 13 22:02:31 2014
From: engineer.demo2020 at gmail.com (Mr Smith)
Date: Fri, 14 Feb 2014 09:32:31 +0330
Subject: [Bro] Fwd: Bro Anomaly Detection
In-Reply-To: <CAEL_4HJ8dAzxNrQ0+1n0NTYYCU99Kef97xq_Ez=JvhJSvdh_Yw@mail.gmail.com>
References: <CAEL_4HJ8dAzxNrQ0+1n0NTYYCU99Kef97xq_Ez=JvhJSvdh_Yw@mail.gmail.com>
Message-ID: <CAEL_4H+UqF1MjCT1=_7LscRsJDVkcnFSx7VawvD76vy8VZZVUw@mail.gmail.com>

Hi, I have two questions regarding the Bro anomaly detection capability.
1.How does the Bro detect anomalies? Using writing rules(anomaly rules) or
using a separate module ?
2.Is it possible to run the signature-based and anomaly-based parts of Bro
separately?
I mean, can the Bro be used only for the detection of anomalies.If it is
possible, how?
Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140214/ad497ea8/attachment.html 

From anthony.kasza at gmail.com  Thu Feb 13 22:21:01 2014
From: anthony.kasza at gmail.com (anthony kasza)
Date: Thu, 13 Feb 2014 22:21:01 -0800
Subject: [Bro] Fwd: Bro Anomaly Detection
In-Reply-To: <CAEL_4H+UqF1MjCT1=_7LscRsJDVkcnFSx7VawvD76vy8VZZVUw@mail.gmail.com>
References: <CAEL_4HJ8dAzxNrQ0+1n0NTYYCU99Kef97xq_Ez=JvhJSvdh_Yw@mail.gmail.com>
	<CAEL_4H+UqF1MjCT1=_7LscRsJDVkcnFSx7VawvD76vy8VZZVUw@mail.gmail.com>
Message-ID: <CAEZw2byMPG3+vMM23UkOS1cZX5necmoptqaDTtwxeK_iAa4p8A@mail.gmail.com>

Technically, many of Bro's protocol identification capabilities (use by
Bro's anomaly detection capabilities) utilize Bro's signature framework.
On Feb 13, 2014 10:04 PM, "Mr Smith" <engineer.demo2020 at gmail.com> wrote:

>
>
>
> Hi, I have two questions regarding the Bro anomaly detection capability.
> 1.How does the Bro detect anomalies? Using writing rules(anomaly rules) or
> using a separate module ?
> 2.Is it possible to run the signature-based and anomaly-based parts of Bro
> separately?
> I mean, can the Bro be used only for the detection of anomalies.If it is
> possible, how?
> Thanks
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140213/014d72b9/attachment.html 

From shaleta.bennett at gmail.com  Fri Feb 14 08:19:32 2014
From: shaleta.bennett at gmail.com (Shaleta Bennett)
Date: Fri, 14 Feb 2014 11:19:32 -0500
Subject: [Bro] &write_expire and &expire_Func
Message-ID: <CAOQCEU9vouKhkjAFM7zBpMRhRKro_-h0EbUHH+-DEJ4aXULGPw@mail.gmail.com>

I'm very new to Bro as this is one of my first scripts. Can anyone explain
to me how the &write_expire and &expire_func work on a table? I would like
to change a value in a table when an event occurs and pass the table into a
function.

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140214/b3e5d490/attachment.html 

From christopher.p.crawford at gmail.com  Fri Feb 14 10:07:03 2014
From: christopher.p.crawford at gmail.com (Chris Crawford)
Date: Fri, 14 Feb 2014 13:07:03 -0500
Subject: [Bro] Additional Records in DNS
In-Reply-To: <112786D8-8E6A-49DE-908A-25EC787083E5@illinois.edu>
References: <CAG1VgObKzCKtdF183-p+GkKrv9f=wiQaY7jZysAW5HCXrHU4ZQ@mail.gmail.com>
	<BACA0BD9-5409-4EF7-A6BF-F35602879491@icir.org>
	<CAG1VgOZfunFwBkA8pcbsLXJ+H-9+O2A_r-DmFGGzbW0+KDOK+A@mail.gmail.com>
	<CFD20297-E929-4504-989F-D2DFEB9CEF3B@icir.org>
	<CAG1VgOaLprodutFzF49hc1O-BK=di22pbKJAd05o7S_UvNkjbw@mail.gmail.com>
	<112786D8-8E6A-49DE-908A-25EC787083E5@illinois.edu>
Message-ID: <CAG1VgObapeVFUege0pOYYeS6Rt1kd76kvPM10xF8AFMBV7bEFg@mail.gmail.com>

scripts/policy/protocols/dns/auth-addl.bro is exactly what I was looking
for.  Thanks!


On Thu, Feb 13, 2014 at 4:45 PM, Siwek, Jonathan Luke
<jsiwek at illinois.edu>wrote:

>
> On Feb 12, 2014, at 3:44 PM, Chris Crawford <
> christopher.p.crawford at gmail.com> wrote:
>
> > I finally got a round to giving this a try on bro 2.2, but it looks like
> dns_EDNS_addl is still unimplemented.
>
> It's not integrated in the default DNS script, but the DNS parser does
> seem like it can generate that event.
>
> > Am I on the right track?
>
> It's not clear from your original email if you actually need EDNS support
> (a particular type of resource record) or just to get the stuff from the
> Authority and Additional sections of a DNS reply?
>
> If it's the later, looking at scripts/policy/protocols/dns/auth-addl.bro
> may help (if not already do exactly what you want).  You'll see the trick
> about that script are the redefs of "dns_skip_all_auth" and
> "dns_skip_all_addl" -- by default Bro will skip parsing Authority/Additional
> sections (for "performance reasons" I suppose) unless explicitly told not
> to.
>
> - Jon
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140214/f3ab46e4/attachment.html 

From netanelmaman0 at gmail.com  Sun Feb 16 08:43:12 2014
From: netanelmaman0 at gmail.com (=?UTF-8?B?16DXqteg15DXnCDXntee158=?=)
Date: Sun, 16 Feb 2014 18:43:12 +0200
Subject: [Bro] Dump reassembled packets
Message-ID: <CA+ttJBL4-N_1qMEYzPx_yaMfB9yRK7ZuoW7VJHs=qUhOjBA-6g@mail.gmail.com>

Hey,

First, sorry about my english.

Im try to dump reassembled http request with "set_record_packets" when i
see intresting thing in my bro rules.

The problem is that this option dump only the *last* truncated packet and
the rest of connection.

Can i get previous truncated packets of known connection?

I tried a few hours but don't understand how to.

Thanks,

Net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140216/ff5a365c/attachment.html 

From engineer.demo2020 at gmail.com  Tue Feb 18 05:10:20 2014
From: engineer.demo2020 at gmail.com (Mr Smith)
Date: Tue, 18 Feb 2014 16:40:20 +0330
Subject: [Bro] Bro Anomaly Detection
Message-ID: <CAEL_4HL3rUwRxrMnR0iv5Z=w-QMzOLar-QnHpTj88-KXXRkGbw@mail.gmail.com>

Hi, I have two questions regarding the Bro anomaly detection capability.
1.How does the Bro detect anomalies? Using writing rules(anomaly rules) or
using a separate module ?
2.Is it possible to run the signature-based and anomaly-based parts of Bro
separately?
I mean, can the Bro be used only for the detection of anomalies.If it is
possible, how?
Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140218/29705ee3/attachment.html 

From jbabio at po-box.esu.edu  Tue Feb 18 05:32:34 2014
From: jbabio at po-box.esu.edu (John Babio)
Date: Tue, 18 Feb 2014 13:32:34 +0000
Subject: [Bro] intel or notice on client software signed with a cert
Message-ID: <CF28CB29.E070%jbabio@po-box.esu.edu>

Can you utilize the intel framework for this type of alerting? I want to alert on client software signed with a certificate containing a particular name or serial.




From seth at icir.org  Tue Feb 18 06:42:09 2014
From: seth at icir.org (Seth Hall)
Date: Tue, 18 Feb 2014 09:42:09 -0500
Subject: [Bro] intel or notice on client software signed with a cert
In-Reply-To: <CF28CB29.E070%jbabio@po-box.esu.edu>
References: <CF28CB29.E070%jbabio@po-box.esu.edu>
Message-ID: <E0A887D5-0F77-4A2E-BD44-E24480DBC65F@icir.org>


On Feb 18, 2014, at 8:32 AM, John Babio <jbabio at po-box.esu.edu> wrote:

> Can you utilize the intel framework for this type of alerting? I want to alert on client software signed with a certificate containing a particular name or serial.


We aren't quite at the point yet where certificates are parsed out of executables.  We are working in that direction though and it *should* be possible in the future do exactly this.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140218/43370d96/attachment.bin 

From slagell at illinois.edu  Tue Feb 18 07:49:03 2014
From: slagell at illinois.edu (Slagell, Adam J)
Date: Tue, 18 Feb 2014 15:49:03 +0000
Subject: [Bro] Bro Anomaly Detection
In-Reply-To: <CAEL_4HL3rUwRxrMnR0iv5Z=w-QMzOLar-QnHpTj88-KXXRkGbw@mail.gmail.com>
References: <CAEL_4HL3rUwRxrMnR0iv5Z=w-QMzOLar-QnHpTj88-KXXRkGbw@mail.gmail.com>
Message-ID: <2D59C682-0DBB-4C98-A398-15427C7F08DE@illinois.edu>

Bro doesn't fit well into either the anomaly-based or signature based paradigm and is often referred to as a specification-based IDS. However, it is probably best understood as more than an IDS, as a network analysis framework that combines a powerful state engine with a full computer language aimed at network analysis.

So to answer your question, there are not separate "modules". There are a set of scripts [1] that come with Bro, and the ability to customize and add to these. If you are interested in doing signature-based detection, look at [2].

I hope this helps to get you started.

:Adam Slagell


[1] http://www.bro.org/sphinx/scripts/index.html
[2] http://www.bro.org/sphinx/frameworks/signatures.html

On Feb 18, 2014, at 7:10 AM, Mr Smith <engineer.demo2020 at gmail.com<mailto:engineer.demo2020 at gmail.com>> wrote:

Hi, I have two questions regarding the Bro anomaly detection capability.
1.How does the Bro detect anomalies? Using writing rules(anomaly rules) or using a separate module ?
2.Is it possible to run the signature-based and anomaly-based parts of Bro separately?
I mean, can the Bro be used only for the detection of anomalies.If it is possible, how?
Thanks
_______________________________________________
Bro mailing list
bro at bro-ids.org<mailto:bro at bro-ids.org>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

------

Adam J. Slagell
Chief Information Security Officer
Assistant Director, Cybersecurity
National Center for Supercomputing ApplicationsUniversity of Illinois at Urbana-Champaign
www.ncsa.illinois.edu/~slagell/<http://www.ncsa.illinois.edu/~slagell/>

"Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure."

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140218/8b488421/attachment.html 

From benson.mathews at gmail.com  Tue Feb 18 09:50:17 2014
From: benson.mathews at gmail.com (Benson Mathews)
Date: Tue, 18 Feb 2014 12:50:17 -0500
Subject: [Bro] Bro and flood protection
In-Reply-To: <20140129232901.GD92282@icir.org>
References: <CAND-VUchaa-+COMczbtCj+Qhgw7qQqFunqtthSDhvGgTATCneg@mail.gmail.com>
	<20140129232901.GD92282@icir.org>
Message-ID: <CAND-VUd_09xnX7_Vu=yD46XqMsNVuqnziBYv4A9d6xS=x_rd1g@mail.gmail.com>

Hi Robin,

Thanks for the response. I've modified the script slightly and commented
out the install and uninstall addr_filters. See attached.

I tried running the script against a sample pcap that I generated using
hping3 (hping3 --rand-source 192.168.146.130 --flood -S -L 0 -p 80) and it
seemed to be logging the SYN attack alerts in the notice logs, atleast the
start of the attack:
1392741015.684576       -       -       -       -       -       -
-       -       -       SynFloodStart   Start of syn-flood against
192.168.146.130; sampling packets now        -       192.168.146.130
-       -       -       bro Notice::ACTION_LOG       3600.000000
F       -       -       -       -       -
1392741015.740914       -       -       -       -       -       -
-       -       -       SynFloodStart   Start of syn-flood against
192.168.146.130; sampling packets now        -       192.168.146.130
-       -       -       bro Notice::ACTION_LOG       3600.000000
F       -       -       -       -       -
1392741015.797104       -       -       -       -       -       -
-       -       -       SynFloodStart   Start of syn-flood against
192.168.146.130; sampling packets now        -       192.168.146.130
-       -       -       bro Notice::ACTION_LOG       3600.000000
F       -       -       -       -       -


But I also keep getting several runtime errors: *no such index
(current_victims[ip]) *on line 66 of the attached script. And I'm not sure
how to fix this. Any thoughts?


Thanks,
Benson



On Wed, Jan 29, 2014 at 6:29 PM, Robin Sommer <robin at icir.org> wrote:

>
>
> On Wed, Jan 29, 2014 at 14:14 -0500, you wrote:
>
> > synflood script that was provided on previous versions of Bro. Wondering
> if
> > there's something similar on Bro 2.2 or if the 1.5.x version would still
> > work?
>
> As far as I recall, the 1.5 script should still work.
>
> Robin
>
> --
> Robin Sommer * Phone +1 (510) 722-6541 *     robin at icir.org
> ICSI/LBNL    * Fax   +1 (510) 666-2956 * www.icir.org/robin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140218/aef04a88/attachment.html 
-------------- next part --------------
export {
	redef enum Notice::Type += {
	SynFloodStart,    # start of syn-flood against a certain victim
	SynFloodEnd,      # end of syn-flood against a certain victim
	SynFloodStatus,   # report of ongoing syn-flood
	};


# We report a syn-flood if more than SYNFLOOD_THRESHOLD new connections
# have been reported within the last SYNFLOOD_INTERVAL for a certain IP.
# (We sample the conns by one out of SYNFLOOD_SAMPLE_RATE, so the attempt
# counter is an estimated value.). If a victim is identified, we install a
# filter via install_dst_filter and sample the packets targeting it by
# SYNFLOOD_VICTIM_SAMPLE_RATE.
#
# Ongoing syn-floods are reported every SYNFLOOD_REPORT_INTERVAL.

global SYNFLOOD_THRESHOLD = 15000 &redef;
global SYNFLOOD_INTERVAL = 60 secs &redef;
global SYNFLOOD_REPORT_INTERVAL = 1 mins &redef;

# Sample connections by one out of x.
global SYNFLOOD_SAMPLE_RATE = 100 &redef;

# Sample packets to known victims with probability x.
global SYNFLOOD_VICTIM_SAMPLE_RATE = 0.01 &redef;

global conn_attempts: table[addr] of count &default = 0;
global victim_attempts: table[addr,addr] of count
	&default = 0  &read_expire = 5mins;

# We remember up to this many number of sources per victim.
global max_sources = 100;
global current_victims: table[addr] of set[addr] &read_expire = 60mins;
global accumulated_conn_attempts: table[addr] of count &default = 0;

global sample_count = 0;
global interval_start: time = 0;
}

# Using new_connection() can be quite expensive but connection_attempt() has
# a rather large lag that may lead to detecting flood too late. Additionally,
# it does not cover UDP/ICMP traffic.
event new_connection(c: connection)
	{
	if ( c$id$resp_h in current_victims )
		{
		++conn_attempts[c$id$resp_h];

		local srcs = current_victims[c$id$resp_h];
		if ( |srcs| < max_sources )
			add srcs[c$id$orig_h];
		return;
		}

	if ( ++sample_count % SYNFLOOD_SAMPLE_RATE == 0 )
		{
		local ip = c$id$resp_h;

		if ( ++conn_attempts[ip] * SYNFLOOD_SAMPLE_RATE >
		     SYNFLOOD_THRESHOLD )
			{
			NOTICE([$note=SynFloodStart, $src=ip,
				   $msg=fmt("Start of syn-flood against %s; sampling packets now", ip)]);

			add current_victims[ip][c$id$orig_h];

			# Drop most packets to victim.
			#install_dst_addr_filter(ip, 0,
			#		1 - SYNFLOOD_VICTIM_SAMPLE_RATE);
			# Drop all packets from victim.
			#install_src_addr_filter(ip, 0, 1.0);
			}
		}
	}

event check_synflood()
	{
	for ( ip in current_victims )
		{
		accumulated_conn_attempts[ip] =
			accumulated_conn_attempts[ip] + conn_attempts[ip];

		if ( conn_attempts[ip] * (1 / SYNFLOOD_VICTIM_SAMPLE_RATE) <
		     SYNFLOOD_THRESHOLD )
			{
			NOTICE([$note=SynFloodEnd, $src=ip, $n=|current_victims[ip]|,
				   $msg=fmt("end of syn-flood against %s; stopping sampling",
					ip)]);
 
			delete current_victims[ip];
			#uninstall_dst_addr_filter(ip);
			#uninstall_src_addr_filter(ip);
			}
		}

	clear_table(conn_attempts);
	schedule SYNFLOOD_INTERVAL { check_synflood() };
	}

event report_synflood()
	{
	for ( ip in current_victims )
		{
		local est_num_conn = accumulated_conn_attempts[ip] *
					(1 / SYNFLOOD_VICTIM_SAMPLE_RATE);

		local interv: interval;

		if ( interval_start != 0 )
			interv = network_time() - interval_start;
		else
			interv = SYNFLOOD_INTERVAL;

		NOTICE([$note=SynFloodStatus, $src=ip, $n=|current_victims[ip]|,
			   $msg=fmt("syn-flood against %s; estimated %.0f connections in last %s",
				    ip, est_num_conn, interv)]);
		}

	clear_table(accumulated_conn_attempts);

	schedule SYNFLOOD_REPORT_INTERVAL { report_synflood() };
	interval_start = network_time();
	}

event bro_init()
	{
	schedule SYNFLOOD_INTERVAL { check_synflood() };
	schedule SYNFLOOD_REPORT_INTERVAL { report_synflood() };
	}


From mhamilton at 21ct.com  Fri Feb 21 06:43:31 2014
From: mhamilton at 21ct.com (Mike Hamilton)
Date: Fri, 21 Feb 2014 08:43:31 -0600
Subject: [Bro] 2.1 file analysis logging in 2.2
Message-ID: <CF2CC233.B5CF%mhamilton@21ct.com>

I'm fairly new to Bro, but had a question I was hoping to get answered.  I've combed documentation and source code, but can't quite get to what I'm looking for.

I believe in 2.2, the file analysis engine was modified such that the HTTP, SMTP,etc. file analysis logs were merged into a single files.log file.  Some of the guys around the office thought they remembered a presentation back in August on being able to configure Bro to still report files in the 2.1 mode.

Am I way off base, or is there a way to do this in 2.2 to report like 2.1?

Thanks,
Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140221/e2c12ea0/attachment.html 

From seth at icir.org  Fri Feb 21 08:10:49 2014
From: seth at icir.org (Seth Hall)
Date: Fri, 21 Feb 2014 11:10:49 -0500
Subject: [Bro] 2.1 file analysis logging in 2.2
In-Reply-To: <CF2CC233.B5CF%mhamilton@21ct.com>
References: <CF2CC233.B5CF%mhamilton@21ct.com>
Message-ID: <1DDCD06F-5760-4C8F-95F5-3E889B668D01@icir.org>


On Feb 21, 2014, at 9:43 AM, Mike Hamilton <mhamilton at 21ct.com> wrote:

> I believe in 2.2, the file analysis engine was modified such that the HTTP, SMTP,etc. file analysis logs were merged into a single files.log file.

Yep, sort of.  There is still some information about the files pulled back into the protocol logs too. (and you could write scripts that pull more back).

> Some of the guys around the office thought they remembered a presentation back in August on being able to configure Bro to still report files in the 2.1 mode.  

I think you're going to need to describe what is missing that you want back.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140221/8bc9a62a/attachment.bin 

From engineer.demo2020 at gmail.com  Fri Feb 21 09:01:24 2014
From: engineer.demo2020 at gmail.com (Mr Smith)
Date: Fri, 21 Feb 2014 20:31:24 +0330
Subject: [Bro] Bro Signatures
Message-ID: <CAEL_4H+4qR=3b-Rr3-xyag-4VVsQZKd=0kXvLDF4oYpj-ZfgLA@mail.gmail.com>

Hi
I have a question about Bro-ids.
How many signatures are exist on the latest version of the Bro?
And, how i can deactivate or activate some of these signatures?
Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140221/8b0291d7/attachment.html 

From engineer.demo2020 at gmail.com  Fri Feb 21 09:01:24 2014
From: engineer.demo2020 at gmail.com (Mr Smith)
Date: Fri, 21 Feb 2014 20:31:24 +0330
Subject: [Bro] Bro Signatures
Message-ID: <CAEL_4H+4qR=3b-Rr3-xyag-4VVsQZKd=0kXvLDF4oYpj-ZfgLA@mail.gmail.com>

Hi
I have a question about Bro-ids.
How many signatures are exist on the latest version of the Bro?
And, how i can deactivate or activate some of these signatures?
Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140221/8b0291d7/attachment-0001.html 

From mhamilton at 21ct.com  Fri Feb 21 09:06:34 2014
From: mhamilton at 21ct.com (Mike Hamilton)
Date: Fri, 21 Feb 2014 11:06:34 -0600
Subject: [Bro] 2.1 file analysis logging in 2.2
In-Reply-To: <1DDCD06F-5760-4C8F-95F5-3E889B668D01@icir.org>
Message-ID: <CF2CE24A.B6E5%mhamilton@21ct.com>



On 2/21/14 10:10 AM, "Seth Hall" <seth at icir.org> wrote:

>
>On Feb 21, 2014, at 9:43 AM, Mike Hamilton <mhamilton at 21ct.com> wrote:
>
>> I believe in 2.2, the file analysis engine was modified such that the
>>HTTP, SMTP,etc. file analysis logs were merged into a single files.log
>>file.
>
>Yep, sort of.  There is still some information about the files pulled
>back into the protocol logs too. (and you could write scripts that pull
>more back).
>
>> Some of the guys around the office thought they remembered a
>>presentation back in August on being able to configure Bro to still
>>report files in the 2.1 mode.
>
>I think you're going to need to describe what is missing that you want
>back.

So in 2.1, the http.log format had field/column values for 'mime_type' and
'md5', both simple strings.  That appears to have been expanded
significantly in 2.2 such that there are now 4 columns:
'orig_fuids','orig_mime_types', 'resp_fuids', 'resp_mime_types', which are
vectors of strings that reference the fuid's in files.log(if my
understanding is correct).

Is there a simple way to add back those two old columns to the http.log
file?  Understanding that the new mime_types fields are vectors instead of
straight strings, do either of the new mime_type fields correspond to the
old mime_type column?




>
>  .Seth
>
>--
>Seth Hall
>International Computer Science Institute
>(Bro) because everyone has a network
>http://www.bro.org/
>




From seth at icir.org  Fri Feb 21 12:30:59 2014
From: seth at icir.org (Seth Hall)
Date: Fri, 21 Feb 2014 15:30:59 -0500
Subject: [Bro] 2.1 file analysis logging in 2.2
In-Reply-To: <CF2CE24A.B6E5%mhamilton@21ct.com>
References: <CF2CE24A.B6E5%mhamilton@21ct.com>
Message-ID: <0064EEAB-629D-4D30-B837-3E5DB7BD6650@icir.org>


On Feb 21, 2014, at 12:06 PM, Mike Hamilton <mhamilton at 21ct.com> wrote:

> Is there a simple way to add back those two old columns to the http.log
> file?  Understanding that the new mime_types fields are vectors instead of
> straight strings, do either of the new mime_type fields correspond to the
> old mime_type column?


You can certainly do that, but I do want to point out that the old log was incorrect.  HTTP uses MIME to transfer data so you can send multiple files and receive multiple files in a single request or response.

mime_type is basically the same as the resp_mime_types field except that it can represent more than one file.  Hashes are not included in the HTTP log at all anymore but you can add it back by basically copying how the resp_mime_types field is populated. 

Finally I should probably point out that the resp_fuids and orig_fuids fields correspond to the second field (fuid) in the files.log. 

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140221/1e933871/attachment.bin 

From martin at gta.ufrj.br  Fri Feb 21 15:08:28 2014
From: martin at gta.ufrj.br (Martin Andreoni)
Date: Fri, 21 Feb 2014 20:08:28 -0300
Subject: [Bro]  Cluster+PF_RING doubt
In-Reply-To: <5307D6B9.7020102@gta.ufrj.br>
References: <5307D6B9.7020102@gta.ufrj.br>
Message-ID: <5307DC6C.9030305@gta.ufrj.br>


Hello community , I will ask you a newbie question about Bro and PF_RING

I am working in Bro Cluster topology, as it can be seen in the figure 
attached or in http://www.freeimagehosting.net/newuploads/oevq9.png, all 
are XEN virtual machines working with the PF_Ring.
My intention is to make a load balancing. I am doing a UDP Flood attack 
from several machines in the 123.123.X.X network to one victim in the 
192.168.1.X network. As you can imagine, I need to deviate the traffic 
to a Bro cluster to analyze it.
As it can be seen in the figure, I put a worker sniffing the eth4 
interface from the TAP (worker-5 is in the TAP VM). So here is my doubt. 
Should Bro, through the PF_RING lib, automatically load balancing the 
traffic to all workers? or there is a mistake in my topology? As it is 
working now, I am just receiving all flow in the worker-5 and there is 
no balancing.

  Thanks for your help.

Below are the commands showing that Bro+PF_ring are working:
*
[BroControl] > status*
Name         Type       Host          Status    Pid Peers  Started
manager      manager    192.168.0.61  running   25665 4      20 Feb 17:15:44
proxy-1      proxy      192.168.0.61  running   25712 4      20 Feb 17:15:47
worker-2     worker     192.168.0.62  running   12326 2      20 Feb 17:15:55
worker-3     worker     192.168.0.64  running   22115 2      20 Feb 17:15:58
worker-4     worker     192.168.0.61  running   25768 2      20 Feb 17:15:59
worker-5     worker     192.168.0.100 running   15901 2      20 Feb 17:16:02
*
**root at manager:/usr/local/bro/etc# broctl config |grep pfring*
pfringclusterid = 15
pfringclustertype = 4-tuple

*root at manager:/usr/local/bro/etc# cat node.cfg*
#Cluster Config

[manager]
type=manager
host=192.168.0.61

[proxy-1]
type=proxy
host=192.168.0.61

[worker-2]
type=worker
host=192.168.0.62
interface=eth2
lb_method=pf_ring

[worker-3]
type=worker
host=192.168.0.64
interface=eth2
lb_method=pf_ring

[worker-4]
type=worker
host=192.168.0.61
interface=eth2
lb_method=pf_ring

[worker-5]
type=worker
host=192.168.0.100
interface=eth4
lb_method=pf_ring

*root at Test1:# ldd /usr/local/bro/bin/bro | grep pcap**
*    libpcap.so.1 => /usr/local/pfring/lib/libpcap.so.1 (0x00007f596d4e3000)


*root at worker-5:/proc/net/pf_ring# cat 15901-eth4.115 *
Bound Device(s)    : eth4
Active             : 1
Breed              : Non-DNA
Sampling Rate      : 1
Capture Direction  : RX+TX
Socket Mode        : RX+TX
Appl. Name         : <unknown>
IP Defragment      : No
BPF Filtering      : Enabled
# Sw Filt. Rules   : 0
# Hw Filt. Rules   : 0
Poll Pkt Watermark : 1
Num Poll Calls     : 4053459
Channel Id Mask    : 0xFFFFFFFF
Cluster Id         : 15
Slot Version       : 15 [5.6.3]
Min Num Slots      : 8151
Bucket Len         : 8192
Slot Len           : 8232 [bucket+header]
Tot Memory         : 67108864
Tot Packets        : 963
Tot Pkt Lost       : 0
Tot Insert         : 963
Tot Read           : 963
Insert Offset      : 298168
Remove Offset      : 298168
TX: Send Ok        : 0
TX: Send Errors    : 0
Reflect: Fwd Ok    : 0
Reflect: Fwd Errors: 0
Num Free Slots     : 8151



-- 

  

------------------------------------------------------------------------

  

*Martin Andreoni  *

/Msc. Student/

/Grupo de Teleinform?tica e Automa??o (GTA)/

/Programa de Engenharia El?trica (PEE)/

/Universidade Federal do Rio de Janeiro (UFRJ)/

/www.gta.ufrj.br/~martin  <http://www.gta.ufrj.br/%7Emartin>/









-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140221/4427b02f/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PF_RING.png
Type: image/png
Size: 45946 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140221/4427b02f/attachment.bin 

From seth at icir.org  Fri Feb 21 17:50:32 2014
From: seth at icir.org (Seth Hall)
Date: Fri, 21 Feb 2014 20:50:32 -0500
Subject: [Bro] Cluster+PF_RING doubt
In-Reply-To: <5307DC6C.9030305@gta.ufrj.br>
References: <5307D6B9.7020102@gta.ufrj.br> <5307DC6C.9030305@gta.ufrj.br>
Message-ID: <F161DD63-80B3-4D86-BF73-1FD981475F10@icir.org>


On Feb 21, 2014, at 6:08 PM, Martin Andreoni <martin at gta.ufrj.br> wrote:

> [worker-4]
> type=worker
> host=192.168.0.61
> interface=eth2
> lb_method=pf_ring
> 
> [worker-5]
> type=worker
> host=192.168.0.100
> interface=eth4
> lb_method=pf_ring

This is definitely not right.  You aren't defining how many processes you want to spread the traffic across (with lb_procs) and you have your other workers sniffing eth2 on their respective installations.  PF_Ring spreads the traffic on a single host, but you're running 4 separate hosts (ignoring the fact that they're VMs.  Couple of questions?

  - Why VMs?
  - Why aren't you just allocating more CPU cores to worker-5 and using pf_ring to balance the traffic across those cores?
 
If you allocated more CPU cores to worker-5, your config would look like this?

[worker-5]
type=worker
host=192.168.0.100
interface=eth4
lb_method=pf_ring
lb_procs=4

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140221/c5482f9a/attachment.bin 

From meetalivaidya at gmail.com  Fri Feb 21 23:22:38 2014
From: meetalivaidya at gmail.com (mv)
Date: Sat, 22 Feb 2014 07:22:38 +0000 (UTC)
Subject: [Bro] Bro problem - no software.log written
References: <1377696968.732000-70822245-31691@walla.co.il>
	<66B82052-33E9-488D-A1CE-57117831367F@gmail.com>
Message-ID: <loom.20140222T081500-641@post.gmane.org>

> If sniffing an interface, as a first step check that the software scripts 
are being loaded:
> 
> 
> $ pwd
> /path/to/bro/logs/2013-08-28
> 
> $ zgrep software loaded_scripts.16\:59\:36-17\:00\:00.log.gz?
> ? /usr/local/bro/share/bro/base/frameworks/software/__load__.bro
> ? /usr/local/bro/share/bro/base/frameworks/software/./main.bro
> ? /usr/local/bro/share/bro/policy/frameworks/software/vulnerable.bro
> ? /usr/local/bro/share/bro/policy/frameworks/software/version-changes.bro
> ? /usr/local/bro/share/bro/policy/protocols/ftp/software.bro
> ? /usr/local/bro/share/bro/policy/protocols/smtp/software.bro
> ? /usr/local/bro/share/bro/policy/protocols/ssh/software.bro
> ? /usr/local/bro/share/bro/policy/protocols/http/software.bro

I have included the detect-webapps script in local.bro. It is supposed to 
show the logs in software.log. But the logs are not see. 

I checked that the software scripts are being loaded.

I am not running against a pcap.

Is there any way to debug why software.log is not written. Also, is there 
any other way I can see logs generated by detect-webapps.bro script which 
uses signatures.

Thanks.



From sconzo at visiblerisk.com  Sat Feb 22 05:59:45 2014
From: sconzo at visiblerisk.com (Mike Sconzo)
Date: Sat, 22 Feb 2014 07:59:45 -0600
Subject: [Bro] Bro problem - no software.log written
In-Reply-To: <loom.20140222T081500-641@post.gmane.org>
References: <1377696968.732000-70822245-31691@walla.co.il>
	<66B82052-33E9-488D-A1CE-57117831367F@gmail.com>
	<loom.20140222T081500-641@post.gmane.org>
Message-ID: <CAGZoo6ETH8Qp0oF-V2yYj+jxyTZ0kps_CALhXzWu=b2P3s_acA@mail.gmail.com>

Make sure to set your Sites::local_net variable. If you set it to
0.0.0.0/0 you should get an entry in software.log for every connection
that bro can find qualifying entries for.

On Sat, Feb 22, 2014 at 1:22 AM, mv <meetalivaidya at gmail.com> wrote:
>> If sniffing an interface, as a first step check that the software scripts
> are being loaded:
>>
>>
>> $ pwd
>> /path/to/bro/logs/2013-08-28
>>
>> $ zgrep software loaded_scripts.16\:59\:36-17\:00\:00.log.gz
>>   /usr/local/bro/share/bro/base/frameworks/software/__load__.bro
>>   /usr/local/bro/share/bro/base/frameworks/software/./main.bro
>>   /usr/local/bro/share/bro/policy/frameworks/software/vulnerable.bro
>>   /usr/local/bro/share/bro/policy/frameworks/software/version-changes.bro
>>   /usr/local/bro/share/bro/policy/protocols/ftp/software.bro
>>   /usr/local/bro/share/bro/policy/protocols/smtp/software.bro
>>   /usr/local/bro/share/bro/policy/protocols/ssh/software.bro
>>   /usr/local/bro/share/bro/policy/protocols/http/software.bro
>
> I have included the detect-webapps script in local.bro. It is supposed to
> show the logs in software.log. But the logs are not see.
>
> I checked that the software scripts are being loaded.
>
> I am not running against a pcap.
>
> Is there any way to debug why software.log is not written. Also, is there
> any other way I can see logs generated by detect-webapps.bro script which
> uses signatures.
>
> Thanks.
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



-- 
cat ~/.bash_history > documentation.txt


From engineer.demo2020 at gmail.com  Sun Feb 23 02:51:44 2014
From: engineer.demo2020 at gmail.com (Mr Smith)
Date: Sun, 23 Feb 2014 14:21:44 +0330
Subject: [Bro] Bro Signatures
Message-ID: <CAEL_4HK8Omc50cH6fpXBcGzpwfcvwEGMTB+UdWFgSzYRdp9Xww@mail.gmail.com>

Hi
I have a question about Bro-ids.
How many signatures are exist on the latest version of the Bro?
And, how i can deactivate or activate some of these signatures?
Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140223/14e03eb1/attachment.html 

From vladg at cmu.edu  Sun Feb 23 06:57:27 2014
From: vladg at cmu.edu (Vlad Grigorescu)
Date: Sun, 23 Feb 2014 14:57:27 +0000
Subject: [Bro] Bro Signatures
In-Reply-To: <CAEL_4HK8Omc50cH6fpXBcGzpwfcvwEGMTB+UdWFgSzYRdp9Xww@mail.gmail.com>
References: <CAEL_4HK8Omc50cH6fpXBcGzpwfcvwEGMTB+UdWFgSzYRdp9Xww@mail.gmail.com>
Message-ID: <4418B63E-DECD-491A-A3DF-C15135015246@andrew.cmu.edu>

Bro isn't a signature based IDS. While there is a signature framework, it's not used for large scale signature matching as your e-mail seems to imply. I'd encourage you to review some of the documentation and training material on bro.org to get a better understanding of Bro. If you have any specific questions at that point, we'd love to hear them and help you with them. Of course, it'd be useful if you could give us more than the weekend to reply to you before sending another e-mail...

  --Vlad

On Feb 23, 2014, at 5:51 AM, Mr Smith <engineer.demo2020 at gmail.com> wrote:

> Hi
> I have a question about Bro-ids.
> How many signatures are exist on the latest version of the Bro?
> And, how i can deactivate or activate some of these signatures? 
> Thanks
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140223/ae5d239f/attachment.bin 

From carlopmart at gmail.com  Mon Feb 24 04:07:00 2014
From: carlopmart at gmail.com (C. L. Martinez)
Date: Mon, 24 Feb 2014 12:07:00 +0000
Subject: [Bro] Trying to use different bpf filters on every worker
Message-ID: <CAEjQA5K=gUY_UsEz3NUf9+sxQdgfuDWuTYrRdc7tivf=8xQx1A@mail.gmail.com>

Hi all,

 After installing my first bro cluster with one manager and three
workers, now I need to configure different bpf filters for every bro
worker.

 If I am not worng, I need to put an entry like this:

redef cmd_line_bpf_filter = "ip and port 80 or port 443"; (for example)

 under local.bro file on every worker node. Is this correct for Bro 2.2??

Thanks.


From seth at icir.org  Mon Feb 24 05:16:29 2014
From: seth at icir.org (Seth Hall)
Date: Mon, 24 Feb 2014 08:16:29 -0500
Subject: [Bro] Bro problem - no software.log written
In-Reply-To: <CAGZoo6ETH8Qp0oF-V2yYj+jxyTZ0kps_CALhXzWu=b2P3s_acA@mail.gmail.com>
References: <1377696968.732000-70822245-31691@walla.co.il>
	<66B82052-33E9-488D-A1CE-57117831367F@gmail.com>
	<loom.20140222T081500-641@post.gmane.org>
	<CAGZoo6ETH8Qp0oF-V2yYj+jxyTZ0kps_CALhXzWu=b2P3s_acA@mail.gmail.com>
Message-ID: <3E75ACDC-8030-4EC3-87B2-C8C66495354A@icir.org>


On Feb 22, 2014, at 8:59 AM, Mike Sconzo <sconzo at visiblerisk.com> wrote:

> Make sure to set your Sites::local_net variable. If you set it to
> 0.0.0.0/0 you should get an entry in software.log for every connection
> that bro can find qualifying entries for.


That will have side effects in other areas of Bro.  If you want to log all software seen, it's probably better to use?

redef Software::asset_tracking = ALL_HOSTS;

Keep in mind though that this will have consequences in memory because it will store all of the seen software in memory.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140224/e374ce75/attachment.bin 

From carlopmart at gmail.com  Mon Feb 24 06:22:23 2014
From: carlopmart at gmail.com (C. L. Martinez)
Date: Mon, 24 Feb 2014 14:22:23 +0000
Subject: [Bro] Trying to use different bpf filters on every worker
In-Reply-To: <CAEjQA5K=gUY_UsEz3NUf9+sxQdgfuDWuTYrRdc7tivf=8xQx1A@mail.gmail.com>
References: <CAEjQA5K=gUY_UsEz3NUf9+sxQdgfuDWuTYrRdc7tivf=8xQx1A@mail.gmail.com>
Message-ID: <CAEjQA5Kd1tLAeETit4xQ9BNS=m3i+bW45+XWuqYyutLiyRZuog@mail.gmail.com>

On Mon, Feb 24, 2014 at 12:07 PM, C. L. Martinez <carlopmart at gmail.com> wrote:
> Hi all,
>
>  After installing my first bro cluster with one manager and three
> workers, now I need to configure different bpf filters for every bro
> worker.
>
>  If I am not worng, I need to put an entry like this:
>
> redef cmd_line_bpf_filter = "ip and port 80 or port 443"; (for example)
>
>  under local.bro file on every worker node. Is this correct for Bro 2.2??
>
> Thanks.

Oops, my mistake. I have modified local.bro in one of the workers:

##! Local site policy. Customize as appropriate.
##!
##! This file will not be overwritten when upgrading or reinstalling!

redef PacketFilter::enable_auto_protocol_capture_filters = F;
redef capture_filters = { ["all"] = "ip or not ip" };
redef restrict_filters = { ["not-hosts"] = "not host 10.10.1.15" };

but it doesn't works:

[BroControl] > print restrict_filters
   manager   restrict_filters = {

}
   proxy-1   restrict_filters = {

}
  worker-1   restrict_filters = {

}
[BroControl] >

 What am I doing wrong??


From carlopmart at gmail.com  Mon Feb 24 06:56:36 2014
From: carlopmart at gmail.com (C. L. Martinez)
Date: Mon, 24 Feb 2014 14:56:36 +0000
Subject: [Bro] Trying to use different bpf filters on every worker
In-Reply-To: <CAEjQA5Kd1tLAeETit4xQ9BNS=m3i+bW45+XWuqYyutLiyRZuog@mail.gmail.com>
References: <CAEjQA5K=gUY_UsEz3NUf9+sxQdgfuDWuTYrRdc7tivf=8xQx1A@mail.gmail.com>
	<CAEjQA5Kd1tLAeETit4xQ9BNS=m3i+bW45+XWuqYyutLiyRZuog@mail.gmail.com>
Message-ID: <CAEjQA5+t38Jh3nyekspguLBcoBYu_ezERi=BXSNe-kZdg5mpOw@mail.gmail.com>

On Mon, Feb 24, 2014 at 2:22 PM, C. L. Martinez <carlopmart at gmail.com> wrote:
> On Mon, Feb 24, 2014 at 12:07 PM, C. L. Martinez <carlopmart at gmail.com> wrote:
>> Hi all,
>>
>>  After installing my first bro cluster with one manager and three
>> workers, now I need to configure different bpf filters for every bro
>> worker.
>>
>>  If I am not worng, I need to put an entry like this:
>>
>> redef cmd_line_bpf_filter = "ip and port 80 or port 443"; (for example)
>>
>>  under local.bro file on every worker node. Is this correct for Bro 2.2??
>>
>> Thanks.
>
> Oops, my mistake. I have modified local.bro in one of the workers:
>
> ##! Local site policy. Customize as appropriate.
> ##!
> ##! This file will not be overwritten when upgrading or reinstalling!
>
> redef PacketFilter::enable_auto_protocol_capture_filters = F;
> redef capture_filters = { ["all"] = "ip or not ip" };
> redef restrict_filters = { ["not-hosts"] = "not host 10.10.1.15" };
>
> but it doesn't works:
>
> [BroControl] > print restrict_filters
>    manager   restrict_filters = {
>
> }
>    proxy-1   restrict_filters = {
>
> }
>   worker-1   restrict_filters = {
>
> }
> [BroControl] >
>
>  What am I doing wrong??

Uhmm ok, doing same config using local-worker.bro file, it works:

[BroControl] > print restrict_filters
   manager   restrict_filters = {

}
   proxy-1   restrict_filters = {

}
  worker-1   restrict_filters = {
[not-hosts] = not host 10.196.0.15
}


Then, how can I discriminate by worker and assign a different bpf
filter to each one??


From lists at g-clef.net  Mon Feb 24 07:00:03 2014
From: lists at g-clef.net (Aaron Gee-Clough)
Date: Mon, 24 Feb 2014 10:00:03 -0500
Subject: [Bro] DNS timeout -> memory usage?
Message-ID: <530B5E73.80002@g-clef.net>


All,

Has anyone else seen an impact in changing the dns_session_timeout 
parameter in bro?

I have been wrestling with Bro's memory usage for a while now (using bro 
2.2 from securityonion to monitor DNS server traffic), and recently 
tried changing the dns_session_timeout value from the default of 10 
seconds to 1 second. That has changed bro's memory consumption 
dramatically. While at the default 10 second timeout, Bro was slowly 
growing in RAM usage until the Linux OOM manager killed it (and broctl 
cron automatically restarted it...lather, rinse, repeat). With the 1 
second timeout bro's been steady at ~200MB/worker for the past couple days.

While I'm happy that this seems to have fixed a problem, I'm wondering 
what other impact that change has had. Obviously, if the DNS server 
starts responding slowly bro will see the request and response as 
separate sessions...I think I can live with that. Is that the only 
impact of changing the dns_session_timeout variable?

Thanks.

aaron


From jsiwek at illinois.edu  Mon Feb 24 08:17:07 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Mon, 24 Feb 2014 16:17:07 +0000
Subject: [Bro] DNS timeout -> memory usage?
In-Reply-To: <530B5E73.80002@g-clef.net>
References: <530B5E73.80002@g-clef.net>
Message-ID: <B423A8C5-40B3-489E-9AA0-E0E5BC471F74@illinois.edu>


On Feb 24, 2014, at 9:00 AM, Aaron Gee-Clough <lists at g-clef.net> wrote:

> While I'm happy that this seems to have fixed a problem, I'm wondering 
> what other impact that change has had. Obviously, if the DNS server 
> starts responding slowly bro will see the request and response as 
> separate sessions...I think I can live with that. Is that the only 
> impact of changing the dns_session_timeout variable?

I?d expect another impact besides decreasing ?accuracy? of dns.log to be increased entries in conn.log due to some DNS replies falling outside the timeout threshold.  Also, I think too low of a threshold could also have some performance cost due to increased amount of events being raised which end up expiring sessions too early.

You may be interested in recent commits in Bro?s git repository to the DNS scripts that (for one thing) attempt to put more well-defined bounds on how much state they hold on to.  It could be your traffic hits certain corner cases that, in Bro 2.2 and earlier, could cause the DNS script?s state management logic to do poorly.  If that?s the case, I can see setting a low value for ?dns_session_timeout? being a workaround because it may coincidentally force state to be expired more frequently.

- Jon


From martin at gta.ufrj.br  Mon Feb 24 11:38:39 2014
From: martin at gta.ufrj.br (Martin Andreoni)
Date: Mon, 24 Feb 2014 16:38:39 -0300
Subject: [Bro] Cluster+PF_RING doubt
In-Reply-To: <F161DD63-80B3-4D86-BF73-1FD981475F10@icir.org>
References: <5307D6B9.7020102@gta.ufrj.br> <5307DC6C.9030305@gta.ufrj.br>
	<F161DD63-80B3-4D86-BF73-1FD981475F10@icir.org>
Message-ID: <530B9FBF.9010400@gta.ufrj.br>

Hi Seth, thanks for your answer,

>>- Why VMs?

We are using VM, because we are working in the Application layer of the 
cloud, we allocate VM dynamically. The Idea was to allocate VMs (Host) 
on the fly, doing load balancing between these hosts.

>> PF_Ring spreads the traffic on a single host.

According to you, PF_RING only spread network traffic inside the host 
between the CPU cores. Is it right this phrase? The cluster made by Bro 
should be just Inside a host?
To make the load balancing between several hosts, like I am trying to 
do, I should use OpenFlow or another solution? Any suggestions?

Thank you.



On 02/21/2014 10:50 PM, Seth Hall wrote:
> On Feb 21, 2014, at 6:08 PM, Martin Andreoni<martin at gta.ufrj.br>  wrote:
>
>> [worker-4]
>> type=worker
>> host=192.168.0.61
>> interface=eth2
>> lb_method=pf_ring
>>
>> [worker-5]
>> type=worker
>> host=192.168.0.100
>> interface=eth4
>> lb_method=pf_ring
> This is definitely not right.  You aren't defining how many processes you want to spread the traffic across (with lb_procs) and you have your other workers sniffing eth2 on their respective installations.

> , but you're running 4 separate hosts (ignoring the fact that they're VMs.  Couple of questions?
>
>    - Why VMs?
>    - Why aren't you just allocating more CPU cores to worker-5 and using pf_ring to balance the traffic across those cores?
>   
> If you allocated more CPU cores to worker-5, your config would look like this?
>
> [worker-5]
> type=worker
> host=192.168.0.100
> interface=eth4
> lb_method=pf_ring
> lb_procs=4
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>


-- 

  

------------------------------------------------------------------------

  

*Martin Andreoni  *

/Msc. Student/

/Grupo de Teleinform?tica e Automa??o (GTA)/

/Programa de Engenharia El?trica (PEE)/

/Universidade Federal do Rio de Janeiro (UFRJ)/

/www.gta.ufrj.br/~martin  <http://www.gta.ufrj.br/%7Emartin>/



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140224/e6fc1309/attachment.html 

From seth at icir.org  Mon Feb 24 13:09:47 2014
From: seth at icir.org (Seth Hall)
Date: Mon, 24 Feb 2014 16:09:47 -0500
Subject: [Bro] Cluster+PF_RING doubt
In-Reply-To: <530B9FBF.9010400@gta.ufrj.br>
References: <5307D6B9.7020102@gta.ufrj.br> <5307DC6C.9030305@gta.ufrj.br>
	<F161DD63-80B3-4D86-BF73-1FD981475F10@icir.org>
	<530B9FBF.9010400@gta.ufrj.br>
Message-ID: <71D623A8-7426-4ECB-828D-A0855BB96E73@icir.org>


On Feb 24, 2014, at 2:38 PM, Martin Andreoni <martin at gta.ufrj.br> wrote:

> >> PF_Ring spreads the traffic on a single host.
> 
> According to you, PF_RING only spread network traffic inside the host between the CPU cores. Is it right this phrase? The cluster made by Bro should be just Inside a host?
> To make the load balancing between several hosts, like I am trying to do, I should use OpenFlow or another solution? Any suggestions?

You have to figure something out.  Unfortunately I have no suggestions.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140224/c2e64508/attachment.bin 

From sconzo at visiblerisk.com  Tue Feb 25 00:13:54 2014
From: sconzo at visiblerisk.com (Mike Sconzo)
Date: Tue, 25 Feb 2014 02:13:54 -0600
Subject: [Bro] Bro problem - no software.log written
In-Reply-To: <3E75ACDC-8030-4EC3-87B2-C8C66495354A@icir.org>
References: <1377696968.732000-70822245-31691@walla.co.il>
	<66B82052-33E9-488D-A1CE-57117831367F@gmail.com>
	<loom.20140222T081500-641@post.gmane.org>
	<CAGZoo6ETH8Qp0oF-V2yYj+jxyTZ0kps_CALhXzWu=b2P3s_acA@mail.gmail.com>
	<3E75ACDC-8030-4EC3-87B2-C8C66495354A@icir.org>
Message-ID: <CAGZoo6E-_awxispFnCsRz_g3nWFF=-yx05KBKdsXtcqmP1bTzA@mail.gmail.com>

Figured it'd have side effects, didn't really matter for my use cases.
However, the ALL_HOSTS setting is useful, and I didn't know that.

Thanks!

On Mon, Feb 24, 2014 at 7:16 AM, Seth Hall <seth at icir.org> wrote:
>
> On Feb 22, 2014, at 8:59 AM, Mike Sconzo <sconzo at visiblerisk.com> wrote:
>
>> Make sure to set your Sites::local_net variable. If you set it to
>> 0.0.0.0/0 you should get an entry in software.log for every connection
>> that bro can find qualifying entries for.
>
>
> That will have side effects in other areas of Bro.  If you want to log all software seen, it's probably better to use...
>
> redef Software::asset_tracking = ALL_HOSTS;
>
> Keep in mind though that this will have consequences in memory because it will store all of the seen software in memory.
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>



-- 
cat ~/.bash_history > documentation.txt


From meetalivaidya at gmail.com  Tue Feb 25 01:13:50 2014
From: meetalivaidya at gmail.com (Meetali Vaidya)
Date: Tue, 25 Feb 2014 14:43:50 +0530
Subject: [Bro] Bro problem - no software.log written
In-Reply-To: <CAGZoo6E-_awxispFnCsRz_g3nWFF=-yx05KBKdsXtcqmP1bTzA@mail.gmail.com>
References: <1377696968.732000-70822245-31691@walla.co.il>
	<66B82052-33E9-488D-A1CE-57117831367F@gmail.com>
	<loom.20140222T081500-641@post.gmane.org>
	<CAGZoo6ETH8Qp0oF-V2yYj+jxyTZ0kps_CALhXzWu=b2P3s_acA@mail.gmail.com>
	<3E75ACDC-8030-4EC3-87B2-C8C66495354A@icir.org>
	<CAGZoo6E-_awxispFnCsRz_g3nWFF=-yx05KBKdsXtcqmP1bTzA@mail.gmail.com>
Message-ID: <CACrPO6aBjkhHA5XpKZwX4aOL_316+ee-+22_SG4FcV=Oy-310A@mail.gmail.com>

Hello,
the ALL_HOSTS setting is not useful in my case. In my case, I want to
detect some webapps. I have enabled the detect-webapps.bro script. But logs
are not created. Am I missing anything else that needs to be changed?

Thanks.



On Tue, Feb 25, 2014 at 1:43 PM, Mike Sconzo <sconzo at visiblerisk.com> wrote:

> Figured it'd have side effects, didn't really matter for my use cases.
> However, the ALL_HOSTS setting is useful, and I didn't know that.
>
> Thanks!
>
> On Mon, Feb 24, 2014 at 7:16 AM, Seth Hall <seth at icir.org> wrote:
> >
> > On Feb 22, 2014, at 8:59 AM, Mike Sconzo <sconzo at visiblerisk.com> wrote:
> >
> >> Make sure to set your Sites::local_net variable. If you set it to
> >> 0.0.0.0/0 you should get an entry in software.log for every connection
> >> that bro can find qualifying entries for.
> >
> >
> > That will have side effects in other areas of Bro.  If you want to log
> all software seen, it's probably better to use...
> >
> > redef Software::asset_tracking = ALL_HOSTS;
> >
> > Keep in mind though that this will have consequences in memory because
> it will store all of the seen software in memory.
> >
> >   .Seth
> >
> > --
> > Seth Hall
> > International Computer Science Institute
> > (Bro) because everyone has a network
> > http://www.bro.org/
> >
>
>
>
> --
> cat ~/.bash_history > documentation.txt
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140225/c8ae7d34/attachment.html 

From seth at icir.org  Tue Feb 25 06:03:47 2014
From: seth at icir.org (Seth Hall)
Date: Tue, 25 Feb 2014 09:03:47 -0500
Subject: [Bro] Bro problem - no software.log written
In-Reply-To: <CACrPO6aBjkhHA5XpKZwX4aOL_316+ee-+22_SG4FcV=Oy-310A@mail.gmail.com>
References: <1377696968.732000-70822245-31691@walla.co.il>
	<66B82052-33E9-488D-A1CE-57117831367F@gmail.com>
	<loom.20140222T081500-641@post.gmane.org>
	<CAGZoo6ETH8Qp0oF-V2yYj+jxyTZ0kps_CALhXzWu=b2P3s_acA@mail.gmail.com>
	<3E75ACDC-8030-4EC3-87B2-C8C66495354A@icir.org>
	<CAGZoo6E-_awxispFnCsRz_g3nWFF=-yx05KBKdsXtcqmP1bTzA@mail.gmail.com>
	<CACrPO6aBjkhHA5XpKZwX4aOL_316+ee-+22_SG4FcV=Oy-310A@mail.gmail.com>
Message-ID: <62C6BFE1-A007-45B8-9A39-4CB321CF5B8D@icir.org>


On Feb 25, 2014, at 4:13 AM, Meetali Vaidya <meetalivaidya at gmail.com> wrote:

> Hello,
> the ALL_HOSTS setting is not useful in my case. In my case, I want to detect some webapps. I have enabled the detect-webapps.bro script. But logs are not created. Am I missing anything else that needs to be changed?

Are you loading the policy/protocols/http/detect-webapps.bro script?  That script isn't very good either, so don't expect anything magic. :)

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140225/2bc47819/attachment.bin 

From antsankov at gmail.com  Tue Feb 25 15:47:49 2014
From: antsankov at gmail.com (Alexander Tsankov)
Date: Tue, 25 Feb 2014 16:47:49 -0700
Subject: [Bro] Question about Controller Framework
Message-ID: <8237E570-E3F7-42CA-9DD6-67A611A67E22@gmail.com>

Hi, I am trying to work with the bro control framework and I had two main questions about it: 

1) Is it possible for one BRO script (Script 1) to send a request to another BRO script(Script 2), possibly on the same device , and for Script 2 to return a list of all of its local/global variables without Script 1 having known about any of them? 

I am working on an SDN project and my goal is to have a BRO box running a master script and whenever I send a generic bro script to the box, I am trying to get the new script  to return a list of all variables to the master bro script(script 1), and from the master script, send it back to the original sender.  

2) Is it possible for Script 1 to modify variables on script 2 on the fly? 

Any help is appreciated. 

- Alex


From jsiwek at illinois.edu  Wed Feb 26 07:55:51 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Wed, 26 Feb 2014 15:55:51 +0000
Subject: [Bro] Question about Controller Framework
In-Reply-To: <8237E570-E3F7-42CA-9DD6-67A611A67E22@gmail.com>
References: <8237E570-E3F7-42CA-9DD6-67A611A67E22@gmail.com>
Message-ID: <FBD8B140-8348-4CA6-932C-AA074E898446@illinois.edu>


On Feb 25, 2014, at 5:47 PM, Alexander Tsankov <antsankov at gmail.com> wrote:

> 1) Is it possible for one BRO script (Script 1) to send a request to another BRO script(Script 2), possibly on the same device , and for Script 2 to return a list of all of its local/global variables without Script 1 having known about any of them? 

There is the global_ids() function [1] to discover globals and whether they?re ?exported?, which might be what you want in terms of telling whether it?s a ?local? variable ? local usually refers to function scope within Bro which probably isn?t what you want.  The other two scopes are module (i.e. private to the modules namespace) and global (i.e. visible across module namespaces), which are determined by whether it?s exported

As far as sending the return value of global_ids() from one Bro instance to another, that should be possible ? Bro instances commonly exchange events w/ one another, so it?s just a matter of defining an event to carry this information.

> 2) Is it possible for Script 1 to modify variables on script 2 on the fly? 

The send_id() function [2] may be one way to do that.

- Jon

[1] http://bro.org/sphinx-git/scripts/base/bif/bro.bif.bro.html#id-global_ids
[2] http://bro.org/sphinx-git/scripts/base/bif/bro.bif.bro.html#id-send_id


From bkellogg at dresser-rand.com  Thu Feb 27 11:23:25 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Thu, 27 Feb 2014 19:23:25 +0000
Subject: [Bro] vector array of string used as a pattern for matching
Message-ID: <BDF3793B557F624699D4C68A3B8FCF820CB992@DRTEXW238VM.na.d-rco.com>

I'm trying to create an array of domain names that I want to use as a pattern to search on.  I know the below is wrong; just looking for someone to educate me on how to do this in a Bro script if it can be done.  thanks

global ignoreDomains: vector of string = vector("webex.com", "pwc.com", "messagelabs.com","akamaitechnologies.com");

                when (local dst = lookup_addr(c$id$resp_h))
                        {
                        if (/ignoreDomains$/ in dst)
                                return;
                        }


Thank you,
Brian Kellogg
Security Analyst; IT Governance, Risk, and Compliance
500 Paul Clark Drive, Olean,  NY 14760
T: (716) 375-3186 | F: (716) 375-3557
www.dresser-rand.com<http://www.dresser-rand.com/>     NYSE: DRC

[Description: Description: Description: Description: Description: Description: d-r_wordraster3R-hi]
Bringing energy and the environment into harmony(r)
IMPORTANT NOTICE:
This email may be confidential, may be legally privileged, and is for the intended recipient only. Unauthorized access, disclosure, copying, distribution, or reliance on any of it by anyone else is prohibited and may be a criminal offense. Please delete if obtained in error and email confirmation to the sender.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140227/38ecfd3d/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2366 bytes
Desc: image001.jpg
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140227/38ecfd3d/attachment.jpg 

From anthony.kasza at gmail.com  Thu Feb 27 11:56:27 2014
From: anthony.kasza at gmail.com (anthony kasza)
Date: Thu, 27 Feb 2014 11:56:27 -0800
Subject: [Bro] vector array of string used as a pattern for matching
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF820CB992@DRTEXW238VM.na.d-rco.com>
References: <BDF3793B557F624699D4C68A3B8FCF820CB992@DRTEXW238VM.na.d-rco.com>
Message-ID: <CAEZw2bwwPcLcXy7wmGinVTCgjbxZ-QJ35vM4d0-xsMwNGnDpBg@mail.gmail.com>

You could use a set of patterns.

foo: set[pattern] = YourPatterns
for (each in foo)
{
  if (each in DomainInQuestion)
    DoSomething
}

-AK
On Feb 27, 2014 11:44 AM, "Kellogg, Brian D (OLN)" <
bkellogg at dresser-rand.com> wrote:

>  I'm trying to create an array of domain names that I want to use as a
> pattern to search on.  I know the below is wrong; just looking for someone
> to educate me on how to do this in a Bro script if it can be done.  thanks
>
>
>
> global ignoreDomains: vector of string = vector("webex.com", "pwc.com", "
> messagelabs.com","akamaitechnologies.com");
>
>
>
>                 when (local dst = lookup_addr(c$id$resp_h))
>
>                         {
>
>                         if (/ignoreDomains$/ in dst)
>
>                                 return;
>
>                         }
>
>
>
>
>
> Thank you,
>
> *Brian Kellogg*
>
> Security Analyst; IT Governance, Risk, and Compliance
>
> 500 Paul Clark Drive, Olean,  NY 14760
>
> T: (716) 375-3186 | F: (716) 375-3557
>
> www.dresser-rand.com     NYSE: DRC
>
>
>
> [image: Description: Description: Description: Description: Description:
> Description: d-r_wordraster3R-hi]
>
> Bringing energy and the environment into harmony(R)
>
>
> *IMPORTANT NOTICE: This email may be confidential, may be legally
> privileged, and is for the intended recipient only. Unauthorized access,
> disclosure, copying, distribution, or reliance on any of it by anyone else
> is prohibited and may be a criminal offense. Please delete if obtained in
> error and email confirmation to the sender.*
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140227/716ebf64/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2366 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140227/716ebf64/attachment.jpg 

From bkellogg at dresser-rand.com  Thu Feb 27 11:58:35 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Thu, 27 Feb 2014 19:58:35 +0000
Subject: [Bro] vector array of string used as a pattern for matching
In-Reply-To: <CAEZw2bwwPcLcXy7wmGinVTCgjbxZ-QJ35vM4d0-xsMwNGnDpBg@mail.gmail.com>
References: <BDF3793B557F624699D4C68A3B8FCF820CB992@DRTEXW238VM.na.d-rco.com>
	<CAEZw2bwwPcLcXy7wmGinVTCgjbxZ-QJ35vM4d0-xsMwNGnDpBg@mail.gmail.com>
Message-ID: <BDF3793B557F624699D4C68A3B8FCF820CB9D1@DRTEXW238VM.na.d-rco.com>

Thanks, I thought of that as well.  I was trying to not use a loop if at all possible.

Thanks,
Brian

From: anthony kasza [mailto:anthony.kasza at gmail.com]
Sent: Thursday, February 27, 2014 2:56 PM
To: Kellogg, Brian D (OLN)
Cc: bro at bro.org
Subject: Re: [Bro] vector array of string used as a pattern for matching


You could use a set of patterns.

foo: set[pattern] = YourPatterns
for (each in foo)
{
  if (each in DomainInQuestion)
    DoSomething
}

-AK
On Feb 27, 2014 11:44 AM, "Kellogg, Brian D (OLN)" <bkellogg at dresser-rand.com<mailto:bkellogg at dresser-rand.com>> wrote:
I'm trying to create an array of domain names that I want to use as a pattern to search on.  I know the below is wrong; just looking for someone to educate me on how to do this in a Bro script if it can be done.  thanks

global ignoreDomains: vector of string = vector("webex.com<http://webex.com>", "pwc.com<http://pwc.com>", "messagelabs.com<http://messagelabs.com>","akamaitechnologies.com<http://akamaitechnologies.com>");

                when (local dst = lookup_addr(c$id$resp_h))
                        {
                        if (/ignoreDomains$/ in dst)
                                return;
                        }


Thank you,
Brian Kellogg
Security Analyst; IT Governance, Risk, and Compliance
500 Paul Clark Drive, Olean,  NY 14760
T: (716) 375-3186<tel:%28716%29%20375-3186> | F: (716) 375-3557<tel:%28716%29%20375-3557>
www.dresser-rand.com<http://www.dresser-rand.com/>     NYSE: DRC

[Description: Description: Description: Description: Description: Description: d-r_wordraster3R-hi]
Bringing energy and the environment into harmony(r)
IMPORTANT NOTICE:
This email may be confidential, may be legally privileged, and is for the intended recipient only. Unauthorized access, disclosure, copying, distribution, or reliance on any of it by anyone else is prohibited and may be a criminal offense. Please delete if obtained in error and email confirmation to the sender.


_______________________________________________
Bro mailing list
bro at bro-ids.org<mailto:bro at bro-ids.org>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140227/b9f374eb/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2366 bytes
Desc: image001.jpg
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140227/b9f374eb/attachment.jpg 

From shaleta.bennett at gmail.com  Thu Feb 27 12:16:27 2014
From: shaleta.bennett at gmail.com (Shaleta Bennett)
Date: Thu, 27 Feb 2014 15:16:27 -0500
Subject: [Bro] Send notices to a new log file instead of notice.log
Message-ID: <CAOQCEU-=cQQXO9cXnsOj0YsrTpUv7stvc9Yq3M787TkDczNd4w@mail.gmail.com>

Is it possible to send notices to new log file instead of notice.log? If
so, can anyone explain how this can be done?

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140227/605baaa8/attachment.html 

From jmellander at lbl.gov  Thu Feb 27 12:28:31 2014
From: jmellander at lbl.gov (Jim Mellander)
Date: Thu, 27 Feb 2014 12:28:31 -0800
Subject: [Bro] vector array of string used as a pattern for matching
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF820CB9D1@DRTEXW238VM.na.d-rco.com>
References: <BDF3793B557F624699D4C68A3B8FCF820CB992@DRTEXW238VM.na.d-rco.com>
	<CAEZw2bwwPcLcXy7wmGinVTCgjbxZ-QJ35vM4d0-xsMwNGnDpBg@mail.gmail.com>
	<BDF3793B557F624699D4C68A3B8FCF820CB9D1@DRTEXW238VM.na.d-rco.com>
Message-ID: <CADju=b7YkcEb6347LHwLR0ScO5OokcYrpokTyVPZ0qN3eMvdWg@mail.gmail.com>

I wrote some fairly elaborate code (called Stomper) a number of years ago
that performed URL/domain matching on a blacklist, and killed the
connections in realtime, probably could be adapted to your use case.  Aside
from the other actions, the domain matching is done by successively
splitting the domain into smaller parts & check for set membership

An example - given a domain www.badguy.com we would check:

www.badguy.com
badguy.com
.com

for membership in the set, and act on it accordingly - of course, unless
you're interested in tracking by TLD you wouldn't go all the way down to
.com, in this example.

If you're interested in the code, contact me offline




On Thu, Feb 27, 2014 at 11:58 AM, Kellogg, Brian D (OLN) <
bkellogg at dresser-rand.com> wrote:

>  Thanks, I thought of that as well.  I was trying to not use a loop if at
> all possible.
>
>
>
> Thanks,
>
> Brian
>
>
>
> *From:* anthony kasza [mailto:anthony.kasza at gmail.com]
> *Sent:* Thursday, February 27, 2014 2:56 PM
> *To:* Kellogg, Brian D (OLN)
> *Cc:* bro at bro.org
> *Subject:* Re: [Bro] vector array of string used as a pattern for matching
>
>
>
> You could use a set of patterns.
>
> foo: set[pattern] = YourPatterns
> for (each in foo)
> {
>   if (each in DomainInQuestion)
>     DoSomething
> }
>
> -AK
>
> On Feb 27, 2014 11:44 AM, "Kellogg, Brian D (OLN)" <
> bkellogg at dresser-rand.com> wrote:
>
> I'm trying to create an array of domain names that I want to use as a
> pattern to search on.  I know the below is wrong; just looking for someone
> to educate me on how to do this in a Bro script if it can be done.  thanks
>
>
>
> global ignoreDomains: vector of string = vector("webex.com", "pwc.com", "
> messagelabs.com","akamaitechnologies.com");
>
>
>
>                 when (local dst = lookup_addr(c$id$resp_h))
>
>                         {
>
>                         if (/ignoreDomains$/ in dst)
>
>                                 return;
>
>                         }
>
>
>
>
>
> Thank you,
>
> *Brian Kellogg*
>
> Security Analyst; IT Governance, Risk, and Compliance
>
> 500 Paul Clark Drive, Olean,  NY 14760
>
> T: (716) 375-3186 | F: (716) 375-3557
>
> www.dresser-rand.com     NYSE: DRC
>
>
>
> [image: Description: Description: Description: Description: Description:
> Description: d-r_wordraster3R-hi]
>
> Bringing energy and the environment into harmony(R)
>
>
> *IMPORTANT NOTICE: This email may be confidential, may be legally
> privileged, and is for the intended recipient only. Unauthorized access,
> disclosure, copying, distribution, or reliance on any of it by anyone else
> is prohibited and may be a criminal offense. Please delete if obtained in
> error and email confirmation to the sender.*
>
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140227/c2941382/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2366 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140227/c2941382/attachment.jpg 

From antsankov at gmail.com  Thu Feb 27 15:15:58 2014
From: antsankov at gmail.com (Alexander Tsankov)
Date: Thu, 27 Feb 2014 16:15:58 -0700
Subject: [Bro] Question about Controller Framework
In-Reply-To: <FBD8B140-8348-4CA6-932C-AA074E898446@illinois.edu>
References: <8237E570-E3F7-42CA-9DD6-67A611A67E22@gmail.com>
	<FBD8B140-8348-4CA6-932C-AA074E898446@illinois.edu>
Message-ID: <EAC817F2-BF86-4DB4-9187-51757641C5E4@gmail.com>

Sounds great! I?m playing around with the global_ids() function, but I?m having the problem of getting too much information. My goal is to only identify global variables that are created in the script, but I am getting internal global variables that I never actually created. 

In my sample bro script I define: 
===========================
global hello: string ?Hello World? 
===========================

But when I run the global_ids function I get records for other things listed below, but not limited to, KEEP_ORIG_ADDR, script_id, etc. My goal is to only print out the script-defined variables. In this case I only want the ?hello? record to be printed. 

===========================
KEEP_ORIG_ADDR
[type_name=enum, exported=F, constant=F, enum_constant=T, redefinable=F, value=<uninitialized>]
script_id
[type_name=record, exported=F, constant=F, enum_constant=F, redefinable=F, value=<uninitialized>]
gtp_teid1
[type_name=count, exported=F, constant=F, enum_constant=F, redefinable=F, value=<uninitialized>]
hello
[type_name=string, exported=F, constant=F, enum_constant=F, redefinable=F, value=hello world]
===========================

I assume that there must be some table for this, otherwise BRO wouldn?t known when to spit back an error when you try to define the something twice.

===========================
global hello: string ?Hello World? 
global hello: string ?Hi"
===========================

Is there anything I can do about this information overload? 


Regards, 

Alex Tsankov

On Feb 26, 2014, at 8:55 AM, Siwek, Jonathan Luke <jsiwek at illinois.edu> wrote:

> 
> On Feb 25, 2014, at 5:47 PM, Alexander Tsankov <antsankov at gmail.com> wrote:
> 
>> 1) Is it possible for one BRO script (Script 1) to send a request to another BRO script(Script 2), possibly on the same device , and for Script 2 to return a list of all of its local/global variables without Script 1 having known about any of them? 
> 
> There is the global_ids() function [1] to discover globals and whether they?re ?exported?, which might be what you want in terms of telling whether it?s a ?local? variable ? local usually refers to function scope within Bro which probably isn?t what you want.  The other two scopes are module (i.e. private to the modules namespace) and global (i.e. visible across module namespaces), which are determined by whether it?s exported
> 
> As far as sending the return value of global_ids() from one Bro instance to another, that should be possible ? Bro instances commonly exchange events w/ one another, so it?s just a matter of defining an event to carry this information.
> 
>> 2) Is it possible for Script 1 to modify variables on script 2 on the fly? 
> 
> The send_id() function [2] may be one way to do that.
> 
> - Jon
> 
> [1] http://bro.org/sphinx-git/scripts/base/bif/bro.bif.bro.html#id-global_ids
> [2] http://bro.org/sphinx-git/scripts/base/bif/bro.bif.bro.html#id-send_id




From carlopmart at gmail.com  Fri Feb 28 06:40:00 2014
From: carlopmart at gmail.com (C. L. Martinez)
Date: Fri, 28 Feb 2014 14:40:00 +0000
Subject: [Bro] Mix different OSes as a workers in the same Bro cluster
Message-ID: <CAEjQA5Lvj6ob18FZ=b00e+OJ_jgn9HDMnuMeS0T2A+fVpF8xVg@mail.gmail.com>

Hi all,

 Is it possible to use different operating systems as a workers inside
the same Bro cluster? I have a CentOS host acting as a Bro
manager/proxy and 5 CentOS acting as a workers. Now, I need to add two
FreeBSD hosts inside this cluster.

 I have do it a simple test in a VM environment, and CentOS manager
has transferred Bro libraries and binaries to a FreeBSD vm configured
as a worker inside the cluster.

 Is it possible to avoid this situation??

 Thanks.


From lists at g-clef.net  Fri Feb 28 06:47:14 2014
From: lists at g-clef.net (Aaron Gee-Clough)
Date: Fri, 28 Feb 2014 09:47:14 -0500
Subject: [Bro] vector array of string used as a pattern for matching
References: <BDF3793B557F624699D4C68A3B8FCF820CB992@DRTEXW238VM.na.d-rco.com>
	<CAEZw2bwwPcLcXy7wmGinVTCgjbxZ-QJ35vM4d0-xsMwNGnDpBg@mail.gmail.com>
	<BDF3793B557F624699D4C68A3B8FCF820CB9D1@DRTEXW238VM.na.d-rco.com>
Message-ID: <5310A172.1070609@g-clef.net>


Wouldn't this be a good use case for a Bloom filter?

http://www.bro.org/sphinx/scripts/base/bif/bloom-filter.bif.html

aaron

On 02/27/2014 02:58 PM, Kellogg, Brian D (OLN) wrote:
> Thanks, I thought of that as well.  I was trying to not use a loop if at
> all possible.
>
> Thanks,
>
> Brian
>
> *From:*anthony kasza [mailto:anthony.kasza at gmail.com]
> *Sent:* Thursday, February 27, 2014 2:56 PM
> *To:* Kellogg, Brian D (OLN)
> *Cc:* bro at bro.org
> *Subject:* Re: [Bro] vector array of string used as a pattern for matching
>
> You could use a set of patterns.
>
> foo: set[pattern] = YourPatterns
> for (each in foo)
> {
>    if (each in DomainInQuestion)
>      DoSomething
> }
>
> -AK
>
> On Feb 27, 2014 11:44 AM, "Kellogg, Brian D (OLN)"
> <bkellogg at dresser-rand.com <mailto:bkellogg at dresser-rand.com>> wrote:
>
> I?m trying to create an array of domain names that I want to use as a
> pattern to search on.  I know the below is wrong; just looking for
> someone to educate me on how to do this in a Bro script if it can be
> done.  thanks
>
> global ignoreDomains: vector of string = vector("webex.com
> <http://webex.com>", "pwc.com <http://pwc.com>", "messagelabs.com
> <http://messagelabs.com>","akamaitechnologies.com
> <http://akamaitechnologies.com>");
>
>                  when (local dst = lookup_addr(c$id$resp_h))
>
>                          {
>
>                          if (/ignoreDomains$/ in dst)
>
>                                  return;
>
>                          }
>
> Thank you,
>
> *Brian Kellogg*
>
> Security Analyst; IT Governance, Risk, and Compliance
>
> 500 Paul Clark Drive, Olean,  NY 14760
>
> T: (716) 375-3186 <tel:%28716%29%20375-3186> | F: (716) 375-3557
> <tel:%28716%29%20375-3557>
>
> www.dresser-rand.com <http://www.dresser-rand..com/>NYSE: DRC
>
> Description: Description: Description: Description: Description:
> Description: d-r_wordraster3R-hi
>
> Bringing energy and the environment into harmony^?
>
> *IMPORTANT NOTICE:
> This email may be confidential, may be legally privileged, and is for
> the intended recipient only. Unauthorized access, disclosure, copying,
> distribution, or reliance on any of it by anyone else is prohibited and
> may be a criminal offense. Please delete if obtained in error and email
> confirmation to the sender.*
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org <mailto:bro at bro-ids.org>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>


From seth at icir.org  Fri Feb 28 07:12:23 2014
From: seth at icir.org (Seth Hall)
Date: Fri, 28 Feb 2014 10:12:23 -0500
Subject: [Bro] vector array of string used as a pattern for matching
In-Reply-To: <BDF3793B557F624699D4C68A3B8FCF820CB992@DRTEXW238VM.na.d-rco.com>
References: <BDF3793B557F624699D4C68A3B8FCF820CB992@DRTEXW238VM.na.d-rco.com>
Message-ID: <EE64A765-B9E5-48FA-854A-715967D9C96C@icir.org>


On Feb 27, 2014, at 2:23 PM, "Kellogg, Brian D (OLN)" <bkellogg at dresser-rand.com> wrote:

> I?m trying to create an array of domain names that I want to use as a pattern to search on.  I know the below is wrong; just looking for someone to educate me on how to do this in a Bro script if it can be done.  thanks

# Create a set of domain suffixes.
global ignore_domains = set(".webex.com", ".pwc.com", ".messagelabs.com", ".akamaitechnologies.com");
# Create an empty pattern where we're going to automatically create.
global my_domain_suffixes = /MATCH_NOTHING/; # There is bug with setting blank patterns at the moment.

event bro_init() &priority=10
	{
	# Create the my_domain_suffixes pattern by auto constructing it from the ignore_domains set.
	my_domain_suffixes = set_to_regex(ignore_domains, "(^\\.?|\\.)(~~)$");
	}

# I'll give an example event like you want.
event whatever(c: connection)
	{
	when (local name = lookup_addr(c$id$resp_h))
		{
		if (my_domain_suffixes in name)
			return;
		}
	}


One thing to keep in mind with this script is the amount of DNS traffic you could easily cause if you handle an event that fires a lot (like the connection_established event).  You may want to do some name caching or restrictions for when the look ups are done.  We are also getting some evidence that overusing when statements is causing trouble for a few people.

Please when you are writing scripts, put them into a namespace (with "module MySpecialModule;" at the beginning of your script) to help us avoid stomping on identifier names that you are using.

Thanks,
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140228/f8f4bc60/attachment.bin 

From seth at icir.org  Fri Feb 28 07:37:07 2014
From: seth at icir.org (Seth Hall)
Date: Fri, 28 Feb 2014 10:37:07 -0500
Subject: [Bro] vector array of string used as a pattern for matching
In-Reply-To: <CADju=b7YkcEb6347LHwLR0ScO5OokcYrpokTyVPZ0qN3eMvdWg@mail.gmail.com>
References: <BDF3793B557F624699D4C68A3B8FCF820CB992@DRTEXW238VM.na.d-rco.com>
	<CAEZw2bwwPcLcXy7wmGinVTCgjbxZ-QJ35vM4d0-xsMwNGnDpBg@mail.gmail.com>
	<BDF3793B557F624699D4C68A3B8FCF820CB9D1@DRTEXW238VM.na.d-rco.com>
	<CADju=b7YkcEb6347LHwLR0ScO5OokcYrpokTyVPZ0qN3eMvdWg@mail.gmail.com>
Message-ID: <A951A4EA-DFD0-4FB9-BA24-F577728A5873@icir.org>


On Feb 27, 2014, at 3:28 PM, Jim Mellander <jmellander at LBL.GOV> wrote:

> the domain matching is done by successively splitting the domain into smaller parts & check for set membership

This has a lot of overhead in Bro at the moment due to the amount of string manipulation.  I have an example module of a faster way to do this that doesn't involve any string manipulation.

	https://github.com/sethhall/bro-junk-drawer/tree/master/domain-tld

It includes Mozilla's list of "effective TLDs".  Things like co.uk are counted as TLDs.

> @load domain-tld
> DomainTLD::effective_tld("www.google.co.uk");
  => co.uk
> DomainTLD::effective_tld("www.google.com");
  => com
> DomainTLD::effective_domain("whatever.www.blah.google.co.uk");
  => google.co.uk
> DomainTLD::effective_domain("whatever.www.blah.google.com");
  => google.com

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140228/8bba98ac/attachment.bin 

From kmcmahon at mitre.org  Fri Feb 28 07:41:58 2014
From: kmcmahon at mitre.org (McMahon, Kevin J)
Date: Fri, 28 Feb 2014 15:41:58 +0000
Subject: [Bro] SMB Event Prototype Issue
In-Reply-To: <C41581D1AA533F48AB7F9E17CE26D6DF529393A9@EXMB02.srn.sandia.gov>
References: <C41581D1AA533F48AB7F9E17CE26D6DF529393A9@EXMB02.srn.sandia.gov>
Message-ID: <00D3CD29F7C24A44B4D23450BB8E55B30AB7C115@IMCMBX03.MITRE.ORG>

David,

I've been interested in utilizing the SMB analyzer on Bro as well.  Where did you find Seth's github data on SMB?

And, are you interested in collaborating on the porting of the SMB scripts?

Kevin


From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Pearson, David
Sent: Wednesday, January 15, 2014 6:08 PM
To: bro at bro.org
Subject: Re: [Bro] SMB Event Prototype Issue

Hello again,

You can ignore my last question; I found Seth's topic github branch on SMB.

Thanks,
David


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140228/19b1500e/attachment.html 

From bkellogg at dresser-rand.com  Fri Feb 28 07:46:03 2014
From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN))
Date: Fri, 28 Feb 2014 15:46:03 +0000
Subject: [Bro] vector array of string used as a pattern for matching
In-Reply-To: <EE64A765-B9E5-48FA-854A-715967D9C96C@icir.org>
References: <BDF3793B557F624699D4C68A3B8FCF820CB992@DRTEXW238VM.na.d-rco.com>
	<EE64A765-B9E5-48FA-854A-715967D9C96C@icir.org>
Message-ID: <BDF3793B557F624699D4C68A3B8FCF820CC6EC@DRTEXW238VM.na.d-rco.com>

Thank you!

This is for a script that alerts on large outgoing Tx's.  So the domain lookups are not going to be that frequent; at least they better not be.  

Hopefully I'll get some time next week to work this solution in.  Wish I had more time to spend on Bro.  It is an incredible and invaluable tool for any NSM solution.


Thank you,
Brian Kellogg
?
-----Original Message-----
From: Seth Hall [mailto:seth at icir.org] 
Sent: Friday, February 28, 2014 10:12 AM
To: Kellogg, Brian D (OLN)
Cc: bro at bro.org
Subject: Re: [Bro] vector array of string used as a pattern for matching


On Feb 27, 2014, at 2:23 PM, "Kellogg, Brian D (OLN)" <bkellogg at dresser-rand.com> wrote:

> I'm trying to create an array of domain names that I want to use as a 
> pattern to search on.  I know the below is wrong; just looking for 
> someone to educate me on how to do this in a Bro script if it can be 
> done.  thanks

# Create a set of domain suffixes.
global ignore_domains = set(".webex.com", ".pwc.com", ".messagelabs.com", ".akamaitechnologies.com"); # Create an empty pattern where we're going to automatically create.
global my_domain_suffixes = /MATCH_NOTHING/; # There is bug with setting blank patterns at the moment.

event bro_init() &priority=10
	{
	# Create the my_domain_suffixes pattern by auto constructing it from the ignore_domains set.
	my_domain_suffixes = set_to_regex(ignore_domains, "(^\\.?|\\.)(~~)$");
	}

# I'll give an example event like you want.
event whatever(c: connection)
	{
	when (local name = lookup_addr(c$id$resp_h))
		{
		if (my_domain_suffixes in name)
			return;
		}
	}


One thing to keep in mind with this script is the amount of DNS traffic you could easily cause if you handle an event that fires a lot (like the connection_established event).  You may want to do some name caching or restrictions for when the look ups are done.  We are also getting some evidence that overusing when statements is causing trouble for a few people.

Please when you are writing scripts, put them into a namespace (with "module MySpecialModule;" at the beginning of your script) to help us avoid stomping on identifier names that you are using.

Thanks,
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/




From jsiwek at illinois.edu  Fri Feb 28 07:49:35 2014
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Fri, 28 Feb 2014 15:49:35 +0000
Subject: [Bro] Question about Controller Framework
In-Reply-To: <EAC817F2-BF86-4DB4-9187-51757641C5E4@gmail.com>
References: <8237E570-E3F7-42CA-9DD6-67A611A67E22@gmail.com>
	<FBD8B140-8348-4CA6-932C-AA074E898446@illinois.edu>
	<EAC817F2-BF86-4DB4-9187-51757641C5E4@gmail.com>
Message-ID: <1754F885-B99F-405B-B5E8-2592AFD911DE@illinois.edu>


On Feb 27, 2014, at 5:15 PM, Alexander Tsankov <antsankov at gmail.com> wrote:

> Sounds great! I?m playing around with the global_ids() function, but I?m having the problem of getting too much information. My goal is to only identify global variables that are created in the script, but I am getting internal global variables that I never actually created. 

> I assume that there must be some table for this, otherwise BRO wouldn?t known when to spit back an error when you try to define the something twice.
> 
> Is there anything I can do about this information overload? 

I don?t think that location info (file & line number) is currently exposed for use in script-layer logic, but it is available internally (that?s why error messages are able to provide it).

I can think of a few methods to patch the functionality in.  One would be to change global_ids() to accept an optional argument to filter results to a particular script and/or have its return value provide the location information for each identifier.  Another would be to add a builtin-function specifically for retrieving location info given a single identifier.

Either seems like a reasonable request if you want to create a ticket at tracker.bro.org.  A patch would probably also be welcomed/accepted if you?re able/willing to try to make one.

- Jon


From robin at icir.org  Fri Feb 28 08:04:51 2014
From: robin at icir.org (Robin Sommer)
Date: Fri, 28 Feb 2014 08:04:51 -0800
Subject: [Bro] Mix different OSes as a workers in the same Bro cluster
In-Reply-To: <CAEjQA5Lvj6ob18FZ=b00e+OJ_jgn9HDMnuMeS0T2A+fVpF8xVg@mail.gmail.com>
References: <CAEjQA5Lvj6ob18FZ=b00e+OJ_jgn9HDMnuMeS0T2A+fVpF8xVg@mail.gmail.com>
Message-ID: <20140228160451.GA12442@icir.org>



On Fri, Feb 28, 2014 at 14:40 +0000, C. L. Martinez wrote:

> Is it possible to use different operating systems as a workers inside
> the same Bro cluster?

No, that's not supported at the moment. The OS must match on all
cluster systems.

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 *     robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 * www.icir.org/robin


From seth at icir.org  Fri Feb 28 09:21:14 2014
From: seth at icir.org (Seth Hall)
Date: Fri, 28 Feb 2014 12:21:14 -0500
Subject: [Bro] SMB Event Prototype Issue
In-Reply-To: <00D3CD29F7C24A44B4D23450BB8E55B30AB7C115@IMCMBX03.MITRE.ORG>
References: <C41581D1AA533F48AB7F9E17CE26D6DF529393A9@EXMB02.srn.sandia.gov>
	<00D3CD29F7C24A44B4D23450BB8E55B30AB7C115@IMCMBX03.MITRE.ORG>
Message-ID: <99CC886C-99D6-45DB-9CC5-9608AE1B8C1B@icir.org>


On Feb 28, 2014, at 10:41 AM, "McMahon, Kevin J" <kmcmahon at mitre.org> wrote:

> I?ve been interested in utilizing the SMB analyzer on Bro as well.  Where did you find Seth?s github data on SMB?
>  
> And, are you interested in collaborating on the porting of the SMB scripts?=

Unfortunately that branch is extremely broken.  

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140228/b67334df/attachment.bin 

From kmcmahon at mitre.org  Fri Feb 28 10:54:29 2014
From: kmcmahon at mitre.org (McMahon, Kevin J)
Date: Fri, 28 Feb 2014 18:54:29 +0000
Subject: [Bro] SMB Event Prototype Issue
In-Reply-To: <99CC886C-99D6-45DB-9CC5-9608AE1B8C1B@icir.org>
References: <C41581D1AA533F48AB7F9E17CE26D6DF529393A9@EXMB02.srn.sandia.gov>
	<00D3CD29F7C24A44B4D23450BB8E55B30AB7C115@IMCMBX03.MITRE.ORG>
	<99CC886C-99D6-45DB-9CC5-9608AE1B8C1B@icir.org>
Message-ID: <00D3CD29F7C24A44B4D23450BB8E55B30AB7C278@IMCMBX03.MITRE.ORG>

How broken is it?  I'm willing to start fixing it -- if it's a reasonably good starting point.

While I'm reasonably new to Bro, I have made quite a few mods to my FTP and HTTP scripts for local customization as well as a few minor mods in the analyzers for the same reasons (one of which is a bug that I've been negligent in supplying back to the community -- but I'm working on it).

Kevin


-----Original Message-----
From: Seth Hall [mailto:seth at icir.org] 
Sent: Friday, February 28, 2014 12:21 PM
To: McMahon, Kevin J
Cc: 'Pearson, David'; bro at bro.org
Subject: Re: [Bro] SMB Event Prototype Issue


On Feb 28, 2014, at 10:41 AM, "McMahon, Kevin J" <kmcmahon at mitre.org> wrote:

> I've been interested in utilizing the SMB analyzer on Bro as well.  Where did you find Seth's github data on SMB?
>  
> And, are you interested in collaborating on the porting of the SMB scripts?=

Unfortunately that branch is extremely broken.  

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/