From dcaldwell at colsa.com Tue Jan 2 10:17:21 2007 From: dcaldwell at colsa.com (David Caldwell) Date: Tue, 2 Jan 2007 12:17:21 -0600 Subject: [Bro] My reports are not updating Message-ID: <08853852-2120-4BC4-AD6D-8B8101CAA22D@colsa.com> Since I got everything working the system seems to be working fine with little exception. It seems that after several days of being run the reports are not updating. For example, I had reports on the 23rd, 24th, 25th (all on time as per the cron job setup). The 26th came up and created the report, but did not post it to email like it had in the previous 3 days. I manually ran the scripts to runt he report, and it generated a new report, and emailed it out. Now I have in the files 2 reports from the 26th, nothing from the 27th, 28th, a hand run from the 29th, and the same for today with the other days missing. I looked back into the crontab file and removed teh NICE function from the head of the command to generate reports. Is the NICE function necessary in a Linux system? We will see what the system does with the report for this evening, and I will update this one as I can figure out what is happening or can't as they case may very well be. If some one else has seen behavior like this please let me know what the fix was if you have managed to find one. TIA David Caldwell From dcaldwell at colsa.com Thu Jan 4 09:05:06 2007 From: dcaldwell at colsa.com (David Caldwell) Date: Thu, 4 Jan 2007 11:05:06 -0600 Subject: [Bro] Report generation not running properly Message-ID: <15CA6FE8-9F8F-499E-B6C3-C85F1F5FA2E4@colsa.com> It seems that if I leave Bro top its own devices I get copies of the previous days reports instead of a current report on the current day. If I force the report generation script to run by hand it will generate a new report, and send it. If I leave it alone afterward I get an updated report at midnight for that day. After that I get a copy of the same report....the previous days report. What do I need to post here to get some extra knowledgeable eyeballs to look at this? TIA David Caldwell From diogo_c at brturbo.com.br Fri Jan 5 06:30:49 2007 From: diogo_c at brturbo.com.br (Diogo Corteletti de Oliveira) Date: Fri, 05 Jan 2007 12:30:49 -0200 Subject: [Bro] Log Rotate Message-ID: <459E6119.504@brturbo.com.br> Hello guys, I'm new to this mailing list and i don't know if my question was asked before. But here it goes: I have configured the rotate-logs.bro policy to rotate logs every one hour. It's working for every log except the dns.bro logs. What can be the cause of that? Tks, Diogo Corteletti de Oliveira From robin at icir.org Mon Jan 8 16:31:56 2007 From: robin at icir.org (Robin Sommer) Date: Mon, 8 Jan 2007 16:31:56 -0800 Subject: [Bro] Dynamic Protocol Detection In-Reply-To: <1bb5dd90612231822mce522abt15d1718a36d53913@mail.gmail.com> References: <1bb5dd90612231822mce522abt15d1718a36d53913@mail.gmail.com> Message-ID: <20070109003156.GA12240@icir.org> On Sun, Dec 24, 2006 at 10:22 +0800, CS Lee wrote: > seems to be normal http session from 1.2.3.5 to 1.2.3.4. Thus I'm wondering > why it happens as if the http analyzer is disabled then the ids can be > evaded. Hard to say without seeing the actual packets. Can you send me the trace of that connection? > redef restrict_filters += [ ["cpanel2"] = "not (port 7777)" ]; > redef restrict_filters += [ ["cpanel3"] = "not (port 7778)" ]; [...] > redef restrict_filters += [ ["cpanel3"] = "not (port 7785)" ]; Assuming that this is not just a typo in the mail, you're using the same index twice (cpanel3), and therefore the second entry for that index overrides the first one, and you are going to see packets on port 7778. If you load print-filter.bro, you see what Bro's packet filter looks like. Note that another way for suppressing alerts for certain ports/servers is redefining ProtocolDetector::valids. See detect-protocols.bro for exampleas. > BRO_CREATE_TRACE_FILE=NO > # BRO_CREATE_TRACE_FILE=NO (Can't say much about this, but probably it's a small bug in the shell script.) Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Mon Jan 8 16:33:57 2007 From: robin at icir.org (Robin Sommer) Date: Mon, 8 Jan 2007 16:33:57 -0800 Subject: [Bro] Toggling traffic direction in reports/logs? In-Reply-To: References: Message-ID: <20070109003357.GB12240@icir.org> On Tue, Dec 26, 2006 at 10:45 -0600, Eric Wages wrote: > This is the exact opposite of what is the actual traffic pattern. Is > there a way that I can tell Bro that my /28 subnet is "local" and > everything else is "remote"? I don't seem to see anything like that > in the configuration files. You can redefine local_nets in site.bro. Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Mon Jan 8 17:19:06 2007 From: robin at icir.org (Robin Sommer) Date: Mon, 8 Jan 2007 17:19:06 -0800 Subject: [Bro] Log Rotate In-Reply-To: <459E6119.504@brturbo.com.br> References: <459E6119.504@brturbo.com.br> Message-ID: <20070109011906.GD13618@icir.org> On Fri, Jan 05, 2007 at 12:30 -0200, Diogo Corteletti de Oliveira wrote: > I have configured the rotate-logs.bro policy to rotate logs every one > hour. It's working for every log except the dns.bro logs. What can be > the cause of that? A bug. Can you try the attached patch to see if that helps? Thanks! Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org -------------- next part -------------- =================================================================== --- branches/robin/work/src/File.cc 2007-01-09 00:20:09 UTC (rev 3911) +++ branches/robin/work/src/File.cc 2007-01-09 01:17:47 UTC (rev 3912) @@ -42,6 +42,7 @@ #include "Timer.h" #include "Expr.h" #include "NetVar.h" +#include "Net.h" #include "Serializer.h" #include "Event.h" @@ -91,6 +92,9 @@ static BroFile* head = 0; static BroFile* tail = 0; +double BroFile::default_rotation_interval = 0; +double BroFile::default_rotation_size = 0; + // Maximizes the number of open file descriptors and returns the number // that we should use for the cache. static int maximize_num_fds() @@ -165,6 +169,10 @@ bool BroFile::Open(FILE* file) { + // Don't open any further files when we're shutting down. + if ( terminating ) + return false; + open_time = network_time ? network_time : current_time(); if ( ! max_files_in_cache ) @@ -176,6 +184,16 @@ f = file; + if ( default_rotation_interval && + (! attrs || ! attrs->FindAttr(ATTR_ROTATE_INTERVAL)) ) + rotate_interval = default_rotation_interval; + + if ( default_rotation_size && + (! attrs || ! attrs->FindAttr(ATTR_ROTATE_SIZE)) ) + rotate_size = default_rotation_size; + + InstallRotateTimer(); + if ( ! f ) { f = fopen(name, access); @@ -558,6 +576,9 @@ if ( ! (f->attrs && f->attrs->FindAttr(ATTR_ROTATE_SIZE)) ) f->rotate_size = max_size; } + + default_rotation_interval = interval; + default_rotation_size = max_size; } void BroFile::CloseCachedFiles() Modified: branches/robin/work/src/File.h =================================================================== --- branches/robin/work/src/File.h 2007-01-09 00:20:09 UTC (rev 3911) +++ branches/robin/work/src/File.h 2007-01-09 01:17:47 UTC (rev 3912) @@ -153,6 +153,9 @@ bool dont_rotate; // See InstallRotateTimer() bool print_hook; + static double default_rotation_interval; + static double default_rotation_size; + #ifdef USE_OPENSSL EVP_PKEY* pub_key; EVP_CIPHER_CTX* cipher_ctx; _______________________________________________ Bro-dev mailing list Bro-dev at george.lbl.gov http://dsd.lbl.gov/mailman/listinfo/bro-dev From diogo_c at brturbo.com.br Tue Jan 9 05:43:45 2007 From: diogo_c at brturbo.com.br (Diogo Corteletti de Oliveira) Date: Tue, 09 Jan 2007 11:43:45 -0200 Subject: [Bro] Log Rotate In-Reply-To: <20070109011906.GD13618@icir.org> References: <459E6119.504@brturbo.com.br> <20070109011906.GD13618@icir.org> Message-ID: <45A39C11.1030601@brturbo.com.br> Robin, I've aplied the patch, re-compiled and re-intalled bro but it didn't worked. Look bellow that every log rotated except the dns.log: -rw-r--r-- 1 bro wheel 0B Jan 9 09:56 alarm.bro.07-01-09_09.56.48 -rw-r--r-- 1 bro wheel 7.0K Jan 9 10:31 alarm.bro.07-01-09_09.58.34 -rw-r--r-- 1 bro wheel 90K Jan 9 09:57 conn.bro.07-01-09_09.56.48 -rw-r--r-- 1 bro wheel 7.2M Jan 9 10:38 conn.bro.07-01-09_09.58.34 -rw-r--r-- 1 bro wheel 6.9M Jan 9 10:38 dns.bro.07-01-09_09.58.34 -rw-r--r-- 1 bro wheel 5.0K Jan 9 09:57 ftp.bro.07-01-09_09.56.48 -rw-r--r-- 1 bro wheel 92K Jan 9 10:38 ftp.bro.07-01-09_09.58.34 -rw-r--r-- 1 bro wheel 57K Jan 9 09:57 http.bro.07-01-09_09.56.48 -rw-r--r-- 1 bro wheel 5.4M Jan 9 10:38 http.bro.07-01-09_09.58.34 -rw-r--r-- 1 bro wheel 548B Jan 9 09:57 info.bro.07-01-09_09.56.48 -rw-r--r-- 1 bro wheel 485B Jan 9 09:58 info.bro.07-01-09_09.58.34 -rw-r--r-- 1 bro wheel 0B Jan 9 09:56 irc.bro.07-01-09_09.56.48 -rw-r--r-- 1 bro wheel 0B Jan 9 09:58 irc.bro.07-01-09_09.58.34 -rw-r--r-- 1 bro wheel 790B Jan 9 09:57 notice.bro.07-01-09_09.56.48 -rw-r--r-- 1 bro wheel 52K Jan 9 10:38 notice.bro.07-01-09_09.58.34 -rw-r--r-- 1 bro wheel 0B Jan 9 09:56 software.bro.07-01-09_09.56.48 -rw-r--r-- 1 bro wheel 0B Jan 9 09:58 software.bro.07-01-09_09.58.34 -rw-r--r-- 1 bro wheel 0B Jan 9 09:57 step.bro.07-01-09_09.56.48 -rw-r--r-- 1 bro wheel 0B Jan 9 09:56 step.bro.07-01-09_09.56.48-07-01-09_09.56.48 -rw-r--r-- 1 bro wheel 0B Jan 9 09:58 step.bro.07-01-09_09.58.34 -rw-r--r-- 1 bro wheel 4.6K Jan 9 09:57 weird.bro.07-01-09_09.56.48 -rw-r--r-- 1 bro wheel 252K Jan 9 10:38 weird.bro.07-01-09_09.58.34 Tks Diogo Corteletti Robin Sommer escreveu: > On Fri, Jan 05, 2007 at 12:30 -0200, Diogo Corteletti de Oliveira wrote: > > >> I have configured the rotate-logs.bro policy to rotate logs every one >> hour. It's working for every log except the dns.bro logs. What can be >> the cause of that? >> > > A bug. Can you try the attached patch to see if that helps? Thanks! > > Robin > > > ------------------------------------------------------------------------ > > =================================================================== > --- branches/robin/work/src/File.cc 2007-01-09 00:20:09 UTC (rev 3911) > +++ branches/robin/work/src/File.cc 2007-01-09 01:17:47 UTC (rev 3912) > @@ -42,6 +42,7 @@ > #include "Timer.h" > #include "Expr.h" > #include "NetVar.h" > +#include "Net.h" > #include "Serializer.h" > #include "Event.h" > > @@ -91,6 +92,9 @@ > static BroFile* head = 0; > static BroFile* tail = 0; > > +double BroFile::default_rotation_interval = 0; > +double BroFile::default_rotation_size = 0; > + > // Maximizes the number of open file descriptors and returns the number > // that we should use for the cache. > static int maximize_num_fds() > @@ -165,6 +169,10 @@ > > bool BroFile::Open(FILE* file) > { > + // Don't open any further files when we're shutting down. > + if ( terminating ) > + return false; > + > open_time = network_time ? network_time : current_time(); > > if ( ! max_files_in_cache ) > @@ -176,6 +184,16 @@ > > f = file; > > + if ( default_rotation_interval && > + (! attrs || ! attrs->FindAttr(ATTR_ROTATE_INTERVAL)) ) > + rotate_interval = default_rotation_interval; > + > + if ( default_rotation_size && > + (! attrs || ! attrs->FindAttr(ATTR_ROTATE_SIZE)) ) > + rotate_size = default_rotation_size; > + > + InstallRotateTimer(); > + > if ( ! f ) > { > f = fopen(name, access); > @@ -558,6 +576,9 @@ > if ( ! (f->attrs && f->attrs->FindAttr(ATTR_ROTATE_SIZE)) ) > f->rotate_size = max_size; > } > + > + default_rotation_interval = interval; > + default_rotation_size = max_size; > } > > void BroFile::CloseCachedFiles() > > Modified: branches/robin/work/src/File.h > =================================================================== > --- branches/robin/work/src/File.h 2007-01-09 00:20:09 UTC (rev 3911) > +++ branches/robin/work/src/File.h 2007-01-09 01:17:47 UTC (rev 3912) > @@ -153,6 +153,9 @@ > bool dont_rotate; // See InstallRotateTimer() > bool print_hook; > > + static double default_rotation_interval; > + static double default_rotation_size; > + > #ifdef USE_OPENSSL > EVP_PKEY* pub_key; > EVP_CIPHER_CTX* cipher_ctx; > > _______________________________________________ > Bro-dev mailing list > Bro-dev at george.lbl.gov > http://dsd.lbl.gov/mailman/listinfo/bro-dev > From robin at icir.org Tue Jan 9 09:03:14 2007 From: robin at icir.org (Robin Sommer) Date: Tue, 9 Jan 2007 09:03:14 -0800 Subject: [Bro] Log Rotate In-Reply-To: <45A39C11.1030601@brturbo.com.br> References: <459E6119.504@brturbo.com.br> <20070109011906.GD13618@icir.org> <45A39C11.1030601@brturbo.com.br> Message-ID: <20070109170314.GC28187@icir.org> On Tue, Jan 09, 2007 at 11:43 -0200, Diogo Corteletti de Oliveira wrote: > I've aplied the patch, re-compiled and re-intalled bro but > it didn't worked. Look bellow that every log rotated except the dns.log: Hmm ... I did for me but I was using a development version of the code. I'll look into it. Thanks for trying it, Robin P.S.: The patch did apply cleanly, didn't it? -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From diogo_c at brturbo.com.br Tue Jan 9 09:11:35 2007 From: diogo_c at brturbo.com.br (Diogo Corteletti de Oliveira) Date: Tue, 09 Jan 2007 15:11:35 -0200 Subject: [Bro] Log Rotate In-Reply-To: <20070109170314.GC28187@icir.org> References: <459E6119.504@brturbo.com.br> <20070109011906.GD13618@icir.org> <45A39C11.1030601@brturbo.com.br> <20070109170314.GC28187@icir.org> Message-ID: <45A3CCC7.5030606@brturbo.com.br> Not at first!! I had to change the path for the two files for /usr/src/bro-1.1d/src/ and had to run patch with -l . =) Tks Diogo Corteletti Robin Sommer escreveu: > On Tue, Jan 09, 2007 at 11:43 -0200, Diogo Corteletti de Oliveira wrote: > > >> I've aplied the patch, re-compiled and re-intalled bro but >> it didn't worked. Look bellow that every log rotated except the dns.log: >> > > Hmm ... I did for me but I was using a development version of the > code. I'll look into it. > > Thanks for trying it, > > Robin > > P.S.: The patch did apply cleanly, didn't it? > > From diogo_c at brturbo.com.br Wed Jan 10 09:48:00 2007 From: diogo_c at brturbo.com.br (Diogo Corteletti de Oliveira) Date: Wed, 10 Jan 2007 15:48:00 -0200 Subject: [Bro] Another Doubt Message-ID: <45A526D0.4050104@brturbo.com.br> Hello Guys, I have another doubt about logs in bro. I have changed the following line in the rotate-logs.bro file: global date_format = "%y-%m-%d_%H.%M.%S" &redef; to global date_format = "%y%m%d%H%M%S" &redef; But the logs are still being created with the old format. I have searched all files in the policy directory for a redef of this variable but it doesn't exist. conn.bro.07-01-10_14.27.21 Thanks, Diogo Corteletti From christian at whoop.org Wed Jan 10 10:02:07 2007 From: christian at whoop.org (Christian Kreibich) Date: Wed, 10 Jan 2007 10:02:07 -0800 Subject: [Bro] Compiling Bro Under Solaris In-Reply-To: <1166617681.1785.142.camel@strangepork> References: <1166543028.458808b485268@staff-webmail.lboro.ac.uk> <1166617681.1785.142.camel@strangepork> Message-ID: <1168452127.14748.20.camel@strangepork> FYI, those issues were largely solved by using the 1.2 release, except for a minor Broccoli configuration issue for which I have already checked in a fix. Cheers, Christian. -- ________________________________________________________________________ http://www.cl.cam.ac.uk/~cpk25 http://www.whoop.org From vern at icir.org Wed Jan 10 13:01:47 2007 From: vern at icir.org (Vern Paxson) Date: Wed, 10 Jan 2007 13:01:47 -0800 Subject: [Bro] Another Doubt In-Reply-To: <45A526D0.4050104@brturbo.com.br> (Wed, 10 Jan 2007 15:48:00 -0200). Message-ID: <200701102101.l0AL1lFL078132@jaguar.icir.org> > I have another doubt about logs in bro. I have changed > the following line in the rotate-logs.bro file: > > global date_format = "%y-%m-%d_%H.%M.%S" &redef; > > to > > global date_format = "%y%m%d%H%M%S" &redef; > > > But the logs are still being created with the old format. This, plus the patch Robin sent you failing to fix the other problem, imply very strongly that you have some search path issues such that you're not executing the version of Bro that you think you are. One thing to do in this regard is run Bro and then use ls -lu to see whether the access times on the your modified files such as rotate-logs.bro change. If not, then you know they're not being read at run-time. Vern From robin at icir.org Thu Jan 11 09:45:56 2007 From: robin at icir.org (Robin Sommer) Date: Thu, 11 Jan 2007 09:45:56 -0800 Subject: [Bro] Log Rotate In-Reply-To: <20070109011906.GD13618@icir.org> References: <459E6119.504@brturbo.com.br> <20070109011906.GD13618@icir.org> Message-ID: <20070111174556.GA18575@icir.org> On Mon, Jan 08, 2007 at 17:19 -0800, I wrote: > A bug. Can you try the attached patch to see if that helps? Thanks! To follow up on this, it seems that the patch does actually solve the problem so we will put into the next Bro release. The other file name problem Diogo mentioned was unrelated: the bro.rc script defines a suffix, containing a timestamp, which is appended to all names of log files. On rotation, the files are then renamed as defined in rotate-logs.bro. Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From christian.novello at gmail.com Mon Jan 15 06:59:02 2007 From: christian.novello at gmail.com (Christian Novello) Date: Mon, 15 Jan 2007 15:59:02 +0100 Subject: [Bro] Application Layer Classification Message-ID: <339de9bb0701150659l151dfd95i3718850192e63374@mail.gmail.com> Dear all, here at Turin Polytechnic (Italy) we're working with Bro 1.2.1 and we're having some trouble in classifying packets that do not use a standard port. Unfortunately, a large part of our traffic does not belong to standard ports and therefore the validity of results we get from Bro are rather limited. Is there any way to let Bro recognize any HTTP session (for example) even if it does not have port 80 or 8080 or such? And... is it possible to generalize this behavior on any protocol? (Obviously, we can also modify the code; we should be extremely grateful if we can provide us some hints, just to start). Cheers, Christian -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070115/a293909a/attachment.html From jp.luiggi at free.fr Mon Jan 15 08:25:33 2007 From: jp.luiggi at free.fr (Jean-Philippe Luiggi) Date: Mon, 15 Jan 2007 11:25:33 -0500 Subject: [Bro] Application Layer Classification In-Reply-To: <339de9bb0701150659l151dfd95i3718850192e63374@mail.gmail.com> References: <339de9bb0701150659l151dfd95i3718850192e63374@mail.gmail.com> Message-ID: <20070115162533.GA15000@armada.mynetwork.local> Hello Christian, As far i know, Bro's able to catch this problem but you need to use the "DPD.bro" module. >From brolite.bro : ==== ## Dynamic Protocol Detection configuration # # This is off by default, as it requires a more powerful Bro host. # Uncomment next line to activate. # const use_dpd = T; @ifdef ( use_dpd ) @load dpd @load irc-bot ==== Just uncomment "const use_dpd = T;" and you'll get it (it works as is at home) Best regards. On Mon, Jan 15, 2007 at 03:59:02PM +0100, Christian Novello wrote: > Dear all, > > here at Turin Polytechnic (Italy) we're working with Bro 1.2.1 and we're > having some trouble in classifying packets that do not use a standard port. > Unfortunately, a large part of our traffic does not belong to standard ports > and therefore the validity of results we get from Bro are rather limited. > > Is there any way to let Bro recognize any HTTP session (for example) even if > it does not have port 80 or 8080 or such? And... is it possible to > generalize this behavior on any protocol? > > (Obviously, we can also modify the code; we should be extremely grateful if > we can provide us some hints, just to start). > > Cheers, > > Christian > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From christian at whoop.org Mon Jan 15 11:13:09 2007 From: christian at whoop.org (Christian Kreibich) Date: Mon, 15 Jan 2007 11:13:09 -0800 Subject: [Bro] Application Layer Classification In-Reply-To: <339de9bb0701150659l151dfd95i3718850192e63374@mail.gmail.com> References: <339de9bb0701150659l151dfd95i3718850192e63374@mail.gmail.com> Message-ID: <1168888389.20195.89.camel@strangepork> Hi Christian, On Mon, 2007-01-15 at 15:59 +0100, Christian Novello wrote: > Is there any way to let Bro recognize any HTTP session (for example) > even if it does not have port 80 or 8080 or such? And... is it > possible to generalize this behavior on any protocol? yes, see http://www.bro-ids.org/wiki/index.php/DynamicProtocolDetection for the details. Cheers, Christian. -- ________________________________________________________________________ http://www.cl.cam.ac.uk/~cpk25 http://www.whoop.org From geek00l at gmail.com Mon Jan 15 17:22:40 2007 From: geek00l at gmail.com (CS Lee) Date: Tue, 16 Jan 2007 09:22:40 +0800 Subject: [Bro] Bro Digest, Vol 9, Issue 7 In-Reply-To: References: Message-ID: <1bb5dd90701151722q39b28ef6teb40c7c4fa728765@mail.gmail.com> Christian, As far as I know, dynamic protocol detection is the answer, while the signatures right now are limited(you can add it anytime), I have it detected http traffics on non standard ports. Cheers. -- Best Regards, CS Lee -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070116/794fd164/attachment.html From ju at klipix.org Mon Jan 15 17:32:38 2007 From: ju at klipix.org (Julien Desfossez) Date: Mon, 15 Jan 2007 20:32:38 -0500 Subject: [Bro] IPv6 support in Bro Message-ID: <45AC2B36.3050803@klipix.org> Hello, I would like to contribute to the IPv6 part of Bro development. So I would like to know what is roadmap in this area and how I could help. Cheers, Julien From ml at grid.einherjar.de Tue Jan 16 01:53:02 2007 From: ml at grid.einherjar.de (Thorolf) Date: Tue, 16 Jan 2007 10:53:02 +0100 Subject: [Bro] Bro is running, but ... Message-ID: <45ACA07E.8060207@grid.einherjar.de> Hello everyone, and happy new year! I am observing some wired things regarding to bro. fw1-net1# /usr/local/etc/rc.d/bro.sh checkpoint bro.rc: Beginning the checkpoint process bro.rc: No current instance of Bro is running. fw1-net1# ps -aux | grep bro root 157 0.0 0.1 1776 1124 ?? I Mon12AM 0:00.01 /bin/sh /usr/local/bro/etc/bro.rc start root 165 0.0 3.5 40340 36556 ?? S Mon12AM 42:12.20 /usr/local/bro/bin/bro -W -i re1 local.site.bro I have to kill the bro process and start it again. I'm running bro 1.1c on FreeBSD 6.2-PRERELEASE. We have custom rules which react to events using system(), and calling pfctl to extend specific tables in the firewall ruleset. Everything is working fine, but time to time, lets say one time a week, bro doesn't react as expected. We have logfiles that events ware there but tables are not extended to orign IP addresses. Does anyone knows what can be wrong or maybe someone observed the same behavior? The custom site-rule isn't different from conn.bro just triggered on specific traffic. Regards, /rl From M9315927 at mail.ntust.edu.tw Wed Jan 17 00:26:24 2007 From: M9315927 at mail.ntust.edu.tw (=?big5?B?sWm/QaXN?=) Date: Wed, 17 Jan 2007 16:26:24 +0800 Subject: [Bro] About Bro drop packet.... Message-ID: <000e01c73a11$2d2d5870$939b768c@gitadmlab> My simulate environment below : Bro 1.2, load local.lite.bro, run in linux fedora 5, Bro`s ip is 192.168.0.1, and replay tcpdump file`s machine is 192.168.0.3. I use tcpreplay to replay Darpa 2000 LLDOS 1.0 DMZ dumpfile to Bro`s machine in real close network. My question is: In info.localhost.06-12-27_13.16.39 file , I find a lots of packets be droped, why? is it right? if not, and how to improve it? Thanks your help!! Gita in NTUST tcpreplay command is below: tcpreplay LLDOS_1.0_dump_file -i 192.168.0.3 info.localhost.06-12-27_13.16.39 file content below ---------------------------------------------------------------- /usr/local/bro/policy/scan.bro, line 92: warning: no such host: j5004.inktomisearch.com /usr/local/bro/policy/scan.bro, line 92: warning: no such host: j5005.inktomisearch.com /usr/local/bro/policy/scan.bro, line 93: warning: no such host: j5006.inktomisearch.com /usr/local/bro/policy/scan.bro, line 93: warning: no such host: j100.inktomi.com /usr/local/bro/policy/scan.bro, line 93: warning: no such host: j101.inktomi.com /usr/local/bro/policy/scan.bro, line 94: warning: no such host: j3002.inktomi.com /usr/local/bro/policy/scan.bro, line 94: warning: no such host: si3000.inktomi.com /usr/local/bro/policy/scan.bro, line 94: warning: no such host: si3001.inktomi.com /usr/local/bro/policy/scan.bro, line 95: warning: no such host: si3002.inktomi.com /usr/local/bro/policy/scan.bro, line 95: warning: no such host: si3003.inktomi.com /usr/local/bro/policy/scan.bro, line 95: warning: no such host: si4000.inktomi.com /usr/local/bro/policy/scan.bro, line 96: warning: no such host: si4001.inktomi.com /usr/local/bro/policy/scan.bro, line 96: warning: no such host: si4002.inktomi.com /usr/local/bro/policy/scan.bro, line 96: warning: no such host: wm3018.inktomi.com /usr/local/bro/policy/scan.bro, line 99: warning: no such host: test-scooter.av.pa-x.dec.com /usr/local/bro/policy/scan.bro, line 130: warning: no such host: a.root-servers.net /usr/local/bro/policy/scan.bro, line 130: warning: no such host: b.root-servers.net /usr/local/bro/policy/scan.bro, line 130: warning: no such host: c.root-servers.net /usr/local/bro/policy/scan.bro, line 131: warning: no such host: d.root-servers.net /usr/local/bro/policy/scan.bro, line 131: warning: no such host: e.root-servers.net /usr/local/bro/policy/scan.bro, line 131: warning: no such host: f.root-servers.net /usr/local/bro/policy/scan.bro, line 132: warning: no such host: g.root-servers.net /usr/local/bro/policy/scan.bro, line 132: warning: no such host: h.root-servers.net /usr/local/bro/policy/scan.bro, line 132: warning: no such host: i.root-servers.net /usr/local/bro/policy/scan.bro, line 133: warning: no such host: j.root-servers.net /usr/local/bro/policy/scan.bro, line 133: warning: no such host: k.root-servers.net /usr/local/bro/policy/scan.bro, line 133: warning: no such host: l.root-servers.net /usr/local/bro/policy/scan.bro, line 134: warning: no such host: m.root-servers.net /usr/local/bro/policy/scan.bro, line 138: warning: no such host: a.gtld-servers.net /usr/local/bro/policy/scan.bro, line 138: warning: no such host: b.gtld-servers.net /usr/local/bro/policy/scan.bro, line 138: warning: no such host: c.gtld-servers.net /usr/local/bro/policy/scan.bro, line 139: warning: no such host: d.gtld-servers.net /usr/local/bro/policy/scan.bro, line 139: warning: no such host: e.gtld-servers.net /usr/local/bro/policy/scan.bro, line 139: warning: no such host: f.gtld-servers.net /usr/local/bro/policy/scan.bro, line 140: warning: no such host: g.gtld-servers.net /usr/local/bro/policy/scan.bro, line 140: warning: no such host: h.gtld-servers.net /usr/local/bro/policy/scan.bro, line 140: warning: no such host: i.gtld-servers.net /usr/local/bro/policy/scan.bro, line 141: warning: no such host: j.gtld-servers.net /usr/local/bro/policy/scan.bro, line 141: warning: no such host: k.gtld-servers.net /usr/local/bro/policy/scan.bro, line 141: warning: no such host: l.gtld-servers.net /usr/local/bro/policy/scan.bro, line 142: warning: no such host: m.gtld-servers.net /usr/local/bro/policy/ftp.bro, line 74: warning: no such host: gvaona1.cns.hp.com /usr/local/bro/policy/portmapper.bro, line 146: warning: no such host: sun-rpc.mcast.net listening on eth0 Bro Version: 1.2 Started with the following command line options: -W -i eth0 local.lite.bro Capture filter: ((((((((((((((((((((((port ftp) or (port 143)) or (port 111)) or (udp port 69)) or (port 6666)) or (tcp[2:2] > 32770 and tcp[2:2] < 32901 and tcp[0:2] != 80 and tcp[0:2] != 22 and tcp[0:2] != 139)) or ( icmp)) or (port 512 or port 513 or port 515)) or (port ftp)) or (port telnet or tcp port 513)) or (port smtp)) or (tcp port 80 or tcp port 8080 or tcp port 8000 or tcp port 8001)) or (port smtp)) or ((ip[6:2] & 0x3fff != 0) and tcp)) or (tcp dst port 80 or tcp dst port 8080 or tcp dst port 8000)) or (dst port 135 or dst port 137 or dst port 139 or dst port 445)) or (port telnet)) or (port 161 or port 162)) or (port 53)) or (port 6667)) or (port 111)) or (tcp[13] & 7 != 0)) or (tcp src port 80 or tcp src port 8080 or tcp src port 8000) 1168837833.287204 received termination signal 334036 packets received on interface eth0, 12343464 dropped -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070117/a4f96952/attachment.html From robin at icir.org Wed Jan 17 08:53:17 2007 From: robin at icir.org (Robin Sommer) Date: Wed, 17 Jan 2007 08:53:17 -0800 Subject: [Bro] About Bro drop packet.... In-Reply-To: <000e01c73a11$2d2d5870$939b768c@gitadmlab> References: <000e01c73a11$2d2d5870$939b768c@gitadmlab> Message-ID: <20070117165317.GA11449@icir.org> On Wed, Jan 17, 2007 at 16:26 +0800, ?i?A?? wrote: > In info.localhost.06-12-27_13.16.39 file , I find a lots of packets be droped, why? is it right? Well, it's not "right" in the sense that naturally you don't want to drop any packets at all. Drops occur if the system cannot keep up with the packet stream, which can have various reasons. > if not, and how to improve it? What kind of system are you using? In general, Linux is not the best choice for packet capturing, yet you might be able to improve performance somewhat with a little bit of tweaking, see http://www.net.t-labs.tu-berlin.de/research/bpcs Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From bltierney at lbl.gov Wed Jan 17 09:07:52 2007 From: bltierney at lbl.gov (Brian Tierney) Date: Wed, 17 Jan 2007 09:07:52 -0800 Subject: [Bro] Bro is running, but ... In-Reply-To: <45ACA07E.8060207@grid.einherjar.de> References: <45ACA07E.8060207@grid.einherjar.de> Message-ID: <45AE57E8.9000603@lbl.gov> Have you set the env variable BROHOME? This script looks for PID of the current bro to kill by looking at BRO_RUNTIME_DIR, which is defined in $BROHOME/etc/bro.cfg Thorolf wrote: > Hello everyone, > and happy new year! > > I am observing some wired things regarding to bro. > > fw1-net1# /usr/local/etc/rc.d/bro.sh checkpoint > bro.rc: Beginning the checkpoint process > bro.rc: No current instance of Bro is running. > > fw1-net1# ps -aux | grep bro > root 157 0.0 0.1 1776 1124 ?? I Mon12AM 0:00.01 /bin/sh > /usr/local/bro/etc/bro.rc start > root 165 0.0 3.5 40340 36556 ?? S Mon12AM 42:12.20 > /usr/local/bro/bin/bro -W -i re1 local.site.bro > > I have to kill the bro process and start it again. > I'm running bro 1.1c on FreeBSD 6.2-PRERELEASE. > > We have custom rules which react to events using system(), and calling > pfctl to extend specific tables in the firewall ruleset. Everything is > working fine, but time to time, lets say one time a week, bro doesn't > react as expected. We have logfiles that events ware there but tables > are not extended to orign IP addresses. > > Does anyone knows what can be wrong or maybe someone observed the same > behavior? > > The custom site-rule isn't different from conn.bro just triggered on > specific traffic. > > Regards, > /rl > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- ------------------------------------------------------------------------ Brian L. Tierney, Lawrence Berkeley National Laboratory (LBNL) 1 Cyclotron Rd. MS: 50B-2239, Berkeley, CA 94720 tel: 510-486-7381 fax: 510-495-2998 efax: 425-642-4558 bltierney at lbl.gov http://www-didc.lbl.gov/~tierney ------------------------------------------------------------------------ From jp.luiggi at free.fr Wed Jan 17 10:06:41 2007 From: jp.luiggi at free.fr (Jean-Philippe Luiggi) Date: Wed, 17 Jan 2007 13:06:41 -0500 Subject: [Bro] About Bro drop packet.... In-Reply-To: <000e01c73a11$2d2d5870$939b768c@gitadmlab> References: <000e01c73a11$2d2d5870$939b768c@gitadmlab> Message-ID: <20070117180640.GA5085@armada.mynetwork.local> Hello, Just a question, what's your hardware ? Just check at http://bro-ids.org/Bro-user-manual/Network-Tap.html#Network-Tap if you've the requirements needed. Depending on your configuration, it may be difficult for your IDS to follow the network's stream. Best regards. On Wed, Jan 17, 2007 at 04:26:24PM +0800, ?i?A?? wrote: > My simulate environment below : > Bro 1.2, load local.lite.bro, run in linux fedora 5, Bro`s ip is 192.168.0.1, and replay tcpdump file`s machine is 192.168.0.3. > > I use tcpreplay to replay Darpa 2000 LLDOS 1.0 DMZ dumpfile to Bro`s machine in real close network. > > My question is: > In info.localhost.06-12-27_13.16.39 file , I find a lots of packets be droped, why? is it right? if not, and how to improve it? > > Thanks your help!! > > Gita in NTUST > > tcpreplay command is below: > tcpreplay LLDOS_1.0_dump_file -i 192.168.0.3 > > info.localhost.06-12-27_13.16.39 file content below > ---------------------------------------------------------------- > /usr/local/bro/policy/scan.bro, line 92: warning: no such host: j5004.inktomisearch.com > /usr/local/bro/policy/scan.bro, line 92: warning: no such host: j5005.inktomisearch.com > /usr/local/bro/policy/scan.bro, line 93: warning: no such host: j5006.inktomisearch.com > /usr/local/bro/policy/scan.bro, line 93: warning: no such host: j100.inktomi.com > /usr/local/bro/policy/scan.bro, line 93: warning: no such host: j101.inktomi.com > /usr/local/bro/policy/scan.bro, line 94: warning: no such host: j3002.inktomi.com > /usr/local/bro/policy/scan.bro, line 94: warning: no such host: si3000.inktomi.com > /usr/local/bro/policy/scan.bro, line 94: warning: no such host: si3001.inktomi.com > /usr/local/bro/policy/scan.bro, line 95: warning: no such host: si3002.inktomi.com > /usr/local/bro/policy/scan.bro, line 95: warning: no such host: si3003.inktomi.com > /usr/local/bro/policy/scan.bro, line 95: warning: no such host: si4000.inktomi.com > /usr/local/bro/policy/scan.bro, line 96: warning: no such host: si4001.inktomi.com > /usr/local/bro/policy/scan.bro, line 96: warning: no such host: si4002.inktomi.com > /usr/local/bro/policy/scan.bro, line 96: warning: no such host: wm3018.inktomi.com > /usr/local/bro/policy/scan.bro, line 99: warning: no such host: test-scooter.av.pa-x.dec.com > /usr/local/bro/policy/scan.bro, line 130: warning: no such host: a.root-servers.net > /usr/local/bro/policy/scan.bro, line 130: warning: no such host: b.root-servers.net > /usr/local/bro/policy/scan.bro, line 130: warning: no such host: c.root-servers.net > /usr/local/bro/policy/scan.bro, line 131: warning: no such host: d.root-servers.net > /usr/local/bro/policy/scan.bro, line 131: warning: no such host: e.root-servers.net > /usr/local/bro/policy/scan.bro, line 131: warning: no such host: f.root-servers.net > /usr/local/bro/policy/scan.bro, line 132: warning: no such host: g.root-servers.net > /usr/local/bro/policy/scan.bro, line 132: warning: no such host: h.root-servers.net > /usr/local/bro/policy/scan.bro, line 132: warning: no such host: i.root-servers.net > /usr/local/bro/policy/scan.bro, line 133: warning: no such host: j.root-servers.net > /usr/local/bro/policy/scan.bro, line 133: warning: no such host: k.root-servers.net > /usr/local/bro/policy/scan.bro, line 133: warning: no such host: l.root-servers.net > /usr/local/bro/policy/scan.bro, line 134: warning: no such host: m.root-servers.net > /usr/local/bro/policy/scan.bro, line 138: warning: no such host: a.gtld-servers.net > /usr/local/bro/policy/scan.bro, line 138: warning: no such host: b.gtld-servers.net > /usr/local/bro/policy/scan.bro, line 138: warning: no such host: c.gtld-servers.net > /usr/local/bro/policy/scan.bro, line 139: warning: no such host: d.gtld-servers.net > /usr/local/bro/policy/scan.bro, line 139: warning: no such host: e.gtld-servers.net > /usr/local/bro/policy/scan.bro, line 139: warning: no such host: f.gtld-servers.net > /usr/local/bro/policy/scan.bro, line 140: warning: no such host: g.gtld-servers.net > /usr/local/bro/policy/scan.bro, line 140: warning: no such host: h.gtld-servers.net > /usr/local/bro/policy/scan.bro, line 140: warning: no such host: i.gtld-servers.net > /usr/local/bro/policy/scan.bro, line 141: warning: no such host: j.gtld-servers.net > /usr/local/bro/policy/scan.bro, line 141: warning: no such host: k.gtld-servers.net > /usr/local/bro/policy/scan.bro, line 141: warning: no such host: l.gtld-servers.net > /usr/local/bro/policy/scan.bro, line 142: warning: no such host: m.gtld-servers.net > /usr/local/bro/policy/ftp.bro, line 74: warning: no such host: gvaona1.cns.hp.com > /usr/local/bro/policy/portmapper.bro, line 146: warning: no such host: sun-rpc.mcast.net > listening on eth0 > Bro Version: 1.2 > Started with the following command line options: -W -i eth0 local.lite.bro > Capture filter: ((((((((((((((((((((((port ftp) or (port 143)) or (port 111)) or (udp port 69)) or (port 6666)) or (tcp[2:2] > 32770 and tcp[2:2] < 32901 and tcp[0:2] != 80 and tcp[0:2] != 22 and tcp[0:2] != 139)) or ( icmp)) or (port 512 or port 513 or port 515)) or (port ftp)) or (port telnet or tcp port 513)) or (port smtp)) or (tcp port 80 or tcp port 8080 or tcp port 8000 or tcp port 8001)) or (port smtp)) or ((ip[6:2] & 0x3fff != 0) and tcp)) or (tcp dst port 80 or tcp dst port 8080 or tcp dst port 8000)) or (dst port 135 or dst port 137 or dst port 139 or dst port 445)) or (port telnet)) or (port 161 or port 162)) or (port 53)) or (port 6667)) or (port 111)) or (tcp[13] & 7 != 0)) or (tcp src port 80 or tcp src port 8080 or tcp src port 8000) > 1168837833.287204 received termination signal > 334036 packets received on interface eth0, 12343464 dropped > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From robin at icir.org Wed Jan 17 13:50:37 2007 From: robin at icir.org (Robin Sommer) Date: Wed, 17 Jan 2007 13:50:37 -0800 Subject: [Bro] IPv6 support in Bro In-Reply-To: <45AC2B36.3050803@klipix.org> References: <45AC2B36.3050803@klipix.org> Message-ID: <20070117215037.GC12630@icir.org> On Mon, Jan 15, 2007 at 20:32 -0500, Julien Desfossez wrote: > I would like to contribute to the IPv6 part of Bro development. > So I would like to know what is roadmap in this area and how I could help. We don't have any roadmap but you can already compile in IPv6 support by configuring with --enable-brov6. What specificially would you like to see? Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From ju at klipix.org Wed Jan 17 15:19:45 2007 From: ju at klipix.org (Julien Desfossez) Date: Wed, 17 Jan 2007 18:19:45 -0500 Subject: [Bro] IPv6 support in Bro In-Reply-To: <20070117215037.GC12630@icir.org> References: <45AC2B36.3050803@klipix.org> <20070117215037.GC12630@icir.org> Message-ID: <45AEAF11.6000106@klipix.org> I want to improve Bro IPv6 support, and I'm just wondering where to start (header fields, extensions, ICMPv6...). Thanks Julien Robin Sommer a ?crit : > On Mon, Jan 15, 2007 at 20:32 -0500, Julien Desfossez wrote: > > >> I would like to contribute to the IPv6 part of Bro development. >> So I would like to know what is roadmap in this area and how I could help. >> > > We don't have any roadmap but you can already compile in IPv6 > support by configuring with --enable-brov6. > > What specificially would you like to see? > > Robin > > From ml at grid.einherjar.de Thu Jan 18 00:41:46 2007 From: ml at grid.einherjar.de (Thorolf) Date: Thu, 18 Jan 2007 09:41:46 +0100 Subject: [Bro] Bro is running, but ... In-Reply-To: <45AE57E8.9000603@lbl.gov> References: <45ACA07E.8060207@grid.einherjar.de> <45AE57E8.9000603@lbl.gov> Message-ID: <45AF32CA.6060808@grid.einherjar.de> Brian Tierney wrote: > Have you set the env variable BROHOME? > > This script looks for PID of the current bro to kill by looking > at BRO_RUNTIME_DIR, which is defined in $BROHOME/etc/bro.cfg > Of course, everything is working fine for few days and then happens what I described in my last email. fw1-net1# ./bro.sh status Bro is running (pid: 66617) Autorestart: ON Running since: Thu Jan 18 04:03:04 UTC 2007 Bro Version: 1.1c Active log suffix: fw1-net1.07-01-18_04.03.01 I'm now restarting bro using cron for test once a day. thx, /rl From robin at icir.org Thu Jan 18 15:44:32 2007 From: robin at icir.org (Robin Sommer) Date: Thu, 18 Jan 2007 15:44:32 -0800 Subject: [Bro] IPv6 support in Bro In-Reply-To: <45AEAF11.6000106@klipix.org> References: <45AC2B36.3050803@klipix.org> <20070117215037.GC12630@icir.org> <45AEAF11.6000106@klipix.org> Message-ID: <20070118234432.GA23713@icir.org> On Wed, Jan 17, 2007 at 18:19 -0500, Julien Desfossez wrote: > I want to improve Bro IPv6 support, and I'm just wondering where to > start (header fields, extensions, ICMPv6...). As far as I know there's nobody currently working on something along these lines. I guess the best approach for you might be to follow the flow of a packet through Bro's core, starting with the NetSessions::NextPacket() method in Sessions.cc, which is the main entry point for packets. Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From ju at klipix.org Fri Jan 19 07:41:21 2007 From: ju at klipix.org (Julien Desfossez) Date: Fri, 19 Jan 2007 10:41:21 -0500 Subject: [Bro] IPv6 support in Bro In-Reply-To: <20070118234432.GA23713@icir.org> References: <45AC2B36.3050803@klipix.org> <20070117215037.GC12630@icir.org> <45AEAF11.6000106@klipix.org> <20070118234432.GA23713@icir.org> Message-ID: <45B0E6A1.5060706@klipix.org> OK thank you, I start working on it. Julien Robin Sommer a ?crit : > On Wed, Jan 17, 2007 at 18:19 -0500, Julien Desfossez wrote: > > >> I want to improve Bro IPv6 support, and I'm just wondering where to >> start (header fields, extensions, ICMPv6...). >> > > As far as I know there's nobody currently working on something along > these lines. I guess the best approach for you might be to follow > the flow of a packet through Bro's core, starting with the > NetSessions::NextPacket() method in Sessions.cc, which is the main > entry point for packets. > > Robin > > From ducha at cse.buffalo.edu Mon Jan 22 13:55:04 2007 From: ducha at cse.buffalo.edu (Duc T Ha) Date: Mon, 22 Jan 2007 16:55:04 -0500 Subject: [Bro] Traffic characteristics extraction with Bro Message-ID: <45B532B8.9060406@cse.buffalo.edu> Hi, I am trying to extract some flow characteristics from static data with Bro. I've checked the analyzer Conn.bro, but didn't find any suitable information. At present, the characteristics I need are: mean packet size and mean packet inter-arrival time, all per flow. Future work may require packet related information, also per flow. Does anybody know how to do this ? Bro's manual doesn't provide much information about static traffic analysis. Where should I start and what should I do now? I am also quite new with Bro, so any detail is appreciated. Thanks in advance, Duc. -------------- next part -------------- A non-text attachment was scrubbed... Name: ducha.vcf Type: text/x-vcard Size: 263 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070122/30c7287c/attachment.vcf From ducha at cse.buffalo.edu Tue Jan 23 17:09:52 2007 From: ducha at cse.buffalo.edu (Duc T Ha) Date: Tue, 23 Jan 2007 20:09:52 -0500 Subject: [Bro] (Traffic characteristics extraction with Bro - Continue) Message-ID: <45B6B1E0.8030208@cse.buffalo.edu> Dear all, I have looked at some files while still waiting for some hints for my previous question. Then I found the event "new_packet" , which looks promising. However, I DON"T know if the packet reported there is already "processed", i.e only valid packets for the connection are considered while others such as: duplicate or out of order packets are not reported to the handler? Does anybody know how this event works ? I REALLY need some answers to go on. Thanks, Duc. -------------- next part -------------- A non-text attachment was scrubbed... Name: ducha.vcf Type: text/x-vcard Size: 263 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070123/c50ad2b0/attachment.vcf From robin at icir.org Tue Jan 23 20:47:16 2007 From: robin at icir.org (Robin Sommer) Date: Tue, 23 Jan 2007 20:47:16 -0800 Subject: [Bro] Traffic characteristics extraction with Bro In-Reply-To: <45B6B1E0.8030208@cse.buffalo.edu> <45B532B8.9060406@cse.buffalo.edu> References: <45B6B1E0.8030208@cse.buffalo.edu> <45B532B8.9060406@cse.buffalo.edu> Message-ID: <20070124044716.GB10910@icir.org> On Mon, Jan 22, 2007 at 16:55 -0500, you wrote: > At present, the characteristics I need are: mean packet size and mean > packet inter-arrival time, all per flow. In general, such statistics are a little bit difficult to get with Bro because its analysis works primarily with higher abstractions of activity, i.e., you write scripts without relying on packet semantics. To still get stats such us inter-arrival times there are two options: (i) you can indeed use the new_packet() event which you already found yourself. Disadvantage: this is not very efficient because script code needs to be executed for each input packet. (ii) you add a module to Bro's core which does the required analysis and just reports results back to script-level. Bro's stepping stone analyzer is an example of how to do this (it does examine packet timings). Disadvantage: you need to hack Bro's source code. On Tue, Jan 23, 2007 at 20:09 -0500, you wrote: > However, I DON"T know if the packet reported there is already > "processed", i.e only valid packets for the connection are considered > while others such as: duplicate or out of order packets are not reported > to the handler? The event is raised for all packets which pass some very basic sanity checks, such as being fully captured and having correct checksums. So, this should indeed work for you. Hope this helps, Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From dick at uchicago.edu Wed Jan 24 07:03:27 2007 From: dick at uchicago.edu (Jacob Yocom-Piatt) Date: Wed, 24 Jan 2007 09:03:27 -0600 (CST) Subject: [Bro] web interface Message-ID: <20070124090327.AJP96010@m4500-00.uchicago.edu> before i get my hands dirty and start installing and configuring bro, i want to know if there is a web interface from which to view alerts, graphs, etc. it is not critical that i have this sort of interface, although it would be nice. if there are any known problems running bro on openbsd, do tell. cheers, jake From jp.luiggi at free.fr Wed Jan 24 08:35:52 2007 From: jp.luiggi at free.fr (Jean-Philippe Luiggi) Date: Wed, 24 Jan 2007 11:35:52 -0500 Subject: [Bro] web interface In-Reply-To: <20070124090327.AJP96010@m4500-00.uchicago.edu> References: <20070124090327.AJP96010@m4500-00.uchicago.edu> Message-ID: <20070124163552.GA20536@armada.mynetwork.local> Hello, I'm not sure about the web interface but speaking of OpenBSD vs Bro, it works. Not directly (build problems) if you use the official .tar.gz but with some patchs, it works very well. Best regards. On Wed, Jan 24, 2007 at 09:03:27AM -0600, Jacob Yocom-Piatt wrote: > before i get my hands dirty and start installing and configuring bro, i > want to know if there is a web interface from which to view alerts, > graphs, etc. it is not critical that i have this sort of interface, > although it would be nice. > > if there are any known problems running bro on openbsd, do tell. > > cheers, > jake > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From stephen.lau at ucsf.edu Wed Jan 24 09:11:48 2007 From: stephen.lau at ucsf.edu (Stephen Lau) Date: Wed, 24 Jan 2007 09:11:48 -0800 Subject: [Bro] web interface In-Reply-To: <20070124090327.AJP96010@m4500-00.uchicago.edu> References: <20070124090327.AJP96010@m4500-00.uchicago.edu> Message-ID: <45B79354.4010402@ucsf.edu> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jacob Yocom-Piatt wrote: > before i get my hands dirty and start installing and configuring bro, i > want to know if there is a web interface from which to view alerts, > graphs, etc. it is not critical that i have this sort of interface, > although it would be nice. > Mike Haberman of NCSA has developed a NetLens web/Java interface to some of the Bro logs. It's still under development however it has seen some operational use. Steve - -- +--------------------------------------------------------------------- Stephen Lau - Stephen.Lau at ucsf.edu Information Security Policy and Program Manager University of California, San Francisco 1855 Folsom, Suite 602, Box 0707, San Francisco, CA 94143 +1(415) 476-3106 (Work) +1(415) 476-1717 (Fax) PGP: 44C8 C9CB C15E 2AE1 7B0A 544E 9A04 AB2B F63F 748B +--------------------------------------------------------------------- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (MingW32) iD8DBQFFt5NUmgSrK/Y/dIsRAq+AAKCh1L7ajhqbeaiqzz2FFUbfBUPUdwCdF20Y dM7t1/WibeOEyHTR/WxJKcg= =kj+e -----END PGP SIGNATURE----- From vern at icir.org Thu Jan 25 15:20:52 2007 From: vern at icir.org (Vern Paxson) Date: Thu, 25 Jan 2007 15:20:52 -0800 Subject: [Bro] Traffic characteristics extraction with Bro In-Reply-To: <45B532B8.9060406@cse.buffalo.edu> (Mon, 22 Jan 2007 16:55:04 EST). Message-ID: <200701252320.l0PNKqIt086857@jaguar.icir.org> > I am trying to extract some flow characteristics from static data with > Bro. I've checked the analyzer Conn.bro, but didn't find any suitable > information. Check out analy.bro, which does this sort of analysis on whatever connections Bro is processing (so you need to load additional scripts to capture the packets of interest). You may need to extend it by editing TCPStats_Endpoint::DataSent in TCP.cc. > At present, the characteristics I need are: mean packet size and mean > packet inter-arrival time, all per flow. Note, if you just want means, then you can track this quite cheaply. (And mean inter-arrival time is just duration divided by number of packets.) Vern From vern at icir.org Thu Jan 25 15:24:38 2007 From: vern at icir.org (Vern Paxson) Date: Thu, 25 Jan 2007 15:24:38 -0800 Subject: [Bro] IPv6 support in Bro In-Reply-To: <45AEAF11.6000106@klipix.org> (Wed, 17 Jan 2007 18:19:45 EST). Message-ID: <200701252324.l0PNOccq087122@jaguar.icir.org> > I want to improve Bro IPv6 support That would be terrific! > and I'm just wondering where to > start (header fields, extensions, ICMPv6...). I think the highest priority is dealing with encapsulated headers (e.g., options), as these are the most immediate shortcoming in Bro's IPv6 analysis that come to mind. Vern From ducha at cse.buffalo.edu Fri Jan 26 08:04:59 2007 From: ducha at cse.buffalo.edu (Duc T Ha) Date: Fri, 26 Jan 2007 11:04:59 -0500 Subject: [Bro] Traffic characteristics extraction with Bro In-Reply-To: <200701252320.l0PNKqIt086857@jaguar.icir.org> References: <200701252320.l0PNKqIt086857@jaguar.icir.org> Message-ID: <45BA26AB.6070104@cse.buffalo.edu> Thank you all for the tips. For those who are interested, here're some observations: 1> I used "tcp_packet" event to capture packets. To test this event, I summed up all packet sizes from the input argument "len" and "is_orig" for each flow. Finally, I compared the sum values (for both directions) with the values returned by "conn_size", as in the analy.bro The result is interesting: for ALL connections, the total size from the ORIGINATOR side is an exact match, but for many connections, my values for the RESPONDER side are higher and the discrepancies depend on the total size. Is there a bug, Vern ? 2> Applying similar method for event "new_packet" (using the field "dl" in tcp_hdr) gived me the discrepancies of ten folds, for each direction. So I guess tcp_packet is more suitable :), although I don't know why. 3> I found that handling packet level events (such as tcp_packet) made Bro run out of memory when analyzing a CRedII trace with lots of scans - even if the handler does nothing. Bro works fine, though, if I don't capture these events. 4> It would be nice if there's an overview explanation about the Bro CC code, for someone who needs to extend or modify the code. Doesn't have to be long, one or 2 pages are fine. Also it would be great if we have a page for people to share useful policy/scripts files. I'd be happy to contribute. 5> I really like the Bro language and is learning a lot from Bro. Thanks for creating such a wonderful tool. Duc Ha Vern Paxson wrote: >> I am trying to extract some flow characteristics from static data with >> Bro. I've checked the analyzer Conn.bro, but didn't find any suitable >> information. >> > > Check out analy.bro, which does this sort of analysis on whatever connections > Bro is processing (so you need to load additional scripts to capture the > packets of interest). You may need to extend it by editing > TCPStats_Endpoint::DataSent in TCP.cc. > > >> At present, the characteristics I need are: mean packet size and mean >> packet inter-arrival time, all per flow. >> > > Note, if you just want means, then you can track this quite cheaply. > (And mean inter-arrival time is just duration divided by number of packets.) > > Vern > -------------- next part -------------- A non-text attachment was scrubbed... Name: ducha.vcf Type: text/x-vcard Size: 263 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070126/382d693a/attachment.vcf From christian at whoop.org Fri Jan 26 10:51:31 2007 From: christian at whoop.org (Christian Kreibich) Date: Fri, 26 Jan 2007 10:51:31 -0800 Subject: [Bro] web interface In-Reply-To: <20070124163552.GA20536@armada.mynetwork.local> References: <20070124090327.AJP96010@m4500-00.uchicago.edu> <20070124163552.GA20536@armada.mynetwork.local> Message-ID: <1169837491.3638.145.camel@strangepork> Hi there, On Wed, 2007-01-24 at 11:35 -0500, Jean-Philippe Luiggi wrote: > Hello, > > I'm not sure about the web interface but speaking of OpenBSD vs Bro, it works. > Not directly (build problems) if you use the official .tar.gz but with some patchs, > it works very well. has there been any progress on the nonblocking DNS front on OpenBSD? If anyone has this working, please let us know what was required so we can write according configure tests/instructions. Thanks! Cheers, Christian. -- ________________________________________________________________________ http://www.cl.cam.ac.uk/~cpk25 http://www.whoop.org From matthew.weiner at us.army.mil Mon Jan 29 08:19:07 2007 From: matthew.weiner at us.army.mil (matthew.weiner at us.army.mil) Date: Mon, 29 Jan 2007 11:19:07 -0500 Subject: [Bro] Change for 2007 on Daylight Savings Time in Bro. Message-ID: Is Bro affected by the new 2007 change in Daylight Savings Time (DST)? Where does Bro pull its time from? Beginning in 2007, Daylight Saving Time is extended one month and begins for most of the United States at: 2 a.m. on the Second Sunday in March to 2 a.m. on the First Sunday of November. If so, are there any current patches or fixes to ensure Bro pulls the correct time. From robin at icir.org Mon Jan 29 09:04:33 2007 From: robin at icir.org (Robin Sommer) Date: Mon, 29 Jan 2007 09:04:33 -0800 Subject: [Bro] Change for 2007 on Daylight Savings Time in Bro. In-Reply-To: References: Message-ID: <20070129170432.GA15585@icir.org> On Mon, Jan 29, 2007 at 11:19 -0500, matthew.weiner at us.army.mil wrote: > Is Bro affected by the new 2007 change in Daylight Savings Time (DST)? Where does Bro pull its time from? Bro relies on the operating system for getting the current time. So, if the OS is updated to reflect the DST changes, Bro should be fine too. Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From dopheide at ncsa.uiuc.edu Mon Jan 29 12:22:26 2007 From: dopheide at ncsa.uiuc.edu (Mike Dopheide) Date: Mon, 29 Jan 2007 14:22:26 -0600 Subject: [Bro] UDP contents Message-ID: <45BE5782.9000100@ncsa.uiuc.edu> I'm having a slight problem getting the contents of Kerberos UDP packets. This is my first attempt at Bro so hopefully my error is something simple. Bro version 1.1d When a client requests an initial kerberos ticket it sends a request to the server (AS_REQ) and the server reply is usually either the ticket or an error. I want to watch the initial AS_REQ, but all I'm seeing is the response from the server. In this case, /tmp/trace2.out is a tcpdump of a couple kerberos requests from the client's perspective and the AS_REQ's are there when looking at the dump via ethereal. /usr/local/bro/bin/bro -r /tmp/trace2.out hostname.bro ====== policy/bro.init ============= ... const udp_content_deliver_all_orig = T &redef; const udp_content_deliver_all_resp = T &redef; ... ====== site/hostname.bro ======== @prefixes = local @load site @load conn.bro # not really needed global dop = open_log_file("dop") &redef; event udp_contents(u: connection, is_orig: bool, contents: string){ local id = u$id; print dop, fmt("KDC %s %s",id$orig_p,id$resp_p); print dop, fmt("contents %s",contents); } =========================== Sample output from one of the requests, this is the server responding back to the client. Again, Bro is running on the client. KDC 32898/udp 88/udp contents ~\x82^A^I0\x82^A^E\xa0^C^B^A^E\xa1^C^B^A\x1e\xa2^Q^X^O20070125213047Z\xa4^Q^X^O20070125213048Z\xa5^E^B^C^F\x80\xb5\xa6^C^B^A^Y\xa7^J\x1b^HNCSA.EDU\xa8^U0^S\xa0^C^B^A^A\xa1^L0^J\x1b^Hdopheide\xa9^J\x1b^HNCSA.EDU\xaa\x1d0\x1b\xa0^C^B^A\0\xa1^T0^R\x1b^Fkrbtgt\x1b^HNCSA.EDU\xab^Q\x1b^ONEEDED_PREAUTH\0\xacf^Dd0b0^I\xa1^C^B^A^B\xa2^B^D\00J\xa1^C^B^A^S\xa2C^DA0?0^E\xa0^C^B^A^R0^E\xa0^C^B^A^P0^E\xa0^C^B^A^A0^E\xa0^C^B^A^C0^I\xa0^C^B^A^A\xa1^B\x1b\00^V\xa0^C^B^A^A\xa1^J\x1b^HNCSA.EDU\xa2^C^D^A^A0^I\xa1^C^B^A^M\xa2^B^D\0 Any thoughts? Is it just because the AS_REQ is outgoing on the system where Bro is running? (And why would that matter?) -Mike From jp.luiggi at free.fr Mon Jan 29 07:14:16 2007 From: jp.luiggi at free.fr (Jean-Philippe Luiggi) Date: Mon, 29 Jan 2007 10:14:16 -0500 Subject: [Bro] web interface In-Reply-To: <1169837491.3638.145.camel@strangepork> References: <20070124090327.AJP96010@m4500-00.uchicago.edu> <20070124163552.GA20536@armada.mynetwork.local> <1169837491.3638.145.camel@strangepork> Message-ID: <20070129151416.GA20100@armada.mynetwork.local> Hello, I'm working on this, the easy way for this is (IMHO) is to use the following package (from OpenBSD's ports) : libbind (libbind-9.3.2p0.tgz as of today in current release). Then and here where i am, tweak "configure.in" in order to use it. Best regards. On Fri, Jan 26, 2007 at 10:51:31AM -0800, Christian Kreibich wrote: > Hi there, > > On Wed, 2007-01-24 at 11:35 -0500, Jean-Philippe Luiggi wrote: > > Hello, > > > > I'm not sure about the web interface but speaking of OpenBSD vs Bro, it works. > > Not directly (build problems) if you use the official .tar.gz but with some patchs, > > it works very well. > > has there been any progress on the nonblocking DNS front on OpenBSD? If > anyone has this working, please let us know what was required so we can > write according configure tests/instructions. Thanks! > > Cheers, > Christian. > -- > ________________________________________________________________________ > http://www.cl.cam.ac.uk/~cpk25 > http://www.whoop.org > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From christian at whoop.org Mon Jan 29 15:15:27 2007 From: christian at whoop.org (Christian Kreibich) Date: Mon, 29 Jan 2007 15:15:27 -0800 Subject: [Bro] UDP contents In-Reply-To: <45BE5782.9000100@ncsa.uiuc.edu> References: <45BE5782.9000100@ncsa.uiuc.edu> Message-ID: <1170112527.5678.45.camel@strangepork> Hi Mike, it doesn't look like you're doing anything wrong. However, I've just tried dumping some UDP contents with Bro 1.2.1 and it works fine, so we need to figure out why it's not working for you. I have this traffic: 22:26:10.089582 IP 1.2.3.211.53 > 200.33.146.222.32789: 3628*- 1/4/4 PTR [...]. (245) 22:26:10.089582 IP 1.2.3.211.32789 > 200.33.146.222.53: 18275 A? [...]. (44) 22:26:10.229573 IP 1.2.3.211.53 > 200.33.146.222.32789: 18275*- 1/3/3 A [...] (187) 22:30:39.981424 IP 1.2.3.211.32789 > 200.33.146.222.53: 36142 PTR? [...]. (44) 22:30:40.141413 IP 1.2.3.211.53 > 200.33.146.222.32789: 36142* 1/2/2 PTR [...]. (16 22:30:40.151412 IP 1.2.3.211.32789 > 200.33.146.222.53: 20837 A? [...]. (52) 22:30:40.311401 IP 1.2.3.211.53 > 200.33.146.222.32789: 20837 NXDomain* 0/1/0 (120) and this policy, in which I don't print out the contents, but have added output of the is_orig flag that shows the directionality of the packet: ---- global dop = open_log_file("dop") &redef; event udp_contents(u: connection, is_orig: bool, contents: string){ local id = u$id; print dop, fmt("KDC %s %s %s",id$orig_p,id$resp_p, is_orig); } ---- I get output from both directions as expected: KDC 32789/udp 53/udp F KDC 32789/udp 53/udp T KDC 32789/udp 53/udp F KDC 32789/udp 53/udp T KDC 32789/udp 53/udp F KDC 32789/udp 53/udp T KDC 32789/udp 53/udp F Could validate via is_orig whether you see the right directions, and use Bro 1.2.1 for your tests? On Mon, 2007-01-29 at 14:22 -0600, Mike Dopheide wrote: > I'm having a slight problem getting the contents of Kerberos UDP > packets. This is my first attempt at Bro so hopefully my error is > something simple. [...] Cheers, Christian. -- ________________________________________________________________________ http://www.cl.cam.ac.uk/~cpk25 http://www.whoop.org From dopheide at ncsa.uiuc.edu Mon Jan 29 17:06:59 2007 From: dopheide at ncsa.uiuc.edu (Mike Dopheide) Date: Mon, 29 Jan 2007 19:06:59 -0600 Subject: [Bro] UDP contents In-Reply-To: <1170112527.5678.45.camel@strangepork> References: <45BE5782.9000100@ncsa.uiuc.edu> <1170112527.5678.45.camel@strangepork> Message-ID: <45BE9A33.8050003@ncsa.uiuc.edu> I've upgraded to Bro 1.2.1 and here are four packets that correspond to a DNS request and the kerberos request: 18:15:49.693120 IP 141.142.222.33.32909 > 141.142.2.2.domain: 24696+ SRV? _kerberos-master._tcp.NCSA.EDU. (48) 18:15:49.693493 IP 141.142.2.2.domain > 141.142.222.33.32909: 24696 NXDomain* 0/1/0 (105) 18:15:49.693545 IP 141.142.222.33.32909 > 141.142.3.16.kerberos: v5 18:15:49.694244 IP 141.142.3.16.kerberos > 141.142.222.33.32909: And my bro output: 32909/udp 53/udp F 32909/udp 88/udp F In this case, 141.142.222.33 is the local client where bro is running. Interestingly enough, I did a tcpdump from two other clients. One in the same building gave similar results, one in the building with the kerberos server gave me the results we should expect: 36209/udp 53/udp T 36209/udp 53/udp F 36209/udp 88/udp T 36209/udp 88/udp F Now might be a good time to mention that I also see these messages locally, but I had always assumed they were unrelated and due to other local traffic: 1170117967.140200 weird: bad_UDP_checksum 1170117967.140751 weird: bad_UDP_checksum 1170117967.141191 weird: bad_UDP_checksum 1170117967.142015 weird: bad_TCP_checksum 1170117967.142807 weird: bad_TCP_checksum So.. my current theory is there's something screwy with our local network and I intend to find out what's causing it. I'll let you know when I do. This UDP traffic 'works' so I think Bro should be detecting it regardless of whether some networking equipment might be mangling the packets a bit. -Mike Christian Kreibich wrote: > Hi Mike, > > it doesn't look like you're doing anything wrong. However, I've just > tried dumping some UDP contents with Bro 1.2.1 and it works fine, so we > need to figure out why it's not working for you. > > I have this traffic: > > 22:26:10.089582 IP 1.2.3.211.53 > 200.33.146.222.32789: 3628*- 1/4/4 PTR [...]. (245) > 22:26:10.089582 IP 1.2.3.211.32789 > 200.33.146.222.53: 18275 A? [...]. (44) > 22:26:10.229573 IP 1.2.3.211.53 > 200.33.146.222.32789: 18275*- 1/3/3 A [...] (187) > 22:30:39.981424 IP 1.2.3.211.32789 > 200.33.146.222.53: 36142 PTR? [...]. (44) > 22:30:40.141413 IP 1.2.3.211.53 > 200.33.146.222.32789: 36142* 1/2/2 PTR [...]. (16 > 22:30:40.151412 IP 1.2.3.211.32789 > 200.33.146.222.53: 20837 A? [...]. (52) > 22:30:40.311401 IP 1.2.3.211.53 > 200.33.146.222.32789: 20837 NXDomain* 0/1/0 (120) > > and this policy, in which I don't print out the contents, but have added > output of the is_orig flag that shows the directionality of the packet: > > ---- > global dop = open_log_file("dop") &redef; > > event udp_contents(u: connection, is_orig: bool, contents: string){ > local id = u$id; > print dop, fmt("KDC %s %s %s",id$orig_p,id$resp_p, is_orig); > } > ---- > > I get output from both directions as expected: > > KDC 32789/udp 53/udp F > KDC 32789/udp 53/udp T > KDC 32789/udp 53/udp F > KDC 32789/udp 53/udp T > KDC 32789/udp 53/udp F > KDC 32789/udp 53/udp T > KDC 32789/udp 53/udp F > > Could validate via is_orig whether you see the right directions, and use > Bro 1.2.1 for your tests? > > On Mon, 2007-01-29 at 14:22 -0600, Mike Dopheide wrote: >> I'm having a slight problem getting the contents of Kerberos UDP >> packets. This is my first attempt at Bro so hopefully my error is >> something simple. > [...] > > Cheers, > Christian. From vern at icir.org Mon Jan 29 17:20:04 2007 From: vern at icir.org (Vern Paxson) Date: Mon, 29 Jan 2007 17:20:04 -0800 Subject: [Bro] UDP contents In-Reply-To: <45BE9A33.8050003@ncsa.uiuc.edu> (Mon, 29 Jan 2007 19:06:59 CST). Message-ID: <200701300120.l0U1K47f057755@jaguar.icir.org> > 1170117967.140200 weird: bad_UDP_checksum > 1170117967.140751 weird: bad_UDP_checksum > 1170117967.141191 weird: bad_UDP_checksum > 1170117967.142015 weird: bad_TCP_checksum > 1170117967.142807 weird: bad_TCP_checksum > > So.. my current theory is there's something screwy with our local > network and I intend to find out what's causing it. It's very likely the problem is that for outbound packets, your local packet capture occurs at a point in the kernel prior to when the checksums are computed (this can especially be the case if your system has a form of TCP offboard acceleration). > This UDP traffic 'works' so I think Bro should be detecting > it regardless of whether some networking equipment might be mangling the > packets a bit. You can test this by running Bro with -C to tell it to ignore checksum errors. Vern From robin at icir.org Mon Jan 29 17:47:36 2007 From: robin at icir.org (Robin Sommer) Date: Mon, 29 Jan 2007 17:47:36 -0800 Subject: [Bro] UDP contents In-Reply-To: <45BE9A33.8050003@ncsa.uiuc.edu> References: <45BE5782.9000100@ncsa.uiuc.edu> <1170112527.5678.45.camel@strangepork> <45BE9A33.8050003@ncsa.uiuc.edu> Message-ID: <20070130014736.GA18035@icir.org> On Mon, Jan 29, 2007 at 19:06 -0600, Mike Dopheide wrote: > when I do. This UDP traffic 'works' so I think Bro should be detecting > it regardless of whether some networking equipment might be mangling the > packets a bit. You can start Bro with -C to ignore broken checksums. Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From archit at cs.wisc.edu Mon Jan 29 19:42:58 2007 From: archit at cs.wisc.edu (Archit Gupta) Date: Mon, 29 Jan 2007 21:42:58 -0600 Subject: [Bro] Using Dynamic protocol detection to create a HTTP trace Message-ID: Hi, Given a trace with all traffic, is it possible to use Bro to produce a trace with just the HTTP traffic? Any hints would be appreciated. Thanks and best regards, Archit From dopheide at ncsa.uiuc.edu Mon Jan 29 21:32:37 2007 From: dopheide at ncsa.uiuc.edu (Mike Dopheide) Date: Mon, 29 Jan 2007 23:32:37 -0600 Subject: [Bro] UDP contents In-Reply-To: <200701300120.l0U1K47f057755@jaguar.icir.org> References: <200701300120.l0U1K47f057755@jaguar.icir.org> Message-ID: <45BED875.60207@ncsa.uiuc.edu> That makes everything work as expected. Thanks! -Mike > You can test this by running Bro with -C to tell it to ignore checksum errors. > > Vern > > From buraglio at ncsa.edu Mon Jan 29 22:47:05 2007 From: buraglio at ncsa.edu (Nick Buraglio) Date: Tue, 30 Jan 2007 00:47:05 -0600 Subject: [Bro] web interface In-Reply-To: <45B79354.4010402@ucsf.edu> References: <20070124090327.AJP96010@m4500-00.uchicago.edu> <45B79354.4010402@ucsf.edu> Message-ID: <9c1a7c5e0701292247pc87c028m9776dba2c895125@mail.gmail.com> At some point this (NetLens) will likely become open source. nb On 1/24/07, Stephen Lau wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Jacob Yocom-Piatt wrote: > > before i get my hands dirty and start installing and configuring bro, i > > want to know if there is a web interface from which to view alerts, > > graphs, etc. it is not critical that i have this sort of interface, > > although it would be nice. > > > > Mike Haberman of NCSA has developed a NetLens web/Java interface > to some of the Bro logs. It's still under development however it > has seen some operational use. > > Steve > > - -- > +--------------------------------------------------------------------- > Stephen Lau - Stephen.Lau at ucsf.edu > Information Security Policy and Program Manager > University of California, San Francisco > 1855 Folsom, Suite 602, Box 0707, San Francisco, CA 94143 > +1(415) 476-3106 (Work) +1(415) 476-1717 (Fax) > PGP: 44C8 C9CB C15E 2AE1 7B0A 544E 9A04 AB2B F63F 748B > +--------------------------------------------------------------------- > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.2.2 (MingW32) > > iD8DBQFFt5NUmgSrK/Y/dIsRAq+AAKCh1L7ajhqbeaiqzz2FFUbfBUPUdwCdF20Y > dM7t1/WibeOEyHTR/WxJKcg= > =kj+e > -----END PGP SIGNATURE----- > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -- - -- Nick Buraglio Network Engineer, NCSA Phone: 217.244.6428 - -- From christian at whoop.org Tue Jan 30 10:56:05 2007 From: christian at whoop.org (Christian Kreibich) Date: Tue, 30 Jan 2007 10:56:05 -0800 Subject: [Bro] web interface In-Reply-To: <20070129151416.GA20100@armada.mynetwork.local> References: <20070124090327.AJP96010@m4500-00.uchicago.edu> <20070124163552.GA20536@armada.mynetwork.local> <1169837491.3638.145.camel@strangepork> <20070129151416.GA20100@armada.mynetwork.local> Message-ID: <1170183366.5678.69.camel@strangepork> On Mon, 2007-01-29 at 10:14 -0500, Jean-Philippe Luiggi wrote: > Hello, > > I'm working on this, the easy way for this is (IMHO) is to use > the following package (from OpenBSD's ports) : libbind > (libbind-9.3.2p0.tgz as of today in current release). Thanks, that's good to know. Could you let me know where in the filesystem that libbind gets installed on OpenBSD? Cheers, Christian. -- ________________________________________________________________________ http://www.cl.cam.ac.uk/~cpk25 http://www.whoop.org From jp.luiggi at free.fr Tue Jan 30 11:49:06 2007 From: jp.luiggi at free.fr (Jean-Philippe Luiggi) Date: Tue, 30 Jan 2007 14:49:06 -0500 Subject: [Bro] web interface In-Reply-To: <1170183366.5678.69.camel@strangepork> References: <20070124090327.AJP96010@m4500-00.uchicago.edu> <20070124163552.GA20536@armada.mynetwork.local> <1169837491.3638.145.camel@strangepork> <20070129151416.GA20100@armada.mynetwork.local> <1170183366.5678.69.camel@strangepork> Message-ID: <20070130194906.GA16971@armada.mynetwork.local> Hello Christian, Libraries are in "/usr/local/lib/" Includes are in "/usr/local/include/bind" So what do you think of something likes a "--with-openbsd-bind-includes" and "--with-openbsd-bind-libraries" with relevant informations ? Best regards. On Tue, Jan 30, 2007 at 10:56:05AM -0800, Christian Kreibich wrote: > On Mon, 2007-01-29 at 10:14 -0500, Jean-Philippe Luiggi wrote: > > Hello, > > > > I'm working on this, the easy way for this is (IMHO) is to use > > the following package (from OpenBSD's ports) : libbind > > (libbind-9.3.2p0.tgz as of today in current release). > > Thanks, that's good to know. Could you let me know where in the > filesystem that libbind gets installed on OpenBSD? > > Cheers, > Christian. > -- > ________________________________________________________________________ > http://www.cl.cam.ac.uk/~cpk25 > http://www.whoop.org > From christian at whoop.org Tue Jan 30 11:57:04 2007 From: christian at whoop.org (Christian Kreibich) Date: Tue, 30 Jan 2007 11:57:04 -0800 Subject: [Bro] web interface In-Reply-To: <20070130194906.GA16971@armada.mynetwork.local> References: <20070124090327.AJP96010@m4500-00.uchicago.edu> <20070124163552.GA20536@armada.mynetwork.local> <1169837491.3638.145.camel@strangepork> <20070129151416.GA20100@armada.mynetwork.local> <1170183366.5678.69.camel@strangepork> <20070130194906.GA16971@armada.mynetwork.local> Message-ID: <1170187024.5678.79.camel@strangepork> On Tue, 2007-01-30 at 14:49 -0500, Jean-Philippe Luiggi wrote: > Hello Christian, > > Libraries are in "/usr/local/lib/" > Includes are in "/usr/local/include/bind" Thanks! > So what do you think of something likes a "--with-openbsd-bind-includes" and > "--with-openbsd-bind-libraries" with relevant informations ? I think I'll just have configure look for libbind in the default locations as the final fallback before giving up on nonblocking DNS, so I don't think with-openbsd-... is necessary. What we should add though is --with-bind-libs= and --with-bind-includes=, to remain flexible in case people have private libbind installations. Cheers, Christian. -- ________________________________________________________________________ http://www.cl.cam.ac.uk/~cpk25 http://www.whoop.org From jp.luiggi at free.fr Tue Jan 30 14:00:36 2007 From: jp.luiggi at free.fr (Jean-Philippe Luiggi) Date: Tue, 30 Jan 2007 17:00:36 -0500 Subject: [Bro] web interface In-Reply-To: <1170187024.5678.79.camel@strangepork> References: <20070124090327.AJP96010@m4500-00.uchicago.edu> <20070124163552.GA20536@armada.mynetwork.local> <1169837491.3638.145.camel@strangepork> <20070129151416.GA20100@armada.mynetwork.local> <1170183366.5678.69.camel@strangepork> <20070130194906.GA16971@armada.mynetwork.local> <1170187024.5678.79.camel@strangepork> Message-ID: <20070130220036.GA4010@armada.mynetwork.local> Hello Christian, On Tue, Jan 30, 2007 at 11:57:04AM -0800, Christian Kreibich wrote: > > I think I'll just have configure look for libbind in the default > locations as the final fallback before giving up on nonblocking DNS, so > I don't think with-openbsd-... is necessary. What we should add though > is --with-bind-libs= and --with-bind-includes=, to remain > flexible in case people have private libbind installations. Good idea, you're right. We don't need to specify the OS we use, just having the correct informations about where to find the libraries. Are we going to have a new release soonly ? In such a case likes this one, i think i could make a focus on the full OpenBSD's integration with Bro. Best regards. From christian at whoop.org Tue Jan 30 14:07:07 2007 From: christian at whoop.org (Christian Kreibich) Date: Tue, 30 Jan 2007 14:07:07 -0800 Subject: [Bro] web interface In-Reply-To: <20070130220036.GA4010@armada.mynetwork.local> References: <20070124090327.AJP96010@m4500-00.uchicago.edu> <20070124163552.GA20536@armada.mynetwork.local> <1169837491.3638.145.camel@strangepork> <20070129151416.GA20100@armada.mynetwork.local> <1170183366.5678.69.camel@strangepork> <20070130194906.GA16971@armada.mynetwork.local> <1170187024.5678.79.camel@strangepork> <20070130220036.GA4010@armada.mynetwork.local> Message-ID: <1170194827.5678.86.camel@strangepork> On Tue, 2007-01-30 at 17:00 -0500, Jean-Philippe Luiggi wrote: > Hello Christian, > > Good idea, you're right. We don't need to specify the OS we use, just > having the correct informations about where to find the libraries. Right. > Are we going to have a new release soonly ? > In such a case likes this one, i think i could make a focus on the full > OpenBSD's integration with Bro. We always have a new release coming up pretty soon. ;^) You're welcome to send patches any time. Cheers, Christian. -- ________________________________________________________________________ http://www.cl.cam.ac.uk/~cpk25 http://www.whoop.org From geek00l at gmail.com Wed Jan 31 06:01:09 2007 From: geek00l at gmail.com (CS Lee) Date: Wed, 31 Jan 2007 22:01:09 +0800 Subject: [Bro] Multiple bro nodes Message-ID: <1bb5dd90701310601r64ed9b9y7aee739b6ce915de@mail.gmail.com> Hi, I haven't seen any discussion on this matter yet, while I have heard how bro developers fully utilize bro-ids system. What's the good and standard management and maintenance process when one deploy multiple bro-ids nodes in the site? This is tricky, as most of security admins always have their own way of administration, but I would like to know how bro-ids developers such as Vern, Christian or Robin doing it or others who would like to share the idea. How are the analysis and correlation process that can be done through multiple bro-ids node? I know bro-ids documentation is improving especially after wiki is launched. But I still hardly find the answer for the questions above. I would like to know how it is done practically. Thanks. -- Best Regards, CS Lee -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070131/4df7848b/attachment.html From robin at icir.org Wed Jan 31 09:18:40 2007 From: robin at icir.org (Robin Sommer) Date: Wed, 31 Jan 2007 09:18:40 -0800 Subject: [Bro] Using Dynamic protocol detection to create a HTTP trace In-Reply-To: References: Message-ID: <20070131171840.GA31125@icir.org> On Mon, Jan 29, 2007 at 21:42 -0600, you wrote: > Given a trace with all traffic, is it possible to use Bro to produce a > trace with just the HTTP traffic? Any hints would be appreciated. No, not directly (one of the reasons is that DPD may take the decision whether a connection uses HTTP only after a couple of packets have already passed; it buffers the payload so that it can still analyze the full HTTP session but it does not keep the headers so that they could be written out into another trace file). But perhaps you can use Bro to find all the HTTP connections and then extract them in a second post-processing step. Either you browse through the notices reported, or you add an event handler for protocol_confirmation (also see detect-protocols.bro) which then adds some tag to the connection summaries in conn.log . Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From dopheide at ncsa.uiuc.edu Wed Jan 31 12:03:47 2007 From: dopheide at ncsa.uiuc.edu (Mike Dopheide) Date: Wed, 31 Jan 2007 14:03:47 -0600 Subject: [Bro] three things Message-ID: <45C0F623.1060301@ncsa.uiuc.edu> I'm back and I'm stirring up trouble. I've come across three different issues in the last couple days that the list might like to know about. None of this is a show stopper for me, just curiosities. 1) I've spent quite a bit of time trying to get a regular expression to match packet contents returned by udp_contents(). An example snippet of the contents would be: dopheide\xa2^J\x1b^HNCSA.EDU I'd like to match against /.*NCSA.*/, but the match fails. This morning I finally found clean() which makes the pattern work, but then my bytes count gets shifted all around. (Maybe I should be using signatures...) 2) While looking at (1) I found that all patterns fail with bro-1.2.1 on Fedora Core 5: line 54: run-time error: error compiling pattern /^?.*(.*NCSA.*)/ It happens with patterns I write or any patterns in the provided .bro files. bro-1.1d works just fine on FC5 and bro-1.2.1 works fine on RHEL4 3) bro-1.2.1 won't compile on RHEL3 u8, but will on RHEL4. On RHEL3, make fails with this error: g++ -DHAVE_CONFIG_H -I. -I. -I.. -I. -I../src/binpac/lib -I../src -I. -I.. -Ilibedit -I/usr/kerberos/include -I../linux-include -O -W -Wall -Wno-unused -I/usr/kerberos/include -I../linux-include -g -O2 -c -o bif_parse.o bif_parse.cc In file included from /afs/.ncsa.uiuc.edu/packages/GNU/gcc-2.95.2/Linux/lib/gcc-lib/i686-pc-linux-gnu/2.95.2/../../../../include/g++-3/iostream.h:31, from /afs/.ncsa.uiuc.edu/packages/GNU/gcc-2.95.2/Linux/lib/gcc-lib/i686-pc-linux-gnu/2.95.2/../../../../include/g++-3/stl_algobase.h:53, from /afs/.ncsa.uiuc.edu/packages/GNU/gcc-2.95.2/Linux/lib/gcc-lib/i686-pc-linux-gnu/2.95.2/../../../../include/g++-3/vector:30, from ../src/builtin-func.y:2: /afs/.ncsa.uiuc.edu/packages/GNU/gcc-2.95.2/Linux/lib/gcc-lib/i686-pc-linux-gnu/2.95.2/../../../../include/g++-3/streambuf.h: In method `struct streampos streambuf::pubseekoff(long long int, ios::seek_dir, int = 3)': /afs/.ncsa.uiuc.edu/packages/GNU/gcc-2.95.2/Linux/lib/gcc-lib/i686-pc-linux-gnu/2.95.2/../../../../include/g++-3/streambuf.h:362: conversion from `__off64_t' to non-scalar type `streampos' requested /afs/.ncsa.uiuc.edu/packages/GNU/gcc-2.95.2/Linux/lib/gcc-lib/i686-pc-linux-gnu/2.95.2/../../../../include/g++-3/streambuf.h: In method `struct streampos streambuf::pubseekpos(_G_fpos64_t, int = 3)': /afs/.ncsa.uiuc.edu/packages/GNU/gcc-2.95.2/Linux/lib/gcc-lib/i686-pc-linux-gnu/2.95.2/../../../../include/g++-3/streambuf.h:364: `struct streampos' used where a `long long int' was expected /afs/.ncsa.uiuc.edu/packages/GNU/gcc-2.95.2/Linux/lib/gcc-lib/i686-pc-linux-gnu/2.95.2/../../../../include/g++-3/streambuf.h:364: warning: control reaches end of non-void function `streambuf::pubseekpos(_G_fpos64_t, int)' make[2]: *** [bif_parse.o] Error 1 make[2]: Leaving directory `/tmp/dopheide/bro-1.2.1/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/tmp/dopheide/bro-1.2.1' make: *** [all] Error 2 From christian at whoop.org Wed Jan 31 13:57:39 2007 From: christian at whoop.org (Christian Kreibich) Date: Wed, 31 Jan 2007 13:57:39 -0800 Subject: [Bro] three things In-Reply-To: <45C0F623.1060301@ncsa.uiuc.edu> References: <45C0F623.1060301@ncsa.uiuc.edu> Message-ID: <1170280659.8770.41.camel@strangepork> Hi Mike, On Wed, 2007-01-31 at 14:03 -0600, Mike Dopheide wrote: > I'm back and I'm stirring up trouble. uh-oh. :) I'll just comment on issues #2 and #3: > 2) > While looking at (1) I found that all patterns fail with bro-1.2.1 on > Fedora Core 5: > > line 54: run-time error: error compiling pattern /^?.*(.*NCSA.*)/ > > It happens with patterns I write or any patterns in the provided .bro > files. bro-1.1d works just fine on FC5 and bro-1.2.1 works fine on RHEL4 Thanks for this. We've received another report of this problem, but it had us scratching our heads. It's great to know more precisely now where it occurs. We'll look into it. > 3) > bro-1.2.1 won't compile on RHEL3 u8, but will on RHEL4. On RHEL3, make > fails with this error: Ouch. I'm afraid those errors are the installation's fault, since they're all rooted in streambuf code. :( Cheers, Christian. -- ________________________________________________________________________ http://www.cl.cam.ac.uk/~cpk25 http://www.whoop.org From robin at icir.org Wed Jan 31 17:09:34 2007 From: robin at icir.org (Robin Sommer) Date: Wed, 31 Jan 2007 17:09:34 -0800 Subject: [Bro] three things In-Reply-To: <45C0F623.1060301@ncsa.uiuc.edu> References: <45C0F623.1060301@ncsa.uiuc.edu> Message-ID: <20070201010934.GB32680@icir.org> On Wed, Jan 31, 2007 at 14:03 -0600, Mike Dopheide wrote: > I've spent quite a bit of time trying to get a regular expression to > match packet contents returned by udp_contents(). Just tried it with a DNS packet and this script works for me: redef udp_content_deliver_all_orig = T; event udp_contents(u: connection, is_orig: bool, contents: string) { print contents; print /.*bro-ids./ in contents; } Output: /\xb8^A\0\0^A\0\0\0\0\0\0^Cwww^Gbro-ids^Corg\0\0^A\0^A T Can you send me a trace and your script? Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From dick at uchicago.edu Wed Jan 31 17:50:30 2007 From: dick at uchicago.edu (Jacob Yocom-Piatt) Date: Wed, 31 Jan 2007 19:50:30 -0600 (CST) Subject: [Bro] no reports generated Message-ID: <20070131195030.AJX66951@m4500-00.uchicago.edu> bro-1.2.1 is now running on a 4.0-release openbsd machine here (thx j-p!) and generating logs, however i got an empty report email this morning. i mean empty as in there was no body to the email i received. when i installed bro, i had to manually add the bro user and change ownership of some of the directories in its install directory: # ls -al /usr/local/bro total 68 drwxr-xr-x 15 root wheel 512 Jan 30 20:05 . drwxr-xr-x 15 root wheel 512 Jan 30 19:52 .. drwxr-xr-x 2 bro wheel 512 Jan 30 19:52 archive drwxr-xr-x 2 root wheel 512 Jan 30 20:02 bin drwxr-xr-x 2 root wheel 512 Jan 30 20:10 etc drwxr-xr-x 2 root wheel 512 Jan 30 20:00 include drwxr-xr-x 2 root wheel 512 Jan 30 20:00 lib drwxr-xr-x 3 bro wheel 3584 Jan 31 03:00 logs drwxr-xr-x 4 root wheel 512 Jan 30 20:05 perl drwxr-xr-x 3 root wheel 4096 Jan 30 20:05 policy drwxr-xr-x 2 bro wheel 512 Jan 30 19:52 reports drwxr-xr-x 2 root wheel 512 Jan 30 20:05 scripts drwxr-xr-x 4 root wheel 512 Jan 30 19:52 share drwxr-xr-x 2 root wheel 512 Jan 30 20:05 site drwxr-xr-x 2 bro wheel 512 Jan 30 21:07 var perhaps this has something to do with the empty report? i've looked through the system logs and cannot find anything indicating why the report was not generated. cluesticking appreciated. cheers, jake From dopheide at ncsa.uiuc.edu Wed Jan 31 16:01:44 2007 From: dopheide at ncsa.uiuc.edu (Mike Dopheide) Date: Wed, 31 Jan 2007 18:01:44 -0600 Subject: [Bro] three things In-Reply-To: <20070201010934.GB32680@icir.org> References: <45C0F623.1060301@ncsa.uiuc.edu> <20070201010934.GB32680@icir.org> Message-ID: <45C12DE8.9000903@ncsa.uiuc.edu> Trace attached. You'll need to run bro with -C to ignore checksum errors. -Mike Robin Sommer wrote: > On Wed, Jan 31, 2007 at 14:03 -0600, Mike Dopheide wrote: > >> I've spent quite a bit of time trying to get a regular expression to >> match packet contents returned by udp_contents(). > > Just tried it with a DNS packet and this script works for me: > > redef udp_content_deliver_all_orig = T; > > event udp_contents(u: connection, is_orig: bool, contents: string) > { > print contents; > print /.*bro-ids./ in contents; > } > > > Output: > > /\xb8^A\0\0^A\0\0\0\0\0\0^Cwww^Gbro-ids^Corg\0\0^A\0^A > T > > Can you send me a trace and your script? > > Robin > -------------- next part -------------- A non-text attachment was scrubbed... Name: trace2.out Type: application/octet-stream Size: 14028 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070131/6772b7d1/attachment.obj