From dianazeaiter at hotmail.com Sun Jul 5 13:01:24 2009 From: dianazeaiter at hotmail.com (Diana Zeaiter Joumblatt) Date: Sun, 5 Jul 2009 20:01:24 +0000 Subject: [Bro] bro hangs Message-ID: Hi, I just started using bro for doing traffic analysis. It seems to be hanging for some of the traces and i don't know how to use the -t (timeout) option because the problem might be related to unanswered dns requests. $bro -r trace.pcap tcp alarm weird /opt/local/share/bro/policy/scan.bro, line 92: warning: no such host: j5004.inktomisearch.com/opt/local/share/bro/policy/scan.bro, line 92: warning: no such host: j5005.inktomisearch.com/opt/local/share/bro/policy/scan.bro, line 93: warning: no such host: j5006.inktomisearch.com/opt/local/share/bro/policy/scan.bro, line 93: warning: no such host: j100.inktomi.com/opt/local/share/bro/policy/scan.bro, line 93: warning: no such host: j101.inktomi.com/opt/local/share/bro/policy/scan.bro, line 94: warning: no such host: j3002.inktomi.com/opt/local/share/bro/policy/scan.bro, line 94: warning: no such host: si3000.inktomi.com/opt/local/share/bro/policy/scan.bro, line 94: warning: no such host: si3001.inktomi.com/opt/local/share/bro/policy/scan.bro, line 95: warning: no such host: si3002.inktomi.com/opt/local/share/bro/policy/scan.bro, line 95: warning: no such host: si3003.inktomi.com/opt/local/share/bro/policy/scan.bro, line 95: warning: no such host: si4000.inktomi.com/opt/local/share/bro/policy/scan.bro, line 96: warning: no such host: si4001.inktomi.com/opt/local/share/bro/policy/scan.bro, line 96: warning: no such host: si4002.inktomi.com/opt/local/share/bro/policy/scan.bro, line 96: warning: no such host: wm3018.inktomi.comline 1: warning: event handlers never invoked:line 1: warning: account_tried Thanks for your help, diana -- Diana Zeaiter Joumblatt Phd Student Universit? Pierre & Marie Curie - LIP6/CNRS 104 Avenue du Pr?sident Kennedy 75016 Paris, France Desk 705 (+33) 01 44 27 87 75 What can you do with the new Windows Live? Find out What can you do with the new Windows Live? Find out _________________________________________________________________ Drag n? drop?Get easy photo sharing with Windows Live? Photos. http://www.microsoft.com/windows/windowslive/products/photos.aspx -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20090705/94f5fe9e/attachment.html From jasstin at 163.com Thu Jul 16 01:44:33 2009 From: jasstin at 163.com (=?gbk?B?x66zzMK3?=) Date: Thu, 16 Jul 2009 16:44:33 +0800 (CST) Subject: [Bro] Questions About UDP Events in Bro Message-ID: <5486973.265911247733873856.JavaMail.coremail@bj163app27.163.com> Dear Bro Developers: I'm a student from Sun Yet-sen Uinversity in China. I'm mailing you for some questions about using bro. I'm now using bro to analyze UDP network traffic flows. I've got to know that bro can trace TCP connections and have the event "tcp_packet" to get the tcp payload. However, it seems that bro doesn't handle UDP flows well. (1)Is there some different definitions between TCP and UDP connection? And (2)how to get the payload of an UDP packet? I have noticed that the event "udp_content" maybe the handler, but (3)I don't know how to ivoke it. Since UDP flows continue to increase on the Internet, I'm doing some analyze on it and need tools to handle it. And I think Bro should be a good one. Looking forward to you reply. Best Regards. Sincerely Yours Jasstin 16th, July 200????,????,????? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20090716/d6888891/attachment.html From sullivan at cs.ucsb.edu Fri Jul 24 16:53:41 2009 From: sullivan at cs.ucsb.edu (Lorenzo Cavallaro) Date: Fri, 24 Jul 2009 16:53:41 -0700 Subject: [Bro] Returning local variables and garbage collection Message-ID: <20090724235340.GA12447@galilei> Hi, I'm debugging a custom policy script that was causing Bro to use too much memory. The script has been stripped down just to do some logging and nothing more, but the memory usage is still pretty "high" i.e., WAY higher then with compared to having conn.bro loaded, for instance. The only particular thing the script is currently doing is just to return local variables. Basically, I'm considering them similar to ptr in C (but I might have misunderstood their semantic, tho). For instance, I've something like the following: type custom_conn_t: record { id: conn_id; # custom type involved here! ... }; function conn_init(c: connection) { local __c: custom_conn_t; __c$id = c$id; # assign other __c's fields from c return __c; } function foobar(c: connection) { local __c: custom_conn_t; __c = conn_init(c); do_log(__c); } event X(c: connection) { foobar(c); } I'm monitoring live-traffic so it's pretty hard to provide a representative trace. However, conn.bro produces a low memory footprint (~50MB over 3/4 hrs) and it gets stable pretty soon. The aforementioned script reached 200MB in less time and it keeps growing. I'm just wondering what it happens when I return __c in conn_init(). I'm expecting a new object to be created and the local one declared in conn_init to be destroyed. Then, eventually, whenever the newly created __c is not needed anymore (say, after do_log, or, however, after event X returns), I'd expect it to be free'd by the garbage collector. Or, is the object the same and just internal refcnt are increased or decreased? If so, it shouldn't really make any difference as refcnt should be going to 0 after X finishes. However, I'm experiencing and (almost linear) increasing memory consumption, and that's weird (bug?). Any idea? TIA, bye Lorenzo -- Lorenzo `Gigi Sullivan' Cavallaro GPG key at http://www.cs.ucsb.edu/~sullivan/sullivan.asc Until I loved, life had no beauty; I did not know I lived until I had loved. (Theodor Korner) See the reality in your eyes, when the hate makes you blind. (A.H.X) From robin at icir.org Mon Jul 27 10:30:41 2009 From: robin at icir.org (Robin Sommer) Date: Mon, 27 Jul 2009 10:30:41 -0700 Subject: [Bro] Save the date: Another Bro Workshop Message-ID: <20090727173041.GE85031@icir.org> Bro Workshop 2009, the 2nd. =========================== The Bro team and the Lawrence Berkeley National Lab are pleased to announce a further "Bro Workshop", a 2.5-day Bro training event that will take place in Berkeley, CA, on October 13-15, 2009. The workshop is primarily targeted at site security personnel wishing to learn more about how Bro works, how to use its scripting language and how to generally customize the system based on a site's local policy. Similar to previous workshops, the agenda will be an informal mix of tutorial-style presentations and hands-on lab sessions. No prior knowledge about using Bro is assumed though attendees should be familiar with Unix shell usage as well as with typical networking tools like tcpdump and Wireshark. All participants are expected to bring a Unix-based (Linux, Mac OS X, FreeBSD) laptop with a working Bro configuration. We will provide sample trace files to work with. This workshop will again be hosted by the Lawrence Berkeley National Lab, and it will be located at the Hotel Durant in Berkeley. We will soon provide a web site with more detailed registration and location information. To facilitate a productive lab environment, the number of attendees will be limited to 30 people. A registration fee of $125 will be charged. We also expect to have time for 2-3 case-study presentations from people using Bro in their environments. If you have something you would like to talk about, please send me a mail. Looking forward to a great workshop, Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Mon Jul 27 10:39:31 2009 From: robin at icir.org (Robin Sommer) Date: Mon, 27 Jul 2009 10:39:31 -0700 Subject: [Bro] Returning local variables and garbage collection In-Reply-To: <20090724235340.GA12447@galilei> References: <20090724235340.GA12447@galilei> Message-ID: <20090727173931.GF85031@icir.org> On Fri, Jul 24, 2009 at 16:53 -0700, you wrote: > Or, is the object the same and just internal refcnt are increased or > decreased? If so, it shouldn't really make any difference as refcnt > should be going to 0 after X finishes. This is indeed what happens. Non-atomic objects are passed around as references, with reference counts adjusted as necessary. > However, I'm experiencing and (almost linear) increasing memory > consumption, and that's weird (bug?). Any idea? Not sure right now, the code excerpts you showed look ok. One thing to do is running with profiling.bro, that will let Bro generate a file prof.log with various memory statistics. Feel free to send me the output if it's too cryptic. If that doesn't help, some leak checking/profiling could help illuminating what's going on, see http://blog.icir.org/2008/02/making-sure-your-bro-code-does-not-leak.html Robin P.S.: Are you creating any cyclic reference structures? -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From adayadil.thomas at gmail.com Mon Jul 27 11:21:45 2009 From: adayadil.thomas at gmail.com (Adayadil Thomas) Date: Mon, 27 Jul 2009 14:21:45 -0400 Subject: [Bro] Save the date: Another Bro Workshop In-Reply-To: <20090727173041.GE85031@icir.org> References: <20090727173041.GE85031@icir.org> Message-ID: This is great. Will the information (slides/talk etc) be available for the public? Thanks On Mon, Jul 27, 2009 at 1:30 PM, Robin Sommer wrote: > > Bro Workshop 2009, the 2nd. > =========================== > > The Bro team and the Lawrence Berkeley National Lab are pleased to > announce a further "Bro Workshop", a 2.5-day Bro training event that > will take place in Berkeley, CA, on October 13-15, 2009. > > The workshop is primarily targeted at site security personnel > wishing to learn more about how Bro works, how to use its scripting > language and how to generally customize the system based on a site's > local policy. > > Similar to previous workshops, the agenda will be an informal mix of > tutorial-style presentations and hands-on lab sessions. No prior > knowledge about using Bro is assumed though attendees should be > familiar with Unix shell usage as well as with typical networking > tools like tcpdump and Wireshark. > > All participants are expected to bring a Unix-based (Linux, Mac OS X, > FreeBSD) laptop with a working Bro configuration. We will provide > sample trace files to work with. > > This workshop will again be hosted by the Lawrence Berkeley National > Lab, and it will be located at the Hotel Durant in Berkeley. We will > soon provide a web site with more detailed registration and location > information. To facilitate a productive lab environment, the number > of attendees will be limited to 30 people. A registration fee of > $125 will be charged. > > We also expect to have time for 2-3 case-study presentations from > people using Bro in their environments. If you have something you > would like to talk about, please send me a mail. > > Looking forward to a great workshop, > > Robin > > -- > Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org > ICSI/LBNL ? ?* Fax ? +1 (510) 666-2956 * ? www.icir.org > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > From sullivan at cs.ucsb.edu Mon Jul 27 11:24:19 2009 From: sullivan at cs.ucsb.edu (Lorenzo Cavallaro) Date: Mon, 27 Jul 2009 11:24:19 -0700 Subject: [Bro] Returning local variables and garbage collection In-Reply-To: <20090727173931.GF85031@icir.org> References: <20090724235340.GA12447@galilei> <20090727173931.GF85031@icir.org> Message-ID: <20090727182419.GH4739@galilei> Robin, On Mon, Jul 27, 2009 at 10:39:31AM -0700, Robin Sommer wrote: > > Or, is the object the same and just internal refcnt are increased or > > decreased? If so, it shouldn't really make any difference as refcnt > > should be going to 0 after X finishes. > > This is indeed what happens. Non-atomic objects are passed around as > references, with reference counts adjusted as necessary. This is what I was thinking, so there shouldn't be any problem. > to do is running with profiling.bro, that will let Bro generate a > file prof.log with various memory statistics. Feel free to send me > the output if it's too cryptic. Perfect, thanks. I'm actually returning "custom" type, i.e., those for which Bro doesn't know anything about, internally. That is, there are no corresponding RecordVal declaration nor "initialization" by means of internal_type("...")->AsRecordType(). Could Bro mess things up if those are missing? In addition, the memory consumption lowered down when I removed the handler for connection_timeout (my code is called when a bunch of connection_* events are triggered). However, I just suppose this happens because less events of this type are raised. > P.S.: Are you creating any cyclic reference structures? I don't think so, but I'll double check. TIA, bye Lorenzo -- Lorenzo `Gigi Sullivan' Cavallaro GPG key at http://www.cs.ucsb.edu/~sullivan/sullivan.asc Until I loved, life had no beauty; I did not know I lived until I had loved. (Theodor Korner) See the reality in your eyes, when the hate makes you blind. (A.H.X) From robin at icir.org Mon Jul 27 11:54:33 2009 From: robin at icir.org (Robin Sommer) Date: Mon, 27 Jul 2009 11:54:33 -0700 Subject: [Bro] Save the date: Another Bro Workshop In-Reply-To: References: <20090727173041.GE85031@icir.org> Message-ID: <20090727185433.GQ85031@icir.org> On Mon, Jul 27, 2009 at 14:21 -0400, Adayadil Thomas wrote: > This is great. Will the information (slides/talk etc) be available for > the public? Yes, they will. The material will however not be too different from that of previous workshops, though updated where appropiate. (That btw also means that the workshop will likely be less interesting for folks who have already attended one in the past.) Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Mon Jul 27 12:45:55 2009 From: robin at icir.org (Robin Sommer) Date: Mon, 27 Jul 2009 12:45:55 -0700 Subject: [Bro] Returning local variables and garbage collection In-Reply-To: <20090727182419.GH4739@galilei> References: <20090724235340.GA12447@galilei> <20090727173931.GF85031@icir.org> <20090727182419.GH4739@galilei> Message-ID: <20090727194555.GB97127@icir.org> On Mon, Jul 27, 2009 at 11:24 -0700, you wrote: > Robin, > Perfect, thanks. I'm actually returning "custom" type, i.e., those > for which Bro doesn't know anything about, internally. By "custom", you mean declared as a record on the script-level, right? That's fine, Bro knows how to handle them when passing around. The internal stuff ("internal_type()" etc.) is only needed for types which the C++ accesses itself in some way (e.g., because it wants to modify an instance of that type). > In addition, the memory consumption lowered down when I removed the > handler for connection_timeout The prof.log output should indicate whether there are a lot of connections hanging out in memory for some reason, which could potentially be a problem. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From kosinovsky1 at llnl.gov Tue Jul 28 11:11:43 2009 From: kosinovsky1 at llnl.gov (Greg Kosinovsky) Date: Tue, 28 Jul 2009 11:11:43 -0700 Subject: [Bro] getting first results Message-ID: <80955b$225c2b@smtp.llnl.gov> I am trying to get my first results with BRO. I am just running mt.bro on an existing tcpdump file (containing some DNS data). My exact command is "bin/bro -r dns.cap share/bro/mt.bro" This command run to completion without error and creates empty log files for a number of policies loaded inside mt.bro. Also, if I put a print statement inside mt.bro, I can see the output. However, If I put print statements inside any of the functions defined in policies loaded by mt.bro ("dns-lookup", "weird", etc.,) I cannot see any results -- presumably these policies have to be invoked. The scripts I am modifying to try to see these results are in share/bro (share/bro/weird.bro, share/bro/dns-lookup.bro, etc.). It appears to be straightforward, but I must be missing something conceptually. Thank you, Greg From robin at icir.org Wed Jul 29 11:04:50 2009 From: robin at icir.org (Robin Sommer) Date: Wed, 29 Jul 2009 11:04:50 -0700 Subject: [Bro] getting first results In-Reply-To: <80955b$225c2b@smtp.llnl.gov> References: <80955b$225c2b@smtp.llnl.gov> Message-ID: <20090729180450.GF87362@icir.org> On Tue, Jul 28, 2009 at 11:11 -0700, you wrote: > I am trying to get my first results with BRO. I am just running > mt.bro on an existing tcpdump file (containing some DNS data). My > exact command is "bin/bro -r dns.cap share/bro/mt.bro" Note that mt is not loading dns.bro, which is the main DNS analysis script. Try "bro -r dns.cap mt dns". Without dns.bro, Bro will not look at DNS packets at all as they won't pass the packet filter, which is also why you aren't seeing much activity at all. (dns-lookup.bro tracks Bro's *own* DNS queries but nothing else). Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org