From ju at klipix.org Mon Apr 2 20:27:17 2007 From: ju at klipix.org (Julien Desfossez) Date: Mon, 02 Apr 2007 23:27:17 -0400 Subject: [Bro] Progress in the IPv6 support Message-ID: <4611C995.2060100@klipix.org> Hi, Some months ago, I told you I wanted to improve Bro IPv6 support. After a long time of source code reading I've started to code something. To begin, I've primarily focused on the support for extensions. So now when Bro receives a TCP or UDP packet with an arbitrary number of IPv6 extensions (hop-by-hop option, destination option and routing header for the moment) it can parse the content of the upper layer. It's a beginning, but now I have some questions : - what should I do with the data of the extensions header ? - what do you think is the "best pratice" in Bro to code the support of ICMPv6 (it's a major update compared to ICMPv4) ? - do you think binpac could help me ? I have not yet catched all the details of the analysers and event handlers, and the most of the doc I have found focuses on upper layer protocols, so any help would be greatly appreciated ! After that I will start dealing with the fragmentation, IPsec headers and eventually the transition mechanisms. Thank you for your help. Julien Desfossez P.S : sorry but my code isn't clean enough yet to attach the patch, as soon as I have something fully functionnal/tested I'll send it to the list. From bindiyavs at tataelxsi.co.in Mon Apr 2 22:32:48 2007 From: bindiyavs at tataelxsi.co.in (Bindiya V S) Date: Tue, 3 Apr 2007 11:02:48 +0530 (IST) Subject: [Bro] giving IPv6 capture filter support Message-ID: <20070403110248.CHX93887@mail.tataelxsi.co.in> Hello all, I want BRO to detect IPv6 packets. I tried giving capture filter as redef capture_filters = { ["tcp"]= "tcp", ["udp"] = "udp", ["icmp"] = "icmp", ["ipv6"] = "ether proto 0x86dd"}; BRO is not complaining, but the packets are not even recognised at Sesssions.cc NextPacket. Thank you Bindiya From ju at klipix.org Tue Apr 3 04:55:48 2007 From: ju at klipix.org (Julien Desfossez) Date: Tue, 03 Apr 2007 07:55:48 -0400 Subject: [Bro] giving IPv6 capture filter support In-Reply-To: <20070403110248.CHX93887@mail.tataelxsi.co.in> References: <20070403110248.CHX93887@mail.tataelxsi.co.in> Message-ID: <461240C4.3070300@klipix.org> Hello, You can use the following syntax to add IPv6 support : redef capture_filters += { ["ipv6"] = "ip6" }; With that filter, Bro will capture all IPv6 traffic. Sessions.cc recognises TCP and UDP over IPv6 if there is no extension header. ICMPv6 has a different protocol number than ICMP (v4). So you must replace the if ( proto != IPPROTO_TCP && proto != IPPROTO_UDP && proto != IPPROTO_ICMP ) {...} in DoNextPacket by : if ( proto != IPPROTO_TCP && proto != IPPROTO_UDP && proto != IPPROTO_ICMP && proto != IPPROTO_ICMPV6) {...} So your packet won't be dumped. With that, you'll see in your Weird logs something like "unknow protocol 58" when you ping6 for example. Because the "switch (proto) {}" doesn't look for IPPROTO_ICMPV6. HTH, Julien Desfossez Bindiya V S a ?crit : > Hello all, > I want BRO to detect IPv6 packets. I tried giving capture > filter as > redef capture_filters = { ["tcp"]= "tcp", ["udp"] = "udp", > ["icmp"] = "icmp", ["ipv6"] = "ether proto 0x86dd"}; > > BRO is not complaining, but the packets are not even > recognised at Sesssions.cc NextPacket. > > Thank you > Bindiya > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > From JHally at epnet.com Tue Apr 3 11:55:08 2007 From: JHally at epnet.com (John Hally) Date: Tue, 3 Apr 2007 14:55:08 -0400 Subject: [Bro] Bro 1.1d on FC6 Message-ID: <65B32A9DA9B3054D8EA9E004CBCDDF55162729B3@epmail.epnet.com> Hello all, I?m trying to get bro 1.1d compiled on Fedora Core6 but get the following. Configure seems to find everything ok, but make returns this: make[4]: Entering directory `/usr/local/src/bro-1.1d/src/binpac' if g++ -DHAVE_CONFIG_H -I. -I. -I../.. -W -Wall -Wno-unused -I../../linux-include -g -O2 -MT pac_parse.o -MD -MP -MF ".deps/pac_parse.Tpo" -c -o pac_parse.o pac_parse.cc; \ then mv -f ".deps/pac_parse.Tpo" ".deps/pac_parse.Po"; else rm -f ".deps/pac_parse.Tpo"; exit 1; fi pac.h:109: warning: ???class Evaluatable??? has virtual functions but non-virtual destructor pac.h:615: error: extra qualification ???Field::??? on member ???getFieldBegin??? pac.h:616: error: extra qualification ???Field::??? on member ???getFieldEnd??? pac.h:658: warning: ???class LetDef??? has virtual functions but non-virtual destructor make[4]: *** [pac_parse.o] Error 1 make[4]: Leaving directory `/usr/local/src/bro-1.1d/src/binpac' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/usr/local/src/bro-1.1d/src' make[2]: *** [all] Error 2 make[2]: Leaving directory `/usr/local/src/bro-1.1d/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/src/bro-1.1d' make: *** [all] Error 2 any suggestions? Thanks in advance! John. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070403/023b3b0c/attachment.html From christian at whoop.org Wed Apr 4 17:42:18 2007 From: christian at whoop.org (Christian Kreibich) Date: Wed, 04 Apr 2007 17:42:18 -0700 Subject: [Bro] Using Broccoli to config Bro agents remotely In-Reply-To: <4db30bc0703242109r4aeaebdcwf0121538a00c9e91@mail.gmail.com> References: <4db30bc0703242109r4aeaebdcwf0121538a00c9e91@mail.gmail.com> Message-ID: <1175733738.21299.28.camel@strangepork> Hi there, apologies for the slow reply. On Sun, 2007-03-25 at 11:09 +0700, Nguoi Khong Mang Ho wrote: > Hi all, > > Do you ever use Broccoli to config, register or implement your own > event handlers on remote Bro agents from the central one? Is there any > docs or experiences on doing that? I'm not sure I understand, sorry, but I'll try to answer by pointing out a few related things that we currently can(not) do: - First, if you are looking at Bro-Bro communication, then Broccoli is not of interest. Broccoli is used when you want to make a *non-Bro* entity talk to Bro. Broccoli also currently doesn't know how to serialize policy code (since non-Bro endpoints trypically won't know what to do with them). - You can currently *full* configuration dumps of a Bro node and transfer this policy configuration + run-time state to another node. If this is what you want, then let us know and we'll clarify. - You can not transfer policy code at the granularity of individual event handlers. While we can serialize them, the bit of infrastructure required to patch in such bits of code dynamically isn't currently there (unless I'm missing recent developments). Hope that helps. Cheers, Christian -- ________________________________________________________________________ http://www.icir.org/christian http://www.whoop.org From christian at whoop.org Wed Apr 4 18:48:54 2007 From: christian at whoop.org (Christian Kreibich) Date: Wed, 04 Apr 2007 18:48:54 -0700 Subject: [Bro] Bro 1.1d on FC6 In-Reply-To: <65B32A9DA9B3054D8EA9E004CBCDDF55162729B3@epmail.epnet.com> References: <65B32A9DA9B3054D8EA9E004CBCDDF55162729B3@epmail.epnet.com> Message-ID: <1175737734.21299.39.camel@strangepork> Hi John, On Tue, 2007-04-03 at 14:55 -0400, John Hally wrote: > Hello all, > > I?m trying to get bro 1.1d compiled on Fedora Core6 but get the > following. Configure seems to find everything ok, but make returns > this: Thanks, these are known issues and already fixed in 1.2.1. Cheers, Christian -- ________________________________________________________________________ http://www.icir.org/christian http://www.whoop.org From vern at icir.org Wed Apr 11 01:19:52 2007 From: vern at icir.org (Vern Paxson) Date: Wed, 11 Apr 2007 01:19:52 -0700 Subject: [Bro] Progress in the IPv6 support In-Reply-To: <4611C995.2060100@klipix.org> (Mon, 02 Apr 2007 23:27:17 EDT). Message-ID: <200704110819.l3B8JqLw029446@jaguar.icir.org> Great to hear that you're pursuing this! > So now when Bro receives a TCP or UDP packet with an arbitrary number of > IPv6 extensions (hop-by-hop option, destination option and routing > header for the moment) it can parse the content of the upper layer. > > It's a beginning, but now I have some questions : > - what should I do with the data of the extensions header ? Ideal would be to generate an event per header (assuming that the policy script defines a corresponding handler). > - what do you think is the "best pratice" in Bro to code the support of > ICMPv6 (it's a major update compared to ICMPv4) ? I don't know the specifics of ICMPv6. What are the main ways in which it's a major update? > - do you think binpac could help me ? It will need extensions to do so. You should contact Ruoming Pang to see what degree he has interest/cycles in adding the necessary support. > After that I will start dealing with the fragmentation, IPsec headers > and eventually the transition mechanisms. This all sounds great ... Vern From rpang at cs.princeton.edu Wed Apr 11 06:59:37 2007 From: rpang at cs.princeton.edu (Ruoming Pang) Date: Wed, 11 Apr 2007 09:59:37 -0400 Subject: [Bro] Progress in the IPv6 support In-Reply-To: <200704110819.l3B8JqLw029446@jaguar.icir.org> References: <4611C995.2060100@klipix.org> <200704110819.l3B8JqLw029446@jaguar.icir.org> Message-ID: > > - do you think binpac could help me ? > > It will need extensions to do so. You should contact Ruoming Pang > to see what degree he has interest/cycles > in adding the necessary support. Julien, It can probably help you. The difference between using binpac for application level protocols and for IPv6 is that you won't have the notion of connection/flow in IPv6. But binpac can still help you parse the packet headers and generate events. For an example of using binpac without connection/flow, please take a look at smb.pac (it's written before we had the notion of connection/flow in binpac). Do not hesitate to contact me if you have any question. Ruoming From ju at klipix.org Thu Apr 12 07:25:57 2007 From: ju at klipix.org (Julien Desfossez) Date: Thu, 12 Apr 2007 10:25:57 -0400 Subject: [Bro] Progress in the IPv6 support In-Reply-To: <200704110819.l3B8JqLw029446@jaguar.icir.org> References: <200704110819.l3B8JqLw029446@jaguar.icir.org> Message-ID: <461E4175.9070507@klipix.org> Hello, >> - what do you think is the "best pratice" in Bro to code the support of >> ICMPv6 (it's a major update compared to ICMPv4) ? >> > > I don't know the specifics of ICMPv6. What are the main ways in which > it's a major update? By major update I think about : - the replacement of IGMP by MLD (Multicast Listener Discovery) - the Neighbor Discovery Protocol which replaces ARP (broadcast disappears, so does ARP) and adds features such as neighbor unreachability detection, stateless autoconfiguration, duplicate address detection, parameters discovery... So I think there is some work to do to integrate it in Bro (more than just adapting the ICMPv4 part). As soon as I'm done with the extensions and fragments I will start working on it. Any ideas are welcome :-) Best regards Julien Desfossez From mjxie2004 at gmail.com Mon Apr 16 12:22:41 2007 From: mjxie2004 at gmail.com (Mengjun Xie) Date: Mon, 16 Apr 2007 15:22:41 -0400 Subject: [Bro] Question on snort2bro Message-ID: Hello, I find that snort2bro doesn't support "pcre" option in snort rules. Could anyone tell me the reason? I guess the complexity of dealing with non-regular-expression features such as backreference in PCRE could be one of the reasons. But I want to get a confirmed answer. Thanks! Mengjun Xie -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070416/9773739f/attachment.html From robin at icir.org Mon Apr 16 13:33:32 2007 From: robin at icir.org (Robin Sommer) Date: Mon, 16 Apr 2007 13:33:32 -0700 Subject: [Bro] Question on snort2bro In-Reply-To: References: Message-ID: <20070416203332.GB15364@icir.org> On Mon, Apr 16, 2007 at 15:22 -0400, Mengjun Xie wrote: > anyone tell me the reason? I guess the complexity of dealing with > non-regular-expression features such as backreference in PCRE could be one > of the reasons. But I want to get a confirmed answer. Yes, right, or more generally it's the problem of translating between the regexp syntax of system A and the syntax of system B. Actually some time ago I started to write a tool which would convert between a set of different regexp dialects (as far as possible) but that never reached a mature state. Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From jmzhou.ml at gmail.com Wed Apr 18 11:25:15 2007 From: jmzhou.ml at gmail.com (jmzhou.ml at gmail.com) Date: Wed, 18 Apr 2007 11:25:15 -0700 (PDT) Subject: [Bro] small patch for binpac (fwd) Message-ID: Hi, Attached is a small patch against binpac in bro-1.2.1-devel.tar.gz. I think its meaning is clear. I hope to contribute more patches later when I get time. :-) Best wishes, Jimmy -------------- next part -------------- diff -urNp bro-1.2.1.orig/src/binpac/pac_common.h bro-1.2.1/src/binpac/pac_common.h --- bro-1.2.1.orig/src/binpac/pac_common.h 2006-07-26 15:02:40.000000000 -0700 +++ bro-1.2.1/src/binpac/pac_common.h 2007-04-18 11:09:16.000000000 -0700 @@ -105,6 +105,11 @@ typedef vector StateVarList; if ( pc ) \ for ( ct::iterator i = (pc)->begin(); i != (pc)->end(); ++i ) +#define foreach_rev(i, ct, pc) \ + if ( pc ) \ + for ( ct::iterator i = (pc)->end(); i-- != (pc)->begin(); ) + + #define delete_list(ct, pc) \ { \ foreach(delete_list_i, ct, pc) \ diff -urNp bro-1.2.1.orig/src/binpac/pac_expr.def bro-1.2.1/src/binpac/pac_expr.def --- bro-1.2.1.orig/src/binpac/pac_expr.def 2006-07-26 15:02:39.000000000 -0700 +++ bro-1.2.1/src/binpac/pac_expr.def 2007-04-18 11:06:47.000000000 -0700 @@ -14,7 +14,7 @@ EXPR_DEF(EXPR_PLUS, 2, "%s + %s") EXPR_DEF(EXPR_MINUS, 2, "%s - %s") EXPR_DEF(EXPR_TIMES, 2, "%s * %s") EXPR_DEF(EXPR_DIV, 2, "%s / %s") -EXPR_DEF(EXPR_MOD, 2, "%s % %s") +EXPR_DEF(EXPR_MOD, 2, "%s %% %s") EXPR_DEF(EXPR_BITNOT, 1, "~%s") EXPR_DEF(EXPR_BITAND, 2, "%s & %s") EXPR_DEF(EXPR_BITOR, 2, "%s | %s") diff -urNp bro-1.2.1.orig/src/binpac/pac_type.cc bro-1.2.1/src/binpac/pac_type.cc --- bro-1.2.1.orig/src/binpac/pac_type.cc 2006-07-26 15:02:40.000000000 -0700 +++ bro-1.2.1/src/binpac/pac_type.cc 2007-04-18 11:09:48.000000000 -0700 @@ -367,7 +367,7 @@ void Type::GenInitCode(Output* out_cc, E void Type::GenCleanUpCode(Output* out_cc, Env* env) { - foreach (i, FieldList, fields_) + foreach_rev (i, FieldList, fields_) { Field *f = *i; if ( f->tof() != CASE_FIELD ) From rpang at cs.princeton.edu Wed Apr 18 19:16:43 2007 From: rpang at cs.princeton.edu (Ruoming Pang) Date: Wed, 18 Apr 2007 22:16:43 -0400 Subject: [Bro] small patch for binpac (fwd) In-Reply-To: References: Message-ID: Hi Jimmy, Thanks for your interest in contributing the patches. Could you please explain to me offline what this patch is for? Thanks. Ruoming On 4/18/07, jmzhou.ml at gmail.com wrote: > Hi, > > Attached is a small patch against binpac in bro-1.2.1-devel.tar.gz. > I think its meaning is clear. I hope to contribute more patches > later when I get time. :-) > > Best wishes, > > Jimmy > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > From jmzhou.ml at gmail.com Thu Apr 19 09:35:00 2007 From: jmzhou.ml at gmail.com (jmzhou.ml at gmail.com) Date: Thu, 19 Apr 2007 09:35:00 -0700 (PDT) Subject: [Bro] new patch for binpac in bro-1.2.1-dev.tar.gz Message-ID: Hi, This includes the patch I sent yesterday, and a new fix to solve a problem of dependency issue in Type::Prepare. BTW, is there a CVS/SVN tree to track the latest code? It will make patching easier. Cheers, Jimmy ____________________________________________________________ The future is not set. There is no fate but what we make for ourselves. - Terminator II, Judgment Day ------------------------------------------------------------ -------------- next part -------------- diff -urNp bro-1.2.1.orig/src/binpac/pac_common.h bro-1.2.1/src/binpac/pac_common.h --- bro-1.2.1.orig/src/binpac/pac_common.h 2006-07-26 15:02:40.000000000 -0700 +++ bro-1.2.1/src/binpac/pac_common.h 2007-04-18 11:09:16.000000000 -0700 @@ -105,6 +105,11 @@ typedef vector StateVarList; if ( pc ) \ for ( ct::iterator i = (pc)->begin(); i != (pc)->end(); ++i ) +#define foreach_rev(i, ct, pc) \ + if ( pc ) \ + for ( ct::iterator i = (pc)->end(); i-- != (pc)->begin(); ) + + #define delete_list(ct, pc) \ { \ foreach(delete_list_i, ct, pc) \ diff -urNp bro-1.2.1.orig/src/binpac/pac_expr.def bro-1.2.1/src/binpac/pac_expr.def --- bro-1.2.1.orig/src/binpac/pac_expr.def 2006-07-26 15:02:39.000000000 -0700 +++ bro-1.2.1/src/binpac/pac_expr.def 2007-04-18 11:06:47.000000000 -0700 @@ -14,7 +14,7 @@ EXPR_DEF(EXPR_PLUS, 2, "%s + %s") EXPR_DEF(EXPR_MINUS, 2, "%s - %s") EXPR_DEF(EXPR_TIMES, 2, "%s * %s") EXPR_DEF(EXPR_DIV, 2, "%s / %s") -EXPR_DEF(EXPR_MOD, 2, "%s % %s") +EXPR_DEF(EXPR_MOD, 2, "%s %% %s") EXPR_DEF(EXPR_BITNOT, 1, "~%s") EXPR_DEF(EXPR_BITAND, 2, "%s & %s") EXPR_DEF(EXPR_BITOR, 2, "%s | %s") diff -urNp bro-1.2.1.orig/src/binpac/pac_field.cc bro-1.2.1/src/binpac/pac_field.cc --- bro-1.2.1.orig/src/binpac/pac_field.cc 2006-07-26 15:02:39.000000000 -0700 +++ bro-1.2.1/src/binpac/pac_field.cc 2007-04-19 09:23:59.000000000 -0700 @@ -12,6 +12,7 @@ Field::Field(FieldType tof, int flags, I decl_id_ = current_decl_id; field_id_str_ = strfmt("%s:%s", decl_id()->Name(), id_->Name()); attrs_ = 0; + prepared_ = false; } Field::~Field() @@ -73,6 +74,11 @@ int Field::ValueVarType() const void Field::Prepare(Env *env) { + /* prevent infinite recursion */ + if ( prepared_ ) + return ; + prepared_ = true; + if ( type_ ) { if ( anonymous_field() ) diff -urNp bro-1.2.1.orig/src/binpac/pac_field.h bro-1.2.1/src/binpac/pac_field.h --- bro-1.2.1.orig/src/binpac/pac_field.h 2006-07-26 15:02:40.000000000 -0700 +++ bro-1.2.1/src/binpac/pac_field.h 2007-04-19 09:24:05.000000000 -0700 @@ -74,6 +74,7 @@ protected: protected: FieldType tof_; int flags_; + bool prepared_; ID* id_; Type *type_; const ID* decl_id_; diff -urNp bro-1.2.1.orig/src/binpac/pac_type.cc bro-1.2.1/src/binpac/pac_type.cc --- bro-1.2.1.orig/src/binpac/pac_type.cc 2006-07-26 15:02:40.000000000 -0700 +++ bro-1.2.1/src/binpac/pac_type.cc 2007-04-19 09:29:56.000000000 -0700 @@ -305,9 +305,40 @@ void Type::Prepare(Env* env, int flags) } } + /* prepare parameter fields first as they may be dependency */ foreach (i, FieldList, fields_) { Field *f = *i; + if (PARAM_FIELD == f->tof()) + f->Prepare(env); + } + + foreach (i, FieldList, fields_) + { + Field *f = *i; + Expr* req; + if (NULL == f->type() /* can be a padding field */ || + NULL == (req = f->type()->attr_requires())) + { + /* there is no &requires dependents, good! */ + f->Prepare(env); + continue; + } + + /* there are dependents on some &let fields :-( */ + ASSERT(Expr::EXPR_CALLARGS == req->expr_type()); + + FieldList::iterator newi = i+1; + for (; newi != fields_->end(); ++ newi) + { + /* not a field of &let, ignore it */ + Field* newf = *newi; + if (LET_FIELD != newf->tof()) + continue; + + if (req->HasReference (newf->id())) + newf->Prepare(env); + } f->Prepare(env); } } @@ -367,7 +398,7 @@ void Type::GenInitCode(Output* out_cc, E void Type::GenCleanUpCode(Output* out_cc, Env* env) { - foreach (i, FieldList, fields_) + foreach_rev (i, FieldList, fields_) { Field *f = *i; if ( f->tof() != CASE_FIELD ) diff -urNp bro-1.2.1.orig/src/binpac/pac_type.h bro-1.2.1/src/binpac/pac_type.h --- bro-1.2.1.orig/src/binpac/pac_type.h 2006-10-17 15:46:32.000000000 -0700 +++ bro-1.2.1/src/binpac/pac_type.h 2007-04-19 09:28:18.000000000 -0700 @@ -155,6 +155,7 @@ public: Expr *attr_if_expr() const { return attr_if_expr_; } // TODO: generate the length expression automatically. Expr *attr_length_expr() const { return attr_length_expr_; } + Expr *attr_requires() const { return attr_requires_; } bool attr_refcount() const { return attr_refcount_; } bool attr_transient() const { return attr_transient_; } From sriranjan.u at gmail.com Thu Apr 19 23:40:37 2007 From: sriranjan.u at gmail.com (SRIRANJAN.) Date: Fri, 20 Apr 2007 12:10:37 +0530 Subject: [Bro] Signatur match issue with Bro 1.2.1 Message-ID: <9d83443a0704192340h44710435j1de1c78433c3e191@mail.gmail.com> I am facing issue with signature match for the traffic without handshake signals I wrote following signature signature ftp_220response { ip-proto == tcp payload /[\n\r ]*(120|220)[^0-9].*\r\n/ event "FTP Response 220 matched" } this signature match will happen for the traffic with initial SYN,SYN-ACK, and ACK followed by the 220 response FTP packet, but if I gave the traffic without the initial handshakes (i.e. only the 220 response FTP packet ) the same signature will not match, but the signature without line "ip-proto == tcp" will work i.e. signature ftp_220response { payload /[\n\r ]*(120|220)[^0-9].*\r\n/ event "FTP Response 220 matched" } the above signature will work for the both the cases(with and without handshake packets), but i am curious to know why the signatures with the line "ip-proto == tcp" will not work for the asymmetric case. is it the desired way of working?? -Sri -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070420/e93c314f/attachment.html From yuppie4ever at gmail.com Sun Apr 22 13:13:22 2007 From: yuppie4ever at gmail.com (Yuppie) Date: Mon, 23 Apr 2007 01:43:22 +0530 Subject: [Bro] Creating new Val() in multi-threaded bro In-Reply-To: References: Message-ID: <2999b2f00704221313t155cfbedle4063549a3903271@mail.gmail.com> Bro Gurus, I'm trying to make Bro run multi-threaded... so wanted to ask if you guys have any experience with that. First of all, is Bro written to be multi-threaded? If not, can you guys point me to the places which would need some work to make it multi-threaded. Apparently with the complex interplay of data structures, it is hard to find those. I have fixed one such place in Type.cc in: BroType* base_type(TypeTag tag) where it was using a static base_types array. There may more lurking elsewhere, which I haven't been able to find. Esp, I'm facing this issue, where I'm trying to create a new Val() object and bro coredumps in one of the threads. thanks! -y From christian at whoop.org Mon Apr 23 15:55:54 2007 From: christian at whoop.org (Christian Kreibich) Date: Mon, 23 Apr 2007 15:55:54 -0700 Subject: [Bro] Creating new Val() in multi-threaded bro In-Reply-To: <2999b2f00704221313t155cfbedle4063549a3903271@mail.gmail.com> References: <2999b2f00704221313t155cfbedle4063549a3903271@mail.gmail.com> Message-ID: <1177368954.18483.33.camel@strangepork> Hi, On Mon, 2007-04-23 at 01:43 +0530, Yuppie wrote: > Bro Gurus, > > I'm trying to make Bro run multi-threaded... so wanted to ask if you > guys have any experience with that. First of all, is Bro written to be > multi-threaded? Not currently, no. > If not, can you guys point me to the places which > would need some work to make it multi-threaded. Apparently with the > complex interplay of data structures, it is hard to find those. I have > fixed one such place in Type.cc in: > BroType* base_type(TypeTag tag) > where it was using a static base_types array. There may more lurking > elsewhere, which I haven't been able to find. > > Esp, I'm facing this issue, where I'm trying to create a new Val() > object and bro coredumps in one of the threads. There are surely many places in the code in which static variables may collide with multi-threaded operation. There will also be a number of synchronization issues. The real question is what you are actually trying to achieve, so you can adjust the architecture to run the relevant tasks in separate threads. This may be non-trivial. That said, it would clearly be interesting to parallelize the operation of individual analyzers, for example. Cheers, Christian -- ________________________________________________________________________ http://www.icir.org/christian http://www.whoop.org From alen at wiretap.net Mon Apr 23 19:41:30 2007 From: alen at wiretap.net (Alen Capalik) Date: Mon, 23 Apr 2007 19:41:30 -0700 Subject: [Bro] Question about creating custom conn logs. Message-ID: Hi all, I was wondering if someone can point me in the right direction regarding creating custom connection logs in Bro. I'm sorry in advance if this is a question already asked before, but I could not find the answer or something remotely close to an answer. I want to add some fields to current conn..log files (namely for instance tcp sequence numbers) for all tcp connections or I want to create new connection log files with new fields. I have written new function (similar to record_connection()) in .bro file as well as new event calling that function. The log files get created, but nothing is ever logged into them. I guess my question would be, how do I create an event calling this function that will just record all tcp network traffic into customized log files? I really don't want to do anything special to it, I just want to log the traffic with the format I defined in the function. I have been using pkt_hdr, ip_hdr, tcp_hdr, udp_hdr data types from bro.init to add additional fields in fmt(). If you can just give me a quick format of the event that would do that, it would help tremendously, I can do the rest myself. I hope this is enough information, if not please let me know. Thank you in advance for any help. Regards, Alen From yuppie4ever at gmail.com Tue Apr 24 07:19:18 2007 From: yuppie4ever at gmail.com (Yuppie) Date: Tue, 24 Apr 2007 19:49:18 +0530 Subject: [Bro] Connection dictionaries Message-ID: <2999b2f00704240719t69b77d53u3b6c548780d06416@mail.gmail.com> I see that there are different connection dictionaries for tcp, udp and icmp connections. These are indexed by originator/responder ip/port 4-tuple. Is there a specific reason (apart from performance maybe?) for going with this approach rather than creating a single dictionary indexed by a 5-tuple, 5th-tuple being the protocol? thanks -y From mtdedlow at lbl.gov Tue Apr 24 07:45:41 2007 From: mtdedlow at lbl.gov (Mark Dedlow) Date: Tue, 24 Apr 2007 07:45:41 -0700 Subject: [Bro] Question about creating custom conn logs. In-Reply-To: References: Message-ID: <462E1815.10004@lbl.gov> Alen, I'm not a bro expert, just a novice user, but here's what I think. First, if you have questions about your code, post the code. It's hard to tell what exactly you're doing from the description. You don't need to create an event to do what you want. Rather, you create an *event handler*. Bro generates the events, and runs your handler at the appropriate time. For example, here's an event handler that gets run when a connection is finished, that is, this is a handler for the built-in 'connection_finished' event. The handler receives a single parameter, which is a connection record. event connection_finished(c: connection) { print fmt("conn %s:%d -> %s:%d", c$id$orig_h, c$id$orig_p, c$id$resp_h, c$id$resp_p) ; } Print this to a file, and that is a minimal example of a custom connection logging policy. (note that in a real app, connection_finished is not the event you want, since it will not run for a connection that never finishes explicitly (FIN). The event connection_state_remove fires shortly after connection_finished, or when an unfinished connection is timed out) As far as logging sequence numbers, keep in mind that connections are at a different level of abstraction than sequence numbers, which exist at the packet level. The bro 'connection' record has no record of any individual packets. To get sequence numbers, you need to write handlers for packet-level events. Here's an example of a policy that would record the sequence number of the SYN packet for a connection. Note that the built-in event 'new_packet' is passed parameters of connection record and the packet, so you can correlate packets with connections. global conns_syn_seq: table[conn_id] of count; event new_packet(c: connection, p: pkt_hdr) { # keep the SYN sequence number if ( p$tcp$flags == 2 ) conns_syn_seq[c$id] = p$tcp$seq; } event connection_finished(c: connection) { print fmt("conn %s:%d -> %s:%d (SYNseq#=%d)", c$id$orig_h, c$id$orig_p, c$id$resp_h, c$id$resp_p, conns_syn_seq[c$id] ); } Mark Alen Capalik wrote: > Hi all, > > I was wondering if someone can point me in the right direction > regarding creating custom connection logs in Bro. I'm sorry in > advance if this is a question already asked before, but I could not > find the answer or something remotely close to an answer. I want to > add some fields to current conn..log files (namely for instance > tcp sequence numbers) for all tcp connections or I want to create new > connection log files with new fields. I have written new function > (similar to record_connection()) in .bro file as well as new > event calling that function. The log files get created, but nothing > is ever logged into them. > > I guess my question would be, how do I create an event calling this > function that will just record all tcp network traffic into customized > log files? I really don't want to do anything special to it, I just > want to log the traffic with the format I defined in the function. I > have been using pkt_hdr, ip_hdr, tcp_hdr, udp_hdr data types from > bro.init to add additional fields in fmt(). If you can just give me a > quick format of the event that would do that, it would help > tremendously, I can do the rest myself. > > I hope this is enough information, if not please let me know. Thank > you in advance for any help. > > Regards, > Alen > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From yuppie4ever at gmail.com Tue Apr 24 07:57:08 2007 From: yuppie4ever at gmail.com (Yuppie) Date: Tue, 24 Apr 2007 20:27:08 +0530 Subject: [Bro] Creating new Val() in multi-threaded bro In-Reply-To: <1177368954.18483.33.camel@strangepork> References: <2999b2f00704221313t155cfbedle4063549a3903271@mail.gmail.com> <1177368954.18483.33.camel@strangepork> Message-ID: <2999b2f00704240757w6a6fa1fas5ea5b93908aa14d8@mail.gmail.com> On 4/24/07, Christian Kreibich wrote: > There are surely many places in the code in which static variables may > collide with multi-threaded operation. There will also be a number of > synchronization issues. The real question is what you are actually > trying to achieve, so you can adjust the architecture to run the > relevant tasks in separate threads. This may be non-trivial. That said, > it would clearly be interesting to parallelize the operation of > individual analyzers, for example. I'm playing with the idea of remotely controlling Bro operation/configuration. Sorry, I'm not interested in Broccoli as that is a non-standard interface. For that I'm creating a separate thread that accepts SOAP messages and controls Bro on-the-fly. I could really do it as a single thread, but it's cleaner the other way. I'm curious though, if the broccoli interface/api is a binary or a text one! Your idea of parallelizing the various analyzers is something I have considered. Really, it'll only make sense parallelizing the analyzers on a single packet. Analyzing multiple packets at the same instant would create synchronization problems of their own sort. What if the analysis of second packet ends up finishing before the first though it was dependent on the first? This is true not just for packets from same connection. Distinct connections may be related too and may need synchronous processing. Talking of analyzing single packets (at an instant) using multiple analyzers, don't know how beneficial that is really. Can't think of too many cases where this is helpful. Especially, even in these cases, after analyzing the first packet, mostly only a single analyzer remains interested. And from what I know, subsequent packets belonging to the same connection can re-use the analyzer information stored in the connection entry. Having said that, parallelizing Event dispatches would be interesting, since there's no guarantee of order in Event Handler execution for a given event anyways. cheers -y From vern at icir.org Tue Apr 24 13:32:13 2007 From: vern at icir.org (Vern Paxson) Date: Tue, 24 Apr 2007 13:32:13 -0700 Subject: [Bro] Creating new Val() in multi-threaded bro In-Reply-To: <2999b2f00704240757w6a6fa1fas5ea5b93908aa14d8@mail.gmail.com> (Tue, 24 Apr 2007 20:27:08 +0530). Message-ID: <200704242032.l3OKWDdf009612@jaguar.icir.org> > Sorry, I'm not interested in Broccoli as that > is a non-standard interface. For that I'm creating a separate thread > that accepts SOAP messages and controls Bro on-the-fly. Well sorry likewise, as we won't be interested in your possible contribution in that case. > Your idea of parallelizing the various analyzers is something I have > considered. Really, it'll only make sense parallelizing the analyzers > on a single packet. Please see our papers which explore the possibilities in depth: Rethinking Hardware Support for Network Analysis and Intrusion Prevention, V. Paxson et al., Proc. USENIX Hot Security, August 2006. http://www.icir.org/vern/papers/hotsec06.pdf An Architecture for Exploiting Multi-Core Processors to Parallelize Network Intrusion Prevention, V. Paxson, R. Sommer, and N. Weaver, Proc. IEEE Sarnoff Symposium, May 2007, to appear. http://www.icir.org/vern/papers/multicore-sarnoff07.pdf - Vern From vern at icir.org Tue Apr 24 13:36:16 2007 From: vern at icir.org (Vern Paxson) Date: Tue, 24 Apr 2007 13:36:16 -0700 Subject: [Bro] Connection dictionaries In-Reply-To: <2999b2f00704240719t69b77d53u3b6c548780d06416@mail.gmail.com> (Tue, 24 Apr 2007 19:49:18 +0530). Message-ID: <200704242036.l3OKaGTp009916@jaguar.icir.org> > Is there a specific reason (apart from performance > maybe?) for going with this approach rather than creating a single > dictionary indexed by a 5-tuple, 5th-tuple being the protocol? That structure arose due to how the code evolved. Using 4-tuples instead of 5-tuples saves a small amount of memory, too. Vern From robin at icir.org Tue Apr 24 16:50:29 2007 From: robin at icir.org (Robin Sommer) Date: Tue, 24 Apr 2007 16:50:29 -0700 Subject: [Bro] Signatur match issue with Bro 1.2.1 In-Reply-To: <9d83443a0704192340h44710435j1de1c78433c3e191@mail.gmail.com> References: <9d83443a0704192340h44710435j1de1c78433c3e191@mail.gmail.com> Message-ID: <20070424235029.GA678@icir.org> On Fri, Apr 20, 2007 at 12:10 +0530, you wrote: > handshake packets), but i am curious to know why the signatures with the > line "ip-proto == tcp" will not work for the asymmetric case. That's indeed an interesting difference. Can you send me the two traces? Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From yuppie4ever at gmail.com Tue Apr 24 22:28:51 2007 From: yuppie4ever at gmail.com (Yuppie) Date: Wed, 25 Apr 2007 10:58:51 +0530 Subject: [Bro] Creating new Val() in multi-threaded bro In-Reply-To: <200704242032.l3OKWDdf009612@jaguar.icir.org> References: <2999b2f00704240757w6a6fa1fas5ea5b93908aa14d8@mail.gmail.com> <200704242032.l3OKWDdf009612@jaguar.icir.org> Message-ID: <2999b2f00704242228v78b1ad2fi1af9ed89487af814@mail.gmail.com> On 4/25/07, Vern Paxson wrote: > > Sorry, I'm not interested in Broccoli as that > > is a non-standard interface. For that I'm creating a separate thread > > that accepts SOAP messages and controls Bro on-the-fly. > > Well sorry likewise, as we won't be interested in your possible contribution > in that case. I hope my comment didn't come across as forthright sacrilege, or a snide advice for that matter. I just meant that requirements differ in my case. I trust that Broccoli does well at what it is inteded to. > Please see our papers which explore the possibilities in depth: Thanks for the pointers to these. -y From ewages at colsa.com Thu Apr 26 06:59:02 2007 From: ewages at colsa.com (Eric Wages) Date: Thu, 26 Apr 2007 08:59:02 -0500 Subject: [Bro] Alerting question on multi-homed bro box Message-ID: <21D34621-6C6B-4DC0-8526-D55E84908DEE@colsa.com> We have bro configured to span a firewall, so we are watching the outside traffic, both inbound and outbound, as well as the internal interface both in and out. We're doing this as a sanity check to verify that, when attacks occur, if they penetrate and are successful. One thing that I'm seeing is that, for example, we can successful triggers like IRC nickname changes, occur with the server & outside IP address, but we don't see an equivalent trigger on the internal interface. Consider the following alarm: t=1177511147.420130 no=SensitiveSignature na=NOTICE_ALARM_ALWAYS sa=A.B.C.D sp=1145/tcp da=E.F.G.H dp=6667/tcp file=s2b-542-10 msg=A.B.C.D:\ CHAT\ IRC\ nick\ change sub=NICK\ Homerpf|CHAMP^J tag=@29 where A.B.C.D is one of the outside IP's associated with either the NAT or PAT range on our firewall. Should I also not see an equivalent trigger like: t=1177511147.420130 no=SensitiveSignature na=NOTICE_ALARM_ALWAYS sa=192.168.x.x sp=1145/tcp da=E.F.G.H dp=6667/tcp file=s2b-542-10 msg=192.168.x.x:\ CHAT\ IRC\ nick\ change sub=NICK\ Homerpf|CHAMP^J tag=@29 Where the source is the IP of the internal machine? If not, is there any way we can configure bro to show those internal entries, since it will help us find machines that are acting in a naughty manner. Thanks, -Eric Eric Wages From vern at icir.org Fri Apr 27 21:15:45 2007 From: vern at icir.org (Vern Paxson) Date: Fri, 27 Apr 2007 21:15:45 -0700 Subject: [Bro] public SVN (Re: new patch for binpac in bro-1.2.1-dev.tar.gz) In-Reply-To: (Thu, 19 Apr 2007 09:35:00 PDT). Message-ID: <200704280415.l3S4Fjkk006300@jaguar.icir.org> > BTW, is there a CVS/SVN tree to track the latest code? It will > make patching easier. There isn't, and we don't have any established plans to do so. However, in general for integration it has worked for contributors to send us context diffs with respect to the latest public release. Vern From vern at icir.org Fri Apr 27 21:16:16 2007 From: vern at icir.org (Vern Paxson) Date: Fri, 27 Apr 2007 21:16:16 -0700 Subject: [Bro] Alerting question on multi-homed bro box In-Reply-To: <21D34621-6C6B-4DC0-8526-D55E84908DEE@colsa.com> (Thu, 26 Apr 2007 08:59:02 CDT). Message-ID: <200704280416.l3S4GGHa006355@jaguar.icir.org> To make sure I understand the scenario correctly: you have a single Bro that has NICs to passively monitor both sides of your firewall, *and* the firewall does NAT, so in fact across the two NICS Bro sees two distinct connections rather than redundant packets for the same connections - is that right? If so, then I'm puzzled by the problem you're seeing. The first step in debugging it would be to capture traces using both NICs (say using two instances of tcpdump, one reading each NIC) and then bro -r trace1 -r trace2 to process them together and see whether the same behavior manifests. If it does, the next thing would be to merge thet traces (easiest with ipsumdump --collate) and see whether the behavior still manifiests from the single trace. Vern