From geek00l at gmail.com Mon Sep 17 20:05:18 2007 From: geek00l at gmail.com (CS Lee) Date: Tue, 18 Sep 2007 11:05:18 +0800 Subject: [Bro] Rotate log time issue - bro seg fault Message-ID: <1bb5dd90709172005j56a05b16ra73b3fd166ca2ee8@mail.gmail.com> Hi, We have installed bro 1.3.2(expect the edge ;]) on Ubuntu 7.04 without much hassles, and we are currently practicing on writing the bro script, but during the loading of brolite policy script, the bro crashed with segmentation fault. It goes in this way - gdb bro GNU gdb 6.6-debian Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i486-linux-gnu"... Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1". (gdb) run -r ../fl0p-skype-sig.pcap brolite Starting program: /usr/local/bin/bro -r ../fl0p-skype-sig.pcap brolite Program received signal SIGSEGV, Segmentation fault. 0x086a67d7 in ?? () (gdb) backtrace #0 0x086a67d7 in ?? () #1 0x080de4a7 in BroFile::InstallRotateTimer (this=0x8990480) at File.cc:562 #2 0x080de5f8 in BroFile::Open (this=0x8990480, file=0x891c218) at File.cc :192 #3 0x080df663 in BroFile::Rotate (this=0x8990480) at File.cc:528 #4 0x080f8314 in bro_rotate_file (frame=0x88b1598, BiF_ARGS=0x8a5b5c8) at bro.bif:2393 #5 0x080e8a4d in BuiltinFunc::Call (this=0x8362020, args=0x8a5b5c8, parent=0x88b1598) at Func.cc:467 #6 0x080da56c in CallExpr::Eval (this=0x8a2b3f0, f=0x88b1598) at Expr.cc :4501 #7 0x080c4a5f in AssignExpr::Eval (this=0x8a2b200, f=0x88b1598) at Expr.cc :2562 #8 0x08179cdc in ExprStmt::Exec (this=0x8a2b590, f=0x88b1598, flow=@0xbff49924) at Stmt.cc:395 #9 0x081756c9 in StmtList::Exec (this=0x8a2b020, f=0x88b1598, flow=@0xbff49924) at Stmt.cc:1391 #10 0x080e8e24 in BroFunc::Call (this=0x8a2bc58, args=0x8a5c258, parent=0x88aca08) at Func.cc:324 #11 0x080da56c in CallExpr::Eval (this=0x8a2f820, f=0x88aca08) at Expr.cc :4501 #12 0x08179cdc in ExprStmt::Exec (this=0x8a2f880, f=0x88aca08, flow=@0xbff49a74) at Stmt.cc:395 #13 0x081756c9 in StmtList::Exec (this=0x8a2f118, f=0x88aca08, flow=@0xbff49a74) at Stmt.cc:1391 #14 0x080e8e24 in BroFunc::Call (this=0x8a2f8e0, args=0x828d698, parent=0x0) at Func.cc:324 #15 0x080a8cf6 in EventHandler::Call (this=0x8a2f9b0, vl=0x828d698, no_remote=true) at EventHandler.cc:64 #16 0x080dfaf3 in BroFile::CloseCachedFiles () at Event.h:59 #17 0x080501aa in main (argc=553648128, argv=0xbff49eb4) at main.cc:1017 (gdb) frame 1 #2 0x080de4a7 in BroFile::InstallRotateTimer (this=0x837c5f8) at File.cc:562 562 timer_mgr->Add(rotate_timer); (gdb) frame 2 #3 0x080de5f8 in BroFile::Open (this=0x837c5f8, file=0x837c720) at File.cc :192 192 InstallRotateTimer(); (gdb) frame 3 #4 0x080df663 in BroFile::Rotate (this=0x837c5f8) at File.cc:528 528 Open(newf); (gdb) frame 4 #5 0x080f8314 in bro_rotate_file (frame=0x84e79e0, BiF_ARGS=0x84e5f10) at bro.bif:2393 2393 RecordVal* info = f->Rotate(); This lead us to believe something wrong with the log rotation(time issue), therefore we tried running bro with this bro -r fl0p-skype-sig.pcap tcp rotate-logs Immediately it crashes, and if we disable the log rotation in brolite, everything goes fine. Looking at our pcap file metadata - capinfos fl0p-skype-sig.pcapFile name: fl0p-skype-sig.pcap File type: Wireshark/tcpdump/... - libpcap Number of packets: 368874 File size: 75144608 bytes Data size: 69242600 bytes Capture duration: 3892.835282 seconds Start time: Sun Sep 9 10:02:58 2007 End time: Sun Sep 9 11:07:51 2007 Data rate: 17787.19 bytes/s Data rate: 142297.52 bits/s Average packet size: 187.71 bytes So this pcap timeline span is around 1 hour, we tune the interval of the log rotation and it may crash in different points and that seems to be the time issue. Btw, we don't have such issue when using bro-1.2 on MacOSX, Gentoo and bro-1.3.2 on FreeBSD 6.2. Thanks. -- Best Regards, CS Lee http://geek00l.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070918/b91becba/attachment.html From geek00l at gmail.com Mon Sep 17 21:29:45 2007 From: geek00l at gmail.com (CS Lee) Date: Tue, 18 Sep 2007 12:29:45 +0800 Subject: [Bro] Rotate log time issue - bro seg fault In-Reply-To: <46EF4C38.5020907@lbl.gov> References: <1bb5dd90709172005j56a05b16ra73b3fd166ca2ee8@mail.gmail.com> <46EF4C38.5020907@lbl.gov> Message-ID: <1bb5dd90709172129l335fbd59qe112f09eff9c673a@mail.gmail.com> Hi Scott, Here's the last part of result in trace 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:102 event called: rotate_interval(f = 'file "weird.log" of string') 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:99 Builtin Function called: bro_is_terminating() 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:99 Function return: T 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:71 function called: RotateLogs::rotate(f = 'file "weird.log" of s tring') 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:66 Builtin Function called: rotate_file(f = 'file "weird. log" of string') 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:66 Function return: [old_name=weird.log, new_name=weird.l og.27507.1189307168.792985.tmp, open=1190089477.64701, close= 1189307168.79298] 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:60 function called: RotateLogs::run_pp(info = '[old_name= weird.log, new_name=weird.log.27507.1189307168.792985.tmp, open= 1190089477.64701, close=1189307168.79298]') 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:41 function called: RotateLogs::build_name(info = '[old_name=weird.log, new_name=weird.log.27507.1189307168.792985.tmp, open= 1190089477.64701, close=1189307168.79298]') 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:41 Builtin Function called: strftime(fmt = '%y-%m-%d_%H.%M.%S', d = '1190089477.64701') 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:41 Function return: 07-09-18_12.24.37 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:41 Builtin Function called: fmt(va_args = '%s-%s', vararg0 = 'weird.log', vararg1 = '07-09-18_12.24.37') 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:41 Function return: weird.log-07-09-18_12 .24.37 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:41 Function return: weird.log-07-09-18_12.24.37 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:60 Builtin Function called: fmt(va_args = '/bin/m v %s %s', vararg0 = 'weird.log.27507.1189307168.792985.tmp', vararg1 = ' weird.log-07-09-18_12.24.37') 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:60 Function return: /bin/mv weird.log.27507.11893 07168.792985.tmp weird.log-07-09-18_12.24.37 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:60 Builtin Function called: system(str = '/bin/mv weird.log.27507.1189307168.792985.tmp weird.log-07-09-18_12.24.37') 1189307168.792985 /usr/local/stow/bro-1.3.2/policy/rotate-logs.bro:60 Function return: 0 On 9/18/07, scott campbell wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I have seen a similar problem with InstallRotateTimer when bro is > exiting (ie with bro.rc --checkpoint) on FreeBSD 6.2, but not at > startup. You might get a little better information if you start up the > instance with the trace option (-t ) which will let you know > exactly what the policy side is doing. > > > scott > CS Lee wrote: > > Hi, > > > > We have installed bro 1.3.2(expect the edge ;]) on Ubuntu 7.04 without > much > > hassles, and we are currently practicing on writing the bro script, but > > during the loading of brolite policy script, the bro crashed > > with segmentation fault. It goes in this way - > > > > gdb bro > > GNU gdb 6.6-debian > > Copyright (C) 2006 Free Software Foundation, Inc. > > GDB is free software, covered by the GNU General Public License, and you > are > > welcome to change it and/or distribute copies of it under certain > > conditions. > > Type "show copying" to see the conditions. > > There is absolutely no warranty for GDB. Type "show warranty" for > details. > > This GDB was configured as "i486-linux-gnu"... > > Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1". > > > > (gdb) run -r ../fl0p-skype-sig.pcap brolite > > Starting program: /usr/local/bin/bro -r ../fl0p-skype-sig.pcap brolite > > > > Program received signal SIGSEGV, Segmentation fault. > > 0x086a67d7 in ?? () > > (gdb) backtrace > > #0 0x086a67d7 in ?? () > > #1 0x080de4a7 in BroFile::InstallRotateTimer (this=0x8990480) at File.cc > :562 > > #2 0x080de5f8 in BroFile::Open (this=0x8990480, file=0x891c218) at > File.cc > > :192 > > #3 0x080df663 in BroFile::Rotate (this=0x8990480) at File.cc:528 > > #4 0x080f8314 in bro_rotate_file (frame=0x88b1598, BiF_ARGS=0x8a5b5c8) > at > > bro.bif:2393 > > #5 0x080e8a4d in BuiltinFunc::Call (this=0x8362020, args=0x8a5b5c8, > > parent=0x88b1598) at Func.cc:467 > > #6 0x080da56c in CallExpr::Eval (this=0x8a2b3f0, f=0x88b1598) at Expr.cc > > :4501 > > #7 0x080c4a5f in AssignExpr::Eval (this=0x8a2b200, f=0x88b1598) at > Expr.cc > > :2562 > > #8 0x08179cdc in ExprStmt::Exec (this=0x8a2b590, f=0x88b1598, > > flow=@0xbff49924) at Stmt.cc:395 > > #9 0x081756c9 in StmtList::Exec (this=0x8a2b020, f=0x88b1598, > > flow=@0xbff49924) at Stmt.cc:1391 > > #10 0x080e8e24 in BroFunc::Call (this=0x8a2bc58, args=0x8a5c258, > > parent=0x88aca08) at Func.cc:324 > > #11 0x080da56c in CallExpr::Eval (this=0x8a2f820, f=0x88aca08) at > Expr.cc > > :4501 > > #12 0x08179cdc in ExprStmt::Exec (this=0x8a2f880, f=0x88aca08, > > flow=@0xbff49a74) at Stmt.cc:395 > > #13 0x081756c9 in StmtList::Exec (this=0x8a2f118, f=0x88aca08, > > flow=@0xbff49a74) at Stmt.cc:1391 > > #14 0x080e8e24 in BroFunc::Call (this=0x8a2f8e0, args=0x828d698, > parent=0x0) > > at Func.cc:324 > > #15 0x080a8cf6 in EventHandler::Call (this=0x8a2f9b0, vl=0x828d698, > > no_remote=true) at EventHandler.cc:64 > > #16 0x080dfaf3 in BroFile::CloseCachedFiles () at Event.h:59 > > #17 0x080501aa in main (argc=553648128, argv=0xbff49eb4) at main.cc:1017 > > > > (gdb) frame 1 > > #2 0x080de4a7 in BroFile::InstallRotateTimer (this=0x837c5f8) at File.cc > :562 > > 562 timer_mgr->Add(rotate_timer); > > (gdb) frame 2 > > #3 0x080de5f8 in BroFile::Open (this=0x837c5f8, file=0x837c720) at > File.cc > > :192 > > 192 InstallRotateTimer(); > > (gdb) frame 3 > > #4 0x080df663 in BroFile::Rotate (this=0x837c5f8) at File.cc:528 > > 528 Open(newf); > > (gdb) frame 4 > > #5 0x080f8314 in bro_rotate_file (frame=0x84e79e0, BiF_ARGS=0x84e5f10) > at > > bro.bif:2393 > > 2393 RecordVal* info = f->Rotate(); > > > > This lead us to believe something wrong with the log rotation(time > issue), > > therefore we tried running bro with this > > > > bro -r fl0p-skype-sig.pcap tcp rotate-logs > > > > Immediately it crashes, and if we disable the log rotation in brolite, > > everything goes fine. Looking at our pcap file metadata - > > > > capinfos fl0p-skype-sig.pcapFile > > name: fl0p-skype-sig.pcap > > File type: Wireshark/tcpdump/... - libpcap > > Number of packets: 368874 > > File size: 75144608 bytes > > Data size: 69242600 bytes > > Capture duration: 3892.835282 seconds > > Start time: Sun Sep 9 10:02:58 2007 > > End time: Sun Sep 9 11:07:51 2007 > > Data rate: 17787.19 bytes/s > > Data rate: 142297.52 bits/s > > Average packet size: 187.71 bytes > > > > So this pcap timeline span is around 1 hour, we tune the interval of the > log > > rotation and it may crash in different points and that seems to be the > time > > issue. > > > > Btw, we don't have such issue when using bro-1.2 on MacOSX, Gentoo and > > bro-1.3.2 on FreeBSD 6.2. > > > > Thanks. > > > > > > > > > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.2.6 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFG70w4K2Plq8B7ZBwRArP8AKCv3j3B6OdYUuvNKI5hsUGibSMv4wCeJ7e4 > SunJ7vjlH+urau1+KqPXUs8= > =vfny > -----END PGP SIGNATURE----- > -- Best Regards, CS Lee -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070918/20424dac/attachment.html From vern at icir.org Mon Sep 17 22:51:53 2007 From: vern at icir.org (Vern Paxson) Date: Mon, 17 Sep 2007 22:51:53 -0700 Subject: [Bro] Rotate log time issue - bro seg fault In-Reply-To: <1bb5dd90709172005j56a05b16ra73b3fd166ca2ee8@mail.gmail.com> (Tue, 18 Sep 2007 11:05:18 +0800). Message-ID: <200709180551.l8I5prOX018822@jaguar.icir.org> > #1 0x080de4a7 in BroFile::InstallRotateTimer (this=0x8990480) at File.cc:562 > .... > #16 0x080dfaf3 in BroFile::CloseCachedFiles () at Event.h:59 > #17 0x080501aa in main (argc=553648128, argv=0xbff49eb4) at main.cc:1017 Try the appended and see if it does the trick. Vern Index: main.cc =================================================================== --- main.cc (revision 4656) +++ main.cc (revision 4657) @@ -281,12 +281,12 @@ done_with_network(); net_delete(); + terminate_bro(); + // Close files after net_delete(), because net_delete() // might write to connection content files. BroFile::CloseCachedFiles(); - terminate_bro(); - delete rule_matcher; exit(0); @@ -1010,12 +1010,12 @@ done_with_network(); net_delete(); + terminate_bro(); + // Close files after net_delete(), because net_delete() // might write to connection content files. BroFile::CloseCachedFiles(); - terminate_bro(); - #ifdef USE_MPATROL fputs( "Stopping mpatrol logging...", stderr ); sig_mpatrol(SIGCONT); From geek00l at gmail.com Tue Sep 18 00:21:37 2007 From: geek00l at gmail.com (CS Lee) Date: Tue, 18 Sep 2007 15:21:37 +0800 Subject: [Bro] Rotate log time issue - bro seg fault In-Reply-To: <200709180551.l8I5prOX018822@jaguar.icir.org> References: <1bb5dd90709172005j56a05b16ra73b3fd166ca2ee8@mail.gmail.com> <200709180551.l8I5prOX018822@jaguar.icir.org> Message-ID: <1bb5dd90709180021o64522519ldf4cfc21ded56094@mail.gmail.com> Hi Vern, The bro-1.3.2. that we installed having the same revision which is 4657 - head -1 main.cc // $Id: main.cc 4657 2007-07-24 20:37:07Z vern $ And it seems to be the same. On 9/18/07, Vern Paxson wrote: > > > #1 0x080de4a7 in BroFile::InstallRotateTimer (this=0x8990480) at File.cc > :562 > > .... > > #16 0x080dfaf3 in BroFile::CloseCachedFiles () at Event.h:59 > > #17 0x080501aa in main (argc=553648128, argv=0xbff49eb4) at main.cc:1017 > > Try the appended and see if it does the trick. > > Vern > > > Index: main.cc > =================================================================== > --- main.cc (revision 4656) > +++ main.cc (revision 4657) > @@ -281,12 +281,12 @@ > done_with_network(); > net_delete(); > > + terminate_bro(); > + > // Close files after net_delete(), because net_delete() > // might write to connection content files. > BroFile::CloseCachedFiles(); > > - terminate_bro(); > - > delete rule_matcher; > > exit(0); > @@ -1010,12 +1010,12 @@ > done_with_network(); > net_delete(); > > + terminate_bro(); > + > // Close files after net_delete(), because net_delete() > // might write to connection content files. > BroFile::CloseCachedFiles(); > > - terminate_bro(); > - > #ifdef USE_MPATROL > fputs( "Stopping mpatrol logging...", stderr ); > sig_mpatrol(SIGCONT); > -- Best Regards, CS Lee -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070918/e7d7495f/attachment.html From geek00l at gmail.com Tue Sep 18 00:39:48 2007 From: geek00l at gmail.com (CS Lee) Date: Tue, 18 Sep 2007 15:39:48 +0800 Subject: [Bro] UDP flow anomaly Message-ID: <1bb5dd90709180039g7770d9c8tf5e2fd31f0fa2164@mail.gmail.com> Hi all, We are trying to write the simple script where it will check if the udp reply is 18 bytes and will return the src ip, dst ip as well as its bytes exchanges as illustrate as below src ip ------> dst ip src ip <----- dst ip (18 bytes) It is pretty simple and works well to detect the udp probing, and we are happy with it but when we test it on one of the packet capture which supposingly having 3 udp packets with 18 bytes return, it only detects one of it but didn't see the rest. And running through tcpdump, we figure this - 2007-09-13 07:32:30.298295 IP 192.168.0.185.63025 > 210.79.186.143.23074: UDP, length 22 2007-09-13 07:32:30.566344 IP 210.79.186.143.23074 > 192.168.0.185.63025: UDP, length 18 2007-09-13 07:32:34.790093 IP 192.168.0.185.63025 > 89.37.157.114.28763: UDP, length 41 2007-09-13 07:32:34.791688 IP 192.168.0.185.63025 > 200.83.176.80.49847: UDP, length 41 2007-09-13 07:32:34.809467 IP 192.168.0.185.63025 > 89.37.157.114.28763: UDP, length 33 2007-09-13 07:32:34.809531 IP 192.168.0.185.63025 > 200.83.176.80.49847: UDP, length 33 2007-09-13 07:32:35.331256 IP 200.83.176.80.49847 > 192.168.0.185.63025: UDP, length 19 *2007-09-13 07:32:35.350840 IP 200.83.176.80.49847 > 192.168.0.185.63025: UDP, length 18* 2007-09-13 07:32:35.403002 IP 89.37.157.114.28763 > 192.168.0.185.63025: UDP, length 19 *2007-09-13 07:32:35.407810 IP 89.37.157.114.28763 > 192.168.0.185.63025: UDP, length 18** * The script can locate the 210.79.186.143 but not 200.83.176.80 and 89.37.157.114. That lead us to believe that bro understand the flow in semantic level. In fact if we do the matching to 18+19 = 37 bytes, it detects the other 2. And just learn about the trace feature from scott, we immediately tried the trace and we found this - 1189639955.350840 /usr/local/stow/bro-1.3.2/policy/hot.bro:153 function called: check_hot(c = '[id=[orig_h=192.168.0.185, orig_p=6302 5/udp, resp_h=200.83.176.80, resp_p=49847/udp], orig=[size=74, state=1], resp=[*size=37*, state=1], start_time=1189639954.79169, duration= 0.559152126312256, service=, addl=, hot=0, history=Dd]', state = '2') .....blablabla another one Here's our simple and shameful script - *@load udp-common redef capture_filters += { ["udp"] = "udp" }; redef local_nets: set[subnet] = { 192.168.0.0/24 }; ** event udp_reply(u: connection) { local orig = u$id$orig_h; local resp = u$id$resp_h; local origs = u$orig$size; local resps = u$resp$size; if ( u$resp$size == 18) { print ("Suspected udp probe"); print fmt("%20s %5s %5s %5s", orig, resp, origs, resps); }* } I don't see we can use udp time out for this as the interval of return packet is too low. Or is there workaround to examine the first corresponding udp reply packet size. Thanks! -- Best Regards, CS Lee -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070918/3e416497/attachment.html From mel at hackinthebox.org Tue Sep 18 00:40:36 2007 From: mel at hackinthebox.org (mel at hackinthebox.org) Date: Tue, 18 Sep 2007 15:40:36 +0800 (MYT) Subject: [Bro] howto: getting the port number only Message-ID: <42809.211.25.195.202.1190101236.squirrel@mail.hackinthebox.org> Hi all, I have: global destinations: set[addr,port]; The port data type will store port information in the following format: 443/tcp, 22/tcp, 53/udp, etc. However, I'm only interested in the port number, not the protocol. How do I get the port number only? --mel From fabian at net.t-labs.tu-berlin.de Tue Sep 18 04:07:44 2007 From: fabian at net.t-labs.tu-berlin.de (Fabian Schneider) Date: Tue, 18 Sep 2007 13:07:44 +0200 (CEST) Subject: [Bro] Bro segfaults with unknown linktype Message-ID: Hi, we found a bug that segfaults bro. If you try to open a pcap file with a data link type that is unknown to libpcap (e.g. data link type 0x32), then it immediately crashes. The reason is in PktSrc.cc: 1.) In the constructor of PktFileSrc you first do an pcap_open_offline() and get a the pcap handle in pd. 2.) The you call SetHdrSize(), which calls Close() if no link type is found. 3.) Close closes pd, sets pd=0 and returns. 4.) back in PktFileSrc immediatly after SetHdrSize() you call pcap_file(pd) with the nulled pd, which leads to the Segfault in pcap. I have added the relevant code fragments and a backtrace of gdb: ---------------------------- void PktSrc::SetHdrSize() { int dl = pcap_datalink(pd); hdr_size = get_link_header_size(dl); if ( hdr_size < 0 ) { safe_snprintf(errbuf, sizeof(errbuf), "unknown data link type 0x%x", dl); Close(); } datalink = dl; } void PktSrc::Close() { if ( pd ) { pcap_close(pd); pd = 0; closed = true; } } -----skiping lines------- PktFileSrc::PktFileSrc(const char* arg_readfile, const char* filter, PktSrc_Filter_Type ft) : PktSrc() { readfile = copy_string(arg_readfile); filter_type = ft; memcpy(errbuf, "xxxxxxxxx", 10); pd = pcap_open_offline((char*) readfile, errbuf); if ( pd && PrecompileFilter(0, filter) && SetFilter(0) ) { SetHdrSize(); #ifdef USE_SELECT_LOOP // We don't put file sources into non-blocking mode as // otherwise we would not be able to identify the EOF // via next_packet(). selectable_fd = fileno(pcap_file(pd)); if ( selectable_fd < 0 ) internal_error("OS does not support selectable pcap fd"); #endif } else closed = true; } --------------------------------- gdb Backtrace: #0 0xb7f4f84b in pcap_file (p=0x0) at ./pcap.c:543 #1 0x08190525 in PktFileSrc (this=0x83ef048, arg_readfile=0xbfc9e594 "mpls-test-100KB-1stpacket.pcap", filter=0x8244f7f "tcp or udp", ft=TYPE_FILTER_NORMAL) at PktSrc.cc:461 #2 0x0817d891 in net_init (interfaces=@0xbfc9c774, readfiles=@0xbfc9c764, writefile=0x0, transformed_writefile=0x0, filter=0x8244f7f "tcp or udp", secondary_filter=0x0, do_watchdog=0) at Net.cc:196 #3 0x08091b35 in main (argc=3, argv=0xbfc9c8a4) at main.cc:832 ---------------------------------- bye Fabian Schneider -- Fabian Schneider, An-Institut Deutsche Telekom Laboratories Technische Universit?t Berlin, Fakult?t IV -- Elektrotechnik und Informatik address: Sekr. TEL 4, FG INET, Ernst-Reuter-Platz 7, 10587 Berlin e-mail: fabian at net.t-labs.tu-berlin.de, WWW: http://www.net.in.tum.de/~schneifa phone: +49 30 8353 - 58513, mobile: +49 160 479 43 97 From fabian at net.t-labs.tu-berlin.de Tue Sep 18 04:28:07 2007 From: fabian at net.t-labs.tu-berlin.de (Fabian Schneider) Date: Tue, 18 Sep 2007 13:28:07 +0200 (CEST) Subject: [Bro] MPLS support for Bro Message-ID: Hi, is there a chance to get MPLS support into bro? Since we are going to analyze MPLS links with Bro we would be very interested in getting that. We would need to add a link type in get_link_header_size for PPP_SERIAL (header length 0), and enable bro to read MPLS packets. The problem of MPLS is that there can be multiple MPLS labels per packet. Thus a per packet loop through the stack of MPLS labels would be required. Perhaps it would even be interesting to keep that information for analysis purposes. This is how MPLS works (from wikipedia): Each label stack entry contains four fields: * a 20-bit label value. * a 3-bit field for QoS (Quality of Service) priority (experimental). * a 1-bit bottom of stack flag. If this is set, it signifies that the current label is the last in the stack. * an 8-bit TTL (time to live) field. bye Fabian -- Fabian Schneider, An-Institut Deutsche Telekom Laboratories Technische Universit?t Berlin, Fakult?t IV -- Elektrotechnik und Informatik address: Sekr. TEL 4, FG INET, Ernst-Reuter-Platz 7, 10587 Berlin e-mail: fabian at net.t-labs.tu-berlin.de, WWW: http://www.net.in.tum.de/~schneifa phone: +49 30 8353 - 58513, mobile: +49 160 479 43 97 From jbabbinlists at gmail.com Tue Sep 18 10:49:21 2007 From: jbabbinlists at gmail.com (Jake Mailinglists) Date: Tue, 18 Sep 2007 13:49:21 -0400 Subject: [Bro] howto: getting the port number only In-Reply-To: <42809.211.25.195.202.1190101236.squirrel@mail.hackinthebox.org> References: <42809.211.25.195.202.1190101236.squirrel@mail.hackinthebox.org> Message-ID: <118136780709181049y7b05b3fcxa8a9ae0ee2cb246c@mail.gmail.com> Mel, you should be able to split the string into an array and use the part you want only. Ex. local dst_port_proto = c$id$resp_p; local port_pair = split(dst_port_proto, /\//); # taking the connection destination port/proto pairing and spliting it into an array with the split occuring on the "/" local port_num = port_pair[1]; local port_proto = port_pair[2]; Jake On 9/18/07, mel at hackinthebox.org wrote: > > Hi all, > > > I have: global destinations: set[addr,port]; > > The port data type will store port information in the following format: > > 443/tcp, 22/tcp, 53/udp, etc. > > However, I'm only interested in the port number, not the protocol. How do > I get the port number only? > > --mel > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070918/f4100327/attachment.html From robin at icir.org Tue Sep 18 19:54:06 2007 From: robin at icir.org (Robin Sommer) Date: Tue, 18 Sep 2007 19:54:06 -0700 Subject: [Bro] howto: getting the port number only In-Reply-To: <42809.211.25.195.202.1190101236.squirrel@mail.hackinthebox.org> References: <42809.211.25.195.202.1190101236.squirrel@mail.hackinthebox.org> Message-ID: <20070919025406.GD1322@icir.org> On Tue, Sep 18, 2007 at 15:40 +0800, mel at hackinthebox.org wrote: > However, I'm only interested in the port number, not the protocol. How do > I get the port number only? This gives you the port as a string: fmt("%d", 42/tcp) Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Tue Sep 18 19:57:31 2007 From: robin at icir.org (Robin Sommer) Date: Tue, 18 Sep 2007 19:57:31 -0700 Subject: [Bro] Bro segfaults with unknown linktype In-Reply-To: References: Message-ID: <20070919025731.GF1322@icir.org> On Tue, Sep 18, 2007 at 13:07 +0200, Fabian Schneider wrote: > we found a bug that segfaults bro. If you try to open a pcap file with a Thanks for tracking this down. I'll fix it. Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Tue Sep 18 20:01:51 2007 From: robin at icir.org (Robin Sommer) Date: Tue, 18 Sep 2007 20:01:51 -0700 Subject: [Bro] MPLS support for Bro In-Reply-To: References: Message-ID: <20070919030151.GG1322@icir.org> On Tue, Sep 18, 2007 at 13:28 +0200, Fabian Schneider wrote: > is there a chance to get MPLS support into bro? Should be pretty easy to add. Can you send me a short trace with MPLS packets (ideally with varying stacking depths)? Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Tue Sep 18 20:12:18 2007 From: robin at icir.org (Robin Sommer) Date: Tue, 18 Sep 2007 20:12:18 -0700 Subject: [Bro] Rotate log time issue - bro seg fault In-Reply-To: <1bb5dd90709180021o64522519ldf4cfc21ded56094@mail.gmail.com> References: <1bb5dd90709172005j56a05b16ra73b3fd166ca2ee8@mail.gmail.com> <200709180551.l8I5prOX018822@jaguar.icir.org> <1bb5dd90709180021o64522519ldf4cfc21ded56094@mail.gmail.com> Message-ID: <20070919031217.GI1322@icir.org> On Tue, Sep 18, 2007 at 15:21 +0800, CS Lee wrote: > And it seems to be the same. I can't reproduce the problem unfortunately. Can you send me your trace? (I suppose you didn't change any of the policy scripts, did you?) Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Tue Sep 18 20:25:10 2007 From: robin at icir.org (Robin Sommer) Date: Tue, 18 Sep 2007 20:25:10 -0700 Subject: [Bro] UDP flow anomaly In-Reply-To: <1bb5dd90709180039g7770d9c8tf5e2fd31f0fa2164@mail.gmail.com> References: <1bb5dd90709180039g7770d9c8tf5e2fd31f0fa2164@mail.gmail.com> Message-ID: <20070919032510.GK1322@icir.org> On Tue, Sep 18, 2007 at 15:39 +0800, CS Lee wrote: > The script can locate the 210.79.186.143 but not 200.83.176.80 and > 89.37.157.114. That lead us to believe that bro understand the flow in > semantic level. In fact if we do the matching to 18+19 = 37 bytes, That's right, the size in the endpoint record is cumulative and reflects the total size of the flow so far. I see two options for you: - you could remember the flows' size with every udp_reply and then calculate the increase when the next udp_reply comes in. - you could use the new_packet() event which gives you the size for each packet. None of the two approaches is very nice and both can also turn out to be pretty expensive. The main problem here is that Bro isn't really well-suited for expressing policies at the level of indivdual packets as it tries to abstract from packets o high-level activity as much as possible. Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From vern at icir.org Tue Sep 18 20:38:55 2007 From: vern at icir.org (Vern Paxson) Date: Tue, 18 Sep 2007 20:38:55 -0700 Subject: [Bro] Rotate log time issue - bro seg fault In-Reply-To: <1bb5dd90709180021o64522519ldf4cfc21ded56094@mail.gmail.com> (Tue, 18 Sep 2007 15:21:37 +0800). Message-ID: <200709190338.l8J3ctKk088277@jaguar.icir.org> > The bro-1.3.2. that we installed having the same revision which is 4657 - Ah. Then please try this one. Vern Index: File.cc =================================================================== --- File.cc (revision 4763) +++ File.cc (revision 4773) @@ -539,6 +539,9 @@ void BroFile::InstallRotateTimer() { + if ( terminating ) + return; + if ( rotate_timer ) { timer_mgr->Cancel(rotate_timer); From geek00l at gmail.com Tue Sep 18 21:50:23 2007 From: geek00l at gmail.com (CS Lee) Date: Wed, 19 Sep 2007 12:50:23 +0800 Subject: [Bro] Rotate log time issue - bro seg fault In-Reply-To: <200709190338.l8J3ctKk088277@jaguar.icir.org> References: <1bb5dd90709180021o64522519ldf4cfc21ded56094@mail.gmail.com> <200709190338.l8J3ctKk088277@jaguar.icir.org> Message-ID: <1bb5dd90709182150p3c70d038qc6e0b198d4d55724@mail.gmail.com> Hi Vern, The patch works! There's no more segfault after the patch is applied. Thanks. On 9/19/07, Vern Paxson wrote: > > > The bro-1.3.2. that we installed having the same revision which is 4657 > - > > Ah. Then please try this one. > > Vern > > > Index: File.cc > =================================================================== > --- File.cc (revision 4763) > +++ File.cc (revision 4773) > @@ -539,6 +539,9 @@ > > void BroFile::InstallRotateTimer() > { > + if ( terminating ) > + return; > + > if ( rotate_timer ) > { > timer_mgr->Cancel(rotate_timer); > -- Best Regards, CS Lee -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070919/48b9e9cd/attachment.html From scampbell at lbl.gov Tue Sep 18 22:54:47 2007 From: scampbell at lbl.gov (scott campbell) Date: Tue, 18 Sep 2007 22:54:47 -0700 Subject: [Bro] using lookup_hostname() Message-ID: <46F0B9A7.8000809@lbl.gov> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I am having issues implementing the new lookup_hostname() bif. The problem involves the necessity for a boolean test while getting the results from an addr_set. Does anybody have a working example of this? thanks, scott -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG8LmnK2Plq8B7ZBwRAl/iAJwNGL82cFhHxnewwmwM8wHkzIUH7ACdFhUN hvXNJ/NhtzEy2KJLDIy/AFQ= =1X+N -----END PGP SIGNATURE----- From robin at icir.org Wed Sep 19 10:26:43 2007 From: robin at icir.org (Robin Sommer) Date: Wed, 19 Sep 2007 10:26:43 -0700 Subject: [Bro] using lookup_hostname() In-Reply-To: <46F0B9A7.8000809@lbl.gov> References: <46F0B9A7.8000809@lbl.gov> Message-ID: <20070919172643.GK4916@icir.org> On Tue, Sep 18, 2007 at 22:54 -0700, Scott Campbell wrote: > I am having issues implementing the new lookup_hostname() bif. The > problem involves the necessity for a boolean test while getting the > results from an addr_set. I'm not sure I understand the question but I've been using this in an older script: when ( local h = lookup_addr(c$id$resp_h) ) { ... } Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Wed Sep 19 11:27:55 2007 From: robin at icir.org (Robin Sommer) Date: Wed, 19 Sep 2007 11:27:55 -0700 Subject: [Bro] using lookup_hostname() In-Reply-To: <20070919172643.GK4916@icir.org> References: <46F0B9A7.8000809@lbl.gov> <20070919172643.GK4916@icir.org> Message-ID: <20070919182755.GB5361@icir.org> On Wed, Sep 19, 2007 at 10:26 -0700, I wrote: > when ( local h = lookup_addr(c$id$resp_h) ) Oops, copied the wrong snippet. I've also been using this: when ( local addresses = lookup_hostname(pu$host) ) { for ( a in addresses ) [...] } Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From geek00l at gmail.com Thu Sep 20 04:32:11 2007 From: geek00l at gmail.com (CS Lee) Date: Thu, 20 Sep 2007 19:32:11 +0800 Subject: [Bro] UDP flow anomaly In-Reply-To: <20070919032510.GK1322@icir.org> References: <1bb5dd90709180039g7770d9c8tf5e2fd31f0fa2164@mail.gmail.com> <20070919032510.GK1322@icir.org> Message-ID: <1bb5dd90709200432o4246dec2t1aeb0dd14479181a@mail.gmail.com> Hi Robin, Thanks, actually it just need to check if the packet that replying is with 18 bytes length so I guess its good to go. Back to word, I would like to know if anyone working on skype policy script, in case there's merge of same interest. On 9/19/07, Robin Sommer wrote: > > > On Tue, Sep 18, 2007 at 15:39 +0800, CS Lee wrote: > > > The script can locate the 210.79.186.143 but not 200.83.176.80 and > > 89.37.157.114. That lead us to believe that bro understand the flow in > > semantic level. In fact if we do the matching to 18+19 = 37 bytes, > > That's right, the size in the endpoint record is cumulative and > reflects the total size of the flow so far. > > I see two options for you: > > - you could remember the flows' size with every udp_reply and then > calculate the increase when the next udp_reply comes in. > > - you could use the new_packet() event which gives you the size for > each packet. > > None of the two approaches is very nice and both can also turn out to > be pretty expensive. The main problem here is that Bro isn't really > well-suited for expressing policies at the level of indivdual packets > as it tries to abstract from packets o high-level activity as much as > possible. > > Robin > > -- > Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org > LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org > -- Best Regards, CS Lee -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20070920/901dd9fd/attachment.html From muscletot at gmail.com Fri Sep 21 17:25:40 2007 From: muscletot at gmail.com (Mike Wood) Date: Fri, 21 Sep 2007 17:25:40 -0700 Subject: [Bro] nfs analysis Message-ID: <7f3e32390709211725w701de3e5geaeb1831d843a8c8@mail.gmail.com> Quick question: is the documentation regarding NFS on the Bro-wiki accurate? "Deficiency: Bro's notion of NFS is currently confined to just knowledge of the existence of these services. It does not analyze the particulars of different NFS operations." I am trying to extract some NFS file access events from a trace and cannot seem to get the nfs_request_* nfs_attempt_* event handlers to trigger. Should I be able to? Thanks, Mike From christian at whoop.org Fri Sep 21 18:52:18 2007 From: christian at whoop.org (Christian Kreibich) Date: Fri, 21 Sep 2007 18:52:18 -0700 Subject: [Bro] How to get anything into c$service Message-ID: <1190425938.5755.24.camel@strangepork> Hi, I seem to have lost the mojo for getting any services to show up in the connection records' service set. I have: @load conn redef dpd_conn_logs = T; However, the service set remains empty in new_connection and connection_finished events. I guess that makes sense for the former event, but not the latter, so what else do I need? (Basically, I'd like to have a reliable way to generate content in c$service for the broconn Broccoli test case.) Cheers, Christian -- ________________________________________________________________________ http://www.icir.org/christian http://www.whoop.org From robin at icir.org Sat Sep 22 11:23:27 2007 From: robin at icir.org (Robin Sommer) Date: Sat, 22 Sep 2007 11:23:27 -0700 Subject: [Bro] How to get anything into c$service In-Reply-To: <1190425938.5755.24.camel@strangepork> References: <1190425938.5755.24.camel@strangepork> Message-ID: <20070922182327.GA18325@icir.org> On Fri, Sep 21, 2007 at 18:52 -0700, you wrote: > However, the service set remains empty in new_connection and > connection_finished events. I guess that makes sense for the former > event, but not the latter, so what else do I need? "services" is set at a few locations whenever some script believes it has recognized a service. Most importantly that's DPD's protocol detection[1] but also, e.g., ftp-data and portmapper connections. The crucial point is that you need to have some analyzer running which takes the decision. Assuming dpd_conn_logs=T, I get for example service={HTTP} for HTTP sessions once I load http-request. Robin [1] Also applies to standard ports, i.e., even without running the DPD signatures. -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From christian at whoop.org Sat Sep 22 14:50:02 2007 From: christian at whoop.org (Christian Kreibich) Date: Sat, 22 Sep 2007 14:50:02 -0700 Subject: [Bro] nfs analysis In-Reply-To: <7f3e32390709211725w701de3e5geaeb1831d843a8c8@mail.gmail.com> References: <7f3e32390709211725w701de3e5geaeb1831d843a8c8@mail.gmail.com> Message-ID: <1190497802.5755.29.camel@strangepork> On Fri, 2007-09-21 at 17:25 -0700, Mike Wood wrote: > Quick question: is the documentation regarding NFS on the Bro-wiki accurate? > > "Deficiency: Bro's notion of NFS is currently confined to just > knowledge of the existence of these services. It does not analyze the > particulars of different NFS operations." > > I am trying to extract some NFS file access events from a trace and > cannot seem to get the nfs_request_* nfs_attempt_* event handlers to > trigger. Should I be able to? Hey Mike! I believe the documentation is once again misleading. :( Do you get nothing at all when you load nfs.bro? Cheers, Christian -- ________________________________________________________________________ http://www.icir.org/christian http://www.whoop.org From christian at whoop.org Sun Sep 23 22:51:53 2007 From: christian at whoop.org (Christian Kreibich) Date: Sun, 23 Sep 2007 22:51:53 -0700 Subject: [Bro] How to get anything into c$service In-Reply-To: <20070922182327.GA18325@icir.org> References: <1190425938.5755.24.camel@strangepork> <20070922182327.GA18325@icir.org> Message-ID: <1190613113.5755.88.camel@strangepork> On Sat, 2007-09-22 at 11:23 -0700, Robin Sommer wrote: > The crucial point is that you need to have some analyzer running > which takes the decision. Assuming dpd_conn_logs=T, I get for > example service={HTTP} for HTTP sessions once I load http-request. Thanks! Mhmm ... I don't quite see this. When I use @load conn @load http-request redef dpd_conn_logs = T; then I no longer seem to get connection_finished events(!), despite seeing the teardown on the wire. I do however see connection_state_remove, but without anything in c$service. When I use @load conn @load dpd redef dpd_conn_logs = T; all is well: I get both connection_finished and connection_state_remove, and both carry HTTP in c$service (since in that case the capture filter ends up being "tcp or udp or icmp"). Cheers, Christian -- ________________________________________________________________________ http://www.icir.org/christian http://www.whoop.org From robin at icir.org Sun Sep 23 23:17:13 2007 From: robin at icir.org (Robin Sommer) Date: Sun, 23 Sep 2007 23:17:13 -0700 Subject: [Bro] How to get anything into c$service In-Reply-To: <1190613113.5755.88.camel@strangepork> References: <1190425938.5755.24.camel@strangepork> <20070922182327.GA18325@icir.org> <1190613113.5755.88.camel@strangepork> Message-ID: <20070924061713.GA18940@icir.org> On Sun, Sep 23, 2007 at 22:51 -0700, Christian Kreibich wrote: > @load conn [...] > then I no longer seem to get connection_finished events(!), despite Does loading tcp.bro instead of conn.bro help? > @load conn > @load dpd > redef dpd_conn_logs = T; > > all is well: I get both connection_finished and connection_state_remove, > and both carry HTTP in c$service (since in that case the capture filter > ends up being "tcp or udp or icmp"). My last reply actually simplified things a bit, sorry. For services added via the DPD mechanism (i.e., verifying the presence of the protocol by having the analyzer parse it), this is what is needed: - the (core) analyzer needs to see the packets. That's the case with Bro's fall-back default "tcp or udp or icmp" but not anymore once you load any script which modifies the default (e.g., tcp.bro). If so, you either need to set the filter manually or load the corresponding analyzer script which then makes sure the packets are included. That's actually why I refered to http-request.bro - you need to load conn.bro (which almost always gets pulled in by some other script anyway). conn.bro has the handler for the protocol_confirmation() event, which adds the entry to the service field once an analyzer believes it's indeed its protocol. For other services (i.e., non-DPD) the corresponding script sets the services. E.g., ftp.bro adds an entry "ftp-data" to services for data sessions. Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From muscletot at gmail.com Mon Sep 24 10:39:20 2007 From: muscletot at gmail.com (Mike Wood) Date: Mon, 24 Sep 2007 10:39:20 -0700 Subject: [Bro] nfs analysis In-Reply-To: <1190497802.5755.29.camel@strangepork> References: <7f3e32390709211725w701de3e5geaeb1831d843a8c8@mail.gmail.com> <1190497802.5755.29.camel@strangepork> Message-ID: <7f3e32390709241039m197f30d4r2378bc715d194c7a@mail.gmail.com> On 9/22/07, Christian Kreibich wrote: > On Fri, 2007-09-21 at 17:25 -0700, Mike Wood wrote: > > > > "Deficiency: Bro's notion of NFS is currently confined to just > > knowledge of the existence of these services. It does not analyze the > > particulars of different NFS operations." > > > > I am trying to extract some NFS file access events from a trace and > > cannot seem to get the nfs_request_* nfs_attempt_* event handlers to > > trigger. Should I be able to? > > Hey Mike! I believe the documentation is once again misleading. :( Do > you get nothing at all when you load nfs.bro? Sadly, yes I do not get any output from nfs.bro. The tcpdump output for my trace looks like: 16:01:13.467628 IP client.host.name.4160508447 > server.host.name.nfs: 132 getattr [|nfs] 16:01:13.467879 IP server.host.name.nfs > client.host.name.4160508447: reply ok 96 getattr DIR 40755 ids 10013/6007 sz 1024 16:01:13.467940 IP client.host.name.4177285663 > server.host.name.nfs: 148 lookup [|nfs] 16:01:13.468130 IP server.host.name.nfs > client.host.name.4177285663: reply ok 128 lookup [|nfs] ... So I would assume I have some valid NFS traffic there... but when I run bro -r mytrace.pcap nfs.bro I just get plenty of weird messages: 1190415715.190522 weird: bad_RPC 1190415715.190781 weird: unpaired_RPC_response So, perhaps this is all the NFS traffic not getting recognized. Have you seen this before? I am running Bro 1.2.9... if that helps at all. Thanks, Mike From rporada at ll.mit.edu Wed Sep 26 12:16:23 2007 From: rporada at ll.mit.edu (Reed Porada) Date: Wed, 26 Sep 2007 15:16:23 -0400 Subject: [Bro] Questions about Bro Capabilities Message-ID: <3F143535-4B01-4A12-856F-E28FD40CA812@ll.mit.edu> I am looking at extending Bro to help with traffic isolation. What I need to be able to do is differentiate between traffic that matches a given set of criteria and that which does not. In general, I know this can be done through the policies, and I believe I can do most of what I want within a policy. There are a few things that from reading the documentation and some initial policy testing that I am not certain about. 1) Is it possible to denote particular packets in a capture? I know most of the analysis is done on a flow/connection basis, but I was wondering if any information regarding the pcap was kept in the streams/records that are passed? 2) Is it possible to get the content from http sessions? I want to be able to validate that the content is that which I know to be on a given site. I know there is a content_length and data_length values in the http_message record type, but I do not see much relating to the actual content. Thanks for any help, -Reed From rreitz at fnal.gov Wed Sep 26 14:37:57 2007 From: rreitz at fnal.gov (Randolph Reitz) Date: Wed, 26 Sep 2007 16:37:57 -0500 Subject: [Bro] inbound PortScans that aren't really... Message-ID: <04488F89-3613-47A4-826B-CC8D4B8F7476@fnal.gov> I'm running a minimal set of BRO (1.3.2) policies, scan.bro plus a few others, on the Fermilab traffic. I see a lot of inbound scans that appear to be bogus. For example... 1190841523.673433:PortScan:NOTICE_ALARM_ALWAYS:: 216.7.172.212::216.7.172.212:80/tcp::::::216.7.172.212 has scanned 50 ports of 131.225.22.131::@9765 This notice seems to be the result of an internal host visiting a web page (e.g. 212.172.7.216.in-addr.arpa domain name pointer forums.snapstream.com) where the web browser is incrementing the source port for each TCP connection to the destination port 80 web server. In scan.bro, this looks like the remote system is (inbound) port scanning the internal host. Have I missed a configuration in scan.bro that will ignore this? Thanks, Randy Reitz From christian at whoop.org Wed Sep 26 17:04:29 2007 From: christian at whoop.org (Christian Kreibich) Date: Wed, 26 Sep 2007 17:04:29 -0700 Subject: [Bro] nfs analysis In-Reply-To: <7f3e32390709241039m197f30d4r2378bc715d194c7a@mail.gmail.com> References: <7f3e32390709211725w701de3e5geaeb1831d843a8c8@mail.gmail.com> <1190497802.5755.29.camel@strangepork> <7f3e32390709241039m197f30d4r2378bc715d194c7a@mail.gmail.com> Message-ID: <1190851469.5755.344.camel@strangepork> On Mon, 2007-09-24 at 10:39 -0700, Mike Wood wrote: > Sadly, yes I do not get any output from nfs.bro. > > The tcpdump output for my trace looks like: > > 16:01:13.467628 IP client.host.name.4160508447 > server.host.name.nfs: > 132 getattr [|nfs] (I feel I won't be able to give the definitive answer to this one, so others are very welcome to jump in.) I wonder whether it could be that Bro doesn't read all of the traffic -- check whether the resulting filter looks decent by adding print-filter.bro at the end of your invocation? Also, I'm wondering how the source port can be 4160508447 in your tcpdump!? Cheers, Christian -- ________________________________________________________________________ http://www.icir.org/christian http://www.whoop.org From robin at icir.org Wed Sep 26 18:43:02 2007 From: robin at icir.org (Robin Sommer) Date: Wed, 26 Sep 2007 18:43:02 -0700 Subject: [Bro] Questions about Bro Capabilities In-Reply-To: <3F143535-4B01-4A12-856F-E28FD40CA812@ll.mit.edu> References: <3F143535-4B01-4A12-856F-E28FD40CA812@ll.mit.edu> Message-ID: <20070927014302.GA8872@icir.org> On Wed, Sep 26, 2007 at 15:16 -0400, you wrote: > 1) Is it possible to denote particular packets in a capture? I know No, not really. The main problem here is that the link between most event handlers and the actual packets is pretty weak. In general, Bro does not give guarantees about when a particular event is raised and also doesn't keep track which packet triggered it. There's a function called get_current_packet() which returns the packet Bro currently munching on but when script code is running it's hard to predict which packet that actually is. The only event which directly refers to packets is new_packet() but using that is expensive because it is raised for *all* packets. That said, perhaps we might be able to come up with some idea if you sketch in a bit more detail what you're trying to achieve. > 2) Is it possible to get the content from http sessions? Yes, that's possible. The event for this is http_entity_data(); see http-body.bro for an example that logs HTTP content into http.log. Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From rporada at ll.mit.edu Thu Sep 27 07:50:25 2007 From: rporada at ll.mit.edu (Reed Porada) Date: Thu, 27 Sep 2007 10:50:25 -0400 Subject: [Bro] http- and empty http.log In-Reply-To: <20070927014302.GA8872@icir.org> References: <3F143535-4B01-4A12-856F-E28FD40CA812@ll.mit.edu> <20070927014302.GA8872@icir.org> Message-ID: <8ED3B6CE-6B09-4658-B801-DF6F76B5DC47@ll.mit.edu> In trying to get the contents of http sessions, I have run http- body.bro against a pcap, and there is not output to http.log. This is the same with most http- scripts, except http-reply. Looking at the traces, I am seeing only 'connection_state_remove' from the main HTTP module for http-body, http, etc. With http-reply, I see the events in http-reply, and the 'connection_state_remove' as expected. I am running 1.3.2 on Ubuntu. I have looked at the pcap and run it against tcptrace, and all seems good. Any thoughts? -Reed From robin at icir.org Thu Sep 27 09:46:36 2007 From: robin at icir.org (Robin Sommer) Date: Thu, 27 Sep 2007 09:46:36 -0700 Subject: [Bro] http- and empty http.log In-Reply-To: <8ED3B6CE-6B09-4658-B801-DF6F76B5DC47@ll.mit.edu> References: <3F143535-4B01-4A12-856F-E28FD40CA812@ll.mit.edu> <20070927014302.GA8872@icir.org> <8ED3B6CE-6B09-4658-B801-DF6F76B5DC47@ll.mit.edu> Message-ID: <20070927164636.GC13997@icir.org> On Thu, Sep 27, 2007 at 10:50 -0400, Reed Porada wrote: > In trying to get the contents of http sessions, I have run http- > body.bro against a pcap, and there is not output to http.log. This > is the same with most http- scripts, except http-reply. The HTTP scripts are a bit different from other analyzers in the sense that they are "incremental", i.e., you typically need to load more than one depending on which parts of the HTTP sessions you want to analyze. The minimum is http-request.bro which analyzes client-side requests. You can add http-reply.bro to also see the response code of the servers. Then you can further add, e.g., http-body.bro, to get the session payload and/or http-header.bro to see all HTTP headers. So, in your case, this should do the trick: bro -r trace http-request http-reply http-body Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Thu Sep 27 09:49:33 2007 From: robin at icir.org (Robin Sommer) Date: Thu, 27 Sep 2007 09:49:33 -0700 Subject: [Bro] inbound PortScans that aren't really... In-Reply-To: <04488F89-3613-47A4-826B-CC8D4B8F7476@fnal.gov> References: <04488F89-3613-47A4-826B-CC8D4B8F7476@fnal.gov> Message-ID: <20070927164933.GD13997@icir.org> On Wed, Sep 26, 2007 at 16:37 -0500, Randolph Reitz wrote: > few others, on the Fermilab traffic. I see a lot of inbound scans > that appear to be bogus. For example... Can you send me a trace of one of these scans? (Just TCP control packets is fine if there's content you can't pass on). Robin -- Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org From rporada at ll.mit.edu Thu Sep 27 11:20:12 2007 From: rporada at ll.mit.edu (Reed Porada) Date: Thu, 27 Sep 2007 14:20:12 -0400 Subject: [Bro] http- and empty http.log In-Reply-To: <20070927164636.GC13997@icir.org> References: <3F143535-4B01-4A12-856F-E28FD40CA812@ll.mit.edu> <20070927014302.GA8872@icir.org> <8ED3B6CE-6B09-4658-B801-DF6F76B5DC47@ll.mit.edu> <20070927164636.GC13997@icir.org> Message-ID: <77B7AE16-2B31-4C2C-8659-F937F7BFD5FD@ll.mit.edu> On Sep 27, 2007, at 12:46 PM, Robin Sommer wrote: > > On Thu, Sep 27, 2007 at 10:50 -0400, Reed Porada wrote: > >> In trying to get the contents of http sessions, I have run http- >> body.bro against a pcap, and there is not output to http.log. This >> is the same with most http- scripts, except http-reply. > > The HTTP scripts are a bit different from other analyzers in the > sense that they are "incremental", i.e., you typically need to load > more than one depending on which parts of the HTTP sessions you want > to analyze. > > The minimum is http-request.bro which analyzes client-side requests. > You can add http-reply.bro to also see the response code of the > servers. Then you can further add, e.g., http-body.bro, to get the > session payload and/or http-header.bro to see all HTTP headers. > > So, in your case, this should do the trick: > > bro -r trace http-request http-reply http-body > > Robin > > -- > Robin Sommer * Phone +1 (510) 931-5555 * robin at icir.org > LBNL/ICSI * Fax +1 (510) 666-2956 * www.icir.org > That worked. Thanks. -Reed From jp.luiggi at free.fr Fri Sep 28 09:01:39 2007 From: jp.luiggi at free.fr (Jean-Philippe Luiggi) Date: Fri, 28 Sep 2007 12:01:39 -0400 Subject: [Bro] Nice connection state graph Message-ID: <20070928160139.GA22084@armada.mynetwork.local> Hello, Perhaps a member of the ML did it but as i've didn't see any post about it, i would like to mention a nice connection state graph built with Afterglow. ref : http://secviz.org/?q=node/81 Very good (and nice). Best regards, Jean-philippe.