From dmvasil at ornl.gov Thu Feb 2 08:03:27 2006 From: dmvasil at ornl.gov (David Vasil) Date: Thu, 02 Feb 2006 11:03:27 -0500 Subject: [Bro] Capturing events Message-ID: <43E22D4F.6080002@ornl.gov> I noticed the capture-events.bro policy and loaded it; it has been generating an events.bst file in my logs directory with data inside of it. My question is: what can I get out of this file? Is this just a raw packet capture of anything that is flagged by a policy? Also, I tried replaying the file using bro -R events.bst and it appears to be waiting for standard input. When I try and specify a policy to use on the command line it errors with: [root at endace bro]# bin/bro -R /scratch/bro/logs/events.bst http ./site, line 1: error: read failed with "Is a directory" I searched through the documentation and saw no reference to 'replay' 'events.bst' or even '.bst'. Also, where can I get the start-capture-all script? Is this just a wrapper around tcpdump; or does it grab the data before/after bro looks at the stream? Unfortunately with the DAG capture cards I am testing with, only one application can read from the device at a single time. Any suggestions? Thanks! -- | David Vasil | Oak Ridge National Laboratory NCCS Division | High Performance Computing Systems Administrator | Bldg: 5600-A115 Phone: (865)241-5562 From robin at icir.org Thu Feb 2 10:34:59 2006 From: robin at icir.org (Robin Sommer) Date: Thu, 2 Feb 2006 10:34:59 -0800 Subject: [Bro] Capturing events In-Reply-To: <43E22D4F.6080002@ornl.gov> References: <43E22D4F.6080002@ornl.gov> Message-ID: <20060202183459.GA22074@net.informatik.tu-muenchen.de> On Thu, Feb 02, 2006 at 11:03 -0500, David Vasil wrote: > I noticed the capture-events.bro policy and loaded it; it has been > generating an events.bst file in my logs directory with data inside of > it. My question is: what can I get out of this file? Is this just a > raw packet capture of anything that is flagged by a policy? It's a capture of all events which were generated during run-time, e.g., things like connection_established and http_request. Events.bst contains all raised events including their timestamps and parameters, so it's a very convenient way to understand what's going on during operation (for large traffic streams it gets huge though and may therefore be more suitable for smaller experiments). You can display the contents of the file with the -x option: bro -x events.bst any-scripts-used-in-the-original-Bro-run.bro > Also, I tried replaying the file using bro -R events.bst and it appears > to be waiting for standard input. The replay mechanism might actually be broken right now due to internal changes. I'll look at it. > I searched through the documentation and saw no reference to 'replay' > 'events.bst' or even '.bst'. Uhm, that's right. Unfortunately, the best documentation of new features is still the "CHANGES". Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICIR/ICSI * Fax +1 (510) 666-2956 * www.icir.org From dmvasil at ornl.gov Thu Feb 2 10:47:01 2006 From: dmvasil at ornl.gov (David Vasil) Date: Thu, 02 Feb 2006 13:47:01 -0500 Subject: [Bro] Capturing events In-Reply-To: <20060202183459.GA22074@net.informatik.tu-muenchen.de> References: <43E22D4F.6080002@ornl.gov> <20060202183459.GA22074@net.informatik.tu-muenchen.de> Message-ID: <43E253A5.1060803@ornl.gov> Would you recommend using BRO_CREATE_TRACE_FILE=YES instead of event-capture.bro? Besides being in a raw tcpdump format, what other benefits does the trace file give me? Thanks! -- | David Vasil | Oak Ridge National Laboratory NCCS Division | High Performance Computing Systems Administrator | Bldg: 5600-A115 Phone: (865)241-5562 From robin at icir.org Thu Feb 2 11:25:14 2006 From: robin at icir.org (Robin Sommer) Date: Thu, 2 Feb 2006 11:25:14 -0800 Subject: [Bro] Capturing events In-Reply-To: <43E253A5.1060803@ornl.gov> References: <43E22D4F.6080002@ornl.gov> <20060202183459.GA22074@net.informatik.tu-muenchen.de> <43E253A5.1060803@ornl.gov> Message-ID: <20060202192514.GA16817@net.informatik.tu-muenchen.de> On Thu, Feb 02, 2006 at 13:47 -0500, David Vasil wrote: > Would you recommend using BRO_CREATE_TRACE_FILE=YES instead of > event-capture.bro? Besides being in a raw tcpdump format, what other > benefits does the trace file give me? Thanks! The tracefile gives you a comprehensive view of the network activity: it contains exactly the data on which Bro performed the analysis; by refeeding the trace into Bro again, you get the same results. The single most important advantage of a trace is that you can manually examine it afterwards to see why Bro reported something, in case the usual logs are not sufficient. If your environmnent allows it (in terms of available resources, network volume, and organisational restrictions), capturing a trace is a good thing. events.bst is a bit different: it does not contain the raw traffic but a higher-level abstraction of it; you loose information as you only see the data in a state when Bro has already performed its first step of analysis (i.e. after a large reduction in volume). This may or may not be sufficient to track things down, though usually most of the data contained in events.bst ends up in some log file anyway. I usually do not routinely use capture-events with an operational Bro. I Rather turn it on selectively if I need to understand how exactly Bro generates events for some given input, e.g., to tweak a policy script. Then it's really helpful as you exactly see what the policy scripts see as well, including timing. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICIR/ICSI * Fax +1 (510) 666-2956 * www.icir.org From bltierney at lbl.gov Thu Feb 2 16:37:53 2006 From: bltierney at lbl.gov (Brian Tierney) Date: Thu, 2 Feb 2006 16:37:53 -0800 Subject: [Bro] Capturing events In-Reply-To: <43E253A5.1060803@ornl.gov> References: <43E22D4F.6080002@ornl.gov> <20060202183459.GA22074@net.informatik.tu-muenchen.de> <43E253A5.1060803@ornl.gov> Message-ID: On Feb 2, 2006, at 10:47 AM, David Vasil wrote: > Would you recommend using BRO_CREATE_TRACE_FILE=YES instead of > event-capture.bro? Besides being in a raw tcpdump format, what > other benefits does the trace file give me? Thanks! > I use event-capture.bro mainly for debugging processing of external events sent to Bro via Broccoli. For example, we convert syslog events to something bro understands and send them to Bro for analysis via Broccoli. From bltierney at lbl.gov Thu Feb 2 17:56:40 2006 From: bltierney at lbl.gov (Brian Tierney) Date: Thu, 2 Feb 2006 18:56:40 -0700 Subject: [Bro] updated summary of Bro User Survey results In-Reply-To: <21FAB984-939B-4098-9453-CB888545414B@lbl.gov> References: <20060126070328.GA11030@uiuc.edu> <21FAB984-939B-4098-9453-CB888545414B@lbl.gov> Message-ID: Got a few more replies. Here is the updated list. - Using Bro as the primary production network IDS for your site/group 4 (LBL, NERSC, ESNet, NCSA, UC Davis, 1 small business net) - Using Bro as the secondary production network IDS for your site/group 4 - Using Bro for off-line analysis for forensics, etc. 2 - Using Bro for traffic characterization studies only 2 - just playing around with Bro. 1 - currently just playing around with Bro, but hoping to use it in production soon 5 Other replies: - using Bro as part of a NIDS product - using Bro for research purposes From robin at icir.org Fri Feb 3 17:02:30 2006 From: robin at icir.org (Robin Sommer) Date: Fri, 3 Feb 2006 17:02:30 -0800 Subject: [Bro] Capturing events In-Reply-To: <20060202183459.GA22074@net.informatik.tu-muenchen.de> References: <43E22D4F.6080002@ornl.gov> <20060202183459.GA22074@net.informatik.tu-muenchen.de> Message-ID: <20060204010230.GA1332@net.informatik.tu-muenchen.de> On Thu, Feb 02, 2006 at 10:34 -0800, I wrote: > The replay mechanism might actually be broken right now due to > internal changes. I'll look at it. Will be fixed in the next release. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICIR/ICSI * Fax +1 (510) 666-2956 * www.icir.org From jruggieri at ucdavis.edu Tue Feb 7 11:50:14 2006 From: jruggieri at ucdavis.edu (Joncarlo Ruggieri) Date: Tue, 7 Feb 2006 11:50:14 -0800 (PST) Subject: [Bro] ScanSummary intervals Message-ID: Hi, We are running Bro 0.9a8. I am trying to track down an inconsistency with our alarm logs. Our thresholds for reporting AddressScans from external hosts is defined in scan.bro as follows: const report_peer_scan = { 20, 100, 1000, 10000, 50000, 100000, 250000, 500000, 1000000, } &redef; This is not redefined elsewhere. I see AddressScan alarms for a given host when they reach our first defined threshold of 20. I don't see entries for the next threshold of 100. However, when we checkpoint Bro, we see ScanSummary log entries for higher counts. (We checkpoint Bro every 3 hours.) Also, we can see ScanSummary entries for hosts that did not have AddressScan entries during this last log/checkpoint period. My questions are: 1) Is there something else which might override the report_peer_scan thresholds? 2) Should checkpointing Bro reset the ScanSummary count, or will we need to force that? 2A) How do we force the ScanSummary count to reset? Our ultimate goal is to be able to determine the number of addresses scanned by a host at the end of our 3-hour checkpoint interval. That count could be either the true number or else last threshold reached. Thanks for your help! Joncarlo Ruggieri University of CA, Davis Data Center & Client Services From aashish at uiuc.edu Tue Feb 7 14:22:52 2006 From: aashish at uiuc.edu (Aashish Sharma) Date: Tue, 7 Feb 2006 16:22:52 -0600 Subject: [Bro] RHEL 4.0/endace 10GigE/Bro Message-ID: <20060207222252.GA400@uiuc.edu> We are in the process of testing and setting up bro for 10GigE monitoring and are using endace cards (DAG 6.2S) on redhat enterprise 4.0. Right now we see that bro mis-compiles on RHEL4. This appears to be an issues with the glibc-headers package. gcc -DHAVE_CONFIG_H -I. -I. -I../.. -I/home/mhoneyfield/libpcap-0.9.4 -I../../linux-include -g -O2 -I/include -I/usr/include -c `test -f ef.c || echo './'`ef.c In file included from /usr/include/netinet/ether.h:26, from ef.l:13: ../../linux-include/netinet/if_ether.h:41: error: redefinition of `struct ether_addr' ../../linux-include/netinet/if_ether.h:48: error: redefinition of `struct ether_header' In file included from ef.l:16: ../../linux-include/netinet/if_ether.h:41: error: redefinition of `struct ether_addr' ../../linux-include/netinet/if_ether.h:48: error: redefinition of `struct ether_header' ../../linux-include/netinet/if_ether.h:77: error: redefinition of `struct ether_arp' make[4]: *** [ef.o] Error 1 make[4]: Leaving directory `/home/mhoneyfield/bro-1.0/aux/hf' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/home/mhoneyfield/bro-1.0/aux' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/mhoneyfield/bro-1.0/aux' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/mhoneyfield/bro-1.0' make: *** [all] Error 2 Any thoughts on this ? Also, I am wondering if anyone here running bro with endace DAG 6.2 cards ? Thanks a lot Aashish Sharma -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20060207/f99a3d07/attachment.bin From robin at icir.org Tue Feb 7 19:03:18 2006 From: robin at icir.org (Robin Sommer) Date: Tue, 7 Feb 2006 19:03:18 -0800 Subject: [Bro] RHEL 4.0/endace 10GigE/Bro In-Reply-To: <20060207222252.GA400@uiuc.edu> References: <20060207222252.GA400@uiuc.edu> Message-ID: <20060208030317.GA19974@net.informatik.tu-muenchen.de> On Tue, Feb 07, 2006 at 16:22 -0600, Aashish Sharma wrote: > Right now we see that bro mis-compiles on RHEL4. This appears to be an issues with the glibc-headers package. Hmm... I've seen this before but right now I can't remember what's the fix. Does anybody? (Just a guess: does it help if you remove "-I../../linux-include" from the command line?) > Any thoughts on this ? Also, I am wondering if anyone here running bro with endace DAG 6.2 cards ? We have some prototypical code to add native DAG support to Bro (rather than relying on libpcap). I'd have to check what the current state is but if you're interested to try it, I'll see. (We're using a 4.3GE currently but this shouldn't make a difference in terms of the API.) Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICIR/ICSI * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Tue Feb 7 19:19:57 2006 From: robin at icir.org (Robin Sommer) Date: Tue, 7 Feb 2006 19:19:57 -0800 Subject: [Bro] ScanSummary intervals In-Reply-To: References: Message-ID: <20060208031957.GB19974@net.informatik.tu-muenchen.de> On Tue, Feb 07, 2006 at 11:50 -0800, Joncarlo Ruggieri wrote: > 1) Is there something else which might override the report_peer_scan > thresholds? If I recall correctly, a source is no longer reported when it has performed shut_down_thresh connection attempts (default: 100). Instead Bro then just generates a ScanSummary after 1 day or at termination, whatever comes first. > 2) Should checkpointing Bro reset the ScanSummary count, or will we need > to force that? It will be reset; by default Bro does not carry state across restarts (it may though by declaring things as &persistent). Robin P.S.: You wrote, you're checkpointing every 3 hours. Do you do this primarily to avoid running out of memory? If yes: in newer (development) versions, we've greatly improved the state management, so this may become unneccesarry eventually. -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICIR/ICSI * Fax +1 (510) 666-2956 * www.icir.org From christian at whoop.org Wed Feb 8 05:58:06 2006 From: christian at whoop.org (Christian Kreibich) Date: Wed, 08 Feb 2006 13:58:06 +0000 Subject: [Bro] RHEL 4.0/endace 10GigE/Bro In-Reply-To: <20060208030317.GA19974@net.informatik.tu-muenchen.de> References: <20060207222252.GA400@uiuc.edu> <20060208030317.GA19974@net.informatik.tu-muenchen.de> Message-ID: <1139407087.18017.372.camel@localhost> Hi, On Tue, 2006-02-07 at 19:03 -0800, Robin Sommer wrote: > On Tue, Feb 07, 2006 at 16:22 -0600, Aashish Sharma wrote: > > > Right now we see that bro mis-compiles on RHEL4. This appears to be > an issues with the glibc-headers package. > > Hmm... I've seen this before but right now I can't remember what's > the fix. Does anybody? mhmmm the archive says that in November Vidar Seeberg had a similar problem ("Compiling error on Ubuntu Linux - now: make-problem"), but I cannot seem to find a mail that detailed a fix. Vern pointed out that the ef tool isn't actually needed, and iirc we have removed it in more recent releases(?). http://thread.gmane.org/gmane.comp.security.detection.bro/1204 Is it just in aux/{cf,hf} that things are failing? If you cd into src directly once configure has finished and type "make", does it fail similarly? > (Just a guess: does it help if you remove "-I../../linux-include" > from the command line?) Yeah, a good thing to try! Good luck, Christian. -- ________________________________________________________________________ http://www.cl.cam.ac.uk/~cpk25 http://www.whoop.org From jp.luiggi at free.fr Wed Feb 8 07:15:43 2006 From: jp.luiggi at free.fr (Jean-Philippe Luiggi) Date: Wed, 08 Feb 2006 10:15:43 -0500 Subject: [Bro] Some questions and Netflow. In-Reply-To: <20060201004418.GC8973@net.informatik.tu-muenchen.de> References: <20060130134147.GA178@armada.mynetwork.net> <20060201004418.GC8973@net.informatik.tu-muenchen.de> Message-ID: <20060208151543.GB11618@armada.mynetwork.net> I was on the discussion about the Netflow's feature in the past but you're right, nothing has materialized. So it may be time... Netflow technology is very used today both for network/security purposes so i'll think about this. Best regards. On Tue, Jan 31, 2006 at 04:44:18PM -0800, Robin Sommer wrote: > > On Mon, Jan 30, 2006 at 08:41 -0500, Jean-Philippe Luiggi wrote: > > > - Using Bro as a Netflow concentrator. > > - Using a dedicated tool to capture the flows and then use "Bro" to inspect data. > > Both approaches seem to be reasonable. There has been some > discussion of integrating NetFlow into Bro for some time now but, as > far as I'm aware of, nothing concrete has materialized yet. It's on > our radar though and indeed part of a planned future project (with > no concrete time schedule yet though). > > Robin > > -- > Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org > ICIR/ICSI * Fax +1 (510) 666-2956 * www.icir.org > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From jp.luiggi at free.fr Wed Feb 8 07:21:31 2006 From: jp.luiggi at free.fr (Jean-Philippe Luiggi) Date: Wed, 08 Feb 2006 10:21:31 -0500 Subject: [Bro] ScanSummary intervals In-Reply-To: <20060208031957.GB19974@net.informatik.tu-muenchen.de> References: <20060208031957.GB19974@net.informatik.tu-muenchen.de> Message-ID: <20060208152131.GC11618@armada.mynetwork.net> Hello, Do you've any release date ? Thank you. On Tue, Feb 07, 2006 at 07:19:57PM -0800, Robin Sommer wrote: > It will be reset; by default Bro does not carry state across > restarts (it may though by declaring things as &persistent). > > Robin > > P.S.: You wrote, you're checkpointing every 3 hours. Do you do this > primarily to avoid running out of memory? If yes: in newer > (development) versions, we've greatly improved the state management, > so this may become unneccesarry eventually. > > -- > Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org > ICIR/ICSI * Fax +1 (510) 666-2956 * www.icir.org > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From bltierney at lbl.gov Wed Feb 8 07:40:00 2006 From: bltierney at lbl.gov (Brian Tierney) Date: Wed, 8 Feb 2006 08:40:00 -0700 Subject: [Bro] ScanSummary intervals In-Reply-To: <20060208152131.GC11618@armada.mynetwork.net> References: <20060208031957.GB19974@net.informatik.tu-muenchen.de> <20060208152131.GC11618@armada.mynetwork.net> Message-ID: <609CC575-C064-4999-B200-4829FD4CE45C@lbl.gov> The "1.x development release" at http://www.bro-ids.org/download.html has these fixes. On Feb 8, 2006, at 8:21 AM, Jean-Philippe Luiggi wrote: > Hello, > > Do you've any release date ? > > Thank you. > > On Tue, Feb 07, 2006 at 07:19:57PM -0800, Robin Sommer wrote: >> It will be reset; by default Bro does not carry state across >> restarts (it may though by declaring things as &persistent). >> >> Robin >> >> P.S.: You wrote, you're checkpointing every 3 hours. Do you do this >> primarily to avoid running out of memory? If yes: in newer >> (development) versions, we've greatly improved the state management, >> so this may become unneccesarry eventually. >> >> -- >> Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org >> ICIR/ICSI * Fax +1 (510) 666-2956 * www.icir.org >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro ------------------------------------------------------------------------ ------------------- Brian L. Tierney, Lawrence Berkeley National Laboratory (LBNL) 1 Cyclotron Rd. MS: 50B-2239, Berkeley, CA 94720 tel: 510-486-7381 fax: 510-495-2998 efax: 425-642-4558 bltierney at lbl.gov http://dsd.lbl.gov/~tierney ------------------------------------------------------------------------ ------------------ From aashish at uiuc.edu Wed Feb 8 09:16:31 2006 From: aashish at uiuc.edu (Aashish Sharma) Date: Wed, 8 Feb 2006 11:16:31 -0600 Subject: [Bro] RHEL 4.0/endace 10GigE/Bro In-Reply-To: <1139407087.18017.372.camel@localhost> Message-ID: <20060208171631.GA13248@uiuc.edu> On Wed, Feb 08, 2006 at 01:58:06PM +0000, Christian Kreibich wrote: > > On Tue, 2006-02-07 at 19:03 -0800, Robin Sommer wrote: > > On Tue, Feb 07, 2006 at 16:22 -0600, Aashish Sharma wrote: > > > > > Right now we see that bro mis-compiles on RHEL4. This appears to be > > an issues with the glibc-headers package. > > > > Hmm... I've seen this before but right now I can't remember what's > > the fix. Does anybody? > > mhmmm the archive says that in November Vidar Seeberg had a similar > problem ("Compiling error on Ubuntu Linux - now: make-problem"), but I > cannot seem to find a mail that detailed a fix. Vern pointed out that > the ef tool isn't actually needed, and iirc we have removed it in more > recent releases(?). > > http://thread.gmane.org/gmane.comp.security.detection.bro/1204 > Here is the error. It did go a lot further in compilation then initially. Not sure if this is useful for you or I should send you the binpac coredumps as well. PktSrc.o(.text+0x74c): In function `PktSrc::SetHdrSize()': /usr/local/packages/bro/bro-1.0/src/PktSrc.cc:248: undefined reference to `pcap_snprintf' PktSrc.o(.text+0xd7e): In function `PktInterfaceSrc::PktInterfaceSrc(char const*, char const*, PktSrc_Filter_Type)': /usr/local/packages/bro/bro-1.0/src/PktSrc.cc:360: undefined reference to `pcap_snprintf' PktSrc.o(.text+0xdd0):/usr/local/packages/bro/bro-1.0/src/PktSrc.cc:326: undefined reference to `pcap_snprintf' PktSrc.o(.text+0xef6): In function `PktInterfaceSrc::PktInterfaceSrc(char const*, char const*, PktSrc_Filter_Type)': /usr/local/packages/bro/bro-1.0/src/PktSrc.cc:360: undefined reference to `pcap_snprintf' PktSrc.o(.text+0xf48):/usr/local/packages/bro/bro-1.0/src/PktSrc.cc:326: undefined reference to `pcap_snprintf' collect2: ld returned 1 exit status make[2]: *** [bro] Error 1 make[2]: Leaving directory `/usr/local/packages/bro/bro-1.0/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/local/packages/bro/bro-1.0/src' make: *** [all] Error 2 > Is it just in aux/{cf,hf} that things are failing? If you cd into src > directly once configure has finished and type "make", does it fail > similarly? > > > (Just a guess: does it help if you remove "-I../../linux-include" > > from the command line?) > > Yeah, a good thing to try! > I tried removing "-I../../linux-include" by commenting it gets erros. Both of running make after configure as well as running make inside the src directory. If you want to look at the errors I can sent that out as well. Thanks a lot for the prompt responses and all the help. aashish -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20060208/a5418712/attachment.bin From dmvasil at ornl.gov Wed Feb 8 10:13:07 2006 From: dmvasil at ornl.gov (David Vasil) Date: Wed, 08 Feb 2006 13:13:07 -0500 Subject: [Bro] RHEL 4.0/endace 10GigE/Bro In-Reply-To: <20060208171631.GA13248@uiuc.edu> References: <20060208171631.GA13248@uiuc.edu> Message-ID: <43EA34B3.5030601@ornl.gov> Looks like it cant find some header files for libpcap. I have a very similar setup to what you are doing, here is the skinny on how I configured my DAG cards with DAG compatible libpcap and bro: I got the libpcap source from endace.com's website, then compiled with the following: ./configure '--with-dag=/opt/dag' '--prefix=/opt' && make make install For bro using bro-1.X: CFLAGS='-I/opt/include' LDFLAGS='-L/opt/lib' ./configure '--disable-localpcap' '--enable-selectloop' '--prefix=/opt/bro-1.0' && make make install You will need to use endace's libpcap until bro supports DAG devices natively. The endace-libpcap allows the dag devices to be seen as dag0 and dag1 (sort of like eth0/eth1). -- | David Vasil | Oak Ridge National Laboratory NCCS Division | High Performance Computing Systems Administrator | Bldg: 5600-A115 Phone: (865)241-5562 From robin at icir.org Wed Feb 8 11:01:03 2006 From: robin at icir.org (Robin Sommer) Date: Wed, 8 Feb 2006 11:01:03 -0800 Subject: [Bro] RHEL 4.0/endace 10GigE/Bro In-Reply-To: <20060208171631.GA13248@uiuc.edu> References: <1139407087.18017.372.camel@localhost> <20060208171631.GA13248@uiuc.edu> Message-ID: <20060208190103.GA26401@net.informatik.tu-muenchen.de> On Wed, Feb 08, 2006 at 11:16 -0600, Aashish Sharma wrote: > PktSrc.o(.text+0x74c): In function `PktSrc::SetHdrSize()': > /usr/local/packages/bro/bro-1.0/src/PktSrc.cc:248: undefined reference to `pcap_snprintf' > PktSrc.o(.text+0xd7e): In function `PktInterfaceSrc::PktInterfaceSrc(char const*, char const*, PktSrc_Filter_Type)': > /usr/local/packages/bro/bro-1.0/src/PktSrc.cc:360: undefined reference to `pcap_snprintf' > PktSrc.o(.text+0xdd0):/usr/local/packages/bro/bro-1.0/src/PktSrc.cc:326: undefined reference to `pcap_snprintf' > PktSrc.o(.text+0xef6): In function `PktInterfaceSrc::PktInterfaceSrc(char const*, char const*, PktSrc_Filter_Type)': > /usr/local/packages/bro/bro-1.0/src/PktSrc.cc:360: undefined reference to `pcap_snprintf' > PktSrc.o(.text+0xf48):/usr/local/packages/bro/bro-1.0/src/PktSrc.cc:326: undefined reference to `pcap_snprintf' > collect2: ld returned 1 exit status This usually indicates that the used pcap.h does not match the version of libpcap which is linked in. Do you have different versions installed? If not, it may also be the case that Bro's configure confuses the system libpcap with the included libpcap (I forgot whether in 1.0 the pcap which ships with Bro is still the default used by configure; this has changed recently). Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICIR/ICSI * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Wed Feb 8 11:04:28 2006 From: robin at icir.org (Robin Sommer) Date: Wed, 8 Feb 2006 11:04:28 -0800 Subject: [Bro] ScanSummary intervals In-Reply-To: <609CC575-C064-4999-B200-4829FD4CE45C@lbl.gov> References: <20060208031957.GB19974@net.informatik.tu-muenchen.de> <20060208152131.GC11618@armada.mynetwork.net> <609CC575-C064-4999-B200-4829FD4CE45C@lbl.gov> Message-ID: <20060208190428.GB26401@net.informatik.tu-muenchen.de> On Wed, Feb 08, 2006 at 08:40 -0700, Brian Tierney wrote: > The "1.x development release" at http://www.bro-ids.org/download.html > has these fixes. Yet there are still a few more tweaks coming up which are not yet included. Essentially we're going to switch to a memory-efficient default configuration. Instead of requiring to load reduce-memory for effective use in large networks, there'll be the opposite then: an expensive-analysis.bro which can be loaded if the network volume permits. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICIR/ICSI * Fax +1 (510) 666-2956 * www.icir.org From jp.luiggi at free.fr Wed Feb 8 11:43:28 2006 From: jp.luiggi at free.fr (Jean-Philippe Luiggi) Date: Wed, 08 Feb 2006 14:43:28 -0500 Subject: [Bro] ScanSummary intervals In-Reply-To: <609CC575-C064-4999-B200-4829FD4CE45C@lbl.gov> References: <20060208031957.GB19974@net.informatik.tu-muenchen.de> <20060208152131.GC11618@armada.mynetwork.net> <609CC575-C064-4999-B200-4829FD4CE45C@lbl.gov> Message-ID: <20060208194328.GA16334@armada.mynetwork.net> Hello, The one you're speaking about seems to be the 1.0 release (oct 2005). If this isn't a normal release instead of a development's one ? Best regards. On Wed, Feb 08, 2006 at 08:40:00AM -0700, Brian Tierney wrote: > > The "1.x development release" at http://www.bro-ids.org/download.html > has these fixes. > > > On Feb 8, 2006, at 8:21 AM, Jean-Philippe Luiggi wrote: > > >Hello, > > > >Do you've any release date ? From aashish at uiuc.edu Thu Feb 9 09:47:16 2006 From: aashish at uiuc.edu (Aashish Sharma) Date: Thu, 9 Feb 2006 11:47:16 -0600 Subject: [Bro] RHEL 4.0/endace 10GigE/Bro In-Reply-To: <20060208185706.GC26270@net.informatik.tu-muenchen.de> Message-ID: <20060209174716.GA27134@uiuc.edu> Hello All: So I have been able to successfully compile and install bro on RHEL4.0 with dag support. Looks like bro is able to recognize DAG cards as well. There were multiple issues which I ended up fixing off-course with the help from this list. Thanks a lot. Just for future reference : 1) compile libpcap-0.9.4 (latest version which has DAG support) to enable DAG options ./configure --disable-localpcap --libdir=/usr/local/lib --with-dag=/usr/local/dag --prefix=/usr/local CFLAGS="-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64" 2) I removed "aux" from compilation list 3) Compile bro with the following : /configure '--disable-localpcap' '--enable-selectloop' '--prefix=/usr/local/bro' '--libdir=/usr/local/lib' CFLAGS='-I/usr/local/include' LDFLAGS='-L/usr/local/lib' Bro installed successfully and starts/stops just fine but its not capturing any data so far. I have defined dag0 and dag1 as capture interfaces in bro.cfg. The info.bro file is a little unusual. It does not pick up any capture filter. Is this normal for dag* interfaces ? Is there any default filter then ? If not, how can I fix this capture filter issue. I tried redefining capture filter in hostname.bro file in site folder but in vein. Here is the info.bro log : ------------------------------------------- listening on dag0 Bro Version: 1.0 Started with the following command line options: -W -i dag0 -i dag1 mybrobox.bro listening on dag1 Reading .state/state.bst ... Capture filter: -------------------------------------------- Any thoughts ?? Thanks a lot for all the help. Aashish On Wed, Feb 08, 2006 at 10:57:06AM -0800, Robin Sommer wrote: > > On Tue, Feb 07, 2006 at 23:06 -0600, you wrote: > > > Yes we would definately like to try your prototypical code for DAG > > support. Can you please share it with us. > > Great! I think I need to get approval from Endace to give out the > code (the API is subject to non-disclosure) but that shouldn't be a > problem. I'll then update the code to the current devel version and > send you a patch. You won't need much of documentation as it > essentially just acts like any other device. You still need to > setup the DAG card with the Endace tools though as that's not > yet part of the code (the API for these things is undocumented). > > > Also, is there any specific manner to defie dag interfaces in > > bro.cfg ? since dag interfaces don't behave like regular network > > interfaces. > > Not sure if I understand what you mean. With the patch, you'll just > use "dag0" as the capture device and Bro will figure out that it is > a DAG card. When using the pcap wrapper, it should behave like any > other pcap device, should it not? > > > Also, I tried removing "-I../../include-linux" very coarsely by commenting the code in configure file. > > Sorry, then this doesn't help. Was really just a guess as I remember > having solved some similar problem once by getting rid of this -I. > (For the pcap error, see my upcoming post to the list). > > Robin > > -- > Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org > ICIR/ICSI * Fax +1 (510) 666-2956 * www.icir.org -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20060209/935b9984/attachment.bin From frenzy at frenzy.org Fri Feb 17 10:38:45 2006 From: frenzy at frenzy.org (frenzy at frenzy.org) Date: Fri, 17 Feb 2006 11:38:45 -0700 (MST) Subject: [Bro] Daily report and Byte Transfer Pairs Message-ID: I have been noticing that sometimes the daily report Byte Transfer Pair information in the Local bytes and Remote Bytes values can be off by a very large factor from the actual traffic size. Is this caused by the traffic estimation algorithm, and what factors could contribute to that larger size? The transfers in question were some HTTP traffic that didn't get to be above 100 K in size, and Bro reported it as being 1815 M. This is using the current Bro 1.x branch code. Thanks for any input you folks can provide. Randy From vern at icir.org Fri Feb 17 10:44:37 2006 From: vern at icir.org (Vern Paxson) Date: Fri, 17 Feb 2006 10:44:37 -0800 Subject: [Bro] Daily report and Byte Transfer Pairs In-Reply-To: Your message of Fri, 17 Feb 2006 11:38:45 MST. Message-ID: <200602171844.k1HIibdt062956@jaguar.icir.org> > I have been noticing that sometimes the daily report Byte Transfer Pair > information in the Local bytes and Remote Bytes values can be off by a > very large factor from the actual traffic size. The problems are artifacts of Bro's use of sequence numbers to compute connection sizes. The estimates can be too large because of connections that have malformed sequence numbers (especially in RST packets); or too small due to connections for which Bro misses the beginning or end (and hence doesn't compute a size), which in fact is much more likely to happen for big (and thus long-lived) connections than small ones. We have a draft paper on incoporating random sampling into Bro's analysis. This allows it to make more accurate estimates of traffic volume and also the sizes of individual connections. One part of this is already available in the Bro distribution using large-conns.bro. Another part (that does overall traffic profiling) has not yet been integrated. Vern From manuel.crotti at ing.unibs.it Mon Feb 20 01:22:27 2006 From: manuel.crotti at ing.unibs.it (Manuel Crotti) Date: Mon, 20 Feb 2006 10:22:27 +0100 Subject: [Bro] bro and flow splitting Message-ID: <84C930A6-7661-41F7-A8F3-6446391FCF85@ing.unibs.it> Hi all, Bro can extract flow informations from a dumpfile (I use it with mt option). I would split the entire dump in parts, one for each flow included in the dump. Is it possible with the sole bro? TnX, Manuel. From dmvasil at ornl.gov Mon Feb 20 08:35:58 2006 From: dmvasil at ornl.gov (David Vasil) Date: Mon, 20 Feb 2006 11:35:58 -0500 Subject: [Bro] Bro logging Message-ID: <43F9EFEE.6090000@ornl.gov> Hello, I'm using Bro 1.0 with some success at high rates of traffic. I would like to configure some automatic handling of signiture/portscans/etc by parsing log output with SEC and syslog-ng. I set 'redef syslog_alarms = T;' in my site policy after which Bro failed to start giving this warning: line 51 (syslog_alarms): error, "redef" used but not previously defined I tried setting 'global enable_syslog = T &redef;' instead, but it didnt seem to put any of the warnings from signitures in syslog. What is the proper way of doing this? Thanks. -- | David Vasil | Oak Ridge National Laboratory NCCS Division | High Performance Computing Systems Administrator | Bldg: 5600-A115 Phone: (865)241-5562 From christian at whoop.org Mon Feb 20 08:53:38 2006 From: christian at whoop.org (Christian Kreibich) Date: Mon, 20 Feb 2006 16:53:38 +0000 Subject: [Bro] bro and flow splitting In-Reply-To: <84C930A6-7661-41F7-A8F3-6446391FCF85@ing.unibs.it> References: <84C930A6-7661-41F7-A8F3-6446391FCF85@ing.unibs.it> Message-ID: <1140454419.2906.182.camel@localhost> On Mon, 2006-02-20 at 10:22 +0100, Manuel Crotti wrote: > Hi all, > Bro can extract flow informations from a dumpfile (I use it with mt > option). > I would split the entire dump in parts, one for each flow included in > the dump. > Is it possible with the sole bro? Manuel, the demux.bro policy can write the application-layer *contents* of individual flows to separate output files: http://www.bro-ids.org/Bro-reference-manual/demux-Analysis-Script.html If you want to demux the flows' individual *packets*, then check out Netdude's command-line demux plugin. It can demux input traces on per- flow, per-{src,dst}-port, and per-{src,dest}-port+host granularity: http://sourceforge.net/project/showfiles.php?group_id=22071&package_id=108810&release_id=232168 Cheers, Christian. -- ________________________________________________________________________ http://www.cl.cam.ac.uk/~cpk25 http://www.whoop.org From bltierney at lbl.gov Mon Feb 20 14:34:01 2006 From: bltierney at lbl.gov (Brian Tierney) Date: Mon, 20 Feb 2006 14:34:01 -0800 Subject: [Bro] Bro logging In-Reply-To: <43F9EFEE.6090000@ornl.gov> References: <43F9EFEE.6090000@ornl.gov> Message-ID: <78DB2B41-384C-4E6D-8979-A208CB862844@lbl.gov> Sorry, but the manual is not correct re: 'redef syslog_alarms = T;' However by default all alarms should be going to syslog, (see bro.init: const enable_syslog = T &redef;) You have alerts in your alarm file that are not in syslog? Maybe check you syslog.conf file? On Feb 20, 2006, at 8:35 AM, David Vasil wrote: > Hello, I'm using Bro 1.0 with some success at high rates of > traffic. I > would like to configure some automatic handling of > signiture/portscans/etc by parsing log output with SEC and syslog- > ng. I set 'redef syslog_alarms = T;' in my site policy after which > Bro failed to start giving this warning: > > line 51 (syslog_alarms): error, "redef" used but not previously > defined > > I tried setting 'global enable_syslog = T &redef;' instead, but it > didnt seem to put any of the warnings from signitures in syslog. > > What is the proper way of doing this? Thanks. > > -- > | David Vasil > | Oak Ridge National Laboratory NCCS Division > | High Performance Computing Systems Administrator > | Bldg: 5600-A115 Phone: (865)241-5562 > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro ------------------------------------------------------------------------ ------------------- Brian L. Tierney, Lawrence Berkeley National Laboratory (LBNL) 1 Cyclotron Rd. MS: 50B-2239, Berkeley, CA 94720 tel: 510-486-7381 fax: 510-495-2998 efax: 425-642-4558 bltierney at lbl.gov http://dsd.lbl.gov/~tierney ------------------------------------------------------------------------ ------------------ From jbabbin at comcast.net Tue Feb 21 18:19:50 2006 From: jbabbin at comcast.net (jbabbin at comcast.net) Date: Wed, 22 Feb 2006 02:19:50 +0000 Subject: [Bro] question about ssl analyzer Message-ID: <022220060219.5569.43FBCA4600040DAC000015C1220730079302070D0D0E0D06@comcast.net> List, I have a question about the SSL analyzer and the ssl.bro policy file. I'm trying to use this policy to track ssl certificates across a link. However, this policy seems to be broken. Upon starting bro 1.0 with ssl.bro enabled I get line 1: warning: event handlers never invoked: line 1: warning: account_tried line 1: warning: ssl_ciphersuite_seen after the first ssl connection or sometimes the second... bro seg faults with no core or debugging information. Also I have recompiled bro with debugging enabled and retried it with no luck as well. Lastly, I have turned almost everything off in the policy file just to check if thats the issue as well. Anyone been able to run the ssl analyzer successfully? Thanks, Jake Babbin From dmvasil at ornl.gov Wed Feb 22 06:12:34 2006 From: dmvasil at ornl.gov (David Vasil) Date: Wed, 22 Feb 2006 09:12:34 -0500 Subject: [Bro] question about ssl analyzer In-Reply-To: <022220060219.5569.43FBCA4600040DAC000015C1220730079302070D0D0E0D06@comcast.net> References: <022220060219.5569.43FBCA4600040DAC000015C1220730079302070D0D0E0D06@comcast.net> Message-ID: <43FC7152.7060009@ornl.gov> jbabbin at comcast.net wrote: > after the first ssl connection or sometimes the second... bro seg faults with no core or debugging information. Also I have recompiled bro with debugging enabled and retried it with no luck as well. > Lastly, I have turned almost everything off in the policy file just to check if thats the issue as well. > > Anyone been able to run the ssl analyzer successfully? I tried running with the ssl policy but bro segfaulted in the same manner as you are seeing. I also am running Bro 1.0. -- | David Vasil | Oak Ridge National Laboratory NCCS Division | High Performance Computing Systems Administrator | Bldg: 5600-A115 Phone: (865)241-5562 From hdreger at net.in.tum.de Wed Feb 22 08:03:47 2006 From: hdreger at net.in.tum.de (Holger Dreger) Date: Wed, 22 Feb 2006 17:03:47 +0100 Subject: [Bro] question about ssl analyzer In-Reply-To: <022220060219.5569.43FBCA4600040DAC000015C1220730079302070D0D0E0D06@comcast.net> References: <022220060219.5569.43FBCA4600040DAC000015C1220730079302070D0D0E0D06@comcast.net> Message-ID: <93037759-B7CB-434A-BBA3-1C35AFC5526D@net.in.tum.de> On 22.02.2006 at 03:19 jbabbin at comcast.net wrote: > after the first ssl connection or sometimes the second... bro seg > faults with no core or debugging information. Also I have > recompiled bro with debugging enabled and retried it with no luck > as well. > Lastly, I have turned almost everything off in the policy file just > to check if thats the issue as well. > > Anyone been able to run the ssl analyzer successfully? There is a preliminary bugfix appended in the patch (apply in the src directory using "patch -p1"). If it works for you, please let me know if you find any inconsistencies in the analyzer output. Holger -------------- next part -------------- A non-text attachment was scrubbed... Name: bro1.0SSL.patch Type: application/octet-stream Size: 4845 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20060222/b5bd4fd3/attachment.obj