From dklinedinst at lbl.gov Mon Aug 1 16:56:22 2011 From: dklinedinst at lbl.gov (Dan Klinedinst) Date: Mon, 1 Aug 2011 19:56:22 -0400 Subject: [Bro] 802.11 link headers? Message-ID: All, I dumped a bunch of packets off a wireless network to a pcap file. tcpdump says the link-type is IEEE802_11_RADIO. If I try to run Bro against the file, I get "unknown data link type 0x7f". I assume this means Bro doesn't understand the link layer data, since it's not Ethernet. [Error is from PktSrc.cc PktSrc::SetHdrSize()] So, is there a way to tell Bro to just ignore the link layer? Or would it then not know where the layer 3 data starts? And if there is not, anyone know a tool that will strip the 802.11 headers and replace them with fake Ethernet headers so I can use Bro on the traffic?? Thanks Dan -- Dan Klinedinst Lawrence Berkeley National Laboratory 510.486.4219 dklinedinst at lbl.gov From gregor at icir.org Mon Aug 1 17:40:26 2011 From: gregor at icir.org (Gregor Maier) Date: Mon, 01 Aug 2011 17:40:26 -0700 Subject: [Bro] 802.11 link headers? In-Reply-To: References: Message-ID: <4E37477A.4030808@icir.org> Hi, the IEEE802_11_RADIO linktype adds a bunch of information from the radio before the actual ethernet header and it appears that this info is variable length. The problem is that Bro doesn't have support for this linktype and so Bro doesn't know where the IP header starts. Since this linktype adds a variable length header it's not straight forward to add support for it (although it's probably not too hard either). (For fixed length headers one would just add an appropriate case to get_link_header_size() in PktSrc.cc) I've added a feature request to Bro's tracker for that though. If you can capture new traces and depending on your OS and tcpdump version, so can run tcpdump *without* the "-I" option or with a "-y EN10MB" option. The tcpdump records plain old ethernet only headers that Bro can deal with. Unfortunately, I don't know of a tool that can convert from IEEE802_11_RADIO to EN10MB :-( cu Gregor On 8/1/11 16:56 , Dan Klinedinst wrote: > All, > I dumped a bunch of packets off a wireless network to a pcap file. > tcpdump says the link-type is . If I try to run Bro > against the file, I get "unknown data link type 0x7f". I assume this > means Bro doesn't understand the link layer data, since it's not > Ethernet. > > [Error is from PktSrc.cc PktSrc::SetHdrSize()] > > So, is there a way to tell Bro to just ignore the link layer? Or > would it then not know where the layer 3 data starts? And if there is > not, anyone know a tool that will strip the 802.11 headers and replace > them with fake Ethernet headers so I can use Bro on the traffic?? > > Thanks > Dan > -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From dklinedinst at lbl.gov Mon Aug 1 22:01:03 2011 From: dklinedinst at lbl.gov (Dan Klinedinst) Date: Tue, 2 Aug 2011 01:01:03 -0400 Subject: [Bro] 802.11 link headers? In-Reply-To: <4E37477A.4030808@icir.org> References: <4E37477A.4030808@icir.org> Message-ID: All, It turns out that if you force tcpdump to output IEEE802_11 (without the _RADIO), you get a standard, fixed-length 802.11 header of 32 bytes. I added an entry for that in get_link_header_size() in PktSrc.cc and now Bro works like a charm on live WiFi traffic. I'll submit a patch tomorrow. Dan On Mon, Aug 1, 2011 at 8:40 PM, Gregor Maier wrote: > Hi, > > the IEEE802_11_RADIO linktype adds a bunch of information from the radio > before the actual ethernet header and it appears that this info is variable > length. The problem is that Bro doesn't have support for this linktype and > so Bro doesn't know where the IP header starts. Since this linktype adds a > variable length header it's not straight forward to add support for it > (although it's probably not too hard either). (For fixed length headers one > would just add an appropriate case to get_link_header_size() in PktSrc.cc) > > I've added a feature request to Bro's tracker for that though. > > If you can capture new traces and depending on your OS and tcpdump version, > so can run tcpdump *without* the "-I" option or with a "-y EN10MB" option. > The tcpdump records plain old ethernet only headers that Bro can deal with. > > Unfortunately, I don't know of a tool that can convert from IEEE802_11_RADIO > to EN10MB :-( > > > cu > Gregor > > On 8/1/11 16:56 , Dan Klinedinst wrote: >> >> All, >> I dumped a bunch of packets off a wireless network to a pcap file. >> tcpdump says the link-type is . ?If I try to run Bro >> against the file, I get "unknown data link type 0x7f". ?I assume this >> means Bro doesn't understand the link layer data, since it's not >> Ethernet. >> >> [Error is from PktSrc.cc PktSrc::SetHdrSize()] >> >> So, is there a way to tell Bro to just ignore the link layer? ?Or >> would it then not know where the layer 3 data starts? ?And if there is >> not, anyone know a tool that will strip the 802.11 headers and replace >> them with fake Ethernet headers so I can use Bro on the traffic?? >> >> Thanks >> Dan >> > > > -- > Gregor Maier > ? > Int. Computer Science Institute (ICSI) > 1947 Center St., Ste. 600 > Berkeley, CA 94704, USA > http://www.icir.org/gregor/ > -- Dan Klinedinst Lawrence Berkeley National Laboratory 510.486.4219 dklinedinst at lbl.gov From christoph.moebius at mailbox.tu-dresden.de Tue Aug 2 05:35:33 2011 From: christoph.moebius at mailbox.tu-dresden.de (=?UTF-8?B?Q2hyaXN0b3BoIE3DtmJpdXM=?=) Date: Tue, 02 Aug 2011 14:35:33 +0200 Subject: [Bro] Compiling error with bro-1.5.3 Message-ID: <4E37EF15.4050708@mailbox.tu-dresden.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 When compiling the latest version I get an error telling that RE_Matcher was defined twice. src/RE.h:176 aux/binpac/lib/binpac_pcre.h:12 Using gcc 4.2.4 - -- /"\ ASCII Ribbon Campaign \ / Respect for low technology. X Keep e-mail messages readable by any computer system. / \ Keep it ASCII. GPG Fingerprint: 38E7 4163 3EA1 DE55 FECE 419B 1AE5 2085 CEBC 5A14 christoph.moebius at mailbox.tu-dresden.de -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFON+8VGuUghc68WhQRCKbXAJ4xVirPkEG7wc2/pasJNrgziBJNAwCfY9Qi GXlOd6xKJ8tu4VKZ1C2bSxE= =NzVa -----END PGP SIGNATURE----- From gregor at icir.org Tue Aug 2 10:11:00 2011 From: gregor at icir.org (Gregor Maier) Date: Tue, 02 Aug 2011 10:11:00 -0700 Subject: [Bro] 802.11 link headers? In-Reply-To: References: <4E37477A.4030808@icir.org> Message-ID: <4E382FA4.3000700@icir.org> On 8/1/11 22:01 , Dan Klinedinst wrote: > It turns out that if you force tcpdump to output IEEE802_11 (without > the _RADIO), you get a standard, fixed-length 802.11 header of 32 > bytes. I added an entry for that in get_link_header_size() in > PktSrc.cc and now Bro works like a charm on live WiFi traffic. I'll > submit a patch tomorrow. Cool! Note however that libpcap's filter code generation treats both IEEE802_11 and IEEE802_11_RADIO as having a variable length header. It might well be that the variable part only varies between drivers, so it might be a constant 32 bytes with your NIC but not necessarily with others. (I might be wrong though. I didn't find a specification for these DLT's just guessing from glancing at libpcap) cu gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From seth at icir.org Tue Aug 2 10:50:14 2011 From: seth at icir.org (Seth Hall) Date: Tue, 2 Aug 2011 13:50:14 -0400 Subject: [Bro] Bro Workshop 2011 Message-ID: I'm happy to announce that we are finally opening registration for the next Bro workshop. It's going to be held Nov 8-10 at NCSA in Urbana, IL. For more information and the link to the registration form, please visit: http://www.bro-ids.org/community/workshop2011.html I'm looking forward to seeing people there! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From dklinedinst at lbl.gov Tue Aug 2 11:00:09 2011 From: dklinedinst at lbl.gov (Dan Klinedinst) Date: Tue, 2 Aug 2011 14:00:09 -0400 Subject: [Bro] 802.11 link headers? In-Reply-To: <4E382FA4.3000700@icir.org> References: <4E37477A.4030808@icir.org> <4E382FA4.3000700@icir.org> Message-ID: Gregor, Thanks for reminding me - I forgot that the header size will, at a minimum, change if you use WEP/WPA*. I'll take a look at this some more and see if I can write a patch to cover all the cases (at least without the radio headers). Dan On Tue, Aug 2, 2011 at 1:11 PM, Gregor Maier wrote: > On 8/1/11 22:01 , Dan Klinedinst wrote: >> It turns out that if you force tcpdump to output IEEE802_11 (without >> the _RADIO), you get a standard, fixed-length 802.11 header of 32 >> bytes. ?I added an entry for that in get_link_header_size() in >> PktSrc.cc and now Bro works like a charm on live WiFi traffic. ?I'll >> submit a patch tomorrow. > > > Cool! > Note however that libpcap's filter code generation treats both IEEE802_11 > and IEEE802_11_RADIO as having a variable length header. It might well be > that the variable part only varies between drivers, so it might be a > constant 32 bytes with your NIC but not necessarily with others. (I might be > wrong though. I didn't find a specification for these DLT's just guessing > from glancing at libpcap) > > cu > gregor > -- > Gregor Maier > ? > Int. Computer Science Institute (ICSI) > 1947 Center St., Ste. 600 > Berkeley, CA 94704, USA > http://www.icir.org/gregor/ > -- Dan Klinedinst Lawrence Berkeley National Laboratory 510.486.4219 dklinedinst at lbl.gov From seth at icir.org Tue Aug 2 13:41:29 2011 From: seth at icir.org (Seth Hall) Date: Tue, 2 Aug 2011 16:41:29 -0400 Subject: [Bro] Workshop presentations? Message-ID: <9500E928-1E72-4178-B7A5-3DE7BD3A816C@icir.org> Is anyone doing something cool with Bro or with the data that they are getting from Bro and could you talk about it at the workshop? It's great to get real world use of Bro injected here and there around the teaching sections. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From christoph.moebius at mailbox.tu-dresden.de Wed Aug 3 07:40:44 2011 From: christoph.moebius at mailbox.tu-dresden.de (=?UTF-8?B?Q2hyaXN0b3BoIE3DtmJpdXM=?=) Date: Wed, 03 Aug 2011 16:40:44 +0200 Subject: [Bro] Compiling error with bro-1.5.3 Message-ID: <4E395DEC.4050906@mailbox.tu-dresden.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Shame on me. I forgot that I (long ago) added some header files to aux/binpac/lib (as described in the Wiki) in order to decouple it from Bro. - -- /"\ ASCII Ribbon Campaign \ / Respect for low technology. X Keep e-mail messages readable by any computer system. / \ Keep it ASCII. GPG Fingerprint: 38E7 4163 3EA1 DE55 FECE 419B 1AE5 2085 CEBC 5A14 christoph.moebius at mailbox.tu-dresden.de -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFOOV3sGuUghc68WhQRCNrCAJ4g0AcTQUjyWNuUpRU1PRO3yAVC1wCghLQI KpvZUXof4Pzzt/j111Uvaqo= =HSRZ -----END PGP SIGNATURE----- From seth at icir.org Wed Aug 3 09:31:58 2011 From: seth at icir.org (Seth Hall) Date: Wed, 3 Aug 2011 12:31:58 -0400 Subject: [Bro] Workshop presentations? In-Reply-To: <9500E928-1E72-4178-B7A5-3DE7BD3A816C@icir.org> References: <9500E928-1E72-4178-B7A5-3DE7BD3A816C@icir.org> Message-ID: <02D5DF9B-415E-46F7-ADC1-9270C96E68E1@icir.org> On Aug 2, 2011, at 4:41 PM, Seth Hall wrote: > Is anyone doing something cool with Bro or with the data that they are getting from Bro and could you talk about it at the workshop? It's great to get real world use of Bro injected here and there around the teaching sections. Let me provide one small bit of clarification. I think that many people would be very interested to find out how people are working with Bro. The community has always been so fragmented on usage styles and no one knows how anyone else works with Bro, but I think it would be very beneficial if we could start identifying common usage scenarios and integration pattern. If anyone wants to and can talk about what they do with Bro in incident response teams, that would absolutely fit into this category of presentation. We'd be glad to have people talk about cool and novel stuff they're doing with Bro too though. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From mcholste at gmail.com Wed Aug 3 10:58:27 2011 From: mcholste at gmail.com (Martin Holste) Date: Wed, 3 Aug 2011 12:58:27 -0500 Subject: [Bro] Workshop presentations? In-Reply-To: <02D5DF9B-415E-46F7-ADC1-9270C96E68E1@icir.org> References: <9500E928-1E72-4178-B7A5-3DE7BD3A816C@icir.org> <02D5DF9B-415E-46F7-ADC1-9270C96E68E1@icir.org> Message-ID: If I'm able to come, I can certainly talk about how we're using Bro to do SSL profiling; we should be in production by then. On Wednesday, August 3, 2011, Seth Hall wrote: > > On Aug 2, 2011, at 4:41 PM, Seth Hall wrote: > >> Is anyone doing something cool with Bro or with the data that they are getting from Bro and could you talk about it at the workshop? It's great to get real world use of Bro injected here and there around the teaching sections. > > > Let me provide one small bit of clarification. I think that many people would be very interested to find out how people are working with Bro. The community has always been so fragmented on usage styles and no one knows how anyone else works with Bro, but I think it would be very beneficial if we could start identifying common usage scenarios and integration pattern. If anyone wants to and can talk about what they do with Bro in incident response teams, that would absolutely fit into this category of presentation. > > We'd be glad to have people talk about cool and novel stuff they're doing with Bro too though. :) > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110803/15729c11/attachment.html From seth at icir.org Wed Aug 3 11:06:27 2011 From: seth at icir.org (Seth Hall) Date: Wed, 3 Aug 2011 14:06:27 -0400 Subject: [Bro] Workshop presentations? In-Reply-To: References: <9500E928-1E72-4178-B7A5-3DE7BD3A816C@icir.org> <02D5DF9B-415E-46F7-ADC1-9270C96E68E1@icir.org> Message-ID: On Aug 3, 2011, at 1:58 PM, Martin Holste wrote: > If I'm able to come, I can certainly talk about how we're using Bro to do SSL profiling; we should be in production by then. Nice! I'm really interested personally to hear about that. Thanks. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From christoph.moebius at mailbox.tu-dresden.de Thu Aug 4 06:18:45 2011 From: christoph.moebius at mailbox.tu-dresden.de (=?UTF-8?B?Q2hyaXN0b3BoIE3DtmJpdXM=?=) Date: Thu, 04 Aug 2011 15:18:45 +0200 Subject: [Bro] Using binpac standalone Message-ID: <4E3A9C35.4040806@mailbox.tu-dresden.de> Hi all, I want to use binpac as a standalone parser generator. My use case is: -write a PDU description -feed it to a standalone binpac -get the parser files for the PDU -use that files somewhere else in order to access protocol fields programmatically I tried the Howto of 2008 in the Wiki (http://is.gd/pyZ7If) which leads me to a problem: I added the mentioned RE.h, binpac_stdalone.h and binpac_pcre.h to binpac/lib and recompiled in aux/binpac. Is it right that recompiling didn't change the binpac binary in aux/binpac/src? And if yes from where do I call binpac then and where should the .pac file be placed? Even the unbro'ed http.pac file in the Wiki refers to binpac-lib.pac which resides outside the binpac folder. Compiling http.pac then leads to an error since bytestring_to_int is undeclared. Adding binpac-lib.pac to binpac/lib didn't help. Before changing too much and getting lost I better ask :) Even If solve the first problem there's another question: I don't need any functionality of a protocol machine. So I'd like to get rid of the need to define connections and analyzers. I just need access to the fields of a single PDU. Anything else is beyond my focus. Do I really ALWAYS need to define a connection and a flow? So is there now another way than the wikified to use binpac as standalone parser generator? Best regards, Christoph M?bius -- /"\ ASCII Ribbon Campaign \ / Respect for low technology. X Keep e-mail messages readable by any computer system. / \ Keep it ASCII. GPG Fingerprint: 38E7 4163 3EA1 DE55 FECE 419B 1AE5 2085 CEBC 5A14 christoph.moebius at mailbox.tu-dresden.de From giralt at reservoir.com Fri Aug 5 02:15:13 2011 From: giralt at reservoir.com (giralt) Date: Fri, 05 Aug 2011 11:15:13 +0200 Subject: [Bro] Using binpac standalone In-Reply-To: <4E3A9C35.4040806@mailbox.tu-dresden.de> References: <4E3A9C35.4040806@mailbox.tu-dresden.de> Message-ID: <4E3BB4A1.902@reservoir.com> What version of Bro are you running? the wiki is for 1.4, so not sure if that may affect if you are using a newer version. you can place the .pac file anywhere you want, for instance, in a pac folder. Then from that folder, you can call binpac and the output files .cc / .h will be placed in the same folder by default. For binpac to see binpac-lib.pac, you can use the -I option (binpac --help) Looks like you may also need to patch the file binpac_bytestring.h if you want to run the example. I am attaching a patch. On 08/04/2011 03:18 PM, Christoph M?bius wrote: > Hi all, > > I want to use binpac as a standalone parser generator. My use case is: > -write a PDU description > -feed it to a standalone binpac > -get the parser files for the PDU > -use that files somewhere else in order to access protocol fields > programmatically > > I tried the Howto of 2008 in the Wiki (http://is.gd/pyZ7If) which leads > me to a problem: > I added the mentioned RE.h, binpac_stdalone.h and binpac_pcre.h to > binpac/lib and recompiled in aux/binpac. Is it right that recompiling > didn't change the binpac binary in aux/binpac/src? And if yes from where > do I call binpac then and where should the .pac file be placed? Even the > unbro'ed http.pac file in the Wiki refers to binpac-lib.pac which > resides outside the binpac folder. Compiling http.pac then leads to an > error since bytestring_to_int is undeclared. > Adding binpac-lib.pac to binpac/lib didn't help. Before changing too > much and getting lost I better ask :) > > Even If solve the first problem there's another question: I don't need > any functionality of a protocol machine. So I'd like to get rid of the > need to define connections and analyzers. I just need access to the > fields of a single PDU. Anything else is beyond my focus. Do I really > ALWAYS need to define a connection and a flow? > > So is there now another way than the wikified to use binpac as > standalone parser generator? > > > Best regards, > Christoph M?bius > > -- > /"\ ASCII Ribbon Campaign > \ / Respect for low technology. > X Keep e-mail messages readable by any computer system. > / \ Keep it ASCII. > > GPG Fingerprint: 38E7 4163 3EA1 DE55 FECE 419B 1AE5 2085 CEBC 5A14 > christoph.moebius at mailbox.tu-dresden.de > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: binpac_bytestring.patch Type: text/x-patch Size: 592 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110805/be892057/attachment.bin From robin at icir.org Fri Aug 5 08:00:44 2011 From: robin at icir.org (Robin Sommer) Date: Fri, 5 Aug 2011 08:00:44 -0700 Subject: [Bro] Using binpac standalone In-Reply-To: <4E3A9C35.4040806@mailbox.tu-dresden.de> References: <4E3A9C35.4040806@mailbox.tu-dresden.de> Message-ID: <20110805150044.GA81199@icir.org> On Thu, Aug 04, 2011 at 15:18 +0200, Christoph M?bius wrote: > I want to use binpac as a standalone parser generator. I have some example code how to do that. I'll put it together and get back to you. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From hlin33 at illinois.edu Fri Aug 5 09:17:23 2011 From: hlin33 at illinois.edu (Hui Lin) Date: Fri, 5 Aug 2011 09:17:23 -0700 Subject: [Bro] Using binpac standalone In-Reply-To: <20110805150044.GA81199@icir.org> References: <4E3A9C35.4040806@mailbox.tu-dresden.de> <20110805150044.GA81199@icir.org> Message-ID: HI, Robin, Can I get that code too? I tried to run Binpac stand alone before, but not successful. But may need that in recently DNP3 protocol test. Best, Hui On Fri, Aug 5, 2011 at 8:00 AM, Robin Sommer wrote: > > On Thu, Aug 04, 2011 at 15:18 +0200, Christoph M?bius wrote: > > > I want to use binpac as a standalone parser generator. > > I have some example code how to do that. I'll put it together and get > back to you. > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Hui Lin Research Assistant DEPEND Research Group, ECE Department University of Illinois at Urbana-Champaign hlin33 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110805/3f3ebed6/attachment.html From robin at icir.org Fri Aug 5 10:16:51 2011 From: robin at icir.org (Robin Sommer) Date: Fri, 5 Aug 2011 10:16:51 -0700 Subject: [Bro] Using binpac standalone In-Reply-To: References: <4E3A9C35.4040806@mailbox.tu-dresden.de> <20110805150044.GA81199@icir.org> Message-ID: <20110805171651.GB90520@icir.org> On Fri, Aug 05, 2011 at 09:17 -0700, Hui Lin wrote: > Can I get that code too? I tried to run Binpac stand alone before, but not > successful. But may need that in recently DNP3 protocol test. Sure, will send it to you too. Hopefully later today. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From gekkeharry13 at hotmail.com Mon Aug 8 05:36:10 2011 From: gekkeharry13 at hotmail.com (Jeffrey Everling) Date: Mon, 8 Aug 2011 14:36:10 +0200 Subject: [Bro] Wrapper Error with compiling In-Reply-To: <012601cc55c6$b1708a50$14519ef0$@hotmail.com> References: <012601cc55c6$b1708a50$14519ef0$@hotmail.com> Message-ID: Hi guys When I try to compile this bro 1.5.3 I have a problem shown in the screenshot below. I have googled my ass of and couldn't find a thing. Maybe you guys can help? Kind regards Jeffrey Everling -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110808/d76d366a/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/octet-stream Size: 17472 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110808/d76d366a/attachment.obj From seth at icir.org Mon Aug 8 07:30:17 2011 From: seth at icir.org (Seth Hall) Date: Mon, 8 Aug 2011 10:30:17 -0400 Subject: [Bro] Wrapper Error with compiling In-Reply-To: References: <012601cc55c6$b1708a50$14519ef0$@hotmail.com> Message-ID: <362FB759-D4A8-415A-87D1-9CD8976D1250@icir.org> On Aug 8, 2011, at 8:36 AM, Jeffrey Everling wrote: > When I try to compile this bro 1.5.3 I have a problem shown in the screenshot below. I have googled my ass of and couldn?t find a thing. Maybe you guys can help? It looks like you don't have "bash" in your path. I'm not sure what script is requiring bash, but you if make sure that bash is in your path it should work. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From gekkeharry13 at hotmail.com Mon Aug 8 07:51:22 2011 From: gekkeharry13 at hotmail.com (Jeffrey Everling) Date: Mon, 8 Aug 2011 16:51:22 +0200 Subject: [Bro] Wrapper Error with compiling In-Reply-To: <362FB759-D4A8-415A-87D1-9CD8976D1250@icir.org> References: <012601cc55c6$b1708a50$14519ef0$@hotmail.com> <362FB759-D4A8-415A-87D1-9CD8976D1250@icir.org> Message-ID: Thanks a lot. It turned out I needed to install Bash. But now I am stuck up with another error. For some reason I guess he can't find the version file in my install folder? Got any ideas for this one? Thanks for helping out. -----Original Message----- From: Seth Hall [mailto:seth at icir.org] Sent: maandag 8 augustus 2011 16:30 To: Jeffrey Everling Cc: bro at bro-ids.org Subject: Re: [Bro] Wrapper Error with compiling On Aug 8, 2011, at 8:36 AM, Jeffrey Everling wrote: > When I try to compile this bro 1.5.3 I have a problem shown in the screenshot below. I have googled my ass of and couldn't find a thing. Maybe you guys can help? It looks like you don't have "bash" in your path. I'm not sure what script is requiring bash, but you if make sure that bash is in your path it should work. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: Capture.PNG Type: application/octet-stream Size: 23410 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110808/b1d5c4f5/attachment.obj From seth at icir.org Mon Aug 8 08:00:52 2011 From: seth at icir.org (Seth Hall) Date: Mon, 8 Aug 2011 11:00:52 -0400 Subject: [Bro] Wrapper Error with compiling In-Reply-To: References: <012601cc55c6$b1708a50$14519ef0$@hotmail.com> <362FB759-D4A8-415A-87D1-9CD8976D1250@icir.org> Message-ID: <42EBA5C7-D6E8-42C7-B480-6C8EB28BB58B@icir.org> On Aug 8, 2011, at 10:51 AM, Jeffrey Everling wrote: > Thanks a lot. It turned out I needed to install Bash. But now I am stuck up > with another error. For some reason I guess he can't find the version file > in my install folder? > > Got any ideas for this one? It looks like you might want to try doing "make clean" and then starting over. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From gekkeharry13 at hotmail.com Mon Aug 8 08:31:19 2011 From: gekkeharry13 at hotmail.com (Jeffrey Everling) Date: Mon, 8 Aug 2011 17:31:19 +0200 Subject: [Bro] Wrapper Error with compiling In-Reply-To: <42EBA5C7-D6E8-42C7-B480-6C8EB28BB58B@icir.org> References: <012601cc55c6$b1708a50$14519ef0$@hotmail.com> <362FB759-D4A8-415A-87D1-9CD8976D1250@icir.org> <42EBA5C7-D6E8-42C7-B480-6C8EB28BB58B@icir.org> Message-ID: Nope that wasn't the solution. I tried it without and with a reboot. Still the same error. Can it be something with my dirnames from my installfolder or so? -----Original Message----- From: Seth Hall [mailto:seth at icir.org] Sent: maandag 8 augustus 2011 17:01 To: Jeffrey Everling Cc: bro at bro-ids.org Subject: Re: [Bro] Wrapper Error with compiling On Aug 8, 2011, at 10:51 AM, Jeffrey Everling wrote: > Thanks a lot. It turned out I needed to install Bash. But now I am > stuck up with another error. For some reason I guess he can't find the > version file in my install folder? > > Got any ideas for this one? It looks like you might want to try doing "make clean" and then starting over. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From gc355804 at ohio.edu Mon Aug 8 09:10:50 2011 From: gc355804 at ohio.edu (Gilbert Clark) Date: Mon, 08 Aug 2011 12:10:50 -0400 Subject: [Bro] Wrapper Error with compiling In-Reply-To: References: <012601cc55c6$b1708a50$14519ef0$@hotmail.com> <362FB759-D4A8-415A-87D1-9CD8976D1250@icir.org> <42EBA5C7-D6E8-42C7-B480-6C8EB28BB58B@icir.org> Message-ID: <4E400A8A.4020505@ohio.edu> Hmm, it seems like it's looking for the binary itself and can't find it. Have you tried adding /usr/local/bro/bin (or whichever folder into which the bro binary is being installed) to your $PATH? --Gilbert On 8/8/2011 11:31 AM, Jeffrey Everling wrote: > Nope that wasn't the solution. I tried it without and with a reboot. Still > the same error. Can it be something with my dirnames from my installfolder > or so? > > -----Original Message----- > From: Seth Hall [mailto:seth at icir.org] > Sent: maandag 8 augustus 2011 17:01 > To: Jeffrey Everling > Cc: bro at bro-ids.org > Subject: Re: [Bro] Wrapper Error with compiling > > > On Aug 8, 2011, at 10:51 AM, Jeffrey Everling wrote: > >> Thanks a lot. It turned out I needed to install Bash. But now I am >> stuck up with another error. For some reason I guess he can't find the >> version file in my install folder? >> >> Got any ideas for this one? > > It looks like you might want to try doing "make clean" and then starting > over. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From gekkeharry13 at hotmail.com Mon Aug 8 13:56:37 2011 From: gekkeharry13 at hotmail.com (Jeffrey Everling) Date: Mon, 8 Aug 2011 22:56:37 +0200 Subject: [Bro] Wrapper Error with compiling In-Reply-To: <4E400A8A.4020505@ohio.edu> References: <012601cc55c6$b1708a50$14519ef0$@hotmail.com> <362FB759-D4A8-415A-87D1-9CD8976D1250@icir.org> <42EBA5C7-D6E8-42C7-B480-6C8EB28BB58B@icir.org> <4E400A8A.4020505@ohio.edu> Message-ID: Ok managed to get past the install by doing a ./configure in the install folder. They didnt mention it in the tutorial i was reading. It's hard finding decent tutorials for this software. I'm trying to configure it now but to be honost, I don't know if I'm doing it right. Thanks for your help again guys :) -----Oorspronkelijk bericht----- Van: bro-bounces at bro-ids.org [mailto:bro-bounces at bro-ids.org] Namens Gilbert Clark Verzonden: maandag 8 augustus 2011 18:11 Aan: bro at bro-ids.org Onderwerp: Re: [Bro] Wrapper Error with compiling Hmm, it seems like it's looking for the binary itself and can't find it. Have you tried adding /usr/local/bro/bin (or whichever folder into which the bro binary is being installed) to your $PATH? --Gilbert On 8/8/2011 11:31 AM, Jeffrey Everling wrote: > Nope that wasn't the solution. I tried it without and with a reboot. > Still the same error. Can it be something with my dirnames from my > installfolder or so? > > -----Original Message----- > From: Seth Hall [mailto:seth at icir.org] > Sent: maandag 8 augustus 2011 17:01 > To: Jeffrey Everling > Cc: bro at bro-ids.org > Subject: Re: [Bro] Wrapper Error with compiling > > > On Aug 8, 2011, at 10:51 AM, Jeffrey Everling wrote: > >> Thanks a lot. It turned out I needed to install Bash. But now I am >> stuck up with another error. For some reason I guess he can't find >> the version file in my install folder? >> >> Got any ideas for this one? > > It looks like you might want to try doing "make clean" and then > starting over. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From gekkeharry13 at hotmail.com Tue Aug 9 06:22:42 2011 From: gekkeharry13 at hotmail.com (Jeffrey Everling) Date: Tue, 9 Aug 2011 15:22:42 +0200 Subject: [Bro] bro_config Message-ID: Hey guys Thanks a lot for the previous help. I think I am almost there, but when I run bro_config there seems to be a few errors. I 've managed to edit the .cfg manually but the programs keeps mailing me about a subnettree (see capture2.png)). Does this has a connection? Do I miss a port? I have captured my errors from bro_config to in capture.png Kinds Regards, Jeffrey Everling -----Original Message----- From: bro-bounces at bro-ids.org [mailto:bro-bounces at bro-ids.org] On Behalf Of Jeffrey Everling Sent: maandag 8 augustus 2011 22:57 To: 'Gilbert Clark' Cc: bro at bro-ids.org Subject: Re: [Bro] Wrapper Error with compiling Ok managed to get past the install by doing a ./configure in the install folder. They didnt mention it in the tutorial i was reading. It's hard finding decent tutorials for this software. I'm trying to configure it now but to be honost, I don't know if I'm doing it right. Thanks for your help again guys :) -----Oorspronkelijk bericht----- Van: bro-bounces at bro-ids.org [mailto:bro-bounces at bro-ids.org] Namens Gilbert Clark Verzonden: maandag 8 augustus 2011 18:11 Aan: bro at bro-ids.org Onderwerp: Re: [Bro] Wrapper Error with compiling Hmm, it seems like it's looking for the binary itself and can't find it. Have you tried adding /usr/local/bro/bin (or whichever folder into which the bro binary is being installed) to your $PATH? --Gilbert On 8/8/2011 11:31 AM, Jeffrey Everling wrote: > Nope that wasn't the solution. I tried it without and with a reboot. > Still the same error. Can it be something with my dirnames from my > installfolder or so? > > -----Original Message----- > From: Seth Hall [mailto:seth at icir.org] > Sent: maandag 8 augustus 2011 17:01 > To: Jeffrey Everling > Cc: bro at bro-ids.org > Subject: Re: [Bro] Wrapper Error with compiling > > > On Aug 8, 2011, at 10:51 AM, Jeffrey Everling wrote: > >> Thanks a lot. It turned out I needed to install Bash. But now I am >> stuck up with another error. For some reason I guess he can't find >> the version file in my install folder? >> >> Got any ideas for this one? > > It looks like you might want to try doing "make clean" and then > starting over. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: Capture.PNG Type: application/octet-stream Size: 27561 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110809/60518fa4/attachment.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: Capture2.PNG Type: application/octet-stream Size: 6822 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110809/60518fa4/attachment-0001.obj From alvinh999 at gmail.com Tue Aug 9 10:43:15 2011 From: alvinh999 at gmail.com (Alvin Huang) Date: Tue, 9 Aug 2011 13:43:15 -0400 Subject: [Bro] Bro for Beginners/Logging SSL Certificate Message-ID: Hey everyone, This is my first time using Linux as well as using Bro so it has taken a while for me to get it installed and up and running, but finally I think I have it. I am running Bro 1.5.3 on Ubuntu and I have gotten BroCtl to start but I have a couple questions: 1. Where are the rules written that Bro is supposed to alert on? I came from Snort so I know a bit about IDS but I don't know how Bro is set up. 2. Where are the logs produced? /spool/broctl.dt? What I really want to do is to log the packet(s) from an SSL handshake that contain a certificate. I was sort of able to do this in Snort. Snort gave me the right packets but the wrong data. I got the TCP Segment Data rather than the reassembled TCP packet of the whole certificate itself. I was told Bro could do this out of the box so hopefully this will work here. Is this possible? How should I go about doing this. I am a true beginner with Linux and I am having some trouble understanding what is going on. Thanks in advance Alvin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110809/a6389401/attachment.html From seth at icir.org Tue Aug 9 11:11:58 2011 From: seth at icir.org (Seth Hall) Date: Tue, 9 Aug 2011 14:11:58 -0400 Subject: [Bro] Bro for Beginners/Logging SSL Certificate In-Reply-To: References: Message-ID: <276B9759-FE79-4533-A646-9DF0F082AA92@icir.org> On Aug 9, 2011, at 1:43 PM, Alvin Huang wrote: > Is this possible? How should I go about doing this. I am a true beginner with Linux and I am having some trouble understanding what is going on. This isn't quite ready for you then. The next release has greatly improved certificate logging and we will likely have a script specifically for that that task once we have the next release available. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From dklinedinst at lbl.gov Wed Aug 10 07:57:28 2011 From: dklinedinst at lbl.gov (Dan Klinedinst) Date: Wed, 10 Aug 2011 10:57:28 -0400 Subject: [Bro] getting raw bytes? Message-ID: OK, this is possibly a dumb question, but I can't find it in documentation or existing scripts. How can I grab a few specific bytes from a connection? E.g., if I want to look for successful X11 connections, I expect to see the following immediately after the TCP header: 0100 0b00 0000. How do I write something like: if (c$id$resp_p == 6000) if (first_6_bytes_after_tcp_header == 01000b000000) do something ? Thanks. Sorry for the noob questions. Dan -- Dan Klinedinst Lawrence Berkeley National Laboratory 510.486.4219 dklinedinst at lbl.gov From seth at icir.org Wed Aug 10 08:08:20 2011 From: seth at icir.org (Seth Hall) Date: Wed, 10 Aug 2011 11:08:20 -0400 Subject: [Bro] getting raw bytes? In-Reply-To: References: Message-ID: On Aug 10, 2011, at 10:57 AM, Dan Klinedinst wrote: > if (c$id$resp_p == 6000) > if (first_6_bytes_after_tcp_header == 01000b000000) > do something You are just looking to write a signature... ==== x11.sigs ===== signature x11_6_special_bytes { ip-proto == tcp dst-port == 6000 payload /\x01\x00\x0b\x00\x00\x00/ tcp-state responder } ====== end x11.sigs =========== ==== start x11.bro ======= redef signature_files += "x11.sigs"; event signature_match(state: signature_state, msg: string, data: string) { if ( state$sig_id == "x11_6_special_bytes" ) { # do something. } } =======end x11.bro========== Make sure both of those are in your BROPATH and load the x11.bro script. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From gregor at icir.org Wed Aug 10 08:57:35 2011 From: gregor at icir.org (Gregor Maier) Date: Wed, 10 Aug 2011 08:57:35 -0700 Subject: [Bro] getting raw bytes? In-Reply-To: References: Message-ID: <4E42AA6F.5070108@icir.org> > You are just looking to write a signature... More info on signatures: http://www.bro-ids.org/documentation/signatures.html > ==== x11.sigs ===== > signature x11_6_special_bytes { > ip-proto == tcp > dst-port == 6000 > payload /\x01\x00\x0b\x00\x00\x00/ > tcp-state responder event "foo" is missing here. cu Gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From dklinedinst at lbl.gov Wed Aug 10 09:00:35 2011 From: dklinedinst at lbl.gov (Dan Klinedinst) Date: Wed, 10 Aug 2011 12:00:35 -0400 Subject: [Bro] getting raw bytes? In-Reply-To: References: Message-ID: Thanks Seth. If I read this right, this line: payload /\x01\x00\x0b\x00\x00\x00/ will match that byte pattern anywhere in the packet, no? Is it possible to give it a specific position / offset? E.g., with regex: /^\x01/ to specify the first byte must be x01, or /.{8}\x01/ would match it at the 9th byte. But I think this will match from the beginning of the packet, if it works at all. I'll test it. Anyway, I'm on the right path now, thanks! Dan On Wed, Aug 10, 2011 at 11:08 AM, Seth Hall wrote: > > On Aug 10, 2011, at 10:57 AM, Dan Klinedinst wrote: > >> if (c$id$resp_p == 6000) >> ?if (first_6_bytes_after_tcp_header == 01000b000000) >> ? ?do something > > > You are just looking to write a signature... > > ==== x11.sigs ===== > signature x11_6_special_bytes { > ?ip-proto == tcp > ?dst-port == 6000 > ?payload /\x01\x00\x0b\x00\x00\x00/ > ?tcp-state responder > } > ====== end x11.sigs =========== > > ==== start x11.bro ======= > redef signature_files += "x11.sigs"; > > event signature_match(state: signature_state, msg: string, data: string) > ? ? ? ?{ > ? ? ? ?if ( state$sig_id == "x11_6_special_bytes" ) > ? ? ? ? ? ? ? ?{ > ? ? ? ? ? ? ? ?# do something. > ? ? ? ? ? ? ? ?} > ? ? ? ?} > =======end x11.bro========== > > Make sure both of those are in your BROPATH and load the x11.bro script. > > ?.Seth > > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > -- Dan Klinedinst Lawrence Berkeley National Laboratory 510.486.4219 dklinedinst at lbl.gov From seth at icir.org Wed Aug 10 09:39:03 2011 From: seth at icir.org (Seth Hall) Date: Wed, 10 Aug 2011 12:39:03 -0400 Subject: [Bro] getting raw bytes? In-Reply-To: References: Message-ID: <295FA823-93F3-47DA-A699-DCD1CF7FF277@icir.org> On Aug 10, 2011, at 12:00 PM, Dan Klinedinst wrote: > But I think this will match from the beginning of the packet, if it > works at all. I'll test it. Signatures are implicitly anchored at the beginning of the stream. :) I could have anchored it myself like you were thinking, but I just chose not to since it's implicit anyway. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From dklinedinst at lbl.gov Wed Aug 10 09:50:08 2011 From: dklinedinst at lbl.gov (Dan Klinedinst) Date: Wed, 10 Aug 2011 12:50:08 -0400 Subject: [Bro] getting raw bytes? In-Reply-To: <295FA823-93F3-47DA-A699-DCD1CF7FF277@icir.org> References: <295FA823-93F3-47DA-A699-DCD1CF7FF277@icir.org> Message-ID: On Wed, Aug 10, 2011 at 12:39 PM, Seth Hall wrote: > > Signatures are implicitly anchored at the beginning of the stream. :) Awesome! Regex on binary data, I love it! Thanks Seth. BTW, everyone, I used Bro to process wifi traffic at DefCon this past weekend and got almost as many questions about Bro as I did about my viz software. I expect you'll see a bunch of downloads this week.... :-) Dan -- Dan Klinedinst Lawrence Berkeley National Laboratory 510.486.4219 dklinedinst at lbl.gov From seth at icir.org Wed Aug 10 09:57:54 2011 From: seth at icir.org (Seth Hall) Date: Wed, 10 Aug 2011 12:57:54 -0400 Subject: [Bro] getting raw bytes? In-Reply-To: References: <295FA823-93F3-47DA-A699-DCD1CF7FF277@icir.org> Message-ID: On Aug 10, 2011, at 12:50 PM, Dan Klinedinst wrote: > BTW, everyone, I used Bro to process wifi traffic at DefCon this past > weekend and got almost as many questions about Bro as I did about my > viz software. I expect you'll see a bunch of downloads this week.... > :-) Awesome! Thanks. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Fri Aug 12 09:12:04 2011 From: seth at icir.org (Seth Hall) Date: Fri, 12 Aug 2011 12:12:04 -0400 Subject: [Bro] Workshop hotel room block Message-ID: <93438C3B-0AE0-49BC-9427-A38B29A629D0@icir.org> Hi all, We just got a hotel group rate established for the upcoming workshop. It's going to be $109+tax/night. More information can be found on the workshop page on our website: http://www.bro-ids.org/community/workshop2011.html .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From redlamb19 at gmail.com Sat Aug 13 18:45:38 2011 From: redlamb19 at gmail.com (Peter Erickson) Date: Sat, 13 Aug 2011 20:45:38 -0500 Subject: [Bro] Split path into directory and filename Message-ID: <20110814014538.GA5993@does.not.exist> Is there a way use regex to extract portions of a string? I'm trying to write a function that accepts a path and breaks it into a directory and filename (/tmp/file.txt => [ /tmp, file.txt ]). I would like to do something as easy as /(\/.+)/([^\/]+)$/, but am not sure it's possible with bro (I wrote the expr quick so there are probably typos). Right now I have the following, but wondering if there is a better way: function path_split(path: string): string_array { local cpath = split(path, /\//); local ret_val: string_array; ret_val[2] = cpath[length(cpath)]; delete cpath[length(cpath)]; ret_val[1] = join_string_array("/", cpath); return ret_val; } The reason I ask is I'm looking to modify the http/file-extract.bro script so that the http responses are saved into a directory structure based on the src and dst ip addresses (e.g. http-items/src_ip/dst_ip). I plan to modify the generate_extraction_filename to create this path and then send the filename to a function to create the directory structure. (I know that modifying generate_extraction_filename will have adverse affects on other scripts, but I plan to update those as well.) If anyone cares, here is the function I wrote to recursively create the directory structure. function mkdirs(dir: string): bool { local path_split = split1(dir, /\/[^\/]*$/); local parent = path_split[1]; if ( parent == "" || length(path_split) == 1 ) return mkdir(dir); else { if ( ! mkdirs(parent) ) return F; return mkdir(dir); } return T; } Thanks in advance. -- Peter Erickson redlamb19 _at_ gmail _dot_ com From seth at icir.org Mon Aug 15 06:20:59 2011 From: seth at icir.org (Seth Hall) Date: Mon, 15 Aug 2011 09:20:59 -0400 Subject: [Bro] Split path into directory and filename In-Reply-To: <20110814014538.GA5993@does.not.exist> References: <20110814014538.GA5993@does.not.exist> Message-ID: <9B745DD3-CB54-4A8A-9B6B-A886A5547497@icir.org> On Aug 13, 2011, at 9:45 PM, Peter Erickson wrote: > Is there a way use regex to extract portions of a string? I'm trying to > write a function that accepts a path and breaks it into a directory and > filename (/tmp/file.txt => [ /tmp, file.txt ]). I would like to do > something as easy as /(\/.+)/([^\/]+)$/, but am not sure it's possible > with bro (I wrote the expr quick so there are probably typos). Nope, Bro's regular expressions don't support captures. You did it exactly the same way that I would have, by splitting on /\// and taking the last value as the file name and the rest as the path. > The reason I ask is I'm looking to modify the http/file-extract.bro > script so that the http responses are saved into a directory structure > based on the src and dst ip addresses (e.g. http-items/src_ip/dst_ip). Ah, that's interesting. We need to rework the way that works to put more control of the file naming in users hands, it's a definite shortcoming in the current iteration. I'll refactor it a little bit soon so that you can accomplish what you want without having to rewrite bits of functionality. :) > I plan to modify the generate_extraction_filename to create this path > and then send the filename to a function to create the directory > structure. (I know that modifying generate_extraction_filename will have > adverse affects on other scripts, but I plan to update those as well.) Yeah, I generally don't like the way I wrote that. > function mkdirs(dir: string): bool { Thanks for this function. I'll integrate it in some form soon. Since I see that using the code from the repository, I'd be happy to find how your experience with it has been if you are interested in sharing. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From redlamb19 at gmail.com Mon Aug 15 08:07:01 2011 From: redlamb19 at gmail.com (Peter Erickson) Date: Mon, 15 Aug 2011 10:07:01 -0500 Subject: [Bro] Split path into directory and filename In-Reply-To: <9B745DD3-CB54-4A8A-9B6B-A886A5547497@icir.org> References: <20110814014538.GA5993@does.not.exist> <9B745DD3-CB54-4A8A-9B6B-A886A5547497@icir.org> Message-ID: <20110815150700.GA23156@does.not.exist> ** Seth Hall [2011-08-15 09:20:59 -0400] ** > On Aug 13, 2011, at 9:45 PM, Peter Erickson wrote: > > > The reason I ask is I'm looking to modify the http/file-extract.bro > > script so that the http responses are saved into a directory structure > > based on the src and dst ip addresses (e.g. http-items/src_ip/dst_ip). > > Ah, that's interesting. We need to rework the way that works to put > more control of the file naming in users hands, it's a definite > shortcoming in the current iteration. I'll refactor it a little bit > soon so that you can accomplish what you want without having to > rewrite bits of functionality. :) No need to spend your time doing it. I got it working over the weekend. I updated the generate_extraction_filename to include a directory path as the first argument... and then left everything else the same. After obtaining the filename to use, I call the mkdirs command to create the directory structure. I also updated the file-extract.bro script to extract the client request payload as well. I'll try to attach my updated scripts to this email, but it they are stripped let me know and I'll send them to you directly. One thing I did notice over the weekend was a potential problem in file-extract (I'm using current as opposed to 1.5) with respect to http POST requests. The file-extract script watches for first_chunk = T before it starts capturing data, however with POST requests the first_chunk is set, and subsequently set to F, within the client request. Once the response gets processed, the first_chunk is F and the payload is never saved (Hopefully that makes sense). I fixed this by creating the following event which resets the first_chunk and mime_type in preparation for the response. The -15 priority will make sure that it executes AFTER logging the message to the logfile. event http_message_done(c: connection, is_orig: bool, stat: http_message_stat) &priority=-15 { c$http$first_chunk = T; delete c$http$mime_type; } > Since I see that using the code from the repository, I'd be happy to > find how your experience with it has been if you are interested in > sharing. I'm not sure why I started playing with the current version in the repo as opposed to 1.5, but I like it. The way the scripts are loaded and the directory structure makes much more sense to me as opposed to having them all in one directory. I also like the addition of the __load__.bro scripts. As seen above in my fix for the http POST problems, the new overloaded delete operator was a nice addition which made solving the problem almost trivial. So far, I haven't seen any problems with the current, but I have been running it one pcap files as opposed to live traffic. -- Peter Erickson redlamb19 _at_ gmail _dot_ com -------------- next part -------------- function generate_extraction_filename2(dir: string, prefix: string, c: connection, suffix: string): string { local conn_info = fmt("%s:%d-%s:%d", c$id$orig_h, c$id$orig_p, c$id$resp_h, c$id$resp_p); if ( prefix != "" ) conn_info = fmt("%s_%s", prefix, conn_info); if ( suffix != "" ) conn_info = fmt("%s_%s", conn_info, suffix); if ( dir != "" ) conn_info = fmt("%s/%s/%s/%s", dir, c$id$orig_h, c$id$resp_h, conn_info); return conn_info; } -------------- next part -------------- function mkdirs(dir: string): bool { local path_split = split1(dir, /\/[^\/]*$/); local parent = path_split[1]; if ( parent == "" || length(path_split) == 1 ) return mkdir(dir); else { if ( ! mkdirs(parent) ) return F; return mkdir(dir); } return T; } function path_dirname(path: string): string { #return path_split(path)[1] return split1(path, /\/[^\/]*$/)[1]; } function path_filename(path: string): string { #return path_split(path)[2] local cpath = split(path, /\//); return cpath[length(cpath)]; } function path_split(path: string): string_array { local cpath = split(path, /\//); local ret_val: string_array; ret_val[2] = cpath[length(cpath)]; delete cpath[length(cpath)]; ret_val[1] = join_string_array("/", cpath); return ret_val; } -------------- next part -------------- ##! Extracts the items from HTTP traffic, one per file. At this time only ##! the message body from the server can be extracted with this script. module HTTP; export { ## Pattern of file mime types to extract from HTTP entity bodies. const extract_file_types = /NO_DEFAULT/ &redef; ## The on-disk prefix for files to be extracted from HTTP entity bodies. const extraction_prefix = "http-item" &redef; const extraction_dir = "" &redef; const extract_requests = F &redef; const extract_responses = F &redef; redef record Info += { ## This field can be set per-connection to determine if the entity body ## will be extracted. It must be set to T on or before the first ## entity_body_data event. extracting_file: bool &default=F; ## This is the holder for the file handle as the file is being written ## to disk. extraction_file: file &log &optional; request_file: file &log &optional; response_file: file &log &optional; }; redef record State += { entity_bodies: count &default=0; }; } event http_entity_data(c: connection, is_orig: bool, length: count, data: string) &priority=5 { # Client body extraction is not currently supported in this script. if ( ! c$http$first_chunk ) return; if ( c$http$first_chunk ) { if ( is_orig && extract_requests || ! is_orig && (extract_responses || c$http?$mime_type && extract_file_types in c$http$mime_type) ) { c$http$extracting_file = T; local suffix = fmt("%s_%d.dat", is_orig ? "orig" : "resp", ++c$http_state$entity_bodies); local fname = generate_extraction_filename2(extraction_dir, extraction_prefix, c, suffix); if ( extraction_dir != "" ) mkdirs(path_dirname(fname)); if ( is_orig ) { c$http$request_file = open(fname); enable_raw_output(c$http$request_file); } else { c$http$response_file = open(fname); enable_raw_output(c$http$response_file); } } if ( c$http$extracting_file ) if ( is_orig ) print c$http$request_file, data; else print c$http$response_file, data; } } event http_end_entity(c: connection, is_orig: bool) { if ( c$http$extracting_file ) if ( is_orig && c$http?$request_file ) close(c$http$request_file); else if ( c$http?$response_file ) close(c$http$response_file); } From sychan at lbl.gov Wed Aug 17 10:42:31 2011 From: sychan at lbl.gov (Stephen Chan) Date: Wed, 17 Aug 2011 10:42:31 -0700 Subject: [Bro] Bro and Shibboleth Message-ID: Are there any Bro installations out there that have done any work with Shibboleth? We put Shibboleth into production here a couple of months back, and I'd like to start getting some basic Bro visibility into Shib traffic. The most basic level I wanted to get working was to generate Bro events when it looks like the Shibboleth server is being pounded on with dictionary attacks. Has anyone out there done any other Shib/Bro work? I'd like to get in touch and trade notes/code/brainstorming. Steve From mcholste at gmail.com Fri Aug 19 18:53:21 2011 From: mcholste at gmail.com (Martin Holste) Date: Fri, 19 Aug 2011 20:53:21 -0500 Subject: [Bro] Bro quickstart Message-ID: FYI, I've put up a very short quickstart on my blog (http://ossectools.blogspot.com/2011/08/monitoring-ssl-connections-with-bro.html) on getting Bro up and running for monitoring SSL connections. The write-up is for Ubuntu, but it should be helpful for anyone just starting with Bro. I will have follow-up posts on dealing with Bro output, but this should be enough to get anyone just starting out something to play with. Thanks again to Seth for all of the help! Thanks, Martin From edwardfjellskaal at gmail.com Sat Aug 20 10:16:51 2011 From: edwardfjellskaal at gmail.com (=?ISO-8859-1?Q?Edward_Fjellsk=E5l?=) Date: Sat, 20 Aug 2011 19:16:51 +0200 Subject: [Bro] Bro quickstart In-Reply-To: References: Message-ID: <4E4FEC03.2030402@gmail.com> On 08/20/2011 03:53 AM, Martin Holste wrote: > FYI, I've put up a very short quickstart on my blog > (http://ossectools.blogspot.com/2011/08/monitoring-ssl-connections-with-bro.html) > on getting Bro up and running for monitoring SSL connections. The > write-up is for Ubuntu, but it should be helpful for anyone just > starting with Bro. I will have follow-up posts on dealing with Bro > output, but this should be enough to get anyone just starting out > something to play with. Thanks again to Seth for all of the help! > > Thanks, > > Martin Nice, A sample output of we would expect to see in the "ssl.log" would be nice :) E From seth at icir.org Sat Aug 20 18:45:55 2011 From: seth at icir.org (Seth Hall) Date: Sat, 20 Aug 2011 21:45:55 -0400 Subject: [Bro] Bro quickstart In-Reply-To: <4E4FEC03.2030402@gmail.com> References: <4E4FEC03.2030402@gmail.com> Message-ID: On Aug 20, 2011, at 1:16 PM, Edward Fjellsk?l wrote: > A sample output of we would expect to see in the "ssl.log" would be nice :) I have most of the blog post written for that and even tried to publish it once, but the style sheet on the blog screwed it up. I'll try and post it again soon. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From mcholste at gmail.com Sat Aug 20 20:43:50 2011 From: mcholste at gmail.com (Martin Holste) Date: Sat, 20 Aug 2011 22:43:50 -0500 Subject: [Bro] Bro quickstart In-Reply-To: <4E4FEC03.2030402@gmail.com> References: <4E4FEC03.2030402@gmail.com> Message-ID: Indeed: I added a small sample of the ssl.log (wraps horribly, but the reader gets the gist) as well as a few other small edits based on info from Seth. If others have suggestions or tips for other platforms, etc., please let me know. I have no problem making this blog post into something of a wiki for the next few days! On Sat, Aug 20, 2011 at 12:16 PM, Edward Fjellsk?l wrote: > On 08/20/2011 03:53 AM, Martin Holste wrote: >> FYI, I've put up a very short quickstart on my blog >> (http://ossectools.blogspot.com/2011/08/monitoring-ssl-connections-with-bro.html) >> on getting Bro up and running for monitoring SSL connections. ?The >> write-up is for Ubuntu, but it should be helpful for anyone just >> starting with Bro. ?I will have follow-up posts on dealing with Bro >> output, but this should be enough to get anyone just starting out >> something to play with. ?Thanks again to Seth for all of the help! >> >> Thanks, >> >> Martin > > Nice, > > A sample output of we would expect to see in the "ssl.log" would be nice :) > > E > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > From jekrous at lbl.gov Tue Aug 23 12:31:59 2011 From: jekrous at lbl.gov (Jay Krous) Date: Tue, 23 Aug 2011 12:31:59 -0700 Subject: [Bro] open cyber security position Lawrence Berkeley National Lab Message-ID: Lawrence Berkeley National Lab has an immediate opening for a cyber security engineer. http:/go.lbl.gov/cyber-position A few reasons why it's awesome to work at Berkeley Lab. - Mission - Protect Science http://go.lbl.gov/mission (pdf) - Smart colleagues - You will teach and learn - Location - Famous California Bay Area weather, activities, and food - Work Environment - Science driven environment, less politics than usual - Benefits - Excellent benefits and retirement -Jay -- Jack (Jay) E. Krous III Cyber Security, Information Technology Division Lawrence Berkeley National Laboratory http://www.lbl.gov/cyber/pgp-krous.txt (510) 495-2522 From rodrigue.alahassa at gmail.com Sun Aug 28 05:06:16 2011 From: rodrigue.alahassa at gmail.com (Rodrigue ALAHASSA) Date: Sun, 28 Aug 2011 14:06:16 +0200 Subject: [Bro] Bro Signatures Message-ID: Hi, 1 - What's the difference between these type of signature ? What I'm trying to understand is when it could become handy to split the payload over many regular expressions. signature sid-542{ ip-proto = tcp payload /.* EHLO *. MAIL FROM *./ event sid-542 } signature sid-543{ ip-proto = tcp payload /*.EHLO*./ payload /*. MAIL FROM *./ event sid-543 } Is the order of appearance of signature attributes important for bro to trigger an alert ? Thanks for your help. -- SLt COC ALAHASSA 161 POL Professeur Georges LEMAITRE -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110828/60692a2f/attachment.html From mcholste at gmail.com Tue Aug 30 13:53:28 2011 From: mcholste at gmail.com (Martin Holste) Date: Tue, 30 Aug 2011 15:53:28 -0500 Subject: [Bro] Bro Web Frontend Message-ID: In order to really drill down on some SSL stuff, I decided it was time to finally put a lightweight web framework on top of my Bro logs. So, unless the server shoots it down, attached is a tarball that contains an alpha-quality framework for reading Bro logs and writing them to syslog or a database or both. If written to a database, they can be accessed through a very light web frontend included which can be run right from the Bro server, or from a traditional web server, if you prefer. The web frontend solely consists of a query bar for searching on terms contained in the tab separated Bro logs. So you can do things like this: subject:CN=mail.google.com* -validation_status:ok This would look for any connections to Gmail with an invalid certificate. Or you could do id.resp_h:74.125.* Which would find any traffic destined for Google's 74.125/16, or just 74.125.* which would do the same thing, but for either originator or responder. At this point, there's not much else you can do, as I mainly wanted to get a quick web frontend up so I could more easily explore the data that Bro outputs. The key part of the framework is that it doesn't care what the fields are. It will take whatever is at the top of the log files and use that as the field names. If it can't find the field list, then it won't use the file. The database doesn't have a standard table structure, rather it uses a key-value store, so the schema is very flexible. It's been tested on MySQL, but it should work on almost any database. It would be very easy to create entirely ad-hoc reporting using this structure. Aside from cosmetics, the big to-do is database table rollover, but if you just want syslog output from Bro, this will get you going very quickly. See the INSTALL doc for a quick how-to, or email if you have any questions; feedback is appreciated! If there is any interest, I will create a project page for the files for future downloading. Thanks, Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: broweb.tar.gz Type: application/x-gzip Size: 5824 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110830/eb778736/attachment.gz From gc355804 at ohio.edu Tue Aug 30 15:20:05 2011 From: gc355804 at ohio.edu (Gilbert Clark) Date: Tue, 30 Aug 2011 18:20:05 -0400 Subject: [Bro] Bro Web Frontend In-Reply-To: References: Message-ID: <4E5D6215.8000707@ohio.edu> Hi Martin: This is cool! A few notes: *) There's a ticket open (http://tracker.bro-ids.org/bro/ticket/558) which is looking at revising the ASCII log header (topic/gilbert/ascii-header is a branch containing what's described in that ticket) to offer more information about the log fields themselves, and to support better classification of log files after they've been rotated / compressed. The idea there is to use that type information to automate the table creation process and / or do some simple type conversion. *) topic/gilbert/log-util is (what sounds like, at least) a similar log wrapper I'm building in Python. The library's in a holding pattern while the new log header format gets pushed out, but should find itself in reasonable shape after that. If you're interested, I'd love to migrate over to bro-dev and chat about stuff that's useful to have in this kind of a library :) --Gilbert Clark On 8/30/2011 4:53 PM, Martin Holste wrote: > In order to really drill down on some SSL stuff, I decided it was time > to finally put a lightweight web framework on top of my Bro logs. So, > unless the server shoots it down, attached is a tarball that contains > an alpha-quality framework for reading Bro logs and writing them to > syslog or a database or both. If written to a database, they can be > accessed through a very light web frontend included which can be run > right from the Bro server, or from a traditional web server, if you > prefer. The web frontend solely consists of a query bar for searching > on terms contained in the tab separated Bro logs. So you can do > things like this: > > subject:CN=mail.google.com* -validation_status:ok > > This would look for any connections to Gmail with an invalid certificate. > > Or you could do > > id.resp_h:74.125.* > > Which would find any traffic destined for Google's 74.125/16, or just > 74.125.* > which would do the same thing, but for either originator or responder. > > At this point, there's not much else you can do, as I mainly wanted to > get a quick web frontend up so I could more easily explore the data > that Bro outputs. The key part of the framework is that it doesn't > care what the fields are. It will take whatever is at the top of the > log files and use that as the field names. If it can't find the field > list, then it won't use the file. The database doesn't have a > standard table structure, rather it uses a key-value store, so the > schema is very flexible. It's been tested on MySQL, but it should > work on almost any database. It would be very easy to create entirely > ad-hoc reporting using this structure. > > Aside from cosmetics, the big to-do is database table rollover, but if > you just want syslog output from Bro, this will get you going very > quickly. See the INSTALL doc for a quick how-to, or email if you have > any questions; feedback is appreciated! If there is any interest, I > will create a project page for the files for future downloading. > > Thanks, > > Martin