From robin at icir.org Tue Sep 6 08:10:11 2011 From: robin at icir.org (Robin Sommer) Date: Tue, 6 Sep 2011 08:10:11 -0700 Subject: [Bro] NIDS panel at RAID 2011 Message-ID: <20110906151011.GG88857@icir.org> Just wanted to send a note that there'll be a panel on open-source NIDS systems at RAID 2011 in Menlo Park, http://www.raid2011.org/panel.shtml, with representatives of the major systems (Snort, Suricata, Bro), moderated by Ron Gula of Tenable. Don't miss it! Members of the Bro team will be around during the whole conference. Also, Suricata is holding a community meeting right before RAID at the same location if you're interested in that as well. See you at RAID, Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From urbanski at vt.edu Wed Sep 7 06:21:22 2011 From: urbanski at vt.edu (Will Urbanski) Date: Wed, 07 Sep 2011 09:21:22 -0400 Subject: [Bro] Bro crashes regularly Message-ID: <4E676FD2.8020202@vt.edu> Hello! I am experiencing an issue where bro regularly crashes. A crash log is appended to this email. I am running a recent version of bro checked out of GIT last week. The OS is FreeBSD 8.2 x64. If there is no bug fix at this time, does anyone have a script that will check if bro has crashed and will restart in broctl? If the workers have crashed and I run 'check' in broctl it says that all workers and proxys are OK, but then if i do 'restart' it says that they have crashed. This might be another bug. Any info / tips would be greatly appreciated. Cheers, Will -------- Original Message -------- Subject: [Bro] Crash report from proxy-1 Date: Wed, 07 Sep 2011 09:17:51 -0400 (EDT) From: Big Brother To: bro.core Core was generated by `bro'. Program terminated with signal 11, Segmentation fault. #0 LogMgr::FindWriter (this=0x7cb3c0, writer=0x16cd260) at /usr/home/sensor/bro/src/LogMgr.cc:447 447 if ( winfo->writer == writer ) #0 LogMgr::FindWriter (this=0x7cb3c0, writer=0x16cd260) at /usr/home/sensor/bro/src/LogMgr.cc:447 #1 0x000000000053da1d in LogMgr::FinishedRotation (this=0x7cb3c0, writer=) at /usr/home/sensor/bro/src/LogMgr.cc:1508 #2 0x00000000005459e4 in LogWriter::FinishedRotation (this=0x16cd260, new_name=@0x7fffffffe190, old_name=) at /usr/home/sensor/bro/src/LogWriter.cc:157 #3 0x0000000000546fa0 in LogWriterAscii::DoRotate (this=0x16cd260, rotated_path=) at /usr/home/sensor/bro/src/LogWriterAscii.cc:257 #4 0x0000000000545d03 in LogWriter::Rotate (this=0x16cd260, rotated_path=) at /usr/home/sensor/bro/src/LogWriter.cc:95 #5 0x000000000053e3d8 in LogMgr::Rotate (this=) at /usr/home/sensor/bro/src/LogMgr.cc:1499 #6 0x000000000053ec6d in RotationTimer::Dispatch (this=0x16d13e8, t=) at /usr/home/sensor/bro/src/LogMgr.cc:1409 #7 0x00000000005b8720 in PQ_TimerMgr::DoAdvance (this=0x7c2968, new_t=1315368000.0016401, max_expire=300) at /usr/home/sensor/bro/src/Timer.cc:164 #8 0x0000000000556182 in expire_timers (src_ps=0x0) at /usr/home/sensor/bro/src/Net.cc:310 #9 0x00000000005568e9 in net_run () at /usr/home/sensor/bro/src/Net.cc:479 #10 0x00000000004688da in main (argc=22254392, argv=) at /usr/home/sensor/bro/src/main.cc:1011 ==== No reporter.log ==== stderr.log /usr/local/bro/share/broctl/scripts/run-bro: line 62: 9144 Segmentation fault: 11 (core dumped) nohup $mybro $@ ==== stdout.log unlimited 33554432 unlimited ==== .cmdline -U .status -p broctl -p broctl-live -p local -p proxy-1 broctl broctl/nodes/proxy local ==== .env_vars PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/home/sensor/bin BROPATH=/usr/local/bro/spool/policy/site:/usr/local/bro/spool/policy/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site CLUSTER_NODE=proxy-1 ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log -- [Automatically generated.] From seth at icir.org Wed Sep 7 06:45:21 2011 From: seth at icir.org (Seth Hall) Date: Wed, 7 Sep 2011 09:45:21 -0400 Subject: [Bro] Bro crashes regularly In-Reply-To: <4E676FD2.8020202@vt.edu> References: <4E676FD2.8020202@vt.edu> Message-ID: <92803996-4761-4E6F-98AC-B35317EFA4EC@icir.org> On Sep 7, 2011, at 9:21 AM, Will Urbanski wrote: > Hello! I am experiencing an issue where bro regularly crashes. A crash > log is appended to this email We have a ticket filed for that bug and I expect a fix in the next day or two. > If there is no bug fix at this time, does anyone have a script that will > check if bro has crashed and will restart in broctl? If the workers have > crashed and I run 'check' in broctl it says that all workers and proxys > are OK, but then if i do 'restart' it says that they have crashed. This > might be another bug. Oh sorry, BroControl should be managing this for you already. You just need to set a cron job that runs BroControl's "cron" command every 5 minutes or so. Here's what I usually use... */5 * * * * /bro/bin/broctl cron You just need to adjust the path based on where you installed broctl obviously. :) That will check to make sure everything is running and take care of a few other minor details as well. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Thu Sep 8 09:13:46 2011 From: robin at icir.org (Robin Sommer) Date: Thu, 8 Sep 2011 09:13:46 -0700 Subject: [Bro] Bro crashes regularly In-Reply-To: <4E676FD2.8020202@vt.edu> References: <4E676FD2.8020202@vt.edu> Message-ID: <20110908161346.GF71984@icir.org> On Wed, Sep 07, 2011 at 09:21 -0400, Will Urbanski wrote: > Hello! I am experiencing an issue where bro regularly crashes. A crash > log is appended to this email. There's a fix now in master that seems to fix it. Could you try? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From iduckhd at hotmail.com Tue Sep 13 08:57:57 2011 From: iduckhd at hotmail.com (Ioannis WiCom) Date: Tue, 13 Sep 2011 15:57:57 +0000 Subject: [Bro] HTTP Object length calculation Message-ID: Hello, I am trying to use Bro 1.5..1 to calculate the HTTP object length from a test packet trace. I have observed that in several HTTP transactions the calculated object length (stat$body_length) is higher than the "Content-Length" (msg$content_length) r For example: GET /tools/services?XXX (200 "OK" ["1945 ", 11182]) I have isolated an example TCP connection, and measured the bytes using wireshark. The real object length is equal to the "Content-Length", but the reported by bro is much higher. Therefore, I cannot understand what the value stat$body_length represents. Any help would be highly appreciated. Thank you, Yannis -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110913/f27bbc2f/attachment.html From seth at icir.org Tue Sep 13 10:05:44 2011 From: seth at icir.org (Seth Hall) Date: Tue, 13 Sep 2011 13:05:44 -0400 Subject: [Bro] HTTP Object length calculation In-Reply-To: References: Message-ID: <28C59012-A02C-407C-8FBA-26857EE534C1@icir.org> On Sep 13, 2011, at 11:57 AM, Ioannis WiCom wrote: > I have isolated an example TCP connection, and measured the bytes using wireshark. The real object length is equal to the "Content-Length", but the reported by bro is much higher. Therefore, I cannot understand what the value stat$body_length represents. stat$body_length *should* be the actual counted number of bytes that were in the body. If you see a disparity between the two numbers, the web server could be reporting an incorrect length for the data it's sending. Could you send the trace file privately? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From gregor at icir.org Tue Sep 13 13:12:50 2011 From: gregor at icir.org (Gregor Maier) Date: Tue, 13 Sep 2011 13:12:50 -0700 Subject: [Bro] HTTP Object length calculation In-Reply-To: <28C59012-A02C-407C-8FBA-26857EE534C1@icir.org> References: <28C59012-A02C-407C-8FBA-26857EE534C1@icir.org> Message-ID: <4E6FB942.7060505@icir.org> On 9/13/11 10:05 , Seth Hall wrote: > > On Sep 13, 2011, at 11:57 AM, Ioannis WiCom wrote: > >> I have isolated an example TCP connection, and measured the bytes using wireshark. The real object length is equal to the "Content-Length", but the reported by bro is much higher. Therefore, I cannot understand what the value stat$body_length represents. > > stat$body_length *should* be the actual counted number of bytes that were in the body. If you see a disparity between the two numbers, the web server could be reporting an incorrect length for the data it's sending. Could you send the trace file privately? Actually that's not exactly the case. Bro reports the body length *after decompression* (for transfer-encodings that use compressions). In addition, the Content-Length header is often unreliable. E.g., if an HTTP transfer is interrupted fewer bytes are transferred that reported by Content-Length. This can happen often with (misconfigured) download managers. Or the Content-Length header can also be just plain wrong (HTTP server sends garbage). We did a study with residential traffic and found that the Content-Length header will on average over-report the volume by a factor of about 5 (with some spikes reaching several 100(!)) cu Gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From iduckhd at hotmail.com Tue Sep 13 16:17:41 2011 From: iduckhd at hotmail.com (Ioannis WiCom) Date: Tue, 13 Sep 2011 23:17:41 +0000 Subject: [Bro] HTTP Object length calculation In-Reply-To: <4E6FB942.7060505@icir.org> References: <28C59012-A02C-407C-8FBA-26857EE534C1@icir.org>, <4E6FB942.7060505@icir.org> Message-ID: Thank you both for getting back to me. I understand your point regarding the content-length. However, it looks to me that there is a contradiction in the calculation. When the host receives the full object and it is encoded, bro reports the size of the object in the server (before compression), not the actual bytes in the network (after compression). When the object is partially downloaded (object_length Date: Tue, 13 Sep 2011 13:12:50 -0700 > From: gregor at icir.org > To: seth at icir.org > CC: iduckhd at hotmail.com; bro at bro-ids.org > Subject: Re: [Bro] HTTP Object length calculation > > On 9/13/11 10:05 , Seth Hall wrote: > > > > On Sep 13, 2011, at 11:57 AM, Ioannis WiCom wrote: > > > >> I have isolated an example TCP connection, and measured the bytes using wireshark. The real object length is equal to the "Content-Length", but the reported by bro is much higher. Therefore, I cannot understand what the value stat$body_length represents. > > > > stat$body_length *should* be the actual counted number of bytes that were in the body. If you see a disparity between the two numbers, the web server could be reporting an incorrect length for the data it's sending. Could you send the trace file privately? > > Actually that's not exactly the case. Bro reports the body length *after > decompression* (for transfer-encodings that use compressions). > > In addition, the Content-Length header is often unreliable. E.g., if an > HTTP transfer is interrupted fewer bytes are transferred that reported > by Content-Length. This can happen often with (misconfigured) download > managers. Or the Content-Length header can also be just plain wrong > (HTTP server sends garbage). We did a study with residential traffic and > found that the Content-Length header will on average over-report the > volume by a factor of about 5 (with some spikes reaching several 100(!)) > > cu > Gregor > -- > Gregor Maier > > Int. Computer Science Institute (ICSI) > 1947 Center St., Ste. 600 > Berkeley, CA 94704, USA > http://www.icir.org/gregor/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110913/d1a30f7a/attachment.html From gregor at icir.org Wed Sep 14 17:29:21 2011 From: gregor at icir.org (Gregor Maier) Date: Wed, 14 Sep 2011 17:29:21 -0700 Subject: [Bro] HTTP Object length calculation In-Reply-To: References: <28C59012-A02C-407C-8FBA-26857EE534C1@icir.org>, <4E6FB942.7060505@icir.org> Message-ID: <4E7146E1.30801@icir.org> On 9/13/11 16:17 , Ioannis WiCom wrote: > When the host receives the full object and it is encoded, bro reports > the size of the object in the server (before compression), not the > actual bytes in the network (after compression). Correct. > When the object is partially downloaded (object_length and assuming it was encoded, bro reports the actual bytes in the network > (since partial decompression cannot be performed). No, the compression schemes can be done on the fly. I.e., decompression can start before the whole object is downloaded. That's what Bro is doing. So if an encoded transfer is interrupted, Bro will report the amount of bytes it has decompressed so far. > Is there a way to get the actual bytes transferred? Unfortunately not. cu gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From seth at icir.org Fri Sep 16 10:14:32 2011 From: seth at icir.org (Seth Hall) Date: Fri, 16 Sep 2011 13:14:32 -0400 Subject: [Bro] Workshop attendence Message-ID: <2E733B13-A623-4DD0-8DDF-AAD7B231B6D7@icir.org> Hi all, As a warning, the workshop is nearly full. If anyone is still working on getting approval please, please, please let me know! I can reserve a spot for you if you let me know off list. If you don't and the workshop fills completely we will be unable to make new seats available. http://www.bro-ids.org/community/workshop2011.html Thanks! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From kristen.eisenberg at yahoo.com Sat Sep 17 11:37:38 2011 From: kristen.eisenberg at yahoo.com (Kristen Eisenberg) Date: Sat, 17 Sep 2011 11:37:38 -0700 (PDT) Subject: [Bro] (no subject) Message-ID: <1316284658.1761.YahooMailNeo@web122306.mail.ne1.yahoo.com> [Bro] Signature payload matching Hi all, I'm working for automation of signature generation for Bro from pcap trace files. I would like to know if the matching of the payload as a condition is done against all the session data or more like per packet matching. Thanks ? Kristen Eisenberg Billige Fl?ge Marketing GmbH Emanuelstr. 3, 10317 Berlin Deutschland Telefon: +49 (33) 5310967 Email: utebachmeier at gmail.com Site: http://flug.airego.de- Billige Fl?ge vergleichen -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110917/7d4f9263/attachment.html From rodrigue.alahassa at gmail.com Sat Sep 17 12:22:30 2011 From: rodrigue.alahassa at gmail.com (Rodrigue ALAHASSA) Date: Sat, 17 Sep 2011 21:22:30 +0200 Subject: [Bro] Signature payload matching Message-ID: The matching is done against all the session for TCP sessions. For UDP sessions, it is much like a per packet matching. R. Alahassa On Sat, Sep 17, 2011 at 9:00 PM, wrote: > Send Bro mailing list submissions to > bro at bro-ids.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > or, via email, send a message with subject or body 'help' to > bro-request at bro-ids.org > > You can reach the person managing the list at > bro-owner at bro-ids.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Bro digest..." > > Today's Topics: > > 1. (no subject) (Kristen Eisenberg) > > > ---------- Forwarded message ---------- > From: Kristen Eisenberg > To: "bro at bro-ids.org" > Date: Sat, 17 Sep 2011 11:37:38 -0700 (PDT) > Subject: [Bro] (no subject) > [Bro] Signature payload matching > > Hi all, > > I'm working for automation of signature generation for Bro from pcap trace > files. > I would like to know if the matching of the payload as a condition is done > against all the session data or more like per packet matching. > > Thanks > > ** ** > Kristen Eisenberg > Billige Fl?ge > Marketing GmbH > Emanuelstr. 3, > 10317 Berlin > Deutschland > Telefon: +49 (33) > 5310967 > Email: > utebachmeier at gmail.com > Site: > http://flug.airego.de > - Billige Fl?ge vergleichen > > _______________________________________________ > Bro mailing list > Bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -- SLt COC ALAHASSA 161 POL Professeur Georges LEMAITRE -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110917/5819a19f/attachment.html From marco-oweber at gmx.de Sat Sep 17 13:00:24 2011 From: marco-oweber at gmx.de (Marc Weber) Date: Sat, 17 Sep 2011 22:00:24 +0200 Subject: [Bro] Looking for a tool detecting abusing IPs Message-ID: <1316289083-sup-8384@nixos> Hi mailinglist, The company I'm working for has been attacked by SYN and ddos within the last three weeks. Now we'd like to optimize our setup so that we can cope with most common attacks with minimal resources. To do so we want to block IPs abusing our server eg by requesting too many page views or sending SYN attacks (if the source IP has not been spoofed) etc. Is bro-ids the right tool to do so? If not which alternative would you recommend? Is there someone who would be interested in providing payed support? Which documentation about pro-ids should I read first? I'm little bit lost cause the wiki says most of its information is outdated. We only have to protect the very common services - HTTP - POP3 - SMTP Thanks for any guidance. Of course I could use libpcap and code up a tool myself. However I hope that with your knowledge we're up and running much faster. I know about fail2ban - however I'd prefer something not requiring huge logfiles.. Marc Weber From robin at icir.org Sun Sep 18 21:05:42 2011 From: robin at icir.org (Robin Sommer) Date: Sun, 18 Sep 2011 21:05:42 -0700 Subject: [Bro] (no subject) In-Reply-To: <1316284658.1761.YahooMailNeo@web122306.mail.ne1.yahoo.com> References: <1316284658.1761.YahooMailNeo@web122306.mail.ne1.yahoo.com> Message-ID: <20110919040542.GK97650@icir.org> On Sat, Sep 17, 2011 at 11:37 -0700, you wrote: > I would like to know if the matching of the payload as a condition is done > against all the session data or more like per packet matching. For TCP it's all session data. Also see: http://www.bro-ids.org/documentation/signatures.html Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From rodrigue.alahassa at gmail.com Mon Sep 19 07:53:56 2011 From: rodrigue.alahassa at gmail.com (Rodrigue ALAHASSA) Date: Mon, 19 Sep 2011 14:53:56 +0000 Subject: [Bro] Looking for a tool detecting abusing IPs Message-ID: Hi, Bro can cope with your requirements. You just need to write a policy script to handle that. You declare global variables to count connexion attemps to your servers. You should take a look at the policy script "signatures.bro". There is an example to detect vertical and horizontal scans. Hope it's helpful. R. Alahassa On Sun, Sep 18, 2011 at 7:00 PM, wrote: > Send Bro mailing list submissions to > bro at bro-ids.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > or, via email, send a message with subject or body 'help' to > bro-request at bro-ids.org > > You can reach the person managing the list at > bro-owner at bro-ids.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Bro digest..." > > Today's Topics: > > 1. Re: Signature payload matching (Rodrigue ALAHASSA) > 2. Looking for a tool detecting abusing IPs (Marc Weber) > > > ---------- Forwarded message ---------- > From: Rodrigue ALAHASSA > To: bro at bro-ids.org > Date: Sat, 17 Sep 2011 21:22:30 +0200 > Subject: Re: [Bro] Signature payload matching > The matching is done against all the session for TCP sessions. > For UDP sessions, it is much like a per packet matching. > > R. Alahassa > > On Sat, Sep 17, 2011 at 9:00 PM, wrote: > >> Send Bro mailing list submissions to >> bro at bro-ids.org >> >> To subscribe or unsubscribe via the World Wide Web, visit >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> or, via email, send a message with subject or body 'help' to >> bro-request at bro-ids.org >> >> You can reach the person managing the list at >> bro-owner at bro-ids.org >> >> When replying, please edit your Subject line so it is more specific >> than "Re: Contents of Bro digest..." >> >> Today's Topics: >> >> 1. (no subject) (Kristen Eisenberg) >> >> >> ---------- Forwarded message ---------- >> From: Kristen Eisenberg >> To: "bro at bro-ids.org" >> Date: Sat, 17 Sep 2011 11:37:38 -0700 (PDT) >> Subject: [Bro] (no subject) >> [Bro] Signature payload matching >> >> Hi all, >> >> I'm working for automation of signature generation for Bro from pcap trace >> files. >> I would like to know if the matching of the payload as a condition is done >> against all the session data or more like per packet matching. >> >> Thanks >> >> ** ** >> Kristen Eisenberg >> Billige Fl?ge >> Marketing GmbH >> Emanuelstr. 3, >> 10317 Berlin >> Deutschland >> Telefon: +49 (33) >> 5310967 >> Email: >> utebachmeier at gmail.com >> Site: >> http://flug.airego.de >> - Billige Fl?ge vergleichen >> >> _______________________________________________ >> Bro mailing list >> Bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> > > > -- > SLt COC ALAHASSA > 161 POL > Professeur Georges LEMAITRE > > > ---------- Forwarded message ---------- > From: Marc Weber > To: bro > Date: Sat, 17 Sep 2011 22:00:24 +0200 > Subject: [Bro] Looking for a tool detecting abusing IPs > Hi mailinglist, > > The company I'm working for has been attacked by SYN and ddos within the > last three weeks. > > Now we'd like to optimize our setup so that we can cope with most common > attacks with minimal resources. To do so we want to block IPs abusing > our server eg by requesting too many page views or sending SYN attacks > (if the source IP has not been spoofed) etc. > > Is bro-ids the right tool to do so? > If not which alternative would you recommend? > > Is there someone who would be interested in providing payed support? > > Which documentation about pro-ids should I read first? > I'm little bit lost cause the wiki says most of its information is > outdated. > > We only have to protect the very common services > - HTTP > - POP3 > - SMTP > > Thanks for any guidance. > > Of course I could use libpcap and code up a tool myself. > However I hope that with your knowledge we're up and running much > faster. > > I know about fail2ban - however I'd prefer something not requiring huge > logfiles.. > > Marc Weber > > > _______________________________________________ > Bro mailing list > Bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -- SLt COC ALAHASSA 161 POL Professeur Georges LEMAITRE -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110919/49fb239e/attachment.html From robin at icir.org Mon Sep 19 08:38:04 2011 From: robin at icir.org (Robin Sommer) Date: Mon, 19 Sep 2011 08:38:04 -0700 Subject: [Bro] Looking for a tool detecting abusing IPs In-Reply-To: <1316289083-sup-8384@nixos> References: <1316289083-sup-8384@nixos> Message-ID: <20110919153804.GD35703@icir.org> On Sat, Sep 17, 2011 at 22:00 +0200, you wrote: > Now we'd like to optimize our setup so that we can cope with most common > attacks with minimal resources. To do so we want to block IPs abusing > our server eg by requesting too many page views or sending SYN attacks > (if the source IP has not been spoofed) etc. Yes, Bro is an excellent tool for such things. There's the default scan.bro script which reports TCP scans (also UDP if udp.bro is loaded; and icmp.bro can find ICMP scans). Generally, it's pretty straight-forward to add custom logic for fine-tuning reporting or finding other types of scans. We're also in the process of adding a new Metrics framework that generalizes "counting stuff", and it will be able reports scans of all kinds. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Mon Sep 19 08:50:16 2011 From: robin at icir.org (Robin Sommer) Date: Mon, 19 Sep 2011 08:50:16 -0700 Subject: [Bro] Bro Tutorial at ACSAC Message-ID: <20110919155016.GF35703@icir.org> For folks who can't make the Bro Workshop at NCSA in November, we're happy to announce a one-day Bro tutorial at ACSAC 2011 in Orlando, FL, on December 6. For more information, see: http://www.acsac.org/2011/program/courses/view.php?t=8 (Note that this is part of the ACSAC's "Professional Development Courses" and requires separate registration.) Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From wathsala at opensource.lk Tue Sep 27 04:11:46 2011 From: wathsala at opensource.lk (Wathsala Vithanage) Date: Tue, 27 Sep 2011 16:41:46 +0530 Subject: [Bro] Can Bro Anonymize the Data it Captures? Message-ID: Hi, Is Bro capable of anonymizing the logs it generate? Thanks. From mcholste at gmail.com Tue Sep 27 08:17:42 2011 From: mcholste at gmail.com (Martin Holste) Date: Tue, 27 Sep 2011 10:17:42 -0500 Subject: [Bro] Quickstart for Bro Cluster Message-ID: I'm cross-posting this because I think Bro is a very helpful supplement to anyone running an IDS, and it sounded like that was pretty much the consensus at RAID 2011. If you're looking to get Bro up and running as a proof-of-concept, check out my first post on it here: http://ossectools.blogspot.com/2011/08/monitoring-ssl-connections-with-bro.html. If you want it to scale up to a large pipe (anything over 80 Mb/sec), check out my new post on Bro cluster (http://ossectools.blogspot.com/2011/09/bro-quickstart-cluster-edition.html) which will show how to set it up to take advantage of a multi-core system and forward its logs to an SIEM or central syslog. If you're not currently using Bro and are wondering why you should bother, consider that Bro provides a great way to survey the SSL traffic that's on your network, and a lot of malware uses SSL for command-and-control channels. It's a terrific way of seeing what email and attachments are being transferred, which can help you spot suspicious attachments, phishing, etc. In addition, it will record the MD5 and URL of every executable downloaded, which can be a real help during incident response. It has many more features (like being able to receive Snort alerts), but these are just some of the immediate benefits you get from running it alongside your current IDS. Thanks, Martin From robin at icir.org Tue Sep 27 08:49:03 2011 From: robin at icir.org (Robin Sommer) Date: Tue, 27 Sep 2011 08:49:03 -0700 Subject: [Bro] Can Bro Anonymize the Data it Captures? In-Reply-To: References: Message-ID: <20110927154903.GJ72342@icir.org> On Tue, Sep 27, 2011 at 16:41 +0530, Wathsala Vithanage wrote: > Is Bro capable of anonymizing the logs it generate? Kind of. Bro 1.5 comes with the anonymization systems described in this paper: http://conferences.sigcomm.org/sigcomm/2003/papers/p339-pang.pdf This is very cool stuff. However, the code hasn't been maintained for a long time already and, due to bit rot, there are various pieces here and there that aren't working right anymore. For the upcoming release, we have thus completely removed that functionality. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From wathsala at opensource.lk Tue Sep 27 09:55:49 2011 From: wathsala at opensource.lk (Wathsala Vithanage) Date: Tue, 27 Sep 2011 22:25:49 +0530 Subject: [Bro] Can Bro Anonymize the Data it Captures? In-Reply-To: <20110927154903.GJ72342@icir.org> References: <20110927154903.GJ72342@icir.org> Message-ID: Thanks Robin, > Kind of. Bro 1.5 comes with the anonymization systems described in > this paper: I have already installed Bro version 2.0. I'm happy to revert back to version 1.5 if it has the logging framework that will be available in version 2.0. > This is very cool stuff. However, the code hasn't been maintained for > a long time already and, due to bit rot, there are various pieces here > and there that aren't working right anymore. Cool stuff indeed! Can you provide me few pointers to the relevant code? Thanks From gregor at icir.org Tue Sep 27 10:55:04 2011 From: gregor at icir.org (Gregor Maier) Date: Tue, 27 Sep 2011 10:55:04 -0700 Subject: [Bro] Can Bro Anonymize the Data it Captures? In-Reply-To: <20110927154903.GJ72342@icir.org> References: <20110927154903.GJ72342@icir.org> Message-ID: <4E820DF8.3060807@icir.org> > Kind of. Bro 1.5 comes with the anonymization systems described in > this paper: > > http://conferences.sigcomm.org/sigcomm/2003/papers/p339-pang.pdf > > This is very cool stuff. However, the code hasn't been maintained for > a long time already and, due to bit rot, there are various pieces here > and there that aren't working right anymore. For the upcoming release, > we have thus completely removed that functionality. Hmm. I'm actually wondering whether all the flexibility of the new logging framework would enable us to anonymize log files as a transparent add-on on the script layer..... I guess that in any case one could always modify / anonymize the c$PROTOCOL record just before it gets logged.... cu Gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From seth at icir.org Tue Sep 27 11:00:18 2011 From: seth at icir.org (Seth Hall) Date: Tue, 27 Sep 2011 14:00:18 -0400 Subject: [Bro] Quickstart for Bro Cluster In-Reply-To: References: Message-ID: On Sep 27, 2011, at 11:17 AM, Martin Holste wrote: > I'm cross-posting this because I think Bro is a very helpful > supplement to anyone running an IDS, and it sounded like that was > pretty much the consensus at RAID 2011. Thanks for writing that quickstart guide Martin! :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From carlopmart at gmail.com Wed Sep 28 10:32:15 2011 From: carlopmart at gmail.com (carlopmart) Date: Wed, 28 Sep 2011 19:32:15 +0200 Subject: [Bro] Integrating bro-ids on sguil or snorby Message-ID: <4E835A1F.7050000@gmail.com> Hi all, Sorry if this question sounds stupid, but I am very new using bro as an IDS. Is it possible to integrate bro logs on sguil or snorby or some type of front-ends like these ones?? Thanks. -- CL Martinez carlopmart {at} gmail {d0t} com From seth at icir.org Wed Sep 28 10:42:13 2011 From: seth at icir.org (Seth Hall) Date: Wed, 28 Sep 2011 13:42:13 -0400 Subject: [Bro] Integrating bro-ids on sguil or snorby In-Reply-To: <4E835A1F.7050000@gmail.com> References: <4E835A1F.7050000@gmail.com> Message-ID: <41EC3D1E-64B0-4DB7-935B-4E0EF26E0512@icir.org> On Sep 28, 2011, at 1:32 PM, carlopmart wrote: > Sorry if this question sounds stupid, but I am very new using bro as > an IDS. Is it possible to integrate bro logs on sguil or snorby or some > type of front-ends like these ones?? I'm going to say no with the caveat that we will almost certainly have some sort of integration with those in the future. If you only look at the output of Bro in those interfaces though, you'd currently be missing out on much of the benefit since Bro does extensive protocol logging. Are you running from our repository or a released version? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From carlopmart at gmail.com Wed Sep 28 10:43:27 2011 From: carlopmart at gmail.com (carlopmart) Date: Wed, 28 Sep 2011 19:43:27 +0200 Subject: [Bro] Integrating bro-ids on sguil or snorby In-Reply-To: <41EC3D1E-64B0-4DB7-935B-4E0EF26E0512@icir.org> References: <4E835A1F.7050000@gmail.com> <41EC3D1E-64B0-4DB7-935B-4E0EF26E0512@icir.org> Message-ID: <4E835CBF.1020201@gmail.com> On 09/28/2011 07:42 PM, Seth Hall wrote: > > On Sep 28, 2011, at 1:32 PM, carlopmart wrote: > >> Sorry if this question sounds stupid, but I am very new using bro as >> an IDS. Is it possible to integrate bro logs on sguil or snorby or some >> type of front-ends like these ones?? > > > I'm going to say no with the caveat that we will almost certainly have some sort of integration with those in the future. > > If you only look at the output of Bro in those interfaces though, you'd currently be missing out on much of the benefit since Bro does extensive protocol logging. Are you running from our repository or a released version? > > .Seth I have installed released version: 1.5.3 ... -- CL Martinez carlopmart {at} gmail {d0t} com From mcholste at gmail.com Wed Sep 28 11:04:06 2011 From: mcholste at gmail.com (Martin Holste) Date: Wed, 28 Sep 2011 13:04:06 -0500 Subject: [Bro] Integrating bro-ids on sguil or snorby In-Reply-To: <4E835CBF.1020201@gmail.com> References: <4E835A1F.7050000@gmail.com> <41EC3D1E-64B0-4DB7-935B-4E0EF26E0512@icir.org> <4E835CBF.1020201@gmail.com> Message-ID: >>> ? Sorry if this question sounds stupid, but I am very new using bro as >>> an IDS. Is it possible to integrate bro logs on sguil or snorby or some >>> type of front-ends like these ones?? As Seth pointed out, Bro's not really the same kind of "alerting" device that Snort is, so it doesn't fit the SIEM mold very well. However, if you have a log management solution, you can forward your Bro logs into it. If you look at the email I sent the list two days ago regarding the Bro cluster quickstart, you'll see a link to my osssectools.blogspot.com post where I show how to forward Bro logs using rsyslog. A similar setup could be achieved with syslog-ng if that's already on the box. Hopefully your log management solution will let you explore Bro's outputs a bit better and provide some alerting capabilties. Otherwise, don't ever be afraid to plow into the logs with grep and sort. From roger.larsen at hig.no Wed Sep 28 13:41:17 2011 From: roger.larsen at hig.no (=?iso-8859-1?Q?Roger_Larsen_-_H=F8gskolen_i_Gj=F8vik?=) Date: Wed, 28 Sep 2011 22:41:17 +0200 Subject: [Bro] Bro policy script language documentation Message-ID: <003701cc7e1e$f94d1980$ebe74c80$@hig.no> Dear Bro Team/Community, I am studying information security in Gj?vik University College (www.hig.no), master degree. Present I am writing an article about Bro. In this case I struggle in finding detailed documentation regarding The Bro Policy Script Language. Can You please help me in this matter? Thanks! Best Regards, Roger Larsen Network manager & student J -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110928/b4c739de/attachment.html From marcos.e.rodriguez at gmail.com Wed Sep 28 14:14:08 2011 From: marcos.e.rodriguez at gmail.com (Marcos Rodriguez) Date: Wed, 28 Sep 2011 17:14:08 -0400 Subject: [Bro] Bro policy script language documentation In-Reply-To: <003701cc7e1e$f94d1980$ebe74c80$@hig.no> References: <003701cc7e1e$f94d1980$ebe74c80$@hig.no> Message-ID: 2011/9/28 Roger Larsen - H?gskolen i Gj?vik > Dear Bro Team/Community,**** > > ** ** > > I am studying information security in Gj?vik University College ( > www.hig.no), master degree.**** > > Present I am writing an article about Bro. In this case I struggle in > finding detailed documentation regarding The Bro Policy Script Language.** > ** > > Can You please help me in this matter?**** > > ** ** > > Thanks!**** > > ** ** > > Best Regards,**** > > ** ** > > ** ** > > Roger Larsen**** > > Network manager & student J**** > > ** ** > > ** ** > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > Hi Roger, The Bro team is overhauling their documentation, but all is not lost! My first suggestion would be to check out their workshop here: http://www-old.bro-ids.org/bro-workshop-2009-2/agenda.html Also, the documentation is included in the 1.5.3 tarball, however, the docs are dated to about 2004. http://www-old.bro-ids.org has a wiki with more updated docs (2007, I believe). Also, get a feel for the scripts included with the tarball, as they are very illuminating. They are the *.bro files in the /policy directory after you've extracted the tarball. Bro is very powerful from what little I've seen so far. I'm a Snort and Suricata guy, and just recently read Vern Paxson's, et al, *"Robust TCP Reassembly in the Presence of Adversaries" *paper and had to dive into Bro. Martin Holste is a frequent poster here, and has actually written some nice posts on his blog regarding Bro setup and clustering. Check it out here: http://ossectools.blogspot.com/ Hope this helps! marcos -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20110928/f241b855/attachment.html From carlopmart at gmail.com Thu Sep 29 01:44:52 2011 From: carlopmart at gmail.com (carlopmart) Date: Thu, 29 Sep 2011 10:44:52 +0200 Subject: [Bro] Integrating bro-ids on sguil or snorby In-Reply-To: References: <4E835A1F.7050000@gmail.com> <41EC3D1E-64B0-4DB7-935B-4E0EF26E0512@icir.org> <4E835CBF.1020201@gmail.com> Message-ID: <4E843004.1040902@gmail.com> On 09/28/2011 08:04 PM, Martin Holste wrote: >>>> Sorry if this question sounds stupid, but I am very new using bro as >>>> an IDS. Is it possible to integrate bro logs on sguil or snorby or some >>>> type of front-ends like these ones?? > > As Seth pointed out, Bro's not really the same kind of "alerting" > device that Snort is, so it doesn't fit the SIEM mold very well. > However, if you have a log management solution, you can forward your > Bro logs into it. If you look at the email I sent the list two days > ago regarding the Bro cluster quickstart, you'll see a link to my > osssectools.blogspot.com post where I show how to forward Bro logs > using rsyslog. A similar setup could be achieved with syslog-ng if > that's already on the box. Hopefully your log management solution > will let you explore Bro's outputs a bit better and provide some > alerting capabilties. Otherwise, don't ever be afraid to plow into > the logs with grep and sort. Thansk Martin. I have do it some google searches about this, and i think the best option is to use an OSSEC agent, and then from the OSSEC server forward all logs to a splunk server to collect statistics, etc. Another option is to use rsyslog like appears in your blog's post like this: rsyslog -> ossec_agent -> ossec_server -> splunk What is your opinion?? -- CL Martinez carlopmart {at} gmail {d0t} com From seth at icir.org Thu Sep 29 04:11:58 2011 From: seth at icir.org (Seth Hall) Date: Thu, 29 Sep 2011 07:11:58 -0400 Subject: [Bro] Integrating bro-ids on sguil or snorby In-Reply-To: <4E843004.1040902@gmail.com> References: <4E835A1F.7050000@gmail.com> <41EC3D1E-64B0-4DB7-935B-4E0EF26E0512@icir.org> <4E835CBF.1020201@gmail.com> <4E843004.1040902@gmail.com> Message-ID: <5CF17279-1CA6-4A3D-83D1-F5485F8A0999@icir.org> On Sep 29, 2011, at 4:44 AM, carlopmart wrote: > rsyslog -> ossec_agent -> ossec_server -> splunk > > What is your opinion?? I want to support splunk as a direct output for Bro eventually, we already have some users that are very successfully using that model. With my "ext" scripts available from http://www.github.com/sethhall/bro_scripts people have been using the Splunk forwarder to the send those logs directly to splunk which automatically does field extraction and it correctly recognizes the epoch time timestamps at the beginning of the lines as what they are. We are planning on doing closer integration with OSSEC once we figure out what that means, but what does that 4 step pipeline gain you over just using the splunk forwarder directly? BTW, I don't really recommend people begin using my "ext" scripts at this point. We're going to be doing a new release very soon and all of the scripts have incorporated "lessons learned" from my experience in writing the ext scripts. It makes more sense to follow our in-development quickstart guide[1] and Martin's recent blog post[2]. 1. http://www.bro-ids.org/documentation/quickstart.html#compiling-bro-source-code 2. http://ossectools.blogspot.com/2011/09/bro-quickstart-cluster-edition.html .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From lruppert at syr.edu Thu Sep 29 05:14:02 2011 From: lruppert at syr.edu (Louis F Ruppert) Date: Thu, 29 Sep 2011 12:14:02 +0000 Subject: [Bro] Integrating bro-ids on sguil or snorby In-Reply-To: <5CF17279-1CA6-4A3D-83D1-F5485F8A0999@icir.org> References: <4E835A1F.7050000@gmail.com> <41EC3D1E-64B0-4DB7-935B-4E0EF26E0512@icir.org> <4E835CBF.1020201@gmail.com> <4E843004.1040902@gmail.com>, <5CF17279-1CA6-4A3D-83D1-F5485F8A0999@icir.org> Message-ID: <18FF0D6321F4A24D901F7651020E931904CAFC@SUEX10-mbx-05.ad.syr.edu> Another option is to use the syslog output and pipe that into prelude-ids/prewikka for handling. I've done that with 1.5.x using its native syslog output. I've been experimenting with doing the same with the development version. Prelude is nice because it's a fairly distributed model, and it encrypts traffic from sensors to manager/display. And its development bits are Python, so there's potential for much tighter integration. Distributed is nice too, if you tend to move back and forth between using multiple specialized bro clusters and the one all-uniting fearsome bro mega-cluster. The model I tend to use looks like this: bro->syslog->prelude-ids But picture it with HIDS output, commercial IDS output, and other syslog output all correlated by IP in my diagram. -- Lou Ruppert Intrusion Analyst, GCFA Information Security Syracuse University ________________________________________ From: bro-bounces at bro-ids.org [bro-bounces at bro-ids.org] on behalf of Seth Hall [seth at icir.org] Sent: Thursday, September 29, 2011 7:11 AM To: carlopmart Cc: bro at bro-ids.org Subject: Re: [Bro] Integrating bro-ids on sguil or snorby On Sep 29, 2011, at 4:44 AM, carlopmart wrote: > rsyslog -> ossec_agent -> ossec_server -> splunk > > What is your opinion?? I want to support splunk as a direct output for Bro eventually, we already have some users that are very successfully using that model. With my "ext" scripts available from http://www.github.com/sethhall/bro_scripts people have been using the Splunk forwarder to the send those logs directly to splunk which automatically does field extraction and it correctly recognizes the epoch time timestamps at the beginning of the lines as what they are. We are planning on doing closer integration with OSSEC once we figure out what that means, but what does that 4 step pipeline gain you over just using the splunk forwarder directly? BTW, I don't really recommend people begin using my "ext" scripts at this point. We're going to be doing a new release very soon and all of the scripts have incorporated "lessons learned" from my experience in writing the ext scripts. It makes more sense to follow our in-development quickstart guide[1] and Martin's recent blog post[2]. 1. http://www.bro-ids.org/documentation/quickstart.html#compiling-bro-source-code 2. http://ossectools.blogspot.com/2011/09/bro-quickstart-cluster-edition.html .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro