From seth at icir.org Tue Apr 1 11:29:50 2014 From: seth at icir.org (Seth Hall) Date: Tue, 1 Apr 2014 14:29:50 -0400 Subject: [Bro] SMTP entities log doesn't appears In-Reply-To: References: <2e9247d7e6fc0ae4af533108a73689a4@localhost> Message-ID: On Mar 28, 2014, at 3:03 AM, C. L. Martinez wrote: > Any more ideas please?? What version of Bro are you running? (2.1 I suppose?) Also, are you positive that your script is being loaded by workers? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140401/9ec73f21/attachment.bin From jlay at slave-tothe-box.net Tue Apr 1 16:28:04 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Tue, 01 Apr 2014 17:28:04 -0600 Subject: [Bro] SMTP entities log doesn't appears In-Reply-To: References: <2e9247d7e6fc0ae4af533108a73689a4@localhost> Message-ID: <1396394884.3282.1.camel@JamesiMac> On Tue, 2014-04-01 at 14:29 -0400, Seth Hall wrote: > On Mar 28, 2014, at 3:03 AM, C. L. Martinez wrote: > > > Any more ideas please?? > > What version of Bro are you running? (2.1 I suppose?) > > Also, are you positive that your script is being loaded by workers? > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro I can confirm this. [17:26:20 @gateway:~/current$] bro --version bro version 2.2 [17:26:47 @gateway:~/current$] ls -l total 27420 -rw-r--r-- 1 root root 6322917 Apr 1 17:26 conn.log -rw-r--r-- 1 root root 5882 Apr 1 17:06 dhcp.log -rw-r--r-- 1 root root 6468780 Apr 1 17:27 dns.log -rw-r--r-- 1 root root 451 Apr 1 12:48 dpd.log -rw-r--r-- 1 root root 3269780 Apr 1 17:26 files.log -rw-r--r-- 1 root root 11706144 Apr 1 17:26 http.log -rw-r--r-- 1 root root 678 Apr 1 12:55 known_hosts.log -rw-r--r-- 1 root root 419 Apr 1 03:00 known_services.log -rw-r--r-- 1 root root 14606 Mar 31 23:58 loaded_scripts.log -rw-r--r-- 1 root root 568 Mar 31 23:58 packet_filter.log -rw-r--r-- 1 root root 494 Mar 31 23:58 reporter.log -rw-r--r-- 1 root root 110446 Apr 1 17:15 smtp.log -rw-r--r-- 1 root root 27098 Apr 1 17:24 software.log -rw-r--r-- 1 root root 1956 Apr 1 16:36 ssh.log -rw-r--r-- 1 root root 991 Apr 1 16:16 tunnel.log -rw-r--r-- 1 root root 56270 Apr 1 17:24 weird.log [17:27:05 @gateway:~/current$] cat loaded_scripts.log | grep smtp /usr/local/bro/share/bro/base/protocols/smtp/__load__.bro /usr/local/bro/share/bro/base/protocols/smtp/main.bro /usr/local/bro/share/bro/base/protocols/smtp/entities.bro /usr/local/bro/share/bro/base/protocols/smtp/files.bro /usr/local/bro/share/bro/policy/protocols/smtp/software.bro James -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140401/4a8ac1ac/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140401/4a8ac1ac/attachment.bin From cf at npulsetech.com Tue Apr 1 16:39:57 2014 From: cf at npulsetech.com (Chris Fauerbach) Date: Tue, 1 Apr 2014 19:39:57 -0400 Subject: [Bro] SMTP entities log doesn't appears In-Reply-To: <1396394884.3282.1.camel@JamesiMac> References: <2e9247d7e6fc0ae4af533108a73689a4@localhost> <1396394884.3282.1.camel@JamesiMac> Message-ID: files.log should have all your file (http, email, etc) information in it, since you're running bro 2.2 On Tue, Apr 1, 2014 at 7:28 PM, James Lay wrote: > On Tue, 2014-04-01 at 14:29 -0400, Seth Hall wrote: > > On Mar 28, 2014, at 3:03 AM, C. L. Martinez wrote: > > Any more ideas please?? > > What version of Bro are you running? (2.1 I suppose?) > > Also, are you positive that your script is being loaded by workers? > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a networkhttp://www.bro.org/ > > _______________________________________________ > Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > I can confirm this. > > [17:26:20 @gateway :~/current$] bro --version > bro version 2.2 > > [17:26:47 @gateway :~/current$] ls -l > total 27420 > -rw-r--r-- 1 root root 6322917 Apr 1 17:26 conn.log > -rw-r--r-- 1 root root 5882 Apr 1 17:06 dhcp.log > -rw-r--r-- 1 root root 6468780 Apr 1 17:27 dns.log > -rw-r--r-- 1 root root 451 Apr 1 12:48 dpd.log > -rw-r--r-- 1 root root 3269780 Apr 1 17:26 files.log > -rw-r--r-- 1 root root 11706144 Apr 1 17:26 http.log > -rw-r--r-- 1 root root 678 Apr 1 12:55 known_hosts.log > -rw-r--r-- 1 root root 419 Apr 1 03:00 known_services.log > -rw-r--r-- 1 root root 14606 Mar 31 23:58 loaded_scripts.log > -rw-r--r-- 1 root root 568 Mar 31 23:58 packet_filter.log > -rw-r--r-- 1 root root 494 Mar 31 23:58 reporter.log > -rw-r--r-- 1 root root 110446 Apr 1 17:15 smtp.log > -rw-r--r-- 1 root root 27098 Apr 1 17:24 software.log > -rw-r--r-- 1 root root 1956 Apr 1 16:36 ssh.log > -rw-r--r-- 1 root root 991 Apr 1 16:16 tunnel.log > -rw-r--r-- 1 root root 56270 Apr 1 17:24 weird.log > > [17:27:05 @gateway :~/current$] cat loaded_scripts.log | > grep smtp > /usr/local/bro/share/bro/base/protocols/smtp/__load__.bro > /usr/local/bro/share/bro/base/protocols/smtp/main.bro > /usr/local/bro/share/bro/base/protocols/smtp/entities.bro > /usr/local/bro/share/bro/base/protocols/smtp/files.bro > /usr/local/bro/share/bro/policy/protocols/smtp/software.bro > > James > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- -- --Chris Fauerbach VP, Software Engineering nPulse Technologies *Network Forensics for the 10 Gig World* http://www.npulsetech.com 703.969.2186 cf at npulsetech.com -------------------------------------- The information contained herein is for the exclusive use of the original recipient. This information is granted for limited distribution within the recipient's organization for planning purposes only. Further dissemination, whether private or public, is prohibited and may be covered under a non-disclosure agreement. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140401/ddd439b5/attachment.html From seth at icir.org Tue Apr 1 18:15:31 2014 From: seth at icir.org (Seth Hall) Date: Tue, 1 Apr 2014 21:15:31 -0400 Subject: [Bro] SMTP entities log doesn't appears In-Reply-To: <1396394884.3282.1.camel@JamesiMac> References: <2e9247d7e6fc0ae4af533108a73689a4@localhost> <1396394884.3282.1.camel@JamesiMac> Message-ID: On Apr 1, 2014, at 7:28 PM, James Lay wrote: > [17:26:20 @gateway:~/current$] bro --version > bro version 2.2 Ah, yep, files.log became a larger abstraction which was informed by the smtp_entities.log. :) So with 2.2+, look to files.log when you formerly would have looked to smtp_entities. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140401/5072603c/attachment.bin From jlay at slave-tothe-box.net Tue Apr 1 19:59:19 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Tue, 01 Apr 2014 20:59:19 -0600 Subject: [Bro] SMTP entities log doesn't appears In-Reply-To: References: <2e9247d7e6fc0ae4af533108a73689a4@localhost> <1396394884.3282.1.camel@JamesiMac> Message-ID: <1396407559.2432.0.camel@JamesiMac> On Tue, 2014-04-01 at 19:39 -0400, Chris Fauerbach wrote: > files.log should have all your file (http, email, etc) information in > it, since you're running bro 2.2 > > > > > > > On Tue, Apr 1, 2014 at 7:28 PM, James Lay > wrote: > > On Tue, 2014-04-01 at 14:29 -0400, Seth Hall wrote: > > > On Mar 28, 2014, at 3:03 AM, C. L. Martinez wrote: > > > > > Any more ideas please?? > > > > What version of Bro are you running? (2.1 I suppose?) > > > > Also, are you positive that your script is being loaded by workers? > > > > .Seth > > > > -- > > Seth Hall > > International Computer Science Institute > > (Bro) because everyone has a network > > http://www.bro.org/ > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > I can confirm this. > > [17:26:20 @gateway:~/current$] bro --version > bro version 2.2 > > [17:26:47 @gateway:~/current$] ls -l > total 27420 > -rw-r--r-- 1 root root 6322917 Apr 1 17:26 conn.log > -rw-r--r-- 1 root root 5882 Apr 1 17:06 dhcp.log > -rw-r--r-- 1 root root 6468780 Apr 1 17:27 dns.log > -rw-r--r-- 1 root root 451 Apr 1 12:48 dpd.log > -rw-r--r-- 1 root root 3269780 Apr 1 17:26 files.log > -rw-r--r-- 1 root root 11706144 Apr 1 17:26 http.log > -rw-r--r-- 1 root root 678 Apr 1 12:55 known_hosts.log > -rw-r--r-- 1 root root 419 Apr 1 03:00 > known_services.log > -rw-r--r-- 1 root root 14606 Mar 31 23:58 > loaded_scripts.log > -rw-r--r-- 1 root root 568 Mar 31 23:58 packet_filter.log > -rw-r--r-- 1 root root 494 Mar 31 23:58 reporter.log > -rw-r--r-- 1 root root 110446 Apr 1 17:15 smtp.log > -rw-r--r-- 1 root root 27098 Apr 1 17:24 software.log > -rw-r--r-- 1 root root 1956 Apr 1 16:36 ssh.log > -rw-r--r-- 1 root root 991 Apr 1 16:16 tunnel.log > -rw-r--r-- 1 root root 56270 Apr 1 17:24 weird.log > > [17:27:05 @gateway:~/current$] cat loaded_scripts.log | grep > smtp > /usr/local/bro/share/bro/base/protocols/smtp/__load__.bro > /usr/local/bro/share/bro/base/protocols/smtp/main.bro > /usr/local/bro/share/bro/base/protocols/smtp/entities.bro > /usr/local/bro/share/bro/base/protocols/smtp/files.bro > /usr/local/bro/share/bro/policy/protocols/smtp/software.bro > > James > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > > > > -- > > -- > > --Chris Fauerbach > > > VP, Software Engineering > > nPulse Technologies > > Network Forensics for the 10 Gig World > > http://www.npulsetech.com > > 703.969.2186 > > > cf at npulsetech.com > > > -------------------------------------- > The information contained herein is for the exclusive use of the > original recipient. This information is granted for limited > distribution within the recipient's organization for planning purposes > only. Further dissemination, whether private or public, is prohibited > and may be covered under a non-disclosure agreement. > Thanks for the quick answer Chris and Seth. James -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140401/8231982b/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140401/8231982b/attachment.bin From carlopmart at gmail.com Tue Apr 1 22:56:49 2014 From: carlopmart at gmail.com (C. L. Martinez) Date: Wed, 2 Apr 2014 05:56:49 +0000 Subject: [Bro] SMTP entities log doesn't appears In-Reply-To: <1396407559.2432.0.camel@JamesiMac> References: <2e9247d7e6fc0ae4af533108a73689a4@localhost> <1396394884.3282.1.camel@JamesiMac> <1396407559.2432.0.camel@JamesiMac> Message-ID: On Wed, Apr 2, 2014 at 2:59 AM, James Lay wrote: > On Tue, 2014-04-01 at 19:39 -0400, Chris Fauerbach wrote: > > files.log should have all your file (http, email, etc) information in it, > since you're running bro 2.2 > Yes I am running bro 2.2, and it is correct: info appears under files.log. Many thanks to all. From iamreck at gmail.com Wed Apr 2 06:36:04 2014 From: iamreck at gmail.com (Ryan) Date: Wed, 2 Apr 2014 09:36:04 -0400 Subject: [Bro] Sanity check - Grabbing platform tokens from browser user agents (was p0f) In-Reply-To: <52F9117D.4070609@doit.wisc.edu> References: <52E99052.3060807@doit.wisc.edu> <20140201001242.GE8640@datacomm.albany.edu> <52EC7E85.9020401@doit.wisc.edu> <52F9117D.4070609@doit.wisc.edu> Message-ID: Gary, This looks very nice. I'm curious if you had any more updates or improvements for this? Ryan Peck On Mon, Feb 10, 2014 at 12:50 PM, Gary Faulkner wrote: > After running various iterations of the original script against several > pcaps of our local traffic (and a couple days of live traffic) I ended up > finding a lot of user agents that would match against the desktop/server OS > rules, but were not necessarily desktops or servers. I ended up adding to > the matching rules in part to parse out these things and also to detect > other things we were interested in. Checking for more things seems to incur > a performance penalty, so I also made some effort to move some of the more > common matches sooner in the if/else statements to avoid having to check > all of the less likely items first. The create_expire statement still > doesn't behave as I expected, as each match is logged once per log rotation > as opposed to once per day, but the matching seems to work with the > exception that it doesn't check for every possible user agent case. I may > also be missing explicitly including scripts that are already commonly > loaded. > > ======================== Begin Script ======================== > > @load base/utils/site > > module BrowserPlatform; > > export > { > # The fully resolved name for this log will be BrowserPlatform::LOG > redef enum Log::ID += { LOG }; > > type Info: record { > ts: time &log &optional; > uid: string &log &optional; > host: addr &log &optional; > platform_token: string &log &optional; > unparsed_version: string &log &optional; > }; > > # A set of seen IP + OS combinations. Used to prevent logging the same > combo repeatedly. > global seen_browser_platforms: set[string] &create_expire = 1.0 day > &synchronized &redef; > } > > event bro_init() &priority=5 > { > Log::create_stream(BrowserPlatform::LOG,[$columns=Info]); > } > > event http_header(c: connection, is_orig: bool, name: string, value: > string) > { > local platform = "Unknown OS"; > if (!is_orig || name != "USER-AGENT" || !Site::is_local_addr(c$id$ > orig_h)) > return; > > # Parse out Apple IOS and Android variants first as some apps will dispay > as compatible with a desktop OS version > > if ( /iPhone/ in value ) > platform = "iPhone"; > else if ( /iPad/ in value ) > platform = "iPad"; > else if ( /iPod/ in value ) > platform = "iPod"; > else if ( /Android/ in value ) > platform = "Android"; > > # Once we've parsed out mobiles move onto desktop/server OS > # User agents listed in order of expected use or to pre-parse user-agents > that might otherwise match multiple rules. > > else if ( /Windows/ in value ) > { > if ( /Xbox/ in value ) # often includes a Windows OS version or > identifies as a Mobile browser > platform = "Xbox"; > else if ( /Phone/ in value || /Mobile/ in value ) # often includes > Windows OS version > platform = "Windows Phone"; > else if ( /Windows NT 6.1/ in value ) > platform = "Windows 7"; > else if ( /Windows NT 5.1/ in value ) > platform = "Windows XP"; > else if ( /Windows NT 5.2/ in value && /WOW64/ in value ) > platform = "Windows XP x64"; > else if ( /Windows NT 6.0/ in value ) > platform = "Windows Vista"; > else if ( /Windows NT 6.2/ in value ) > platform = "Windows 8"; > else if ( /Windows NT 6.3/ in value ) > platform = "Windows 8.1"; > else if ( /Windows 95/ in value ) > platform = "Windows 95"; > else if ( /Windows 98/ in value && /4.90/ !in value ) > platform = "Windows 98"; > else if ( /Win 9x 4.90/ in value ) > platform = "Windows Me"; > else if ( /Windows NT 4.0/ in value ) > platform = "Windows NT 4.0"; > else if ( /Windows NT 5.0/ in value || /Windows 2000/ in value ) > platform = "Windows 2000"; > # Catch-all for identifying less common user-agents. Can be noisy. > # else > # platform = "Windows Other"; > } > else if ( /Mac OS X/ in value ) > { > if ( /Mac OS X 10_9/ in value || /Mac OS X 10.9/ in value ) > platform = "Mac OS X 10.9"; > else if ( /Mac OS X 10_8/ in value || /Mac OS X 10.8/ in value ) > platform = "Mac OS X 10.8"; > else if ( /Mac OS X 10_7/ in value || /Mac OS X 10.7/ in value ) > platform = "Mac OS X 10.7"; > else if ( /Mac OS X 10_6/ in value || /Mac OS X 10.6/ in value ) > platform = "Mac OS X 10.6"; > else if ( /Mac OS X 10_5/ in value || /Mac OS X 10.5/ in value ) > platform = "Mac OS X 10.5"; > else if ( /Mac OS X 10_4/ in value || /Mac OS X 10.4/ in value ) > platform = "Mac OS X 10.4"; > # Catch-all for identifying less common user-agents. Can be noisy. > # else > # platform = "Mac OS X Other"; > } > else if ( /Linux/ in value ) > platform = "Linux"; > > # Check to see if IP+OS combo already logged and if not log it and add it > to the list of tracked combos. > > > local saw = cat(c$id$orig_h,platform); #There is probably a less ugly > way to do this than cat, but it seems to work > if ( platform != "Unknown OS" && saw !in seen_browser_platforms ) > { > local rec: BrowserPlatform::Info = [$ts=network_time(), > $uid=c$uid, $host=c$id$orig_h, $platform_token=platform, > $unparsed_version=value]; > Log::write(BrowserPlatform::LOG, rec); > add seen_browser_platforms[saw]; > } > } > > ======================== End Script ======================== > > On 1/31/2014 10:56 PM, Gary Faulkner wrote: > >> Thanks for the suggestions, that cleans that bit up quite nicely. I >> actually started by trying to deconstruct the various software.bro >> scripts and work my way backwards through the framework to see what was >> doing what. I'm still trying to navigate my way through that code, but I >> agree that it would make more sense to leverage it directly than create >> a derivative just to pull out a specific bit of the data. I'm not >> currently running Splunk in any production sense, but that is pretty >> much what I'm trying to do in Bro. Thanks for sharing it! >> >> Regards, >> Gary >> >> On 1/31/2014 6:12 PM, Justin Azoff wrote: >> >>> On Wed, Jan 29, 2014 at 05:35:46PM -0600, Gary Faulkner wrote: >>> >>>> event http_header(c: connection, is_orig: bool, name: string, value: >>>> string) >>>> { >>>> local platform = "Unknown OS"; >>>> if ( is_orig ) >>>> { >>>> if ( name == "USER-AGENT" && /Windows NT 5.1/ in value ) >>>> { >>>> platform = "Windows XP"; >>>> } >>>> else if ( name == "USER-AGENT" && /Windows NT 6.0/ in value ) >>>> { >>>> platform = "Windows Vista"; >>>> } >>>> else if ( name == "USER-AGENT" && /Windows NT 6.1/ in value ) >>>> { >>>> platform = "Windows 7"; >>>> } >>>> >>> .. >>> >>> Modifying the http_header event handler as follows will increase >>> performance: >>> >>> event http_header(c: connection, is_orig: bool, name: string, value: >>> string) >>> { >>> if(!is_orig || name != "USER-AGENT") >>> return; >>> if(/Windows NT 5.1/ in value) >>> platform = "Windows XP"; >>> else if ... >>> >>> FWIW, I used to do this kind of thing outside of bro using splunk: >>> >>> https://github.com/JustinAzoff/splunk-scripts/blob/master/ua2os.py >>> >>> One thing you may want to do is rather than use the http_header event >>> use >>> >>> event log_software(rec: Info) >>> { >>> ... >>> } >>> >>> which will be raised every time a new software version is seen. The >>> software framework is already pulling most of the info out that you >>> might need, so you can piggy back on the work that it is doing. >>> >>> > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140402/20d512cf/attachment.html From gary at doit.wisc.edu Wed Apr 2 08:30:59 2014 From: gary at doit.wisc.edu (Gary Faulkner) Date: Wed, 02 Apr 2014 10:30:59 -0500 Subject: [Bro] Sanity check - Grabbing platform tokens from browser user agents (was p0f) In-Reply-To: References: <52E99052.3060807@doit.wisc.edu> <20140201001242.GE8640@datacomm.albany.edu> <52EC7E85.9020401@doit.wisc.edu> <52F9117D.4070609@doit.wisc.edu> Message-ID: <533C2D33.2010901@doit.wisc.edu> I haven't updated it from this point yet as I've been struggling with hooking into the existing software logging as well as having problems keeping track of state to prevent/reduce duplicate log entries. There were also some performance concerns raised so I've been hesitant to post any in progress work that might inadvertently cause someone else grief. My observations after running the script continuously for the last month is that it probably needs the ability to exclude specific subnets. An example might be wireless networks that may have high client IP churn, short DHCP lease times, are more likely to have mobile devices with apps that have ugly user-agents, and just generally likely to provide unreliable data. Regards, Gary On 4/2/2014 8:36 AM, Ryan wrote: > Gary, > > This looks very nice. I'm curious if you had any more updates or > improvements for this? > > Ryan Peck > > > > On Mon, Feb 10, 2014 at 12:50 PM, Gary Faulkner wrote: > >> After running various iterations of the original script against several >> pcaps of our local traffic (and a couple days of live traffic) I ended up >> finding a lot of user agents that would match against the desktop/server OS >> rules, but were not necessarily desktops or servers. I ended up adding to >> the matching rules in part to parse out these things and also to detect >> other things we were interested in. Checking for more things seems to incur >> a performance penalty, so I also made some effort to move some of the more >> common matches sooner in the if/else statements to avoid having to check >> all of the less likely items first. The create_expire statement still >> doesn't behave as I expected, as each match is logged once per log rotation >> as opposed to once per day, but the matching seems to work with the >> exception that it doesn't check for every possible user agent case. I may >> also be missing explicitly including scripts that are already commonly >> loaded. >> >> ======================== Begin Script ======================== >> >> @load base/utils/site >> >> module BrowserPlatform; >> >> export >> { >> # The fully resolved name for this log will be BrowserPlatform::LOG >> redef enum Log::ID += { LOG }; >> >> type Info: record { >> ts: time &log &optional; >> uid: string &log &optional; >> host: addr &log &optional; >> platform_token: string &log &optional; >> unparsed_version: string &log &optional; >> }; >> >> # A set of seen IP + OS combinations. Used to prevent logging the same >> combo repeatedly. >> global seen_browser_platforms: set[string] &create_expire = 1.0 day >> &synchronized &redef; >> } >> >> event bro_init() &priority=5 >> { >> Log::create_stream(BrowserPlatform::LOG,[$columns=Info]); >> } >> >> event http_header(c: connection, is_orig: bool, name: string, value: >> string) >> { >> local platform = "Unknown OS"; >> if (!is_orig || name != "USER-AGENT" || !Site::is_local_addr(c$id$ >> orig_h)) >> return; >> >> # Parse out Apple IOS and Android variants first as some apps will dispay >> as compatible with a desktop OS version >> >> if ( /iPhone/ in value ) >> platform = "iPhone"; >> else if ( /iPad/ in value ) >> platform = "iPad"; >> else if ( /iPod/ in value ) >> platform = "iPod"; >> else if ( /Android/ in value ) >> platform = "Android"; >> >> # Once we've parsed out mobiles move onto desktop/server OS >> # User agents listed in order of expected use or to pre-parse user-agents >> that might otherwise match multiple rules. >> >> else if ( /Windows/ in value ) >> { >> if ( /Xbox/ in value ) # often includes a Windows OS version or >> identifies as a Mobile browser >> platform = "Xbox"; >> else if ( /Phone/ in value || /Mobile/ in value ) # often includes >> Windows OS version >> platform = "Windows Phone"; >> else if ( /Windows NT 6.1/ in value ) >> platform = "Windows 7"; >> else if ( /Windows NT 5.1/ in value ) >> platform = "Windows XP"; >> else if ( /Windows NT 5.2/ in value && /WOW64/ in value ) >> platform = "Windows XP x64"; >> else if ( /Windows NT 6.0/ in value ) >> platform = "Windows Vista"; >> else if ( /Windows NT 6.2/ in value ) >> platform = "Windows 8"; >> else if ( /Windows NT 6.3/ in value ) >> platform = "Windows 8.1"; >> else if ( /Windows 95/ in value ) >> platform = "Windows 95"; >> else if ( /Windows 98/ in value && /4.90/ !in value ) >> platform = "Windows 98"; >> else if ( /Win 9x 4.90/ in value ) >> platform = "Windows Me"; >> else if ( /Windows NT 4.0/ in value ) >> platform = "Windows NT 4.0"; >> else if ( /Windows NT 5.0/ in value || /Windows 2000/ in value ) >> platform = "Windows 2000"; >> # Catch-all for identifying less common user-agents. Can be noisy. >> # else >> # platform = "Windows Other"; >> } >> else if ( /Mac OS X/ in value ) >> { >> if ( /Mac OS X 10_9/ in value || /Mac OS X 10.9/ in value ) >> platform = "Mac OS X 10.9"; >> else if ( /Mac OS X 10_8/ in value || /Mac OS X 10.8/ in value ) >> platform = "Mac OS X 10.8"; >> else if ( /Mac OS X 10_7/ in value || /Mac OS X 10.7/ in value ) >> platform = "Mac OS X 10.7"; >> else if ( /Mac OS X 10_6/ in value || /Mac OS X 10.6/ in value ) >> platform = "Mac OS X 10.6"; >> else if ( /Mac OS X 10_5/ in value || /Mac OS X 10.5/ in value ) >> platform = "Mac OS X 10.5"; >> else if ( /Mac OS X 10_4/ in value || /Mac OS X 10.4/ in value ) >> platform = "Mac OS X 10.4"; >> # Catch-all for identifying less common user-agents. Can be noisy. >> # else >> # platform = "Mac OS X Other"; >> } >> else if ( /Linux/ in value ) >> platform = "Linux"; >> >> # Check to see if IP+OS combo already logged and if not log it and add it >> to the list of tracked combos. >> >> >> local saw = cat(c$id$orig_h,platform); #There is probably a less ugly >> way to do this than cat, but it seems to work >> if ( platform != "Unknown OS" && saw !in seen_browser_platforms ) >> { >> local rec: BrowserPlatform::Info = [$ts=network_time(), >> $uid=c$uid, $host=c$id$orig_h, $platform_token=platform, >> $unparsed_version=value]; >> Log::write(BrowserPlatform::LOG, rec); >> add seen_browser_platforms[saw]; >> } >> } >> >> ======================== End Script ======================== >> >> On 1/31/2014 10:56 PM, Gary Faulkner wrote: >> >>> Thanks for the suggestions, that cleans that bit up quite nicely. I >>> actually started by trying to deconstruct the various software.bro >>> scripts and work my way backwards through the framework to see what was >>> doing what. I'm still trying to navigate my way through that code, but I >>> agree that it would make more sense to leverage it directly than create >>> a derivative just to pull out a specific bit of the data. I'm not >>> currently running Splunk in any production sense, but that is pretty >>> much what I'm trying to do in Bro. Thanks for sharing it! >>> >>> Regards, >>> Gary >>> >>> On 1/31/2014 6:12 PM, Justin Azoff wrote: >>> >>>> On Wed, Jan 29, 2014 at 05:35:46PM -0600, Gary Faulkner wrote: >>>> >>>>> event http_header(c: connection, is_orig: bool, name: string, value: >>>>> string) >>>>> { >>>>> local platform = "Unknown OS"; >>>>> if ( is_orig ) >>>>> { >>>>> if ( name == "USER-AGENT" && /Windows NT 5.1/ in value ) >>>>> { >>>>> platform = "Windows XP"; >>>>> } >>>>> else if ( name == "USER-AGENT" && /Windows NT 6.0/ in value ) >>>>> { >>>>> platform = "Windows Vista"; >>>>> } >>>>> else if ( name == "USER-AGENT" && /Windows NT 6.1/ in value ) >>>>> { >>>>> platform = "Windows 7"; >>>>> } >>>>> >>>> .. >>>> >>>> Modifying the http_header event handler as follows will increase >>>> performance: >>>> >>>> event http_header(c: connection, is_orig: bool, name: string, value: >>>> string) >>>> { >>>> if(!is_orig || name != "USER-AGENT") >>>> return; >>>> if(/Windows NT 5.1/ in value) >>>> platform = "Windows XP"; >>>> else if ... >>>> >>>> FWIW, I used to do this kind of thing outside of bro using splunk: >>>> >>>> https://github.com/JustinAzoff/splunk-scripts/blob/master/ua2os.py >>>> >>>> One thing you may want to do is rather than use the http_header event >>>> use >>>> >>>> event log_software(rec: Info) >>>> { >>>> ... >>>> } >>>> >>>> which will be raised every time a new software version is seen. The >>>> software framework is already pulling most of the info out that you >>>> might need, so you can piggy back on the work that it is doing. >>>> >>>> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6257 bytes Desc: S/MIME Cryptographic Signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140402/3c2d3e61/attachment.bin From jlay at slave-tothe-box.net Wed Apr 2 11:14:02 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Wed, 02 Apr 2014 12:14:02 -0600 Subject: [Bro] Interesting observation with ssh on non-ssh port Message-ID: <0a7a8fcf6dc160c2e1d72aefee392ff2@localhost> All, I resolved to get monitoring of large amounts of outbound data (the exfiltrate and largeTx type scripts) working today. My test setup is two internal machines (192.168.1.). My hope was that I could just test scp'ing a file from a machine running bro. I haven't got any scripts to function, so I looked at the conn.log. Interestingly, the port I run ssh on doesn't show up. I see my connected sessions fine in ssh.log, but there's no trace of it in conn.log. This obviously explains why I couldn't get the large outbound transfer scripts working, but now I'm curious...is there a reason why this TCP session doesn't show up in conn.log? Running bro 2.2...thank you. James From seth at icir.org Wed Apr 2 13:08:12 2014 From: seth at icir.org (Seth Hall) Date: Wed, 2 Apr 2014 16:08:12 -0400 Subject: [Bro] Interesting observation with ssh on non-ssh port In-Reply-To: <0a7a8fcf6dc160c2e1d72aefee392ff2@localhost> References: <0a7a8fcf6dc160c2e1d72aefee392ff2@localhost> Message-ID: <132F71BE-89AB-4F86-A8FD-9FF451E11115@icir.org> On Apr 2, 2014, at 2:14 PM, James Lay wrote: > but there's no trace of it in conn.log. This obviously explains why I > couldn't get the large outbound transfer scripts working, but now I'm > curious...is there a reason why this TCP session doesn't show up in > conn.log? Running bro 2.2...thank you. Did you close down the connection? Bro doesn?t log anything until the connection ends. In scripts though you could use the ConnPolling thing that Jon Siwek mentioned in another thread to monitor the connection in-flight. Typically I don?t recommend relying on the log events except for very simple tasks. Doing any sort of in-progress monitoring of connections is almost intrinsically not a simple task. .SEth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140402/e294af06/attachment.bin From jlay at slave-tothe-box.net Wed Apr 2 13:11:45 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Wed, 02 Apr 2014 14:11:45 -0600 Subject: [Bro] Interesting observation with ssh on non-ssh port In-Reply-To: <132F71BE-89AB-4F86-A8FD-9FF451E11115@icir.org> References: <0a7a8fcf6dc160c2e1d72aefee392ff2@localhost> <132F71BE-89AB-4F86-A8FD-9FF451E11115@icir.org> Message-ID: <47b60737c0b940fe25dc7c136732636f@localhost> On 2014-04-02 14:08, Seth Hall wrote: > On Apr 2, 2014, at 2:14 PM, James Lay > wrote: > >> but there's no trace of it in conn.log. This obviously explains why >> I >> couldn't get the large outbound transfer scripts working, but now >> I'm >> curious...is there a reason why this TCP session doesn't show up in >> conn.log? Running bro 2.2...thank you. > > Did you close down the connection? Bro doesn?t log anything until > the connection ends. In scripts though you could use the ConnPolling > thing that Jon Siwek mentioned in another thread to monitor the > connection in-flight. Typically I don?t recommend relying on the log > events except for very simple tasks. Doing any sort of in-progress > monitoring of connections is almost intrinsically not a simple task. > > .SEth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ Hi Seth, I absolutely did. I was originally scp'ing a file, then started to test with ssh'ing to the host and exiting out. Is there a way to debug this? I think I even compiled it for debugging :) James From seth at icir.org Wed Apr 2 13:58:11 2014 From: seth at icir.org (Seth Hall) Date: Wed, 2 Apr 2014 16:58:11 -0400 Subject: [Bro] Interesting observation with ssh on non-ssh port In-Reply-To: <47b60737c0b940fe25dc7c136732636f@localhost> References: <0a7a8fcf6dc160c2e1d72aefee392ff2@localhost> <132F71BE-89AB-4F86-A8FD-9FF451E11115@icir.org> <47b60737c0b940fe25dc7c136732636f@localhost> Message-ID: <0E176CDA-B692-43FE-B400-62E59D1EC112@icir.org> On Apr 2, 2014, at 4:11 PM, James Lay wrote: > I absolutely did. I was originally scp'ing a file, then started to test with ssh'ing to the host and exiting out. Is there a way to debug this? I think I even compiled it for debugging :) You?re sure that your box was seeing the traffic you think it was? Also, if this is on a real network you want to be careful with building with debugging enabled because it can cause a large performance hit which can then cause it?s own problems. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140402/ea25de3b/attachment.bin From jlay at slave-tothe-box.net Wed Apr 2 14:10:24 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Wed, 02 Apr 2014 15:10:24 -0600 Subject: [Bro] Interesting observation with ssh on non-ssh port In-Reply-To: <0E176CDA-B692-43FE-B400-62E59D1EC112@icir.org> References: <0a7a8fcf6dc160c2e1d72aefee392ff2@localhost> <132F71BE-89AB-4F86-A8FD-9FF451E11115@icir.org> <47b60737c0b940fe25dc7c136732636f@localhost> <0E176CDA-B692-43FE-B400-62E59D1EC112@icir.org> Message-ID: <90c3b79b7a2fca0293e869064f5c4413@localhost> On 2014-04-02 14:58, Seth Hall wrote: > On Apr 2, 2014, at 4:11 PM, James Lay > wrote: > >> I absolutely did. I was originally scp'ing a file, then started to >> test with ssh'ing to the host and exiting out. Is there a way to >> debug this? I think I even compiled it for debugging :) > > You?re sure that your box was seeing the traffic you think it was? > Also, if this is on a real network you want to be careful with > building with debugging enabled because it can cause a large > performance hit which can then cause it?s own problems. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ Bah...the port I was listening on was a spanned port 8-| Actually looping in on itself...shut that off and now I see things just fine...sorry for the waste of bits on the list. Thanks Seth. James From jsiwek at illinois.edu Wed Apr 2 14:15:20 2014 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Wed, 2 Apr 2014 21:15:20 +0000 Subject: [Bro] Interesting observation with ssh on non-ssh port In-Reply-To: <0a7a8fcf6dc160c2e1d72aefee392ff2@localhost> References: <0a7a8fcf6dc160c2e1d72aefee392ff2@localhost> Message-ID: On Apr 2, 2014, at 1:14 PM, James Lay wrote: > I see my connected sessions fine in ssh.log, > but there's no trace of it in conn.log. This obviously explains why I > couldn't get the large outbound transfer scripts working, but now I'm > curious...is there a reason why this TCP session doesn't show up in > conn.log? No immediate idea on why the TCP session isn?t showing in conn.log, but one thing to be aware of is SSH::skip_processing_after_detection. If you?ve redef?d that to true, then any large-transfer detection is bound to fail for SSH sessions. Generally, any connection on which the skip_further_processing() built-in function is called won?t have accurate size/packet counts. - Jon From jlay at slave-tothe-box.net Wed Apr 2 14:20:19 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Wed, 02 Apr 2014 15:20:19 -0600 Subject: [Bro] Interesting observation with ssh on non-ssh port In-Reply-To: References: <0a7a8fcf6dc160c2e1d72aefee392ff2@localhost> Message-ID: <4c8fe38f9f880276ce53cb4f32bf13eb@localhost> On 2014-04-02 15:15, Siwek, Jonathan Luke wrote: > On Apr 2, 2014, at 1:14 PM, James Lay > wrote: > >> I see my connected sessions fine in ssh.log, >> but there's no trace of it in conn.log. This obviously explains why >> I >> couldn't get the large outbound transfer scripts working, but now >> I'm >> curious...is there a reason why this TCP session doesn't show up in >> conn.log? > > No immediate idea on why the TCP session isn?t showing in conn.log, > but one thing to be aware of is SSH::skip_processing_after_detection. > If you?ve redef?d that to true, then any large-transfer detection is > bound to fail for SSH sessions. Generally, any connection on which > the skip_further_processing() built-in function is called won?t have > accurate size/packet counts. > > - Jon Thanks Jon I'll keep that in mind. Short answer was I'm an idiot :) James From anthony.kasza at gmail.com Wed Apr 2 19:00:05 2014 From: anthony.kasza at gmail.com (anthony kasza) Date: Wed, 2 Apr 2014 19:00:05 -0700 Subject: [Bro] Log Stream Types Message-ID: Hey Bro List, I'm hoping someone could explain why %prefix%bro/share/bro/base/frameworks/logging/main.bro (from an installation) defines a Log::Stream type as a record of two any types but bro/src/logging/Manager.cc (line 335 from Github) seems to enforce Log::Stream types to consist of an event type. I'm curious to see if it is possible to take immediate action upon a log line being ready with a function or hook instead of having to wait for an event to be handled. Thanks, -AK From jlay at slave-tothe-box.net Thu Apr 3 14:39:05 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Thu, 03 Apr 2014 15:39:05 -0600 Subject: [Bro] Large file ex-filtration revisited Message-ID: So first off a HUGE thank you to Robert Rotsted who posted the original after_hours_exfiltrate.bro. (http://mailman.icsi.berkeley.edu/pipermail/bro/2014-March/007510.html). Here's how I've modified this: module Exfil; export { redef enum Notice::Type += { Large_File_Upload, }; } ## Each time a connection is logged execute the following code event Conn::log_conn(rec: Conn::Info) { ## Ensure orig_bytes and resp_bytes exist, if not, return. if (! (rec?$orig_bytes || rec?$resp_bytes)) return; ## Is this connection between a local originator and a ## remote responder? ## Are the sent bytes greater that 10 x the received bytes? ## Has the originator sent more than 3 Megabytes? if ( rec$id$orig_h in Site::local_nets && rec$id$resp_h !in Site::local_nets && rec$orig_bytes > (20 * rec$resp_bytes) && rec$orig_bytes >= 13145728 ) { NOTICE([$note=Large_File_Upload, $id=rec$id, $identifier=cat(rec$uid), $msg=fmt("Sent Bytes: %s, Received Bytes: %s", rec$orig_bytes, rec$resp_bytes)]); } } I noticed today an anomaly I guess: 2014-04-03T13:38:45-0600 - x.x.x.x 55023 4.71.33.182 80 - - - tcp Exfil::Large_File_Upload Sent Bytes: 1213381425, Received Bytes: 0 - x.x.x.x 4.71.33.182 80 - bro Notice::ACTION_LOG 3600.000000 F - - - -- 2014-04-03T13:38:42-0600 CSZCCe4mZI1T7iJogg x.x.x.x 55023 4.71.33.182 80 tcp - 0.035191 1213381425 0 RSTOS0 T 0 SaR 2 88 1 40 (empty) I found a RST packet in the capture that matched close to the sent bytes: Transmission Control Protocol, Src Port: 55023 (55023), Dst Port: http (80), Seq: 1213381426, Len: 0 Did I hose the script by removing the hourly constraint? Thanks for the the assist...this has helped me better understand the scripting (though I'm still just at the copy and paste level :)). James From jsiwek at illinois.edu Thu Apr 3 15:18:08 2014 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Thu, 3 Apr 2014 22:18:08 +0000 Subject: [Bro] Large file ex-filtration revisited In-Reply-To: References: Message-ID: On Apr 3, 2014, at 4:39 PM, James Lay wrote: > 2014-04-03T13:38:42-0600 CSZCCe4mZI1T7iJogg x.x.x.x > 55023 4.71.33.182 80 tcp - 0.035191 > 1213381425 0 RSTOS0 T 0 SaR 2 88 > 1 40 (empty) This looks like it may be a ?half-open? TCP connection, and Bro may report inaccurate {orig,resp}_bytes unless you?re running a development version from the git repo which has a fix for this situation. What version of Bro are you running? A way to improve your detection with only script changes could be to include {orig,resp}_ip_bytes in the criteria. The difference is that field counts total bytes of IP packets, not just payload data. It?s also more sensitive to packet loss, where {orig,resp}_bytes should still work since it?s monitoring the TCP sequence space. - Jon From jlay at slave-tothe-box.net Thu Apr 3 15:45:46 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Thu, 03 Apr 2014 16:45:46 -0600 Subject: [Bro] Large file ex-filtration revisited In-Reply-To: References: Message-ID: <333a080cf6ca1f57298e5f93d673fff9@localhost> On 2014-04-03 16:18, Siwek, Jonathan Luke wrote: > On Apr 3, 2014, at 4:39 PM, James Lay > wrote: > >> 2014-04-03T13:38:42-0600 CSZCCe4mZI1T7iJogg x.x.x.x >> 55023 4.71.33.182 80 tcp - 0.035191 >> 1213381425 0 RSTOS0 T 0 SaR 2 88 >> 1 40 (empty) > > This looks like it may be a ?half-open? TCP connection, and Bro may > report inaccurate {orig,resp}_bytes unless you?re running a > development version from the git repo which has a fix for this > situation. What version of Bro are you running? > > A way to improve your detection with only script changes could be to > include {orig,resp}_ip_bytes in the criteria. The difference is that > field counts total bytes of IP packets, not just payload data. It?s > also more sensitive to packet loss, where {orig,resp}_bytes should > still work since it?s monitoring the TCP sequence space. > > - Jon Thanks Jon, I'm on 2.2 here. I'm going to start fiddling with the script now...thanks again for the help and response. James From jlay at slave-tothe-box.net Thu Apr 3 15:59:42 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Thu, 03 Apr 2014 16:59:42 -0600 Subject: [Bro] Large file ex-filtration revisited In-Reply-To: <333a080cf6ca1f57298e5f93d673fff9@localhost> References: <333a080cf6ca1f57298e5f93d673fff9@localhost> Message-ID: On 2014-04-03 16:45, James Lay wrote: > On 2014-04-03 16:18, Siwek, Jonathan Luke wrote: >> On Apr 3, 2014, at 4:39 PM, James Lay >> wrote: >> >>> 2014-04-03T13:38:42-0600 CSZCCe4mZI1T7iJogg x.x.x.x >>> 55023 4.71.33.182 80 tcp - 0.035191 >>> 1213381425 0 RSTOS0 T 0 SaR 2 88 >>> 1 40 (empty) >> >> This looks like it may be a ?half-open? TCP connection, and Bro may >> report inaccurate {orig,resp}_bytes unless you?re running a >> development version from the git repo which has a fix for this >> situation. What version of Bro are you running? >> >> A way to improve your detection with only script changes could be to >> include {orig,resp}_ip_bytes in the criteria. The difference is >> that >> field counts total bytes of IP packets, not just payload data. It?s >> also more sensitive to packet loss, where {orig,resp}_bytes should >> still work since it?s monitoring the TCP sequence space. >> >> - Jon > > Thanks Jon, > > I'm on 2.2 here. I'm going to start fiddling with the script > now...thanks again for the help and response. > > James Ok...I've made the below modification: if ( rec$id$orig_h in Site::local_nets && rec$id$resp_h in Site::local_nets && rec$orig_bytes > (10 * rec$resp_bytes) && rec$orig_bytes > (10 * rec$resp_ip_bytes) && rec$orig_bytes >= 3145728 ) { This works in dev when sending a large file, so going to test this out in production....thank you. James From r.bortolameotti at gmail.com Fri Apr 4 02:21:13 2014 From: r.bortolameotti at gmail.com (Riccardo Bortolameotti) Date: Fri, 04 Apr 2014 11:21:13 +0200 Subject: [Bro] Error SSL::Info server_name - not present Message-ID: <1396603273.3897.7.camel@stud169130.mobiel.utwente.nl> Hello, I am a student, and I started few days ago to approach the bro's scripting language. I am writing a script that needs to get some information regarding SSL communications. In my script I do: - local server_n = c$ssl$server_name; - local subj = c$ssl$subject; Of course these fields are not always in the table (e.g. in SSLv3 server_name does not exist, therefore no values), then my script raise a warning while it runs, saying that those values do not exist. What can I do to avoid that warning output? I have tried to do some checks like: if (c$ssl$server_name=="-" || c$ssl$server_name=="empty") do something ... But it does not work, I do not know how it is represented it is not in the table. thank you in advance, glad to join this community, R. From bernhard at ICSI.Berkeley.EDU Fri Apr 4 05:24:10 2014 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Fri, 4 Apr 2014 05:24:10 -0700 Subject: [Bro] Error SSL::Info server_name - not present In-Reply-To: <1396603273.3897.7.camel@stud169130.mobiel.utwente.nl> References: <1396603273.3897.7.camel@stud169130.mobiel.utwente.nl> Message-ID: <0A2179DA-A814-4EE9-9D79-EABAC27EB649@icsi.berkeley.edu> Hello Riccardo, to check if a record entry is present, you can use the ?$ operator. So - if ( c$ssl?$server_name) do something? Bernhard On Apr 4, 2014, at 2:21 AM, Riccardo Bortolameotti wrote: > Hello, > > I am a student, and I started few days ago to approach the bro's > scripting language. > > I am writing a script that needs to get some information regarding SSL > communications. In my script I do: > - local server_n = c$ssl$server_name; > - local subj = c$ssl$subject; > Of course these fields are not always in the table (e.g. in SSLv3 > server_name does not exist, therefore no values), then my script raise a > warning while it runs, saying that those values do not exist. > > What can I do to avoid that warning output? > > I have tried to do some checks like: > > if (c$ssl$server_name=="-" || c$ssl$server_name=="empty") > do something ... > > But it does not work, I do not know how it is represented it is not in > the table. > > thank you in advance, > > glad to join this community, > > R. > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > From rotsted at reservoir.com Fri Apr 4 07:38:50 2014 From: rotsted at reservoir.com (Robert Rotsted) Date: Fri, 4 Apr 2014 07:38:50 -0700 Subject: [Bro] Large file ex-filtration revisited In-Reply-To: References: Message-ID: James, Glad to hear that the script was helpful! --bob On Thu, Apr 3, 2014 at 2:39 PM, James Lay wrote: > So first off a HUGE thank you to Robert Rotsted who posted the original > after_hours_exfiltrate.bro. > (http://mailman.icsi.berkeley.edu/pipermail/bro/2014-March/007510.html). > Here's how I've modified this: > > module Exfil; > > export { > > redef enum Notice::Type += { > Large_File_Upload, > }; > } > > > ## Each time a connection is logged execute the following code > event Conn::log_conn(rec: Conn::Info) { > > ## Ensure orig_bytes and resp_bytes exist, if not, return. > if (! (rec?$orig_bytes || rec?$resp_bytes)) > return; > > ## Is this connection between a local originator and a > ## remote responder? > ## Are the sent bytes greater that 10 x the received bytes? > ## Has the originator sent more than 3 Megabytes? > if ( rec$id$orig_h in Site::local_nets && > rec$id$resp_h !in Site::local_nets && > rec$orig_bytes > (20 * rec$resp_bytes) && > rec$orig_bytes >= 13145728 ) > { > > NOTICE([$note=Large_File_Upload, > $id=rec$id, > $identifier=cat(rec$uid), > $msg=fmt("Sent Bytes: %s, Received Bytes: %s", > rec$orig_bytes, rec$resp_bytes)]); > } > > } > > I noticed today an anomaly I guess: > > > 2014-04-03T13:38:45-0600 - x.x.x.x 55023 4.71.33.182 > 80 - - - tcp Exfil::Large_File_Upload > Sent Bytes: 1213381425, Received Bytes: 0 - x.x.x.x > 4.71.33.182 80 - bro Notice::ACTION_LOG > 3600.000000 F - - - -- > > > 2014-04-03T13:38:42-0600 CSZCCe4mZI1T7iJogg x.x.x.x > 55023 4.71.33.182 80 tcp - 0.035191 > 1213381425 0 RSTOS0 T 0 SaR 2 88 > 1 40 (empty) > > > I found a RST packet in the capture that matched close to the sent > bytes: > > Transmission Control Protocol, Src Port: 55023 (55023), Dst Port: http > (80), Seq: 1213381426, Len: 0 > > Did I hose the script by removing the hourly constraint? Thanks for > the the assist...this has helped me better understand the scripting > (though I'm still just at the copy and paste level :)). > > James > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Bob Rotsted Senior Engineer Reservoir Labs, Inc. 503-225-0583 x138 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140404/1ea779aa/attachment.html From jlay at slave-tothe-box.net Fri Apr 4 08:36:09 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 04 Apr 2014 09:36:09 -0600 Subject: [Bro] Large file ex-filtration revisited In-Reply-To: References: <333a080cf6ca1f57298e5f93d673fff9@localhost> Message-ID: <02b6eab49e6a3b86d4a9e9884afc705b@localhost> On 2014-04-03 16:59, James Lay wrote: > On 2014-04-03 16:45, James Lay wrote: >> On 2014-04-03 16:18, Siwek, Jonathan Luke wrote: >>> On Apr 3, 2014, at 4:39 PM, James Lay >>> wrote: >>> >>>> 2014-04-03T13:38:42-0600 CSZCCe4mZI1T7iJogg x.x.x.x >>>> 55023 4.71.33.182 80 tcp - 0.035191 >>>> 1213381425 0 RSTOS0 T 0 SaR 2 88 >>>> 1 40 (empty) >>> >>> This looks like it may be a ?half-open? TCP connection, and Bro may >>> report inaccurate {orig,resp}_bytes unless you?re running a >>> development version from the git repo which has a fix for this >>> situation. What version of Bro are you running? >>> >>> A way to improve your detection with only script changes could be >>> to >>> include {orig,resp}_ip_bytes in the criteria. The difference is >>> that >>> field counts total bytes of IP packets, not just payload data. >>> It?s >>> also more sensitive to packet loss, where {orig,resp}_bytes should >>> still work since it?s monitoring the TCP sequence space. >>> >>> - Jon >> >> Thanks Jon, >> >> I'm on 2.2 here. I'm going to start fiddling with the script >> now...thanks again for the help and response. >> >> James > > Ok...I've made the below modification: > > if ( rec$id$orig_h in Site::local_nets && > rec$id$resp_h in Site::local_nets && > rec$orig_bytes > (10 * rec$resp_bytes) && > rec$orig_bytes > (10 * rec$resp_ip_bytes) && > rec$orig_bytes >= 3145728 ) > > { > > This works in dev when sending a large file, so going to test this > out > in production....thank you. > > James Well shoot..still seeing these: 1396617228.413862 - x.x.x.x 51859 x.x.x.x 80 - - - tcp Exfil::Large_File_Upload Sent Bytes: 1029838798, Received Bytes: 0 - x.x.x.x x.x.x.x 80 - bro Notice::ACTION_LOG 3600.000000 F - - - - - 1396620769.215111 - x.x.x.x 53522 x.x.x.x 80 - - - tcp Exfil::Large_File_Upload Sent Bytes: 569497424, Received Bytes: 0 - x.x.x.x x.x.x.x 80 - bro Notice::ACTION_LOG 3600.000000 F - - - - - 2014-04-04T07:13:45-0600 CGZlLW2nctAkETr18c x.x.x.x 51859 x.x.x.x 80 tcp - 0.064546 1029838798 0 RSTOS0 T 0 SaR 2 92 1 52 (empty) 2014-04-04T08:12:46-0600 C7E5mt24LSdkhFVcI5 x.x.x.x 53522 x.x.x.x 80 tcp - 0.064791 569497424 0 RSTOS0 T 0 SaR 2 92 1 52 (empty) Should I just take the plunge to the latest git? Side question...how to get the bro id (CGZlLW2nctAkETr18c) in the notice file? I have: NOTICE([$note=Large_File_Upload, $id=rec$id, $identifier=cat(rec$uid), $msg=fmt("Sent Bytes: %s, Received Bytes: %s", rec$orig_bytes, rec$resp_bytes)]); Thank you. James From jsiwek at illinois.edu Fri Apr 4 08:50:13 2014 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 4 Apr 2014 15:50:13 +0000 Subject: [Bro] Large file ex-filtration revisited In-Reply-To: References: <333a080cf6ca1f57298e5f93d673fff9@localhost> Message-ID: On Apr 3, 2014, at 5:59 PM, James Lay wrote: > Ok...I've made the below modification: > > if ( rec$id$orig_h in Site::local_nets && > rec$id$resp_h in Site::local_nets && > rec$orig_bytes > (10 * rec$resp_bytes) && > rec$orig_bytes > (10 * rec$resp_ip_bytes) && > rec$orig_bytes >= 3145728 ) rec$orig_bytes > (10 * rec$resp_ip_bytes) is probably still going to be true if the calculation of orig_bytes was botched and incorrectly reported as too large. You probably meant rec$orig_ip_bytes > (10 * rec$resp_bytes) ? Another idea might be to just check for ?d? or ?D? in the history field to verify the value is sane ? absence of ?d? or ?D? means no payload data was seen, just control packets, so large values of {orig,resp}_bytes can?t possibly make sense. - Jon From jlay at slave-tothe-box.net Fri Apr 4 10:35:29 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 04 Apr 2014 11:35:29 -0600 Subject: [Bro] Impact of HUPping bro process Message-ID: All, Curious...I don't use broctl, choosing instead to run bro. What is the impact of HUPping the current running bro process? Will it re-read the local.bro for example? Or something else? Thank you. James From paul.veenstra at kahuna.nl Sat Apr 5 07:04:15 2014 From: paul.veenstra at kahuna.nl (Paul Veenstra) Date: Sat, 5 Apr 2014 16:04:15 +0200 Subject: [Bro] Writing logs in cef format Message-ID: Hi, Would it be possible to write out logs directly in CEF format? If yes, how to do it? Thanks, /Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140405/262a0a8a/attachment.html From daniel.guerra69 at gmail.com Tue Apr 8 13:52:49 2014 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Tue, 8 Apr 2014 22:52:49 +0200 Subject: [Bro] Detect openssl heartbleed Message-ID: I was wondering if it would be possible to detect the openssl heartbleed attack. In this site are details & some scripts to test this attackhttp://blog.fox-it.com/2014/04/08/openssl-heartbleed-bug-live-blog/ I found a suricata sollutionhttp://blog.inliniac.net/2014/04/08/detecting-openssl-heartbleed-with-suricata/ Maybe someone has such a script for bro ? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140408/f552c63c/attachment.html From bernhard at ICSI.Berkeley.EDU Tue Apr 8 14:05:02 2014 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Tue, 8 Apr 2014 14:05:02 -0700 Subject: [Bro] Detect openssl heartbleed In-Reply-To: References: Message-ID: <49D0360E-F7C7-4862-89C4-B245853ECCE4@icsi.berkeley.edu> We just wrote a blog post about this. See http://blog.bro.org/2014/04/detecting-heartbleed-bug-using-bro.html Bernhard On Apr 8, 2014, at 1:52 PM, Daniel Guerra wrote: > I was wondering if it would be possible to detect the openssl > heartbleed attack. > In this site are details & some scripts to test this attack > > http://blog.fox-it.com/2014/04/08/openssl-heartbleed-bug-live-blog/ > > > I found a suricata sollution > > http://blog.inliniac.net/2014/04/08/detecting-openssl-heartbleed-with-suricata/ > > > Maybe someone has such a script for bro ? > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From jep at g-c-i.net Wed Apr 9 05:17:17 2014 From: jep at g-c-i.net (Parker, Jonathan E.) Date: Wed, 9 Apr 2014 12:17:17 +0000 Subject: [Bro] Bro website offline? Message-ID: <36C06B73C5A9D845A5435F7AAE88802B8464B7EC@Mail10.Corporate.net> I am unable to get to the Bro website (bro.org). Something up? Jon P. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140409/b97116fb/attachment.html From mike.patterson at uwaterloo.ca Wed Apr 9 05:40:53 2014 From: mike.patterson at uwaterloo.ca (Mike Patterson) Date: Wed, 9 Apr 2014 12:40:53 +0000 Subject: [Bro] Bro website offline? In-Reply-To: <36C06B73C5A9D845A5435F7AAE88802B8464B7EC@Mail10.Corporate.net> References: <36C06B73C5A9D845A5435F7AAE88802B8464B7EC@Mail10.Corporate.net> Message-ID: <519FDF81-1E1F-4A63-8C20-B3BC555AF6B2@uwaterloo.ca> http://www.downforeveryoneorjustme.com/bro.org works fine here :-) Mike -- ... it is commonly the case with technologies that you can get the best insight about how they work by watching them fail. - Neal Stephenson, _In The Beginning Was The Command Line_ On Apr 9, 2014, at 8:17 AM, Parker, Jonathan E. wrote: I am unable to get to the Bro website (bro.org). Something up? Jon P. _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140409/cc84c71d/attachment.html From jep at g-c-i.net Wed Apr 9 05:46:50 2014 From: jep at g-c-i.net (Parker, Jonathan E.) Date: Wed, 9 Apr 2014 12:46:50 +0000 Subject: [Bro] Bro website offline? In-Reply-To: <36C06B73C5A9D845A5435F7AAE88802B8464B7EC@Mail10.Corporate.net> References: <36C06B73C5A9D845A5435F7AAE88802B8464B7EC@Mail10.Corporate.net> Message-ID: <36C06B73C5A9D845A5435F7AAE88802B8464C808@Mail10.Corporate.net> Yes, sorry, they https version is what I had bookmarked. I can get to http version. Thanks. ________________________________ From: Parker, Jonathan E. Sent: Wednesday, April 09, 2014 8:17 AM To: bro at bro.org Subject: Bro website offline? I am unable to get to the Bro website (bro.org). Something up? Jon P. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140409/31d336d0/attachment.html From bernhard at ICSI.Berkeley.EDU Wed Apr 9 05:57:28 2014 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Wed, 9 Apr 2014 05:57:28 -0700 Subject: [Bro] Bro website offline? In-Reply-To: <36C06B73C5A9D845A5435F7AAE88802B8464C808@Mail10.Corporate.net> References: <36C06B73C5A9D845A5435F7AAE88802B8464B7EC@Mail10.Corporate.net> <36C06B73C5A9D845A5435F7AAE88802B8464C808@Mail10.Corporate.net> Message-ID: <065565BB-DA8E-4519-AAE3-669A825B1981@icsi.berkeley.edu> On Apr 9, 2014, at 5:46 AM, Parker, Jonathan E. wrote: > Yes, sorry, they https version is what I had bookmarked. I can get to http version. Thanks. Sorry about that. We disabled https for the moment due to the heartbleed bug. It will be back in a bit though :) Bernhard From seth at icir.org Wed Apr 9 05:59:35 2014 From: seth at icir.org (Seth Hall) Date: Wed, 9 Apr 2014 08:59:35 -0400 Subject: [Bro] Bro website offline? In-Reply-To: <36C06B73C5A9D845A5435F7AAE88802B8464B7EC@Mail10.Corporate.net> References: <36C06B73C5A9D845A5435F7AAE88802B8464B7EC@Mail10.Corporate.net> Message-ID: <7D64C73C-4F17-4330-B149-908C1D888CF8@icir.org> On Apr 9, 2014, at 8:17 AM, Parker, Jonathan E. wrote: > I am unable to get to the Bro website (bro.org). Something up? SSL was disabled yesterday. Try visiting the non-SSL site. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From seth at icir.org Wed Apr 9 06:03:46 2014 From: seth at icir.org (Seth Hall) Date: Wed, 9 Apr 2014 09:03:46 -0400 Subject: [Bro] Writing logs in cef format In-Reply-To: References: Message-ID: On Apr 5, 2014, at 10:04 AM, Paul Veenstra wrote: > Would it be possible to write out logs directly in CEF format? > If yes, how to do it? Best would be to create a log writer. Alternately you might be able to do it with print statements but I?d really recommend doing the log writer instead. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140409/df5cac05/attachment.bin From seth at icir.org Wed Apr 9 06:06:12 2014 From: seth at icir.org (Seth Hall) Date: Wed, 9 Apr 2014 09:06:12 -0400 Subject: [Bro] Impact of HUPping bro process In-Reply-To: References: Message-ID: <1F9272A4-A34F-47D5-A14B-16F2DBA725AF@icir.org> On Apr 4, 2014, at 1:35 PM, James Lay wrote: > Curious...I don't use broctl, choosing instead to run bro. What is the > impact of HUPping the current running bro process? Nothing. Well, let me backpedal a bit. It will force Bro to checkpoint all persistent state to disk but that feature of Bro isn?t used very much (and we don?t recommend using it much either due to various problems with it). If you want to update the scripts that Bro is running you have to restart Bro in almost all cases. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140409/e72214e8/attachment.bin From jlay at slave-tothe-box.net Wed Apr 9 06:11:52 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Wed, 09 Apr 2014 07:11:52 -0600 Subject: [Bro] Impact of HUPping bro process In-Reply-To: <1F9272A4-A34F-47D5-A14B-16F2DBA725AF@icir.org> References: <1F9272A4-A34F-47D5-A14B-16F2DBA725AF@icir.org> Message-ID: <1397049112.2388.18.camel@JamesiMac> On Wed, 2014-04-09 at 09:06 -0400, Seth Hall wrote: > On Apr 4, 2014, at 1:35 PM, James Lay wrote: > > > Curious...I don't use broctl, choosing instead to run bro. What is the > > impact of HUPping the current running bro process? > > Nothing. Well, let me backpedal a bit. It will force Bro to checkpoint all persistent state to disk but that feature of Bro isn?t used very much (and we don?t recommend using it much either due to various problems with it). > > If you want to update the scripts that Bro is running you have to restart Bro in almost all cases. > > .Seth > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > Good info..thanks Seth. James -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140409/814cd2cc/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140409/814cd2cc/attachment.bin From seth at icir.org Wed Apr 9 06:34:38 2014 From: seth at icir.org (Seth Hall) Date: Wed, 9 Apr 2014 09:34:38 -0400 Subject: [Bro] Log Stream Types In-Reply-To: References: Message-ID: <21A5E001-AEA5-4F85-BDAF-2467C1A4941B@icir.org> On Apr 2, 2014, at 10:00 PM, anthony kasza wrote: > I'm hoping someone could explain why > %prefix%bro/share/bro/base/frameworks/logging/main.bro (from an > installation) defines a Log::Stream type as a record of two any types > but bro/src/logging/Manager.cc (line 335 from Github) seems to enforce > Log::Stream types to consist of an event type. That?s a hack. :) It?s because internally, the $columns field is a TypeType type which allows us to specify a type as a value (I know, kind of weird). $ev is declared as any at script land because the type of an event includes the full parameter list but most events being provided to that field are of different types because they carry different record types in their parameter lists. Those hacks have bugged us (me at least!) for quite a while and if there is anything that is constant in our community, it?s that change is constant and we?ll probably be back around to work on this issue again before long. :) > I'm curious to see if > it is possible to take immediate action upon a log line being ready > with a function or hook instead of having to wait for an event to be > handled. Typically when writing scripts that have specific requirements like it sounds like yours has, I don?t recommend that people hang off of the logging events. You are always going to run into problems like you are here. Find the event that you really want to hang your functionality off of and do that. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140409/6ff5e102/attachment.bin From seth at icir.org Wed Apr 9 06:36:19 2014 From: seth at icir.org (Seth Hall) Date: Wed, 9 Apr 2014 09:36:19 -0400 Subject: [Bro] Broctl appears to be creating invalid gz files In-Reply-To: <1395920020.2597.1.camel@JamesiMac> References: <1395920020.2597.1.camel@JamesiMac> Message-ID: <12772F24-5C1D-4DFB-83E2-EE6933443473@icir.org> On Mar 27, 2014, at 7:33 AM, James Lay wrote: > Topic says it...was going to look at something this morning and: > > gzip: syslog.00:00:00-00:00:00.log.gz: invalid compressed data--format violated > > gzip: weird.00:00:00-00:00:00.log.gz: decompression OK, trailing garbage ignored Is this still happening for you? It seems really weird because the time range that file seems to encompass is zero seconds. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140409/c8a3fc0f/attachment.bin From anthony.kasza at gmail.com Wed Apr 9 10:10:11 2014 From: anthony.kasza at gmail.com (anthony kasza) Date: Wed, 9 Apr 2014 10:10:11 -0700 Subject: [Bro] Log Stream Types In-Reply-To: <21A5E001-AEA5-4F85-BDAF-2467C1A4941B@icir.org> References: <21A5E001-AEA5-4F85-BDAF-2467C1A4941B@icir.org> Message-ID: Ah! The typing makes sense now. You're right about the specific requirements, too. I'm looking for a way to feed log lines to a process similar to a Python generator. I've tried the Python broccoli bindings but wasn't satisfied with type conversions. I'll keep looking and post here if I find anything worth while. Thanks for the insight, Seth! -AK On Apr 9, 2014 6:34 AM, "Seth Hall" wrote: > > On Apr 2, 2014, at 10:00 PM, anthony kasza > wrote: > > > I'm hoping someone could explain why > > %prefix%bro/share/bro/base/frameworks/logging/main.bro (from an > > installation) defines a Log::Stream type as a record of two any types > > but bro/src/logging/Manager.cc (line 335 from Github) seems to enforce > > Log::Stream types to consist of an event type. > > That's a hack. :) It's because internally, the $columns field is a > TypeType type which allows us to specify a type as a value (I know, kind of > weird). $ev is declared as any at script land because the type of an event > includes the full parameter list but most events being provided to that > field are of different types because they carry different record types in > their parameter lists. > > Those hacks have bugged us (me at least!) for quite a while and if there > is anything that is constant in our community, it's that change is constant > and we'll probably be back around to work on this issue again before long. > :) > > > I'm curious to see if > > it is possible to take immediate action upon a log line being ready > > with a function or hook instead of having to wait for an event to be > > handled. > > Typically when writing scripts that have specific requirements like it > sounds like yours has, I don't recommend that people hang off of the > logging events. You are always going to run into problems like you are > here. Find the event that you really want to hang your functionality off > of and do that. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140409/45565122/attachment.html From jlay at slave-tothe-box.net Thu Apr 10 05:24:44 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Thu, 10 Apr 2014 06:24:44 -0600 Subject: [Bro] Detecting heartbleed activity Message-ID: <1397132684.2460.7.camel@JamesiMac> So...I'd like to be able to see if any heartbleed activity was happening before everyone knew about it. I'm thinking I'd see this in the conn.log with data leaving the server. Any thoughts or pointers we could use to check? Thanks all. James -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/95136359/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/95136359/attachment.bin From mkhan04 at gmail.com Thu Apr 10 05:37:11 2014 From: mkhan04 at gmail.com (M K) Date: Thu, 10 Apr 2014 08:37:11 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: <1397132684.2460.7.camel@JamesiMac> References: <1397132684.2460.7.camel@JamesiMac> Message-ID: Heartbleed is extremely difficult to detect without doing an inspection of the TLS traffic. The biggest indicator is that a heartbeat message response is larger than the request. But with regular https traffic, as an example, you'll see similar looking payloads, just as a matter of course. I.e., a GET request which is X kB will garner a response that is Y kB (where Y is greater than X). If you take a look at the branch that the bro folks released to detect heartbleed, they specifically inspect the heartbeat message if a cipher spec hasn't been chosen. And if a cipher has been chosen, compare the payload sizes of the heartbeat request/response (TLS Record information is in the clear, even if the actual record is encrypted). I'm not sure you can come up with a reliable and simple means of finding it through the information in the connection log. But I'd like to be wrong in this instance. If anybody disagrees with me, I'd really really like to know as it'd would help me (and a bunch of folks) out. On Thu, Apr 10, 2014 at 8:24 AM, James Lay wrote: > So...I'd like to be able to see if any heartbleed activity was happening > before everyone knew about it. I'm thinking I'd see this in the conn.log > with data leaving the server. Any thoughts or pointers we could use to > check? Thanks all. > > James > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/e5f02d65/attachment.html From seth at icir.org Thu Apr 10 06:01:42 2014 From: seth at icir.org (Seth Hall) Date: Thu, 10 Apr 2014 09:01:42 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <1397132684.2460.7.camel@JamesiMac> Message-ID: On Apr 10, 2014, at 8:37 AM, M K wrote: > I'm not sure you can come up with a reliable and simple means of finding it through the information in the connection log. You?re correct, I don?t believe that the attack is apparent in any Bro logs in our releases. The only way at the moment to detect heartbleed with Bro is to use Bernhard?s branch (although I?d love to be proven wrong if someone figures out a way to catch it!) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/805b30a3/attachment.bin From jlanders at paymetric.com Thu Apr 10 06:33:40 2014 From: jlanders at paymetric.com (John Landers) Date: Thu, 10 Apr 2014 08:33:40 -0500 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <1397132684.2460.7.camel@JamesiMac> Message-ID: <199F5CD38D5E984990F92D2CDE955F664C664BD07D@34093-MBX-C12.mex07a.mlsrvr.com> >From using the proof-of-concept code on my servers to validate before/after patching the vulnerability, I found the following common criteria in the Bro conn log: orig_bytes=241 protocol=tcp resp_port=443 Using this criteria in a search, and grouping by orig_ip, I was able to find all of the known attempts in my log from April 8. When I searched through historical logs, I had one false positive using the search but I don't know how many false negatives this produced. Whether or not this search would work is completely dependent on your standard SSL traffic. You could try searching for orig_bytes < 250 and resp_bytes > 30000 but I suspect that won't work if your organization is offering downloads or rich web content... John Landers -----Original Message----- From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Seth Hall Sent: Thursday, April 10, 2014 8:02 AM To: M K Cc: Bro-IDS Subject: Re: [Bro] Detecting heartbleed activity On Apr 10, 2014, at 8:37 AM, M K wrote: > I'm not sure you can come up with a reliable and simple means of finding it through the information in the connection log. You're correct, I don't believe that the attack is apparent in any Bro logs in our releases. The only way at the moment to detect heartbleed with Bro is to use Bernhard's branch (although I'd love to be proven wrong if someone figures out a way to catch it!) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From bernhard at ICSI.Berkeley.EDU Thu Apr 10 06:45:25 2014 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Thu, 10 Apr 2014 06:45:25 -0700 Subject: [Bro] Detecting heartbleed activity In-Reply-To: <199F5CD38D5E984990F92D2CDE955F664C664BD07D@34093-MBX-C12.mex07a.mlsrvr.com> References: <1397132684.2460.7.camel@JamesiMac> <199F5CD38D5E984990F92D2CDE955F664C664BD07D@34093-MBX-C12.mex07a.mlsrvr.com> Message-ID: Note that there is an almost 100% chance that this (and similar) approaches will not work on any attacks that might have happened before the bug was publicly released. You basically just fingerprint one of the (more common) tools. There is next to no chance that an earlier attack (or even someone using a different tool) will exhibit the same characteristics. All of them will send the TLS records in a slightly different way, perhaps enable encryption before sending them, or ? perhaps even send a https request before the heartbeat to make it less obvious in logs. Bernhard On Apr 10, 2014, at 6:33 AM, John Landers wrote: >> From using the proof-of-concept code on my servers to validate before/after patching the vulnerability, I found the following common criteria in the Bro conn log: > > orig_bytes=241 > protocol=tcp > resp_port=443 > > Using this criteria in a search, and grouping by orig_ip, I was able to find all of the known attempts in my log from April 8. When I searched through historical logs, I had one false positive using the search but I don't know how many false negatives this produced. > > Whether or not this search would work is completely dependent on your standard SSL traffic. You could try searching for orig_bytes < 250 and resp_bytes > 30000 but I suspect that won't work if your organization is offering downloads or rich web content... > > > > John Landers > > -----Original Message----- > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Seth Hall > Sent: Thursday, April 10, 2014 8:02 AM > To: M K > Cc: Bro-IDS > Subject: Re: [Bro] Detecting heartbleed activity > > > On Apr 10, 2014, at 8:37 AM, M K wrote: > >> I'm not sure you can come up with a reliable and simple means of finding it through the information in the connection log. > > You're correct, I don't believe that the attack is apparent in any Bro logs in our releases. The only way at the moment to detect heartbleed with Bro is to use Bernhard's branch (although I'd love to be proven wrong if someone figures out a way to catch it!) > > .Seth > > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > From jlanders at paymetric.com Thu Apr 10 06:50:38 2014 From: jlanders at paymetric.com (John Landers) Date: Thu, 10 Apr 2014 08:50:38 -0500 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <1397132684.2460.7.camel@JamesiMac> <199F5CD38D5E984990F92D2CDE955F664C664BD07D@34093-MBX-C12.mex07a.mlsrvr.com> Message-ID: <199F5CD38D5E984990F92D2CDE955F664C664BD087@34093-MBX-C12.mex07a.mlsrvr.com> Yep. It's pretty much an impossible task. John Landers -----Original Message----- From: Bernhard Amann [mailto:bernhard at ICSI.Berkeley.EDU] Sent: Thursday, April 10, 2014 8:45 AM To: John Landers Cc: Seth Hall; M K; Bro-IDS Subject: Re: [Bro] Detecting heartbleed activity Note that there is an almost 100% chance that this (and similar) approaches will not work on any attacks that might have happened before the bug was publicly released. You basically just fingerprint one of the (more common) tools. There is next to no chance that an earlier attack (or even someone using a different tool) will exhibit the same characteristics. All of them will send the TLS records in a slightly different way, perhaps enable encryption before sending them, or - perhaps even send a https request before the heartbeat to make it less obvious in logs. Bernhard On Apr 10, 2014, at 6:33 AM, John Landers wrote: >> From using the proof-of-concept code on my servers to validate before/after patching the vulnerability, I found the following common criteria in the Bro conn log: > > orig_bytes=241 > protocol=tcp > resp_port=443 > > Using this criteria in a search, and grouping by orig_ip, I was able to find all of the known attempts in my log from April 8. When I searched through historical logs, I had one false positive using the search but I don't know how many false negatives this produced. > > Whether or not this search would work is completely dependent on your standard SSL traffic. You could try searching for orig_bytes < 250 and resp_bytes > 30000 but I suspect that won't work if your organization is offering downloads or rich web content... > > > > John Landers > > -----Original Message----- > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Seth Hall > Sent: Thursday, April 10, 2014 8:02 AM > To: M K > Cc: Bro-IDS > Subject: Re: [Bro] Detecting heartbleed activity > > > On Apr 10, 2014, at 8:37 AM, M K wrote: > >> I'm not sure you can come up with a reliable and simple means of finding it through the information in the connection log. > > You're correct, I don't believe that the attack is apparent in any Bro logs in our releases. The only way at the moment to detect heartbleed with Bro is to use Bernhard's branch (although I'd love to be proven wrong if someone figures out a way to catch it!) > > .Seth > > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > From daniel.guerra69 at gmail.com Thu Apr 10 07:00:25 2014 From: daniel.guerra69 at gmail.com (daniel.guerra69) Date: Thu, 10 Apr 2014 16:00:25 +0200 Subject: [Bro] Detecting heartbleed activity In-Reply-To: <199F5CD38D5E984990F92D2CDE955F664C664BD07D@34093-MBX-C12.mex07a.mlsrvr.com> References: <1397132684.2460.7.camel@JamesiMac> <199F5CD38D5E984990F92D2CDE955F664C664BD07D@34093-MBX-C12.mex07a.mlsrvr.com> Message-ID: <5346A3F9.70701@gmail.com> Check this blog http://blog.bro.org/2014/04/detecting-heartbleed-bug-using-bro.html use this git git clone --recursive git://git.bro.org/bro -b topic/bernhard/heartbeat From jmellander at lbl.gov Thu Apr 10 09:39:23 2014 From: jmellander at lbl.gov (Jim Mellander) Date: Thu, 10 Apr 2014 09:39:23 -0700 Subject: [Bro] Detecting heartbleed activity In-Reply-To: <5346A3F9.70701@gmail.com> References: <1397132684.2460.7.camel@JamesiMac> <199F5CD38D5E984990F92D2CDE955F664C664BD07D@34093-MBX-C12.mex07a.mlsrvr.com> <5346A3F9.70701@gmail.com> Message-ID: As a data point, NERSC ran our timemachine data (goes back several months) thru Bernhard's code, and saw no SSL Heartbeat attempts until the morning of Apr 8. On Thu, Apr 10, 2014 at 7:00 AM, daniel.guerra69 wrote: > Check this blog > > http://blog.bro.org/2014/04/detecting-heartbleed-bug-using-bro.html > > > use this git > > git clone --recursive git://git.bro.org/bro -b topic/bernhard/heartbeat > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/56be4541/attachment.html From jlay at slave-tothe-box.net Thu Apr 10 09:51:00 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Thu, 10 Apr 2014 10:51:00 -0600 Subject: [Bro] Detecting heartbleed activity In-Reply-To: <1397132684.2460.7.camel@JamesiMac> References: <1397132684.2460.7.camel@JamesiMac> Message-ID: On 2014-04-10 06:24, James Lay wrote: > So...I'd like to be able to see if any heartbleed activity was > happening before everyone knew about it. I'm thinking I'd see this in > the conn.log with data leaving the server. Any thoughts or pointers > we > could use to check? Thanks all. > > James Thanks for the feedback all..very helpful. James From john.h.hoyt at gmail.com Thu Apr 10 11:12:28 2014 From: john.h.hoyt at gmail.com (John Hoyt) Date: Thu, 10 Apr 2014 14:12:28 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <1397132684.2460.7.camel@JamesiMac> Message-ID: I'm attempting to add an email alert for these, but I'm getting an error. This is my first time attempting this, so I may have something wrong with syntax. Here is what I've added to local.bro. hook Notice::policy(n: Notice::Info) { if ( n$note == SSL::SSL_Heartbeat_Attack_Success ) add n$actions[Notice::ACTION_EMAIL]; } Here is the error: error in /bro/share/bro/site/local.bro, line 96: unknown identifier SSL::SSL_Heartbeat_Attack_Success, at or near "SSL::SSL_Heartbeat_Attack_Success" -John On Thu, Apr 10, 2014 at 12:51 PM, James Lay wrote: > On 2014-04-10 06:24, James Lay wrote: > > So...I'd like to be able to see if any heartbleed activity was > > happening before everyone knew about it. I'm thinking I'd see this in > > the conn.log with data leaving the server. Any thoughts or pointers > > we > > could use to check? Thanks all. > > > > James > > Thanks for the feedback all..very helpful. > > James > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/af89d2c7/attachment.html From seth at icir.org Thu Apr 10 11:19:19 2014 From: seth at icir.org (Seth Hall) Date: Thu, 10 Apr 2014 14:19:19 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <1397132684.2460.7.camel@JamesiMac> Message-ID: <3C8402FD-952F-47D9-B490-E31D55D15FAC@icir.org> On Apr 10, 2014, at 2:12 PM, John Hoyt wrote: > error in /bro/share/bro/site/local.bro, line 96: unknown identifier SSL::SSL_Heartbeat_Attack_Success, at or near "SSL::SSL_Heartbeat_Attack_Success" Heartbleed::SSL_Heartbeat_Attack_Success Easy mistake. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/6d7801c4/attachment.bin From JAzoff at albany.edu Thu Apr 10 11:20:26 2014 From: JAzoff at albany.edu (Justin Azoff) Date: Thu, 10 Apr 2014 14:20:26 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <1397132684.2460.7.camel@JamesiMac> Message-ID: <20140410182026.GC27258@datacomm.albany.edu> On Thu, Apr 10, 2014 at 02:12:28PM -0400, John Hoyt wrote: > I'm attempting to add an email alert for these, but I'm getting an error. This > is my first time attempting this, so I may have something wrong with syntax. > > Here is what I've added to local.bro. > > > hook Notice::policy(n: Notice::Info) > > { > > if ( n$note == SSL::SSL_Heartbeat_Attack_Success ) > > add n$actions[Notice::ACTION_EMAIL]; > > } The heartbleed module is in the Heartbleed namespace so the notice is Heartbleed::SSL_Heartbeat_Attack_Success Also, there is a helper for that sort of thing, you can simply: redef Notice::emailed_types += { Heartbleed::SSL_Heartbeat_Attack_Success, }; -- -- Justin Azoff From john.h.hoyt at gmail.com Thu Apr 10 11:32:29 2014 From: john.h.hoyt at gmail.com (John Hoyt) Date: Thu, 10 Apr 2014 14:32:29 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: <20140410182026.GC27258@datacomm.albany.edu> References: <1397132684.2460.7.camel@JamesiMac> <20140410182026.GC27258@datacomm.albany.edu> Message-ID: Thanks Justin, I changed it to what you listed, but I'm still getting the following error: error in /bro/share/bro/site/local.bro, line 95: unknown identifier Heartbleed::SSL_Heartbeat_Attack_Success, at or near "Heartbleed::SSL_Heartbeat_Attack_Success" On Thu, Apr 10, 2014 at 2:20 PM, Justin Azoff wrote: > On Thu, Apr 10, 2014 at 02:12:28PM -0400, John Hoyt wrote: > > I'm attempting to add an email alert for these, but I'm getting an > error. This > > is my first time attempting this, so I may have something wrong with > syntax. > > > > Here is what I've added to local.bro. > > > > > > hook Notice::policy(n: Notice::Info) > > > > { > > > > if ( n$note == SSL::SSL_Heartbeat_Attack_Success ) > > > > add n$actions[Notice::ACTION_EMAIL]; > > > > } > > The heartbleed module is in the Heartbleed namespace so the notice is > > Heartbleed::SSL_Heartbeat_Attack_Success > > Also, there is a helper for that sort of thing, you can simply: > > redef Notice::emailed_types += { > Heartbleed::SSL_Heartbeat_Attack_Success, > }; > > -- > -- Justin Azoff > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/10ce25c8/attachment.html From JAzoff at albany.edu Thu Apr 10 11:40:53 2014 From: JAzoff at albany.edu (Justin Azoff) Date: Thu, 10 Apr 2014 14:40:53 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <1397132684.2460.7.camel@JamesiMac> <20140410182026.GC27258@datacomm.albany.edu> Message-ID: <20140410184053.GD27258@datacomm.albany.edu> On Thu, Apr 10, 2014 at 02:32:29PM -0400, John Hoyt wrote: > Thanks Justin, > > I changed it to what you listed, but I'm still getting the following error: > > error in /bro/share/bro/site/local.bro, line 95: unknown identifier > Heartbleed::SSL_Heartbeat_Attack_Success, at or near > "Heartbleed::SSL_Heartbeat_Attack_Success" Do you have @load policy/protocols/ssl/heartbleed at some point before that? -- -- Justin Azoff From bernhard at ICSI.Berkeley.EDU Thu Apr 10 11:42:20 2014 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Thu, 10 Apr 2014 11:42:20 -0700 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <1397132684.2460.7.camel@JamesiMac> <20140410182026.GC27258@datacomm.albany.edu> Message-ID: <381020D0-79A7-4F81-AD80-4A75B4A3B735@icsi.berkeley.edu> Did you add that after the line that @loads the heartbleed script? On Apr 10, 2014, at 11:32 AM, John Hoyt wrote: > Thanks Justin, > > I changed it to what you listed, but I'm still getting the following error: > > error in /bro/share/bro/site/local.bro, line 95: unknown identifier Heartbleed::SSL_Heartbeat_Attack_Success, at or near "Heartbleed::SSL_Heartbeat_Attack_Success" > > > On Thu, Apr 10, 2014 at 2:20 PM, Justin Azoff wrote: > On Thu, Apr 10, 2014 at 02:12:28PM -0400, John Hoyt wrote: > > I'm attempting to add an email alert for these, but I'm getting an error. This > > is my first time attempting this, so I may have something wrong with syntax. > > > > Here is what I've added to local.bro. > > > > > > hook Notice::policy(n: Notice::Info) > > > > { > > > > if ( n$note == SSL::SSL_Heartbeat_Attack_Success ) > > > > add n$actions[Notice::ACTION_EMAIL]; > > > > } > > The heartbleed module is in the Heartbleed namespace so the notice is > > Heartbleed::SSL_Heartbeat_Attack_Success > > Also, there is a helper for that sort of thing, you can simply: > > redef Notice::emailed_types += { > Heartbleed::SSL_Heartbeat_Attack_Success, > }; > > -- > -- Justin Azoff > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From john.h.hoyt at gmail.com Thu Apr 10 12:03:24 2014 From: john.h.hoyt at gmail.com (John Hoyt) Date: Thu, 10 Apr 2014 15:03:24 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: <381020D0-79A7-4F81-AD80-4A75B4A3B735@icsi.berkeley.edu> References: <1397132684.2460.7.camel@JamesiMac> <20140410182026.GC27258@datacomm.albany.edu> <381020D0-79A7-4F81-AD80-4A75B4A3B735@icsi.berkeley.edu> Message-ID: That did it. :-) Thanks! On Thu, Apr 10, 2014 at 2:42 PM, Bernhard Amann wrote: > Did you add that after the line that @loads the heartbleed script? > > On Apr 10, 2014, at 11:32 AM, John Hoyt wrote: > > > Thanks Justin, > > > > I changed it to what you listed, but I'm still getting the following > error: > > > > error in /bro/share/bro/site/local.bro, line 95: unknown identifier > Heartbleed::SSL_Heartbeat_Attack_Success, at or near > "Heartbleed::SSL_Heartbeat_Attack_Success" > > > > > > On Thu, Apr 10, 2014 at 2:20 PM, Justin Azoff wrote: > > On Thu, Apr 10, 2014 at 02:12:28PM -0400, John Hoyt wrote: > > > I'm attempting to add an email alert for these, but I'm getting an > error. This > > > is my first time attempting this, so I may have something wrong with > syntax. > > > > > > Here is what I've added to local.bro. > > > > > > > > > hook Notice::policy(n: Notice::Info) > > > > > > { > > > > > > if ( n$note == SSL::SSL_Heartbeat_Attack_Success ) > > > > > > add n$actions[Notice::ACTION_EMAIL]; > > > > > > } > > > > The heartbleed module is in the Heartbleed namespace so the notice is > > > > Heartbleed::SSL_Heartbeat_Attack_Success > > > > Also, there is a helper for that sort of thing, you can simply: > > > > redef Notice::emailed_types += { > > Heartbleed::SSL_Heartbeat_Attack_Success, > > }; > > > > -- > > -- Justin Azoff > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/b6d03dd4/attachment.html From gary at doit.wisc.edu Thu Apr 10 13:11:19 2014 From: gary at doit.wisc.edu (Gary Faulkner) Date: Thu, 10 Apr 2014 15:11:19 -0500 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <1397132684.2460.7.camel@JamesiMac> <20140410182026.GC27258@datacomm.albany.edu> <381020D0-79A7-4F81-AD80-4A75B4A3B735@icsi.berkeley.edu> Message-ID: <5346FAE7.90502@doit.wisc.edu> Just curious how the heartbleed Bro build is running for folks. Any problems? On 4/10/2014 2:03 PM, John Hoyt wrote: > That did it. :-) > > Thanks! > > > On Thu, Apr 10, 2014 at 2:42 PM, Bernhard Amann > > wrote: > > Did you add that after the line that @loads the heartbleed script? > > On Apr 10, 2014, at 11:32 AM, John Hoyt > wrote: > > > Thanks Justin, > > > > I changed it to what you listed, but I'm still getting the > following error: > > > > error in /bro/share/bro/site/local.bro, line 95: unknown > identifier Heartbleed::SSL_Heartbeat_Attack_Success, at or near > "Heartbleed::SSL_Heartbeat_Attack_Success" > > > > > > On Thu, Apr 10, 2014 at 2:20 PM, Justin Azoff > wrote: > > On Thu, Apr 10, 2014 at 02:12:28PM -0400, John Hoyt wrote: > > > I'm attempting to add an email alert for these, but I'm > getting an error. This > > > is my first time attempting this, so I may have something > wrong with syntax. > > > > > > Here is what I've added to local.bro. > > > > > > > > > hook Notice::policy(n: Notice::Info) > > > > > > { > > > > > > if ( n$note == SSL::SSL_Heartbeat_Attack_Success ) > > > > > > add n$actions[Notice::ACTION_EMAIL]; > > > > > > } > > > > The heartbleed module is in the Heartbleed namespace so the > notice is > > > > Heartbleed::SSL_Heartbeat_Attack_Success > > > > Also, there is a helper for that sort of thing, you can simply: > > > > redef Notice::emailed_types += { > > Heartbleed::SSL_Heartbeat_Attack_Success, > > }; > > > > -- > > -- Justin Azoff > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/0b51acfc/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6257 bytes Desc: S/MIME Cryptographic Signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/0b51acfc/attachment.bin From john.h.hoyt at gmail.com Thu Apr 10 13:29:27 2014 From: john.h.hoyt at gmail.com (John Hoyt) Date: Thu, 10 Apr 2014 16:29:27 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: <5346FAE7.90502@doit.wisc.edu> References: <1397132684.2460.7.camel@JamesiMac> <20140410182026.GC27258@datacomm.albany.edu> <381020D0-79A7-4F81-AD80-4A75B4A3B735@icsi.berkeley.edu> <5346FAE7.90502@doit.wisc.edu> Message-ID: After implementing it just a little while ago, I've had eight notifications. Half of which look to be vulnerable servers. So, I'd say so far good. -John On Thu, Apr 10, 2014 at 4:11 PM, Gary Faulkner wrote: > Just curious how the heartbleed Bro build is running for folks. Any > problems? > > On 4/10/2014 2:03 PM, John Hoyt wrote: > > That did it. :-) > > Thanks! > > > On Thu, Apr 10, 2014 at 2:42 PM, Bernhard Amann < > bernhard at icsi.berkeley.edu> wrote: > >> Did you add that after the line that @loads the heartbleed script? >> >> On Apr 10, 2014, at 11:32 AM, John Hoyt wrote: >> >> > Thanks Justin, >> > >> > I changed it to what you listed, but I'm still getting the following >> error: >> > >> > error in /bro/share/bro/site/local.bro, line 95: unknown identifier >> Heartbleed::SSL_Heartbeat_Attack_Success, at or near >> "Heartbleed::SSL_Heartbeat_Attack_Success" >> > >> > >> > On Thu, Apr 10, 2014 at 2:20 PM, Justin Azoff >> wrote: >> > On Thu, Apr 10, 2014 at 02:12:28PM -0400, John Hoyt wrote: >> > > I'm attempting to add an email alert for these, but I'm getting an >> error. This >> > > is my first time attempting this, so I may have something wrong with >> syntax. >> > > >> > > Here is what I've added to local.bro. >> > > >> > > >> > > hook Notice::policy(n: Notice::Info) >> > > >> > > { >> > > >> > > if ( n$note == SSL::SSL_Heartbeat_Attack_Success ) >> > > >> > > add n$actions[Notice::ACTION_EMAIL]; >> > > >> > > } >> > >> > The heartbleed module is in the Heartbleed namespace so the notice is >> > >> > Heartbleed::SSL_Heartbeat_Attack_Success >> > >> > Also, there is a helper for that sort of thing, you can simply: >> > >> > redef Notice::emailed_types += { >> > Heartbleed::SSL_Heartbeat_Attack_Success, >> > }; >> > >> > -- >> > -- Justin Azoff >> > >> > _______________________________________________ >> > Bro mailing list >> > bro at bro-ids.org >> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> > > > _______________________________________________ > Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/aac47d7d/attachment.html From jbabio at me.com Thu Apr 10 15:22:49 2014 From: jbabio at me.com (John Babio) Date: Thu, 10 Apr 2014 22:22:49 +0000 (GMT) Subject: [Bro] Detecting heartbleed activity In-Reply-To: Message-ID: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> Do you have a github with this script in it? Thanks?! On Apr 10, 2014, at 04:29 PM, John Hoyt wrote: > After implementing it just a little while ago, I've had eight notifications. Half of which look to be vulnerable servers. > > So, I'd say so far good. > > -John > > > On Thu, Apr 10, 2014 at 4:11 PM, Gary Faulkner wrote: > > Just curious how the heartbleed Bro build is running for folks. Any problems? > > On 4/10/2014 2:03 PM, John Hoyt wrote: >> That did it. :-) >> >> Thanks! >> >> >> On Thu, Apr 10, 2014 at 2:42 PM, Bernhard Amann wrote: >> >> Did you add that after the line that @loads the heartbleed script? >> >> On Apr 10, 2014, at 11:32 AM, John Hoyt wrote: >> >> > Thanks Justin, >> > >> > I changed it to what you listed, but I'm still getting the following error: >> > >> > error in /bro/share/bro/site/local.bro, line 95: unknown identifier Heartbleed::SSL_Heartbeat_Attack_Success, at or near "Heartbleed::SSL_Heartbeat_Attack_Success" >> > >> > >> > On Thu, Apr 10, 2014 at 2:20 PM, Justin Azoff wrote: >> > On Thu, Apr 10, 2014 at 02:12:28PM -0400, John Hoyt wrote: >> > > I'm attempting to add an email alert for these, but I'm getting an error. This >> > > is my first time attempting this, so I may have something wrong with syntax. >> > > >> > > Here is what I've added to local.bro. >> > > >> > > >> > > hook Notice::policy(n: Notice::Info) >> > > >> > > { >> > > >> > > if ( n$note == SSL::SSL_Heartbeat_Attack_Success ) >> > > >> > > add n$actions[Notice::ACTION_EMAIL]; >> > > >> > > } >> > >> > The heartbleed module is in the Heartbleed namespace so the notice is >> > >> > Heartbleed::SSL_Heartbeat_Attack_Success >> > >> > Also, there is a helper for that sort of thing, you can simply: >> > >> > redef Notice::emailed_types += { >> > Heartbleed::SSL_Heartbeat_Attack_Success, >> > }; >> > >> > -- >> > -- Justin Azoff >> > >> > _______________________________________________ >> > Bro mailing list >> > bro at bro-ids.org >> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/0567d65a/attachment.html From bernhard at ICSI.Berkeley.EDU Thu Apr 10 15:46:00 2014 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Thu, 10 Apr 2014 15:46:00 -0700 Subject: [Bro] Detecting heartbleed activity In-Reply-To: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> References: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> Message-ID: https://github.com/bro/bro/tree/topic/bernhard/heartbeat - the script is in scripts/policy/protocols/ssl/heartbleed.bro Make sure to use the linked branch (topic/bernhard/heartbeat) Bernhard On Apr 10, 2014, at 3:22 PM, John Babio wrote: > Do you have a github with this script in it? Thanks! > > On Apr 10, 2014, at 04:29 PM, John Hoyt wrote: > >> After implementing it just a little while ago, I've had eight notifications. Half of which look to be vulnerable servers. >> >> So, I'd say so far good. >> >> -John >> >> >> On Thu, Apr 10, 2014 at 4:11 PM, Gary Faulkner wrote: >> Just curious how the heartbleed Bro build is running for folks. Any problems? >> >> On 4/10/2014 2:03 PM, John Hoyt wrote: >>> That did it. :-) >>> >>> Thanks! >>> >>> >>> On Thu, Apr 10, 2014 at 2:42 PM, Bernhard Amann wrote: >>> Did you add that after the line that @loads the heartbleed script? >>> >>> On Apr 10, 2014, at 11:32 AM, John Hoyt wrote: >>> >>>> Thanks Justin, >>>> >>>> I changed it to what you listed, but I'm still getting the following error: >>>> >>>> error in /bro/share/bro/site/local.bro, line 95: unknown identifier Heartbleed::SSL_Heartbeat_Attack_Success, at or near "Heartbleed::SSL_Heartbeat_Attack_Success" >>>> >>>> >>>> On Thu, Apr 10, 2014 at 2:20 PM, Justin Azoff wrote: >>>> On Thu, Apr 10, 2014 at 02:12:28PM -0400, John Hoyt wrote: >>>>> I'm attempting to add an email alert for these, but I'm getting an error. This >>>>> is my first time attempting this, so I may have something wrong with syntax. >>>>> >>>>> Here is what I've added to local.bro. >>>>> >>>>> >>>>> hook Notice::policy(n: Notice::Info) >>>>> >>>>> { >>>>> >>>>> if ( n$note == SSL::SSL_Heartbeat_Attack_Success ) >>>>> >>>>> add n$actions[Notice::ACTION_EMAIL]; >>>>> >>>>> } >>>> >>>> The heartbleed module is in the Heartbleed namespace so the notice is >>>> >>>> Heartbleed::SSL_Heartbeat_Attack_Success >>>> >>>> Also, there is a helper for that sort of thing, you can simply: >>>> >>>> redef Notice::emailed_types += { >>>> Heartbleed::SSL_Heartbeat_Attack_Success, >>>> }; >>>> >>>> -- >>>> -- Justin Azoff >>>> >>>> _______________________________________________ >>>> Bro mailing list >>>> bro at bro-ids.org >>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> >>> >>> >>> >>> _______________________________________________ >>> Bro mailing list >>> >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From alexwis at gmail.com Thu Apr 10 16:19:02 2014 From: alexwis at gmail.com (Alex Waher) Date: Thu, 10 Apr 2014 16:19:02 -0700 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> Message-ID: I had the heartbeat branch running for a few hours (very successfully detecting activity!) and noticed it eventually had the manager worker consuming +70gb of memory. Wasn't sure if the leak was from the heartbeat capability itself or something else along the current git repo.. ymmv! On Thu, Apr 10, 2014 at 3:46 PM, Bernhard Amann wrote: > https://github.com/bro/bro/tree/topic/bernhard/heartbeat - the script is > in scripts/policy/protocols/ssl/heartbleed.bro > > Make sure to use the linked branch (topic/bernhard/heartbeat) > > Bernhard > > On Apr 10, 2014, at 3:22 PM, John Babio wrote: > > > Do you have a github with this script in it? Thanks! > > > > On Apr 10, 2014, at 04:29 PM, John Hoyt wrote: > > > >> After implementing it just a little while ago, I've had eight > notifications. Half of which look to be vulnerable servers. > >> > >> So, I'd say so far good. > >> > >> -John > >> > >> > >> On Thu, Apr 10, 2014 at 4:11 PM, Gary Faulkner > wrote: > >> Just curious how the heartbleed Bro build is running for folks. Any > problems? > >> > >> On 4/10/2014 2:03 PM, John Hoyt wrote: > >>> That did it. :-) > >>> > >>> Thanks! > >>> > >>> > >>> On Thu, Apr 10, 2014 at 2:42 PM, Bernhard Amann < > bernhard at icsi.berkeley.edu> wrote: > >>> Did you add that after the line that @loads the heartbleed script? > >>> > >>> On Apr 10, 2014, at 11:32 AM, John Hoyt wrote: > >>> > >>>> Thanks Justin, > >>>> > >>>> I changed it to what you listed, but I'm still getting the following > error: > >>>> > >>>> error in /bro/share/bro/site/local.bro, line 95: unknown identifier > Heartbleed::SSL_Heartbeat_Attack_Success, at or near > "Heartbleed::SSL_Heartbeat_Attack_Success" > >>>> > >>>> > >>>> On Thu, Apr 10, 2014 at 2:20 PM, Justin Azoff > wrote: > >>>> On Thu, Apr 10, 2014 at 02:12:28PM -0400, John Hoyt wrote: > >>>>> I'm attempting to add an email alert for these, but I'm getting an > error. This > >>>>> is my first time attempting this, so I may have something wrong with > syntax. > >>>>> > >>>>> Here is what I've added to local.bro. > >>>>> > >>>>> > >>>>> hook Notice::policy(n: Notice::Info) > >>>>> > >>>>> { > >>>>> > >>>>> if ( n$note == SSL::SSL_Heartbeat_Attack_Success ) > >>>>> > >>>>> add n$actions[Notice::ACTION_EMAIL]; > >>>>> > >>>>> } > >>>> > >>>> The heartbleed module is in the Heartbleed namespace so the notice is > >>>> > >>>> Heartbleed::SSL_Heartbeat_Attack_Success > >>>> > >>>> Also, there is a helper for that sort of thing, you can simply: > >>>> > >>>> redef Notice::emailed_types += { > >>>> Heartbleed::SSL_Heartbeat_Attack_Success, > >>>> }; > >>>> > >>>> -- > >>>> -- Justin Azoff > >>>> > >>>> _______________________________________________ > >>>> Bro mailing list > >>>> bro at bro-ids.org > >>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > >>> > >>> > >>> > >>> > >>> _______________________________________________ > >>> Bro mailing list > >>> > >>> bro at bro-ids.org > >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > >> > >> > >> _______________________________________________ > >> Bro mailing list > >> bro at bro-ids.org > >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > >> > >> _______________________________________________ > >> Bro mailing list > >> bro at bro-ids.org > >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/cb08cfea/attachment.html From gary at doit.wisc.edu Thu Apr 10 16:54:00 2014 From: gary at doit.wisc.edu (Gary Faulkner) Date: Thu, 10 Apr 2014 18:54:00 -0500 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> Message-ID: <53472F18.7020508@doit.wisc.edu> I may have experienced something similar; although I only have 64G of RAM on my worker node at present so my workers would reach about 7-8G committed memory within 5-15 minutes before being OOM-killed. I've had memory issues in the past on 2.2-release, so I can't rule out my environment; although I've been problem free for the last month after moving to 2.2-184, so I just went back to that. When it was up and running I got a lot of good confirmed detections though. Regards On 4/10/2014 6:19 PM, Alex Waher wrote: > I had the heartbeat branch running for a few hours (very successfully > detecting activity!) and noticed it eventually had the manager worker > consuming +70gb of memory. Wasn't sure if the leak was from the heartbeat > capability itself or something else along the current git repo.. ymmv! > > > On Thu, Apr 10, 2014 at 3:46 PM, Bernhard Amann > wrote: > >> https://github.com/bro/bro/tree/topic/bernhard/heartbeat - the script is >> in scripts/policy/protocols/ssl/heartbleed.bro >> >> Make sure to use the linked branch (topic/bernhard/heartbeat) >> >> Bernhard >> >> On Apr 10, 2014, at 3:22 PM, John Babio wrote: >> >>> Do you have a github with this script in it? Thanks! >>> >>> On Apr 10, 2014, at 04:29 PM, John Hoyt wrote: >>> >>>> After implementing it just a little while ago, I've had eight >> notifications. Half of which look to be vulnerable servers. >>>> So, I'd say so far good. >>>> >>>> -John >>>> >>>> >>>> On Thu, Apr 10, 2014 at 4:11 PM, Gary Faulkner >> wrote: >>>> Just curious how the heartbleed Bro build is running for folks. Any >> problems? >>>> On 4/10/2014 2:03 PM, John Hoyt wrote: >>>>> That did it. :-) >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> On Thu, Apr 10, 2014 at 2:42 PM, Bernhard Amann < >> bernhard at icsi.berkeley.edu> wrote: >>>>> Did you add that after the line that @loads the heartbleed script? >>>>> >>>>> On Apr 10, 2014, at 11:32 AM, John Hoyt wrote: >>>>> >>>>>> Thanks Justin, >>>>>> >>>>>> I changed it to what you listed, but I'm still getting the following >> error: >>>>>> error in /bro/share/bro/site/local.bro, line 95: unknown identifier >> Heartbleed::SSL_Heartbeat_Attack_Success, at or near >> "Heartbleed::SSL_Heartbeat_Attack_Success" >>>>>> >>>>>> On Thu, Apr 10, 2014 at 2:20 PM, Justin Azoff >> wrote: >>>>>> On Thu, Apr 10, 2014 at 02:12:28PM -0400, John Hoyt wrote: >>>>>>> I'm attempting to add an email alert for these, but I'm getting an >> error. This >>>>>>> is my first time attempting this, so I may have something wrong with >> syntax. >>>>>>> Here is what I've added to local.bro. >>>>>>> >>>>>>> >>>>>>> hook Notice::policy(n: Notice::Info) >>>>>>> >>>>>>> { >>>>>>> >>>>>>> if ( n$note == SSL::SSL_Heartbeat_Attack_Success ) >>>>>>> >>>>>>> add n$actions[Notice::ACTION_EMAIL]; >>>>>>> >>>>>>> } >>>>>> The heartbleed module is in the Heartbleed namespace so the notice is >>>>>> >>>>>> Heartbleed::SSL_Heartbeat_Attack_Success >>>>>> >>>>>> Also, there is a helper for that sort of thing, you can simply: >>>>>> >>>>>> redef Notice::emailed_types += { >>>>>> Heartbleed::SSL_Heartbeat_Attack_Success, >>>>>> }; >>>>>> >>>>>> -- >>>>>> -- Justin Azoff >>>>>> >>>>>> _______________________________________________ >>>>>> Bro mailing list >>>>>> bro at bro-ids.org >>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Bro mailing list >>>>> >>>>> bro at bro-ids.org >>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>> >>>> _______________________________________________ >>>> Bro mailing list >>>> bro at bro-ids.org >>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>> >>>> _______________________________________________ >>>> Bro mailing list >>>> bro at bro-ids.org >>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Gary Faulkner -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/ee3d6e57/attachment.html From seth at icir.org Thu Apr 10 19:42:54 2014 From: seth at icir.org (Seth Hall) Date: Thu, 10 Apr 2014 22:42:54 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> Message-ID: On Apr 10, 2014, at 7:19 PM, Alex Waher wrote: > I had the heartbeat branch running for a few hours (very successfully detecting activity!) and noticed it eventually had the manager worker consuming +70gb of memory. Wasn't sure if the leak was from the heartbeat capability itself or something else along the current git repo.. ymmv! Sorry about that. We?ve been battling a bug in git master for a while now that causes high memory use on the manager of clusters (production-only memory leaks are really difficult). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140410/226231f5/attachment.bin From 45070198 at qq.com Mon Apr 14 19:42:16 2014 From: 45070198 at qq.com (=?gb18030?B?o6hwZXRlcqOp?=) Date: Tue, 15 Apr 2014 10:42:16 +0800 Subject: [Bro] How to port the pop3 script to Bro2.x Message-ID: Hi all, I'm new in Bro. I want to know how to port the pop3 script to the Bro2.x. Thank you. ---------- peter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140415/962d87cd/attachment.html From 45070198 at qq.com Tue Apr 15 05:36:43 2014 From: 45070198 at qq.com (=?gb18030?B?o6hwZXRlcqOp?=) Date: Tue, 15 Apr 2014 20:36:43 +0800 Subject: [Bro] Bro Error: fatal error in : Val::CONVERTER (string/record) Message-ID: Hi all, I add a file named ?main.bro? into the directory /usr/local/bro/share/bro/base/protocols/pop3/. The file?s content is as following: module POP3; export { redef enum Log::ID += { LOG }; type Info: record { ts: time &log; src: addr &log; srcport: port &log; dst: addr &log; dstport: port &log; }; global log_pop: event(rec: Info); } redef record connection += { pop3: Info &optional; }; const ports = { 110/tcp }; redef likely_server_ports += { ports }; event bro_init() &priority=5 { Log::create_stream(LOG, [$columns=Info, $ev=log_pop]); Analyzer::register_for_ports(Analyzer::ANALYZER_POP3, ports); } event pop3_request(c: connection, is_orig: bool, command: string, arg: string) &priority=5 { Log::write(LOG, command); } And I also modify the file _load_.bro, it?s content is as following: @load-sigs ./dpd.sig @load ./main OK, after the modify, I run the command stop, then start in the broctl, it work ok! But when I receive mail from pop3 server with my mail client, the bro is crash, and I found the error message in the logs/current/stderr.log: [root at VPS2 logs]# more current/stderr.log listening on em2, capture length 8192 bytes 1397564041.380190 fatal error in : Val::CONVERTER (string/record) (CAPA) I don?t know what happened, and what should I do next. Does anyone?s bro support the pop3? Could you tell me how to do it? Thank you. ------------- peter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140415/0b0f0a15/attachment.html From prateekgupta.3991 at gmail.com Tue Apr 15 05:53:27 2014 From: prateekgupta.3991 at gmail.com (Prateek Gupta) Date: Tue, 15 Apr 2014 18:23:27 +0530 Subject: [Bro] information exchange between binpac and analyzer Message-ID: Hello, I am working on Bro-IDS as my academic project and want some information. I want to know what are the data structures implemented in analyzer and binpac and how are these data structures passed between them. Its urgent. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140415/f2fd2e1e/attachment.html From prateekgupta.3991 at gmail.com Tue Apr 15 05:53:27 2014 From: prateekgupta.3991 at gmail.com (Prateek Gupta) Date: Tue, 15 Apr 2014 18:23:27 +0530 Subject: [Bro] information exchange between binpac and analyzer Message-ID: Hello, I am working on Bro-IDS as my academic project and want some information. I want to know what are the data structures implemented in analyzer and binpac and how are these data structures passed between them. Its urgent. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140415/f2fd2e1e/attachment-0001.html From vladg at cmu.edu Tue Apr 15 06:02:52 2014 From: vladg at cmu.edu (Vlad Grigorescu) Date: Tue, 15 Apr 2014 13:02:52 +0000 Subject: [Bro] Bro Error: fatal error in : Val::CONVERTER (string/record) In-Reply-To: References: Message-ID: On Apr 15, 2014, at 8:36 AM, ?peter? <45070198 at qq.com> wrote: > event pop3_request(c: connection, is_orig: bool, command: string, arg: string) &priority=5 > { > Log::write(LOG, command); > } Command needs to be an Info record. You're passing a string. You'll need to fill out an Info record and log that. > type Info: record { > ts: time &log; > src: addr &log; > srcport: port &log; > dst: addr &log; > dstport: port &log; > }; Take a look at the other Bro scripts. This isn't a good Bro Info record. You're even using different terminology (Bro doesn't have the concept of a "source" or "destination" - it's "originator" and "responder.") --Vlad -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140415/6a48caa0/attachment.bin From vladg at cmu.edu Tue Apr 15 06:10:54 2014 From: vladg at cmu.edu (Vlad Grigorescu) Date: Tue, 15 Apr 2014 13:10:54 +0000 Subject: [Bro] information exchange between binpac and analyzer In-Reply-To: References: Message-ID: <71FB3710-A4A1-4497-BD88-D36D302BA18D@andrew.cmu.edu> I don't understand the question. BinPAC is a compiler. It takes one or more .pac files, and compiles them to a .cc and .h file. Those then get compiled with the rest of Bro. You can look at these .cc and .h files when you build Bro - build/src/analyzer/protocol/ssl/ssl_pac.cc, for example. Data structures will be in those files. Have you seen the documentation? http://www.icir.org/vern/papers/binpac.IMC06.pdf https://www.bro.org/download/README.binpac.html http://www.bro.org/development/howtos/binpac-sample-analyzer.html Let us know if you have a specific question. --Vlad On Apr 15, 2014, at 8:53 AM, Prateek Gupta wrote: > Hello, > I am working on Bro-IDS as my academic project and want some information. > I want to know what are the data structures implemented in analyzer and binpac and how are these data structures passed between them. > Its urgent. > > Thank you. > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140415/5626f7bf/attachment.bin From 45070198 at qq.com Tue Apr 15 06:28:30 2014 From: 45070198 at qq.com (=?gb18030?B?o6hwZXRlcqOp?=) Date: Tue, 15 Apr 2014 21:28:30 +0800 Subject: [Bro] Bro Error: fatal error in : Val::CONVERTER(string/record) In-Reply-To: References: Message-ID: Hi vladg, Thank you very much. I modify the code follow your direction, it works now. The code which works well is here, maybe someone need it :) module POP3; export { redef enum Log::ID += { LOG }; type Info: record { ts: time &log; orig_h: addr &log; orig_p: port &log; resp_h: addr &log; resp_p: port &log; command: string &log; arg: string &log; }; global log_pop: event(rec: Info); } redef record connection += { pop3: Info &optional; }; const ports = { 110/tcp }; redef likely_server_ports += { ports }; event bro_init() &priority=5 { Log::create_stream(POP3::LOG, [$columns=POP3::Info, $ev=log_pop]); Analyzer::register_for_ports(Analyzer::ANALYZER_POP3, ports); } function set_session(c: connection, command: string, arg: string): Info { local l: Info; l$ts = network_time(); l$orig_h = c$id$orig_h; l$orig_p = c$id$orig_p; l$resp_h = c$id$resp_h; l$resp_p = c$id$resp_p; l$command = command; l$arg = arg; return l; } event pop3_request(c: connection, is_orig: bool, command: string, arg: string) &priority=5 { local myinfo: Info; myinfo = set_session(c, command, arg); Log::write(POP3::LOG, myinfo); } ------------------ Original ------------------ From: "Vlad Grigorescu";; Date: Tue, Apr 15, 2014 09:02 PM To: "?peter?"<45070198 at qq.com>; Cc: "bro"; Subject: Re: [Bro] Bro Error: fatal error in : Val::CONVERTER(string/record) On Apr 15, 2014, at 8:36 AM, ?peter? <45070198 at qq.com> wrote: > event pop3_request(c: connection, is_orig: bool, command: string, arg: string) &priority=5 > { > Log::write(LOG, command); > } Command needs to be an Info record. You're passing a string. You'll need to fill out an Info record and log that. > type Info: record { > ts: time &log; > src: addr &log; > srcport: port &log; > dst: addr &log; > dstport: port &log; > }; Take a look at the other Bro scripts. This isn't a good Bro Info record. You're even using different terminology (Bro doesn't have the concept of a "source" or "destination" - it's "originator" and "responder.") --Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140415/e8d783f5/attachment.html From prateekgupta.3991 at gmail.com Tue Apr 15 06:49:59 2014 From: prateekgupta.3991 at gmail.com (Prateek Gupta) Date: Tue, 15 Apr 2014 19:19:59 +0530 Subject: [Bro] information exchange between binpac and analyzer In-Reply-To: <71FB3710-A4A1-4497-BD88-D36D302BA18D@andrew.cmu.edu> References: <71FB3710-A4A1-4497-BD88-D36D302BA18D@andrew.cmu.edu> Message-ID: Hello Vlad, Thank you for your reply. Though I framed the question completely wrong, yet your answer served almost all my queries. I have another doubt and please correct my mistakes. The C++ code is generated of the .pac files by the binpac in the build. After "make" and "make install", is this the final analyzer ? What do the .cc files along with the .pac files in the analyzer have as content if I intend to write a custom protocol? As binpac calls the event function for the protocols when a particular "type" is detected, are those event functions present in these .cc files? Are the data structures present in these binpac compiled .cc files used for information exchange? Thank you. On Tue, Apr 15, 2014 at 6:40 PM, Vlad Grigorescu wrote: > I don't understand the question. BinPAC is a compiler. It takes one or > more .pac files, and compiles them to a .cc and .h file. Those then get > compiled with the rest of Bro. > > You can look at these .cc and .h files when you build Bro - > build/src/analyzer/protocol/ssl/ssl_pac.cc, for example. Data structures > will be in those files. > > Have you seen the documentation? > > http://www.icir.org/vern/papers/binpac.IMC06.pdf > https://www.bro.org/download/README.binpac.html > http://www.bro.org/development/howtos/binpac-sample-analyzer.html > > Let us know if you have a specific question. > > --Vlad > > On Apr 15, 2014, at 8:53 AM, Prateek Gupta > wrote: > > > Hello, > > I am working on Bro-IDS as my academic project and want some information. > > I want to know what are the data structures implemented in analyzer and > binpac and how are these data structures passed between them. > > Its urgent. > > > > Thank you. > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140415/56d015b0/attachment.html From 45070198 at qq.com Tue Apr 15 20:18:53 2014 From: 45070198 at qq.com (=?gb18030?B?o6hwZXRlcqOp?=) Date: Wed, 16 Apr 2014 11:18:53 +0800 Subject: [Bro] POP3: pop3_login_failure event could not work Message-ID: Hi all, I add a script to Bro2.x, which make it could support pop3 now. The pop3_login_success event could work ok, it output the logs I need. But the event pop3_login_failure failed. I use a wrong user and password to login pop3 server, I can find the login failure through the wireshark, BUT the Bro could not generate the corresponding logs. I do not know that whether or not the Bro pop3_login_failure could not discover the real pop3 login failure? --------------- peter. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140416/f6bd1ad6/attachment.html From 45070198 at qq.com Wed Apr 16 02:13:28 2014 From: 45070198 at qq.com (=?gb18030?B?o6hwZXRlcqOp?=) Date: Wed, 16 Apr 2014 17:13:28 +0800 Subject: [Bro] Do the Bro support LDAP protocol Message-ID: Hi, Do the Bro2.x support LDAP protocol ? Does anyone have the experience of making the Bro support the ldap protocol? Thank you. ---------- peter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140416/67c1b0f7/attachment.html From r.bortolameotti at gmail.com Wed Apr 16 08:03:41 2014 From: r.bortolameotti at gmail.com (Riccardo Bortolameotti) Date: Wed, 16 Apr 2014 17:03:41 +0200 Subject: [Bro] External commands - Error Message-ID: <1397660621.2141.12.camel@stud169130.mobiel.utwente.nl> Hello everybody, I am having a problem regarding external command execution. This is the piece of code: function f (s : string) : Exec::Result { local anagram : Exec::Command; local res : Exec::Result; res = [$exit_code = 0]; local prog = "/home/riccardo/ngram-ids/ngram-ids/ngram-ids"; local param = "-l /home/riccardo/ngram-ids/ngram-ids/save -m test -t 0 -U " + s; anagram = [$cmd = prog, $stdin = param]; res = Exec::run(anagram); return res; } Basically the problem is that the variable res do not receive any value from the execution of the program. Since this execution should be run several times because is within a loop, I do not know if there are problems of threading (like wait for the result). I also receive an error like this: 1394205441.982764 warning: non-void function returns without a value: Exec::run 1394205441.982764 error: return trigger in context which does not allow delaying result thank you in advance, R. From bernhard at ICSI.Berkeley.EDU Wed Apr 16 10:04:12 2014 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Wed, 16 Apr 2014 10:04:12 -0700 Subject: [Bro] External commands - Error In-Reply-To: <1397660621.2141.12.camel@stud169130.mobiel.utwente.nl> References: <1397660621.2141.12.camel@stud169130.mobiel.utwente.nl> Message-ID: <74455FDD-D1B2-4AAF-A053-04C0BB1149D8@icsi.berkeley.edu> Hi, you have to use Exec::run inside of a when statement - because the command is run asynchronously results are not immediately available. You can e.g. see https://github.com/bro/bro/blob/master/scripts/base/utils/dir.bro for an example that uses it. Bernhard On Apr 16, 2014, at 8:03 AM, Riccardo Bortolameotti wrote: > > Hello everybody, > > I am having a problem regarding external command execution. > > This is the piece of code: > function f (s : string) : Exec::Result > { > local anagram : Exec::Command; > local res : Exec::Result; > res = [$exit_code = 0]; > local prog = "/home/riccardo/ngram-ids/ngram-ids/ngram-ids"; > local param = "-l /home/riccardo/ngram-ids/ngram-ids/save -m test -t 0 -U " + s; > anagram = [$cmd = prog, $stdin = param]; > res = Exec::run(anagram); > return res; > } > Basically the problem is that the variable res do not receive any value > from the execution of the program. Since this execution should be run > several times because is within a loop, I do not know if there are > problems of threading (like wait for the result). I also receive an > error like this: > > 1394205441.982764 warning: non-void function returns without a value: > Exec::run > 1394205441.982764 error: return trigger in context which does not allow > delaying result > > thank you in advance, > > R. > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > From john.h.hoyt at gmail.com Wed Apr 16 10:17:16 2014 From: john.h.hoyt at gmail.com (John Hoyt) Date: Wed, 16 Apr 2014 13:17:16 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> Message-ID: Seth, Any luck on a fix for this memory leak? Currently, this is causing my Bro server to come to a slow death. Thanks, John On Thu, Apr 10, 2014 at 10:42 PM, Seth Hall wrote: > > On Apr 10, 2014, at 7:19 PM, Alex Waher wrote: > > > I had the heartbeat branch running for a few hours (very successfully > detecting activity!) and noticed it eventually had the manager worker > consuming +70gb of memory. Wasn't sure if the leak was from the heartbeat > capability itself or something else along the current git repo.. ymmv! > > Sorry about that. We?ve been battling a bug in git master for a while now > that causes high memory use on the manager of clusters (production-only > memory leaks are really difficult). > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140416/72761483/attachment.html From r.bortolameotti at gmail.com Wed Apr 16 10:28:59 2014 From: r.bortolameotti at gmail.com (Riccardo Bortolameotti) Date: Wed, 16 Apr 2014 19:28:59 +0200 Subject: [Bro] External commands - Error In-Reply-To: <74455FDD-D1B2-4AAF-A053-04C0BB1149D8@icsi.berkeley.edu> References: <1397660621.2141.12.camel@stud169130.mobiel.utwente.nl> <74455FDD-D1B2-4AAF-A053-04C0BB1149D8@icsi.berkeley.edu> Message-ID: Thank you very much guys! On 16 Apr 2014 19:04, "Bernhard Amann" wrote: > Hi, > > you have to use Exec::run inside of a when statement - because the command > is run asynchronously results are not immediately available. > > You can e.g. see > https://github.com/bro/bro/blob/master/scripts/base/utils/dir.bro for an > example that uses it. > > Bernhard > > On Apr 16, 2014, at 8:03 AM, Riccardo Bortolameotti < > r.bortolameotti at gmail.com> wrote: > > > > > Hello everybody, > > > > I am having a problem regarding external command execution. > > > > This is the piece of code: > > function f (s : string) : Exec::Result > > { > > local anagram : Exec::Command; > > local res : Exec::Result; > > res = [$exit_code = 0]; > > local prog = "/home/riccardo/ngram-ids/ngram-ids/ngram-ids"; > > local param = "-l /home/riccardo/ngram-ids/ngram-ids/save -m test > -t 0 -U " + s; > > anagram = [$cmd = prog, $stdin = param]; > > res = Exec::run(anagram); > > return res; > > } > > Basically the problem is that the variable res do not receive any value > > from the execution of the program. Since this execution should be run > > several times because is within a loop, I do not know if there are > > problems of threading (like wait for the result). I also receive an > > error like this: > > > > 1394205441.982764 warning: non-void function returns without a value: > > Exec::run > > 1394205441.982764 error: return trigger in context which does not allow > > delaying result > > > > thank you in advance, > > > > R. > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140416/5612d865/attachment.html From vladg at cmu.edu Thu Apr 17 08:14:27 2014 From: vladg at cmu.edu (Vlad Grigorescu) Date: Thu, 17 Apr 2014 15:14:27 +0000 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> Message-ID: John, Try merging in this branch: https://github.com/bro/bro/tree/topic/jsiwek/ascii-log-memleak-fix A number of people have tested it and reported that it fixed the memory issue. --Vlad On Apr 16, 2014, at 1:17 PM, John Hoyt wrote: > Seth, > > Any luck on a fix for this memory leak? Currently, this is causing my Bro server to come to a slow death. > > Thanks, > John > > > On Thu, Apr 10, 2014 at 10:42 PM, Seth Hall wrote: > > On Apr 10, 2014, at 7:19 PM, Alex Waher wrote: > > > I had the heartbeat branch running for a few hours (very successfully detecting activity!) and noticed it eventually had the manager worker consuming +70gb of memory. Wasn't sure if the leak was from the heartbeat capability itself or something else along the current git repo.. ymmv! > > Sorry about that. We?ve been battling a bug in git master for a while now that causes high memory use on the manager of clusters (production-only memory leaks are really difficult). > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140417/08d4b69c/attachment.bin From gary at doit.wisc.edu Thu Apr 17 16:56:02 2014 From: gary at doit.wisc.edu (Gary Faulkner) Date: Thu, 17 Apr 2014 18:56:02 -0500 Subject: [Bro] FYI for folks that use PF_RING + DNA + Libzero with Bro > PF_RING ZC is here Message-ID: <53506A12.80702@doit.wisc.edu> FYI, I was just going to look at the Libzero documentation on the NTOP site when I noticed the message "PF_RING ZC (Zero Copy) is the successor of Libzero and DNA. Please consider using it." It looks to be a single license now for the driver and libraries as opposed to one for each per MAC, but it also looks like there is an entirely worked API and example applications such as pfdnacluster_master get replaced with reworked and renamed versions. I hadn't seen any mention of it (or missed it) on the NTOP-misc list, so thought others may have also missed it as well. See here: http://www.ntop.org/pf_ring/introducing-pf_ring-zc-zero-copy/ Regards, -- Gary Faulkner UW Madison Office of Campus Information Security 608-262-8591 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6257 bytes Desc: S/MIME Cryptographic Signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140417/984ac49e/attachment.bin From liburdi.joshua at gmail.com Fri Apr 18 08:15:33 2014 From: liburdi.joshua at gmail.com (Josh Liburdi) Date: Fri, 18 Apr 2014 11:15:33 -0400 Subject: [Bro] Intel + Sumstats frameworks Message-ID: Having some unidentifiable issues working with the intelligence and Sumstats frameworks that I hope the development team or community can comment on. I've written multiple Sumstats scripts that return results, but I have an example where it doesn't when running on a cluster. Locally, the following snip of script runs and returns the expected results: @load base/frameworks/sumstats @load base/frameworks/notice @load base/frameworks/intel module Intel; export { redef enum Notice::Type += { Test_Indicators }; } event log_intel(rec: Info) { SumStats::observe("intel.indicators", [$str=cat(rec$uid,"`",rec$id$orig_h,"`",rec$id$resp_h)], [$str=rec$seen$indicator]); } event bro_init() { local r1: SumStats::Reducer = [$stream="intel.indicators", $apply=set(SumStats::UNIQUE)]; SumStats::create([$name="test-intel", $epoch=2mins, $reducers=set(r1), $epoch_result(ts: time, key: SumStats::Key, result: SumStats::Result) = { NOTICE([$note=Test_Indicators, #$src=to_addr(parts[3]), #$dst=to_addr(parts[5]), $msg="passed sumstats", #$sub=sub_msg, $identifier=key$str]); }]); } Each time a log is sent to intel.log, this observes the indicators seen in unique connections. For testing purposes, I have it writing a notice whenever the Sumstats event finishes. While this works as expected locally, when run in production (on a cluster), no notices are written-- that suggests to me that the data is not being sent to Sumstats. (And I am generating lines in intel.log in prod, so it cannot be the lack of intel.log activity.) The only identifiable difference between my local version of Bro and the one running in production is clustering. Is this expected behavior? I have used log_ events successfully with Sumstats in the past, so I can't think of what is preventing this notice from firing. I also verified the syntax of the above script by changing the event from log_intel to something more common (http_reply) and it worked both locally and in prod; changing log_intel to Intel::match (mimicking intel/do_notice.bro) worked locally but not in prod. Interested in reading thoughts on this... Thanks for reading, Josh Liburdi -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140418/ee85f2e4/attachment.html From jdopheid at illinois.edu Fri Apr 18 09:54:55 2014 From: jdopheid at illinois.edu (Dopheide, Jeannette M) Date: Fri, 18 Apr 2014 16:54:55 +0000 Subject: [Bro] BroCon '14: August 18-20 @ NCSA Message-ID: Bro Community, Mark your calendars. We're happy to announce the official dates and locations or BroCon '14! Dates: Monday, August 18th - Wednesday, August 20th Location: National Center for Supercomputing Applications(NCSA), Urbana-Champaign, IL If you are traveling by air we suggest you book your flights soon. The conference begins with breakfast at 8am so make sure you fly out by Sunday at the latest. We will be updating the site with hotel and registration information soon. Looking forward to seeing you, The Bro Team ------ Jeannette M. Dopheide Bro Outreach Coordinator National Center for Supercomputing Applications University of Illinois at Urbana-Champaign From jdopheid at illinois.edu Fri Apr 18 12:25:43 2014 From: jdopheid at illinois.edu (Dopheide, Jeannette M) Date: Fri, 18 Apr 2014 19:25:43 +0000 Subject: [Bro] Bro's YouTube Channel and Google+ pages moved Message-ID: Bro Community, We recently moved our YouTube channel to a page with a more Bro-centric name: https://www.youtube.com/user/BroPlatform We also moved our Google+ Page to: https://plus.google.com/+BroOrgProject/ If you have bookmarked or subscribed to these pages please update your settings. Thanks for your continued support. Sincerely, The Bro Team ------ Jeannette M. Dopheide Bro Outreach Coordinator National Center for Supercomputing Applications University of Illinois at Urbana-Champaign From abc423321c at gmail.com Sat Apr 19 10:43:13 2014 From: abc423321c at gmail.com (Amazon Student) Date: Sat, 19 Apr 2014 13:43:13 -0400 Subject: [Bro] Free for students: Get 20% off your first order and FREE Two-Day Shipping on Amazon for one year! Message-ID: My Amazon Account You are invited to join Amazon Student Join for FREE and get: FREE Two-Day Shipping on millions of items FREE release-date delivery on video games, DVDs, books and more Deals and promotions exclusively for Student Members Don't want Amazon Invitation e-mail? Unsubscribe here ? 2014 Amazon.com, Inc. and its affiliates. Amazon.com, Inc., 410 Terry Avenue North, Seattle, WA 98109-5210. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140419/0ac7a3cb/attachment.html From julien.t43 at gmail.com Sun Apr 20 22:06:15 2014 From: julien.t43 at gmail.com (Julien T) Date: Mon, 21 Apr 2014 01:06:15 -0400 Subject: [Bro] Bro on macos, magic bug Message-ID: Hello, I'm trying to use bro 2.2 on Mac (10.9.2) with macports but when I try to parse a pcap, I got magic errors: >>> $ bro -r /opt/local/share/bro/magic/animation, 193: Warning: Current entry does not yet have a description for adding a MIME type /opt/local/share/bro/magic/animation, 195: Warning: Current entry does not yet have a description for adding a MIME type /opt/local/share/bro/magic/animation, 197: Warning: Current entry does not yet have a description for adding a MIME type /opt/local/share/bro/magic/animation, 199: Warning: Current entry does not yet have a description for adding a MIME type /opt/local/share/bro/magic/animation, 201: Warning: Current entry does not yet have a description for adding a MIME type /opt/local/share/bro/magic/animation, 203: Warning: Current entry does not yet have a description for adding a MIME type /opt/local/share/bro/magic/animation, 205: Warning: Current entry does not yet have a description for adding a MIME type /opt/local/share/bro/magic/animation, 208: Warning: Current entry does not yet have a description for adding a MIME type /opt/local/share/bro/magic/archive, 45: Warning: Current entry does not yet have a description for adding a MIME type /opt/local/share/bro/magic/cafebabe, 19: Warning: Current entry does not yet have a description for adding a MIME type internal error: can't load magic file /opt/local/share/bro/magic: could not find any valid magic files! <<< Those files are from bro, while macports' libmagic (5.18) has /opt/local/share/misc/magic.mgc same if I do MAGIC=/opt/local/share/misc/magic.mgc bro -r as suggested on http://comments.gmane.org/gmane.comp.security.detection.bro/6225 Improvement with https://bro-tracker.atlassian.net/browse/BIT-1143. I gave a try to github head and it works as expected, probably because of above change. So is a fix is expected for 2.2 or 2.3 is sufficiently near release? Thanks Julien -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140421/47efcd18/attachment.html From sangdrax8 at gmail.com Mon Apr 21 05:33:49 2014 From: sangdrax8 at gmail.com (sangdrax8) Date: Mon, 21 Apr 2014 08:33:49 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> Message-ID: It appears that the master branch was merged into this heartbeat branch. Does this by chance include the memleak-fix merge you mentioned? Is this possibly a test before merging these changes into master its self? Also, it has been a while since I did my install, and I can't recall. If I do this on my master, then run the broctrl install, does it push the new install to all the nodes? I know the configurations get pushed out, but I can't recall if the entire install is pushed, or just configuration files. Thank you! On Thu, Apr 17, 2014 at 11:14 AM, Vlad Grigorescu wrote: > John, > > Try merging in this branch: https://github.com/bro/bro/tree/topic/jsiwek/ascii-log-memleak-fix > > A number of people have tested it and reported that it fixed the memory issue. > > --Vlad > > > On Apr 16, 2014, at 1:17 PM, John Hoyt wrote: > >> Seth, >> >> Any luck on a fix for this memory leak? Currently, this is causing my Bro server to come to a slow death. >> >> Thanks, >> John >> >> >> On Thu, Apr 10, 2014 at 10:42 PM, Seth Hall wrote: >> >> On Apr 10, 2014, at 7:19 PM, Alex Waher wrote: >> >> > I had the heartbeat branch running for a few hours (very successfully detecting activity!) and noticed it eventually had the manager worker consuming +70gb of memory. Wasn't sure if the leak was from the heartbeat capability itself or something else along the current git repo.. ymmv! >> >> Sorry about that. We?ve been battling a bug in git master for a while now that causes high memory use on the manager of clusters (production-only memory leaks are really difficult). >> >> .Seth >> >> -- >> Seth Hall >> International Computer Science Institute >> (Bro) because everyone has a network >> http://www.bro.org/ >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From bernhard at ICSI.Berkeley.EDU Mon Apr 21 05:43:31 2014 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Mon, 21 Apr 2014 05:43:31 -0700 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> Message-ID: <5DD3CE1D-E15B-4E2C-A7D2-F6A978FB50EF@icsi.berkeley.edu> On Apr 21, 2014, at 5:33 AM, sangdrax8 wrote: > It appears that the master branch was merged into this heartbeat > branch. Does this by chance include the memleak-fix merge you > mentioned? Is this possibly a test before merging these changes into > master its self? It does include the memory leak fixes that were mentioned, if you update the branch to the current state these are included. > Also, it has been a while since I did my install, and I can't recall. > If I do this on my master, then run the broctrl install, does it push > the new install to all the nodes? I know the configurations get > pushed out, but I can't recall if the entire install is pushed, or > just configuration files. The entire installation is pushed out. > Thank you! You are welcome, Bernhard From sangdrax8 at gmail.com Mon Apr 21 07:32:10 2014 From: sangdrax8 at gmail.com (sangdrax8) Date: Mon, 21 Apr 2014 10:32:10 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: <5DD3CE1D-E15B-4E2C-A7D2-F6A978FB50EF@icsi.berkeley.edu> References: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> <5DD3CE1D-E15B-4E2C-A7D2-F6A978FB50EF@icsi.berkeley.edu> Message-ID: I have pulled the latest branch, installed and pushed to my hosts. I loaded the heartbleed as indicated, then I am testing with the following site (https://filippo.io/Heartbleed/) so I can try and cause a notice. After running the attack, I can't seem to get a notice log. So I figure either the attach generated by this site doesn't trigger the script to insert a log, or I have something not configured right still. Is there some way I can check to see that I am in fact on this branch on all my nodes? Is there a specific version number or something I can verify? I can see the file in place, and the load statement in my local.bro, so not really sure what else to check. Any assistance would be much appreciated. On Mon, Apr 21, 2014 at 8:43 AM, Bernhard Amann wrote: > > On Apr 21, 2014, at 5:33 AM, sangdrax8 wrote: > >> It appears that the master branch was merged into this heartbeat >> branch. Does this by chance include the memleak-fix merge you >> mentioned? Is this possibly a test before merging these changes into >> master its self? > > It does include the memory leak fixes that were mentioned, if you update > the branch to the current state these are included. > >> Also, it has been a while since I did my install, and I can't recall. >> If I do this on my master, then run the broctrl install, does it push >> the new install to all the nodes? I know the configurations get >> pushed out, but I can't recall if the entire install is pushed, or >> just configuration files. > > The entire installation is pushed out. > >> Thank you! > > You are welcome, > Bernhard From sangdrax8 at gmail.com Mon Apr 21 07:50:19 2014 From: sangdrax8 at gmail.com (sangdrax8) Date: Mon, 21 Apr 2014 10:50:19 -0400 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> <5DD3CE1D-E15B-4E2C-A7D2-F6A978FB50EF@icsi.berkeley.edu> Message-ID: Alright, I have checked with multiple other websites, and it seems that the first one I tried isn't detected. I have seen the notice from a few other scans. I'll watch the memory and see if the fix that was merged keeps it in check. On Mon, Apr 21, 2014 at 10:32 AM, sangdrax8 wrote: > I have pulled the latest branch, installed and pushed to my hosts. I > loaded the heartbleed as indicated, then I am testing with the > following site (https://filippo.io/Heartbleed/) so I can try and cause > a notice. After running the attack, I can't seem to get a notice > log. > > So I figure either the attach generated by this site doesn't trigger > the script to insert a log, or I have something not configured right > still. Is there some way I can check to see that I am in fact on this > branch on all my nodes? Is there a specific version number or > something I can verify? > > I can see the file in place, and the load statement in my local.bro, > so not really sure what else to check. Any assistance would be much > appreciated. > > > > On Mon, Apr 21, 2014 at 8:43 AM, Bernhard Amann > wrote: >> >> On Apr 21, 2014, at 5:33 AM, sangdrax8 wrote: >> >>> It appears that the master branch was merged into this heartbeat >>> branch. Does this by chance include the memleak-fix merge you >>> mentioned? Is this possibly a test before merging these changes into >>> master its self? >> >> It does include the memory leak fixes that were mentioned, if you update >> the branch to the current state these are included. >> >>> Also, it has been a while since I did my install, and I can't recall. >>> If I do this on my master, then run the broctrl install, does it push >>> the new install to all the nodes? I know the configurations get >>> pushed out, but I can't recall if the entire install is pushed, or >>> just configuration files. >> >> The entire installation is pushed out. >> >>> Thank you! >> >> You are welcome, >> Bernhard From bernhard at ICSI.Berkeley.EDU Mon Apr 21 07:52:00 2014 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Mon, 21 Apr 2014 07:52:00 -0700 Subject: [Bro] Detecting heartbleed activity In-Reply-To: References: <25b5be03-f29a-4b01-bfd1-4ad6b73fb91d@me.com> <5DD3CE1D-E15B-4E2C-A7D2-F6A978FB50EF@icsi.berkeley.edu> Message-ID: <55B2D2D3-339D-406E-972F-FB7C5ABC0BDE@icsi.berkeley.edu> Hi, that site uses the encrypted variant of the attack (hence it sends the exploit heartbeat frames after encryption has begun). In this case, it is more difficult to detect the attack than in the simple case - we cannot just flag all heartbeats because that would introduce a lot of false positive. Thus, in case the attack is encrypted, you will only get notices if it was successful (we still can determine that by comparing sizes), but not if it was just attempted, sorry. There really is no good way around that. So - you probably tested against a non-vulnerable server. If you test against a vulnerable machine, you should get a notice in your log. I think the heartbeat check by www.ssllabs.com always triggers - they don?t start encryption before sending the heartbeats. Bernhard On Apr 21, 2014, at 7:32 AM, sangdrax8 wrote: > I have pulled the latest branch, installed and pushed to my hosts. I > loaded the heartbleed as indicated, then I am testing with the > following site (https://filippo.io/Heartbleed/) so I can try and cause > a notice. After running the attack, I can't seem to get a notice > log. > > So I figure either the attach generated by this site doesn't trigger > the script to insert a log, or I have something not configured right > still. Is there some way I can check to see that I am in fact on this > branch on all my nodes? Is there a specific version number or > something I can verify? > > I can see the file in place, and the load statement in my local.bro, > so not really sure what else to check. Any assistance would be much > appreciated. > > > > On Mon, Apr 21, 2014 at 8:43 AM, Bernhard Amann > wrote: >> >> On Apr 21, 2014, at 5:33 AM, sangdrax8 wrote: >> >>> It appears that the master branch was merged into this heartbeat >>> branch. Does this by chance include the memleak-fix merge you >>> mentioned? Is this possibly a test before merging these changes into >>> master its self? >> >> It does include the memory leak fixes that were mentioned, if you update >> the branch to the current state these are included. >> >>> Also, it has been a while since I did my install, and I can't recall. >>> If I do this on my master, then run the broctrl install, does it push >>> the new install to all the nodes? I know the configurations get >>> pushed out, but I can't recall if the entire install is pushed, or >>> just configuration files. >> >> The entire installation is pushed out. >> >>> Thank you! >> >> You are welcome, >> Bernhard > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > From jsiwek at illinois.edu Mon Apr 21 07:53:25 2014 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Mon, 21 Apr 2014 14:53:25 +0000 Subject: [Bro] Bro on macos, magic bug In-Reply-To: References: Message-ID: <752C7492-E09A-47DA-B6F0-40EDFA1EA9B0@illinois.edu> On Apr 21, 2014, at 12:06 AM, Julien T wrote: > internal error: can't load magic file /opt/local/share/bro/magic: could not find any valid magic files! > <<< > > Those files are from bro, while macports' libmagic (5.18) has /opt/local/share/misc/magic.mgc > > same if I do > MAGIC=/opt/local/share/misc/magic.mgc bro -r Try also `BROMAGIC=/opt/local/share/misc/magic.mgc MAGIC=/opt/local/share/misc/magic.mgc bro -r `. > So is a fix is expected for 2.2 or 2.3 is sufficiently near release? No 2.2 changes are planned and the 2.3 (beta) isn?t dated, you?ll have to infer from https://bro-tracker.atlassian.net/issues/?filter=10001 - Jon From prateekgupta.3991 at gmail.com Tue Apr 22 08:20:40 2014 From: prateekgupta.3991 at gmail.com (Prateek Gupta) Date: Tue, 22 Apr 2014 20:50:40 +0530 Subject: [Bro] some information Message-ID: Hello everyone, I want to know that which module in the source code supplies the information to the root node of the analyzer tree (i.e. Tansport Layer Analyzer) . Where should I look for in the source code. Urgently need to understand the information flow and the data structures involved to the analyzer layer for an academic project. Also I have been able to use sample analyzer provided in the bro site but how to test run it easily. Can I do something like modifying it like some already made protocol (ex. dhcp etc) and test run it or what? Hoping for a reply. Regards, Prateek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140422/6fb245eb/attachment.html From prateekgupta.3991 at gmail.com Tue Apr 22 08:20:40 2014 From: prateekgupta.3991 at gmail.com (Prateek Gupta) Date: Tue, 22 Apr 2014 20:50:40 +0530 Subject: [Bro] some information Message-ID: Hello everyone, I want to know that which module in the source code supplies the information to the root node of the analyzer tree (i.e. Tansport Layer Analyzer) . Where should I look for in the source code. Urgently need to understand the information flow and the data structures involved to the analyzer layer for an academic project. Also I have been able to use sample analyzer provided in the bro site but how to test run it easily. Can I do something like modifying it like some already made protocol (ex. dhcp etc) and test run it or what? Hoping for a reply. Regards, Prateek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140422/6fb245eb/attachment-0001.html From schworer at gmail.com Tue Apr 22 09:24:58 2014 From: schworer at gmail.com (Andy Schworer) Date: Tue, 22 Apr 2014 09:24:58 -0700 Subject: [Bro] Bro Intel Framework - IP Subnet Message-ID: Does the Bro Intel Framework support Ip address ranges? ex 192.168.0.0/16? https://www.bro.org/sphinx/scripts/base/frameworks/intel/main.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140422/e210dd8d/attachment.html From julien.t43 at gmail.com Tue Apr 22 10:23:36 2014 From: julien.t43 at gmail.com (Julien T) Date: Tue, 22 Apr 2014 13:23:36 -0400 Subject: [Bro] Bro on macos, magic bug In-Reply-To: <752C7492-E09A-47DA-B6F0-40EDFA1EA9B0@illinois.edu> References: <752C7492-E09A-47DA-B6F0-40EDFA1EA9B0@illinois.edu> Message-ID: 2014-04-21 10:53 GMT-04:00 Siwek, Jonathan Luke : > > Try also `BROMAGIC=/opt/local/share/misc/magic.mgc > MAGIC=/opt/local/share/misc/magic.mgc bro -r `. > yes with BROMAGIC, it's working. without environment or only MAGIC, only warnings and no output. > > > So is a fix is expected for 2.2 or 2.3 is sufficiently near release? > > No 2.2 changes are planned and the 2.3 (beta) isn?t dated, you?ll have to > infer from https://bro-tracker.atlassian.net/issues/?filter=10001 > > ok. Thanks a lot! Cheers, Julien -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140422/d98ba1bf/attachment.html From jsiwek at illinois.edu Tue Apr 22 11:10:22 2014 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Tue, 22 Apr 2014 18:10:22 +0000 Subject: [Bro] some information In-Reply-To: References: Message-ID: <1EAB961A-3ED4-4DC7-B070-C18BCCB7ED0F@illinois.edu> On Apr 22, 2014, at 10:20 AM, Prateek Gupta wrote: > I want to know that which module in the source code supplies the information to the root node of the analyzer tree (i.e. Tansport Layer Analyzer) . Where should I look for in the source code. > Urgently need to understand the information flow and the data structures involved to the analyzer layer for an academic project. See Connection::NextPacket which is called from NetSessions::DoNextPacket. > Also I have been able to use sample analyzer provided in the bro site but how to test run it easily. Can I do something like modifying it like some already made protocol (ex. dhcp etc) and test run it or what? You can do whatever you need/like in the implementation of the sample analyzer (e.g. the overrides of Analyzer::DeliverStream or Analyzer::DeliverPacket). Then, to get the sample analyzer attached to particular connections so it will actually process data, there?s a choice of (1) look in to how the Analyzer::register_for_ports script-layer function is used for other protocol analyzers (2) look in to how other analyzers use DPD signatures to automatically attach themselves to a connection when the payload matches a signature (3) hardcode the sample analyzer to be used for every connection. It?s typical to combine (1) and (2). - Jon From grutz at jingojango.net Tue Apr 22 13:25:19 2014 From: grutz at jingojango.net (Kurt Grutzmacher) Date: Tue, 22 Apr 2014 13:25:19 -0700 Subject: [Bro] Github repository clone failure: binpac not found Message-ID: <5356D02F.9060406@jingojango.net> The submodule clone for binpac from git.bro.org seems to be missing and causes a failure when synchronizing from github: $ git clone --recursive https://github.com/bro/bro.git Cloning into 'bro'... remote: Reusing existing pack: 57723, done. remote: Counting objects: 91, done. remote: Compressing objects: 100% (89/89), done. remote: Total 57814 (delta 33), reused 0 (delta 0) Receiving objects: 100% (57814/57814), 40.93 MiB | 175.00 KiB/s, done. Resolving deltas: 100% (37711/37711), done. Checking connectivity... done. Submodule 'aux/binpac' (git://git.bro.org/binpac) registered for path 'aux/binpac' Submodule 'aux/bro-aux' (git://git.bro.org/bro-aux) registered for path 'aux/bro-aux' Submodule 'aux/broccoli' (git://git.bro.org/broccoli) registered for path 'aux/broccoli' Submodule 'aux/broctl' (git://git.bro.org/broctl) registered for path 'aux/broctl' Submodule 'aux/btest' (git://git.bro.org/btest) registered for path 'aux/btest' Submodule 'cmake' (git://git.bro.org/cmake) registered for path 'cmake' Submodule 'src/3rdparty' (git://git.bro.org/bro-3rdparty) registered for path 'src/3rdparty' Cloning into 'aux/binpac'... fatal: repository 'https://git.bro.org/binpac/' not found Clone of 'git://git.bro.org/binpac' into submodule path 'aux/binpac' failed Changing the .gitmodules URL to git://github.com/bro/binpac works. -- - grutz; From r.bortolameotti at gmail.com Wed Apr 23 03:00:38 2014 From: r.bortolameotti at gmail.com (Riccardo Bortolameotti) Date: Wed, 23 Apr 2014 12:00:38 +0200 Subject: [Bro] External commands - Error In-Reply-To: <74455FDD-D1B2-4AAF-A053-04C0BB1149D8@icsi.berkeley.edu> References: <1397660621.2141.12.camel@stud169130.mobiel.utwente.nl> <74455FDD-D1B2-4AAF-A053-04C0BB1149D8@icsi.berkeley.edu> Message-ID: <1398247238.7056.2.camel@stud169130.mobiel.utwente.nl> Hi guys, I have inserted the command within the when statement { local s = "whatever"; local anagram : Exec::Command; local param = "-l /home/riccardo/ngram-ids/ngram-ids/save -m test -t 0 -U " + s; local prog = "/home/riccardo/ngram-ids/ngram-ids/ngram-ids" + param; anagram = [$cmd = prog]; when ( local res = Exec::run(anagram) ) { print "I AM IN! - Debug"; } return "Not Executed!"; } However seems it is not able to enter in the block. Like the program is not executed at all. If I run the command from my command line it perfectly works. I do not really understand why it does not go within that block. thank you for help, R. On Wed, 2014-04-16 at 10:04 -0700, Bernhard Amann wrote: > Hi, > > you have to use Exec::run inside of a when statement - because the command is run asynchronously results are not immediately available. > > You can e.g. see https://github.com/bro/bro/blob/master/scripts/base/utils/dir.bro for an example that uses it. > > Bernhard > > On Apr 16, 2014, at 8:03 AM, Riccardo Bortolameotti wrote: > > > > > Hello everybody, > > > > I am having a problem regarding external command execution. > > > > This is the piece of code: > > function f (s : string) : Exec::Result > > { > > local anagram : Exec::Command; > > local res : Exec::Result; > > res = [$exit_code = 0]; > > local prog = "/home/riccardo/ngram-ids/ngram-ids/ngram-ids"; > > local param = "-l /home/riccardo/ngram-ids/ngram-ids/save -m test -t 0 -U " + s; > > anagram = [$cmd = prog, $stdin = param]; > > res = Exec::run(anagram); > > return res; > > } > > Basically the problem is that the variable res do not receive any value > > from the execution of the program. Since this execution should be run > > several times because is within a loop, I do not know if there are > > problems of threading (like wait for the result). I also receive an > > error like this: > > > > 1394205441.982764 warning: non-void function returns without a value: > > Exec::run > > 1394205441.982764 error: return trigger in context which does not allow > > delaying result > > > > thank you in advance, > > > > R. > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > From r.bortolameotti at gmail.com Wed Apr 23 03:03:05 2014 From: r.bortolameotti at gmail.com (Riccardo Bortolameotti) Date: Wed, 23 Apr 2014 12:03:05 +0200 Subject: [Bro] External commands - Error In-Reply-To: <1398247238.7056.2.camel@stud169130.mobiel.utwente.nl> References: <1397660621.2141.12.camel@stud169130.mobiel.utwente.nl> <74455FDD-D1B2-4AAF-A053-04C0BB1149D8@icsi.berkeley.edu> <1398247238.7056.2.camel@stud169130.mobiel.utwente.nl> Message-ID: <1398247385.7056.4.camel@stud169130.mobiel.utwente.nl> Sorry for writing another email. This is a simple script that does not work, and basically follows the same concept. @load base/utils/exec event bro_init() { local cmd = Exec::Command($cmd="echo 'hello'"); when (local res = Exec::run(cmd)) { print "hello"; print res$stdout; } } On Wed, 2014-04-23 at 12:00 +0200, Riccardo Bortolameotti wrote: > Hi guys, > > I have inserted the command within the when statement > { > local s = "whatever"; > local anagram : Exec::Command; > local param = "-l /home/riccardo/ngram-ids/ngram-ids/save -m test -t 0 > -U " + s; > local prog = "/home/riccardo/ngram-ids/ngram-ids/ngram-ids" + param; > anagram = [$cmd = prog]; > when ( local res = Exec::run(anagram) ) > { > print "I AM IN! - Debug"; > } > > return "Not Executed!"; > } > > However seems it is not able to enter in the block. Like the program is > not executed at all. If I run the command from my command line it > perfectly works. I do not really understand why it does not go within > that block. > > thank you for help, > > R. > > On Wed, 2014-04-16 at 10:04 -0700, Bernhard Amann wrote: > > Hi, > > > > you have to use Exec::run inside of a when statement - because the command is run asynchronously results are not immediately available. > > > > You can e.g. see https://github.com/bro/bro/blob/master/scripts/base/utils/dir.bro for an example that uses it. > > > > Bernhard > > > > On Apr 16, 2014, at 8:03 AM, Riccardo Bortolameotti wrote: > > > > > > > > Hello everybody, > > > > > > I am having a problem regarding external command execution. > > > > > > This is the piece of code: > > > function f (s : string) : Exec::Result > > > { > > > local anagram : Exec::Command; > > > local res : Exec::Result; > > > res = [$exit_code = 0]; > > > local prog = "/home/riccardo/ngram-ids/ngram-ids/ngram-ids"; > > > local param = "-l /home/riccardo/ngram-ids/ngram-ids/save -m test -t 0 -U " + s; > > > anagram = [$cmd = prog, $stdin = param]; > > > res = Exec::run(anagram); > > > return res; > > > } > > > Basically the problem is that the variable res do not receive any value > > > from the execution of the program. Since this execution should be run > > > several times because is within a loop, I do not know if there are > > > problems of threading (like wait for the result). I also receive an > > > error like this: > > > > > > 1394205441.982764 warning: non-void function returns without a value: > > > Exec::run > > > 1394205441.982764 error: return trigger in context which does not allow > > > delaying result > > > > > > thank you in advance, > > > > > > R. > > > > > > _______________________________________________ > > > Bro mailing list > > > bro at bro-ids.org > > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > > > > > From seth at icir.org Wed Apr 23 06:39:16 2014 From: seth at icir.org (Seth Hall) Date: Wed, 23 Apr 2014 09:39:16 -0400 Subject: [Bro] External commands - Error In-Reply-To: <1398247385.7056.4.camel@stud169130.mobiel.utwente.nl> References: <1397660621.2141.12.camel@stud169130.mobiel.utwente.nl> <74455FDD-D1B2-4AAF-A053-04C0BB1149D8@icsi.berkeley.edu> <1398247238.7056.2.camel@stud169130.mobiel.utwente.nl> <1398247385.7056.4.camel@stud169130.mobiel.utwente.nl> Message-ID: <869ACE40-FC88-40F1-80D8-5A9B70133925@icir.org> On Apr 23, 2014, at 6:03 AM, Riccardo Bortolameotti wrote: > This is a simple script that does not work, and basically follows the > same concept. At the beginning of your script, add? redef exit_only_after_terminate=T; Bro normally shuts down if there is nothing feeding the event queue (and when statements are asynchronous and don?t count as feeding the event queue). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140423/fb0d8ecf/attachment.bin From prateekgupta.3991 at gmail.com Wed Apr 23 10:17:59 2014 From: prateekgupta.3991 at gmail.com (Prateek Gupta) Date: Wed, 23 Apr 2014 22:47:59 +0530 Subject: [Bro] information between TLanallyzer and Application protocol analyzer Message-ID: Hello, I have a query. In TCP.cc , the information is trasferred to the child analyzers (i.e. application protocol analyzer) using statement LOOP_OVER_GIVEN_ and calling NextPacket() function but in the case of UDP I dont find any mechanism to transfer info to the application layer protocol. Do the UDP based application protocol get their own information from connection via their respective .pac analyzer? If not can you please explain in brief. Thank you, Prateek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140423/a9b84479/attachment.html From prateekgupta.3991 at gmail.com Wed Apr 23 10:17:59 2014 From: prateekgupta.3991 at gmail.com (Prateek Gupta) Date: Wed, 23 Apr 2014 22:47:59 +0530 Subject: [Bro] information between TLanallyzer and Application protocol analyzer Message-ID: Hello, I have a query. In TCP.cc , the information is trasferred to the child analyzers (i.e. application protocol analyzer) using statement LOOP_OVER_GIVEN_ and calling NextPacket() function but in the case of UDP I dont find any mechanism to transfer info to the application layer protocol. Do the UDP based application protocol get their own information from connection via their respective .pac analyzer? If not can you please explain in brief. Thank you, Prateek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140423/a9b84479/attachment-0001.html From prateekgupta.3991 at gmail.com Wed Apr 23 10:21:24 2014 From: prateekgupta.3991 at gmail.com (Prateek Gupta) Date: Wed, 23 Apr 2014 22:51:24 +0530 Subject: [Bro] information between TLanallyzer and Application protocol analyzer In-Reply-To: References: Message-ID: Is ForwardPacket() function supplyin the info to the child analyzer? On Wed, Apr 23, 2014 at 10:47 PM, Prateek Gupta wrote: > Hello, > I have a query. In TCP.cc , the information is trasferred to the child > analyzers (i.e. application protocol analyzer) using statement > LOOP_OVER_GIVEN_ and calling NextPacket() function > > but in the case of UDP I dont find any mechanism to transfer info to the > application layer protocol. Do the UDP based application protocol get their > own information from connection via their respective .pac analyzer? If not > can you please explain in brief. > > Thank you, > Prateek > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140423/2146aed1/attachment.html From prateekgupta.3991 at gmail.com Wed Apr 23 10:21:24 2014 From: prateekgupta.3991 at gmail.com (Prateek Gupta) Date: Wed, 23 Apr 2014 22:51:24 +0530 Subject: [Bro] information between TLanallyzer and Application protocol analyzer In-Reply-To: References: Message-ID: Is ForwardPacket() function supplyin the info to the child analyzer? On Wed, Apr 23, 2014 at 10:47 PM, Prateek Gupta wrote: > Hello, > I have a query. In TCP.cc , the information is trasferred to the child > analyzers (i.e. application protocol analyzer) using statement > LOOP_OVER_GIVEN_ and calling NextPacket() function > > but in the case of UDP I dont find any mechanism to transfer info to the > application layer protocol. Do the UDP based application protocol get their > own information from connection via their respective .pac analyzer? If not > can you please explain in brief. > > Thank you, > Prateek > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140423/2146aed1/attachment-0001.html From jsiwek at illinois.edu Wed Apr 23 11:22:31 2014 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Wed, 23 Apr 2014 18:22:31 +0000 Subject: [Bro] information between TLanallyzer and Application protocol analyzer In-Reply-To: References: Message-ID: <25D14D5E-598B-489A-9158-43B04A4A2EDE@illinois.edu> On Apr 23, 2014, at 12:17 PM, Prateek Gupta wrote: > I have a query. In TCP.cc , the information is trasferred to the child analyzers (i.e. application protocol analyzer) using statement > LOOP_OVER_GIVEN_ and calling NextPacket() function > > but in the case of UDP I dont find any mechanism to transfer info to the application layer protocol. Do the UDP based application protocol get their own information from connection via their respective .pac analyzer? If not can you please explain in brief. Each connection starts with an analyzer tree that looks like (see analyzer::Manager::BuildInitialAnalyzerTree): UDP: UDP_Analyzer -> PIA_UDP, any analyzers registered for a well-known UDP resp port TCP: TCP_Analyzer -> PIA_TCP, any analyzers registered for a well-known TCP resp port The PIA_*, Port Independent Analysis (I think), are responsible for automatically attaching new analyzers if payload content matches provided signatures. Children of UDP_Analyzer which override Analyzer::DeliverPacket will start receiving packets immediately from Analyzer::ForwardPacket. Children of TCP_Analyzer which override Analyzer::DeliverPacket will start receiving packets immediately from (1) Analyzer::ForwardPacket if reassembly is not enabled (2) Analyzer::NextPacket if they were explicitly added as children via TCP_Analyzer::AddChildPacketAnalyzer. It?s more typical for children of TCP_Analyzer to be overriding Analyzer::DeliverStream in order to receive input as reassembled TCP segments. i.e. protocols on top of TCP may choose between packet-wise and stream-wise input, but the later is more common. - Jon From jdopheid at illinois.edu Thu Apr 24 12:23:25 2014 From: jdopheid at illinois.edu (Dopheide, Jeannette M) Date: Thu, 24 Apr 2014 19:23:25 +0000 Subject: [Bro] BroCon '14 Registration and Events page Message-ID: Bro Community, We are happy to announce that registration fro BroCon '14(August 18th - 20th) is now open! Register here: regonline.com/brocon2014 Event details are a work in progress. Updates, agenda, and hotel information will be posted here: http://bro.org/community/brocon2014.html We are working to confirm a discounted rate at the Hampton Inn and Holiday Inn hotels. We hope to have an update posted soon. We are accepting presentation proposals. More info here: http://bro.org/community/brocon2014.html#call-forpresentations Interested in sponsoring BroCon '14? More info here: http://bro.org/community/brocon2014.html#sponsorship See you in August, The Bro Team ------ Jeannette M. Dopheide Bro Outreach Coordinator National Center for Supercomputing Applications University of Illinois at Urbana-Champaign From jep at g-c-i.net Fri Apr 25 07:23:17 2014 From: jep at g-c-i.net (Parker, Jonathan E.) Date: Fri, 25 Apr 2014 14:23:17 +0000 Subject: [Bro] File extraction and archive files Message-ID: <36C06B73C5A9D845A5435F7AAE88802B846651F0@Mail10.Corporate.net> I've been tasked to find files with a specific "signature" in the file header, where the file will be within an archive of files. This needs to be agnostic of the protocol that transported the archive file. I'm thinking the way to do this is to use the new File Analysis framework. Does Bro provide a mechanism to "automagically" extract the contents of an archive when it is an archive file that is being extracted from a protocol, or is this something I'm going to have to script myself? How can I know that a file has been fully received such that I can begin my analysis? Thanks - Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140425/3c7a2704/attachment.html From zryzregister at 163.com Sat Apr 26 08:25:57 2014 From: zryzregister at 163.com (=?GBK?B?1dTcx9Sq?=) Date: Sat, 26 Apr 2014 23:25:57 +0800 (CST) Subject: [Bro] Problems in Communicating with BroCluster Using Broccoli Message-ID: <567eab66.23c10.1459ea4947f.Coremail.zryzregister@163.com> Hi all, Sorry to trouble! Now I encountered a problem when I tries to implement a programme that is capable of communicating with Bro CLuster. The programme functions well when communicating with single Bro, but has problem when I installed the scripts to BroCluster. The Bro-end is something like: @load frameworks/communication/listen redef Communication::listen_port = 49889/tcp; redef Communication::listen_ssl = F; global ping_log = open_log_file("alert"); global alert: event(t: string, id: string, sip: string, sp: string, dip: string, dp: string); redef Communication::nodes += { ["pingevent"] = [$host = 202.197.165.213, $events = /alert/, $connect=T, $ssl=F] }; event alert(t: string, id: string, sip: string, sp: string, dip: string, dp: string) { print ping_log, fmt("Alert received, %s",id); print fmt("alert event recieved! %s %s %s %s %s %s",t,id,sip,sp,dip,dp); } } And the other end sends Bro events to the Bro-end. Now, the problem is while this script works well when communicating with single Bro which uses the command "bro", but it has problems when I install them to the Bro Cluster nodes, e.g. local-manager.bro or local-worker.bro. My script and programme is listed in the attachment. Hope for help!! Thanks a lot! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140426/a63583e8/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: local-worker.bro Type: application/octet-stream Size: 750 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140426/a63583e8/attachment.obj -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: pingevent.c Url: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140426/a63583e8/attachment.c From liburdi.joshua at gmail.com Sat Apr 26 09:28:21 2014 From: liburdi.joshua at gmail.com (Josh Liburdi) Date: Sat, 26 Apr 2014 12:28:21 -0400 Subject: [Bro] Using for loop w/ &expire_func Message-ID: I'm having a problem iterating over a table in an &expire_func function. I receive this error: "value used but not set (t)" When using this function with my table... &expire_func=function(t: table[string] of set[string,string,addr], idx: string): interval { for ( [x,y,z] in t[idx] ) { print x; } return 0secs; }; Iterating over the table works with events, but I'd like to process indexes as they are removed from the table. Has anyone run into this before or know what might be causing the error? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140426/4dbbbc0b/attachment.html From josh.liburdi at ge.com Mon Apr 28 06:40:11 2014 From: josh.liburdi at ge.com (Liburdi, Josh (GE Corporate)) Date: Mon, 28 Apr 2014 13:40:11 +0000 Subject: [Bro] Using for loop w/ &expire_func In-Reply-To: References: Message-ID: Nevermind? I mistakenly had my table in the export section of the script. Moving it fixed my problem. - Josh From: Josh Liburdi Date: Saturday, April 26, 2014 at 12:28 PM To: bro Subject: [Bro] Using for loop w/ &expire_func I'm having a problem iterating over a table in an &expire_func function. I receive this error: "value used but not set (t)" When using this function with my table... &expire_func=function(t: table[string] of set[string,string,addr], idx: string): interval { for ( [x,y,z] in t[idx] ) { print x; } return 0secs; }; Iterating over the table works with events, but I'd like to process indexes as they are removed from the table. Has anyone run into this before or know what might be causing the error? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140428/969cf53b/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4424 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140428/969cf53b/attachment.bin From jsiwek at illinois.edu Mon Apr 28 08:29:18 2014 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Mon, 28 Apr 2014 15:29:18 +0000 Subject: [Bro] File extraction and archive files In-Reply-To: <36C06B73C5A9D845A5435F7AAE88802B846651F0@Mail10.Corporate.net> References: <36C06B73C5A9D845A5435F7AAE88802B846651F0@Mail10.Corporate.net> Message-ID: On Apr 25, 2014, at 9:23 AM, Parker, Jonathan E. wrote: > I've been tasked to find files with a specific "signature" in the file header, where the file will be within an archive of files. This needs to be agnostic of the protocol that transported the archive file. > > I'm thinking the way to do this is to use the new File Analysis framework. Does Bro provide a mechanism to "automagically" extract the contents of an archive when it is an archive file that is being extracted from a protocol, It does not currently recurse on the contents of archive files ?on-the-fly?. > or is this something I'm going to have to script myself? A way (that I can think of) to possibly do this only in Bro scripts would be to extract the full archive to disk using the File Analysis Framework, then use Bro?s Exec module to expand the archive, and finally use the Input framework to feed the contents back in to the File Analysis Framework. > How can I know that a file has been fully received such that I can begin my analysis? The ?file_state_remove? event is when you?ve got as much data of the file as you?re going to get. Whether it?s actually the full file: sometimes you can?t tell and the best you can to is check that missing_bytes is zero (doesn?t appear to have been missed packets), but other times you may be able to check that seen_bytes == total_bytes. - Jon From prateekgupta.3991 at gmail.com Mon Apr 28 12:43:05 2014 From: prateekgupta.3991 at gmail.com (Prateek Gupta) Date: Tue, 29 Apr 2014 01:13:05 +0530 Subject: [Bro] bro sequence diagram Message-ID: Hello, I wanted to know if there is an easy way to represent bro working in a sequence diagram? Any help is appreciated. Thank you, Prateek Gupta -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140429/77fff506/attachment.html From coen_bakkers at symantec.com Tue Apr 29 03:46:56 2014 From: coen_bakkers at symantec.com (Coen Bakkers) Date: Tue, 29 Apr 2014 03:46:56 -0700 Subject: [Bro] Bro 2.1 support for sniffing on multiple interfaces faces Message-ID: Does Bro 2.1 support sniffing on several interfaces at the same time? I have tried this now on a dozen of nodes, and the behavior does not seem to be consistent. Note that I am not trying to sniff an outbound and an inbound stream that are related, but I have a tap port on a separate network that I also interested in in covering. Sometimes multiple interfaces in node.cfg will work, but sometimes it makes Bro just hang and not record any of the http, dns, ftp logs etc.. From doug.burks at gmail.com Tue Apr 29 04:03:21 2014 From: doug.burks at gmail.com (Doug Burks) Date: Tue, 29 Apr 2014 07:03:21 -0400 Subject: [Bro] Bro 2.1 support for sniffing on multiple interfaces faces In-Reply-To: References: Message-ID: Hi Coen, Are you perhaps using PF_RING? https://bro-tracker.atlassian.net/browse/BIT-943 The PF_RING multiple interface issue was resolved in Bro 2.2. On Tue, Apr 29, 2014 at 6:46 AM, Coen Bakkers wrote: > Does Bro 2.1 support sniffing on several interfaces at the same time? I have tried this now on a dozen of nodes, and the behavior does not seem to be consistent. > Note that I am not trying to sniff an outbound and an inbound stream that are related, but I have a tap port on a separate network that I also interested in in covering. > Sometimes multiple interfaces in node.cfg will work, but sometimes it makes Bro just hang and not record any of the http, dns, ftp logs etc.. > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Doug Burks From jlay at slave-tothe-box.net Tue Apr 29 04:05:04 2014 From: jlay at slave-tothe-box.net (James Lay) Date: Tue, 29 Apr 2014 05:05:04 -0600 Subject: [Bro] Bro 2.1 support for sniffing on multiple interfaces faces In-Reply-To: References: Message-ID: <1398769504.3680.6.camel@JamesiMac> On Tue, 2014-04-29 at 03:46 -0700, Coen Bakkers wrote: > Does Bro 2.1 support sniffing on several interfaces at the same time? I have tried this now on a dozen of nodes, and the behavior does not seem to be consistent. > Note that I am not trying to sniff an outbound and an inbound stream that are related, but I have a tap port on a separate network that I also interested in in covering. > Sometimes multiple interfaces in node.cfg will work, but sometimes it makes Bro just hang and not record any of the http, dns, ftp logs etc.. > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro I have had great success with starting bro with: bro -i eth0 -i eth1 I am not using broctl. James -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140429/3ba8a38c/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 473 bytes Desc: This is a digitally signed message part Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140429/3ba8a38c/attachment.bin From coen_bakkers at symantec.com Tue Apr 29 04:08:10 2014 From: coen_bakkers at symantec.com (Coen Bakkers) Date: Tue, 29 Apr 2014 04:08:10 -0700 Subject: [Bro] Bro 2.1 support for sniffing on multiple interfaces faces In-Reply-To: References: Message-ID: Hi Doug, I am not using PF_RING, for now we have [bro] type=standalone host=localhost interface=eth0 interface=eth1 interface=eth2 interface=eth3 interface=eth5 interface=eth6 Where I am noticing that when leaving all of these interfaces enabled it may or may not break its working for some reason. When I switched to a single interface it started working but the configuration one of the nodes with above settings seems to works though and I have no clue why. Regards, Coen -----Original Message----- From: Doug Burks [mailto:doug.burks at gmail.com] Sent: Dienstag, 29. April 2014 13:03 To: Coen Bakkers Cc: bro at bro.org Subject: Re: [Bro] Bro 2.1 support for sniffing on multiple interfaces faces Hi Coen, Are you perhaps using PF_RING? https://bro-tracker.atlassian.net/browse/BIT-943 The PF_RING multiple interface issue was resolved in Bro 2.2. On Tue, Apr 29, 2014 at 6:46 AM, Coen Bakkers wrote: > Does Bro 2.1 support sniffing on several interfaces at the same time? I have tried this now on a dozen of nodes, and the behavior does not seem to be consistent. > Note that I am not trying to sniff an outbound and an inbound stream that are related, but I have a tap port on a separate network that I also interested in in covering. > Sometimes multiple interfaces in node.cfg will work, but sometimes it makes Bro just hang and not record any of the http, dns, ftp logs etc.. > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Doug Burks From doug.burks at gmail.com Tue Apr 29 04:16:40 2014 From: doug.burks at gmail.com (Doug Burks) Date: Tue, 29 Apr 2014 07:16:40 -0400 Subject: [Bro] Bro 2.1 support for sniffing on multiple interfaces faces In-Reply-To: References: Message-ID: I think type=standalone only supports one interface. Have you tried replacing the standalone config with a clustered config? https://github.com/bro/broctl/blob/master/etc/node.cfg On Tue, Apr 29, 2014 at 7:08 AM, Coen Bakkers wrote: > Hi Doug, > > I am not using PF_RING, for now we have > > > [bro] > type=standalone > host=localhost > interface=eth0 > interface=eth1 > interface=eth2 > interface=eth3 > interface=eth5 > interface=eth6 > > Where I am noticing that when leaving all of these interfaces enabled it may or may not break its working for some reason. When I switched to a single interface it started working but the configuration one of the nodes with above settings seems to works though and I have no clue why. > > Regards, > > Coen > > -----Original Message----- > From: Doug Burks [mailto:doug.burks at gmail.com] > Sent: Dienstag, 29. April 2014 13:03 > To: Coen Bakkers > Cc: bro at bro.org > Subject: Re: [Bro] Bro 2.1 support for sniffing on multiple interfaces faces > > Hi Coen, > > Are you perhaps using PF_RING? > https://bro-tracker.atlassian.net/browse/BIT-943 > > The PF_RING multiple interface issue was resolved in Bro 2.2. > > On Tue, Apr 29, 2014 at 6:46 AM, Coen Bakkers wrote: >> Does Bro 2.1 support sniffing on several interfaces at the same time? I have tried this now on a dozen of nodes, and the behavior does not seem to be consistent. >> Note that I am not trying to sniff an outbound and an inbound stream that are related, but I have a tap port on a separate network that I also interested in in covering. >> Sometimes multiple interfaces in node.cfg will work, but sometimes it makes Bro just hang and not record any of the http, dns, ftp logs etc.. >> >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > -- > Doug Burks -- Doug Burks From mkhan04 at gmail.com Tue Apr 29 06:17:15 2014 From: mkhan04 at gmail.com (M K) Date: Tue, 29 Apr 2014 09:17:15 -0400 Subject: [Bro] Broctl policy files Message-ID: Is there any guidance/information as to how things should be split up between the 3 types of site policies (manager, proxy, worker). Can it actually make a difference in performance or is it mainly there for organization purposes? As far as I can tell the docs only mention that notice filtering needs to be done on the manager and everything else can go into the generic local.bro file. Is there any further guidance? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140429/25a563c5/attachment.html From seth at icir.org Tue Apr 29 06:37:58 2014 From: seth at icir.org (Seth Hall) Date: Tue, 29 Apr 2014 09:37:58 -0400 Subject: [Bro] Broctl policy files In-Reply-To: References: Message-ID: <82D342D2-1971-464F-93D7-C73802DDF471@icir.org> On Apr 29, 2014, at 9:17 AM, M K wrote: > Is there any guidance/information as to how things should be split up between the 3 types of site policies (manager, proxy, worker). Can it actually make a difference in performance or is it mainly there for organization purposes? This has been an especially weak area for us regarding documentation. I've actually been considering removing the local-manager.bro, local-worker.bro, and local-proxy.bro files for quite a while now because in most cases the frameworks are cluster capable and you don't need to do anything special (i.e. the right stuff runs in the right place automatically). > As far as I can tell the docs only mention that notice filtering needs to be done on the manager and everything else can go into the generic local.bro file. Is there any further guidance? You can do the filtering in local.bro. The local-*.bro files are a hold over from back when we were still very unclear on how to achieve cluster transparency in a lot of code we were writing. As more of our code has grown to cope with clusters automatically we've never found a strong need for users to be exposed to the node differentiation which is frequently quite difficult to get right anyway. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140429/39edd981/attachment.bin From mkhan04 at gmail.com Tue Apr 29 08:40:20 2014 From: mkhan04 at gmail.com (M K) Date: Tue, 29 Apr 2014 11:40:20 -0400 Subject: [Bro] Bro Cluster Elasticsearch rotation_interval ignored Message-ID: I'm testing out the ElasticSearch writer in a Bro Cluster (2.2 release) along with the Ascii writer. I've set LogRotationInterval to an hour (3600) in broctl.cfg which I know sets or overrides Log::default_rotation_interval and in my local.bro I've overridden the rotation_interval parameter of the ElasticSearch Logger (defined in logs-to-elasticsearch policy) to be every 24 hours. Apparently, Bro seems to be ignoring the rotation_interval value. I've tried not setting LogRotationInterval and setting Log::default_rotation_interval in my local.bro file but i got similar results. Is there anyway to have the Ascii writer use a 1hr rotation interval while the ElasticSearch writer uses a different one? Looking through the docs/code it doesn't look like LogAscii has a rotation_interval of its own. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140429/8822c8e2/attachment.html From bruisebrotherprobert at gmail.com Tue Apr 29 14:49:04 2014 From: bruisebrotherprobert at gmail.com (Bob Probert) Date: Tue, 29 Apr 2014 14:49:04 -0700 Subject: [Bro] Filenames not extracted in files.log Message-ID: Hi all, After looking at an aggregate 30 days of files.log in Splunk, I noticed that 98% of the files identified by Bro have no filenames associated with them. While I haven't done any rigorous testing of this, it just seems wrong. Is this a known bug? Is anyone else experiencing this? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140429/a29f36e1/attachment.html From charles.fair at mac.com Tue Apr 29 19:42:20 2014 From: charles.fair at mac.com (Charles A. Fair) Date: Tue, 29 Apr 2014 22:42:20 -0400 Subject: [Bro] Filenames not extracted in files.log In-Reply-To: References: Message-ID: <0672E9AE-25B1-44C9-93F2-D0AF6B6176F8@mac.com> The file analysis framework does not annotate the original file names as I understand it. I am not sure why this is. What it does do is assign a Unique File ID to each file that can be used to search search across different Bro logs. Chuck On Apr 29, 2014, at 5:49 PM, Bob Probert wrote: > Hi all, > > After looking at an aggregate 30 days of files.log in Splunk, I noticed that 98% of the files identified by Bro have no filenames associated with them. > > While I haven't done any rigorous testing of this, it just seems wrong. Is this a known bug? Is anyone else experiencing this? > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From seth at icir.org Tue Apr 29 20:08:57 2014 From: seth at icir.org (Seth Hall) Date: Tue, 29 Apr 2014 23:08:57 -0400 Subject: [Bro] Filenames not extracted in files.log In-Reply-To: References: Message-ID: On Apr 29, 2014, at 5:49 PM, Bob Probert wrote: > After looking at an aggregate 30 days of files.log in Splunk, I noticed that 98% of the files identified by Bro have no filenames associated with them. It's because 98% of files transferred over the internet have no reliable name associated with them. :)  Since most of the "files" in your files.log are http content and with HTTP there is a mechanism for transferring a file name along with the data (content-disposition header) it's a pretty bad idea to trust anything in the url as a file name. You would end up with lots of "files" being transferred named "index.php" and "index.asp" which I don't think you want either. We heavily tend toward conservatism in cases where an incorrect interpretation could arise. All of that said, this is something that you could write an extension script to add to your files.log if you really want it. I'll leave it as an exercise to you to write the script though. ;) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140429/69018e04/attachment.bin From seth at icir.org Tue Apr 29 20:16:44 2014 From: seth at icir.org (Seth Hall) Date: Tue, 29 Apr 2014 23:16:44 -0400 Subject: [Bro] Filenames not extracted in files.log In-Reply-To: <0672E9AE-25B1-44C9-93F2-D0AF6B6176F8@mac.com> References: <0672E9AE-25B1-44C9-93F2-D0AF6B6176F8@mac.com> Message-ID: On Apr 29, 2014, at 10:42 PM, Charles A. Fair wrote: > The file analysis framework does not annotate the original file names as I understand it. The file analysis framework itself doesn't do it. Some of the protocol scripts poke forward into files transferred and annotate the files log with a file name if a suitable one was found. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140429/0bc8f9a9/attachment.bin From mkhan04 at gmail.com Wed Apr 30 06:42:59 2014 From: mkhan04 at gmail.com (M K) Date: Wed, 30 Apr 2014 09:42:59 -0400 Subject: [Bro] Bro Cluster Dropped Packets Message-ID: Is there any way to determine the cause of dropped packets? I'm running Bro Cluster (2.2) on a single machine with 1 manager, 1 proxy and 10 workers. The total number of workers is much less than the number of cpus in this machine (system load doesn't usually get higher than 2 and individual worker processes hover at around 30-40% cpu utilization). The machine has PF_Ring and related ethernet drivers installed. After looking at netstats there's always some dropped packets. The occasional dropped packet isn't usually a cause for concern but some workers show large numbers of dropped packets. I'd like to know what part of the process is bottle-necked and causing packets to be dropped. The documentation mentions that broctl cron logs stats but doesn't mention where they're located (didn't see anything in spool that looked like cluster runtime stats) or how to view the data. Anyone have any ideas? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/51cba00f/attachment.html From bruisebrotherprobert at gmail.com Wed Apr 30 06:47:43 2014 From: bruisebrotherprobert at gmail.com (Bob Probert) Date: Wed, 30 Apr 2014 06:47:43 -0700 Subject: [Bro] Filenames not extracted in files.log In-Reply-To: References: <0672E9AE-25B1-44C9-93F2-D0AF6B6176F8@mac.com> Message-ID: Thanks for the thoughtful replies Chuck and Seth. I will add this field to my files log and name it "inferred_filename". For everyone else on the list, I will forward this along when I'm finished. Seth - I don't agree with your assumption that I don't want to see the filename from the URL, I think that this is pretty relevant data, especially when viewed from a security context. I do however agree that one should definitely not "trust" the URL. This is the beauty of Bro - I can add and remove this data at my discretion :-). Thanks again!! On Tue, Apr 29, 2014 at 8:16 PM, Seth Hall wrote: > > On Apr 29, 2014, at 10:42 PM, Charles A. Fair > wrote: > > > The file analysis framework does not annotate the original file names as > I understand it. > > The file analysis framework itself doesn't do it. Some of the protocol > scripts poke forward into files transferred and annotate the files log with > a file name if a suitable one was found. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/30655a17/attachment.html From seth at icir.org Wed Apr 30 06:56:56 2014 From: seth at icir.org (Seth Hall) Date: Wed, 30 Apr 2014 09:56:56 -0400 Subject: [Bro] Filenames not extracted in files.log In-Reply-To: References: <0672E9AE-25B1-44C9-93F2-D0AF6B6176F8@mac.com> Message-ID: On Apr 30, 2014, at 9:47 AM, Bob Probert wrote: > This is the beauty of Bro - I can add and remove this data at my discretion :-). Exactly! :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/18c05a51/attachment.bin From jessebowling at gmail.com Wed Apr 30 06:59:50 2014 From: jessebowling at gmail.com (Jesse Bowling) Date: Wed, 30 Apr 2014 09:59:50 -0400 Subject: [Bro] Bro Cluster Dropped Packets In-Reply-To: References: Message-ID: Hello MK, Would you happen to be running PF_RING 5.6.2? If so, you might want to join in on this thread on the ntop-misc list: http://www.gossamer-threads.com/lists/ntop/misc/34343 To speak more directly to the question you asked, you can certainly look at the stats from ifconfig to see if your card is dropping packets (something I'm seeing with the above issue), and you can also look at the stats in /proc/net/pf_ring/${PID_FROM_EACH_BRO_WORKER}* . I'm not sure where any Bro specific stats may be kept... Cheers, Jesse On Wed, Apr 30, 2014 at 9:42 AM, M K wrote: > Is there any way to determine the cause of dropped packets? I'm running > Bro Cluster (2.2) on a single machine with 1 manager, 1 proxy and 10 > workers. The total number of workers is much less than the number of cpus > in this machine (system load doesn't usually get higher than 2 and > individual worker processes hover at around 30-40% cpu utilization). The > machine has PF_Ring and related ethernet drivers installed. After looking > at netstats there's always some dropped packets. The occasional dropped > packet isn't usually a cause for concern but some workers show large > numbers of dropped packets. I'd like to know what part of the process is > bottle-necked and causing packets to be dropped. > > > The documentation mentions that broctl cron logs stats but doesn't mention > where they're located (didn't see anything in spool that looked like > cluster runtime stats) or how to view the data. > > Anyone have any ideas? > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Jesse Bowling -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/5ae32569/attachment.html From seth at icir.org Wed Apr 30 07:03:56 2014 From: seth at icir.org (Seth Hall) Date: Wed, 30 Apr 2014 10:03:56 -0400 Subject: [Bro] Bro Cluster Dropped Packets In-Reply-To: References: Message-ID: <1940835E-49F5-4F10-BED5-E910FF46B011@icir.org> On Apr 30, 2014, at 9:42 AM, M K wrote: > After looking at netstats there's always some dropped packets. Generally with network monitoring you're going to have some degree of dropped packets even on the most appropriately scaled systems. What you generally want to do is fight to keep the percentage of dropped packets as consistently low as possible. Also, when you're using things like PF_Ring that do odd things with nic buffers you have to be very leery of the stats reported from the NIC. Even in the best of cases those stats aren't very trustable. What we generally recommend for our users is to run the misc/capture-loss script. You can load it by adding this line to local.bro (and doing install then restart in broctl) @load misc/capture-loss This will create a capture-loss.log file that is written to every 15 minutes (by default) which will tell your apparent packet loss measured by watching non-seen but acked data segments in TCP streams. This can also be confusing for people sometimes it will measure traffic loss happening upstream in your network. Here is a blog post where someone had packet loss happening on a network device before the packets were even sent to their box running Bro: http://brotocols.blogspot.com/2013/08/bro-knows-packet-loss.html .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/c460ab3b/attachment.bin From john.h.hoyt at gmail.com Wed Apr 30 07:09:17 2014 From: john.h.hoyt at gmail.com (John Hoyt) Date: Wed, 30 Apr 2014 10:09:17 -0400 Subject: [Bro] Module pf_ring not found. Message-ID: I followed the steps for configuring load balancing here: http://www.bro.org/documentation/load-balancing.html Everything worked great, but I had to restart the server and now I can't load the PF_RING module. sudo modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 FATAL: Module pf_ring not found. When I run: ldd /bro/bin/bro | grep pcap It get the following: libpcap.so.1 => /opt/pfring/lib/libpcap.so.1 (0x00007f3aeb8ff000) In the instructions it stated: "Refer to the documentation for your Linux distribution on how to load the pf_ring module at boot time." But I wasn't able to find any details on how to do so. Any suggestions? Thanks, John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/98fe98da/attachment.html From jessebowling at gmail.com Wed Apr 30 07:27:50 2014 From: jessebowling at gmail.com (Jesse Bowling) Date: Wed, 30 Apr 2014 10:27:50 -0400 Subject: [Bro] Module pf_ring not found. In-Reply-To: References: Message-ID: On Wed, Apr 30, 2014 at 10:09 AM, John Hoyt wrote: > FATAL: Module pf_ring not found. Hi John, FATAL: Module pf_ring not found. This would seem to indicate that the kernel can't find the PF_RING module...Did you happen to update your kernel before that reboot? If so, you'd likely need to re-compile the PF_RING/kernel and PF_RING/driver sections to work with the new kernel. Cheers, Jesse -- Jesse Bowling -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/03f8b375/attachment.html From JAzoff at albany.edu Wed Apr 30 07:32:03 2014 From: JAzoff at albany.edu (Justin Azoff) Date: Wed, 30 Apr 2014 10:32:03 -0400 Subject: [Bro] Module pf_ring not found. In-Reply-To: References: Message-ID: <20140430143203.GG13320@datacomm.albany.edu> On Wed, Apr 30, 2014 at 10:09:17AM -0400, John Hoyt wrote: > I followed the steps for configuring load balancing here: > > http://www.bro.org/documentation/load-balancing.html > > Everything worked great, but I had to restart the server and now I can't load > the PF_RING module. > > > sudo modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > > FATAL: Module pf_ring not found. Did you upgrade your kernel version before the reboot? You may need to reinstall the kernel module Try going back into the PF_RING-5.6.1/kernel directory and running make install upstream pf_ring does not use dkms so if you upgrade the kernel you will need to reinstall the modules.. -- -- Justin Azoff From john.h.hoyt at gmail.com Wed Apr 30 08:45:25 2014 From: john.h.hoyt at gmail.com (John Hoyt) Date: Wed, 30 Apr 2014 11:45:25 -0400 Subject: [Bro] Module pf_ring not found. In-Reply-To: <20140430143203.GG13320@datacomm.albany.edu> References: <20140430143203.GG13320@datacomm.albany.edu> Message-ID: I went back and re-ran make install from the kernel directory and got the following output: mkdir -p /lib/modules/3.8.0-38-generic/kernel/net/pf_ring cp *.ko /lib/modules/3.8.0-38-generic/kernel/net/pf_ring cp linux/pf_ring.h /usr/include/linux /sbin/depmod 3.8.0-38-generic Then I re-ran the command: modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 And got the error: FATAL: Error inserting pf_ring (/lib/modules/3.8.0-38-generic/kernel/net/pf_ring/pf_ring.ko): Invalid module format Then, I went back through the steps of re-running make install, beginning with the directoryPF_RING-5.6.1/userland/lib Still no luck. I checked the make file for each before re-running the install to make sure that the prefix was still set. Thanks, John On Wed, Apr 30, 2014 at 10:32 AM, Justin Azoff wrote: > On Wed, Apr 30, 2014 at 10:09:17AM -0400, John Hoyt wrote: > > I followed the steps for configuring load balancing here: > > > > http://www.bro.org/documentation/load-balancing.html > > > > Everything worked great, but I had to restart the server and now I can't > load > > the PF_RING module. > > > > > > sudo modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > > > > FATAL: Module pf_ring not found. > > Did you upgrade your kernel version before the reboot? You may need to > reinstall the kernel module > > Try going back into the PF_RING-5.6.1/kernel directory and running > > make install > > upstream pf_ring does not use dkms so if you upgrade the kernel you will > need to reinstall the modules.. > > -- > -- Justin Azoff > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/5520a1f9/attachment.html From JAzoff at albany.edu Wed Apr 30 09:02:36 2014 From: JAzoff at albany.edu (Justin Azoff) Date: Wed, 30 Apr 2014 12:02:36 -0400 Subject: [Bro] Module pf_ring not found. In-Reply-To: References: <20140430143203.GG13320@datacomm.albany.edu> Message-ID: <20140430160236.GH13320@datacomm.albany.edu> On Wed, Apr 30, 2014 at 11:45:25AM -0400, John Hoyt wrote: > I went back and re-ran make install from the kernel directory and got the > following output: > > > mkdir -p /lib/modules/3.8.0-38-generic/kernel/net/pf_ring > > cp *.ko /lib/modules/3.8.0-38-generic/kernel/net/pf_ring > > cp linux/pf_ring.h /usr/include/linux > > /sbin/depmod 3.8.0-38-generic > > Then I re-ran the command: > > modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > > And got the error:? > > FATAL: Error inserting pf_ring (/lib/modules/3.8.0-38-generic/kernel/net/ > pf_ring/pf_ring.ko): Invalid module format Ah, try doing a make clean and then a make install. That should install a working module build for the right kernel. > > Then, I went back through the steps of re-running make install, beginning with > the directoryPF_RING-5.6.1/userland/lib? Your problem is with the kernel module so reinstalling the stuff in lib/ would not have helped. -- -- Justin Azoff From wsladekjr at hotmail.com Wed Apr 30 09:09:07 2014 From: wsladekjr at hotmail.com (Ward Sladek) Date: Wed, 30 Apr 2014 11:09:07 -0500 Subject: [Bro] Module pf_ring not found. In-Reply-To: References: , <20140430143203.GG13320@datacomm.albany.edu>, Message-ID: PF_RING doesn't/didn't support kernel 3.8 and I'm not sure if that is still the case today. You might try a newer version of PF_RING. Date: Wed, 30 Apr 2014 11:45:25 -0400 From: john.h.hoyt at gmail.com To: JAzoff at albany.edu CC: bro at bro.org Subject: Re: [Bro] Module pf_ring not found. I went back and re-ran make install from the kernel directory and got the following output: mkdir -p /lib/modules/3.8.0-38-generic/kernel/net/pf_ring cp *.ko /lib/modules/3.8.0-38-generic/kernel/net/pf_ring cp linux/pf_ring.h /usr/include/linux /sbin/depmod 3.8.0-38-generic Then I re-ran the command:modprobe pf_ring enable_tx_capture=0 min_num_slots=32768And got the error: FATAL: Error inserting pf_ring (/lib/modules/3.8.0-38-generic/kernel/net/pf_ring/pf_ring.ko): Invalid module format Then, I went back through the steps of re-running make install, beginning with the directoryPF_RING-5.6.1/userland/lib Still no luck. I checked the make file for each before re-running the install to make sure that the prefix was still set. Thanks,John On Wed, Apr 30, 2014 at 10:32 AM, Justin Azoff wrote: On Wed, Apr 30, 2014 at 10:09:17AM -0400, John Hoyt wrote: > I followed the steps for configuring load balancing here: > > http://www.bro.org/documentation/load-balancing.html > > Everything worked great, but I had to restart the server and now I can't load > the PF_RING module. > > > sudo modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > > FATAL: Module pf_ring not found. Did you upgrade your kernel version before the reboot? You may need to reinstall the kernel module Try going back into the PF_RING-5.6.1/kernel directory and running make install upstream pf_ring does not use dkms so if you upgrade the kernel you will need to reinstall the modules.. -- -- Justin Azoff _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/79b99f97/attachment.html From john.h.hoyt at gmail.com Wed Apr 30 10:03:12 2014 From: john.h.hoyt at gmail.com (John Hoyt) Date: Wed, 30 Apr 2014 13:03:12 -0400 Subject: [Bro] Module pf_ring not found. In-Reply-To: References: <20140430143203.GG13320@datacomm.albany.edu> Message-ID: I'm using PF_RING 6.0. The first time I went through the steps it was working. It was only after restarting, that I couldn't reload the module. On Wed, Apr 30, 2014 at 12:09 PM, Ward Sladek wrote: > PF_RING doesn't/didn't support kernel 3.8 and I'm not sure if that is > still the case today. You might try a newer version of PF_RING. > > ------------------------------ > Date: Wed, 30 Apr 2014 11:45:25 -0400 > From: john.h.hoyt at gmail.com > To: JAzoff at albany.edu > CC: bro at bro.org > Subject: Re: [Bro] Module pf_ring not found. > > > I went back and re-ran make install from the kernel directory and got the > following output: > > mkdir -p /lib/modules/3.8.0-38-generic/kernel/net/pf_ring > cp *.ko /lib/modules/3.8.0-38-generic/kernel/net/pf_ring > cp linux/pf_ring.h /usr/include/linux > /sbin/depmod 3.8.0-38-generic > > Then I re-ran the command: > > modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > > And got the error: > FATAL: Error inserting pf_ring > (/lib/modules/3.8.0-38-generic/kernel/net/pf_ring/pf_ring.ko): Invalid > module format > > > Then, I went back through the steps of re-running make install, beginning > with the directoryPF_RING-5.6.1/userland/lib > > Still no luck. > > I checked the make file for each before re-running the install to make > sure that the prefix was still set. > > Thanks, > John > > > > > > On Wed, Apr 30, 2014 at 10:32 AM, Justin Azoff wrote: > > On Wed, Apr 30, 2014 at 10:09:17AM -0400, John Hoyt wrote: > > I followed the steps for configuring load balancing here: > > > > http://www.bro.org/documentation/load-balancing.html > > > > Everything worked great, but I had to restart the server and now I can't > load > > the PF_RING module. > > > > > > sudo modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > > > > FATAL: Module pf_ring not found. > > Did you upgrade your kernel version before the reboot? You may need to > reinstall the kernel module > > Try going back into the PF_RING-5.6.1/kernel directory and running > > make install > > upstream pf_ring does not use dkms so if you upgrade the kernel you will > need to reinstall the modules.. > > -- > -- Justin Azoff > > > > _______________________________________________ Bro mailing list > bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/a8d0fe9a/attachment.html From jxbatchelor at gmail.com Wed Apr 30 10:48:43 2014 From: jxbatchelor at gmail.com (Jason Batchelor) Date: Wed, 30 Apr 2014 12:48:43 -0500 Subject: [Bro] Bro Log Filename Question Message-ID: Hello Bro Community: I was wondering if there was an easy way to modify log filenames that are placed into the spool directory. All I would like to to, is to simply append 'bro.' to the beginning of each filename. I searched around a bit thinking there may be a simple configuration option I could modify in the broctl.cfg file. Unfortunately however, I have not come upon any solution yet and feel like I am likely missing something obvious. As an example, I would like the prefix to be something like 'bro.conn.log' instead of 'conn.log' for all files being written to the '/var/opt/bro/spool/bro' directory. Is there a simple way to do this using the Bro application? Thanks very much for your time and assistance. -Jason -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/3bb026d4/attachment.html From kmcmahon at mitre.org Wed Apr 30 12:10:44 2014 From: kmcmahon at mitre.org (McMahon, Kevin J) Date: Wed, 30 Apr 2014 19:10:44 +0000 Subject: [Bro] Bro Log Filename Question In-Reply-To: References: Message-ID: <00D3CD29F7C24A44B4D23450BB8E55B30AB9AC9D@IMCMBX03.MITRE.ORG> Here?s what I did in Bro 2.1 (I haven?t tried this particular option in 2.2 yet). It?s a little hacky, but it works and I can use different values for different instantiations: Change the frameworks/logging/main.bro script to include a ?const log_prefix = ?? &redef, then change the default_path_func to include this prefix when the function returns by cat?ing the prefix with whatever was being returned (three places in 2.1). Then you can add: Redef Log::log_prefix = ?bro.?; in your run specific file to allow for variations. A quick look at 2.2 seems to indicate that the same operation will work with that version. From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Jason Batchelor Sent: Wednesday, April 30, 2014 1:49 PM To: bro at bro.org Subject: [Bro] Bro Log Filename Question Hello Bro Community: I was wondering if there was an easy way to modify log filenames that are placed into the spool directory. All I would like to to, is to simply append 'bro.' to the beginning of each filename. I searched around a bit thinking there may be a simple configuration option I could modify in the broctl.cfg file. Unfortunately however, I have not come upon any solution yet and feel like I am likely missing something obvious. As an example, I would like the prefix to be something like 'bro.conn.log' instead of 'conn.log' for all files being written to the '/var/opt/bro/spool/bro' directory. Is there a simple way to do this using the Bro application? Thanks very much for your time and assistance. -Jason -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/4615df15/attachment.html From seth at icir.org Wed Apr 30 12:49:52 2014 From: seth at icir.org (Seth Hall) Date: Wed, 30 Apr 2014 15:49:52 -0400 Subject: [Bro] Bro Log Filename Question In-Reply-To: <00D3CD29F7C24A44B4D23450BB8E55B30AB9AC9D@IMCMBX03.MITRE.ORG> References: <00D3CD29F7C24A44B4D23450BB8E55B30AB9AC9D@IMCMBX03.MITRE.ORG> Message-ID: <724C29FB-19F8-4FE2-B148-80F0752D48E2@icir.org> On Apr 30, 2014, at 3:10 PM, McMahon, Kevin J wrote: > Redef Log::log_prefix = ?bro.?; Nice! Submit a patch perhaps? I think that's something we could reasonably take, especially since there are already two people obviously wanting it. ;) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140430/e2239371/attachment.bin