From zer0d0y at foxmail.com Mon Jul 1 08:19:42 2019 From: zer0d0y at foxmail.com (=?ISO-8859-1?B?WmVyMGQweQ==?=) Date: Mon, 1 Jul 2019 23:19:42 +0800 Subject: [Zeek] Help to detect CVE-2019-11479 Message-ID: Hi all, Recently,Netflix has identified several TCP networking vulnerabilities in FreeBSD and Linux kernels. (#CVE-2019-11479 : Excess Resource Consumption Due to Low MSS Values (all Linux versions) We want to detecting this flaw with Zeek,but looks like there's no way to get the MSS(Maximum segment size) value of TCP Option,any ideas? Thanks, ------------------ Zer0d0y Threat Detection & Hunting -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190701/c7901c3c/attachment.html From fyiohhai at gmail.com Mon Jul 1 14:16:17 2019 From: fyiohhai at gmail.com (Enki) Date: Mon, 01 Jul 2019 15:16:17 -0600 Subject: [Zeek] (no subject) In-Reply-To: References: Message-ID: <16baf6540e8.2825.f532124d3e5409d082c18343f797a353@gmail.com> I took a look at the dp3 files, but I couldn't find anything that helps with my use case - - maybe I'm just blind and I missed it. However, I did find this older question that fits pretty close to mine: https://marc.info/?l=bro&m=146194027831545&w=2 I still feel like there's probably a better way to solve this issue than what's presented. I'll try it out though, unless anyone knows of any better methods. On June 27, 2019 2:09:31 PM Hugo wrote: > Hi Enki, > > I have not read C37.118 in details before. But I contributed the DNP3 > analyzer in Bro both on top of TCP and UPD, may be you can take a look. > DNP3 also have some similar characteristics, like the parsing of the > current packets depends on the previous packet. Hope this helps. > > Best, > > Hui Lin > > On Thu, Jun 27, 2019 at 10:09 AM Enki wrote: > I?m trying to create my first protocol analyzer with BinPac > for the synchrophasor protocol (IEEE Std C37.118) ? from what I can tell, > nobody has made an analyzer for it yet. I'm trying to define the message format > in synchrophasor-protocol.pac. However, stuff like the format of data > packets are based on a previously sent configuration packet. How do I write > synchrophasor-protocol.pac so I can parse them based on the previously > sent packet? Here?s some documentation on the protocol > if you need it: http://smartgridcenter.tamu.edu/resume/pdf/1/SynPhasor_std.pdf > > Again, this is my first time trying to write a protocol > analyzer with BinPac, so sorry if this is obvious. > > Thank you > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek Sent with AquaMail for Android https://www.mobisystems.com/aqua-mail -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190701/79b54a14/attachment.html From tet68mt at gmail.com Mon Jul 1 16:33:19 2019 From: tet68mt at gmail.com (Matt Trostel) Date: Mon, 1 Jul 2019 18:33:19 -0500 Subject: [Zeek] Help to detect CVE-2019-11479 In-Reply-To: References: Message-ID: Hi Zer0d0y, You should be able to pull these values from the connection_SYN_packet event (https://docs.zeek.org/en/stable/script-reference/proto-analyzers.html#id-connection_SYN_packet ). The SYN packet (https://docs.zeek.org/en/stable/scripts/base/init-bare.bro.html#type-SYN_packet ) contains the MSS value. I hope this helps. - Matt > On Jul 1, 2019, at 10:19, Zer0d0y wrote: > > Hi all, > Recently,Netflix has identified several TCP networking vulnerabilities in FreeBSD and Linux kernels. (#CVE-2019-11479 : Excess Resource Consumption Due to Low MSS Values (all Linux versions) > > We want to detecting this flaw with Zeek,but looks like there's no way to get the MSS(Maximum segment size) value of TCP Option,any ideas? > > Thanks, > > ------------------ > > Zer0d0y > Threat Detection & Hunting > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190701/3ba91e21/attachment.html From tet68mt at gmail.com Mon Jul 1 16:47:28 2019 From: tet68mt at gmail.com (Matt Trostel) Date: Mon, 1 Jul 2019 18:47:28 -0500 Subject: [Zeek] Help to detect CVE-2019-11479 In-Reply-To: References: Message-ID: <07D1E37C-C1C4-446A-9997-5AEE6EDAC4E8@gmail.com> It just dawned on me. I did this for CVE-2019-11477 the other day. The below should add ?mss? and ?sack_ok? fields to your CONN log for all TCP connections. I?m not great at Zeek scripting, so take this with some caution. I?m sure there are folks here on the list that could better optimize this. :) redef record Conn::Info += { mss: count &optional &log; sack_ok: bool &optional &log; }; redef record connection += { mss: count &optional &log; sack_ok: bool &optiional &log; }; event connection_SYN_packet(c: connection, pkt: SYN_packet) { c$mss = pkt$MSS; c$sack_ok = pkt$SACK_OK; } event connection_state_remove(c: connection) { if ( c ?$ mss ) c$conn$mss = c$mss; if (c ?$ sack_ok ) c$conn$sack_ok = c$sack_ok; } > On Jul 1, 2019, at 18:33, Matt Trostel wrote: > > Hi Zer0d0y, > > You should be able to pull these values from the connection_SYN_packet event (https://docs.zeek.org/en/stable/script-reference/proto-analyzers.html#id-connection_SYN_packet ). > > The SYN packet (https://docs.zeek.org/en/stable/scripts/base/init-bare.bro.html#type-SYN_packet ) contains the MSS value. > > I hope this helps. > > - Matt > > >> On Jul 1, 2019, at 10:19, Zer0d0y > wrote: >> >> Hi all, >> Recently,Netflix has identified several TCP networking vulnerabilities in FreeBSD and Linux kernels. (#CVE-2019-11479 : Excess Resource Consumption Due to Low MSS Values (all Linux versions) >> >> We want to detecting this flaw with Zeek,but looks like there's no way to get the MSS(Maximum segment size) value of TCP Option,any ideas? >> >> Thanks, >> >> ------------------ >> >> Zer0d0y >> Threat Detection & Hunting >> _______________________________________________ >> Zeek mailing list >> zeek at zeek.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190701/88c346b0/attachment-0001.html From zer0d0y at foxmail.com Mon Jul 1 17:37:55 2019 From: zer0d0y at foxmail.com (=?gb18030?B?WmVyMGQweQ==?=) Date: Tue, 2 Jul 2019 08:37:55 +0800 Subject: [Zeek] Help to detect CVE-2019-11479 In-Reply-To: <07D1E37C-C1C4-446A-9997-5AEE6EDAC4E8@gmail.com> References: <07D1E37C-C1C4-446A-9997-5AEE6EDAC4E8@gmail.com> Message-ID: Matt, Thanks for the reply.That works! ------------------ Zer0d0y Threat Detection & Hunting ------------------ Original ------------------ From: "Matt Trostel";; Send time: Tuesday, Jul 2, 2019 7:47 AM To: "Zer0d0y"; Cc: "zeek"; Subject: Re: [Zeek] Help to detect CVE-2019-11479 It just dawned on me. I did this for CVE-2019-11477 the other day. The below should add ?mss? and ?sack_ok? fields to your CONN log for all TCP connections.I?m not great at Zeek scripting, so take this with some caution. I?m sure there are folks here on the list that could better optimize this. :) redef record Conn::Info += { mss: count &optional &log; sack_ok: bool &optional &log; }; redef record connection += { mss: count &optional &log; sack_ok: bool &optiional &log; }; event connection_SYN_packet(c: connection, pkt: SYN_packet) { c$mss = pkt$MSS; c$sack_ok = pkt$SACK_OK; } event connection_state_remove(c: connection) { if ( c ?$ mss ) c$conn$mss = c$mss; if (c ?$ sack_ok ) c$conn$sack_ok = c$sack_ok; } On Jul 1, 2019, at 18:33, Matt Trostel wrote: Hi Zer0d0y, You should be able to pull these values from the connection_SYN_packet event (https://docs.zeek.org/en/stable/script-reference/proto-analyzers.html#id-connection_SYN_packet). The SYN packet (https://docs.zeek.org/en/stable/scripts/base/init-bare.bro.html#type-SYN_packet) contains the MSS value. I hope this helps. - Matt On Jul 1, 2019, at 10:19, Zer0d0y wrote: Hi all, Recently,Netflix has identified several TCP networking vulnerabilities in FreeBSD and Linux kernels. (#CVE-2019-11479 : Excess Resource Consumption Due to Low MSS Values (all Linux versions) We want to detecting this flaw with Zeek,but looks like there's no way to get the MSS(Maximum segment size) value of TCP Option,any ideas? Thanks, ------------------ Zer0d0y Threat Detection & Hunting _______________________________________________ Zeek mailing list zeek at zeek.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190702/8729b3fd/attachment.html From ambros.novak.89 at gmail.com Tue Jul 2 05:48:46 2019 From: ambros.novak.89 at gmail.com (Ambros Novak) Date: Tue, 2 Jul 2019 08:48:46 -0400 Subject: [Zeek] Changing logging defaults Message-ID: <708764A4-45E8-42D9-A9A0-2A1E1B1D5A04@gmail.com> Good Day, I?m running an older version of Bro and would like to change some default logging options using the local.bro. For example, in trying to change DNS?s rejected to T because it?s defaulted to F. I?m able to change it in the module file, but I would rather do it in local.bro. Thanks you for the assistance. Ambros ?? From apumphrey at bricata.com Tue Jul 2 12:26:19 2019 From: apumphrey at bricata.com (Adam Pumphrey) Date: Tue, 2 Jul 2019 19:26:19 +0000 Subject: [Zeek] Changing logging defaults In-Reply-To: <708764A4-45E8-42D9-A9A0-2A1E1B1D5A04@gmail.com> References: <708764A4-45E8-42D9-A9A0-2A1E1B1D5A04@gmail.com> Message-ID: <52F2F32A-B9C0-4BB1-94F7-BE933CA51A06@bricata.com> Hey Ambros You can use Filters in local.zeek (local.bro in your case) to customize many of Zeek's default logging options. A "default" filter is applied to each log stream, you have the option to customize the default filter or remove it and add a new one (or add multiple filters). That's all done inside of a zeek_init() event handler. To learn more I suggest reading this https://docs.zeek.org/en/stable/frameworks/logging.html#filters and checking out this blog post https://blog.zeek.org/2012/02/filtering-logs-with-bro.html (its old but still relevant). To manipulate field values, or add new ones, you'll need to use the appropriate event handlers. This is one of those "it depends" situations. In many cases you can use a protocol's log_* event (log_dns for example) to do what you need but, depending on the situation, there may be others that are a better fit. I caution against modifying the values of built-in fields unless you have a compelling reason to do so though. Take the "rejected" field in the DNS stream for example, it tells you whether the DNS server refused to respond to a client request for policy reasons. It only gets set to T if the server actually rejected the query; in other words "not rejected" is assumed unless the server says otherwise. That said, why would you need to change the value? Maybe you're actually trying to filter which DNS events get logged? If so, you can accomplish that using the Filters I mentioned above with a predicate (pred) function. Take a look at this for more info https://docs.zeek.org/en/stable/frameworks/logging.html#filter-log-records. Hope that helps. Adam ?On 7/2/19, 9:53 AM, "zeek-bounces at zeek.org on behalf of Ambros Novak" wrote: Good Day, I?m running an older version of Bro and would like to change some default logging options using the local.bro. For example, in trying to change DNS?s rejected to T because it?s defaulted to F. I?m able to change it in the module file, but I would rather do it in local.bro. Thanks you for the assistance. Ambros ?? _______________________________________________ Zeek mailing list zeek at zeek.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek From mauro.palumbo at aizoon.it Wed Jul 3 01:17:12 2019 From: mauro.palumbo at aizoon.it (Palumbo Mauro) Date: Wed, 3 Jul 2019 08:17:12 +0000 Subject: [Zeek] json formatter Message-ID: <8fecd8246ee14b3194d0c4fd891497c9@SRVEX03.aizoon.local> Hi everybody, we had some issues with the json formatter, which we are using with kafka. We temporarily patched this and it is working fine for now. However, probably a more general revision could be better, possibly using an external library for the conversion to json format. I saw in github somebody is already working on it, right? So, I guess we just have to wait for next zeek's version? Cheers Mauro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190703/37bf8a42/attachment.html From johanna at icir.org Wed Jul 3 01:43:24 2019 From: johanna at icir.org (Johanna Amann) Date: Wed, 3 Jul 2019 10:43:24 +0200 Subject: [Zeek] json formatter In-Reply-To: <8fecd8246ee14b3194d0c4fd891497c9@SRVEX03.aizoon.local> References: <8fecd8246ee14b3194d0c4fd891497c9@SRVEX03.aizoon.local> Message-ID: <20190703084318.huy3zi5y47rqwdbh@Tranquility.local> Hi Mauro, > we had some issues with the json formatter, which we are using with > kafka. We temporarily patched this and it is working fine for now. > However, probably a more general revision could be better, possibly > using an external library for the conversion to json format. I saw in > github somebody is already working on it, right? So, I guess we just > have to wait for next zeek's version? Yup, that is the plan. For more information see the github bug at https://github.com/zeek/zeek/issues/150 and the pull request at https://github.com/zeek/zeek/pull/451; I assume this will be merged into master in not too long. Johanna From mauro.palumbo at aizoon.it Wed Jul 3 01:47:21 2019 From: mauro.palumbo at aizoon.it (Palumbo Mauro) Date: Wed, 3 Jul 2019 08:47:21 +0000 Subject: [Zeek] R: json formatter In-Reply-To: <20190703084318.huy3zi5y47rqwdbh@Tranquility.local> References: <8fecd8246ee14b3194d0c4fd891497c9@SRVEX03.aizoon.local> <20190703084318.huy3zi5y47rqwdbh@Tranquility.local> Message-ID: Hi Johanna, great! Then we just have to wait... ;-) Thanks, Mauro -----Messaggio originale----- Da: Johanna Amann [mailto:johanna at icir.org] Inviato: mercoled? 3 luglio 2019 10:43 A: Palumbo Mauro Cc: zeek at zeek.org Oggetto: Re: [Zeek] json formatter Hi Mauro, > we had some issues with the json formatter, which we are using with > kafka. We temporarily patched this and it is working fine for now. > However, probably a more general revision could be better, possibly > using an external library for the conversion to json format. I saw in > github somebody is already working on it, right? So, I guess we just > have to wait for next zeek's version? Yup, that is the plan. For more information see the github bug at https://github.com/zeek/zeek/issues/150 and the pull request at https://github.com/zeek/zeek/pull/451; I assume this will be merged into master in not too long. Johanna From mauro.palumbo at aizoon.it Mon Jul 8 03:19:17 2019 From: mauro.palumbo at aizoon.it (Palumbo Mauro) Date: Mon, 8 Jul 2019 10:19:17 +0000 Subject: [Zeek] known services Message-ID: <19048895f0a54a4a820c32d65efb859d@SRVEX03.aizoon.local> Hi all, I am looking at the known-services log and it seems to me that when multiple services are detected on the conn.log, not all of them are reported in the known-services.log. For example, http+ssl in the conn.log is logged in known-services.log as only http, while other multiple protocols (for exmaple NTLM,DCE_RPC or even as many as SMB,DCE_RPC,KRB,GSSAPI) are correctly logged. Is there any rationale for this behaviour or it is just a bug? I saw there is an issue (#419) open on github about it, but it's not clear to me why this happens only for some combinations of multiple protocols. Besides, I noticed that the know-services script does not detect all DNS conns. I opened an issue on this #455. Last a minor thing. In the script known-services.zeek, in the event connection_state_remove, there is an if statement (below) which is filtering all non-estabilshed tcp conns, but also all udp conns. if ( c$resp$state != TCP_ESTABLISHED ) return; Despite this, everything works fine because all udp analyzers rise an event protocol_confirmation. Would it be better changing the if statement into something like: if ( c$resp$state != TCP_ESTABLISHED && c$resp$state != UDP_ACTIVE ) return; In this way, if an udp analyzer does not rise the event protocol_confirmation, the connection will still be logged into known-services. Any thoughts? Thanks. Mauro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190708/9d8c692e/attachment.html From justin at corelight.com Mon Jul 8 09:02:00 2019 From: justin at corelight.com (Justin Azoff) Date: Mon, 8 Jul 2019 12:02:00 -0400 Subject: [Zeek] known services In-Reply-To: <19048895f0a54a4a820c32d65efb859d@SRVEX03.aizoon.local> References: <19048895f0a54a4a820c32d65efb859d@SRVEX03.aizoon.local> Message-ID: On Mon, Jul 8, 2019 at 6:27 AM Palumbo Mauro wrote: > I am looking at the known-services log and it seems to me that when multiple services are detected on the conn.log, not all of them are reported in the known-services.log. For example, http+ssl in the conn.log is logged in known-services.log as only http, while other multiple protocols (for exmaple NTLM,DCE_RPC or even as many as SMB,DCE_RPC,KRB,GSSAPI) are correctly logged. Is there any rationale for this behaviour or it is just a bug? I saw there is an issue (#419) open on github about it, but it?s not clear to me why this happens only for some combinations of multiple protocols. Some connections are decoded as multiple protocols. Something like a SMTP connection that runs STARTTLS and turns into SSL. This will end up in the conn log as smtp,ssl and also show up in known services as smtp,ssl. The problem is that services are tracked by ip+port, instead of ip+port+service, so whatever the protocol was on the first seen connection is the one that gets logged. This means that if the first seen connection is just 'smtp', it will get logged as 'smtp' and then further 'smtp,ssl' connections will not get logged. I had an earlier patch to update the service tracking to include the service, it just needs to be updated for 2.6 and tested. -- Justin From klehigh at iu.edu Mon Jul 8 13:15:07 2019 From: klehigh at iu.edu (Keith Lehigh) Date: Mon, 08 Jul 2019 16:15:07 -0400 Subject: [Zeek] ZeekWeek 2019 CFP reminder Message-ID: <9562E69A-AE7E-4EF5-BAE2-CE8960431484@iu.edu> Hi Folks, Just a friendly reminder that the CFP deadline for ZeekWeek 2019 is this Friday, July 12th. We?re looking for talks about tools being built for Zeek as well as interesting ways you may using data. Got stories about how Zeek helped solve problems? We?d love to hear those too! Have some novel way to manage your Zeek herd? That?s fair game for consideration. Ready to get that submission going? Head on over to https://www.zeekweek.com and hit the ?Call For Presentations? link at the top. - Keith -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3700 bytes Desc: S/MIME digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190708/b7762477/attachment.bin From mauro.palumbo at aizoon.it Tue Jul 9 00:30:32 2019 From: mauro.palumbo at aizoon.it (Palumbo Mauro) Date: Tue, 9 Jul 2019 07:30:32 +0000 Subject: [Zeek] R: known services In-Reply-To: References: <19048895f0a54a4a820c32d65efb859d@SRVEX03.aizoon.local> Message-ID: Ok, thanks. Are you planning to release the patch soon? -----Messaggio originale----- Da: Justin Azoff [mailto:justin at corelight.com] Inviato: luned? 8 luglio 2019 18:02 A: Palumbo Mauro Cc: zeek at zeek.org Oggetto: Re: [Zeek] known services On Mon, Jul 8, 2019 at 6:27 AM Palumbo Mauro wrote: > I am looking at the known-services log and it seems to me that when multiple services are detected on the conn.log, not all of them are reported in the known-services.log. For example, http+ssl in the conn.log is logged in known-services.log as only http, while other multiple protocols (for exmaple NTLM,DCE_RPC or even as many as SMB,DCE_RPC,KRB,GSSAPI) are correctly logged. Is there any rationale for this behaviour or it is just a bug? I saw there is an issue (#419) open on github about it, but it?s not clear to me why this happens only for some combinations of multiple protocols. Some connections are decoded as multiple protocols. Something like a SMTP connection that runs STARTTLS and turns into SSL. This will end up in the conn log as smtp,ssl and also show up in known services as smtp,ssl. The problem is that services are tracked by ip+port, instead of ip+port+service, so whatever the protocol was on the first seen connection is the one that gets logged. This means that if the first seen connection is just 'smtp', it will get logged as 'smtp' and then further 'smtp,ssl' connections will not get logged. I had an earlier patch to update the service tracking to include the service, it just needs to be updated for 2.6 and tested. -- Justin From smshahzaibshah at gmail.com Tue Jul 9 02:18:51 2019 From: smshahzaibshah at gmail.com (Shahzaib Shah) Date: Tue, 9 Jul 2019 14:48:51 +0530 Subject: [Zeek] Configuring incomplete, errors occurred! Message-ID: CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage Configuring incomplete, errors occurred! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190709/61c37b27/attachment.html From justin at corelight.com Tue Jul 9 07:09:53 2019 From: justin at corelight.com (Justin Azoff) Date: Tue, 9 Jul 2019 10:09:53 -0400 Subject: [Zeek] known services In-Reply-To: References: <19048895f0a54a4a820c32d65efb859d@SRVEX03.aizoon.local> Message-ID: On Tue, Jul 9, 2019 at 3:30 AM Palumbo Mauro wrote: > > Ok, thanks. Are you planning to release the patch soon? Hopefully.. it's not super complicated but making sure all of the edge cases are tested properly is most of the work. -- Justin From jmellander at lbl.gov Tue Jul 9 11:07:29 2019 From: jmellander at lbl.gov (Jim Mellander) Date: Tue, 9 Jul 2019 11:07:29 -0700 Subject: [Zeek] Configuring incomplete, errors occurred! In-Reply-To: References: Message-ID: Running ./configure --help will show the various options to the configure script. Is it possible that there are some problems with the CC or CXX environment variables? On Tue, Jul 9, 2019 at 2:27 AM Shahzaib Shah wrote: > CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage Configuring incomplete, errors occurred! > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190709/bd3164e3/attachment-0001.html From jsiwek at corelight.com Tue Jul 9 11:12:36 2019 From: jsiwek at corelight.com (Jon Siwek) Date: Tue, 9 Jul 2019 11:12:36 -0700 Subject: [Zeek] Configuring incomplete, errors occurred! In-Reply-To: References: Message-ID: On Tue, Jul 9, 2019 at 2:27 AM Shahzaib Shah wrote: > > CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage Configuring incomplete, errors occurred! Sounds like there's no compiler installed ? Maybe see if these installation instructions help: https://docs.zeek.org/en/stable/install/install.html - Jon From william.dieterich at gmail.com Tue Jul 9 11:27:10 2019 From: william.dieterich at gmail.com (William Dieterich) Date: Tue, 9 Jul 2019 13:27:10 -0500 Subject: [Zeek] Issues with Intel::FILE_NAME not working. Message-ID: Using the Intel Framework I cannot get Intel::FILE_NAME to fire. It is working with any other type so my script and read file is good. I am loading the following scripts Policy/frameworks/intel/seen policy/frameworks/intel/do_notice frameworks/file/hash-all-files.bro base/frameworks/intel/files.bro Loading hash-all-files.bro is there so that Intel::FILE_HASH works, is there a better way? I am taking filenames from both my files.log and http.log files so I know the files exist. I am getting no errors in recorder.log and am running from the command line and no errors are there. Any ideas on what I am doing wrong? From deltah24 at gmail.com Tue Jul 9 18:44:35 2019 From: deltah24 at gmail.com (Aaron Heller) Date: Tue, 9 Jul 2019 21:44:35 -0400 Subject: [Zeek] New Analyzer Message-ID: Hi everyone, I'm working on a BACnet protocol analyzer for Zeek and am having problems getting the analyzer to fire. I've been working with Zeek version 2.6.2 and the analyzer was created using binpac_quickstart. BACnet is a UDP based building automation and control protocol (think furnaces, security/access systems, lighting, etc.). Not sure what info would be most helpful, if anyone is willing to lend some insight as why the analyzer isn't firing off? The analyzer is supposed to be signature based and bro -N shows it as built-in and active. If bro -s option is used to specify the signature file then the analyzer will fire off appropriately, but I'm looking for it to auto-magically be included in the UDP analyzer tree. Greatly appreciate any help or thought for where to look first, Aaron Virus-free. www.avg.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190709/a8579081/attachment.html From mauro.palumbo at aizoon.it Wed Jul 10 00:30:18 2019 From: mauro.palumbo at aizoon.it (Palumbo Mauro) Date: Wed, 10 Jul 2019 07:30:18 +0000 Subject: [Zeek] R: New Analyzer In-Reply-To: References: Message-ID: <4f87e95ceddc4eee8b2951225fb61a86@SRVEX03.aizoon.local> Hi Aaron, not sure what you have done so far, but maybe you are missing something on the script side? To activate signature recognition for analyzers, you must write a script with the proper signature (usually called dpd.sig) and load it (usually with @load-sigs ./dpd.sig in the main.zeek script for the analyzer). Have a look at the script side of some other analyzers to see some examples. Mauro Da: zeek-bounces at zeek.org [mailto:zeek-bounces at zeek.org] Per conto di Aaron Heller Inviato: mercoled? 10 luglio 2019 03:45 A: zeek at zeek.org Oggetto: [Zeek] New Analyzer Hi everyone, I'm working on a BACnet protocol analyzer for Zeek and am having problems getting the analyzer to fire. I've been working with Zeek version 2.6.2 and the analyzer was created using binpac_quickstart. BACnet is a UDP based building automation and control protocol (think furnaces, security/access systems, lighting, etc.). Not sure what info would be most helpful, if anyone is willing to lend some insight as why the analyzer isn't firing off? The analyzer is supposed to be signature based and bro -N shows it as built-in and active. If bro -s option is used to specify the signature file then the analyzer will fire off appropriately, but I'm looking for it to auto-magically be included in the UDP analyzer tree. Greatly appreciate any help or thought for where to look first, Aaron [Immagine rimossa dal mittente.] Virus-free. www.avg.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/e3119293/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 362 bytes Desc: image001.jpg Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/e3119293/attachment.jpg From jan.grashoefer at gmail.com Wed Jul 10 03:03:36 2019 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Wed, 10 Jul 2019 12:03:36 +0200 Subject: [Zeek] Issues with Intel::FILE_NAME not working. In-Reply-To: References: Message-ID: <29e7d9b3-c68f-f959-0485-85f952532183@gmail.com> Hi William, the script seen/file-names.zeek [1] defines how file names are reported to the intel framework. To match, the indicator has to be identical to f$info$filename. Jan [1] https://github.com/zeek/zeek/blob/master/scripts/policy/frameworks/intel/seen/file-names.zeek On 09/07/2019 20:27, William Dieterich wrote: > Using the Intel Framework I cannot get Intel::FILE_NAME to fire. It > is working with any other type so my script and read file is good. > > I am loading the following scripts > > Policy/frameworks/intel/seen > policy/frameworks/intel/do_notice > frameworks/file/hash-all-files.bro > base/frameworks/intel/files.bro > > Loading hash-all-files.bro is there so that Intel::FILE_HASH works, is > there a better way? > > I am taking filenames from both my files.log and http.log files so I > know the files exist. I am getting no errors in recorder.log and am > running from the command line and no errors are there. Any ideas on > what I am doing wrong? > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > From nothinrandom at gmail.com Wed Jul 10 15:51:27 2019 From: nothinrandom at gmail.com (TQ) Date: Wed, 10 Jul 2019 15:51:27 -0700 Subject: [Zeek] New Analyzer In-Reply-To: References: Message-ID: Hi Aaron, Silly question, but did you tell it to listen to port 47808 in your main.zeek script? Probably something like this: ## define listening ports const ports = { 47808/udp }; redef likely_server_ports += { ports }; I do have a working analyzer and could walk you through if you could share what you have so far. Thanks, On Wed, Jul 10, 2019 at 12:00 PM wrote: > Send Zeek mailing list submissions to > zeek at zeek.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > or, via email, send a message with subject or body 'help' to > zeek-request at zeek.org > > You can reach the person managing the list at > zeek-owner at zeek.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Zeek digest..." > > > Today's Topics: > > 1. New Analyzer (Aaron Heller) > 2. R: New Analyzer (Palumbo Mauro) > 3. Re: Issues with Intel::FILE_NAME not working. (Jan Grash?fer) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 9 Jul 2019 21:44:35 -0400 > From: Aaron Heller > Subject: [Zeek] New Analyzer > To: zeek at zeek.org > Message-ID: > w+bgHsVk9oKG8qk7QC9s+h6Dg at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi everyone, > I'm working on a BACnet protocol analyzer for Zeek and am having problems > getting the analyzer to fire. I've been working with Zeek version 2.6.2 > and the analyzer was created using binpac_quickstart. > > BACnet is a UDP based building automation and control protocol (think > furnaces, security/access systems, lighting, etc.). > > Not sure what info would be most helpful, if anyone is willing to lend some > insight as why the analyzer isn't firing off? The analyzer is supposed to > be signature based and bro -N shows it as built-in and active. If bro -s > option is used to specify the signature file then the analyzer will fire > off appropriately, but I'm looking for it to auto-magically be included in > the UDP analyzer tree. > > Greatly appreciate any help or thought for where to look first, > Aaron > > < > http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > Virus-free. > www.avg.com > < > http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190709/a8579081/attachment-0001.html > > ------------------------------ > > Message: 2 > Date: Wed, 10 Jul 2019 07:30:18 +0000 > From: Palumbo Mauro > Subject: [Zeek] R: New Analyzer > To: Aaron Heller , "zeek at zeek.org" > Message-ID: <4f87e95ceddc4eee8b2951225fb61a86 at SRVEX03.aizoon.local> > Content-Type: text/plain; charset="utf-8" > > Hi Aaron, > not sure what you have done so far, but maybe you are missing something > on the script side? > > To activate signature recognition for analyzers, you must write a script > with the proper signature (usually called dpd.sig) and load it (usually > with @load-sigs ./dpd.sig in the main.zeek script for the analyzer). > > Have a look at the script side of some other analyzers to see some > examples. > > Mauro > > > > > Da: zeek-bounces at zeek.org [mailto:zeek-bounces at zeek.org] Per conto di > Aaron Heller > Inviato: mercoled? 10 luglio 2019 03:45 > A: zeek at zeek.org > Oggetto: [Zeek] New Analyzer > > Hi everyone, > I'm working on a BACnet protocol analyzer for Zeek and am having problems > getting the analyzer to fire. I've been working with Zeek version 2.6.2 > and the analyzer was created using binpac_quickstart. > > BACnet is a UDP based building automation and control protocol (think > furnaces, security/access systems, lighting, etc.). > > Not sure what info would be most helpful, if anyone is willing to lend > some insight as why the analyzer isn't firing off? The analyzer is supposed > to be signature based and bro -N shows it as built-in and active. If bro > -s option is used to specify the signature file then the analyzer will fire > off appropriately, but I'm looking for it to auto-magically be included in > the UDP analyzer tree. > > Greatly appreciate any help or thought for where to look first, > Aaron > > [Immagine rimossa dal mittente.]< > http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > > Virus-free. www.avg.com< > http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/e3119293/attachment-0001.html > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: image001.jpg > Type: image/jpeg > Size: 362 bytes > Desc: image001.jpg > Url : > http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/e3119293/attachment-0001.jpg > > ------------------------------ > > Message: 3 > Date: Wed, 10 Jul 2019 12:03:36 +0200 > From: Jan Grash?fer > Subject: Re: [Zeek] Issues with Intel::FILE_NAME not working. > To: zeek at zeek.org > Message-ID: <29e7d9b3-c68f-f959-0485-85f952532183 at gmail.com> > Content-Type: text/plain; charset=utf-8; format=flowed > > Hi William, > > the script seen/file-names.zeek [1] defines how file names are reported > to the intel framework. To match, the indicator has to be identical to > f$info$filename. > > Jan > > [1] > > https://github.com/zeek/zeek/blob/master/scripts/policy/frameworks/intel/seen/file-names.zeek > > On 09/07/2019 20:27, William Dieterich wrote: > > Using the Intel Framework I cannot get Intel::FILE_NAME to fire. It > > is working with any other type so my script and read file is good. > > > > I am loading the following scripts > > > > Policy/frameworks/intel/seen > > policy/frameworks/intel/do_notice > > frameworks/file/hash-all-files.bro > > base/frameworks/intel/files.bro > > > > Loading hash-all-files.bro is there so that Intel::FILE_HASH works, is > > there a better way? > > > > I am taking filenames from both my files.log and http.log files so I > > know the files exist. I am getting no errors in recorder.log and am > > running from the command line and no errors are there. Any ideas on > > what I am doing wrong? > > _______________________________________________ > > Zeek mailing list > > zeek at zeek.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > > > > ------------------------------ > > _______________________________________________ > Zeek mailing list > Zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > > End of Zeek Digest, Vol 159, Issue 8 > ************************************ > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/f0d4608f/attachment.html From deltah24 at gmail.com Wed Jul 10 17:07:45 2019 From: deltah24 at gmail.com (Aaron Heller) Date: Wed, 10 Jul 2019 20:07:45 -0400 Subject: [Zeek] New Analyzer In-Reply-To: <4f87e95ceddc4eee8b2951225fb61a86@SRVEX03.aizoon.local> References: <4f87e95ceddc4eee8b2951225fb61a86@SRVEX03.aizoon.local> Message-ID: Hi Mauro, Thanks much for the idea/insight. I didn't have a @load-sigs line in the main.bro script, but there is one in the __load__.bro file. It looks consistent with the other protocols that appear to be signature based (dnp3, ftp, pop3, etc.). I tried adding the @load-sigs ./dpd.sig line to the main.bro script but still no joy. Any other thoughts? I didn't think to include it in the original email, but when zeek is run with the -s option and a signature file is specified, the 'C' portion of the analyzer fires off (i.e., the ../zeek/src/analyzers/protocol/bacnet/bacnet.cc, Plugin.cc, and events.bif), but the script side that should generate a log file does not (../zeek/scripts/base/protocols/bacnet/main.bro, __load__.bro, and dpd.sig). Maybe that and the analyzer not automatically firing off indicates an issue with the bacnet script not being called appropriately? I'm grasping at straws, so any thoughts are greatly appreciated! Thanks again, Aaron On Wed, Jul 10, 2019 at 3:30 AM Palumbo Mauro wrote: > Hi Aaron, > > not sure what you have done so far, but maybe you are missing something > on the script side? > > > > To activate signature recognition for analyzers, you must write a script > with the proper signature (usually called dpd.sig) and load it (usually > with @load-sigs ./dpd.sig in the main.zeek script for the analyzer). > > > > Have a look at the script side of some other analyzers to see some > examples. > > > > Mauro > > > > > > > > > > *Da:* zeek-bounces at zeek.org [mailto:zeek-bounces at zeek.org] *Per conto di *Aaron > Heller > *Inviato:* mercoled? 10 luglio 2019 03:45 > *A:* zeek at zeek.org > *Oggetto:* [Zeek] New Analyzer > > > > Hi everyone, > > I'm working on a BACnet protocol analyzer for Zeek and am having problems > getting the analyzer to fire. I've been working with Zeek version 2.6.2 > and the analyzer was created using binpac_quickstart. > > > > BACnet is a UDP based building automation and control protocol (think > furnaces, security/access systems, lighting, etc.). > > > > Not sure what info would be most helpful, if anyone is willing to lend > some insight as why the analyzer isn't firing off? The analyzer is supposed > to be signature based and bro -N shows it as built-in and active. If bro > -s option is used to specify the signature file then the analyzer will fire > off appropriately, but I'm looking for it to auto-magically be included in > the UDP analyzer tree. > > > > Greatly appreciate any help or thought for where to look first, > > Aaron > > > > [image: Immagine rimossa dal mittente.] > > > Virus-free. www.avg.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/7ff16c1a/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 362 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/7ff16c1a/attachment-0001.jpg From justin at corelight.com Wed Jul 10 17:20:20 2019 From: justin at corelight.com (Justin Azoff) Date: Wed, 10 Jul 2019 20:20:20 -0400 Subject: [Zeek] New Analyzer In-Reply-To: References: <4f87e95ceddc4eee8b2951225fb61a86@SRVEX03.aizoon.local> Message-ID: On Wed, Jul 10, 2019 at 8:16 PM Aaron Heller wrote: > Maybe that and the analyzer not automatically firing off indicates an > issue with the bacnet script not being called appropriately? I'm grasping > at straws, so any thoughts are greatly appreciated! > I don't think you are loading the scripts at all.. which is also why the sigs aren't loaded. Are you building this as an in-tree analyzer or as an external plugin? -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/7c9ec473/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 362 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/7c9ec473/attachment.jpg From deltah24 at gmail.com Wed Jul 10 17:39:07 2019 From: deltah24 at gmail.com (Aaron Heller) Date: Wed, 10 Jul 2019 20:39:07 -0400 Subject: [Zeek] New Analyzer In-Reply-To: References: <4f87e95ceddc4eee8b2951225fb61a86@SRVEX03.aizoon.local> Message-ID: Hi Justin, I started off using the binpac_quickstart script, which I thought created an external plugin? Thanks, Aaron On Wed, Jul 10, 2019 at 8:20 PM Justin Azoff wrote: > On Wed, Jul 10, 2019 at 8:16 PM Aaron Heller wrote: > >> Maybe that and the analyzer not automatically firing off indicates an >> issue with the bacnet script not being called appropriately? I'm grasping >> at straws, so any thoughts are greatly appreciated! >> > > I don't think you are loading the scripts at all.. which is also why the > sigs aren't loaded. > > Are you building this as an in-tree analyzer or as an external plugin? > > -- > Justin > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/4c34f170/attachment.html From deltah24 at gmail.com Wed Jul 10 17:46:56 2019 From: deltah24 at gmail.com (Aaron Heller) Date: Wed, 10 Jul 2019 20:46:56 -0400 Subject: [Zeek] Zeek Digest, Vol 159, Issue 9 In-Reply-To: References: Message-ID: Hi TQ, Thanks for the thoughts. I tried as you suggested, but nothing changed when specifying 47808/udp. Below is what's in the main.bro file - any thoughts are welcome and appreciated. Also - what's best practice for this mailing list when a conversation gets multiple threads going (you, Mauro, and Justin have all offered help)? Keep everyone's thoughts/replies in one thread or break out into direct emails? Aaron --------------------------------------------------------------------------------------------------------------------- ##! Implements base functionality for bacnet analysis. ##! Generates the Bacnet.log file. # Generated by binpac_quickstart module Bacnet; export { print "Export statement!"; redef enum Log::ID += { LOG }; type Info: record { ## Timestamp for when the event happened. ts: time &log; ## Unique ID for the connection. uid: string &log; ## The connection's 4-tuple of endpoint addresses/ports. id: conn_id &log; # ## TODO: Add other fields here that you'd like to log. }; ## Event that can be handled to access the bacnet record as it is sent on ## to the loggin framework. global log_bacnet: event(rec: Info); } # TODO: The recommended method to do dynamic protocol detection # (DPD) is with the signatures in dpd.sig. If you can't come up # with any signatures, then you can do port-based detection by # uncommenting the following and specifying the port(s): ################################################################################################################# # # Per your suggestion, I modified and uncommented the following 2 lines and tried running again, # along with uncommenting the "Analyzer::register_for_ports(Analyzer::ANALYZER_BACNET, ports);" line. No joy. :( # ################################################################################################################# # const ports = { 47808/udp }; # redef likely_server_ports += { ports }; event bro_init() &priority=5 { Log::create_stream(Bacnet::LOG, [$columns=Info, $ev=log_bacnet, $path="bacnet"]); print "Init Statement"; # TODO: If you're using port-based DPD, uncomment this. # Analyzer::register_for_ports(Analyzer::ANALYZER_BACNET, ports); } event bacnet_ethernet_BVLC_Result(c: connection, BVLC_Type : count, BVLC_Function : count) { local info: Info; info$ts = network_time(); info$uid = c$uid; info$id = c$id; print "Result seen! ", BVLC_Function; Log::write(Bacnet::LOG, info); } On Wed, Jul 10, 2019 at 8:08 PM wrote: > Send Zeek mailing list submissions to > zeek at zeek.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > or, via email, send a message with subject or body 'help' to > zeek-request at zeek.org > > You can reach the person managing the list at > zeek-owner at zeek.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Zeek digest..." > > > Today's Topics: > > 1. Re: New Analyzer (TQ) > 2. Re: New Analyzer (Aaron Heller) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 10 Jul 2019 15:51:27 -0700 > From: TQ > Subject: Re: [Zeek] New Analyzer > To: zeek > Message-ID: > YGhp0mcnTB3oBg at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi Aaron, > > Silly question, but did you tell it to listen to port 47808 in your > main.zeek script? Probably something like this: > > ## define listening ports > const ports = { 47808/udp }; > redef likely_server_ports += { ports }; > > I do have a working analyzer and could walk you through if you could share > what you have so far. > > Thanks, > > On Wed, Jul 10, 2019 at 12:00 PM wrote: > > > Send Zeek mailing list submissions to > > zeek at zeek.org > > > > To subscribe or unsubscribe via the World Wide Web, visit > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > or, via email, send a message with subject or body 'help' to > > zeek-request at zeek.org > > > > You can reach the person managing the list at > > zeek-owner at zeek.org > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of Zeek digest..." > > > > > > Today's Topics: > > > > 1. New Analyzer (Aaron Heller) > > 2. R: New Analyzer (Palumbo Mauro) > > 3. Re: Issues with Intel::FILE_NAME not working. (Jan Grash?fer) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Tue, 9 Jul 2019 21:44:35 -0400 > > From: Aaron Heller > > Subject: [Zeek] New Analyzer > > To: zeek at zeek.org > > Message-ID: > > > w+bgHsVk9oKG8qk7QC9s+h6Dg at mail.gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > Hi everyone, > > I'm working on a BACnet protocol analyzer for Zeek and am having problems > > getting the analyzer to fire. I've been working with Zeek version 2.6.2 > > and the analyzer was created using binpac_quickstart. > > > > BACnet is a UDP based building automation and control protocol (think > > furnaces, security/access systems, lighting, etc.). > > > > Not sure what info would be most helpful, if anyone is willing to lend > some > > insight as why the analyzer isn't firing off? The analyzer is supposed to > > be signature based and bro -N shows it as built-in and active. If bro -s > > option is used to specify the signature file then the analyzer will fire > > off appropriately, but I'm looking for it to auto-magically be included > in > > the UDP analyzer tree. > > > > Greatly appreciate any help or thought for where to look first, > > Aaron > > > > < > > > http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > > > Virus-free. > > www.avg.com > > < > > > http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > > > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: > > > http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190709/a8579081/attachment-0001.html > > > > ------------------------------ > > > > Message: 2 > > Date: Wed, 10 Jul 2019 07:30:18 +0000 > > From: Palumbo Mauro > > Subject: [Zeek] R: New Analyzer > > To: Aaron Heller , "zeek at zeek.org" > > Message-ID: <4f87e95ceddc4eee8b2951225fb61a86 at SRVEX03.aizoon.local> > > Content-Type: text/plain; charset="utf-8" > > > > Hi Aaron, > > not sure what you have done so far, but maybe you are missing > something > > on the script side? > > > > To activate signature recognition for analyzers, you must write a script > > with the proper signature (usually called dpd.sig) and load it (usually > > with @load-sigs ./dpd.sig in the main.zeek script for the analyzer). > > > > Have a look at the script side of some other analyzers to see some > > examples. > > > > Mauro > > > > > > > > > > Da: zeek-bounces at zeek.org [mailto:zeek-bounces at zeek.org] Per conto di > > Aaron Heller > > Inviato: mercoled? 10 luglio 2019 03:45 > > A: zeek at zeek.org > > Oggetto: [Zeek] New Analyzer > > > > Hi everyone, > > I'm working on a BACnet protocol analyzer for Zeek and am having problems > > getting the analyzer to fire. I've been working with Zeek version 2.6.2 > > and the analyzer was created using binpac_quickstart. > > > > BACnet is a UDP based building automation and control protocol (think > > furnaces, security/access systems, lighting, etc.). > > > > Not sure what info would be most helpful, if anyone is willing to lend > > some insight as why the analyzer isn't firing off? The analyzer is > supposed > > to be signature based and bro -N shows it as built-in and active. If bro > > -s option is used to specify the signature file then the analyzer will > fire > > off appropriately, but I'm looking for it to auto-magically be included > in > > the UDP analyzer tree. > > > > Greatly appreciate any help or thought for where to look first, > > Aaron > > > > [Immagine rimossa dal mittente.]< > > > http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > > > > > Virus-free. www.avg.com< > > > http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > > > > > > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: > > > http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/e3119293/attachment-0001.html > > -------------- next part -------------- > > A non-text attachment was scrubbed... > > Name: image001.jpg > > Type: image/jpeg > > Size: 362 bytes > > Desc: image001.jpg > > Url : > > > http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/e3119293/attachment-0001.jpg > > > > ------------------------------ > > > > Message: 3 > > Date: Wed, 10 Jul 2019 12:03:36 +0200 > > From: Jan Grash?fer > > Subject: Re: [Zeek] Issues with Intel::FILE_NAME not working. > > To: zeek at zeek.org > > Message-ID: <29e7d9b3-c68f-f959-0485-85f952532183 at gmail.com> > > Content-Type: text/plain; charset=utf-8; format=flowed > > > > Hi William, > > > > the script seen/file-names.zeek [1] defines how file names are reported > > to the intel framework. To match, the indicator has to be identical to > > f$info$filename. > > > > Jan > > > > [1] > > > > > https://github.com/zeek/zeek/blob/master/scripts/policy/frameworks/intel/seen/file-names.zeek > > > > On 09/07/2019 20:27, William Dieterich wrote: > > > Using the Intel Framework I cannot get Intel::FILE_NAME to fire. It > > > is working with any other type so my script and read file is good. > > > > > > I am loading the following scripts > > > > > > Policy/frameworks/intel/seen > > > policy/frameworks/intel/do_notice > > > frameworks/file/hash-all-files.bro > > > base/frameworks/intel/files.bro > > > > > > Loading hash-all-files.bro is there so that Intel::FILE_HASH works, is > > > there a better way? > > > > > > I am taking filenames from both my files.log and http.log files so I > > > know the files exist. I am getting no errors in recorder.log and am > > > running from the command line and no errors are there. Any ideas on > > > what I am doing wrong? > > > _______________________________________________ > > > Zeek mailing list > > > zeek at zeek.org > > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > > > > > > > > ------------------------------ > > > > _______________________________________________ > > Zeek mailing list > > Zeek at zeek.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > > > > > End of Zeek Digest, Vol 159, Issue 8 > > ************************************ > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/f0d4608f/attachment-0001.html > > ------------------------------ > > Message: 2 > Date: Wed, 10 Jul 2019 20:07:45 -0400 > From: Aaron Heller > Subject: Re: [Zeek] New Analyzer > To: Palumbo Mauro > Cc: "zeek at zeek.org" > Message-ID: > a9MWjOQ at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi Mauro, > Thanks much for the idea/insight. > > I didn't have a @load-sigs line in the main.bro script, but there is one in > the __load__.bro file. It looks consistent with the other protocols that > appear to be signature based (dnp3, ftp, pop3, etc.). I tried adding the > @load-sigs ./dpd.sig line to the main.bro script but still no joy. Any > other thoughts? > > I didn't think to include it in the original email, but when zeek is run > with the -s option and a signature file is specified, the 'C' portion of > the analyzer fires off (i.e., the > ../zeek/src/analyzers/protocol/bacnet/bacnet.cc, Plugin.cc, and > events.bif), but the script side that should generate a log file does not > (../zeek/scripts/base/protocols/bacnet/main.bro, __load__.bro, and > dpd.sig). Maybe that and the analyzer not automatically firing off > indicates an issue with the bacnet script not being called appropriately? > I'm grasping at straws, so any thoughts are greatly appreciated! > > > Thanks again, > Aaron > > On Wed, Jul 10, 2019 at 3:30 AM Palumbo Mauro > wrote: > > > Hi Aaron, > > > > not sure what you have done so far, but maybe you are missing > something > > on the script side? > > > > > > > > To activate signature recognition for analyzers, you must write a script > > with the proper signature (usually called dpd.sig) and load it (usually > > with @load-sigs ./dpd.sig in the main.zeek script for the analyzer). > > > > > > > > Have a look at the script side of some other analyzers to see some > > examples. > > > > > > > > Mauro > > > > > > > > > > > > > > > > > > > > *Da:* zeek-bounces at zeek.org [mailto:zeek-bounces at zeek.org] *Per conto > di *Aaron > > Heller > > *Inviato:* mercoled? 10 luglio 2019 03:45 > > *A:* zeek at zeek.org > > *Oggetto:* [Zeek] New Analyzer > > > > > > > > Hi everyone, > > > > I'm working on a BACnet protocol analyzer for Zeek and am having problems > > getting the analyzer to fire. I've been working with Zeek version 2.6.2 > > and the analyzer was created using binpac_quickstart. > > > > > > > > BACnet is a UDP based building automation and control protocol (think > > furnaces, security/access systems, lighting, etc.). > > > > > > > > Not sure what info would be most helpful, if anyone is willing to lend > > some insight as why the analyzer isn't firing off? The analyzer is > supposed > > to be signature based and bro -N shows it as built-in and active. If bro > > -s option is used to specify the signature file then the analyzer will > fire > > off appropriately, but I'm looking for it to auto-magically be included > in > > the UDP analyzer tree. > > > > > > > > Greatly appreciate any help or thought for where to look first, > > > > Aaron > > > > > > > > [image: Immagine rimossa dal mittente.] > > < > http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > > > > Virus-free. www.avg.com > > < > http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > > > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/7ff16c1a/attachment.html > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: image001.jpg > Type: image/jpeg > Size: 362 bytes > Desc: not available > Url : > http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/7ff16c1a/attachment.jpg > > ------------------------------ > > _______________________________________________ > Zeek mailing list > Zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > > End of Zeek Digest, Vol 159, Issue 9 > ************************************ > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/c0faa59e/attachment-0001.html From justin at corelight.com Wed Jul 10 17:51:41 2019 From: justin at corelight.com (Justin Azoff) Date: Wed, 10 Jul 2019 20:51:41 -0400 Subject: [Zeek] New Analyzer In-Reply-To: References: <4f87e95ceddc4eee8b2951225fb61a86@SRVEX03.aizoon.local> Message-ID: did you run that with --plugin? On Wed, Jul 10, 2019 at 8:39 PM Aaron Heller wrote: > > Hi Justin, > I started off using the binpac_quickstart script, which I thought created an external plugin? > > Thanks, > Aaron > > On Wed, Jul 10, 2019 at 8:20 PM Justin Azoff wrote: >> >> On Wed, Jul 10, 2019 at 8:16 PM Aaron Heller wrote: >>> >>> Maybe that and the analyzer not automatically firing off indicates an issue with the bacnet script not being called appropriately? I'm grasping at straws, so any thoughts are greatly appreciated! >> >> >> I don't think you are loading the scripts at all.. which is also why the sigs aren't loaded. >> >> Are you building this as an in-tree analyzer or as an external plugin? >> >> -- >> Justin -- Justin From justin at corelight.com Wed Jul 10 18:01:10 2019 From: justin at corelight.com (Justin Azoff) Date: Wed, 10 Jul 2019 21:01:10 -0400 Subject: [Zeek] New Analyzer In-Reply-To: References: <4f87e95ceddc4eee8b2951225fb61a86@SRVEX03.aizoon.local> Message-ID: Oh, looking at this closer you probably want to use zeek/aux/zeek-aux/plugin-support/init-plugin to create the plugin skeleton. the binpac quickstart I think is a bit out of date at this point for how to setup an external plugin+package. The binpac parts it genrates should still be fine though. so I would use init-plugin to make a new package and copy your existing code over it. that should give you a working self-contained external package that you can install. It also takes advantage of the new bro-config bits which make building and installing the plugin work without the full source checkout. On Wed, Jul 10, 2019 at 8:51 PM Justin Azoff wrote: > > did you run that with --plugin? > > On Wed, Jul 10, 2019 at 8:39 PM Aaron Heller wrote: > > > > Hi Justin, > > I started off using the binpac_quickstart script, which I thought created an external plugin? > > > > Thanks, > > Aaron > > > > On Wed, Jul 10, 2019 at 8:20 PM Justin Azoff wrote: > >> > >> On Wed, Jul 10, 2019 at 8:16 PM Aaron Heller wrote: > >>> > >>> Maybe that and the analyzer not automatically firing off indicates an issue with the bacnet script not being called appropriately? I'm grasping at straws, so any thoughts are greatly appreciated! > >> > >> > >> I don't think you are loading the scripts at all.. which is also why the sigs aren't loaded. > >> > >> Are you building this as an in-tree analyzer or as an external plugin? > >> > >> -- > >> Justin > > > > -- > Justin -- Justin From deltah24 at gmail.com Wed Jul 10 18:11:46 2019 From: deltah24 at gmail.com (Aaron Heller) Date: Wed, 10 Jul 2019 21:11:46 -0400 Subject: [Zeek] New Analyzer In-Reply-To: References: <4f87e95ceddc4eee8b2951225fb61a86@SRVEX03.aizoon.local> Message-ID: I did try running with the bacnet plugin specified and it didn't work, so I'll give the init-plugin a shot tomorrow. Thanks much all for the thoughts and help, Aaron On Wed, Jul 10, 2019 at 9:01 PM Justin Azoff wrote: > Oh, looking at this closer you probably want to use > > zeek/aux/zeek-aux/plugin-support/init-plugin > > to create the plugin skeleton. the binpac quickstart I think is a bit > out of date at this point for how to setup an external plugin+package. > The binpac parts it genrates should still be fine though. > > so I would use init-plugin to make a new package and copy your > existing code over it. that should give you a working self-contained > external package that you can install. It also takes advantage of the > new bro-config bits which make building and installing the plugin work > without the full source checkout. > > On Wed, Jul 10, 2019 at 8:51 PM Justin Azoff wrote: > > > > did you run that with --plugin? > > > > On Wed, Jul 10, 2019 at 8:39 PM Aaron Heller wrote: > > > > > > Hi Justin, > > > I started off using the binpac_quickstart script, which I thought > created an external plugin? > > > > > > Thanks, > > > Aaron > > > > > > On Wed, Jul 10, 2019 at 8:20 PM Justin Azoff > wrote: > > >> > > >> On Wed, Jul 10, 2019 at 8:16 PM Aaron Heller > wrote: > > >>> > > >>> Maybe that and the analyzer not automatically firing off indicates > an issue with the bacnet script not being called appropriately? I'm > grasping at straws, so any thoughts are greatly appreciated! > > >> > > >> > > >> I don't think you are loading the scripts at all.. which is also why > the sigs aren't loaded. > > >> > > >> Are you building this as an in-tree analyzer or as an external plugin? > > >> > > >> -- > > >> Justin > > > > > > > > -- > > Justin > > > > -- > Justin > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190710/2c9cb4bf/attachment.html From mauro.palumbo at aizoon.it Thu Jul 11 01:12:07 2019 From: mauro.palumbo at aizoon.it (Palumbo Mauro) Date: Thu, 11 Jul 2019 08:12:07 +0000 Subject: [Zeek] dpd framework and DCE_RPC/NTLM analyzers Message-ID: <631e8f2601ce4692b38e67bba9478ff7@SRVEX03.aizoon.local> Hi everybody, is there any particular reason why the DCE_RPC/NTLM protocols are disabled by default in the DPD framework? (both protocols are in DPD::ignore_violations). Thanks Mauro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190711/38d6cccd/attachment.html From mauro.palumbo at aizoon.it Thu Jul 11 01:15:25 2019 From: mauro.palumbo at aizoon.it (Palumbo Mauro) Date: Thu, 11 Jul 2019 08:15:25 +0000 Subject: [Zeek] R: New Analyzer In-Reply-To: References: <4f87e95ceddc4eee8b2951225fb61a86@SRVEX03.aizoon.local> Message-ID: <3207fd834b1a403fa416602ad56b0c37@SRVEX03.aizoon.local> Hi Aaron, I can confirm the binpac quickstart is a bit out of date. I tried to use it a couple of months ago and run into some issues. You can still use it but then have to edit some files manually. Mauro Da: Aaron Heller [mailto:deltah24 at gmail.com] Inviato: gioved? 11 luglio 2019 03:12 A: Justin Azoff Cc: Palumbo Mauro ; zeek at zeek.org Oggetto: Re: [Zeek] New Analyzer I did try running with the bacnet plugin specified and it didn't work, so I'll give the init-plugin a shot tomorrow. Thanks much all for the thoughts and help, Aaron On Wed, Jul 10, 2019 at 9:01 PM Justin Azoff > wrote: Oh, looking at this closer you probably want to use zeek/aux/zeek-aux/plugin-support/init-plugin to create the plugin skeleton. the binpac quickstart I think is a bit out of date at this point for how to setup an external plugin+package. The binpac parts it genrates should still be fine though. so I would use init-plugin to make a new package and copy your existing code over it. that should give you a working self-contained external package that you can install. It also takes advantage of the new bro-config bits which make building and installing the plugin work without the full source checkout. On Wed, Jul 10, 2019 at 8:51 PM Justin Azoff > wrote: > > did you run that with --plugin? > > On Wed, Jul 10, 2019 at 8:39 PM Aaron Heller > wrote: > > > > Hi Justin, > > I started off using the binpac_quickstart script, which I thought created an external plugin? > > > > Thanks, > > Aaron > > > > On Wed, Jul 10, 2019 at 8:20 PM Justin Azoff > wrote: > >> > >> On Wed, Jul 10, 2019 at 8:16 PM Aaron Heller > wrote: > >>> > >>> Maybe that and the analyzer not automatically firing off indicates an issue with the bacnet script not being called appropriately? I'm grasping at straws, so any thoughts are greatly appreciated! > >> > >> > >> I don't think you are loading the scripts at all.. which is also why the sigs aren't loaded. > >> > >> Are you building this as an in-tree analyzer or as an external plugin? > >> > >> -- > >> Justin > > > > -- > Justin -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190711/82fcbee0/attachment-0001.html From jsiwek at corelight.com Thu Jul 11 09:48:59 2019 From: jsiwek at corelight.com (Jon Siwek) Date: Thu, 11 Jul 2019 09:48:59 -0700 Subject: [Zeek] dpd framework and DCE_RPC/NTLM analyzers In-Reply-To: <631e8f2601ce4692b38e67bba9478ff7@SRVEX03.aizoon.local> References: <631e8f2601ce4692b38e67bba9478ff7@SRVEX03.aizoon.local> Message-ID: On Thu, Jul 11, 2019 at 1:20 AM Palumbo Mauro wrote: > is there any particular reason why the DCE_RPC/NTLM protocols are disabled by default in the DPD framework? (both protocols are in DPD::ignore_violations). Being in DPD::ignore_violations doesn't exactly mean "DPD is disabled for those analyzers". It's more like "if an analyzer has previously issued a protocol confirmation signal, but later issues a protocol violation signal, then disable that analyzer except if it's in DPD::ignore_violations". So it's actually used to prevent the disabling of analyzers. However, I don't know the origins of DPD::ignore_violations, why it works that way, or why the DCE_RPC/NTLM protocols are in that set. - Jon From manoj.petshali at paytm.com Thu Jul 11 21:51:19 2019 From: manoj.petshali at paytm.com (Manoj Petshali) Date: Fri, 12 Jul 2019 10:21:19 +0530 Subject: [Zeek] Query reagrding Bro Ids Message-ID: Hi Team, I am very eager about the Bro and need to know below information : -We are working in india's biggest transactional system and facing many issues e.g. : if some user request is coming from pubic or private network (Internal request) and traverses across many servers and if user receives timeout ( e.g. connection time out, read time out ,rst etc) then we need to know the deep analysis of the same means : : Why/where the request timed out ? : Upto which hop the request travelled? : Network latency between these hopes to know if the latency is the issue? : tcp handshake and ssl handshake latency and the reason for the same? : Applicatency latency ? means if the network latency is fine We searched on wen and got feeling that the Bro is more oriented toward security and do deep packe inspection.But we have many problems like above to resolve .May you please let us know that how Bro can help us to resolve above issues? Thanks Manoj Petshali Sr. Manager - Payments Engineering Mobile +91-9891066456 www.paytm.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190712/381a3ee7/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: noname Type: image/png Size: 944 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190712/381a3ee7/attachment.bin From manoj.petshali at paytm.com Fri Jul 12 01:24:43 2019 From: manoj.petshali at paytm.com (Manoj Petshali) Date: Fri, 12 Jul 2019 13:54:43 +0530 Subject: [Zeek] Query reagrding Bro Ids In-Reply-To: References: Message-ID: Hi Team, Please respond as we need to implement the same at the earliest. Thanks Manoj Petshali Sr. Manager - Payments Engineering Mobile +91-9891066456 www.paytm.com On Fri, Jul 12, 2019 at 10:21 AM Manoj Petshali wrote: > Hi Team, > > I am very eager about the Bro and need to know below information : > > -We are working in india's biggest transactional system and facing many > issues e.g. > > : if some user request is coming from pubic or private network (Internal > request) and traverses across many servers and if user receives timeout ( > e.g. connection time out, read time out ,rst etc) then we need to know the > deep analysis of the same means : > > : Why/where the request timed out ? > : Upto which hop the request travelled? > : Network latency between these hopes to know if the latency is the issue? > : tcp handshake and ssl handshake latency and the reason for the same? > : Applicatency latency ? means if the network latency is fine > > We searched on wen and got feeling that the Bro is more oriented toward > security and do deep packe inspection.But we have many problems like above > to resolve .May you please let us know that how Bro can help us to resolve > above issues? > > Thanks > Manoj Petshali > Sr. Manager - Payments Engineering > Mobile +91-9891066456 > > www.paytm.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190712/0fdef4b1/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: noname Type: image/png Size: 944 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190712/0fdef4b1/attachment.bin From salwa.alem at univ-ubs.fr Fri Jul 12 02:09:29 2019 From: salwa.alem at univ-ubs.fr (Salwa Alem) Date: Fri, 12 Jul 2019 11:09:29 +0200 (CEST) Subject: [Zeek] Deleting my email address Message-ID: <321842000.2159668.1562922569481.JavaMail.zimbra@univ-ubs.fr> Hello, How can I delete my email address from zeek mailing list please ? Thanks in advance. Best regards, Salwa Alem -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190712/f5f8d775/attachment.html From dracosp at gmail.com Fri Jul 12 04:02:42 2019 From: dracosp at gmail.com (Panos_D.) Date: Fri, 12 Jul 2019 14:02:42 +0300 Subject: [Zeek] Fwd: Deleting my email address In-Reply-To: References: <321842000.2159668.1562922569481.JavaMail.zimbra@univ-ubs.fr> Message-ID: Hello Salwa, The same way you subscribed in I guess from the link http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek unsubscribe or edit options BRs Panos On Fri, 12 Jul 2019 at 12:11, Salwa Alem wrote: > Hello, > > How can I delete my email address from zeek mailing list please ? > > Thanks in advance. > > Best regards, > Salwa Alem > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190712/ffe82839/attachment.html From jmellander at lbl.gov Fri Jul 12 13:17:25 2019 From: jmellander at lbl.gov (Jim Mellander) Date: Fri, 12 Jul 2019 13:17:25 -0700 Subject: [Zeek] Query reagrding Bro Ids In-Reply-To: References: Message-ID: Hi Manoj: The issue you described seems more on the networking side, rather than the IDS side. However, it seems likely that a much bigger issue that a business like yours would face would be that of cybersecurity, in particular, securing your servers from unauthorized intrusion and data exfiltration. In this, Zeek (the opensource IDS formerly known as Bro) can play an important role in early detection of possible intrusions. Hope this helps, Jim On Fri, Jul 12, 2019 at 1:33 AM Manoj Petshali wrote: > Hi Team, > > Please respond as we need to implement the same at the earliest. > > Thanks > Manoj Petshali > Sr. Manager - Payments Engineering > Mobile +91-9891066456 > > www.paytm.com > > > > On Fri, Jul 12, 2019 at 10:21 AM Manoj Petshali > wrote: > >> Hi Team, >> >> I am very eager about the Bro and need to know below information : >> >> -We are working in india's biggest transactional system and facing many >> issues e.g. >> >> : if some user request is coming from pubic or private network (Internal >> request) and traverses across many servers and if user receives timeout ( >> e.g. connection time out, read time out ,rst etc) then we need to know the >> deep analysis of the same means : >> >> : Why/where the request timed out ? >> : Upto which hop the request travelled? >> : Network latency between these hopes to know if the latency is the issue? >> : tcp handshake and ssl handshake latency and the reason for the same? >> : Applicatency latency ? means if the network latency is fine >> >> We searched on wen and got feeling that the Bro is more oriented toward >> security and do deep packe inspection.But we have many problems like above >> to resolve .May you please let us know that how Bro can help us to resolve >> above issues? >> >> Thanks >> Manoj Petshali >> Sr. Manager - Payments Engineering >> Mobile +91-9891066456 >> >> www.paytm.com >> >> _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190712/b233dc24/attachment.html From asharma at lbl.gov Fri Jul 12 13:34:41 2019 From: asharma at lbl.gov (Aashish Sharma) Date: Fri, 12 Jul 2019 13:34:41 -0700 Subject: [Zeek] Query reagrding Bro Ids In-Reply-To: References: Message-ID: <20190712203440.GH2789@MacPro-2331.local> Hello Manoj, you can sure use zeek to get more visibility into your traffic and connections. It has a pretty good and powerful tcp analysis engine built into. I am sure zeek can get you a lot of diagnostic data - I say that from our experience at Berkeley Lab where we do a lot of proactive blocking and always rely on zeek's conn.log (and similar) to look into connectivity issues. So to me what you seek, is not too difficult. The difficult part for you is going to be getting this traffic into zeek or putting taps/sensors at the right places. Do you have taps on the points you want to monitor ? Aashish On Fri, Jul 12, 2019 at 01:54:43PM +0530, Manoj Petshali wrote: > Hi Team, > > Please respond as we need to implement the same at the earliest. > > Thanks > Manoj Petshali > Sr. Manager - Payments Engineering > Mobile +91-9891066456 > > www.paytm.com > > > > On Fri, Jul 12, 2019 at 10:21 AM Manoj Petshali > wrote: > > > Hi Team, > > > > I am very eager about the Bro and need to know below information : > > > > -We are working in india's biggest transactional system and facing many > > issues e.g. > > > > : if some user request is coming from pubic or private network (Internal > > request) and traverses across many servers and if user receives timeout ( > > e.g. connection time out, read time out ,rst etc) then we need to know the > > deep analysis of the same means : > > > > : Why/where the request timed out ? > > : Upto which hop the request travelled? > > : Network latency between these hopes to know if the latency is the issue? > > : tcp handshake and ssl handshake latency and the reason for the same? > > : Applicatency latency ? means if the network latency is fine > > > > We searched on wen and got feeling that the Bro is more oriented toward > > security and do deep packe inspection.But we have many problems like above > > to resolve .May you please let us know that how Bro can help us to resolve > > above issues? > > > > Thanks > > Manoj Petshali > > Sr. Manager - Payments Engineering > > Mobile +91-9891066456 > > > > www.paytm.com > > > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek From manoj.petshali at paytm.com Sat Jul 13 01:07:04 2019 From: manoj.petshali at paytm.com (Manoj Petshali) Date: Sat, 13 Jul 2019 13:37:04 +0530 Subject: [Zeek] Query reagrding Bro Ids In-Reply-To: <20190712203440.GH2789@MacPro-2331.local> References: <20190712203440.GH2789@MacPro-2331.local> Message-ID: Hi Ashish, Thanks a lot for your response. we do not have taps/sensors as of now. if we have taps placed at right places , may you elaborate what kind of difficulty we may face? Also let me know if we can filter and send the traffic (without payload) according to our requirement e.g. flags only like syn, synack,ack, timeout etc to zeek for troubleshooting. May you please share some data/charts depicting the information we are looking for (as per the trail mail ) so that we may proceed further. Thanks Manoj Petshali Sr. Manager - Payments Engineering Mobile +91-9891066456 www.paytm.com On Sat, Jul 13, 2019 at 2:04 AM Aashish Sharma wrote: > Hello Manoj, > > you can sure use zeek to get more visibility into your traffic and > connections. > It has a pretty good and powerful tcp analysis engine built into. I am > sure zeek > can get you a lot of diagnostic data - I say that from our experience at > Berkeley Lab where we do a lot of proactive blocking and always rely on > zeek's > conn.log (and similar) to look into connectivity issues. So to me what you > seek, is not too difficult. > > The difficult part for you is going to be getting this traffic into zeek > or > putting taps/sensors at the right places. > > Do you have taps on the points you want to monitor ? > > Aashish > > On Fri, Jul 12, 2019 at 01:54:43PM +0530, Manoj Petshali wrote: > > Hi Team, > > > > Please respond as we need to implement the same at the earliest. > > > > Thanks > > Manoj Petshali > > Sr. Manager - Payments Engineering > > Mobile +91-9891066456 > > > > www.paytm.com > > > > > > > > On Fri, Jul 12, 2019 at 10:21 AM Manoj Petshali < > manoj.petshali at paytm.com> > > wrote: > > > > > Hi Team, > > > > > > I am very eager about the Bro and need to know below information : > > > > > > -We are working in india's biggest transactional system and facing many > > > issues e.g. > > > > > > : if some user request is coming from pubic or private network > (Internal > > > request) and traverses across many servers and if user receives > timeout ( > > > e.g. connection time out, read time out ,rst etc) then we need to know > the > > > deep analysis of the same means : > > > > > > : Why/where the request timed out ? > > > : Upto which hop the request travelled? > > > : Network latency between these hopes to know if the latency is the > issue? > > > : tcp handshake and ssl handshake latency and the reason for the same? > > > : Applicatency latency ? means if the network latency is fine > > > > > > We searched on wen and got feeling that the Bro is more oriented toward > > > security and do deep packe inspection.But we have many problems like > above > > > to resolve .May you please let us know that how Bro can help us to > resolve > > > above issues? > > > > > > Thanks > > > Manoj Petshali > > > Sr. Manager - Payments Engineering > > > Mobile +91-9891066456 > > > > > > www.paytm.com > > > > > > > > > > > _______________________________________________ > > Zeek mailing list > > zeek at zeek.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190713/c66ed52a/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: noname Type: image/png Size: 944 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190713/c66ed52a/attachment-0001.bin From klehigh at iu.edu Mon Jul 15 07:27:57 2019 From: klehigh at iu.edu (Keith Lehigh) Date: Mon, 15 Jul 2019 10:27:57 -0400 Subject: [Zeek] ZeekWeek 2019 CFP Extension Message-ID: Greetings! We?d like to give the community a bit more time to get presentations submitted, so the CFP deadline has been extended to July 26th and we will get responses to all submitters by August 9th. The hotel group rate deadline is September 16th, so you?ll still have plenty of time to plan your travel, if talk acceptance is necessary to permit your attendance. If you?ve already submitted, and you want to tweak your entry, you?ll be able to do so with the login you created. If you?re still on the fence about submitting, now is the time to get a submission in! If you?ve got questions about the process, please feel free to email me. - Keith ZeekWeek 2019 Program Chair -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190715/3ce41475/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2511 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190715/3ce41475/attachment.bin From asharma at lbl.gov Wed Jul 17 13:02:18 2019 From: asharma at lbl.gov (Aashish Sharma) Date: Wed, 17 Jul 2019 13:02:18 -0700 Subject: [Zeek] Query reagrding Bro Ids In-Reply-To: References: <20190712203440.GH2789@MacPro-2331.local> Message-ID: <20190717200215.GF31547@MacPro-2331.local> Manoj, (Apologies for the delayed reply!) > we do not have taps/sensors as of now. if we have taps placed at right > places , may you elaborate what kind of difficulty we may face? That is generally the most difficult part - to put the taps in right places to be able to sniff the bytes - gain visibility. YOu might have issues with encryption - in which case you'd still see connection info but not the contents. I know some sites have workaround where the taps are 'beyond encryption' - ie you might want to tap behind load balancers where SSL terminates etc. If you are able to do that, you should be able to get zeek running and seeing the traffic and also reporting tcp flags/states etc. > Also let me know if we can filter and send the traffic (without payload) > according to our requirement e.g. flags only like syn, synack,ack, timeout > etc to zeek for troubleshooting. Yes, you can do that - as long as control packets are sent, zeek is able to handle most, if not all, of connection info. We at Berkeley Lab do this for one of our deployment. > May you please share some data/charts depicting the information we are > looking for (as per the trail mail ) so that we may proceed further. I am afraid I don't have data/charts for information you are looking for handy with me. I'd advice you should run zeek on a laptop/linux box - feed it some data and see if you are seeing what you desire. If so, you can scale up to your needs. roughly 4 years ago we did write a document which shows how you'd deploy zeek: go.lbl.gov/100g - may be useful. But as far as what you seek, you should look at conn.log and try to understand it: read this page -- has pretty detailed info on connection record: https://docs.zeek.org/en/stable/scripts/base/protocols/conn/main.bro.html Hope this helps. Aashish On Sat, Jul 13, 2019 at 01:37:04PM +0530, Manoj Petshali wrote: > Hi Ashish, > > Thanks a lot for your response. > we do not have taps/sensors as of now. if we have taps placed at right > places , may you elaborate what kind of difficulty we may face? > Also let me know if we can filter and send the traffic (without payload) > according to our requirement e.g. flags only like syn, synack,ack, timeout > etc to zeek for troubleshooting. > May you please share some data/charts depicting the information we are > looking for (as per the trail mail ) so that we may proceed further. > > > Thanks > Manoj Petshali > Sr. Manager - Payments Engineering > Mobile +91-9891066456 > > www.paytm.com > > > > On Sat, Jul 13, 2019 at 2:04 AM Aashish Sharma wrote: > > > Hello Manoj, > > > > you can sure use zeek to get more visibility into your traffic and > > connections. > > It has a pretty good and powerful tcp analysis engine built into. I am > > sure zeek > > can get you a lot of diagnostic data - I say that from our experience at > > Berkeley Lab where we do a lot of proactive blocking and always rely on > > zeek's > > conn.log (and similar) to look into connectivity issues. So to me what you > > seek, is not too difficult. > > > > The difficult part for you is going to be getting this traffic into zeek > > or > > putting taps/sensors at the right places. > > > > Do you have taps on the points you want to monitor ? > > > > Aashish > > > > On Fri, Jul 12, 2019 at 01:54:43PM +0530, Manoj Petshali wrote: > > > Hi Team, > > > > > > Please respond as we need to implement the same at the earliest. > > > > > > Thanks > > > Manoj Petshali > > > Sr. Manager - Payments Engineering > > > Mobile +91-9891066456 > > > > > > www.paytm.com > > > > > > > > > > > > On Fri, Jul 12, 2019 at 10:21 AM Manoj Petshali < > > manoj.petshali at paytm.com> > > > wrote: > > > > > > > Hi Team, > > > > > > > > I am very eager about the Bro and need to know below information : > > > > > > > > -We are working in india's biggest transactional system and facing many > > > > issues e.g. > > > > > > > > : if some user request is coming from pubic or private network > > (Internal > > > > request) and traverses across many servers and if user receives > > timeout ( > > > > e.g. connection time out, read time out ,rst etc) then we need to know > > the > > > > deep analysis of the same means : > > > > > > > > : Why/where the request timed out ? > > > > : Upto which hop the request travelled? > > > > : Network latency between these hopes to know if the latency is the > > issue? > > > > : tcp handshake and ssl handshake latency and the reason for the same? > > > > : Applicatency latency ? means if the network latency is fine > > > > > > > > We searched on wen and got feeling that the Bro is more oriented toward > > > > security and do deep packe inspection.But we have many problems like > > above > > > > to resolve .May you please let us know that how Bro can help us to > > resolve > > > > above issues? > > > > > > > > Thanks > > > > Manoj Petshali > > > > Sr. Manager - Payments Engineering > > > > Mobile +91-9891066456 > > > > > > > > www.paytm.com > > > > > > > > > > > > > > > > > _______________________________________________ > > > Zeek mailing list > > > zeek at zeek.org > > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > > > From neslog at gmail.com Wed Jul 17 13:45:48 2019 From: neslog at gmail.com (Neslog) Date: Wed, 17 Jul 2019 16:45:48 -0400 Subject: [Zeek] Broker & Python 3 Message-ID: I just compiled Broker and getting errors with python3x. Here's the output. # python3 -c 'import broker; print(broker.__file__)' Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'broker' # python3.6 -c 'import broker; print(broker.__file__)' Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'broker' Is this my system or something in the Broker build? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190717/375bb457/attachment.html From jsiwek at corelight.com Wed Jul 17 14:31:12 2019 From: jsiwek at corelight.com (Jon Siwek) Date: Wed, 17 Jul 2019 14:31:12 -0700 Subject: [Zeek] Broker & Python 3 In-Reply-To: References: Message-ID: On Wed, Jul 17, 2019 at 1:54 PM Neslog wrote: > > I just compiled Broker and getting errors with python3x. Here's the output. > > # python3 -c 'import broker; print(broker.__file__)' > Traceback (most recent call last): > File "", line 1, in > ModuleNotFoundError: No module named 'broker' That error is saying Python can't locate the broker module, so the general thing you need to do is figure out why it can't locate it. First thing to check would be if the Python bindings are being built at all. Does the summary of ./configure say "Python bindings: yes" ? If they're built, then it also depends on whether you're installing it or trying to run from the build directory ? If you're installing it, are you doing it as part of a Bro/Zeek install or separately on its own ? If you're not installing, then you need to tell Python where to locate it within the build directory: $ PYTHONPATH=./build/python python3 -c 'import broker; print(broker.__file__)' /Users/jon If you're installing as part of Bro/Zeek, it's likely not installed at a path where Python will locate it by default: $ PYTHONPATH=/usr/local/zeek/lib/zeekctl python3 -c 'import broker; print(broker.__file__)' /usr/local/zeek/lib/zeekctl/broker/__init__.py If you're installing Broker by itself, then it may choose a standard place where it will be located by Python by default (e.g. site-packages), but really depends on your system and how Broker is configured, so might help to get more details from the output of your ./configure - Jon From dnj0496 at gmail.com Wed Jul 17 17:07:32 2019 From: dnj0496 at gmail.com (Dk Jack) Date: Wed, 17 Jul 2019 17:07:32 -0700 Subject: [Zeek] capture filters. Message-ID: Hi, I am trying to test capture filters and I am having a hard time to get them to work. Here's my configuration: redef capture_filters += { ["host"] = "host 10.16.138.55", ["port"] = "port 443" }; redef restrict_filters += { }; Here's the broctl print command output: root at ip-10-50-30-33:/opt/bro/logs/current# /opt/bro/bin/broctl print capture_filters bro capture_filters = { [host] = host 10.16.138.55, [port] = port 443 } root at ip-10-50-30-33:/opt/bro/logs/current# Is there some other option I need to enable to get this to work? Thanks for the help... Dk. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190717/8619b1b0/attachment.html From manoj.petshali at paytm.com Wed Jul 17 21:15:31 2019 From: manoj.petshali at paytm.com (Manoj Petshali) Date: Thu, 18 Jul 2019 09:45:31 +0530 Subject: [Zeek] Query reagrding Bro Ids In-Reply-To: <20190717200215.GF31547@MacPro-2331.local> References: <20190712203440.GH2789@MacPro-2331.local> <20190717200215.GF31547@MacPro-2331.local> Message-ID: Hi Aashish, Thanks for knowledge sharing. We will check the docs and will contact you as and when required. Thanks Manoj Petshali Sr. Manager - Payments Engineering Mobile +91-9891066456 www.paytm.com On Thu, Jul 18, 2019 at 1:32 AM Aashish Sharma wrote: > Manoj, > > (Apologies for the delayed reply!) > > > we do not have taps/sensors as of now. if we have taps placed at right > > places , may you elaborate what kind of difficulty we may face? > > That is generally the most difficult part - to put the taps in right > places to be > able to sniff the bytes - gain visibility. > > YOu might have issues with encryption - in which case you'd still see > connection > info but not the contents. I know some sites have workaround where the > taps are > 'beyond encryption' - ie you might want to tap behind load balancers where > SSL > terminates etc. > > If you are able to do that, you should be able to get zeek running and > seeing > the traffic and also reporting tcp flags/states etc. > > > Also let me know if we can filter and send the traffic (without payload) > > according to our requirement e.g. flags only like syn, synack,ack, > timeout > > etc to zeek for troubleshooting. > > Yes, you can do that - as long as control packets are sent, zeek is able to > handle most, if not all, of connection info. We at Berkeley Lab do this > for one > of our deployment. > > > May you please share some data/charts depicting the information we are > > looking for (as per the trail mail ) so that we may proceed further. > > I am afraid I don't have data/charts for information you are looking for > handy > with me. I'd advice you should run zeek on a laptop/linux box - feed it > some > data and see if you are seeing what you desire. If so, you can scale up to > your > needs. > > roughly 4 years ago we did write a document which shows how you'd deploy > zeek: > go.lbl.gov/100g - may be useful. > > But as far as what you seek, you should look at conn.log and try to > understand > it: read this page -- has pretty detailed info on connection record: > > https://docs.zeek.org/en/stable/scripts/base/protocols/conn/main.bro.html > > Hope this helps. > > Aashish > > > > On Sat, Jul 13, 2019 at 01:37:04PM +0530, Manoj Petshali wrote: > > Hi Ashish, > > > > Thanks a lot for your response. > > we do not have taps/sensors as of now. if we have taps placed at right > > places , may you elaborate what kind of difficulty we may face? > > Also let me know if we can filter and send the traffic (without payload) > > according to our requirement e.g. flags only like syn, synack,ack, > timeout > > etc to zeek for troubleshooting. > > May you please share some data/charts depicting the information we are > > looking for (as per the trail mail ) so that we may proceed further. > > > > > > Thanks > > Manoj Petshali > > Sr. Manager - Payments Engineering > > Mobile +91-9891066456 > > > > www.paytm.com > > > > > > > > On Sat, Jul 13, 2019 at 2:04 AM Aashish Sharma wrote: > > > > > Hello Manoj, > > > > > > you can sure use zeek to get more visibility into your traffic and > > > connections. > > > It has a pretty good and powerful tcp analysis engine built into. I am > > > sure zeek > > > can get you a lot of diagnostic data - I say that from our experience > at > > > Berkeley Lab where we do a lot of proactive blocking and always rely on > > > zeek's > > > conn.log (and similar) to look into connectivity issues. So to me > what you > > > seek, is not too difficult. > > > > > > The difficult part for you is going to be getting this traffic into > zeek > > > or > > > putting taps/sensors at the right places. > > > > > > Do you have taps on the points you want to monitor ? > > > > > > Aashish > > > > > > On Fri, Jul 12, 2019 at 01:54:43PM +0530, Manoj Petshali wrote: > > > > Hi Team, > > > > > > > > Please respond as we need to implement the same at the earliest. > > > > > > > > Thanks > > > > Manoj Petshali > > > > Sr. Manager - Payments Engineering > > > > Mobile +91-9891066456 > > > > > > > > www.paytm.com > > > > > > > > > > > > > > > > On Fri, Jul 12, 2019 at 10:21 AM Manoj Petshali < > > > manoj.petshali at paytm.com> > > > > wrote: > > > > > > > > > Hi Team, > > > > > > > > > > I am very eager about the Bro and need to know below information : > > > > > > > > > > -We are working in india's biggest transactional system and facing > many > > > > > issues e.g. > > > > > > > > > > : if some user request is coming from pubic or private network > > > (Internal > > > > > request) and traverses across many servers and if user receives > > > timeout ( > > > > > e.g. connection time out, read time out ,rst etc) then we need to > know > > > the > > > > > deep analysis of the same means : > > > > > > > > > > : Why/where the request timed out ? > > > > > : Upto which hop the request travelled? > > > > > : Network latency between these hopes to know if the latency is the > > > issue? > > > > > : tcp handshake and ssl handshake latency and the reason for the > same? > > > > > : Applicatency latency ? means if the network latency is fine > > > > > > > > > > We searched on wen and got feeling that the Bro is more oriented > toward > > > > > security and do deep packe inspection.But we have many problems > like > > > above > > > > > to resolve .May you please let us know that how Bro can help us to > > > resolve > > > > > above issues? > > > > > > > > > > Thanks > > > > > Manoj Petshali > > > > > Sr. Manager - Payments Engineering > > > > > Mobile +91-9891066456 > > > > > > > > > > www.paytm.com > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > Zeek mailing list > > > > zeek at zeek.org > > > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190718/cf3b6f2e/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: noname Type: image/png Size: 944 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190718/cf3b6f2e/attachment-0001.bin From neslog at gmail.com Thu Jul 18 08:12:55 2019 From: neslog at gmail.com (Neslog) Date: Thu, 18 Jul 2019 11:12:55 -0400 Subject: [Zeek] Broker & Python 3 In-Reply-To: References: Message-ID: Broker was installed with Bro. Python wasn't working so it was installed via source also. Python 2.7 bindings was installed in the right place. But Python3.6 didn't install right. I'm not seeing broker in any python3.x folder. # python2.7 -c 'import broker; print(broker.__file__)' /usr/lib/python2.7/dist-packages/broker/__init__.pyc # python2 -c 'import broker; print(broker.__file__)' /usr/lib/python2.7/dist-packages/broker/__init__.pyc # python3 -c 'import broker; print(broker.__file__)' Traceback (most recent call last): File "", line 1, in AttributeError: module 'broker' has no attribute '__file__' # python3.6 -c 'import broker; print(broker.__file__)' Traceback (most recent call last): File "", line 1, in AttributeError: module 'broker' has no attribute '__file__' On Wed, Jul 17, 2019 at 5:31 PM Jon Siwek wrote: > On Wed, Jul 17, 2019 at 1:54 PM Neslog wrote: > > > > I just compiled Broker and getting errors with python3x. Here's the > output. > > > > # python3 -c 'import broker; print(broker.__file__)' > > Traceback (most recent call last): > > File "", line 1, in > > ModuleNotFoundError: No module named 'broker' > > That error is saying Python can't locate the broker module, so the > general thing you need to do is figure out why it can't locate it. > > First thing to check would be if the Python bindings are being built > at all. Does the summary of ./configure say "Python bindings: yes" ? > > If they're built, then it also depends on whether you're installing it > or trying to run from the build directory ? If you're installing it, > are you doing it as part of a Bro/Zeek install or separately on its > own ? > > If you're not installing, then you need to tell Python where to locate > it within the build directory: > > $ PYTHONPATH=./build/python python3 -c 'import broker; > print(broker.__file__)' > /Users/jon > > If you're installing as part of Bro/Zeek, it's likely not installed at > a path where Python will locate it by default: > > $ PYTHONPATH=/usr/local/zeek/lib/zeekctl python3 -c 'import broker; > print(broker.__file__)' > /usr/local/zeek/lib/zeekctl/broker/__init__.py > > If you're installing Broker by itself, then it may choose a standard > place where it will be located by Python by default (e.g. > site-packages), but really depends on your system and how Broker is > configured, so might help to get more details from the output of your > ./configure > > - Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190718/37f3741e/attachment.html From jsiwek at corelight.com Thu Jul 18 09:14:50 2019 From: jsiwek at corelight.com (Jon Siwek) Date: Thu, 18 Jul 2019 09:14:50 -0700 Subject: [Zeek] Broker & Python 3 In-Reply-To: References: Message-ID: On Thu, Jul 18, 2019 at 8:13 AM Neslog wrote: > > Python 2.7 bindings was installed in the right place. But Python3.6 didn't install right. I'm not seeing broker in any python3.x folder. Any given build/install of Broker only picks a single version of Python for which to provide bindings. The output of ./configure tells you which one (likely whatever is first in your PATH). There's also the --with-python= and --with-python-config options if you need them. - Jon From hbhania28 at gmail.com Mon Jul 22 04:35:56 2019 From: hbhania28 at gmail.com (Hania) Date: Mon, 22 Jul 2019 16:35:56 +0500 Subject: [Zeek] Use cases of Bro for Threat hunting Message-ID: Hi all, Can you please share some use cases of Bro in threat hunting. Examples like Bro logs in term of validating particular hypothesis for threat hunting. Will really appreciate if you can share some great resources here. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190722/1dd55cb9/attachment.html From akgraner at corelight.com Mon Jul 22 06:06:26 2019 From: akgraner at corelight.com (Amber Graner) Date: Mon, 22 Jul 2019 08:06:26 -0500 Subject: [Zeek] Use cases of Bro for Threat hunting In-Reply-To: References: Message-ID: Hania, Here's a link to some uses case examples - https://docs.zeek.org/en/stable/examples/ I'm sure others on the list can point you to more specific uses cases. Thanks, ~Amber On Mon, Jul 22, 2019 at 6:39 AM Hania wrote: > Hi all, > > Can you please share some use cases of Bro in threat hunting. Examples > like Bro logs in term of validating particular hypothesis for threat > hunting. Will really appreciate if you can share some great resources here. > > Thanks > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190722/de39d919/attachment.html From chris at cwalsh.org Mon Jul 22 10:02:04 2019 From: chris at cwalsh.org (Chris Walsh) Date: Mon, 22 Jul 2019 12:02:04 -0500 Subject: [Zeek] Use cases of Bro for Threat hunting In-Reply-To: References: Message-ID: One useful source might be Liam Randall?s training materials, which you can find at https://github.com/LiamRandall An example of identifying and tracing the behavior of specific malware samples is https://github.com/LiamRandall/BroMalware-Exercise Additionally, some of the presentations at BroCon 2017 went into detail about identifying/analysing specific real-world threats: https://www.zeek.org/community/brocon2017.html Chris > On Jul 22, 2019, at 8:06 AM, Amber Graner wrote: > > Hania, > > Here's a link to some uses case examples - https://docs.zeek.org/en/stable/examples/ > > I'm sure others on the list can point you to more specific uses cases. > From j.blakey at rheagroup.com Tue Jul 23 05:08:14 2019 From: j.blakey at rheagroup.com (Jason Blakey) Date: Tue, 23 Jul 2019 12:08:14 +0000 Subject: [Zeek] Question about missing cert expired notice... Message-ID: Hi all, We're running Bro 2.5.3, and I've noticed that what seems to be the same certificate sometimes get flagged as expired, sometimes not. For instance, here are the log entries associated with two connection UIDs, one set that generated an 'expired cert' notice, and one that didn't. Given that the hashes for the cert are the same, i would think both should have been classified one way or the other. I've replaced the two IPs with 10.x.x.x and 178.x.x.x, but they were consistent across both connection UIDs. CERT APPARENTLY VALID grep CNPeEy2dKUx3LGty0k * conn.21:23:00-21:24:00.log:1563845011.183787 CNPeEy2dKUx3LGty0k 10.x.x.x 55847 178.x.x.x 443 tcp ssl 1.673746 373 3246 SF T F 190 ShADadFf 7475 8 3574 (empty) files.21:23:00-21:24:00.log:1563845011.430465 FWRyJzWEVQN1qjrxd 178.x.x.x 10.x.x.x CNPeEy2dKUx3LGty0k SSL 0 SHA256,X509,SHA1,MD5 application/pkix-cert - 0.000000 FF 1368 - 0 0 F - 8f63ea9982d47eaedde789dc7d81c4bb a4875d27ad671e92703eec7145b76297ad2ee476 76221dfe013cc6fd2b46294e823349be51617b5eee8069c53da32502b6b1a099 -- - 10.x.x.x 178.x.x.x tcp - - - - - files.21:23:00-21:24:00.log:1563845011.430465 FiWpkz4BkQGSWA36O7 178.x.x.x 10.x.x.x CNPeEy2dKUx3LGty0k SSL 0 SHA256,X509,SHA1,MD5 application/pkix-cert - 0.000000 FF 1174 - 0 0 F - b15409274f54ad8f023d3b85a5ecec5d e6a3b45b062d509b3382282d196efe97d5956ccb 25847d668eb4f04fdd40b12b6b0740c567da7d024308eb6c2c96fe41d9de218d -- - 10.x.x.x 178.x.x.x tcp - - - - - ssl.21:23:00-21:24:00.log:1563845011.305071 CNPeEy2dKUx3LGty0k 10.x.x.x 55847 178.x.x.x 443 TLSv12 TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 - adserver.video F - - FFWRyJzWEVQN1qjrxd,FiWpkz4BkQGSWA36O7 (empty) - - - - - - - - - - - - - weird.21:23:00-21:24:00.log:1563845011.430440 CNPeEy2dKUx3LGty0k 10.x.x.x 55847 178.x.x.x 443 window_recision - F main-s0f0-1 WHOOPS - LOOKS LIKE CERT IS INVALID grep CsnEaj44xdEyJva9W8 * conn.21:23:00-21:24:00.log:1563845012.953787 CsnEaj44xdEyJva9W8 10.x.x.x 55936 178.x.x.x 443 tcp ssl 0.436407 405 3246 SF T F 0 ShADadFf 7697 8 3574 (empty) files.21:23:00-21:24:00.log:1563845013.144430 F7hMB24QQdYAnV68Ia 178.x.x.x 10.x.x.x CsnEaj44xdEyJva9W8 SSL 0 SHA256,X509,SHA1,MD5 application/pkix-cert - 0.000000 FF 1368 - 0 0 F - 8f63ea9982d47eaedde789dc7d81c4bb a4875d27ad671e92703eec7145b76297ad2ee476 76221dfe013cc6fd2b46294e823349be51617b5eee8069c53da32502b6b1a099 -- - 10.x.x.x 178.x.x.x tcp - - - - - files.21:23:00-21:24:00.log:1563845013.144430 FjcGCY1SisKJ6ZzYD2 178.x.x.x 10.x.x.x CsnEaj44xdEyJva9W8 SSL 0 SHA256,X509,SHA1,MD5 application/pkix-cert - 0.000000 FF 1174 - 0 0 F - b15409274f54ad8f023d3b85a5ecec5d e6a3b45b062d509b3382282d196efe97d5956ccb 25847d668eb4f04fdd40b12b6b0740c567da7d024308eb6c2c96fe41d9de218d -- - 10.x.x.x 178.x.x.x tcp - - - - - notice.21:23:00-21:24:00.log:1563845013.441985 CsnEaj44xdEyJva9W8 10.x.x.x 55936 178.x.x.x - - - tcp SSL::Invalid_Server_Cert SSL certificate validation failed with (certificate has expired) CN=adserver.video 10.x.x.x 178.x.x.x 443 - main-s0f0-1 Notice::ACTION_LOG 3600.000000 F - - - - - ssl.21:23:00-21:24:00.log:1563845013.048317 CsnEaj44xdEyJva9W8 10.x.x.x 55936 178.x.x.x 443 TLSv12 TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 - adserver.video F - - TF7hMB24QQdYAnV68Ia,FjcGCY1SisKJ6ZzYD2 (empty) CN=adserver.video CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US - - certificate has expired - - - - - - -- weird.21:23:00-21:24:00.log:1563845013.142121 CsnEaj44xdEyJva9W8 10.x.x.x 55936 178.x.x.x 443 window_recision - F main-s0f0-1 Does anyone have any thoughts? Are these actually the same certs? Anyone know if this 1) is a bug and 2) is fixed in later versions? Thanks, jason -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190723/9bd223b1/attachment.html From johanna at icir.org Tue Jul 23 07:51:34 2019 From: johanna at icir.org (Johanna Amann) Date: Tue, 23 Jul 2019 07:51:34 -0700 Subject: [Zeek] Question about missing cert expired notice... In-Reply-To: References: Message-ID: <20190723145134.5jpq3emjrzltwuyw@wifi189.sys.ICSI.Berkeley.EDU> Hi Jason, > We're running Bro 2.5.3, first - to state the obvious - please upgrade your installtion. There are _multiple_ security issues in 2.5.3. These can be at least used to crash your installation. > ssl.21:23:00-21:24:00.log:1563845011.305071 CNPeEy2dKUx3LGty0k 10.x.x.x 55847 178.x.x.x 443 TLSv12 TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 - adserver.video F - - FFWRyJzWEVQN1qjrxd,FiWpkz4BkQGSWA36O7 (empty) - - - - - - - - - - - - - Actually - no certificate validation was performed at allhere (the entries that would state anything about validity simply are not present, meaning that the validation code was not run). The reason for that is probably a bug in 2.5.x - in versions before 2.6, certificate validation was only performed if the connection was recognized as established. If packets were missing at the right time of the connection, this did not happen; there also were some edge-cases were the detection failed. This is fixed in 2.6.x Johanna From akgraner at corelight.com Wed Jul 24 10:47:02 2019 From: akgraner at corelight.com (Amber Graner) Date: Wed, 24 Jul 2019 12:47:02 -0500 Subject: [Zeek] ZeekWeek 19 Keynote Speaker Announced - Freddy DeZeure Message-ID: You can find out more about Freddy via the blog post. Link is below. Complacency is not an option - Freddy DeZeure to keynote ZeekWeek 2019 - https://blog.zeek.org/2019/07/complacency-is-not-option-freddy.html Registration is open - http://bit.ly/zeekweek19_registration Call for Papers ends this Friday 26 July 2019 - http://bit.ly/zeekweek19talksubmission If you have any questions, please let me know. See you in Seattle, ~Amber -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190724/2213a580/attachment.html From x.faith at gmail.com Wed Jul 24 13:52:21 2019 From: x.faith at gmail.com (David Decker) Date: Wed, 24 Jul 2019 13:52:21 -0700 Subject: [Zeek] Workers Message-ID: Is it normal for the workers to stop? Test environment, so not actively passing traffic, but need to run broctl restart of broctl deploy to get the workers to start back up. What are good things to look to see what the issues that is causing it. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190724/3ca855a9/attachment.html From jlay at slave-tothe-box.net Wed Jul 24 14:05:30 2019 From: jlay at slave-tothe-box.net (James Lay) Date: Wed, 24 Jul 2019 15:05:30 -0600 Subject: [Zeek] Workers In-Reply-To: References: Message-ID: On 2019-07-24 14:52, David Decker wrote: > Is it normal for the workers to stop? Test environment, so not > actively passing traffic, but need to run broctl restart of broctl > deploy to get the workers to start back up. > > What are good things to look to see what the issues that is causing > it. > > Thanks > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek Make sure you've added: 0-59/5 * * * * root /broctl cron to your crontab. James From cherish_139 at foxmail.com Thu Jul 25 05:41:13 2019 From: cherish_139 at foxmail.com (=?gb18030?B?Y2hlcmlzaA==?=) Date: Thu, 25 Jul 2019 20:41:13 +0800 Subject: [Zeek] help Message-ID: ????iPhone ------------------ Original ------------------ From: zeek-request Date: Sat,Jul 13,2019 4:07 PM To: zeek Subject: Re: Zeek Digest, Vol 159, Issue 14 Send Zeek mailing list submissions to zeek at zeek.org To subscribe or unsubscribe via the World Wide Web, visit http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek or, via email, send a message with subject or body 'help' to zeek-request at zeek.org You can reach the person managing the list at zeek-owner at zeek.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Zeek digest..." Today's Topics: 1. Re: Query reagrding Bro Ids (Jim Mellander) 2. Re: Query reagrding Bro Ids (Aashish Sharma) 3. Re: Query reagrding Bro Ids (Manoj Petshali) ---------------------------------------------------------------------- Message: 1 Date: Fri, 12 Jul 2019 13:17:25 -0700 From: Jim Mellander Subject: Re: [Zeek] Query reagrding Bro Ids To: Manoj Petshali Cc: Payments Network Team ,zeek Message-ID: Content-Type: text/plain; charset="utf-8" Hi Manoj: The issue you described seems more on the networking side, rather than the IDS side. However, it seems likely that a much bigger issue that a business like yours would face would be that of cybersecurity, in particular, securing your servers from unauthorized intrusion and data exfiltration. In this, Zeek (the opensource IDS formerly known as Bro) can play an important role in early detection of possible intrusions. Hope this helps, Jim On Fri, Jul 12, 2019 at 1:33 AM Manoj Petshali wrote: > Hi Team, > > Please respond as we need to implement the same at the earliest. > > Thanks > Manoj Petshali > Sr. Manager - Payments Engineering > Mobile +91-9891066456 > > www.paytm.com > > > > On Fri, Jul 12, 2019 at 10:21 AM Manoj Petshali > wrote: > >> Hi Team, >> >> I am very eager about the Bro and need to know below information : >> >> -We are working in india's biggest transactional system and facing many >> issues e.g. >> >> : if some user request is coming from pubic or private network (Internal >> request) and traverses across many servers and if user receives timeout ( >> e.g. connection time out, read time out ,rst etc) then we need to know the >> deep analysis of the same means : >> >> : Why/where the request timed out ? >> : Upto which hop the request travelled? >> : Network latency between these hopes to know if the latency is the issue? >> : tcp handshake and ssl handshake latency and the reason for the same? >> : Applicatency latency ? means if the network latency is fine >> >> We searched on wen and got feeling that the Bro is more oriented toward >> security and do deep packe inspection.But we have many problems like above >> to resolve .May you please let us know that how Bro can help us to resolve >> above issues? >> >> Thanks >> Manoj Petshali >> Sr. Manager - Payments Engineering >> Mobile +91-9891066456 >> >> www.paytm.com >> >> _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190712/b233dc24/attachment-0001.html ------------------------------ Message: 2 Date: Fri, 12 Jul 2019 13:34:41 -0700 From: Aashish Sharma Subject: Re: [Zeek] Query reagrding Bro Ids To: Manoj Petshali Cc: Payments Network Team , zeek at zeek.org Message-ID: <20190712203440.GH2789 at MacPro-2331.local> Content-Type: text/plain; charset=us-ascii Hello Manoj, you can sure use zeek to get more visibility into your traffic and connections. It has a pretty good and powerful tcp analysis engine built into. I am sure zeek can get you a lot of diagnostic data - I say that from our experience at Berkeley Lab where we do a lot of proactive blocking and always rely on zeek's conn.log (and similar) to look into connectivity issues. So to me what you seek, is not too difficult. The difficult part for you is going to be getting this traffic into zeek or putting taps/sensors at the right places. Do you have taps on the points you want to monitor ? Aashish On Fri, Jul 12, 2019 at 01:54:43PM +0530, Manoj Petshali wrote: > Hi Team, > > Please respond as we need to implement the same at the earliest. > > Thanks > Manoj Petshali > Sr. Manager - Payments Engineering > Mobile +91-9891066456 > > www.paytm.com > > > > On Fri, Jul 12, 2019 at 10:21 AM Manoj Petshali > wrote: > > > Hi Team, > > > > I am very eager about the Bro and need to know below information : > > > > -We are working in india's biggest transactional system and facing many > > issues e.g. > > > > : if some user request is coming from pubic or private network (Internal > > request) and traverses across many servers and if user receives timeout ( > > e.g. connection time out, read time out ,rst etc) then we need to know the > > deep analysis of the same means : > > > > : Why/where the request timed out ? > > : Upto which hop the request travelled? > > : Network latency between these hopes to know if the latency is the issue? > > : tcp handshake and ssl handshake latency and the reason for the same? > > : Applicatency latency ? means if the network latency is fine > > > > We searched on wen and got feeling that the Bro is more oriented toward > > security and do deep packe inspection.But we have many problems like above > > to resolve .May you please let us know that how Bro can help us to resolve > > above issues? > > > > Thanks > > Manoj Petshali > > Sr. Manager - Payments Engineering > > Mobile +91-9891066456 > > > > www.paytm.com > > > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek ------------------------------ Message: 3 Date: Sat, 13 Jul 2019 13:37:04 +0530 From: Manoj Petshali Subject: Re: [Zeek] Query reagrding Bro Ids To: Aashish Sharma Cc: Payments Network Team , zeek at zeek.org Message-ID: Content-Type: text/plain; charset="utf-8" Hi Ashish, Thanks a lot for your response. we do not have taps/sensors as of now. if we have taps placed at right places , may you elaborate what kind of difficulty we may face? Also let me know if we can filter and send the traffic (without payload) according to our requirement e.g. flags only like syn, synack,ack, timeout etc to zeek for troubleshooting. May you please share some data/charts depicting the information we are looking for (as per the trail mail ) so that we may proceed further. Thanks Manoj Petshali Sr. Manager - Payments Engineering Mobile +91-9891066456 www.paytm.com On Sat, Jul 13, 2019 at 2:04 AM Aashish Sharma wrote: > Hello Manoj, > > you can sure use zeek to get more visibility into your traffic and > connections. > It has a pretty good and powerful tcp analysis engine built into. I am > sure zeek > can get you a lot of diagnostic data - I say that from our experience at > Berkeley Lab where we do a lot of proactive blocking and always rely on > zeek's > conn.log (and similar) to look into connectivity issues. So to me what you > seek, is not too difficult. > > The difficult part for you is going to be getting this traffic into zeek > or > putting taps/sensors at the right places. > > Do you have taps on the points you want to monitor ? > > Aashish > > On Fri, Jul 12, 2019 at 01:54:43PM +0530, Manoj Petshali wrote: > > Hi Team, > > > > Please respond as we need to implement the same at the earliest. > > > > Thanks > > Manoj Petshali > > Sr. Manager - Payments Engineering > > Mobile +91-9891066456 > > > > www.paytm.com > > > > > > > > On Fri, Jul 12, 2019 at 10:21 AM Manoj Petshali < > manoj.petshali at paytm.com> > > wrote: > > > > > Hi Team, > > > > > > I am very eager about the Bro and need to know below information : > > > > > > -We are working in india's biggest transactional system and facing many > > > issues e.g. > > > > > > : if some user request is coming from pubic or private network > (Internal > > > request) and traverses across many servers and if user receives > timeout ( > > > e.g. connection time out, read time out ,rst etc) then we need to know > the > > > deep analysis of the same means : > > > > > > : Why/where the request timed out ? > > > : Upto which hop the request travelled? > > > : Network latency between these hopes to know if the latency is the > issue? > > > : tcp handshake and ssl handshake latency and the reason for the same? > > > : Applicatency latency ? means if the network latency is fine > > > > > > We searched on wen and got feeling that the Bro is more oriented toward > > > security and do deep packe inspection.But we have many problems like > above > > > to resolve .May you please let us know that how Bro can help us to > resolve > > > above issues? > > > > > > Thanks > > > Manoj Petshali > > > Sr. Manager - Payments Engineering > > > Mobile +91-9891066456 > > > > > > www.paytm.com > > > > > > > > > > > _______________________________________________ > > Zeek mailing list > > zeek at zeek.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190713/c66ed52a/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: noname Type: image/png Size: 944 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190713/c66ed52a/attachment.bin ------------------------------ _______________________________________________ Zeek mailing list Zeek at zeek.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek End of Zeek Digest, Vol 159, Issue 14 ************************************* -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190725/a478ed69/attachment-0001.html From akgraner at corelight.com Thu Jul 25 15:16:26 2019 From: akgraner at corelight.com (Amber Graner) Date: Thu, 25 Jul 2019 17:16:26 -0500 Subject: [Zeek] Announcing The Zeek Package Contest Message-ID: Hi all, I'm excited to announce the Zeek Package Contest. - Are you a Zeek user? - Do you enjoy writing Zeek scripts? - Do you like being recognized for your awesome work? - Do you want to make the world?s networks safer? - Do you like winning prizes and claiming bragging rights? - Do you want the opportunity to present your work at Zeek events? More information and details on how you can participate can be found on the Zeek Blog at: https://blog.zeek.org/2019/07/zeek-package-contest_25.html Please let us know if you have any questions. Thanks, ~Amber -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190725/e7f41284/attachment.html From akgraner at corelight.com Fri Jul 26 09:13:57 2019 From: akgraner at corelight.com (Amber Graner) Date: Fri, 26 Jul 2019 11:13:57 -0500 Subject: [Zeek] ZeekDays - Technical User Workshops - Poll Message-ID: Hi all, Various organizations are considering sponsoring a series of single-day Zeek User Workshops across multiple geographies, starting in Atlanta, GA this fall. I've created a short (7 question) survey poll to find out what topics are of interest to you. This will allow me work with the sponsoring organizations to provide the most relevant content. Please take a moment to give your feedback - https://www.surveymonkey.com/r/G7PLPDZ Also if you or your organization would like to host a Zeek Event, please let me know. Thanks, ~Amber -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190726/c768fb9b/attachment.html From akgraner at corelight.com Fri Jul 26 10:02:05 2019 From: akgraner at corelight.com (Amber Graner) Date: Fri, 26 Jul 2019 12:02:05 -0500 Subject: [Zeek] Reminder - Call for Papers for ZeekWeek Ends Today Message-ID: Hi all, ZeekWeek 2019 Call for Papers ends tonight. There's still a few hours left to get that talk in. Submission Link - http://bit.ly/zeekweek19talksubmission Thanks, ~Amber -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190726/9b8c0d34/attachment.html From vinod76 at gmail.com Mon Jul 29 05:39:51 2019 From: vinod76 at gmail.com (Vinod) Date: Mon, 29 Jul 2019 18:09:51 +0530 Subject: [Zeek] (no subject) Message-ID: Hi, I am trying to compile the Zeek XDP plugin ( https://github.com/irtimmer/bro-xdp_packet-plugin) and running into lots of errors. posting the configuration and errors below Host: Ubuntu 18.04 Kernels tested with: 4.18.0.18 and 5.1 Any help will be greatly appreciated. neo at base-ubuntu:~/bro-xdp_packet-plugin$ ./configure --bro-dist=/home/neo/zeek --with-kernel=/usr/src/linux-5.1 --with-bpf=/usr/local --with-clang=/usr/bin/clang --with-llc=/usr/bin/llc Build Directory : build Bro Source Directory : /home/neo/zeek -- Bro executable : /home/neo/zeek/build/src/bro -- Bro source : /home/neo/zeek -- Bro build : /home/neo/zeek/build -- Bro install prefix : /opt/bro -- Bro plugin directory: /opt/bro/lib/bro/plugins -- Bro debug mode : false -- Found KernelHeaders: /usr/src/linux-5.1 -- Configuring done -- Generating done -- Build files have been written to: /home/neo/bro-xdp_packet-plugin/build neo at base-ubuntu:~/bro-xdp_packet-plugin$ make Makefile:11: recipe for target 'build-it' failed make: [build-it] Error 1 (ignored) ( cd build && make ) make[1]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' make[2]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' make[3]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' Scanning dependencies of target bif-plugin-irtimmer_af_xdp-af_xdp.bif make[3]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' make[3]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' [ 7%] [BIFCL] Processing src/af_xdp.bif make[3]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' [ 7%] Built target bif-plugin-irtimmer_af_xdp-af_xdp.bif make[3]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' Scanning dependencies of target generate_outputs make[3]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' [ 7%] Built target generate_outputs make[3]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' Scanning dependencies of target af_xdp_bpf make[3]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' make[3]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' [ 15%] Generating af_xdp_bpf.ll In file included from /home/neo/bro-xdp_packet-plugin/src/bpf/bpf.c:5: */home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:75:11: error: use of undeclared identifier 'BPF_FUNC_sock_ops_cb_flags_set' (void *) BPF_FUNC_sock_ops_cb_flags_set; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:79:11: error: use of undeclared identifier 'BPF_FUNC_sk_redirect_hash' (void *) BPF_FUNC_sk_redirect_hash; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:85:11: error: use of undeclared identifier 'BPF_FUNC_sock_hash_update' (void *) BPF_FUNC_sock_hash_update; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:93:11: error: use of undeclared identifier 'BPF_FUNC_override_return' (void *) BPF_FUNC_override_return; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:95:11: error: use of undeclared identifier 'BPF_FUNC_msg_redirect_map' (void *) BPF_FUNC_msg_redirect_map; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:98:11: error: use of undeclared identifier 'BPF_FUNC_msg_redirect_hash' (void *) BPF_FUNC_msg_redirect_hash; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:100:11: error: use of undeclared identifier 'BPF_FUNC_msg_apply_bytes' (void *) BPF_FUNC_msg_apply_bytes; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:102:11: error: use of undeclared identifier 'BPF_FUNC_msg_cork_bytes' (void *) BPF_FUNC_msg_cork_bytes; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:104:11: error: use of undeclared identifier 'BPF_FUNC_msg_pull_data' (void *) BPF_FUNC_msg_pull_data; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:106:11: error: use of undeclared identifier 'BPF_FUNC_bind' (void *) BPF_FUNC_bind; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:108:11: error: use of undeclared identifier 'BPF_FUNC_xdp_adjust_tail' (void *) BPF_FUNC_xdp_adjust_tail; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:111:11: error: use of undeclared identifier 'BPF_FUNC_skb_get_xfrm_state' (void *) BPF_FUNC_skb_get_xfrm_state; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:113:11: error: use of undeclared identifier 'BPF_FUNC_get_stack' (void *) BPF_FUNC_get_stack; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:114:48: warning: declaration of 'struct bpf_fib_lookup' will not be visible outside of this function [-Wvisibility]static int (*bpf_fib_lookup)(void *ctx, struct bpf_fib_lookup *params, ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:116:11: error: use of undeclared identifier 'BPF_FUNC_fib_lookup' (void *) BPF_FUNC_fib_lookup; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:119:11: error: use of undeclared identifier 'BPF_FUNC_lwt_push_encap' (void *) BPF_FUNC_lwt_push_encap; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:122:11: error: use of undeclared identifier 'BPF_FUNC_lwt_seg6_store_bytes' (void *) BPF_FUNC_lwt_seg6_store_bytes; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:125:11: error: use of undeclared identifier 'BPF_FUNC_lwt_seg6_action' (void *) BPF_FUNC_lwt_seg6_action; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:128:11: error: use of undeclared identifier 'BPF_FUNC_lwt_seg6_adjust_srh' (void *) BPF_FUNC_lwt_seg6_adjust_srh; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:130:11: error: use of undeclared identifier 'BPF_FUNC_rc_repeat' (void *) BPF_FUNC_rc_repeat;* ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 1 warning and 20 errors generated. CMakeFiles/af_xdp_bpf.dir/build.make:64: recipe for target 'af_xdp_bpf.ll' failed make[3]: *** [af_xdp_bpf.ll] Error 1 make[3]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' CMakeFiles/Makefile2:169: recipe for target 'CMakeFiles/af_xdp_bpf.dir/all' failed make[2]: *** [CMakeFiles/af_xdp_bpf.dir/all] Error 2 make[2]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' Makefile:151: recipe for target 'all' failed make[1]: *** [all] Error 2 make[1]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' Makefile:11: recipe for target 'build-it' failed make: *** [build-it] Error 2 Regards, Vinod -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190729/2dfcaa28/attachment.html From vinod76 at gmail.com Mon Jul 29 05:43:43 2019 From: vinod76 at gmail.com (Vinod) Date: Mon, 29 Jul 2019 18:13:43 +0530 Subject: [Zeek] Zeek XDP Plugin - Ubuntu Compile problem Message-ID: Hi, I am trying to compile the Zeek XDP plugin ( https://github.com/irtimmer/bro-xdp_packet-plugin) and running into lots of errors. posting the configuration and errors below Host: Ubuntu 18.04 Kernels tested with: 4.18.0.18 and 5.1 Any help will be greatly appreciated. neo at base-ubuntu:~/bro-xdp_packet-plugin$ ./configure --bro-dist=/home/neo/zeek --with-kernel=/usr/src/linux-5.1 --with-bpf=/usr/local --with-clang=/usr/bin/clang --with-llc=/usr/bin/llc Build Directory : build Bro Source Directory : /home/neo/zeek -- Bro executable : /home/neo/zeek/build/src/bro -- Bro source : /home/neo/zeek -- Bro build : /home/neo/zeek/build -- Bro install prefix : /opt/bro -- Bro plugin directory: /opt/bro/lib/bro/plugins -- Bro debug mode : false -- Found KernelHeaders: /usr/src/linux-5.1 -- Configuring done -- Generating done -- Build files have been written to: /home/neo/bro-xdp_packet-plugin/build neo at base-ubuntu:~/bro-xdp_packet-plugin$ make Makefile:11: recipe for target 'build-it' failed make: [build-it] Error 1 (ignored) ( cd build && make ) make[1]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' make[2]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' make[3]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' Scanning dependencies of target bif-plugin-irtimmer_af_xdp-af_xdp.bif make[3]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' make[3]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' [ 7%] [BIFCL] Processing src/af_xdp.bif make[3]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' [ 7%] Built target bif-plugin-irtimmer_af_xdp-af_xdp.bif make[3]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' Scanning dependencies of target generate_outputs make[3]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' [ 7%] Built target generate_outputs make[3]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' Scanning dependencies of target af_xdp_bpf make[3]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' make[3]: Entering directory '/home/neo/bro-xdp_packet-plugin/build' [ 15%] Generating af_xdp_bpf.ll In file included from /home/neo/bro-xdp_packet-plugin/src/bpf/bpf.c:5: */home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:75:11: error: use of undeclared identifier 'BPF_FUNC_sock_ops_cb_flags_set' (void *) BPF_FUNC_sock_ops_cb_flags_set; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:79:11: error: use of undeclared identifier 'BPF_FUNC_sk_redirect_hash' (void *) BPF_FUNC_sk_redirect_hash; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:85:11: error: use of undeclared identifier 'BPF_FUNC_sock_hash_update' (void *) BPF_FUNC_sock_hash_update; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:93:11: error: use of undeclared identifier 'BPF_FUNC_override_return' (void *) BPF_FUNC_override_return; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:95:11: error: use of undeclared identifier 'BPF_FUNC_msg_redirect_map' (void *) BPF_FUNC_msg_redirect_map; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:98:11: error: use of undeclared identifier 'BPF_FUNC_msg_redirect_hash' (void *) BPF_FUNC_msg_redirect_hash; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:100:11: error: use of undeclared identifier 'BPF_FUNC_msg_apply_bytes' (void *) BPF_FUNC_msg_apply_bytes; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:102:11: error: use of undeclared identifier 'BPF_FUNC_msg_cork_bytes' (void *) BPF_FUNC_msg_cork_bytes; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:104:11: error: use of undeclared identifier 'BPF_FUNC_msg_pull_data' (void *) BPF_FUNC_msg_pull_data; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:106:11: error: use of undeclared identifier 'BPF_FUNC_bind' (void *) BPF_FUNC_bind; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:108:11: error: use of undeclared identifier 'BPF_FUNC_xdp_adjust_tail' (void *) BPF_FUNC_xdp_adjust_tail; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:111:11: error: use of undeclared identifier 'BPF_FUNC_skb_get_xfrm_state' (void *) BPF_FUNC_skb_get_xfrm_state; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:113:11: error: use of undeclared identifier 'BPF_FUNC_get_stack' (void *) BPF_FUNC_get_stack; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:114:48: warning: declaration of 'struct bpf_fib_lookup' will not be visible outside of this function [-Wvisibility]static int (*bpf_fib_lookup)(void *ctx, struct bpf_fib_lookup *params, ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:116:11: error: use of undeclared identifier 'BPF_FUNC_fib_lookup' (void *) BPF_FUNC_fib_lookup; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:119:11: error: use of undeclared identifier 'BPF_FUNC_lwt_push_encap' (void *) BPF_FUNC_lwt_push_encap; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:122:11: error: use of undeclared identifier 'BPF_FUNC_lwt_seg6_store_bytes' (void *) BPF_FUNC_lwt_seg6_store_bytes; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:125:11: error: use of undeclared identifier 'BPF_FUNC_lwt_seg6_action' (void *) BPF_FUNC_lwt_seg6_action; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:128:11: error: use of undeclared identifier 'BPF_FUNC_lwt_seg6_adjust_srh' (void *) BPF_FUNC_lwt_seg6_adjust_srh; ^/home/neo/bro-xdp_packet-plugin/src/bpf/bpf_helpers.h:130:11: error: use of undeclared identifier 'BPF_FUNC_rc_repeat' (void *) BPF_FUNC_rc_repeat;* ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 1 warning and 20 errors generated. CMakeFiles/af_xdp_bpf.dir/build.make:64: recipe for target 'af_xdp_bpf.ll' failed make[3]: *** [af_xdp_bpf.ll] Error 1 make[3]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' CMakeFiles/Makefile2:169: recipe for target 'CMakeFiles/af_xdp_bpf.dir/all' failed make[2]: *** [CMakeFiles/af_xdp_bpf.dir/all] Error 2 make[2]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' Makefile:151: recipe for target 'all' failed make[1]: *** [all] Error 2 make[1]: Leaving directory '/home/neo/bro-xdp_packet-plugin/build' Makefile:11: recipe for target 'build-it' failed make: *** [build-it] Error 2 Regards, Vinod -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190729/e2802392/attachment-0001.html From dnj0496 at gmail.com Mon Jul 29 18:54:21 2019 From: dnj0496 at gmail.com (Dk Jack) Date: Mon, 29 Jul 2019 18:54:21 -0700 Subject: [Zeek] http body q. Message-ID: Hi, I am trying to understand the behavior of bro with respect to logging http request when the http request has a large body. In my script, I am trying to log http body. I agree, http bodies can be large. However, I need the body for further parsing and analysis of traffic based on the content of the body content. To capture the body, I am setup events for http_entity_data and http_end_entity. In the 'http_entity_data' event, I am accumulating the body data into a request variable. In the end_entity event I am encoding body data using base64_encode (since body can include non printable characters). This seems to work fine for small bodies. However, for large bodies, I noticed that the log gets written without the body getting encoded. To debug, I added a log filter. In the log predicate call, I can see the http log writing happening before the end_entity even is called. Is this how it's supposed to work? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190729/9b10fc0f/attachment.html From dnj0496 at gmail.com Mon Jul 29 18:56:46 2019 From: dnj0496 at gmail.com (Dk Jack) Date: Mon, 29 Jul 2019 18:56:46 -0700 Subject: [Zeek] http body q. In-Reply-To: References: Message-ID: Sorry, hit send before I finished my message. Is there something I can do to ensure my end_entity event is invoked before http_log event is called? Any input is appreciated. Thanks. Dk. On Mon, Jul 29, 2019 at 6:54 PM Dk Jack wrote: > Hi, > I am trying to understand the behavior of bro with respect to logging http > request when the http request has a large body. > > In my script, I am trying to log http body. I agree, http bodies can be > large. However, I need the body for further parsing and analysis of traffic > based on the content of the body content. To capture the body, I am setup > events for http_entity_data and http_end_entity. In the 'http_entity_data' > event, I am accumulating the body data into a request variable. In the > end_entity event I am encoding body data using base64_encode (since body > can include non printable characters). > > This seems to work fine for small bodies. However, for large bodies, I > noticed that the log gets written without the body getting encoded. To > debug, I added a log filter. In the log predicate call, I can see the http > log writing happening before the end_entity even is called. > > Is this how it's supposed to work? > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190729/216ca818/attachment.html From mauro.palumbo at aizoon.it Tue Jul 30 01:01:02 2019 From: mauro.palumbo at aizoon.it (Palumbo Mauro) Date: Tue, 30 Jul 2019 08:01:02 +0000 Subject: [Zeek] known_* Message-ID: Hi everybody, there are a number of scripts (known_services, known_hosts, known_certs) which are implemented both using a broker store and sending broker events. It is possible to switch from one mode to the other using the option use_service_store. Is there any particular reason for this? Is one option more efficient than the other? Thanks, Mauro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190730/77d33c39/attachment.html From dopheide at gmail.com Tue Jul 30 09:37:53 2019 From: dopheide at gmail.com (Mike Dopheide) Date: Tue, 30 Jul 2019 11:37:53 -0500 Subject: [Zeek] known_* In-Reply-To: References: Message-ID: I think it can get kind of confusing with the naming of different functions. For instance, in bro_init() you'll see Cluster::create_store(), but later you'll see Broker::put_unique(). Those are both required bits for the data store to work and the Cluster* functions use Broker underneath. The variables like use_service_store toggle whether or not to use the data store method for persistent data or if you'd rather just use Cluster::publish_hrw. I imagine that's faster because it's doing less, then then you obviously lose the persistence. -Dop On Tue, Jul 30, 2019 at 3:09 AM Palumbo Mauro wrote: > Hi everybody, > > there are a number of scripts (known_services, known_hosts, > known_certs) which are implemented both using a broker store and sending > broker events. It is possible to switch from one mode to the other using > the option use_service_store. > > Is there any particular reason for this? Is one option more efficient than > the other? > > > > Thanks, > > Mauro > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190730/faa5cf6b/attachment.html From jsiwek at corelight.com Tue Jul 30 09:41:47 2019 From: jsiwek at corelight.com (Jon Siwek) Date: Tue, 30 Jul 2019 09:41:47 -0700 Subject: [Zeek] http body q. In-Reply-To: References: Message-ID: On Mon, Jul 29, 2019 at 6:59 PM Dk Jack wrote: >> This seems to work fine for small bodies. However, for large bodies, I noticed that the log gets written without the body getting encoded. For performance reasons, there's an option which throttles the maximum amount of data provided in entity_data events to 1500 bytes by default: https://docs.zeek.org/en/stable/scripts/base/init-bare.bro.html#id-http_entity_data_delivery_size So you can see if changing that `http_entity_data_delivery_size` option works for you. > Is there something I can do to ensure my end_entity event is invoked before http_log event is called? Any input is appreciated. Thanks. I think normally that should not be the order of things. Might be easier to explain what's going on if you provide an example script and pcap that reproduces what you are seeing. - Jon From jsiwek at corelight.com Tue Jul 30 09:55:21 2019 From: jsiwek at corelight.com (Jon Siwek) Date: Tue, 30 Jul 2019 09:55:21 -0700 Subject: [Zeek] known_* In-Reply-To: References: Message-ID: On Tue, Jul 30, 2019 at 1:09 AM Palumbo Mauro wrote: > there are a number of scripts (known_services, known_hosts, known_certs) which are implemented both using a broker store and sending broker events. It is possible to switch from one mode to the other using the option use_service_store. > > Is there any particular reason for this? Mostly due to not knowing which version better fits the common use-case. > Is one option more efficient than the other? No, there's two choices because there's a trade-off: using the Broker store version is theoretically less scalable because it relies on a single, centralized node, but the version using events partitions the data across many nodes in a consistent way via HRW. With the Broker store version you have the option of turning on persistence (e.g. saving data across restarts can prevent duplicate logs, etc.). With the version using events there's no option for persistence, although there's nothing preventing one from combining the two approaches except complexity of implementing it right: partitioned events + storing data inside a Broker store. - Jon From akgraner at corelight.com Tue Jul 30 12:51:33 2019 From: akgraner at corelight.com (Amber Graner) Date: Tue, 30 Jul 2019 14:51:33 -0500 Subject: [Zeek] 26 July 2019 - Zeek Leadership Team (LT) Meeting Minutes - Now Available Message-ID: Hi all, The LT Meeting minutes from the 26 July 2019 LT Meeting are now available at: https://blog.zeek.org/2019/07/open-source-zeek-leadership-team.html Please let me know if you have any questions. Thanks, ~Amber PS - Don't forget registration for ZeekWeek is still open - https://www.zeekweek.com -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190730/51581665/attachment.html From dnj0496 at gmail.com Tue Jul 30 16:04:09 2019 From: dnj0496 at gmail.com (Dk Jack) Date: Tue, 30 Jul 2019 16:04:09 -0700 Subject: [Zeek] http body q. In-Reply-To: References: Message-ID: Hi Jon, I've attached a minimal script that does what I am describing. However, when I try this minimal script on try.bro.org it works correctly as I expected. However, in my cluster setup, the end_entity event from response body is coming first, then log-filter call, followed by end entity call for request. Since I have other events I am processing for the request, not sure if they are affecting the order. Giving a priority of 10 for the end_entity event did not fix it either. Running the minimalist script I gave here also causes the problem on my system but not try.bro.org website. Really puzzling :( fyi, I've included the logs I see in my reporter log which indicates the order of the events at the end of this email Dk. PS: pcap link https://www.dropbox.com/s/epx2nz60d06uor9/long-user.pcap?dl=0 @load base/protocols/http/main module HTTP; export { redef record Info += { request_body: string &log &optional; request_body_complete: bool &log &default=F; response_body: string &log &optional; response_body_complete: bool &log &default=F; log_pred: bool &default=T; }; } event bro_init() { Log::remove_default_filter(HTTP::LOG); Log::add_filter(HTTP::LOG, [$name="new-default", $pred(rec: HTTP::Info) = { Reporter::info(fmt("request body complete: %s, bsize=%d" , rec$request_body_complete?"true":"false" , |rec$request_body|)); return rec$log_pred; } ]); } event http_entity_data(c: connection, is_orig: bool, length: count, data: string) { if (is_orig) { # request body accumulation if (is_orig && !c$http?$request_body) c$http$request_body = ""; c$http$request_body += data; } else { # response body accumulation. if (!is_orig && !c$http?$response_body) c$http$response_body = ""; c$http$response_body += data; } } event http_end_entity(c: connection, is_orig: bool) { Reporter::info(fmt("%s: end_entity called for %s, bsize=%d" , c$uid, is_orig?"request":"response", |c$http$request_body|)); if (is_orig) { if (|c$http$request_body| > 0) { c$http$request_body = encode_base64(c$http$request_body); c$http$request_body_complete = T; } } else { if (|c$http$response_body| > 0) { c$http$response_body = encode_base64(c$http$response_body); c$http$response_body_complete = T; } } } Logs order on my setup: -------------------------------- 1564527256.318960 Reporter::INFO CHhIoL3bq5EEA89mE6: end_entity called for response, resp_bsize=14182 /opt/bro/spool/installed-scripts-do-not-touch/site/large-body.bro, lines 51-52 1564527256.318960 Reporter::INFO log-filter: body complete: false, bsize=5494 /opt/bro/spool/installed-scripts-do-not-touch/site/large-body.bro, lines 21-23 1564527256.318960 Reporter::INFO CX7yqa4u6bAe5Jiyx8: end_entity called for request, req_bsize=5494 /opt/bro/spool/installed-scripts-do-not-touch/site/large-body.bro, lines 51-52 Logs order on try.bro.org: ---------------------------------- 1564517218.391577 ././trybro.bro, lines 51-52: CS3JwH1fT1mceM8uF5: end_entity called for request, req_bsize=42179 1564517218.391807 ././trybro.bro, lines 51-52: CS3JwH1fT1mceM8uF5: end_entity called for response, req_bsize=56240 1564517218.391807 ././trybro.bro, lines 21-23: log-filter body complete: true, bsize=56240 On Tue, Jul 30, 2019 at 9:41 AM Jon Siwek wrote: > On Mon, Jul 29, 2019 at 6:59 PM Dk Jack wrote: > > >> This seems to work fine for small bodies. However, for large bodies, I > noticed that the log gets written without the body getting encoded. > > For performance reasons, there's an option which throttles the maximum > amount of data provided in entity_data events to 1500 bytes by > default: > > > https://docs.zeek.org/en/stable/scripts/base/init-bare.bro.html#id-http_entity_data_delivery_size > > So you can see if changing that `http_entity_data_delivery_size` > option works for you. > > > Is there something I can do to ensure my end_entity event is invoked > before http_log event is called? Any input is appreciated. Thanks. > > I think normally that should not be the order of things. Might be > easier to explain what's going on if you provide an example script and > pcap that reproduces what you are seeing. > > - Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190730/8deb7142/attachment.html From jsiwek at corelight.com Tue Jul 30 17:46:04 2019 From: jsiwek at corelight.com (Jon Siwek) Date: Tue, 30 Jul 2019 17:46:04 -0700 Subject: [Zeek] http body q. In-Reply-To: References: Message-ID: On Tue, Jul 30, 2019 at 4:04 PM Dk Jack wrote: > However, in my cluster setup, the end_entity event from response body is coming first, then log-filter call, followed by end entity call for request. For the pcap you gave, it looks like that is actually the real order of the packets: the client is in the middle of sending the request body, but we see the server's OK response before the request is even finished (server also starts trying to reset connection at that point). So seems like it's weirdness to blame on that particular HTTP server? Or do you generally see this same pattern for other servers, too? - Jon From dnj0496 at gmail.com Tue Jul 30 18:37:36 2019 From: dnj0496 at gmail.com (Dk Jack) Date: Tue, 30 Jul 2019 18:37:36 -0700 Subject: [Zeek] http body q. In-Reply-To: References: Message-ID: Uhm! didn't notice that. Thanks for pointing out. Looks like my earlier experiment on try.bro.org was with another server. I am getting the same behavior now on my setup and try.bro.org setup with the two pcaps. On Tue, Jul 30, 2019 at 5:46 PM Jon Siwek wrote: > On Tue, Jul 30, 2019 at 4:04 PM Dk Jack wrote: > > > However, in my cluster setup, the end_entity event from response body is > coming first, then log-filter call, followed by end entity call for request. > > For the pcap you gave, it looks like that is actually the real order > of the packets: the client is in the middle of sending the request > body, but we see the server's OK response before the request is even > finished (server also starts trying to reset connection at that > point). So seems like it's weirdness to blame on that particular HTTP > server? Or do you generally see this same pattern for other servers, > too? > > - Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190730/70bc292a/attachment-0001.html From dnj0496 at gmail.com Wed Jul 31 18:43:01 2019 From: dnj0496 at gmail.com (Dk Jack) Date: Wed, 31 Jul 2019 18:43:01 -0700 Subject: [Zeek] gre capture filter Message-ID: Hi, I am trying to write a capture filter to filter GRE traffic based on the inside IP of a GRE packet. Based on the advice given in the link below: http://novalidhostsfound.blogspot.com/2015/03/how-to-filter-ip-addresses-inside-gre.html I wrote my capture filter (see at end of the email). With the capture filter, I am getting the following error: "Invalid capture_filter named 'inside_ip' - 'proto gre and (ip[50:4]=0xac1c0203 or ip[54:4]=0xac1c0203)'" when I use the same filter with tcpdump i.e. 'tcpdump -r