From akgraner at corelight.com Wed Apr 1 06:31:08 2020 From: akgraner at corelight.com (Amber Graner) Date: Wed, 1 Apr 2020 09:31:08 -0400 Subject: [Zeek] Interviews - Keith Lehigh of Indiana University and the Zeek Leadership Team Message-ID: Hi all, The latest People of Zeek Interview is with Keith Lehigh. Many of you know him from the Zeek Community and have probably met him at ZeekWeek or other Zeek events. In this interview you can get to know a little more about Keith and what excites him about Zeek. https://zeek.org/2020/03/30/people-of-zeek-interview-series-keith-lehigh-of-indiana-university-and-the-zeek-leadership-team/ Are you or someone you know doing great things with Zeek? We've got a lot more being queued up, but everyone has a story and I'd like to share yours! You can email me or news at zeek.org if you'd like to be included in the series or if there is someone you'd like me to reach out to. Thanks again, Keith for taking time to do the interview. And thanks in advance for all those I've already talked to about doing these. We're working through the list. With gratitude, ~Amber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200401/1e2d5bdd/attachment.html From akgraner at corelight.com Wed Apr 1 07:57:31 2020 From: akgraner at corelight.com (akgraner at corelight.com) Date: Wed, 01 Apr 2020 14:57:31 +0000 Subject: [Zeek] Canceled event: Reoccurring Zeek Community Call @ Fri Apr 3, 2020 3pm - 3:45pm (EDT) (zeek@zeek.org) Message-ID: <0000000000004fe5ca05a23be777@google.com> This event has been canceled. Title: Reoccurring Zeek Community Call We want to hear your suggestions and what you would like to see us focus on. Bring your questions and ideas.  We want to hear from you all. AGENDAZeekWeek 2020 - CancellationVirtual In-Person Different LocationZeek From Home - Webinar SeriesWhat is it?Submission CriteriaScheduleZeek Package Contest (ZPC-2)FocusTimelineLaunch Date(Public call w/anyone who wants to join)??????????Amber Graner is inviting you to a scheduled Zoom meeting.Join Zoom Meetinghttps://corelight.zoom.us/j/898658920Meeting ID: 898 658 920One tap mobile+16465588656,,898658920# US (New York)+16699006833,,898658920# US (San Jose)Dial by your location +1 646 558 8656 US (New York) +1 669 900 6833 US (San Jose) 877 853 5257 US Toll-free 888 475 4499 US Toll-freeMeeting ID: 898 658 920Find your local number: https://corelight.zoom.us/u/acY5L1LN7?????????? When: Fri Apr 3, 2020 3pm ? 3:45pm Eastern Time - New York Where: https://corelight.zoom.us/j/898658920 Calendar: zeek at zeek.org Who: * akgraner at corelight.com - organizer Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account zeek at zeek.org because you are an attendee of this event. To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar. Forwarding this invitation could allow any recipient to send a response to the organizer and be added to the guest list, or invite others regardless of their own invitation status, or to modify your RSVP. Learn more at https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200401/45ffc25c/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 2505 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200401/45ffc25c/attachment-0002.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 2563 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200401/45ffc25c/attachment-0003.bin From raubvogel at gmail.com Wed Apr 1 08:23:59 2020 From: raubvogel at gmail.com (Mauricio Tavares) Date: Wed, 1 Apr 2020 11:23:59 -0400 Subject: [Zeek] How to turn verbose on (Can't load packages)? In-Reply-To: References: Message-ID: On Tue, Mar 31, 2020 at 2:52 PM Jon Siwek wrote: > > On Tue, Mar 31, 2020 at 11:39 AM Mauricio Tavares wrote: > > > In that case, I think I am in trouble because I did use zkg > > What's the output of `zkg config` ? > > Did you previously run `zkg autoconfig` ? > > If you don't do any configuration, the default location for zkg to > install packages is in $HOME/.zkg rather than inside your Zeek install > prefix and could explain why your `@load packages` doesn't find > anything. > > - Jon Thanks for everyone's replies! Yes, I was indeed missing `zkg autoconfig` as you all guessed. After running I have [root at bro scratch]# zkg config [sources] zeek = https://github.com/zeek/packages [paths] state_dir = /root/.zkg script_dir = /usr/local/bro/share/bro/site plugin_dir = /usr/local/bro/lib/bro/plugins zeek_dist = /usr/local/bro [root at bro scratch]# I then installed the packages and then ran `broctl check` whose errors now at least indicate I have progress: [root at bro scratch]# broctl check Hint: Run the broctl "deploy" command to get started. manager scripts failed. error in /usr/local/bro/share/bro/site/packages/__load__.bro, line 3: Failed to open package '/usr/local/bro/share/bro/site/packages/./add-node-names': missing '__load__.bro' file fatal error in /usr/local/bro/share/bro/site/packages/__load__.bro, line 3: can't open /usr/local/bro/share/bro/site/packages/./add-node-names/__load__.bro [...] [root at bro scratch]# Now, if I look at __load__.bro [root at bro scratch]# cat /usr/local/bro/share/bro/site/packages/__load__.bro # WARNING: This file is managed by zkg. # Do not make direct modifications here. @load ./add-node-names @load ./bro-shellshock @load ./credit-card-exposure @load ./domain-tld @load ./file-extraction @load ./ssn-exposure @load ./top-dns @load ./venom @load ./zeek-cryptomining [root at bro scratch]# all is it doing is loading files off the /usr/local/bro/share/bro/site/packages/ dir [root at bro scratch]# ls -l /usr/local/bro/share/bro/site/packages/ total 8 -rw-r--r-- 1 root root 98 Mar 31 19:24 README lrwxrwxrwx 1 root root 13 Apr 1 11:56 __load__.bro -> packages.zeek lrwxrwxrwx 1 root root 13 Apr 1 11:56 __load__.zeek -> packages.zeek drwxr-xr-x 2 root root 54 Mar 31 19:28 add-node-names drwxr-xr-x 2 root root 95 Mar 31 19:27 bro-shellshock drwxr-xr-x 2 root root 62 Mar 31 19:28 credit-card-exposure drwxr-xr-x 2 root root 62 Mar 31 19:27 domain-tld drwxr-xr-x 3 root root 106 Mar 31 19:27 file-extraction lrwxrwxrwx 1 root root 13 Apr 1 11:56 packages.bro -> packages.zeek -rw-r--r-- 1 root root 276 Apr 1 11:56 packages.zeek drwxr-xr-x 2 root root 42 Mar 31 19:28 ssn-exposure drwxr-xr-x 2 root root 42 Mar 31 19:27 top-dns drwxr-xr-x 2 root root 60 Mar 31 19:27 venom drwxr-xr-x 3 root root 112 Mar 31 19:28 zeek-cryptomining [root at bro scratch]# I do not know how it is going from that to trying to open /usr/local/bro/share/bro/site/packages/./add-node-names/__load__.bro From tyrone at udel.edu Wed Apr 1 09:39:07 2020 From: tyrone at udel.edu (Tyrone Smith) Date: Wed, 1 Apr 2020 12:39:07 -0400 Subject: [Zeek] Method to detect tcp urgent flag Message-ID: Hi, First time posting. Links to documentation and RTM comments are welcome responses. Please be patient. Is there an easy method to trigger an event based on the TCP urgent flag? I am looking at Zeek::TCP tcp_packet but it is listed as a low level and noisy event. I'd like to setup something that leverages Zeek instead of a separate tcpdump. I'd like to use connection events but I do not see an easy way to detect if the urgent flag is set. If anyone has ideas/solutions better than tcp_packet or tcpdump, I'd love to get that feedback. Thanks. Tyrone Smith University of Delaware Security Operations -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200401/3abe3e07/attachment.html From justin at corelight.com Wed Apr 1 11:26:11 2020 From: justin at corelight.com (Justin Azoff) Date: Wed, 1 Apr 2020 14:26:11 -0400 Subject: [Zeek] How to turn verbose on (Can't load packages)? In-Reply-To: References: Message-ID: Ah, you're trying to install some packages that have been updated to work with zeek on an older bro installation. Can you upgrade to zeek 3.0.3 ? On Wed, Apr 1, 2020 at 11:24 AM Mauricio Tavares wrote: > > On Tue, Mar 31, 2020 at 2:52 PM Jon Siwek wrote: > > > > On Tue, Mar 31, 2020 at 11:39 AM Mauricio Tavares wrote: > > > > > In that case, I think I am in trouble because I did use zkg > > > > What's the output of `zkg config` ? > > > > Did you previously run `zkg autoconfig` ? > > > > If you don't do any configuration, the default location for zkg to > > install packages is in $HOME/.zkg rather than inside your Zeek install > > prefix and could explain why your `@load packages` doesn't find > > anything. > > > > - Jon > > Thanks for everyone's replies! Yes, I was indeed missing `zkg > autoconfig` as you all guessed. After running I have > > [root at bro scratch]# zkg config > [sources] > zeek = https://github.com/zeek/packages > > [paths] > state_dir = /root/.zkg > script_dir = /usr/local/bro/share/bro/site > plugin_dir = /usr/local/bro/lib/bro/plugins > zeek_dist = /usr/local/bro > > [root at bro scratch]# > > I then installed the packages and then ran `broctl check` whose errors > now at least indicate I have progress: > > [root at bro scratch]# broctl check > Hint: Run the broctl "deploy" command to get started. > manager scripts failed. > error in /usr/local/bro/share/bro/site/packages/__load__.bro, line 3: > Failed to open package > '/usr/local/bro/share/bro/site/packages/./add-node-names': missing > '__load__.bro' file > fatal error in /usr/local/bro/share/bro/site/packages/__load__.bro, > line 3: can't open > /usr/local/bro/share/bro/site/packages/./add-node-names/__load__.bro > [...] > [root at bro scratch]# > > Now, if I look at __load__.bro > > [root at bro scratch]# cat /usr/local/bro/share/bro/site/packages/__load__.bro > # WARNING: This file is managed by zkg. > # Do not make direct modifications here. > @load ./add-node-names > @load ./bro-shellshock > @load ./credit-card-exposure > @load ./domain-tld > @load ./file-extraction > @load ./ssn-exposure > @load ./top-dns > @load ./venom > @load ./zeek-cryptomining > [root at bro scratch]# > > all is it doing is loading files off the > /usr/local/bro/share/bro/site/packages/ dir > > [root at bro scratch]# ls -l /usr/local/bro/share/bro/site/packages/ > total 8 > -rw-r--r-- 1 root root 98 Mar 31 19:24 README > lrwxrwxrwx 1 root root 13 Apr 1 11:56 __load__.bro -> packages.zeek > lrwxrwxrwx 1 root root 13 Apr 1 11:56 __load__.zeek -> packages.zeek > drwxr-xr-x 2 root root 54 Mar 31 19:28 add-node-names > drwxr-xr-x 2 root root 95 Mar 31 19:27 bro-shellshock > drwxr-xr-x 2 root root 62 Mar 31 19:28 credit-card-exposure > drwxr-xr-x 2 root root 62 Mar 31 19:27 domain-tld > drwxr-xr-x 3 root root 106 Mar 31 19:27 file-extraction > lrwxrwxrwx 1 root root 13 Apr 1 11:56 packages.bro -> packages.zeek > -rw-r--r-- 1 root root 276 Apr 1 11:56 packages.zeek > drwxr-xr-x 2 root root 42 Mar 31 19:28 ssn-exposure > drwxr-xr-x 2 root root 42 Mar 31 19:27 top-dns > drwxr-xr-x 2 root root 60 Mar 31 19:27 venom > drwxr-xr-x 3 root root 112 Mar 31 19:28 zeek-cryptomining > [root at bro scratch]# > > I do not know how it is going from that to trying to open > /usr/local/bro/share/bro/site/packages/./add-node-names/__load__.bro -- Justin From raubvogel at gmail.com Wed Apr 1 11:55:17 2020 From: raubvogel at gmail.com (Mauricio Tavares) Date: Wed, 1 Apr 2020 14:55:17 -0400 Subject: [Zeek] How to turn verbose on (Can't load packages)? In-Reply-To: References: Message-ID: On Wed, Apr 1, 2020 at 2:26 PM Justin Azoff wrote: > > Ah, you're trying to install some packages that have been updated to > work with zeek on an older bro installation. Can you upgrade to zeek > 3.0.3 ? > Unfortunately not as of now. A few months later, maybe. Am I SOL or there is a way to get the older versions of said packages if that is what it takes? > On Wed, Apr 1, 2020 at 11:24 AM Mauricio Tavares wrote: > > > > On Tue, Mar 31, 2020 at 2:52 PM Jon Siwek wrote: > > > > > > On Tue, Mar 31, 2020 at 11:39 AM Mauricio Tavares wrote: > > > > > > > In that case, I think I am in trouble because I did use zkg > > > > > > What's the output of `zkg config` ? > > > > > > Did you previously run `zkg autoconfig` ? > > > > > > If you don't do any configuration, the default location for zkg to > > > install packages is in $HOME/.zkg rather than inside your Zeek install > > > prefix and could explain why your `@load packages` doesn't find > > > anything. > > > > > > - Jon > > > > Thanks for everyone's replies! Yes, I was indeed missing `zkg > > autoconfig` as you all guessed. After running I have > > > > [root at bro scratch]# zkg config > > [sources] > > zeek = https://github.com/zeek/packages > > > > [paths] > > state_dir = /root/.zkg > > script_dir = /usr/local/bro/share/bro/site > > plugin_dir = /usr/local/bro/lib/bro/plugins > > zeek_dist = /usr/local/bro > > > > [root at bro scratch]# > > > > I then installed the packages and then ran `broctl check` whose errors > > now at least indicate I have progress: > > > > [root at bro scratch]# broctl check > > Hint: Run the broctl "deploy" command to get started. > > manager scripts failed. > > error in /usr/local/bro/share/bro/site/packages/__load__.bro, line 3: > > Failed to open package > > '/usr/local/bro/share/bro/site/packages/./add-node-names': missing > > '__load__.bro' file > > fatal error in /usr/local/bro/share/bro/site/packages/__load__.bro, > > line 3: can't open > > /usr/local/bro/share/bro/site/packages/./add-node-names/__load__.bro > > [...] > > [root at bro scratch]# > > > > Now, if I look at __load__.bro > > > > [root at bro scratch]# cat /usr/local/bro/share/bro/site/packages/__load__.bro > > # WARNING: This file is managed by zkg. > > # Do not make direct modifications here. > > @load ./add-node-names > > @load ./bro-shellshock > > @load ./credit-card-exposure > > @load ./domain-tld > > @load ./file-extraction > > @load ./ssn-exposure > > @load ./top-dns > > @load ./venom > > @load ./zeek-cryptomining > > [root at bro scratch]# > > > > all is it doing is loading files off the > > /usr/local/bro/share/bro/site/packages/ dir > > > > [root at bro scratch]# ls -l /usr/local/bro/share/bro/site/packages/ > > total 8 > > -rw-r--r-- 1 root root 98 Mar 31 19:24 README > > lrwxrwxrwx 1 root root 13 Apr 1 11:56 __load__.bro -> packages.zeek > > lrwxrwxrwx 1 root root 13 Apr 1 11:56 __load__.zeek -> packages.zeek > > drwxr-xr-x 2 root root 54 Mar 31 19:28 add-node-names > > drwxr-xr-x 2 root root 95 Mar 31 19:27 bro-shellshock > > drwxr-xr-x 2 root root 62 Mar 31 19:28 credit-card-exposure > > drwxr-xr-x 2 root root 62 Mar 31 19:27 domain-tld > > drwxr-xr-x 3 root root 106 Mar 31 19:27 file-extraction > > lrwxrwxrwx 1 root root 13 Apr 1 11:56 packages.bro -> packages.zeek > > -rw-r--r-- 1 root root 276 Apr 1 11:56 packages.zeek > > drwxr-xr-x 2 root root 42 Mar 31 19:28 ssn-exposure > > drwxr-xr-x 2 root root 42 Mar 31 19:27 top-dns > > drwxr-xr-x 2 root root 60 Mar 31 19:27 venom > > drwxr-xr-x 3 root root 112 Mar 31 19:28 zeek-cryptomining > > [root at bro scratch]# > > > > I do not know how it is going from that to trying to open > > /usr/local/bro/share/bro/site/packages/./add-node-names/__load__.bro > > > > -- > Justin From jsiwek at corelight.com Wed Apr 1 12:33:01 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Wed, 1 Apr 2020 12:33:01 -0700 Subject: [Zeek] How to turn verbose on (Can't load packages)? In-Reply-To: References: Message-ID: On Wed, Apr 1, 2020 at 11:57 AM Mauricio Tavares wrote: > Unfortunately not as of now. A few months later, maybe. Am I SOL > or there is a way to get the older versions of said packages if that > is what it takes? You can try `zkg install --version ...` (see [1]) but I don't know what version of the particular package(s) in question may work. Alternatively, you can always fork, make the required changes and then install your version. - Jon [1] https://docs.zeek.org/projects/package-manager/en/stable/zkg.html#install From akgraner at corelight.com Wed Apr 1 12:46:40 2020 From: akgraner at corelight.com (Amber Graner) Date: Wed, 1 Apr 2020 15:46:40 -0400 Subject: [Zeek] Reminder - Monthly Community Call - 3 April - 3pm EST (noon PST) Message-ID: Hi all, Just a reminder that the Monthly Zeek Call will be this Friday 3 April at 3pm EST/Noon PST. Below is the agenda: We want to hear your suggestions and what you would like to see us focus on. Bring your questions and ideas. We want to hear from you all. ===AGENDA=== * ZeekWeek 2020 - Austin - Cancellation - Virtual - In-Person Different Location * Zeek From Home - Webinar Series - What is it? - Submission Criteria - Schedule * Zeek Package Contest (ZPC-2) - Focus - Timeline - Launch Date * Other Topics. ===Joining the Call=== Join Zoom Meeting https://corelight.zoom.us/j/898658920 Meeting ID: 898 658 920 One tap mobile +16465588656,,898658920# US (New York) +16699006833,,898658920# US (San Jose) Dial by your location +1 646 558 8656 <+1%20646%20558%208656> US (New York) +1 669 900 6833 <+1%20669%20900%206833> US (San Jose) 877 853 5257 US Toll-free 888 475 4499 US Toll-free Meeting ID: 898 658 920 Find your local number: https://corelight.zoom.us/u/acY5L1LN7 ===Opt In=== if you'd like to be added to the calendar reminder for the monthly calls you can opt-in at: https://www.surveymonkey.com/r/X5W5YQZ Please let me know if you have any questions or other topics you'd like to discuss. Thanks, ~Amber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200401/981d96f9/attachment.html From andrew at aklaus.ca Thu Apr 2 09:29:30 2020 From: andrew at aklaus.ca (Andrew Klaus) Date: Thu, 2 Apr 2020 10:29:30 -0600 Subject: [Zeek] Adding MySQL TLS Functionality Message-ID: Hello, I'm working on adding TLS support for MySQL in the Zeek master branch. Our mysql.log is filling up with a lot of encrypted TLS traffic which isn't overly helpful to us. Rather than ignore this log entirely, I'd like to just add TLS handshake functionality to the analyzer. I found that there' a `Client_Capabilities` enum type within the MySQL Analyzer that I've added the `CLIENT_SSL` bitmask to, which seems to compare against the right bits in the initial handshake. However, when I try to create a new TLSHandshake() function in MySQL.cc to call when this type of handshake is detected, similar to how the IMAP analyzer's StartTLS function, I can't quite get it to compile: Error: ------------------ /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc: In member function ?bool binpac::MySQL::MySQL_Flow::proc_mysql_handshake_response_packet(binpac::MySQL::Handshake_Response_Packet*)?: /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:35: error: ?class analyzer::Analyzer? has no member named ?TLSHandshake? connection()->bro_analyzer()->TLSHandshake(); ^~~~~~~~~~~~ src/analyzer/protocol/mysql/CMakeFiles/plugin-Zeek-MySQL.dir/build.make:190: recipe for target 'src/analyzer/protocol/mysql/CMakeFiles/plugin-Zeek-MySQL.dir/mysql_pac.cc.o' failed make[3]: *** [src/analyzer/protocol/mysql/CMakeFiles/plugin-Zeek-MySQL.dir/mysql_pac.cc.o] Error 1 ----------------- I know it's probably trivial, but I can't quite figure it out. From what I can tell, I've added the necessary function declaration to the MySQL.h header file, as well as the function to MySQL.cc. This is the branch I'm working off, with the diff of Zeek master here: https://github.com/zeek/zeek/compare/master...precurse:mysql-ssl Any help would be really appreciated. I haven't written an analyzer, so if there's anything that I could do better please let me know. Thanks! Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200402/5e08ad61/attachment-0001.html From jsiwek at corelight.com Thu Apr 2 09:59:09 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Thu, 2 Apr 2020 09:59:09 -0700 Subject: [Zeek] Adding MySQL TLS Functionality In-Reply-To: References: Message-ID: On Thu, Apr 2, 2020 at 9:33 AM Andrew Klaus wrote: > /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:35: error: ?class analyzer::Analyzer? has no member named ?TLSHandshake? > connection()->bro_analyzer()->TLSHandshake(); Seems like you just need to cast: static_cast(connection()->bro_analyzer())->TLSHandshake(); That is, bro_analyzer() returns an analyzer::Analyzer*, which is the base class that has no TLSHandshake() method, thus the compiler error. - Jon From andrew at aklaus.ca Thu Apr 2 11:11:45 2020 From: andrew at aklaus.ca (Andrew Klaus) Date: Thu, 2 Apr 2020 12:11:45 -0600 Subject: [Zeek] Adding MySQL TLS Functionality In-Reply-To: References: Message-ID: Thanks for the reply! I'm now getting this error: ------------------- /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc: In member function ?bool binpac::MySQL::MySQL_Flow::proc_mysql_handshake_response_packet(binpac::MySQL::Handshake_Response_Packet*)?: /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:27:* error: ?MySQL? in namespace ?analyzer? does not name a type* static_cast(connection()->bro_analyzer())->TLSHandshake(); ^~~~~ /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:32: error: expected ?>? before ?::? token static_cast(connection()->bro_analyzer())->TLSHandshake(); ^~ /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:32: error: expected ?(? before ?::? token /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:34: error: ?::MySQL_Analyzer? has not been declared static_cast(connection()->bro_analyzer())->TLSHandshake(); ^~~~~~~~~~~~~~ /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:34: note: suggested alternative: ?FlowAnalyzer? static_cast(connection()->bro_analyzer())->TLSHandshake(); ^~~~~~~~~~~~~~ FlowAnalyzer /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:81: error: ?class analyzer::Analyzer? has no member named ?TLSHandshake? static_cast(connection()->bro_analyzer())->TLSHandshake(); ^~~~~~~~~~~~ /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:95: error: expected ?)? before ?;? token static_cast(connection()->bro_analyzer())->TLSHandshake(); ^ src/analyzer/protocol/mysql/CMakeFiles/plugin-Zeek-MySQL.dir/build.make:190: recipe for target 'src/analyzer/protocol/mysql/CMakeFiles/plugin-Zeek-MySQL.dir/mysql_pac.cc.o' failed make[3]: *** [src/analyzer/protocol/mysql/CMakeFiles/plugin-Zeek-MySQL.dir/mysql_pac.cc.o] Error 1 -------------------------------------------- Some more differences I've noticed compared to the IMAP analyzer: imap.pac has "connection IMAP_Conn(bro_analyzer: IMAPAnalyzer)" https://github.com/zeek/zeek/blob/master/src/analyzer/protocol/imap/imap.pac#L27 mysql.pac has: "connection MySQL_Conn(bro_analyzer: BroAnalyzer)" https://github.com/zeek/zeek/blob/master/src/analyzer/protocol/mysql/mysql.pac#L20 also in imap.pac, but not mysql.pac: "namespace analyzer { namespace imap { class IMAP_Analyzer; } } typedef analyzer::imap::IMAP_Analyzer* IMAPAnalyzer; extern type IMAPAnalyzer;" I've tried porting these differences over as well, without much luck. I'll keep seeing what I can try to get this working. Thanks, Andrew On Thu, Apr 2, 2020 at 10:59 AM Jon Siwek wrote: > On Thu, Apr 2, 2020 at 9:33 AM Andrew Klaus wrote: > > > /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:35: > error: ?class analyzer::Analyzer? has no member named ?TLSHandshake? > > connection()->bro_analyzer()->TLSHandshake(); > > Seems like you just need to cast: > > > static_cast(connection()->bro_analyzer())->TLSHandshake(); > > That is, bro_analyzer() returns an analyzer::Analyzer*, which is the > base class that has no TLSHandshake() method, thus the compiler error. > > - Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200402/13a9f224/attachment.html From jsiwek at corelight.com Thu Apr 2 12:08:57 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Thu, 2 Apr 2020 12:08:57 -0700 Subject: [Zeek] Adding MySQL TLS Functionality In-Reply-To: References: Message-ID: On Thu, Apr 2, 2020 at 11:11 AM Andrew Klaus wrote: > /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc: In member function ?bool binpac::MySQL::MySQL_Flow::proc_mysql_handshake_response_packet(binpac::MySQL::Handshake_Response_Packet*)?: > /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:27: error: ?MySQL? in namespace ?analyzer? does not name a type > static_cast(connection()->bro_analyzer())->TLSHandshake(); > ^~~~~ You may need to shuffle some of the header includes around, see if the attached patch helps. I also had a typo in the example cast, it should have been casting to a pointer-type with '*', like: static_cast(connection()->bro_analyzer()); - Jon -------------- next part -------------- A non-text attachment was scrubbed... Name: mysql-analyzer.patch Type: application/octet-stream Size: 1114 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200402/01954aa5/attachment.obj From andrew at aklaus.ca Thu Apr 2 12:43:41 2020 From: andrew at aklaus.ca (Andrew Klaus) Date: Thu, 2 Apr 2020 13:43:41 -0600 Subject: [Zeek] Adding MySQL TLS Functionality In-Reply-To: References: Message-ID: That worked, thanks! It's compiling now. Andrew On Thu, Apr 2, 2020 at 1:09 PM Jon Siwek wrote: > On Thu, Apr 2, 2020 at 11:11 AM Andrew Klaus wrote: > > > /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc: In member > function ?bool > binpac::MySQL::MySQL_Flow::proc_mysql_handshake_response_packet(binpac::MySQL::Handshake_Response_Packet*)?: > > /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:27: > error: ?MySQL? in namespace ?analyzer? does not name a type > > > static_cast(connection()->bro_analyzer())->TLSHandshake(); > > ^~~~~ > > You may need to shuffle some of the header includes around, see if the > attached patch helps. > > I also had a typo in the example cast, it should have been casting to > a pointer-type with '*', like: > > > static_cast(connection()->bro_analyzer()); > > - Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200402/b12143f6/attachment.html From andrew at aklaus.ca Thu Apr 2 15:34:49 2020 From: andrew at aklaus.ca (Andrew Klaus) Date: Thu, 2 Apr 2020 16:34:49 -0600 Subject: [Zeek] Adding MySQL TLS Functionality In-Reply-To: References: Message-ID: So now that it's compiling, I'm able to work further on this. I was successful in generating ssl.log for SSL connections. However, I'm running into some logic issues where ALL connections are getting flagged as being SSL-enabled handshakes. I've narrowed it down to the logic where `msg.v10_response.client_ssl` is being set to true for both SSL-handshake and non-encrypted connections. This boolean is being generated from: client_ssl: bool = $context.connection.set_client_ssl(cap_flags & CLIENT_SSL); Looking at the MySQL documentation, I see that the CLIENT_DEPRECATE_EOF matches correctly: https://github.com/zeek/zeek/blob/master/src/analyzer/protocol/mysql/mysql-protocol.pac#L156 with the MySQL Documentation (0x01000000) ( https://dev.mysql.com/doc/internals/en/capability-flags.html#packet-Protocol::CapabilityFlags ) I've appended the CLIENT_SSL (Value 0x00000800) to the enum type, which is bitmasked against the Handshake_Response_Packet_v10.cap_flags (uint32) record. When I printf() the msg.v10_response.client_ssl here:: function proc_mysql_handshake_response_packet(msg: Handshake_Response_Packet): bool %{ ... if ( mysql_handshake ) { if ( ${msg.version} == 10 && ${msg.v10_response.client_ssl}) { fprintf(stderr, "%u\n",${msg.v10_response.cap_flags}); ... I'm seeing the value of 4026597376 set for this variable when I use the zeek cli to parse my MySQL SSL-handshake PCAP. The value that is being returned for this field between Zeek and Wireshark are very different. This is for the same connection: Zeek returns: 4026597376 (1111 0000 0000 0001 0000 0000 0000 0000) Wireshark shows: (Client Capabilities section) 1010 1010 10000 1101 (Extended capabilities section) 0000 0001 1011 1111 I _think_ I'm comparing what should be the same fields. Let me know any of these steps don't seem right. Thanks for the help! Andrew On Thu, Apr 2, 2020 at 1:09 PM Jon Siwek wrote: > On Thu, Apr 2, 2020 at 11:11 AM Andrew Klaus wrote: > > > /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc: In member > function ?bool > binpac::MySQL::MySQL_Flow::proc_mysql_handshake_response_packet(binpac::MySQL::Handshake_Response_Packet*)?: > > /home/zeek/build/src/analyzer/protocol/mysql/mysql_pac.cc:3042:27: > error: ?MySQL? in namespace ?analyzer? does not name a type > > > static_cast(connection()->bro_analyzer())->TLSHandshake(); > > ^~~~~ > > You may need to shuffle some of the header includes around, see if the > attached patch helps. > > I also had a typo in the example cast, it should have been casting to > a pointer-type with '*', like: > > > static_cast(connection()->bro_analyzer()); > > - Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200402/7ac99074/attachment-0001.html From Garcia_G2 at outlook.es Fri Apr 3 03:51:06 2020 From: Garcia_G2 at outlook.es (Alejandro Garcia) Date: Fri, 3 Apr 2020 10:51:06 +0000 Subject: [Zeek] High Chargue in Cpu with 1,5GB/s Message-ID: Hi everyone, We have been having troubles with Zeek. I hope that you can help me to solve this. We have 4 machines with this Hardware: SFP+, SR, transceptor ?ptico, Intel,10 Gb-1 Gb, PowerEdge R340 Server [PowerEdge R340 - Full Configuration - [EMEA_R340_VI_VP]] PowerEdge R340 Motherboard Intel Xeon E-2136 3.3GHz, 12M cache, 6Cores/12Threads, turbo (80W) 555-BCKN Adaptador PCIe Intel X710 two interfaces 10 GbE SFP+ 64GB 2666MT/s DDR4 ECC UDIMM This machines are running just the Zeek processes and a Filebeat to send the logs to our SIEM. The thing is that we need to process a maximun of 4-5 GB/s per machine. Now we are just processing 1,5GB/s and we have all the cores at 70% of charge, which we think that is too much for this amount of traffic. Our workers config in the node.cfg looks like this: [Zeek-1-W-1] type=worker host=localhost interface=p1p1 lb_method=pf_ring lb_procs=5 pin_cpus=0,1,2,3,4 [Zeek-1-W-2] type=worker host=localhost interface=p1p2 lb_method=pf_ring lb_procs=5 pin_cpus=5,6,7,8,9 We have tested with different number of RSS queues with no success. Now we are using PF_RING but we have tested with AF_Packet plugin with the same results. Also we have tested different driver updates for our network card with the same results: i40e-2.11.25_sourceforge i40e-2.10.19.82_intel We also have tried with diferent versions of Zeek 3.0.1 and 3.1.1 with the same results. Hope you can help me to improve the performance of our machines, Thank you. Best Regards -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200403/814da821/attachment.html From jgarciar at sia.es Fri Apr 3 03:57:49 2020 From: jgarciar at sia.es (Jorge Garcia Rodriguez) Date: Fri, 3 Apr 2020 10:57:49 +0000 Subject: [Zeek] Error with filters Message-ID: <0d3234ebb2a84c39bddc3b296015abfc@sia.es> Hi guys. We have recently updated the versi?n of our Zeeks to the last stable 3.1.1 from 3.0.1 My problem is that in the previous version we had a filter that worked perfectly and now in the new version it doesn't work anymore. The filter is in the end of the local.zeek file and is the next one: redef restrict_filters += { ["hosts"] = " not host 172.22.96.200" }; I don't know if it is a bug or we have to write the filter differently but it doesn't work in the new version. Thank you, Regards. Jorge Garc?a Rodr?guez Technical Consultant Security Infrastructures jgarciar at sia.es Grupo SIA Avda.Europa,2 - Alcor Plaza, Edificio B - Parque Oeste Alcorc?n 28922 Alcorc?n - Madrid Tlf: +34 902 480 580 Fax: +34 91 307 79 80 www.siainternational.com delivering value This e-mail and any attached files are intended solely for the addresse/s identified herein. It may contain confidential and/or legally privileged information and may not necessarily represent the opinion of SIA. No legally binding commitments will be created by this E-mail message. Where we intend to create legally binding commitments these will be made through hard copy correspondence or documents. If you receive this message by mistake, please immediately notify the sender and delete it since you are not authorized to use, disclose, distribute, print or copy all or part of the contained information Thank you. It is understood that the message was sent to you accidentally, although you appear as the addressee, you can see from the frame of existing relations that you were not the final addressee. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200403/f5d954f1/attachment-0001.html From akgraner at corelight.com Fri Apr 3 09:02:47 2020 From: akgraner at corelight.com (Amber Graner) Date: Fri, 3 Apr 2020 12:02:47 -0400 Subject: [Zeek] New Blog Post - The New IO Loop in Zeek 3.1 Message-ID: Happy Friday Below is the link to a new blog post about The New IO Loop in Zeek 3.1 by Tim Wojtulewicz - https://zeek.org/2020/04/03/the-new-io-loop-in-zeek-3-1/ Thanks, ~Amber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200403/3a52192a/attachment.html From tim at corelight.com Fri Apr 3 16:37:35 2020 From: tim at corelight.com (Tim Wojtulewicz) Date: Fri, 3 Apr 2020 16:37:35 -0700 Subject: [Zeek] Error with filters In-Reply-To: <0d3234ebb2a84c39bddc3b296015abfc@sia.es> References: <0d3234ebb2a84c39bddc3b296015abfc@sia.es> Message-ID: <808EE973-01B9-45C2-BF6C-C93E35C13867@corelight.com> Is zeek returning an error or is it just not filtering? Is the filter listed correctly in packet_filter.log? I tested the same filter redef here with a test file and it appeared to be working. Tim > On Apr 3, 2020, at 3:57 AM, Jorge Garcia Rodriguez wrote: > > Hi guys. > > We have recently updated the versi?n of our Zeeks to the last stable 3.1.1 from 3.0.1 > > My problem is that in the previous version we had a filter that worked perfectly and now in the new version it doesn?t work anymore. > > The filter is in the end of the local.zeek file and is the next one: > > redef restrict_filters += { ["hosts"] = " not host 172.22.96.200" }; > > I don?t know if it is a bug or we have to write the filter differently but it doesn?t work in the new version. > > Thank you, > > Regards. > > > > Jorge Garc?a Rodr?guez > Technical Consultant > Security Infrastructures > jgarciar at sia.es > Grupo SIA > Avda.Europa,2 - Alcor Plaza, Edificio B - Parque Oeste Alcorc?n > 28922 Alcorc?n - Madrid > Tlf: +34 902 480 580 Fax: +34 91 307 79 80 > www.siainternational.com > delivering value > This e-mail and any attached files are intended solely for the addresse/s identified herein. It may contain confidential and/or legally privileged information and may not necessarily represent the opinion of SIA. > No legally binding commitments will be created by this E-mail message. Where we intend to create legally binding commitments these will be made through hard copy correspondence or documents. If you receive this message by mistake, please immediately notify the sender and delete it since you are not authorized to use, disclose, distribute, print or copy all or part of the contained information Thank you. It is understood that the message was sent to you accidentally, although you appear as the addressee, you can see from the frame of existing relations that you were not the final addressee. > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200403/badd8bb4/attachment.html From nabilmemon.ec at gmail.com Sat Apr 4 21:51:34 2020 From: nabilmemon.ec at gmail.com (Nabil Memon) Date: Sun, 5 Apr 2020 10:21:34 +0530 Subject: [Zeek] Regarding udp_content event Message-ID: Hi, I am going through UDP events bro generates. I have a one use case, for example, UDP transaction is initiated by a control point to auto discover devices present in the network. Control point broadcasts, UDP request for lets say, IP1(192.168.1.1)->Broadcast(192.168.1.255)->SP(54632->DP(3702)->UDP and the contents exchanged in the request. Devices who receives those broadcast requests, responds with unicast message saying I am the one you are looking for, IP2(192.168.1.2)->IP1(192.168.1.1)->DP(3702)->SP(54632)->UDP and the contents exchanged in the response. There will be two different connections for bro because of two different 5 tuples. What I wish to extract is the raw content exchanged over both request and response packet. I came across udp_content event. After looking in the implementation, configuration supports only destination ports to be checked. But for response packet, bro has maintained a different connection in which source port is the one I am interested in. Flow also will not be flipped by bro because the port is not added in *"likely_server_ports"*. Also I tried adding port 3702 in *"likely_server_ports" *list, bro still did not raise udp_content event. Is there any way I can extract UDP contents from both request(no problem extracting request content) and response without adding ports in " *likely_server_ports" *list?? Or Even when I am adding those ports in the list, I don't get the event. *Here's how the configuration looks like,* const udp_content_delivery_ports_orig: table[port] of bool = {[3702/udp] = T} &redef; const udp_content_delivery_ports_resp: table[port] of bool = {[3702/udp] = T} &redef; const udp_content_deliver_all_orig = F &redef; const udp_content_deliver_all_resp = F &redef; *test.bro* const ports = {37020/udp, 1900/udp, 3702/udp}; redef likely_server_ports += { ports }; event udp_contents(c: connection, is_orig: bool, contents: string) { print "-----------------------------------------------------"; print "Contents:", c$id, is_orig, |contents|, contents; print "-----------------------------------------------------"; } Output: ----------------------------------------------------- Contents:, [orig_h=10.113.14.94, orig_p=50818/udp, resp_h=239.255.255.250, resp_p=3702/udp], T, 488, ----------------------------------------------------- Bro did not raise an event for response packet. *When I change configuration to,* const udp_content_deliver_all_orig = T &redef; const udp_content_deliver_all_resp = T &redef; *Output:* ----------------------------------------------------- Contents:, [orig_h=10.113.14.94, orig_p=50818/udp, resp_h=239.255.255.250, resp_p=3702/udp], T, 488, ----------------------------------------------------- ----------------------------------------------------- Contents:, [orig_h=10.113.14.94, orig_p=50818/udp, resp_h=10.113.14.197, resp_p=3702/udp], F, 3289, ----------------------------------------------------- I got event for both request and response. What am I doing wrong? Thanks and Regards, Nabil Phone: +91 81477 17034 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200405/897c7929/attachment.html From jgarciar at sia.es Mon Apr 6 02:20:27 2020 From: jgarciar at sia.es (Jorge Garcia Rodriguez) Date: Mon, 6 Apr 2020 09:20:27 +0000 Subject: [Zeek] Error with filters Message-ID: <7dbb84e10a324229abf2eb7cb74ddcfa@sia.es> Hi! Zeek doesn?t return an error at the start. It just not filtering. If I run de command ?zeekctl print restrict_filters? I get this output for every workers: Zeek-1-W-1-1 restrict_filters = { [hosts] = not host 172.22.96.200 } But the log ?packet_filter.log? is not being recorded. However in the machines that the filter works correctly, this log is being recorded. Thank you for your reply. Regards. Jorge Garc?a Rodr?guez Technical Consultant Security Infrastructures jgarciar at sia.es Grupo SIA Avda.Europa,2 - Alcor Plaza, Edificio B - Parque Oeste Alcorc?n 28922 Alcorc?n - Madrid Tlf: +34 902 480 580 Fax: +34 91 307 79 80 www.siainternational.com delivering value This e-mail and any attached files are intended solely for the addresse/s identified herein. It may contain confidential and/or legally privileged information and may not necessarily represent the opinion of SIA. No legally binding commitments will be created by this E-mail message. Where we intend to create legally binding commitments these will be made through hard copy correspondence or documents. If you receive this message by mistake, please immediately notify the sender and delete it since you are not authorized to use, disclose, distribute, print or copy all or part of the contained information Thank you. It is understood that the message was sent to you accidentally, although you appear as the addressee, you can see from the frame of existing relations that you were not the final addressee. De: Tim Wojtulewicz [mailto:tim at corelight.com] Enviado el: s?bado, 4 de abril de 2020 1:38 Para: Jorge Garcia Rodriguez CC: zeek at zeek.org Asunto: Re: [Zeek] Error with filters Is zeek returning an error or is it just not filtering? Is the filter listed correctly in packet_filter.log? I tested the same filter redef here with a test file and it appeared to be working. Tim On Apr 3, 2020, at 3:57 AM, Jorge Garcia Rodriguez > wrote: Hi guys. We have recently updated the versi?n of our Zeeks to the last stable 3.1.1 from 3.0.1 My problem is that in the previous version we had a filter that worked perfectly and now in the new version it doesn?t work anymore. The filter is in the end of the local.zeek file and is the next one: redef restrict_filters += { ["hosts"] = " not host 172.22.96.200" }; I don?t know if it is a bug or we have to write the filter differently but it doesn?t work in the new version. Thank you, Regards. Jorge Garc?a Rodr?guez Technical Consultant Security Infrastructures jgarciar at sia.es Grupo SIA Avda.Europa,2 - Alcor Plaza, Edificio B - Parque Oeste Alcorc?n 28922 Alcorc?n - Madrid Tlf: +34 902 480 580 Fax: +34 91 307 79 80 www.siainternational.com delivering value This e-mail and any attached files are intended solely for the addresse/s identified herein. It may contain confidential and/or legally privileged information and may not necessarily represent the opinion of SIA. No legally binding commitments will be created by this E-mail message. Where we intend to create legally binding commitments these will be made through hard copy correspondence or documents. If you receive this message by mistake, please immediately notify the sender and delete it since you are not authorized to use, disclose, distribute, print or copy all or part of the contained information Thank you. It is understood that the message was sent to you accidentally, although you appear as the addressee, you can see from the frame of existing relations that you were not the final addressee. _______________________________________________ Zeek mailing list zeek at zeek.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200406/b3ce1a69/attachment-0001.html From raubvogel at gmail.com Mon Apr 6 06:47:58 2020 From: raubvogel at gmail.com (Mauricio Tavares) Date: Mon, 6 Apr 2020 09:47:58 -0400 Subject: [Zeek] Finding out what is making zeekctl/broctl deploy hang up hang up Message-ID: So I start it broctl deploy checking configurations ... installing ... creating policy directories ... installing site policies ... generating cluster-layout.bro ... generating local-networks.bro ... generating broctl-config.bro ... generating broctl-config.sh ... stopping ... bro-bond0-1 not running bro-bond0-2 not running bro-bond0-3 not running bro-bond0-4 not running bro-bond0-5 not running proxy not running manager not running starting ... starting manager ... starting proxy ... starting bro-bond0-1 ... starting bro-bond0-2 ... starting bro-bond0-3 ... starting bro-bond0-4 ... starting bro-bond0-5 ... and it has been there for a while. Is there a way to see what it is doing? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200406/c7fa0c8c/attachment.html From akgraner at corelight.com Mon Apr 6 12:37:03 2020 From: akgraner at corelight.com (Amber Graner) Date: Mon, 6 Apr 2020 15:37:03 -0400 Subject: [Zeek] 2019 Zeek Package Contest Summary and Winners (ZPC-1) Message-ID: Happy Monday!! In case you weren't at ZeekWeek last year, here's the list of winning submissions and a summary of each Package contributed to the first #Zeek Package Contest (ZPC-1) Many thanks to all those who made it a success! - https://zeek.org/2020/04/06/2019-zeek-package-contest-summary-and-winners-zpc-1/ Thanks, ~Amber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200406/daba45f1/attachment.html From akgraner at corelight.com Mon Apr 6 13:30:29 2020 From: akgraner at corelight.com (Amber Graner) Date: Mon, 6 Apr 2020 16:30:29 -0400 Subject: [Zeek] Announcing - A New Zeek Package Contest - ZPC-2 Message-ID: More exciting news!!! Announcing a new Zeek Package Contest (ZPC-2). This contest will focus on the MITRE ATT&CK? Framework, more specifically packages that help detect C2 Techniques. Hope you'll take a look at the details (link below) and participate. https://zeek.org/2020/04/06/zeek-package-contest-zpc-2/ If you have any questions can email contest at zeek.org or join #packages on slack . Thanks, ~Amber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200406/dba14495/attachment.html From jsiwek at corelight.com Mon Apr 6 13:43:57 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Mon, 6 Apr 2020 13:43:57 -0700 Subject: [Zeek] Long running connection using threshold In-Reply-To: <38442acf-7430-2e23-0528-e85a098437b7@etnetera.cz> References: <38442acf-7430-2e23-0528-e85a098437b7@etnetera.cz> Message-ID: Hi Petr, Your example code looked correct to me, but I found what simply looked like a bug in the connection thresholding code that did the duration comparison in reverse of what it should. Here's my proposed patch: https://github.com/zeek/zeek/pull/899 - Jon On Tue, Mar 31, 2020 at 12:25 PM Petr Medonos wrote: > > Hi, > I tried to write simple script to detect long running connection using > zeek (3.0) threshold. I set duration in connection established event and > then using duration_threshold_crossed logged connection above the limit. > But Notice log is then flooded with every new established connection. > Simple PoC bellow. Did I missed something? Is there any better way to > detect long running connection? I tried Corelight bro-long-connections > but there is lot overhead in my environment. Thanks for pointing me the > right way! > > > -- > Petr > > > PoC: > > @load base/protocols/conn > > module LongConnection; > > export { > redef enum Log::ID += { LOG }; > > redef enum Notice::Type += { > LongConnection::found > }; > > const duration: interval = 12hr &redef; > } > > event connection_established(c: connection) > { > ConnThreshold::set_duration_threshold(c, duration); > } > > event ConnThreshold::duration_threshold_crossed(c: connection, > threshold: interval, is_orig: bool) > { > local message = fmt("%s:%s -> %s:%s remained alive for longer > than %s", c$id$orig_h, c$id$orig_p, c$id$resp_h, c$id$resp_p, threshold); > > NOTICE([$note=LongConnection::found, > $msg=message, > $sub=fmt("%.2f", threshold), > $conn=c]); > > } > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek From jsiwek at corelight.com Mon Apr 6 15:05:37 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Mon, 6 Apr 2020 15:05:37 -0700 Subject: [Zeek] Regarding udp_content event In-Reply-To: References: Message-ID: On Sat, Apr 4, 2020 at 9:55 PM Nabil Memon wrote: > Is there any way I can extract UDP contents from both request(no problem extracting request content) and response without adding ports in "likely_server_ports" list?? Think modifying "likely_server_ports" is the right approach here. > Even when I am adding those ports in the list, I don't get the event. Yeah, that looks like a bit of a deficiency in how UDP contents generally works for those "content delivery ports" tables: it's just tracking the exact "destination port" per UDP packet, so I'm suggesting to add an additional option to instead track according to the Connection's "responder" port. That will also correctly track any role flipping that occurs from the "likely server ports" logic. The PR for this is here: https://github.com/zeek/zeek/pull/900 - Jon From nabilmemon.ec at gmail.com Tue Apr 7 02:11:47 2020 From: nabilmemon.ec at gmail.com (Nabil Memon) Date: Tue, 7 Apr 2020 14:41:47 +0530 Subject: [Zeek] Regarding udp_content event In-Reply-To: References: Message-ID: Awesome, thanks! On Tue, Apr 7, 2020 at 3:35 AM Jon Siwek wrote: > On Sat, Apr 4, 2020 at 9:55 PM Nabil Memon > wrote: > > > Is there any way I can extract UDP contents from both request(no problem > extracting request content) and response without adding ports in > "likely_server_ports" list?? > > Think modifying "likely_server_ports" is the right approach here. > > > Even when I am adding those ports in the list, I don't get the event. > > Yeah, that looks like a bit of a deficiency in how UDP contents > generally works for those "content delivery ports" tables: it's just > tracking the exact "destination port" per UDP packet, so I'm > suggesting to add an additional option to instead track according to > the Connection's "responder" port. That will also correctly track any > role flipping that occurs from the "likely server ports" logic. The > PR for this is here: > > https://github.com/zeek/zeek/pull/900 > > - Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200407/9f259b24/attachment.html From nabilmemon.ec at gmail.com Tue Apr 7 02:43:35 2020 From: nabilmemon.ec at gmail.com (Nabil Memon) Date: Tue, 7 Apr 2020 15:13:35 +0530 Subject: [Zeek] Regarding udp_content event In-Reply-To: References: Message-ID: Hi Jon, Instead configuring zeek to say these are likely to be server ports. What would happen if we introduce a check for source port as well with the destination port? Did you consider this approach? Thanks and Regards, Nabil Phone: +91 81477 17034 On Tue, Apr 7, 2020 at 2:41 PM Nabil Memon wrote: > Awesome, thanks! > > On Tue, Apr 7, 2020 at 3:35 AM Jon Siwek wrote: > >> On Sat, Apr 4, 2020 at 9:55 PM Nabil Memon >> wrote: >> >> > Is there any way I can extract UDP contents from both request(no >> problem extracting request content) and response without adding ports in >> "likely_server_ports" list?? >> >> Think modifying "likely_server_ports" is the right approach here. >> >> > Even when I am adding those ports in the list, I don't get the event. >> >> Yeah, that looks like a bit of a deficiency in how UDP contents >> generally works for those "content delivery ports" tables: it's just >> tracking the exact "destination port" per UDP packet, so I'm >> suggesting to add an additional option to instead track according to >> the Connection's "responder" port. That will also correctly track any >> role flipping that occurs from the "likely server ports" logic. The >> PR for this is here: >> >> https://github.com/zeek/zeek/pull/900 >> >> - Jon >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200407/b7472bbf/attachment.html From jsiwek at corelight.com Tue Apr 7 13:10:38 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Tue, 7 Apr 2020 13:10:38 -0700 Subject: [Zeek] Regarding udp_content event In-Reply-To: References: Message-ID: On Tue, Apr 7, 2020 at 2:43 AM Nabil Memon wrote: > Instead configuring zeek to say these are likely to be server ports. > What would happen if we introduce a check for source port as well with the destination port? > Did you consider this approach? Yeah, that's an alternate idea that would work. I added such an option, called "udp_content_ports", to the Pull Request if you find it more convenient, although configuring likely server ports may still generally be useful if you commonly find inspected-traffic where the originator/responder roles would better to have been flipped to reflect a known-server. - Jon From akgraner at corelight.com Tue Apr 7 13:28:47 2020 From: akgraner at corelight.com (Amber Graner) Date: Tue, 7 Apr 2020 16:28:47 -0400 Subject: [Zeek] =?utf-8?q?Zeek_Monthly_Newsletter_=E2=80=93_Issue_3_?= =?utf-8?q?=E2=80=93_April_2020?= Message-ID: Below is Issue 3 of the Zeek Monthly Newsletter. You can also find it at: https://zeek.org/2020/04/07/zeek-monthly-newsletter-issue-3-april-2020/ ==Issue 3 - April 2020== Welcome to the Zeek Monthly Newsletter, Issue 3 covers March 2020 as well as upcoming events. ===In this Issue:=== * General Community News/Updates * Development Updates * Zeek in the News * Zeek In the Community * Interviews * Threat of the Month * Upcoming Events * New Zeek Related Packages * Publication Schedule * Get Involved ===General Community News/Updates=== * New Zeek Package Contest Announced - ZPC-2 - The ZPC contest series is intended to inspire Zeek users to demonstrate their creativity and ingenuity while winning the admiration of their peers, and giving back to the community. The ZPC-2 contest will focus on the MITRE ATT&CK? Framework, more specifically packages that help detect C2 Techniques. Find out more about how you can participate in ZPC-2 at: https://zeek.org/2020/04/06/zeek-package-contest-zpc-2/ * Zeek From Home - Weekly Webinar Series - If you have a Zeek Related talk (even one that you?ve given at past Zeek events) submit those today and let?s get you scheduled for a Zeek From Home presentation. Find out more at: https://zeek.org/2020/03/31/zeek-from-home/ * Zeek Slack Workspace Announced - This post will give you more information about the Slack Space and how you can join. https://zeek.org/2020/03/04/zeek-slack-channel-announced/ * New Zeek Website announced - We hope you?ve had a chance to look around the new site. This post tells you more about the site and the meaning of the new Zeek Logo - https://zeek.org/2020/03/11/announcing-the-new-zeek-website/ * ZeekWeek 2020 Austin ? Cancelled ? Open Letter to the Community - Given the uncertainty, we?ve made the difficult decision to cancel ZeekWeek 2020 in Austin. Rest assured that we are looking at other options to bring the community together as things improve and become more predictable. Those options include a virtual event during the same time frame, and if it?s safe to bring people together, then we will look at holding a smaller event in a different location. However, we won?t know until we get closer to October. You can read morte about this at: https://zeek.org/2020/03/31/zeekweek-2020-austin-cancelled-open-letter-to-the-community/ ===Development Updates=== * Announcing the New Zeek Agent - an open source endpoint agent that turns host activity into Zeek events as it happens. You can find out more about the Zeek Agent in the blog post at: https://zeek.org/2020/03/11/announcing-the-new-zeek-website/ and on the Zeek Mailing list at: http://mailman.icsi.berkeley.edu/pipermail/zeek/2020-March/015187.html ===Zeek in the News=== * Zeek and Jitsi: 2 open source projects we need now - Long proven but not well known, these network security monitoring and video conferencing tools couldn?t be more timely says Matt Asay. You can find out more at: https://www.infoworld.com/article/3533999/zeek-and-jitsi-2-open-source-projects-we-need-now.html * Researchers identify novel cybersecurity approach to protect Army systems - From this post, "Our approach uses symbolic execution to explore the state of TCP implementation of an endhost to identify ways to reach critical points in the code," Chan said. "If such a point is found, then packets can be inserted and be undetected by DPI. This method is evaluated against several state-of-the-art DPI systems such as Zeek and Snort and identifies previously known evasion strategies in addition to new ones that were not previously documented." You can find out more at: https://techxplore.com/news/2020-03-cybersecurity-approach-army.html ===Zeek in the Community=== * Security Onion 16.04.6.5 ISO image now available featuring Zeek 3.0.3, Suricata 4.1.7, Elastic 6.8.7, CyberChef 9.18.2, and more! - https://blog.securityonion.net/2020/03/security-onion-160465-iso-image-now.html * Brim Security - Desktop App - open sourced - In a tweet, Brim Security announced, ?We've open sourced our desktop application Brim! It lets you easily work with huge pcaps: it uses Zeek to generate logs you can search with intuitive queries, and then lets you extract just the interesting packets into Wireshark.? You can find out more at: https://www.brimsecurity.com/download/ * Getting Network Visibility into East-West Traffic by Bricata- https://securityboulevard.com/2020/03/getting-network-visibility-into-east-west-traffic/ ===Interviews=== * Doug Burks of Security Onion - https://zeek.org/2020/03/25/people-of-zeek-interview-series-doug-burks-of-security-onion/ * Keith Lehigh of Indiana University and the Zeek Leadership Team - https://zeek.org/2020/03/30/people-of-zeek-interview-series-keith-lehigh-of-indiana-university-and-the-zeek-leadership-team/ ===Threat of the Month=== Do you have a threat you?d like to share with the community and how using Zeek in your security stack helped you identify that threat? Please email news at zeek.org and we?ll work with you to get it written up and shared in the next newsletter. ===Upcoming Events=== ====Ask the Zeeksperts==== Ask the Zeeksperts is a one hour bi-weekly call that is hosted by various ?Zeeksperts? in the community. This is where you can drop by and ask your Zeek Related questions. The webinars are free to attend, but registration is required. * 9 April 2020 - 12:30pm PST/3:30pm EST - https://attendee.gotowebinar.com/register/2632319203581363981 * 23 April 2020 - 12:30pm PST/3:30pm EST - https://attendee.gotowebinar.com/register/1763308093940786957 ====Zeek From Home==== This is a new weekly webinar series, where the community can share their Zeek Related presentations (scripts, use cases, how to?s, unique usages, lessons learned etc). These will be recorded. * 15 April 2020 - 2pm EST/11am PST (registration details will be announced on the Zeek Mailing list, Twitter, Slack and the website) ====Virtual CTF - Hunt From Home==== Corelight Virtual Hunt from Home - A free, 2-hour Virtual Capture the Flag event hosted by Corelight, where players compete to answer security challenges using Zeek data in Splunk and Elastic. The security challenges model realistic IR and hunting queries and can help you uplevel your Zeek log proficiency. Corelight experts will be on hand during the game to guide players of all skill levels through two exciting hunt scenarios. Sign up for one of eight virtual CTF spots in April. Game winners will take home bragging rights and a $100 Amazon Gift Card. https://www3.corelight.com/ctf/hunt-from-home If you know of any Zeek related events that you would like to share with the community in the monthly newsletter, please email news at zeek.org or share on the Zeek mailing list (zeek at zeek.org). ===Zeek Related Packages=== * RDP Fingerprinting - Profiling RDP Clients with JA3 and RDFP - Adel K announced this package. You can find out more about it at: https://medium.com/@0x4d31/rdp-client-fingerprinting-9e7ac219f7f4 ===Publication Schedule (Updated)=== * Issue 1 - January 2020 (Covers December 2019) - 14 January 2020 * Issue 2 - March 2020 (Covers January and February 2020) - 2 March 2020 * Issue 3 - April 2020 (Covers March 2020) - 7 April 2020 * Issue 4 - May 2020 (Covers April 2020) - 4 May 2020 * Issue 5 - June 2020 (Covers May 2020) - 1 June 2020 * Issue 6 - July 2020 (Covers June 2020) - 6 July 2020 * Issue 7 - August 2020 (Covers July 2020) - 3 August 2020 * Issue 8 - September 2020 (Covers August 2020) - 7 September 2020 * Issue 9 - Special Issue 1 - September 2020 (Covers ZeekWeek 2020) - 21 September 2020 * Issue 10 - October 2020 (Covers September 2020) - 5 October 2020 * Issue 11 - November 2020 (Covers October 2020) - 2 November 2020 * Issue 12 - December 2020 (Covers November 2020) - 7 December 2020 * Issue 13 - Special Issue 2 - (Year End Review) - 21 December 2020 ===Get Involved=== If you are interested in getting involved with the Zeek Newsletter, please email news at zeek.org. Stay up to date by subscribing to the Zeek Mailing List. Follow us on Twitter Join the Slack Channel. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200407/ba35a3c4/attachment-0001.html From robin at corelight.com Wed Apr 8 02:17:51 2020 From: robin at corelight.com (Robin Sommer) Date: Wed, 8 Apr 2020 09:17:51 +0000 Subject: [Zeek] Zeek and CLA In-Reply-To: References: Message-ID: <20200408091751.GA10024@corelight.com> > I wanted to ask whether contributing code to Zeek requires signing a > Contributor License Agreement (CLA). No, it doesn't, but we do assume that all contributions are licensed under Zeek's BSD license. Robin -- Robin Sommer * Corelight, Inc. * robin at corelight.com * www.corelight.com From petr.medonos at etnetera.cz Wed Apr 8 09:20:58 2020 From: petr.medonos at etnetera.cz (Petr Medonos) Date: Wed, 8 Apr 2020 18:20:58 +0200 Subject: [Zeek] Long running connection using threshold In-Reply-To: References: <38442acf-7430-2e23-0528-e85a098437b7@etnetera.cz> Message-ID: <10a4a2ba-906d-3bb6-ce7c-66da0018729c@etnetera.cz> Hi Jon, thanks for the fix. Works like a charm! -- Petr On 06. 04. 20 22:43, Jon Siwek wrote: > Hi Petr, > > Your example code looked correct to me, but I found what simply looked > like a bug in the connection thresholding code that did the duration > comparison in reverse of what it should. Here's my proposed patch: > https://github.com/zeek/zeek/pull/899 > > - Jon > > On Tue, Mar 31, 2020 at 12:25 PM Petr Medonos wrote: >> >> Hi, >> I tried to write simple script to detect long running connection using >> zeek (3.0) threshold. I set duration in connection established event and >> then using duration_threshold_crossed logged connection above the limit. >> But Notice log is then flooded with every new established connection. >> Simple PoC bellow. Did I missed something? Is there any better way to >> detect long running connection? I tried Corelight bro-long-connections >> but there is lot overhead in my environment. Thanks for pointing me the >> right way! >> >> >> -- >> Petr >> >> >> PoC: >> >> @load base/protocols/conn >> >> module LongConnection; >> >> export { >> redef enum Log::ID += { LOG }; >> >> redef enum Notice::Type += { >> LongConnection::found >> }; >> >> const duration: interval = 12hr &redef; >> } >> >> event connection_established(c: connection) >> { >> ConnThreshold::set_duration_threshold(c, duration); >> } >> >> event ConnThreshold::duration_threshold_crossed(c: connection, >> threshold: interval, is_orig: bool) >> { >> local message = fmt("%s:%s -> %s:%s remained alive for longer >> than %s", c$id$orig_h, c$id$orig_p, c$id$resp_h, c$id$resp_p, threshold); >> >> NOTICE([$note=LongConnection::found, >> $msg=message, >> $sub=fmt("%.2f", threshold), >> $conn=c]); >> >> } >> >> >> _______________________________________________ >> Zeek mailing list >> zeek at zeek.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200408/9dbd296f/attachment.bin From nahum at us.ibm.com Wed Apr 8 12:37:37 2020 From: nahum at us.ibm.com (Erich M Nahum) Date: Wed, 8 Apr 2020 19:37:37 +0000 Subject: [Zeek] kafka plugin silently fails Message-ID: An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200408/5ed8ef06/attachment.html From akgraner at corelight.com Thu Apr 9 06:59:52 2020 From: akgraner at corelight.com (Amber Graner) Date: Thu, 9 Apr 2020 09:59:52 -0400 Subject: [Zeek] Reminder - ASK THE ZEEKSPERTS Webinar Today 9 April 2020 Message-ID: Hi all, Fatema Bannat Wala will be hosting today's ASK THE ZEEKSPERTS Webinar at 3:30 - 4:30pm Eastern/12:30-1:30pm Pacific. If you're new to Zeek, have installation questions, questions about the weird.log and more then you don't want to miss this call with Fatema. You can join via the registration link: https://attendee.gotowebinar.com/register/2632319203581363981 Hope to see you on the call. Thanks, ~Amber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200409/356c05c7/attachment.html From ttomek.koziak at gmail.com Thu Apr 9 12:05:50 2020 From: ttomek.koziak at gmail.com (Tomek Koziak) Date: Thu, 9 Apr 2020 21:05:50 +0200 Subject: [Zeek] Does Zeek allow to inspect RTP headers? Message-ID: Hi all, As far as I see here no RTP analizer has been added yet. So I have an another question regarding this topic. Is there any existing guide or tutorial explaining how I can develop an analizer for a protocol myself or should I just base it on the already existing code? Best regards and happy Easter. Tomasz -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200409/d1199679/attachment.html From vincyforce at gmail.com Thu Apr 9 13:25:13 2020 From: vincyforce at gmail.com (Vincenzo) Date: Thu, 9 Apr 2020 22:25:13 +0200 Subject: [Zeek] Load Unload Script Signature RunTime Message-ID: Hi, is there a way to enable scripts or signatures in runtime, without doing zeekctl restart or zeekctl deploy? Same speech to disable them Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200409/3965f9f6/attachment.html From anthony.kasza at gmail.com Thu Apr 9 14:35:27 2020 From: anthony.kasza at gmail.com (anthony kasza) Date: Thu, 9 Apr 2020 15:35:27 -0600 Subject: [Zeek] Does Zeek allow to inspect RTP headers? In-Reply-To: References: Message-ID: This is a great question and something I recently went through with RDPEUDP. The Syslog analyzer is a good example of a simple analyzer. The SSL analyzer is a good example of a complex analyzer. I found comparing existing analyzers to the files which binpac_quickstart outputs very helpful. It turns out much of the code which composes an analyzer is template/boilerplate. *-protocol.pac and *-analzer.pac is where most of the analyzer will live. The README of binpac explains its DSL fairly well. It's much smaller than Zeek's scripting language but harder, in my opinion, to debug. Some random thoughts on binpac: - I had issues using nested cases, so don't use them - I ended up using temporary or "throw away" fields than I thought would be necessary - There are conventions but there doesn't seem to be one way of using binpac - printf from proc_* functions is basically all the debugging info you get Reading the Zeek docs on DPD, PIA, and the Signature Framework were also useful if you want your analyzer to attach to connections in a robust manner. I hope this helps! -AK On Thu, Apr 9, 2020, 13:10 Tomek Koziak wrote: > Hi all, > > As far as I see here > no > RTP analizer has been added yet. So I have an another question regarding > this topic. Is there any existing guide or tutorial explaining how I can > develop an analizer for a protocol myself or should I just base it on the > already existing code? > > Best regards and happy Easter. > > Tomasz > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200409/0b831ed5/attachment.html From nabilmemon.ec at gmail.com Thu Apr 9 23:06:54 2020 From: nabilmemon.ec at gmail.com (Nabil Memon) Date: Fri, 10 Apr 2020 11:36:54 +0530 Subject: [Zeek] Regarding udp_content event In-Reply-To: References: Message-ID: Fair enough. Thanks Jon. On Wed, Apr 8, 2020 at 1:40 AM Jon Siwek wrote: > On Tue, Apr 7, 2020 at 2:43 AM Nabil Memon > wrote: > > > Instead configuring zeek to say these are likely to be server ports. > > What would happen if we introduce a check for source port as well with > the destination port? > > Did you consider this approach? > > Yeah, that's an alternate idea that would work. I added such an > option, called "udp_content_ports", to the Pull Request if you find it > more convenient, although configuring likely server ports may still > generally be useful if you commonly find inspected-traffic where the > originator/responder roles would better to have been flipped to > reflect a known-server. > > - Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200410/df9f3b34/attachment-0001.html From nabilmemon.ec at gmail.com Thu Apr 9 23:58:32 2020 From: nabilmemon.ec at gmail.com (Nabil Memon) Date: Fri, 10 Apr 2020 12:28:32 +0530 Subject: [Zeek] Performance hit with long lived flows Message-ID: Hi Zeek, Hope you're all doing well. I have a big 4GB sized PCAP and I am running many iterations of it at 10 Gbps with the help of load balancer and multiple instance of bro running on top of it. It takes around 3.5 seconds to finish one iteration. I have to run multiple iteration of the same pcap because of not having a test network which can pump 10Gbps traffic to my software. I don't even have a very large pcap so that I can run only one iteration for a long time. Other than proper(SYN-SYNACK-ACK--------FIN/RST) TCP flows, bro is able to hold all the other connections. If the run is for let's say an hour, it notifies about the connection after the test is over. This particular scenario is a test specific, but the need to tackle long lived flows is a valid one. I tired, *"connection_status_update" *way of handling this. If the update interval is configured to 10 min, it starts dropping exactly around 40 mins. If the interval is kept to 1 min, it starts having problem at around 4 min. I could not figure out why bro behaves this way, what is causing at (interval * 4) mins(there is one parameter i think is playing a role which is the time taken by a PCAP to complete one iteration, but still it doesn't help coming up with any theory). So, after it starts dropping, the number of broccoli sockets seems to be increasing and bro then goes into unresponsive state. I tried Connection polling using ConnPolling::watch(), this approach is way better than *connection_status_update* for sure is what I observed, this takes a little while to drop but it doesn't go into the very bad state of sockets being increasing and the unresponsive state. I also tried schedule, and it didn't serve my purpose either. After trying out whatever bro suggests me to handle this, I came up with my own implementation. -------------------------------------------------------------------------------------------------------------------- redef record connection += { loop_count: count &default=1; }; global connection_status_interval = 1 min; global connTable: table[string] of conn_id = table(); event connection_state_remove(c: connection) { delete connTable[c$uid]; } event new_connection(c: connection) { connTable[c$uid] = c$id; } event checkConnectionInterval() { local conn: connection; for (uid1 in connTable) { conn = lookup_connection(connTable[uid1]); if (conn$duration >= connection_status_interval * conn$loop_count) { handle_connection_data(conn, T); conn$loop_count += 1; } } schedule 30secs {checkConnectionInterval()}; } schedule 30secs {checkConnectionInterval()}; -------------------------------------------------------------------------------------------------------------------- As you can see, I maintained my own connection table, and with the help of schedule, I am managed to scan the table every 30 seconds and compare the connection's duration with the time configured. I haven't explore if schedule routines works in the same main thread?? If yes, then obviously, this can hold bro's main packet processing thread and we may have a serious damage going through a big list of such table entries. But I also thought of scanning in some batches, with the help of *Two_Such_Tables* and a *flag_For_In_Which_To_Fill_NewConnections* approach. Scanning in some batches will surely help in overall balancing of the software. With this approach, I am having a successful run. I just want to know what do you guys feel about this by keeping everything(test scenario/overall system's condition etc.) in mind. Can bro's suggested approach work in real 10Gbps network traffic?? Any suggestions how I can simulate 10Gbps real network traffic with the packet containing protocols or conversations I am interested in?? Regards, Nabil Jada -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200410/2e212ad9/attachment.html From jsiwek at corelight.com Fri Apr 10 10:27:03 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Fri, 10 Apr 2020 10:27:03 -0700 Subject: [Zeek] Performance hit with long lived flows In-Reply-To: References: Message-ID: On Fri, Apr 10, 2020 at 12:07 AM Nabil Memon wrote: > I haven't explore if schedule routines works in the same main thread?? All script code is currently executed on the main thread. > If yes, then obviously, this can hold bro's main packet processing thread and we may have a serious damage going through a big list of such table entries. But I also thought of scanning in some batches, Right, there's potential for scripts that do a lot of work at one time to interfere w/ packet processing and batching the work across time is a possible solution. Generalized coroutine support might also make it a bit easier to structure such batch-and-yield logic, but don't think there's near-term plans to add that feature. - Jon From jsiwek at corelight.com Fri Apr 10 10:40:32 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Fri, 10 Apr 2020 10:40:32 -0700 Subject: [Zeek] Load Unload Script Signature RunTime In-Reply-To: References: Message-ID: On Thu, Apr 9, 2020 at 1:29 PM Vincenzo wrote: > > Hi, is there a way to enable scripts or signatures in runtime, without doing zeekctl restart or zeekctl deploy? For signatures: no, don't know (or never seen) a way to dynamically enable/disable them at run-time For scripts: also no direct/generic way to dynamically enable/disable, but if you have control of the script, it's always possible to conditionalize logic on some flag and then flip that flag state whenever you want. - Jon From glwallum at gmail.com Fri Apr 10 13:55:54 2020 From: glwallum at gmail.com (Gordon Wallum) Date: Fri, 10 Apr 2020 14:55:54 -0600 Subject: [Zeek] Notice framework - able to send syslog? Message-ID: Hello! We are looking to integrate the Zeek notices with our separate SIEM system. Is it possible to have the Notice framework send syslogs? Or could this be accomplished in a different way? Thank you, Gordon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200410/3587566a/attachment.html From greg.grasmehr at caltech.edu Fri Apr 10 15:57:15 2020 From: greg.grasmehr at caltech.edu (Greg Grasmehr) Date: Fri, 10 Apr 2020 15:57:15 -0700 Subject: [Zeek] zkg after CentOS zeek 3.1.1 rpm install Message-ID: <20200410225715.GG3456@dakine> Greetings, I'm trying to use zkg to install pf_ring, I have zeek 3.1.1 source available, and the following in the config, but it still errors with Cannot determine Bro source directory, use --bro-dist=DIR Having bro_dist defined in the config makes no difference cat .zkg/config [sources] zeek = https://github.com/zeek/packages [paths] state_dir = /home/zeek/.zkg script_dir = /opt/zeek/share/zeek/site plugin_dir = /opt/zeek/lib/zeek/plugins zeek_dist = /tmp/zeek-3.1.1 bro_dist = /tmp/zeek-3.1.1 zkg --verbose install zeek/ntop/bro-pf_ring The following packages will be INSTALLED: zeek/ntop/bro-pf_ring (master) Proceed? [Y/n] Y Running unit tests for "zeek/ntop/bro-pf_ring" error: failed to run tests for zeek/ntop/bro-pf_ring: package build_command failed, see log in /home/zeek/.zkg/logs/bro-pf_ring-build.log Proceed to install anyway? [N/y] N Abort. cat /home/zeek/.zkg/logs/bro-pf_ring-build.log === STDERR === === STDOUT === Cannot determine Bro source directory, use --bro-dist=DIR. Thanks for any pointers. Stay Safe, Greg From ericooi at gmail.com Fri Apr 10 16:24:11 2020 From: ericooi at gmail.com (Eric Ooi) Date: Fri, 10 Apr 2020 18:24:11 -0500 Subject: [Zeek] zkg after CentOS zeek 3.1.1 rpm install In-Reply-To: <20200410225715.GG3456@dakine> References: <20200410225715.GG3456@dakine> Message-ID: <8A8C3A9A-493A-43F0-AFFE-D84CBEAFD026@gmail.com> The package likely isn?t compatible with Zeek 3.1 yet, since it introduced significant changes. However, you can install pf_ring without using zkg: https://docs.zeek.org/en/current/configuration/#installing-pf-ring That said, the general recommendation nowadays is to go with af_packet, of which there is a Zeek 3.1 compatible package: https://packages.zeek.org/packages/view/6dedb9cc-5916-11ea-9321-0a645a3f3086 If you go that route, I?ve written a guide to get you started: https://www.ericooi.com/zeekurity-zen-part-ii-zeek-package-manager/ Hope that helps! Eric > On Apr 10, 2020, at 5:57 PM, Greg Grasmehr wrote: > > Greetings, > > I'm trying to use zkg to install pf_ring, I have zeek 3.1.1 source > available, and the following in the config, but it still errors with > > Cannot determine Bro source directory, use --bro-dist=DIR > > Having bro_dist defined in the config makes no difference > > cat .zkg/config > [sources] > zeek = https://github.com/zeek/packages > > [paths] > state_dir = /home/zeek/.zkg > script_dir = /opt/zeek/share/zeek/site > plugin_dir = /opt/zeek/lib/zeek/plugins > zeek_dist = /tmp/zeek-3.1.1 > bro_dist = /tmp/zeek-3.1.1 > > > > > zkg --verbose install zeek/ntop/bro-pf_ring > The following packages will be INSTALLED: > zeek/ntop/bro-pf_ring (master) > > Proceed? [Y/n] Y > Running unit tests for "zeek/ntop/bro-pf_ring" > error: failed to run tests for zeek/ntop/bro-pf_ring: package build_command failed, see log in /home/zeek/.zkg/logs/bro-pf_ring-build.log > Proceed to install anyway? [N/y] N > Abort. > > > > cat /home/zeek/.zkg/logs/bro-pf_ring-build.log > === STDERR === > === STDOUT === > Cannot determine Bro source directory, use --bro-dist=DIR. > > Thanks for any pointers. > > Stay Safe, > > Greg > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200410/b3d6134a/attachment.html From johanna at icir.org Fri Apr 10 16:47:17 2020 From: johanna at icir.org (Johanna Amann) Date: Fri, 10 Apr 2020 16:47:17 -0700 Subject: [Zeek] zkg after CentOS zeek 3.1.1 rpm install In-Reply-To: <8A8C3A9A-493A-43F0-AFFE-D84CBEAFD026@gmail.com> References: <20200410225715.GG3456@dakine> <8A8C3A9A-493A-43F0-AFFE-D84CBEAFD026@gmail.com> Message-ID: <119C2E5F-8D15-4952-99FE-0537922015AC@icir.org> Also - make sure that you install the zeek development headers (assuming you use our rpms, the package is called zeek-devel Johanna On 10 Apr 2020, at 16:24, Eric Ooi wrote: > The package likely isn?t compatible with Zeek 3.1 yet, since it > introduced significant changes. However, you can install pf_ring > without using zkg: > https://docs.zeek.org/en/current/configuration/#installing-pf-ring > > > That said, the general recommendation nowadays is to go with > af_packet, of which there is a Zeek 3.1 compatible package: > https://packages.zeek.org/packages/view/6dedb9cc-5916-11ea-9321-0a645a3f3086 > > > If you go that route, I?ve written a guide to get you started: > https://www.ericooi.com/zeekurity-zen-part-ii-zeek-package-manager/ > > > Hope that helps! > Eric > > >> On Apr 10, 2020, at 5:57 PM, Greg Grasmehr >> wrote: >> >> Greetings, >> >> I'm trying to use zkg to install pf_ring, I have zeek 3.1.1 source >> available, and the following in the config, but it still errors with >> >> Cannot determine Bro source directory, use --bro-dist=DIR >> >> Having bro_dist defined in the config makes no difference >> >> cat .zkg/config >> [sources] >> zeek = https://github.com/zeek/packages >> >> [paths] >> state_dir = /home/zeek/.zkg >> script_dir = /opt/zeek/share/zeek/site >> plugin_dir = /opt/zeek/lib/zeek/plugins >> zeek_dist = /tmp/zeek-3.1.1 >> bro_dist = /tmp/zeek-3.1.1 >> >> >> >> >> zkg --verbose install zeek/ntop/bro-pf_ring >> The following packages will be INSTALLED: >> zeek/ntop/bro-pf_ring (master) >> >> Proceed? [Y/n] Y >> Running unit tests for "zeek/ntop/bro-pf_ring" >> error: failed to run tests for zeek/ntop/bro-pf_ring: package >> build_command failed, see log in >> /home/zeek/.zkg/logs/bro-pf_ring-build.log >> Proceed to install anyway? [N/y] N >> Abort. >> >> >> >> cat /home/zeek/.zkg/logs/bro-pf_ring-build.log >> === STDERR === >> === STDOUT === >> Cannot determine Bro source directory, use --bro-dist=DIR. >> >> Thanks for any pointers. >> >> Stay Safe, >> >> Greg >> _______________________________________________ >> Zeek mailing list >> zeek at zeek.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek From abattou at gmail.com Sat Apr 11 08:19:45 2020 From: abattou at gmail.com (Abdella Battou) Date: Sat, 11 Apr 2020 11:19:45 -0400 Subject: [Zeek] changing conn logging to SQLite Message-ID: I am new to Zeek and I would like to redist the conn logging to SQLite. The documentation says that this is natively supported. I found this filter "sqlite-conn-filte.zeek" in one of the post event zeek_init() { local filter: Log::Filter = [ $name="sqlite", $path="/var/db/conn", $config=table(["tablename"] = "conn"), $writer=Log::WRITER_SQLITE ]; Log::add_filter(Conn::LOG, filter); } my question is where to put (which directory) ? and do I need to invoke it somewhere ? cheers, Abdella -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200411/ba980b94/attachment.html From akgraner at corelight.com Tue Apr 14 06:10:53 2020 From: akgraner at corelight.com (Amber Graner) Date: Tue, 14 Apr 2020 09:10:53 -0400 Subject: [Zeek] Zeek From Home - 15 April 2020 - Zeek Agent - presented by Seth Hall Message-ID: Hi all, ===You are invited to a Zoom webinar.=== * When: Apr 15, 2020 * Time: 11am AM Pacific Time/ 2:00 PM Eastern Time (US and Canada) * Topic: Zeek From Home - Zeek-Agent - host Seth Hall Zeek-Agent is an endpoint monitoring agent that provides host activity to Zeek ===How to Register=== Register in advance for this webinar*: https://corelight.zoom.us/webinar/register/WN_SK5uujKBQL-DyV7Jcq-I9w After registering, you will receive a confirmation email containing information about joining the webinar. **PLEASE NOTE: * While these webinars are hosted by Corelight, your registration information is not shared with anyone outside of myself and will only be used to update you about this webinar. ===More information about the Zeek Agent=== More information can be found about the Zeek Agent at: Zeek Blog Announcement - Announcing The Zeek Agent Github Repository - https://github.com/zeek/zeek-agent ===Presenting a Zeek From Home Session=== Please let me know if you would like to present on a Zeek related topic. More information can be found at: https://zeek.org/2020/03/31/zeek-from-home/ Please let me know if you have any questions. Thanks, ~Amber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200414/4c36938a/attachment.html From shadowx787 at gmail.com Tue Apr 14 09:58:31 2020 From: shadowx787 at gmail.com (Justin Mullins) Date: Tue, 14 Apr 2020 12:58:31 -0400 Subject: [Zeek] Zeek Plugins Message-ID: Question, do all Zeek/Bro plugins require access to Zeek source? Is it possible to pre-compile a plugin or should I always expect to run Zeek from source if I need to run plugins? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200414/5ac7f1b0/attachment.html From jsiwek at corelight.com Tue Apr 14 13:10:40 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Tue, 14 Apr 2020 13:10:40 -0700 Subject: [Zeek] Zeek 3.0.4 and 3.1.2 release (security + bug fixes) Message-ID: Downloads for Zeek 3.0.4 (LTS) and Zeek 3.1.2 are available: https://zeek.org/get-zeek These releases fix several bugs, including one potential security issue due to a stack overflow in the POP3 analyzer (thanks to Matteo Rizzo for the report). See the release notes for details: https://github.com/zeek/zeek/releases/tag/v3.0.4 https://github.com/zeek/zeek/releases/tag/v3.1.2 From jsiwek at corelight.com Tue Apr 14 15:04:08 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Tue, 14 Apr 2020 15:04:08 -0700 Subject: [Zeek] Zeek 3.0.4 and 3.1.2 release (security + bug fixes) In-Reply-To: References: Message-ID: There's also now Zeek 3.0.5 (LTS), with only difference from 3.0.4 being a fix for compilation on various platforms with older compilers (e.g. GCC 4.8.x). On Tue, Apr 14, 2020 at 1:10 PM Jon Siwek wrote: > > Downloads for Zeek 3.0.4 (LTS) and Zeek 3.1.2 are available: > > https://zeek.org/get-zeek > > These releases fix several bugs, including one potential security > issue due to a stack overflow in the POP3 analyzer (thanks to Matteo > Rizzo for the report). See the release notes for details: > > https://github.com/zeek/zeek/releases/tag/v3.0.4 > https://github.com/zeek/zeek/releases/tag/v3.1.2 From ben.reardon at corelight.com Tue Apr 14 16:17:54 2020 From: ben.reardon at corelight.com (Ben Reardon) Date: Wed, 15 Apr 2020 09:17:54 +1000 Subject: [Zeek] Got Zoom? Message-ID: This may be helpful for some out there. It's a simple package that works on Zoom TLS traffic. https://zeek.org/2020/04/14/got-zoom/ Cheers Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200415/23062f53/attachment.html From jsiwek at corelight.com Tue Apr 14 17:22:45 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Tue, 14 Apr 2020 17:22:45 -0700 Subject: [Zeek] Zeek Plugins In-Reply-To: References: Message-ID: On Tue, Apr 14, 2020 at 11:05 AM Justin Mullins wrote: > > Question, do all Zeek/Bro plugins require access to Zeek source? Compiling plugins requires access to Zeek headers. Those are either available in the original source tree, or they'll get installed after compiling/installing from source (you can remove the original source tree afterward if you want), or various binary packages for Zeek should also be installing headers, likely as separate dev/devel package. > Is it possible to pre-compile a plugin or should I always expect to run Zeek from source if I need to run plugins? It is technically possible to pre-compile a plugin for a specific platform and Zeek version and distribute it (e.g. in zkg package), but don't think many will go that route, so yeah, expect to have the Zeek headers installed if you'll be using plugins. - Jon From jsiwek at corelight.com Tue Apr 14 17:32:31 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Tue, 14 Apr 2020 17:32:31 -0700 Subject: [Zeek] changing conn logging to SQLite In-Reply-To: References: Message-ID: On Sat, Apr 11, 2020 at 8:29 AM Abdella Battou wrote: > > I am new to Zeek and I would like to redist the conn logging to SQLite. The documentation says that this is natively supported. > > I found this filter "sqlite-conn-filte.zeek" in one of the post > > event zeek_init() > { > local filter: Log::Filter = > [ > $name="sqlite", > $path="/var/db/conn", > $config=table(["tablename"] = "conn"), > $writer=Log::WRITER_SQLITE > ]; > > Log::add_filter(Conn::LOG, filter); > } > > my question is where to put (which directory) ? and do I need to invoke it somewhere ? Where you put that depends on how you run/deploy Zeek, but the usual way involving ZeekControl means you could just add it to the end of your local.zeek file which gets installed by default (if built from source) at /usr/local/zeek/share/zeek/site/local.zeek - Jon From yuri.neves at pan-net.eu Wed Apr 15 01:58:35 2020 From: yuri.neves at pan-net.eu (Neves, Yuri) Date: Wed, 15 Apr 2020 08:58:35 +0000 Subject: [Zeek] SumStat key request for UID took longer than 1 minute Message-ID: <1586941115744.30866@Pan-net.eu> Hello, I'm fairly new to Zeek and I'm trying to install and configure a Zeek cluster as a Proof of Concept for enterprise deployment. The environment consists of 3 hosts - 1 manager and 2 workers (Zeek has been compiled with PF_RING to leverage load balancing capabilities). The installation is successful, zeekctl deploy didn't yield any errors, and all nodes appear as 'running'; however I cannot see any logs (HTTP, DNS, SSL etc.). The workers don't seem to be working. Digging a little bit, the logger process yields a report.log, in which the following entry shows up repeatedly: {"ts":1586866086.934979,"level":"Reporter::WARNING","message":"SumStat key request for the j1158rc4kei SumStat uid took longer than 1 minute and was automatically cancelled.","location":"/usr/local/zeek/share/zeek/base/frameworks/sumstats/./cluster.zeek, line 226"}. I've found a similar issue here but I made sure that scan.zeek policy is commented out. Also, the manager process outputs the following in the stderr.log: warning in /usr/local/zeek/share/zeek/base/frameworks/netcontrol/./cluster.zeek, line 117: &default on parameter 'msg' has no effect (not a event declaration). Not sure if this is even relevant, but I did not recall seeing this when I installed zeek as a standalone. Could you help shed a light on this? I'm sharing as much information as possible from the cluster below: node.cfg: [manager] type=manager host=nids [proxy-1] type=proxy host=nids [logger] type=logger host=nids [worker-1] type=worker host=192.168.2.31 interface=ens3 lb_method=pf_ring lb_procs=3 pin_cpus=0,1,2 [worker-2] type=worker host=192.168.2.36 interface=ens3 lb_method=pf_ring lb_procs=3 pin_cpus=0,1,2 zeekctl status: Name Type Host Status Pid Started logger logger nids running 12620 14 Apr 11:52:04 manager manager nids running 12668 14 Apr 11:52:05 proxy-1 proxy nids running 12715 14 Apr 11:52:07 worker-1-1 worker 192.168.2.31 running 24440 14 Apr 11:52:08 worker-1-2 worker 192.168.2.31 running 24436 14 Apr 11:52:08 worker-1-3 worker 192.168.2.31 running 24439 14 Apr 11:52:08 worker-2-1 worker 192.168.2.36 running 24619 14 Apr 11:52:08 worker-2-2 worker 192.168.2.36 running 24617 14 Apr 11:52:08 worker-2-3 worker 192.168.2.36 running 24616 14 Apr 11:52:08 zeekctl top: Name Type Host Pid VSize Rss Cpu Cmd logger logger nids 12620 1G 107M 0% zeek manager manager nids 12668 678M 108M 0% zeek proxy-1 proxy nids 12715 676M 106M 0% zeek worker-1-1 worker 192.168.2.31 24440 683M 112M 0% zeek worker-1-2 worker 192.168.2.31 24436 683M 112M 0% zeek worker-1-3 worker 192.168.2.31 24439 683M 113M 0% zeek worker-2-1 worker 192.168.2.36 24619 685M 115M 0% zeek worker-2-2 worker 192.168.2.36 24617 683M 113M 0% zeek worker-2-3 worker 192.168.2.36 24616 684M 114M 0% zeek zeekctl config: bindir = /usr/local/zeek/bin capstatspath = /usr/local/zeek/bin/capstats cfgdir = /usr/local/zeek/etc commandtimeout = 60 commtimeout = 10 compresscmd = gzip compressextension = gz compresslogs = 1 compresslogsinflight = 0 configchksum = cc8e3228f42668759783d0165ac9181f751e6e76 confignodechksum = 29aa08b5f6adaf65cfe2f550452d9abd7a76a699 controltopic = zeek/control crashexpireinterval = 0 croncmd = cronenabled = True debug = 0 debuglog = /usr/local/zeek/spool/debug.log defaultstoredir = /usr/local/zeek/spool/stores env_vars = global-hash-seed = a776fc25 hash-nodecfg = 05042402823ed87a824dd5042ad63f8f679b6761 hash-zeekctlcfg = 583b8364fa01143dead8af7fbbcdb01fc98762f2 havenfs = 0 helperdir = /usr/local/zeek/share/zeekctl/scripts/helpers keeplogs = lb_custom.interfaceprefix = lb_custom.interfacesuffix = libdir = /usr/local/zeek/lib libdir64 = /usr/local/zeek/lib64 libdirinternal = /usr/local/zeek/lib/zeekctl localnetscfg = /usr/local/zeek/etc/networks.cfg lockfile = /usr/local/zeek/spool/lock logdir = /usr/local/zeek/logs logexpireinterval = 0 logexpireminutes = 0 logger-crashed = False logger-expect-running = True logger-host = nids logger-pid = 12620 logger-port = 47763 logrotationinterval = 3600 mailalarmsinterval = 86400 mailalarmsto = root at localhost mailarchivelogfail = 1 mailconnectionsummary = True mailfrom = Zeek mailhostupdown = True mailreceivingpackets = 1 mailreplyto = mailsubjectprefix = [Zeek] mailto = root at localhost makearchivename = /usr/local/zeek/share/zeekctl/scripts/make-archive-name manager-crashed = False manager-expect-running = True manager-host = nids manager-pid = 12668 manager-port = 47764 memlimit = unlimited mindiskspace = 5 nodecfg = /usr/local/zeek/etc/node.cfg os = Linux pcapbufsize = 128 pcapsnaplen = 9216 pfringclusterid = 21 pfringclustertype = 4-tuple pfringfirstappinstance = 0 pin_command = taskset -c plugindir = /usr/local/zeek/lib/zeekctl/plugins pluginzeekdir = /usr/local/zeek/lib/zeek/plugins policydir = /usr/local/zeek/share/zeek policydirsiteinstall = /usr/local/zeek/spool/installed-scripts-do-not-touch/site policydirsiteinstallauto = /usr/local/zeek/spool/installed-scripts-do-not-touch/auto postprocdir = /usr/local/zeek/share/zeekctl/scripts/postprocessors prefixes = local proxy-1-crashed = False proxy-1-expect-running = True proxy-1-host = nids proxy-1-pid = 12715 proxy-1-port = 47765 savetraces = 0 scriptsdir = /usr/local/zeek/share/zeekctl/scripts sendmail = /usr/sbin/sendmail sitepluginpath = sitepolicypath = /usr/local/zeek/share/zeek/site sitepolicyscripts = local.zeek spooldir = /usr/local/zeek/spool standalone = False statefile = /usr/local/zeek/spool/state.db staticdir = /usr/local/zeek/share/zeekctl statsdir = /usr/local/zeek/logs/stats statslog = /usr/local/zeek/spool/stats.log statslogenable = True statslogexpireinterval = 0 statuscmdshowall = False stoptimeout = 60 stopwait = 0 test.enabled = False test.foo = 1 time = /usr/bin/time timefmt = %d %b %H:%M:%S timemachinehost = timemachineport = 47757/tcp tmpdir = /usr/local/zeek/spool/tmp tmpexecdir = /usr/local/zeek/spool/tmp tracesummary = /usr/local/zeek/bin/trace-summary version = 2.1.0-11 worker-1-1-crashed = False worker-1-1-expect-running = True worker-1-1-host = 192.168.2.31 worker-1-1-pid = 24440 worker-1-1-port = 47766 worker-1-2-crashed = False worker-1-2-expect-running = True worker-1-2-host = 192.168.2.31 worker-1-2-pid = 24436 worker-1-2-port = 47767 worker-1-3-crashed = False worker-1-3-expect-running = True worker-1-3-host = 192.168.2.31 worker-1-3-pid = 24439 worker-1-3-port = 47768 worker-2-1-crashed = False worker-2-1-expect-running = True worker-2-1-host = 192.168.2.36 worker-2-1-pid = 24619 worker-2-1-port = 47769 worker-2-2-crashed = False worker-2-2-expect-running = True worker-2-2-host = 192.168.2.36 worker-2-2-pid = 24617 worker-2-2-port = 47770 worker-2-3-crashed = False worker-2-3-expect-running = True worker-2-3-host = 192.168.2.36 worker-2-3-pid = 24616 worker-2-3-port = 47771 zeek = /usr/local/zeek/bin/zeek zeekargs = zeekbase = /usr/local/zeek zeekctlconfigdir = /usr/local/zeek/spool zeekport = 47762 zeekscriptdir = /usr/local/zeek/share/zeek zeekversion = 3.2.0-dev.391 zeekctl diag: [logger] No core file found. Zeek 3.2.0-dev.391-debug Linux 4.15.0-36-generic Zeek plugins: (none found) ==== reporter.log {"ts":1586865786.930556,"level":"Reporter::WARNING","message":"SumStat key request for the 5dLj9RAlW1g SumStat uid took longer than 1 minute and was automatically cancelled.","location":"/usr/local/zeek/share/zeek/base/frameworks/sumstats/./cluster.zeek, line 226"} {"ts":1586865786.930556,"level":"Reporter::WARNING","message":"SumStat key request for the JXG5gNSXhlj SumStat uid took longer than 1 minute and was automatically cancelled.","location":"/usr/local/zeek/share/zeek/base/frameworks/sumstats/./cluster.zeek, line 226"} {"ts":1586866086.934979,"level":"Reporter::WARNING","message":"SumStat key request for the j1158rc4kei SumStat uid took longer than 1 minute and was automatically cancelled.","location":"/usr/local/zeek/share/zeek/base/frameworks/sumstats/./cluster.zeek, line 226"} {"ts":1586866086.934979,"level":"Reporter::WARNING","message":"SumStat key request for the 8eFeFUPsW01 SumStat uid took longer than 1 minute and was automatically cancelled.","location":"/usr/local/zeek/share/zeek/base/frameworks/sumstats/./cluster.zeek, line 226"} ==== stderr.log ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -U .status -p zeekctl -p zeekctl-live -p local -p logger local.zeek zeekctl base/frameworks/cluster zeekctl/auto ==== .env_vars PATH=/usr/local/zeek/bin:/usr/local/zeek/share/zeekctl/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin ZEEKPATH=/usr/local/zeek/spool/installed-scripts-do-not-touch/site::/usr/local/zeek/spool/installed-scripts-do-not-touch/auto:/usr/local/zeek/share/zeek:/usr/local/zeek/share/zeek/policy:/usr/local/zeek/share/zeek/site CLUSTER_NODE=logger ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log [manager] No core file found. Zeek 3.2.0-dev.391-debug Linux 4.15.0-36-generic Zeek plugins: (none found) ==== No reporter.log ==== stderr.log warning in /usr/local/zeek/share/zeek/base/frameworks/netcontrol/./cluster.zeek, line 117: &default on parameter 'msg' has no effect (not a event declaration) warning in /usr/local/zeek/share/zeek/base/frameworks/netcontrol/./cluster.zeek, line 125: &default on parameter 'msg' has no effect (not a event declaration) warning in /usr/local/zeek/share/zeek/base/frameworks/netcontrol/./cluster.zeek, line 133: &default on parameter 'msg' has no effect (not a event declaration) warning in /usr/local/zeek/share/zeek/base/frameworks/netcontrol/./cluster.zeek, line 143: &default on parameter 'msg' has no effect (not a event declaration) ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -U .status -p zeekctl -p zeekctl-live -p local -p manager local.zeek zeekctl base/frameworks/cluster zeekctl/auto ==== .env_vars PATH=/usr/local/zeek/bin:/usr/local/zeek/share/zeekctl/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin ZEEKPATH=/usr/local/zeek/spool/installed-scripts-do-not-touch/site::/usr/local/zeek/spool/installed-scripts-do-not-touch/auto:/usr/local/zeek/share/zeek:/usr/local/zeek/share/zeek/policy:/usr/local/zeek/share/zeek/site CLUSTER_NODE=manager ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log [proxy-1] No core file found. Zeek 3.2.0-dev.391-debug Linux 4.15.0-36-generic Zeek plugins: (none found) ==== No reporter.log ==== stderr.log ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -U .status -p zeekctl -p zeekctl-live -p local -p proxy-1 local.zeek zeekctl base/frameworks/cluster zeekctl/auto ==== .env_vars PATH=/usr/local/zeek/bin:/usr/local/zeek/share/zeekctl/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin ZEEKPATH=/usr/local/zeek/spool/installed-scripts-do-not-touch/site::/usr/local/zeek/spool/installed-scripts-do-not-touch/auto:/usr/local/zeek/share/zeek:/usr/local/zeek/share/zeek/policy:/usr/local/zeek/share/zeek/site CLUSTER_NODE=proxy-1 ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log [worker-1-1] No core file found. Zeek 3.2.0-dev.391-debug Linux 4.15.0-36-generic Zeek plugins: (none found) ==== No reporter.log ==== stderr.log listening on ens3 ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -i ens3 -U .status -p zeekctl -p zeekctl-live -p local -p worker-1-1 local.zeek zeekctl base/frameworks/cluster zeekctl/auto ==== .env_vars PATH=/usr/local/zeek/bin:/usr/local/zeek/share/zeekctl/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games ZEEKPATH=/usr/local/zeek/spool/installed-scripts-do-not-touch/site::/usr/local/zeek/spool/installed-scripts-do-not-touch/auto:/usr/local/zeek/share/zeek:/usr/local/zeek/share/zeek/policy:/usr/local/zeek/share/zeek/site CLUSTER_NODE=worker-1-1 ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log [worker-1-2] No core file found. Zeek 3.2.0-dev.391-debug Linux 4.15.0-36-generic Zeek plugins: (none found) ==== No reporter.log ==== stderr.log listening on ens3 ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -i ens3 -U .status -p zeekctl -p zeekctl-live -p local -p worker-1-2 local.zeek zeekctl base/frameworks/cluster zeekctl/auto ==== .env_vars PATH=/usr/local/zeek/bin:/usr/local/zeek/share/zeekctl/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games ZEEKPATH=/usr/local/zeek/spool/installed-scripts-do-not-touch/site::/usr/local/zeek/spool/installed-scripts-do-not-touch/auto:/usr/local/zeek/share/zeek:/usr/local/zeek/share/zeek/policy:/usr/local/zeek/share/zeek/site CLUSTER_NODE=worker-1-2 ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log [worker-1-3] No core file found. Zeek 3.2.0-dev.391-debug Linux 4.15.0-36-generic Zeek plugins: (none found) ==== No reporter.log ==== stderr.log listening on ens3 ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -i ens3 -U .status -p zeekctl -p zeekctl-live -p local -p worker-1-3 local.zeek zeekctl base/frameworks/cluster zeekctl/auto ==== .env_vars PATH=/usr/local/zeek/bin:/usr/local/zeek/share/zeekctl/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games ZEEKPATH=/usr/local/zeek/spool/installed-scripts-do-not-touch/site::/usr/local/zeek/spool/installed-scripts-do-not-touch/auto:/usr/local/zeek/share/zeek:/usr/local/zeek/share/zeek/policy:/usr/local/zeek/share/zeek/site CLUSTER_NODE=worker-1-3 ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log [worker-2-1] No core file found. Zeek 3.2.0-dev.391-debug Linux 4.15.0-36-generic Zeek plugins: (none found) ==== No reporter.log ==== stderr.log listening on ens3 ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -i ens3 -U .status -p zeekctl -p zeekctl-live -p local -p worker-2-1 local.zeek zeekctl base/frameworks/cluster zeekctl/auto ==== .env_vars PATH=/usr/local/zeek/bin:/usr/local/zeek/share/zeekctl/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games ZEEKPATH=/usr/local/zeek/spool/installed-scripts-do-not-touch/site::/usr/local/zeek/spool/installed-scripts-do-not-touch/auto:/usr/local/zeek/share/zeek:/usr/local/zeek/share/zeek/policy:/usr/local/zeek/share/zeek/site CLUSTER_NODE=worker-2-1 ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log [worker-2-2] No core file found. Zeek 3.2.0-dev.391-debug Linux 4.15.0-36-generic Zeek plugins: (none found) ==== No reporter.log ==== stderr.log listening on ens3 ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -i ens3 -U .status -p zeekctl -p zeekctl-live -p local -p worker-2-2 local.zeek zeekctl base/frameworks/cluster zeekctl/auto ==== .env_vars PATH=/usr/local/zeek/bin:/usr/local/zeek/share/zeekctl/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games ZEEKPATH=/usr/local/zeek/spool/installed-scripts-do-not-touch/site::/usr/local/zeek/spool/installed-scripts-do-not-touch/auto:/usr/local/zeek/share/zeek:/usr/local/zeek/share/zeek/policy:/usr/local/zeek/share/zeek/site CLUSTER_NODE=worker-2-2 ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log [worker-2-3] No core file found. Zeek 3.2.0-dev.391-debug Linux 4.15.0-36-generic Zeek plugins: (none found) ==== No reporter.log ==== stderr.log listening on ens3 ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -i ens3 -U .status -p zeekctl -p zeekctl-live -p local -p worker-2-3 local.zeek zeekctl base/frameworks/cluster zeekctl/auto ==== .env_vars PATH=/usr/local/zeek/bin:/usr/local/zeek/share/zeekctl/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games ZEEKPATH=/usr/local/zeek/spool/installed-scripts-do-not-touch/site::/usr/local/zeek/spool/installed-scripts-do-not-touch/auto:/usr/local/zeek/share/zeek:/usr/local/zeek/share/zeek/policy:/usr/local/zeek/share/zeek/site CLUSTER_NODE=worker-2-3 ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200415/81a843db/attachment-0001.html From clopmz at outlook.com Thu Apr 16 09:09:32 2020 From: clopmz at outlook.com (Carlos Lopez) Date: Thu, 16 Apr 2020 16:09:32 +0000 Subject: [Zeek] Some error in kerberos log Message-ID: <739D8DE1-A315-416B-A7B7-6808A9BA86A2@outlook.com> Hi all, From time to time I am seeing some errors in my Zeek?s cluster like these: /opt/zeek/share/zeek/base/protocols/krb/./main.zeek, line 175 2020-04-16T15:13:49.866557Z Reporter::ERROR no such index (KRB::cipher_name[KRB::msg$ticket$cipher]) /opt/zeek/share/zeek/base/protocols/krb/./main.zeek, line 175 2020-04-16T14:37:05.530502Z Reporter::ERROR no such index (KRB::cipher_name[KRB::msg$ticket$cipher]) What does it matter? I am using Zeek 3.0.5 ? Regards, C. L. Martinez -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200416/9e31e70e/attachment.html From jsiwek at corelight.com Thu Apr 16 12:43:03 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Thu, 16 Apr 2020 12:43:03 -0700 Subject: [Zeek] Some error in kerberos log In-Reply-To: <739D8DE1-A315-416B-A7B7-6808A9BA86A2@outlook.com> References: <739D8DE1-A315-416B-A7B7-6808A9BA86A2@outlook.com> Message-ID: On Thu, Apr 16, 2020 at 9:18 AM Carlos Lopez wrote: > /opt/zeek/share/zeek/base/protocols/krb/./main.zeek, line 175 2020-04-16T14:37:05.530502Z Reporter::ERROR no such index (KRB::cipher_name[KRB::msg$ticket$cipher]) > > What does it matter? I am using Zeek 3.0.5 ? Thanks for reporting that, it's a bug that could cause some incorrect analysis/processing of the associated connection or even minor memory leak over time in that Zeek version. Proposed patch is on the way here: https://github.com/zeek/zeek/pull/918 - Jon From clopmz at outlook.com Thu Apr 16 23:40:04 2020 From: clopmz at outlook.com (Carlos Lopez) Date: Fri, 17 Apr 2020 06:40:04 +0000 Subject: [Zeek] Some error in kerberos log In-Reply-To: References: <739D8DE1-A315-416B-A7B7-6808A9BA86A2@outlook.com> Message-ID: <9484EBA7-09EF-4A01-B494-551B65D52203@outlook.com> Many thanks Jon. ?On 16/04/2020, 21:43, "Jon Siwek" wrote: On Thu, Apr 16, 2020 at 9:18 AM Carlos Lopez wrote: > /opt/zeek/share/zeek/base/protocols/krb/./main.zeek, line 175 2020-04-16T14:37:05.530502Z Reporter::ERROR no such index (KRB::cipher_name[KRB::msg$ticket$cipher]) > > What does it matter? I am using Zeek 3.0.5 ? Thanks for reporting that, it's a bug that could cause some incorrect analysis/processing of the associated connection or even minor memory leak over time in that Zeek version. Proposed patch is on the way here: https://github.com/zeek/zeek/pull/918 - Jon From ttomek.koziak at gmail.com Fri Apr 17 05:58:15 2020 From: ttomek.koziak at gmail.com (Tomek Koziak) Date: Fri, 17 Apr 2020 14:58:15 +0200 Subject: [Zeek] Zeek doesn't see the MQTT traffic. Message-ID: I am trying to work with the *mqtt_publish *and the* mqtt_connect *events*. *I have a mosquitto broker which is running locally on which I send messages from a sensor. I want to investigate those messages. However, I am not able to observe any values that should like c: connection. Even when I run it on the mqtt.pcap from here and again the previously mentioned events, it doesn't produce any output. What may cause this problem? BR Tomasz -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200417/9d89e533/attachment.html From jsiwek at corelight.com Fri Apr 17 11:42:44 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Fri, 17 Apr 2020 11:42:44 -0700 Subject: [Zeek] Zeek doesn't see the MQTT traffic. In-Reply-To: References: Message-ID: On Fri, Apr 17, 2020 at 6:00 AM Tomek Koziak wrote: > > Even when I run it on the mqtt.pcap from here and again the previously mentioned events, it doesn't produce any output. What may cause this problem? MQTT analysis isn't enabled by default, you can `@load policy/protocols/mqtt` to enable it. $ zeek -r mqtt_packets_tcpdump.pcap protocols/mqtt $ ls mqtt_*.log mqtt_connect.log mqtt_publish.log mqtt_subscribe.log - Jon From akgraner at corelight.com Fri Apr 17 13:28:03 2020 From: akgraner at corelight.com (Amber Graner) Date: Fri, 17 Apr 2020 16:28:03 -0400 Subject: [Zeek] Zeek From Home - Zeek Agent - Recording now Available. Message-ID: Hi all and Happy Friday!! The recording for Zeek-Agent- Zeek From Home Webinar is now available. * Link to the full blog post: https://zeek.org/2020/04/17/zeek-from-home-episode-1-zeek-agent-recording-now-available/ * Video: https://www.dropbox.com/s/c3tt7qdi84wrqha/GMT20200415-175213_Zeek-From-_2560x1440.mp4?dl=0 * Audio Only: https://www.dropbox.com/s/hn6srwmu0oten10/GMT20200415-175213_Zeek-From-.m4a?dl=0 * Slides: https://www.dropbox.com/s/bgww42hlnxzvcx2/Zeek%20at%20Home%20-%20Zeek-Agent%20-%20Seth%20Hall.pdf?dl=0 More information on the Zeek-Agent can be found at: https://zeek.org/2020/03/23/announcing-the-zeek-agent/ ===How can you or your organization get on the Zeek From Home Schedule?=== Have you presented at past Zeek Weeks or Zeek events? Do you have a Zeek related topic you?d like to share with the Zeek Community? If so, take a look at the Zeek from Home announcement . Please let me know if you have any questions or if you would like to present. With gratitude, ~Amber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200417/b3ade51c/attachment.html From vincyforce at gmail.com Mon Apr 20 01:49:05 2020 From: vincyforce at gmail.com (Vincenzo) Date: Mon, 20 Apr 2020 10:49:05 +0200 Subject: [Zeek] Support bro-netmap for FreeBSD 12 Message-ID: Hi the plugin bro-netmap is only compatibile with FreeBSD 11. Are there any development plans, to bring it to freebsd 12? thank you very much -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200420/6edd589e/attachment.html From raubvogel at gmail.com Mon Apr 20 05:30:43 2020 From: raubvogel at gmail.com (Mauricio Tavares) Date: Mon, 20 Apr 2020 08:30:43 -0400 Subject: [Zeek] Letting system handle log rotation Message-ID: I have the system's syslog to do the log rotation, including renaming, just the way I want. If I set LogRotationInterval = 0, would zeek then let the system do its thing? From raubvogel at gmail.com Tue Apr 21 05:34:07 2020 From: raubvogel at gmail.com (Mauricio Tavares) Date: Tue, 21 Apr 2020 08:34:07 -0400 Subject: [Zeek] Letting system handle log rotation In-Reply-To: References: Message-ID: On Mon, Apr 20, 2020 at 8:30 AM Mauricio Tavares wrote: > > I have the system's syslog to do the log rotation, including > renaming, just the way I want. If I set LogRotationInterval = 0, would > zeek then let the system do its thing? Got it to work: [raub at testcentos log]$ sudo ls -lh /var/log/bro/old total 16M -rw-r--r-- 1 root root 14K Apr 21 03:39 capture_loss.log-20200421 -rw-r--r-- 1 root root 3.6M Apr 21 03:39 communication.log-20200421 -rw-r--r-- 1 root root 6.4M Apr 21 03:39 conn.log-20200421 -rw-r--r-- 1 root root 970K Apr 21 03:39 dns.log-20200421 -rw-r--r-- 1 root root 177K Apr 21 03:39 files.log-20200421 -rw-r--r-- 1 root root 120K Apr 21 03:39 http.log-20200421 -rw-r--r-- 1 root root 27K Apr 21 03:39 loaded_scripts.log-20200421 -rw-r--r-- 1 root root 187 Apr 21 03:39 packet_filter.log-20200421 -rw-r--r-- 1 root root 529 Apr 21 03:39 reporter.log-20200421 -rw-r--r-- 1 root root 30K Apr 21 03:39 sip.log-20200421 -rw-r--r-- 1 root root 24K Apr 21 03:39 ssl.log-20200421 -rw-r--r-- 1 root root 118K Apr 21 03:39 stats.log-20200421 -rw-r--r-- 1 root root 188 Apr 21 03:39 stdout.log-20200421 -rw-r--r-- 1 root root 580 Apr 21 03:39 top_dns.log-20200421 -rw-r--r-- 1 root root 3.8M Apr 21 03:39 weird.log-20200421 [raub at testcentos log]$ sudo ls -lh /var/log/bro/current total 12M -rw-r--r-- 1 root root 22K Apr 21 12:13 capture_loss.log -rw-r--r-- 1 root root 5.7M Apr 21 12:22 communication.log -rw-r--r-- 1 root root 11M Apr 21 12:22 conn.log -rw-r--r-- 1 root root 1.6M Apr 21 12:22 dns.log -rw-r--r-- 1 root root 283K Apr 21 12:22 files.log -rw-r--r-- 1 root root 191K Apr 21 12:22 http.log -rw-r--r-- 1 root root 0 Apr 21 03:39 loaded_scripts.log -rw-r--r-- 1 root root 784 Apr 20 20:42 notice.log -rw-r--r-- 1 root root 0 Apr 21 03:39 packet_filter.log -rw-r--r-- 1 root root 0 Apr 21 03:39 reporter.log -rw-r--r-- 1 root root 42K Apr 21 12:03 sip.log -rw-r--r-- 1 root root 36K Apr 21 12:21 ssl.log -rw-r--r-- 1 root root 190K Apr 21 12:19 stats.log -rw-r--r-- 1 root root 0 Apr 20 13:28 stderr.log -rw-r--r-- 1 root root 0 Apr 21 03:39 stdout.log -rw-r--r-- 1 root root 0 Apr 21 03:39 top_dns.log -rw-r--r-- 1 root root 6.1M Apr 21 12:22 weird.log -rw-r--r-- 1 root root 1.3K Apr 21 02:26 x509.log [raub at testcentos log]$ From akgraner at corelight.com Tue Apr 21 06:15:52 2020 From: akgraner at corelight.com (Amber Graner) Date: Tue, 21 Apr 2020 09:15:52 -0400 Subject: [Zeek] [Reminder]- Zeek From Home - Zeek Package Contest Getting Started - Amber Graner, Seth Hall Message-ID: Hi all, If you're interested in the Zeek Package Contest or have questions about writing Zeek Packages then tomorrow's webinar is for you. You are invited to a Zoom webinar. *When:* Apr 22, 2020 02:00 PM Eastern Time (US and Canada) *Topic:* Zeek From Home - Zeek Package Contest Getting Started - Amber Graner, Seth Hall ****THIS LINK HAS BEEN UPDATED***:* Register in advance for this webinar: https://corelight.zoom.us/webinar/register/WN_bJgqTBXTSAebz4N6QWJScg After registering, you will receive a confirmation email containing information about joining the webinar. Thanks, ~Amber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200421/4aae3d87/attachment.html From raubvogel at gmail.com Tue Apr 21 09:10:06 2020 From: raubvogel at gmail.com (Mauricio Tavares) Date: Tue, 21 Apr 2020 12:10:06 -0400 Subject: [Zeek] Letting system handle log rotation In-Reply-To: References: Message-ID: On Tue, Apr 21, 2020 at 8:34 AM Mauricio Tavares wrote: > > On Mon, Apr 20, 2020 at 8:30 AM Mauricio Tavares wrote: > > > > I have the system's syslog to do the log rotation, including > > renaming, just the way I want. If I set LogRotationInterval = 0, would > > zeek then let the system do its thing? > > Got it to work: > > [raub at testcentos log]$ sudo ls -lh /var/log/bro/old > total 16M > -rw-r--r-- 1 root root 14K Apr 21 03:39 capture_loss.log-20200421 > -rw-r--r-- 1 root root 3.6M Apr 21 03:39 communication.log-20200421 > -rw-r--r-- 1 root root 6.4M Apr 21 03:39 conn.log-20200421 > -rw-r--r-- 1 root root 970K Apr 21 03:39 dns.log-20200421 > -rw-r--r-- 1 root root 177K Apr 21 03:39 files.log-20200421 > -rw-r--r-- 1 root root 120K Apr 21 03:39 http.log-20200421 > -rw-r--r-- 1 root root 27K Apr 21 03:39 loaded_scripts.log-20200421 > -rw-r--r-- 1 root root 187 Apr 21 03:39 packet_filter.log-20200421 > -rw-r--r-- 1 root root 529 Apr 21 03:39 reporter.log-20200421 > -rw-r--r-- 1 root root 30K Apr 21 03:39 sip.log-20200421 > -rw-r--r-- 1 root root 24K Apr 21 03:39 ssl.log-20200421 > -rw-r--r-- 1 root root 118K Apr 21 03:39 stats.log-20200421 > -rw-r--r-- 1 root root 188 Apr 21 03:39 stdout.log-20200421 > -rw-r--r-- 1 root root 580 Apr 21 03:39 top_dns.log-20200421 > -rw-r--r-- 1 root root 3.8M Apr 21 03:39 weird.log-20200421 > [raub at testcentos log]$ sudo ls -lh /var/log/bro/current > total 12M > -rw-r--r-- 1 root root 22K Apr 21 12:13 capture_loss.log > -rw-r--r-- 1 root root 5.7M Apr 21 12:22 communication.log > -rw-r--r-- 1 root root 11M Apr 21 12:22 conn.log > -rw-r--r-- 1 root root 1.6M Apr 21 12:22 dns.log > -rw-r--r-- 1 root root 283K Apr 21 12:22 files.log > -rw-r--r-- 1 root root 191K Apr 21 12:22 http.log > -rw-r--r-- 1 root root 0 Apr 21 03:39 loaded_scripts.log > -rw-r--r-- 1 root root 784 Apr 20 20:42 notice.log > -rw-r--r-- 1 root root 0 Apr 21 03:39 packet_filter.log > -rw-r--r-- 1 root root 0 Apr 21 03:39 reporter.log > -rw-r--r-- 1 root root 42K Apr 21 12:03 sip.log > -rw-r--r-- 1 root root 36K Apr 21 12:21 ssl.log > -rw-r--r-- 1 root root 190K Apr 21 12:19 stats.log > -rw-r--r-- 1 root root 0 Apr 20 13:28 stderr.log > -rw-r--r-- 1 root root 0 Apr 21 03:39 stdout.log > -rw-r--r-- 1 root root 0 Apr 21 03:39 top_dns.log > -rw-r--r-- 1 root root 6.1M Apr 21 12:22 weird.log > -rw-r--r-- 1 root root 1.3K Apr 21 02:26 x509.log > [raub at testcentos log]$ But now I broke mail summary: # Mail connection summary reports each log rotation interval. A value of 1 # means mail connection summaries, and a value of 0 means do not mail # connection summaries. This option has no effect if the trace-summary # script is not available. MailConnectionSummary = 1 From nabilmemon.ec at gmail.com Wed Apr 22 00:13:27 2020 From: nabilmemon.ec at gmail.com (Nabil Memon) Date: Wed, 22 Apr 2020 12:43:27 +0530 Subject: [Zeek] Flow stats in dns_end event Message-ID: Hi, event dns_end(c: connection, msg: dns_msg) { print "dns_end: -------------------------------"; print c$orig, c$resp, c$id, c$dns; print "dns_end: -------------------------------"; } Output: ==================================================== Request: dns_end: ------------------------------- [size=32, state=1, *num_pkts=0, num_bytes_ip=0*, flow_label=0, l2_addr=00:e0:18:b1:0c:ad], [size=0, state=0, num_pkts=0, num_bytes_ip=0, flow_label=0, l2_addr=00:c0:9f:32:41:8c] Request: dns_end: ------------------------------- ==================================================== ==================================================== Reply: dns_end: ------------------------------- dns_end: ------------------------------- [size=32, state=1, num_pkts=1, num_bytes_ip=60, flow_label=0, l2_addr=00:e0:18:b1:0c:ad], [size=60, state=1, *num_pkts=0, num_bytes_ip=0*, flow_label=0, l2_addr=00:c0:9f:32:41:8c] Reply: dns_end: ------------------------------- ==================================================== When bro sees DNS request/reply, it raises dns_end() event for both the packets at the end. In the reply packet's DNS event I see flow stats info is 0 in c$orig and c$resp as highlighted. Stats gets updated in connection record after dns_end() event raised???? I have a use case, where I want to gather DNS req/rep data with the flow stats, but I see dns object from connection record is deleted in the last dns_end() event inside dns main.bro file. I assume the reason of same 5 tuple being used for different DNS exchanges. Regards, Nabil -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200422/a3eb1064/attachment.html From jsiwek at corelight.com Wed Apr 22 10:22:28 2020 From: jsiwek at corelight.com (Jon Siwek) Date: Wed, 22 Apr 2020 10:22:28 -0700 Subject: [Zeek] Flow stats in dns_end event In-Reply-To: References: Message-ID: On Wed, Apr 22, 2020 at 12:15 AM Nabil Memon wrote: > Stats gets updated in connection record after dns_end() event raised???? Yes, the "dns_end" event and associated values can get created/enqueued before the states are updated. However, the updated stats *are* technically available by the time the event gets dispatched, so one trick to refresh the connection record value would be to use "lookup_connection()". See if this helps: event dns_end(c: connection, msg: dns_msg) { c = lookup_connection(c$id); print "dns_end: -------------------------------"; print c$orig, c$resp, c$id, c$dns; print "dns_end: -------------------------------"; } - Jon From nabilmemon.ec at gmail.com Wed Apr 22 11:54:42 2020 From: nabilmemon.ec at gmail.com (Nabil Memon) Date: Thu, 23 Apr 2020 00:24:42 +0530 Subject: [Zeek] Flow stats in dns_end event In-Reply-To: References: Message-ID: Awesome, thanks! It worked. Nabil On Wed, Apr 22, 2020 at 10:52 PM Jon Siwek wrote: > On Wed, Apr 22, 2020 at 12:15 AM Nabil Memon > wrote: > > > Stats gets updated in connection record after dns_end() event raised???? > > Yes, the "dns_end" event and associated values can get > created/enqueued before the states are updated. However, the updated > stats *are* technically available by the time the event gets > dispatched, so one trick to refresh the connection record value would > be to use "lookup_connection()". See if this helps: > > event dns_end(c: connection, msg: dns_msg) > { > c = lookup_connection(c$id); > print "dns_end: -------------------------------"; > print c$orig, c$resp, c$id, c$dns; > print "dns_end: -------------------------------"; > } > > - Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200423/55d03311/attachment.html From vincyforce at gmail.com Thu Apr 23 05:35:33 2020 From: vincyforce at gmail.com (Vincenzo) Date: Thu, 23 Apr 2020 14:35:33 +0200 Subject: [Zeek] Converting Rule suricata to zeek Message-ID: I am working on a suricata signature converter and converting them for Zeek, starting from this development https://github.com/adi928/brocata (which currently does not work), and I am doing various bug fixing and expanding it. But I have only one problem, it concerns the conversion of the rules containing the suricata pcre into expressions compatible with zeek ("flex"). has anyone ever approached this development and could you give me some advice? Anyone knows other development for this scope? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200423/25d72ddc/attachment.html From richard at corelight.com Thu Apr 23 06:29:26 2020 From: richard at corelight.com (Richard Bejtlich) Date: Thu, 23 Apr 2020 09:29:26 -0400 Subject: [Zeek] Converting Rule suricata to zeek In-Reply-To: References: Message-ID: Hi Vincenzo, I am not a developer, so I can't comment on the programming aspects. However, from what little I know about the optimizations and use cases for Zeek compared to Suricata, it makes sense to run each tool in the manner for which it was designed. In other words, depending on the number of signatures you want to port to Zeek, and that they work as expected, it's possible you will cripple your Zeek deployment. Can you tell us a little bit more about your expected use case? It might be better to just run both tools in parallel. Sincerely, Richard On Thu, Apr 23, 2020 at 8:38 AM Vincenzo wrote: > I am working on a suricata signature converter and converting them for > Zeek, starting from this development https://github.com/adi928/brocata > (which currently does not work), and I am doing various bug fixing and > expanding it. > But I have only one problem, it concerns the conversion of the rules > containing the suricata pcre into expressions compatible with zeek ("flex"). > has anyone ever approached this development and could you give me some > advice? > > Anyone knows other development for this scope? > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Richard Bejtlich Principal Security Strategist, Corelight https://corelight.blog/author/richardbejtlich/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200423/dc356563/attachment.html From vincyforce at gmail.com Thu Apr 23 06:49:18 2020 From: vincyforce at gmail.com (Vincenzo) Date: Thu, 23 Apr 2020 15:49:18 +0200 Subject: [Zeek] Converting Rule suricata to zeek In-Reply-To: References: Message-ID: Hi download a tar from this emerging threaths https://rules.emergingthreats.net/open/suricata-5.0/ and Zeek has loaded all signature (29670) excluding pcre option from suricata rule, but i included (content,ip,port,flow,nocase of content etc), and Zeek rose correctly. Yes, I know they are tools that are made to work in parallel, but these are the design requirements. Il giorno gio 23 apr 2020 alle ore 15:29 Richard Bejtlich < richard at corelight.com> ha scritto: > Hi Vincenzo, > > I am not a developer, so I can't comment on the programming aspects. > However, from what little I know about the optimizations and use cases for > Zeek compared to Suricata, it makes sense to run each tool in the manner > for which it was designed. > > In other words, depending on the number of signatures you want to port to > Zeek, and that they work as expected, it's possible you will cripple your > Zeek deployment. Can you tell us a little bit more about your expected use > case? It might be better to just run both tools in parallel. > > Sincerely, > > Richard > > On Thu, Apr 23, 2020 at 8:38 AM Vincenzo wrote: > >> I am working on a suricata signature converter and converting them for >> Zeek, starting from this development https://github.com/adi928/brocata >> (which currently does not work), and I am doing various bug fixing and >> expanding it. >> But I have only one problem, it concerns the conversion of the rules >> containing the suricata pcre into expressions compatible with zeek ("flex"). >> has anyone ever approached this development and could you give me some >> advice? >> >> Anyone knows other development for this scope? >> _______________________________________________ >> Zeek mailing list >> zeek at zeek.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > > > -- > Richard Bejtlich > Principal Security Strategist, Corelight > https://corelight.blog/author/richardbejtlich/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200423/1330488f/attachment.html From akgraner at corelight.com Thu Apr 23 07:22:48 2020 From: akgraner at corelight.com (Amber Graner) Date: Thu, 23 Apr 2020 10:22:48 -0400 Subject: [Zeek] Ask The Zeeksperts - Reminders and Ideas Message-ID: Hi all, Thank you to everyone who "drops by" the Ask The Zeeksperts calls and just joins the conversation by bringing your questions. It's great to collaborate and hear your questions! ===Reminder Webinar today=== We have another Ask The Zeeksperts call today at 3:30pm Eastern/12:30pm Pacific as well. If you'd like to join here is the registration link: https://attendee.gotowebinar.com/register/1763308093940786957 Feel free to send over your questions in advance. You can also drop them into the #webinar channel on slack. ===Ask The Zeeksperts Topics/Ideas Wanted === While the idea of these webinars was for users to be able to have a place/time with "office hours" to bring your Zeek related questions too, we also been asked to theme these from time to time. What topics/themes would you like to see us base these Q&A bi weekly sessions on? Also, is there a specific Zeek related topic that you're comfortable with people asking you questions about? Please let me know and I'd be happy to get you scheduled. Thanks again everyone! Let me know if you have any questions. With gratitude, ~Amber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200423/58a898f8/attachment-0001.html From patrick.kelley at criticalpathsecurity.com Fri Apr 24 07:40:43 2020 From: patrick.kelley at criticalpathsecurity.com (Patrick Kelley) Date: Fri, 24 Apr 2020 10:40:43 -0400 Subject: [Zeek] COVID-19 CTI LEAGUE and CRITICAL PATH SECURITY Intel feed Message-ID: Community, We wanted to share an updated COVID-19 threat feed for Zeek. It includes COVID-19 CTI public data, our collection from dns.log, as well as data from PREDICT. It will be updated as often as possible. https://github.com/CriticalPathSecurity/COVID-THREAT-INTEL-PUBLIC-ZEEK/blob/master/README.md -- *Patrick Kelley, CISSP, C|EH, ITIL* *CTO* patrick.kelley at criticalpathsecurity.com (o) 770-224-6482 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200424/8b8d8017/attachment.html From hovsep.sanjay.levi at gmail.com Sun Apr 26 18:10:58 2020 From: hovsep.sanjay.levi at gmail.com (Hovsep Levi) Date: Mon, 27 Apr 2020 01:10:58 +0000 Subject: [Zeek] Zeek 3.1.2 and Kafka - No data flow Message-ID: Hello Zeeks Has anyone succeeded to enable Kafka plugin with Zeek 3.1.2 ? I am trying to modernize the metron-kafka plugin and have partial success. My problem seems to be with script-land referencing. The logger node is loading the plugin OK and connects to the Kafka broker. The broker IP is redef information found from site/local.zeek. $ bin/zeekctl diag logger-1 [logger-1] No core file found. Zeek 3.1.2-debug Zeek plugins: Apache::Kafka - Writes logs to Kafka (dynamic, version 0.3.0) ==== No reporter.log ==== stderr.log %7|1587948661.341|RECV|rdkafka#producer-3| [..kafka messages..] .. But the worker node has a problem referencing existing variable declaration. The logs-to-kafka.bro script expects it. There is also suspicion with the Zeek plugins info that is different from the logger node and maybe the problem. $ bin/zeekctl diag worker-1-1 [worker-1-1] No core file found. Zeek 3.1.2-debug Zeek plugins: (none found) <<< ??? Normal for worker node ??? ==== No reporter.log ==== stderr.log error in /opt/zeek/spool/installed-scripts-do-not-touch/site/custom_plugins/APACHE_KAFKA/scripts/Apache/Kafka/./logs-to-kafka.bro, line 24: unknown identifier logs_to_send, at or near "logs_to_send" The configuration is not default and explained below: The Kafka logger was installed to site/custom_plugins/APACHE_KAFKA share/zeek/site/local.zeek uses: @load custom_plugins/APACHE_KAFKA/scripts/Apache/Kafka lib/zeek/plugins/custom_plugins is a symlink to share/zeek/site/custom_plugins Using the lib symlink seems to be the only way to load the plugin, then the @load statement brings redef customizations and scripts. This works ok for the logger node but not the worker who cannot interface with the plugin ? Another idea is have non-logger nodes bypass loading logs-to-kafka.bro but this isn't fully understood. TIA /hovsep -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200427/e9f7b683/attachment.html From zeolla at gmail.com Mon Apr 27 03:39:29 2020 From: zeolla at gmail.com (Zeolla@GMail.com) Date: Mon, 27 Apr 2020 06:39:29 -0400 Subject: [Zeek] Zeek 3.1.2 and Kafka - No data flow In-Reply-To: References: Message-ID: I have not run it on 3.1.2 yet but I recommend making your changes to the plugin and running the end to end testing script at https://github.com/apache/metron-bro-plugin-kafka/blob/master/docker/run_end_to_end.sh It was meant to help isolate issues when making changes to the plugin. Also, we welcome PRs against the project so please feel free to contribute. Thanks, Jon Zeolla On Sun, Apr 26, 2020, 9:12 PM Hovsep Levi wrote: > Hello Zeeks > > > Has anyone succeeded to enable Kafka plugin with Zeek 3.1.2 ? I am trying > to modernize the metron-kafka plugin and have partial success. My problem > seems to be with script-land referencing. > > The logger node is loading the plugin OK and connects to the Kafka > broker. The broker IP is redef information found from site/local.zeek. > > $ bin/zeekctl diag logger-1 > [logger-1] > > No core file found. > > Zeek 3.1.2-debug > > Zeek plugins: > Apache::Kafka - Writes logs to Kafka (dynamic, version 0.3.0) > > ==== No reporter.log > > ==== stderr.log > %7|1587948661.341|RECV|rdkafka#producer-3| [..kafka messages..] .. > > > > > But the worker node has a problem referencing existing variable > declaration. The logs-to-kafka.bro script expects it. There is also > suspicion with the Zeek plugins info that is different from the logger node > and maybe the problem. > > $ bin/zeekctl diag worker-1-1 > [worker-1-1] > > No core file found. > > Zeek 3.1.2-debug > > Zeek plugins: (none found) <<< ??? Normal for worker node ??? > > ==== No reporter.log > > ==== stderr.log > > error in > /opt/zeek/spool/installed-scripts-do-not-touch/site/custom_plugins/APACHE_KAFKA/scripts/Apache/Kafka/./logs-to-kafka.bro, > line 24: unknown identifier logs_to_send, at or near "logs_to_send" > > > > The configuration is not default and explained below: > > > The Kafka logger was installed to site/custom_plugins/APACHE_KAFKA > > > share/zeek/site/local.zeek uses: > > @load custom_plugins/APACHE_KAFKA/scripts/Apache/Kafka > > > > lib/zeek/plugins/custom_plugins is a symlink to > share/zeek/site/custom_plugins > > > Using the lib symlink seems to be the only way to load the plugin, then > the @load statement brings redef customizations and scripts. This works ok > for the logger node but not the worker who cannot interface with the plugin > ? > > Another idea is have non-logger nodes bypass loading logs-to-kafka.bro but > this isn't fully understood. > > > TIA > > /hovsep > > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200427/835fc3bd/attachment.html From Kayode_Enwerem at ao.uscourts.gov Mon Apr 27 14:08:08 2020 From: Kayode_Enwerem at ao.uscourts.gov (Kayode Enwerem) Date: Mon, 27 Apr 2020 21:08:08 +0000 Subject: [Zeek] File extraction package Message-ID: Hello, We are trying to do some customization to the file extraction package https://github.com/hosom/file-extraction Does any one have any suggestions on how I can get any of these done? 1. Is there a way to define what network you want the "file extracting package" to extract the files from? Instead of extracting files from all the networks defined in network.cfg. Example: if I have 7 subnets defined in network.cfg but I only the file extracting package to extract files from 2 out of the 7. 2. Is there a way to dedup the extracted files. Example: If a file was sent to 20 people, I only want to see the file 1 time instead of 20 times. 3. We would also like to exclude certain file types based coming via SMB. Example: excluding all .pdf files I just want to exclude .pdf files coming via SMB. Zeek version we are running is 3.0.3. Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200427/8072b508/attachment.html From akgraner at corelight.com Tue Apr 28 14:09:06 2020 From: akgraner at corelight.com (Amber Graner) Date: Tue, 28 Apr 2020 17:09:06 -0400 Subject: [Zeek] [Reminder] Zeek Package Contest Still Open Message-ID: Hi all, The Zeek Package Contest (ZPC-2) is still open. There is still time to potentially win some cash and prizes. Head on over to the Zeek Blog to find out more information: https://zeek.org/2020/04/06/zeek-package-contest-zpc-2/ If you have any questions please let me know. Thanks, ~Amber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200428/6b90379c/attachment.html From vincyforce at gmail.com Wed Apr 29 03:46:34 2020 From: vincyforce at gmail.com (Vincenzo) Date: Wed, 29 Apr 2020 12:46:34 +0200 Subject: [Zeek] More in payload in a signature Message-ID: Hi everyone, in Zeek's signature framework, is it possible to set multiple payloads in "AND" and not in "OR" within a signature? Example my-first-sig signature { ip-proto == tcp dst-port == 80 payload /.*root/ payload / Hello / event "Found root!" } >From the tests carried out it seems that the two payloads are in 'OR' and not in 'AND' conditions, do you have any suggestions? Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200429/d9bbd856/attachment.html From jlay at slave-tothe-box.net Wed Apr 29 08:13:03 2020 From: jlay at slave-tothe-box.net (James Lay) Date: Wed, 29 Apr 2020 09:13:03 -0600 Subject: [Zeek] More in payload in a signature In-Reply-To: References: Message-ID: <5f092fa1f0313ddf9fd700d064013c0d821e7ff3.camel@slave-tothe-box.net> On Wed, 2020-04-29 at 12:46 +0200, Vincenzo wrote: > Hi everyone, in Zeek's signature framework, is it possible to set > multiple payloads in "AND" and not in "OR" within a signature? > Example > my-first-sig signature { > ip-proto == tcp > dst-port == 80 > payload /.*root/ > payload / Hello / > > event "Found root!" > } > > From the tests carried out it seems that the two payloads are in 'OR' > and not in 'AND' conditions, do you have any suggestions? > Thank you > > _______________________________________________Zeek mailing > listzeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek Not that I found. Have to make two separate sigs. James -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200429/50955828/attachment.html From justin at corelight.com Wed Apr 29 10:02:53 2020 From: justin at corelight.com (Justin Azoff) Date: Wed, 29 Apr 2020 13:02:53 -0400 Subject: [Zeek] File extraction package In-Reply-To: References: Message-ID: On Mon, Apr 27, 2020 at 5:10 PM Kayode Enwerem wrote: > > Hello, > > We are trying to do some customization to the file extraction package https://github.com/hosom/file-extraction > > Does any one have any suggestions on how I can get any of these done? > > Is there a way to define what network you want the ?file extracting package? to extract the files from? Instead of extracting files from all the networks defined in network.cfg. Example: if I have 7 subnets defined in network.cfg but I only the file extracting package to extract files from 2 out of the 7. yes, just make a set[subnet] and add the networks you want to it. the networks.cfg just auto generates one for you called Site::local_nets > Is there a way to dedup the extracted files. Example: If a file was sent to 20 people, I only want to see the file 1 time instead of 20 times. easiest way to do this part is to just name the file the hash, but you could track recent files with a set[string]. > We would also like to exclude certain file types based coming via SMB. Example: excluding all .pdf files I just want to exclude .pdf files coming via SMB. If you look at how the plugins in that package are written, they are just small scripts containing an if statement: https://github.com/hosom/file-extraction/blob/master/scripts/plugins/extract-pdf.zeek so you would just need something like const pdf_types: set[string] = { "application/pdf" }; hook FileExtraction::extract(f: fa_file, meta: fa_metadata) &priority=5 { if ( f$source != "SMB" && meta$mime_type in pdf_types ) break; } or keep extracting all pdfs and ignore the ones that come from smb. hook FileExtraction::ignore(f: fa_file, meta: fa_metadata) { if ( f$source == "SMB" && meta$mime_type in pdf_types ) break; } -- Justin From akgraner at corelight.com Wed Apr 29 11:48:13 2020 From: akgraner at corelight.com (Amber Graner) Date: Wed, 29 Apr 2020 14:48:13 -0400 Subject: [Zeek] [Reminder] - Community Call - Friday 1 May 3pm ET Message-ID: Hi all, Just a reminder that we have our Monthly call this Friday at 3pm Eastern. Please LET ME KNOW IF YOU NEED AN INVITE to the call. I'll get add you added and send you the Zoom link. I'll update the links for future calls so it's not just an open Zoom link. Thanks in advance for understanding. ==APRIL CALL SUMMARY== For those who weren't able to attend last month below is a summary of that call. Unfortunately, I wasn't able to record that call; however, we will be able to record the call this Friday. ===AGENDA=== * ZeekWeek 2020 - Cancellation ( https://zeek.org/2020/03/31/zeekweek-2020-austin-cancelled-open-letter-to-the-community/ ) - Virtual - We are looking at options for holding a virtual ZeekWeek. Suggestions were to make sure it wasn't an all day event and to give options that made it easy to both present and view for varied time zones. - In-Person Different Location - As it gets closer to Oct we will look at the possibility of holding a smaller in-person event possibly in Santa Clara or other locations in the Bay Area (if it is safe to do so. * Zeek From Home - Webinar Series ( https://zeek.org/2020/03/31/zeek-from-home/) - What is it? - A weekly one hour recorded webinar series meant to highlight anything Zeek related to include adjacent technologies. - Submission Criteria - It doesn?t matter where you get your Zeek: The Zeek Project, Security Onion, Rock NSM, Bricata, Brim Security, Corelight or others. Consider submitting a talk for this webinar series and share what you?ve learned, best practice, challenges or tips and tricks. (As long as it's not a sales/marketing or product pitch. See link above for more details.) - Schedule - Wednesdays at 2pm Eastern. However, We are still working on the details and currently scheduling for May, June and July. If you would like to give a talk please let me know and we'll get the details worked out. We'll do weekly if we have enough talks. * Zeek Package Contest (ZPC-2) ( https://zeek.org/2020/04/06/zeek-package-contest-zpc-2/) - Focus - The ZPC-2 contest will focus on the MITRE ATT&CK? Framework, more specifically packages that help detect C2 Techniques. - Timeline - o Submission opens: April 6, 2020 o Submission deadline: May 15, 2020 o Notification: June 1, 2020 o Announcement of results: June 15th, 2020 - Launch Date - 6 April (See link above for more details. ===NOTES=== We had about 17 people on the call. We discussed ZeekWeek, Zeek From Home and the package contest (notes for those items above). In addition to these topics we also discussed the following: * Unique and Interesting ways to Use Zeek: Also folks on the call suggested getting people to share how they use Zeek especially around interesting ways people can or are using Zeek data outside of scripting or network security monitoring. Do you use Zeek to model threats? * Community Questions into Blog Posts: We also discussed turning mailinglist and slack questions into blog posts. A couple people from the community have offered to help with this., but if you would like to help and be a guest blogger, please let me know. I'll be reaching out to folks as we make content plans for the Zeek Blog for May, June and July. * Updating tags and categories for Zeek Blog posts: When we switched to the new site a lot of the blog posts aren't tagged or categorized. In order to make sure when people search for various Zeek related topics we want to make sure related blog posts show up in the searches. Thank you to those who volunteered, we're getting everything ready to give you access and plan out the tasks. ***THANK YOU*** so much to all those who attend each month. Please invite others and let me know if there are topics you'd like to see discussed on these calls. As a reminder: These calls are not for Zeek Technical/Development discussions, but for how we can all come together to make the community stronger and build awareness around The Zeek Project. If you have ideas or would like to get more involved please let me know. With gratitude, ~Amber -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200429/09d90c81/attachment.html From zeolla at gmail.com Wed Apr 29 12:32:50 2020 From: zeolla at gmail.com (Zeolla@GMail.com) Date: Wed, 29 Apr 2020 15:32:50 -0400 Subject: [Zeek] kafka plugin silently fails In-Reply-To: References: Message-ID: I didn't take a close look but I believe this is a known bug. Take a look at https://github.com/apache/metron-bro-plugin-kafka/pull/40 I followed up on that PR to see if we can get it merged. - Jon Zeolla Zeolla at GMail.Com On Wed, Apr 8, 2020 at 3:40 PM Erich M Nahum wrote: > Howdy, > > I'm currently using the latest kafka package manager on Bro 3.0.2. The > plugin is configured to send to two kafka brokers. Unfortunately, it seems > to work for some time and then quietly stops sending to the second broker. > > Here's my zeek config for kafka: > > @load packages/metron-bro-plugin-kafka > redef Kafka::topic_name = ""; > redef Kafka::kafka_conf = table( > ["metadata.broker.list"] = "broker1:9092, broker2:9092" > ); > > event zeek_init() > { > local protocol_list = table( > ["conn" ] = Conn::LOG, > ["dhcp" ] = DHCP::LOG, > ["dns" ] = DNS::LOG, > ["ftp" ] = FTP::LOG, > ["http"] = HTTP::LOG, > ["ssl"] = SSL::LOG, > ["x509"] = X509::LOG > ); > for (proto, log_id in protocol_list ) { > local this_filter: Log::Filter = [ > $name = "kafka-" + proto, > $writer = Log::WRITER_KAFKAWRITER, > $config = table( > ["metadata.broker.list"] = "broker1:9092, broker2:9092" > ), > $path = proto > ]; > Log::add_filter(log_id, this_filter); > } > } > > > Does anyone see anything wrong with my config? It works fine for a single > broker. > > I notice the failure since all the netstat entries disappear for broker2, > and it stops receiving data. Broker1 is fine. > > Thanks, > > -Erich > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200429/9b3c4c60/attachment.html From zeolla at gmail.com Wed Apr 29 12:35:01 2020 From: zeolla at gmail.com (Zeolla@GMail.com) Date: Wed, 29 Apr 2020 15:35:01 -0400 Subject: [Zeek] Zeek 3.1.2 and Kafka - No data flow In-Reply-To: References: Message-ID: Were you able to get this working? I'm planning to work on the bro to zeek cutover for the plugin soon. - Jon Zeolla Zeolla at GMail.Com On Mon, Apr 27, 2020 at 6:39 AM Zeolla at GMail.com wrote: > I have not run it on 3.1.2 yet but I recommend making your changes to the > plugin and running the end to end testing script at > https://github.com/apache/metron-bro-plugin-kafka/blob/master/docker/run_end_to_end.sh > > It was meant to help isolate issues when making changes to the plugin. > Also, we welcome PRs against the project so please feel free to > contribute. Thanks, > > Jon Zeolla > > On Sun, Apr 26, 2020, 9:12 PM Hovsep Levi > wrote: > >> Hello Zeeks >> >> >> Has anyone succeeded to enable Kafka plugin with Zeek 3.1.2 ? I am >> trying to modernize the metron-kafka plugin and have partial success. My >> problem seems to be with script-land referencing. >> >> The logger node is loading the plugin OK and connects to the Kafka >> broker. The broker IP is redef information found from site/local.zeek. >> >> $ bin/zeekctl diag logger-1 >> [logger-1] >> >> No core file found. >> >> Zeek 3.1.2-debug >> >> Zeek plugins: >> Apache::Kafka - Writes logs to Kafka (dynamic, version 0.3.0) >> >> ==== No reporter.log >> >> ==== stderr.log >> %7|1587948661.341|RECV|rdkafka#producer-3| [..kafka messages..] .. >> >> >> >> >> But the worker node has a problem referencing existing variable >> declaration. The logs-to-kafka.bro script expects it. There is also >> suspicion with the Zeek plugins info that is different from the logger node >> and maybe the problem. >> >> $ bin/zeekctl diag worker-1-1 >> [worker-1-1] >> >> No core file found. >> >> Zeek 3.1.2-debug >> >> Zeek plugins: (none found) <<< ??? Normal for worker node ??? >> >> ==== No reporter.log >> >> ==== stderr.log >> >> error in >> /opt/zeek/spool/installed-scripts-do-not-touch/site/custom_plugins/APACHE_KAFKA/scripts/Apache/Kafka/./logs-to-kafka.bro, >> line 24: unknown identifier logs_to_send, at or near "logs_to_send" >> >> >> >> The configuration is not default and explained below: >> >> >> The Kafka logger was installed to site/custom_plugins/APACHE_KAFKA >> >> >> share/zeek/site/local.zeek uses: >> >> @load custom_plugins/APACHE_KAFKA/scripts/Apache/Kafka >> >> >> >> lib/zeek/plugins/custom_plugins is a symlink to >> share/zeek/site/custom_plugins >> >> >> Using the lib symlink seems to be the only way to load the plugin, then >> the @load statement brings redef customizations and scripts. This works ok >> for the logger node but not the worker who cannot interface with the plugin >> ? >> >> Another idea is have non-logger nodes bypass loading logs-to-kafka.bro >> but this isn't fully understood. >> >> >> TIA >> >> /hovsep >> >> >> >> _______________________________________________ >> Zeek mailing list >> zeek at zeek.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200429/1c88a345/attachment.html From Kayode_Enwerem at ao.uscourts.gov Wed Apr 29 12:59:34 2020 From: Kayode_Enwerem at ao.uscourts.gov (Kayode Enwerem) Date: Wed, 29 Apr 2020 19:59:34 +0000 Subject: [Zeek] File extraction package In-Reply-To: References: Message-ID: Thanks for the response Justin. How do I make a "set[subnet]" and what file do I add it in? -----Original Message----- From: Justin Azoff Sent: Wednesday, April 29, 2020 1:03 PM To: Kayode Enwerem Cc: zeek at zeek.org Subject: Re: [Zeek] File extraction package On Mon, Apr 27, 2020 at 5:10 PM Kayode Enwerem wrote: > > Hello, > > We are trying to do some customization to the file extraction package > https://github.com/hosom/file-extraction > > Does any one have any suggestions on how I can get any of these done? > > Is there a way to define what network you want the ?file extracting package? to extract the files from? Instead of extracting files from all the networks defined in network.cfg. Example: if I have 7 subnets defined in network.cfg but I only the file extracting package to extract files from 2 out of the 7. yes, just make a set[subnet] and add the networks you want to it. the networks.cfg just auto generates one for you called Site::local_nets > Is there a way to dedup the extracted files. Example: If a file was sent to 20 people, I only want to see the file 1 time instead of 20 times. easiest way to do this part is to just name the file the hash, but you could track recent files with a set[string]. > We would also like to exclude certain file types based coming via SMB. Example: excluding all .pdf files I just want to exclude .pdf files coming via SMB. If you look at how the plugins in that package are written, they are just small scripts containing an if statement: https://github.com/hosom/file-extraction/blob/master/scripts/plugins/extract-pdf.zeek so you would just need something like const pdf_types: set[string] = { "application/pdf" }; hook FileExtraction::extract(f: fa_file, meta: fa_metadata) &priority=5 { if ( f$source != "SMB" && meta$mime_type in pdf_types ) break; } or keep extracting all pdfs and ignore the ones that come from smb. hook FileExtraction::ignore(f: fa_file, meta: fa_metadata) { if ( f$source == "SMB" && meta$mime_type in pdf_types ) break; } -- Justin From fatema.bannatwala at gmail.com Thu Apr 30 09:48:13 2020 From: fatema.bannatwala at gmail.com (fatema bannatwala) Date: Thu, 30 Apr 2020 09:48:13 -0700 Subject: [Zeek] [Reminder] - Community Call - Friday 1 May 3pm ET (Amber Graner) Message-ID: Hi All! Just a humble reminder, Zeek Package Contest is still on and great prizes are up for winning! A small back story, last year I was already thinking of a problem that I wanted to solve with Zeek and it was getting procrastinated as it wasn't high priority task that time, but when I saw ZPC'19 announced (and frankly the prizes!) it kicked off the motivation to finish it and get submitted for the consideration for the contest! *That was I think one of my quickest Zeek packages that I wrote and deployed in production* ;-) So, if you are already thinking/working towards a challenge to solve with Zeek (this time the theme is focused on the C2 techniques), then it's the time to take it up and finish it, and submit it towards ZPC'20! Lastly, here's the details that Amber shared in her previous email regarding ZPC'20: * Zeek Package Contest (ZPC-2) ( https://zeek.org/2020/04/06/zeek-package-contest-zpc-2/) - Focus - The ZPC-2 contest will focus on the MITRE ATT&CK? Framework, more specifically packages that help detect C2 Techniques. - Timeline - o Submission opens: April 6, 2020 o Submission deadline: May 15, 2020 o Notification: June 1, 2020 o Announcement of results: June 15th, 2020 - Launch Date - 6 April (See link above for more details. *Zeek and you shall find!* -Fatema -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200430/ddfb7967/attachment.html From Kayode_Enwerem at ao.uscourts.gov Thu Apr 30 13:54:38 2020 From: Kayode_Enwerem at ao.uscourts.gov (Kayode Enwerem) Date: Thu, 30 Apr 2020 20:54:38 +0000 Subject: [Zeek] File extraction package In-Reply-To: References: Message-ID: Also is there a way to have Zeek organize my extracted files on an hourly basis. So I want zeek to store all extracted files from each hour in a separate timestamped folder. I currently have the extracted files being stored in this directory: /logs/bro/spool/extracted_files/ Which I created and defined in: /usr/local/zeek/share/zeek/site/file-extraction/ config.zeek redef path = "/logs/bro/spool/extracted_files/"; -----Original Message----- From: zeek-bounces at zeek.org On Behalf Of Kayode Enwerem Sent: Wednesday, April 29, 2020 4:00 PM To: Justin Azoff Cc: zeek at zeek.org Subject: Re: [Zeek] File extraction package Thanks for the response Justin. How do I make a "set[subnet]" and what file do I add it in? -----Original Message----- From: Justin Azoff Sent: Wednesday, April 29, 2020 1:03 PM To: Kayode Enwerem Cc: zeek at zeek.org Subject: Re: [Zeek] File extraction package On Mon, Apr 27, 2020 at 5:10 PM Kayode Enwerem wrote: > > Hello, > > We are trying to do some customization to the file extraction package > https://github.com/hosom/file-extraction > > Does any one have any suggestions on how I can get any of these done? > > Is there a way to define what network you want the ?file extracting package? to extract the files from? Instead of extracting files from all the networks defined in network.cfg. Example: if I have 7 subnets defined in network.cfg but I only the file extracting package to extract files from 2 out of the 7. yes, just make a set[subnet] and add the networks you want to it. the networks.cfg just auto generates one for you called Site::local_nets > Is there a way to dedup the extracted files. Example: If a file was sent to 20 people, I only want to see the file 1 time instead of 20 times. easiest way to do this part is to just name the file the hash, but you could track recent files with a set[string]. > We would also like to exclude certain file types based coming via SMB. Example: excluding all .pdf files I just want to exclude .pdf files coming via SMB. If you look at how the plugins in that package are written, they are just small scripts containing an if statement: https://github.com/hosom/file-extraction/blob/master/scripts/plugins/extract-pdf.zeek so you would just need something like const pdf_types: set[string] = { "application/pdf" }; hook FileExtraction::extract(f: fa_file, meta: fa_metadata) &priority=5 { if ( f$source != "SMB" && meta$mime_type in pdf_types ) break; } or keep extracting all pdfs and ignore the ones that come from smb. hook FileExtraction::ignore(f: fa_file, meta: fa_metadata) { if ( f$source == "SMB" && meta$mime_type in pdf_types ) break; } -- Justin _______________________________________________ Zeek mailing list zeek at zeek.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek