From jsiwek at corelight.com Mon Dec 2 10:26:56 2019 From: jsiwek at corelight.com (Jon Siwek) Date: Mon, 2 Dec 2019 10:26:56 -0800 Subject: [Zeek] Absolute ack and seq number of tcp packet In-Reply-To: References: Message-ID: On Thu, Nov 28, 2019 at 11:38 AM Hui Lin (Hugo) wrote: > In the tcp_packet event, how can I obtain the absolute values (found in the tcp header), not the relative values of ack and seq numbers. The `get_current_packet_header()` BIF likely works for you: https://docs.zeek.org/en/stable/scripts/base/bif/zeek.bif.zeek.html#id-get_current_packet_header Or else the `raw_packet` event is also something that uses the `raw_pkt_hdr` type which should have the absolute sequence numbers. - Jon From mauro.palumbo at aizoon.it Tue Dec 3 02:14:08 2019 From: mauro.palumbo at aizoon.it (Palumbo Mauro) Date: Tue, 3 Dec 2019 10:14:08 +0000 Subject: [Zeek] R: tcp partial connections In-Reply-To: References: <09a8e3bfc8a34d56ae8439e193d219c4@SRVEX03.aizoon.local> Message-ID: Hi Jon, Thanks for your reply. As a follow up to my previous question here, I am analyzing zeek processing network traffic which is quite messy, lots of retransmissions, duplicate acks, etc. and a weird log reporting a lot of potential issues. We are looking into fixing these issues. In this context, tcp conns are detected as partials quite often by Zeek and several analyzers do not process correctly the related traffic. As I would like to get a better understanding of what is going on, I examined a single tcp+http connection in more details. Zeek starts processing it correctly at first, there is the tcp handshake and the first GET/POST are logged correctly. Then Zeek stops processing the http traffic. By debugging, I noticed that in the middle of the conn, zeek calls again the ctor of the tcp analyzer and and as a consequence resets "is_partial" and other variables. Again from debugging it seems this happens because of the timer TCPConnectionExpireTimer is dispatched at a certain point. Is this what you would expect in the middle of a conn? It doesn't seem to me that there is a long inactivity on this conn. Best wishes, Mauro -----Messaggio originale----- Da: Jon Siwek [mailto:jsiwek at corelight.com] Inviato: gioved? 28 novembre 2019 17:06 A: Palumbo Mauro Cc: zeek at zeek.org Oggetto: Re: [Zeek] tcp partial connections On Thu, Nov 28, 2019 at 7:06 AM Palumbo Mauro wrote: > if ( TCP() && TCP()->IsPartial() ) > return; > > This is true for example for the HTTP, SSH, SSL analyzers and more. My understanding is that this is to prevent app layer analyzers or scripts relying on them from breaking down or missing some information when processing packets with possible missing bytes. Mre related to the "breaking down" part: current protocol parsers don't have any type of "re-synchronization" mechanism so particularly if we miss the TCP handshake and assume we may be starting in the middle of the app-layer protocol stream (or else have a content gap), the parser won't know what to do with the incoming data and so the IsPartial() checks just exit early, before attempting to parse further. > How much reliable is this check TCP()->IsPartial() for partial tcp sessions in the tcp analyzer? Should be reliable in detecting the problematic scenario AFAIK, but in the case where just the TCP handshake packets are missing and not any segment data, analyzers that exit early like that are skipping streams they actually should be able to parse. - Jon From SHARRIS at hollywoodfl.org Tue Dec 3 06:24:01 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Tue, 3 Dec 2019 14:24:01 +0000 Subject: [Zeek] Cluster configuration zeekctl status hangs Message-ID: Install Zeek 3.0 on Centos 8. Have been working through the setup of zeek using two machines in a cluster. The cluster appears to be working. I can zeekctl install and zeekctl start the cluster. On the remote machine I see the workers start up. On the local machine the services and workers appear to startup. Remote machine: zeek 25985 1 0 08:58 ? 00:00:00 /usr/bin/bash /opt/zeek/share/zeekctl/scripts/run-zeek 3 -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-3-2 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 25986 1 0 08:58 ? 00:00:00 /usr/bin/bash /opt/zeek/share/zeekctl/scripts/run-zeek 2 -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-3-1 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 25990 1 0 08:58 ? 00:00:00 /usr/bin/bash /opt/zeek/share/zeekctl/scripts/run-zeek 4 -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-4-1 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 25992 1 0 08:58 ? 00:00:00 /usr/bin/bash /opt/zeek/share/zeekctl/scripts/run-zeek 5 -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-4-2 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 26012 25985 9 08:58 ? 00:01:31 /opt/zeek/bin/zeek -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-3-2 local.zee zeekctl base/frameworks/cluster zeekctl/auto zeek 26013 25986 9 08:58 ? 00:01:31 /opt/zeek/bin/zeek -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-3-1 local.zee zeekctl base/frameworks/cluster zeekctl/auto zeek 26016 25992 9 08:58 ? 00:01:31 /opt/zeek/bin/zeek -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-4-2 local.zee zeekctl base/frameworks/cluster zeekctl/auto zeek 26017 25990 9 08:58 ? 00:01:30 /opt/zeek/bin/zeek -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-4-1 local.zee zeekctl base/frameworks/cluster zeekctl/auto Local (manager) machine: zeek 8314 1 0 08:57 ? 00:00:00 /usr/bin/bash /opt/zeek/share/zeekctl/scripts/run-zeek -1 -U .status -p zeekctl -p zeekctl-live -p local -p logger local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8320 8314 5 08:57 ? 00:00:58 /opt/zeek/bin/zeek -U .status -p zeekctl -p zeekctl-live -p local -p logger local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8361 1 0 08:57 ? 00:00:00 /usr/bin/bash /opt/zeek/share/zeekctl/scripts/run-zeek -1 -U .status -p zeekctl -p zeekctl-live -p local -p manager local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8367 8361 5 08:57 ? 00:00:59 /opt/zeek/bin/zeek -U .status -p zeekctl -p zeekctl-live -p local -p manager local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8406 1 0 08:58 ? 00:00:00 /usr/bin/bash /opt/zeek/share/zeekctl/scripts/run-zeek -1 -U .status -p zeekctl -p zeekctl-live -p local -p proxy-1 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8412 8406 1 08:58 ? 00:00:21 /opt/zeek/bin/zeek -U .status -p zeekctl -p zeekctl-live -p local -p proxy-1 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8471 1 0 08:58 ? 00:00:00 /usr/bin/bash /opt/zeek/share/zeekctl/scripts/run-zeek 2 -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-1-1 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8474 1 0 08:58 ? 00:00:00 /usr/bin/bash /opt/zeek/share/zeekctl/scripts/run-zeek 3 -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-1-2 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8477 1 0 08:58 ? 00:00:00 /usr/bin/bash /opt/zeek/share/zeekctl/scripts/run-zeek 5 -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-2-2 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8479 1 0 08:58 ? 00:00:00 /usr/bin/bash /opt/zeek/share/zeekctl/scripts/run-zeek 4 -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-2-1 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8499 8471 17 08:58 ? 00:03:09 /opt/zeek/bin/zeek -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-1-1 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8502 8474 21 08:58 ? 00:03:47 /opt/zeek/bin/zeek -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-1-2 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8503 8477 17 08:58 ? 00:03:09 /opt/zeek/bin/zeek -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-2-2 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8504 8479 18 08:58 ? 00:03:17 /opt/zeek/bin/zeek -i af_packet::eno1 -U .status -p zeekctl -p zeekctl-live -p local -p worker-2-1 local.zeek zeekctl base/frameworks/cluster zeekctl/auto zeek 8593 3011 6 08:58 pts/0 00:01:04 /usr/bin/python3.6 /opt/zeek/bin/zeekctl status The problem is that when I run zeekctl status that request hangs: [zeek at heimdallr etc]$ zeekctl status Warning: ZeekControl plugin uses legacy BroControl API. Use 'import ZeekControl.plugin' instead of 'import BroControl.plugin' Getting process status ... Getting peer status ... Only way to resolve this is to kill process 8593. Any ideas on why this is hanging? Secondary problem with a work around available: Also have to follow the following steps for the cluster to work. 1. zeekctl install 2. setcap cap_net_raw=eip /opt/zeek/bin/zeek (on the remote peer) 3. zeekctl start Attempts to use zeekctl deploy does not work as the setcap command needs to be run on the remote peer after the install is completed. Running zeek 3.0. __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191203/68c47ee5/attachment-0001.html From justin at corelight.com Tue Dec 3 06:55:41 2019 From: justin at corelight.com (Justin Azoff) Date: Tue, 3 Dec 2019 09:55:41 -0500 Subject: [Zeek] Cluster configuration zeekctl status hangs In-Reply-To: References: Message-ID: On Tue, Dec 3, 2019 at 9:28 AM Scot Harris wrote: > > > > The problem is that when I run zeekctl status that request hangs: > > > > > > > > [zeek at heimdallr etc]$ zeekctl status > > > > Warning: ZeekControl plugin uses legacy BroControl API. Use > > 'import ZeekControl.plugin' instead of 'import BroControl.plugin' > > > > Getting process status ... > > Getting peer status ... > > > > Only way to resolve this is to kill process 8593. > > > > Any ideas on why this is hanging? > Odd that it's even doing that.. did you change this option in zeekctl.cfg? # Show all output of the zeekctl status command. If set to 1, then all output # is shown. If set to 0, then zeekctl status will not collect or show the peer # information (and the command will run faster). StatusCmdShowAll = 0 The default is to skip the "peer status" stuff, which causes zeekctl to connect to each worker on the broker port. You may have firewall rules or something preventing this from working. Does the zeekctl netstats command also hang? > > > Secondary problem with a work around available: > > > > Also have to follow the following steps for the cluster to work. > > > > 1. zeekctl install > > 2. setcap cap_net_raw=eip /opt/zeek/bin/zeek (on the remote peer) > > 3. zeekctl start > > > > Attempts to use zeekctl deploy does not work as the setcap command needs > to be run on the remote peer after the install is completed. > This should do what you want: https://github.com/PingTrip/broctl-setcap -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191203/e046ab46/attachment.html From SHARRIS at hollywoodfl.org Tue Dec 3 07:29:07 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Tue, 3 Dec 2019 15:29:07 +0000 Subject: [Zeek] [EXT]Re: Cluster configuration zeekctl status hangs In-Reply-To: References: Message-ID: Justin, That option did resolve the status problem I was seeing. What peer data is it trying to pull that causes it to hang? Now get the expected results: [zeek at heimdallr etc]$ zeekctl status Warning: ZeekControl plugin uses legacy BroControl API. Use 'import ZeekControl.plugin' instead of 'import BroControl.plugin' Name Type Host Status Pid Started logger logger 10.1.1.15 running 18323 03 Dec 10:26:15 manager manager 10.1.1.15 running 18370 03 Dec 10:26:16 proxy-1 proxy 10.1.1.15 running 18415 03 Dec 10:26:17 worker-1-1 worker 10.1.1.15 running 18505 03 Dec 10:26:19 worker-1-2 worker 10.1.1.15 running 18501 03 Dec 10:26:19 worker-2-1 worker 10.1.1.15 running 18506 03 Dec 10:26:19 worker-2-2 worker 10.1.1.15 running 18507 03 Dec 10:26:19 worker-3-1 worker 10.1.7.186 running 28032 03 Dec 10:26:19 worker-3-2 worker 10.1.7.186 running 28033 03 Dec 10:26:19 worker-4-1 worker 10.1.7.186 running 28035 03 Dec 10:26:19 worker-4-2 worker 10.1.7.186 running 28036 03 Dec 10:26:19 Will try the other fix shortly. Thank you! Scot From: Justin Azoff [mailto:justin at corelight.com] Sent: Tuesday, December 3, 2019 9:56 AM To: Scot Harris Cc: zeek at zeek.org Subject: [EXT]Re: [Zeek] Cluster configuration zeekctl status hangs On Tue, Dec 3, 2019 at 9:28 AM Scot Harris > wrote: The problem is that when I run zeekctl status that request hangs: [zeek at heimdallr etc]$ zeekctl status Warning: ZeekControl plugin uses legacy BroControl API. Use 'import ZeekControl.plugin' instead of 'import BroControl.plugin' Getting process status ... Getting peer status ... Only way to resolve this is to kill process 8593. Any ideas on why this is hanging? Odd that it's even doing that.. did you change this option in zeekctl.cfg? # Show all output of the zeekctl status command. If set to 1, then all output # is shown. If set to 0, then zeekctl status will not collect or show the peer # information (and the command will run faster). StatusCmdShowAll = 0 The default is to skip the "peer status" stuff, which causes zeekctl to connect to each worker on the broker port. You may have firewall rules or something preventing this from working. Does the zeekctl netstats command also hang? Secondary problem with a work around available: Also have to follow the following steps for the cluster to work. 1. zeekctl install 2. setcap cap_net_raw=eip /opt/zeek/bin/zeek (on the remote peer) 3. zeekctl start Attempts to use zeekctl deploy does not work as the setcap command needs to be run on the remote peer after the install is completed. This should do what you want: https://github.com/PingTrip/broctl-setcap -- Justin CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191203/0c430f52/attachment-0001.html From SHARRIS at hollywoodfl.org Tue Dec 3 08:27:57 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Tue, 3 Dec 2019 16:27:57 +0000 Subject: [Zeek] [EXT]Re: Cluster configuration zeekctl status hangs In-Reply-To: References: Message-ID: Justin, Was able to get that setcap script to work. Required editing to get paths correct and remove extras that were not required. But it does work now! Thank you. From: zeek-bounces at zeek.org [mailto:zeek-bounces at zeek.org] On Behalf Of Scot Harris Sent: Tuesday, December 3, 2019 10:29 AM To: Justin Azoff Cc: zeek at zeek.org Subject: Re: [Zeek] [EXT]Re: Cluster configuration zeekctl status hangs Justin, That option did resolve the status problem I was seeing. What peer data is it trying to pull that causes it to hang? Now get the expected results: [zeek at heimdallr etc]$ zeekctl status Warning: ZeekControl plugin uses legacy BroControl API. Use 'import ZeekControl.plugin' instead of 'import BroControl.plugin' Name Type Host Status Pid Started logger logger 10.1.1.15 running 18323 03 Dec 10:26:15 manager manager 10.1.1.15 running 18370 03 Dec 10:26:16 proxy-1 proxy 10.1.1.15 running 18415 03 Dec 10:26:17 worker-1-1 worker 10.1.1.15 running 18505 03 Dec 10:26:19 worker-1-2 worker 10.1.1.15 running 18501 03 Dec 10:26:19 worker-2-1 worker 10.1.1.15 running 18506 03 Dec 10:26:19 worker-2-2 worker 10.1.1.15 running 18507 03 Dec 10:26:19 worker-3-1 worker 10.1.7.186 running 28032 03 Dec 10:26:19 worker-3-2 worker 10.1.7.186 running 28033 03 Dec 10:26:19 worker-4-1 worker 10.1.7.186 running 28035 03 Dec 10:26:19 worker-4-2 worker 10.1.7.186 running 28036 03 Dec 10:26:19 Will try the other fix shortly. Thank you! Scot From: Justin Azoff [mailto:justin at corelight.com] Sent: Tuesday, December 3, 2019 9:56 AM To: Scot Harris > Cc: zeek at zeek.org Subject: [EXT]Re: [Zeek] Cluster configuration zeekctl status hangs On Tue, Dec 3, 2019 at 9:28 AM Scot Harris > wrote: The problem is that when I run zeekctl status that request hangs: [zeek at heimdallr etc]$ zeekctl status Warning: ZeekControl plugin uses legacy BroControl API. Use 'import ZeekControl.plugin' instead of 'import BroControl.plugin' Getting process status ... Getting peer status ... Only way to resolve this is to kill process 8593. Any ideas on why this is hanging? Odd that it's even doing that.. did you change this option in zeekctl.cfg? # Show all output of the zeekctl status command. If set to 1, then all output # is shown. If set to 0, then zeekctl status will not collect or show the peer # information (and the command will run faster). StatusCmdShowAll = 0 The default is to skip the "peer status" stuff, which causes zeekctl to connect to each worker on the broker port. You may have firewall rules or something preventing this from working. Does the zeekctl netstats command also hang? Secondary problem with a work around available: Also have to follow the following steps for the cluster to work. 1. zeekctl install 2. setcap cap_net_raw=eip /opt/zeek/bin/zeek (on the remote peer) 3. zeekctl start Attempts to use zeekctl deploy does not work as the setcap command needs to be run on the remote peer after the install is completed. This should do what you want: https://github.com/PingTrip/broctl-setcap -- Justin CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191203/beb03e68/attachment-0001.html From jsiwek at corelight.com Tue Dec 3 10:40:01 2019 From: jsiwek at corelight.com (Jon Siwek) Date: Tue, 3 Dec 2019 10:40:01 -0800 Subject: [Zeek] tcp partial connections In-Reply-To: References: <09a8e3bfc8a34d56ae8439e193d219c4@SRVEX03.aizoon.local> Message-ID: On Tue, Dec 3, 2019 at 2:14 AM Palumbo Mauro wrote: > As I would like to get a better understanding of what is going on, I examined a single tcp+http connection in more details. Zeek starts processing it correctly at first, there is the tcp handshake and the first GET/POST are logged correctly. Then Zeek stops processing the http traffic. By debugging, I noticed that in the middle of the conn, zeek calls again the ctor of the tcp analyzer and and as a consequence resets "is_partial" and other variables. Again from debugging it seems this happens because of the timer TCPConnectionExpireTimer is dispatched at a certain point. > > Is this what you would expect in the middle of a conn? It doesn't seem to me that there is a long inactivity on this conn. Hard to say without looking directly at a pcap which reproduces the behavior, but yes, there do exist various inactivity timers you might expect/suspect to interfere or cause things like that to happen. You might look more closely at TCP_Analyzer::ExpireTimer() to find which condition is being met and see if it makes sense for the particular connection(s). You'll also find out from that whether there's particular timeout interval options to try tuning for your use-case. E.g. the relevant ones look like they're 5-6 seconds by default: "tcp_SYN_timeout", "tcp_session_timer", and "tcp_connection_linger". - Jon From mauro.palumbo at aizoon.it Wed Dec 4 00:35:08 2019 From: mauro.palumbo at aizoon.it (Palumbo Mauro) Date: Wed, 4 Dec 2019 08:35:08 +0000 Subject: [Zeek] R: tcp partial connections In-Reply-To: References: <09a8e3bfc8a34d56ae8439e193d219c4@SRVEX03.aizoon.local> Message-ID: <6ab08b9c4adc42db96d3f8b6fa86dac1@SRVEX03.aizoon.local> Actually the pcap file I was looking into had only half the traffic in it (only one direction). That's why I was seeing a quite odd behavior. After fixing this, zeek disables the http analyzer when it detects some gaps in the TCP flux, i.e. some packets are missing. This makes much more sense. Thanks again, Mauro -----Messaggio originale----- Da: Jon Siwek [mailto:jsiwek at corelight.com] Inviato: marted? 3 dicembre 2019 19:40 A: Palumbo Mauro Cc: zeek at zeek.org Oggetto: Re: [Zeek] tcp partial connections On Tue, Dec 3, 2019 at 2:14 AM Palumbo Mauro wrote: > As I would like to get a better understanding of what is going on, I examined a single tcp+http connection in more details. Zeek starts processing it correctly at first, there is the tcp handshake and the first GET/POST are logged correctly. Then Zeek stops processing the http traffic. By debugging, I noticed that in the middle of the conn, zeek calls again the ctor of the tcp analyzer and and as a consequence resets "is_partial" and other variables. Again from debugging it seems this happens because of the timer TCPConnectionExpireTimer is dispatched at a certain point. > > Is this what you would expect in the middle of a conn? It doesn't seem to me that there is a long inactivity on this conn. Hard to say without looking directly at a pcap which reproduces the behavior, but yes, there do exist various inactivity timers you might expect/suspect to interfere or cause things like that to happen. You might look more closely at TCP_Analyzer::ExpireTimer() to find which condition is being met and see if it makes sense for the particular connection(s). You'll also find out from that whether there's particular timeout interval options to try tuning for your use-case. E.g. the relevant ones look like they're 5-6 seconds by default: "tcp_SYN_timeout", "tcp_session_timer", and "tcp_connection_linger". - Jon From SHARRIS at hollywoodfl.org Wed Dec 4 14:31:22 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Wed, 4 Dec 2019 22:31:22 +0000 Subject: [Zeek] capstats Message-ID: Noticed this while running zeekctl Ran the capstats command with this result: [ZeekControl] > capstats Interface kpps mbps (10s average) ---------------------------------------- worker-1-1: capstats failed (error: eno1: You don't have permission to capture on that device (socket: Operation not permitted)) worker-3-1: capstats failed (error: eno1: You don't have permission to capture on that device (socket: Operation not permitted)) Run status: [ZeekControl] > status Name Type Host Status Pid Started logger logger 10.1.1.15 running 5656 03 Dec 11:24:51 manager manager 10.1.1.15 running 5783 03 Dec 11:24:53 proxy-1 proxy 10.1.1.15 running 5834 03 Dec 11:24:54 worker-1-1 worker 10.1.1.15 running 6026 03 Dec 11:24:55 worker-1-2 worker 10.1.1.15 running 6027 03 Dec 11:24:55 worker-2-1 worker 10.1.1.15 running 6031 03 Dec 11:24:55 worker-2-2 worker 10.1.1.15 running 6030 03 Dec 11:24:55 worker-3-1 worker 10.1.7.186 running 9937 04 Dec 17:16:56 worker-3-2 worker 10.1.7.186 running 9995 04 Dec 17:17:23 worker-4-1 worker 10.1.7.186 running 10040 04 Dec 17:17:29 worker-4-2 worker 10.1.7.186 running 10085 04 Dec 17:17:32 Seems like the capstats command is incorrect. They system is collecting data. The workers that are listed are the first ones on each device. This is zeek 3.0. __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191204/08d74ca4/attachment.html From justin at corelight.com Wed Dec 4 17:25:49 2019 From: justin at corelight.com (Justin Azoff) Date: Wed, 4 Dec 2019 20:25:49 -0500 Subject: [Zeek] capstats In-Reply-To: References: Message-ID: If you're using that setcap plugin you also need to run setcap on ..../ bin/capstats On Wed, Dec 4, 2019 at 5:34 PM Scot Harris wrote: > Noticed this while running zeekctl > > > > Ran the capstats command with this result: > > > > [ZeekControl] > capstats > > Interface kpps mbps (10s average) > > ---------------------------------------- > > worker-1-1: capstats failed (error: eno1: You don't have permission to > capture on that device (socket: Operation not permitted)) > > worker-3-1: capstats failed (error: eno1: You don't have permission to > capture on that device (socket: Operation not permitted)) > > > > > > > > Run status: > > > > > > [ZeekControl] > status > > Name Type Host Status Pid Started > > logger logger 10.1.1.15 running 5656 03 Dec 11:24:51 > > manager manager 10.1.1.15 running 5783 03 Dec 11:24:53 > > proxy-1 proxy 10.1.1.15 running 5834 03 Dec 11:24:54 > > worker-1-1 worker 10.1.1.15 running 6026 03 Dec 11:24:55 > > worker-1-2 worker 10.1.1.15 running 6027 03 Dec 11:24:55 > > worker-2-1 worker 10.1.1.15 running 6031 03 Dec 11:24:55 > > worker-2-2 worker 10.1.1.15 running 6030 03 Dec 11:24:55 > > worker-3-1 worker 10.1.7.186 running 9937 04 Dec 17:16:56 > > worker-3-2 worker 10.1.7.186 running 9995 04 Dec 17:17:23 > > worker-4-1 worker 10.1.7.186 running 10040 04 Dec 17:17:29 > > worker-4-2 worker 10.1.7.186 running 10085 04 Dec 17:17:32 > > > > > > Seems like the capstats command is incorrect. > > > > They system is collecting data. > > > > The workers that are listed are the first ones on each device. > > > > This is zeek 3.0. > > > > > __________________________________________ > *Scot Harris* > Network Engineer > City of Hollywood > Information Technology > > P.O. Box 229045 > Hollywood, FL 33022-9045 > Office: 954-921-3304 > E-mail: SHARRIS at hollywoodfl.org > [image: www.hollywoodfl.org] > Notice: Florida has a broad public records law. All correspondence sent to > the City of Hollywood via e-mail may be subject to disclosure as a matter > of public record. > __________________________________________ > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191204/5c3ec3ac/attachment.html From SHARRIS at hollywoodfl.org Thu Dec 5 04:43:27 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Thu, 5 Dec 2019 12:43:27 +0000 Subject: [Zeek] [EXT]Re: capstats In-Reply-To: References: Message-ID: Justin, Thank you. That was the issue. From: Justin Azoff [mailto:justin at corelight.com] Sent: Wednesday, December 4, 2019 8:26 PM To: Scot Harris Cc: zeek at zeek.org Subject: [EXT]Re: [Zeek] capstats If you're using that setcap plugin you also need to run setcap on ..../bin/capstats On Wed, Dec 4, 2019 at 5:34 PM Scot Harris > wrote: Noticed this while running zeekctl Ran the capstats command with this result: [ZeekControl] > capstats Interface kpps mbps (10s average) ---------------------------------------- worker-1-1: capstats failed (error: eno1: You don't have permission to capture on that device (socket: Operation not permitted)) worker-3-1: capstats failed (error: eno1: You don't have permission to capture on that device (socket: Operation not permitted)) Run status: [ZeekControl] > status Name Type Host Status Pid Started logger logger 10.1.1.15 running 5656 03 Dec 11:24:51 manager manager 10.1.1.15 running 5783 03 Dec 11:24:53 proxy-1 proxy 10.1.1.15 running 5834 03 Dec 11:24:54 worker-1-1 worker 10.1.1.15 running 6026 03 Dec 11:24:55 worker-1-2 worker 10.1.1.15 running 6027 03 Dec 11:24:55 worker-2-1 worker 10.1.1.15 running 6031 03 Dec 11:24:55 worker-2-2 worker 10.1.1.15 running 6030 03 Dec 11:24:55 worker-3-1 worker 10.1.7.186 running 9937 04 Dec 17:16:56 worker-3-2 worker 10.1.7.186 running 9995 04 Dec 17:17:23 worker-4-1 worker 10.1.7.186 running 10040 04 Dec 17:17:29 worker-4-2 worker 10.1.7.186 running 10085 04 Dec 17:17:32 Seems like the capstats command is incorrect. They system is collecting data. The workers that are listed are the first ones on each device. This is zeek 3.0. __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191205/f7bfc6a6/attachment-0001.html From SHARRIS at hollywoodfl.org Thu Dec 5 05:07:37 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Thu, 5 Dec 2019 13:07:37 +0000 Subject: [Zeek] Warning: ZeekControl plugin uses legacy BroControl API. Message-ID: Get a lot of these warnings reported in email notices. Warning: ZeekControl plugin uses legacy BroControl API. Use 'import ZeekControl.plugin' instead of 'import BroControl.plugin' Found this thread but no final resolution. https://github.com/J-Gras/bro-af_packet-plugin/issues/11 Running zeek 3.0 __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191205/08456c1a/attachment.html From SHARRIS at hollywoodfl.org Thu Dec 5 06:19:28 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Thu, 5 Dec 2019 14:19:28 +0000 Subject: [Zeek] Ports for workers Message-ID: Have two machines, one running manager, logger, proxy, worker. Second machine running worker. Cluster appears to be working. However, some of the commands in zeekctl hang. (peerstatus, netstats) Appears to be due to firewall on the second machine. Disabling firewall the commands work as expected. What ports on the second machine need to be opened? I found ports for: Logger 47761 Manager 47762 Proxy 47763 Thank you. __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191205/078e255b/attachment.html From justin at corelight.com Thu Dec 5 06:43:57 2019 From: justin at corelight.com (Justin Azoff) Date: Thu, 5 Dec 2019 09:43:57 -0500 Subject: [Zeek] Ports for workers In-Reply-To: References: Message-ID: It's explained here: https://github.com/zeek/zeekctl/blob/master/doc/main.rst#zeek-communication You can also see exactly what ports have been allocated if you look at /usr/local/bro/spool/installed-scripts-do-not-touch/auto/cluster-layout.bro On Thu, Dec 5, 2019 at 9:22 AM Scot Harris wrote: > Have two machines, one running manager, logger, proxy, worker. > > > > Second machine running worker. > > > > Cluster appears to be working. > > > > However, some of the commands in zeekctl hang. (peerstatus, netstats) > > > > Appears to be due to firewall on the second machine. Disabling firewall > the commands work as expected. > > > > What ports on the second machine need to be opened? > > > > I found ports for: > > > > Logger 47761 > > Manager 47762 > > Proxy 47763 > > > > Thank you. > > > __________________________________________ > *Scot Harris* > Network Engineer > City of Hollywood > Information Technology > > P.O. Box 229045 > Hollywood, FL 33022-9045 > Office: 954-921-3304 > E-mail: SHARRIS at hollywoodfl.org > [image: www.hollywoodfl.org] > Notice: Florida has a broad public records law. All correspondence sent to > the City of Hollywood via e-mail may be subject to disclosure as a matter > of public record. > __________________________________________ > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191205/4d46abd8/attachment-0001.html From akgraner at corelight.com Thu Dec 5 07:02:44 2019 From: akgraner at corelight.com (akgraner at corelight.com) Date: Thu, 05 Dec 2019 15:02:44 +0000 Subject: [Zeek] Invitation: Reoccurring Zeek Community Call @ Fri Dec 6, 2019 3pm - 3:45pm (EST) (zeek@zeek.org) Message-ID: <000000000000b92baa0598f63810@google.com> You have been invited to the following event. Title: Reoccurring Zeek Community Call (Public call w/anyone who wants to join) Moving this to the 6th as I am getting everything set up for a reoccurring cadence. I'll share with the community this week and get some input. Gathering input - this first call will be on Zoom but following calls will be Go To Meeting AGENDA: * Kick off these monthly calls and gather feedback * Discuss - Slack, Discourse and Matrix * Announce the Kick off of related Zeek Webinars ?????????? Amber Graner is inviting you to a scheduled Zoom meeting. Join Zoom Meeting https://corelight.zoom.us/j/471894896 Meeting ID: 471 894 896 One tap mobile +16465588656,,471894896# US (New York) +16699006833,,471894896# US (San Jose) Dial by your location +1 646 558 8656 US (New York) +1 669 900 6833 US (San Jose) 877 853 5257 US Toll-free 888 475 4499 US Toll-free Meeting ID: 471 894 896 Find your local number: https://corelight.zoom.us/u/acY5L1LN7 ?????????? When: Fri Dec 6, 2019 3pm ? 3:45pm Eastern Time - New York Where: https://corelight.zoom.us/j/471894896 Calendar: zeek at zeek.org Who: * akgraner at corelight.com - organizer * jan.grashoefer at gmail.com * dopheide at es.net * dopheide at gmail.com * fatema.bannatwala at gmail.com * tet68mt at gmail.com * phil at brimsecurity.com * zeek at zeek.org Event details: https://www.google.com/calendar/event?action=VIEW&eid=NzE2dXJ1MXAzNDBsa2g4aGZ2ZGVibXAxam1fMjAxOTEyMDZUMjAwMDAwWiB6ZWVrQHplZWsub3Jn&tok=MjIjYWtncmFuZXJAY29yZWxpZ2h0LmNvbTlkNWRhNmQ1MWNlNjY5MjljMDlkZmU4NTUyOWZiMzU5Yzc2NmEzYTk&ctz=America%2FNew_York&hl=en&es=0 Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account zeek at zeek.org because you are an attendee of this event. To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar. Forwarding this invitation could allow any recipient to send a response to the organizer and be added to the guest list, or invite others regardless of their own invitation status, or to modify your RSVP. Learn more at https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191205/cc04b0fc/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 3818 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191205/cc04b0fc/attachment-0002.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 3896 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191205/cc04b0fc/attachment-0003.bin From akgraner at corelight.com Thu Dec 5 07:01:56 2019 From: akgraner at corelight.com (Amber Graner) Date: Thu, 5 Dec 2019 10:01:56 -0500 Subject: [Zeek] Upcoming Zeek Community events - Participation Opportunities Message-ID: Hi all, First, I apologize for the late notice for the following, but hope you can join us: ============= *6 Dec 3:00pm ET - *First Open Community Call (this will be recorded so those that can't make will be able to review the discussion). I'll add the list to the invite, but here is the link if you can join tomorrow - https://corelight.zoom.us/j/471894896 AGENDA * Kick off these monthly calls and gather feedback * Discuss - Slack, Discourse and Matrix * Announce the Kick off of related Zeek Webinars ============== *7 Dec 10am-5pm Ft. Lauderdale, FL - *Winter Hacker Fest Conference and CTF=== You can register at - https://www.meetup.com/hackmiami/events/264309540/ The Zeek Workshop will include: * An Intro to Zeek (Amber Graner, Seth Hall and Aaron Soto) * An intro to Zeek Scripting (Seth Hall) * A 2 Hour Capture the Flag event. (Aaron Soto) There will also be a couple of Raspberry Pi 4 Giveaways (sponsored by Corelight). ============== *Ask the Zeeksperts -Webinar Series* This will be a drop in, non recorded series of webinars where folks like Seth Hall, Robin Sommer, Vern Paxon and many others will be holding office hours for you to ask them questions. I'll add the list to the webinar invite. We'll do one in December then starting in January 2020 we'll do two a month. So get those questions ready. Are you a Zeekspert? Do you want to be added to the schedule? Please let me know. ============== Thanks everyone, again, apologies for the late notice. With gratitude, ~Amber -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191205/d0e86884/attachment.html From SHARRIS at hollywoodfl.org Thu Dec 5 14:39:59 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Thu, 5 Dec 2019 22:39:59 +0000 Subject: [Zeek] pkg add-interfaces Message-ID: Used zkg to add the add-interfaces package to zeek 3.0 Seeing the following warning messages in stderr log: [9] => warning: non-void function returning without a value: AddInterfaces::interface_ext_func [10] => warning: non-void function returning without a value: AddInterfaces::interface_ext_func [11] => warning: non-void function returning without a value: AddInterfaces::interface_ext_func [12] => warning: non-void function returning without a value: AddInterfaces::interface_ext_func [13] => warning: non-void function returning without a value: AddInterfaces::interface_ext_func [14] => warning: non-void function returning without a value: AddInterfaces::interface_ext_func [15] => warning: non-void function returning without a value: AddInterfaces::interface_ext_func [16] => warning: non-void function returning without a value: AddInterfaces::interface_ext_func Looking at the code for the function interface_ext_func looks like the condition is most likely failing and there is no return value specified. function interface_ext_func(path: string): AddedFields { if ( Cluster::nodes[Cluster::node]?$interface ) return AddedFields($interface = Cluster::nodes[Cluster::node]$interface); } __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191205/fda87edb/attachment.html From akgraner at corelight.com Fri Dec 6 01:11:04 2019 From: akgraner at corelight.com (Amber Graner) Date: Fri, 6 Dec 2019 04:11:04 -0500 Subject: [Zeek] [New Blog Post] - How to Add a JPEG file analyzer to #Zeek Message-ID: Hi all, Have you ever wanted to add a JPEG file analyzer to Zeek? Keith J. Jones, Ph.D (more information below) has contributed the first in a series of Zeek Blogs posts on how to do that. https://blog.zeek.org/2019/12/how-to-add-jpeg-file-analyzer-to-zeek.html Do you have ideas for *Tweaking your Zeek*? Have you written a how-to you would like to contribute? Please let me know. In the meantime, "Happy Zeeking!" Thanks, ~Amber --------------- ==About Keith J. Jones, Ph.D== Dr. Jones is an internationally industry-recognized expert with over two decades of experience in cyber security, incident response, and computer forensics. His expertise includes software development, innovative prototyping, information security consulting, application security, malware analysis & reverse engineering, software analysis/design and image/video/audio analysis. Dr. Jones holds an Electrical Engineering and Computer Engineering undergraduate degrees from Michigan State University. He also earned a Master of Science degree in Electrical Engineering from MSU. Dr. Jones recently completed his Ph.D. in Cyber Operations from Dakota State University in 2019. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191206/9d7a7da0/attachment.html From fatema.bannatwala at gmail.com Fri Dec 6 14:00:16 2019 From: fatema.bannatwala at gmail.com (fatema bannatwala) Date: Fri, 6 Dec 2019 17:00:16 -0500 Subject: [Zeek] Noticing "SumStat key request for the.." in reporter.log Zeek 3.0 Message-ID: Hi Everyone, I upgraded our external zeek cluster right before ThanksGiving to zeek 3.0, and have started noticing a fair amount of following warnings in reporter.log file: "SumStat key request for the 7PJNSqZOUs8 SumStat uid took longer than 1 minute and was automatically cancelled." Also, interesting thing is that after the upgrade, generation of software.log file has become pretty sporadic (no software.log file for last one week).. Anyone else noticing this behavior? Any thoughts? Something needs to get back ported for software.log to work correctly again in zeek 3.0? Thanks! Fatema -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191206/67bb5466/attachment.html From jsiwek at corelight.com Fri Dec 6 17:26:43 2019 From: jsiwek at corelight.com (Jon Siwek) Date: Fri, 6 Dec 2019 17:26:43 -0800 Subject: [Zeek] Noticing "SumStat key request for the.." in reporter.log Zeek 3.0 In-Reply-To: References: Message-ID: On Fri, Dec 6, 2019 at 2:06 PM fatema bannatwala wrote: > I upgraded our external zeek cluster right before ThanksGiving to zeek 3.0, and have started noticing a fair amount of following warnings in reporter.log file: > > "SumStat key request for the 7PJNSqZOUs8 SumStat uid took longer than 1 minute and was automatically cancelled." Did you happen to copy over a previous local.bro that still has "@load misc/scan" in it? The new local.zeek has that commented out due to it being frequent cause of performance issues. > Also, interesting thing is that after the upgrade, generation of software.log file has become pretty sporadic (no software.log file for last one week).. One reason for that may be if you don't have any proxy nodes in your cluster config (or they aren't reachable for some reason). - Jon From nothinrandom at gmail.com Fri Dec 6 19:33:25 2019 From: nothinrandom at gmail.com (TQ) Date: Fri, 6 Dec 2019 19:33:25 -0800 Subject: [Zeek] Part of const array with same value Message-ID: Hello Zeekers, What is the best way to represent indices of an array with the same value? Let's say I declare a constant string array used for enumeration called 'test'. Values 0 and 1 has some unique value, but anything from 2 and 7 has a constant value of 'test2'. Is there a quicker way of representing this or would I need to manually set it? const test = { [0] = "test0", [1] = "test1", [2] = "test2", [3] = "test2", [4] = "test2", [5] = "test2", [6] = "test2", [7] = "test2", [8] = "test3", } &default=function(i: count):string { return fmt("test(%x)", i); } &redef; Thanks, -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191206/44ff7cc1/attachment.html From kilotao at gmail.com Sat Dec 7 06:00:29 2019 From: kilotao at gmail.com (kilotao at gmail.com) Date: Sat, 7 Dec 2019 09:00:29 -0500 Subject: [Zeek] Part of const array with same value In-Reply-To: References: Message-ID: What's the use of this array? On Fri, Dec 6, 2019, 10:36 PM TQ wrote: > Hello Zeekers, > > What is the best way to represent indices of an array with the same > value? Let's say I declare a constant string array used for enumeration > called 'test'. Values 0 and 1 has some unique value, but anything from 2 > and 7 has a constant value of 'test2'. Is there a quicker way of > representing this or would I need to manually set it? > > const test = { > [0] = "test0", > [1] = "test1", > [2] = "test2", > [3] = "test2", > [4] = "test2", > [5] = "test2", > [6] = "test2", > [7] = "test2", > [8] = "test3", > } &default=function(i: count):string { return fmt("test(%x)", i); } > &redef; > > Thanks, > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191207/680a4362/attachment.html From nothinrandom at gmail.com Sat Dec 7 11:31:11 2019 From: nothinrandom at gmail.com (TQ) Date: Sat, 7 Dec 2019 11:31:11 -0800 Subject: [Zeek] Part of const array with same value In-Reply-To: References: Message-ID: My use case is for enumeration. This array is typically declared in consts.zeek; you would call it in main.zeek. I mean you could write a switch statement, but now main.zeek would definitely blow up quite a bit. Thanks, On Sat, Dec 7, 2019 at 6:00 AM kilotao at gmail.com wrote: > What's the use of this array? > > On Fri, Dec 6, 2019, 10:36 PM TQ wrote: > >> Hello Zeekers, >> >> What is the best way to represent indices of an array with the same >> value? Let's say I declare a constant string array used for enumeration >> called 'test'. Values 0 and 1 has some unique value, but anything from 2 >> and 7 has a constant value of 'test2'. Is there a quicker way of >> representing this or would I need to manually set it? >> >> const test = { >> [0] = "test0", >> [1] = "test1", >> [2] = "test2", >> [3] = "test2", >> [4] = "test2", >> [5] = "test2", >> [6] = "test2", >> [7] = "test2", >> [8] = "test3", >> } &default=function(i: count):string { return fmt("test(%x)", i); } >> &redef; >> >> Thanks, >> _______________________________________________ >> Zeek mailing list >> zeek at zeek.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191207/edbf8c6f/attachment.html From akgraner at corelight.com Sat Dec 7 13:49:21 2019 From: akgraner at corelight.com (Amber Graner) Date: Sat, 7 Dec 2019 16:49:21 -0500 Subject: [Zeek] 6 Dec 2019 - Community Call - Summary and Links Message-ID: Hi all, Below is the link to the folder that includes Slides, Audio, and Video from the 6 Dec 2019 call. http://bit.ly/ZeekCommunityCall_6Dec19 Thank you so much to all those who participated. ------------ Next Call ------------ 3 January 2020 - 3pm ET ------------- Summary ------------- We had 26 participants on the first monthly community call. Thank you to those who presented on alternatives to IRC: Alternatives to IRC Discussion/Presentations: - Matrix - Jan Grash?fer (alternative to IRC - Slack integration available) - Slack - Michael Dopheide (alternative to IRC) - Discourse - Matt Trostel (alternative to Mailman, includes integration with Slack and Matrix) Amber to send poll to the community about all options. Community to review options between 9-13 December; poll to be sent out week of 16-20 December. ----------------- Other Topics Mentioned: ----------------- - More Content Posted on the Zeek Blog - If you have an idea for a blog post, please let Amber know or send to the Zeek Mailing list. - Monthly Newsletter - Matt Trostel volunteered to help, but if you would like to help please let Amber know or send to the mailing list. - Raspberry Pi Zeek Images - Image in call folder (no instructions - Need to create a howto blog post - ZeekWeek 2020 - Increased Training and sponsor opportunities - Dual track formats (One track geared toward threat hunters/incident responders other traditional developer track) - Look for location and date to be announced in January 2020 - Zeek Packages - If you have written packages that extend the capabilities of Zeek, please consider opening them through the Zeek Package Manager. - If you need help with this process please reach out to the list or to Amber. Please let know if you have any questions, comments, feedback or thoughts for the January meeting. With gratitude, ~Amber -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191207/34809166/attachment.html From fatema.bannatwala at gmail.com Sat Dec 7 16:24:49 2019 From: fatema.bannatwala at gmail.com (fatema bannatwala) Date: Sat, 7 Dec 2019 19:24:49 -0500 Subject: [Zeek] Noticing "SumStat key request for the.." in reporter.log Zeek 3.0 In-Reply-To: References: Message-ID: Hi Jon, Thanks for the insights. I don't have the misc/scan enabled in local.zeek, actually using Justin's simple scan detection script. Also, checked the local scripts that are currently enabled in local.zeek and found two scripts - detect-ms15-034.bro and http-basic-auth-bruteforce.bro that use SumStat framework. I have disabled them to see if the SumStat warnings are reduced in the reporter.log. Thanks! Fatema On Fri, Dec 6, 2019 at 8:26 PM Jon Siwek wrote: > On Fri, Dec 6, 2019 at 2:06 PM fatema bannatwala > wrote: > > > I upgraded our external zeek cluster right before ThanksGiving to zeek > 3.0, and have started noticing a fair amount of following warnings in > reporter.log file: > > > > "SumStat key request for the 7PJNSqZOUs8 SumStat uid took longer than 1 > minute and was automatically cancelled." > > Did you happen to copy over a previous local.bro that still has "@load > misc/scan" in it? The new local.zeek has that commented out due to it > being frequent cause of performance issues. > > > Also, interesting thing is that after the upgrade, generation of > software.log file has become pretty sporadic (no software.log file for last > one week).. > > One reason for that may be if you don't have any proxy nodes in your > cluster config (or they aren't reachable for some reason). > > - Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191207/1741d858/attachment.html From jan.grashoefer at gmail.com Mon Dec 9 09:38:35 2019 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Mon, 9 Dec 2019 18:38:35 +0100 Subject: [Zeek] pkg add-interfaces In-Reply-To: References: Message-ID: <1ed1cfc6-d980-b226-df54-715c882b68dc@gmail.com> Hi Scot, unfortunately I cannot reproduce the issue. > Looking at the code for the function interface_ext_func looks like the > condition is most likely failing and there is no return value specified. > > function interface_ext_func(path: string): AddedFields > > ??????? { > > ??????? if ( Cluster::nodes[Cluster::node]?$interface ) > > ??????????????? return AddedFields($interface = > Cluster::nodes[Cluster::node]$interface); > > ??????? } You might try to add something like the following to deal with timing issues: else return AddedFields($interface = fmt("%s:unknown-interface", Cluster::node)); However, if the interfaces do not appear in the logs after an initial warm-up, there might be something else wrong. Jan From justin at corelight.com Mon Dec 9 09:49:49 2019 From: justin at corelight.com (Justin Azoff) Date: Mon, 9 Dec 2019 12:49:49 -0500 Subject: [Zeek] pkg add-interfaces In-Reply-To: <1ed1cfc6-d980-b226-df54-715c882b68dc@gmail.com> References: <1ed1cfc6-d980-b226-df54-715c882b68dc@gmail.com> Message-ID: On Mon, Dec 9, 2019 at 12:40 PM Jan Grash?fer wrote: > Hi Scot, > > unfortunately I cannot reproduce the issue. > Hmm.. that code could run into an issue on non worker nodes.. so if a manager or proxy logs something directly you will hit that edge case. -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191209/0e84a9ce/attachment.html From SHARRIS at hollywoodfl.org Mon Dec 9 13:20:20 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Mon, 9 Dec 2019 21:20:20 +0000 Subject: [Zeek] Large file detection Message-ID: Running zeek 3.0. Installed zeek/theflakes/bro-large_uploads (installed: master) - Raise notices on outgoing files over X bytes in size. Getting a lot of events logged in notices log files. Fairly certain at this time that these events are due to Cylance application sending data to Cylance cloud services for analysis. Unable to get a specific list of aws ec2 servers as they are using a lot of them and they change regularly. Any ideas on how to reduce these notices so the unusual events are more apparent? Since it is looking at network packets I don't think there is any way to tie the file transfer back to the application. Examples of the events found in the notices log files below. _interface ts uid id.orig_h id.orig_p id.resp_h id.resp_p fuid file_mime_type file_desc proto note msg sub src dst p n peer_descr actions suppress_for remote_location.country_code remote_location.region remote_location.city remote_location.latitude remote_location.longitude af_packet::eno1 2019-12-09T00:00:59-0500 COFjRo4ZBvf3xSXVK2 10.1.7.205 59028 3.231.142.14 443 - - - tcp LargeUploads::Very_Large_Outgoing_Tx Orig transmitted 29666232 bytes to resp. Duration 206837.349578 sec. Source is 07984coh.hollywood.local. Destination is ec2-3-231-142-14.compute-1.amazonaws.com. Connection UID COFjRo4ZBvf3xSXVK2. Tx start: 12/06/2019 14:33:37 UTC, end: 12/09/2019 00:00:54 UTC 10.1.7.205 3.231.142.14 443 - worker-1-1 Notice::ACTION_LOG 3600.000000 - - - - - af_packet::eno1 2019-12-09T00:01:00-0500 CyrMsW3v1LqtqWVEu2 10.1.100.83 58699 3.224.236.241 443 - - - tcp LargeUploads::Very_Large_Outgoing_Tx Orig transmitted 17038390 bytes to resp. Duration 206903.826323 sec. Source is rfidsrv.hollywood.local. Destination is ec2-3-224-236-241.compute-1.amazonaws.com. Connection UID CyrMsW3v1LqtqWVEu2. Tx start: 12/06/2019 14:32:31 UTC, end: 12/09/2019 00:00:54 UTC 10.1.100.83 3.224.236.241 443 - worker-2-1 Notice::ACTION_LOG 3600.000000 - - - - - af_packet::eno1 2019-12-09T00:01:01-0500 CarVkYRqSh34QSiOl 10.1.23.90 57968 52.200.205.157 443 - - - tcp LargeUploads::Very_Large_Outgoing_Tx Orig transmitted 17852337 bytes to resp. Duration 206996.104661 sec. Source is . Destination is ec2-52-200-205-157.compute-1.amazonaws.com. Connection UID CarVkYRqSh34QSiOl. Tx start: 12/06/2019 14:31:00 UTC, end: 12/09/2019 00:00:56 UTC 10.1.23.90 52.200.205.157 443 - worker-2-1 Notice::ACTION_LOG 3600.000000 - - - - - af_packet::eno1 2019-12-09T00:01:02-0500 CooJjR1HWcj5B6Cwt7 10.1.41.74 58770 35.170.28.255 443 - - - tcp LargeUploads::Very_Large_Outgoing_Tx Orig transmitted 16385139 bytes to resp. Duration 205678.150834 sec. Source is 06208coh.hollywood.local. Destination is ec2-35-170-28-255.compute-1.amazonaws.com. Connection UID CooJjR1HWcj5B6Cwt7. Tx start: 12/06/2019 14:52:59 UTC, end: 12/09/2019 00:00:57 UTC 10.1.41.74 35.170.28.255 443 - worker-1-2 Notice::ACTION_LOG 3600.000000 - - - - - Thank you. __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191209/6b68bac5/attachment-0001.html From michalpurzynski1 at gmail.com Mon Dec 9 13:44:02 2019 From: michalpurzynski1 at gmail.com (=?utf-8?Q?Micha=C5=82_Purzy=C5=84ski?=) Date: Mon, 9 Dec 2019 13:44:02 -0800 Subject: [Zeek] Large file detection In-Reply-To: References: Message-ID: Just an idea without a code You could have a lookup table created from ssl events that keeps the list or recent IP addresses associated with your vendor, recognized by x509 certificate details and then a avoid alerting if there?s a March. > On Dec 9, 2019, at 1:25 PM, Scot Harris wrote: > > ? > Running zeek 3.0. > > Installed zeek/theflakes/bro-large_uploads (installed: master) - Raise notices on outgoing files over X bytes in size. > > Getting a lot of events logged in notices log files. Fairly certain at this time that these events are due to Cylance application sending data to Cylance cloud services for analysis. > > Unable to get a specific list of aws ec2 servers as they are using a lot of them and they change regularly. > > Any ideas on how to reduce these notices so the unusual events are more apparent? > > Since it is looking at network packets I don?t think there is any way to tie the file transfer back to the application. > > Examples of the events found in the notices log files below. > > > > > > _interface > ts > uid > id.orig_h > id.orig_p > id.resp_h > id.resp_p > fuid > file_mime_type > file_desc > proto > note > msg > sub > src > dst > p > n > peer_descr > actions > suppress_for > remote_location.country_code > remote_location.region > remote_location.city > remote_location.latitude > remote_location.longitude > af_packet::eno1 > 2019-12-09T00:00:59-0500 > COFjRo4ZBvf3xSXVK2 > 10.1.7.205 > 59028 > 3.231.142.14 > 443 > - > - > - > tcp > LargeUploads::Very_Large_Outgoing_Tx > Orig transmitted 29666232 bytes to resp. Duration 206837.349578 sec. Source is 07984coh.hollywood.local. Destination is ec2-3-231-142-14.compute-1.amazonaws.com. Connection UID COFjRo4ZBvf3xSXVK2. > Tx start: 12/06/2019 14:33:37 UTC, end: 12/09/2019 00:00:54 UTC > 10.1.7.205 > 3.231.142.14 > 443 > - > worker-1-1 > Notice::ACTION_LOG > 3600.000000 > - > - > - > - > - > af_packet::eno1 > 2019-12-09T00:01:00-0500 > CyrMsW3v1LqtqWVEu2 > 10.1.100.83 > 58699 > 3.224.236.241 > 443 > - > - > - > tcp > LargeUploads::Very_Large_Outgoing_Tx > Orig transmitted 17038390 bytes to resp. Duration 206903.826323 sec. Source is rfidsrv.hollywood.local. Destination is ec2-3-224-236-241.compute-1.amazonaws.com. Connection UID CyrMsW3v1LqtqWVEu2. > Tx start: 12/06/2019 14:32:31 UTC, end: 12/09/2019 00:00:54 UTC > 10.1.100.83 > 3.224.236.241 > 443 > - > worker-2-1 > Notice::ACTION_LOG > 3600.000000 > - > - > - > - > - > af_packet::eno1 > 2019-12-09T00:01:01-0500 > CarVkYRqSh34QSiOl > 10.1.23.90 > 57968 > 52.200.205.157 > 443 > - > - > - > tcp > LargeUploads::Very_Large_Outgoing_Tx > Orig transmitted 17852337 bytes to resp. Duration 206996.104661 sec. Source is . Destination is ec2-52-200-205-157.compute-1.amazonaws.com. Connection UID CarVkYRqSh34QSiOl. > Tx start: 12/06/2019 14:31:00 UTC, end: 12/09/2019 00:00:56 UTC > 10.1.23.90 > 52.200.205.157 > 443 > - > worker-2-1 > Notice::ACTION_LOG > 3600.000000 > - > - > - > - > - > af_packet::eno1 > 2019-12-09T00:01:02-0500 > CooJjR1HWcj5B6Cwt7 > 10.1.41.74 > 58770 > 35.170.28.255 > 443 > - > - > - > tcp > LargeUploads::Very_Large_Outgoing_Tx > Orig transmitted 16385139 bytes to resp. Duration 205678.150834 sec. Source is 06208coh.hollywood.local. Destination is ec2-35-170-28-255.compute-1.amazonaws.com. Connection UID CooJjR1HWcj5B6Cwt7. > Tx start: 12/06/2019 14:52:59 UTC, end: 12/09/2019 00:00:57 UTC > 10.1.41.74 > 35.170.28.255 > 443 > - > worker-1-2 > Notice::ACTION_LOG > 3600.000000 > - > - > - > - > - > > > Thank you. > > __________________________________________ > Scot Harris > Network Engineer > City of Hollywood > Information Technology > > P.O. Box 229045 > Hollywood, FL 33022-9045 > Office: 954-921-3304 > E-mail: SHARRIS at hollywoodfl.org > > Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. > __________________________________________ > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191209/4a12fdd5/attachment-0001.html From SHARRIS at hollywoodfl.org Tue Dec 10 11:52:59 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Tue, 10 Dec 2019 19:52:59 +0000 Subject: [Zeek] WriterFronend Message-ID: Getting this in the reporter log files: Reporter::WARNING WriterFrontend cluster/Log::WRITER_ASCII expected 3 fields in write, got 4. Skipping line. Is there a fix? __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191210/6f349239/attachment.html From SHARRIS at hollywoodfl.org Tue Dec 10 13:04:08 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Tue, 10 Dec 2019 21:04:08 +0000 Subject: [Zeek] log files Message-ID: Running the bro-large_uploads installed using zkg. I would like to have the information it is logging moved from notices to a new log file. Is there an easy way to make that change? Thank you. __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191210/44f970af/attachment.html From jsiwek at corelight.com Tue Dec 10 13:46:54 2019 From: jsiwek at corelight.com (Jon Siwek) Date: Tue, 10 Dec 2019 13:46:54 -0800 Subject: [Zeek] Zeek 3.0.1 release available Message-ID: Zeek release 3.0.1 is now available: https://www.zeek.org/downloads/zeek-3.0.1.tar.gz https://www.zeek.org/downloads/zeek-3.0.1.tar.gz.asc This is a bug-fix release that most notably addresses a JSON logging performance regression in 3.0.0, but also fixes other minor bugs. A list which details the changes can be found here: https://github.com/zeek/zeek/releases/tag/v3.0.1 -------------- next part -------------- A non-text attachment was scrubbed... Name: 3.0.1-announce.txt.asc Type: application/octet-stream Size: 1264 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191210/948f1799/attachment.obj From jgarciar at sia.es Wed Dec 11 03:29:54 2019 From: jgarciar at sia.es (Jorge Garcia Rodriguez) Date: Wed, 11 Dec 2019 11:29:54 +0000 Subject: [Zeek] Error with zeekctl netstats command Message-ID: Hi everyone Im facing the next error whenever I try to use the command "zeekctl netstats" to check the packets dropped. worker-1-1: And this with every other worker in 2 of 4 zeeks that we have deployed. I have other 2 nodes in which this command run without any problem and all the machines has been installed following the same procedure. Also this 2 nodes in which the command runs well, don't receive any traffic at this moment. So I don't know if this is something that we missed in the installation or is something produced by the amount of traffic. Hope you can help me to resolve this issue. Thanks you all! Regards. Jorge Garc?a Rodr?guez Technical Consultant Security Infrastructures jgarciar at sia.es Grupo SIA Avda.Europa,2 - Alcor Plaza, Edificio B - Parque Oeste Alcorc?n 28922 Alcorc?n - Madrid Tlf: +34 902 480 580 Fax: +34 91 307 79 80 www.siainternational.com delivering value This e-mail and any attached files are intended solely for the addresse/s identified herein. It may contain confidential and/or legally privileged information and may not necessarily represent the opinion of SIA. No legally binding commitments will be created by this E-mail message. Where we intend to create legally binding commitments these will be made through hard copy correspondence or documents. If you receive this message by mistake, please immediately notify the sender and delete it since you are not authorized to use, disclose, distribute, print or copy all or part of the contained information Thank you. It is understood that the message was sent to you accidentally, although you appear as the addressee, you can see from the frame of existing relations that you were not the final addressee. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191211/9baa531b/attachment.html From jsiwek at corelight.com Wed Dec 11 04:42:16 2019 From: jsiwek at corelight.com (Jon Siwek) Date: Wed, 11 Dec 2019 04:42:16 -0800 Subject: [Zeek] Error with zeekctl netstats command In-Reply-To: References: Message-ID: On Wed, Dec 11, 2019 at 3:32 AM Jorge Garcia Rodriguez wrote: > Im facing the next error whenever I try to use the command ?zeekctl netstats? to check the packets dropped. > > worker-1-1: > > And this with every other worker in 2 of 4 zeeks that we have deployed. The nodes with that error are using Python 2 and don't have the required "ipaddress" backport from Python 3 installed (distros in the RHEL or Debian families usually call their package "python-ipaddress"). The nodes without that error are either using Python 2 and do have "ipaddress" or are already using Python 3. Aligning all nodes to use Python 3 is the ideal path since Python 2 is EOL in ~20 days (Jan. 1). - Jon From SHARRIS at hollywoodfl.org Thu Dec 12 05:26:27 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Thu, 12 Dec 2019 13:26:27 +0000 Subject: [Zeek] sethhall/credit-card-exposure Message-ID: Does anyone have experience with the sethhall/credit-card-exposure package? I installed it and it is generating some results that does not seem valid. Running zeek 3.0 with this package installed using zkg. The odd data includes packets that go from my workstation to the zeek main server on port 80 that is flagged as having credit card numbers in it. I don't think that actually occurred. So was wondering if someone else had that package and what kind of results they are getting. Thank you. __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191212/98b3bef5/attachment.html From nick_turley at byu.edu Thu Dec 12 06:07:46 2019 From: nick_turley at byu.edu (Nick Turley) Date: Thu, 12 Dec 2019 14:07:46 +0000 Subject: [Zeek] sethhall/credit-card-exposure In-Reply-To: References: Message-ID: We?ve had pretty good luck with the package but we had to make modifications to get it working the way we wanted. We also modified it so it would work on Corelight. We?ve been running it on our Bro 2.6 cluster for some time. SSN detection is a high false positive game in a large environment like ours, so our analysts are still required to review the extracted payload and make a determination. Some of the modifications include extracting a chunk of the payload where the SSN was detected and including that in the notice log. We also added the protocol that was detected and associated info. For example, if SMB, we include the file name and location identified. As I recall, there was also a bug we fixed that wasn?t masking the SSNs correctly. We also feed in all 50 state historical SSN prefixes and include the state data in the notice log. However, SSNs after 2011 I believe are now randomized so this will be less effective over time. While we get a number of false positives, the module has also helped us discover some fairly serious security issues. When I get to the office, I would be happy to share our code. Nick Turley Security Architect CES Security Operations Center Office: (801) 422-4994 | Cell: (801) 310-3816 | nick_turley at byu.edu ________________________________ From: zeek-bounces at zeek.org on behalf of Scot Harris Sent: Thursday, December 12, 2019 6:26:27 AM To: zeek at zeek.org Subject: [Zeek] sethhall/credit-card-exposure Does anyone have experience with the sethhall/credit-card-exposure package? I installed it and it is generating some results that does not seem valid. Running zeek 3.0 with this package installed using zkg. The odd data includes packets that go from my workstation to the zeek main server on port 80 that is flagged as having credit card numbers in it. I don?t think that actually occurred. So was wondering if someone else had that package and what kind of results they are getting. Thank you. __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191212/264509ad/attachment.html From shirkdog.bsd at gmail.com Thu Dec 12 06:19:53 2019 From: shirkdog.bsd at gmail.com (Michael Shirk) Date: Thu, 12 Dec 2019 09:19:53 -0500 Subject: [Zeek] sethhall/credit-card-exposure In-Reply-To: References: Message-ID: You can submit a pull request to Seth's GitHub repo if you can share the modifications with the community. -- Michael Shirk Daemon Security, Inc. https://www.daemon-security.com On Thu, Dec 12, 2019, 09:18 Nick Turley wrote: > We?ve had pretty good luck with the package but we had to make > modifications to get it working the way we wanted. We also modified it so > it would work on Corelight. We?ve been running it on our Bro 2.6 cluster > for some time. SSN detection is a high false positive game in a large > environment like ours, so our analysts are still required to review the > extracted payload and make a determination. > > Some of the modifications include extracting a chunk of the payload where > the SSN was detected and including that in the notice log. We also added > the protocol that was detected and associated info. For example, if SMB, we > include the file name and location identified. As I recall, there was also > a bug we fixed that wasn?t masking the SSNs correctly. > > We also feed in all 50 state historical SSN prefixes and include the state > data in the notice log. However, SSNs after 2011 I believe are now > randomized so this will be less effective over time. > > While we get a number of false positives, the module has also helped us > discover some fairly serious security issues. > > When I get to the office, I would be happy to share our code. > > Nick Turley > Security Architect > CES Security Operations Center > Office: (801) 422-4994 | Cell: (801) 310-3816 | nick_turley at byu.edu > ------------------------------ > *From:* zeek-bounces at zeek.org on behalf of Scot > Harris > *Sent:* Thursday, December 12, 2019 6:26:27 AM > *To:* zeek at zeek.org > *Subject:* [Zeek] sethhall/credit-card-exposure > > > Does anyone have experience with the sethhall/credit-card-exposure package? > > > > I installed it and it is generating some results that does not seem valid. > > > > Running zeek 3.0 with this package installed using zkg. > > > > The odd data includes packets that go from my workstation to the zeek main > server on port 80 that is flagged as having credit card numbers in it. > > > > I don?t think that actually occurred. > > > > So was wondering if someone else had that package and what kind of results > they are getting. > > > > Thank you. > > > > > > > __________________________________________ > *Scot Harris* > Network Engineer > City of Hollywood > Information Technology > > P.O. Box 229045 > Hollywood, FL 33022-9045 > Office: 954-921-3304 > E-mail: SHARRIS at hollywoodfl.org > [image: www.hollywoodfl.org] > Notice: Florida has a broad public records law. All correspondence sent to > the City of Hollywood via e-mail may be subject to disclosure as a matter > of public record. > __________________________________________ > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191212/1ef7aa10/attachment-0001.html From nick_turley at byu.edu Thu Dec 12 06:47:51 2019 From: nick_turley at byu.edu (Nick Turley) Date: Thu, 12 Dec 2019 14:47:51 +0000 Subject: [Zeek] sethhall/credit-card-exposure In-Reply-To: References: , Message-ID: We?ve been meaning to share some of our work with the community so this has prompted a call to action :) Nick Turley Security Architect CES Security Operations Center Office: (801) 422-4994 | Cell: (801) 310-3816 | nick_turley at byu.edu ________________________________ From: Michael Shirk Sent: Thursday, December 12, 2019 7:19:53 AM To: Nick Turley Cc: Scot Harris ; zeek at zeek.org Subject: Re: [Zeek] sethhall/credit-card-exposure You can submit a pull request to Seth's GitHub repo if you can share the modifications with the community. -- Michael Shirk Daemon Security, Inc. https://www.daemon-security.com On Thu, Dec 12, 2019, 09:18 Nick Turley > wrote: We?ve had pretty good luck with the package but we had to make modifications to get it working the way we wanted. We also modified it so it would work on Corelight. We?ve been running it on our Bro 2.6 cluster for some time. SSN detection is a high false positive game in a large environment like ours, so our analysts are still required to review the extracted payload and make a determination. Some of the modifications include extracting a chunk of the payload where the SSN was detected and including that in the notice log. We also added the protocol that was detected and associated info. For example, if SMB, we include the file name and location identified. As I recall, there was also a bug we fixed that wasn?t masking the SSNs correctly. We also feed in all 50 state historical SSN prefixes and include the state data in the notice log. However, SSNs after 2011 I believe are now randomized so this will be less effective over time. While we get a number of false positives, the module has also helped us discover some fairly serious security issues. When I get to the office, I would be happy to share our code. Nick Turley Security Architect CES Security Operations Center Office: (801) 422-4994 | Cell: (801) 310-3816 | nick_turley at byu.edu ________________________________ From: zeek-bounces at zeek.org > on behalf of Scot Harris > Sent: Thursday, December 12, 2019 6:26:27 AM To: zeek at zeek.org > Subject: [Zeek] sethhall/credit-card-exposure Does anyone have experience with the sethhall/credit-card-exposure package? I installed it and it is generating some results that does not seem valid. Running zeek 3.0 with this package installed using zkg. The odd data includes packets that go from my workstation to the zeek main server on port 80 that is flagged as having credit card numbers in it. I don?t think that actually occurred. So was wondering if someone else had that package and what kind of results they are getting. Thank you. __________________________________________ Scot Harris Network Engineer City of Hollywood Information Technology P.O. Box 229045 Hollywood, FL 33022-9045 Office: 954-921-3304 E-mail: SHARRIS at hollywoodfl.org [www.hollywoodfl.org] Notice: Florida has a broad public records law. All correspondence sent to the City of Hollywood via e-mail may be subject to disclosure as a matter of public record. __________________________________________ _______________________________________________ Zeek mailing list zeek at zeek.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191212/f0408b0b/attachment.html From vern at corelight.com Thu Dec 12 13:21:35 2019 From: vern at corelight.com (Vern Paxson) Date: Thu, 12 Dec 2019 13:21:35 -0800 Subject: [Zeek] sethhall/credit-card-exposure In-Reply-To: (Thu, 12 Dec 2019 14:07:46 GMT). Message-ID: <20191212212135.BEB062C4015@rock.ICSI.Berkeley.EDU> Great - this will be really cool to have! Vern From akgraner at corelight.com Fri Dec 13 11:29:06 2019 From: akgraner at corelight.com (Amber Graner) Date: Fri, 13 Dec 2019 14:29:06 -0500 Subject: [Zeek] Ask the Zeeksperts - NEW Webinar Series Message-ID: Hi all - Happy Friday!! I'm excited to announce a new monthly webinar series--Ask the Zeeksperts. (A calendar invite will go out to the list) Have you ever wanted to ask folks like Seth Hall, Vern Paxon, Robin Sommer and many other Zeek experts questions about Zeek? Now is your chance. We'll start with a monthly webinar and if there is enough interest move to two webinars per month. ======== DETAILS: ======== * When: 19 Dec 2019 * Time: 10am EST* * Where: online at: https://register.gotowebinar.com/register/6208971152786498573 (*We will adjust the time based on demand. This is just the first in the series.) ============== Other information ============== These webinars are free and open to the public, but registration is required. These sessions will NOT be recorded. This series is meant to be a place where you can drop in and have a conversation with the Zeekspert on the call that day. This webinar series is being sponsored by Corelight, but your registration information will NOT be shared with Corelight and that information will NOT be used for marketing purposes. Please let me know if you have any questions Thanks, ~Amber -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 Schedule time on my calendar here. * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191213/295d89c4/attachment.html From akgraner at corelight.com Fri Dec 13 11:31:49 2019 From: akgraner at corelight.com (akgraner at corelight.com) Date: Fri, 13 Dec 2019 19:31:49 +0000 Subject: [Zeek] Invitation: Ask the Zeeksperts - Webinar (Seth Hall) @ Thu Dec 19, 2019 10am - 11am (EST) (zeek@zeek.org) Message-ID: <000000000000c844c905999ae9d3@google.com> You have been invited to the following event. Title: Ask the Zeeksperts - Webinar (Seth Hall) The "Ask The Zeeksperts" webinar series is a monthly series where Zeek Experts join the webinar and answer questions from the community.Seth Hall will be the Zeekspert on this call. This webinar series will NOT be recorded as we want to encourage people to ask any and all Zeek related questions.The webinar series is free to attend, but registration is required. Please note, while Corelight is sponsoring this series, your registration information will not be shared unless you opt in to sharing it. You can register at: https://register.gotowebinar.com/register/6208971152786498573 When: Thu Dec 19, 2019 10am ? 11am Eastern Time - New York Where: https://register.gotowebinar.com/register/6208971152786498573 Calendar: zeek at zeek.org Who: * akgraner at corelight.com - organizer * Robin Sommer * Seth Hall * john at corelight.com * vern at corelight.com * zeek at zeek.org Event details: https://www.google.com/calendar/event?action=VIEW&eid=NGg0anVuZTZoNDBtaTZ2MzUwNWZjYWdjaGEgemVla0B6ZWVrLm9yZw&tok=MjIjYWtncmFuZXJAY29yZWxpZ2h0LmNvbWM4ZjdhNjBkOWZiNzBkNzE4ZWFmNTVkOWQ3ZDMyNDg2ZDczZmEyYTE&ctz=America%2FNew_York&hl=en&es=0 Invitation from Google Calendar: https://www.google.com/calendar/ You are receiving this courtesy email at the account zeek at zeek.org because you are an attendee of this event. To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar. Forwarding this invitation could allow any recipient to send a response to the organizer and be added to the guest list, or invite others regardless of their own invitation status, or to modify your RSVP. Learn more at https://support.google.com/calendar/answer/37135#forwarding -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191213/553fe68c/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 2706 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191213/553fe68c/attachment-0002.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: invite.ics Type: application/ics Size: 2755 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191213/553fe68c/attachment-0003.bin From don.thomas.cissp at gmail.com Fri Dec 13 11:48:21 2019 From: don.thomas.cissp at gmail.com (Don Thomas) Date: Fri, 13 Dec 2019 11:48:21 -0800 Subject: [Zeek] Ask the Zeeksperts - NEW Webinar Series In-Reply-To: References: Message-ID: 7:00 AM ! Ouch. I will put that on my calendar... but... For the next ones... can you push the meeting to like noon EST (9:00 AM PST) ? Please ? Thank you. *Don Thomas, CISSP, CISA* On Fri, Dec 13, 2019 at 11:34 AM Amber Graner wrote: > Hi all - Happy Friday!! > > I'm excited to announce a new monthly webinar series--Ask the Zeeksperts. > (A calendar invite will go out to the list) > > Have you ever wanted to ask folks like Seth Hall, Vern Paxon, Robin Sommer > and many other Zeek experts questions about Zeek? > > Now is your chance. We'll start with a monthly webinar and if there is > enough interest move to two webinars per month. > > ======== > DETAILS: > ======== > > * When: 19 Dec 2019 > * Time: 10am EST* > * Where: online at: > https://register.gotowebinar.com/register/6208971152786498573 > > (*We will adjust the time based on demand. This is just the first in the > series.) > > ============== > Other information > ============== > > These webinars are free and open to the public, but registration is > required. These sessions will NOT be recorded. This series is meant to be > a place where you can drop in and have a conversation with the Zeekspert on > the call that day. > > This webinar series is being sponsored by Corelight, but your registration > information will NOT be shared with Corelight and that information will NOT > be used for marketing purposes. > > Please let me know if you have any questions > > Thanks, > ~Amber > > > > -- > > *Amber Graner* > Director of Community > Corelight, Inc > > 828.582.9469 > > Schedule time on my calendar here. > > > > * Ask me about how you can participate in the Zeek (formerly Bro) > community. > * Remember - ZEEK AND YOU SHALL FIND!! > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191213/ada966ed/attachment.html From akgraner at corelight.com Fri Dec 13 11:52:15 2019 From: akgraner at corelight.com (Amber Graner) Date: Fri, 13 Dec 2019 14:52:15 -0500 Subject: [Zeek] Ask the Zeeksperts - NEW Webinar Series In-Reply-To: References: Message-ID: This is just the first one. We'll definitely look into times that work for the west coast going forward. Thanks! ~Amber On Fri, Dec 13, 2019 at 2:48 PM Don Thomas wrote: > 7:00 AM ! Ouch. > > I will put that on my calendar... but... > > For the next ones... can you push the meeting to like noon EST (9:00 AM > PST) ? > > Please ? > > > Thank you. > > *Don Thomas, CISSP, CISA* > > > > On Fri, Dec 13, 2019 at 11:34 AM Amber Graner > wrote: > >> Hi all - Happy Friday!! >> >> I'm excited to announce a new monthly webinar series--Ask the Zeeksperts. >> (A calendar invite will go out to the list) >> >> Have you ever wanted to ask folks like Seth Hall, Vern Paxon, Robin >> Sommer and many other Zeek experts questions about Zeek? >> >> Now is your chance. We'll start with a monthly webinar and if there is >> enough interest move to two webinars per month. >> >> ======== >> DETAILS: >> ======== >> >> * When: 19 Dec 2019 >> * Time: 10am EST* >> * Where: online at: >> https://register.gotowebinar.com/register/6208971152786498573 >> >> (*We will adjust the time based on demand. This is just the first in the >> series.) >> >> ============== >> Other information >> ============== >> >> These webinars are free and open to the public, but registration is >> required. These sessions will NOT be recorded. This series is meant to be >> a place where you can drop in and have a conversation with the Zeekspert on >> the call that day. >> >> This webinar series is being sponsored by Corelight, but your >> registration information will NOT be shared with Corelight and that >> information will NOT be used for marketing purposes. >> >> Please let me know if you have any questions >> >> Thanks, >> ~Amber >> >> >> >> -- >> >> *Amber Graner* >> Director of Community >> Corelight, Inc >> >> 828.582.9469 >> >> Schedule time on my calendar here. >> >> >> >> * Ask me about how you can participate in the Zeek (formerly Bro) >> community. >> * Remember - ZEEK AND YOU SHALL FIND!! >> >> >> _______________________________________________ >> Zeek mailing list >> zeek at zeek.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 Schedule time on my calendar here. * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191213/664b6234/attachment.html From jgarciar at sia.es Mon Dec 16 03:38:20 2019 From: jgarciar at sia.es (Jorge Garcia Rodriguez) Date: Mon, 16 Dec 2019 11:38:20 +0000 Subject: [Zeek] High CPU Usage Message-ID: Hi, everyone I'm facing an issue regarding a high CPU usage in a Zeek machine, this cause packets dropped whenever a core reach 100% usage. We always have 1 core at 100% load and the others are around 60-80% Name Type Host Pid VSize Rss Cpu logger logger localhost 4666 2G 121M 53% manager manager localhost 4712 584M 114M 40% proxy-1 proxy localhost 4757 639M 148M 20% worker-1-1 worker localhost 4934 884M 393M 53% worker-1-2 worker localhost 4893 1G 596M 73% worker-1-3 worker localhost 4890 1G 592M 80% worker-1-4 worker localhost 4895 887M 395M 46% worker-1-5 worker localhost 4935 4G 3G 106% worker-1-6 worker localhost 4901 877M 385M 40% worker-1-7 worker localhost 4911 1G 581M 66% worker-1-8 worker localhost 4906 879M 389M 40% worker-1-9 worker localhost 4937 1G 576M 80% worker-1-10 worker localhost 4920 881M 391M 46% We have the next specifications : -x 1Intel Xeon E-2136 3.3GHz, 12M cache, 6C/12T, turbo (80W) -64GB RAM - And we are using PF_Ring to balance de traffic. The traffic that this Zeek manage is about 1,5GB/s with peaks of 2,5 at max. We don't know if this is a normal behavior or we need more Hardware to manage this amount of traffic or something that we have bad in the configuration. The node.cfg is the next one: [logger] type=logger host=localhost [manager] type=manager host=localhost [proxy-1] type=proxy host=localhost [worker-1] type=worker host=localhost interface=p1p1 lb_method=pf_ring lb_procs=10 pin_cpus=0,1,2,3,4,5,6,7,8,9 We have been testing different solutions posted before but nothing seems to take effect. I hope you can help me improve this. Also, is there a way to reduce the amount of CPU that Zeek use? For example disabling some scripts or something like that? Thank you all. Best Regards! Jorge Garc?a Rodr?guez Technical Consultant Security Infrastructures jgarciar at sia.es Grupo SIA Avda.Europa,2 - Alcor Plaza, Edificio B - Parque Oeste Alcorc?n 28922 Alcorc?n - Madrid Tlf: +34 902 480 580 Fax: +34 91 307 79 80 www.siainternational.com delivering value This e-mail and any attached files are intended solely for the addresse/s identified herein. It may contain confidential and/or legally privileged information and may not necessarily represent the opinion of SIA. No legally binding commitments will be created by this E-mail message. Where we intend to create legally binding commitments these will be made through hard copy correspondence or documents. If you receive this message by mistake, please immediately notify the sender and delete it since you are not authorized to use, disclose, distribute, print or copy all or part of the contained information Thank you. It is understood that the message was sent to you accidentally, although you appear as the addressee, you can see from the frame of existing relations that you were not the final addressee. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191216/ce37fcab/attachment.html From akgraner at corelight.com Mon Dec 16 12:00:03 2019 From: akgraner at corelight.com (Amber Graner) Date: Mon, 16 Dec 2019 15:00:03 -0500 Subject: [Zeek] 6 Dec 2019 - Community Call - Summary and Links In-Reply-To: References: Message-ID: Hi all, Below is the link to the follow-up poll from the call 6 December Zeek Community call. https://www.surveymonkey.com/r/alternate_to_IRC This poll will remain open until 7pm PST on Friday 20 December 2019. If you missed the call and would like to review the slides or listen to the recording, those can be found at: http://bit.ly/ZeekCommunityCall_6Dec19 Thank you in advance for your time and continued support of and participation in the Zeek Community. Please let me know if you have any questions. With gratitude, ~Amber On Sat, Dec 7, 2019 at 4:49 PM Amber Graner wrote: > Hi all, > > Below is the link to the folder that includes Slides, Audio, and Video > from the 6 Dec 2019 call. > > http://bit.ly/ZeekCommunityCall_6Dec19 > > Thank you so much to all those who participated. > > ------------ > Next Call > ------------ > > 3 January 2020 - 3pm ET > > ------------- > Summary > ------------- > We had 26 participants on the first monthly community call. > > Thank you to those who presented on alternatives to IRC: > > Alternatives to IRC Discussion/Presentations: > > - Matrix - Jan Grash?fer (alternative to IRC - Slack > integration available) > - Slack - Michael Dopheide (alternative to IRC) > - Discourse - Matt Trostel (alternative to Mailman, > includes integration with Slack and Matrix) > > Amber to send poll to the community about all options. Community to > review options between 9-13 December; poll to be sent out week of 16-20 > December. > > ----------------- > Other Topics Mentioned: > ----------------- > > - More Content Posted on the Zeek Blog > - If you have an idea for a blog post, please let Amber know or > send to the Zeek Mailing list. > - Monthly Newsletter > - Matt Trostel volunteered to help, but if you would like to help > please let Amber know or send to the mailing list. > - Raspberry Pi Zeek Images > - Image in call folder (no instructions > - Need to create a howto blog post > - ZeekWeek 2020 > - Increased Training and sponsor opportunities > - Dual track formats (One track geared toward threat > hunters/incident responders other traditional developer track) > - Look for location and date to be announced in January 2020 > - Zeek Packages > - If you have written packages that extend the capabilities of > Zeek, please consider opening them through the Zeek Package Manager. > - If you need help with this process please reach out to the list > or to Amber. > > Please let know if you have any questions, comments, feedback or thoughts > for the January meeting. > > With gratitude, > ~Amber > > -- > > *Amber Graner* > Director of Community > Corelight, Inc > > 828.582.9469 > > > > * Ask me about how you can participate in the Zeek (formerly Bro) > community. > * Remember - ZEEK AND YOU SHALL FIND!! > > > -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 Schedule time on my calendar here. * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191216/abf27c23/attachment.html From 13426106235 at 139.com Mon Dec 16 23:23:43 2019 From: 13426106235 at 139.com (=?utf-8?B?5ZGo5paM?=) Date: Tue, 17 Dec 2019 15:23:43 +0800 (CST) Subject: [Zeek] About FlipRoles function Message-ID: <2afd5df8758f564-00053.Richmail.01084752452839438249@139.com> Hi everybody, Sorry my English. I have noticed that in conn.cc(zeek-3.0.1\src) file there is a address translation in the method Connection::FlipRoles. The source code is: IPAddr tmp_addr = resp_addr resp_addr = orig_addr orig_addr = tmp_addr uint32 tmp_port = resp_port resp_port = orig_port orig_port = tmp_port I have tow questions: 1. When the function(Connection::FlipRoles) was called? 2. Not need to think of MAC address? And I've run into some technical problems recently. In conn.log, You can see: "id.orig_h":"Source IP","id.resp_h":"Destination IP",......"orig_l2_addr":"Destination MAC","resp_l2_addr":"Source MAC". Thanks, Zhoubin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191217/5f5da16b/attachment.html From robin at corelight.com Mon Dec 16 23:38:33 2019 From: robin at corelight.com (Robin Sommer) Date: Tue, 17 Dec 2019 07:38:33 +0000 Subject: [Zeek] About FlipRoles function In-Reply-To: <2afd5df8758f564-00053.Richmail.01084752452839438249@139.com> References: <2afd5df8758f564-00053.Richmail.01084752452839438249@139.com> Message-ID: <20191217073833.GF40519@corelight.com> On Tue, Dec 17, 2019 at 15:23 +0800, ?? wrote: > 1. When the function(Connection::FlipRoles) was called? There are a couple of places but the main one is when Zeek sees a partial connection that has a well-known port on the *originator* side. It then assumes that it must have missed the actual first packet because the well-known port would normally be on the responder side. So it flips the direction internally before doing anything further. > 2. Not need to think of MAC address? It should be flipping that, too, see the code for Connection::FlipRoles(). > And I've run into some technical problems recently. In conn.log, You can see: > "id.orig_h":"Source IP","id.resp_h":"Destination IP",......"orig_l2_addr":"Destination MAC","resp_l2_addr":"Source MAC". I'm not quite sure if you're saying you aren't seeing the MAC address being flipped? Or *they* are flipped, but not the IP addresses? Do you have a trace that shows what you're observing? Robin -- Robin Sommer * Corelight, Inc. * robin at corelight.com * www.corelight.com From akgraner at corelight.com Tue Dec 17 06:32:45 2019 From: akgraner at corelight.com (Amber Graner) Date: Tue, 17 Dec 2019 09:32:45 -0500 Subject: [Zeek] [New Blog Post] - Zeek Community Resources - Contributed by Johanna Amann Message-ID: Hi all, Check out the latest blog post on blog.zeek.org entitled 'Zeek Community Resources: Or, how can I get involved" by Johanna Amann. ( https://blog.zeek.org/2019/12/zeek-community-resources.html) In this post, Johanna goes through all the ways that Zeek Users can participate in the Zeek Community and Zeek out with users and contributors like yourselves. Check it out and if you have any questions about getting involved or if you have ideas for a blog post, meetups or workshops please let us know. Thanks and Happy Zeeking! ~Amber -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 Schedule time on my calendar here. * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191217/41e098ea/attachment.html From JorgeGarcia.1995 at outlook.es Wed Dec 18 02:29:33 2019 From: JorgeGarcia.1995 at outlook.es (=?iso-8859-1?Q?Jorge_Garc=EDa_Rodr=EDguez?=) Date: Wed, 18 Dec 2019 10:29:33 +0000 Subject: [Zeek] Zeek + PF_Ring Issue Message-ID: Hi Zeekers! I need to resolve a problem attached to Zeek when its configured to work with PF_Ring. The thing is that we receive between 1.0 and 2.5 GB/s in a fiber interface. Also when we lauch the command "Zeekctl top" to check the Cpu usage and the traffic managed in each worker, we see that the sum of the traffic of all workers is greater than the traffic we receive through the interface. This makes me think that we have something badly configured in PF_Ring or somehow Zeek is generating some kind of loop. For example, receiving 2Gb/s, i execute "Zeekctl top" and the result is the next one: Name Type Host Pid VSize Rss Cpu Cmd logger logger localhost 11474 3G 118M 50% zeek manager manager localhost 11520 589M 98M 25% zeek proxy-1 proxy localhost 11565 610M 113M 18% zeek worker-1-1 worker localhost 11693 1G 570M 62% zeek worker-1-2 worker localhost 11701 1G 574M 62% zeek worker-1-3 worker localhost 11711 1G 573M 68% zeek worker-1-4 worker localhost 11713 1G 572M 50% zeek worker-1-5 worker localhost 11718 3G 2G 106% zeek worker-1-6 worker localhost 11719 1G 567M 62% zeek worker-1-7 worker localhost 11726 1G 579M 68% zeek worker-1-8 worker localhost 11732 1G 575M 56% zeek worker-1-9 worker localhost 11733 1G 571M 68% zeek worker-1-10 worker localhost 11735 1G 558M 62% zeek Hope someone of you can help me to resolve this. Really thank you. Best Regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191218/343f5dfb/attachment.html From phil at brimsecurity.com Wed Dec 18 07:13:24 2019 From: phil at brimsecurity.com (Phil Rzewski) Date: Wed, 18 Dec 2019 07:13:24 -0800 Subject: [Zeek] Zeek + PF_Ring Issue In-Reply-To: References: Message-ID: Jorge, Have you checked for duplicate events in Zeek? I recall when I set up Zeek with PF_RING, I followed the instructions at https://www.zeek.org/documentation/load-balancing.html and only followed the instructions through the "Using PF_RING" paragraph. In my case I was pinning to four CPUs, and what I found was that I was getting four copies of the all sniffed network traffic onto my Zeek environment, one going to each worker. The symptom that tipped me off is that I would see was four "conn" events for a given connection, each with all the same source/dest/byte counts/etc. but each had a different UID. I suspect that if I continued on to additional paragraphs I would have been able to get past this problem (note how in the paragraph "Using PF_RING+DNA with symmetric RSS" it says "You can sniff each packet only once"... don't we always want that? :) ) Alas, I'm not 100% sure of the solution as I started using a different Zeek approach instead. Hope it helps though. -- Phil > On Dec 18, 2019, at 2:29 AM, Jorge Garc?a Rodr?guez wrote: > > Hi Zeekers! > > I need to resolve a problem attached to Zeek when its configured to work with PF_Ring. > > The thing is that we receive between 1.0 and 2.5 GB/s in a fiber interface. Also when we lauch the command "Zeekctl top" to check the Cpu usage and the traffic managed in each worker, we see that the sum of the traffic of all workers is greater than the traffic we receive through the interface. > > This makes me think that we have something badly configured in PF_Ring or somehow Zeek is generating some kind of loop. > > For example, receiving 2Gb/s, i execute "Zeekctl top" and the result is the next one: > > Name Type Host Pid VSize Rss Cpu Cmd > logger logger localhost 11474 3G 118M 50% zeek > manager manager localhost 11520 589M 98M 25% zeek > proxy-1 proxy localhost 11565 610M 113M 18% zeek > worker-1-1 worker localhost 11693 1G 570M 62% zeek > worker-1-2 worker localhost 11701 1G 574M 62% zeek > worker-1-3 worker localhost 11711 1G 573M 68% zeek > worker-1-4 worker localhost 11713 1G 572M 50% zeek > worker-1-5 worker localhost 11718 3G 2G 106% zeek > worker-1-6 worker localhost 11719 1G 567M 62% zeek > worker-1-7 worker localhost 11726 1G 579M 68% zeek > worker-1-8 worker localhost 11732 1G 575M 56% zeek > worker-1-9 worker localhost 11733 1G 571M 68% zeek > worker-1-10 worker localhost 11735 1G 558M 62% zeek > > Hope someone of you can help me to resolve this. > > Really thank you. > > Best Regards! > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191218/ae530746/attachment-0001.html From seth at corelight.com Wed Dec 18 12:48:39 2019 From: seth at corelight.com (Seth Hall) Date: Wed, 18 Dec 2019 15:48:39 -0500 Subject: [Zeek] sethhall/credit-card-exposure In-Reply-To: References: Message-ID: <672F6CAD-84F8-4500-9867-3F509CE43180@corelight.com> Awesome! Looking forward to any changes. And I agree about the results of that script, I've seen a few catches with that thing that are pretty bad and catching them was very nice. .Seth On 12 Dec 2019, at 9:47, Nick Turley wrote: > We?ve been meaning to share some of our work with the community so > this has prompted a call to action :) > > Nick Turley > Security Architect > CES Security Operations Center > Office: (801) 422-4994 | Cell: (801) 310-3816 | nick_turley at byu.edu > ________________________________ > From: Michael Shirk > Sent: Thursday, December 12, 2019 7:19:53 AM > To: Nick Turley > Cc: Scot Harris ; zeek at zeek.org > > Subject: Re: [Zeek] sethhall/credit-card-exposure > > You can submit a pull request to Seth's GitHub repo if you can share > the modifications with the community. > > -- > Michael Shirk > Daemon Security, Inc. > https://www.daemon-security.com > > On Thu, Dec 12, 2019, 09:18 Nick Turley > > wrote: > We?ve had pretty good luck with the package but we had to make > modifications to get it working the way we wanted. We also modified it > so it would work on Corelight. We?ve been running it on our Bro 2.6 > cluster for some time. SSN detection is a high false positive game in > a large environment like ours, so our analysts are still required to > review the extracted payload and make a determination. > > Some of the modifications include extracting a chunk of the payload > where the SSN was detected and including that in the notice log. We > also added the protocol that was detected and associated info. For > example, if SMB, we include the file name and location identified. As > I recall, there was also a bug we fixed that wasn?t masking the SSNs > correctly. > > We also feed in all 50 state historical SSN prefixes and include the > state data in the notice log. However, SSNs after 2011 I believe are > now randomized so this will be less effective over time. > > While we get a number of false positives, the module has also helped > us discover some fairly serious security issues. > > When I get to the office, I would be happy to share our code. > > Nick Turley > Security Architect > CES Security Operations Center > Office: (801) 422-4994 | Cell: (801) 310-3816 | > nick_turley at byu.edu > ________________________________ > From: zeek-bounces at zeek.org > > on behalf of > Scot Harris > > Sent: Thursday, December 12, 2019 6:26:27 AM > To: zeek at zeek.org > > > Subject: [Zeek] sethhall/credit-card-exposure > > > Does anyone have experience with the sethhall/credit-card-exposure > package? > > > > I installed it and it is generating some results that does not seem > valid. > > > > Running zeek 3.0 with this package installed using zkg. > > > > The odd data includes packets that go from my workstation to the zeek > main server on port 80 that is flagged as having credit card numbers > in it. > > > > I don?t think that actually occurred. > > > > So was wondering if someone else had that package and what kind of > results they are getting. > > > > Thank you. > > > > > > > > __________________________________________ > Scot Harris > Network Engineer > City of Hollywood > Information Technology > > P.O. Box 229045 > Hollywood, FL 33022-9045 > Office: 954-921-3304 > E-mail: SHARRIS at hollywoodfl.org > [www.hollywoodfl.org] > Notice: Florida has a broad public records law. All correspondence > sent to the City of Hollywood via e-mail may be subject to disclosure > as a matter of public record. > __________________________________________ > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Seth Hall * Corelight, Inc * www.corelight.com From phatbuckett at gmail.com Wed Dec 18 13:17:32 2019 From: phatbuckett at gmail.com (Darren S.) Date: Wed, 18 Dec 2019 14:17:32 -0700 Subject: [Zeek] Zeek + PF_Ring Issue In-Reply-To: References: Message-ID: I'm not certain if it's the exact root cause, but does the advice on PCAP_PF_RING_CLUSTER_ID at https://www.ntop.org/guides/pf_ring/thirdparty/bro.html apply? > ...Bro needs to setup a pf_ring kernel cluster in order to split the traffic across the processes (otherwise your get duplicated data). - Darren On Wed, Dec 18, 2019 at 8:16 AM Phil Rzewski wrote: > > Jorge, > > Have you checked for duplicate events in Zeek? I recall when I set up Zeek with PF_RING, I followed the instructions at https://www.zeek.org/documentation/load-balancing.html and only followed the instructions through the "Using PF_RING" paragraph. In my case I was pinning to four CPUs, and what I found was that I was getting four copies of the all sniffed network traffic onto my Zeek environment, one going to each worker. The symptom that tipped me off is that I would see was four "conn" events for a given connection, each with all the same source/dest/byte counts/etc. but each had a different UID. I suspect that if I continued on to additional paragraphs I would have been able to get past this problem (note how in the paragraph "Using PF_RING+DNA with symmetric RSS" it says "You can sniff each packet only once"... don't we always want that? :) ) Alas, I'm not 100% sure of the solution as I started using a different Zeek approach instead. Hope it helps though. > > -- > Phil > > > On Dec 18, 2019, at 2:29 AM, Jorge Garc?a Rodr?guez wrote: > > Hi Zeekers! > > I need to resolve a problem attached to Zeek when its configured to work with PF_Ring. > > The thing is that we receive between 1.0 and 2.5 GB/s in a fiber interface. Also when we lauch the command "Zeekctl top" to check the Cpu usage and the traffic managed in each worker, we see that the sum of the traffic of all workers is greater than the traffic we receive through the interface. > > This makes me think that we have something badly configured in PF_Ring or somehow Zeek is generating some kind of loop. > > For example, receiving 2Gb/s, i execute "Zeekctl top" and the result is the next one: > > Name Type Host Pid VSize Rss Cpu Cmd > logger logger localhost 11474 3G 118M 50% zeek > manager manager localhost 11520 589M 98M 25% zeek > proxy-1 proxy localhost 11565 610M 113M 18% zeek > worker-1-1 worker localhost 11693 1G 570M 62% zeek > worker-1-2 worker localhost 11701 1G 574M 62% zeek > worker-1-3 worker localhost 11711 1G 573M 68% zeek > worker-1-4 worker localhost 11713 1G 572M 50% zeek > worker-1-5 worker localhost 11718 3G 2G 106% zeek > worker-1-6 worker localhost 11719 1G 567M 62% zeek > worker-1-7 worker localhost 11726 1G 579M 68% zeek > worker-1-8 worker localhost 11732 1G 575M 56% zeek > worker-1-9 worker localhost 11733 1G 571M 68% zeek > worker-1-10 worker localhost 11735 1G 558M 62% zeek > > Hope someone of you can help me to resolve this. > > Really thank you. > > Best Regards! > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Darren Spruell phatbuckett at gmail.com From justin at corelight.com Wed Dec 18 13:29:37 2019 From: justin at corelight.com (Justin Azoff) Date: Wed, 18 Dec 2019 16:29:37 -0500 Subject: [Zeek] Zeek + PF_Ring Issue In-Reply-To: References: Message-ID: Can you run bro-doctor: https://packages.bro.org/packages/view/1251f948-f435-11e9-9321-0a645a3f3086 (works with zeek, just didn't change the name). that will likely tell you what is wrong. You're probably not actually using pf_ring and should use the native plugin and not the pcap wrapper. On Wed, Dec 18, 2019 at 5:31 AM Jorge Garc?a Rodr?guez < JorgeGarcia.1995 at outlook.es> wrote: > Hi Zeekers! > > I need to resolve a problem attached to Zeek when its configured to work > with PF_Ring. > > The thing is that we receive between 1.0 and 2.5 GB/s in a fiber > interface. Also when we lauch the command "Zeekctl top" to check the Cpu > usage and the traffic managed in each worker, we see that the sum of the > traffic of all workers is greater than the traffic we receive through the > interface. > > This makes me think that we have something badly configured in PF_Ring or > somehow Zeek is generating some kind of loop. > > For example, receiving 2Gb/s, i execute "Zeekctl top" and the result is > the next one: > > Name Type Host Pid VSize Rss Cpu Cmd > logger logger localhost 11474 3G 118M 50% zeek > manager manager localhost 11520 589M 98M 25% zeek > proxy-1 proxy localhost 11565 610M 113M 18% zeek > worker-1-1 worker localhost 11693 1G 570M 62% zeek > worker-1-2 worker localhost 11701 1G 574M 62% zeek > worker-1-3 worker localhost 11711 1G 573M 68% zeek > worker-1-4 worker localhost 11713 1G 572M 50% zeek > worker-1-5 worker localhost 11718 3G 2G 106% zeek > worker-1-6 worker localhost 11719 1G 567M 62% zeek > worker-1-7 worker localhost 11726 1G 579M 68% zeek > worker-1-8 worker localhost 11732 1G 575M 56% zeek > worker-1-9 worker localhost 11733 1G 571M 68% zeek > worker-1-10 worker localhost 11735 1G 558M 62% zeek > > Hope someone of you can help me to resolve this. > > Really thank you. > > Best Regards! > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191218/17ad11cc/attachment.html From SHARRIS at hollywoodfl.org Wed Dec 18 15:02:22 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Wed, 18 Dec 2019 23:02:22 +0000 Subject: [Zeek] Time value errors Message-ID: Noted what appear to be errors in the ntp.log file. Using following command: cat ntp.log | zeek-cut -d | less af_packet::eno1 2019-12-18T17:44:39-0500 C7MULpTngYof10ymf 10.1.45.35 123 10.1.5.60 123 2 3 4 64.000000 0.000004 0.070786 0.113083 10.1.5.60 2019-12-18T17:43:35-0500 2019-12-18T17:43:35-0500 2019-12-18T17:43:35-0500 2019-12-18T17:44:39-0500 0 af_packet::eno1 2019-12-18T17:44:39-0500 C7MULpTngYof10ymf 10.1.45.35 123 10.1.5.60 123 3 4 3 64.000000 0.015625 0.069839 0.077545 23.239.26.89 2019-:zeek-cut: time value out-of-range: -586465861.545972 zeek-cut: time value out-of-range: -586465861.545972 12-18T17:42:18-0500 2019-12-18T17:44:39-0500 2019-12-18T17:44:39-0500 2019-12-18T17:44:39-0500 0 af_packet::eno1 2019-12-18T17:44:39-0500 C5GF2T1ozzCZptCbjf 10.1.204.212 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-18T17:44:46-0500 0 af_packet::eno1 2019-12-18T17:44:40-0500 CxaJ6KeJfxVcN8Fw2 10.1.201.150 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-18T17:44:48-0500 0 af_packet::eno1 2019-12-18T17:44:40-0500 C8dZCI37SuYRZB9L7g 10.1.13.61 123 10.1.5.60 123 3 3 4 64.000000 0.007812 0.069839 0.402298 60.5.1.10 2019-12-18T17:43:37-0500 2019-12-18T17:43:36-0500 2019-12-18T17:43:37-0500 2019-12-18T17:44:41-0500 0 af_packet::eno1 2019-12-18T17:44:41-0500 CBz4Ww4jjCjKgHYfwc 10.1.221.30 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19:zeek-cut: time value out-of-range: -1114760693.379112 zeek-cut: time value out-of-range: -1114760693.379112 zeek-cut: time value out-of-range: -1115340513.842638 :00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-18T17:44:44-0500 0 af_packet::eno1 2019-12-18T17:44:40-0500 C4akh61szBCsYCPJn6 10.1.223.28 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19: Have not noticed these errors previously. ____________________________________________________ Scot Harris Network Engineer | IT | City of Hollywood (P) 954-921-3304 | sharris at hollywoodfl.org [CISSP_small] -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191218/99114053/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 1475 bytes Desc: image001.png Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191218/99114053/attachment-0001.bin From justin at corelight.com Wed Dec 18 15:49:59 2019 From: justin at corelight.com (Justin Azoff) Date: Wed, 18 Dec 2019 18:49:59 -0500 Subject: [Zeek] Time value errors In-Reply-To: References: Message-ID: If you run that without the -d option, what does the line containing negative times look like? There should be 4 times at the end of each record: ref_time org_time rec_time xmt_time, knowing which one(s) have the out of range value would help. Something like cat ntp.log |zeek-cut uid ref_time org_time rec_time xmt_time | fgrep -- - may help see them better. On Wed, Dec 18, 2019 at 6:08 PM Scot Harris wrote: > > Noted what appear to be errors in the ntp.log file. > > > > Using following command: > > > > cat ntp.log | zeek-cut ?d | less > > > > > > > > af_packet::eno1 2019-12-18T17:44:39-0500 C7MULpTngYof10ymf 10.1.45.35 123 10.1.5.60 123 2 3 4 64.000000 0.000004 0.070786 0.113083 10.1.5.60 2019-12-18T17:43:35-0500 2019-12-18T17:43:35-0500 2019-12-18T17:43:35-0500 2019-12-18T17:44:39-0500 0 > > af_packet::eno1 2019-12-18T17:44:39-0500 C7MULpTngYof10ymf 10.1.45.35 123 10.1.5.60 123 3 4 3 64.000000 0.015625 0.069839 0.077545 23.239.26.89 2019-:zeek-cut: time value out-of-range: -586465861.545972 > > zeek-cut: time value out-of-range: -586465861.545972 > > 12-18T17:42:18-0500 2019-12-18T17:44:39-0500 2019-12-18T17:44:39-0500 2019-12-18T17:44:39-0500 0 > > af_packet::eno1 2019-12-18T17:44:39-0500 C5GF2T1ozzCZptCbjf 10.1.204.212 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-18T17:44:46-0500 0 > > af_packet::eno1 2019-12-18T17:44:40-0500 CxaJ6KeJfxVcN8Fw2 10.1.201.150 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-18T17:44:48-0500 0 > > af_packet::eno1 2019-12-18T17:44:40-0500 C8dZCI37SuYRZB9L7g 10.1.13.61 123 10.1.5.60 123 3 3 4 64.000000 0.007812 0.069839 0.402298 60.5.1.10 2019-12-18T17:43:37-0500 2019-12-18T17:43:36-0500 2019-12-18T17:43:37-0500 2019-12-18T17:44:41-0500 0 > > af_packet::eno1 2019-12-18T17:44:41-0500 CBz4Ww4jjCjKgHYfwc 10.1.221.30 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19:zeek-cut: time value out-of-range: -1114760693.379112 > > zeek-cut: time value out-of-range: -1114760693.379112 > > zeek-cut: time value out-of-range: -1115340513.842638 > > :00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-18T17:44:44-0500 0 > > af_packet::eno1 2019-12-18T17:44:40-0500 C4akh61szBCsYCPJn6 10.1.223.28 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19: > > > > Have not noticed these errors previously. > > > > > > > > ____________________________________________________ > Scot Harris > Network Engineer | IT | City of Hollywood > > (P) 954-921-3304 | sharris at hollywoodfl.org > > > > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191218/2ecf6b58/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 1475 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191218/2ecf6b58/attachment.bin From JorgeGarcia.1995 at outlook.es Thu Dec 19 04:16:24 2019 From: JorgeGarcia.1995 at outlook.es (=?iso-8859-1?Q?Jorge_Garc=EDa_Rodr=EDguez?=) Date: Thu, 19 Dec 2019 12:16:24 +0000 Subject: [Zeek] Zeek + PF_Ring Issue In-Reply-To: References: , Message-ID: I have ran bro-doctor as you said and certainly I saw interesting things, for example: ################################################################### # Checking if connections are unevenly distributed across workers # ################################################################### error: The distribution of connections across workers seems uneven: worker-1-5: 462 connections worker-1-4: 890 connections worker-1-7: 874 connections worker-1-6: 4122 connections worker-1-1: 432 connections worker-1-3: 930 connections worker-1-2: 907 connections worker-1-9: 451 connections worker-1-8: 435 connections worker-1-10: 497 connections ############################################################################################################################### # Checking if anything is in the deprecated local-logger.bro, local-manager.bro, local-proxy.bro, or local-worker.bro scripts # ############################################################################################################################### Nothing found ###################################################################### # Checking if any recent connections have been logged multiple times # ###################################################################### ok, only 0.00%, 0 out of 2429 connections appear to be duplicate ################################## # Checking pf_ring configuration # ################################## configured to use pf_ring=True pcap=True plugin=False ############################################################################################################################### Let me know what do you think about the report. I have checked about the PF_Ring plugin but it gives me an error, im not sure if im following the last update of this plugin. https://github.com/ntop/bro-pf_ring Also doing a further investigation it seems that the script that is overcharguing the cpu is the weird.zeek ?Is there a way to disable this script? Thank you all for your replies. ________________________________ De: Justin Azoff Enviado: mi?rcoles, 18 de diciembre de 2019 22:29 Para: Jorge Garc?a Rodr?guez Cc: zeek at zeek.org Asunto: Re: [Zeek] Zeek + PF_Ring Issue Can you run bro-doctor: https://packages.bro.org/packages/view/1251f948-f435-11e9-9321-0a645a3f3086 (works with zeek, just didn't change the name). that will likely tell you what is wrong. You're probably not actually using pf_ring and should use the native plugin and not the pcap wrapper. On Wed, Dec 18, 2019 at 5:31 AM Jorge Garc?a Rodr?guez > wrote: Hi Zeekers! I need to resolve a problem attached to Zeek when its configured to work with PF_Ring. The thing is that we receive between 1.0 and 2.5 GB/s in a fiber interface. Also when we lauch the command "Zeekctl top" to check the Cpu usage and the traffic managed in each worker, we see that the sum of the traffic of all workers is greater than the traffic we receive through the interface. This makes me think that we have something badly configured in PF_Ring or somehow Zeek is generating some kind of loop. For example, receiving 2Gb/s, i execute "Zeekctl top" and the result is the next one: Name Type Host Pid VSize Rss Cpu Cmd logger logger localhost 11474 3G 118M 50% zeek manager manager localhost 11520 589M 98M 25% zeek proxy-1 proxy localhost 11565 610M 113M 18% zeek worker-1-1 worker localhost 11693 1G 570M 62% zeek worker-1-2 worker localhost 11701 1G 574M 62% zeek worker-1-3 worker localhost 11711 1G 573M 68% zeek worker-1-4 worker localhost 11713 1G 572M 50% zeek worker-1-5 worker localhost 11718 3G 2G 106% zeek worker-1-6 worker localhost 11719 1G 567M 62% zeek worker-1-7 worker localhost 11726 1G 579M 68% zeek worker-1-8 worker localhost 11732 1G 575M 56% zeek worker-1-9 worker localhost 11733 1G 571M 68% zeek worker-1-10 worker localhost 11735 1G 558M 62% zeek Hope someone of you can help me to resolve this. Really thank you. Best Regards! _______________________________________________ Zeek mailing list zeek at zeek.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191219/83ca26c3/attachment-0001.html From SHARRIS at hollywoodfl.org Thu Dec 19 05:21:18 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Thu, 19 Dec 2019 13:21:18 +0000 Subject: [Zeek] [EXT]Re: Time value errors In-Reply-To: References: Message-ID: Bigger issue possibly. A lot of zero values I check date/time on both zeek boxes and they are set correctly. CPURlj0fxNnhawrQk 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-19T06:59:59-0500 CPURlj0fxNnhawrQk 0.000000 0.000000 0.000000 1576756799.000161 Even the transmit date on some of the records are 1969. CQ2SXD4XyRGPpQCu9e 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-19T07:00:02-0500 CLpVPg3841dexUbAu6 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 CLpVPg3841dexUbAu6 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 CxTT3e4BKVjQ9ogjng 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-19T07:00:00-0500 CGFPn54Ff0m4cIkr5e 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-19T07:00:04-0500 CXMvpj1SJy4aBwQ81i 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-19T07:00:04-0500 CQ2SXD4XyRGPpQCu9e 0.000000 0.000000 0.000000 1576756802.000145 CLpVPg3841dexUbAu6 0.000000 0.000000 0.000000 0.000000 CLpVPg3841dexUbAu6 0.000000 0.000000 0.000000 0.000000 CxTT3e4BKVjQ9ogjng 0.000000 0.000000 0.000000 1576756800.889992 CGFPn54Ff0m4cIkr5e 0.000000 0.000000 0.000000 1576756804.000129 CXMvpj1SJy4aBwQ81i 0.000000 0.000000 0.000000 1576756804.000076 Should those fields have zero values? That is why they are being displayed as start of epoch. From: Justin Azoff [mailto:justin at corelight.com] Sent: Wednesday, December 18, 2019 6:50 PM To: Scot Harris Cc: zeek at zeek.org Subject: [EXT]Re: [Zeek] Time value errors If you run that without the -d option, what does the line containing negative times look like? There should be 4 times at the end of each record: ref_time org_time rec_time xmt_time, knowing which one(s) have the out of range value would help. Something like cat ntp.log |zeek-cut uid ref_time org_time rec_time xmt_time | fgrep -- - may help see them better. On Wed, Dec 18, 2019 at 6:08 PM Scot Harris > wrote: > > Noted what appear to be errors in the ntp.log file. > > > > Using following command: > > > > cat ntp.log | zeek-cut ?d | less > > > > > > > > af_packet::eno1 2019-12-18T17:44:39-0500 C7MULpTngYof10ymf 10.1.45.35 123 10.1.5.60 123 2 3 4 64.000000 0.000004 0.070786 0.113083 10.1.5.60 2019-12-18T17:43:35-0500 2019-12-18T17:43:35-0500 2019-12-18T17:43:35-0500 2019-12-18T17:44:39-0500 0 > > af_packet::eno1 2019-12-18T17:44:39-0500 C7MULpTngYof10ymf 10.1.45.35 123 10.1.5.60 123 3 4 3 64.000000 0.015625 0.069839 0.077545 23.239.26.89 2019-:zeek-cut: time value out-of-range: -586465861.545972 > > zeek-cut: time value out-of-range: -586465861.545972 > > 12-18T17:42:18-0500 2019-12-18T17:44:39-0500 2019-12-18T17:44:39-0500 2019-12-18T17:44:39-0500 0 > > af_packet::eno1 2019-12-18T17:44:39-0500 C5GF2T1ozzCZptCbjf 10.1.204.212 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-18T17:44:46-0500 0 > > af_packet::eno1 2019-12-18T17:44:40-0500 CxaJ6KeJfxVcN8Fw2 10.1.201.150 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-18T17:44:48-0500 0 > > af_packet::eno1 2019-12-18T17:44:40-0500 C8dZCI37SuYRZB9L7g 10.1.13.61 123 10.1.5.60 123 3 3 4 64.000000 0.007812 0.069839 0.402298 60.5.1.10 2019-12-18T17:43:37-0500 2019-12-18T17:43:36-0500 2019-12-18T17:43:37-0500 2019-12-18T17:44:41-0500 0 > > af_packet::eno1 2019-12-18T17:44:41-0500 CBz4Ww4jjCjKgHYfwc 10.1.221.30 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19:zeek-cut: time value out-of-range: -1114760693.379112 > > zeek-cut: time value out-of-range: -1114760693.379112 > > zeek-cut: time value out-of-range: -1115340513.842638 > > :00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-18T17:44:44-0500 0 > > af_packet::eno1 2019-12-18T17:44:40-0500 C4akh61szBCsYCPJn6 10.1.223.28 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19: > > > > Have not noticed these errors previously. > > > > > > > > > > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Justin CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191219/1a969742/attachment-0001.html From justin at corelight.com Thu Dec 19 07:01:52 2019 From: justin at corelight.com (Justin Azoff) Date: Thu, 19 Dec 2019 10:01:52 -0500 Subject: [Zeek] [EXT]Re: Time value errors In-Reply-To: References: Message-ID: On Thu, Dec 19, 2019 at 8:21 AM Scot Harris wrote: > Bigger issue possibly. A lot of zero values > > That's probably a smaller issue :-) it's logging unknown or unset values as 0, when it should maybe just be optional fields that are logged as unset values.. unfortunately with timestamps 0 gets turned into 1969.. we can probably fix that in zeek-cut to not format 0 as a timestamp though. -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191219/5bccab37/attachment.html From justin at corelight.com Thu Dec 19 07:06:47 2019 From: justin at corelight.com (Justin Azoff) Date: Thu, 19 Dec 2019 10:06:47 -0500 Subject: [Zeek] Zeek + PF_Ring Issue In-Reply-To: References: Message-ID: On Thu, Dec 19, 2019 at 7:18 AM Jorge Garc?a Rodr?guez < JorgeGarcia.1995 at outlook.es> wrote: > I have ran bro-doctor as you said and certainly I saw interesting things, > for example: > > > ################################################################### > # Checking if connections are unevenly distributed across workers # > ################################################################### > error: The distribution of connections across workers seems uneven: > worker-1-5: 462 connections > worker-1-4: 890 connections > worker-1-7: 874 connections > worker-1-6: 4122 connections > worker-1-1: 432 connections > worker-1-3: 930 connections > worker-1-2: 907 connections > worker-1-9: 451 connections > worker-1-8: 435 connections > worker-1-10: 497 connections > Interesting indeed. If you look at your conn log can you tell anything about all those connections that worker-1-6 is seeing? > Let me know what do you think about the report. > > I have checked about the PF_Ring plugin but it gives me an error, im not > sure if im following the last update of this plugin. > https://github.com/ntop/bro-pf_ring > you should be able to zkg install bro-pf_ring.. or install it manually with ./configure && make && sudo make install. are you getting an error when you do that? > Also doing a further investigation it seems that the script that is > overcharguing the cpu is the weird.zeek ?Is there a way to disable this > script? > Do you say that because you have a lot of entries in the weird log? that points to traffic issues that need to be fixed... disabling the weird logs will just ignore the problem. What are the top weirds that you are seeing? cat /usr/local/zeek/logs/current/weird.log |zeek-cut name|sort|uniq -c|sort -rn What did you see as the result from this check? # Checking if many recent connections have a SAD or had history -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191219/8c47dc7f/attachment.html From JorgeGarcia.1995 at outlook.es Thu Dec 19 07:55:47 2019 From: JorgeGarcia.1995 at outlook.es (=?iso-8859-1?Q?Jorge_Garc=EDa_Rodr=EDguez?=) Date: Thu, 19 Dec 2019 15:55:47 +0000 Subject: [Zeek] Zeek + PF_Ring Issue In-Reply-To: References: , Message-ID: Enviado desde Outlook ________________________________ De: Justin Azoff Enviado: jueves, 19 de diciembre de 2019 16:06 Para: Jorge Garc?a Rodr?guez Cc: zeek at zeek.org Asunto: Re: [Zeek] Zeek + PF_Ring Issue Thank you for your reply Also doing a further investigation it seems that the script that is overcharguing the cpu is the weird.zeek ?Is there a way to disable this script? Do you say that because you have a lot of entries in the weird log? that points to traffic issues that need to be fixed... disabling the weird logs will just ignore the problem. What are the top weirds that you are seeing? cat /usr/local/zeek/logs/current/weird.log |zeek-cut name|sort|uniq -c|sort -rn I have 160167 entries in like 10 minutes. After 20 mins I have a total of 329572 entries and 245954 of them are bad_HTTP_request What did you see as the result from this check? # Checking if many recent connections have a SAD or had history ################################################################# # Checking if many recent connections have a SAD or had history # ################################################################# error: 52.91%, 33795 out of 63873 connections are half duplex Best Regards! -- Jorge -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191219/32d65aaf/attachment.html From johanna at icir.org Fri Dec 20 00:02:35 2019 From: johanna at icir.org (Johanna Amann) Date: Fri, 20 Dec 2019 09:02:35 +0100 Subject: [Zeek] 6 Dec 2019 - Community Call - Summary and Links In-Reply-To: References: Message-ID: <72DBE687-32C6-48E8-827B-6BD3120D31D0@icir.org> Hi, In news related to this - Mozilla just went through a long process to decide on what to do for synchronous messaging. They published a thread with their decision at https://discourse.mozilla.org/t/synchronous-messaging-at-mozilla-the-decision/50620, which might be worth a read. The short version is that they chose Matrix, after evaluating a lot of options; two of the main reasons are community safety and accessibility. Johanna On 16 Dec 2019, at 21:00, Amber Graner wrote: > Hi all, > > Below is the link to the follow-up poll from the call 6 December Zeek > Community call. > > https://www.surveymonkey.com/r/alternate_to_IRC > > > This poll will remain open until 7pm PST on Friday 20 December 2019. > > If you missed the call and would like to review the slides or listen > to the > recording, those can be found at: > > http://bit.ly/ZeekCommunityCall_6Dec19 > > Thank you in advance for your time and continued support of and > participation in the Zeek Community. > > Please let me know if you have any questions. > > With gratitude, > ~Amber > > On Sat, Dec 7, 2019 at 4:49 PM Amber Graner > wrote: > >> Hi all, >> >> Below is the link to the folder that includes Slides, Audio, and >> Video >> from the 6 Dec 2019 call. >> >> http://bit.ly/ZeekCommunityCall_6Dec19 >> >> Thank you so much to all those who participated. >> >> ------------ >> Next Call >> ------------ >> >> 3 January 2020 - 3pm ET >> >> ------------- >> Summary >> ------------- >> We had 26 participants on the first monthly community call. >> >> Thank you to those who presented on alternatives to IRC: >> >> Alternatives to IRC Discussion/Presentations: >> >> - Matrix - Jan Grash?fer (alternative to IRC - Slack >> integration available) >> - Slack - Michael Dopheide (alternative to IRC) >> - Discourse - Matt Trostel (alternative to Mailman, >> includes integration with Slack and Matrix) >> >> Amber to send poll to the community about all options. Community to >> review options between 9-13 December; poll to be sent out week of >> 16-20 >> December. >> >> ----------------- >> Other Topics Mentioned: >> ----------------- >> >> - More Content Posted on the Zeek Blog >> - If you have an idea for a blog post, please let Amber know or >> send to the Zeek Mailing list. >> - Monthly Newsletter >> - Matt Trostel volunteered to help, but if you would like to >> help >> please let Amber know or send to the mailing list. >> - Raspberry Pi Zeek Images >> - Image in call folder (no instructions >> - Need to create a howto blog post >> - ZeekWeek 2020 >> - Increased Training and sponsor opportunities >> - Dual track formats (One track geared toward threat >> hunters/incident responders other traditional developer track) >> - Look for location and date to be announced in January 2020 >> - Zeek Packages >> - If you have written packages that extend the capabilities of >> Zeek, please consider opening them through the Zeek Package >> Manager. >> - If you need help with this process please reach out to the >> list >> or to Amber. >> >> Please let know if you have any questions, comments, feedback or >> thoughts >> for the January meeting. >> >> With gratitude, >> ~Amber >> >> -- >> >> *Amber Graner* >> Director of Community >> Corelight, Inc >> >> 828.582.9469 >> >> >> >> * Ask me about how you can participate in the Zeek (formerly Bro) >> community. >> * Remember - ZEEK AND YOU SHALL FIND!! >> >> >> > > -- > > *Amber Graner* > Director of Community > Corelight, Inc > > 828.582.9469 > > Schedule time on my calendar here. > > > > * Ask me about how you can participate in the Zeek (formerly Bro) > community. > * Remember - ZEEK AND YOU SHALL FIND!! > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek From clopmz at outlook.com Fri Dec 20 00:46:56 2019 From: clopmz at outlook.com (Carlos Lopez) Date: Fri, 20 Dec 2019 08:46:56 +0000 Subject: [Zeek] Debbuging zeekctl errors Message-ID: <98309A0D-2BA7-4DF7-9407-DF90F46BB951@outlook.com> Hi all, I have tried to upgrade to Zeek 3.0.1 from Zeek 3.0.0 but I am seeing a lot of errors: Warning: ZeekControl plugin uses legacy BroControl API. Use 'import ZeekControl.plugin' instead of 'import BroControl.plugin' removing old policies in /nsm/zeek/spool/installed-scripts-do-not-touch/site ... removing old policies in /nsm/zeek/spool/installed-scripts-do-not-touch/auto ... creating policy directories ... installing site policies ... generating cluster-layout.zeek ... generating local-networks.zeek ... generating zeekctl-config.zeek ... generating zeekctl-config.sh ... updating nodes ... sh: line 1: /home/zeek/venv/bin/python: No such file or directory sh: line 1: /home/zeek/venv/bin/python: No such file or directory sh: line 2: [/bin/echo,: No such file or directory sh: line 3: syntax error near unexpected token `done' sh: line 3: `done' sh: line 2: [/bin/echo,: No such file or directory sh: line 3: syntax error near unexpected token `done' sh: line 3: `done' Error: cannot create a directory on node worker-1 Error: Failed to establish ssh connection to host 172.22.58.2 setcap plugin: executing setcap on each node: sh: line 1: /home/zeek/venv/bin/python: No such file or directory sh: line 1: /home/zeek/venv/bin/python: No such file or directory sh: line 2: [/bin/echo,: No such file or directory sh: line 3: syntax error near unexpected token `done' sh: line 3: `done' sh: line 2: [/bin/echo,: No such file or directory sh: line 3: syntax error near unexpected token `done' sh: line 3: `done' How can I try to debug these errors? ?In 3.0.0 release all works well ? For example, : /home/zeek/venv/ is python?s virtualenv for zkg ? and I don?t understand why it is a requirement for nodes ? -- Regards, C. L. Martinez -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/37ab47cb/attachment.html From clopmz at outlook.com Fri Dec 20 00:53:47 2019 From: clopmz at outlook.com (Carlos Lopez) Date: Fri, 20 Dec 2019 08:53:47 +0000 Subject: [Zeek] Documentation about Corelight's Splunk Apps for Zeek Message-ID: <7F534E24-79F1-4F87-8CDF-71611D8255FA@outlook.com> Hi all, I would like to install Corelight App For Splunk and TA for Corelight, but there is no documentation about how to accomplish it ? All info points to https://www.corelight.com/support/, but there is no docs in there ? Any idea? -- Regards, C. L. Martinez -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/06541808/attachment-0001.html From akgraner at corelight.com Fri Dec 20 04:03:48 2019 From: akgraner at corelight.com (Amber Graner) Date: Fri, 20 Dec 2019 07:03:48 -0500 Subject: [Zeek] Documentation about Corelight's Splunk Apps for Zeek In-Reply-To: <7F534E24-79F1-4F87-8CDF-71611D8255FA@outlook.com> References: <7F534E24-79F1-4F87-8CDF-71611D8255FA@outlook.com> Message-ID: Hi Carlos, As that is a Corelight offering and not something maintained by the Zeek Project or the community, we?d have to refer you to Corelight. Let me find out who you need to talk to and I?ll make introductions. Thanks, ~Amber On Fri, Dec 20, 2019 at 3:56 AM Carlos Lopez wrote: > Hi all, > > > > I would like to install Corelight App For Splunk and TA for Corelight, but > there is no documentation about how to accomplish it ? All info points to > https://www.corelight.com/support/, but there is no docs in there ? > > > > Any idea? > > > > > > -- > > Regards, > > C. L. Martinez > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 Schedule time on my calendar here. * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/f4bc058d/attachment.html From clopmz at outlook.com Fri Dec 20 04:06:35 2019 From: clopmz at outlook.com (Carlos Lopez) Date: Fri, 20 Dec 2019 12:06:35 +0000 Subject: [Zeek] Debbuging zeekctl errors (SOLVED) Message-ID: Problem solved. I had to do something wrong during the upgrade process. I reinstalled from sources by restoring the configuration that worked for me and everything is ok now. Sorry for the noise ? -- Regards, C. L. Martinez From: on behalf of Carlos Lopez Date: Friday, 20 December 2019 at 09:52 To: "zeek at zeek.org" Subject: [Zeek] Debbuging zeekctl errors Hi all, I have tried to upgrade to Zeek 3.0.1 from Zeek 3.0.0 but I am seeing a lot of errors: Warning: ZeekControl plugin uses legacy BroControl API. Use 'import ZeekControl.plugin' instead of 'import BroControl.plugin' removing old policies in /nsm/zeek/spool/installed-scripts-do-not-touch/site ... removing old policies in /nsm/zeek/spool/installed-scripts-do-not-touch/auto ... creating policy directories ... installing site policies ... generating cluster-layout.zeek ... generating local-networks.zeek ... generating zeekctl-config.zeek ... generating zeekctl-config.sh ... updating nodes ... sh: line 1: /home/zeek/venv/bin/python: No such file or directory sh: line 1: /home/zeek/venv/bin/python: No such file or directory sh: line 2: [/bin/echo,: No such file or directory sh: line 3: syntax error near unexpected token `done' sh: line 3: `done' sh: line 2: [/bin/echo,: No such file or directory sh: line 3: syntax error near unexpected token `done' sh: line 3: `done' Error: cannot create a directory on node worker-1 Error: Failed to establish ssh connection to host 172.22.58.2 setcap plugin: executing setcap on each node: sh: line 1: /home/zeek/venv/bin/python: No such file or directory sh: line 1: /home/zeek/venv/bin/python: No such file or directory sh: line 2: [/bin/echo,: No such file or directory sh: line 3: syntax error near unexpected token `done' sh: line 3: `done' sh: line 2: [/bin/echo,: No such file or directory sh: line 3: syntax error near unexpected token `done' sh: line 3: `done' How can I try to debug these errors? ?In 3.0.0 release all works well ? For example, : /home/zeek/venv/ is python?s virtualenv for zkg ? and I don?t understand why it is a requirement for nodes ? -- Regards, C. L. Martinez -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/7e3cc6be/attachment.html From ericooi at gmail.com Fri Dec 20 04:52:40 2019 From: ericooi at gmail.com (Eric Ooi) Date: Fri, 20 Dec 2019 12:52:40 +0000 Subject: [Zeek] Documentation about Corelight's Splunk Apps for Zeek In-Reply-To: References: <7F534E24-79F1-4F87-8CDF-71611D8255FA@outlook.com>, Message-ID: Assuming you?re doing an install on a standalone Splunk server, you can use my guide here: https://www.ericooi.com/zeekurity-zen-part-iii-how-to-send-zeek-bro-logs-to-splunk/ ________________________________ From: zeek-bounces at zeek.org on behalf of Amber Graner Sent: Friday, December 20, 2019 6:13 AM To: Carlos Lopez Cc: zeek at zeek.org Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek Hi Carlos, As that is a Corelight offering and not something maintained by the Zeek Project or the community, we?d have to refer you to Corelight. Let me find out who you need to talk to and I?ll make introductions. Thanks, ~Amber On Fri, Dec 20, 2019 at 3:56 AM Carlos Lopez > wrote: Hi all, I would like to install Corelight App For Splunk and TA for Corelight, but there is no documentation about how to accomplish it ? All info points to https://www.corelight.com/support/, but there is no docs in there ? Any idea? -- Regards, C. L. Martinez _______________________________________________ Zeek mailing list zeek at zeek.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Amber Graner Director of Community Corelight, Inc 828.582.9469 Schedule time on my calendar here. [https://docs.google.com/uc?export=download&id=1Pst1UL56ibuX2pFmisz34iKLVlU8LSjd&revid=0B0wAXWxdbUhfQmZtTHhhTlJtS3ZuZnN1QkRDRW80UjZnTno4PQ] [https://docs.google.com/uc?export=download&id=1eqoMcNL7TDN2I-N9dIlYt0Yidwg04aeu&revid=0B0wAXWxdbUhfSTdRUThMWitxYUE5MTczVHpIUUFIVE5jdmFvPQ] * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/82ef1bac/attachment-0001.html From clopmz at outlook.com Fri Dec 20 06:22:45 2019 From: clopmz at outlook.com (Carlos Lopez) Date: Fri, 20 Dec 2019 14:22:45 +0000 Subject: [Zeek] Documentation about Corelight's Splunk Apps for Zeek In-Reply-To: References: <7F534E24-79F1-4F87-8CDF-71611D8255FA@outlook.com> Message-ID: <84B4DEB5-7131-445F-951E-DF4AA2248FBA@outlook.com> Thanks Eric. But I have doubt with your setup. For inputs.conf, maybe this configuration is best? [monitor:///opt/zeek/logs/spool/current] disabled = 0 sourcetype = zeek:json whitelist = \.log$ instead of to put file by file? -- Regards, C. L. Martinez From: Eric Ooi Date: Friday, 20 December 2019 at 13:52 To: Amber Graner , Carlos Lopez Cc: "zeek at zeek.org" Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek Assuming you?re doing an install on a standalone Splunk server, you can use my guide here: https://www.ericooi.com/zeekurity-zen-part-iii-how-to-send-zeek-bro-logs-to-splunk/ ________________________________ From: zeek-bounces at zeek.org on behalf of Amber Graner Sent: Friday, December 20, 2019 6:13 AM To: Carlos Lopez Cc: zeek at zeek.org Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek Hi Carlos, As that is a Corelight offering and not something maintained by the Zeek Project or the community, we?d have to refer you to Corelight. Let me find out who you need to talk to and I?ll make introductions. Thanks, ~Amber On Fri, Dec 20, 2019 at 3:56 AM Carlos Lopez > wrote: Hi all, I would like to install Corelight App For Splunk and TA for Corelight, but there is no documentation about how to accomplish it ? All info points to https://www.corelight.com/support/, but there is no docs in there ? Any idea? -- Regards, C. L. Martinez _______________________________________________ Zeek mailing list zeek at zeek.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Amber Graner Director of Community Corelight, Inc 828.582.9469 Schedule time on my calendar here. [Image removed by sender.] [Image removed by sender.] * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/2411d84c/attachment.html From ericooi at gmail.com Fri Dec 20 06:27:44 2019 From: ericooi at gmail.com (ericooi at gmail.com) Date: Fri, 20 Dec 2019 08:27:44 -0600 Subject: [Zeek] Documentation about Corelight's Splunk Apps for Zeek In-Reply-To: <84B4DEB5-7131-445F-951E-DF4AA2248FBA@outlook.com> References: <7F534E24-79F1-4F87-8CDF-71611D8255FA@outlook.com> <84B4DEB5-7131-445F-951E-DF4AA2248FBA@outlook.com> Message-ID: <5CC1EA46-DCEA-43BD-AD4D-76DBB1FCA701@gmail.com> Hi Carlos, ?Best? is subjective. For someone who wants all logs and a short inputs.conf file, your suggestion will work. My example is geared towards the fact that these logs are large and depending on your Splunk license and requirements, you may not actually want to ingest every single log file into your system. Ultimately, you know your environment and needs best which is why I also state in the writeup: "An example inputs.conf is below but may or may not include the logs you wish to ingest...Modify the index and sourcetype configurations to your needs.? Hope that helps! Eric > On Dec 20, 2019, at 8:22 AM, Carlos Lopez wrote: > > Thanks Eric. But I have doubt with your setup. For inputs.conf, maybe this configuration is best? > > [monitor:///opt/zeek/logs/spool/current ] > disabled = 0 > sourcetype = zeek:json > whitelist = \.log$ > > instead of to put file by file? > -- > Regards, > C. L. Martinez > > From: Eric Ooi > Date: Friday, 20 December 2019 at 13:52 > To: Amber Graner , Carlos Lopez > Cc: "zeek at zeek.org" > Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek > > Assuming you?re doing an install on a standalone Splunk server, you can use my guide here: https://www.ericooi.com/zeekurity-zen-part-iii-how-to-send-zeek-bro-logs-to-splunk/ > > > From: zeek-bounces at zeek.org on behalf of Amber Graner > Sent: Friday, December 20, 2019 6:13 AM > To: Carlos Lopez > Cc: zeek at zeek.org > Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek > > Hi Carlos, > > As that is a Corelight offering and not something maintained by the Zeek Project or the community, we?d have to refer you to Corelight. > > Let me find out who you need to talk to and I?ll make introductions. > > Thanks, > ~Amber > > On Fri, Dec 20, 2019 at 3:56 AM Carlos Lopez > wrote: > Hi all, > > I would like to install Corelight App For Splunk and TA for Corelight, but there is no documentation about how to accomplish it ? All info points tohttps://www.corelight.com/support/ , but there is no docs in there ? > > Any idea? > > > -- > Regards, > C. L. Martinez > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > -- > > Amber Graner > Director of Community > Corelight, Inc > > 828.582.9469 > > Schedule time on my calendar here. > > > > * Ask me about how you can participate in the Zeek (formerly Bro) community. > * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/7daac152/attachment-0001.html From patrick.kelley at criticalpathsecurity.com Fri Dec 20 06:35:15 2019 From: patrick.kelley at criticalpathsecurity.com (Patrick Kelley) Date: Fri, 20 Dec 2019 09:35:15 -0500 Subject: [Zeek] Documentation about Corelight's Splunk Apps for Zeek In-Reply-To: <5CC1EA46-DCEA-43BD-AD4D-76DBB1FCA701@gmail.com> References: <7F534E24-79F1-4F87-8CDF-71611D8255FA@outlook.com> <84B4DEB5-7131-445F-951E-DF4AA2248FBA@outlook.com> <5CC1EA46-DCEA-43BD-AD4D-76DBB1FCA701@gmail.com> Message-ID: Seconding the statements of Eric, Splunk costs can get to be expensive extremely quick with Zeek. My only secondary suggestion is that you ingest individual logs to provide a bit more granularity and control. You might not wish to ingest every log due to the processing and storage costs. In the past, I've tried leveraging Splunk multiple times due to my familiarity. In the end, we've built our stack around Elastic. We were just spending too much time servicing the hammer, instead of building the house. On Fri, Dec 20, 2019 at 9:29 AM ericooi at gmail.com wrote: > Hi Carlos, > > ?Best? is subjective. For someone who wants all logs and a short > inputs.conf file, your suggestion will work. My example is geared towards > the fact that these logs are large and depending on your Splunk license and > requirements, you may not actually want to ingest every single log file > into your system. Ultimately, you know your environment and needs best > which is why I also state in the writeup: > > "An example inputs.conf is below but may or may not include the logs you > wish to ingest...Modify the index and sourcetype configurations to your > needs.? > > Hope that helps! > Eric > > On Dec 20, 2019, at 8:22 AM, Carlos Lopez wrote: > > Thanks Eric. But I have doubt with your setup. For inputs.conf, maybe this > configuration is best? > > [monitor:///opt/zeek/logs/spool/current] > disabled = 0 > sourcetype = zeek:json > whitelist = \.log$ > > instead of to put file by file? > -- > Regards, > C. L. Martinez > > *From: *Eric Ooi > *Date: *Friday, 20 December 2019 at 13:52 > *To: *Amber Graner , Carlos Lopez < > clopmz at outlook.com> > *Cc: *"zeek at zeek.org" > *Subject: *Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek > > Assuming you?re doing an install on a standalone Splunk server, you can > use my guide here: > https://www.ericooi.com/zeekurity-zen-part-iii-how-to-send-zeek-bro-logs-to-splunk/ > > > ------------------------------ > *From:* zeek-bounces at zeek.org on behalf of Amber Graner < > akgraner at corelight.com> > *Sent:* Friday, December 20, 2019 6:13 AM > *To:* Carlos Lopez > *Cc:* zeek at zeek.org > *Subject:* Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek > > > Hi Carlos, > > As that is a Corelight offering and not something maintained by the Zeek > Project or the community, we?d have to refer you to Corelight. > > Let me find out who you need to talk to and I?ll make introductions. > > Thanks, > ~Amber > > On Fri, Dec 20, 2019 at 3:56 AM Carlos Lopez wrote: > > Hi all, > > I would like to install Corelight App For Splunk and TA for Corelight, but > there is no documentation about how to accomplish it ? All info points to > https://www.corelight.com/support/, but there is no docs in there ? > > Any idea? > > > -- > Regards, > C. L. Martinez > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > > -- > > *Amber Graner* > Director of Community > Corelight, Inc > > 828.582.9469 > > Schedule time on my calendar here. > > [image: Image removed by sender.] [image: Image removed by sender.] > > * Ask me about how you can participate in the Zeek (formerly Bro) > community. > * Remember - ZEEK AND YOU SHALL FIND!! > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- *Patrick Kelley, CISSP, C|EH, ITIL* *CTO* patrick.kelley at criticalpathsecurity.com (o) 770-224-6482 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/a5d306ca/attachment.html From ericooi at gmail.com Fri Dec 20 06:37:49 2019 From: ericooi at gmail.com (ericooi at gmail.com) Date: Fri, 20 Dec 2019 08:37:49 -0600 Subject: [Zeek] Documentation about Corelight's Splunk Apps for Zeek In-Reply-To: References: <7F534E24-79F1-4F87-8CDF-71611D8255FA@outlook.com> <84B4DEB5-7131-445F-951E-DF4AA2248FBA@outlook.com> <5CC1EA46-DCEA-43BD-AD4D-76DBB1FCA701@gmail.com> Message-ID: <75B332E1-20D4-451B-B727-0594A65DBBCE@gmail.com> Yep, definitely agree on the granularity and control. May also help with troubleshooting to split it out like that. And Elastic is what I?m looking into next. :P > On Dec 20, 2019, at 8:35 AM, Patrick Kelley wrote: > > Seconding the statements of Eric, Splunk costs can get to be expensive extremely quick with Zeek. > > My only secondary suggestion is that you ingest individual logs to provide a bit more granularity and control. You might not wish to ingest every log due to the processing and storage costs. > > In the past, I've tried leveraging Splunk multiple times due to my familiarity. In the end, we've built our stack around Elastic. > > We were just spending too much time servicing the hammer, instead of building the house. > > On Fri, Dec 20, 2019 at 9:29 AM ericooi at gmail.com > wrote: > Hi Carlos, > > ?Best? is subjective. For someone who wants all logs and a short inputs.conf file, your suggestion will work. My example is geared towards the fact that these logs are large and depending on your Splunk license and requirements, you may not actually want to ingest every single log file into your system. Ultimately, you know your environment and needs best which is why I also state in the writeup: > > "An example inputs.conf is below but may or may not include the logs you wish to ingest...Modify the index and sourcetype configurations to your needs.? > > Hope that helps! > Eric > >> On Dec 20, 2019, at 8:22 AM, Carlos Lopez > wrote: >> >> Thanks Eric. But I have doubt with your setup. For inputs.conf, maybe this configuration is best? >> >> [monitor:///opt/zeek/logs/spool/current <>] >> disabled = 0 >> sourcetype = zeek:json >> whitelist = \.log$ >> >> instead of to put file by file? >> -- >> Regards, >> C. L. Martinez >> >> From: Eric Ooi > >> Date: Friday, 20 December 2019 at 13:52 >> To: Amber Graner >, Carlos Lopez > >> Cc: "zeek at zeek.org " > >> Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek >> >> Assuming you?re doing an install on a standalone Splunk server, you can use my guide here: https://www.ericooi.com/zeekurity-zen-part-iii-how-to-send-zeek-bro-logs-to-splunk/ >> >> >> From: zeek-bounces at zeek.org on behalf of Amber Graner > >> Sent: Friday, December 20, 2019 6:13 AM >> To: Carlos Lopez >> Cc: zeek at zeek.org >> Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek >> >> Hi Carlos, >> >> As that is a Corelight offering and not something maintained by the Zeek Project or the community, we?d have to refer you to Corelight. >> >> Let me find out who you need to talk to and I?ll make introductions. >> >> Thanks, >> ~Amber >> >> On Fri, Dec 20, 2019 at 3:56 AM Carlos Lopez > wrote: >> Hi all, >> >> I would like to install Corelight App For Splunk and TA for Corelight, but there is no documentation about how to accomplish it ? All info points tohttps://www.corelight.com/support/ , but there is no docs in there ? >> >> Any idea? >> >> >> -- >> Regards, >> C. L. Martinez >> _______________________________________________ >> Zeek mailing list >> zeek at zeek.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek >> -- >> >> Amber Graner >> Director of Community >> Corelight, Inc >> >> 828.582.9469 >> >> Schedule time on my calendar here. >> >> >> >> * Ask me about how you can participate in the Zeek (formerly Bro) community. >> * Remember - ZEEK AND YOU SHALL FIND!! > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > -- > > Patrick Kelley, CISSP, C|EH, ITIL > CTO > patrick.kelley at criticalpathsecurity.com > (o) 770-224-6482 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/e7acabb7/attachment-0001.html From clopmz at outlook.com Fri Dec 20 06:41:23 2019 From: clopmz at outlook.com (Carlos Lopez) Date: Fri, 20 Dec 2019 14:41:23 +0000 Subject: [Zeek] Documentation about Corelight's Splunk Apps for Zeek In-Reply-To: References: <7F534E24-79F1-4F87-8CDF-71611D8255FA@outlook.com> <84B4DEB5-7131-445F-951E-DF4AA2248FBA@outlook.com> <5CC1EA46-DCEA-43BD-AD4D-76DBB1FCA701@gmail.com> Message-ID: <10E6F70D-1617-447F-BE84-3C73BF007660@outlook.com> I agree with both of you. But this is a little lab to accomplish some tests using Splunk free version ( I don?t expect more than 500 MiB daily logs ??). On the other side, Elastic is too expensive in maintenance and for me it is not an option in my case. With splunk things just work ?? -- Regards, C. L. Martinez From: Patrick Kelley Date: Friday, 20 December 2019 at 15:35 To: "ericooi at gmail.com" Cc: Carlos Lopez , "zeek at zeek.org" Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek Seconding the statements of Eric, Splunk costs can get to be expensive extremely quick with Zeek. My only secondary suggestion is that you ingest individual logs to provide a bit more granularity and control. You might not wish to ingest every log due to the processing and storage costs. In the past, I've tried leveraging Splunk multiple times due to my familiarity. In the end, we've built our stack around Elastic. We were just spending too much time servicing the hammer, instead of building the house. On Fri, Dec 20, 2019 at 9:29 AM ericooi at gmail.com > wrote: Hi Carlos, ?Best? is subjective. For someone who wants all logs and a short inputs.conf file, your suggestion will work. My example is geared towards the fact that these logs are large and depending on your Splunk license and requirements, you may not actually want to ingest every single log file into your system. Ultimately, you know your environment and needs best which is why I also state in the writeup: "An example inputs.conf is below but may or may not include the logs you wish to ingest...Modify the index and sourcetype configurations to your needs.? Hope that helps! Eric On Dec 20, 2019, at 8:22 AM, Carlos Lopez > wrote: Thanks Eric. But I have doubt with your setup. For inputs.conf, maybe this configuration is best? [monitor:///opt/zeek/logs/spool/current] disabled = 0 sourcetype = zeek:json whitelist = \.log$ instead of to put file by file? -- Regards, C. L. Martinez From: Eric Ooi > Date: Friday, 20 December 2019 at 13:52 To: Amber Graner >, Carlos Lopez > Cc: "zeek at zeek.org" > Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek Assuming you?re doing an install on a standalone Splunk server, you can use my guide here: https://www.ericooi.com/zeekurity-zen-part-iii-how-to-send-zeek-bro-logs-to-splunk/ ________________________________ From: zeek-bounces at zeek.org on behalf of Amber Graner > Sent: Friday, December 20, 2019 6:13 AM To: Carlos Lopez Cc: zeek at zeek.org Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek Hi Carlos, As that is a Corelight offering and not something maintained by the Zeek Project or the community, we?d have to refer you to Corelight. Let me find out who you need to talk to and I?ll make introductions. Thanks, ~Amber On Fri, Dec 20, 2019 at 3:56 AM Carlos Lopez > wrote: Hi all, I would like to install Corelight App For Splunk and TA for Corelight, but there is no documentation about how to accomplish it ? All info points tohttps://www.corelight.com/support/, but there is no docs in there ? Any idea? -- Regards, C. L. Martinez _______________________________________________ Zeek mailing list zeek at zeek.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Amber Graner Director of Community Corelight, Inc 828.582.9469 Schedule time on my calendar here. Error! Filename not specified. Error! Filename not specified. * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! _______________________________________________ Zeek mailing list zeek at zeek.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Patrick Kelley, CISSP, C|EH, ITIL CTO patrick.kelley at criticalpathsecurity.com (o) 770-224-6482 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/5bddbac1/attachment-0001.html From ericooi at gmail.com Fri Dec 20 06:43:12 2019 From: ericooi at gmail.com (ericooi at gmail.com) Date: Fri, 20 Dec 2019 08:43:12 -0600 Subject: [Zeek] Documentation about Corelight's Splunk Apps for Zeek In-Reply-To: <10E6F70D-1617-447F-BE84-3C73BF007660@outlook.com> References: <7F534E24-79F1-4F87-8CDF-71611D8255FA@outlook.com> <84B4DEB5-7131-445F-951E-DF4AA2248FBA@outlook.com> <5CC1EA46-DCEA-43BD-AD4D-76DBB1FCA701@gmail.com> <10E6F70D-1617-447F-BE84-3C73BF007660@outlook.com> Message-ID: <32A3DCE1-F7D5-4964-84D2-AA677A8150B9@gmail.com> Cool, then that should work. Like I said, your environment and requirements will be unique and to adjust as needed. The entire guide is meant just as a way to help people get started. It?s not meant to be a one size fits all solution. > On Dec 20, 2019, at 8:41 AM, Carlos Lopez wrote: > > I agree with both of you. But this is a little lab to accomplish some tests using Splunk free version ( I don?t expect more than 500 MiB daily logs ??). > > On the other side, Elastic is too expensive in maintenance and for me it is not an option in my case. With splunk things just work ?? > > -- > Regards, > C. L. Martinez > > From: Patrick Kelley > > Date: Friday, 20 December 2019 at 15:35 > To: "ericooi at gmail.com " > > Cc: Carlos Lopez >, "zeek at zeek.org " > > Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek > > Seconding the statements of Eric, Splunk costs can get to be expensive extremely quick with Zeek. > > My only secondary suggestion is that you ingest individual logs to provide a bit more granularity and control. You might not wish to ingest every log due to the processing and storage costs. > > In the past, I've tried leveraging Splunk multiple times due to my familiarity. In the end, we've built our stack around Elastic. > > We were just spending too much time servicing the hammer, instead of building the house. > > On Fri, Dec 20, 2019 at 9:29 AM ericooi at gmail.com > wrote: > Hi Carlos, > > ?Best? is subjective. For someone who wants all logs and a short inputs.conf file, your suggestion will work. My example is geared towards the fact that these logs are large and depending on your Splunk license and requirements, you may not actually want to ingest every single log file into your system. Ultimately, you know your environment and needs best which is why I also state in the writeup: > > "An example inputs.conf is below but may or may not include the logs you wish to ingest...Modify the index and sourcetype configurations to your needs.? > > Hope that helps! > Eric > > On Dec 20, 2019, at 8:22 AM, Carlos Lopez > wrote: > > Thanks Eric. But I have doubt with your setup. For inputs.conf, maybe this configuration is best? > > [monitor:///opt/zeek/logs/spool/current ] > disabled = 0 > sourcetype = zeek:json > whitelist = \.log$ > > instead of to put file by file? > -- > Regards, > C. L. Martinez > > From: Eric Ooi > > Date: Friday, 20 December 2019 at 13:52 > To: Amber Graner >, Carlos Lopez > > Cc: "zeek at zeek.org " > > Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek > > Assuming you?re doing an install on a standalone Splunk server, you can use my guide here: https://www.ericooi.com/zeekurity-zen-part-iii-how-to-send-zeek-bro-logs-to-splunk/ > > > From: zeek-bounces at zeek.org on behalf of Amber Graner > > Sent: Friday, December 20, 2019 6:13 AM > To: Carlos Lopez > Cc: zeek at zeek.org > Subject: Re: [Zeek] Documentation about Corelight's Splunk Apps for Zeek > > Hi Carlos, > > As that is a Corelight offering and not something maintained by the Zeek Project or the community, we?d have to refer you to Corelight. > > Let me find out who you need to talk to and I?ll make introductions. > > Thanks, > ~Amber > > On Fri, Dec 20, 2019 at 3:56 AM Carlos Lopez > wrote: > Hi all, > > I would like to install Corelight App For Splunk and TA for Corelight, but there is no documentation about how to accomplish it ? All info points tohttps://www.corelight.com/support/ , but there is no docs in there ? > > Any idea? > > > -- > Regards, > C. L. Martinez > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > -- > > Amber Graner > Director of Community > Corelight, Inc > > 828.582.9469 > > Schedule time on my calendar here. > > Error! Filename not specified. Error! Filename not specified. > > * Ask me about how you can participate in the Zeek (formerly Bro) community. > * Remember - ZEEK AND YOU SHALL FIND!! > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > > -- > > Patrick Kelley, CISSP, C|EH, ITIL > CTO > patrick.kelley at criticalpathsecurity.com > (o) 770-224-6482 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/59093efe/attachment-0001.html From mauro.palumbo at aizoon.it Fri Dec 20 07:00:48 2019 From: mauro.palumbo at aizoon.it (Palumbo Mauro) Date: Fri, 20 Dec 2019 15:00:48 +0000 Subject: [Zeek] R: [EXT]Re: Time value errors In-Reply-To: References: Message-ID: <44e441cec7ac412c90b1760a7c214787@SRVEX03.aizoon.local> Hi Scot, If you try: tcpdump -i your_interface udp port 123 ?vv you?ll see that sometimes there are zero values in ref time, orig time, etc. I don?t think it?s an issue with the analyzer and the NTP protocol does not require all timestamps fields to have a non-zero value. Mauro Da: zeek-bounces at zeek.org [mailto:zeek-bounces at zeek.org] Per conto di Scot Harris Inviato: gioved? 19 dicembre 2019 14:21 A: Justin Azoff Cc: zeek at zeek.org Oggetto: Re: [Zeek] [EXT]Re: Time value errors Bigger issue possibly. A lot of zero values I check date/time on both zeek boxes and they are set correctly. CPURlj0fxNnhawrQk 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-19T06:59:59-0500 CPURlj0fxNnhawrQk 0.000000 0.000000 0.000000 1576756799.000161 Even the transmit date on some of the records are 1969. CQ2SXD4XyRGPpQCu9e 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-19T07:00:02-0500 CLpVPg3841dexUbAu6 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 CLpVPg3841dexUbAu6 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 CxTT3e4BKVjQ9ogjng 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-19T07:00:00-0500 CGFPn54Ff0m4cIkr5e 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-19T07:00:04-0500 CXMvpj1SJy4aBwQ81i 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-19T07:00:04-0500 CQ2SXD4XyRGPpQCu9e 0.000000 0.000000 0.000000 1576756802.000145 CLpVPg3841dexUbAu6 0.000000 0.000000 0.000000 0.000000 CLpVPg3841dexUbAu6 0.000000 0.000000 0.000000 0.000000 CxTT3e4BKVjQ9ogjng 0.000000 0.000000 0.000000 1576756800.889992 CGFPn54Ff0m4cIkr5e 0.000000 0.000000 0.000000 1576756804.000129 CXMvpj1SJy4aBwQ81i 0.000000 0.000000 0.000000 1576756804.000076 Should those fields have zero values? That is why they are being displayed as start of epoch. From: Justin Azoff [mailto:justin at corelight.com] Sent: Wednesday, December 18, 2019 6:50 PM To: Scot Harris > Cc: zeek at zeek.org Subject: [EXT]Re: [Zeek] Time value errors If you run that without the -d option, what does the line containing negative times look like? There should be 4 times at the end of each record: ref_time org_time rec_time xmt_time, knowing which one(s) have the out of range value would help. Something like cat ntp.log |zeek-cut uid ref_time org_time rec_time xmt_time | fgrep -- - may help see them better. On Wed, Dec 18, 2019 at 6:08 PM Scot Harris > wrote: > > Noted what appear to be errors in the ntp.log file. > > > > Using following command: > > > > cat ntp.log | zeek-cut ?d | less > > > > > > > > af_packet::eno1 2019-12-18T17:44:39-0500 C7MULpTngYof10ymf 10.1.45.35 123 10.1.5.60 123 2 3 4 64.000000 0.000004 0.070786 0.113083 10.1.5.60 2019-12-18T17:43:35-0500 2019-12-18T17:43:35-0500 2019-12-18T17:43:35-0500 2019-12-18T17:44:39-0500 0 > > af_packet::eno1 2019-12-18T17:44:39-0500 C7MULpTngYof10ymf 10.1.45.35 123 10.1.5.60 123 3 4 3 64.000000 0.015625 0.069839 0.077545 23.239.26.89 2019-:zeek-cut: time value out-of-range: -586465861.545972 > > zeek-cut: time value out-of-range: -586465861.545972 > > 12-18T17:42:18-0500 2019-12-18T17:44:39-0500 2019-12-18T17:44:39-0500 2019-12-18T17:44:39-0500 0 > > af_packet::eno1 2019-12-18T17:44:39-0500 C5GF2T1ozzCZptCbjf 10.1.204.212 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-18T17:44:46-0500 0 > > af_packet::eno1 2019-12-18T17:44:40-0500 CxaJ6KeJfxVcN8Fw2 10.1.201.150 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-18T17:44:48-0500 0 > > af_packet::eno1 2019-12-18T17:44:40-0500 C8dZCI37SuYRZB9L7g 10.1.13.61 123 10.1.5.60 123 3 3 4 64.000000 0.007812 0.069839 0.402298 60.5.1.10 2019-12-18T17:43:37-0500 2019-12-18T17:43:36-0500 2019-12-18T17:43:37-0500 2019-12-18T17:44:41-0500 0 > > af_packet::eno1 2019-12-18T17:44:41-0500 CBz4Ww4jjCjKgHYfwc 10.1.221.30 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19:zeek-cut: time value out-of-range: -1114760693.379112 > > zeek-cut: time value out-of-range: -1114760693.379112 > > zeek-cut: time value out-of-range: -1115340513.842638 > > :00:00-0500 1969-12-31T19:00:00-0500 1969-12-31T19:00:00-0500 2019-12-18T17:44:44-0500 0 > > af_packet::eno1 2019-12-18T17:44:40-0500 C4akh61szBCsYCPJn6 10.1.223.28 123 10.1.5.180 123 3 3 15 64.000000 0.007812 0.000000 2.009995 0.0.0.0 1969-12-31T19: > > > > Have not noticed these errors previously. > > > > > > > > > > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Justin CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/d10e77f5/attachment-0001.html From akgraner at corelight.com Fri Dec 20 07:36:51 2019 From: akgraner at corelight.com (Amber Graner) Date: Fri, 20 Dec 2019 10:36:51 -0500 Subject: [Zeek] Documentation about Corelight's Splunk Apps for Zeek In-Reply-To: References: <7F534E24-79F1-4F87-8CDF-71611D8255FA@outlook.com> Message-ID: Hi Carlos, I reached out to Corelight and below is the response: Docs and downloads can be found here: App - https://splunkbase.splunk.com/app/3884/ TA - https://splunkbase.splunk.com/app/3885/ At a very basic level the install guidance is this: - For stand alone Splunk instances - install the Corelight App for Splunk ONLY using the Splunk Web UI - For distributed instances - installed the Corelight App for Splunk on search head(s) using the Splunk Web UI, install the TA on indexers and/or heavy forwarders using install method of choice (cli, web ui, or deployment server) Thanks, ~Amber On Fri, Dec 20, 2019 at 7:03 AM Amber Graner wrote: > Hi Carlos, > > As that is a Corelight offering and not something maintained by the Zeek > Project or the community, we?d have to refer you to Corelight. > > Let me find out who you need to talk to and I?ll make introductions. > > Thanks, > ~Amber > > On Fri, Dec 20, 2019 at 3:56 AM Carlos Lopez wrote: > >> Hi all, >> >> >> >> I would like to install Corelight App For Splunk and TA for Corelight, >> but there is no documentation about how to accomplish it ? All info points >> to https://www.corelight.com/support/, but there is no docs in there ? >> >> >> >> Any idea? >> >> >> >> >> >> -- >> >> Regards, >> >> C. L. Martinez >> _______________________________________________ >> Zeek mailing list >> zeek at zeek.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek > > -- > > *Amber Graner* > Director of Community > Corelight, Inc > > 828.582.9469 > > Schedule time on my calendar here. > > > > * Ask me about how you can participate in the Zeek (formerly Bro) > community. > * Remember - ZEEK AND YOU SHALL FIND!! > > > -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 Schedule time on my calendar here. * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/de5188b0/attachment.html From smoot at corelight.com Fri Dec 20 08:07:35 2019 From: smoot at corelight.com (Steve Smoot) Date: Fri, 20 Dec 2019 08:07:35 -0800 Subject: [Zeek] Documentation about Corelight's Splunk Apps for Zeek In-Reply-To: References: <7F534E24-79F1-4F87-8CDF-71611D8255FA@outlook.com> Message-ID: Also in the Apps that Amber pointed to in the "Details" tab, there are instructions for install/config. Though depending on the environment specific instructions around sourcetypes and index names will differ due to local configuration. -s On Fri, Dec 20, 2019 at 7:45 AM Amber Graner wrote: > Hi Carlos, > > I reached out to Corelight and below is the response: > > Docs and downloads can be found here: > App - https://splunkbase.splunk.com/app/3884/ > TA - https://splunkbase.splunk.com/app/3885/ > > At a very basic level the install guidance is this: > > - For stand alone Splunk instances - install the Corelight App for > Splunk ONLY using the Splunk Web UI > - For distributed instances - installed the Corelight App for Splunk > on search head(s) using the Splunk Web UI, install the TA on indexers > and/or heavy forwarders using install method of choice (cli, web ui, or > deployment server) > > Thanks, > ~Amber > > On Fri, Dec 20, 2019 at 7:03 AM Amber Graner > wrote: > >> Hi Carlos, >> >> As that is a Corelight offering and not something maintained by the Zeek >> Project or the community, we?d have to refer you to Corelight. >> >> Let me find out who you need to talk to and I?ll make introductions. >> >> Thanks, >> ~Amber >> >> On Fri, Dec 20, 2019 at 3:56 AM Carlos Lopez wrote: >> >>> Hi all, >>> >>> >>> >>> I would like to install Corelight App For Splunk and TA for Corelight, >>> but there is no documentation about how to accomplish it ? All info points >>> to https://www.corelight.com/support/, but there is no docs in there ? >>> >>> >>> >>> Any idea? >>> >>> >>> >>> >>> >>> -- >>> >>> Regards, >>> >>> C. L. Martinez >>> _______________________________________________ >>> Zeek mailing list >>> zeek at zeek.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek >> >> -- >> >> *Amber Graner* >> Director of Community >> Corelight, Inc >> >> 828.582.9469 >> >> Schedule time on my calendar here. >> >> >> >> * Ask me about how you can participate in the Zeek (formerly Bro) >> community. >> * Remember - ZEEK AND YOU SHALL FIND!! >> >> >> > > -- > > *Amber Graner* > Director of Community > Corelight, Inc > > 828.582.9469 > > Schedule time on my calendar here. > > > > * Ask me about how you can participate in the Zeek (formerly Bro) > community. > * Remember - ZEEK AND YOU SHALL FIND!! > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- *Stephen R. Smoot, PhD* VP, Customer Success Corelight -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191220/85927aa5/attachment.html From akgraner at corelight.com Mon Dec 23 06:11:53 2019 From: akgraner at corelight.com (Amber Graner) Date: Mon, 23 Dec 2019 09:11:53 -0500 Subject: [Zeek] [Blog] - How to Add a JPEG File Analyzer to Zeek - Part 4 Message-ID: Hi all, Part 4 of How to Add a JPEG File Analyzer to Zeek is now available on the Zeek blog at: https://blog.zeek.org/2019/12/how-to-add-jpeg-file-analyzer-to-zeek_23.html Just in case you may have missed Parts 1-3 below are the links: Part 3 - https://blog.zeek.org/2019/12/how-to-add-jpeg-file-analyzer-to-zeek_20.html Part 2 - https://blog.zeek.org/2019/12/how-to-add-jpeg-file-analyzer-to-zeek_19.html Part 1 - https://blog.zeek.org/2019/12/how-to-add-jpeg-file-analyzer-to-zeek.html Thanks again to Keith J. Jones, Ph.D. for this series. If you or someone you know has a great idea for another howto series, please let me know so we can get it scheduled. Thanks in advance and Happy Zeeking! ~Amber -- *Amber Graner* Director of Community Corelight, Inc 828.582.9469 Schedule time on my calendar here. * Ask me about how you can participate in the Zeek (formerly Bro) community. * Remember - ZEEK AND YOU SHALL FIND!! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191223/e320477d/attachment.html From mxd6 at comcast.net Wed Dec 25 07:32:03 2019 From: mxd6 at comcast.net (mxd6) Date: Wed, 25 Dec 2019 07:32:03 -0800 Subject: [Zeek] Live capture configuration Message-ID: <20191225153206.E90322C4010@rock.ICSI.Berkeley.EDU> I'm trying to setup my ZEEK system to do both MODBUS and DNP3 live capture on the network. When I include the following statements into my local.zeek file it fails on deployment with the error "fatal error in ../../../local.zeek. Can't find protocols/modbus/software"Here is my load statements in local.zeek:@load protocols/modbus/software at load protocols/dnp3/softwareAny suggestion or examples on configurations for live capture for these two protocols?Any help would be greatly appreciated! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191225/7b094a51/attachment.html From justin at corelight.com Thu Dec 26 06:05:22 2019 From: justin at corelight.com (Justin Azoff) Date: Thu, 26 Dec 2019 09:05:22 -0500 Subject: [Zeek] Live capture configuration In-Reply-To: <20191225153206.E90322C4010@rock.ICSI.Berkeley.EDU> References: <20191225153206.E90322C4010@rock.ICSI.Berkeley.EDU> Message-ID: On Wed, Dec 25, 2019 at 10:34 AM mxd6 wrote: > I'm trying to setup my ZEEK system to do both MODBUS and DNP3 live capture > on the network. When I include the following statements into my local.zeek > file it fails on deployment with the error "fatal error in > ../../../local.zeek. Can't find protocols/modbus/software" > > Here is my load statements in local.zeek: > > @load protocols/modbus/software > @load protocols/dnp3/software > > Any suggestion or examples on configurations for live capture for these > two protocols? > > Any help would be greatly appreciated! > Those scripts don't actually exist, only a few protocols have the 'software' scripts; scripts/policy/protocols/ftp/software.zeek scripts/policy/protocols/ssh/software.zeek scripts/policy/protocols/http/software-browser-plugins.zeek scripts/policy/protocols/http/software.zeek scripts/policy/protocols/smtp/software.zeek scripts/policy/protocols/mysql/software.zeek scripts/policy/protocols/dhcp/software.zeek The default scripts/base/init-default.zeek already contains the @load statements for the base dnp3 and modbus scripts; @load base/protocols/dnp3 @load base/protocols/modbus so you shouldn't have to do any configuration to analyze these protocols. -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191226/f6790036/attachment.html From Denny.Sabu at sensato.co Thu Dec 26 12:42:07 2019 From: Denny.Sabu at sensato.co (Denny Sabu) Date: Thu, 26 Dec 2019 20:42:07 +0000 Subject: [Zeek] Manager, Proxy and Worker all logging the same notice to notice.log Message-ID: Hello, I have a clustered deployment of Zeek (v3.0.0) consisting of a manager, a proxy and 16 workers. In notice.log, I see 3 notices for what appears to be a single event. The 3 notices have the same ts, source, destination, IPs, ports, fuids, notes, and msgs but the uid is different for all 3 notices. In addition the 'peer_descr' value is different for each, with one being the manager, one the proxy and one the worker. Any help/guidance on the matter would be greatly appreciated. Best, Denny Sabu Software Engineer Sensato www.sensato.co [http://cdn2.hubspot.net/hubfs/184235/dev_images/signature_app/facebook_sig.png] [http://cdn2.hubspot.net/hubfs/184235/dev_images/signature_app/twitter_sig.png] [http://cdn2.hubspot.net/hubfs/184235/dev_images/signature_app/linkedin_sig.png] -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191226/42f30ef0/attachment.html From justin at corelight.com Thu Dec 26 14:01:52 2019 From: justin at corelight.com (Justin Azoff) Date: Thu, 26 Dec 2019 17:01:52 -0500 Subject: [Zeek] Manager, Proxy and Worker all logging the same notice to notice.log In-Reply-To: References: Message-ID: What is the notice? What does your node.cfg look like? On Thu, Dec 26, 2019 at 3:44 PM Denny Sabu wrote: > Hello, > > I have a clustered deployment of Zeek (v3.0.0) consisting of a manager, a > proxy and 16 workers. In notice.log, I see 3 notices for what appears to be > a single event. The 3 notices have the same ts, source, destination, IPs, > ports, fuids, notes, and msgs but the uid is different for all 3 notices. > In addition the 'peer_descr' value is different for each, with one being > the manager, one the proxy and one the worker. > > Any help/guidance on the matter would be greatly appreciated. > > Best, > > Denny Sabu > *Software Engineer* > *Sensato* > www.sensato.co > > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191226/d31cfc69/attachment.html From Denny.Sabu at sensato.co Thu Dec 26 15:01:32 2019 From: Denny.Sabu at sensato.co (Denny Sabu) Date: Thu, 26 Dec 2019 23:01:32 +0000 Subject: [Zeek] Manager, Proxy and Worker all logging the same notice to notice.log In-Reply-To: References: , Message-ID: node.cfg is as follows: [manager] type=manager host=localhost interface=enp101s0f1 [proxy-1] type=proxy host=localhost interface=enp101s0f1 [worker-1] type=worker host=localhost interface=enp101s0f1 lb_method=pf_ring lb_procs=16 pin_cpus=4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19 The notice is SSL::Invalid_Server_Cert Denny Sabu Software Engineer Sensato www.sensato.co [http://cdn2.hubspot.net/hubfs/184235/dev_images/signature_app/facebook_sig.png] [http://cdn2.hubspot.net/hubfs/184235/dev_images/signature_app/twitter_sig.png] [http://cdn2.hubspot.net/hubfs/184235/dev_images/signature_app/linkedin_sig.png] ________________________________ From: Justin Azoff Sent: Thursday, December 26, 2019 5:01 PM To: Denny Sabu Cc: zeek at zeek.org Subject: Re: [Zeek] Manager, Proxy and Worker all logging the same notice to notice.log What is the notice? What does your node.cfg look like? On Thu, Dec 26, 2019 at 3:44 PM Denny Sabu > wrote: Hello, I have a clustered deployment of Zeek (v3.0.0) consisting of a manager, a proxy and 16 workers. In notice.log, I see 3 notices for what appears to be a single event. The 3 notices have the same ts, source, destination, IPs, ports, fuids, notes, and msgs but the uid is different for all 3 notices. In addition the 'peer_descr' value is different for each, with one being the manager, one the proxy and one the worker. Any help/guidance on the matter would be greatly appreciated. Best, Denny Sabu Software Engineer Sensato www.sensato.co [http://cdn2.hubspot.net/hubfs/184235/dev_images/signature_app/facebook_sig.png] [http://cdn2.hubspot.net/hubfs/184235/dev_images/signature_app/twitter_sig.png] [http://cdn2.hubspot.net/hubfs/184235/dev_images/signature_app/linkedin_sig.png] _______________________________________________ Zeek mailing list zeek at zeek.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Justin [EXTERNAL MESSAGE]: This e-mail was sent from an external source - use caution and vigilance and don't become attacker fodder! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191226/69d8638f/attachment-0001.html From justin at corelight.com Thu Dec 26 16:00:53 2019 From: justin at corelight.com (Justin Azoff) Date: Thu, 26 Dec 2019 19:00:53 -0500 Subject: [Zeek] Manager, Proxy and Worker all logging the same notice to notice.log In-Reply-To: References: Message-ID: On Thu, Dec 26, 2019 at 6:01 PM Denny Sabu wrote: > node.cfg is as follows: > > [manager] > type=manager > host=localhost > interface=enp101s0f1 > > [proxy-1] > type=proxy > host=localhost > interface=enp101s0f1 > > [worker-1] > type=worker > host=localhost > interface=enp101s0f1 > lb_method=pf_ring > lb_procs=16 > pin_cpus=4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19 > You've told the manager and proxy to capture from enp101s0f1.. remove those lines and this problem will go away. Also, you should add a logger section. -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191226/7af3dd61/attachment.html From SHARRIS at hollywoodfl.org Mon Dec 30 13:33:12 2019 From: SHARRIS at hollywoodfl.org (Scot Harris) Date: Mon, 30 Dec 2019 21:33:12 +0000 Subject: [Zeek] TEMP files under /opt/zeek/spool/worker-*-*/extract_files Message-ID: While running some system checks, I noted that on the two zeek 3.0 boxes I have in a cluster that drive space was being taken up in the following directory: /opt/zeek/spool/worker-3-1/extract_files. These files not large but numerous. total 15884720 -rw-rw-r-- 1 zeek zeek 50 Dec 30 16:26 CZi12w40mQqZt08H24_FWsM9w4uCR965XUybc__10.1.24.161_c$Windows_TEMP_fstmp_fs_smbfile_10.1.24.161-93.txt -rw-rw-r-- 1 zeek zeek 326 Dec 30 16:26 CZi12w40mQqZt08H24_FEJBtK3MwBWHCM3ET__10.1.24.161_c$Windows_TEMP_fstmp_fs_action_110812.bat -rw-rw-r-- 1 zeek zeek 44 Dec 30 16:26 CZi12w40mQqZt08H24_FzgNR24m7uP0dbeI54__10.1.24.161_c$Windows_TEMP_fstmp_fs_smbfile_10.1.24.161-92.txt -rw-rw-r-- 1 zeek zeek 315 Dec 30 16:26 CZi12w40mQqZt08H24_FlcyIJ91gLM90QJhe__10.1.24.161_c$Windows_TEMP_fstmp_fs_action_110811.bat -rw-rw-r-- 1 zeek zeek 23 Dec 30 16:26 CZi12w40mQqZt08H24_FXnI1p4BopgQQE4jye__10.1.24.161_c$Windows_TEMP_fstmp_fs_smbfile_10.1.24.161-91.txt -rw-rw-r-- 1 zeek zeek 285 Dec 30 16:26 CZi12w40mQqZt08H24_FXROsS11QBGTbOJGNd__10.1.24.161_c$Windows_TEMP_fstmp_fs_action_110810.bat -rw-rw-r-- 1 zeek zeek 19 Dec 30 16:25 CZi12w40mQqZt08H24_FZ3vCj4O7BqZZYdgT__10.1.24.161_c$Windows_TEMP_fstmp_fs_smbfile_10.1.24.161-90.txt -rw-rw-r-- 1 zeek zeek 330 Dec 30 16:25 CZi12w40mQqZt08H24_FozvDx13eGGG5Sssyc__10.1.24.161_c$Windows_TEMP_fstmp_fs_action_110793.bat -rw-rw-r-- 1 zeek zeek 77 Dec 30 16:25 CZi12w40mQqZt08H24_FIwGEp2TW6WJ8IbEcd__10.1.24.161_c$Windows_TEMP_fstmp_fs_smbfile_10.1.24.161-89.txt -rw-rw-r-- 1 zeek zeek 313 Dec 30 16:25 CZi12w40mQqZt08H24_FWm9sG44On6hV5AoWj__10.1.24.161_c$Windows_TEMP_fstmp_fs_action_110781.bat -rw-rw-r-- 1 zeek zeek 10 Dec 30 16:25 CZi12w40mQqZt08H24_Fo3rTB2pdq5ooABUCa__10.1.24.161_c$Windows_TEMP_fstmp_fs_smbfile_10.1.24.161-88.txt Can these files be purged periodically? Seeing this on both the master box and the cluster node. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191230/05b80d8a/attachment.html From justin at corelight.com Tue Dec 31 06:27:19 2019 From: justin at corelight.com (Justin Azoff) Date: Tue, 31 Dec 2019 09:27:19 -0500 Subject: [Zeek] TEMP files under /opt/zeek/spool/worker-*-*/extract_files In-Reply-To: References: Message-ID: Those aren't TEMP files as far as zeek is concerned. Those are being extracted by the BZAR script: https://github.com/mitre-attack/bzar/blob/master/scripts/bzar_files.bro they just happen to be from c:\windows\TEMP\ on the server. You should be analyzing those files, or if you don't want them at all the bzar script has BZAR::file_extract_option or some other ways of filtering things to turn that feature off. On Mon, Dec 30, 2019 at 4:35 PM Scot Harris wrote: > While running some system checks, I noted that on the two zeek 3.0 boxes I > have in a cluster that drive space was being taken up in the following > directory: > > > > /opt/zeek/spool/worker-3-1/extract_files. > > > > These files not large but numerous. > > > > total 15884720 > > -rw-rw-r-- 1 zeek zeek 50 Dec 30 16:26 > CZi12w40mQqZt08H24_FWsM9w4uCR965XUybc__10.1.24.161_c$Windows_TEMP_fstmp_fs_smbfile_10.1.24.161-93.txt > > -rw-rw-r-- 1 zeek zeek 326 Dec 30 16:26 > CZi12w40mQqZt08H24_FEJBtK3MwBWHCM3ET__10.1.24.161_c$Windows_TEMP_fstmp_fs_action_110812.bat > > -rw-rw-r-- 1 zeek zeek 44 Dec 30 16:26 > CZi12w40mQqZt08H24_FzgNR24m7uP0dbeI54__10.1.24.161_c$Windows_TEMP_fstmp_fs_smbfile_10.1.24.161-92.txt > > -rw-rw-r-- 1 zeek zeek 315 Dec 30 16:26 > CZi12w40mQqZt08H24_FlcyIJ91gLM90QJhe__10.1.24.161_c$Windows_TEMP_fstmp_fs_action_110811.bat > > -rw-rw-r-- 1 zeek zeek 23 Dec 30 16:26 > CZi12w40mQqZt08H24_FXnI1p4BopgQQE4jye__10.1.24.161_c$Windows_TEMP_fstmp_fs_smbfile_10.1.24.161-91.txt > > -rw-rw-r-- 1 zeek zeek 285 Dec 30 16:26 > CZi12w40mQqZt08H24_FXROsS11QBGTbOJGNd__10.1.24.161_c$Windows_TEMP_fstmp_fs_action_110810.bat > > -rw-rw-r-- 1 zeek zeek 19 Dec 30 16:25 > CZi12w40mQqZt08H24_FZ3vCj4O7BqZZYdgT__10.1.24.161_c$Windows_TEMP_fstmp_fs_smbfile_10.1.24.161-90.txt > > -rw-rw-r-- 1 zeek zeek 330 Dec 30 16:25 > CZi12w40mQqZt08H24_FozvDx13eGGG5Sssyc__10.1.24.161_c$Windows_TEMP_fstmp_fs_action_110793.bat > > -rw-rw-r-- 1 zeek zeek 77 Dec 30 16:25 > CZi12w40mQqZt08H24_FIwGEp2TW6WJ8IbEcd__10.1.24.161_c$Windows_TEMP_fstmp_fs_smbfile_10.1.24.161-89.txt > > -rw-rw-r-- 1 zeek zeek 313 Dec 30 16:25 > CZi12w40mQqZt08H24_FWm9sG44On6hV5AoWj__10.1.24.161_c$Windows_TEMP_fstmp_fs_action_110781.bat > > -rw-rw-r-- 1 zeek zeek 10 Dec 30 16:25 > CZi12w40mQqZt08H24_Fo3rTB2pdq5ooABUCa__10.1.24.161_c$Windows_TEMP_fstmp_fs_smbfile_10.1.24.161-88.txt > > > > Can these files be purged periodically? > > > > Seeing this on both the master box and the cluster node. > > > > Thank you. > > > > > _______________________________________________ > Zeek mailing list > zeek at zeek.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek -- Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191231/335bc9aa/attachment.html