From akgraner at corelight.com  Tue Oct  1 03:25:21 2019
From: akgraner at corelight.com (Amber Graner)
Date: Tue, 1 Oct 2019 06:25:21 -0400
Subject: [Zeek] ZeekWeek 2019
Message-ID: <CAJhOzupJYPSGFkSA11mQ2u=O-xxQ4Au_Ts_-ox_wKTJHrAYNaQ@mail.gmail.com>

Hi all,

ZeekWeek 2019 is less than a week a way?  Want to go?

Registration is still open for both the training sessions (8 Oct) and
ZeekWeek (9-11 Oct).

==Registration Link - http://bit.ly/zeekweek19_registration

If you want to know more about using Zeek as a network monitoring tool or a
language then this is the place to me.

Also, if you're interested in networking with the Zeek Leadership Team,
Zeek Maintainers, or any of the sponsoring companies, now is your chance.

==Agenda Link - http://bit.ly/zeekweek19agenda

Missed the early bird registration but you still want to go?  Contact me
for a 20% off discount code.

Hope to see you in Seattle next week.

With gratitude,
~Amber


-- 

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191001/e5ee458e/attachment.html 

From jlay at slave-tothe-box.net  Tue Oct  1 09:28:38 2019
From: jlay at slave-tothe-box.net (James Lay)
Date: Tue, 01 Oct 2019 10:28:38 -0600
Subject: [Zeek] Zeek and packages
Message-ID: <d112fdafeae986a1c7a174521ed0235b@slave-tothe-box.net>

Soooooo....in testing, all my third party packages are muffed 
(deprecated (bro_init)).  What's a reasonable time frame to start to bug 
package maintainers?

Thank you!

James

P.S. See you next week.

From zeolla at gmail.com  Tue Oct  1 10:40:08 2019
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Tue, 1 Oct 2019 13:40:08 -0400
Subject: [Zeek] Zeek and packages
In-Reply-To: <d112fdafeae986a1c7a174521ed0235b@slave-tothe-box.net>
References: <d112fdafeae986a1c7a174521ed0235b@slave-tothe-box.net>
Message-ID: <CAM1mTCpjKxn8-7oV5hZVZEVecZ3wHUgyKB1B+HsVJam5CfemTw@mail.gmail.com>

Now!  =)

As a maintainer of the kafka plugin we're working on a release right now
(pre-zeek rename), and plan to fix 3.0 upgrade related issues as a part of
the next release (likely a 1.0.0).

- Jon Zeolla
Zeolla at GMail.Com


On Tue, Oct 1, 2019 at 12:36 PM James Lay <jlay at slave-tothe-box.net> wrote:

> Soooooo....in testing, all my third party packages are muffed
> (deprecated (bro_init)).  What's a reasonable time frame to start to bug
> package maintainers?
>
> Thank you!
>
> James
>
> P.S. See you next week.
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191001/b21d0c6a/attachment.html 

From jlay at slave-tothe-box.net  Tue Oct  1 10:43:23 2019
From: jlay at slave-tothe-box.net (James Lay)
Date: Tue, 01 Oct 2019 11:43:23 -0600
Subject: [Zeek] Zeek and packages
In-Reply-To: <CAM1mTCpjKxn8-7oV5hZVZEVecZ3wHUgyKB1B+HsVJam5CfemTw@mail.gmail.com>
References: <d112fdafeae986a1c7a174521ed0235b@slave-tothe-box.net>
	<CAM1mTCpjKxn8-7oV5hZVZEVecZ3wHUgyKB1B+HsVJam5CfemTw@mail.gmail.com>
Message-ID: <dad2fa2c58bf80a392b502c9330cc16f@slave-tothe-box.net>

Pimpy..thanks Jon!

James

On 2019-10-01 11:40, Zeolla at GMail.com wrote:
> Now!  =)
> 
> As a maintainer of the kafka plugin we're working on a release right
> now (pre-zeek rename), and plan to fix 3.0 upgrade related issues as a
> part of the next release (likely a 1.0.0).
> 
> - Jon Zeolla
> Zeolla at GMail.Com
> 
> On Tue, Oct 1, 2019 at 12:36 PM James Lay <jlay at slave-tothe-box.net>
> wrote:
> 
>> Soooooo....in testing, all my third party packages are muffed
>> (deprecated (bro_init)).  What's a reasonable time frame to start to
>> bug
>> package maintainers?
>> 
>> Thank you!
>> 
>> James
>> 
>> P.S. See you next week.
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

From nothinrandom at gmail.com  Tue Oct  1 16:20:07 2019
From: nothinrandom at gmail.com (TQ)
Date: Tue, 1 Oct 2019 16:20:07 -0700
Subject: [Zeek] Segmentation Fault on Zeek 3.0.0
In-Reply-To: <CAMzgZ0LiyNmCcwS1Xz=v4Uu4YYQwFVy=uDGKrhHMB3qMUVN=LA@mail.gmail.com>
References: <CAM2MKSpcxdzScdfW8pOuRjaPaxrmFzOPgWX=Y32P728dQz5YKg@mail.gmail.com>
	<CAMzgZ0JQoH8n5xu0K=OTXHvAHGkHEibZLzR3vZEQ7obDBhZeBg@mail.gmail.com>
	<CAM2MKSoXv+hOBi1bg=Jz-ULE4=_Uke9=NdE53aNJ2_dfy7D-fQ@mail.gmail.com>
	<CAMzgZ0LiyNmCcwS1Xz=v4Uu4YYQwFVy=uDGKrhHMB3qMUVN=LA@mail.gmail.com>
Message-ID: <CAM2MKSpWcOrVpK3=Yt-WvS8Vd8ZEeT+F=Kjwk5V+Xt1xP2ttwQ@mail.gmail.com>

Hey Jon,

Unfortunately, I'm not at liberty to share right now.  However, my
colleague Blake Johnson and I might give a talk at Zeek Week next week (
https://twitter.com/voteblake/status/1178787539999526912?s=20).  Will you
be there?  I'd love to catch up with you and show you in person instead.
It's probably some silly issue/misconfiguration on my end.

Thanks,

On Mon, Sep 30, 2019 at 9:37 AM Jon Siwek <jsiwek at corelight.com> wrote:

> Can you provide more information on how to reproduce the issue (exact
> scripts/plugins/pcaps that crash every time) ?  There's still a bug in
> Zeek to fix here, but just adding `-t` and trying a few things hasn't
> triggered it for me.
>
> - Jon
>
> On Fri, Sep 27, 2019 at 7:57 PM TQ <nothinrandom at gmail.com> wrote:
> >
> > Hey Jon,
> >
> > Thanks for guidance on this!  You are absolutely right.  If I remove "-t
> ~/Desktop/logs/output.log", then that segmentation fault goes away.  I have
> not a clue why as it works fine for 2.6.2.  I thought something was wrong
> with the actual code.  Again, thanks for helping out with this!
> >
> > Thanks,
> >
> > On Fri, Sep 27, 2019 at 5:16 PM Jon Siwek <jsiwek at corelight.com> wrote:
> >>
> >> On Fri, Sep 27, 2019 at 9:47 AM TQ <nothinrandom at gmail.com> wrote:
> >>
> >> > cd ~/Desktop/logs/ && sudo rm -f *.log && zeek -C -t
> ~/Desktop/logs/output.log -r ~/Desktop/pcap/ testPlugin1_pcap_1.pcapng
> >>
> >> The `-t` option isn't commonly used and could see it accidentally
> >> breaking without anyone noticing.  It does still seem to work for me,
> >> but you might try removing it to see if it makes a difference.
> >>
> >> But the best thing would be if you can provide the full directions to
> >> be able to reproduce the segfault -- e.g. the plugin/script code along
> >> with pcap and command-line you're using.
> >>
> >> If you can't share those, then next best thing would be if you can run
> >> in a debugger (gdb, lldb) and share a stack trace of the segfault.
> >>
> >> - Jon
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191001/49926fa8/attachment.html 

From jsiwek at corelight.com  Tue Oct  1 19:23:42 2019
From: jsiwek at corelight.com (Jon Siwek)
Date: Tue, 1 Oct 2019 19:23:42 -0700
Subject: [Zeek] Segmentation Fault on Zeek 3.0.0
In-Reply-To: <CAM2MKSpWcOrVpK3=Yt-WvS8Vd8ZEeT+F=Kjwk5V+Xt1xP2ttwQ@mail.gmail.com>
References: <CAM2MKSpcxdzScdfW8pOuRjaPaxrmFzOPgWX=Y32P728dQz5YKg@mail.gmail.com>
	<CAMzgZ0JQoH8n5xu0K=OTXHvAHGkHEibZLzR3vZEQ7obDBhZeBg@mail.gmail.com>
	<CAM2MKSoXv+hOBi1bg=Jz-ULE4=_Uke9=NdE53aNJ2_dfy7D-fQ@mail.gmail.com>
	<CAMzgZ0LiyNmCcwS1Xz=v4Uu4YYQwFVy=uDGKrhHMB3qMUVN=LA@mail.gmail.com>
	<CAM2MKSpWcOrVpK3=Yt-WvS8Vd8ZEeT+F=Kjwk5V+Xt1xP2ttwQ@mail.gmail.com>
Message-ID: <CAMzgZ0+4WvOL2o25hR=+WH_tvb4jH75VScKvz_iNSRWi0z1CWQ@mail.gmail.com>

Yes, I'll be at ZeekWeek and happy to take a look then.  Else if you
had any minimal reproducer and/or stack trace you can actually share,
feel free to send that along in the meantime.

- Jon

On Tue, Oct 1, 2019 at 4:20 PM TQ <nothinrandom at gmail.com> wrote:
>
> Hey Jon,
>
> Unfortunately, I'm not at liberty to share right now.  However, my colleague Blake Johnson and I might give a talk at Zeek Week next week (https://twitter.com/voteblake/status/1178787539999526912?s=20).  Will you be there?  I'd love to catch up with you and show you in person instead.  It's probably some silly issue/misconfiguration on my end.
>
> Thanks,
>
> On Mon, Sep 30, 2019 at 9:37 AM Jon Siwek <jsiwek at corelight.com> wrote:
>>
>> Can you provide more information on how to reproduce the issue (exact
>> scripts/plugins/pcaps that crash every time) ?  There's still a bug in
>> Zeek to fix here, but just adding `-t` and trying a few things hasn't
>> triggered it for me.
>>
>> - Jon
>>
>> On Fri, Sep 27, 2019 at 7:57 PM TQ <nothinrandom at gmail.com> wrote:
>> >
>> > Hey Jon,
>> >
>> > Thanks for guidance on this!  You are absolutely right.  If I remove "-t ~/Desktop/logs/output.log", then that segmentation fault goes away.  I have not a clue why as it works fine for 2.6.2.  I thought something was wrong with the actual code.  Again, thanks for helping out with this!
>> >
>> > Thanks,
>> >
>> > On Fri, Sep 27, 2019 at 5:16 PM Jon Siwek <jsiwek at corelight.com> wrote:
>> >>
>> >> On Fri, Sep 27, 2019 at 9:47 AM TQ <nothinrandom at gmail.com> wrote:
>> >>
>> >> > cd ~/Desktop/logs/ && sudo rm -f *.log && zeek -C -t ~/Desktop/logs/output.log -r ~/Desktop/pcap/ testPlugin1_pcap_1.pcapng
>> >>
>> >> The `-t` option isn't commonly used and could see it accidentally
>> >> breaking without anyone noticing.  It does still seem to work for me,
>> >> but you might try removing it to see if it makes a difference.
>> >>
>> >> But the best thing would be if you can provide the full directions to
>> >> be able to reproduce the segfault -- e.g. the plugin/script code along
>> >> with pcap and command-line you're using.
>> >>
>> >> If you can't share those, then next best thing would be if you can run
>> >> in a debugger (gdb, lldb) and share a stack trace of the segfault.
>> >>
>> >> - Jon


From alajal at gmail.com  Wed Oct  2 02:04:43 2019
From: alajal at gmail.com (Mustafa Qasim)
Date: Wed, 2 Oct 2019 19:04:43 +1000
Subject: [Zeek] Investigating logger crash/restart events
Message-ID: <CAFhjsXJPD-2SUQGAvcj5y8Ff7BqxJ59muZZVPUW4GmL+o1Mb_w@mail.gmail.com>

Hi,

I have an instance where log files were terminated during the day and
started the next day around the same time. broctl status shows that logger
started exactly a few seconds before the time logging was resumed.

What procedures one can follow to investigate such disruption?

------
*Mustafa Qasim*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191002/09f1bffc/attachment.html 

From bill.de.ping at gmail.com  Wed Oct  2 04:02:02 2019
From: bill.de.ping at gmail.com (william de ping)
Date: Wed, 2 Oct 2019 14:02:02 +0300
Subject: [Zeek] Why does my logger keep crashing - bro version 2.6.3
In-Reply-To: <BN6PR09MB11544ACBD5A65A117708FABDA1810@BN6PR09MB1154.namprd09.prod.outlook.com>
References: <BN6PR09MB11542B9F128BD9B7413F842BA1890@BN6PR09MB1154.namprd09.prod.outlook.com>
	<BN6PR09MB115443C06D491DA302F640F3A1890@BN6PR09MB1154.namprd09.prod.outlook.com>
	<CAKZA8xjFZpcsZQ-JVXyT5ZrAaZ0bL7VObV+69NRejn6Wyy4AzQ@mail.gmail.com>
	<BN6PR09MB11542783738CE7EFBDEF8324A1850@BN6PR09MB1154.namprd09.prod.outlook.com>
	<CAPqbkwtZSuqUSTLJFDGfpWegyA=gucOZmGbdqvqxoSboLs55jQ@mail.gmail.com>
	<BN6PR09MB1154540BEAE66A7A8991C63EA1840@BN6PR09MB1154.namprd09.prod.outlook.com>
	<CAKZA8xih9SnfgCBODS0cqKj=mh8jtKdFxR96j8rNMw8E6A+p=w@mail.gmail.com>
	<BN6PR09MB11544ACBD5A65A117708FABDA1810@BN6PR09MB1154.namprd09.prod.outlook.com>
Message-ID: <CAKZA8xh0qsq_=T8n6uXEzqpoiT5RAuKkhqUmDhPHkS783dNstA@mail.gmail.com>

Hi,

Can you please share your entire node.cfg file ?

It looks like you've added 3 more workers. I would check if the CPUs you
are pinning has a direct PCI lane to the NIC you are listening on.
Check the numa node the NIC is attached to and make sure you are pinning
the correct CPUs first

B


On Fri, Sep 27, 2019 at 7:27 PM Kayode Enwerem <
Kayode_Enwerem at ao.uscourts.gov> wrote:

> Looks like setting up 2 loggers resolved the issue of my logger crashing
> but my Dropped packets are pretty high on my workers. Can someone assist me
> with how I can reduce my dropped packets.
>
>
>
> cat capture_loss.log
>
> #separator \x09
>
> #set_separator  ,
>
> #empty_field    (empty)
>
> #unset_field    -
>
> #path   capture_loss
>
> #open   2019-09-27-12-05-05
>
> #fields ts      ts_delta        peer    gaps    acks    percent_lost
>
> #types  time    interval        string  count   count   double
>
> 1569600304.774215       900.000013      worker-1-1      126463  3246542
> 3.895314
>
> 1569600304.783703       900.000064      worker-1-3      106904  4465333
> 2.394088
>
> 1569600304.785983       900.000212      worker-1-11     123729  3768503
> 3.28324
>
> 1569600304.802244       900.000098      worker-1-14     144154  3584013
> 4.022139
>
> 1569600304.823378       900.000095      worker-1-18     137507  3503583
> 3.924754
>
> 1569600304.892559       900.000470      worker-1-13     148904  3448544
> 4.31788
>
> 1569600305.010986       900.000030      worker-1-8      174213  3409819
> 5.109157
>
> 1569600305.938686       901.043465      worker-1-15     509268  1072199
> 47.497526
>
> 1569600304.806850       900.000047      worker-1-22     591232  1234893
> 47.877185
>
> 1569601204.762382       900.000786      worker-1-16     120086  4491072
> 2.673883
>
> 1569601204.774220       900.000005      worker-1-1      127257  3461349
> 3.676515
>
> 1569601204.802447       900.000203      worker-1-14     125481  3171663
> 3.956316
>
> 1569601204.884438       900.000029      worker-1-19     125037  3566663
> 3.505714
>
> 1569601204.891746       900.000015      worker-1-23     120553  3078889
> 3.915471
>
> 1569601205.110098       900.000139      worker-1-10     108016  3442813
> 3.137434
>
> 1569601205.938906       900.000220      worker-1-15     565536  1156759
> 48.8897
>
> 1569601218.120290       900.000047      worker-1-6      456312  753749
> 60.538986
>
>
>
> Below are some of my settings:
>
>
>
> I have 23 workers defined and I pinned CPU.
>
> [worker-1]
>
> type=worker
>
> host=localhost
>
> interface=af_packet::ens2f0
>
> lb_method=custom
>
> #lb_method=pf_ring
>
> lb_procs=23
>
> pin_cpus=5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27
>
> af_packet_fanout_id=25
>
> af_packet_fanout_mode=AF_Packet::FANOUT_HASH
>
>
>
> Can someone assist me with this.
>
>
>
> Thanks.
>
>
>
>
>
> *From:* william de ping <bill.de.ping at gmail.com>
> *Sent:* Wednesday, September 25, 2019 4:00 AM
> *To:* Kayode Enwerem <Kayode_Enwerem at ao.uscourts.gov>
> *Cc:* zeek at zeek.org
> *Subject:* Re: [Zeek] Why does my logger keep crashing - bro version 2.6.3
>
>
>
> Hi
>
>
>
> Try using the None writer instead of the ASCII one.
>
> In local.bro add :
>
> redef Log::default_writer=Log::WRITER_NONE;
>
>
>
> If the logger instance still crashes then the issue is not related to an
> IO bottleneck.
>
>
>
> B
>
>
>
> On Tue, Sep 24, 2019 at 7:49 PM Kayode Enwerem <
> Kayode_Enwerem at ao.uscourts.gov> wrote:
>
> Thanks for your response.
>
> I do see the following OOM message in my system logs on the logger process
> ID:
> Sep 23 18:48:00 kernel: Out of memory: Kill process 10439 (bro) score 787
> or sacrifice child
> Sep 23 18:48:00 kernel: Killed process 10439 (bro), UID 0,
> total-vm:301983900kB, anon-rss:195261772kB, file-rss:2592kB, shmem-rss:0kB
>
> Wonder why its taking so much memory, I have 251G and 99G swap on this
> server.
> total        used        free      shared  buff/cache   available
> Mem:           251G         66G        185G        4.2M        488M
> 184G
> Swap:           99G        1.1G         98G
>
> Below is the output of "broctl diag logger", ran after the logger crashed.
>
>  [logger]
>
> No core file found.
>
> Bro 2.6.3
> Linux 3.10.0-1062.1.1.el7.x86_64
>
> Bro plugins:
> Bro::AF_Packet - Packet acquisition via AF_Packet (dynamic, version 1.4)
>
> ==== No reporter.log
>
> ==== stderr.log
> /usr/local/bro/share/broctl/scripts/run-bro: line 110: 10439 Killed
>           nohup "$mybro" "$@"
>
> ==== stdout.log
> max memory size         (kbytes, -m) unlimited
> data seg size           (kbytes, -d) unlimited
> virtual memory          (kbytes, -v) unlimited
> core file size          (blocks, -c) unlimited
>
> ==== .cmdline
> -U .status -p broctl -p broctl-live -p local -p logger local.bro broctl
> base/frameworks/cluster broctl/auto
>
> ==== .env_vars
>
> PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bro/bin
>
> BROPATH=/logs/bro/spool/installed-scripts-do-not-touch/site::/logs/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site
> CLUSTER_NODE=logger
>
> ==== .status
> RUNNING [net_run]
>
> ==== No prof.log
>
> ==== No packet_filter.log
>
> ==== No loaded_scripts.log
>
> Thoughts? Any suggestions.
>
> -----Original Message-----
> From: Vlad Grigorescu <vlad at es.net>
> Sent: Monday, September 23, 2019 10:20 AM
> To: Kayode Enwerem <Kayode_Enwerem at ao.uscourts.gov>
> Cc: william de ping <bill.de.ping at gmail.com>; zeek at zeek.org
> Subject: Re: [Zeek] Why does my logger keep crashing - bro version 2.6.3
>
> The logger is threaded, so seeing CPU > 100% is not necessarily a problem.
>
> Have you tried running "broctl diag logger" to see why the logger is
> crashing? Do you have any messages in your system logs about processing
> being killed for out of memory (OOM)?
>
>   --Vlad
>
> On Mon, Sep 23, 2019 at 1:32 PM Kayode Enwerem <
> Kayode_Enwerem at ao.uscourts.gov> wrote:
> >
> > Thanks for your response. The CPU usage for the logger is at 311%. (look
> below).
> >
> >
> >
> > broctl top
> >
> > Name         Type    Host             Pid     VSize  Rss  Cpu   Cmd
> >
> > logger       logger  localhost        22867    12G     9G 311%  bro
> >
> >
> >
> > I wasn?t aware that you could set up multiple loggers, I tried checking
> the docs to see if that was an option. Does anyone know how to do this?
> >
> >
> >
> > From: william de ping <bill.de.ping at gmail.com>
> > Sent: Sunday, September 22, 2019 6:42 AM
> > To: Kayode Enwerem <Kayode_Enwerem at ao.uscourts.gov>
> > Cc: zeek at zeek.org
> > Subject: Re: [Zeek] Why does my logger keep crashing - bro version
> > 2.6.3
> >
> >
> >
> > Hi,
> >
> >
> >
> > I would try to monitor the cpu \ mem usage of the logger instance.
> >
> > Try running broctl top, my guess is that you will see that the logger
> process will have a very high cpu usage.
> >
> >
> >
> > I know of an option to have multiple loggers but I am not sure how to
> set it up.
> >
> >
> >
> > Are you writing to a file ?
> >
> >
> >
> > B
> >
> >
> >
> > On Thu, Sep 19, 2019 at 7:14 PM Kayode Enwerem <
> Kayode_Enwerem at ao.uscourts.gov> wrote:
> >
> > Hello,
> >
> >
> >
> > Why does my logger keep crashing? Can someone please help me with this.
> I have provided some system information below:
> >
> >
> >
> > I am running bro version 2.6.3
> >
> >
> >
> > Output of broctl status. The logger is crashed but the manager, proxy
> and workers are still running.
> >
> > broctl status
> >
> > Name         Type    Host             Status    Pid    Started
> >
> > logger       logger  localhost        crashed
> >
> > manager      manager localhost        running   17356  09 Sep 15:42:24
> >
> > proxy-1      proxy   localhost        running   17401  09 Sep 15:42:25
> >
> > worker-1-1   worker  localhost        running   17573  09 Sep 15:42:27
> >
> > worker-1-2   worker  localhost        running   17569  09 Sep 15:42:27
> >
> > worker-1-3   worker  localhost        running   17572  09 Sep 15:42:27
> >
> > worker-1-4   worker  localhost        running   17587  09 Sep 15:42:27
> >
> > worker-1-5   worker  localhost        running   17619  09 Sep 15:42:27
> >
> > worker-1-6   worker  localhost        running   17614  09 Sep 15:42:27
> >
> > worker-1-7   worker  localhost        running   17625  09 Sep 15:42:27
> >
> > worker-1-8   worker  localhost        running   17646  09 Sep 15:42:27
> >
> > worker-1-9   worker  localhost        running   17671  09 Sep 15:42:27
> >
> > worker-1-10  worker  localhost        running   17663  09 Sep 15:42:27
> >
> > worker-1-11  worker  localhost        running   17679  09 Sep 15:42:27
> >
> > worker-1-12  worker  localhost        running   17685  09 Sep 15:42:27
> >
> > worker-1-13  worker  localhost        running   17698  09 Sep 15:42:27
> >
> > worker-1-14  worker  localhost        running   17703  09 Sep 15:42:27
> >
> > worker-1-15  worker  localhost        running   17710  09 Sep 15:42:27
> >
> > worker-1-16  worker  localhost        running   17717  09 Sep 15:42:27
> >
> > worker-1-17  worker  localhost        running   17720  09 Sep 15:42:27
> >
> > worker-1-18  worker  localhost        running   17727  09 Sep 15:42:27
> >
> > worker-1-19  worker  localhost        running   17728  09 Sep 15:42:27
> >
> > worker-1-20  worker  localhost        running   17731  09 Sep 15:42:27
> >
> >
> >
> > Here?s my node.cfg settings
> >
> > [logger]
> >
> > type=logger
> >
> > host=localhost
> >
> >
> >
> > [manager]
> >
> > type=manager
> >
> > host=localhost
> >
> >
> >
> > [proxy-1]
> >
> > type=proxy
> >
> > host=localhost
> >
> >
> >
> > [worker-1]
> >
> > type=worker
> >
> > host=localhost
> >
> > interface=af_packet::ens2f0
> >
> > lb_method=custom
> >
> > #lb_method=pf_ring
> >
> > lb_procs=20
> >
> > pin_cpus=6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25
> >
> > af_packet_fanout_id=25
> >
> > af_packet_fanout_mode=AF_Packet::FANOUT_HASH
> >
> >
> >
> > Heres more information on my CPU. 32 CPUs, model name ? AMD, CPU max
> > MHz is 2800.0000
> >
> > Architecture:          x86_64
> >
> > CPU op-mode(s):        32-bit, 64-bit
> >
> > Byte Order:            Little Endian
> >
> > CPU(s):                32
> >
> > On-line CPU(s) list:   0-31
> >
> > Thread(s) per core:    2
> >
> > Core(s) per socket:    8
> >
> > Socket(s):             2
> >
> > NUMA node(s):          4
> >
> > Vendor ID:             AuthenticAMD
> >
> > CPU family:            21
> >
> > Model:                 2
> >
> > Model name:            AMD Opteron(tm) Processor 6386 SE
> >
> > Stepping:              0
> >
> > CPU MHz:               1960.000
> >
> > CPU max MHz:           2800.0000
> >
> > CPU min MHz:           1400.0000
> >
> > BogoMIPS:              5585.93
> >
> > Virtualization:        AMD-V
> >
> > L1d cache:             16K
> >
> > L1i cache:             64K
> >
> > L2 cache:              2048K
> >
> > L3 cache:              6144K
> >
> > NUMA node0 CPU(s):     0,2,4,6,8,10,12,14
> >
> > NUMA node1 CPU(s):     16,18,20,22,24,26,28,30
> >
> > NUMA node2 CPU(s):     1,3,5,7,9,11,13,15
> >
> > NUMA node3 CPU(s):     17,19,21,23,25,27,29,31
> >
> >
> >
> > Would also like to know how I can reduce my packet loss. Below is the
> > output of broctl netstats
> >
> > broctl netstats
> >
> > worker-1-1: 1568908277.861813 recvd=12248845422 dropped=5171188999
> > link=17420313882
> >
> > worker-1-2: 1568908298.313954 recvd=8636707266 dropped=971489
> > link=8637678939
> >
> > worker-1-3: 1568908278.425888 recvd=11684808853 dropped=5617381647
> > link=17302473791
> >
> > worker-1-4: 1568908285.731130 recvd=12567242226 dropped=4339688288
> > link=16907212802
> >
> > worker-1-5: 1568908298.363911 recvd=8620499351 dropped=24595149
> > link=8645095758
> >
> > worker-1-6: 1568908298.372892 recvd=8710565757 dropped=1731022
> > link=8712297432
> >
> > worker-1-7: 1568908298.266010 recvd=9065207444 dropped=53523232
> > link=9118737229
> >
> > worker-1-8: 1568908286.935607 recvd=11377790124 dropped=3680887247
> > link=15058934491
> >
> > worker-1-9: 1568908298.419657 recvd=8931903322 dropped=39696184
> > link=8971604219
> >
> > worker-1-10: 1568908298.478576 recvd=8842874030 dropped=2501252
> > link=8845376352
> >
> > worker-1-11: 1568908298.506649 recvd=8692769329 dropped=2253413
> > link=8695025626
> >
> > worker-1-12: 1568908298.520830 recvd=8749977028 dropped=2314733
> > link=8752293714
> >
> > worker-1-13: 1568908298.544573 recvd=9101243757 dropped=1779460
> > link=9103025399
> >
> > worker-1-14: 1568908291.370011 recvd=10876925726 dropped=775722632
> > link=11652810353
> >
> > worker-1-15: 1568908298.579721 recvd=8503097394 dropped=1420699
> > link=8504520066
> >
> > worker-1-16: 1568908298.594942 recvd=8515164266 dropped=1840977
> > link=8517006779
> >
> > worker-1-17: 1568908298.646966 recvd=10666567717 dropped=466489754
> > link=11133059283
> >
> > worker-1-18: 1568908298.671246 recvd=9023603573 dropped=2037607
> > link=9025642263
> >
> > worker-1-19: 1568908298.704675 recvd=8907784186 dropped=1164594
> > link=8908950238
> >
> > worker-1-20: 1568908298.718084 recvd=9140525444 dropped=2028593
> > link=9142555259
> >
> >
> >
> > Thanks,
> >
> > _______________________________________________
> > Zeek mailing list
> > zeek at zeek.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> >
> > _______________________________________________
> > Zeek mailing list
> > zeek at zeek.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191002/52eddc77/attachment-0001.html 

From mauro.palumbo at aizoon.it  Thu Oct  3 05:35:23 2019
From: mauro.palumbo at aizoon.it (Palumbo Mauro)
Date: Thu, 3 Oct 2019 12:35:23 +0000
Subject: [Zeek] duplicated intel logs DNS::IN_REQUEST
Message-ID: <f14f812365414364b5cea609820b7fdd@SRVEX03.aizoon.local>

Hi everybody,

  I am having an issue with the intel.log file, I am getting duplicated lines for the same dns request such as:

#fields ts      uid     id.orig_h       id.orig_p       id.resp_h       id.resp_p       seen.indicator  seen.indicator_type     seen.where      seen.node       matched sources fuid    file_mime_type  file_desc       cif.tags        cif.confidence  cif.source      cif.description cif.firstseen   cif.lastseen
#types  time    string  addr    port    addr    port    string  enum    enum    string  set[enum]       set[string]     string  string  string  string  double  string  string  string  string
1570105259.197420       CP1BZx1QgzdPpfEyda      172.17.0.186    43283   172.16.1.10     53      opencalphad.com Intel::DOMAIN   DNS::IN_REQUEST worker-1        Intel::DOMAIN   0       -       -       -       -       85.0    -       -       -       -
1570105259.197420       CP1BZx1QgzdPpfEyda      172.17.0.186    43283   172.16.1.10     53      opencalphad.com Intel::DOMAIN   DNS::IN_REQUEST worker-1        Intel::DOMAIN   0       -       -       -       -       85.0    -       -       -       -
1570105259.207335       CJZASAQTB2qgPSYw7       172.17.0.186    59553   172.16.1.10     53      opencalphad.com Intel::DOMAIN   DNS::IN_REQUEST worker-1        Intel::DOMAIN   0       -       -       -       -       85.0    -       -       -       -
1570105259.211927       CJZASAQTB2qgPSYw7       172.17.0.186    59553   172.16.1.10     53      opencalphad.com Intel::DOMAIN   DNS::IN_REQUEST worker-1        Intel::DOMAIN   0       -       -       -       -       85.0    -       -

As you can see, some lines are identical, same uid, same worker, same timestamp, etc...

>From my tests, it appears that the problem is not in the intel framework but possibly in the dns analyzer as the event "dns_request" is raised twice (in the same worker) even if the dns packet contains a single query. This happens in a cluster configuration. The manager then receives twice the event dns_request  and its intel framework matches it twice as well. Hence, two logs... As far as I read in the doc, the event "dns_request" is raised more than once if the dns packet contains multiple queries, but this doesn't seem to be the case here.

I have the same issue on different machines and I tried both bro v.2.6.1 and latest zeek from github.

Does anyone have a clue about what is happening?

Thanks,
Mauro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191003/bd334737/attachment.html 

From Melissa.Carpenter1 at gdit.com  Thu Oct  3 05:38:11 2019
From: Melissa.Carpenter1 at gdit.com (Carpenter, Melissa)
Date: Thu, 3 Oct 2019 12:38:11 +0000
Subject: [Zeek] Zeek Week Registration Question
Message-ID: <fb3b6c7ef2064a3ea636c39e40cb423d@HQ-EXCHMBX06.ad.local>

Hey All, 

Is it possible to transfer my registration to one of my coworkers? I will no
longer be with the organization and would hate for someone to miss the
opportunity to go. 

Thanks, 

Melissa

-----Original Message-----
From: zeek-bounces at zeek.org <zeek-bounces at zeek.org> On Behalf Of
zeek-request at zeek.org
Sent: Wednesday, October 2, 2019 6:59 AM
To: zeek at zeek.org
Subject: Zeek Digest, Vol 162, Issue 2

Send Zeek mailing list submissions to
	zeek at zeek.org

To subscribe or unsubscribe via the World Wide Web, visit
	http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
or, via email, send a message with subject or body 'help' to
	zeek-request at zeek.org

You can reach the person managing the list at
	zeek-owner at zeek.org

When replying, please edit your Subject line so it is more specific than
"Re: Contents of Zeek digest..."


Today's Topics:

   1. Re: Segmentation Fault on Zeek 3.0.0 (TQ)
   2. Re: Segmentation Fault on Zeek 3.0.0 (Jon Siwek)
   3. Investigating logger crash/restart events (Mustafa Qasim)
   4. Re: Why does my logger keep crashing - bro version 2.6.3
      (william de ping)


----------------------------------------------------------------------

Message: 1
Date: Tue, 1 Oct 2019 16:20:07 -0700
From: TQ <nothinrandom at gmail.com>
Subject: Re: [Zeek] Segmentation Fault on Zeek 3.0.0
To: Jon Siwek <jsiwek at corelight.com>
Cc: zeek <zeek at zeek.org>
Message-ID:
	<CAM2MKSpWcOrVpK3=Yt-WvS8Vd8ZEeT+F=Kjwk5V+Xt1xP2ttwQ at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hey Jon,

Unfortunately, I'm not at liberty to share right now.  However, my colleague
Blake Johnson and I might give a talk at Zeek Week next week (
https://twitter.com/voteblake/status/1178787539999526912?s=20).  Will you be
there?  I'd love to catch up with you and show you in person instead.
It's probably some silly issue/misconfiguration on my end.

Thanks,

On Mon, Sep 30, 2019 at 9:37 AM Jon Siwek <jsiwek at corelight.com> wrote:

> Can you provide more information on how to reproduce the issue (exact 
> scripts/plugins/pcaps that crash every time) ?  There's still a bug in 
> Zeek to fix here, but just adding `-t` and trying a few things hasn't 
> triggered it for me.
>
> - Jon
>
> On Fri, Sep 27, 2019 at 7:57 PM TQ <nothinrandom at gmail.com> wrote:
> >
> > Hey Jon,
> >
> > Thanks for guidance on this!  You are absolutely right.  If I remove 
> > "-t
> ~/Desktop/logs/output.log", then that segmentation fault goes away.  I 
> have not a clue why as it works fine for 2.6.2.  I thought something 
> was wrong with the actual code.  Again, thanks for helping out with this!
> >
> > Thanks,
> >
> > On Fri, Sep 27, 2019 at 5:16 PM Jon Siwek <jsiwek at corelight.com> wrote:
> >>
> >> On Fri, Sep 27, 2019 at 9:47 AM TQ <nothinrandom at gmail.com> wrote:
> >>
> >> > cd ~/Desktop/logs/ && sudo rm -f *.log && zeek -C -t
> ~/Desktop/logs/output.log -r ~/Desktop/pcap/ testPlugin1_pcap_1.pcapng
> >>
> >> The `-t` option isn't commonly used and could see it accidentally 
> >> breaking without anyone noticing.  It does still seem to work for 
> >> me, but you might try removing it to see if it makes a difference.
> >>
> >> But the best thing would be if you can provide the full directions 
> >> to be able to reproduce the segfault -- e.g. the plugin/script code 
> >> along with pcap and command-line you're using.
> >>
> >> If you can't share those, then next best thing would be if you can 
> >> run in a debugger (gdb, lldb) and share a stack trace of the segfault.
> >>
> >> - Jon
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191001/49926fa
8/attachment-0001.html 

------------------------------

Message: 2
Date: Tue, 1 Oct 2019 19:23:42 -0700
From: Jon Siwek <jsiwek at corelight.com>
Subject: Re: [Zeek] Segmentation Fault on Zeek 3.0.0
To: TQ <nothinrandom at gmail.com>
Cc: zeek <zeek at zeek.org>
Message-ID:
	<CAMzgZ0+4WvOL2o25hR=+WH_tvb4jH75VScKvz_iNSRWi0z1CWQ at mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"

Yes, I'll be at ZeekWeek and happy to take a look then.  Else if you had any
minimal reproducer and/or stack trace you can actually share, feel free to
send that along in the meantime.

- Jon

On Tue, Oct 1, 2019 at 4:20 PM TQ <nothinrandom at gmail.com> wrote:
>
> Hey Jon,
>
> Unfortunately, I'm not at liberty to share right now.  However, my
colleague Blake Johnson and I might give a talk at Zeek Week next week
(https://twitter.com/voteblake/status/1178787539999526912?s=20).  Will you
be there?  I'd love to catch up with you and show you in person instead.
It's probably some silly issue/misconfiguration on my end.
>
> Thanks,
>
> On Mon, Sep 30, 2019 at 9:37 AM Jon Siwek <jsiwek at corelight.com> wrote:
>>
>> Can you provide more information on how to reproduce the issue (exact 
>> scripts/plugins/pcaps that crash every time) ?  There's still a bug 
>> in Zeek to fix here, but just adding `-t` and trying a few things 
>> hasn't triggered it for me.
>>
>> - Jon
>>
>> On Fri, Sep 27, 2019 at 7:57 PM TQ <nothinrandom at gmail.com> wrote:
>> >
>> > Hey Jon,
>> >
>> > Thanks for guidance on this!  You are absolutely right.  If I remove
"-t ~/Desktop/logs/output.log", then that segmentation fault goes away.  I
have not a clue why as it works fine for 2.6.2.  I thought something was
wrong with the actual code.  Again, thanks for helping out with this!
>> >
>> > Thanks,
>> >
>> > On Fri, Sep 27, 2019 at 5:16 PM Jon Siwek <jsiwek at corelight.com> wrote:
>> >>
>> >> On Fri, Sep 27, 2019 at 9:47 AM TQ <nothinrandom at gmail.com> wrote:
>> >>
>> >> > cd ~/Desktop/logs/ && sudo rm -f *.log && zeek -C -t 
>> >> > ~/Desktop/logs/output.log -r ~/Desktop/pcap/ 
>> >> > testPlugin1_pcap_1.pcapng
>> >>
>> >> The `-t` option isn't commonly used and could see it accidentally 
>> >> breaking without anyone noticing.  It does still seem to work for 
>> >> me, but you might try removing it to see if it makes a difference.
>> >>
>> >> But the best thing would be if you can provide the full directions 
>> >> to be able to reproduce the segfault -- e.g. the plugin/script 
>> >> code along with pcap and command-line you're using.
>> >>
>> >> If you can't share those, then next best thing would be if you can 
>> >> run in a debugger (gdb, lldb) and share a stack trace of the segfault.
>> >>
>> >> - Jon



------------------------------

Message: 3
Date: Wed, 2 Oct 2019 19:04:43 +1000
From: Mustafa Qasim <alajal at gmail.com>
Subject: [Zeek] Investigating logger crash/restart events
To: zeek <Zeek at zeek.org>
Message-ID:
	<CAFhjsXJPD-2SUQGAvcj5y8Ff7BqxJ59muZZVPUW4GmL+o1Mb_w at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

I have an instance where log files were terminated during the day and
started the next day around the same time. broctl status shows that logger
started exactly a few seconds before the time logging was resumed.

What procedures one can follow to investigate such disruption?

------
*Mustafa Qasim*
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191002/09f1bff
c/attachment-0001.html 

------------------------------

Message: 4
Date: Wed, 2 Oct 2019 14:02:02 +0300
From: william de ping <bill.de.ping at gmail.com>
Subject: Re: [Zeek] Why does my logger keep crashing - bro version
	2.6.3
To: Kayode Enwerem <Kayode_Enwerem at ao.uscourts.gov>
Cc: "zeek at zeek.org" <zeek at zeek.org>
Message-ID:
	<CAKZA8xh0qsq_=T8n6uXEzqpoiT5RAuKkhqUmDhPHkS783dNstA at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

Can you please share your entire node.cfg file ?

It looks like you've added 3 more workers. I would check if the CPUs you are
pinning has a direct PCI lane to the NIC you are listening on.
Check the numa node the NIC is attached to and make sure you are pinning the
correct CPUs first

B


On Fri, Sep 27, 2019 at 7:27 PM Kayode Enwerem <
Kayode_Enwerem at ao.uscourts.gov> wrote:

> Looks like setting up 2 loggers resolved the issue of my logger 
> crashing but my Dropped packets are pretty high on my workers. Can 
> someone assist me with how I can reduce my dropped packets.
>
>
>
> cat capture_loss.log
>
> #separator \x09
>
> #set_separator  ,
>
> #empty_field    (empty)
>
> #unset_field    -
>
> #path   capture_loss
>
> #open   2019-09-27-12-05-05
>
> #fields ts      ts_delta        peer    gaps    acks    percent_lost
>
> #types  time    interval        string  count   count   double
>
> 1569600304.774215       900.000013      worker-1-1      126463  3246542
> 3.895314
>
> 1569600304.783703       900.000064      worker-1-3      106904  4465333
> 2.394088
>
> 1569600304.785983       900.000212      worker-1-11     123729  3768503
> 3.28324
>
> 1569600304.802244       900.000098      worker-1-14     144154  3584013
> 4.022139
>
> 1569600304.823378       900.000095      worker-1-18     137507  3503583
> 3.924754
>
> 1569600304.892559       900.000470      worker-1-13     148904  3448544
> 4.31788
>
> 1569600305.010986       900.000030      worker-1-8      174213  3409819
> 5.109157
>
> 1569600305.938686       901.043465      worker-1-15     509268  1072199
> 47.497526
>
> 1569600304.806850       900.000047      worker-1-22     591232  1234893
> 47.877185
>
> 1569601204.762382       900.000786      worker-1-16     120086  4491072
> 2.673883
>
> 1569601204.774220       900.000005      worker-1-1      127257  3461349
> 3.676515
>
> 1569601204.802447       900.000203      worker-1-14     125481  3171663
> 3.956316
>
> 1569601204.884438       900.000029      worker-1-19     125037  3566663
> 3.505714
>
> 1569601204.891746       900.000015      worker-1-23     120553  3078889
> 3.915471
>
> 1569601205.110098       900.000139      worker-1-10     108016  3442813
> 3.137434
>
> 1569601205.938906       900.000220      worker-1-15     565536  1156759
> 48.8897
>
> 1569601218.120290       900.000047      worker-1-6      456312  753749
> 60.538986
>
>
>
> Below are some of my settings:
>
>
>
> I have 23 workers defined and I pinned CPU.
>
> [worker-1]
>
> type=worker
>
> host=localhost
>
> interface=af_packet::ens2f0
>
> lb_method=custom
>
> #lb_method=pf_ring
>
> lb_procs=23
>
> pin_cpus=5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,
> 27
>
> af_packet_fanout_id=25
>
> af_packet_fanout_mode=AF_Packet::FANOUT_HASH
>
>
>
> Can someone assist me with this.
>
>
>
> Thanks.
>
>
>
>
>
> *From:* william de ping <bill.de.ping at gmail.com>
> *Sent:* Wednesday, September 25, 2019 4:00 AM
> *To:* Kayode Enwerem <Kayode_Enwerem at ao.uscourts.gov>
> *Cc:* zeek at zeek.org
> *Subject:* Re: [Zeek] Why does my logger keep crashing - bro version 
> 2.6.3
>
>
>
> Hi
>
>
>
> Try using the None writer instead of the ASCII one.
>
> In local.bro add :
>
> redef Log::default_writer=Log::WRITER_NONE;
>
>
>
> If the logger instance still crashes then the issue is not related to 
> an IO bottleneck.
>
>
>
> B
>
>
>
> On Tue, Sep 24, 2019 at 7:49 PM Kayode Enwerem < 
> Kayode_Enwerem at ao.uscourts.gov> wrote:
>
> Thanks for your response.
>
> I do see the following OOM message in my system logs on the logger 
> process
> ID:
> Sep 23 18:48:00 kernel: Out of memory: Kill process 10439 (bro) score 
> 787 or sacrifice child Sep 23 18:48:00 kernel: Killed process 10439 
> (bro), UID 0, total-vm:301983900kB, anon-rss:195261772kB, 
> file-rss:2592kB, shmem-rss:0kB
>
> Wonder why its taking so much memory, I have 251G and 99G swap on this 
> server.
> total        used        free      shared  buff/cache   available
> Mem:           251G         66G        185G        4.2M        488M
> 184G
> Swap:           99G        1.1G         98G
>
> Below is the output of "broctl diag logger", ran after the logger crashed.
>
>  [logger]
>
> No core file found.
>
> Bro 2.6.3
> Linux 3.10.0-1062.1.1.el7.x86_64
>
> Bro plugins:
> Bro::AF_Packet - Packet acquisition via AF_Packet (dynamic, version 
> 1.4)
>
> ==== No reporter.log
>
> ==== stderr.log
> /usr/local/bro/share/broctl/scripts/run-bro: line 110: 10439 Killed
>           nohup "$mybro" "$@"
>
> ==== stdout.log
> max memory size         (kbytes, -m) unlimited
> data seg size           (kbytes, -d) unlimited
> virtual memory          (kbytes, -v) unlimited
> core file size          (blocks, -c) unlimited
>
> ==== .cmdline
> -U .status -p broctl -p broctl-live -p local -p logger local.bro 
> broctl base/frameworks/cluster broctl/auto
>
> ==== .env_vars
>
> PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/sbin:/bin
> :/usr/sbin:/usr/bin:/usr/local/bro/bin
>
> BROPATH=/logs/bro/spool/installed-scripts-do-not-touch/site::/logs/bro
> /spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/u
> sr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site
> CLUSTER_NODE=logger
>
> ==== .status
> RUNNING [net_run]
>
> ==== No prof.log
>
> ==== No packet_filter.log
>
> ==== No loaded_scripts.log
>
> Thoughts? Any suggestions.
>
> -----Original Message-----
> From: Vlad Grigorescu <vlad at es.net>
> Sent: Monday, September 23, 2019 10:20 AM
> To: Kayode Enwerem <Kayode_Enwerem at ao.uscourts.gov>
> Cc: william de ping <bill.de.ping at gmail.com>; zeek at zeek.org
> Subject: Re: [Zeek] Why does my logger keep crashing - bro version 
> 2.6.3
>
> The logger is threaded, so seeing CPU > 100% is not necessarily a problem.
>
> Have you tried running "broctl diag logger" to see why the logger is 
> crashing? Do you have any messages in your system logs about 
> processing being killed for out of memory (OOM)?
>
>   --Vlad
>
> On Mon, Sep 23, 2019 at 1:32 PM Kayode Enwerem < 
> Kayode_Enwerem at ao.uscourts.gov> wrote:
> >
> > Thanks for your response. The CPU usage for the logger is at 311%. 
> > (look
> below).
> >
> >
> >
> > broctl top
> >
> > Name         Type    Host             Pid     VSize  Rss  Cpu   Cmd
> >
> > logger       logger  localhost        22867    12G     9G 311%  bro
> >
> >
> >
> > I wasn?t aware that you could set up multiple loggers, I tried 
> > checking
> the docs to see if that was an option. Does anyone know how to do this?
> >
> >
> >
> > From: william de ping <bill.de.ping at gmail.com>
> > Sent: Sunday, September 22, 2019 6:42 AM
> > To: Kayode Enwerem <Kayode_Enwerem at ao.uscourts.gov>
> > Cc: zeek at zeek.org
> > Subject: Re: [Zeek] Why does my logger keep crashing - bro version
> > 2.6.3
> >
> >
> >
> > Hi,
> >
> >
> >
> > I would try to monitor the cpu \ mem usage of the logger instance.
> >
> > Try running broctl top, my guess is that you will see that the 
> > logger
> process will have a very high cpu usage.
> >
> >
> >
> > I know of an option to have multiple loggers but I am not sure how 
> > to
> set it up.
> >
> >
> >
> > Are you writing to a file ?
> >
> >
> >
> > B
> >
> >
> >
> > On Thu, Sep 19, 2019 at 7:14 PM Kayode Enwerem <
> Kayode_Enwerem at ao.uscourts.gov> wrote:
> >
> > Hello,
> >
> >
> >
> > Why does my logger keep crashing? Can someone please help me with this.
> I have provided some system information below:
> >
> >
> >
> > I am running bro version 2.6.3
> >
> >
> >
> > Output of broctl status. The logger is crashed but the manager, 
> > proxy
> and workers are still running.
> >
> > broctl status
> >
> > Name         Type    Host             Status    Pid    Started
> >
> > logger       logger  localhost        crashed
> >
> > manager      manager localhost        running   17356  09 Sep 15:42:24
> >
> > proxy-1      proxy   localhost        running   17401  09 Sep 15:42:25
> >
> > worker-1-1   worker  localhost        running   17573  09 Sep 15:42:27
> >
> > worker-1-2   worker  localhost        running   17569  09 Sep 15:42:27
> >
> > worker-1-3   worker  localhost        running   17572  09 Sep 15:42:27
> >
> > worker-1-4   worker  localhost        running   17587  09 Sep 15:42:27
> >
> > worker-1-5   worker  localhost        running   17619  09 Sep 15:42:27
> >
> > worker-1-6   worker  localhost        running   17614  09 Sep 15:42:27
> >
> > worker-1-7   worker  localhost        running   17625  09 Sep 15:42:27
> >
> > worker-1-8   worker  localhost        running   17646  09 Sep 15:42:27
> >
> > worker-1-9   worker  localhost        running   17671  09 Sep 15:42:27
> >
> > worker-1-10  worker  localhost        running   17663  09 Sep 15:42:27
> >
> > worker-1-11  worker  localhost        running   17679  09 Sep 15:42:27
> >
> > worker-1-12  worker  localhost        running   17685  09 Sep 15:42:27
> >
> > worker-1-13  worker  localhost        running   17698  09 Sep 15:42:27
> >
> > worker-1-14  worker  localhost        running   17703  09 Sep 15:42:27
> >
> > worker-1-15  worker  localhost        running   17710  09 Sep 15:42:27
> >
> > worker-1-16  worker  localhost        running   17717  09 Sep 15:42:27
> >
> > worker-1-17  worker  localhost        running   17720  09 Sep 15:42:27
> >
> > worker-1-18  worker  localhost        running   17727  09 Sep 15:42:27
> >
> > worker-1-19  worker  localhost        running   17728  09 Sep 15:42:27
> >
> > worker-1-20  worker  localhost        running   17731  09 Sep 15:42:27
> >
> >
> >
> > Here?s my node.cfg settings
> >
> > [logger]
> >
> > type=logger
> >
> > host=localhost
> >
> >
> >
> > [manager]
> >
> > type=manager
> >
> > host=localhost
> >
> >
> >
> > [proxy-1]
> >
> > type=proxy
> >
> > host=localhost
> >
> >
> >
> > [worker-1]
> >
> > type=worker
> >
> > host=localhost
> >
> > interface=af_packet::ens2f0
> >
> > lb_method=custom
> >
> > #lb_method=pf_ring
> >
> > lb_procs=20
> >
> > pin_cpus=6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25
> >
> > af_packet_fanout_id=25
> >
> > af_packet_fanout_mode=AF_Packet::FANOUT_HASH
> >
> >
> >
> > Heres more information on my CPU. 32 CPUs, model name ? AMD, CPU max 
> > MHz is 2800.0000
> >
> > Architecture:          x86_64
> >
> > CPU op-mode(s):        32-bit, 64-bit
> >
> > Byte Order:            Little Endian
> >
> > CPU(s):                32
> >
> > On-line CPU(s) list:   0-31
> >
> > Thread(s) per core:    2
> >
> > Core(s) per socket:    8
> >
> > Socket(s):             2
> >
> > NUMA node(s):          4
> >
> > Vendor ID:             AuthenticAMD
> >
> > CPU family:            21
> >
> > Model:                 2
> >
> > Model name:            AMD Opteron(tm) Processor 6386 SE
> >
> > Stepping:              0
> >
> > CPU MHz:               1960.000
> >
> > CPU max MHz:           2800.0000
> >
> > CPU min MHz:           1400.0000
> >
> > BogoMIPS:              5585.93
> >
> > Virtualization:        AMD-V
> >
> > L1d cache:             16K
> >
> > L1i cache:             64K
> >
> > L2 cache:              2048K
> >
> > L3 cache:              6144K
> >
> > NUMA node0 CPU(s):     0,2,4,6,8,10,12,14
> >
> > NUMA node1 CPU(s):     16,18,20,22,24,26,28,30
> >
> > NUMA node2 CPU(s):     1,3,5,7,9,11,13,15
> >
> > NUMA node3 CPU(s):     17,19,21,23,25,27,29,31
> >
> >
> >
> > Would also like to know how I can reduce my packet loss. Below is 
> > the output of broctl netstats
> >
> > broctl netstats
> >
> > worker-1-1: 1568908277.861813 recvd=12248845422 dropped=5171188999
> > link=17420313882
> >
> > worker-1-2: 1568908298.313954 recvd=8636707266 dropped=971489
> > link=8637678939
> >
> > worker-1-3: 1568908278.425888 recvd=11684808853 dropped=5617381647
> > link=17302473791
> >
> > worker-1-4: 1568908285.731130 recvd=12567242226 dropped=4339688288
> > link=16907212802
> >
> > worker-1-5: 1568908298.363911 recvd=8620499351 dropped=24595149
> > link=8645095758
> >
> > worker-1-6: 1568908298.372892 recvd=8710565757 dropped=1731022
> > link=8712297432
> >
> > worker-1-7: 1568908298.266010 recvd=9065207444 dropped=53523232
> > link=9118737229
> >
> > worker-1-8: 1568908286.935607 recvd=11377790124 dropped=3680887247
> > link=15058934491
> >
> > worker-1-9: 1568908298.419657 recvd=8931903322 dropped=39696184
> > link=8971604219
> >
> > worker-1-10: 1568908298.478576 recvd=8842874030 dropped=2501252
> > link=8845376352
> >
> > worker-1-11: 1568908298.506649 recvd=8692769329 dropped=2253413
> > link=8695025626
> >
> > worker-1-12: 1568908298.520830 recvd=8749977028 dropped=2314733
> > link=8752293714
> >
> > worker-1-13: 1568908298.544573 recvd=9101243757 dropped=1779460
> > link=9103025399
> >
> > worker-1-14: 1568908291.370011 recvd=10876925726 dropped=775722632
> > link=11652810353
> >
> > worker-1-15: 1568908298.579721 recvd=8503097394 dropped=1420699
> > link=8504520066
> >
> > worker-1-16: 1568908298.594942 recvd=8515164266 dropped=1840977
> > link=8517006779
> >
> > worker-1-17: 1568908298.646966 recvd=10666567717 dropped=466489754
> > link=11133059283
> >
> > worker-1-18: 1568908298.671246 recvd=9023603573 dropped=2037607
> > link=9025642263
> >
> > worker-1-19: 1568908298.704675 recvd=8907784186 dropped=1164594
> > link=8908950238
> >
> > worker-1-20: 1568908298.718084 recvd=9140525444 dropped=2028593
> > link=9142555259
> >
> >
> >
> > Thanks,
> >
> > _______________________________________________
> > Zeek mailing list
> > zeek at zeek.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> >
> > _______________________________________________
> > Zeek mailing list
> > zeek at zeek.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191002/52eddc7
7/attachment.html 

------------------------------

_______________________________________________
Zeek mailing list
Zeek at zeek.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek


End of Zeek Digest, Vol 162, Issue 2
************************************
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 7171 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191003/ddf25ae0/attachment-0001.bin 

From seth at corelight.com  Thu Oct  3 06:21:26 2019
From: seth at corelight.com (Seth Hall)
Date: Thu, 03 Oct 2019 09:21:26 -0400
Subject: [Zeek] duplicated intel logs DNS::IN_REQUEST
In-Reply-To: <f14f812365414364b5cea609820b7fdd@SRVEX03.aizoon.local>
References: <f14f812365414364b5cea609820b7fdd@SRVEX03.aizoon.local>
Message-ID: <F20D1A77-B710-4C4F-9C3E-7E215B4F4622@corelight.com>



On 3 Oct 2019, at 8:35, Palumbo Mauro wrote:

> 1570105259.207335       CJZASAQTB2qgPSYw7       172.17.0.186    59553  
>  172.16.1.10     53      opencalphad.com Intel::DOMAIN   
> DNS::IN_REQUEST worker-1        Intel::DOMAIN   0       -       -      
>  -       -       85.0    -       -       -       -
> 1570105259.211927       CJZASAQTB2qgPSYw7       172.17.0.186    59553  
>  172.16.1.10     53      opencalphad.com Intel::DOMAIN   
> DNS::IN_REQUEST worker-1        Intel::DOMAIN   0       -       -      
>  -       -       85.0    -       -
>
> As you can see, some lines are identical, same uid, same worker, same 
> timestamp, etc...

Would it be possible to grab a pcap that recreates this behavior?  
Certainly not the correct behavior and it sounds like you've thought 
through the potential issues pretty thoroughly already and I agree with 
your thoughts.  We might be at the point of just needing the PCAP to see 
what's causing it.

   .Seth

--
Seth Hall * Corelight, Inc * www.corelight.com

From seth at corelight.com  Thu Oct  3 06:22:23 2019
From: seth at corelight.com (Seth Hall)
Date: Thu, 03 Oct 2019 09:22:23 -0400
Subject: [Zeek] Zeek and packages
In-Reply-To: <CAM1mTCpjKxn8-7oV5hZVZEVecZ3wHUgyKB1B+HsVJam5CfemTw@mail.gmail.com>
References: <d112fdafeae986a1c7a174521ed0235b@slave-tothe-box.net>
	<CAM1mTCpjKxn8-7oV5hZVZEVecZ3wHUgyKB1B+HsVJam5CfemTw@mail.gmail.com>
Message-ID: <D1E591F4-CCD6-460B-8ADC-4C6D63F7798D@corelight.com>



On 1 Oct 2019, at 13:40, Zeolla at GMail.com wrote:

> Now!  =)

+1 on that!  Actually, I don't think the community would complain if you 
automated your complaints. :)

   .Seth

--
Seth Hall * Corelight, Inc * www.corelight.com

From mauro.palumbo at aizoon.it  Thu Oct  3 07:14:06 2019
From: mauro.palumbo at aizoon.it (Palumbo Mauro)
Date: Thu, 3 Oct 2019 14:14:06 +0000
Subject: [Zeek] R:  duplicated intel logs DNS::IN_REQUEST
In-Reply-To: <F20D1A77-B710-4C4F-9C3E-7E215B4F4622@corelight.com>
References: <f14f812365414364b5cea609820b7fdd@SRVEX03.aizoon.local>
	<F20D1A77-B710-4C4F-9C3E-7E215B4F4622@corelight.com>
Message-ID: <1e9d664089f94131ba7dfdb888256b1a@SRVEX03.aizoon.local>

It turned out that there is an issue in our network and we are in fact getting duplicated dns packets on the span port...

So bro sees only one dns session in dns.log (and only one uid in conn.log), but the event  dns_request is raised more than once and hence we get multiple intel matches.

Thanks and sorry for the false alarm...

Mauro

-----Messaggio originale-----
Da: Seth Hall [mailto:seth at corelight.com] 
Inviato: gioved? 3 ottobre 2019 15:21
A: Palumbo Mauro <mauro.palumbo at aizoon.it>
Cc: zeek <zeek at zeek.org>
Oggetto: Re: [Zeek] duplicated intel logs DNS::IN_REQUEST



On 3 Oct 2019, at 8:35, Palumbo Mauro wrote:

> 1570105259.207335       CJZASAQTB2qgPSYw7       172.17.0.186    59553  
>  172.16.1.10     53      opencalphad.com Intel::DOMAIN   
> DNS::IN_REQUEST worker-1        Intel::DOMAIN   0       -       -      
>  -       -       85.0    -       -       -       -
> 1570105259.211927       CJZASAQTB2qgPSYw7       172.17.0.186    59553  
>  172.16.1.10     53      opencalphad.com Intel::DOMAIN   
> DNS::IN_REQUEST worker-1        Intel::DOMAIN   0       -       -      
>  -       -       85.0    -       -
>
> As you can see, some lines are identical, same uid, same worker, same 
> timestamp, etc...

Would it be possible to grab a pcap that recreates this behavior?  
Certainly not the correct behavior and it sounds like you've thought through the potential issues pretty thoroughly already and I agree with your thoughts.  We might be at the point of just needing the PCAP to see what's causing it.

   .Seth

--
Seth Hall * Corelight, Inc * www.corelight.com


From justin at corelight.com  Thu Oct  3 08:58:00 2019
From: justin at corelight.com (Justin Azoff)
Date: Thu, 3 Oct 2019 11:58:00 -0400
Subject: [Zeek] duplicated intel logs DNS::IN_REQUEST
In-Reply-To: <f14f812365414364b5cea609820b7fdd@SRVEX03.aizoon.local>
References: <f14f812365414364b5cea609820b7fdd@SRVEX03.aizoon.local>
Message-ID: <CAPfnCugkXtoU6XZ4nZCiGdG7nbygB0QkcXYYPGN4Q2ubUj+GcA@mail.gmail.com>

On Thu, Oct 3, 2019 at 8:38 AM Palumbo Mauro <mauro.palumbo at aizoon.it>
wrote:

> Hi everybody,
>
>
>
>   I am having an issue with the intel.log file, I am getting duplicated
> lines for the same dns request such as:
>
>
>
> #fields ts      uid     id.orig_h       id.orig_p       id.resp_h
> id.resp_p       seen.indicator  seen.indicator_type     seen.where
> seen.node       matched sources fuid    file_mime_type  file_desc
> cif.tags        cif.confidence  cif.source      cif.description
> cif.firstseen   cif.lastseen
>
> #types  time    string  addr    port    addr    port    string  enum
> enum    string  set[enum]       set[string]     string  string  string
> string  double  string  string  string  string
>
> 1570105259.197420       CP1BZx1QgzdPpfEyda      172.17.0.186    43283
> 172.16.1.10     53      opencalphad.com Intel::DOMAIN   DNS::IN_REQUEST
> worker-1        Intel::DOMAIN   0       -       -       -       -
> 85.0    -       -       -       -
>
> 1570105259.197420       CP1BZx1QgzdPpfEyda      172.17.0.186    43283
> 172.16.1.10     53      opencalphad.com Intel::DOMAIN   DNS::IN_REQUEST
> worker-1        Intel::DOMAIN   0       -       -       -       -
> 85.0    -       -       -       -
>
> 1570105259.207335       CJZASAQTB2qgPSYw7       172.17.0.186    59553
> 172.16.1.10     53      opencalphad.com Intel::DOMAIN   DNS::IN_REQUEST
> worker-1        Intel::DOMAIN   0       -       -       -       -
> 85.0    -       -       -       -
>
> 1570105259.211927       CJZASAQTB2qgPSYw7       172.17.0.186    59553
> 172.16.1.10     53      opencalphad.com Intel::DOMAIN   DNS::IN_REQUEST
> worker-1        Intel::DOMAIN   0       -       -       -       -
> 85.0    -       -
>
>
>
> As you can see, some lines are identical, same uid, same worker, same
> timestamp, etc...
>

The usual case for this is that you are tapping the same traffic twice.
 If you look up the CP1BZx1QgzdPpfEyda connection in the conn.log  and look
at orig_pkts and resp_pkts  you should see 1 and 1.   If you see 2,2 or 2,1
then you are seeing duplicate packets.

justin at mbp:/tmp/b$ cat dns.log |bro-cut  uid qtype_name query
Cu1Xq04w0nXaBiFiD A opencalphad.com
CJYuzY33KkZubxHXMc AAAA opencalphad.com
CdgXOb43ML2PJSv84a MX opencalphad.com
justin at mbp:/tmp/b$ cat conn.log |bro-cut uid orig_pkts resp_pkts |fgrep
Cu1Xq04w0nXaBiFiD
Cu1Xq04w0nXaBiFiD 1 1



-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191003/80a9689c/attachment.html 

From akgraner at corelight.com  Thu Oct  3 09:26:35 2019
From: akgraner at corelight.com (Amber Graner)
Date: Thu, 3 Oct 2019 12:26:35 -0400
Subject: [Zeek] ZeekWeek 2019 - Thank you to our Sponsors
Message-ID: <CAJhOzuph_nhmahGyAdfLuY72X2ebFbsZ6BOc6=FoqanKMeRK6Q@mail.gmail.com>

Hi all,

Join us in giving a shout out and many thank you's to our ZeekWeek 2019
sponsors. With their support ZeekWeek and other Zeek events would not be
possible.

Please share the blog post or retweet the Twitter post:

Blog post -
https://blog.zeek.org/2019/10/zeekweek-2019-thank-you-to-our-sponsors.html
Twitter Link - https://twitter.com/Zeekurity/status/1179786728158220288?s=20

Sponsors for this years event include:

*Exclusive Sponsorships*

Reception Sponsor - Bricata
Pre-Conference Training Sponsor - Humio
Lunch Sponsor - AlphaSoc

*40 GIG Sponsors*

Reservoir Labs
BluVector
BRIM
Gigamon
LookingGlass

*10 GIG Sponsors*

Amazon
Salesforce

*Host*

Corelight

If you'd like to meet with any of the sponsors, host, or Zeek Leadership
Team registration is open - http://bit.ly/zeekweek19_registration

With gratitude,
~Amber
-- 

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191003/e74d8a1f/attachment-0001.html 

From akgraner at corelight.com  Thu Oct  3 15:11:38 2019
From: akgraner at corelight.com (Amber Graner)
Date: Thu, 3 Oct 2019 18:11:38 -0400
Subject: [Zeek] Seeking Volunteers for ZeekWeek - Seattle, WA
Message-ID: <CAJhOzuoohpYw0KQeU8GvvQ63dW7MhiQKtjt7V8E8LjoyyNLXdg@mail.gmail.com>

Hi all,

Are you in the Seattle area?  Would you like to attend ZeekWeek?

We need to staff our registration desk for ZeekWeek starting on Wednesday,
Oct. 9 and ending on Friday, Oct.11.

If you volunteer you'll be given access to the other days event. For
example, if you staff the registration desk on Wednesday you can attend the
Thursday and Friday Sessions.  If you volunteer for the Thursday time slot
you can attend Wednesday and Friday.

Below are the details (parking is currently not complimentary/covered by
us, but I will look into getting the staff vouchers):

Hilton Embassy Suites Seattle 255 S King St, Seattle, WA 98104
*Everything will take place on the 6th floor for the entire day*

Wednesday, Oct. 9th (two people needed)
6:00am - 5:00pm

Thursday, Oct. 10th (just one person needed)
7:00am - 3:30pm

Friday, Oct. 11th (just one person needed)
7:30am - 1:30pm

Below is the link to the conference website, my company Corelight is just
hosting/paying for the event which is why I am scheduling the temp staff:
https://www.zeekweek.com/event/e0f22534-4348-4055-bdeb-c0496ac24050/websitePage:645d57e4-75eb-4769-b2c0-f201a0bfc6ce?5S%2CM3%2Ce0f22534-4348-4055-bdeb-c0496ac24050=

Please let me know if you have any questions.Thank you for your help.

~Amber

-- 

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191003/6ed3bbf1/attachment.html 

From akgraner at corelight.com  Thu Oct  3 17:01:35 2019
From: akgraner at corelight.com (Amber Graner)
Date: Thu, 3 Oct 2019 20:01:35 -0400
Subject: [Zeek] ZeekDays (Technical User Workshops) - 23 Oct 2019 - Atlanta,
	GA
Message-ID: <CAJhOzurw8-Zbg77sCAmg3tCC-YYuJsBAb1fDE1RmHRmQqeHRFQ@mail.gmail.com>

Are you in, near, or around Atlanta, GA?   Aren't  able to attend ZeekWeek this
year?


Are you interested in knowing more about Zeek (Formerly Bro)?


Does scripting with Zeek interest you?


What about Threat Hunting with Elastic + Zeek?


Check out our first in a series ZeekDays Events (Technical User Workshop)
that will be held on 23 Oct in Atlanta, GA.


The workshops are free, but REGISTRATION IS REQUIRED.


There will be a Happy Hour Social sponsored by Corelight, Inc following the
event.


More information can be found at:
https://www.meetup.com/Zeek-Days-Zeek-Bro-Technical-User-Workshop-Atlanta-GA/events/265369805/


If you're interested in sponsoring a Zeek event please let me know.


Thanks,

~Amber

-- 

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191003/47f77ca9/attachment.html 

From hinconszlc at gmail.com  Thu Oct  3 19:02:40 2019
From: hinconszlc at gmail.com (lc z)
Date: Fri, 4 Oct 2019 10:02:40 +0800
Subject: [Zeek] how can i config bro to let it only capture and analyze http
	packages?
Message-ID: <CADUtY13wBKGg3wW-4NQAX-PhTHd_Vio02Psa0MRt37zyzTXzxg@mail.gmail.com>

Does it have this function?I just want to only analyze http packages.And
Does it can reduce capture loss rate via analyzing less packages?  Thanks a
lot.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191004/f44af0b4/attachment.html 

From mauro.palumbo at aizoon.it  Fri Oct  4 01:08:19 2019
From: mauro.palumbo at aizoon.it (Palumbo Mauro)
Date: Fri, 4 Oct 2019 08:08:19 +0000
Subject: [Zeek] R:  duplicated intel logs DNS::IN_REQUEST
In-Reply-To: <CAPfnCugkXtoU6XZ4nZCiGdG7nbygB0QkcXYYPGN4Q2ubUj+GcA@mail.gmail.com>
References: <f14f812365414364b5cea609820b7fdd@SRVEX03.aizoon.local>
	<CAPfnCugkXtoU6XZ4nZCiGdG7nbygB0QkcXYYPGN4Q2ubUj+GcA@mail.gmail.com>
Message-ID: <3b12e2281f604c71b197ac218039692d@SRVEX03.aizoon.local>

Hi Justin,
   I am in fact seeing 2,2 or 2,0 as orig_pkts and resp_pkts. And I confirmed this with tcpdump. So I believe it is an issue with the network we are tapping as I see these duplicated packets only for dns.

Thnaks
Mauro

Da: Justin Azoff [mailto:justin at corelight.com]
Inviato: gioved? 3 ottobre 2019 17:58
A: Palumbo Mauro <mauro.palumbo at aizoon.it>
Cc: zeek <zeek at zeek.org>
Oggetto: Re: [Zeek] duplicated intel logs DNS::IN_REQUEST

On Thu, Oct 3, 2019 at 8:38 AM Palumbo Mauro <mauro.palumbo at aizoon.it<mailto:mauro.palumbo at aizoon.it>> wrote:
Hi everybody,

  I am having an issue with the intel.log file, I am getting duplicated lines for the same dns request such as:

#fields ts      uid     id.orig_h       id.orig_p       id.resp_h       id.resp_p       seen.indicator  seen.indicator_type     seen.where      seen.node       matched sources fuid    file_mime_type  file_desc       cif.tags        cif.confidence  cif.source      cif.description cif.firstseen   cif.lastseen
#types  time    string  addr    port    addr    port    string  enum    enum    string  set[enum]       set[string]     string  string  string  string  double  string  string  string  string
1570105259.197420       CP1BZx1QgzdPpfEyda      172.17.0.186    43283   172.16.1.10     53      opencalphad.com<http://opencalphad.com> Intel::DOMAIN   DNS::IN_REQUEST worker-1        Intel::DOMAIN   0       -       -       -       -       85.0    -       -       -       -
1570105259.197420       CP1BZx1QgzdPpfEyda      172.17.0.186    43283   172.16.1.10     53      opencalphad.com<http://opencalphad.com> Intel::DOMAIN   DNS::IN_REQUEST worker-1        Intel::DOMAIN   0       -       -       -       -       85.0    -       -       -       -
1570105259.207335       CJZASAQTB2qgPSYw7       172.17.0.186    59553   172.16.1.10     53      opencalphad.com<http://opencalphad.com> Intel::DOMAIN   DNS::IN_REQUEST worker-1        Intel::DOMAIN   0       -       -       -       -       85.0    -       -       -       -
1570105259.211927       CJZASAQTB2qgPSYw7       172.17.0.186    59553   172.16.1.10     53      opencalphad.com<http://opencalphad.com> Intel::DOMAIN   DNS::IN_REQUEST worker-1        Intel::DOMAIN   0       -       -       -       -       85.0    -       -

As you can see, some lines are identical, same uid, same worker, same timestamp, etc...

The usual case for this is that you are tapping the same traffic twice.   If you look up the CP1BZx1QgzdPpfEyda connection in the conn.log  and look at orig_pkts and resp_pkts  you should see 1 and 1.   If you see 2,2 or 2,1 then you are seeing duplicate packets.

justin at mbp:/tmp/b$ cat dns.log |bro-cut  uid qtype_name query
Cu1Xq04w0nXaBiFiD A opencalphad.com<http://opencalphad.com>
CJYuzY33KkZubxHXMc AAAA opencalphad.com<http://opencalphad.com>
CdgXOb43ML2PJSv84a MX opencalphad.com<http://opencalphad.com>
justin at mbp:/tmp/b$ cat conn.log |bro-cut uid orig_pkts resp_pkts |fgrep Cu1Xq04w0nXaBiFiD
Cu1Xq04w0nXaBiFiD 1 1



--
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191004/cfd18063/attachment-0001.html 

From akgraner at corelight.com  Fri Oct  4 09:43:40 2019
From: akgraner at corelight.com (Amber Graner)
Date: Fri, 4 Oct 2019 12:43:40 -0400
Subject: [Zeek] LT Meeting today
Message-ID: <CAJhOzurFhGrP-Stv9=SyoG6ACeOcVrhdQbMC_LcPL7fE73Szpw@mail.gmail.com>

We aren't having this right?

It's still on my calendar, so I wanted to make sure.

Thanks,
~Amber

-- 

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191004/ed21aa7f/attachment.html 

From akgraner at corelight.com  Fri Oct  4 10:09:32 2019
From: akgraner at corelight.com (Amber Graner)
Date: Fri, 4 Oct 2019 13:09:32 -0400
Subject: [Zeek] LT Meeting today
In-Reply-To: <CAJhOzurFhGrP-Stv9=SyoG6ACeOcVrhdQbMC_LcPL7fE73Szpw@mail.gmail.com>
References: <CAJhOzurFhGrP-Stv9=SyoG6ACeOcVrhdQbMC_LcPL7fE73Szpw@mail.gmail.com>
Message-ID: <CAJhOzur9Wr6YP3Wq7XRqGxS+0LBY0=dcZXb0xdpu3mzo=vzZSQ@mail.gmail.com>

Oops. Wrong list.

Apologies for the noise.

Hope to see many of you at ZeekWeek in Seattle next week!

Thanks,
-Amber


On Fri, Oct 4, 2019, 12:43 PM Amber Graner <akgraner at corelight.com> wrote:

> We aren't having this right?
>
> It's still on my calendar, so I wanted to make sure.
>
> Thanks,
> ~Amber
>
> --
>
> *Amber Graner*
> Director of Community
> Corelight, Inc
>
> 828.582.9469
>
>
>  * Ask me about how you can participate in the Zeek (formerly Bro)
> community.
>  * Remember - ZEEK AND YOU SHALL FIND!!
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191004/d44ec305/attachment.html 

From justin at corelight.com  Fri Oct  4 10:19:08 2019
From: justin at corelight.com (Justin Azoff)
Date: Fri, 4 Oct 2019 13:19:08 -0400
Subject: [Zeek] duplicated intel logs DNS::IN_REQUEST
In-Reply-To: <3b12e2281f604c71b197ac218039692d@SRVEX03.aizoon.local>
References: <f14f812365414364b5cea609820b7fdd@SRVEX03.aizoon.local>
	<CAPfnCugkXtoU6XZ4nZCiGdG7nbygB0QkcXYYPGN4Q2ubUj+GcA@mail.gmail.com>
	<3b12e2281f604c71b197ac218039692d@SRVEX03.aizoon.local>
Message-ID: <CAPfnCugzEbPtLqsLgQartOcvi6Z2OTVSgNerTyMs0KMNW7wuQQ@mail.gmail.com>

On Fri, Oct 4, 2019 at 4:08 AM Palumbo Mauro <mauro.palumbo at aizoon.it>
wrote:

> Hi Justin,
>
>    I am in fact seeing 2,2 or 2,0 as orig_pkts and resp_pkts. And I
> confirmed this with tcpdump. So I believe it is an issue with the network
> we are tapping as I see these duplicated packets only for dns.
>

Possibly, but you may have duplicates everywhere.  The tcp reassembler can
use the sequence numbers to avoid analyzing the same traffic twice, but UDP
doesn't have anything like that.  DNS is just the place you tend to notice
the duplicate traffic the most.


-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191004/2fa86e11/attachment.html 

From eva.c.e.seipel at gmail.com  Mon Oct  7 01:11:16 2019
From: eva.c.e.seipel at gmail.com (Eva Seipel)
Date: Mon, 7 Oct 2019 10:11:16 +0200
Subject: [Zeek] Capture Loss using pcap file
Message-ID: <CAPoSYBedorGrUOyuAVOiDB98=JTK8VbbeckGa_kx_5e9HzaHKA@mail.gmail.com>

Dear all,

when I run Zeek/Bro (Version 2.6.3) against a rather large pcap file of
about 8GB (one from the CICIDS2017 dataset) I get values in between 17 and
65% in capture_loss.log. What could be the reason for that? I am pretty new
to the topic and couldn't find anything about that via search. Is it a
problem with Zeek like to much data or was the loss already in the pcap and
has nothing to do with Zeek?

Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191007/078f4181/attachment.html 

From akgraner at corelight.com  Wed Oct  9 11:45:17 2019
From: akgraner at corelight.com (Amber Graner)
Date: Wed, 9 Oct 2019 14:45:17 -0400
Subject: [Zeek] ZeekDays Workshop in Atlanta on 23 OCT
Message-ID: <CAJhOzuoXsamgoQc_kckO9fPuuLcTVE1irjKr_ARx_iQs14Uy6w@mail.gmail.com>

Hi all,

Maybe you or someone you know wasn't able to make it out to Seattle this
week. If so, then check out the ZeekDays Workshop that will take place in
Atlanta, Georgia on 23 Oct 2019 from 9 - 5pm.

This workshop will include:
* An Intro to Zeek (tool)
* An Intro to Zeek Scripting (language)
* Threat Hunting with Elastic +Zeek (data)

Time
9-10am - Registration
10-10:15am - Welcome - Amber Graner, Zeek Director of Community, Corelight
10:15-11am - Intro to Zeek - Seth Hall, Zeek Leadership Team Member,
Co-Founder and Chief Evangelist, Corelight
11am-11:15am - Break
11:15am-12pm - Intro to Zeek Scripting - Seth Hall
12-1pm - Lunch (provided)
1-2pm - Threat Hunting with Elastic+Zeek - Alex Kirk
2-2:15pm - Break
2-3pm - Session 4 - Threat Hunting with Elastic+Zeek (cont) - Alex Kirk
3-3:30pm - Q&A/Wrap up - Seth/Alex/Amber and others TBD
3:30-5pm - Happy Hour sponsored by Corelight - onsite

This event is free to the public, but registration is required.

You can sign up at:
https://www.meetup.com/Zeek-Days-Zeek-Bro-Technical-User-Workshop-Atlanta-GA/events/265369805/

Please let me know if you have any questions.

Thanks,
~Amber

-- 

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191009/ab094df7/attachment.html 

From sethdgrover at gmail.com  Wed Oct  9 14:18:52 2019
From: sethdgrover at gmail.com (Seth Grover)
Date: Wed, 9 Oct 2019 15:18:52 -0600
Subject: [Zeek] error message "&optional is not valid for global variables"
Message-ID: <CAD13zvSxo1y1oGpvG6dPw8gWE8QTeaKhYsP_R2x=hg0r0vAUZA@mail.gmail.com>

Greetings! I am migrating some scripts from 2.6.4 to 3.0.0. The scripts
reside here in github:

https://github.com/idaholab/Malcolm/tree/master/moloch/zeek

These worked fine in 2.6.4, but in 3.0.0 running zeek fails with something
like this:

/opt/zeek/share/zeek/site/./extractor_params.zeek, line 936: &optional is
not valid for global variables (&optional, &redef, &default=dat, &optional)

when running the command:

zeek -r whatever.pcap local extractor.zeek
extractor_override.interesting.zeek

It appears the problem is this: in extractor_params.zeek I have a global
table that looks like this:

  const extractor_mime_to_ext_map : table[string] of string = {
    ["application/acad"]= "dwg",
    ["application/andrew-inset"]= "ez",
...
    ["x-world/x-vrml"]= "wrl"
  } &default="bin" &redef;

This is the default. In extractor_override.interesting.zeek I have a redef
table in the export section that looks like this:

  redef extractor_mime_to_ext_map : table[string] of string = {
    ["application/binary"]= "bin",
...
  } &default="dat";

I found this issue (https://github.com/zeek/zeek/issues/157) and this
commit (
https://github.com/zeek/zeek/commit/db79041b1924e95d0bbde81acfbfb8d8ba1814b5)
with the comment "This disallows &default for global values that are not
tables, and &optional for all globals." However, in my case aren't these
*both* tables?

In the end, I don't *think* it's a big deal. If I remove the &default="dat"
from the extractor_override.interesting.zeek table it does run okay. But I
would like to understand what is happening here. Especially, what will
happen if the table is redef'ed and the lookup value is not found? I don't
think it will use the original global table, right? It seems to me like I
should be able to have defaults on both since they are, in fact, tables.

Thanks,

-SG
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191009/c3a1365f/attachment.html 

From vern at corelight.com  Thu Oct 10 13:00:42 2019
From: vern at corelight.com (Vern Paxson)
Date: Thu, 10 Oct 2019 13:00:42 -0700
Subject: [Zeek] Capture Loss using pcap file
In-Reply-To: <CAPoSYBedorGrUOyuAVOiDB98=JTK8VbbeckGa_kx_5e9HzaHKA@mail.gmail.com>
	(Mon, 07 Oct 2019 10:11:16 +0200).
Message-ID: <20191010200042.5F4B42C4021@rock.ICSI.Berkeley.EDU>

Hi Eva,

> ... Is it a
> problem with Zeek like to much data or was the loss already in the pcap and
> has nothing to do with Zeek?

The loss was already in the pcap.  When running on pcaps, Zeek does not
drop any packets present in the pcap.

		Vern

From stu.h at live.com  Thu Oct 10 14:07:48 2019
From: stu.h at live.com (Stuart H)
Date: Thu, 10 Oct 2019 21:07:48 +0000
Subject: [Zeek] Communication channels
In-Reply-To: <CAJhOzupbWXdfUQ1179smjW6xxrZVkF2_fBUidn0A8=Sv4vzzKA@mail.gmail.com>
References: <CALiK309iTPp9GjM-+nh63pn3eX8F7BDZn64sLAtRGao64L_5Dg@mail.gmail.com>
	<CAEZw2bz9byw287vkmw6hCny8cFTx10_GTFi9EtKu0Hy_6YKA0Q@mail.gmail.com>
	<CAG=_CgBi9As+rY=dwDXw2NORezb4bGtYLGkBjwCqEMYWgBp-iw@mail.gmail.com>
	<CALiK3086v6UKskGrDr=d6sLMN4BBrm2PVVA9M-_0CkfrO=3K5A@mail.gmail.com>
	<CAJhOzupbWXdfUQ1179smjW6xxrZVkF2_fBUidn0A8=Sv4vzzKA@mail.gmail.com>
Message-ID: <F579C7B8-13BA-4D88-94C1-245EB4A53D06@live.com>

After Amber?s talk today at ZeekWeek I thought I?d pitch in some ideas to help build the community.

I think there are a few types of interaction the community has, there may be others:

  *   Asking questions
  *   Reporting bugs
  *   Announcing things

Asking questions - We could use some form of group instant messenger or more of a forum style platform. Group instant messengers include Slack, Discord, Rocket chat, Discourse etc. The key thing is to make barrier to entry low, while I use IRC it?s not exactly user friendly. Forum style platforms include Stackexchange, Discourse etc. ? I see these as replacements for the Mailing list which may find difficult to use, search etc.

Reporting bug ? I guess we should be using GitHub to report the actual bug but normally you need troubleshoot by asking questions or chatting.

Announcing things ? Perhaps some sort of distribution list that you subscribe to? Or perhaps just pinning topics on the forum platform?

From what I?ve looked at so far Discourse looks pretty good and for open source projects you can apply for free hosting: https://blog.discourse.org/2018/11/free-hosting-for-open-source-v2/

What do you think? Any more thoughts?

Cheers,
Stu

From: <zeek-bounces at zeek.org> on behalf of Amber Graner <akgraner at corelight.com>
Date: Monday, 3 June 2019 at 14:35
To: Woot4moo <tscheponik at gmail.com>
Cc: zeek <zeek at zeek.org>
Subject: Re: [Zeek] Communication channels

I?ll research some options and ask for the LT to review at the next meeting.

Please continue to add your your thoughts.

Thanks,
~Amber

On Mon, Jun 3, 2019 at 7:34 AM Woot4moo <tscheponik at gmail.com<mailto:tscheponik at gmail.com>> wrote:
Agree. While giphy integration is a good time killer, I am far more interested in modern amenities such as threads and history. I presume Matrix would get us there or could be close with some pull requests.

On Mon, Jun 3, 2019 at 8:24 AM Mark Gardner <mkg at vt.edu<mailto:mkg at vt.edu>> wrote:
On Sun, Jun 2, 2019 at 11:45 AM anthony kasza <anthony.kasza at gmail.com<mailto:anthony.kasza at gmail.com>> wrote:
I don't use the IRC channel but I would lurk in a Zeek Slack channel.

Please choose an open standard rather than a walled garden. Someone above suggested Matrix as a possibility.
_______________________________________________
Zeek mailing list
zeek at zeek.org<mailto:zeek at zeek.org>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
--
Amber Graner
Director of Community
Corelight, Inc

828.582.9469
 [Image removed by sender.]  [Image removed by sender.]

 * Ask me about how you can participate in the Zeek (formerly Bro) community.
 * Remember - ZEEK AND YOU SHALL FIND!!


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191010/b6f0de9a/attachment.html 

From dopheide at gmail.com  Thu Oct 10 15:02:29 2019
From: dopheide at gmail.com (Mike Dopheide)
Date: Thu, 10 Oct 2019 17:02:29 -0500
Subject: [Zeek] Communication channels
In-Reply-To: <F579C7B8-13BA-4D88-94C1-245EB4A53D06@live.com>
References: <CALiK309iTPp9GjM-+nh63pn3eX8F7BDZn64sLAtRGao64L_5Dg@mail.gmail.com>
	<CAEZw2bz9byw287vkmw6hCny8cFTx10_GTFi9EtKu0Hy_6YKA0Q@mail.gmail.com>
	<CAG=_CgBi9As+rY=dwDXw2NORezb4bGtYLGkBjwCqEMYWgBp-iw@mail.gmail.com>
	<CALiK3086v6UKskGrDr=d6sLMN4BBrm2PVVA9M-_0CkfrO=3K5A@mail.gmail.com>
	<CAJhOzupbWXdfUQ1179smjW6xxrZVkF2_fBUidn0A8=Sv4vzzKA@mail.gmail.com>
	<F579C7B8-13BA-4D88-94C1-245EB4A53D06@live.com>
Message-ID: <CAPy2kFaDhcD=+saLRFVb67uwzG0kBO615EK_ctEq=-7ZU9PvJg@mail.gmail.com>

There are feelings within the community that a Slack channel for the
community should be 'owned/run/administered' outside of Corelight.

-Dop

On Thu, Oct 10, 2019 at 4:09 PM Stuart H <stu.h at live.com> wrote:

> After Amber?s talk today at ZeekWeek I thought I?d pitch in some ideas to
> help build the community.
>
>
>
> I think there are a few types of interaction the community has, there may
> be others:
>
>    - Asking questions
>    - Reporting bugs
>    - Announcing things
>
>
>
> Asking questions - We could use some form of group instant messenger or
> more of a forum style platform. Group instant messengers include Slack,
> Discord, Rocket chat, Discourse etc. The key thing is to make barrier to
> entry low, while I use IRC it?s not exactly user friendly. Forum style
> platforms include Stackexchange, Discourse etc. ? I see these as
> replacements for the Mailing list which may find difficult to use, search
> etc.
>
>
>
> Reporting bug ? I guess we should be using GitHub to report the actual bug
> but normally you need troubleshoot by asking questions or chatting.
>
>
>
> Announcing things ? Perhaps some sort of distribution list that you
> subscribe to? Or perhaps just pinning topics on the forum platform?
>
>
>
> From what I?ve looked at so far Discourse looks pretty good and for open
> source projects you can apply for free hosting:
> https://blog.discourse.org/2018/11/free-hosting-for-open-source-v2/
>
>
>
> What do you think? Any more thoughts?
>
>
>
> Cheers,
>
> Stu
>
>
>
> *From: *<zeek-bounces at zeek.org> on behalf of Amber Graner <
> akgraner at corelight.com>
> *Date: *Monday, 3 June 2019 at 14:35
> *To: *Woot4moo <tscheponik at gmail.com>
> *Cc: *zeek <zeek at zeek.org>
> *Subject: *Re: [Zeek] Communication channels
>
>
>
> I?ll research some options and ask for the LT to review at the next
> meeting.
>
>
>
> Please continue to add your your thoughts.
>
>
>
> Thanks,
>
> ~Amber
>
>
>
> On Mon, Jun 3, 2019 at 7:34 AM Woot4moo <tscheponik at gmail.com> wrote:
>
> Agree. While giphy integration is a good time killer, I am far more
> interested in modern amenities such as threads and history. I presume
> Matrix would get us there or could be close with some pull requests.
>
>
>
> On Mon, Jun 3, 2019 at 8:24 AM Mark Gardner <mkg at vt.edu> wrote:
>
> On Sun, Jun 2, 2019 at 11:45 AM anthony kasza <anthony.kasza at gmail.com>
> wrote:
>
> I don't use the IRC channel but I would lurk in a Zeek Slack channel.
>
>
>
> Please choose an open standard rather than a walled garden. Someone above
> suggested Matrix as a possibility.
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
> --
>
> *Amber Graner*
>
> Director of Community
>
> Corelight, Inc
>
>
>
> 828.582.9469
>
>  [image: Image removed by sender.] [image: Image removed by sender.]
>
>
>
>  * Ask me about how you can participate in the Zeek (formerly Bro)
> community.
>
>  * Remember - ZEEK AND YOU SHALL FIND!!
>
>
>
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191010/171cbb94/attachment-0001.html 

From akgraner at corelight.com  Thu Oct 10 15:24:41 2019
From: akgraner at corelight.com (Amber Graner)
Date: Thu, 10 Oct 2019 15:24:41 -0700
Subject: [Zeek] Communication channels
In-Reply-To: <CAPy2kFaDhcD=+saLRFVb67uwzG0kBO615EK_ctEq=-7ZU9PvJg@mail.gmail.com>
References: <CALiK309iTPp9GjM-+nh63pn3eX8F7BDZn64sLAtRGao64L_5Dg@mail.gmail.com>
	<CAEZw2bz9byw287vkmw6hCny8cFTx10_GTFi9EtKu0Hy_6YKA0Q@mail.gmail.com>
	<CAG=_CgBi9As+rY=dwDXw2NORezb4bGtYLGkBjwCqEMYWgBp-iw@mail.gmail.com>
	<CALiK3086v6UKskGrDr=d6sLMN4BBrm2PVVA9M-_0CkfrO=3K5A@mail.gmail.com>
	<CAJhOzupbWXdfUQ1179smjW6xxrZVkF2_fBUidn0A8=Sv4vzzKA@mail.gmail.com>
	<F579C7B8-13BA-4D88-94C1-245EB4A53D06@live.com>
	<CAPy2kFaDhcD=+saLRFVb67uwzG0kBO615EK_ctEq=-7ZU9PvJg@mail.gmail.com>
Message-ID: <CAJhOzur8vOwyFpCHdP9nAjYJUOgHNCWARjehLaHjqdV=N=jj4A@mail.gmail.com>

That?s certainly a possibility but for an official Zeek tool the Zeek
Leadership Team would need to make that decision.

Any new tooling is discussed with Zeek Leadership team. Corelight doesn?t
make those  decisions. I hope I didn?t lead anyone to think that in my talk
today.

The Zeek LT and I just need to be aware of things that bear the official
Zeek ?title? if you will.

We?d have to figure out who would have oversite and management
responsibilities and that?s something I?ll bring up with the LT when we
discuss this.

Thanks,
~Amber

On Thu, Oct 10, 2019 at 3:02 PM Mike Dopheide <dopheide at gmail.com> wrote:

> There are feelings within the community that a Slack channel for the
> community should be 'owned/run/administered' outside of Corelight.
>
> -Dop
>
> On Thu, Oct 10, 2019 at 4:09 PM Stuart H <stu.h at live.com> wrote:
>
>> After Amber?s talk today at ZeekWeek I thought I?d pitch in some ideas to
>> help build the community.
>>
>>
>>
>> I think there are a few types of interaction the community has, there may
>> be others:
>>
>>    - Asking questions
>>    - Reporting bugs
>>    - Announcing things
>>
>>
>>
>> Asking questions - We could use some form of group instant messenger or
>> more of a forum style platform. Group instant messengers include Slack,
>> Discord, Rocket chat, Discourse etc. The key thing is to make barrier to
>> entry low, while I use IRC it?s not exactly user friendly. Forum style
>> platforms include Stackexchange, Discourse etc. ? I see these as
>> replacements for the Mailing list which may find difficult to use, search
>> etc.
>>
>>
>>
>> Reporting bug ? I guess we should be using GitHub to report the actual
>> bug but normally you need troubleshoot by asking questions or chatting.
>>
>>
>>
>> Announcing things ? Perhaps some sort of distribution list that you
>> subscribe to? Or perhaps just pinning topics on the forum platform?
>>
>>
>>
>> From what I?ve looked at so far Discourse looks pretty good and for open
>> source projects you can apply for free hosting:
>> https://blog.discourse.org/2018/11/free-hosting-for-open-source-v2/
>>
>>
>>
>> What do you think? Any more thoughts?
>>
>>
>>
>> Cheers,
>>
>> Stu
>>
>>
>>
>> *From: *<zeek-bounces at zeek.org> on behalf of Amber Graner <
>> akgraner at corelight.com>
>> *Date: *Monday, 3 June 2019 at 14:35
>> *To: *Woot4moo <tscheponik at gmail.com>
>> *Cc: *zeek <zeek at zeek.org>
>> *Subject: *Re: [Zeek] Communication channels
>>
>>
>>
>> I?ll research some options and ask for the LT to review at the next
>> meeting.
>>
>>
>>
>> Please continue to add your your thoughts.
>>
>>
>>
>> Thanks,
>>
>> ~Amber
>>
>>
>>
>> On Mon, Jun 3, 2019 at 7:34 AM Woot4moo <tscheponik at gmail.com> wrote:
>>
>> Agree. While giphy integration is a good time killer, I am far more
>> interested in modern amenities such as threads and history. I presume
>> Matrix would get us there or could be close with some pull requests.
>>
>>
>>
>> On Mon, Jun 3, 2019 at 8:24 AM Mark Gardner <mkg at vt.edu> wrote:
>>
>> On Sun, Jun 2, 2019 at 11:45 AM anthony kasza <anthony.kasza at gmail.com>
>> wrote:
>>
>> I don't use the IRC channel but I would lurk in a Zeek Slack channel.
>>
>>
>>
>> Please choose an open standard rather than a walled garden. Someone above
>> suggested Matrix as a possibility.
>>
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>
>> --
>>
>> *Amber Graner*
>>
>> Director of Community
>>
>> Corelight, Inc
>>
>>
>>
>> 828.582.9469
>>
>>  [image: Image removed by sender.] [image: Image removed by sender.]
>>
>>
>>
>>  * Ask me about how you can participate in the Zeek (formerly Bro)
>> community.
>>
>>  * Remember - ZEEK AND YOU SHALL FIND!!
>>
>>
>>
>>
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
> --

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191010/4e5982ea/attachment.html 

From jsiwek at corelight.com  Fri Oct 11 19:07:57 2019
From: jsiwek at corelight.com (Jon Siwek)
Date: Fri, 11 Oct 2019 19:07:57 -0700
Subject: [Zeek] error message "&optional is not valid for global
	variables"
In-Reply-To: <CAD13zvSxo1y1oGpvG6dPw8gWE8QTeaKhYsP_R2x=hg0r0vAUZA@mail.gmail.com>
References: <CAD13zvSxo1y1oGpvG6dPw8gWE8QTeaKhYsP_R2x=hg0r0vAUZA@mail.gmail.com>
Message-ID: <CAMzgZ0K-+nay_wJ1m7ytFpYpubu+qqw9pZ=X5oRsV4R3y=-aEA@mail.gmail.com>

On Wed, Oct 9, 2019 at 2:21 PM Seth Grover <sethdgrover at gmail.com> wrote:

> https://github.com/idaholab/Malcolm/tree/master/moloch/zeek
>
> These worked fine in 2.6.4, but in 3.0.0 running zeek fails with something like this:
>
> /opt/zeek/share/zeek/site/./extractor_params.zeek, line 936: &optional is not valid for global variables (&optional, &redef, &default=dat, &optional)

Looks like a bug/regression, thanks for reporting it.  I've proposed a
patch that would eventually make it into a 3.0.1 release:

https://github.com/zeek/zeek/pull/632

- Jon

From akgraner at corelight.com  Mon Oct 14 18:41:54 2019
From: akgraner at corelight.com (Amber Graner)
Date: Mon, 14 Oct 2019 20:41:54 -0500
Subject: [Zeek] Official Zeek Community Slack Channel Poll
Message-ID: <CAJhOzupLoFGh7RYR_kOCcEHaqBVE5S7OX1sf1BRf=yZ6TQ5QFA@mail.gmail.com>

Hi all,

There was a question on the mailing list earlier this year and again at
ZeekWeek19 about plans to create an Official Zeek Community Slack Channel.

Below is a 4 question 1 minute survey, please take a moment to respond.  We
will leave the survey up until 21 Oct 2019 5pm PT.

https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll

We will publish the results to the mailing list on 25 October 2019.

<https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll>
Please let me know if you have any questions.

Thanks in advance for taking the time to answer these questions.

with gratitude,
~Amber

-- 

*Amber Graner*
Zeek, Director of Community
Corelight, Inc

828.582.9469



 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191014/bf3bb40a/attachment.html 

From jan.grashoefer at gmail.com  Tue Oct 15 01:25:00 2019
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Tue, 15 Oct 2019 10:25:00 +0200
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <CAJhOzupLoFGh7RYR_kOCcEHaqBVE5S7OX1sf1BRf=yZ6TQ5QFA@mail.gmail.com>
References: <CAJhOzupLoFGh7RYR_kOCcEHaqBVE5S7OX1sf1BRf=yZ6TQ5QFA@mail.gmail.com>
Message-ID: <63e702a9-e218-412f-1d74-bf1ec2cc09b9@gmail.com>

I am sorry to open the can of worms again but is Slack already set as 
the platform? Personally, I am in favor of a more open and decentralized 
approach like matrix [1], which would even allow to bridge the old IRC 
channel.

Jan

[1] https://en.wikipedia.org/wiki/Matrix_(protocol)

On 15/10/2019 03:41, Amber Graner wrote:
> Hi all,
> 
> There was a question on the mailing list earlier this year and again at
> ZeekWeek19 about plans to create an Official Zeek Community Slack Channel.
> 
> Below is a 4 question 1 minute survey, please take a moment to respond.  We
> will leave the survey up until 21 Oct 2019 5pm PT.
> 
> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
> 
> We will publish the results to the mailing list on 25 October 2019.
> 
> <https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll>
> Please let me know if you have any questions.
> 
> Thanks in advance for taking the time to answer these questions.
> 
> with gratitude,
> ~Amber
> 
> 
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> 

From michalpurzynski1 at gmail.com  Tue Oct 15 01:52:20 2019
From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=)
Date: Tue, 15 Oct 2019 01:52:20 -0700
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <63e702a9-e218-412f-1d74-bf1ec2cc09b9@gmail.com>
References: <CAJhOzupLoFGh7RYR_kOCcEHaqBVE5S7OX1sf1BRf=yZ6TQ5QFA@mail.gmail.com>
	<63e702a9-e218-412f-1d74-bf1ec2cc09b9@gmail.com>
Message-ID: <CAJ6bFK3Mw1Ajsfjjxr+cAE0NUpedPY_j9+uEcrG9E42mYm5wjA@mail.gmail.com>

My private 2c here, not MoCo.

I strongly oppose having a closed platform chosen as a communication medium.

Slack has a long story of being hostile to projects as soon as there is an
opportunity to earn 0.25 cents.

Slack lied about the IRC compatibility, making sure to first close projects
within the slack world, promising IRC bridge would stay, and then silently
murdering it.
I'm pretty sure it was planned for the beginning, to lure projects that
would not move to slack otherwise.

Slack is notoriously behaving in a counter-inclusive manner, asking people
to switch to a certain browser, completely ignoring user's choice.

Slack user interface is so bad (on any browser) that it requires a
specialized client or it will drain your battery in no time.
I refuse installing a 3rd party software or a client from a company with a
long history of security ignorance.

Slack ignores Internet standards and bugs in web compatibility.

The list could go on, but I am strongly against closed solutions as they
give away the power over the project to a single hostile company, for them
to do whatever they want in the future.

Michal Purzynski, speaking privately, aka not MoCo. Feel free to quote me.

On Tue, Oct 15, 2019 at 1:27 AM Jan Grash?fer <jan.grashoefer at gmail.com>
wrote:

> I am sorry to open the can of worms again but is Slack already set as
> the platform? Personally, I am in favor of a more open and decentralized
> approach like matrix [1], which would even allow to bridge the old IRC
> channel.
>
> Jan
>
> [1] https://en.wikipedia.org/wiki/Matrix_(protocol)
>
> On 15/10/2019 03:41, Amber Graner wrote:
> > Hi all,
> >
> > There was a question on the mailing list earlier this year and again at
> > ZeekWeek19 about plans to create an Official Zeek Community Slack
> Channel.
> >
> > Below is a 4 question 1 minute survey, please take a moment to respond.
> We
> > will leave the survey up until 21 Oct 2019 5pm PT.
> >
> >
> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
> >
> > We will publish the results to the mailing list on 25 October 2019.
> >
> > <
> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll>
> > Please let me know if you have any questions.
> >
> > Thanks in advance for taking the time to answer these questions.
> >
> > with gratitude,
> > ~Amber
> >
> >
> > _______________________________________________
> > Zeek mailing list
> > zeek at zeek.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> >
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191015/c9b3a1e4/attachment.html 

From akgraner at corelight.com  Tue Oct 15 03:29:39 2019
From: akgraner at corelight.com (Amber Graner)
Date: Tue, 15 Oct 2019 05:29:39 -0500
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <CAJ6bFK3Mw1Ajsfjjxr+cAE0NUpedPY_j9+uEcrG9E42mYm5wjA@mail.gmail.com>
References: <CAJhOzupLoFGh7RYR_kOCcEHaqBVE5S7OX1sf1BRf=yZ6TQ5QFA@mail.gmail.com>
	<63e702a9-e218-412f-1d74-bf1ec2cc09b9@gmail.com>
	<CAJ6bFK3Mw1Ajsfjjxr+cAE0NUpedPY_j9+uEcrG9E42mYm5wjA@mail.gmail.com>
Message-ID: <CAJhOzuq1X4Jm-FTwK8_knku5wO5WD1_Mf1N+_KYLthcZs0JAHQ@mail.gmail.com>

Jan, thank you for that suggestion, no need to apologize. And no, slack has
not been settled on as the defacto choice. However, I wanted to
followup with this topic since it has been brought up twice now by the
community.

Michal, I understand the concern not only with Slack as tool, but also
using closed source tools for an open source project. In addition to
Matrix, which has been suggested by Jan, are there any other alternatives
to Slack or IRC that you would recommend?

I think the important thing to note, and I believe what the poll will
confirm, is that at the moment very few in the community are using IRC and
many in the community have expressed that they would participate more if
there was an alternative to IRC. I think most people are familiar with
Slack because it's a tool they use in their professional environments and
something they are already familiar with that fits into their current
workflow.  All of which would lower the barrier to entry and promote more
participation. However, let's look into other options as well now that it's
been brought up.

I'll leave the survey up open as planned, but will prepare to present to
the community and to the Zeek LT a comparison of all alternatives to IRC.

What other suggestions does anyone have in addition to the following:

 - IRC
 - Slack
 - Matrix
 - Discourse (chat)

If you have any other suggestions, please respond here or to me personally
before end of day on 18 Oct 2019, so that I can prepare the presentation to
you, the community and to the Zeek LT.

This discussion is great!

Thanks in advance for the suggestions and feedback.

With gratitude,
~Amber







On Tue, Oct 15, 2019 at 3:54 AM Micha? Purzy?ski <michalpurzynski1 at gmail.com>
wrote:

> My private 2c here, not MoCo.
>
> I strongly oppose having a closed platform chosen as a communication
> medium.
>
> Slack has a long story of being hostile to projects as soon as there is an
> opportunity to earn 0.25 cents.
>
> Slack lied about the IRC compatibility, making sure to first close
> projects within the slack world, promising IRC bridge would stay, and then
> silently murdering it.
> I'm pretty sure it was planned for the beginning, to lure projects that
> would not move to slack otherwise.
>
> Slack is notoriously behaving in a counter-inclusive manner, asking people
> to switch to a certain browser, completely ignoring user's choice.
>
> Slack user interface is so bad (on any browser) that it requires a
> specialized client or it will drain your battery in no time.
> I refuse installing a 3rd party software or a client from a company with a
> long history of security ignorance.
>
> Slack ignores Internet standards and bugs in web compatibility.
>
> The list could go on, but I am strongly against closed solutions as they
> give away the power over the project to a single hostile company, for them
> to do whatever they want in the future.
>
> Michal Purzynski, speaking privately, aka not MoCo. Feel free to quote me.
>
> On Tue, Oct 15, 2019 at 1:27 AM Jan Grash?fer <jan.grashoefer at gmail.com>
> wrote:
>
>> I am sorry to open the can of worms again but is Slack already set as
>> the platform? Personally, I am in favor of a more open and decentralized
>> approach like matrix [1], which would even allow to bridge the old IRC
>> channel.
>>
>> Jan
>>
>> [1] https://en.wikipedia.org/wiki/Matrix_(protocol)
>>
>> On 15/10/2019 03:41, Amber Graner wrote:
>> > Hi all,
>> >
>> > There was a question on the mailing list earlier this year and again at
>> > ZeekWeek19 about plans to create an Official Zeek Community Slack
>> Channel.
>> >
>> > Below is a 4 question 1 minute survey, please take a moment to
>> respond.  We
>> > will leave the survey up until 21 Oct 2019 5pm PT.
>> >
>> >
>> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
>> >
>> > We will publish the results to the mailing list on 25 October 2019.
>> >
>> > <
>> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
>> >
>> > Please let me know if you have any questions.
>> >
>> > Thanks in advance for taking the time to answer these questions.
>> >
>> > with gratitude,
>> > ~Amber
>> >
>> >
>> > _______________________________________________
>> > Zeek mailing list
>> > zeek at zeek.org
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>> >
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469



 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191015/74a55da9/attachment-0001.html 

From anthony.kasza at gmail.com  Tue Oct 15 05:34:49 2019
From: anthony.kasza at gmail.com (anthony kasza)
Date: Tue, 15 Oct 2019 06:34:49 -0600
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <CAJhOzuq1X4Jm-FTwK8_knku5wO5WD1_Mf1N+_KYLthcZs0JAHQ@mail.gmail.com>
References: <CAJhOzupLoFGh7RYR_kOCcEHaqBVE5S7OX1sf1BRf=yZ6TQ5QFA@mail.gmail.com>
	<63e702a9-e218-412f-1d74-bf1ec2cc09b9@gmail.com>
	<CAJ6bFK3Mw1Ajsfjjxr+cAE0NUpedPY_j9+uEcrG9E42mYm5wjA@mail.gmail.com>
	<CAJhOzuq1X4Jm-FTwK8_knku5wO5WD1_Mf1N+_KYLthcZs0JAHQ@mail.gmail.com>
Message-ID: <CAEZw2byUJG2V6ds+ewnwue-BpZkhdbfMEDwT6Z-A9b0PtuCicQ@mail.gmail.com>

Slack is easy for beginners to use and is adopted by many other
communities. What's popular isn't always right and Slack definitely has its
limitations, as pointed out by others.

If other low maintenance, easy-to-use solutions come to light let's try to
use them. I agree with Amber; IRC is not growing the community and
something else might.

-AK

On Tue, Oct 15, 2019, 04:32 Amber Graner <akgraner at corelight.com> wrote:

>
> Jan, thank you for that suggestion, no need to apologize. And no, slack
> has not been settled on as the defacto choice. However, I wanted to
> followup with this topic since it has been brought up twice now by the
> community.
>
> Michal, I understand the concern not only with Slack as tool, but also
> using closed source tools for an open source project. In addition to
> Matrix, which has been suggested by Jan, are there any other alternatives
> to Slack or IRC that you would recommend?
>
> I think the important thing to note, and I believe what the poll will
> confirm, is that at the moment very few in the community are using IRC and
> many in the community have expressed that they would participate more if
> there was an alternative to IRC. I think most people are familiar with
> Slack because it's a tool they use in their professional environments and
> something they are already familiar with that fits into their current
> workflow.  All of which would lower the barrier to entry and promote more
> participation. However, let's look into other options as well now that it's
> been brought up.
>
> I'll leave the survey up open as planned, but will prepare to present to
> the community and to the Zeek LT a comparison of all alternatives to IRC.
>
> What other suggestions does anyone have in addition to the following:
>
>  - IRC
>  - Slack
>  - Matrix
>  - Discourse (chat)
>
> If you have any other suggestions, please respond here or to me personally
> before end of day on 18 Oct 2019, so that I can prepare the presentation to
> you, the community and to the Zeek LT.
>
> This discussion is great!
>
> Thanks in advance for the suggestions and feedback.
>
> With gratitude,
> ~Amber
>
>
>
>
>
>
>
> On Tue, Oct 15, 2019 at 3:54 AM Micha? Purzy?ski <
> michalpurzynski1 at gmail.com> wrote:
>
>> My private 2c here, not MoCo.
>>
>> I strongly oppose having a closed platform chosen as a communication
>> medium.
>>
>> Slack has a long story of being hostile to projects as soon as there is
>> an opportunity to earn 0.25 cents.
>>
>> Slack lied about the IRC compatibility, making sure to first close
>> projects within the slack world, promising IRC bridge would stay, and then
>> silently murdering it.
>> I'm pretty sure it was planned for the beginning, to lure projects that
>> would not move to slack otherwise.
>>
>> Slack is notoriously behaving in a counter-inclusive manner, asking
>> people to switch to a certain browser, completely ignoring user's choice.
>>
>> Slack user interface is so bad (on any browser) that it requires a
>> specialized client or it will drain your battery in no time.
>> I refuse installing a 3rd party software or a client from a company with
>> a long history of security ignorance.
>>
>> Slack ignores Internet standards and bugs in web compatibility.
>>
>> The list could go on, but I am strongly against closed solutions as they
>> give away the power over the project to a single hostile company, for them
>> to do whatever they want in the future.
>>
>> Michal Purzynski, speaking privately, aka not MoCo. Feel free to quote me.
>>
>> On Tue, Oct 15, 2019 at 1:27 AM Jan Grash?fer <jan.grashoefer at gmail.com>
>> wrote:
>>
>>> I am sorry to open the can of worms again but is Slack already set as
>>> the platform? Personally, I am in favor of a more open and decentralized
>>> approach like matrix [1], which would even allow to bridge the old IRC
>>> channel.
>>>
>>> Jan
>>>
>>> [1] https://en.wikipedia.org/wiki/Matrix_(protocol)
>>>
>>> On 15/10/2019 03:41, Amber Graner wrote:
>>> > Hi all,
>>> >
>>> > There was a question on the mailing list earlier this year and again at
>>> > ZeekWeek19 about plans to create an Official Zeek Community Slack
>>> Channel.
>>> >
>>> > Below is a 4 question 1 minute survey, please take a moment to
>>> respond.  We
>>> > will leave the survey up until 21 Oct 2019 5pm PT.
>>> >
>>> >
>>> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
>>> >
>>> > We will publish the results to the mailing list on 25 October 2019.
>>> >
>>> > <
>>> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
>>> >
>>> > Please let me know if you have any questions.
>>> >
>>> > Thanks in advance for taking the time to answer these questions.
>>> >
>>> > with gratitude,
>>> > ~Amber
>>> >
>>> >
>>> > _______________________________________________
>>> > Zeek mailing list
>>> > zeek at zeek.org
>>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>> >
>>> _______________________________________________
>>> Zeek mailing list
>>> zeek at zeek.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
>
> --
>
> *Amber Graner*
> Director of Community
> Corelight, Inc
>
> 828.582.9469
>
>
>
>  * Ask me about how you can participate in the Zeek (formerly Bro)
> community.
>  * Remember - ZEEK AND YOU SHALL FIND!!
>
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191015/a6295346/attachment.html 

From richard at corelight.com  Tue Oct 15 06:46:17 2019
From: richard at corelight.com (Richard Bejtlich)
Date: Tue, 15 Oct 2019 09:46:17 -0400
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <CAEZw2byUJG2V6ds+ewnwue-BpZkhdbfMEDwT6Z-A9b0PtuCicQ@mail.gmail.com>
References: <CAJhOzupLoFGh7RYR_kOCcEHaqBVE5S7OX1sf1BRf=yZ6TQ5QFA@mail.gmail.com>
	<63e702a9-e218-412f-1d74-bf1ec2cc09b9@gmail.com>
	<CAJ6bFK3Mw1Ajsfjjxr+cAE0NUpedPY_j9+uEcrG9E42mYm5wjA@mail.gmail.com>
	<CAJhOzuq1X4Jm-FTwK8_knku5wO5WD1_Mf1N+_KYLthcZs0JAHQ@mail.gmail.com>
	<CAEZw2byUJG2V6ds+ewnwue-BpZkhdbfMEDwT6Z-A9b0PtuCicQ@mail.gmail.com>
Message-ID: <CAOtSMjZOTijSTnF9x00UkWqdmeoLG2d13+Weg0pZt4-cqhedYg@mail.gmail.com>

Hello everyone,

I agree with Anthony. However, I had never heard of Matrix until Jan's
post. I checked it out via the Riot Web client, but I couldn't tell if the
bridge with the #bro IRC channel worked or not. Furthermore, bridging the
channel required having a registered nick with Freenode, which I had to
re-activate. Eventually I was able to access Freenode using their Web chat,
but I don't know if I was able to access #bro as it was quiet.

This is the sort of friction that causes new people to give up on chatting
about Zeek. Slack at least makes it very easy for new users to learn about
what happens in a channel.

Also, is there a concept of chat history in Matrix? That's one of the best
features of Slack, in my opinion. Reading back through time is a great way
to learn.

Sincerely,

Richard

On Tue, Oct 15, 2019 at 8:43 AM anthony kasza <anthony.kasza at gmail.com>
wrote:

> Slack is easy for beginners to use and is adopted by many other
> communities. What's popular isn't always right and Slack definitely has its
> limitations, as pointed out by others.
>
> If other low maintenance, easy-to-use solutions come to light let's try to
> use them. I agree with Amber; IRC is not growing the community and
> something else might.
>
> -AK
>
> On Tue, Oct 15, 2019, 04:32 Amber Graner <akgraner at corelight.com> wrote:
>
>>
>> Jan, thank you for that suggestion, no need to apologize. And no, slack
>> has not been settled on as the defacto choice. However, I wanted to
>> followup with this topic since it has been brought up twice now by the
>> community.
>>
>> Michal, I understand the concern not only with Slack as tool, but also
>> using closed source tools for an open source project. In addition to
>> Matrix, which has been suggested by Jan, are there any other alternatives
>> to Slack or IRC that you would recommend?
>>
>> I think the important thing to note, and I believe what the poll will
>> confirm, is that at the moment very few in the community are using IRC and
>> many in the community have expressed that they would participate more if
>> there was an alternative to IRC. I think most people are familiar with
>> Slack because it's a tool they use in their professional environments and
>> something they are already familiar with that fits into their current
>> workflow.  All of which would lower the barrier to entry and promote more
>> participation. However, let's look into other options as well now that it's
>> been brought up.
>>
>> I'll leave the survey up open as planned, but will prepare to present to
>> the community and to the Zeek LT a comparison of all alternatives to IRC.
>>
>> What other suggestions does anyone have in addition to the following:
>>
>>  - IRC
>>  - Slack
>>  - Matrix
>>  - Discourse (chat)
>>
>> If you have any other suggestions, please respond here or to me
>> personally before end of day on 18 Oct 2019, so that I can prepare the
>> presentation to you, the community and to the Zeek LT.
>>
>> This discussion is great!
>>
>> Thanks in advance for the suggestions and feedback.
>>
>> With gratitude,
>> ~Amber
>>
>>
>>
>>
>>
>>
>>
>> On Tue, Oct 15, 2019 at 3:54 AM Micha? Purzy?ski <
>> michalpurzynski1 at gmail.com> wrote:
>>
>>> My private 2c here, not MoCo.
>>>
>>> I strongly oppose having a closed platform chosen as a communication
>>> medium.
>>>
>>> Slack has a long story of being hostile to projects as soon as there is
>>> an opportunity to earn 0.25 cents.
>>>
>>> Slack lied about the IRC compatibility, making sure to first close
>>> projects within the slack world, promising IRC bridge would stay, and then
>>> silently murdering it.
>>> I'm pretty sure it was planned for the beginning, to lure projects that
>>> would not move to slack otherwise.
>>>
>>> Slack is notoriously behaving in a counter-inclusive manner, asking
>>> people to switch to a certain browser, completely ignoring user's choice.
>>>
>>> Slack user interface is so bad (on any browser) that it requires a
>>> specialized client or it will drain your battery in no time.
>>> I refuse installing a 3rd party software or a client from a company with
>>> a long history of security ignorance.
>>>
>>> Slack ignores Internet standards and bugs in web compatibility.
>>>
>>> The list could go on, but I am strongly against closed solutions as they
>>> give away the power over the project to a single hostile company, for them
>>> to do whatever they want in the future.
>>>
>>> Michal Purzynski, speaking privately, aka not MoCo. Feel free to quote
>>> me.
>>>
>>> On Tue, Oct 15, 2019 at 1:27 AM Jan Grash?fer <jan.grashoefer at gmail.com>
>>> wrote:
>>>
>>>> I am sorry to open the can of worms again but is Slack already set as
>>>> the platform? Personally, I am in favor of a more open and
>>>> decentralized
>>>> approach like matrix [1], which would even allow to bridge the old IRC
>>>> channel.
>>>>
>>>> Jan
>>>>
>>>> [1] https://en.wikipedia.org/wiki/Matrix_(protocol)
>>>>
>>>> On 15/10/2019 03:41, Amber Graner wrote:
>>>> > Hi all,
>>>> >
>>>> > There was a question on the mailing list earlier this year and again
>>>> at
>>>> > ZeekWeek19 about plans to create an Official Zeek Community Slack
>>>> Channel.
>>>> >
>>>> > Below is a 4 question 1 minute survey, please take a moment to
>>>> respond.  We
>>>> > will leave the survey up until 21 Oct 2019 5pm PT.
>>>> >
>>>> >
>>>> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
>>>> >
>>>> > We will publish the results to the mailing list on 25 October 2019.
>>>> >
>>>> > <
>>>> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
>>>> >
>>>> > Please let me know if you have any questions.
>>>> >
>>>> > Thanks in advance for taking the time to answer these questions.
>>>> >
>>>> > with gratitude,
>>>> > ~Amber
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > Zeek mailing list
>>>> > zeek at zeek.org
>>>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>> >
>>>> _______________________________________________
>>>> Zeek mailing list
>>>> zeek at zeek.org
>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>>
>>> _______________________________________________
>>> Zeek mailing list
>>> zeek at zeek.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>
>>
>>
>> --
>>
>> *Amber Graner*
>> Director of Community
>> Corelight, Inc
>>
>> 828.582.9469
>>
>>
>>
>>  * Ask me about how you can participate in the Zeek (formerly Bro)
>> community.
>>  * Remember - ZEEK AND YOU SHALL FIND!!
>>
>>
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
Richard Bejtlich
Principal Security Strategist, Corelight
https://corelight.blog/author/richardbejtlich/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191015/427ae7a6/attachment-0001.html 

From fatema.bannatwala at gmail.com  Tue Oct 15 07:24:30 2019
From: fatema.bannatwala at gmail.com (fatema bannatwala)
Date: Tue, 15 Oct 2019 10:24:30 -0400
Subject: [Zeek] Official Zeek Community Slack Channel Poll
Message-ID: <CACX0rURsNPYSCozicOLcNKMKJ7MtGP8MH2OMCR-3SjcYuu39Vw@mail.gmail.com>

Thanks Amber for setting up the poll for the audience input.
All the points mentioned regarding pros and cons are noted, and think one
of the ways, to see what will work best for the community to stay connected
via a communication channel, is to ask for their input. What works for us
or few of us might not work at all for the community and we definitely
would want to make their life easy by providing an easy access to the
community channel.

For the survey, looks good, maybe providing the third option of "Maybe"
with a comment box for the people to write their comments, if slack doesn't
work for them, could help us getting more info? I am sure there are better
ways to incorporate the users feedback, but just a thought!

Regards,
Fatema
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191015/0c2a4bcf/attachment.html 

From vallentin at icir.org  Tue Oct 15 09:42:01 2019
From: vallentin at icir.org (Matthias Vallentin)
Date: Tue, 15 Oct 2019 18:42:01 +0200
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <CAOtSMjZOTijSTnF9x00UkWqdmeoLG2d13+Weg0pZt4-cqhedYg@mail.gmail.com>
References: <CAJhOzupLoFGh7RYR_kOCcEHaqBVE5S7OX1sf1BRf=yZ6TQ5QFA@mail.gmail.com>
	<63e702a9-e218-412f-1d74-bf1ec2cc09b9@gmail.com>
	<CAJ6bFK3Mw1Ajsfjjxr+cAE0NUpedPY_j9+uEcrG9E42mYm5wjA@mail.gmail.com>
	<CAJhOzuq1X4Jm-FTwK8_knku5wO5WD1_Mf1N+_KYLthcZs0JAHQ@mail.gmail.com>
	<CAEZw2byUJG2V6ds+ewnwue-BpZkhdbfMEDwT6Z-A9b0PtuCicQ@mail.gmail.com>
	<CAOtSMjZOTijSTnF9x00UkWqdmeoLG2d13+Weg0pZt4-cqhedYg@mail.gmail.com>
Message-ID: <CADTpMNaqn+piu_8pG2Sy5dTHLYxPG9UrNrFqgAopLOhrrnVkoA@mail.gmail.com>

A befriended startup recommended Rocket Chat to me the other day:
https://rocket.chat.

When evaluating other options, this might be a contender. It seems there's
IRC integration available as well [1].

    Matthias

[1] https://rocket.chat/2018/07/17/v66-release-post/

On Tue, Oct 15, 2019 at 3:55 PM Richard Bejtlich <richard at corelight.com>
wrote:

> Hello everyone,
>
> I agree with Anthony. However, I had never heard of Matrix until Jan's
> post. I checked it out via the Riot Web client, but I couldn't tell if the
> bridge with the #bro IRC channel worked or not. Furthermore, bridging the
> channel required having a registered nick with Freenode, which I had to
> re-activate. Eventually I was able to access Freenode using their Web chat,
> but I don't know if I was able to access #bro as it was quiet.
>
> This is the sort of friction that causes new people to give up on chatting
> about Zeek. Slack at least makes it very easy for new users to learn about
> what happens in a channel.
>
> Also, is there a concept of chat history in Matrix? That's one of the best
> features of Slack, in my opinion. Reading back through time is a great way
> to learn.
>
> Sincerely,
>
> Richard
>
> On Tue, Oct 15, 2019 at 8:43 AM anthony kasza <anthony.kasza at gmail.com>
> wrote:
>
>> Slack is easy for beginners to use and is adopted by many other
>> communities. What's popular isn't always right and Slack definitely has its
>> limitations, as pointed out by others.
>>
>> If other low maintenance, easy-to-use solutions come to light let's try
>> to use them. I agree with Amber; IRC is not growing the community and
>> something else might.
>>
>> -AK
>>
>> On Tue, Oct 15, 2019, 04:32 Amber Graner <akgraner at corelight.com> wrote:
>>
>>>
>>> Jan, thank you for that suggestion, no need to apologize. And no, slack
>>> has not been settled on as the defacto choice. However, I wanted to
>>> followup with this topic since it has been brought up twice now by the
>>> community.
>>>
>>> Michal, I understand the concern not only with Slack as tool, but also
>>> using closed source tools for an open source project. In addition to
>>> Matrix, which has been suggested by Jan, are there any other alternatives
>>> to Slack or IRC that you would recommend?
>>>
>>> I think the important thing to note, and I believe what the poll will
>>> confirm, is that at the moment very few in the community are using IRC and
>>> many in the community have expressed that they would participate more if
>>> there was an alternative to IRC. I think most people are familiar with
>>> Slack because it's a tool they use in their professional environments and
>>> something they are already familiar with that fits into their current
>>> workflow.  All of which would lower the barrier to entry and promote more
>>> participation. However, let's look into other options as well now that it's
>>> been brought up.
>>>
>>> I'll leave the survey up open as planned, but will prepare to present to
>>> the community and to the Zeek LT a comparison of all alternatives to IRC.
>>>
>>> What other suggestions does anyone have in addition to the following:
>>>
>>>  - IRC
>>>  - Slack
>>>  - Matrix
>>>  - Discourse (chat)
>>>
>>> If you have any other suggestions, please respond here or to me
>>> personally before end of day on 18 Oct 2019, so that I can prepare the
>>> presentation to you, the community and to the Zeek LT.
>>>
>>> This discussion is great!
>>>
>>> Thanks in advance for the suggestions and feedback.
>>>
>>> With gratitude,
>>> ~Amber
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Oct 15, 2019 at 3:54 AM Micha? Purzy?ski <
>>> michalpurzynski1 at gmail.com> wrote:
>>>
>>>> My private 2c here, not MoCo.
>>>>
>>>> I strongly oppose having a closed platform chosen as a communication
>>>> medium.
>>>>
>>>> Slack has a long story of being hostile to projects as soon as there is
>>>> an opportunity to earn 0.25 cents.
>>>>
>>>> Slack lied about the IRC compatibility, making sure to first close
>>>> projects within the slack world, promising IRC bridge would stay, and then
>>>> silently murdering it.
>>>> I'm pretty sure it was planned for the beginning, to lure projects that
>>>> would not move to slack otherwise.
>>>>
>>>> Slack is notoriously behaving in a counter-inclusive manner, asking
>>>> people to switch to a certain browser, completely ignoring user's choice.
>>>>
>>>> Slack user interface is so bad (on any browser) that it requires a
>>>> specialized client or it will drain your battery in no time.
>>>> I refuse installing a 3rd party software or a client from a company
>>>> with a long history of security ignorance.
>>>>
>>>> Slack ignores Internet standards and bugs in web compatibility.
>>>>
>>>> The list could go on, but I am strongly against closed solutions as
>>>> they give away the power over the project to a single hostile company, for
>>>> them to do whatever they want in the future.
>>>>
>>>> Michal Purzynski, speaking privately, aka not MoCo. Feel free to quote
>>>> me.
>>>>
>>>> On Tue, Oct 15, 2019 at 1:27 AM Jan Grash?fer <jan.grashoefer at gmail.com>
>>>> wrote:
>>>>
>>>>> I am sorry to open the can of worms again but is Slack already set as
>>>>> the platform? Personally, I am in favor of a more open and
>>>>> decentralized
>>>>> approach like matrix [1], which would even allow to bridge the old IRC
>>>>> channel.
>>>>>
>>>>> Jan
>>>>>
>>>>> [1] https://en.wikipedia.org/wiki/Matrix_(protocol)
>>>>>
>>>>> On 15/10/2019 03:41, Amber Graner wrote:
>>>>> > Hi all,
>>>>> >
>>>>> > There was a question on the mailing list earlier this year and again
>>>>> at
>>>>> > ZeekWeek19 about plans to create an Official Zeek Community Slack
>>>>> Channel.
>>>>> >
>>>>> > Below is a 4 question 1 minute survey, please take a moment to
>>>>> respond.  We
>>>>> > will leave the survey up until 21 Oct 2019 5pm PT.
>>>>> >
>>>>> >
>>>>> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
>>>>> >
>>>>> > We will publish the results to the mailing list on 25 October 2019.
>>>>> >
>>>>> > <
>>>>> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
>>>>> >
>>>>> > Please let me know if you have any questions.
>>>>> >
>>>>> > Thanks in advance for taking the time to answer these questions.
>>>>> >
>>>>> > with gratitude,
>>>>> > ~Amber
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > Zeek mailing list
>>>>> > zeek at zeek.org
>>>>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>>> >
>>>>> _______________________________________________
>>>>> Zeek mailing list
>>>>> zeek at zeek.org
>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>>>
>>>> _______________________________________________
>>>> Zeek mailing list
>>>> zeek at zeek.org
>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>
>>>
>>>
>>> --
>>>
>>> *Amber Graner*
>>> Director of Community
>>> Corelight, Inc
>>>
>>> 828.582.9469
>>>
>>>
>>>
>>>  * Ask me about how you can participate in the Zeek (formerly Bro)
>>> community.
>>>  * Remember - ZEEK AND YOU SHALL FIND!!
>>>
>>>
>>> _______________________________________________
>>> Zeek mailing list
>>> zeek at zeek.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
>
> --
> Richard Bejtlich
> Principal Security Strategist, Corelight
> https://corelight.blog/author/richardbejtlich/
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191015/fc36c519/attachment.html 

From akgraner at corelight.com  Tue Oct 15 09:45:54 2019
From: akgraner at corelight.com (Amber Graner)
Date: Tue, 15 Oct 2019 11:45:54 -0500
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <CADTpMNaqn+piu_8pG2Sy5dTHLYxPG9UrNrFqgAopLOhrrnVkoA@mail.gmail.com>
References: <CAJhOzupLoFGh7RYR_kOCcEHaqBVE5S7OX1sf1BRf=yZ6TQ5QFA@mail.gmail.com>
	<63e702a9-e218-412f-1d74-bf1ec2cc09b9@gmail.com>
	<CAJ6bFK3Mw1Ajsfjjxr+cAE0NUpedPY_j9+uEcrG9E42mYm5wjA@mail.gmail.com>
	<CAJhOzuq1X4Jm-FTwK8_knku5wO5WD1_Mf1N+_KYLthcZs0JAHQ@mail.gmail.com>
	<CAEZw2byUJG2V6ds+ewnwue-BpZkhdbfMEDwT6Z-A9b0PtuCicQ@mail.gmail.com>
	<CAOtSMjZOTijSTnF9x00UkWqdmeoLG2d13+Weg0pZt4-cqhedYg@mail.gmail.com>
	<CADTpMNaqn+piu_8pG2Sy5dTHLYxPG9UrNrFqgAopLOhrrnVkoA@mail.gmail.com>
Message-ID: <CAJhOzupJNzqFU9Xu+wougVAYx=+yMoqVqg-aEK_bMT5vvXEG_Q@mail.gmail.com>

Thanks!  Adding it to the list.

~Amber

On Tue, Oct 15, 2019 at 11:44 AM Matthias Vallentin <vallentin at icir.org>
wrote:

> A befriended startup recommended Rocket Chat to me the other day:
> https://rocket.chat.
>
> When evaluating other options, this might be a contender. It seems there's
> IRC integration available as well [1].
>
>     Matthias
>
> [1] https://rocket.chat/2018/07/17/v66-release-post/
>
> On Tue, Oct 15, 2019 at 3:55 PM Richard Bejtlich <richard at corelight.com>
> wrote:
>
>> Hello everyone,
>>
>> I agree with Anthony. However, I had never heard of Matrix until Jan's
>> post. I checked it out via the Riot Web client, but I couldn't tell if the
>> bridge with the #bro IRC channel worked or not. Furthermore, bridging the
>> channel required having a registered nick with Freenode, which I had to
>> re-activate. Eventually I was able to access Freenode using their Web chat,
>> but I don't know if I was able to access #bro as it was quiet.
>>
>> This is the sort of friction that causes new people to give up on
>> chatting about Zeek. Slack at least makes it very easy for new users to
>> learn about what happens in a channel.
>>
>> Also, is there a concept of chat history in Matrix? That's one of the
>> best features of Slack, in my opinion. Reading back through time is a great
>> way to learn.
>>
>> Sincerely,
>>
>> Richard
>>
>> On Tue, Oct 15, 2019 at 8:43 AM anthony kasza <anthony.kasza at gmail.com>
>> wrote:
>>
>>> Slack is easy for beginners to use and is adopted by many other
>>> communities. What's popular isn't always right and Slack definitely has its
>>> limitations, as pointed out by others.
>>>
>>> If other low maintenance, easy-to-use solutions come to light let's try
>>> to use them. I agree with Amber; IRC is not growing the community and
>>> something else might.
>>>
>>> -AK
>>>
>>> On Tue, Oct 15, 2019, 04:32 Amber Graner <akgraner at corelight.com> wrote:
>>>
>>>>
>>>> Jan, thank you for that suggestion, no need to apologize. And no, slack
>>>> has not been settled on as the defacto choice. However, I wanted to
>>>> followup with this topic since it has been brought up twice now by the
>>>> community.
>>>>
>>>> Michal, I understand the concern not only with Slack as tool, but also
>>>> using closed source tools for an open source project. In addition to
>>>> Matrix, which has been suggested by Jan, are there any other alternatives
>>>> to Slack or IRC that you would recommend?
>>>>
>>>> I think the important thing to note, and I believe what the poll will
>>>> confirm, is that at the moment very few in the community are using IRC and
>>>> many in the community have expressed that they would participate more if
>>>> there was an alternative to IRC. I think most people are familiar with
>>>> Slack because it's a tool they use in their professional environments and
>>>> something they are already familiar with that fits into their current
>>>> workflow.  All of which would lower the barrier to entry and promote more
>>>> participation. However, let's look into other options as well now that it's
>>>> been brought up.
>>>>
>>>> I'll leave the survey up open as planned, but will prepare to present
>>>> to the community and to the Zeek LT a comparison of all alternatives to
>>>> IRC.
>>>>
>>>> What other suggestions does anyone have in addition to the following:
>>>>
>>>>  - IRC
>>>>  - Slack
>>>>  - Matrix
>>>>  - Discourse (chat)
>>>>
>>>> If you have any other suggestions, please respond here or to me
>>>> personally before end of day on 18 Oct 2019, so that I can prepare the
>>>> presentation to you, the community and to the Zeek LT.
>>>>
>>>> This discussion is great!
>>>>
>>>> Thanks in advance for the suggestions and feedback.
>>>>
>>>> With gratitude,
>>>> ~Amber
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Oct 15, 2019 at 3:54 AM Micha? Purzy?ski <
>>>> michalpurzynski1 at gmail.com> wrote:
>>>>
>>>>> My private 2c here, not MoCo.
>>>>>
>>>>> I strongly oppose having a closed platform chosen as a communication
>>>>> medium.
>>>>>
>>>>> Slack has a long story of being hostile to projects as soon as there
>>>>> is an opportunity to earn 0.25 cents.
>>>>>
>>>>> Slack lied about the IRC compatibility, making sure to first close
>>>>> projects within the slack world, promising IRC bridge would stay, and then
>>>>> silently murdering it.
>>>>> I'm pretty sure it was planned for the beginning, to lure projects
>>>>> that would not move to slack otherwise.
>>>>>
>>>>> Slack is notoriously behaving in a counter-inclusive manner, asking
>>>>> people to switch to a certain browser, completely ignoring user's choice.
>>>>>
>>>>> Slack user interface is so bad (on any browser) that it requires a
>>>>> specialized client or it will drain your battery in no time.
>>>>> I refuse installing a 3rd party software or a client from a company
>>>>> with a long history of security ignorance.
>>>>>
>>>>> Slack ignores Internet standards and bugs in web compatibility.
>>>>>
>>>>> The list could go on, but I am strongly against closed solutions as
>>>>> they give away the power over the project to a single hostile company, for
>>>>> them to do whatever they want in the future.
>>>>>
>>>>> Michal Purzynski, speaking privately, aka not MoCo. Feel free to quote
>>>>> me.
>>>>>
>>>>> On Tue, Oct 15, 2019 at 1:27 AM Jan Grash?fer <
>>>>> jan.grashoefer at gmail.com> wrote:
>>>>>
>>>>>> I am sorry to open the can of worms again but is Slack already set as
>>>>>> the platform? Personally, I am in favor of a more open and
>>>>>> decentralized
>>>>>> approach like matrix [1], which would even allow to bridge the old
>>>>>> IRC
>>>>>> channel.
>>>>>>
>>>>>> Jan
>>>>>>
>>>>>> [1] https://en.wikipedia.org/wiki/Matrix_(protocol)
>>>>>>
>>>>>> On 15/10/2019 03:41, Amber Graner wrote:
>>>>>> > Hi all,
>>>>>> >
>>>>>> > There was a question on the mailing list earlier this year and
>>>>>> again at
>>>>>> > ZeekWeek19 about plans to create an Official Zeek Community Slack
>>>>>> Channel.
>>>>>> >
>>>>>> > Below is a 4 question 1 minute survey, please take a moment to
>>>>>> respond.  We
>>>>>> > will leave the survey up until 21 Oct 2019 5pm PT.
>>>>>> >
>>>>>> >
>>>>>> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
>>>>>> >
>>>>>> > We will publish the results to the mailing list on 25 October 2019.
>>>>>> >
>>>>>> > <
>>>>>> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
>>>>>> >
>>>>>> > Please let me know if you have any questions.
>>>>>> >
>>>>>> > Thanks in advance for taking the time to answer these questions.
>>>>>> >
>>>>>> > with gratitude,
>>>>>> > ~Amber
>>>>>> >
>>>>>> >
>>>>>> > _______________________________________________
>>>>>> > Zeek mailing list
>>>>>> > zeek at zeek.org
>>>>>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>>>> >
>>>>>> _______________________________________________
>>>>>> Zeek mailing list
>>>>>> zeek at zeek.org
>>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>>>>
>>>>> _______________________________________________
>>>>> Zeek mailing list
>>>>> zeek at zeek.org
>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Amber Graner*
>>>> Director of Community
>>>> Corelight, Inc
>>>>
>>>> 828.582.9469
>>>>
>>>>
>>>>
>>>>  * Ask me about how you can participate in the Zeek (formerly Bro)
>>>> community.
>>>>  * Remember - ZEEK AND YOU SHALL FIND!!
>>>>
>>>>
>>>> _______________________________________________
>>>> Zeek mailing list
>>>> zeek at zeek.org
>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>
>>> _______________________________________________
>>> Zeek mailing list
>>> zeek at zeek.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>
>>
>>
>> --
>> Richard Bejtlich
>> Principal Security Strategist, Corelight
>> https://corelight.blog/author/richardbejtlich/
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469



 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191015/1a690e65/attachment-0001.html 

From jlay at slave-tothe-box.net  Tue Oct 15 09:55:52 2019
From: jlay at slave-tothe-box.net (James Lay)
Date: Tue, 15 Oct 2019 10:55:52 -0600
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <CAJhOzupJNzqFU9Xu+wougVAYx=+yMoqVqg-aEK_bMT5vvXEG_Q@mail.gmail.com>
References: <CAJhOzupLoFGh7RYR_kOCcEHaqBVE5S7OX1sf1BRf=yZ6TQ5QFA@mail.gmail.com>
	<63e702a9-e218-412f-1d74-bf1ec2cc09b9@gmail.com>
	<CAJ6bFK3Mw1Ajsfjjxr+cAE0NUpedPY_j9+uEcrG9E42mYm5wjA@mail.gmail.com>
	<CAJhOzuq1X4Jm-FTwK8_knku5wO5WD1_Mf1N+_KYLthcZs0JAHQ@mail.gmail.com>
	<CAEZw2byUJG2V6ds+ewnwue-BpZkhdbfMEDwT6Z-A9b0PtuCicQ@mail.gmail.com>
	<CAOtSMjZOTijSTnF9x00UkWqdmeoLG2d13+Weg0pZt4-cqhedYg@mail.gmail.com>
	<CADTpMNaqn+piu_8pG2Sy5dTHLYxPG9UrNrFqgAopLOhrrnVkoA@mail.gmail.com>
	<CAJhOzupJNzqFU9Xu+wougVAYx=+yMoqVqg-aEK_bMT5vvXEG_Q@mail.gmail.com>
Message-ID: <1dbd11d63a4f42de6f7e5c6bb5de99cb@slave-tothe-box.net>

I think this depends on your target audience.  Maybe edu folks have 
unfettered port/application access on school networks...I don't know, 
but I can tell you, good luck getting anything resembling IRC on my 
$dayjob network.

James

On 2019-10-15 10:45, Amber Graner wrote:
> Thanks!  Adding it to the list.
> 
> ~Amber
> 
> On Tue, Oct 15, 2019 at 11:44 AM Matthias Vallentin
> <vallentin at icir.org> wrote:
> 
>> A befriended startup recommended Rocket Chat to me the other day:
>> https://rocket.chat.
>> 
>> When evaluating other options, this might be a contender. It seems
>> there's IRC integration available as well [1].
>> 
>> Matthias
>> 
>> [1] https://rocket.chat/2018/07/17/v66-release-post/
>> 
>> On Tue, Oct 15, 2019 at 3:55 PM Richard Bejtlich
>> <richard at corelight.com> wrote:
>> 
>> Hello everyone,
>> 
>> I agree with Anthony. However, I had never heard of Matrix until
>> Jan's post. I checked it out via the Riot Web client, but I couldn't
>> tell if the bridge with the #bro IRC channel worked or not.
>> Furthermore, bridging the channel required having a registered nick
>> with Freenode, which I had to re-activate. Eventually I was able to
>> access Freenode using their Web chat, but I don't know if I was able
>> to access #bro as it was quiet.
>> 
>> This is the sort of friction that causes new people to give up on
>> chatting about Zeek. Slack at least makes it very easy for new users
>> to learn about what happens in a channel.
>> 
>> Also, is there a concept of chat history in Matrix? That's one of
>> the best features of Slack, in my opinion. Reading back through time
>> is a great way to learn.
>> 
>> Sincerely,
>> 
>> Richard
>> 
>> On Tue, Oct 15, 2019 at 8:43 AM anthony kasza
>> <anthony.kasza at gmail.com> wrote:
>> 
>> Slack is easy for beginners to use and is adopted by many other
>> communities. What's popular isn't always right and Slack definitely
>> has its limitations, as pointed out by others.
>> 
>> If other low maintenance, easy-to-use solutions come to light let's
>> try to use them. I agree with Amber; IRC is not growing the
>> community and something else might.
>> 
>> -AK
>> 
>> On Tue, Oct 15, 2019, 04:32 Amber Graner <akgraner at corelight.com>
>> wrote:
>> 
>> Jan, thank you for that suggestion, no need to apologize. And no,
>> slack has not been settled on as the defacto choice. However, I
>> wanted to followup with this topic since it has been brought up
>> twice now by the community.
>> 
>> Michal, I understand the concern not only with Slack as tool, but
>> also using closed source tools for an open source project. In
>> addition to Matrix, which has been suggested by Jan, are there any
>> other alternatives to Slack or IRC that you would recommend?
>> 
>> I think the important thing to note, and I believe what the poll
>> will confirm, is that at the moment very few in the community are
>> using IRC and many in the community have expressed that they would
>> participate more if there was an alternative to IRC. I think most
>> people are familiar with Slack because it's a tool they use in their
>> professional environments and something they are already familiar
>> with that fits into their current workflow.  All of which would
>> lower the barrier to entry and promote more participation. However,
>> let's look into other options as well now that it's been brought up.
>> 
>> 
>> I'll leave the survey up open as planned, but will prepare to
>> present to the community and to the Zeek LT a comparison of all
>> alternatives to IRC.
>> 
>> What other suggestions does anyone have in addition to the
>> following:
>> 
>> - IRC
>> - Slack
>> - Matrix
>> - Discourse (chat)
>> 
>> If you have any other suggestions, please respond here or to me
>> personally before end of day on 18 Oct 2019, so that I can prepare
>> the presentation to you, the community and to the Zeek LT.
>> 
>> This discussion is great!
>> 
>> Thanks in advance for the suggestions and feedback.
>> 
>> With gratitude,
>> ~Amber
>> 
>> On Tue, Oct 15, 2019 at 3:54 AM Micha? Purzy?ski
>> <michalpurzynski1 at gmail.com> wrote:
>> 
>> My private 2c here, not MoCo.
>> 
>> I strongly oppose having a closed platform chosen as a communication
>> medium.
>> 
>> Slack has a long story of being hostile to projects as soon as there
>> is an opportunity to earn 0.25 cents.
>> 
>> Slack lied about the IRC compatibility, making sure to first close
>> projects within the slack world, promising IRC bridge would stay,
>> and then silently murdering it.
>> I'm pretty sure it was planned for the beginning, to lure projects
>> that would not move to slack otherwise.
>> 
>> Slack is notoriously behaving in a counter-inclusive manner, asking
>> people to switch to a certain browser, completely ignoring user's
>> choice.
>> 
>> Slack user interface is so bad (on any browser) that it requires a
>> specialized client or it will drain your battery in no time.
>> I refuse installing a 3rd party software or a client from a company
>> with a long history of security ignorance.
>> 
>> Slack ignores Internet standards and bugs in web compatibility.
>> 
>> The list could go on, but I am strongly against closed solutions as
>> they give away the power over the project to a single hostile
>> company, for them to do whatever they want in the future.
>> 
>> Michal Purzynski, speaking privately, aka not MoCo. Feel free to
>> quote me.
>> 
>> On Tue, Oct 15, 2019 at 1:27 AM Jan Grash?fer
>> <jan.grashoefer at gmail.com> wrote:
>> I am sorry to open the can of worms again but is Slack already set
>> as
>> the platform? Personally, I am in favor of a more open and
>> decentralized
>> approach like matrix [1], which would even allow to bridge the old
>> IRC
>> channel.
>> 
>> Jan
>> 
>> [1] https://en.wikipedia.org/wiki/Matrix_(protocol)
>> 
>> On 15/10/2019 03:41, Amber Graner wrote:
>>> Hi all,
>>> 
>>> There was a question on the mailing list earlier this year and
>> again at
>>> ZeekWeek19 about plans to create an Official Zeek Community Slack
>> Channel.
>>> 
>>> Below is a 4 question 1 minute survey, please take a moment to
>> respond.  We
>>> will leave the survey up until 21 Oct 2019 5pm PT.
>>> 
>>> 
>> 
> https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll
>>> 
>>> We will publish the results to the mailing list on 25 October
>> 2019.
>>> 
>>> 
>> 
> <https://www.surveymonkey.com/r/Official_Zeek_Community_Slack_Channel_Poll>
>>> Please let me know if you have any questions.
>>> 
>>> Thanks in advance for taking the time to answer these questions.
>>> 
>>> with gratitude,
>>> ~Amber
>>> 
>>> 
>>> _______________________________________________
>>> Zeek mailing list
>>> zeek at zeek.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>> 
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> 
> --
> 
> AMBER GRANER
> 
> Director of Community
> Corelight, Inc
> 
> 828.582.9469
> 
>  * Ask me about how you can participate in the Zeek (formerly Bro)
> community.
>  * Remember - ZEEK AND YOU SHALL FIND!!
> 
>  _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> 
> --
> 
> Richard Bejtlich
> Principal Security Strategist, Corelight
> https://corelight.blog/author/richardbejtlich/
>  _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> 
> --
> 
> AMBER GRANER
> 
> Director of Community
> Corelight, Inc
> 
> 828.582.9469
> 
>  * Ask me about how you can participate in the Zeek (formerly Bro)
> community.
>  * Remember - ZEEK AND YOU SHALL FIND!!
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

From dopheide at gmail.com  Tue Oct 15 10:04:59 2019
From: dopheide at gmail.com (Mike Dopheide)
Date: Tue, 15 Oct 2019 12:04:59 -0500
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <CACX0rURsNPYSCozicOLcNKMKJ7MtGP8MH2OMCR-3SjcYuu39Vw@mail.gmail.com>
References: <CACX0rURsNPYSCozicOLcNKMKJ7MtGP8MH2OMCR-3SjcYuu39Vw@mail.gmail.com>
Message-ID: <CAPy2kFYeD5V_TkXDKRYxz0tO-U7B7P5U0YrzHJ-WvQnRpK80Yg@mail.gmail.com>

I think Fatema is spot on here.  It doesn't matter what any of us would
prefer in a perfect world.

If any random open source standard would work, then we'd have a bunch of
people in irc asking questions.  Any solution that requires people to
install a different app or have an extra browser tab open is likely doomed
to a similar fate, but if we can hook into an app like Slack, that they
already have open, we have a better chance of staying connected after
reboots and garnering participation via subtle activity notifications.

-Dop





On Tue, Oct 15, 2019 at 9:26 AM fatema bannatwala <
fatema.bannatwala at gmail.com> wrote:

> Thanks Amber for setting up the poll for the audience input.
> All the points mentioned regarding pros and cons are noted, and think one
> of the ways, to see what will work best for the community to stay connected
> via a communication channel, is to ask for their input. What works for us
> or few of us might not work at all for the community and we definitely
> would want to make their life easy by providing an easy access to the
> community channel.
>
> For the survey, looks good, maybe providing the third option of "Maybe"
> with a comment box for the people to write their comments, if slack doesn't
> work for them, could help us getting more info? I am sure there are better
> ways to incorporate the users feedback, but just a thought!
>
> Regards,
> Fatema
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191015/aeace0c2/attachment.html 

From tet68mt at gmail.com  Tue Oct 15 18:02:14 2019
From: tet68mt at gmail.com (Matt Trostel)
Date: Tue, 15 Oct 2019 20:02:14 -0500
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <CAPy2kFYeD5V_TkXDKRYxz0tO-U7B7P5U0YrzHJ-WvQnRpK80Yg@mail.gmail.com>
References: <CACX0rURsNPYSCozicOLcNKMKJ7MtGP8MH2OMCR-3SjcYuu39Vw@mail.gmail.com>
	<CAPy2kFYeD5V_TkXDKRYxz0tO-U7B7P5U0YrzHJ-WvQnRpK80Yg@mail.gmail.com>
Message-ID: <EC6DD8FC-6856-4707-BDE8-10ED1029B0C9@gmail.com>

I agree with Mike and Fatema.

I?d also like to clarify the Discourse is not just chat. It is more analogous to a forum system. It also has Slack integration, though I?ll admit I don?t know to what extent. Maybe that could prove to be a good middle ground? They also offer free hosting for open source projects.

I like the idea of finding a forum thread versus a mail archive through a web search. They are easier to follow in my opinion.

I think it is also important to note that whatever comes of this, we aren?t bound to the specific platform. As a community, if we recognize that this is a learning experience on how best to organize and communicate -- adoption and change will be smooth whenever it is necessary.

I found this article interesting and a good take on the subject. It?s worth the read.
https://blog.discourse.org/2018/04/effectively-using-discourse-together-with-group-chat/ <https://blog.discourse.org/2018/04/effectively-using-discourse-together-with-group-chat/>

? Matt Trostel


> On Oct 15, 2019, at 12:04, Mike Dopheide <dopheide at gmail.com> wrote:
> 
> I think Fatema is spot on here.  It doesn't matter what any of us would prefer in a perfect world.  
> 
> If any random open source standard would work, then we'd have a bunch of people in irc asking questions.  Any solution that requires people to install a different app or have an extra browser tab open is likely doomed to a similar fate, but if we can hook into an app like Slack, that they already have open, we have a better chance of staying connected after reboots and garnering participation via subtle activity notifications.
> 
> -Dop
> 
> 
> 
> 
> 
> On Tue, Oct 15, 2019 at 9:26 AM fatema bannatwala <fatema.bannatwala at gmail.com <mailto:fatema.bannatwala at gmail.com>> wrote:
> Thanks Amber for setting up the poll for the audience input.
> All the points mentioned regarding pros and cons are noted, and think one of the ways, to see what will work best for the community to stay connected via a communication channel, is to ask for their input. What works for us or few of us might not work at all for the community and we definitely would want to make their life easy by providing an easy access to the community channel.
> 
> For the survey, looks good, maybe providing the third option of "Maybe" with a comment box for the people to write their comments, if slack doesn't work for them, could help us getting more info? I am sure there are better ways to incorporate the users feedback, but just a thought!
> 
> Regards,
> Fatema
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org <mailto:zeek at zeek.org>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek <http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek>_______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191015/cd3739bc/attachment.html 

From jan.grashoefer at gmail.com  Wed Oct 16 06:18:53 2019
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Wed, 16 Oct 2019 15:18:53 +0200
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <CAOtSMjZOTijSTnF9x00UkWqdmeoLG2d13+Weg0pZt4-cqhedYg@mail.gmail.com>
References: <CAJhOzupLoFGh7RYR_kOCcEHaqBVE5S7OX1sf1BRf=yZ6TQ5QFA@mail.gmail.com>
	<63e702a9-e218-412f-1d74-bf1ec2cc09b9@gmail.com>
	<CAJ6bFK3Mw1Ajsfjjxr+cAE0NUpedPY_j9+uEcrG9E42mYm5wjA@mail.gmail.com>
	<CAJhOzuq1X4Jm-FTwK8_knku5wO5WD1_Mf1N+_KYLthcZs0JAHQ@mail.gmail.com>
	<CAEZw2byUJG2V6ds+ewnwue-BpZkhdbfMEDwT6Z-A9b0PtuCicQ@mail.gmail.com>
	<CAOtSMjZOTijSTnF9x00UkWqdmeoLG2d13+Weg0pZt4-cqhedYg@mail.gmail.com>
Message-ID: <7940adbd-7086-544c-373b-8ed693d6f653@gmail.com>

Hi Richard,

On 15/10/2019 15:46, Richard Bejtlich wrote:
> I checked it out via the Riot Web client, but I couldn't tell if the
> bridge with the #bro IRC channel worked or not. Furthermore, bridging the
> channel required having a registered nick with Freenode, which I had to
> re-activate. Eventually I was able to access Freenode using their Web chat,
> but I don't know if I was able to access #bro as it was quiet.

in fact it worked assuming you joined as taosecurity. However, you 
already followed an advanced approach by using a bridge to join the IRC 
channel. The fact that you required a registered nick is due to the fact 
that the #bro IRC channel has the +r mode.

> This is the sort of friction that causes new people to give up on chatting
> about Zeek. Slack at least makes it very easy for new users to learn about
> what happens in a channel.

So does Matrix, you just tried to join an IRC channel using Matrix. A 
Matrix room essentially works like a Slack channel. An IRC bridge is an 
additional feature to keep IRC folks in the loop (kind of backward 
compatibility if you like).

> Also, is there a concept of chat history in Matrix? That's one of the best
> features of Slack, in my opinion. Reading back through time is a great way
> to learn.

Of course Matrix rooms provide a history (e.g. I saw your test message 
although I was not part of the room when you sent it). Furthermore, 
there are different types of bridges. Without going into the details, I 
just created a room in Matrix (#zeek:matrix.org) and bridged it to 
#zeek-test on freenode. If you join that room via Matrix you have access 
to the complete history (including messages sent via IRC). If people 
would like to test additional integrations (Slack, GitHub, RSS, Gitter, 
Discord) just let me know.

The following link should bring you to the test room:
https://riot.im/app/#/room/#zeek:matrix.org

Jan

From akgraner at corelight.com  Wed Oct 16 06:18:59 2019
From: akgraner at corelight.com (Amber Graner)
Date: Wed, 16 Oct 2019 08:18:59 -0500
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <EC6DD8FC-6856-4707-BDE8-10ED1029B0C9@gmail.com>
References: <CACX0rURsNPYSCozicOLcNKMKJ7MtGP8MH2OMCR-3SjcYuu39Vw@mail.gmail.com>
	<CAPy2kFYeD5V_TkXDKRYxz0tO-U7B7P5U0YrzHJ-WvQnRpK80Yg@mail.gmail.com>
	<EC6DD8FC-6856-4707-BDE8-10ED1029B0C9@gmail.com>
Message-ID: <CAJhOzurKhVz_+Z_QBOdnhjxZvEeKKDy9O-Gag0_huZsE+WUC6A@mail.gmail.com>

Thanks Matt, great article suggestion!

Yes, Discourse is a lot more tgan chat. And I'm very familiar with several
communities who use, have converted to it recently, or who are in that
process.

I'm thrilled to see all the suggestions and input here.

This is great!!

Thanks,
Amber

On Tue, Oct 15, 2019, 20:10 Matt Trostel <tet68mt at gmail.com> wrote:

> I agree with Mike and Fatema.
>
> I?d also like to clarify the Discourse is not just chat. It is more
> analogous to a forum system. It also has Slack integration, though I?ll
> admit I don?t know to what extent. Maybe that could prove to be a good
> middle ground? They also offer free hosting for open source projects.
>
> I like the idea of finding a forum thread versus a mail archive through a
> web search. They are easier to follow in my opinion.
>
> I think it is also important to note that whatever comes of this, we
> aren?t bound to the specific platform. As a community, if we recognize that
> this is a learning experience on how best to organize and communicate --
> adoption and change will be smooth whenever it is necessary.
>
> I found this article interesting and a good take on the subject. It?s
> worth the read.
>
> https://blog.discourse.org/2018/04/effectively-using-discourse-together-with-group-chat/
>
> ? Matt Trostel
>
>
> On Oct 15, 2019, at 12:04, Mike Dopheide <dopheide at gmail.com> wrote:
>
> I think Fatema is spot on here.  It doesn't matter what any of us would
> prefer in a perfect world.
>
> If any random open source standard would work, then we'd have a bunch of
> people in irc asking questions.  Any solution that requires people to
> install a different app or have an extra browser tab open is likely doomed
> to a similar fate, but if we can hook into an app like Slack, that they
> already have open, we have a better chance of staying connected after
> reboots and garnering participation via subtle activity notifications.
>
> -Dop
>
>
>
>
>
> On Tue, Oct 15, 2019 at 9:26 AM fatema bannatwala <
> fatema.bannatwala at gmail.com> wrote:
>
>> Thanks Amber for setting up the poll for the audience input.
>> All the points mentioned regarding pros and cons are noted, and think one
>> of the ways, to see what will work best for the community to stay connected
>> via a communication channel, is to ask for their input. What works for us
>> or few of us might not work at all for the community and we definitely
>> would want to make their life easy by providing an easy access to the
>> community channel.
>>
>> For the survey, looks good, maybe providing the third option of "Maybe"
>> with a comment box for the people to write their comments, if slack doesn't
>> work for them, could help us getting more info? I am sure there are better
>> ways to incorporate the users feedback, but just a thought!
>>
>> Regards,
>> Fatema
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>> <http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek>
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191016/a2897fee/attachment-0001.html 

From jan.grashoefer at gmail.com  Wed Oct 16 06:23:14 2019
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Wed, 16 Oct 2019 15:23:14 +0200
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <CAPy2kFYeD5V_TkXDKRYxz0tO-U7B7P5U0YrzHJ-WvQnRpK80Yg@mail.gmail.com>
References: <CACX0rURsNPYSCozicOLcNKMKJ7MtGP8MH2OMCR-3SjcYuu39Vw@mail.gmail.com>
	<CAPy2kFYeD5V_TkXDKRYxz0tO-U7B7P5U0YrzHJ-WvQnRpK80Yg@mail.gmail.com>
Message-ID: <ac41adb8-06bb-67b9-fbbc-9eddfc932128@gmail.com>

Just a side note: I do understand your concerns but I don't buy your 
argument. We do not live in a perfect world, so let's do our best to 
improve the situation. E.g. by using a "random open source" approach in 
popular projects to make it a standard. Furthermore, comparing IRC to 
modern communication approaches is comparing apples to oranges. Any of 
the mentioned alternatives will severely simplify participation and come 
with a bunch of new features. So at the end the question would be 
whether using an open source project is worth the cost of an additional 
browser tab. I would vote for yes.

Jan

P.S.: Matrix provides a Slack bridge ;)

On 15/10/2019 19:04, Mike Dopheide wrote:
> I think Fatema is spot on here.  It doesn't matter what any of us would
> prefer in a perfect world.
> 
> If any random open source standard would work, then we'd have a bunch of
> people in irc asking questions.  Any solution that requires people to
> install a different app or have an extra browser tab open is likely doomed
> to a similar fate, but if we can hook into an app like Slack, that they
> already have open, we have a better chance of staying connected after
> reboots and garnering participation via subtle activity notifications.
> 
> -Dop
> 
> 
> 
> 
> 
> On Tue, Oct 15, 2019 at 9:26 AM fatema bannatwala <
> fatema.bannatwala at gmail.com> wrote:
> 
>> Thanks Amber for setting up the poll for the audience input.
>> All the points mentioned regarding pros and cons are noted, and think one
>> of the ways, to see what will work best for the community to stay connected
>> via a communication channel, is to ask for their input. What works for us
>> or few of us might not work at all for the community and we definitely
>> would want to make their life easy by providing an easy access to the
>> community channel.
>>
>> For the survey, looks good, maybe providing the third option of "Maybe"
>> with a comment box for the people to write their comments, if slack doesn't
>> work for them, could help us getting more info? I am sure there are better
>> ways to incorporate the users feedback, but just a thought!
>>
>> Regards,
>> Fatema
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> 
> 
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> 

From richard at corelight.com  Wed Oct 16 07:14:17 2019
From: richard at corelight.com (Richard Bejtlich)
Date: Wed, 16 Oct 2019 10:14:17 -0400
Subject: [Zeek] Official Zeek Community Slack Channel Poll
In-Reply-To: <7940adbd-7086-544c-373b-8ed693d6f653@gmail.com>
References: <CAJhOzupLoFGh7RYR_kOCcEHaqBVE5S7OX1sf1BRf=yZ6TQ5QFA@mail.gmail.com>
	<63e702a9-e218-412f-1d74-bf1ec2cc09b9@gmail.com>
	<CAJ6bFK3Mw1Ajsfjjxr+cAE0NUpedPY_j9+uEcrG9E42mYm5wjA@mail.gmail.com>
	<CAJhOzuq1X4Jm-FTwK8_knku5wO5WD1_Mf1N+_KYLthcZs0JAHQ@mail.gmail.com>
	<CAEZw2byUJG2V6ds+ewnwue-BpZkhdbfMEDwT6Z-A9b0PtuCicQ@mail.gmail.com>
	<CAOtSMjZOTijSTnF9x00UkWqdmeoLG2d13+Weg0pZt4-cqhedYg@mail.gmail.com>
	<7940adbd-7086-544c-373b-8ed693d6f653@gmail.com>
Message-ID: <CAOtSMjaFd2r7fqHjyQOtU9VqPeL86rxvtB0qoYDkXQzetzBd3w@mail.gmail.com>

Thank you Jan, I will check it out!

Richard

On Wed, Oct 16, 2019 at 9:18 AM Jan Grash?fer <jan.grashoefer at gmail.com>
wrote:

> Hi Richard,
>
> On 15/10/2019 15:46, Richard Bejtlich wrote:
> > I checked it out via the Riot Web client, but I couldn't tell if the
> > bridge with the #bro IRC channel worked or not. Furthermore, bridging the
> > channel required having a registered nick with Freenode, which I had to
> > re-activate. Eventually I was able to access Freenode using their Web
> chat,
> > but I don't know if I was able to access #bro as it was quiet.
>
> in fact it worked assuming you joined as taosecurity. However, you
> already followed an advanced approach by using a bridge to join the IRC
> channel. The fact that you required a registered nick is due to the fact
> that the #bro IRC channel has the +r mode.
>
> > This is the sort of friction that causes new people to give up on
> chatting
> > about Zeek. Slack at least makes it very easy for new users to learn
> about
> > what happens in a channel.
>
> So does Matrix, you just tried to join an IRC channel using Matrix. A
> Matrix room essentially works like a Slack channel. An IRC bridge is an
> additional feature to keep IRC folks in the loop (kind of backward
> compatibility if you like).
>
> > Also, is there a concept of chat history in Matrix? That's one of the
> best
> > features of Slack, in my opinion. Reading back through time is a great
> way
> > to learn.
>
> Of course Matrix rooms provide a history (e.g. I saw your test message
> although I was not part of the room when you sent it). Furthermore,
> there are different types of bridges. Without going into the details, I
> just created a room in Matrix (#zeek:matrix.org) and bridged it to
> #zeek-test on freenode. If you join that room via Matrix you have access
> to the complete history (including messages sent via IRC). If people
> would like to test additional integrations (Slack, GitHub, RSS, Gitter,
> Discord) just let me know.
>
> The following link should bring you to the test room:
> https://riot.im/app/#/room/#zeek:matrix.org
>
> Jan
>


-- 
Richard Bejtlich
Principal Security Strategist, Corelight
https://corelight.blog/author/richardbejtlich/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191016/7d32b91e/attachment.html 

From cherdt at umn.edu  Wed Oct 16 09:20:40 2019
From: cherdt at umn.edu (Chris Herdt)
Date: Wed, 16 Oct 2019 11:20:40 -0500
Subject: [Zeek] log delays and logger CPUs
Message-ID: <CAO8nG4fwQsBGHqfTSoKAtFrZvN5u1jKGY0_M8e5CroWTxf10Cg@mail.gmail.com>

I have a cluster running Bro 2.6.4. One host runs a manager and logger, 8
other hosts run proxy and worker nodes.

Lately the logger node has not been able to keep up with the logs, and I've
noticed that the most recent entries in the current/conn.log are
significantly delayed (I've seen delays as high as 90 minutes).

The logger process has maxed out CPU usage on core 1. The node.cfg file
specifies 8 CPU cores (all on the same NUMA node as the NVMe drive where
the logs are written):

[logger]
type=logger
host=bromanager-01.umn.edu
pin_cpus=1,3,5,7,9,11,13,15

`broctl nodes` shows that only 1 CPU core is pinned:

/usr/local/bro/bin/broctl nodes
         logger - addr=10.x.x.x aux_scripts= brobase= count=1 env_vars=
ether= host=bromanager-01.umn.edu interface= lb_interfaces= lb_method=
lb_procs= name=logger pin_cpus=1 test_mykey= type=logger zone_id=
...

Can pin_cpus be used with a logger node? Any other suggestions for
improving logger performance?


-- 
Chris Herdt
UIS Systems Administrator
cherdt at umn.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191016/88441318/attachment.html 

From justin at corelight.com  Wed Oct 16 10:15:17 2019
From: justin at corelight.com (Justin Azoff)
Date: Wed, 16 Oct 2019 13:15:17 -0400
Subject: [Zeek] log delays and logger CPUs
In-Reply-To: <CAO8nG4fwQsBGHqfTSoKAtFrZvN5u1jKGY0_M8e5CroWTxf10Cg@mail.gmail.com>
References: <CAO8nG4fwQsBGHqfTSoKAtFrZvN5u1jKGY0_M8e5CroWTxf10Cg@mail.gmail.com>
Message-ID: <CAPfnCuhhsF0h0pLdOg30mD_+mnfdRTXVuMrXNfd0r=hqMn=BHA@mail.gmail.com>

On Wed, Oct 16, 2019 at 12:23 PM Chris Herdt <cherdt at umn.edu> wrote:

> I have a cluster running Bro 2.6.4. One host runs a manager and logger, 8
> other hosts run proxy and worker nodes.
>
> Lately the logger node has not been able to keep up with the logs, and
> I've noticed that the most recent entries in the current/conn.log are
> significantly delayed (I've seen delays as high as 90 minutes).
>
> The logger process has maxed out CPU usage on core 1. The node.cfg file
> specifies 8 CPU cores (all on the same NUMA node as the NVMe drive where
> the logs are written):
>
> [logger]
> type=logger
> host=bromanager-01.umn.edu
> pin_cpus=1,3,5,7,9,11,13,15
>
> `broctl nodes` shows that only 1 CPU core is pinned:
>


> Can pin_cpus be used with a logger node? Any other suggestions for
> improving logger performance?
>

Ah.. pin_cpus is more intended to work with multiple worker processes.
It's definitely doing the wrong thing in your case.  It's pinning the first
logger (out of one total) to core 1, and then it never uses 3,5,7....
You're better off removing that setting.  It should run fine across all
cores, even with the numa hit.. the volume of logs that would go across the
numa bus would only be a small fraction of the total bandwidth.

You could still pin it to those cores, but you'd have to do it manually
using taskset for now.  This is probably something that could be fixed in
broctl(zeekctl) to better handle pin_cpus option when only a single process
is being started.

-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191016/8ce2ecd0/attachment.html 

From michalpurzynski1 at gmail.com  Wed Oct 16 10:53:59 2019
From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=)
Date: Wed, 16 Oct 2019 10:53:59 -0700
Subject: [Zeek] log delays and logger CPUs
In-Reply-To: <CAPfnCuhhsF0h0pLdOg30mD_+mnfdRTXVuMrXNfd0r=hqMn=BHA@mail.gmail.com>
References: <CAO8nG4fwQsBGHqfTSoKAtFrZvN5u1jKGY0_M8e5CroWTxf10Cg@mail.gmail.com>
	<CAPfnCuhhsF0h0pLdOg30mD_+mnfdRTXVuMrXNfd0r=hqMn=BHA@mail.gmail.com>
Message-ID: <CAJ6bFK1Q4gSLQwaT7NyjroXDhfHUD4JjRFDxChzkv_qoGkN__w@mail.gmail.com>

So there's a couple of things you could do.

1. Indeed using pin_cpus for the manager or the logging will result in all
threads being pinned to the same CPU, definitely not ideal and worth
filling a bug

2. You could (per Justin's idea from Zeek Week) use the path func and write
a script that will split biggest log files into two or more files - and
each will get a separate thread

https://github.com/michalpurzynski/bro-gramming/blob/master/filter_noise_dns.zeek
https://github.com/michalpurzynski/bro-gramming/blob/master/filter_noise_http.zeek
https://github.com/michalpurzynski/bro-gramming/blob/master/filter_noise_intel.zeek

3. And maybe implement some filtering for traffic you don't care for, if
there is any, examples here

https://github.com/michalpurzynski/bro-gramming/blob/master/filter_noise_files.zeek
https://github.com/michalpurzynski/bro-gramming/blob/master/filter_noise_mysql.zeek
https://github.com/michalpurzynski/bro-gramming/blob/master/filter_noise_ssl.zeek
https://github.com/michalpurzynski/bro-gramming/blob/master/filter_noise_x509.zeek
https://github.com/michalpurzynski/bro-gramming/blob/master/filter_noise_conn.zeek
https://github.com/michalpurzynski/bro-gramming/blob/master/filter_input.zeek


On Wed, Oct 16, 2019 at 10:17 AM Justin Azoff <justin at corelight.com> wrote:

>
>
> On Wed, Oct 16, 2019 at 12:23 PM Chris Herdt <cherdt at umn.edu> wrote:
>
>> I have a cluster running Bro 2.6.4. One host runs a manager and logger, 8
>> other hosts run proxy and worker nodes.
>>
>> Lately the logger node has not been able to keep up with the logs, and
>> I've noticed that the most recent entries in the current/conn.log are
>> significantly delayed (I've seen delays as high as 90 minutes).
>>
>> The logger process has maxed out CPU usage on core 1. The node.cfg file
>> specifies 8 CPU cores (all on the same NUMA node as the NVMe drive where
>> the logs are written):
>>
>> [logger]
>> type=logger
>> host=bromanager-01.umn.edu
>> pin_cpus=1,3,5,7,9,11,13,15
>>
>> `broctl nodes` shows that only 1 CPU core is pinned:
>>
>
>
>> Can pin_cpus be used with a logger node? Any other suggestions for
>> improving logger performance?
>>
>
> Ah.. pin_cpus is more intended to work with multiple worker processes.
> It's definitely doing the wrong thing in your case.  It's pinning the first
> logger (out of one total) to core 1, and then it never uses 3,5,7....
> You're better off removing that setting.  It should run fine across all
> cores, even with the numa hit.. the volume of logs that would go across the
> numa bus would only be a small fraction of the total bandwidth.
>
> You could still pin it to those cores, but you'd have to do it manually
> using taskset for now.  This is probably something that could be fixed in
> broctl(zeekctl) to better handle pin_cpus option when only a single process
> is being started.
>
> --
> Justin
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191016/8c28ee5f/attachment-0001.html 

From henridf at gmail.com  Wed Oct 16 12:45:46 2019
From: henridf at gmail.com (Henri Dubois-Ferriere)
Date: Wed, 16 Oct 2019 21:45:46 +0200
Subject: [Zeek] printing stream columns
Message-ID: <CABcA1ruzNDefM16QHP5yJnOqXL4h=q0=LQwDLatCLaKcHS2aAg@mail.gmail.com>

I'm trying to print the record type for each log stream at startup.
Something like:

 for ( id in Log::active_streams ) {
                 local stream = Log::active_streams[id];
                 print stream$path, stream$columns;
}

doesn't work because $columns is a record type, and gets stringified "<no
value description>".

Is there a way to do this in zeek script?

Thanks,
Henri
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191016/f37d154a/attachment.html 

From anthony.kasza at gmail.com  Wed Oct 16 13:27:16 2019
From: anthony.kasza at gmail.com (anthony kasza)
Date: Wed, 16 Oct 2019 14:27:16 -0600
Subject: [Zeek] printing stream columns
In-Reply-To: <CABcA1ruzNDefM16QHP5yJnOqXL4h=q0=LQwDLatCLaKcHS2aAg@mail.gmail.com>
References: <CABcA1ruzNDefM16QHP5yJnOqXL4h=q0=LQwDLatCLaKcHS2aAg@mail.gmail.com>
Message-ID: <CAEZw2bw6w-p=XeQ-Qx6+4BsivnFi2hQU7o5vZ0ChUcymSE0eEw@mail.gmail.com>

Hi Henri,

Great question.
The logging framework is extremely flexible and allows for log stream
columns to dynamically change during run time. This means at startup, the
bro_init() event, Zeek may not know all the columns of all the logs. Here's
a script I wrote for you which sort of answers your question. If you have
more questions about it, just reach back out to the list.

-AK


function pfunk(rec: any): bool {
  print type_name(rec);
  return T;
}

event bro_init() {
  for (id in Log::active_streams) {
    for (fname in Log::get_filter_names(id)) {
      local filter: Log::Filter;
      filter = Log::get_filter(id, fname);
      filter$pred = pfunk;
      Log::add_filter(id, filter);
    }
  }
}


On Wed, Oct 16, 2019, 13:48 Henri Dubois-Ferriere <henridf at gmail.com> wrote:

> I'm trying to print the record type for each log stream at startup.
> Something like:
>
>  for ( id in Log::active_streams ) {
>                  local stream = Log::active_streams[id];
>                  print stream$path, stream$columns;
> }
>
> doesn't work because $columns is a record type, and gets stringified "<no
> value description>".
>
> Is there a way to do this in zeek script?
>
> Thanks,
> Henri
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191016/65e0ca35/attachment.html 

From jsiwek at corelight.com  Wed Oct 16 13:37:34 2019
From: jsiwek at corelight.com (Jon Siwek)
Date: Wed, 16 Oct 2019 13:37:34 -0700
Subject: [Zeek] printing stream columns
In-Reply-To: <CABcA1ruzNDefM16QHP5yJnOqXL4h=q0=LQwDLatCLaKcHS2aAg@mail.gmail.com>
References: <CABcA1ruzNDefM16QHP5yJnOqXL4h=q0=LQwDLatCLaKcHS2aAg@mail.gmail.com>
Message-ID: <CAMzgZ0KkEk--tNxm7f2dBWuWqCYo8hddfE4DFLgzcOXcV+nAyQ@mail.gmail.com>

On Wed, Oct 16, 2019 at 12:48 PM Henri Dubois-Ferriere
<henridf at gmail.com> wrote:
>
> I'm trying to print the record type for each log stream at startup. Something like:
>
>  for ( id in Log::active_streams ) {
>                  local stream = Log::active_streams[id];
>                  print stream$path, stream$columns;
> }
>
> doesn't work because $columns is a record type, and gets stringified "<no value description>".

Zeek 3.0 should give better descriptions for types.  This was the
relevant patch which is not in any 2.6.x version:

  https://github.com/bro/bro/commit/1f450c05102be6dd7ebcc2c5901d5a3a231cd675

This script may also help demonstrate things related to what you're
trying to do:

  https://gist.github.com/jsiwek/f843b3321f4227b6ec32d110424ebf70

It prints field descriptions of all logs either to stdout or a CSV
file.  Example command:

  ZEEK_ALLOW_INIT_ERRORS=1 zeek print-log-info.bro PrintLogs::csv=F

Sample of output:

known_hosts.log | Hosts with complete TCP handshakes
  ts: time - The timestamp at which the host was detected.
  host: addr - The address that was detected originating or responding
to a TCP connection.

- Jon

From henridf at gmail.com  Wed Oct 16 13:47:05 2019
From: henridf at gmail.com (Henri Dubois-Ferriere)
Date: Wed, 16 Oct 2019 22:47:05 +0200
Subject: [Zeek] printing stream columns
In-Reply-To: <CAMzgZ0KkEk--tNxm7f2dBWuWqCYo8hddfE4DFLgzcOXcV+nAyQ@mail.gmail.com>
References: <CABcA1ruzNDefM16QHP5yJnOqXL4h=q0=LQwDLatCLaKcHS2aAg@mail.gmail.com>
	<CAMzgZ0KkEk--tNxm7f2dBWuWqCYo8hddfE4DFLgzcOXcV+nAyQ@mail.gmail.com>
Message-ID: <CABcA1ruH2qu-3R96UVQNbCJxN+FqdycNze13g2kRg7rSpZ_VbQ@mail.gmail.com>

Thanks Jon and Anthony for the quick responses! print-log-info.bro looks
promising for what I'm trying to do.

On Wed, 16 Oct 2019 at 22:37, Jon Siwek <jsiwek at corelight.com> wrote:

> On Wed, Oct 16, 2019 at 12:48 PM Henri Dubois-Ferriere
> <henridf at gmail.com> wrote:
> >
> > I'm trying to print the record type for each log stream at startup.
> Something like:
> >
> >  for ( id in Log::active_streams ) {
> >                  local stream = Log::active_streams[id];
> >                  print stream$path, stream$columns;
> > }
> >
> > doesn't work because $columns is a record type, and gets stringified
> "<no value description>".
>
> Zeek 3.0 should give better descriptions for types.  This was the
> relevant patch which is not in any 2.6.x version:
>
>
> https://github.com/bro/bro/commit/1f450c05102be6dd7ebcc2c5901d5a3a231cd675
>
> This script may also help demonstrate things related to what you're
> trying to do:
>
>   https://gist.github.com/jsiwek/f843b3321f4227b6ec32d110424ebf70
>
> It prints field descriptions of all logs either to stdout or a CSV
> file.  Example command:
>
>   ZEEK_ALLOW_INIT_ERRORS=1 zeek print-log-info.bro PrintLogs::csv=F
>
> Sample of output:
>
> known_hosts.log | Hosts with complete TCP handshakes
>   ts: time - The timestamp at which the host was detected.
>   host: addr - The address that was detected originating or responding
> to a TCP connection.
>
> - Jon
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191016/2f0a86c7/attachment.html 

From yizhu at shapesecurity.com  Thu Oct 17 14:49:00 2019
From: yizhu at shapesecurity.com (Yi Zhu)
Date: Thu, 17 Oct 2019 14:49:00 -0700
Subject: [Zeek] zeek drops requests with large harders
Message-ID: <CAKNjQ_9=Q8sgLZSbNUdN6vefy3vmcJMC3aprsy2F9PLX3_ZwrA@mail.gmail.com>

Hi,

We are using zeek 3.0.0.
We found that zeek drops requests with large harders.
Is it possible to make zeek catch such requests?
For example,

curl -k -i -vv -X GET http://test/login \

-H 'User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1;
Trident/4.0; YPC 3.2.0; SearchSystem6829992239; SearchSystem9616306563;
SearchSystem6017393645; SearchSystem5219240075; SearchSystem2768350104;
SearchSystem6919669052; SearchSystem1986739074; SearchSystem1555480186;
SearchSystem3376893470; SearchSystem9530642569; SearchSystem4877790286;
SearchSystem8104932799; SearchSystem2313134663; SearchSystem1545325372;
SearchSystem7742471461; SearchSystem9092363703; SearchSystem6992236221;
SearchSystem3507700306; SearchSystem1129983453; SearchSystem1077927937;
SearchSystem2297142691; SearchSystem7813572891; SearchSystem5668754497;
SearchSystem6220295595; SearchSystem4157940963; SearchSystem7656671655;
SearchSystem2865656762; SearchSystem6520604676; SearchSystem4960161466;
.NET CLR 1.1.4322; .NET CLR 2.0.50727; Hotbar 10.2.232.0;
SearchSystem9616306563; SearchSystem6017393645; SearchSystem5219240075;
SearchSystem2768350104; SearchSystem6919669052; SearchSystem1986739074;
SearchSystem1555480186; SearchSystem3376893470; SearchSystem9530642569;
SearchSystem4877790286; SearchSystem8104932799; SearchSystem2313134663;
SearchSystem1545325372; SearchSystem7742471461; SearchSystem9092363703;
SearchSystem6992236221; SearchSystem3507700306; SearchSystem1129983453;
SearchSystem1077927937; SearchSystem2297142691; SearchSystem7813572891;
SearchSystem5668754497; SearchSystem6220295595; SearchSystem4157940963;
SearchSystem7656671655; SearchSystem2865656762; SearchSystem6520604676;
SearchSystem4960161466; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)' \

-H 'Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \

-H 'Accept-Language: en-US,en;q=0.5' --compressed \

-H 'Content-Type: application/x-www-form-urlencoded' \

-H 'Connection: keep-alive' \

-H 'Upgrade-Insecure-Requests: 1' \

-H 'Pragma: no-cache' \

-H 'Cache-Control: no-cache' \

-H 'True-Client-Ip: 2.18.114.25' \

--data 'user=dasD


After I reduced the header size, zeek can catch it.

For example,

curl -k -i -vv -X GET http://test/login \
-H 'User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1;
Trident/4.0; YPC 3.2.0; SearchSystem6829992239; SearchSystem9616306563;
SearchSystem6017393645; SearchSystem5219240075; SearchSystem2768350104;
SearchSystem6919669052; SearchSystem1986739074; SearchSystem1555480186;
SearchSystem3376893470; SearchSystem9530642569; SearchSystem4877790286;
SearchSystem8104932799; SearchSystem2313134663; SearchSystem1545325373;
SearchSystem7742471461;
SearchSystem2313134663; SearchSystem1545325372; SearchSystem7742471462;
SearchSystem2313134663; SearchSystem1545325372; SearchSystem7742471461;
SearchSystem2313134663;x)' \
-H 'Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
-H 'Accept-Language: en-US,en;q=0.5' --compressed \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Connection: keep-alive' \
-H 'Upgrade-Insecure-Requests: 1' \
-H 'Pragma: no-cache' \
-H 'Cache-Control: no-cache' \
-H 'True-Client-Ip: 2.18.114.25' \
--data 'user=dasD'


Thanks,

Yi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191017/5fbacfcd/attachment.html 

From justin at corelight.com  Thu Oct 17 16:47:59 2019
From: justin at corelight.com (Justin Azoff)
Date: Thu, 17 Oct 2019 19:47:59 -0400
Subject: [Zeek] zeek drops requests with large harders
In-Reply-To: <CAKNjQ_9=Q8sgLZSbNUdN6vefy3vmcJMC3aprsy2F9PLX3_ZwrA@mail.gmail.com>
References: <CAKNjQ_9=Q8sgLZSbNUdN6vefy3vmcJMC3aprsy2F9PLX3_ZwrA@mail.gmail.com>
Message-ID: <CAPfnCugoFh6KZuMkB4kzfJQMw2NShjLucAQ+JS5+oQirFNr6=A@mail.gmail.com>

Is that request not on port 80?  You are probably hitting
https://github.com/zeek/zeek/issues/343  Does the problem go away if you
set dpd_buffer_size to 4096 ?

On Thu, Oct 17, 2019 at 5:51 PM Yi Zhu <yizhu at shapesecurity.com> wrote:

> Hi,
>
> We are using zeek 3.0.0.
> We found that zeek drops requests with large harders.
> Is it possible to make zeek catch such requests?
> For example,
>
> curl -k -i -vv -X GET http://test/login \
>
> -H 'User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1;
> Trident/4.0; YPC 3.2.0; SearchSystem6829992239; SearchSystem9616306563;
> SearchSystem6017393645; SearchSystem5219240075; SearchSystem2768350104;
> SearchSystem6919669052; SearchSystem1986739074; SearchSystem1555480186;
> SearchSystem3376893470; SearchSystem9530642569; SearchSystem4877790286;
> SearchSystem8104932799; SearchSystem2313134663; SearchSystem1545325372;
> SearchSystem7742471461; SearchSystem9092363703; SearchSystem6992236221;
> SearchSystem3507700306; SearchSystem1129983453; SearchSystem1077927937;
> SearchSystem2297142691; SearchSystem7813572891; SearchSystem5668754497;
> SearchSystem6220295595; SearchSystem4157940963; SearchSystem7656671655;
> SearchSystem2865656762; SearchSystem6520604676; SearchSystem4960161466;
> .NET CLR 1.1.4322; .NET CLR 2.0.50727; Hotbar 10.2.232.0;
> SearchSystem9616306563; SearchSystem6017393645; SearchSystem5219240075;
> SearchSystem2768350104; SearchSystem6919669052; SearchSystem1986739074;
> SearchSystem1555480186; SearchSystem3376893470; SearchSystem9530642569;
> SearchSystem4877790286; SearchSystem8104932799; SearchSystem2313134663;
> SearchSystem1545325372; SearchSystem7742471461; SearchSystem9092363703;
> SearchSystem6992236221; SearchSystem3507700306; SearchSystem1129983453;
> SearchSystem1077927937; SearchSystem2297142691; SearchSystem7813572891;
> SearchSystem5668754497; SearchSystem6220295595; SearchSystem4157940963;
> SearchSystem7656671655; SearchSystem2865656762; SearchSystem6520604676;
> SearchSystem4960161466; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)' \
>
> -H 'Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
>
> -H 'Accept-Language: en-US,en;q=0.5' --compressed \
>
> -H 'Content-Type: application/x-www-form-urlencoded' \
>
> -H 'Connection: keep-alive' \
>
> -H 'Upgrade-Insecure-Requests: 1' \
>
> -H 'Pragma: no-cache' \
>
> -H 'Cache-Control: no-cache' \
>
> -H 'True-Client-Ip: 2.18.114.25' \
>
> --data 'user=dasD
>
>
> After I reduced the header size, zeek can catch it.
>
> For example,
>
> curl -k -i -vv -X GET http://test/login \
> -H 'User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1;
> Trident/4.0; YPC 3.2.0; SearchSystem6829992239; SearchSystem9616306563;
> SearchSystem6017393645; SearchSystem5219240075; SearchSystem2768350104;
> SearchSystem6919669052; SearchSystem1986739074; SearchSystem1555480186;
> SearchSystem3376893470; SearchSystem9530642569; SearchSystem4877790286;
> SearchSystem8104932799; SearchSystem2313134663; SearchSystem1545325373;
> SearchSystem7742471461;
> SearchSystem2313134663; SearchSystem1545325372; SearchSystem7742471462;
> SearchSystem2313134663; SearchSystem1545325372; SearchSystem7742471461;
> SearchSystem2313134663;x)' \
> -H 'Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
> -H 'Accept-Language: en-US,en;q=0.5' --compressed \
> -H 'Content-Type: application/x-www-form-urlencoded' \
> -H 'Connection: keep-alive' \
> -H 'Upgrade-Insecure-Requests: 1' \
> -H 'Pragma: no-cache' \
> -H 'Cache-Control: no-cache' \
> -H 'True-Client-Ip: 2.18.114.25' \
> --data 'user=dasD'
>
>
> Thanks,
>
> Yi
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191017/477329fd/attachment.html 

From dopheide at gmail.com  Thu Oct 17 17:21:51 2019
From: dopheide at gmail.com (Mike Dopheide)
Date: Thu, 17 Oct 2019 19:21:51 -0500
Subject: [Zeek] myricom driver broken w/ new CentOS 7 kernel
Message-ID: <CAPy2kFbY9oy1_GGfizO7FW1JZgP-kt3beVeaSgrKUH172=ayyg@mail.gmail.com>

This is mostly just a heads-up for folks that use the myri_snf driver.
First, I have to thank Patrick Storm for bringing this to our attention
last week at the conference.

On CentOS 7, the latest 3.10.0-1062.1.2 kernel is going to break your
ability to build the Myricom driver, versions < 3.0.18.  If you don't touch
the existing driver, you may be fine.  If you try to rebuild the driver,
the compile will fail.

There is a myri_snf-3.0.19 available from CSPi, but it's not currently on
the public download page.  With this version, 2/3 systems we've tried it on
work perfectly fine.  The 3rd has so far been a big mystery.  If anyone
tries it and ends up with workers dying with an error like the following
from 'zeekctl diag' please let me know:

fatal error: problem with interface myricom::p3p2:13 (No such device)

I've got a line of communication open with CSPi.  Separately, we have a
potential workaround, but we also don't understand how it's working.  :)

-Dop
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191017/27a4c862/attachment-0001.html 

From ericooi at gmail.com  Thu Oct 17 18:44:27 2019
From: ericooi at gmail.com (ericooi at gmail.com)
Date: Thu, 17 Oct 2019 20:44:27 -0500
Subject: [Zeek] zeek drops requests with large harders
In-Reply-To: <CAPfnCugoFh6KZuMkB4kzfJQMw2NShjLucAQ+JS5+oQirFNr6=A@mail.gmail.com>
References: <CAKNjQ_9=Q8sgLZSbNUdN6vefy3vmcJMC3aprsy2F9PLX3_ZwrA@mail.gmail.com>
	<CAPfnCugoFh6KZuMkB4kzfJQMw2NShjLucAQ+JS5+oQirFNr6=A@mail.gmail.com>
Message-ID: <19ACCF73-2702-469A-AD4C-7620E34EE36F@gmail.com>

Seems to capture fine when I do it on my instance of Zeek 3.0.  Perhaps you?re not capturing the full packet?

https://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html <https://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html>


> On Oct 17, 2019, at 6:47 PM, Justin Azoff <justin at corelight.com> wrote:
> 
> Is that request not on port 80?  You are probably hitting https://github.com/zeek/zeek/issues/343 <https://github.com/zeek/zeek/issues/343>  Does the problem go away if you set dpd_buffer_size to 4096 ?
> 
> On Thu, Oct 17, 2019 at 5:51 PM Yi Zhu <yizhu at shapesecurity.com <mailto:yizhu at shapesecurity.com>> wrote:
> Hi,
> 
> We are using zeek 3.0.0.
> We found that zeek drops requests with large harders.
> Is it possible to make zeek catch such requests?
> For example,
> curl -k -i -vv -X GET http://test/login <http://test/login> \
> -H 'User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; YPC 3.2.0; SearchSystem6829992239; SearchSystem9616306563; SearchSystem6017393645; SearchSystem5219240075; SearchSystem2768350104; SearchSystem6919669052; SearchSystem1986739074; SearchSystem1555480186; SearchSystem3376893470; SearchSystem9530642569; SearchSystem4877790286; SearchSystem8104932799; SearchSystem2313134663; SearchSystem1545325372; SearchSystem7742471461; SearchSystem9092363703; SearchSystem6992236221; SearchSystem3507700306; SearchSystem1129983453; SearchSystem1077927937; SearchSystem2297142691; SearchSystem7813572891; SearchSystem5668754497; SearchSystem6220295595; SearchSystem4157940963; SearchSystem7656671655; SearchSystem2865656762; SearchSystem6520604676; SearchSystem4960161466; .NET CLR 1.1.4322; .NET CLR 2.0.50727; Hotbar 10.2.232.0; SearchSystem9616306563; SearchSystem6017393645; SearchSystem5219240075; SearchSystem2768350104; SearchSystem6919669052; SearchSystem1986739074; SearchSystem1555480186; SearchSystem3376893470; SearchSystem9530642569; SearchSystem4877790286; SearchSystem8104932799; SearchSystem2313134663; SearchSystem1545325372; SearchSystem7742471461; SearchSystem9092363703; SearchSystem6992236221; SearchSystem3507700306; SearchSystem1129983453; SearchSystem1077927937; SearchSystem2297142691; SearchSystem7813572891; SearchSystem5668754497; SearchSystem6220295595; SearchSystem4157940963; SearchSystem7656671655; SearchSystem2865656762; SearchSystem6520604676; SearchSystem4960161466; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)' \
> -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
> -H 'Accept-Language: en-US,en;q=0.5' --compressed \
> -H 'Content-Type: application/x-www-form-urlencoded' \
> -H 'Connection: keep-alive' \
> -H 'Upgrade-Insecure-Requests: 1' \
> -H 'Pragma: no-cache' \
> -H 'Cache-Control: no-cache' \
> -H 'True-Client-Ip: 2.18.114.25' \
> --data 'user=dasD
> 
> After I reduced the header size, zeek can catch it.
> For example,
> curl -k -i -vv -X GET http://test/login <http://test/login> \
> -H 'User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; YPC 3.2.0; SearchSystem6829992239; SearchSystem9616306563; SearchSystem6017393645; SearchSystem5219240075; SearchSystem2768350104; SearchSystem6919669052; SearchSystem1986739074; SearchSystem1555480186; SearchSystem3376893470; SearchSystem9530642569; SearchSystem4877790286; SearchSystem8104932799; SearchSystem2313134663; SearchSystem1545325373; SearchSystem7742471461;
> SearchSystem2313134663; SearchSystem1545325372; SearchSystem7742471462;
> SearchSystem2313134663; SearchSystem1545325372; SearchSystem7742471461;
> SearchSystem2313134663;x)' \
> -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
> -H 'Accept-Language: en-US,en;q=0.5' --compressed \
> -H 'Content-Type: application/x-www-form-urlencoded' \
> -H 'Connection: keep-alive' \
> -H 'Upgrade-Insecure-Requests: 1' \
> -H 'Pragma: no-cache' \
> -H 'Cache-Control: no-cache' \
> -H 'True-Client-Ip: 2.18.114.25' \
> --data 'user=dasD'
> 
> Thanks,
> Yi
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org <mailto:zeek at zeek.org>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek <http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek>
> 
> -- 
> Justin
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191017/1ad88a80/attachment.html 

From justin at corelight.com  Fri Oct 18 06:55:28 2019
From: justin at corelight.com (Justin Azoff)
Date: Fri, 18 Oct 2019 09:55:28 -0400
Subject: [Zeek] zeek drops requests with large harders
In-Reply-To: <CAPfnCugoFh6KZuMkB4kzfJQMw2NShjLucAQ+JS5+oQirFNr6=A@mail.gmail.com>
References: <CAKNjQ_9=Q8sgLZSbNUdN6vefy3vmcJMC3aprsy2F9PLX3_ZwrA@mail.gmail.com>
	<CAPfnCugoFh6KZuMkB4kzfJQMw2NShjLucAQ+JS5+oQirFNr6=A@mail.gmail.com>
Message-ID: <CAPfnCuhegFpt0p_HJtNJ_zmB9Y1w9Y-K8GfRr0OwWL_8mFsb5w@mail.gmail.com>

I  took a closer look at those 2 commands, the first one sends 1900 bytes
or so while the 2nd one is just under 1024.  The default dpd buffer size
would definitely explain this issue if you were sending that request to a
port other than 80 or 8080 or one of the other default http ports.

On Thu, Oct 17, 2019 at 7:47 PM Justin Azoff <justin at corelight.com> wrote:

> Is that request not on port 80?  You are probably hitting
> https://github.com/zeek/zeek/issues/343  Does the problem go away if you
> set dpd_buffer_size to 4096 ?
>
> On Thu, Oct 17, 2019 at 5:51 PM Yi Zhu <yizhu at shapesecurity.com> wrote:
>
>> Hi,
>>
>> We are using zeek 3.0.0.
>> We found that zeek drops requests with large harders.
>> Is it possible to make zeek catch such requests?
>> For example,
>>
>> curl -k -i -vv -X GET http://test/login \
>>
>> -H 'User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1;
>> Trident/4.0; YPC 3.2.0; SearchSystem6829992239; SearchSystem9616306563;
>> SearchSystem6017393645; SearchSystem5219240075; SearchSystem2768350104;
>> SearchSystem6919669052; SearchSystem1986739074; SearchSystem1555480186;
>> SearchSystem3376893470; SearchSystem9530642569; SearchSystem4877790286;
>> SearchSystem8104932799; SearchSystem2313134663; SearchSystem1545325372;
>> SearchSystem7742471461; SearchSystem9092363703; SearchSystem6992236221;
>> SearchSystem3507700306; SearchSystem1129983453; SearchSystem1077927937;
>> SearchSystem2297142691; SearchSystem7813572891; SearchSystem5668754497;
>> SearchSystem6220295595; SearchSystem4157940963; SearchSystem7656671655;
>> SearchSystem2865656762; SearchSystem6520604676; SearchSystem4960161466;
>> .NET CLR 1.1.4322; .NET CLR 2.0.50727; Hotbar 10.2.232.0;
>> SearchSystem9616306563; SearchSystem6017393645; SearchSystem5219240075;
>> SearchSystem2768350104; SearchSystem6919669052; SearchSystem1986739074;
>> SearchSystem1555480186; SearchSystem3376893470; SearchSystem9530642569;
>> SearchSystem4877790286; SearchSystem8104932799; SearchSystem2313134663;
>> SearchSystem1545325372; SearchSystem7742471461; SearchSystem9092363703;
>> SearchSystem6992236221; SearchSystem3507700306; SearchSystem1129983453;
>> SearchSystem1077927937; SearchSystem2297142691; SearchSystem7813572891;
>> SearchSystem5668754497; SearchSystem6220295595; SearchSystem4157940963;
>> SearchSystem7656671655; SearchSystem2865656762; SearchSystem6520604676;
>> SearchSystem4960161466; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)' \
>>
>> -H 'Accept:
>> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
>>
>> -H 'Accept-Language: en-US,en;q=0.5' --compressed \
>>
>> -H 'Content-Type: application/x-www-form-urlencoded' \
>>
>> -H 'Connection: keep-alive' \
>>
>> -H 'Upgrade-Insecure-Requests: 1' \
>>
>> -H 'Pragma: no-cache' \
>>
>> -H 'Cache-Control: no-cache' \
>>
>> -H 'True-Client-Ip: 2.18.114.25' \
>>
>> --data 'user=dasD
>>
>>
>> After I reduced the header size, zeek can catch it.
>>
>> For example,
>>
>> curl -k -i -vv -X GET http://test/login \
>> -H 'User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1;
>> Trident/4.0; YPC 3.2.0; SearchSystem6829992239; SearchSystem9616306563;
>> SearchSystem6017393645; SearchSystem5219240075; SearchSystem2768350104;
>> SearchSystem6919669052; SearchSystem1986739074; SearchSystem1555480186;
>> SearchSystem3376893470; SearchSystem9530642569; SearchSystem4877790286;
>> SearchSystem8104932799; SearchSystem2313134663; SearchSystem1545325373;
>> SearchSystem7742471461;
>> SearchSystem2313134663; SearchSystem1545325372; SearchSystem7742471462;
>> SearchSystem2313134663; SearchSystem1545325372; SearchSystem7742471461;
>> SearchSystem2313134663;x)' \
>> -H 'Accept:
>> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
>> -H 'Accept-Language: en-US,en;q=0.5' --compressed \
>> -H 'Content-Type: application/x-www-form-urlencoded' \
>> -H 'Connection: keep-alive' \
>> -H 'Upgrade-Insecure-Requests: 1' \
>> -H 'Pragma: no-cache' \
>> -H 'Cache-Control: no-cache' \
>> -H 'True-Client-Ip: 2.18.114.25' \
>> --data 'user=dasD'
>>
>>
>> Thanks,
>>
>> Yi
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
>
> --
> Justin
>


-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191018/41c994b1/attachment-0001.html 

From mkg at vt.edu  Fri Oct 18 07:47:21 2019
From: mkg at vt.edu (Mark Gardner)
Date: Fri, 18 Oct 2019 10:47:21 -0400
Subject: [Zeek] Workers dying with "out of memory in new"
Message-ID: <CAG=_CgB6mzrNgExNQkc3jCqew76oGK-DHpruAwu11DY7XxtfMw@mail.gmail.com>

We must have crossed some threshold yesterday. Suddenly we are suffering an
epidemic of workers dying with "out of memory in new" even though we made
no changes. Previously, we would have a few die each day. Now we have had
250 alerts of workers dying and being restarted from 00:00 to 10:00. I have
no idea where to start debugging the problem. Any suggestions?

What causes a worker to die by running out of memory? The sensors have lots
of memory (see below) so I would not expect to have any out of memory
deaths. (To monitor the problem, I am in the process of setting up collectd
and graphana.)

Some details:
- 5 sensors, each with 16-core, AMD Epyc 7351P, 128 GB RAM, Intel X520-T2
- Zeek 2.6.1
- node.cfg: lb_procs=15, pin_cpus=1-15,
af_packet_buffer_size=1*1024*1024*1024
- broctl.cfg: setcap enabled
- Not shunting any traffic

Mark
-- 
Mark Gardner
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191018/4986cef0/attachment.html 

From mus3 at lehigh.edu  Fri Oct 18 08:12:20 2019
From: mus3 at lehigh.edu (Munroe Sollog)
Date: Fri, 18 Oct 2019 11:12:20 -0400
Subject: [Zeek] Workers dying with "out of memory in new"
In-Reply-To: <CAG=_CgB6mzrNgExNQkc3jCqew76oGK-DHpruAwu11DY7XxtfMw@mail.gmail.com>
References: <CAG=_CgB6mzrNgExNQkc3jCqew76oGK-DHpruAwu11DY7XxtfMw@mail.gmail.com>
Message-ID: <CAN3u96-rVzZekQ=5C-N2iLK0Fw-P0sFCUCr=jgk1j_56rO1x8A@mail.gmail.com>

Interestingly enough, we started suffering the same problem at the same
time.

- 1 node with 44 cores, 256GB of RAM
- Zeek 2.5.5
- node.cfg:
  [worker-1]

type=worker

host=localhost

interface=af_packet::ens4f0

lb_method=custom

lb_procs=25

pin_cpus=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24


- broctl.cfg:

MemLimit = 100000000 #100GB

setcap.enabled=1



On Fri, Oct 18, 2019 at 10:48 AM Mark Gardner <mkg at vt.edu> wrote:

> We must have crossed some threshold yesterday. Suddenly we are suffering
> an epidemic of workers dying with "out of memory in new" even though we
> made no changes. Previously, we would have a few die each day. Now we have
> had 250 alerts of workers dying and being restarted from 00:00 to 10:00. I
> have no idea where to start debugging the problem. Any suggestions?
>
> What causes a worker to die by running out of memory? The sensors have
> lots of memory (see below) so I would not expect to have any out of memory
> deaths. (To monitor the problem, I am in the process of setting up collectd
> and graphana.)
>
> Some details:
> - 5 sensors, each with 16-core, AMD Epyc 7351P, 128 GB RAM, Intel X520-T2
> - Zeek 2.6.1
> - node.cfg: lb_procs=15, pin_cpus=1-15,
> af_packet_buffer_size=1*1024*1024*1024
> - broctl.cfg: setcap enabled
> - Not shunting any traffic
>
> Mark
> --
> Mark Gardner
> --
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
Munroe Sollog
Senior Network Engineer
munroe at lehigh.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191018/d6ebb2fd/attachment.html 

From mus3 at lehigh.edu  Fri Oct 18 08:26:13 2019
From: mus3 at lehigh.edu (Munroe Sollog)
Date: Fri, 18 Oct 2019 11:26:13 -0400
Subject: [Zeek] Workers dying with "out of memory in new"
In-Reply-To: <CAN3u96-rVzZekQ=5C-N2iLK0Fw-P0sFCUCr=jgk1j_56rO1x8A@mail.gmail.com>
References: <CAG=_CgB6mzrNgExNQkc3jCqew76oGK-DHpruAwu11DY7XxtfMw@mail.gmail.com>
	<CAN3u96-rVzZekQ=5C-N2iLK0Fw-P0sFCUCr=jgk1j_56rO1x8A@mail.gmail.com>
Message-ID: <CAN3u9695HPyuUAZd6mTkxeWXo3eZt8FqmDJqE7hhwTx_ZD865Q@mail.gmail.com>

For additional reference:

Linux snout 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u5 (2019-08-11)
x86_64 GNU/Linux

on 10-11 I patched libssl,and libc
on 10-17 I upgraded sudo (about 30 mins after the first worker crashed)

[Bro] Crash report from worker-1-12 email received at 16:00

Log output from dpkg for reference:

# less /var/log/dpkg.log |grep "status installed"

2019-10-11 14:59:23 status installed telegraf:amd64 1.12.3-1

2019-10-11 14:59:23 status installed libssl1.0.2:amd64 1.0.2t-1~deb9u1

2019-10-11 14:59:23 status installed libc-bin:amd64 2.24-11+deb9u4

2019-10-11 14:59:23 status installed libssl1.1:amd64 1.1.0l-1~deb9u1

2019-10-11 14:59:23 status installed openssl:amd64 1.1.0l-1~deb9u1

2019-10-11 14:59:24 status installed man-db:amd64 2.7.6.1-2

2019-10-11 14:59:24 status installed libssl1.0-dev:amd64 1.0.2t-1~deb9u1

2019-10-11 14:59:24 status installed libc-bin:amd64 2.24-11+deb9u4

2019-10-17 16:25:47 status installed sudo:amd64 1.8.19p1-2.1+deb9u1

2019-10-17 16:25:47 status installed apache2-utils:amd64 2.4.25-3+deb9u9

2019-10-17 16:25:47 status installed apache2-bin:amd64 2.4.25-3+deb9u9

2019-10-17 16:25:47 status installed apache2-data:all 2.4.25-3+deb9u9

2019-10-17 16:25:47 status installed systemd:amd64 232-25+deb9u12

2019-10-17 16:25:47 status installed man-db:amd64 2.7.6.1-2

2019-10-17 16:25:48 status installed apache2:amd64 2.4.25-3+deb9u9


On Fri, Oct 18, 2019 at 11:12 AM Munroe Sollog <mus3 at lehigh.edu> wrote:

> Interestingly enough, we started suffering the same problem at the same
> time.
>
> - 1 node with 44 cores, 256GB of RAM
> - Zeek 2.5.5
> - node.cfg:
>   [worker-1]
>
> type=worker
>
> host=localhost
>
> interface=af_packet::ens4f0
>
> lb_method=custom
>
> lb_procs=25
>
> pin_cpus=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24
>
>
> - broctl.cfg:
>
> MemLimit = 100000000 #100GB
>
> setcap.enabled=1
>
>
>
> On Fri, Oct 18, 2019 at 10:48 AM Mark Gardner <mkg at vt.edu> wrote:
>
>> We must have crossed some threshold yesterday. Suddenly we are suffering
>> an epidemic of workers dying with "out of memory in new" even though we
>> made no changes. Previously, we would have a few die each day. Now we have
>> had 250 alerts of workers dying and being restarted from 00:00 to 10:00. I
>> have no idea where to start debugging the problem. Any suggestions?
>>
>> What causes a worker to die by running out of memory? The sensors have
>> lots of memory (see below) so I would not expect to have any out of memory
>> deaths. (To monitor the problem, I am in the process of setting up collectd
>> and graphana.)
>>
>> Some details:
>> - 5 sensors, each with 16-core, AMD Epyc 7351P, 128 GB RAM, Intel X520-T2
>> - Zeek 2.6.1
>> - node.cfg: lb_procs=15, pin_cpus=1-15,
>> af_packet_buffer_size=1*1024*1024*1024
>> - broctl.cfg: setcap enabled
>> - Not shunting any traffic
>>
>> Mark
>> --
>> Mark Gardner
>> --
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
>
> --
> Munroe Sollog
> Senior Network Engineer
> munroe at lehigh.edu
>


-- 
Munroe Sollog
Senior Network Engineer
munroe at lehigh.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191018/62ed79f0/attachment-0001.html 

From johanna at corelight.com  Fri Oct 18 16:38:24 2019
From: johanna at corelight.com (Johanna Amann)
Date: Fri, 18 Oct 2019 16:38:24 -0700
Subject: [Zeek] Workers dying with "out of memory in new"
In-Reply-To: <CAN3u9695HPyuUAZd6mTkxeWXo3eZt8FqmDJqE7hhwTx_ZD865Q@mail.gmail.com>
References: <CAG=_CgB6mzrNgExNQkc3jCqew76oGK-DHpruAwu11DY7XxtfMw@mail.gmail.com>
	<CAN3u96-rVzZekQ=5C-N2iLK0Fw-P0sFCUCr=jgk1j_56rO1x8A@mail.gmail.com>
	<CAN3u9695HPyuUAZd6mTkxeWXo3eZt8FqmDJqE7hhwTx_ZD865Q@mail.gmail.com>
Message-ID: <322D17E2-EA2B-4C61-AB81-26B16860D058@corelight.com>

Hi,

both of you are running rather old versions of Zeek.

Both 2.5.5 and 2.6.1 have a number of issues.

One of the issues that was fixed could be the cause for crashes. A bug 
could result in Zeek requesting huge allocations that cannot be 
fulfilled by the operating system; see e.g. 
https://github.com/zeek/zeek/issues/245 for more details. This specific 
issue was fixed on 2.6.3.

So - upgrading to 2.6.4 (or even better - 3.0.0) might fix those 
problems for you.

Besides that - both 2.5.5 and 2.6.1 have several vulnerabilities - and 
you really really really should upgrade them :).

Johanna

On 18 Oct 2019, at 8:26, Munroe Sollog wrote:

> For additional reference:
>
> Linux snout 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u5 (2019-08-11)
> x86_64 GNU/Linux
>
> on 10-11 I patched libssl,and libc
> on 10-17 I upgraded sudo (about 30 mins after the first worker 
> crashed)
>
> [Bro] Crash report from worker-1-12 email received at 16:00
>
> Log output from dpkg for reference:
>
> # less /var/log/dpkg.log |grep "status installed"
>
> 2019-10-11 14:59:23 status installed telegraf:amd64 1.12.3-1
>
> 2019-10-11 14:59:23 status installed libssl1.0.2:amd64 1.0.2t-1~deb9u1
>
> 2019-10-11 14:59:23 status installed libc-bin:amd64 2.24-11+deb9u4
>
> 2019-10-11 14:59:23 status installed libssl1.1:amd64 1.1.0l-1~deb9u1
>
> 2019-10-11 14:59:23 status installed openssl:amd64 1.1.0l-1~deb9u1
>
> 2019-10-11 14:59:24 status installed man-db:amd64 2.7.6.1-2
>
> 2019-10-11 14:59:24 status installed libssl1.0-dev:amd64 
> 1.0.2t-1~deb9u1
>
> 2019-10-11 14:59:24 status installed libc-bin:amd64 2.24-11+deb9u4
>
> 2019-10-17 16:25:47 status installed sudo:amd64 1.8.19p1-2.1+deb9u1
>
> 2019-10-17 16:25:47 status installed apache2-utils:amd64 
> 2.4.25-3+deb9u9
>
> 2019-10-17 16:25:47 status installed apache2-bin:amd64 2.4.25-3+deb9u9
>
> 2019-10-17 16:25:47 status installed apache2-data:all 2.4.25-3+deb9u9
>
> 2019-10-17 16:25:47 status installed systemd:amd64 232-25+deb9u12
>
> 2019-10-17 16:25:47 status installed man-db:amd64 2.7.6.1-2
>
> 2019-10-17 16:25:48 status installed apache2:amd64 2.4.25-3+deb9u9
>
>
> On Fri, Oct 18, 2019 at 11:12 AM Munroe Sollog <mus3 at lehigh.edu> 
> wrote:
>
>> Interestingly enough, we started suffering the same problem at the 
>> same
>> time.
>>
>> - 1 node with 44 cores, 256GB of RAM
>> - Zeek 2.5.5
>> - node.cfg:
>>   [worker-1]
>>
>> type=worker
>>
>> host=localhost
>>
>> interface=af_packet::ens4f0
>>
>> lb_method=custom
>>
>> lb_procs=25
>>
>> pin_cpus=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24
>>
>>
>> - broctl.cfg:
>>
>> MemLimit = 100000000 #100GB
>>
>> setcap.enabled=1
>>
>>
>>
>> On Fri, Oct 18, 2019 at 10:48 AM Mark Gardner <mkg at vt.edu> wrote:
>>
>>> We must have crossed some threshold yesterday. Suddenly we are 
>>> suffering
>>> an epidemic of workers dying with "out of memory in new" even though 
>>> we
>>> made no changes. Previously, we would have a few die each day. Now 
>>> we have
>>> had 250 alerts of workers dying and being restarted from 00:00 to 
>>> 10:00. I
>>> have no idea where to start debugging the problem. Any suggestions?
>>>
>>> What causes a worker to die by running out of memory? The sensors 
>>> have
>>> lots of memory (see below) so I would not expect to have any out of 
>>> memory
>>> deaths. (To monitor the problem, I am in the process of setting up 
>>> collectd
>>> and graphana.)
>>>
>>> Some details:
>>> - 5 sensors, each with 16-core, AMD Epyc 7351P, 128 GB RAM, Intel 
>>> X520-T2
>>> - Zeek 2.6.1
>>> - node.cfg: lb_procs=15, pin_cpus=1-15,
>>> af_packet_buffer_size=1*1024*1024*1024
>>> - broctl.cfg: setcap enabled
>>> - Not shunting any traffic
>>>
>>> Mark
>>> --
>>> Mark Gardner
>>> --
>>> _______________________________________________
>>> Zeek mailing list
>>> zeek at zeek.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>
>>
>>
>> --
>> Munroe Sollog
>> Senior Network Engineer
>> munroe at lehigh.edu
>>
>
>
> -- 
> Munroe Sollog
> Senior Network Engineer
> munroe at lehigh.edu
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

From johanna at corelight.com  Fri Oct 18 16:50:30 2019
From: johanna at corelight.com (Johanna Amann)
Date: Fri, 18 Oct 2019 16:50:30 -0700
Subject: [Zeek] how can i config bro to let it only capture and analyze
 http packages?
In-Reply-To: <CADUtY13wBKGg3wW-4NQAX-PhTHd_Vio02Psa0MRt37zyzTXzxg@mail.gmail.com>
References: <CADUtY13wBKGg3wW-4NQAX-PhTHd_Vio02Psa0MRt37zyzTXzxg@mail.gmail.com>
Message-ID: <A747210C-45C2-465C-935B-C1F9D5B8C6CB@corelight.com>

Hi,

On 3 Oct 2019, at 19:02, lc z wrote:

> Does it have this function?I just want to only analyze http 
> packages.And
> Does it can reduce capture loss rate via analyzing less packages?  
> Thanks a
> lot.

It kind of depends on what you mean. There are basically two approaches 
that can be used to limit yourself to only http.

First - you can use BPF filters to limit the traffic that Zeek sees. So 
- you can e.g. get Zeek to only analyze port 80 traffic by 
PacketFilter::default_capture_filter setting to (ip or not ip) and (tcp 
port 80) or similar.

If you do this you will miss http traffic on ports different from 80 
though.

The other alternative is to not load scripts that do non-http analysis. 
You can e.g. do that by starting zeek in bare mode (passing -b on the 
command line, or setting zeekargs/broargs to -b). You then have to set 
your local.bro to load the protocol analyzers that you want manually - 
in your case you would probably need

@load base/protocols/dpd
@load base/protocols/http

which should only give you http.log. All other traffic will still be 
seen by Zeek, but no protocol analysis will run on it.

If you want conn.log, you also need

@load base/protocols/conn

Both of these approaches will speed Zeek up - the first one more than 
the second one. However they both are no golden bullets - and you will 
not get as much data as you did before.

I hope this helps,
  Johanna

From brian at corelight.com  Sat Oct 19 12:52:32 2019
From: brian at corelight.com (Brian Dye)
Date: Sat, 19 Oct 2019 12:52:32 -0700
Subject: [Zeek] Elastic Common Schema mapping
Message-ID: <CAEC=iOvTurux2ykB+XnWqihWOddGqZWbV2CP5LR_SktE6WAEyA@mail.gmail.com>

All,

Following up on my brief comments at ZeekWeek, happy to share that we've
developed a mapping of Zeek fields to the Elastic Common Schema.  It is
posted at https://github.com/corelight/ecs-mapping - looking forward to
feedback and of course if there are any issues let us know (big thanks to
Richard, cc'd above, for his work as the first deployment!). We'll work to
update this as the ECS revs - there are several field they don't have in
the schema yet. Happy mapping!

Best,

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191019/05dcecee/attachment.html 

From damonrouse at gmail.com  Sat Oct 19 13:08:02 2019
From: damonrouse at gmail.com (Damon Rouse)
Date: Sat, 19 Oct 2019 13:08:02 -0700
Subject: [Zeek] Elastic Common Schema mapping
In-Reply-To: <CAEC=iOvTurux2ykB+XnWqihWOddGqZWbV2CP5LR_SktE6WAEyA@mail.gmail.com>
References: <CAEC=iOvTurux2ykB+XnWqihWOddGqZWbV2CP5LR_SktE6WAEyA@mail.gmail.com>
Message-ID: <CAHoWat62hEYJG21BMhc5-1M0CAU9cChP3aZoJya=1T4tu4XEhw@mail.gmail.com>

This is great, thanks for sharing.

On Sat, Oct 19, 2019 at 1:01 PM Brian Dye <brian at corelight.com> wrote:

> All,
>
> Following up on my brief comments at ZeekWeek, happy to share that we've
> developed a mapping of Zeek fields to the Elastic Common Schema.  It is
> posted at https://github.com/corelight/ecs-mapping - looking forward to
> feedback and of course if there are any issues let us know (big thanks to
> Richard, cc'd above, for his work as the first deployment!). We'll work to
> update this as the ECS revs - there are several field they don't have in
> the schema yet. Happy mapping!
>
> Best,
>
> Brian
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191019/70575aeb/attachment.html 

From phatbuckett at gmail.com  Sat Oct 19 13:29:09 2019
From: phatbuckett at gmail.com (Darren S.)
Date: Sat, 19 Oct 2019 13:29:09 -0700
Subject: [Zeek] Elastic Common Schema mapping
In-Reply-To: <CAEC=iOvTurux2ykB+XnWqihWOddGqZWbV2CP5LR_SktE6WAEyA@mail.gmail.com>
References: <CAEC=iOvTurux2ykB+XnWqihWOddGqZWbV2CP5LR_SktE6WAEyA@mail.gmail.com>
Message-ID: <CAKVSOJUmKTHoCvA70dOrxMM1tDRY=sXPmH-0wsZcBgx-qJznPQ@mail.gmail.com>

On Sat, Oct 19, 2019 at 1:01 PM Brian Dye <brian at corelight.com> wrote:

> All,
>
> Following up on my brief comments at ZeekWeek, happy to share that we've
> developed a mapping of Zeek fields to the Elastic Common Schema.  It is
> posted at https://github.com/corelight/ecs-mapping - looking forward to
> feedback and of course if there are any issues let us know (big thanks to
> Richard, cc'd above, for his work as the first deployment!). We'll work to
> update this as the ECS revs - there are several field they don't have in
> the schema yet. Happy mapping!
>

This is great!

The project README notes:

> The mapping can be done using either an ElasticSearch ingest node or
directly in Kibana

For users that ingest and enrich through a Logstash pipeline, how does this
apply? (i.e. would they then have to maintain ingestion content in multiple
layers)?

-- 
Darren Spruell
phatbuckett at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191019/ccef3f6b/attachment.html 

From clopmz at outlook.com  Sun Oct 20 06:17:26 2019
From: clopmz at outlook.com (Carlos Lopez)
Date: Sun, 20 Oct 2019 13:17:26 +0000
Subject: [Zeek] Seek error when I try to use zeekctl command
Message-ID: <F6C01A44-B817-4560-B0C4-FEEDE627DD6C@outlook.com>

Hi all,

I have doing a clean install on RHEL8 host with Zeek 3.0.0. When I try to use any zeekctl option, it returns the following error:

root at rhel8host:~# zeekctl help

Warning: ZeekControl plugin uses legacy BroControl API. Use
'import ZeekControl.plugin' instead of 'import BroControl.plugin'

Error: no type given for node zeek

Maybe the problem is with python3 that comes with RHEL8? Any idea?

--
Regards,
C. L. Martinez
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191020/84abc9bf/attachment.html 

From ericooi at gmail.com  Sun Oct 20 06:39:57 2019
From: ericooi at gmail.com (ericooi at gmail.com)
Date: Sun, 20 Oct 2019 08:39:57 -0500
Subject: [Zeek] Seek error when I try to use zeekctl command
In-Reply-To: <F6C01A44-B817-4560-B0C4-FEEDE627DD6C@outlook.com>
References: <F6C01A44-B817-4560-B0C4-FEEDE627DD6C@outlook.com>
Message-ID: <FA119CE6-642D-4488-BC6D-1966DF09F69B@gmail.com>

I run Zeek 3.0 on CentOS 8 and I also get this warning, though the error seems to suggest there?s an issue with your /opt/zeek/etc/node.cfg file.

> On Oct 20, 2019, at 8:17 AM, Carlos Lopez <clopmz at outlook.com> wrote:
> 
> Hi all,
>  
> I have doing a clean install on RHEL8 host with Zeek 3.0.0. When I try to use any zeekctl option, it returns the following error:
>  
> root at rhel8host:~# zeekctl help
>  
> Warning: ZeekControl plugin uses legacy BroControl API. Use
> 'import ZeekControl.plugin' instead of 'import BroControl.plugin'
>  
> Error: no type given for node zeek
>  
> Maybe the problem is with python3 that comes with RHEL8? Any idea?
>  
> -- 
> Regards,
> C. L. Martinez
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org <mailto:zeek at zeek.org>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek <http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191020/b6c0cf46/attachment-0001.html 

From clopmz at outlook.com  Sun Oct 20 06:44:47 2019
From: clopmz at outlook.com (Carlos Lopez)
Date: Sun, 20 Oct 2019 13:44:47 +0000
Subject: [Zeek] Seek error when I try to use zeekctl command
In-Reply-To: <FA119CE6-642D-4488-BC6D-1966DF09F69B@gmail.com>
References: <F6C01A44-B817-4560-B0C4-FEEDE627DD6C@outlook.com>
	<FA119CE6-642D-4488-BC6D-1966DF09F69B@gmail.com>
Message-ID: <9CFD3D6C-87F2-42EB-995E-7F521D6BCCD9@outlook.com>

Yep, you are right Ericooi ? There was an error in my nodes.cfg ? Now, it is solved. Many thanks.

On the other side, this:

Warning: ZeekControl plugin uses legacy BroControl API. Use
'import ZeekControl.plugin' instead of 'import BroControl.plugin'

.. appears every time zeekctl runs ?

--
Regards,
C. L. Martinez

From: "ericooi at gmail.com" <ericooi at gmail.com>
Date: Sunday, 20 October 2019 at 15:40
To: Carlos Lopez <clopmz at outlook.com>
Cc: "zeek at zeek.org" <zeek at zeek.org>
Subject: Re: [Zeek] Seek error when I try to use zeekctl command

I run Zeek 3.0 on CentOS 8 and I also get this warning, though the error seems to suggest there?s an issue with your /opt/zeek/etc/node.cfg file.


On Oct 20, 2019, at 8:17 AM, Carlos Lopez <clopmz at outlook.com<mailto:clopmz at outlook.com>> wrote:

Hi all,

I have doing a clean install on RHEL8 host with Zeek 3.0.0. When I try to use any zeekctl option, it returns the following error:

root at rhel8host:~# zeekctl help

Warning: ZeekControl plugin uses legacy BroControl API. Use
'import ZeekControl.plugin' instead of 'import BroControl.plugin'

Error: no type given for node zeek

Maybe the problem is with python3 that comes with RHEL8? Any idea?

--
Regards,
C. L. Martinez
_______________________________________________
Zeek mailing list
zeek at zeek.org<mailto:zeek at zeek.org>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek<http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191020/9e0ec134/attachment.html 

From ericooi at gmail.com  Sun Oct 20 06:46:41 2019
From: ericooi at gmail.com (ericooi at gmail.com)
Date: Sun, 20 Oct 2019 08:46:41 -0500
Subject: [Zeek] Seek error when I try to use zeekctl command
In-Reply-To: <9CFD3D6C-87F2-42EB-995E-7F521D6BCCD9@outlook.com>
References: <F6C01A44-B817-4560-B0C4-FEEDE627DD6C@outlook.com>
	<FA119CE6-642D-4488-BC6D-1966DF09F69B@gmail.com>
	<9CFD3D6C-87F2-42EB-995E-7F521D6BCCD9@outlook.com>
Message-ID: <7C8DC5A0-CEF5-4843-8F5A-17AE066FD27C@gmail.com>

Cool, glad I could help.

Yeah, Zeek 3 was the big transition from Bro to Zeek so there are still areas where not every single command or line of code has been transitioned over to using ?zeek? instead of ?bro.?  My guess is the warnings will go away in future releases of Zeek.  For now, it is nothing to worry about.

> On Oct 20, 2019, at 8:44 AM, Carlos Lopez <clopmz at outlook.com> wrote:
> 
> Yep, you are right Ericooi ? There was an error in my nodes.cfg ? Now, it is solved. Many thanks.
>  
> On the other side, this:
>  
> Warning: ZeekControl plugin uses legacy BroControl API. Use
> 'import ZeekControl.plugin' instead of 'import BroControl.plugin'
>  
> .. appears every time zeekctl runs ?
>  
> -- 
> Regards,
> C. L. Martinez
>  
> From: "ericooi at gmail.com" <ericooi at gmail.com>
> Date: Sunday, 20 October 2019 at 15:40
> To: Carlos Lopez <clopmz at outlook.com>
> Cc: "zeek at zeek.org" <zeek at zeek.org>
> Subject: Re: [Zeek] Seek error when I try to use zeekctl command
>  
> I run Zeek 3.0 on CentOS 8 and I also get this warning, though the error seems to suggest there?s an issue with your /opt/zeek/etc/node.cfg file.
> 
> 
> On Oct 20, 2019, at 8:17 AM, Carlos Lopez <clopmz at outlook.com <mailto:clopmz at outlook.com>> wrote:
>  
> Hi all,
>  
> I have doing a clean install on RHEL8 host with Zeek 3.0.0. When I try to use any zeekctl option, it returns the following error:
>  
> root at rhel8host:~# zeekctl help
>  
> Warning: ZeekControl plugin uses legacy BroControl API. Use
> 'import ZeekControl.plugin' instead of 'import BroControl.plugin'
>  
> Error: no type given for node zeek
>  
> Maybe the problem is with python3 that comes with RHEL8? Any idea?
>  
> -- 
> Regards,
> C. L. Martinez
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org <mailto:zeek at zeek.org>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek <http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191020/ca2951fd/attachment-0001.html 

From akgraner at corelight.com  Mon Oct 21 07:25:06 2019
From: akgraner at corelight.com (Amber Graner)
Date: Mon, 21 Oct 2019 10:25:06 -0400
Subject: [Zeek] Registration Open: Zeek Days Atlanta - 23 Oct 2019 - Attend
 for a chance to win a Raspberry Pi 4B
Message-ID: <CAJhOzuq+x-mHNB3kXG2mcnSW=x+sbG9+wpUwac2k53favaY+8g@mail.gmail.com>

Hi all,

Just a reminder, we're kicking off Zeek Days in Atlanta this week on 23
October 2019.

Zeek Days are free to attend, but registration is required
<https://www.meetup.com/Zeek-Days-Zeek-Bro-Technical-User-Workshop-Atlanta-GA/events/265369805/>
.

Corelight, Inc is sponsoring this event and will be raffling off a
Raspberry Pi 4B running Raspbian with Zeek pre-installed.(
https://lnkd.in/eEgkpsY)

Sessions include:

   - Introduction to Zeek - Amber Graner
   - Script Writing with Zeek - Seth Hall
   - Threat Hunting with Elastic + Zeek - Alex Kirk and Michelle Bennett
   - Profiling in Production (Memory, Core & Script profiling; Problems
   solved and lessons learned) - Justin Azoff


Sign up today -
https://www.meetup.com/Zeek-Days-Zeek-Bro-Technical-User-Workshop-Atlanta-GA/events/265369805/

If you or your organization would like to host a Zeek Hours (Meetups) or
Zeek Days (Workshop) event please let me know.

I look forward to hearing from you and hope to see you all in Atlanta.

With gratitude,
~Amber


-- 

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469



 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191021/6a1dacae/attachment.html 

From bpboci24 at gmail.com  Mon Oct 21 11:09:48 2019
From: bpboci24 at gmail.com (Borivoje Pavlovic)
Date: Mon, 21 Oct 2019 20:09:48 +0200
Subject: [Zeek] Detection of all attacks in pcap file
Message-ID: <CAGWrsfxydKJb_Am6CKhK43yarPh1eU2Sv8qM_OWKfGaKqaRWCg@mail.gmail.com>

Hi all,

I am beginner in Zeek. Currently, I have a task to perform analysis of .
pcap files and detect all possible attacks per time instances. In the other
words I have to test Zeek as an IDS tool and find with which percentage is
Zeek able to classify traffic correctly (True/False positive, True/False
negative indication). Is there possibility to do so? For example, I tried
to run integrated Brute-Forcing.zeek script against my .pcap file but in
the notice.log there is just note that there was an attack which is not
what I am looking. Do I have to search for labeled network in some other
logs?

Thanks in advance

Borivoje
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191021/83129832/attachment.html 

From richard at corelight.com  Mon Oct 21 13:59:26 2019
From: richard at corelight.com (Richard Bejtlich)
Date: Mon, 21 Oct 2019 16:59:26 -0400
Subject: [Zeek] Detection of all attacks in pcap file
In-Reply-To: <CAGWrsfxydKJb_Am6CKhK43yarPh1eU2Sv8qM_OWKfGaKqaRWCg@mail.gmail.com>
References: <CAGWrsfxydKJb_Am6CKhK43yarPh1eU2Sv8qM_OWKfGaKqaRWCg@mail.gmail.com>
Message-ID: <CAOtSMjbCdNBo-frCGmxXwDiPnTzSRDjT7oQvVPb3JGTFVvTL_Q@mail.gmail.com>

Hello,

The notice log would contain any information pertaining to
the policy/protocols/ssh/detect-bruteforcing.zeek script.

However, I'm a little concerned by the nature of your task. Zeek isn't
really designed as an "intrusion detection system" like Snort or Suricata.
Is this a school project?

Sincerely,

Richard

On Mon, Oct 21, 2019 at 2:18 PM Borivoje Pavlovic <bpboci24 at gmail.com>
wrote:

> Hi all,
>
> I am beginner in Zeek. Currently, I have a task to perform analysis of .
> pcap files and detect all possible attacks per time instances. In the other
> words I have to test Zeek as an IDS tool and find with which percentage is
> Zeek able to classify traffic correctly (True/False positive, True/False
> negative indication). Is there possibility to do so? For example, I tried
> to run integrated Brute-Forcing.zeek script against my .pcap file but in
> the notice.log there is just note that there was an attack which is not
> what I am looking. Do I have to search for labeled network in some other
> logs?
>
> Thanks in advance
>
> Borivoje
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
Richard Bejtlich
Principal Security Strategist, Corelight
https://corelight.blog/author/richardbejtlich/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191021/319b8df5/attachment.html 

From richard at corelight.com  Mon Oct 21 15:18:17 2019
From: richard at corelight.com (Richard Bejtlich)
Date: Mon, 21 Oct 2019 18:18:17 -0400
Subject: [Zeek] Detection of all attacks in pcap file
In-Reply-To: <CAGWrsfwor1QweZPq1DaamdAa5zhTruJJOoxn=exgdQrCtaC12Q@mail.gmail.com>
References: <CAGWrsfxydKJb_Am6CKhK43yarPh1eU2Sv8qM_OWKfGaKqaRWCg@mail.gmail.com>
	<CAOtSMjbCdNBo-frCGmxXwDiPnTzSRDjT7oQvVPb3JGTFVvTL_Q@mail.gmail.com>
	<CAGWrsfwor1QweZPq1DaamdAa5zhTruJJOoxn=exgdQrCtaC12Q@mail.gmail.com>
Message-ID: <CAOtSMjb-vT4YBv3TM_qJSrd-rferHfG=h35moq5Jem9dGO7qsA@mail.gmail.com>

Hi Borivoje and Zeek users,

Traditionally, analyst uses Zeek to transform their network traffic into
compact logs that describe a variety of activities. Rather than recording
full content in a .pcap if you're interested in a FTP session, for example,
Zeek will create one or more logs describing the important elements of that
FTP session. There's no concept of "good" or "bad" in that log, or in most
logs.

So, the premise of comparing Zeek as an IDS with Snort or Suricata doesn't
make much sense. You would be better off comparing Snort with Suricata, as
they are both designed as intrusion detection systems, i.e., they render
judgments based on the traffic they observe. Of course you need to provide
rule sets, which contain the essence of "badness" as designed by the rule
creators.

You could conceivably program Zeek to be an IDS if you decided what was bad
on your network and told Zeek to write a notice when it sees that activity.
Running default Zeek against a data set from the Internet is not going to
yield the results your professor is seeking.

Sincerely,

Richard

On Mon, Oct 21, 2019 at 6:05 PM Borivoje Pavlovic <bpboci24 at gmail.com>
wrote:

> Hi Richard,
>
> Thank you for promt response. Actually, it is a part of my thesis at
> faculty. I am required to compare different Intrusion detection systems
> such as Zeek and aforementioned Suricata and Snort based on dataset
> CICIDS2017 which contains malicious (Bruteforce, DoS, Web attacks...) and
> benign traffic. What I need is to classify/label traffic with these
> different IDS tools, but I haven't found the way anywhere how to do that
> with Zeek. Attached, you can find two images. The first one is .csv file
> that contains different flow-based features and labeled traffic (benign or
> ftp patator). I am not sure is Bro able to perform this kind of analysis at
> all. The second image is notice.log made after running
> policy/protocols/ssh/detect-bruteforcing.zeek script against .pcap file. It
> would mean a lot to me if you know is there some kind of custom script
> written in Zeek which can label all the traffic per each instances?
>
> Best regards
>
> Borivoje
>
> On Mon, Oct 21, 2019 at 10:59 PM Richard Bejtlich <richard at corelight.com>
> wrote:
>
>> Hello,
>>
>> The notice log would contain any information pertaining to
>> the policy/protocols/ssh/detect-bruteforcing.zeek script.
>>
>> However, I'm a little concerned by the nature of your task. Zeek isn't
>> really designed as an "intrusion detection system" like Snort or Suricata.
>> Is this a school project?
>>
>> Sincerely,
>>
>> Richard
>>
>> On Mon, Oct 21, 2019 at 2:18 PM Borivoje Pavlovic <bpboci24 at gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> I am beginner in Zeek. Currently, I have a task to perform analysis of .
>>> pcap files and detect all possible attacks per time instances. In the other
>>> words I have to test Zeek as an IDS tool and find with which percentage is
>>> Zeek able to classify traffic correctly (True/False positive, True/False
>>> negative indication). Is there possibility to do so? For example, I tried
>>> to run integrated Brute-Forcing.zeek script against my .pcap file but in
>>> the notice.log there is just note that there was an attack which is not
>>> what I am looking. Do I have to search for labeled network in some other
>>> logs?
>>>
>>> Thanks in advance
>>>
>>> Borivoje
>>>
>>> _______________________________________________
>>> Zeek mailing list
>>> zeek at zeek.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>
>>
>>
>> --
>> Richard Bejtlich
>> Principal Security Strategist, Corelight
>> https://corelight.blog/author/richardbejtlich/
>>
>

-- 
Richard Bejtlich
Principal Security Strategist, Corelight
https://corelight.blog/author/richardbejtlich/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191021/1303233f/attachment.html 

From richard at corelight.com  Mon Oct 21 16:22:08 2019
From: richard at corelight.com (Richard Bejtlich)
Date: Mon, 21 Oct 2019 19:22:08 -0400
Subject: [Zeek] Detection of all attacks in pcap file
In-Reply-To: <CAGWrsfwggxw_aYWXOo0BK=7tnEp3b8wm+=Orw0CWi-WOUEVTpQ@mail.gmail.com>
References: <CAGWrsfxydKJb_Am6CKhK43yarPh1eU2Sv8qM_OWKfGaKqaRWCg@mail.gmail.com>
	<CAOtSMjbCdNBo-frCGmxXwDiPnTzSRDjT7oQvVPb3JGTFVvTL_Q@mail.gmail.com>
	<CAGWrsfwor1QweZPq1DaamdAa5zhTruJJOoxn=exgdQrCtaC12Q@mail.gmail.com>
	<CAOtSMjb-vT4YBv3TM_qJSrd-rferHfG=h35moq5Jem9dGO7qsA@mail.gmail.com>
	<CAGWrsfwggxw_aYWXOo0BK=7tnEp3b8wm+=Orw0CWi-WOUEVTpQ@mail.gmail.com>
Message-ID: <CAOtSMjaBk2SCeK_E7=iOVHmO9YtUpKDzdO1ToJ-BRVOXfJiT-w@mail.gmail.com>

That?s generally what IDS users want to know ? what activity is normal,
suspicious, or malicious?

Richard

On Mon, Oct 21, 2019 at 7:04 PM Borivoje Pavlovic <bpboci24 at gmail.com>
wrote:

> Dear Richard,
>
> Thank you very much for your answer. I have one last question.
> What do you mean by deciding what was bad on network?
>
> Best regards
>
> Borivoje
>
> On Tue, 22 Oct 2019, 00:18 Richard Bejtlich, <richard at corelight.com>
> wrote:
>
>> Hi Borivoje and Zeek users,
>>
>> Traditionally, analyst uses Zeek to transform their network traffic into
>> compact logs that describe a variety of activities. Rather than recording
>> full content in a .pcap if you're interested in a FTP session, for example,
>> Zeek will create one or more logs describing the important elements of that
>> FTP session. There's no concept of "good" or "bad" in that log, or in most
>> logs.
>>
>> So, the premise of comparing Zeek as an IDS with Snort or Suricata
>> doesn't make much sense. You would be better off comparing Snort with
>> Suricata, as they are both designed as intrusion detection systems, i.e.,
>> they render judgments based on the traffic they observe. Of course you need
>> to provide rule sets, which contain the essence of "badness" as designed by
>> the rule creators.
>>
>> You could conceivably program Zeek to be an IDS if you decided what was
>> bad on your network and told Zeek to write a notice when it sees that
>> activity. Running default Zeek against a data set from the Internet is not
>> going to yield the results your professor is seeking.
>>
>> Sincerely,
>>
>> Richard
>>
>> On Mon, Oct 21, 2019 at 6:05 PM Borivoje Pavlovic <bpboci24 at gmail.com>
>> wrote:
>>
>>> Hi Richard,
>>>
>>> Thank you for promt response. Actually, it is a part of my thesis at
>>> faculty. I am required to compare different Intrusion detection systems
>>> such as Zeek and aforementioned Suricata and Snort based on dataset
>>> CICIDS2017 which contains malicious (Bruteforce, DoS, Web attacks...) and
>>> benign traffic. What I need is to classify/label traffic with these
>>> different IDS tools, but I haven't found the way anywhere how to do that
>>> with Zeek. Attached, you can find two images. The first one is .csv file
>>> that contains different flow-based features and labeled traffic (benign or
>>> ftp patator). I am not sure is Bro able to perform this kind of analysis at
>>> all. The second image is notice.log made after running
>>> policy/protocols/ssh/detect-bruteforcing.zeek script against .pcap file. It
>>> would mean a lot to me if you know is there some kind of custom script
>>> written in Zeek which can label all the traffic per each instances?
>>>
>>> Best regards
>>>
>>> Borivoje
>>>
>>> On Mon, Oct 21, 2019 at 10:59 PM Richard Bejtlich <richard at corelight.com>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> The notice log would contain any information pertaining to
>>>> the policy/protocols/ssh/detect-bruteforcing.zeek script.
>>>>
>>>> However, I'm a little concerned by the nature of your task. Zeek isn't
>>>> really designed as an "intrusion detection system" like Snort or Suricata.
>>>> Is this a school project?
>>>>
>>>> Sincerely,
>>>>
>>>> Richard
>>>>
>>>> On Mon, Oct 21, 2019 at 2:18 PM Borivoje Pavlovic <bpboci24 at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I am beginner in Zeek. Currently, I have a task to perform analysis of
>>>>> . pcap files and detect all possible attacks per time instances. In the
>>>>> other words I have to test Zeek as an IDS tool and find with which
>>>>> percentage is Zeek able to classify traffic correctly (True/False positive,
>>>>> True/False negative indication). Is there possibility to do so? For
>>>>> example, I tried to run integrated Brute-Forcing.zeek script against my
>>>>> .pcap file but in the notice.log there is just note that there was an
>>>>> attack which is not what I am looking. Do I have to search for labeled
>>>>> network in some other logs?
>>>>>
>>>>> Thanks in advance
>>>>>
>>>>> Borivoje
>>>>>
>>>>> _______________________________________________
>>>>> Zeek mailing list
>>>>> zeek at zeek.org
>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>>
>>>>
>>>>
>>>> --
>>>> Richard Bejtlich
>>>> Principal Security Strategist, Corelight
>>>> https://corelight.blog/author/richardbejtlich/
>>>>
>>>
>>
>> --
>> Richard Bejtlich
>> Principal Security Strategist, Corelight
>> https://corelight.blog/author/richardbejtlich/
>>
> --
Richard Bejtlich
Principal Security Strategist, Corelight
https://corelight.blog/author/richardbejtlich/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191021/61f4210b/attachment-0001.html 

From michalpurzynski1 at gmail.com  Mon Oct 21 20:20:07 2019
From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=)
Date: Mon, 21 Oct 2019 20:20:07 -0700
Subject: [Zeek] Detection of all attacks in pcap file
In-Reply-To: <CAOtSMjaBk2SCeK_E7=iOVHmO9YtUpKDzdO1ToJ-BRVOXfJiT-w@mail.gmail.com>
References: <CAGWrsfxydKJb_Am6CKhK43yarPh1eU2Sv8qM_OWKfGaKqaRWCg@mail.gmail.com>
	<CAOtSMjbCdNBo-frCGmxXwDiPnTzSRDjT7oQvVPb3JGTFVvTL_Q@mail.gmail.com>
	<CAGWrsfwor1QweZPq1DaamdAa5zhTruJJOoxn=exgdQrCtaC12Q@mail.gmail.com>
	<CAOtSMjb-vT4YBv3TM_qJSrd-rferHfG=h35moq5Jem9dGO7qsA@mail.gmail.com>
	<CAGWrsfwggxw_aYWXOo0BK=7tnEp3b8wm+=Orw0CWi-WOUEVTpQ@mail.gmail.com>
	<CAOtSMjaBk2SCeK_E7=iOVHmO9YtUpKDzdO1ToJ-BRVOXfJiT-w@mail.gmail.com>
Message-ID: <CAJ6bFK142OuhOoNgcHYOxZpaHBG4O-OTs650k_WTFJDWMFmhhw@mail.gmail.com>

It would be extremely difficult to compare IDS systems and here are a
couple of reasons why.

What does it mean, to compare IDS systems? Would you compare the
performance? Sure, this one can be measured, but it's so much ruleset
dependent that it rarely makes sense.

Detection accuracy? All of them basically perform the same job - reassemble
a number of packets in a stream, compare that stream against a huge
dataset, flag matches. If one of those components does not work correctly,
for example, IDS X can correctly reassemble TCP flows 99% of time, it is
either a bug in the engine or there is something wrong with the capture, or
there are performance problems.

Then it comes the comparison of "how many alerts will IDS X generate vs IDS
Y vs Z for the same input". You're not really comparing IDS-es again, but
the set of rules. Some rules can be used by multiple engines, like the
Emerging Threats has versions for both Snort and Suricata. Some don't -
like the commercial Palo Alto Networks.

Finally, you have the "neutral" engines, like Zeek (or Suricata without any
rules and with flow and protocol logging enabled in eve-json). They do not
tell what is good and what is bad - because that's up to you. They merely
tell you about a connection that happened in the past, was from A to B, N
bytes and packets were sent and M bytes and packets were received, it took
5 minutes and it was SSL.
In case of Zeek or Suricata you can have protocol analysis done, so for our
SSL connection, you will see the SNI, ciphersuites negotiated, X509
certificate details and so on.

At this point nothing is technically good or bad. Now, you as an analyst
can feed Zeek with rules saying "connections to or from IP A are always
bad" - and Zeek will let you know when those happen (or try to happen).

Or you can say "all SSL connections with a certificate with a serial number
12345 are bad" or flag a domain name in many places (not just the DNS
traffic), calculate file hashes, analyze PE files, SMB and RPC sessions,
etc.

NSM like Zeek is basically like a giant time machine + a matching engine +
an engine that can do almost arbitrary operations on network flows. It's up
to you to program it.

And that's why I think it cannot be compared with IDSes like Snort (purely
rule based) or Suricata (a combination of a traditional IDS with NSM
functionality).

Comparing Snort vs Suricata doesn't make sense either - because you would
be comparing rulesets, not engines.


On Mon, Oct 21, 2019 at 4:24 PM Richard Bejtlich <richard at corelight.com>
wrote:

> That?s generally what IDS users want to know ? what activity is normal,
> suspicious, or malicious?
>
> Richard
>
> On Mon, Oct 21, 2019 at 7:04 PM Borivoje Pavlovic <bpboci24 at gmail.com>
> wrote:
>
>> Dear Richard,
>>
>> Thank you very much for your answer. I have one last question.
>> What do you mean by deciding what was bad on network?
>>
>> Best regards
>>
>> Borivoje
>>
>> On Tue, 22 Oct 2019, 00:18 Richard Bejtlich, <richard at corelight.com>
>> wrote:
>>
>>> Hi Borivoje and Zeek users,
>>>
>>> Traditionally, analyst uses Zeek to transform their network traffic into
>>> compact logs that describe a variety of activities. Rather than recording
>>> full content in a .pcap if you're interested in a FTP session, for example,
>>> Zeek will create one or more logs describing the important elements of that
>>> FTP session. There's no concept of "good" or "bad" in that log, or in most
>>> logs.
>>>
>>> So, the premise of comparing Zeek as an IDS with Snort or Suricata
>>> doesn't make much sense. You would be better off comparing Snort with
>>> Suricata, as they are both designed as intrusion detection systems, i.e.,
>>> they render judgments based on the traffic they observe. Of course you need
>>> to provide rule sets, which contain the essence of "badness" as designed by
>>> the rule creators.
>>>
>>> You could conceivably program Zeek to be an IDS if you decided what was
>>> bad on your network and told Zeek to write a notice when it sees that
>>> activity. Running default Zeek against a data set from the Internet is not
>>> going to yield the results your professor is seeking.
>>>
>>> Sincerely,
>>>
>>> Richard
>>>
>>> On Mon, Oct 21, 2019 at 6:05 PM Borivoje Pavlovic <bpboci24 at gmail.com>
>>> wrote:
>>>
>>>> Hi Richard,
>>>>
>>>> Thank you for promt response. Actually, it is a part of my thesis at
>>>> faculty. I am required to compare different Intrusion detection systems
>>>> such as Zeek and aforementioned Suricata and Snort based on dataset
>>>> CICIDS2017 which contains malicious (Bruteforce, DoS, Web attacks...) and
>>>> benign traffic. What I need is to classify/label traffic with these
>>>> different IDS tools, but I haven't found the way anywhere how to do that
>>>> with Zeek. Attached, you can find two images. The first one is .csv file
>>>> that contains different flow-based features and labeled traffic (benign or
>>>> ftp patator). I am not sure is Bro able to perform this kind of analysis at
>>>> all. The second image is notice.log made after running
>>>> policy/protocols/ssh/detect-bruteforcing.zeek script against .pcap file. It
>>>> would mean a lot to me if you know is there some kind of custom script
>>>> written in Zeek which can label all the traffic per each instances?
>>>>
>>>> Best regards
>>>>
>>>> Borivoje
>>>>
>>>> On Mon, Oct 21, 2019 at 10:59 PM Richard Bejtlich <
>>>> richard at corelight.com> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> The notice log would contain any information pertaining to
>>>>> the policy/protocols/ssh/detect-bruteforcing.zeek script.
>>>>>
>>>>> However, I'm a little concerned by the nature of your task. Zeek isn't
>>>>> really designed as an "intrusion detection system" like Snort or Suricata.
>>>>> Is this a school project?
>>>>>
>>>>> Sincerely,
>>>>>
>>>>> Richard
>>>>>
>>>>> On Mon, Oct 21, 2019 at 2:18 PM Borivoje Pavlovic <bpboci24 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I am beginner in Zeek. Currently, I have a task to perform analysis
>>>>>> of . pcap files and detect all possible attacks per time instances. In the
>>>>>> other words I have to test Zeek as an IDS tool and find with which
>>>>>> percentage is Zeek able to classify traffic correctly (True/False positive,
>>>>>> True/False negative indication). Is there possibility to do so? For
>>>>>> example, I tried to run integrated Brute-Forcing.zeek script against my
>>>>>> .pcap file but in the notice.log there is just note that there was an
>>>>>> attack which is not what I am looking. Do I have to search for labeled
>>>>>> network in some other logs?
>>>>>>
>>>>>> Thanks in advance
>>>>>>
>>>>>> Borivoje
>>>>>>
>>>>>> _______________________________________________
>>>>>> Zeek mailing list
>>>>>> zeek at zeek.org
>>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Richard Bejtlich
>>>>> Principal Security Strategist, Corelight
>>>>> https://corelight.blog/author/richardbejtlich/
>>>>>
>>>>
>>>
>>> --
>>> Richard Bejtlich
>>> Principal Security Strategist, Corelight
>>> https://corelight.blog/author/richardbejtlich/
>>>
>> --
> Richard Bejtlich
> Principal Security Strategist, Corelight
> https://corelight.blog/author/richardbejtlich/
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191021/1981fffe/attachment.html 

From bramiejim at gmail.com  Mon Oct 21 21:37:41 2019
From: bramiejim at gmail.com (jamie brim)
Date: Mon, 21 Oct 2019 21:37:41 -0700
Subject: [Zeek] Detection of all attacks in pcap file
In-Reply-To: <CAOtSMjb-vT4YBv3TM_qJSrd-rferHfG=h35moq5Jem9dGO7qsA@mail.gmail.com>
References: <CAGWrsfxydKJb_Am6CKhK43yarPh1eU2Sv8qM_OWKfGaKqaRWCg@mail.gmail.com>
	<CAOtSMjbCdNBo-frCGmxXwDiPnTzSRDjT7oQvVPb3JGTFVvTL_Q@mail.gmail.com>
	<CAGWrsfwor1QweZPq1DaamdAa5zhTruJJOoxn=exgdQrCtaC12Q@mail.gmail.com>
	<CAOtSMjb-vT4YBv3TM_qJSrd-rferHfG=h35moq5Jem9dGO7qsA@mail.gmail.com>
Message-ID: <CAMq_K2xQe=TAbqnG5NjfEHaJF6=7nn9DTB=trfiRpvfVvaA-Kw@mail.gmail.com>

Looking at the page I found for CICIDS2017
<https://www.unb.ca/cic/datasets/ids-2017.html>, it seems like there are
specific attacks enumerated on the page that describe the traffic in the
PCAPs. Roughly the captured behavior seems like it can be broken down into
reconnaissance, specific exploits, botnet activity, implant activity, and
DoS. There is included already scanning and ssh brute force detection, an
example
<https://docs.zeek.org/en/stable/examples/ids/index.html#detecting-an-ftp-brute-force-attack-and-notifying>
provided of
how to set up FTP brute force detection, as well as a blog post
<https://blog.zeek.org//2014/04/detecting-heartbleed-bug-using-bro.html> on
Heartbleed detection in Zeek, that was arguably more robust than the
signature based detections that went out at the same time. I encourage you
to check these resources out!

Your results with Zeek will depend on just how deep down the rabbit hole
you are willing to go. The exploits and implants will likely not be
detected by Zeek "out of the box", the same likely goes with the DoS
detection (though the SumStats
<https://docs.zeek.org/en/stable/frameworks/sumstats.html> framework may
help here with some quick wins). Some could likely be caught with good IOC
(domain, IP) feeds, transformed into something usable by Zeek, and fed into
Zeek's intel framework
<https://docs.zeek.org/en/stable/frameworks/intel.html>. Other exploit /
implant detection might require further log analysis or some scripting.
Some might only be detectable by bytestream signatures. It's hard to tell
without digging in further.

As others have explained, Zeek "out of the box" can provide useful
telemetry, in the form of network protocol logs, that could be used
downstream, in either some sort of SIEM, rules engine, or similarly
purposed data infrastructure, to not only detect such attacks, but provide
a fuller context to incident responders running down these detections, who
might need later follow-up information that the pattern matching based
approaches would not have preserved. As others have also said, Zeek makes
more observations than judgments, and it is up to the operator what
response to take to these observations.

Without knowing more about those specific DoS attacks or exploit based
attacks, it's hard to know what would show up in the standard logs, but it
could be an interesting exercise to run the PCAPs through default Zeek,
feed the logs to Splunk or ELK, look for patterns that you might be able to
discern and write rules for them, either in the downstream log processor,
or in Zeek scripting language itself. I encourage you to consider Zeek as a
crucial source component of your overall security detection stack, rather
than a full-stack, fully-calibrated network detection engine in a box.

One cool thing about Zeek vs most signature based IDS systems is that it
Zeek reaches as deep as it can into the protocols it understands, and makes
even more information available than it does by default to operators who
are willing to dive into script land. If you know what you're looking for,
and it's observable on the network, chances are there's a way to analyze it
with a Zeek script. As an example, Corelight recently published an overview
of such work for SSH
<https://corelight.blog/2019/05/07/how-zeek-can-provide-insights-despite-encrypted-communications/>
analysis. Zeek offers a diverse set of Protocol Analyzers
<https://docs.zeek.org/en/stable/script-reference/proto-analyzers.html> that
provide a rich stream of events that can be hooked to suit your every
purpose. There's also a collection of packages <https://packages.zeek.org/> to
explore if you're interested in learning more about scripting. If you have
an hour, or two, I'd also check out this talk
<https://www.youtube.com/watch?v=fGgHgJEzgLc> which I found helpful in
explaining what it is you're signing up yourself up for :)

All the best
Jamie



On Mon, Oct 21, 2019 at 3:20 PM Richard Bejtlich <richard at corelight.com>
wrote:

> Hi Borivoje and Zeek users,
>
> Traditionally, analyst uses Zeek to transform their network traffic into
> compact logs that describe a variety of activities. Rather than recording
> full content in a .pcap if you're interested in a FTP session, for example,
> Zeek will create one or more logs describing the important elements of that
> FTP session. There's no concept of "good" or "bad" in that log, or in most
> logs.
>
> So, the premise of comparing Zeek as an IDS with Snort or Suricata doesn't
> make much sense. You would be better off comparing Snort with Suricata, as
> they are both designed as intrusion detection systems, i.e., they render
> judgments based on the traffic they observe. Of course you need to provide
> rule sets, which contain the essence of "badness" as designed by the rule
> creators.
>
> You could conceivably program Zeek to be an IDS if you decided what was
> bad on your network and told Zeek to write a notice when it sees that
> activity. Running default Zeek against a data set from the Internet is not
> going to yield the results your professor is seeking.
>
> Sincerely,
>
> Richard
>
> On Mon, Oct 21, 2019 at 6:05 PM Borivoje Pavlovic <bpboci24 at gmail.com>
> wrote:
>
>> Hi Richard,
>>
>> Thank you for promt response. Actually, it is a part of my thesis at
>> faculty. I am required to compare different Intrusion detection systems
>> such as Zeek and aforementioned Suricata and Snort based on dataset
>> CICIDS2017 which contains malicious (Bruteforce, DoS, Web attacks...) and
>> benign traffic. What I need is to classify/label traffic with these
>> different IDS tools, but I haven't found the way anywhere how to do that
>> with Zeek. Attached, you can find two images. The first one is .csv file
>> that contains different flow-based features and labeled traffic (benign or
>> ftp patator). I am not sure is Bro able to perform this kind of analysis at
>> all. The second image is notice.log made after running
>> policy/protocols/ssh/detect-bruteforcing.zeek script against .pcap file. It
>> would mean a lot to me if you know is there some kind of custom script
>> written in Zeek which can label all the traffic per each instances?
>>
>> Best regards
>>
>> Borivoje
>>
>> On Mon, Oct 21, 2019 at 10:59 PM Richard Bejtlich <richard at corelight.com>
>> wrote:
>>
>>> Hello,
>>>
>>> The notice log would contain any information pertaining to
>>> the policy/protocols/ssh/detect-bruteforcing.zeek script.
>>>
>>> However, I'm a little concerned by the nature of your task. Zeek isn't
>>> really designed as an "intrusion detection system" like Snort or Suricata.
>>> Is this a school project?
>>>
>>> Sincerely,
>>>
>>> Richard
>>>
>>> On Mon, Oct 21, 2019 at 2:18 PM Borivoje Pavlovic <bpboci24 at gmail.com>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I am beginner in Zeek. Currently, I have a task to perform analysis of
>>>> . pcap files and detect all possible attacks per time instances. In the
>>>> other words I have to test Zeek as an IDS tool and find with which
>>>> percentage is Zeek able to classify traffic correctly (True/False positive,
>>>> True/False negative indication). Is there possibility to do so? For
>>>> example, I tried to run integrated Brute-Forcing.zeek script against my
>>>> .pcap file but in the notice.log there is just note that there was an
>>>> attack which is not what I am looking. Do I have to search for labeled
>>>> network in some other logs?
>>>>
>>>> Thanks in advance
>>>>
>>>> Borivoje
>>>>
>>>> _______________________________________________
>>>> Zeek mailing list
>>>> zeek at zeek.org
>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>
>>>
>>>
>>> --
>>> Richard Bejtlich
>>> Principal Security Strategist, Corelight
>>> https://corelight.blog/author/richardbejtlich/
>>>
>>
>
> --
> Richard Bejtlich
> Principal Security Strategist, Corelight
> https://corelight.blog/author/richardbejtlich/
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191021/9bbec7cc/attachment-0001.html 

From mkg at vt.edu  Tue Oct 22 12:32:16 2019
From: mkg at vt.edu (Mark Gardner)
Date: Tue, 22 Oct 2019 15:32:16 -0400
Subject: [Zeek] how can i config bro to let it only capture and analyze
 http packages?
In-Reply-To: <A747210C-45C2-465C-935B-C1F9D5B8C6CB@corelight.com>
References: <CADUtY13wBKGg3wW-4NQAX-PhTHd_Vio02Psa0MRt37zyzTXzxg@mail.gmail.com>
	<A747210C-45C2-465C-935B-C1F9D5B8C6CB@corelight.com>
Message-ID: <CAG=_CgCvWwmpgXTT3dY4=A6CVBrX9nBNKyFPGnjzDZ+NdM0NUQ@mail.gmail.com>

Thanks Johanna. Learning about the bug which is fixed in 2.6.4 is really
helpful.

I would like to utilize the OpenSUSE build service to install/upgrade 2.6.4
but going to OBS from the zeek.org/download points me to v3.0.0 for Debian
10. How do I access the v2.6.4 package for Debian 10? (I am not ready to
upgrade to 3.0. I want to give it some time for the bugs to shake out.)

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191022/b4bdbbfd/attachment.html 

From edx0004 at gmail.com  Wed Oct 23 00:55:47 2019
From: edx0004 at gmail.com (edX)
Date: Wed, 23 Oct 2019 10:55:47 +0300
Subject: [Zeek] ZEEK AS AN IDS
Message-ID: <CAArApxJ+QormJFPrPE8K=PNKBz-mnxSMaFRrsBwvWSNuCBxEJw@mail.gmail.com>

Hello! I am an intermediate zeek user. I would like a walk-through on how i
can use zeek to detect different types of attacks such as sql injection,
ddos, man in the middle attacks and the likes.
Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/4c8a76ed/attachment.html 

From johanna at corelight.com  Wed Oct 23 01:39:44 2019
From: johanna at corelight.com (Johanna Amann)
Date: Wed, 23 Oct 2019 10:39:44 +0200
Subject: [Zeek] how can i config bro to let it only capture and analyze
 http packages?
In-Reply-To: <CAG=_CgCvWwmpgXTT3dY4=A6CVBrX9nBNKyFPGnjzDZ+NdM0NUQ@mail.gmail.com>
References: <CADUtY13wBKGg3wW-4NQAX-PhTHd_Vio02Psa0MRt37zyzTXzxg@mail.gmail.com>
	<A747210C-45C2-465C-935B-C1F9D5B8C6CB@corelight.com>
	<CAG=_CgCvWwmpgXTT3dY4=A6CVBrX9nBNKyFPGnjzDZ+NdM0NUQ@mail.gmail.com>
Message-ID: <5832443F-321E-4972-9977-A0ADFC59E1CD@corelight.com>

Hi Mark,

the old packages currently are still available at the old location 
https://software.opensuse.org//download.html?project=network%3Abro&package=bro 
/ https://build.opensuse.org/package/show/network:bro/bro

And just for reference - Zeek downloads moved to 
https://software.opensuse.org//download.html?project=security%3Azeek&package=zeek 
/ https://build.opensuse.org/package/show/security:zeek/zeek

Johanna

On 22 Oct 2019, at 21:32, Mark Gardner wrote:

> Thanks Johanna. Learning about the bug which is fixed in 2.6.4 is 
> really
> helpful.
>
> I would like to utilize the OpenSUSE build service to install/upgrade 
> 2.6.4
> but going to OBS from the zeek.org/download points me to v3.0.0 for 
> Debian
> 10. How do I access the v2.6.4 package for Debian 10? (I am not ready 
> to
> upgrade to 3.0. I want to give it some time for the bugs to shake 
> out.)
>
> Mark
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek


From richard at corelight.com  Wed Oct 23 05:49:28 2019
From: richard at corelight.com (Richard Bejtlich)
Date: Wed, 23 Oct 2019 08:49:28 -0400
Subject: [Zeek] ZEEK AS AN IDS
In-Reply-To: <CAArApxJ+QormJFPrPE8K=PNKBz-mnxSMaFRrsBwvWSNuCBxEJw@mail.gmail.com>
References: <CAArApxJ+QormJFPrPE8K=PNKBz-mnxSMaFRrsBwvWSNuCBxEJw@mail.gmail.com>
Message-ID: <CAOtSMja1WzZc1zUw4nYPxqWuET_GWp7OBamkAADo1djz1DnqbA@mail.gmail.com>

Hello,

What research have you done so far?

Richard

On Wed, Oct 23, 2019 at 4:04 AM edX <edx0004 at gmail.com> wrote:

> Hello! I am an intermediate zeek user. I would like a walk-through on how
> i can use zeek to detect different types of attacks such as sql injection,
> ddos, man in the middle attacks and the likes.
> Thanks.
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
Richard Bejtlich
Principal Security Strategist, Corelight
https://corelight.blog/author/richardbejtlich/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/a8f4c9df/attachment.html 

From richard at corelight.com  Wed Oct 23 06:41:48 2019
From: richard at corelight.com (Richard Bejtlich)
Date: Wed, 23 Oct 2019 09:41:48 -0400
Subject: [Zeek] ZEEK AS AN IDS
In-Reply-To: <CAArApx+Az=efB92DC2=QKaw2cx=Rn=YUNiyncKON-yBZhZ5D4g@mail.gmail.com>
References: <CAArApxJ+QormJFPrPE8K=PNKBz-mnxSMaFRrsBwvWSNuCBxEJw@mail.gmail.com>
	<CAOtSMja1WzZc1zUw4nYPxqWuET_GWp7OBamkAADo1djz1DnqbA@mail.gmail.com>
	<CAArApx+Az=efB92DC2=QKaw2cx=Rn=YUNiyncKON-yBZhZ5D4g@mail.gmail.com>
Message-ID: <CAOtSMjYhb5cgG7sWS2M=DLvcaPN-j6NzJdRssUZJr5Vh5ByzUA@mail.gmail.com>

I just Googled

bro sql injection detection

and this paper was the second result, right after a link to the Bro SQL
injection detection script.

https://www.sans.org/reading-room/whitepapers/detection/web-application-attack-analysis-bro-ids-34042


You might have to look for Bro references as the Zeek rename is only a year
old.

Sincerely,

Richard

On Wed, Oct 23, 2019 at 9:26 AM edX <edx0004 at gmail.com> wrote:

> I have done some research on detecting ssh bruteforce attacks. I found
> resource from hold my beer blog.
>
> edx0004.
>
> On Wed, Oct 23, 2019 at 3:49 PM Richard Bejtlich <richard at corelight.com>
> wrote:
>
>> Hello,
>>
>> What research have you done so far?
>>
>> Richard
>>
>> On Wed, Oct 23, 2019 at 4:04 AM edX <edx0004 at gmail.com> wrote:
>>
>>> Hello! I am an intermediate zeek user. I would like a walk-through on
>>> how i can use zeek to detect different types of attacks such as sql
>>> injection, ddos, man in the middle attacks and the likes.
>>> Thanks.
>>> _______________________________________________
>>> Zeek mailing list
>>> zeek at zeek.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>
>>
>>
>> --
>> Richard Bejtlich
>> Principal Security Strategist, Corelight
>> https://corelight.blog/author/richardbejtlich/
>>
>

-- 
Richard Bejtlich
Principal Security Strategist, Corelight
https://corelight.blog/author/richardbejtlich/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/0d8c7a90/attachment.html 

From smoot at corelight.com  Wed Oct 23 07:23:38 2019
From: smoot at corelight.com (Steve Smoot)
Date: Wed, 23 Oct 2019 07:23:38 -0700
Subject: [Zeek] Elastic Common Schema mapping
In-Reply-To: <CAKVSOJUmKTHoCvA70dOrxMM1tDRY=sXPmH-0wsZcBgx-qJznPQ@mail.gmail.com>
References: <CAEC=iOvTurux2ykB+XnWqihWOddGqZWbV2CP5LR_SktE6WAEyA@mail.gmail.com>
	<CAKVSOJUmKTHoCvA70dOrxMM1tDRY=sXPmH-0wsZcBgx-qJznPQ@mail.gmail.com>
Message-ID: <CADOXQ8aTFGaKNJwmjG8Y3UFRV2Jq8yn66gpRZo1WKRmoeXEoEA@mail.gmail.com>

On Sat, Oct 19, 2019 at 1:31 PM Darren S. <phatbuckett at gmail.com> wrote:

> Following up on my brief comments at ZeekWeek, happy to share that we've
>> developed a mapping of Zeek fields to the Elastic Common Schema.  It is
>> posted at https://github.com/corelight/ecs-mapping - looking forward to
>> feedback and of course if there are any issues let us know (big thanks to
>> Richard, cc'd above, for his work as the first deployment!). We'll work to
>> update this as the ECS revs - there are several field they don't have in
>> the schema yet. Happy mapping!
>>
>
> This is great!
>
> The project README notes:
>
> > The mapping can be done using either an ElasticSearch ingest node or
> directly in Kibana
>
> For users that ingest and enrich through a Logstash pipeline, how does
> this apply? (i.e. would they then have to maintain ingestion content in
> multiple layers)?
>

Yes it still applies, when Logstash forwards the data to Elastic it will go
through the ingest pipelines and go through ECS.

-s


>
> --
> Darren Spruell
> phatbuckett at gmail.com
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
*Stephen R. Smoot, PhD*
VP, Customer Success
Corelight
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/0a4d02f1/attachment-0001.html 

From mkg at vt.edu  Wed Oct 23 08:05:07 2019
From: mkg at vt.edu (Mark Gardner)
Date: Wed, 23 Oct 2019 11:05:07 -0400
Subject: [Zeek] Errors installing af_packet-plugin
Message-ID: <CAG=_CgD6h1KFXR95p1zpDKWHT+UGS9x43uqd7uguwfBNM-UAnA@mail.gmail.com>

I installed Zeek 2.6.4 for Debian 10 following the instructions for adding
the repo and manually installing the package. Next, I attempted to "bro-pkg
install bro-af_package-plugin" and received an error message
in bro-af_packet-plugin-build.log (see below). I have been unsuccessful in
searching for information online to help me solve the problem. What should
I be doing?

Note: the kernel header package for the running kernel
(4.19.0-0.bpo.4-amd64) is installed. No idea why cmake can't find them.

Mark

=== STDERR ===
CMake Error at CMakeLists.txt:6 (include):
  include could not find load file:

    BroPlugin


CMake Warning at CMakeLists.txt:8 (find_package):
  By not providing "FindKernelHeaders.cmake" in CMAKE_MODULE_PATH this
  project has asked CMake to find a package configuration file provided by
  "KernelHeaders", but CMake did not find one.

  Could not find a package configuration file provided by "KernelHeaders"
  with any of the following names:

    KernelHeadersConfig.cmake
    kernelheaders-config.cmake

  Add the installation prefix of "KernelHeaders" to CMAKE_PREFIX_PATH or set
  "KernelHeaders_DIR" to a directory containing one of the above files.  If
  "KernelHeaders" provides a separate development package or SDK, be sure it
  has been installed.


CMake Error at CMakeLists.txt:22 (message):
  Kernel headers not found.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/5af70f97/attachment.html 

From jgarciar at sia.es  Wed Oct 23 08:22:32 2019
From: jgarciar at sia.es (Jorge Garcia Rodriguez)
Date: Wed, 23 Oct 2019 15:22:32 +0000
Subject: [Zeek] High Availability with Zeek
Message-ID: <D6E3E7F0075CEF4CB16D0A627912409A35DFD7E1@Newton.sia.es>

Hi everyone.

We trying to use Zeek to monitor 4 interfaces in different machines. The idea is to have 1 Manager with 1 logger in one machine, and 4 workers to monitoring each of the interfaces. But this means that if the Manager crashes, everything goes down, I guess. So my question here is: ?Is possible to configure a second Manager or something to reach high availability?

Regards.

Jorge Garc?a Rodr?guez
Technical Consultant
Security Infrastructures
jgarciar at sia.es<mailto:jgarciar at sia.es>
Grupo SIA
Avda.Europa,2 - Alcor Plaza, Edificio B - Parque Oeste Alcorc?n
28922 Alcorc?n - Madrid
Tlf: +34 902 480 580<nxphone:+34%20902%20480%20580>   Fax: +34 91 307 79 80<nxphone:+34%2091%20307%2079%2080>
www.siainternational.com<http://www.siainternational.com/>
delivering value
This e-mail and any attached files are intended solely for the addresse/s identified herein. It may contain confidential and/or legally privileged information and may not necessarily represent the opinion of SIA.
No legally binding commitments will be created by this E-mail message. Where we intend to create legally binding commitments these will be made through hard copy correspondence or documents. If you receive this message by mistake, please immediately notify the sender and delete it since you are not authorized to use, disclose, distribute, print or copy all or part of the contained information Thank you. It is understood that the message was sent to you accidentally, although you appear as the addressee, you can see from the frame of existing relations that you were not the final addressee.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/08557c59/attachment.html 

From jlay at slave-tothe-box.net  Wed Oct 23 08:33:12 2019
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 23 Oct 2019 09:33:12 -0600
Subject: [Zeek] Errors installing af_packet-plugin
In-Reply-To: <CAG=_CgD6h1KFXR95p1zpDKWHT+UGS9x43uqd7uguwfBNM-UAnA@mail.gmail.com>
References: <CAG=_CgD6h1KFXR95p1zpDKWHT+UGS9x43uqd7uguwfBNM-UAnA@mail.gmail.com>
Message-ID: <594508231e6bd97bdf1970ad28d8c7ab@slave-tothe-box.net>

You need to install deb kernel header package...do:

apt-cache search linux-headers

and find the package that matches your running kernel.

James

On 2019-10-23 09:05, Mark Gardner wrote:
> I installed Zeek 2.6.4 for Debian 10 following the instructions for
> adding the repo and manually installing the package. Next, I attempted
> to "bro-pkg install bro-af_package-plugin" and received an error
> message in bro-af_packet-plugin-build.log (see below). I have been
> unsuccessful in searching for information online to help me solve the
> problem. What should I be doing?
> 
> Note: the kernel header package for the running kernel
> (4.19.0-0.bpo.4-amd64) is installed. No idea why cmake can't find
> them.
> 
> Mark
> 
> === STDERR ===
> CMake Error at CMakeLists.txt:6 (include):
>   include could not find load file:
> 
>     BroPlugin
> 
> CMake Warning at CMakeLists.txt:8 (find_package):
>   By not providing "FindKernelHeaders.cmake" in CMAKE_MODULE_PATH this
>   project has asked CMake to find a package configuration file
> provided by
>   "KernelHeaders", but CMake did not find one.
> 
>   Could not find a package configuration file provided by
> "KernelHeaders"
>   with any of the following names:
> 
>     KernelHeadersConfig.cmake
>     kernelheaders-config.cmake
> 
>   Add the installation prefix of "KernelHeaders" to CMAKE_PREFIX_PATH
> or set
>   "KernelHeaders_DIR" to a directory containing one of the above
> files.  If
>   "KernelHeaders" provides a separate development package or SDK, be
> sure it
>   has been installed.
> 
> CMake Error at CMakeLists.txt:22 (message):
>   Kernel headers not found.
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

From mkg at vt.edu  Wed Oct 23 08:51:01 2019
From: mkg at vt.edu (Mark Gardner)
Date: Wed, 23 Oct 2019 11:51:01 -0400
Subject: [Zeek] Errors installing af_packet-plugin
In-Reply-To: <594508231e6bd97bdf1970ad28d8c7ab@slave-tothe-box.net>
References: <CAG=_CgD6h1KFXR95p1zpDKWHT+UGS9x43uqd7uguwfBNM-UAnA@mail.gmail.com>
	<594508231e6bd97bdf1970ad28d8c7ab@slave-tothe-box.net>
Message-ID: <CAG=_CgAcp_rj726z4g=zQExr1d_d22Q1ntZM+U5Pziq7OH5Esg@mail.gmail.com>

On Wed, Oct 23, 2019 at 11:39 AM James Lay <jlay at slave-tothe-box.net> wrote:

> You need to install deb kernel header package...do:
>
> apt-cache search linux-headers
>
> and find the package that matches your running kernel.
>

I have the kernel headers installed:

$ uname -a
Linux zeekmgr 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2+deb10u1 (2019-09-20)
x86_64 GNU/Linux

$ dpkg -l linux-headers-4.19.0-6-amd64
Desired=Unknown/Install/Remove/Purge/Hold
|
Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                         Version           Architecture Description
+++-============================-=================-============-=====================================
ii  linux-headers-4.19.0-6-amd64 4.19.67-2+deb10u1 amd64        Header
files for Linux 4.19.0-6-amd64

# bro-pkg install bro-af_packet-plugin
The following packages will be INSTALLED:
  bro/j-gras/bro-af_packet-plugin (1.4.0)

Proceed? [Y/n]
Running unit tests for "bro/j-gras/bro-af_packet-plugin"
error: failed to run tests for bro/j-gras/bro-af_packet-plugin: package
build_command failed, see log in
/root/.bro-pkg/logs/bro-af_packet-plugin-build.log
Proceed to install anyway? [N/y]

The error message (sent in the original post and below) suggests that there
are two problems: 1) can't find BroPlugin and 2) can't find the kernel
headers. As I demonstrated above, the headers are indeed installed. I
suspect there is a problem with the Debian package in that it does not
contain FindKernelHeaders.cmake or kernelheaders-config.cmake but then
again I don't know anything about using Cmake.

# cat /root/.bro-pkg/logs/bro-af_packet-plugin-build.log

=== STDERR ===
CMake Error at CMakeLists.txt:6 (include):
  include could not find load file:

    BroPlugin


CMake Warning at CMakeLists.txt:8 (find_package):
  By not providing "FindKernelHeaders.cmake" in CMAKE_MODULE_PATH this
  project has asked CMake to find a package configuration file provided by
  "KernelHeaders", but CMake did not find one.

  Could not find a package configuration file provided by "KernelHeaders"
  with any of the following names:

    KernelHeadersConfig.cmake
    kernelheaders-config.cmake

  Add the installation prefix of "KernelHeaders" to CMAKE_PREFIX_PATH or set
  "KernelHeaders_DIR" to a directory containing one of the above files.  If
  "KernelHeaders" provides a separate development package or SDK, be sure it
  has been installed.


CMake Error at CMakeLists.txt:22 (message):
  Kernel headers not found.


=== STDOUT ===
Build Directory        : build
Bro Source Directory   :
-- The C compiler identification is GNU 8.3.0
-- The CXX compiler identification is GNU 8.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring incomplete, errors occurred!
See also
"/root/.bro-pkg/testing/bro-af_packet-plugin/clones/bro-af_packet-plugin/build/CMakeFiles/CMakeOutput.log".


Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/d1d3177d/attachment-0001.html 

From jlay at slave-tothe-box.net  Wed Oct 23 08:55:51 2019
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 23 Oct 2019 09:55:51 -0600
Subject: [Zeek] Errors installing af_packet-plugin
In-Reply-To: <CAG=_CgAcp_rj726z4g=zQExr1d_d22Q1ntZM+U5Pziq7OH5Esg@mail.gmail.com>
References: <CAG=_CgD6h1KFXR95p1zpDKWHT+UGS9x43uqd7uguwfBNM-UAnA@mail.gmail.com>
	<594508231e6bd97bdf1970ad28d8c7ab@slave-tothe-box.net>
	<CAG=_CgAcp_rj726z4g=zQExr1d_d22Q1ntZM+U5Pziq7OH5Esg@mail.gmail.com>
Message-ID: <2addc970f79c6906aebfb073d5c0fcf6@slave-tothe-box.net>

Ah....so yea you're looking at my point of contention with bro-pkg, so 
per:

https://github.com/J-Gras/bro-af_packet-plugin

it looks like you might have to try and manually install after all.

James

On 2019-10-23 09:51, Mark Gardner wrote:
> On Wed, Oct 23, 2019 at 11:39 AM James Lay <jlay at slave-tothe-box.net>
> wrote:
> 
>> You need to install deb kernel header package...do:
>> 
>> apt-cache search linux-headers
>> 
>> and find the package that matches your running kernel.
> 
> I have the kernel headers installed:
> 
> $ uname -aLinux zeekmgr 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2+deb10u1
> (2019-09-20) x86_64 GNU/Linux
> 
> $ dpkg -l
> linux-headers-4.19.0-6-amd64Desired=Unknown/Install/Remove/Purge/Hold
> |
> Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
> |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
> ||/ Name                         Version           Architecture
> Description
> +++-============================-=================-============-=====================================
> ii  linux-headers-4.19.0-6-amd64 4.19.67-2+deb10u1 amd64        Header
> files for Linux 4.19.0-6-amd64
> 
> # bro-pkg install bro-af_packet-plugin
> The following packages will be INSTALLED:
>   bro/j-gras/bro-af_packet-plugin (1.4.0)
> 
> Proceed? [Y/n]
> Running unit tests for "bro/j-gras/bro-af_packet-plugin"
> error: failed to run tests for bro/j-gras/bro-af_packet-plugin:
> package build_command failed, see log in
> /root/.bro-pkg/logs/bro-af_packet-plugin-build.log
> Proceed to install anyway? [N/y]
> 
> The error message (sent in the original post and below) suggests that
> there are two problems: 1) can't find BroPlugin and 2) can't find the
> kernel headers. As I demonstrated above, the headers are indeed
> installed. I suspect there is a problem with the Debian package in
> that it does not contain FindKernelHeaders.cmake or
> kernelheaders-config.cmake but then again I don't know anything about
> using Cmake.
> 
> # cat /root/.bro-pkg/logs/bro-af_packet-plugin-build.log
> 
> === STDERR ===
> CMake Error at CMakeLists.txt:6 (include):
>   include could not find load file:
> 
>     BroPlugin
> 
> CMake Warning at CMakeLists.txt:8 (find_package):
>   By not providing "FindKernelHeaders.cmake" in CMAKE_MODULE_PATH this
>   project has asked CMake to find a package configuration file
> provided by
>   "KernelHeaders", but CMake did not find one.
> 
>   Could not find a package configuration file provided by
> "KernelHeaders"
>   with any of the following names:
> 
>     KernelHeadersConfig.cmake
>     kernelheaders-config.cmake
> 
>   Add the installation prefix of "KernelHeaders" to CMAKE_PREFIX_PATH
> or set
>   "KernelHeaders_DIR" to a directory containing one of the above
> files.  If
>   "KernelHeaders" provides a separate development package or SDK, be
> sure it
>   has been installed.
> 
> CMake Error at CMakeLists.txt:22 (message):
>   Kernel headers not found.
> 
> === STDOUT ===
> Build Directory        : build
> Bro Source Directory   :
> -- The C compiler identification is GNU 8.3.0
> -- The CXX compiler identification is GNU 8.3.0
> -- Check for working C compiler: /usr/bin/cc
> -- Check for working C compiler: /usr/bin/cc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Detecting C compile features
> -- Detecting C compile features - done
> -- Check for working CXX compiler: /usr/bin/c++
> -- Check for working CXX compiler: /usr/bin/c++ -- works
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> -- Configuring incomplete, errors occurred!
> See also
> "/root/.bro-pkg/testing/bro-af_packet-plugin/clones/bro-af_packet-plugin/build/CMakeFiles/CMakeOutput.log".
> 
> Mark

From mkg at vt.edu  Wed Oct 23 09:08:36 2019
From: mkg at vt.edu (Mark Gardner)
Date: Wed, 23 Oct 2019 12:08:36 -0400
Subject: [Zeek] Errors installing af_packet-plugin
In-Reply-To: <2addc970f79c6906aebfb073d5c0fcf6@slave-tothe-box.net>
References: <CAG=_CgD6h1KFXR95p1zpDKWHT+UGS9x43uqd7uguwfBNM-UAnA@mail.gmail.com>
	<594508231e6bd97bdf1970ad28d8c7ab@slave-tothe-box.net>
	<CAG=_CgAcp_rj726z4g=zQExr1d_d22Q1ntZM+U5Pziq7OH5Esg@mail.gmail.com>
	<2addc970f79c6906aebfb073d5c0fcf6@slave-tothe-box.net>
Message-ID: <CAG=_CgARsm=7RHOSso+U+6sXMFP24oC1+VhEqmZ=fx9Aev0iWQ@mail.gmail.com>

On Wed, Oct 23, 2019 at 11:56 AM James Lay <jlay at slave-tothe-box.net> wrote:

> Ah....so yea you're looking at my point of contention with bro-pkg, so
> per:
>
> https://github.com/J-Gras/bro-af_packet-plugin
>
> it looks like you might have to try and manually install after all.
>

I installed by hand the last time. I was hoping that the problem was fixed.
No problem. Thanks for your help.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/116e869f/attachment.html 

From jlay at slave-tothe-box.net  Wed Oct 23 09:11:56 2019
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 23 Oct 2019 10:11:56 -0600
Subject: [Zeek] Errors installing af_packet-plugin
In-Reply-To: <CAG=_CgARsm=7RHOSso+U+6sXMFP24oC1+VhEqmZ=fx9Aev0iWQ@mail.gmail.com>
References: <CAG=_CgD6h1KFXR95p1zpDKWHT+UGS9x43uqd7uguwfBNM-UAnA@mail.gmail.com>
	<594508231e6bd97bdf1970ad28d8c7ab@slave-tothe-box.net>
	<CAG=_CgAcp_rj726z4g=zQExr1d_d22Q1ntZM+U5Pziq7OH5Esg@mail.gmail.com>
	<2addc970f79c6906aebfb073d5c0fcf6@slave-tothe-box.net>
	<CAG=_CgARsm=7RHOSso+U+6sXMFP24oC1+VhEqmZ=fx9Aev0iWQ@mail.gmail.com>
Message-ID: <195587e7d97772552ff44d8191e10318@slave-tothe-box.net>

On 2019-10-23 10:08, Mark Gardner wrote:
> On Wed, Oct 23, 2019 at 11:56 AM James Lay <jlay at slave-tothe-box.net>
> wrote:
> 
>> Ah....so yea you're looking at my point of contention with bro-pkg,
>> so
>> per:
>> 
>> https://github.com/J-Gras/bro-af_packet-plugin
>> 
>> it looks like you might have to try and manually install after all.
> 
> I installed by hand the last time. I was hoping that the problem was
> fixed. No problem. Thanks for your help.
> 
> Mark

You bet...I was in the same boat...gave up on bro-pkg and just installed 
manually....won't look back...too much hassle.

James

From jan.grashoefer at gmail.com  Wed Oct 23 09:12:38 2019
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Wed, 23 Oct 2019 18:12:38 +0200
Subject: [Zeek] Errors installing af_packet-plugin
In-Reply-To: <2addc970f79c6906aebfb073d5c0fcf6@slave-tothe-box.net>
References: <CAG=_CgD6h1KFXR95p1zpDKWHT+UGS9x43uqd7uguwfBNM-UAnA@mail.gmail.com>
	<594508231e6bd97bdf1970ad28d8c7ab@slave-tothe-box.net>
	<CAG=_CgAcp_rj726z4g=zQExr1d_d22Q1ntZM+U5Pziq7OH5Esg@mail.gmail.com>
	<2addc970f79c6906aebfb073d5c0fcf6@slave-tothe-box.net>
Message-ID: <bd8383dc-ccca-293f-d06c-0e9dd650ec29@gmail.com>

Where are the headers located? Especially where can you find 
"include/linux/user.h"? This is what cmake is looking for...

Jan

On 23/10/2019 17:55, James Lay wrote:
> Ah....so yea you're looking at my point of contention with bro-pkg, so
> per:
> 
> https://github.com/J-Gras/bro-af_packet-plugin
> 
> it looks like you might have to try and manually install after all.
> 
> James
> 
> On 2019-10-23 09:51, Mark Gardner wrote:
>> On Wed, Oct 23, 2019 at 11:39 AM James Lay <jlay at slave-tothe-box.net>
>> wrote:
>>
>>> You need to install deb kernel header package...do:
>>>
>>> apt-cache search linux-headers
>>>
>>> and find the package that matches your running kernel.
>>
>> I have the kernel headers installed:
>>
>> $ uname -aLinux zeekmgr 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2+deb10u1
>> (2019-09-20) x86_64 GNU/Linux
>>
>> $ dpkg -l
>> linux-headers-4.19.0-6-amd64Desired=Unknown/Install/Remove/Purge/Hold
>> |
>> Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
>> |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
>> ||/ Name                         Version           Architecture
>> Description
>> +++-============================-=================-============-=====================================
>> ii  linux-headers-4.19.0-6-amd64 4.19.67-2+deb10u1 amd64        Header
>> files for Linux 4.19.0-6-amd64
>>
>> # bro-pkg install bro-af_packet-plugin
>> The following packages will be INSTALLED:
>>    bro/j-gras/bro-af_packet-plugin (1.4.0)
>>
>> Proceed? [Y/n]
>> Running unit tests for "bro/j-gras/bro-af_packet-plugin"
>> error: failed to run tests for bro/j-gras/bro-af_packet-plugin:
>> package build_command failed, see log in
>> /root/.bro-pkg/logs/bro-af_packet-plugin-build.log
>> Proceed to install anyway? [N/y]
>>
>> The error message (sent in the original post and below) suggests that
>> there are two problems: 1) can't find BroPlugin and 2) can't find the
>> kernel headers. As I demonstrated above, the headers are indeed
>> installed. I suspect there is a problem with the Debian package in
>> that it does not contain FindKernelHeaders.cmake or
>> kernelheaders-config.cmake but then again I don't know anything about
>> using Cmake.
>>
>> # cat /root/.bro-pkg/logs/bro-af_packet-plugin-build.log
>>
>> === STDERR ===
>> CMake Error at CMakeLists.txt:6 (include):
>>    include could not find load file:
>>
>>      BroPlugin
>>
>> CMake Warning at CMakeLists.txt:8 (find_package):
>>    By not providing "FindKernelHeaders.cmake" in CMAKE_MODULE_PATH this
>>    project has asked CMake to find a package configuration file
>> provided by
>>    "KernelHeaders", but CMake did not find one.
>>
>>    Could not find a package configuration file provided by
>> "KernelHeaders"
>>    with any of the following names:
>>
>>      KernelHeadersConfig.cmake
>>      kernelheaders-config.cmake
>>
>>    Add the installation prefix of "KernelHeaders" to CMAKE_PREFIX_PATH
>> or set
>>    "KernelHeaders_DIR" to a directory containing one of the above
>> files.  If
>>    "KernelHeaders" provides a separate development package or SDK, be
>> sure it
>>    has been installed.
>>
>> CMake Error at CMakeLists.txt:22 (message):
>>    Kernel headers not found.
>>
>> === STDOUT ===
>> Build Directory        : build
>> Bro Source Directory   :
>> -- The C compiler identification is GNU 8.3.0
>> -- The CXX compiler identification is GNU 8.3.0
>> -- Check for working C compiler: /usr/bin/cc
>> -- Check for working C compiler: /usr/bin/cc -- works
>> -- Detecting C compiler ABI info
>> -- Detecting C compiler ABI info - done
>> -- Detecting C compile features
>> -- Detecting C compile features - done
>> -- Check for working CXX compiler: /usr/bin/c++
>> -- Check for working CXX compiler: /usr/bin/c++ -- works
>> -- Detecting CXX compiler ABI info
>> -- Detecting CXX compiler ABI info - done
>> -- Detecting CXX compile features
>> -- Detecting CXX compile features - done
>> -- Configuring incomplete, errors occurred!
>> See also
>> "/root/.bro-pkg/testing/bro-af_packet-plugin/clones/bro-af_packet-plugin/build/CMakeFiles/CMakeOutput.log".
>>
>> Mark
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> 

From jlay at slave-tothe-box.net  Wed Oct 23 09:28:18 2019
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 23 Oct 2019 10:28:18 -0600
Subject: [Zeek] Errors installing af_packet-plugin
In-Reply-To: <bd8383dc-ccca-293f-d06c-0e9dd650ec29@gmail.com>
References: <CAG=_CgD6h1KFXR95p1zpDKWHT+UGS9x43uqd7uguwfBNM-UAnA@mail.gmail.com>
	<594508231e6bd97bdf1970ad28d8c7ab@slave-tothe-box.net>
	<CAG=_CgAcp_rj726z4g=zQExr1d_d22Q1ntZM+U5Pziq7OH5Esg@mail.gmail.com>
	<2addc970f79c6906aebfb073d5c0fcf6@slave-tothe-box.net>
	<bd8383dc-ccca-293f-d06c-0e9dd650ec29@gmail.com>
Message-ID: <389b0c6ad85a75bbf270b258343243d9@slave-tothe-box.net>

Oh I see where this might be the issue...looks like common needs 
installed:

/usr/src/linux-headers-4.9.0-3-common/include/linux/user.h

ii  linux-headers-4.9.0-3-amd64            4.9.30-2+deb9u5               
     amd64        Header files for Linux 4.9.0-3-amd64
ii  linux-headers-4.9.0-3-common           4.9.30-2+deb9u5               
     all          Common header files for Linux 4.9.0-3

Debian is funky ;)

James

On 2019-10-23 10:12, Jan Grash?fer wrote:
> Where are the headers located? Especially where can you find
> "include/linux/user.h"? This is what cmake is looking for...
> 
> Jan
> 
> On 23/10/2019 17:55, James Lay wrote:
>> Ah....so yea you're looking at my point of contention with bro-pkg, so
>> per:
>> 
>> https://github.com/J-Gras/bro-af_packet-plugin
>> 
>> it looks like you might have to try and manually install after all.
>> 
>> James
>> 
>> On 2019-10-23 09:51, Mark Gardner wrote:
>>> On Wed, Oct 23, 2019 at 11:39 AM James Lay <jlay at slave-tothe-box.net>
>>> wrote:
>>> 
>>>> You need to install deb kernel header package...do:
>>>> 
>>>> apt-cache search linux-headers
>>>> 
>>>> and find the package that matches your running kernel.
>>> 
>>> I have the kernel headers installed:
>>> 
>>> $ uname -aLinux zeekmgr 4.19.0-6-amd64 #1 SMP Debian 
>>> 4.19.67-2+deb10u1
>>> (2019-09-20) x86_64 GNU/Linux
>>> 
>>> $ dpkg -l
>>> linux-headers-4.19.0-6-amd64Desired=Unknown/Install/Remove/Purge/Hold
>>> |
>>> Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
>>> |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
>>> ||/ Name                         Version           Architecture
>>> Description
>>> +++-============================-=================-============-=====================================
>>> ii  linux-headers-4.19.0-6-amd64 4.19.67-2+deb10u1 amd64        
>>> Header
>>> files for Linux 4.19.0-6-amd64
>>> 
>>> # bro-pkg install bro-af_packet-plugin
>>> The following packages will be INSTALLED:
>>>    bro/j-gras/bro-af_packet-plugin (1.4.0)
>>> 
>>> Proceed? [Y/n]
>>> Running unit tests for "bro/j-gras/bro-af_packet-plugin"
>>> error: failed to run tests for bro/j-gras/bro-af_packet-plugin:
>>> package build_command failed, see log in
>>> /root/.bro-pkg/logs/bro-af_packet-plugin-build.log
>>> Proceed to install anyway? [N/y]
>>> 
>>> The error message (sent in the original post and below) suggests that
>>> there are two problems: 1) can't find BroPlugin and 2) can't find the
>>> kernel headers. As I demonstrated above, the headers are indeed
>>> installed. I suspect there is a problem with the Debian package in
>>> that it does not contain FindKernelHeaders.cmake or
>>> kernelheaders-config.cmake but then again I don't know anything about
>>> using Cmake.
>>> 
>>> # cat /root/.bro-pkg/logs/bro-af_packet-plugin-build.log
>>> 
>>> === STDERR ===
>>> CMake Error at CMakeLists.txt:6 (include):
>>>    include could not find load file:
>>> 
>>>      BroPlugin
>>> 
>>> CMake Warning at CMakeLists.txt:8 (find_package):
>>>    By not providing "FindKernelHeaders.cmake" in CMAKE_MODULE_PATH 
>>> this
>>>    project has asked CMake to find a package configuration file
>>> provided by
>>>    "KernelHeaders", but CMake did not find one.
>>> 
>>>    Could not find a package configuration file provided by
>>> "KernelHeaders"
>>>    with any of the following names:
>>> 
>>>      KernelHeadersConfig.cmake
>>>      kernelheaders-config.cmake
>>> 
>>>    Add the installation prefix of "KernelHeaders" to 
>>> CMAKE_PREFIX_PATH
>>> or set
>>>    "KernelHeaders_DIR" to a directory containing one of the above
>>> files.  If
>>>    "KernelHeaders" provides a separate development package or SDK, be
>>> sure it
>>>    has been installed.
>>> 
>>> CMake Error at CMakeLists.txt:22 (message):
>>>    Kernel headers not found.
>>> 
>>> === STDOUT ===
>>> Build Directory        : build
>>> Bro Source Directory   :
>>> -- The C compiler identification is GNU 8.3.0
>>> -- The CXX compiler identification is GNU 8.3.0
>>> -- Check for working C compiler: /usr/bin/cc
>>> -- Check for working C compiler: /usr/bin/cc -- works
>>> -- Detecting C compiler ABI info
>>> -- Detecting C compiler ABI info - done
>>> -- Detecting C compile features
>>> -- Detecting C compile features - done
>>> -- Check for working CXX compiler: /usr/bin/c++
>>> -- Check for working CXX compiler: /usr/bin/c++ -- works
>>> -- Detecting CXX compiler ABI info
>>> -- Detecting CXX compiler ABI info - done
>>> -- Detecting CXX compile features
>>> -- Detecting CXX compile features - done
>>> -- Configuring incomplete, errors occurred!
>>> See also
>>> "/root/.bro-pkg/testing/bro-af_packet-plugin/clones/bro-af_packet-plugin/build/CMakeFiles/CMakeOutput.log".
>>> 
>>> Mark
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>> 
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

From mkg at vt.edu  Wed Oct 23 09:31:40 2019
From: mkg at vt.edu (Mark Gardner)
Date: Wed, 23 Oct 2019 12:31:40 -0400
Subject: [Zeek] Errors installing af_packet-plugin
In-Reply-To: <bd8383dc-ccca-293f-d06c-0e9dd650ec29@gmail.com>
References: <CAG=_CgD6h1KFXR95p1zpDKWHT+UGS9x43uqd7uguwfBNM-UAnA@mail.gmail.com>
	<594508231e6bd97bdf1970ad28d8c7ab@slave-tothe-box.net>
	<CAG=_CgAcp_rj726z4g=zQExr1d_d22Q1ntZM+U5Pziq7OH5Esg@mail.gmail.com>
	<2addc970f79c6906aebfb073d5c0fcf6@slave-tothe-box.net>
	<bd8383dc-ccca-293f-d06c-0e9dd650ec29@gmail.com>
Message-ID: <CAG=_CgD8=VG6QStsSjc663c_fJhu4H3VHW98=T7h8gK7acmkTQ@mail.gmail.com>

On Wed, Oct 23, 2019 at 12:13 PM Jan Grash?fer <jan.grashoefer at gmail.com>
wrote:

> Where are the headers located? Especially where can you find
> "include/linux/user.h"? This is what cmake is looking for...
>

$ dpkg -L linux-headers-4.19.0-6-common | grep include/linux/user.h
/usr/src/linux-headers-4.19.0-6-common/include/linux/user.h

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/eb0d3beb/attachment.html 

From mkg at vt.edu  Wed Oct 23 10:10:18 2019
From: mkg at vt.edu (Mark Gardner)
Date: Wed, 23 Oct 2019 13:10:18 -0400
Subject: [Zeek] Errors installing af_packet-plugin
In-Reply-To: <389b0c6ad85a75bbf270b258343243d9@slave-tothe-box.net>
References: <CAG=_CgD6h1KFXR95p1zpDKWHT+UGS9x43uqd7uguwfBNM-UAnA@mail.gmail.com>
	<594508231e6bd97bdf1970ad28d8c7ab@slave-tothe-box.net>
	<CAG=_CgAcp_rj726z4g=zQExr1d_d22Q1ntZM+U5Pziq7OH5Esg@mail.gmail.com>
	<2addc970f79c6906aebfb073d5c0fcf6@slave-tothe-box.net>
	<bd8383dc-ccca-293f-d06c-0e9dd650ec29@gmail.com>
	<389b0c6ad85a75bbf270b258343243d9@slave-tothe-box.net>
Message-ID: <CAG=_CgBg3uPiE89hiYMyQKKVeT2BmL4y8aTzFJbM9vXApmbV6A@mail.gmail.com>

On Wed, Oct 23, 2019, 12:40 James Lay <jlay at slave-tothe-box.net> wrote:

> Oh I see where this might be the issue...looks like common needs
> installed:
>
> /usr/src/linux-headers-4.9.0-3-common/include/linux/user.h
>
> ii  linux-headers-4.9.0-3-amd64            4.9.30-2+deb9u5
>      amd64        Header files for Linux 4.9.0-3-amd64
> ii  linux-headers-4.9.0-3-common           4.9.30-2+deb9u5
>      all          Common header files for Linux 4.9.0-3
>
> Debian is funky ;)
>

Both common and amd64 kernel header packages are installed on my system.
(The common package is a dependency of the amd64 package.)

Besides the kernel headed not being found, there was the error message
about not finding BroPlugin. Any idea what that is about?

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/f00c7eba/attachment.html 

From jsbarber60 at gmail.com  Wed Oct 23 11:31:17 2019
From: jsbarber60 at gmail.com (Jeff Barber)
Date: Wed, 23 Oct 2019 12:31:17 -0600
Subject: [Zeek] Event namespaces
Message-ID: <CABLqKG+Ac23EfmooQZxTveVLA60qxfc3W_R9nvLwWVoEKzvSYQ@mail.gmail.com>

At
https://docs.zeek.org/en/stable/frameworks/broker.html#a-reminder-about-events-and-module-namespaces,
following a code sample, there is the statement:

*This code runs without errors, however, the local my_event handler will
never be called and also not any remote handlers either, even if
Broker::auto_publish was used elsewhere for it. *


My tests have not supported that assertion: the event handler is invoked -
even via auto_publish. If it is so, how/when exactly would it manifest? Are
there other factors that might cause it to be true in some cases? (Say, the
same event name in a different namespace?)

Just trying to figure out how careful I need to be of namespace issues. My
tests have generally shown that if you get the namespace of some script
element wrong, the script parsing stage gives you an 'undefined' right out
of the gate.

Thanks,
Jeff
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/fc5a56eb/attachment.html 

From dopheide at gmail.com  Wed Oct 23 12:12:27 2019
From: dopheide at gmail.com (Mike Dopheide)
Date: Wed, 23 Oct 2019 14:12:27 -0500
Subject: [Zeek] Event namespaces
In-Reply-To: <CABLqKG+Ac23EfmooQZxTveVLA60qxfc3W_R9nvLwWVoEKzvSYQ@mail.gmail.com>
References: <CABLqKG+Ac23EfmooQZxTveVLA60qxfc3W_R9nvLwWVoEKzvSYQ@mail.gmail.com>
Message-ID: <CAPy2kFYi9SjekaM3rh_kO-3K8=-9YqadcqjBx+SoaCRCfQ3XGg@mail.gmail.com>

Hhmm...  I get the expected non-working behavior:

====== TEST 1 ======
# cat test.zeek
module MyModule;

export {
    global my_event: event();
}

event my_event()
    {
    print "got my event";
    }

event zeek_init()
    {
    event my_event();
    schedule 10sec { my_event() };
    }

# zeek -i eth0 test.zeek
listening on eth0

(nothing else)

======= TEST 2 =======
# cat test2.zeek
module MyModule;

export {
    global my_event: event();
}

event my_event()
    {
    print "got my event";
    }

event zeek_init()
    {
    event MyModule::my_event();
    schedule 10sec { MyModule::my_event() };
    }

# zeek -i eth0 test2.zeek
listening on eth0

got my event
got my event



On Wed, Oct 23, 2019 at 1:34 PM Jeff Barber <jsbarber60 at gmail.com> wrote:

> At
> https://docs.zeek.org/en/stable/frameworks/broker.html#a-reminder-about-events-and-module-namespaces,
> following a code sample, there is the statement:
>
> *This code runs without errors, however, the local my_event handler will
> never be called and also not any remote handlers either, even if
> Broker::auto_publish was used elsewhere for it. *
>
>
> My tests have not supported that assertion: the event handler is invoked -
> even via auto_publish. If it is so, how/when exactly would it manifest? Are
> there other factors that might cause it to be true in some cases? (Say, the
> same event name in a different namespace?)
>
> Just trying to figure out how careful I need to be of namespace issues. My
> tests have generally shown that if you get the namespace of some script
> element wrong, the script parsing stage gives you an 'undefined' right out
> of the gate.
>
> Thanks,
> Jeff
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/867224ff/attachment.html 

From jmellander at lbl.gov  Wed Oct 23 12:15:17 2019
From: jmellander at lbl.gov (Jim Mellander)
Date: Wed, 23 Oct 2019 12:15:17 -0700
Subject: [Zeek] Event namespaces
In-Reply-To: <CABLqKG+Ac23EfmooQZxTveVLA60qxfc3W_R9nvLwWVoEKzvSYQ@mail.gmail.com>
References: <CABLqKG+Ac23EfmooQZxTveVLA60qxfc3W_R9nvLwWVoEKzvSYQ@mail.gmail.com>
Message-ID: <CADju=b5tvWnCxSHKuf28d=h8JRsxO0y42tmj+v0qdBoHoqUizg@mail.gmail.com>

Take a look at these: https://github.com/zeek/zeek/issues/163 &
https://bro-tracker.atlassian.net/browse/BIT-984

I've run into this issue, and my best practice is to use fully scoped event
names, which is the recommended workaround.

Jim


On Wed, Oct 23, 2019 at 11:34 AM Jeff Barber <jsbarber60 at gmail.com> wrote:

> At
> https://docs.zeek.org/en/stable/frameworks/broker.html#a-reminder-about-events-and-module-namespaces,
> following a code sample, there is the statement:
>
> *This code runs without errors, however, the local my_event handler will
> never be called and also not any remote handlers either, even if
> Broker::auto_publish was used elsewhere for it. *
>
>
> My tests have not supported that assertion: the event handler is invoked -
> even via auto_publish. If it is so, how/when exactly would it manifest? Are
> there other factors that might cause it to be true in some cases? (Say, the
> same event name in a different namespace?)
>
> Just trying to figure out how careful I need to be of namespace issues. My
> tests have generally shown that if you get the namespace of some script
> element wrong, the script parsing stage gives you an 'undefined' right out
> of the gate.
>
> Thanks,
> Jeff
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/28c8c3c3/attachment.html 

From jsbarber60 at gmail.com  Wed Oct 23 16:16:37 2019
From: jsbarber60 at gmail.com (Jeff Barber)
Date: Wed, 23 Oct 2019 17:16:37 -0600
Subject: [Zeek] Event namespaces
In-Reply-To: <CADju=b5tvWnCxSHKuf28d=h8JRsxO0y42tmj+v0qdBoHoqUizg@mail.gmail.com>
References: <CABLqKG+Ac23EfmooQZxTveVLA60qxfc3W_R9nvLwWVoEKzvSYQ@mail.gmail.com>
	<CADju=b5tvWnCxSHKuf28d=h8JRsxO0y42tmj+v0qdBoHoqUizg@mail.gmail.com>
Message-ID: <CABLqKGLGoEs2jgXo_PXU55_e89tVKoigZqvdoCHJuhWtx0fRqg@mail.gmail.com>

Ah. Thanks, Jim.

That's what I wanted to understand: From your second link, it's clear that
the "global" declaration on the event is what causes the issue. If you
simply define an event (without global) and use it within the same script
file, there seems to be no problem.

Cheers,
Jeff



On Wed, Oct 23, 2019 at 1:15 PM Jim Mellander <jmellander at lbl.gov> wrote:

> Take a look at these: https://github.com/zeek/zeek/issues/163 &
> https://bro-tracker.atlassian.net/browse/BIT-984
>
> I've run into this issue, and my best practice is to use fully scoped
> event names, which is the recommended workaround.
>
> Jim
>
>
> On Wed, Oct 23, 2019 at 11:34 AM Jeff Barber <jsbarber60 at gmail.com> wrote:
>
>> At
>> https://docs.zeek.org/en/stable/frameworks/broker.html#a-reminder-about-events-and-module-namespaces,
>> following a code sample, there is the statement:
>>
>> *This code runs without errors, however, the local my_event handler will
>> never be called and also not any remote handlers either, even if
>> Broker::auto_publish was used elsewhere for it. *
>>
>>
>> My tests have not supported that assertion: the event handler is invoked
>> - even via auto_publish. If it is so, how/when exactly would it manifest?
>> Are there other factors that might cause it to be true in some cases? (Say,
>> the same event name in a different namespace?)
>>
>> Just trying to figure out how careful I need to be of namespace issues.
>> My tests have generally shown that if you get the namespace of some script
>> element wrong, the script parsing stage gives you an 'undefined' right out
>> of the gate.
>>
>> Thanks,
>> Jeff
>>
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191023/e4b5ccf5/attachment.html 

From edx0004 at gmail.com  Wed Oct 23 23:27:48 2019
From: edx0004 at gmail.com (edX)
Date: Thu, 24 Oct 2019 09:27:48 +0300
Subject: [Zeek] ZEEK AS AN IDS
In-Reply-To: <CAOtSMjYhb5cgG7sWS2M=DLvcaPN-j6NzJdRssUZJr5Vh5ByzUA@mail.gmail.com>
References: <CAArApxJ+QormJFPrPE8K=PNKBz-mnxSMaFRrsBwvWSNuCBxEJw@mail.gmail.com>
	<CAOtSMja1WzZc1zUw4nYPxqWuET_GWp7OBamkAADo1djz1DnqbA@mail.gmail.com>
	<CAArApx+Az=efB92DC2=QKaw2cx=Rn=YUNiyncKON-yBZhZ5D4g@mail.gmail.com>
	<CAOtSMjYhb5cgG7sWS2M=DLvcaPN-j6NzJdRssUZJr5Vh5ByzUA@mail.gmail.com>
Message-ID: <CAArApxKbufy5tdmfFUNV6pw2cFjiUC7G9KqvmvK+Ao74Dcn23A@mail.gmail.com>

Thanks, I'll check it out.

edX

On Wed, Oct 23, 2019 at 4:41 PM Richard Bejtlich <richard at corelight.com>
wrote:

> I just Googled
>
> bro sql injection detection
>
> and this paper was the second result, right after a link to the Bro SQL
> injection detection script.
>
>
> https://www.sans.org/reading-room/whitepapers/detection/web-application-attack-analysis-bro-ids-34042
>
>
> You might have to look for Bro references as the Zeek rename is only a
> year old.
>
> Sincerely,
>
> Richard
>
> On Wed, Oct 23, 2019 at 9:26 AM edX <edx0004 at gmail.com> wrote:
>
>> I have done some research on detecting ssh bruteforce attacks. I found
>> resource from hold my beer blog.
>>
>> edx0004.
>>
>> On Wed, Oct 23, 2019 at 3:49 PM Richard Bejtlich <richard at corelight.com>
>> wrote:
>>
>>> Hello,
>>>
>>> What research have you done so far?
>>>
>>> Richard
>>>
>>> On Wed, Oct 23, 2019 at 4:04 AM edX <edx0004 at gmail.com> wrote:
>>>
>>>> Hello! I am an intermediate zeek user. I would like a walk-through on
>>>> how i can use zeek to detect different types of attacks such as sql
>>>> injection, ddos, man in the middle attacks and the likes.
>>>> Thanks.
>>>> _______________________________________________
>>>> Zeek mailing list
>>>> zeek at zeek.org
>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>
>>>
>>>
>>> --
>>> Richard Bejtlich
>>> Principal Security Strategist, Corelight
>>> https://corelight.blog/author/richardbejtlich/
>>>
>>
>
> --
> Richard Bejtlich
> Principal Security Strategist, Corelight
> https://corelight.blog/author/richardbejtlich/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191024/51c1f96f/attachment-0001.html 

From jsiwek at corelight.com  Fri Oct 25 11:22:38 2019
From: jsiwek at corelight.com (Jon Siwek)
Date: Fri, 25 Oct 2019 11:22:38 -0700
Subject: [Zeek] High Availability with Zeek
In-Reply-To: <D6E3E7F0075CEF4CB16D0A627912409A35DFD7E1@Newton.sia.es>
References: <D6E3E7F0075CEF4CB16D0A627912409A35DFD7E1@Newton.sia.es>
Message-ID: <CAMzgZ0+WCCvfS4RBrBZWz=CsMog-6POpvehY-7iHsQzAsheW-Q@mail.gmail.com>

On Wed, Oct 23, 2019 at 8:24 AM Jorge Garcia Rodriguez <jgarciar at sia.es> wrote:
>
> So my question here is: ?Is possible to configure a second Manager or something to reach high availability?

The default scripts/configuration more or less depends on there being
exactly 1 Manager.  That doesn't stop someone from writing their own
scripts to handle things differently, but while that's technically
possible, it's not a trivial effort I expect a user undertake.

- Jon


From justin at corelight.com  Fri Oct 25 12:04:37 2019
From: justin at corelight.com (Justin Azoff)
Date: Fri, 25 Oct 2019 15:04:37 -0400
Subject: [Zeek] High Availability with Zeek
In-Reply-To: <CAMzgZ0+WCCvfS4RBrBZWz=CsMog-6POpvehY-7iHsQzAsheW-Q@mail.gmail.com>
References: <D6E3E7F0075CEF4CB16D0A627912409A35DFD7E1@Newton.sia.es>
	<CAMzgZ0+WCCvfS4RBrBZWz=CsMog-6POpvehY-7iHsQzAsheW-Q@mail.gmail.com>
Message-ID: <CAPfnCuhj6+imEV4kAAUureWPwO9-K=EbX=vBsDvWYWZAwdvASA@mail.gmail.com>

On Fri, Oct 25, 2019 at 2:34 PM Jon Siwek <jsiwek at corelight.com> wrote:

> On Wed, Oct 23, 2019 at 8:24 AM Jorge Garcia Rodriguez <jgarciar at sia.es>
> wrote:
> >
> > So my question here is: ?Is possible to configure a second Manager or
> something to reach high availability?
>
> The default scripts/configuration more or less depends on there being
> exactly 1 Manager.  That doesn't stop someone from writing their own
> scripts to handle things differently, but while that's technically
> possible, it's not a trivial effort I expect a user undertake.
>
> - Jon
>

We may be a bit further along than people realize though.  With 2.6+ we
have proxy failover, and I think logger failover works too if you configure
more than one.  If the manager dies the most noticeable issues are intel,
notices, and sumstats would stop working..  so I think  only a few places
need updating.  At some point the manager process won't be doing anything.

-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191025/306c9edc/attachment.html 

From 1766521944 at qq.com  Tue Oct 29 04:24:57 2019
From: 1766521944 at qq.com (=?gb18030?B?0e7S48Ho?=)
Date: Tue, 29 Oct 2019 19:24:57 +0800
Subject: [Zeek] =?gb18030?q?=A1=B8FOR_HELP=A1=B9The_mirrored_traffic_is_he?=
 =?gb18030?q?avily_lost=2E?=
Message-ID: <tencent_29D00AE1D4D975279BD59A2B19E2DB907607@qq.com>

I mirrored the traffic between the core switch of our computer room and the public network firewall, but the zeek report contained a lot of packet loss (30%), and currently uses PFring for packet capture. I confirm that the hardware is fully capable of handling these packet?"Capture loss" and "dropped packets"  have alarms?At the same time, in the werid log, a large number of TCP_seq/ack_underflow_or_misorder logs are included.
So I want to know why there is such a high rate of packet loss, how to trace the cause, and how to solve it.I look forward to receiving your reply.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191029/b75ed80c/attachment.html 

From richard at corelight.com  Tue Oct 29 11:05:32 2019
From: richard at corelight.com (Richard Bejtlich)
Date: Tue, 29 Oct 2019 14:05:32 -0400
Subject: [Zeek]
	=?utf-8?q?=E3=80=8CFOR_HELP=E3=80=8DThe_mirrored_traffic_i?=
	=?utf-8?q?s_heavily_lost=2E?=
In-Reply-To: <tencent_29D00AE1D4D975279BD59A2B19E2DB907607@qq.com>
References: <tencent_29D00AE1D4D975279BD59A2B19E2DB907607@qq.com>
Message-ID: <CAOtSMjZnuBE8zr_qJoraHDDVfXyvPJ4QX_8SvkP05FqymqwBfg@mail.gmail.com>

Hello,

How are you mirroring the traffic? If it?s a switch span port, that could
be the source of the dropped traffic.

Sincerely,

Richard

On Tue, Oct 29, 2019 at 7:30 AM ??? <1766521944 at qq.com> wrote:

> I mirrored the traffic between the core switch of our computer room and
> the public network firewall, but the zeek report contained a lot of packet
> loss (30%), and currently uses PFring for packet capture. I confirm that
> the hardware is fully capable of handling these packet?"Capture loss" and
> "dropped packets"  have alarms?At the same time, in the werid log, a large
> number of TCP_seq/ack_underflow_or_misorder logs are included.
> So I want to know why there is such a high rate of packet loss, how to
> trace the cause, and how to solve it.I look forward to receiving your reply.
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

-- 
Richard Bejtlich
Principal Security Strategist, Corelight
https://corelight.blog/author/richardbejtlich/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191029/eee2299c/attachment.html 

From austin522 at gmail.com  Tue Oct 29 20:31:39 2019
From: austin522 at gmail.com (venkatesh bandari)
Date: Wed, 30 Oct 2019 11:31:39 +0800
Subject: [Zeek] zeek ts conversion
Message-ID: <CAKPzng_ps2YZzJqUy_2ZGWpJHcRrmcixqmRg8sb9RoqfRa0TJg@mail.gmail.com>

Hello team,

we are doing a zeek poc.iam doing the integration with splunk.in the spunk
logs i see the ts value which is not in human readable
format.zeek-cut/bro-cut  on the box can be used to convert ts to human
readable format using -d

the question is how can i do this before sending the json logs to splunk.is
there a way

Thanks
Venkatesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191030/8da3eb23/attachment.html 

From seth at corelight.com  Thu Oct 31 05:08:40 2019
From: seth at corelight.com (Seth Hall)
Date: Thu, 31 Oct 2019 08:08:40 -0400
Subject: [Zeek] zeek ts conversion
In-Reply-To: <CAKPzng_ps2YZzJqUy_2ZGWpJHcRrmcixqmRg8sb9RoqfRa0TJg@mail.gmail.com>
References: <CAKPzng_ps2YZzJqUy_2ZGWpJHcRrmcixqmRg8sb9RoqfRa0TJg@mail.gmail.com>
Message-ID: <988A8876-2DCD-46A5-95D8-1A4A5D6196A8@corelight.com>

In local.bro, add the following line...

```bro
redef LogAscii::json_timestamps = JSON::TS_ISO8601;
```

That should make your log have timestamps in ISO8601 time format which 
most systems natively recognize and understand.

   .Seth

On 29 Oct 2019, at 23:31, venkatesh bandari wrote:

> Hello team,
>
> we are doing a zeek poc.iam doing the integration with splunk.in the 
> spunk
> logs i see the ts value which is not in human readable
> format.zeek-cut/bro-cut  on the box can be used to convert ts to human
> readable format using -d
>
> the question is how can i do this before sending the json logs to 
> splunk.is
> there a way
>
> Thanks
> Venkatesh
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

--
Seth Hall * Corelight, Inc * www.corelight.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191031/499c4e61/attachment.html 

From seth at corelight.com  Thu Oct 31 05:13:17 2019
From: seth at corelight.com (Seth Hall)
Date: Thu, 31 Oct 2019 08:13:17 -0400
Subject: [Zeek] Seek error when I try to use zeekctl command
In-Reply-To: <9CFD3D6C-87F2-42EB-995E-7F521D6BCCD9@outlook.com>
References: <F6C01A44-B817-4560-B0C4-FEEDE627DD6C@outlook.com>
	<FA119CE6-642D-4488-BC6D-1966DF09F69B@gmail.com>
	<9CFD3D6C-87F2-42EB-995E-7F521D6BCCD9@outlook.com>
Message-ID: <E750D6C4-57BE-4935-9219-D12C9FA428D6@corelight.com>

It's possible that a plugin you have installed from the package manager 
or manually contained a zeekctl/broctl plugin and hasn't been updated 
for Zeek 3.0 yet.  Take a look in /usr/local/bro/lib/zeekctl/plugins to 
see what's there and if one of the plugins there is using BroControl.  
If you find something in there using the BroControl API you could then 
go and file a ticket with the plugin that is causing the issue! :)

   .Seth

On 20 Oct 2019, at 9:44, Carlos Lopez wrote:

> Yep, you are right Ericooi ? There was an error in my nodes.cfg ? 
> Now, it is solved. Many thanks.
>
> On the other side, this:
>
> Warning: ZeekControl plugin uses legacy BroControl API. Use
> 'import ZeekControl.plugin' instead of 'import BroControl.plugin'
>
> .. appears every time zeekctl runs ?
>
> --
> Regards,
> C. L. Martinez
>
> From: "ericooi at gmail.com" <ericooi at gmail.com>
> Date: Sunday, 20 October 2019 at 15:40
> To: Carlos Lopez <clopmz at outlook.com>
> Cc: "zeek at zeek.org" <zeek at zeek.org>
> Subject: Re: [Zeek] Seek error when I try to use zeekctl command
>
> I run Zeek 3.0 on CentOS 8 and I also get this warning, though the 
> error seems to suggest there?s an issue with your 
> /opt/zeek/etc/node.cfg file.
>
>
> On Oct 20, 2019, at 8:17 AM, Carlos Lopez 
> <clopmz at outlook.com<mailto:clopmz at outlook.com>> wrote:
>
> Hi all,
>
> I have doing a clean install on RHEL8 host with Zeek 3.0.0. When I try 
> to use any zeekctl option, it returns the following error:
>
> root at rhel8host:~# zeekctl help
>
> Warning: ZeekControl plugin uses legacy BroControl API. Use
> 'import ZeekControl.plugin' instead of 'import BroControl.plugin'
>
> Error: no type given for node zeek
>
> Maybe the problem is with python3 that comes with RHEL8? Any idea?
>
> --
> Regards,
> C. L. Martinez
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org<mailto:zeek at zeek.org>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek<http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek>
>
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

--
Seth Hall * Corelight, Inc * www.corelight.com


From mauro.palumbo at aizoon.it  Thu Oct 31 08:48:32 2019
From: mauro.palumbo at aizoon.it (Palumbo Mauro)
Date: Thu, 31 Oct 2019 15:48:32 +0000
Subject: [Zeek] ssl established but no validation status
Message-ID: <9c259ea7f7c2466a859cf673da9653f7@SRVEX03.aizoon.local>

Hi there,
   I have a question related to the ssl.log. As I am no expert of the SSL protocol, it is higly probable that I am missing something here.

I noticed in the ssl.log several cases where the field "established" is T, but there is no certificate found (no fuids) and the field validation_status in empty (-). In the code I saw that the field "established" is set to T if the event ssl_established  is generated. Is it possible to establish an ssl session without certificates? Is it because some sessions can be resumed with tickets as described in RFC 5077?

I'd appreciate some help to save me some time...

Mauro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191031/e25a4678/attachment.html 

From johanna at corelight.com  Thu Oct 31 14:15:01 2019
From: johanna at corelight.com (Johanna Amann)
Date: Thu, 31 Oct 2019 22:15:01 +0100
Subject: [Zeek] ssl established but no validation status
In-Reply-To: <9c259ea7f7c2466a859cf673da9653f7@SRVEX03.aizoon.local>
References: <9c259ea7f7c2466a859cf673da9653f7@SRVEX03.aizoon.local>
Message-ID: <E8ABEB03-3DA4-4219-BF59-B1ABFFE5546C@corelight.com>

Hi Mauro,

it is probably resumed connections. An indication for that is that there 
are no server certificates present.

Alternatively - for TLS 1.3 connections validation is not possible 
because the certificates are encrypted.

Johanna


On 31 Oct 2019, at 16:48, Palumbo Mauro wrote:

> Hi there,
>    I have a question related to the ssl.log. As I am no expert of the 
> SSL protocol, it is higly probable that I am missing something here.
>
> I noticed in the ssl.log several cases where the field "established" 
> is T, but there is no certificate found (no fuids) and the field 
> validation_status in empty (-). In the code I saw that the field 
> "established" is set to T if the event ssl_established  is generated. 
> Is it possible to establish an ssl session without certificates? Is it 
> because some sessions can be resumed with tickets as described in RFC 
> 5077?
>
> I'd appreciate some help to save me some time...
>
> Mauro
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

From jlay at slave-tothe-box.net  Thu Oct 31 14:46:18 2019
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 31 Oct 2019 15:46:18 -0600
Subject: [Zeek] Status of 3 plugins
Message-ID: <d44d529d10c95a4a4e982157151cff05@slave-tothe-box.net>

Well here I am.  Zeek 3 was released Sep 23rd, and I'm dead in the water 
until these are updated:

https://github.com/J-Gras/bro-af_packet-plugin
https://github.com/salesforce/ja3/tree/master/bro
https://github.com/J-Gras/intel-seen-more

if anyone has inside communication channels or some other form of 
digital cattle prod I'd love it if you could motivate the above to get 
to Zeek 3 compatibility.  Truth be told I haven't even been able to 
start testing yet due to the missing plugins I use.

Thank you.

James

From kilotao at gmail.com  Thu Oct 31 15:21:48 2019
From: kilotao at gmail.com (kilotao at gmail.com)
Date: Thu, 31 Oct 2019 18:21:48 -0400
Subject: [Zeek] zeek3.0.0 high memory usage
Message-ID: <CAC_rUhLaFnPE9JQSLwoMe=gAwvQwiu-5TMFnKtvGhpYYfxbN-A@mail.gmail.com>

After upgrading to zeek 3.0.0, we noticed that memory utilization on zeek
workers were constantly at 1G with vsize=1G.  It was about half of the
usage with 2.6.1.  Any ideas?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191031/1f7ae2b8/attachment.html 

From justin at corelight.com  Thu Oct 31 15:30:08 2019
From: justin at corelight.com (Justin Azoff)
Date: Thu, 31 Oct 2019 18:30:08 -0400
Subject: [Zeek] Status of 3 plugins
In-Reply-To: <d44d529d10c95a4a4e982157151cff05@slave-tothe-box.net>
References: <d44d529d10c95a4a4e982157151cff05@slave-tothe-box.net>
Message-ID: <CAPfnCugK8gvpT=KjkdQHj=Fr6gOLR+iFPaqO0potzZ738=ucZQ@mail.gmail.com>

On Thu, Oct 31, 2019 at 5:55 PM James Lay <jlay at slave-tothe-box.net> wrote:

> Well here I am.  Zeek 3 was released Sep 23rd, and I'm dead in the water
> until these are updated:
>
> https://github.com/J-Gras/bro-af_packet-plugin


appears to work just fine.

https://github.com/salesforce/ja3/tree/master/bro


Other than a warning about using bro_init, works fine.  The code that
causes this looks like dead code that isn't even used anyway, so they can
just be deleted.  There's also the hosom/bro-ja3 which is a cleaned up
version and works without warnings.   The test suite it includes fails to
run properly (fixable btest issues), but installing anyway results in a
functional package.

https://github.com/J-Gras/intel-seen-more


Looks like this hits the issue where it depends on bro/something which are
now all zeek/something  I fixed this in one of my packages by just deleting
the "bro/" part of the dependency but I think this is more a migration
issue that zkg could help resolve.


> if anyone has inside communication channels or some other form of
> digital cattle prod I'd love it if you could motivate the above to get
> to Zeek 3 compatibility.  Truth be told I haven't even been able to
> start testing yet due to the missing plugins I use.
>

Well, 2 out of the 3 already work, and one just needs a minor update that
I'm sure Jan would be happy to make if someone had just told him about it.

-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191031/4aff7389/attachment.html 

From michalpurzynski1 at gmail.com  Thu Oct 31 15:42:54 2019
From: michalpurzynski1 at gmail.com (=?utf-8?Q?Micha=C5=82_Purzy=C5=84ski?=)
Date: Thu, 31 Oct 2019 15:42:54 -0700
Subject: [Zeek] Status of 3 plugins
In-Reply-To: <CAPfnCugK8gvpT=KjkdQHj=Fr6gOLR+iFPaqO0potzZ738=ucZQ@mail.gmail.com>
References: <CAPfnCugK8gvpT=KjkdQHj=Fr6gOLR+iFPaqO0potzZ738=ucZQ@mail.gmail.com>
Message-ID: <51C69B9C-D744-4C7F-B8B7-ADAF8FECDC3D@gmail.com>

All of them seem to work fine here on Zeek 3 pre-release.

> On Oct 31, 2019, at 3:33 PM, Justin Azoff <justin at corelight.com> wrote:
> 
> ?
>> On Thu, Oct 31, 2019 at 5:55 PM James Lay <jlay at slave-tothe-box.net> wrote:
> 
>> Well here I am.  Zeek 3 was released Sep 23rd, and I'm dead in the water 
>> until these are updated:
>> 
>> https://github.com/J-Gras/bro-af_packet-plugin
> 
> appears to work just fine.
> 
>> https://github.com/salesforce/ja3/tree/master/bro
> 
> Other than a warning about using bro_init, works fine.  The code that causes this looks like dead code that isn't even used anyway, so they can just be deleted.  There's also the hosom/bro-ja3 which is a cleaned up version and works without warnings.   The test suite it includes fails to run properly (fixable btest issues), but installing anyway results in a functional package.
> 
>> https://github.com/J-Gras/intel-seen-more
> 
> Looks like this hits the issue where it depends on bro/something which are now all zeek/something  I fixed this in one of my packages by just deleting the "bro/" part of the dependency but I think this is more a migration issue that zkg could help resolve.
>  
>> if anyone has inside communication channels or some other form of 
>> digital cattle prod I'd love it if you could motivate the above to get 
>> to Zeek 3 compatibility.  Truth be told I haven't even been able to 
>> start testing yet due to the missing plugins I use.
> 
> Well, 2 out of the 3 already work, and one just needs a minor update that I'm sure Jan would be happy to make if someone had just told him about it.
> 
> -- 
> Justin
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20191031/9892faf5/attachment.html