[Zeek] Workers occasionally using 102% CPU

[Doug Burks]

I've been able to duplicate this issue so I'm passing along some notes in
the hope that others are able to duplicate the issue as well and perhaps
pinpoint what's going on.

- I've seen this issue on both physical boxes and virtual machines and on
both single CPU socket and multiple CPU socket systems

- I've been able to trigger this issue fairly consistently using VMware
Workstation with the VM set to 4 processors (seems easier to duplicate when
using processors rather than cores)

- 8GB RAM and 2 NICs (one set to NAT for management and the other set to a
custom network and configured for sniffing)

- running our latest Security Onion ISO image which contains Zeek 3.0.1
(I've also duplicated this behavior using Zeek 3.0.2 compiled manually):
https://blog.securityonion.net/2020/02/security-onion-160464-iso-image-now.html

- run sosetup-minimal and choose Evaluation Mode

- once Setup is complete, create some traffic on the sniffing interface:
while :; do sudo so-replay; done

- on my box, Zeek normally runs at about 10% to 20% CPU usage when running
so-replay but after a certain period of time (seems inconsistent, could be
minutes or over an hour), Zeek will go to 100% CPU usage and remain there
even if you kill the so-replay while loop from above

- you can restart Zeek with "sudo so-zeek-restart" and it will go back to
normal operation and normal CPU usage, but after a while of processing
traffic it will go back to 100% CPU usage

- as mentioned above, you can also download Zeek 3.0.2 and compile it
manually according to https://docs.zeek.org/en/v3.0.2/install/install.html
and duplicate the issue there, so this would seem to rule out any possible
issues with our Zeek package or scripts

Please let me know if I can provide any further information to assist in
duplicating and pinpointing this issue.

Thanks!


On Tue, Feb 25, 2020 at 6:41 PM Pete Nelson <petiepooo at gmail.com> wrote:

> Thanks, Jon.
>
> I'll try to digest those links and dig into the code.  Unfortunately,
> it seems running strace on the process keeps it from occurring...  I
> may try to get dtrace working in place, but I need to improve my lab
> setup first before I go too crazy.
> --
> Pete
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>


-- 
Doug Burks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200304/1c3ea34e/attachment.html