[Zeek] Zeek 3.1.2 and Kafka - No data flow

Zeolla@GMail.com zeolla at gmail.com
Mon Apr 27 03:39:29 PDT 2020


I have not run it on 3.1.2 yet but I recommend making your changes to the
plugin and running the end to end testing script at
https://github.com/apache/metron-bro-plugin-kafka/blob/master/docker/run_end_to_end.sh

It was meant to help isolate issues when making changes to the plugin.
Also, we welcome PRs against the project so please feel free to
contribute.  Thanks,

Jon Zeolla

On Sun, Apr 26, 2020, 9:12 PM Hovsep Levi <hovsep.sanjay.levi at gmail.com>
wrote:

> Hello Zeeks
>
>
> Has anyone succeeded to enable Kafka plugin with Zeek 3.1.2 ?  I am trying
> to modernize the metron-kafka plugin and have partial success.  My problem
> seems to be with script-land referencing.
>
> The logger node is loading the plugin OK and connects to the Kafka
> broker.  The broker IP is redef information found from site/local.zeek.
>
> $ bin/zeekctl diag logger-1
> [logger-1]
>
> No core file found.
>
> Zeek 3.1.2-debug
>
> Zeek plugins:
> Apache::Kafka - Writes logs to Kafka (dynamic, version 0.3.0)
>
> ==== No reporter.log
>
> ==== stderr.log
> %7|1587948661.341|RECV|rdkafka#producer-3| [..kafka messages..] ..
>
>
>
>
> But the worker node has a problem referencing existing variable
> declaration.  The logs-to-kafka.bro script expects it.  There is also
> suspicion with the Zeek plugins info that is different from the logger node
> and maybe the problem.
>
> $ bin/zeekctl diag worker-1-1
> [worker-1-1]
>
> No core file found.
>
> Zeek 3.1.2-debug
>
> Zeek plugins: (none found)   <<< ??? Normal for worker node ???
>
> ==== No reporter.log
>
> ==== stderr.log
>
> error in
> /opt/zeek/spool/installed-scripts-do-not-touch/site/custom_plugins/APACHE_KAFKA/scripts/Apache/Kafka/./logs-to-kafka.bro,
> line 24: unknown identifier logs_to_send, at or near "logs_to_send"
>
>
>
> The configuration is not default and explained below:
>
>
> The Kafka logger was installed to site/custom_plugins/APACHE_KAFKA
>
>
> share/zeek/site/local.zeek uses:
>
> @load custom_plugins/APACHE_KAFKA/scripts/Apache/Kafka
>
>
>
> lib/zeek/plugins/custom_plugins is a symlink to
> share/zeek/site/custom_plugins
>
>
> Using the lib symlink seems to be the only way to load the plugin, then
> the @load statement brings redef customizations and scripts.  This works ok
> for the logger node but not the worker who cannot interface with the plugin
> ?
>
> Another idea is have non-logger nodes bypass loading logs-to-kafka.bro but
> this isn't fully understood.
>
>
> TIA
>
> /hovsep
>
>
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200427/835fc3bd/attachment.html 


More information about the Zeek mailing list