[Zeek] Zeek 3.1.2 and Kafka - No data flow

Hovsep Levi hovsep.sanjay.levi at gmail.com
Sun Apr 26 18:10:58 PDT 2020


Hello Zeeks


Has anyone succeeded to enable Kafka plugin with Zeek 3.1.2 ?  I am trying
to modernize the metron-kafka plugin and have partial success.  My problem
seems to be with script-land referencing.

The logger node is loading the plugin OK and connects to the Kafka broker.
The broker IP is redef information found from site/local.zeek.

$ bin/zeekctl diag logger-1
[logger-1]

No core file found.

Zeek 3.1.2-debug

Zeek plugins:
Apache::Kafka - Writes logs to Kafka (dynamic, version 0.3.0)

==== No reporter.log

==== stderr.log
%7|1587948661.341|RECV|rdkafka#producer-3| [..kafka messages..] ..




But the worker node has a problem referencing existing variable
declaration.  The logs-to-kafka.bro script expects it.  There is also
suspicion with the Zeek plugins info that is different from the logger node
and maybe the problem.

$ bin/zeekctl diag worker-1-1
[worker-1-1]

No core file found.

Zeek 3.1.2-debug

Zeek plugins: (none found)   <<< ??? Normal for worker node ???

==== No reporter.log

==== stderr.log

error in
/opt/zeek/spool/installed-scripts-do-not-touch/site/custom_plugins/APACHE_KAFKA/scripts/Apache/Kafka/./logs-to-kafka.bro,
line 24: unknown identifier logs_to_send, at or near "logs_to_send"



The configuration is not default and explained below:


The Kafka logger was installed to site/custom_plugins/APACHE_KAFKA


share/zeek/site/local.zeek uses:

@load custom_plugins/APACHE_KAFKA/scripts/Apache/Kafka



lib/zeek/plugins/custom_plugins is a symlink to
share/zeek/site/custom_plugins


Using the lib symlink seems to be the only way to load the plugin, then
the @load statement brings redef customizations and scripts.  This works ok
for the logger node but not the worker who cannot interface with the plugin
?

Another idea is have non-logger nodes bypass loading logs-to-kafka.bro but
this isn't fully understood.


TIA

/hovsep
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200427/e9f7b683/attachment.html 


More information about the Zeek mailing list