[Zeek] Kafka plugin causes logger to segfault

[Patrick Cain]

Hi,

You don't say what version you're running, but with 2.5 and 2.6 I use these
lines along with the kafka config:

### JSON LOGGING
@load tuning/json-logs
# Set the log separator
redef Log::default_scope_sep = "_";
# Set the time in iso format
redef LogAscii::json_timestamps = JSON::TS_ISO8601;

Your kafka config looks close to mine (I leave the topic_name field blank.)
My kafka emitter has been running on Centos 6, Centos 7 and RHEL7 systems
for about a year.
Can you manually connect to your broker from the zeek box?  I have had
issues in the past when the logger was happy but other things in the pipe to
zookeeper and kafka were unhappy.

Pat
-----Original Message-----
From: zeek-bounces at zeek.org <zeek-bounces at zeek.org> On Behalf Of Weasel,
Gary W CIV DISA RE (US)
Sent: Monday, April 22, 2019 11:10 AM
To: 'zeek at zeek.org' <zeek at zeek.org>
Subject: [Zeek] Kafka plugin causes logger to segfault

All,

I'm currently at my wits end dealing with the Kafka plugin, I'm having great
difficulty stopping it from crashing.

When I use the library of librdkafka as prescribed from
https://packages.zeek.org/packages/view/7388aa77-4fb7-11e8-88be-0a645a3f3086
(librdkafka-0.11.5), my logger crashes immediately after startup.  When
using an alternative version of librdkafka
(librdkakfa1-0.11.4_confluent4.1.3) the logger doesn't immediately crash but
within a minute of starting it usually does.

The stderr.log says the same every time, /run-bro: line 110: <pid>
Segmentation fault   nohup "$mybro" "$@"

I have downloaded the most recent version of
https://github.com/apache/metron-bro-plugin-kafka and still experience this.

I am building an RPM (running CentOS) for the Kafka plugin and installing
that way, since the box is offline and unable to reach bro-packages.  When I
tried to use librdkafka-0.11.5 I've also built an RPM for that.

The following is my only added configuration

@load Apache/Kafka/logs-to-kafka.bro
redef Kafka::logs_to_send = set(Conn::LOG); redef Kafka::kafka_conf = table(
        ["metadata.broker.list"] = "172.16.0.40.9092"
);
redef Kafka::topic_name = "bro";
redef Kafka::tag_json = T;

The interesting thing to note: the logger does not crash if no logs are
being sent (i.e. I comment out the logs_to_send line).

The only other plugins I'm running are Bro::AF_Packet and
Corelight::CommunityID.

Anyone have any insight or doing something different?

v/r
Gary


_______________________________________________
Zeek mailing list
zeek at zeek.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek