[Zeek] Kafka plugin causes logger to segfault

[Zeolla@GMail.com]

172.16.0.40.9092 doesn't appear to be an IP address to me.  Did you mean
172.16.0.40:9092?

- Jon Zeolla
Zeolla at GMail.Com


On Tue, Apr 23, 2019 at 3:16 PM Patrick Cain <pcain at coopercain.com> wrote:

> Hi,
>
> You don't say what version you're running, but with 2.5 and 2.6 I use these
> lines along with the kafka config:
>
> ### JSON LOGGING
> @load tuning/json-logs
> # Set the log separator
> redef Log::default_scope_sep = "_";
> # Set the time in iso format
> redef LogAscii::json_timestamps = JSON::TS_ISO8601;
>
> Your kafka config looks close to mine (I leave the topic_name field blank.)
> My kafka emitter has been running on Centos 6, Centos 7 and RHEL7 systems
> for about a year.
> Can you manually connect to your broker from the zeek box?  I have had
> issues in the past when the logger was happy but other things in the pipe
> to
> zookeeper and kafka were unhappy.
>
> Pat
> -----Original Message-----
> From: zeek-bounces at zeek.org <zeek-bounces at zeek.org> On Behalf Of Weasel,
> Gary W CIV DISA RE (US)
> Sent: Monday, April 22, 2019 11:10 AM
> To: 'zeek at zeek.org' <zeek at zeek.org>
> Subject: [Zeek] Kafka plugin causes logger to segfault
>
> All,
>
> I'm currently at my wits end dealing with the Kafka plugin, I'm having
> great
> difficulty stopping it from crashing.
>
> When I use the library of librdkafka as prescribed from
>
> https://packages.zeek.org/packages/view/7388aa77-4fb7-11e8-88be-0a645a3f3086
> (librdkafka-0.11.5
> <https://packages.zeek.org/packages/view/7388aa77-4fb7-11e8-88be-0a645a3f3086(librdkafka-0.11.5>),
> my logger crashes immediately after startup.  When
> using an alternative version of librdkafka
> (librdkakfa1-0.11.4_confluent4.1.3) the logger doesn't immediately crash
> but
> within a minute of starting it usually does.
>
> The stderr.log says the same every time, /run-bro: line 110: <pid>
> Segmentation fault   nohup "$mybro" "$@"
>
> I have downloaded the most recent version of
> https://github.com/apache/metron-bro-plugin-kafka and still experience
> this.
>
> I am building an RPM (running CentOS) for the Kafka plugin and installing
> that way, since the box is offline and unable to reach bro-packages.  When
> I
> tried to use librdkafka-0.11.5 I've also built an RPM for that.
>
> The following is my only added configuration
>
> @load Apache/Kafka/logs-to-kafka.bro
> redef Kafka::logs_to_send = set(Conn::LOG); redef Kafka::kafka_conf =
> table(
>         ["metadata.broker.list"] = "172.16.0.40.9092"
> );
> redef Kafka::topic_name = "bro";
> redef Kafka::tag_json = T;
>
> The interesting thing to note: the logger does not crash if no logs are
> being sent (i.e. I comment out the logs_to_send line).
>
> The only other plugins I'm running are Bro::AF_Packet and
> Corelight::CommunityID.
>
> Anyone have any insight or doing something different?
>
> v/r
> Gary
>
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
> _______________________________________________
> Zeek mailing list
> zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190423/e3d08845/attachment.html