<div dir="ltr">All,<div><br></div><div>This is helpful. Other than my sysadmins' preference, is there any reason to choose one or the other, between rsyslog and syslog-ng?</div><div><br></div><div>Thanks,</div><div>Jim</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 6, 2020 at 3:00 PM <<a href="mailto:zeek-request@zeek.org">zeek-request@zeek.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Send Zeek mailing list submissions to<br>
<a href="mailto:zeek@zeek.org" target="_blank">zeek@zeek.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek" rel="noreferrer" target="_blank">http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:zeek-request@zeek.org" target="_blank">zeek-request@zeek.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:zeek-owner@zeek.org" target="_blank">zeek-owner@zeek.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of Zeek digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: Zeek with ELK (Darren S.)<br>
2. Re: Zeek with ELK (duhang)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Sun, 5 Jan 2020 17:40:02 -0700<br>
From: "Darren S." <<a href="mailto:phatbuckett@gmail.com" target="_blank">phatbuckett@gmail.com</a>><br>
Subject: Re: [Zeek] Zeek with ELK<br>
To: sec-x sec-x <<a href="mailto:center.mnt@gmail.com" target="_blank">center.mnt@gmail.com</a>><br>
Cc: <a href="mailto:Zeek@zeek.org" target="_blank">Zeek@zeek.org</a><br>
Message-ID:<br>
<<a href="mailto:CAKVSOJWSnfCrzPyreChQtVVU5LS1yEKO3ufV3S2ZdyJXwj4-9w@mail.gmail.com" target="_blank">CAKVSOJWSnfCrzPyreChQtVVU5LS1yEKO3ufV3S2ZdyJXwj4-9w@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="UTF-8"<br>
<br>
I read OP question as "I have Zeek running on FreeBSD, what is a<br>
sensible option for shipping logs from the sensor to an Elastic<br>
Stack?" Apologies if it's the wrong read.<br>
<br>
In that case I wouldn't want to install either the whole stack nor<br>
even Logstash on the sensor as it alone tends to consume an excessive<br>
amount of memory, not what you want on a sensor. Filebeat (a small<br>
footprint data collector/shipper) is the way to go if you're shipping<br>
remotely.<br>
<br>
If Filebeat isn't an option on the platform, maybe explore Fluent Bit:<br>
<br>
<a href="https://github.com/fluent/fluent-bit" rel="noreferrer" target="_blank">https://github.com/fluent/fluent-bit</a><br>
<a href="https://fluentbit.io/" rel="noreferrer" target="_blank">https://fluentbit.io/</a><br>
<br>
Fluent Bit can output directly to Elasticsearch:<br>
<a href="https://fluentbit.io/documentation/0.14/output/elasticsearch.html" rel="noreferrer" target="_blank">https://fluentbit.io/documentation/0.14/output/elasticsearch.html</a><br>
<br>
Even a Fluentd can run with typically lower memory consumption than<br>
Logstash, so perhaps worth exploring both/either:<br>
<br>
<a href="https://github.com/fluent/fluentd" rel="noreferrer" target="_blank">https://github.com/fluent/fluentd</a><br>
<a href="https://www.fluentd.org/" rel="noreferrer" target="_blank">https://www.fluentd.org/</a><br>
<br>
Fluentd can also output to Elasticsearch:<br>
<a href="https://docs.fluentd.org/output/elasticsearch" rel="noreferrer" target="_blank">https://docs.fluentd.org/output/elasticsearch</a><br>
<br>
There are other options for shippers too, such as Syslog-ng:<br>
<a href="https://www.syslog-ng.com/community/b/blog/posts/logging-to-elasticsearch-made-simple-with-syslog-ng" rel="noreferrer" target="_blank">https://www.syslog-ng.com/community/b/blog/posts/logging-to-elasticsearch-made-simple-with-syslog-ng</a><br>
<br>
- Darren<br>
<br>
On Sun, Jan 5, 2020 at 9:11 AM Michael Shirk <<a href="mailto:shirkdog.bsd@gmail.com" target="_blank">shirkdog.bsd@gmail.com</a>> wrote:<br>
><br>
> You should be able to fire up Elastic, Logstash and Kibana on FreeBSD, using recommend Logstash configs to read in the log files from the file system. I can check about the Filebeat port to see if that can be updated or fixed.<br>
><br>
> I myself just use the CLI tools but have been working on something "Not Java" to ingest log files into other than Splunk.<br>
><br>
><br>
> --<br>
> Michael Shirk<br>
> Daemon Security, Inc.<br>
> <a href="https://www.daemon-security.com" rel="noreferrer" target="_blank">https://www.daemon-security.com</a><br>
><br>
> On Sun, Jan 5, 2020, 10:35 sec-x sec-x <<a href="mailto:center.mnt@gmail.com" target="_blank">center.mnt@gmail.com</a>> wrote:<br>
>><br>
>> Hi,<br>
>><br>
>> I recently used zeek IDS on FreeBSD 12.1 - Default Policy (GetTraffic<br>
>> from TAP on the network) and i want to send all the logs to ELK in<br>
>> realtime.<br>
>><br>
>> I saw Filebeat ports on BSD is old and has problems.<br>
>><br>
>> How can i send the logs from the BSD to the Elastic (what is the<br>
>> correct/best way)?<br>
>><br>
>><br>
>> Thanks,<br>
>><br>
>> CM.<br>
>> _______________________________________________<br>
>> Zeek mailing list<br>
>> <a href="mailto:zeek@zeek.org" target="_blank">zeek@zeek.org</a><br>
>> <a href="http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek" rel="noreferrer" target="_blank">http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek</a><br>
><br>
> _______________________________________________<br>
> Zeek mailing list<br>
> <a href="mailto:zeek@zeek.org" target="_blank">zeek@zeek.org</a><br>
> <a href="http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek" rel="noreferrer" target="_blank">http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek</a><br>
<br>
<br>
<br>
-- <br>
Darren Spruell<br>
<a href="mailto:phatbuckett@gmail.com" target="_blank">phatbuckett@gmail.com</a><br>
<br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Mon, 6 Jan 2020 09:52:52 +0800<br>
From: duhang <<a href="mailto:darkheaven1983@gmail.com" target="_blank">darkheaven1983@gmail.com</a>><br>
Subject: Re: [Zeek] Zeek with ELK<br>
To: sec-x sec-x <<a href="mailto:center.mnt@gmail.com" target="_blank">center.mnt@gmail.com</a>><br>
Cc: <a href="mailto:Zeek@zeek.org" target="_blank">Zeek@zeek.org</a><br>
Message-ID:<br>
<<a href="mailto:CAG%2ByijM94rhd5m9PifrbnEAf1yRii-N4aWA8-qfDJzCGnr9u9Q@mail.gmail.com" target="_blank">CAG+yijM94rhd5m9PifrbnEAf1yRii-N4aWA8-qfDJzCGnr9u9Q@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
You can try rsyslog imfile module to send logs to logstash. The following<br>
is my configuration.<br>
<br>
$ModLoad imfile<br>
$InputFileName /usr/local/bro/logs/current/dns.log<br>
$InputFileTag dns:<br>
$InputFileStateFile stat-dns<br>
$InputFileSeverity info<br>
$InputFileFacility local2<br>
$InputRunFileMonitor<br>
<br>
$SystemLogRateLimitInterval 0<br>
$SystemLogRateLimitBurst 0<br>
$MaxMessageSize 64k<br>
<br>
sec-x sec-x <<a href="mailto:center.mnt@gmail.com" target="_blank">center.mnt@gmail.com</a>> ?2020?1?5??? ??11:36???<br>
<br>
> Hi,<br>
><br>
> I recently used zeek IDS on FreeBSD 12.1 - Default Policy (GetTraffic<br>
> from TAP on the network) and i want to send all the logs to ELK in<br>
> realtime.<br>
><br>
> I saw Filebeat ports on BSD is old and has problems.<br>
><br>
> How can i send the logs from the BSD to the Elastic (what is the<br>
> correct/best way)?<br>
><br>
><br>
> Thanks,<br>
><br>
> CM.<br>
> _______________________________________________<br>
> Zeek mailing list<br>
> <a href="mailto:zeek@zeek.org" target="_blank">zeek@zeek.org</a><br>
> <a href="http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek" rel="noreferrer" target="_blank">http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek</a><br>
><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <a href="http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200106/91801d18/attachment-0001.html" rel="noreferrer" target="_blank">http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200106/91801d18/attachment-0001.html</a> <br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
Zeek mailing list<br>
<a href="mailto:Zeek@zeek.org" target="_blank">Zeek@zeek.org</a><br>
<a href="http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek" rel="noreferrer" target="_blank">http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek</a><br>
<br>
<br>
End of Zeek Digest, Vol 165, Issue 6<br>
************************************<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><span style="font-family:Arial,sans-serif;color:black">Jim Offer<br>Network Security Analyst<br>Saint Joseph's University<br>(610) 660-<span>1573</span></span></div></div></div></div>