[Bro] bro elasticsearch plugin + kibana indexing

Wed Mar 8 12:42:29 PST 2017

How TS_ISO8601 timestamp looks like? I ll try to recompile maybe i didn't
apply the patch correctly. Thanks again a lot for your help

2017-03-05 15:41 GMT+02:00 Daniel Guerra <daniel.guerra69 at gmail.com>:

> The patch wasn’t used, your timestamp is not in TS_ISO8601 but in TS_MILIS
> And your separator is a “.” not a “_"
>
> Check my docker
>
> https://hub.docker.com/r/danielguerra/bro-debian-elasticsearch/
>
> On 05 Mar 2017, at 14:12, Alex Kefallonitis <al.kefallonitis at gmail.com>
> wrote:
>
> Nothing changed
>
> <image.png>
>
>
> Although i have logs
>
> tail -f /opt/bro/logs/current/conn.log
> {"ts":1488719244.873684,"uid":"CCToVE1JzVl9n5zDnj","id.orig_
> h":"10.0.0.31","id.orig_p":123,"id.resp_h":"194.177.210.
> 54","id.resp_p":123,"proto":"udp","duration":0.021199,"
> orig_bytes":0,"resp_bytes":48,"conn_state":"SHR","local_
> orig":true,"local_resp":false,"missed_bytes":0,"history":"
> Cd","orig_pkts":0,"orig_ip_bytes":0,"resp_pkts":1,"resp_
> ip_bytes":76,"tunnel_parents":[],"orig_l2_addr":"b8:27:eb:
> 68:1a:49","resp_l2_addr":"36:34:64:31:64:39"}
> {"ts":1488719252.873686,"uid":"ChHXxg3NsigjS6QwXg","id.orig_
> h":"10.0.0.31","id.orig_p":123,"id.resp_h":"62.1.45.120",
> "id.resp_p":123,"proto":"udp","duration":0.020193,"orig_
> bytes":0,"resp_bytes":48,"conn_state":"SHR","local_orig"
> :true,"local_resp":false,"missed_bytes":0,"history":"Cd"
> ,"orig_pkts":0,"orig_ip_bytes":0,"resp_pkts":1,"resp_ip_
> bytes":76,"tunnel_parents":[],"orig_l2_addr":"b8:27:eb:68:
> 1a:49","resp_l2_addr":"36:34:64:31:64:39"}
> {"ts":1488719249.686949,"uid":"CIrHst2VsHafEIR4vk","id.orig_
> h":"10.0.0.3","id.orig_p":123,"id.resp_h":"91.189.89.198","
> id.resp_p":123,"proto":"udp","duration":0.066331,"orig_
> bytes":48,"resp_bytes":48,"conn_state":"SF","local_orig":
> true,"local_resp":false,"missed_bytes":0,"history":"Dd"
> ,"orig_pkts":1,"orig_ip_bytes":76,"resp_pkts":1,"resp_ip_
> bytes":76,"tunnel_parents":[],"orig_l2_addr":"32:38:66:64:
> 64:62","resp_l2_addr":"36:34:64:31:64:39"}
> {"ts":1488719306.835847,"uid":"CqrMPYWEW543RxOX5","id.orig_
> h":"10.0.0.33","id.orig_p":51666,"id.resp_h":"10.0.0.1","
> id.resp_p":53,"proto":"udp","service":"dns","duration":0.
> 000744,"orig_bytes":39,"resp_bytes":98,"conn_state":"SF","
> local_orig":true,"local_resp":true,"missed_bytes":0,"
> history":"Dd","orig_pkts":1,"orig_ip_bytes":67,"resp_pkts":
> 1,"resp_ip_bytes":126,"tunnel_parents":[],"orig_l2_addr":"
> 36:33:62:63:39:61","resp_l2_addr":"36:34:64:31:64:39"}
> {"ts":1488719314.06168,"uid":"CYw1dj2WUBXn6ua8O1","id.orig_
> h":"10.0.0.31","id.orig_p":37456,"id.resp_h":"10.0.0.1","
> id.resp_p":53,"proto":"udp","service":"dns","duration":0.
> 000596,"orig_bytes":0,"resp_bytes":172,"conn_state":"SHR",
> "local_orig":true,"local_resp":true,"missed_bytes":0,"
> history":"Cd","orig_pkts":0,"orig_ip_bytes":0,"resp_pkts":
> 2,"resp_ip_bytes":228,"tunnel_parents":[],"orig_l2_addr":"
> b8:27:eb:68:1a:49","resp_l2_addr":"36:34:64:31:64:39"}
>
> Do i have to change serator to init.bro also?
>
>
> 2017-03-05 14:57 GMT+02:00 Daniel Guerra <daniel.guerra69 at gmail.com>:
>
>> ##! Load this script to enable global log output to an ElasticSearch
>> database.
>>
>> module LogElasticSearch;
>>
>> export {
>> ## An elasticsearch specific rotation interval.
>> const rotation_interval = 1hr &redef;
>>
>> ## Optionally ignore any :bro:type:`Log::ID` from being sent to
>> ## ElasticSearch with this script.
>> const excluded_log_ids: set[Log::ID] &redef;
>>
>> ## If you want to explicitly only send certain :bro:type:`Log::ID`
>> ## streams, add them to this set.  If the set remains empty, all will
>> ## be sent.  The :bro:id:`LogElasticSearch::excluded_log_ids` option
>> ## will remain in effect as well.
>> const send_logs: set[Log::ID] &redef;
>>
>> ## Set the separator
>> redef Log::default_scope_sep = "_";
>> }
>>
>>
>> On 05 Mar 2017, at 13:56, Alex Kefallonitis <al.kefallonitis at gmail.com>
>> wrote:
>>
>> Where do i put this?
>>
>> redef Log::default_scope_sep = "_”;
>>
>> Do i have to enable json output to ascii.bro?
>>
>> 2017-03-05 14:32 GMT+02:00 Daniel Guerra <daniel.guerra69 at gmail.com>:
>>
>>> Don’t forget this in the bro script that starts elasticsearch in the
>>> export part
>>>
>>> redef Log::default_scope_sep = "_”;
>>>
>>>
>>> On 05 Mar 2017, at 11:22, Alex Kefallonitis <al.kefallonitis at gmail.com>
>>> wrote:
>>>
>>> I do patch src/ElasticSearch.cc <http://elasticsearch.cc/>
>>> ./ElasticSearch.cc.patch ./configure && make && make install . Load bro
>>> elasticsearch script and restart bro open kibana
>>>
>>> <image.png>
>>>
>>> 2017-03-05 12:14 GMT+02:00 Alex Kefallonitis <al.kefallonitis at gmail.com>
>>> :
>>>
>>>> I try the patch too but still no timestamp appears i am using ELK 5.2.2
>>>>
>>>> 2017-03-05 10:27 GMT+02:00 Daniel Guerra <daniel.guerra69 at gmail.com>:
>>>>
>>>>> Try this
>>>>>
>>>>> https://github.com/danielguerra69/bro-debian-elasticsearch/b
>>>>> lob/master/bro-patch/ElasticSearch.cc.patch
>>>>>
>>>>>
>>>>> > On 05 Mar 2017, at 02:57, Alex Kefallonitis <
>>>>> al.kefallonitis at gmail.com> wrote:
>>>>> >
>>>>> > ELK + Kibana not indexing bro logs
>>>>> >
>>>>> > Succesfully installed the plugin and ELK but when i add indexing
>>>>> bro-* , index time-field appears empty (@timestamp) so i cannot use bro
>>>>> logs with kibana search. Anyone have same issue?
>>>>> > _______________________________________________
>>>>> > Bro mailing list
>>>>> > bro at bro-ids.org
>>>>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>>> <http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170308/3fd09c8d/attachment-0001.html 