[Bro] Help with Bro & ES

[Azoff, Justin S]

One doesn't need to disable it entirely, you can just not send it to ES by using

redef LogElasticSearch::excluded_log_ids += {LoadedScripts::LOG};
	
-- 
- Justin Azoff

> On Oct 22, 2015, at 12:09 PM, Michael Wenthold <michael.wenthold at gmail.com> wrote:
> 
> The easiest way might be to just disable the loaded scripts log by adding this to local.bro:
> 
> event bro_init()
>     {
>     Log::disable_stream(LoadedScripts::LOG);
>     }
> 
> 
> 
> On Thu, Oct 22, 2015 at 1:39 PM, Chris Williams <cw13 at umbc.edu> wrote:
> Justin, 
> 
> I understand that "loaded scripts" is special, but it is creating issues with the index because there are no timestamps. Is there a recommended method for handling these messages?
> --
> Daniel- my connlog looks the same with the exception of "ts" which I assume you changed?
> --
> 
> Thanks
> 
> 
> On Wed, Oct 21, 2015 at 2:20 PM, Daniel Guerra <daniel.guerra69 at gmail.com> wrote:
> I’m working on elastic for a while
> I changed a bit on the elasticsearch source,
> recompile after doing this.
> sed -i "s/JSON::TS_MILLIS/JSON::TS_ISO8601/g" bro/aux/plugins/elasticsearch/src/ElasticSearch.cc
> 
> my conn.log looks like this in kibana check ts difference
> 
> {
>   "_index": "bro-201509160700",
>   "_type": "conn",
>   "_id": "AVCLfROKixyabuRJCOlt",
>   "_score": null,
>   "_source": {
>     "_timestamp": 1442388234879,
>     "ts": "2015-09-16T07:16:54.185442Z",
>     "uid": "Cv7R6a19zHzfu1H6U4",
>     "id.orig_h": "192.168.1.122",
>     "id.orig_p": 49428,
>     "id.resp_h": "192.168.102.97",
>     "id.resp_p": 514,
>     "proto": "udp",
>     "duration": 360.241984,
>     "orig_bytes": 32096,
>     "resp_bytes": 0,
>     "conn_state": "S0",
>     "missed_bytes": 0,
>     "history": "D",
>     "orig_pkts": 191,
>     "orig_ip_bytes": 37444,
>     "resp_pkts": 0,
>     "resp_ip_bytes": 0,
>     "tunnel_parents": []
>   }
> 
> The next chalenge are coordinates from geoip ..
> 
>> On 21 Oct 2015, at 18:32, Chris Williams <cw13 at umbc.edu> wrote:
>> 
>> I mean... I think they look ok. Again, I understand that I have to learn how to organize the information in such a way that will make sense. This is an example of a conn message:
>> 
>> {
>>   "_index": "bro-201510191500",
>>   "_type": "conn",
>>   "_id": "AVCBxqIWiyISA4W_6X6I",
>>   "_score": 1,
>>   "_source": {
>>     "ts": 1445286221580,
>>     "uid": "CG7qWz2Xgs7J8LcO5d",
>>     "id.orig_h": "*.*.9.119",
>>     "id.orig_p": 123,
>>     "id.resp_h": "*.*.1.3",
>>     "id.resp_p": 123,
>>     "proto": "udp",
>>     "duration": 0.0002,
>>     "orig_bytes": 0,
>>     "resp_bytes": 48,
>>     "conn_state": "SHR",
>>     "local_orig": false,
>>     "local_resp": false,
>>     "missed_bytes": 0,
>>     "history": "Cd",
>>     "orig_pkts": 0,
>>     "orig_ip_bytes": 0,
>>     "resp_pkts": 1,
>>     "resp_ip_bytes": 76,
>>     "tunnel_parents": []
>>   }
>> }
>> 
>> 
>> 
>> 
>> Chris Williams 
>> ---
>> BS Information Systems - '16
>> CWIT Y2 Cyber Scholar
>> Work: 5-0933
>> Cell: (202) 596-5406
>> 
>> "If you think technology can solve your security problems, then you don't understand the problems and you don't understand the technology." Bruce Schneier
>> 
>> 
>> On Wed, Oct 21, 2015 at 12:28 PM, Azoff, Justin S <jazoff at illinois.edu> wrote:
>> 
>> > On Oct 21, 2015, at 12:16 PM, Chris Williams <cw13 at umbc.edu> wrote:
>> >
>> > I recently installed Bro, and I am trying to get it to work with elastic search (with Kibana as a front end.) I have alerts getting to ES and it shows up in Kibana, but it is a mix of unintelligible json messages. For example, some don't have timestamps:
>> 
>> ...
>> 
>> >   "_type": "loaded_scripts",
>> >
>> 
>> The loaded_scripts.log is 'special' and does not have timestamps.  How do entries from things like the conn.log or http.log look?
>> 
>> 
>> --
>> - Justin Azoff
>> 
>> 
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro