[Bro] Bro -> Elastic Search -> Kibana Issues | Default Analyzed Fields

Arash Fallah af7 at umbc.edu
Mon Oct 26 17:47:00 PDT 2015


Exactly, which is why I suggest disabling string analyzation in Elastic Search for this field (and all IP fields). Otherwise, the IPv6 entries are ruining the results.

> On Oct 26, 2015, at 8:00 PM, Mark Parsons <mark.parsons at gmail.com> wrote:
> 
> Arash,
> I imagine that is because elasticsearch currently does not support IPv6 fields. 
> 
> See this elasticsearch github issue for more background
> 
> https://github.com/elastic/elasticsearch/issues/3714 <https://github.com/elastic/elasticsearch/issues/3714>
> 
> Thanks,
> Mark
> 
> Sent from my iPhone
> 
> On Oct 26, 2015, at 3:16 PM, Arash Fallah <af7 at umbc.edu <mailto:af7 at umbc.edu>> wrote:
> 
>> Real Quick Update:
>> 
>> My initial analysis was incorrect. IPv4 fields are processed correctly. The issue is with IPv6 fields. Same concept, different trigger. Here is a picture illustrating the problem:
>> 
>> http://i.imgur.com/pdxRbmX.png <http://i.imgur.com/pdxRbmX.png>
>> 
>> On Mon, Oct 26, 2015 at 2:50 PM, Arash Fallah <af7 at umbc.edu <mailto:af7 at umbc.edu>> wrote:
>> I am having an issue when trying to process Bro data through Elastic Search and Kibana.
>> 
>> Specifically, I am doing basic quantitative statistics such as pulling the Top 5 Originating IP's addresses by id_orig_h. However, Elastic Search and Kibana breaks apart this field as shown below in the screenshot linked below:
>> 
>> http://i.imgur.com/m3BH6LP.png <http://i.imgur.com/m3BH6LP.png>
>> 
>> Basically, for some strings, the default Elastic Search analyzer will segment them into different pieces based on the "." character. For example, 130.85.12.20 will be broken apart into 130, 85, 12, and 80. This is because in the Core Type mappings, the not index attribute is not explicitly set to no (it defaults to yes). There is no way to adjust this for existing fields.
>> 
>> Here is the current mapping created by Bro:
>> 
>> "id.resp_h" : {
>>        "type" : "string"
>> }
>> 
>> It should be:
>> 
>> "id.resp_h" : {
>>        "type" : "string"
>>         "index": "not_analyzed"
>> }
>> 
>> Suggestions?
>> 
>> 
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org <mailto:bro at bro-ids.org>
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro <http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151026/2c41aa57/attachment.html 


More information about the Bro mailing list