[Bro] Bro -> Elasticsearch -> Kibana4beta -> GeoLocation

[Daniel Guerra]

I use the elasticsearch plugin in bro. I know logstash works fine but its
very cpu intensive. Thanx anyway. 
> On 29 Oct 2015, at 23:55, Andrew Ratcliffe <andrew.ratcliffe at nswcsystems.co.uk> wrote:
> 
> I’m not sure if this is what you are looking for but just in case it’s useful. 
> 
> # BRO_connlog ######################
>   if [type] == "BRO_connlog" {
>     grok {
>       match => [ "message", "(?<ts>(.*?))\t(?<uid>(.*?))\t(?<id.orig_h>(.*?))\t(?<id.orig_p>(.*?))\t(?<id.resp_h>(.*?))\t(?<id.resp_p>(.*?))\t(?<proto>(.*?))\t(?<service>(.*?))\t(?<duration>(.*?))\t(?<orig_bytes>(.*?))\t(?<resp_bytes>(.*?))\t(?<conn_state>(.*?))\t(?<local_orig>(.*?))\t(?<missed_bytes>(.*?))\t(?<history>(.*?))\t(?<orig_pkts>(.*?))\t(?<orig_ip_bytes>(.*?))\t(?<resp_pkts>(.*?))\t(?<resp_ip_bytes>(.*?))\t(?<tunnel_parents>(.*))\t(?<orig_cc>(.*))\t(?<resp_cc>(.*))\t(?<orig_longitude>(.*))\t(?<orig_latitude>(.*))\t(?<resp_longitude>(.*))\t(?<resp_latitude>(.*))" ]
>     }
>      mutate {
>            # 'coords' will be kept, 'tmplat' is temporary.
>            # Both of these new fields are strings.
>            add_field => [ "resp_coords", "%{resp_longitude}",
>                      "tmplat", "%{resp_latitude}" ]
>                 }
>      mutate {
>            # Merge 'tmplat' into 'coords'
>            merge => [ "resp_coords", "tmplat" ]
>                 }
>      mutate {
>            # Convert our new array of strings back to float
>            convert => [ "resp_coords", "float" ]
>            # Delete our temporary latitude field
>            remove => [ "tmplat" ]
>                 }
>   }
> 
> I just used the resp but could duplicate for orig. 
> 
> Kind regards,
> Andy
> Andrew.Ratcliffe at NSWCSystems.co.uk <mailto:Andrew.Ratcliffe at nswcsystems.co.uk>
> CISSP, GCIA, GCIH, GPEN, GWAPT, CSTA, CSTP, CWSA, GCFE
> Blog.InfoSecMatters.net <http://blog.infosecmatters.net/>
> 
> 
> 
> 
> 
> 
>> On 28 Oct 2015, at 13:31, Daniel Guerra <daniel.guerra69 at gmail.com <mailto:daniel.guerra69 at gmail.com>> wrote:
>> 
>> To be more clear
>> I use this bro script for geo location
>> 
>> ##! Add geo_location for the originator and responder of a connection
>> ##! to the connection logs.
>> 
>> module Conn;
>> 
>> export
>>  {
>> 	redef record Conn::Info += 
>> 	{
>> 		orig_loc: geo_location &optional &log;
>>     		resp_loc: geo_location &optional &log;
>> 	};
>> }
>> 
>> event connection_state_remove(c: connection) 
>> {
>> 	local orig_loc = lookup_location(c$id$orig_h);
>>   	if (orig_loc?$longitude && orig_loc?$latitude)
>>     		c$conn$orig_loc= orig_loc;
>> 	local resp_loc = lookup_location(c$id$resp_h);
>>   	if (resp_loc?$longitude && resp_loc?$latitude)
>>     		c$conn$resp_loc= resp_loc;
>> }
>> 
>> Produces this output in json example
>> {
>>     "ts": "2013-04-26T12:12:02.341149Z",
>>     "uid": "C0GaiXWHKY4Uj0qke",
>>     "id.orig_h": "83.161.249.149",
>>     "id.orig_p": 49318,
>>     "id.resp_h": "68.232.35.139",
>>     "id.resp_p": 443,
>>     "proto": "tcp",
>>     "conn_state": "SHR",
>>     "missed_bytes": 0,
>>     "history": "f",
>>     "orig_pkts": 0,
>>     "orig_ip_bytes": 0,
>>     "resp_pkts": 1,
>>     "resp_ip_bytes": 67,
>>     "tunnel_parents": [],
>>     "orig_loc.country_code": "NL",
>>     "orig_loc.latitude": 52.366699,
>>     "orig_loc.longitude": 4.9,
>>     "resp_loc.country_code": "US",
>>     "resp_loc.region": "CA",
>>     "resp_loc.city": "Santa Monica",
>>     "resp_loc.latitude": 34.011902,
>>     "resp_loc.longitude": -118.468201
>>   }
>> 
>> According to the elasticsearch documentation i need an output like
>> 
>> “resp_loc”: {
>> 	“lat”: 52.366699,
>> 	“long”:4.9
>> }
>> 
>> https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-geo-point-type.html <https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-geo-point-type.html>
>> 
>> 
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org <mailto:bro at bro-ids.org>
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151030/0f93b987/attachment.html