From andrew.ratcliffe at nswcsystems.co.uk Fri May 1 01:13:17 2015 From: andrew.ratcliffe at nswcsystems.co.uk (Andrew Ratcliffe) Date: Fri, 1 May 2015 08:13:17 +0000 Subject: [Bro] loging to elasticsearch git clone In-Reply-To: <40FFA35A-0161-4E0A-B573-6278DF689ED2@gmail.com> References: <40FFA35A-0161-4E0A-B573-6278DF689ED2@gmail.com> Message-ID: <8ED57A28-EF46-4681-9F02-8B6DE032E262@nswcsystems.co.uk> Also, you could have a look at this for an alternative way of getting Bro into Logstash. http://www.appliednsm.com/parsing-bro-logs-with-logstash/ > On 1 May 2015, at 03:41, Daniel Guerra wrote: > > I log to json files. After this I use logstash to store it in elasticsearch. > Logstash has an embeded elasicsearch + kibana > > in bro edit init-default.bro and add @load policy/tuning/json-logs > > a config i use for logstash might be handy for you > > Regards, > Daniel > > input { > file { > codec => json > path => "/input/*.log" > type => "bro_log" > } > } > > filter { > # Parse the `time` attribute as a UNIX timestamp (seconds since epoch) > # and store it in `@timestamp` attribute. This will be used in Kibana later on. > date { > match => [ "ts", "UNIX" ] > } > translate { > field => "conn_state" > destination => "conn_state_full" > dictionary => [ > "S0", "Attempt", > "S1", "Established", > "S2", "Originator close only", > "S3", "Responder close only", > "SF", "SYN/FIN completion", > "REJ", "Rejected", > "RSTO", "Originator aborted", > "RSTR", "Responder aborted", > "RSTOS0", "Originator SYN + RST", > "RSTRH", "Responder SYN ACK + RST", > "SH", "Originator SYN + FIN", > "SHR", "Responder SYN ACK + FIN", > "OTH", "Midstream traffic" > ] > } > grok { > match => { "path" => ".*\/(?[a-zA-Z0-9]+)\.log$" } > } > } > > > output { > elasticsearch { > embedded => true > } > } > >> On 30 Apr 2015, at 18:27, Mo Jia > wrote: >> >> Hi : >> >> I follow the https://www.bro.org/sphinx/frameworks/logging-elasticsearch.html >> with git clone latest source, seem it can't take effect to find it >> should build elasticsearch. So how can I build elasticsearch with >> latest source? >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150501/538a2363/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150501/538a2363/attachment-0001.bin From daniel.guerra69 at gmail.com Fri May 1 15:29:58 2015 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Sat, 2 May 2015 00:29:58 +0200 Subject: [Bro] loging to elasticsearch git clone In-Reply-To: <8ED57A28-EF46-4681-9F02-8B6DE032E262@nswcsystems.co.uk> References: <40FFA35A-0161-4E0A-B573-6278DF689ED2@gmail.com> <8ED57A28-EF46-4681-9F02-8B6DE032E262@nswcsystems.co.uk> Message-ID: <8B65066E-BBA3-412A-96B3-0D01E3E99C29@gmail.com> Logging local and then parse (the logstash way) it is not really preferred. I have been playing with docker and created a docker image for bro with elasticsearch. This works great bro uses elasticsearch to log, only kibana needs a different timestamp (ts). To check your bro can do elasticsearch do : /usr/local/bro/bin/bro -N Bro::ElasticSearch should give Bro::ElasticSearch - ElasticSearch log writer (dynamic, version 1.0) Setup elasticsearch vi /usr/local/bro/share/bro/base/frameworks/logging/main.bro and set const enable_local_logging = F to avoid local logging vi /usr/local/bro/lib/bro/plugins/Bro_ElasticSearch/scripts/init.bro and set ## Name of the ES cluster. const cluster_name = ?" &redef; ## ES server. const server_host = ?" &redef; to get clustername and ip check with your browser http://:9200/_nodes mkdir /usr/local/bro/share/bro/elasticsearch and copy from the git bro source dir aux/plugins/elasticsearch/scripts/Bro/ElasticSearch/logs-to-elasticsearch.bro to /usr/local/bro/share/bro/elasticsearch add to /usr/local/bro/share/bro/base/init-default.bro @load elasticsearch/logs-to-elasticsearch You are now ready to log to elasticsearch In kibana use bro-* to get your indices or check http://:9200/_cat/indices?v Hopefully bro can log a YYYY:mm:dd HH:MM:ss format for ts, work in progress ??. Regards, Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150502/76c4d0ae/attachment.html From seth at icir.org Fri May 1 21:00:39 2015 From: seth at icir.org (Seth Hall) Date: Sat, 2 May 2015 00:00:39 -0400 Subject: [Bro] loging to elasticsearch git clone In-Reply-To: <8B65066E-BBA3-412A-96B3-0D01E3E99C29@gmail.com> References: <40FFA35A-0161-4E0A-B573-6278DF689ED2@gmail.com> <8ED57A28-EF46-4681-9F02-8B6DE032E262@nswcsystems.co.uk> <8B65066E-BBA3-412A-96B3-0D01E3E99C29@gmail.com> Message-ID: <9524E5E4-AB52-42F9-B2A3-7117C5AE35D4@icir.org> > On May 1, 2015, at 6:29 PM, Daniel Guerra wrote: > > Hopefully bro can log a YYYY:mm:dd HH:MM:ss format for ts, work in progress ??. It can. :) If you want to make JSON logs globally into ISO8601, you can do... redef LogAscii::json_timestamps = JSON::TS_ISO8601; .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150502/80b71404/attachment.bin From life.130815 at gmail.com Sat May 2 07:02:25 2015 From: life.130815 at gmail.com (Mo Jia) Date: Sat, 2 May 2015 22:02:25 +0800 Subject: [Bro] loging to elasticsearch git clone In-Reply-To: <8B65066E-BBA3-412A-96B3-0D01E3E99C29@gmail.com> References: <40FFA35A-0161-4E0A-B573-6278DF689ED2@gmail.com> <8ED57A28-EF46-4681-9F02-8B6DE032E262@nswcsystems.co.uk> <8B65066E-BBA3-412A-96B3-0D01E3E99C29@gmail.com> Message-ID: Do what you said, ElasticSearch was installed . I think I success at first. But after I rm the dir elasticsearch-1.5.2/data/ And do it again it don't work any more . In logs only have stderr.log and stdout.log (disable local log take effect) ikfb at ikfb:/usr/local/bro/logs/current$ cat stderr.log listening on eth0, capture length 8192 bytes ikfb at ikfb:/usr/local/bro/logs/current$ cat stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited seem work fine. I think after I rm elasticsearch-1.5.2/data/ it can rebuild. I don't change bro system. Any suggestion to debug why bro can't connect elasticsearch? I add print in share/bro/elasticsearch/logs-to-elasticsearch.bro event bro_init() &priority=-5 { if ( server_host == "" ) return; print "beofore for"; for ( stream_id in Log::active_streams ) { if ( stream_id in excluded_log_ids || (|send_logs| > 0 && stream_id !in send_logs) ) next; print "after if" local filter: Log::Filter = [$name = "default-es", $writer = Log::WRITER_ELASTICSEARCH, $interv = LogElasticSearch::rotation_interval]; Log::add_filter(stream_id, filter); } } It don't show msg in broctl where I start. I think it may be in current/stdout.log and I am wrong. 2015-05-02 6:29 GMT+08:00 Daniel Guerra : > Logging local and then parse (the logstash way) it is not really preferred. > I have been playing with docker and created a docker image for bro with > elasticsearch. This works great bro uses elasticsearch to log, only kibana > needs a different timestamp (ts). > To check your bro can do elasticsearch do : > /usr/local/bro/bin/bro -N Bro::ElasticSearch > should give > Bro::ElasticSearch - ElasticSearch log writer (dynamic, version 1.0) > > Setup elasticsearch > vi /usr/local/bro/share/bro/base/frameworks/logging/main.bro > and set > const enable_local_logging = F > to avoid local logging > vi /usr/local/bro/lib/bro/plugins/Bro_ElasticSearch/scripts/init.bro > and set by the way : can we just add these line to local.bro @load elasticsearch/logs-to-elasticsearch export { redef Log::enable_local_logging = F; redef LogAscii::json_timestamps = JSON::TS_ISO8601; } > ## Name of the ES cluster. > const cluster_name = ?" &redef; > > ## ES server. > const server_host = ?" &redef; > > to get clustername and ip check with your browser > http://:9200/_nodes > > mkdir /usr/local/bro/share/bro/elasticsearch and copy from the git bro > source dir > aux/plugins/elasticsearch/scripts/Bro/ElasticSearch/logs-to-elasticsearch.bro > to > /usr/local/bro/share/bro/elasticsearch > > add to /usr/local/bro/share/bro/base/init-default.bro > > @load elasticsearch/logs-to-elasticsearch > > You are now ready to log to elasticsearch > > In kibana use bro-* to get your indices or check > http://:9200/_cat/indices?v > > Hopefully bro can log a YYYY:mm:dd HH:MM:ss format for ts, work in progress > ??. > > Regards, > > Daniel > > From daniel.guerra69 at gmail.com Sat May 2 11:24:51 2015 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Sat, 2 May 2015 20:24:51 +0200 Subject: [Bro] loging to elasticsearch git clone In-Reply-To: References: <40FFA35A-0161-4E0A-B573-6278DF689ED2@gmail.com> <8ED57A28-EF46-4681-9F02-8B6DE032E262@nswcsystems.co.uk> <8B65066E-BBA3-412A-96B3-0D01E3E99C29@gmail.com> Message-ID: I have the same it stops after 1549 records(tried twice). After this, even after a restart bro and/or the removal of all bro databases from elasticsearch. Its like it changed the config and not logging is permanent now. Very strange. I used docker to reproduce it. The diff of the before and after elasticsearch logging with bro. C /root C /root/brotest C /root/brotest/.state C /root/brotest/.state/.tmp C /root/brotest/.state/state.bst C /root/elasticsearch-1.5.2 C /root/elasticsearch-1.5.2/data C /root/elasticsearch-1.5.2/data/elasticsearch C /root/elasticsearch-1.5.2/data/elasticsearch/nodes C /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0 C /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/_state D /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/_state/global-0.st A /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/_state/global-1.st C /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/indices C /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/indices/.kibana C /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/indices/.kibana/0 C /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/indices/.kibana/0/_state D /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/indices/.kibana/0/_state/state-1.st A /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/indices/.kibana/0/_state/state-2.st C /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/indices/.kibana/0/translog C /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/indices/.kibana/0/translog/translog-1430433284419 D /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/indices/@bro-meta D /root/elasticsearch-1.5.2/data/elasticsearch/nodes/0/indices/bro-201505011800 A /root/elasticsearch-1.5.2/e.out C /root/elasticsearch-1.5.2/logs C /root/elasticsearch-1.5.2/logs/elasticsearch.log A /root/elasticsearch-1.5.2/logs/elasticsearch.log.2015-05-01 A /root/elasticsearch-1.5.2/o.out C /tmp C /tmp/hsperfdata_root D /tmp/hsperfdata_root/21864 A /tmp/hsperfdata_root/14 The elasticsearch log. [2015-05-02 17:49:52,315][INFO ][node ] [Midnight Man] version[1.5.2], pid[14], build[62ff986/2015-04-27T09:21:06Z] [2015-05-02 17:49:52,316][INFO ][node ] [Midnight Man] initializing ... [2015-05-02 17:49:52,323][INFO ][plugins ] [Midnight Man] loaded [], sites [] [2015-05-02 17:49:56,501][INFO ][node ] [Midnight Man] initialized [2015-05-02 17:49:56,501][INFO ][node ] [Midnight Man] starting ... [2015-05-02 17:49:56,641][INFO ][transport ] [Midnight Man] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/172.17.0.3:9300]} [2015-05-02 17:49:56,661][INFO ][discovery ] [Midnight Man] elasticsearch/vQdCc78tSOi-NDV-EQkItg [2015-05-02 17:50:00,445][INFO ][cluster.service ] [Midnight Man] new_master [Midnight Man][vQdCc78tSOi-NDV-EQkItg][7338ce54205e][inet[/172.17.0.3:9300]], reason: zen-disco-join (elected_as_master) [2015-05-02 17:50:00,517][INFO ][http ] [Midnight Man] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/172.17.0.3:9200]} [2015-05-02 17:50:00,518][INFO ][node ] [Midnight Man] started [2015-05-02 17:50:01,753][INFO ][gateway ] [Midnight Man] recovered [3] indices into cluster_state [2015-05-02 17:55:19,907][INFO ][cluster.metadata ] [Midnight Man] [bro-201505011800] deleting index [2015-05-02 17:55:28,711][INFO ][cluster.metadata ] [Midnight Man] [@bro-meta] deleting index [2015-05-02 18:14:39,947][INFO ][node ] [Midnight Man] stopping ... [2015-05-02 18:14:40,031][INFO ][node ] [Midnight Man] stopped [2015-05-02 18:14:40,033][INFO ][node ] [Midnight Man] closing ... [2015-05-02 18:14:40,042][INFO ][node ] [Midnight Man] closed [2015-05-02 18:15:15,298][INFO ][node ] [Magician] version[1.5.2], pid[146], build[62ff986/2015-04-27T09:21:06Z] [2015-05-02 18:15:15,299][INFO ][node ] [Magician] initializing ... [2015-05-02 18:15:15,308][INFO ][plugins ] [Magician] loaded [], sites [] [2015-05-02 18:15:19,437][INFO ][node ] [Magician] initialized [2015-05-02 18:15:19,438][INFO ][node ] [Magician] starting ... [2015-05-02 18:15:19,635][INFO ][transport ] [Magician] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/172.17.0.3:9300]} [2015-05-02 18:15:19,671][INFO ][discovery ] [Magician] elasticsearch/yPmlXzBLQpS-VK8iFOGmfg [2015-05-02 18:15:23,461][INFO ][cluster.service ] [Magician] new_master [Magician][yPmlXzBLQpS-VK8iFOGmfg][7338ce54205e][inet[/172.17.0.3:9300]], reason: zen-disco-join (elected_as_master) [2015-05-02 18:15:23,507][INFO ][http ] [Magician] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/172.17.0.3:9200]} [2015-05-02 18:15:23,507][INFO ][node ] [Magician] started [2015-05-02 18:15:23,535][INFO ][gateway ] [Magician] recovered [0] indices into cluster_state I removed the elastic data, the tmp and the .state files. Restarted everything but no more elasticsearch loging in this image. Sorry, rather had a nice story here ;) Daniel > On 02 May 2015, at 16:02, Mo Jia wrote: > > Do what you said, ElasticSearch was installed . > > I think I success at first. But after I rm the dir elasticsearch-1.5.2/data/ > And do it again it don't work any more . > > In logs only have stderr.log and stdout.log (disable local log take effect) > > ikfb at ikfb:/usr/local/bro/logs/current$ cat stderr.log > listening on eth0, capture length 8192 bytes > > ikfb at ikfb:/usr/local/bro/logs/current$ cat stdout.log > max memory size (kbytes, -m) unlimited > data seg size (kbytes, -d) unlimited > virtual memory (kbytes, -v) unlimited > core file size (blocks, -c) unlimited > > seem work fine. > I think after I rm elasticsearch-1.5.2/data/ it can rebuild. I don't > change bro system. Any suggestion to debug why bro can't connect > elasticsearch? > > I add print in share/bro/elasticsearch/logs-to-elasticsearch.bro > > event bro_init() &priority=-5 > { > if ( server_host == "" ) > return; > > print "beofore for"; > > for ( stream_id in Log::active_streams ) > { > if ( stream_id in excluded_log_ids || > (|send_logs| > 0 && stream_id !in send_logs) ) > next; > > print "after if" > > local filter: Log::Filter = [$name = "default-es", > $writer = Log::WRITER_ELASTICSEARCH, > $interv = LogElasticSearch::rotation_interval]; > Log::add_filter(stream_id, filter); > } > } > > It don't show msg in broctl where I start. I think it may be in > current/stdout.log > and I am wrong. > > 2015-05-02 6:29 GMT+08:00 Daniel Guerra >: >> Logging local and then parse (the logstash way) it is not really preferred. >> I have been playing with docker and created a docker image for bro with >> elasticsearch. This works great bro uses elasticsearch to log, only kibana >> needs a different timestamp (ts). >> To check your bro can do elasticsearch do : >> /usr/local/bro/bin/bro -N Bro::ElasticSearch >> should give >> Bro::ElasticSearch - ElasticSearch log writer (dynamic, version 1.0) >> >> Setup elasticsearch >> vi /usr/local/bro/share/bro/base/frameworks/logging/main.bro >> and set >> const enable_local_logging = F >> to avoid local logging >> vi /usr/local/bro/lib/bro/plugins/Bro_ElasticSearch/scripts/init.bro >> and set > > by the way : can we just add these line to local.bro > > @load elasticsearch/logs-to-elasticsearch > export { > redef Log::enable_local_logging = F; > redef LogAscii::json_timestamps = JSON::TS_ISO8601; > } > > >> ## Name of the ES cluster. >> const cluster_name = ?" &redef; >> >> ## ES server. >> const server_host = ?" &redef; >> >> to get clustername and ip check with your browser >> http://:9200/_nodes >> >> mkdir /usr/local/bro/share/bro/elasticsearch and copy from the git bro >> source dir >> aux/plugins/elasticsearch/scripts/Bro/ElasticSearch/logs-to-elasticsearch.bro >> to >> /usr/local/bro/share/bro/elasticsearch >> >> add to /usr/local/bro/share/bro/base/init-default.bro >> >> @load elasticsearch/logs-to-elasticsearch >> >> You are now ready to log to elasticsearch >> >> In kibana use bro-* to get your indices or check >> http://:9200/_cat/indices?v >> >> Hopefully bro can log a YYYY:mm:dd HH:MM:ss format for ts, work in progress >> ??. >> >> Regards, >> >> Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150502/a5a1276c/attachment-0001.html From daniel.guerra69 at gmail.com Sun May 3 04:53:19 2015 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Sun, 3 May 2015 13:53:19 +0200 Subject: [Bro] send logs to custom server by socket In-Reply-To: References: Message-ID: <062023BD-18D0-4374-95DA-58A0EA313EA1@gmail.com> Is this a bro only broker or does it communicate amqp ? > On 01 May 2015, at 03:38, Hosom, Stephen M wrote: > > I believe you likely want functionality that technically exists in Master. > > Check out remote logging with Broker... https://www.bro.org/sphinx-git/frameworks/broker.html#remote-logging > > I haven't played with that yet, so I can't be certain it does precisely what you want... > > Alternatively, you could just delete the logs after they rotate and send the logs via syslog with rsyslog, or your syslog daemon of choice. > > Let me know if that helps! > ________________________________________ > From: bro-bounces at bro.org [bro-bounces at bro.org] on behalf of Mo Jia [life.130815 at gmail.com] > Sent: Thursday, April 30, 2015 1:17 AM > To: bro at bro.org > Subject: [Bro] send logs to custom server by socket > > Hello: > > If I don't want log to disk, and want send json logs to a remote > server. When some code like this Log::write(HTTP::LOG, c$http); it was > send http log to my server. Dose this mean I need change > src/logging/writters/ascii ? Or I should add a new writer something > like socket? I don't want change the bro scripts already have, so > Log:write(HTTP::LOG, c$http) should don't change. Or I think is > add a config like > > LOG_SERVER_IP = 192.168.100 > LOG_SERVER_PORT = 8087 > > and all the http , notice and so on all send to the server. > Any suggest? Or does somebody already done before? > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From life.130815 at gmail.com Sun May 3 06:43:35 2015 From: life.130815 at gmail.com (Mo Jia) Date: Sun, 3 May 2015 21:43:35 +0800 Subject: [Bro] send logs to custom server by socket In-Reply-To: <062023BD-18D0-4374-95DA-58A0EA313EA1@gmail.com> References: <062023BD-18D0-4374-95DA-58A0EA313EA1@gmail.com> Message-ID: The bro log should send to a stream handing process, may be kafka or storm for preprocessing the logs .(Which I need to redefine the logs field , and add more precise fields in one proto like http). So directly log to elasticsearch may be not a good method. Also, does bro supports to understand code by switch some macro , So I can see (for example how a packet was handing, from begging to end), I don't want make debug-version and step by step in gdb to see which func was called. Something like a debug log of call orders? 2015-05-03 19:53 GMT+08:00 Daniel Guerra : > Is this a bro only broker or does it communicate amqp ? > >> On 01 May 2015, at 03:38, Hosom, Stephen M wrote: >> >> I believe you likely want functionality that technically exists in Master. >> >> Check out remote logging with Broker... https://www.bro.org/sphinx-git/frameworks/broker.html#remote-logging >> >> I haven't played with that yet, so I can't be certain it does precisely what you want... >> >> Alternatively, you could just delete the logs after they rotate and send the logs via syslog with rsyslog, or your syslog daemon of choice. >> >> Let me know if that helps! >> ________________________________________ >> From: bro-bounces at bro.org [bro-bounces at bro.org] on behalf of Mo Jia [life.130815 at gmail.com] >> Sent: Thursday, April 30, 2015 1:17 AM >> To: bro at bro.org >> Subject: [Bro] send logs to custom server by socket >> >> Hello: >> >> If I don't want log to disk, and want send json logs to a remote >> server. When some code like this Log::write(HTTP::LOG, c$http); it was >> send http log to my server. Dose this mean I need change >> src/logging/writters/ascii ? Or I should add a new writer something >> like socket? I don't want change the bro scripts already have, so >> Log:write(HTTP::LOG, c$http) should don't change. Or I think is >> add a config like >> >> LOG_SERVER_IP = 192.168.100 >> LOG_SERVER_PORT = 8087 >> >> and all the http , notice and so on all send to the server. >> Any suggest? Or does somebody already done before? >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > From life.130815 at gmail.com Sun May 3 20:20:53 2015 From: life.130815 at gmail.com (Mo Jia) Date: Mon, 4 May 2015 11:20:53 +0800 Subject: [Bro] loging to elasticsearch git clone In-Reply-To: References: <40FFA35A-0161-4E0A-B573-6278DF689ED2@gmail.com> <8ED57A28-EF46-4681-9F02-8B6DE032E262@nswcsystems.co.uk> <8B65066E-BBA3-412A-96B3-0D01E3E99C29@gmail.com> <9524E5E4-AB52-42F9-B2A3-7117C5AE35D4@icir.org> Message-ID: Hi, what's your elasticsearch version using? I am using 1.5.2 I only get indice like : health status index pri rep docs.count docs.deleted store.size pri.store.size yellow open .packetbeat-topology 5 1 0 0 3.6kb 3.6kb yellow open kibana-int 5 1 6 0 58.2kb 58.2kb yellow open .kibana 1 1 6 0 20.5kb 20.5kb yellow open bro-201505040900 5 1 33 0 98.6kb 98.6kb yellow open @bro-meta 5 1 1 0 3.4kb 3.4kb yellow open packetbeat-2015.05.04 5 1 780 0 693.2kb 693.2kb no indice for proto analsys. But it is strage after I redef Log::enable_local_logging = T; Seem that I need enable local logging so that the elasticsearch can work? 2015-05-04 10:28 GMT+08:00 Daniel Guerra : > Elasticsearch is working fine, I made some mistakes. > But still no progress on the timestamps, is there an issue > on this ? > >> On 02 May 2015, at 06:00, Seth Hall wrote: >> >> >>> On May 1, 2015, at 6:29 PM, Daniel Guerra wrote: >>> >>> Hopefully bro can log a YYYY:mm:dd HH:MM:ss format for ts, work in progress ??. >> >> It can. :) >> >> If you want to make JSON logs globally into ISO8601, you can do... >> redef LogAscii::json_timestamps = JSON::TS_ISO8601; >> >> .Seth >> >> -- >> Seth Hall >> International Computer Science Institute >> (Bro) because everyone has a network >> http://www.bro.org/ >> > From carlopmart at gmail.com Mon May 4 04:19:57 2015 From: carlopmart at gmail.com (C.L. Martinez) Date: Mon, 04 May 2015 11:19:57 +0000 Subject: [Bro] Lot of weird log entries like DNS_RR_unknown_type Message-ID: <554755DD.3040707@gmail.com> Hi all, Over last days, I am seeing a lot of weird errors like: #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird #open 2015-05-04-11-04-42 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1430737482.215330 CYHJwf46bhQTDkaZV9 172.22.55.1 22237 172.22.55.6 53 DNS_RR_unknown_type - F bro 1430737483.223168 CIpEYq3OXvMER15dG1 172.22.55.1 58971 172.22.55.6 53 DNS_RR_unknown_type - F bro 172.22.55.1 is our internal DNS server, and recursive queries works ok. Then, why this type of weird logs?? Bro 2.3.2 installed under Debian 7 host (fully updated) Thanks. From carlopmart at gmail.com Mon May 4 04:24:16 2015 From: carlopmart at gmail.com (C.L. Martinez) Date: Mon, 04 May 2015 11:24:16 +0000 Subject: [Bro] Lot of weird log entries like DNS_RR_unknown_type In-Reply-To: <554755DD.3040707@gmail.com> References: <554755DD.3040707@gmail.com> Message-ID: <554756E0.2040906@gmail.com> On 05/04/2015 11:19 AM, C.L. Martinez wrote: > Hi all, > > Over last days, I am seeing a lot of weird errors like: > > #separator \x09 > #set_separator , > #empty_field (empty) > #unset_field - > #path weird > #open 2015-05-04-11-04-42 > #fields ts uid id.orig_h id.orig_p id.resp_h > id.resp_p name addl notice peer > #types time string addr port addr port string > string bool string > 1430737482.215330 CYHJwf46bhQTDkaZV9 172.22.55.1 22237 > 172.22.55.6 53 DNS_RR_unknown_type - F bro > 1430737483.223168 CIpEYq3OXvMER15dG1 172.22.55.1 58971 > 172.22.55.6 53 DNS_RR_unknown_type - F bro > > 172.22.55.1 is our internal DNS server, and recursive queries works ok. > Then, why this type of weird logs?? > > Bro 2.3.2 installed under Debian 7 host (fully updated) > > Thanks. Yep, forget it ... Bro logs are correct. Host 172.22.55.6 use an external DNS to resolv. Sorry for the noise. From johanna at icir.org Fri May 8 13:37:14 2015 From: johanna at icir.org (Johanna Amann) Date: Fri, 8 May 2015 13:37:14 -0700 Subject: [Bro] Bro 2.4 beta available Message-ID: <20150508203714.GA30223@wifi86.sys.ICSI.Berkeley.EDU> The beta version for Bro 2.4 is now available for testing and can be downloaded at: https://bro.org/download/index.html Please note that (for the first time for a beta) binary packages are also available. For more information see http://blog.bro.org/2015/05/bro-24-beta.html as well as the NEWS/CHANGES files. Feel free to use this mailing list or the bug tracker (tracker.bro.org) to provide feedback or report problems. Johanna From daniel.guerra69 at gmail.com Fri May 8 16:22:04 2015 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Sat, 9 May 2015 01:22:04 +0200 Subject: [Bro] Bro + elasticsearch + kibana on Docker Message-ID: <7E283136-32AD-4348-8FEF-C67F410033C1@gmail.com> Hi, I have place an image on docker which runs the git from a few days ago. Its an experiment to use workers and log to elasticsearch and view it in kibana. Its in the docker repo danielguerra/bro-node https://registry.hub.docker.com/u/danielguerra/bro-node/ There is always room for improvement... Regards, Daniel (next is 2.4) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150509/f0eb9e6e/attachment.html From cantonweston at gmail.com Fri May 8 18:10:23 2015 From: cantonweston at gmail.com (Canton Weston) Date: Fri, 08 May 2015 21:10:23 -0400 Subject: [Bro] BRO_CFLAG_RECONNECT error Message-ID: <554D5E7F.8050400@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hello Bro Team. I had done a fresh pull down of Fedora 21 and wanted to upgrade bro as well. So I pulled down bro-2.3.2 and compiled. All went fine and no errors. After 'make install' I went to start bro via broctl and got the following error Traceback (most recent call last): File "/usr/local/bro/bin/broctl", line 962, in from BroControl import util File "/usr/local/bro/lib/broctl/BroControl/util.py", line 10, in import config File "/usr/local/bro/lib/broctl/BroControl/config.py", line 9, in import execute File "/usr/local/bro/lib/broctl/BroControl/execute.py", line 17, in import broccoli File "/usr/local/bro/lib/broctl/broccoli.py", line 11, in class Connection: File "/usr/local/bro/lib/broctl/broccoli.py", line 13, in Connection def __init__(self, destination, broclass="", flags=BRO_CFLAG_RECONNECT | BRO_CFLAG_ALWAYS_QUEUE, connect=True): NameError: name 'BRO_CFLAG_RECONNECT' is not defined Now I thought this was due to an upgrade issue. So I deleted everything in /usr/local/bro and reinstalled. Unfortunately, still the same error. Has anyone encountered this before? I have had bro running on Fedora 20 just fine before upgrade so I am wondering what is afoot here. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJVTV5/AAoJEI2M5athlEGVH2QQAKFlc43oLq4j6vG8fbRBgEY+ rx8uGhroG38k5A9n/lC/9uv+LuhIFr1W99ka9LSxGzAhevWjLbrGmkjwZNEbYHyC Cwv5Wv5OXktnMkr9mH5pQOBce2H9PLQDEWKmTCgUQa49iUbqMkeYLq2mJ/yU9i1X SdRQcPSTlkzhWonNpx2CDK+v+exhAmQvI2Iy+TzOgUR+G4hxV1tn88R2UtR3+nDs mU4bDIldhIl6Fp5eBw/j4juNr6hd+5i577ySU6a8+uRv5qgNevMyIYhNRgwDosoe 2W8lpFqQifQNrX1cazLSqm3dJnxMowstmchCl2DI/WMiYjgleYOZ6+pbic7c15ty JDSr14d4e0XFWsJG9lhT4z7R+PXeW3g88N4m4qEuorAuP4HgK/7CvFt+Lwa28QDq rhQYKGkMeZfJ7qBE10vDrRrYKRVveIiXGj/5PeJ+5NZ6WO33VbXucU/VBbOdsJF5 iygtJV6oQaxN+uZlEXXBESKkuYrKXBq5VYh6XbYApk46MfkkEVsogNVPoHtqy7fV QigdFlkNFsZaHfWds/kYbqxzhXw4k+YMi+ZdV7bdZQjrvImC8jvw7Hd9nXtiXEd4 m7w/WDJeXGlDFFrZJADRFbhxsPf8qqigsYpHO0xPO9fFeP3fzytk/dC/rtMLyVDn 7vrHqUr37jf0K7krB3oF =Qxhe -----END PGP SIGNATURE----- From dnthayer at illinois.edu Fri May 8 19:46:26 2015 From: dnthayer at illinois.edu (Daniel Thayer) Date: Fri, 8 May 2015 21:46:26 -0500 Subject: [Bro] BRO_CFLAG_RECONNECT error In-Reply-To: <554D5E7F.8050400@gmail.com> References: <554D5E7F.8050400@gmail.com> Message-ID: <554D7502.1010602@illinois.edu> When you compiled Bro, which version of swig was used? On 05/08/2015 08:10 PM, Canton Weston wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Hello Bro Team. > > I had done a fresh pull down of Fedora 21 and wanted to upgrade bro as > well. So I pulled down bro-2.3.2 and compiled. All went fine and no > errors. After 'make install' I went to start bro via broctl and got > the following error > > Traceback (most recent call last): > File "/usr/local/bro/bin/broctl", line 962, in > from BroControl import util > File "/usr/local/bro/lib/broctl/BroControl/util.py", line 10, in > > import config > File "/usr/local/bro/lib/broctl/BroControl/config.py", line 9, in > > import execute > File "/usr/local/bro/lib/broctl/BroControl/execute.py", line 17, in > > import broccoli > File "/usr/local/bro/lib/broctl/broccoli.py", line 11, in > class Connection: > File "/usr/local/bro/lib/broctl/broccoli.py", line 13, in Connection > def __init__(self, destination, broclass="", > flags=BRO_CFLAG_RECONNECT | BRO_CFLAG_ALWAYS_QUEUE, connect=True): > NameError: name 'BRO_CFLAG_RECONNECT' is not defined > > > Now I thought this was due to an upgrade issue. So I deleted > everything in /usr/local/bro and reinstalled. Unfortunately, still > the same error. Has anyone encountered this before? I have had bro > running on Fedora 20 just fine before upgrade so I am wondering what > is afoot here. > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iQIcBAEBCAAGBQJVTV5/AAoJEI2M5athlEGVH2QQAKFlc43oLq4j6vG8fbRBgEY+ > rx8uGhroG38k5A9n/lC/9uv+LuhIFr1W99ka9LSxGzAhevWjLbrGmkjwZNEbYHyC > Cwv5Wv5OXktnMkr9mH5pQOBce2H9PLQDEWKmTCgUQa49iUbqMkeYLq2mJ/yU9i1X > SdRQcPSTlkzhWonNpx2CDK+v+exhAmQvI2Iy+TzOgUR+G4hxV1tn88R2UtR3+nDs > mU4bDIldhIl6Fp5eBw/j4juNr6hd+5i577ySU6a8+uRv5qgNevMyIYhNRgwDosoe > 2W8lpFqQifQNrX1cazLSqm3dJnxMowstmchCl2DI/WMiYjgleYOZ6+pbic7c15ty > JDSr14d4e0XFWsJG9lhT4z7R+PXeW3g88N4m4qEuorAuP4HgK/7CvFt+Lwa28QDq > rhQYKGkMeZfJ7qBE10vDrRrYKRVveIiXGj/5PeJ+5NZ6WO33VbXucU/VBbOdsJF5 > iygtJV6oQaxN+uZlEXXBESKkuYrKXBq5VYh6XbYApk46MfkkEVsogNVPoHtqy7fV > QigdFlkNFsZaHfWds/kYbqxzhXw4k+YMi+ZdV7bdZQjrvImC8jvw7Hd9nXtiXEd4 > m7w/WDJeXGlDFFrZJADRFbhxsPf8qqigsYpHO0xPO9fFeP3fzytk/dC/rtMLyVDn > 7vrHqUr37jf0K7krB3oF > =Qxhe > -----END PGP SIGNATURE----- > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > From dnthayer at illinois.edu Fri May 8 21:16:02 2015 From: dnthayer at illinois.edu (Daniel Thayer) Date: Fri, 8 May 2015 23:16:02 -0500 Subject: [Bro] BRO_CFLAG_RECONNECT error In-Reply-To: <0C3144B8-04D9-4650-8D3A-EA65A05B0CFC@gmail.com> References: <554D5E7F.8050400@gmail.com> <554D7502.1010602@illinois.edu> <0C3144B8-04D9-4650-8D3A-EA65A05B0CFC@gmail.com> Message-ID: <554D8A02.1010409@illinois.edu> I believe this is an issue with broccoli and newer versions of swig. This should be fixed in a newer version of Bro (the 2.4 beta was released today). Alternatively, if you prefer to use Bro 2.3.2, then you could copy the files CMakeLists.txt and broccoli.py from the git repo https://github.com/bro/broccoli-python and put them in your Bro 2.3.2 source tree (the relative path is aux/broccoli/bindings/broccoli-python/), then rebuild Bro. On 05/08/2015 09:52 PM, Canton Weston wrote: > 3.0.5 > > Sent from my iPad > >> On May 8, 2015, at 10:46 PM, Daniel Thayer wrote: >> >> When you compiled Bro, which version of swig was used? >> >> >> >>> On 05/08/2015 08:10 PM, Canton Weston wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA256 >>> >>> Hello Bro Team. >>> >>> I had done a fresh pull down of Fedora 21 and wanted to upgrade bro as >>> well. So I pulled down bro-2.3.2 and compiled. All went fine and no >>> errors. After 'make install' I went to start bro via broctl and got >>> the following error >>> >>> Traceback (most recent call last): >>> File "/usr/local/bro/bin/broctl", line 962, in >>> from BroControl import util >>> File "/usr/local/bro/lib/broctl/BroControl/util.py", line 10, in >>> >>> import config >>> File "/usr/local/bro/lib/broctl/BroControl/config.py", line 9, in >>> >>> import execute >>> File "/usr/local/bro/lib/broctl/BroControl/execute.py", line 17, in >>> >>> import broccoli >>> File "/usr/local/bro/lib/broctl/broccoli.py", line 11, in >>> class Connection: >>> File "/usr/local/bro/lib/broctl/broccoli.py", line 13, in Connection >>> def __init__(self, destination, broclass="", >>> flags=BRO_CFLAG_RECONNECT | BRO_CFLAG_ALWAYS_QUEUE, connect=True): >>> NameError: name 'BRO_CFLAG_RECONNECT' is not defined >>> >>> >>> Now I thought this was due to an upgrade issue. So I deleted >>> everything in /usr/local/bro and reinstalled. Unfortunately, still >>> the same error. Has anyone encountered this before? I have had bro >>> running on Fedora 20 just fine before upgrade so I am wondering what >>> is afoot here. >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v2 >>> >>> iQIcBAEBCAAGBQJVTV5/AAoJEI2M5athlEGVH2QQAKFlc43oLq4j6vG8fbRBgEY+ >>> rx8uGhroG38k5A9n/lC/9uv+LuhIFr1W99ka9LSxGzAhevWjLbrGmkjwZNEbYHyC >>> Cwv5Wv5OXktnMkr9mH5pQOBce2H9PLQDEWKmTCgUQa49iUbqMkeYLq2mJ/yU9i1X >>> SdRQcPSTlkzhWonNpx2CDK+v+exhAmQvI2Iy+TzOgUR+G4hxV1tn88R2UtR3+nDs >>> mU4bDIldhIl6Fp5eBw/j4juNr6hd+5i577ySU6a8+uRv5qgNevMyIYhNRgwDosoe >>> 2W8lpFqQifQNrX1cazLSqm3dJnxMowstmchCl2DI/WMiYjgleYOZ6+pbic7c15ty >>> JDSr14d4e0XFWsJG9lhT4z7R+PXeW3g88N4m4qEuorAuP4HgK/7CvFt+Lwa28QDq >>> rhQYKGkMeZfJ7qBE10vDrRrYKRVveIiXGj/5PeJ+5NZ6WO33VbXucU/VBbOdsJF5 >>> iygtJV6oQaxN+uZlEXXBESKkuYrKXBq5VYh6XbYApk46MfkkEVsogNVPoHtqy7fV >>> QigdFlkNFsZaHfWds/kYbqxzhXw4k+YMi+ZdV7bdZQjrvImC8jvw7Hd9nXtiXEd4 >>> m7w/WDJeXGlDFFrZJADRFbhxsPf8qqigsYpHO0xPO9fFeP3fzytk/dC/rtMLyVDn >>> 7vrHqUr37jf0K7krB3oF >>> =Qxhe >>> -----END PGP SIGNATURE----- >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> From abhall1 at yahoo.com Sun May 10 07:09:12 2015 From: abhall1 at yahoo.com (abhall1) Date: Sun, 10 May 2015 10:09:12 -0400 Subject: [Bro] BRO_CFLAG_RECONNECT error Message-ID: <7x37anh0w8r0iijvfgl2kwh0.1431266952314@email.android.com> With everything being new, what version of Python is Bro running as? ?I usually change the python declaration in bro/bin/bro,/bro/bin/broctl, and /bro/share/broctl/scripts/stats-2-csv to python2 because they begin with python and the python wrapper will try send it to the system wide python level which may be a python3 version and you will have problems. Hope that helps! Sent via the Samsung Galaxy Note? 4, an AT&T 4G LTE smartphone -------- Original message -------- From: bro-request at bro.org Date: 05/09/2015 3:00 PM (GMT-05:00) To: bro at bro.org Subject: Bro Digest, Vol 109, Issue 5 Send Bro mailing list submissions to bro at bro.org To subscribe or unsubscribe via the World Wide Web, visit http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro or, via email, send a message with subject or body 'help' to bro-request at bro.org You can reach the person managing the list at bro-owner at bro.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Bro digest..." Today's Topics: ?? 1. Bro 2.4 beta available (Johanna Amann) ?? 2.? Bro + elasticsearch + kibana on Docker (Daniel Guerra) ?? 3. BRO_CFLAG_RECONNECT error (Canton Weston) ?? 4. Re: BRO_CFLAG_RECONNECT error (Daniel Thayer) ?? 5. Re: BRO_CFLAG_RECONNECT error (Daniel Thayer) ---------------------------------------------------------------------- Message: 1 Date: Fri, 8 May 2015 13:37:14 -0700 From: Johanna Amann Subject: [Bro] Bro 2.4 beta available To: bro at bro.org Message-ID: <20150508203714.GA30223 at wifi86.sys.ICSI.Berkeley.EDU> Content-Type: text/plain; charset=us-ascii The beta version for Bro 2.4 is now available for testing and can be downloaded at: ???????? https://bro.org/download/index.html Please note that (for the first time for a beta) binary packages are also available. For more information see http://blog.bro.org/2015/05/bro-24-beta.html as well as the NEWS/CHANGES files. Feel free to use this mailing list or the bug tracker (tracker.bro.org) to provide feedback or report problems. Johanna ------------------------------ Message: 2 Date: Sat, 9 May 2015 01:22:04 +0200 From: Daniel Guerra Subject: [Bro]? Bro + elasticsearch + kibana on Docker To: bro at bro.org Message-ID: <7E283136-32AD-4348-8FEF-C67F410033C1 at gmail.com> Content-Type: text/plain; charset="us-ascii" Hi, I have place an image on docker which runs the git from a few days ago. Its an experiment to use workers and log to elasticsearch and view it in kibana. Its in the docker repo danielguerra/bro-node https://registry.hub.docker.com/u/danielguerra/bro-node/ There is always room for improvement... Regards, Daniel (next is 2.4) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150509/f0eb9e6e/attachment-0001.html ------------------------------ Message: 3 Date: Fri, 08 May 2015 21:10:23 -0400 From: Canton Weston Subject: [Bro] BRO_CFLAG_RECONNECT error To: bro at bro.org Message-ID: <554D5E7F.8050400 at gmail.com> Content-Type: text/plain; charset=utf-8 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hello Bro Team. I had done a fresh pull down of Fedora 21 and wanted to upgrade bro as well. So I pulled down bro-2.3.2 and compiled.? All went fine and no errors.? After 'make install' I went to start bro via broctl and got the following error Traceback (most recent call last): ? File "/usr/local/bro/bin/broctl", line 962, in ??? from BroControl import util ? File "/usr/local/bro/lib/broctl/BroControl/util.py", line 10, in ??? import config ? File "/usr/local/bro/lib/broctl/BroControl/config.py", line 9, in ??? import execute ? File "/usr/local/bro/lib/broctl/BroControl/execute.py", line 17, in ??? import broccoli ? File "/usr/local/bro/lib/broctl/broccoli.py", line 11, in ??? class Connection: ? File "/usr/local/bro/lib/broctl/broccoli.py", line 13, in Connection ??? def __init__(self, destination, broclass="", flags=BRO_CFLAG_RECONNECT | BRO_CFLAG_ALWAYS_QUEUE, connect=True): NameError: name 'BRO_CFLAG_RECONNECT' is not defined Now I thought this was due to an upgrade issue.? So I deleted everything in /usr/local/bro and reinstalled.? Unfortunately, still the same error.? Has anyone encountered this before?? I have had bro running on Fedora 20 just fine before upgrade so I am wondering what is afoot here. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJVTV5/AAoJEI2M5athlEGVH2QQAKFlc43oLq4j6vG8fbRBgEY+ rx8uGhroG38k5A9n/lC/9uv+LuhIFr1W99ka9LSxGzAhevWjLbrGmkjwZNEbYHyC Cwv5Wv5OXktnMkr9mH5pQOBce2H9PLQDEWKmTCgUQa49iUbqMkeYLq2mJ/yU9i1X SdRQcPSTlkzhWonNpx2CDK+v+exhAmQvI2Iy+TzOgUR+G4hxV1tn88R2UtR3+nDs mU4bDIldhIl6Fp5eBw/j4juNr6hd+5i577ySU6a8+uRv5qgNevMyIYhNRgwDosoe 2W8lpFqQifQNrX1cazLSqm3dJnxMowstmchCl2DI/WMiYjgleYOZ6+pbic7c15ty JDSr14d4e0XFWsJG9lhT4z7R+PXeW3g88N4m4qEuorAuP4HgK/7CvFt+Lwa28QDq rhQYKGkMeZfJ7qBE10vDrRrYKRVveIiXGj/5PeJ+5NZ6WO33VbXucU/VBbOdsJF5 iygtJV6oQaxN+uZlEXXBESKkuYrKXBq5VYh6XbYApk46MfkkEVsogNVPoHtqy7fV QigdFlkNFsZaHfWds/kYbqxzhXw4k+YMi+ZdV7bdZQjrvImC8jvw7Hd9nXtiXEd4 m7w/WDJeXGlDFFrZJADRFbhxsPf8qqigsYpHO0xPO9fFeP3fzytk/dC/rtMLyVDn 7vrHqUr37jf0K7krB3oF =Qxhe -----END PGP SIGNATURE----- ------------------------------ Message: 4 Date: Fri, 8 May 2015 21:46:26 -0500 From: Daniel Thayer Subject: Re: [Bro] BRO_CFLAG_RECONNECT error To: Canton Weston , Message-ID: <554D7502.1010602 at illinois.edu> Content-Type: text/plain; charset="windows-1252"; format=flowed When you compiled Bro, which version of swig was used? On 05/08/2015 08:10 PM, Canton Weston wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Hello Bro Team. > > I had done a fresh pull down of Fedora 21 and wanted to upgrade bro as > well. So I pulled down bro-2.3.2 and compiled.? All went fine and no > errors.? After 'make install' I went to start bro via broctl and got > the following error > > Traceback (most recent call last): >??? File "/usr/local/bro/bin/broctl", line 962, in >????? from BroControl import util >??? File "/usr/local/bro/lib/broctl/BroControl/util.py", line 10, in > >????? import config >??? File "/usr/local/bro/lib/broctl/BroControl/config.py", line 9, in > >????? import execute >??? File "/usr/local/bro/lib/broctl/BroControl/execute.py", line 17, in > >????? import broccoli >??? File "/usr/local/bro/lib/broctl/broccoli.py", line 11, in >????? class Connection: >??? File "/usr/local/bro/lib/broctl/broccoli.py", line 13, in Connection >????? def __init__(self, destination, broclass="", > flags=BRO_CFLAG_RECONNECT | BRO_CFLAG_ALWAYS_QUEUE, connect=True): > NameError: name 'BRO_CFLAG_RECONNECT' is not defined > > > Now I thought this was due to an upgrade issue.? So I deleted > everything in /usr/local/bro and reinstalled.? Unfortunately, still > the same error.? Has anyone encountered this before?? I have had bro > running on Fedora 20 just fine before upgrade so I am wondering what > is afoot here. > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iQIcBAEBCAAGBQJVTV5/AAoJEI2M5athlEGVH2QQAKFlc43oLq4j6vG8fbRBgEY+ > rx8uGhroG38k5A9n/lC/9uv+LuhIFr1W99ka9LSxGzAhevWjLbrGmkjwZNEbYHyC > Cwv5Wv5OXktnMkr9mH5pQOBce2H9PLQDEWKmTCgUQa49iUbqMkeYLq2mJ/yU9i1X > SdRQcPSTlkzhWonNpx2CDK+v+exhAmQvI2Iy+TzOgUR+G4hxV1tn88R2UtR3+nDs > mU4bDIldhIl6Fp5eBw/j4juNr6hd+5i577ySU6a8+uRv5qgNevMyIYhNRgwDosoe > 2W8lpFqQifQNrX1cazLSqm3dJnxMowstmchCl2DI/WMiYjgleYOZ6+pbic7c15ty > JDSr14d4e0XFWsJG9lhT4z7R+PXeW3g88N4m4qEuorAuP4HgK/7CvFt+Lwa28QDq > rhQYKGkMeZfJ7qBE10vDrRrYKRVveIiXGj/5PeJ+5NZ6WO33VbXucU/VBbOdsJF5 > iygtJV6oQaxN+uZlEXXBESKkuYrKXBq5VYh6XbYApk46MfkkEVsogNVPoHtqy7fV > QigdFlkNFsZaHfWds/kYbqxzhXw4k+YMi+ZdV7bdZQjrvImC8jvw7Hd9nXtiXEd4 > m7w/WDJeXGlDFFrZJADRFbhxsPf8qqigsYpHO0xPO9fFeP3fzytk/dC/rtMLyVDn > 7vrHqUr37jf0K7krB3oF > =Qxhe > -----END PGP SIGNATURE----- > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > ------------------------------ Message: 5 Date: Fri, 8 May 2015 23:16:02 -0500 From: Daniel Thayer Subject: Re: [Bro] BRO_CFLAG_RECONNECT error To: Canton Weston , Message-ID: <554D8A02.1010409 at illinois.edu> Content-Type: text/plain; charset="windows-1252"; format=flowed I believe this is an issue with broccoli and newer versions of swig. This should be fixed in a newer version of Bro (the 2.4 beta was released today).? Alternatively, if you prefer to use Bro 2.3.2, then you could copy the files CMakeLists.txt and broccoli.py from the git repo https://github.com/bro/broccoli-python and put them in your Bro 2.3.2 source tree (the relative path is aux/broccoli/bindings/broccoli-python/), then rebuild Bro. On 05/08/2015 09:52 PM, Canton Weston wrote: > 3.0.5 > > Sent from my iPad > >> On May 8, 2015, at 10:46 PM, Daniel Thayer wrote: >> >> When you compiled Bro, which version of swig was used? >> >> >> >>> On 05/08/2015 08:10 PM, Canton Weston wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA256 >>> >>> Hello Bro Team. >>> >>> I had done a fresh pull down of Fedora 21 and wanted to upgrade bro as >>> well. So I pulled down bro-2.3.2 and compiled.? All went fine and no >>> errors.? After 'make install' I went to start bro via broctl and got >>> the following error >>> >>> Traceback (most recent call last): >>>??? File "/usr/local/bro/bin/broctl", line 962, in >>>????? from BroControl import util >>>??? File "/usr/local/bro/lib/broctl/BroControl/util.py", line 10, in >>> >>>????? import config >>>??? File "/usr/local/bro/lib/broctl/BroControl/config.py", line 9, in >>> >>>????? import execute >>>??? File "/usr/local/bro/lib/broctl/BroControl/execute.py", line 17, in >>> >>>????? import broccoli >>>??? File "/usr/local/bro/lib/broctl/broccoli.py", line 11, in >>>????? class Connection: >>>??? File "/usr/local/bro/lib/broctl/broccoli.py", line 13, in Connection >>>????? def __init__(self, destination, broclass="", >>> flags=BRO_CFLAG_RECONNECT | BRO_CFLAG_ALWAYS_QUEUE, connect=True): >>> NameError: name 'BRO_CFLAG_RECONNECT' is not defined >>> >>> >>> Now I thought this was due to an upgrade issue.? So I deleted >>> everything in /usr/local/bro and reinstalled.? Unfortunately, still >>> the same error.? Has anyone encountered this before?? I have had bro >>> running on Fedora 20 just fine before upgrade so I am wondering what >>> is afoot here. >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v2 >>> >>> iQIcBAEBCAAGBQJVTV5/AAoJEI2M5athlEGVH2QQAKFlc43oLq4j6vG8fbRBgEY+ >>> rx8uGhroG38k5A9n/lC/9uv+LuhIFr1W99ka9LSxGzAhevWjLbrGmkjwZNEbYHyC >>> Cwv5Wv5OXktnMkr9mH5pQOBce2H9PLQDEWKmTCgUQa49iUbqMkeYLq2mJ/yU9i1X >>> SdRQcPSTlkzhWonNpx2CDK+v+exhAmQvI2Iy+TzOgUR+G4hxV1tn88R2UtR3+nDs >>> mU4bDIldhIl6Fp5eBw/j4juNr6hd+5i577ySU6a8+uRv5qgNevMyIYhNRgwDosoe >>> 2W8lpFqQifQNrX1cazLSqm3dJnxMowstmchCl2DI/WMiYjgleYOZ6+pbic7c15ty >>> JDSr14d4e0XFWsJG9lhT4z7R+PXeW3g88N4m4qEuorAuP4HgK/7CvFt+Lwa28QDq >>> rhQYKGkMeZfJ7qBE10vDrRrYKRVveIiXGj/5PeJ+5NZ6WO33VbXucU/VBbOdsJF5 >>> iygtJV6oQaxN+uZlEXXBESKkuYrKXBq5VYh6XbYApk46MfkkEVsogNVPoHtqy7fV >>> QigdFlkNFsZaHfWds/kYbqxzhXw4k+YMi+ZdV7bdZQjrvImC8jvw7Hd9nXtiXEd4 >>> m7w/WDJeXGlDFFrZJADRFbhxsPf8qqigsYpHO0xPO9fFeP3fzytk/dC/rtMLyVDn >>> 7vrHqUr37jf0K7krB3oF >>> =Qxhe >>> -----END PGP SIGNATURE----- >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> ------------------------------ _______________________________________________ Bro mailing list Bro at bro.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro End of Bro Digest, Vol 109, Issue 5 *********************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150510/a591b50e/attachment-0001.html From vitologrillo at gmail.com Mon May 11 01:07:08 2015 From: vitologrillo at gmail.com (Vito Logrillo) Date: Mon, 11 May 2015 10:07:08 +0200 Subject: [Bro] Question about Broker Message-ID: Hello Bro Team, i've tried to use Broker without any result. Below what i've done: 1. i've compiled bro source with ./configure --enable-broker option 2. I've modified BrokerComm::connect() in printing-connector.bro: i've changed "127.0.0.1" to my listener ip 3. ./bro -b printing-connector.bro 4. I've configured the listener with netcat -l 9999 5. netstat -ant on my bro machine The last command shows an established connection between the bro machine and the listener, but there's no data exchange. What's wrong? Vito From silusilusilu at gmail.com Mon May 11 03:24:16 2015 From: silusilusilu at gmail.com (fasf safas) Date: Mon, 11 May 2015 12:24:16 +0200 Subject: [Bro] On conn.log Message-ID: Hi, sometimes the field "service duration" in conn.log is void: why? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150511/d0928d9a/attachment.html From jsiwek at illinois.edu Mon May 11 08:41:24 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Mon, 11 May 2015 15:41:24 +0000 Subject: [Bro] Question about Broker In-Reply-To: References: Message-ID: <6B199FFE-92CE-49CD-8710-3D217EE31D5D@illinois.edu> > On May 11, 2015, at 3:07 AM, Vito Logrillo wrote: > > 4. I've configured the listener with netcat -l 9999 > 5. netstat -ant on my bro machine > > The last command shows an established connection between the bro > machine and the listener, but there's no data exchange. That seems ok to me: the TCP connection gets established, but a plain netcat listener doesn?t have any concept of broker data/messages/connection-handshake to be able to communicate in a meaningful way. Are you just doing this to troubleshoot some other connection problems? What are you expecting or trying to do? - Jon From vitologrillo at gmail.com Mon May 11 08:57:48 2015 From: vitologrillo at gmail.com (Vito Logrillo) Date: Mon, 11 May 2015 17:57:48 +0200 Subject: [Bro] Question about Broker In-Reply-To: <6B199FFE-92CE-49CD-8710-3D217EE31D5D@illinois.edu> References: <6B199FFE-92CE-49CD-8710-3D217EE31D5D@illinois.edu> Message-ID: Hi Jon, thanks for your reply. My final goal is the bro's log transmission from a node to a server: i think that Broker can be useful to my application, so i've tried to use the example code to understand better how it works. If i want to emulate a broker listener, which application can be used? Thanks, Vito 2015-05-11 17:41 GMT+02:00 Siwek, Jon : > >> On May 11, 2015, at 3:07 AM, Vito Logrillo wrote: >> >> 4. I've configured the listener with netcat -l 9999 >> 5. netstat -ant on my bro machine >> >> The last command shows an established connection between the bro >> machine and the listener, but there's no data exchange. > > That seems ok to me: the TCP connection gets established, but a plain netcat listener doesn?t have any concept of broker data/messages/connection-handshake to be able to communicate in a meaningful way. Are you just doing this to troubleshoot some other connection problems? What are you expecting or trying to do? > > - Jon From rjenkins at rmjconsulting.net Mon May 11 09:15:27 2015 From: rjenkins at rmjconsulting.net (Ron M. Jenkins) Date: Mon, 11 May 2015 16:15:27 +0000 Subject: [Bro] P2P Traffic Message-ID: Good morning; Can Bro detected P2P traffic, specially Bitorrent? Thanks! Ron Jenkins (Owner / Senior Architect) RMJ Consulting, LLC. "Bringing Companies and Solutions Together" 11715 Bricksome Ave STE B-7 Baton Rouge, LA 70816 Toll: 855-448-5214 Direct. 225-448-5214 Ext #101 Fax. 225-448-5324 Cell. 225-931-1632 Email. rjenkins at rmjconsulting.net Web. http://www.rmjconsulting.net Log Siphon. http://www.logsiphon.com Linkedin. www.linkedin.com/in/ronmjenkins/ Twitter: www.twitter.com/RMJConsulting Facebook: www.facebook.com/rmjcsconsulting RMJ Consulting's Technology Corner. https://www.rmjconsulting.net/main/paper.php -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150511/bbbb5d08/attachment.html From rjenkins at rmjconsulting.net Mon May 11 09:15:46 2015 From: rjenkins at rmjconsulting.net (Ron M. Jenkins) Date: Mon, 11 May 2015 16:15:46 +0000 Subject: [Bro] P2P Traffic Message-ID: Good morning; Can Bro detected P2P traffic, specially Bitorrent? Thanks! Ron Jenkins (Owner / Senior Architect) RMJ Consulting, LLC. "Bringing Companies and Solutions Together" 11715 Bricksome Ave STE B-7 Baton Rouge, LA 70816 Toll: 855-448-5214 Direct. 225-448-5214 Ext #101 Fax. 225-448-5324 Cell. 225-931-1632 Email. rjenkins at rmjconsulting.net Web. http://www.rmjconsulting.net Log Siphon. http://www.logsiphon.com Linkedin. www.linkedin.com/in/ronmjenkins/ Twitter: www.twitter.com/RMJConsulting Facebook: www.facebook.com/rmjcsconsulting RMJ Consulting's Technology Corner. https://www.rmjconsulting.net/main/paper.php -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150511/bbbb5d08/attachment-0003.html From jsiwek at illinois.edu Mon May 11 09:25:41 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Mon, 11 May 2015 16:25:41 +0000 Subject: [Bro] Question about Broker In-Reply-To: References: <6B199FFE-92CE-49CD-8710-3D217EE31D5D@illinois.edu> Message-ID: > On May 11, 2015, at 10:57 AM, Vito Logrillo wrote: > > If i want to emulate a broker listener, which application can be used? Any application that integrates/uses the Broker library. So you can have a Bro process as the listener side since it uses Broker. You can also write your own C or C++ program using Broker?s API and start that up as the listener side. - Jon From doris at bro.org Mon May 11 09:30:40 2015 From: doris at bro.org (Doris Schioberg) Date: Mon, 11 May 2015 09:30:40 -0700 Subject: [Bro] P2P Traffic In-Reply-To: References: Message-ID: <5550D930.9030308@bro.org> Hi Ron, it that what you are looking for: https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent Doris On 5/11/15 9:15 AM, Ron M. Jenkins wrote: > Good morning; > > Can Bro detected P2P traffic, specially Bitorrent? > > > Thanks! > > > > Ron Jenkins (Owner / Senior Architect) > RMJ Consulting, LLC. "Bringing Companies and Solutions Together" > 11715 Bricksome Ave STE B-7 > Baton Rouge, LA 70816 > Toll: 855-448-5214 > Direct. 225-448-5214 Ext #101 > Fax. 225-448-5324 > Cell. 225-931-1632 > Email. rjenkins at rmjconsulting.net > Web. http://www.rmjconsulting.net > Log Siphon. http://www.logsiphon.com > Linkedin. www.linkedin.com/in/ronmjenkins/ > Twitter: www.twitter.com/RMJConsulting > Facebook: www.facebook.com/rmjcsconsulting > RMJ Consulting's Technology Corner. https://www.rmjconsulting.net/main/paper.php > > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Doris Schioberg Bro Outreach, Training, and Education Coordinator International Computer Science Institute (ICSI Berkeley) Phone: +1 (510) 289-8406 * doris at bro.org From pkelley at hyperionavenue.com Mon May 11 09:31:23 2015 From: pkelley at hyperionavenue.com (Patrick Kelley) Date: Mon, 11 May 2015 09:31:23 -0700 Subject: [Bro] P2P Traffic In-Reply-To: References: Message-ID: I wrote an intel feed some time ago that fires a notice when primary trackers are used. There may be a more technical way of performing it, but it worked great for me. On Mon, May 11, 2015 at 9:15 AM, Ron M. Jenkins wrote: > Good morning; > > > > Can Bro detected P2P traffic, specially Bitorrent? > > > Thanks! > > > > > > > > Ron Jenkins (Owner / Senior Architect) > > *RMJ Consulting, LLC. *"*Bringing Companies and Solutions Together*" > > 11715 Bricksome Ave STE B-7 > > Baton Rouge, LA 70816 > > *Toll: *855-448-5214 > > *Direct*. 225-448-5214 Ext #101 > > *Fax.* 225-448-5324 > > *Cell.* 225-931-1632 > > *Email.* rjenkins at rmjconsulting.net > > *Web.* http://www.rmjconsulting.net > > *Log Siphon*. http://www.logsiphon.com > > *Linkedin.* www.linkedin.com/in/ronmjenkins/ > > *Twitter**:* www.twitter.com/RMJConsulting > > *Facebook: *www.facebook.com/rmjcsconsulting > > *RMJ Consulting?s Technology Corner.* > https://www.rmjconsulting.net/main/paper.php > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Patrick Kelley, CEH Hyperion Avenue Labs http://www.hyperionavenue.com 951.291.8310 *The limit to which you have accepted being comfortable is the limit to which you have grown. Accept new challenges as an opportunity to enrich yourself and not as a point of potential failure.* [image: hal_logo] -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150511/2740d28d/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 12155 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150511/2740d28d/attachment-0001.bin From vitologrillo at gmail.com Mon May 11 10:01:53 2015 From: vitologrillo at gmail.com (Vito Logrillo) Date: Mon, 11 May 2015 19:01:53 +0200 Subject: [Bro] Question about Broker In-Reply-To: References: <6B199FFE-92CE-49CD-8710-3D217EE31D5D@illinois.edu> Message-ID: Could you suggest me an application to test the protocol? I know, i can write my own app: however in this stage i'm much more interested to test the protocol and see if fits my needs. An "already done" application could be very helpful. Thanks, Vito Il 11/mag/2015 18:26, "Siwek, Jon" ha scritto: > > > On May 11, 2015, at 10:57 AM, Vito Logrillo > wrote: > > > > If i want to emulate a broker listener, which application can be used? > > Any application that integrates/uses the Broker library. So you can have > a Bro process as the listener side since it uses Broker. You can also > write your own C or C++ program using Broker?s API and start that up as the > listener side. > > - Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150511/e0444abd/attachment.html From jsiwek at illinois.edu Mon May 11 11:00:25 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Mon, 11 May 2015 18:00:25 +0000 Subject: [Bro] Question about Broker In-Reply-To: References: <6B199FFE-92CE-49CD-8710-3D217EE31D5D@illinois.edu> Message-ID: > On May 11, 2015, at 12:01 PM, Vito Logrillo wrote: > > Could you suggest me an application to test the protocol? I know, i can write my own app: however in this stage i'm much more interested to test the protocol and see if fits my needs. > An "already done" application could be very helpful. Currently, I think the only thing using Broker is probably Bro since it?s so new and no one has really had a chance yet to make any other applications with it. My suggestion is still to either write your own program or use a Bro process to talk to another Bro process. - Jon From mlaterma at ucalgary.ca Mon May 11 13:33:37 2015 From: mlaterma at ucalgary.ca (Michel Laterman) Date: Mon, 11 May 2015 20:33:37 +0000 Subject: [Bro] P2P Traffic In-Reply-To: <5550D930.9030308@bro.org> References: , <5550D930.9030308@bro.org> Message-ID: <1431376416713.8759@ucalgary.ca> I'm not sure that the BitTorrent analyzer has been updated for Bro v2.X. I have written a small signature based script to get an estimate of traffic volumes. Michel ________________________________________ From: bro-bounces at bro.org on behalf of Doris Schioberg Sent: May 11, 2015 10:30 AM To: bro at bro.org Subject: Re: [Bro] P2P Traffic Hi Ron, it that what you are looking for: https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent Doris On 5/11/15 9:15 AM, Ron M. Jenkins wrote: > Good morning; > > Can Bro detected P2P traffic, specially Bitorrent? > > > Thanks! > > > > Ron Jenkins (Owner / Senior Architect) > RMJ Consulting, LLC. "Bringing Companies and Solutions Together" > 11715 Bricksome Ave STE B-7 > Baton Rouge, LA 70816 > Toll: 855-448-5214 > Direct. 225-448-5214 Ext #101 > Fax. 225-448-5324 > Cell. 225-931-1632 > Email. rjenkins at rmjconsulting.net > Web. http://www.rmjconsulting.net > Log Siphon. http://www.logsiphon.com > Linkedin. www.linkedin.com/in/ronmjenkins/ > Twitter: www.twitter.com/RMJConsulting > Facebook: www.facebook.com/rmjcsconsulting > RMJ Consulting's Technology Corner. https://www.rmjconsulting.net/main/paper.php > > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Doris Schioberg Bro Outreach, Training, and Education Coordinator International Computer Science Institute (ICSI Berkeley) Phone: +1 (510) 289-8406 * doris at bro.org _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From richardqa at gmail.com Mon May 11 19:02:46 2015 From: richardqa at gmail.com (Javier Richard Quinto Ancieta) Date: Mon, 11 May 2015 23:02:46 -0300 Subject: [Bro] Run scripts of Python in Bro scripting Message-ID: Hi everyone, I wonder if there is a way of running a script in python at the precise moment that Bro launches some event such as Intel.log file is created?. For example, if a malicious activity is launched to the network, o intel framework detects and create a Intel.log, then my script in python (script.py) read this intel.log file and execute a countermeasure blocking such malicious activity, I would like to know if there is any way to run this Python script within the Bro language to get this automatically. Thank You, Javier -- Saludos Cordiales Javier Richard Quinto Ancieta Est. maestr?a en Ing. de Computaci?n-UNICAMP Br http://www.linkedin.com/in/richardqa CELL: +51 972205099 (Lima), +55 19 99033699 (Campinas-SP) Fingerprint: 52C8 9361 B7B1 0CDE A7FF 0AAF 6911 459E F588 ACFD -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150511/cb9573fc/attachment.html From rjenkins at rmjconsulting.net Tue May 12 06:28:46 2015 From: rjenkins at rmjconsulting.net (Ron M. Jenkins) Date: Tue, 12 May 2015 13:28:46 +0000 Subject: [Bro] P2P Traffic In-Reply-To: <5550D930.9030308@bro.org> References: <5550D930.9030308@bro.org> Message-ID: <2c491c0a0c3f46c284951cc20457b8b2@RMJCS-FS002.RMJCS.LOCAL> Good morning; I see lots of protocol analyzers in the source, but not after complied and install. How do I get all analyzers installed? Thanks! [cid:image001.png at 01D08C8D.A9944BD0] -----Original Message----- From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Doris Schioberg Sent: Monday, May 11, 2015 11:31 AM To: bro at bro.org Subject: Re: [Bro] P2P Traffic Hi Ron, it that what you are looking for: https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent Doris On 5/11/15 9:15 AM, Ron M. Jenkins wrote: > Good morning; > > Can Bro detected P2P traffic, specially Bitorrent? > > > Thanks! > > > > Ron Jenkins (Owner / Senior Architect) RMJ Consulting, LLC. "Bringing > Companies and Solutions Together" > 11715 Bricksome Ave STE B-7 > Baton Rouge, LA 70816 > Toll: 855-448-5214 > Direct. 225-448-5214 Ext #101 > Fax. 225-448-5324 > Cell. 225-931-1632 > Email. rjenkins at rmjconsulting.net> > Web. http://www.rmjconsulting.net> > Log Siphon. http://www.logsiphon.com> > Linkedin. > www.linkedin.com/in/ronmjenkins/ > s/> > Twitter: > www.twitter.com/RMJConsulting> > Facebook: > www.facebook.com/rmjcsconsulting > ng> RMJ Consulting's Technology Corner. > https://www.rmjconsulting.net/main/paper.php > > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Doris Schioberg Bro Outreach, Training, and Education Coordinator International Computer Science Institute (ICSI Berkeley) Phone: +1 (510) 289-8406 * doris at bro.org _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/13538983/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 26807 bytes Desc: image001.png Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/13538983/attachment-0001.bin From dn1nj4 at gmail.com Tue May 12 06:43:46 2015 From: dn1nj4 at gmail.com (Jason) Date: Tue, 12 May 2015 09:43:46 -0400 Subject: [Bro] PPPoE Capture IP Layer Being Stripped Message-ID: Good day all, One of my sites has all PPPoE traffic on the link I'm monitoring. The .log files are all generated correctly, but PCAP files end up with stripped IP layer information. This was easy to reproduce in bro 2.3.1 on Ubuntu by doing: tcpdump -nn -i ethX -w test.pcap bro -r test.pcap -w bro.pcap The tcpdump traffic in test.pcap looks fine, but the bro pcap comes up as Ethernet traffic with an unknown type. Is this a known bug? Or is there perhaps some configuration that needs to be changed in bro support this traffic? Thanks in advance, Jason -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/3e9f5379/attachment.html From vlad at grigorescu.org Tue May 12 07:28:38 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Tue, 12 May 2015 09:28:38 -0500 Subject: [Bro] P2P Traffic In-Reply-To: <2c491c0a0c3f46c284951cc20457b8b2@RMJCS-FS002.RMJCS.LOCAL> References: <5550D930.9030308@bro.org> <2c491c0a0c3f46c284951cc20457b8b2@RMJCS-FS002.RMJCS.LOCAL> Message-ID: There are two parts to each analyzer - the traffic is parsed off the wire in the "core," which is what you showed in your screenshot, and events are generated. Then, Bro scripts handle the events to generate logs, raise notices, etc. Bro scripts also determine which analyzer will be enabled for a certain TCP or UDP connection. Bro protocol analyzers exist in several different states - most protocol analyzers have both core and script layer code, and get enabled properly. It's possible for an analyzer to be enabled, but not have any event handlers to actually *do* anything with the resulting data (I don't think there are any examples of this right now). Finally, the core parsing code could be present, but the analyzer isn't getting enabled, and there are no scripts either. Some analyzers fall into this third category (including bittorrent). Everything in the screenshot should be getting compiled into Bro, and it's available for you to use, but some may require you to write custom scripts to enable the analyzer or generate logs. To see which analyzers are available in your complied version of Bro, you can run: > % bro --print-plugins > Bro::ARP - ARP Parsing (built-in) > Bro::AYIYA - AYIYA Analyzer (built-in) > Bro::BackDoor - Backdoor Analyzer deprecated (built-in) > Bro::BitTorrent - BitTorrent Analyzer (built-in) > Bro::ConnSize - Connection size analyzer (built-in) > Bro::DCE_RPC - DCE-RPC analyzer (built-in) > Bro::DHCP - DHCP analyzer (built-in) > Bro::DNP3 - DNP3 UDP/TCP analyzers (built-in) > ... For example, if you want to enable the BitTorrent analyzer, you could write a dynamic-protocol detection signature for it like this: > # site/bt_dpd.sig > signature dpd_bittorrent { > ip-proto == tcp > payload /\x13BitTorrent protocol.\x00.\x00\x00/ > enable "bittorrent" > } Then, in your site/local.bro, you could load this with "@load-sigs ./dpd.sig". This should be enough to start seeing BitTorrent P2P connections have the service field of conn.log set to "bittorrent." If you want to take this a step further, and start writing out a bittorrent.log file, you could then start handling the BitTorrent events: https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent --Vlad On Tue, May 12, 2015 at 8:28 AM, Ron M. Jenkins wrote: > Good morning; > > > > I see lots of protocol analyzers in the source, but not after complied and > install. > > > > How do I get all analyzers installed? > > > > > > Thanks! > > > > > > > > -----Original Message----- > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Doris > Schioberg > Sent: Monday, May 11, 2015 11:31 AM > To: bro at bro.org > Subject: Re: [Bro] P2P Traffic > > > > Hi Ron, > > > > it that what you are looking for: > > > https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent > > > > Doris > > > > On 5/11/15 9:15 AM, Ron M. Jenkins wrote: > > > Good morning; > > > > > > Can Bro detected P2P traffic, specially Bitorrent? > > > > > > > > > Thanks! > > > > > > > > > > > > Ron Jenkins (Owner / Senior Architect) RMJ Consulting, LLC. "Bringing > > > Companies and Solutions Together" > > > 11715 Bricksome Ave STE B-7 > > > Baton Rouge, LA 70816 > > > Toll: 855-448-5214 > > > Direct. 225-448-5214 Ext #101 > > > Fax. 225-448-5324 > > > Cell. 225-931-1632 > > > Email. rjenkins at rmjconsulting.net > > > Web. http://www.rmjconsulting.net > > > > Log Siphon. http://www.logsiphon.com > > > > Linkedin. > > > www.linkedin.com/in/ronmjenkins/ > > > s/> > > > Twitter: > > > www.twitter.com/RMJConsulting > > > > > Facebook: > > > www.facebook.com/rmjcsconsulting > > > ng> RMJ Consulting's Technology Corner. > > > https://www.rmjconsulting.net/main/paper.php > > > > > > > > > > > > > > > _______________________________________________ > > > Bro mailing list > > > bro at bro-ids.org > > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > > > -- > > Doris Schioberg > > Bro Outreach, Training, and Education Coordinator International Computer > Science Institute (ICSI Berkeley) > > Phone: +1 (510) 289-8406 * doris at bro.org > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/d6fb9d7f/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 26807 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/d6fb9d7f/attachment-0001.bin From rjenkins at rmjconsulting.net Tue May 12 07:31:59 2015 From: rjenkins at rmjconsulting.net (Ron M. Jenkins) Date: Tue, 12 May 2015 14:31:59 +0000 Subject: [Bro] P2P Traffic In-Reply-To: References: <5550D930.9030308@bro.org> <2c491c0a0c3f46c284951cc20457b8b2@RMJCS-FS002.RMJCS.LOCAL> Message-ID: <4d9145d3a2294674a4c0f0a7d43cae6e@RMJCS-FS002.RMJCS.LOCAL> Great explanation! Thank you! From: grigorescu at gmail.com [mailto:grigorescu at gmail.com] On Behalf Of Vlad Grigorescu Sent: Tuesday, May 12, 2015 9:29 AM To: Ron M. Jenkins Cc: bro at bro.org List (bro at bro.org) Subject: Re: [Bro] P2P Traffic There are two parts to each analyzer - the traffic is parsed off the wire in the "core," which is what you showed in your screenshot, and events are generated. Then, Bro scripts handle the events to generate logs, raise notices, etc. Bro scripts also determine which analyzer will be enabled for a certain TCP or UDP connection. Bro protocol analyzers exist in several different states - most protocol analyzers have both core and script layer code, and get enabled properly. It's possible for an analyzer to be enabled, but not have any event handlers to actually *do* anything with the resulting data (I don't think there are any examples of this right now). Finally, the core parsing code could be present, but the analyzer isn't getting enabled, and there are no scripts either. Some analyzers fall into this third category (including bittorrent). Everything in the screenshot should be getting compiled into Bro, and it's available for you to use, but some may require you to write custom scripts to enable the analyzer or generate logs. To see which analyzers are available in your complied version of Bro, you can run: > % bro --print-plugins > Bro::ARP - ARP Parsing (built-in) > Bro::AYIYA - AYIYA Analyzer (built-in) > Bro::BackDoor - Backdoor Analyzer deprecated (built-in) > Bro::BitTorrent - BitTorrent Analyzer (built-in) > Bro::ConnSize - Connection size analyzer (built-in) > Bro::DCE_RPC - DCE-RPC analyzer (built-in) > Bro::DHCP - DHCP analyzer (built-in) > Bro::DNP3 - DNP3 UDP/TCP analyzers (built-in) > ... For example, if you want to enable the BitTorrent analyzer, you could write a dynamic-protocol detection signature for it like this: > # site/bt_dpd.sig > signature dpd_bittorrent { > ip-proto == tcp > payload /\x13BitTorrent protocol.\x00.\x00\x00/ > enable "bittorrent" > } Then, in your site/local.bro, you could load this with "@load-sigs ./dpd.sig". This should be enough to start seeing BitTorrent P2P connections have the service field of conn.log set to "bittorrent." If you want to take this a step further, and start writing out a bittorrent.log file, you could then start handling the BitTorrent events: https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent --Vlad On Tue, May 12, 2015 at 8:28 AM, Ron M. Jenkins > wrote: Good morning; I see lots of protocol analyzers in the source, but not after complied and install. How do I get all analyzers installed? Thanks! [cid:image001.png at 01D08C96.7E28A3C0] -----Original Message----- From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Doris Schioberg Sent: Monday, May 11, 2015 11:31 AM To: bro at bro.org Subject: Re: [Bro] P2P Traffic Hi Ron, it that what you are looking for: https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent Doris On 5/11/15 9:15 AM, Ron M. Jenkins wrote: > Good morning; > > Can Bro detected P2P traffic, specially Bitorrent? > > > Thanks! > > > > Ron Jenkins (Owner / Senior Architect) RMJ Consulting, LLC. "Bringing > Companies and Solutions Together" > 11715 Bricksome Ave STE B-7 > Baton Rouge, LA 70816 > Toll: 855-448-5214 > Direct. 225-448-5214 Ext #101 > Fax. 225-448-5324 > Cell. 225-931-1632 > Email. rjenkins at rmjconsulting.net> > Web. http://www.rmjconsulting.net> > Log Siphon. http://www.logsiphon.com> > Linkedin. > www.linkedin.com/in/ronmjenkins/ > s/> > Twitter: > www.twitter.com/RMJConsulting> > Facebook: > www.facebook.com/rmjcsconsulting > ng> RMJ Consulting's Technology Corner. > https://www.rmjconsulting.net/main/paper.php > > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Doris Schioberg Bro Outreach, Training, and Education Coordinator International Computer Science Institute (ICSI Berkeley) Phone: +1 (510) 289-8406 * doris at bro.org _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/6fb5933e/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 26807 bytes Desc: image001.png Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/6fb5933e/attachment-0001.bin From rjenkins at rmjconsulting.net Tue May 12 07:40:49 2015 From: rjenkins at rmjconsulting.net (Ron M. Jenkins) Date: Tue, 12 May 2015 14:40:49 +0000 Subject: [Bro] P2P Traffic In-Reply-To: References: <5550D930.9030308@bro.org> <2c491c0a0c3f46c284951cc20457b8b2@RMJCS-FS002.RMJCS.LOCAL> Message-ID: Has any already set this to log to a file? Thanks! From: grigorescu at gmail.com [mailto:grigorescu at gmail.com] On Behalf Of Vlad Grigorescu Sent: Tuesday, May 12, 2015 9:29 AM To: Ron M. Jenkins Cc: bro at bro.org List (bro at bro.org) Subject: Re: [Bro] P2P Traffic There are two parts to each analyzer - the traffic is parsed off the wire in the "core," which is what you showed in your screenshot, and events are generated. Then, Bro scripts handle the events to generate logs, raise notices, etc. Bro scripts also determine which analyzer will be enabled for a certain TCP or UDP connection. Bro protocol analyzers exist in several different states - most protocol analyzers have both core and script layer code, and get enabled properly. It's possible for an analyzer to be enabled, but not have any event handlers to actually *do* anything with the resulting data (I don't think there are any examples of this right now). Finally, the core parsing code could be present, but the analyzer isn't getting enabled, and there are no scripts either. Some analyzers fall into this third category (including bittorrent). Everything in the screenshot should be getting compiled into Bro, and it's available for you to use, but some may require you to write custom scripts to enable the analyzer or generate logs. To see which analyzers are available in your complied version of Bro, you can run: > % bro --print-plugins > Bro::ARP - ARP Parsing (built-in) > Bro::AYIYA - AYIYA Analyzer (built-in) > Bro::BackDoor - Backdoor Analyzer deprecated (built-in) > Bro::BitTorrent - BitTorrent Analyzer (built-in) > Bro::ConnSize - Connection size analyzer (built-in) > Bro::DCE_RPC - DCE-RPC analyzer (built-in) > Bro::DHCP - DHCP analyzer (built-in) > Bro::DNP3 - DNP3 UDP/TCP analyzers (built-in) > ... For example, if you want to enable the BitTorrent analyzer, you could write a dynamic-protocol detection signature for it like this: > # site/bt_dpd.sig > signature dpd_bittorrent { > ip-proto == tcp > payload /\x13BitTorrent protocol.\x00.\x00\x00/ > enable "bittorrent" > } Then, in your site/local.bro, you could load this with "@load-sigs ./dpd.sig". This should be enough to start seeing BitTorrent P2P connections have the service field of conn.log set to "bittorrent." If you want to take this a step further, and start writing out a bittorrent.log file, you could then start handling the BitTorrent events: https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent --Vlad On Tue, May 12, 2015 at 8:28 AM, Ron M. Jenkins > wrote: Good morning; I see lots of protocol analyzers in the source, but not after complied and install. How do I get all analyzers installed? Thanks! [cid:image001.png at 01D08C97.BAD0F0B0] -----Original Message----- From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Doris Schioberg Sent: Monday, May 11, 2015 11:31 AM To: bro at bro.org Subject: Re: [Bro] P2P Traffic Hi Ron, it that what you are looking for: https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent Doris On 5/11/15 9:15 AM, Ron M. Jenkins wrote: > Good morning; > > Can Bro detected P2P traffic, specially Bitorrent? > > > Thanks! > > > > Ron Jenkins (Owner / Senior Architect) RMJ Consulting, LLC. "Bringing > Companies and Solutions Together" > 11715 Bricksome Ave STE B-7 > Baton Rouge, LA 70816 > Toll: 855-448-5214 > Direct. 225-448-5214 Ext #101 > Fax. 225-448-5324 > Cell. 225-931-1632 > Email. rjenkins at rmjconsulting.net> > Web. http://www.rmjconsulting.net> > Log Siphon. http://www.logsiphon.com> > Linkedin. > www.linkedin.com/in/ronmjenkins/ > s/> > Twitter: > www.twitter.com/RMJConsulting> > Facebook: > www.facebook.com/rmjcsconsulting > ng> RMJ Consulting's Technology Corner. > https://www.rmjconsulting.net/main/paper.php > > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Doris Schioberg Bro Outreach, Training, and Education Coordinator International Computer Science Institute (ICSI Berkeley) Phone: +1 (510) 289-8406 * doris at bro.org _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/5ffe9d05/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 26807 bytes Desc: image001.png Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/5ffe9d05/attachment-0001.bin From jlay at slave-tothe-box.net Tue May 12 09:04:56 2015 From: jlay at slave-tothe-box.net (James Lay) Date: Tue, 12 May 2015 10:04:56 -0600 Subject: [Bro] PPPoE Capture IP Layer Being Stripped In-Reply-To: References: Message-ID: On 2015-05-12 07:43 AM, Jason wrote: > Good day all, > > One of my sites has all PPPoE traffic on the link I'm monitoring. The > .log files are all generated correctly, but PCAP files end up with > stripped IP layer information. This was easy to reproduce in bro > 2.3.1 on Ubuntu by doing: > > tcpdump -nn -i ethX -w test.pcap > bro -r test.pcap -w bro.pcap > > The tcpdump traffic in test.pcap looks fine, but the bro pcap comes up > as Ethernet traffic with an unknown type. > > Is this a known bug? Or is there perhaps some configuration that > needs to be changed in bro support this traffic? > > Thanks in advance, > > Jason > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro I run bro on ppp0, but I don't think I've seen this issue. Have you tried having bro listen on the physical interface instead? James From anthony.kasza at gmail.com Tue May 12 09:23:31 2015 From: anthony.kasza at gmail.com (anthony kasza) Date: Tue, 12 May 2015 09:23:31 -0700 Subject: [Bro] P2P Traffic In-Reply-To: References: <5550D930.9030308@bro.org> <2c491c0a0c3f46c284951cc20457b8b2@RMJCS-FS002.RMJCS.LOCAL> Message-ID: This might be a silly question, but why does Bro have scripts for analyzers supported by the core? -AK On May 12, 2015 7:32 AM, "Vlad Grigorescu" wrote: > There are two parts to each analyzer - the traffic is parsed off the wire > in the "core," which is what you showed in your screenshot, and events are > generated. Then, Bro scripts handle the events to generate logs, raise > notices, etc. Bro scripts also determine which analyzer will be enabled for > a certain TCP or UDP connection. Bro protocol analyzers exist in several > different states - most protocol analyzers have both core and script layer > code, and get enabled properly. It's possible for an analyzer to be > enabled, but not have any event handlers to actually *do* anything with the > resulting data (I don't think there are any examples of this right now). > Finally, the core parsing code could be present, but the analyzer isn't > getting enabled, and there are no scripts either. Some analyzers fall into > this third category (including bittorrent). > > Everything in the screenshot should be getting compiled into Bro, and it's > available for you to use, but some may require you to write custom scripts > to enable the analyzer or generate logs. To see which analyzers are > available in your complied version of Bro, you can run: > > > % bro --print-plugins > > Bro::ARP - ARP Parsing (built-in) > > Bro::AYIYA - AYIYA Analyzer (built-in) > > Bro::BackDoor - Backdoor Analyzer deprecated (built-in) > > Bro::BitTorrent - BitTorrent Analyzer (built-in) > > Bro::ConnSize - Connection size analyzer (built-in) > > Bro::DCE_RPC - DCE-RPC analyzer (built-in) > > Bro::DHCP - DHCP analyzer (built-in) > > Bro::DNP3 - DNP3 UDP/TCP analyzers (built-in) > > ... > > For example, if you want to enable the BitTorrent analyzer, you could > write a dynamic-protocol detection signature for it like this: > > > # site/bt_dpd.sig > > signature dpd_bittorrent { > > ip-proto == tcp > > payload /\x13BitTorrent protocol.\x00.\x00\x00/ > > enable "bittorrent" > > } > > Then, in your site/local.bro, you could load this with "@load-sigs > ./dpd.sig". This should be enough to start seeing BitTorrent P2P > connections have the service field of conn.log set to "bittorrent." If you > want to take this a step further, and start writing out a bittorrent.log > file, you could then start handling the BitTorrent events: > https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent > > --Vlad > > On Tue, May 12, 2015 at 8:28 AM, Ron M. Jenkins < > rjenkins at rmjconsulting.net> wrote: > >> Good morning; >> >> >> >> I see lots of protocol analyzers in the source, but not after complied >> and install. >> >> >> >> How do I get all analyzers installed? >> >> >> >> >> >> Thanks! >> >> >> >> >> >> >> >> -----Original Message----- >> From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of >> Doris Schioberg >> Sent: Monday, May 11, 2015 11:31 AM >> To: bro at bro.org >> Subject: Re: [Bro] P2P Traffic >> >> >> >> Hi Ron, >> >> >> >> it that what you are looking for: >> >> >> https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent >> >> >> >> Doris >> >> >> >> On 5/11/15 9:15 AM, Ron M. Jenkins wrote: >> >> > Good morning; >> >> > >> >> > Can Bro detected P2P traffic, specially Bitorrent? >> >> > >> >> > >> >> > Thanks! >> >> > >> >> > >> >> > >> >> > Ron Jenkins (Owner / Senior Architect) RMJ Consulting, LLC. "Bringing >> >> > Companies and Solutions Together" >> >> > 11715 Bricksome Ave STE B-7 >> >> > Baton Rouge, LA 70816 >> >> > Toll: 855-448-5214 >> >> > Direct. 225-448-5214 Ext #101 >> >> > Fax. 225-448-5324 >> >> > Cell. 225-931-1632 >> >> > Email. rjenkins at rmjconsulting.net >> >> > Web. http://www.rmjconsulting.net> > >> >> > Log Siphon. http://www.logsiphon.com> > >> >> > Linkedin. >> >> > www.linkedin.com/in/ronmjenkins/> >> >> > s/> >> >> > Twitter: >> >> > www.twitter.com/RMJConsulting> >> > >> >> > Facebook: >> >> > www.facebook.com/rmjcsconsulting> >> >> > ng> RMJ Consulting's Technology Corner. >> >> > https://www.rmjconsulting.net/main/paper.php >> >> > >> >> > >> >> > >> >> > >> >> > _______________________________________________ >> >> > Bro mailing list >> >> > bro at bro-ids.org >> >> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> > >> >> >> >> -- >> >> Doris Schioberg >> >> Bro Outreach, Training, and Education Coordinator International Computer >> Science Institute (ICSI Berkeley) >> >> Phone: +1 (510) 289-8406 * doris at bro.org >> _______________________________________________ >> >> Bro mailing list >> >> bro at bro-ids.org >> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/2f8eff09/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 26807 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/2f8eff09/attachment-0001.bin From anthony.kasza at gmail.com Tue May 12 09:25:04 2015 From: anthony.kasza at gmail.com (anthony kasza) Date: Tue, 12 May 2015 09:25:04 -0700 Subject: [Bro] Run scripts of Python in Bro scripting In-Reply-To: References: Message-ID: Grep for the Exec framework. It should allow you to run a command at the shell from scriptland. -AK On May 11, 2015 7:21 PM, "Javier Richard Quinto Ancieta" < richardqa at gmail.com> wrote: > Hi everyone, > > I wonder if there is a way of running a script in python at the precise > moment that Bro launches some event such as Intel.log file is created?. > > For example, if a malicious activity is launched to the network, o intel > framework detects and create a Intel.log, then my script in python > (script.py) read this intel.log file and execute a countermeasure blocking > such malicious activity, I would like to know if there is any way to run > this Python script within the Bro language to get this automatically. > > Thank You, > Javier > > > -- > Saludos Cordiales > Javier Richard Quinto Ancieta > Est. maestr?a en Ing. de Computaci?n-UNICAMP Br > http://www.linkedin.com/in/richardqa > CELL: +51 972205099 (Lima), +55 19 99033699 (Campinas-SP) > Fingerprint: 52C8 9361 B7B1 0CDE A7FF 0AAF 6911 459E F588 ACFD > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/e5559162/attachment.html From dn1nj4 at gmail.com Tue May 12 09:51:18 2015 From: dn1nj4 at gmail.com (Jason) Date: Tue, 12 May 2015 12:51:18 -0400 Subject: [Bro] Bro Digest, Vol 109, Issue 14 In-Reply-To: References: Message-ID: > Date: Tue, 12 May 2015 10:04:56 -0600 > From: James Lay > Subject: Re: [Bro] PPPoE Capture IP Layer Being Stripped > To: bro at bro.org > Message-ID: > Content-Type: text/plain; charset=US-ASCII; format=flowed > > On 2015-05-12 07:43 AM, Jason wrote: > > Good day all, > > > > One of my sites has all PPPoE traffic on the link I'm monitoring. The > > .log files are all generated correctly, but PCAP files end up with > > stripped IP layer information. This was easy to reproduce in bro > > 2.3.1 on Ubuntu by doing: > > > > tcpdump -nn -i ethX -w test.pcap > > bro -r test.pcap -w bro.pcap > > > > The tcpdump traffic in test.pcap looks fine, but the bro pcap comes up > > as Ethernet traffic with an unknown type. > > > > Is this a known bug? Or is there perhaps some configuration that > > needs to be changed in bro support this traffic? > > > > Thanks in advance, > > > > Jason > > > > I run bro on ppp0, but I don't think I've seen this issue. Have you > tried having bro listen on the physical interface instead? > > James > > > ------------------------------ > > I have indeed. Live capture was where the problem was first noticed. I moved to an offline/tcpdump test as part of my troubleshooting to ensure nothing else was causing problems (link issues, PF_RING, etc). -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/f911b08d/attachment.html From albert.zaharovits at gmail.com Tue May 12 10:12:58 2015 From: albert.zaharovits at gmail.com (Albert Zaharovits) Date: Tue, 12 May 2015 20:12:58 +0300 Subject: [Bro] Extract complete files Message-ID: Hello, I am experimenting with the Files framework in bro 2.4 beta. I would like to extract HTTP files, *without* missing_bytes. Can anyone please help me on this? Thanks, Albert From rjenkins at rmjconsulting.net Tue May 12 12:21:11 2015 From: rjenkins at rmjconsulting.net (Ron M. Jenkins) Date: Tue, 12 May 2015 19:21:11 +0000 Subject: [Bro] P2P Traffic In-Reply-To: References: <5550D930.9030308@bro.org> <2c491c0a0c3f46c284951cc20457b8b2@RMJCS-FS002.RMJCS.LOCAL> Message-ID: Where did you determine the payload part? payload /\x13BitTorrent protocol.\x00.\x00\x00/ Thanks! From: grigorescu at gmail.com [mailto:grigorescu at gmail.com] On Behalf Of Vlad Grigorescu Sent: Tuesday, May 12, 2015 9:29 AM To: Ron M. Jenkins Cc: bro at bro.org List (bro at bro.org) Subject: Re: [Bro] P2P Traffic There are two parts to each analyzer - the traffic is parsed off the wire in the "core," which is what you showed in your screenshot, and events are generated. Then, Bro scripts handle the events to generate logs, raise notices, etc. Bro scripts also determine which analyzer will be enabled for a certain TCP or UDP connection. Bro protocol analyzers exist in several different states - most protocol analyzers have both core and script layer code, and get enabled properly. It's possible for an analyzer to be enabled, but not have any event handlers to actually *do* anything with the resulting data (I don't think there are any examples of this right now). Finally, the core parsing code could be present, but the analyzer isn't getting enabled, and there are no scripts either. Some analyzers fall into this third category (including bittorrent). Everything in the screenshot should be getting compiled into Bro, and it's available for you to use, but some may require you to write custom scripts to enable the analyzer or generate logs. To see which analyzers are available in your complied version of Bro, you can run: > % bro --print-plugins > Bro::ARP - ARP Parsing (built-in) > Bro::AYIYA - AYIYA Analyzer (built-in) > Bro::BackDoor - Backdoor Analyzer deprecated (built-in) > Bro::BitTorrent - BitTorrent Analyzer (built-in) > Bro::ConnSize - Connection size analyzer (built-in) > Bro::DCE_RPC - DCE-RPC analyzer (built-in) > Bro::DHCP - DHCP analyzer (built-in) > Bro::DNP3 - DNP3 UDP/TCP analyzers (built-in) > ... For example, if you want to enable the BitTorrent analyzer, you could write a dynamic-protocol detection signature for it like this: > # site/bt_dpd.sig > signature dpd_bittorrent { > ip-proto == tcp > payload /\x13BitTorrent protocol.\x00.\x00\x00/ > enable "bittorrent" > } Then, in your site/local.bro, you could load this with "@load-sigs ./dpd.sig". This should be enough to start seeing BitTorrent P2P connections have the service field of conn.log set to "bittorrent." If you want to take this a step further, and start writing out a bittorrent.log file, you could then start handling the BitTorrent events: https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent --Vlad On Tue, May 12, 2015 at 8:28 AM, Ron M. Jenkins > wrote: Good morning; I see lots of protocol analyzers in the source, but not after complied and install. How do I get all analyzers installed? Thanks! [cid:image001.png at 01D08C9A.1F5523B0] -----Original Message----- From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Doris Schioberg Sent: Monday, May 11, 2015 11:31 AM To: bro at bro.org Subject: Re: [Bro] P2P Traffic Hi Ron, it that what you are looking for: https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent Doris On 5/11/15 9:15 AM, Ron M. Jenkins wrote: > Good morning; > > Can Bro detected P2P traffic, specially Bitorrent? > > > Thanks! > > > > Ron Jenkins (Owner / Senior Architect) RMJ Consulting, LLC. "Bringing > Companies and Solutions Together" > 11715 Bricksome Ave STE B-7 > Baton Rouge, LA 70816 > Toll: 855-448-5214 > Direct. 225-448-5214 Ext #101 > Fax. 225-448-5324 > Cell. 225-931-1632 > Email. rjenkins at rmjconsulting.net> > Web. http://www.rmjconsulting.net> > Log Siphon. http://www.logsiphon.com> > Linkedin. > www.linkedin.com/in/ronmjenkins/ > s/> > Twitter: > www.twitter.com/RMJConsulting> > Facebook: > www.facebook.com/rmjcsconsulting > ng> RMJ Consulting's Technology Corner. > https://www.rmjconsulting.net/main/paper.php > > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Doris Schioberg Bro Outreach, Training, and Education Coordinator International Computer Science Institute (ICSI Berkeley) Phone: +1 (510) 289-8406 * doris at bro.org _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/47c87803/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 26807 bytes Desc: image001.png Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/47c87803/attachment-0001.bin From vlad at grigorescu.org Tue May 12 14:32:45 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Tue, 12 May 2015 16:32:45 -0500 Subject: [Bro] P2P Traffic In-Reply-To: References: <5550D930.9030308@bro.org> <2c491c0a0c3f46c284951cc20457b8b2@RMJCS-FS002.RMJCS.LOCAL> Message-ID: >From the spec: http://www.bittorrent.org/beps/bep_0003.html > The peer wire protocol consists of a handshake followed by a never-ending stream of length-prefixed messages. The handshake starts with character ninteen (decimal) followed by the string 'BitTorrent protocol'. ... After the fixed headers come eight reserved bytes, which are all zero in all current implementations. There are some extensions that are defined here: http://www.bittorrent.org/beps/bep_0000.html. I couldn't find any extensions that used the 2nd, 4th or 5th extension bytes, so I hard-coded those to 0. On Tue, May 12, 2015 at 2:21 PM, Ron M. Jenkins wrote: > Where did you determine the payload part? > > > > payload /\x13BitTorrent protocol.\x00.\x00\x00/ > > > > > > > > Thanks! > > > > > > > > > > *From:* grigorescu at gmail.com [mailto:grigorescu at gmail.com] *On Behalf Of *Vlad > Grigorescu > *Sent:* Tuesday, May 12, 2015 9:29 AM > *To:* Ron M. Jenkins > *Cc:* bro at bro.org List (bro at bro.org) > *Subject:* Re: [Bro] P2P Traffic > > > > There are two parts to each analyzer - the traffic is parsed off the wire > in the "core," which is what you showed in your screenshot, and events are > generated. Then, Bro scripts handle the events to generate logs, raise > notices, etc. Bro scripts also determine which analyzer will be enabled for > a certain TCP or UDP connection. Bro protocol analyzers exist in several > different states - most protocol analyzers have both core and script layer > code, and get enabled properly. It's possible for an analyzer to be > enabled, but not have any event handlers to actually *do* anything with the > resulting data (I don't think there are any examples of this right now). > Finally, the core parsing code could be present, but the analyzer isn't > getting enabled, and there are no scripts either. Some analyzers fall into > this third category (including bittorrent). > > > > Everything in the screenshot should be getting compiled into Bro, and it's > available for you to use, but some may require you to write custom scripts > to enable the analyzer or generate logs. To see which analyzers are > available in your complied version of Bro, you can run: > > > > > % bro --print-plugins > > > Bro::ARP - ARP Parsing (built-in) > > > Bro::AYIYA - AYIYA Analyzer (built-in) > > > Bro::BackDoor - Backdoor Analyzer deprecated (built-in) > > > Bro::BitTorrent - BitTorrent Analyzer (built-in) > > > Bro::ConnSize - Connection size analyzer (built-in) > > > Bro::DCE_RPC - DCE-RPC analyzer (built-in) > > > Bro::DHCP - DHCP analyzer (built-in) > > > Bro::DNP3 - DNP3 UDP/TCP analyzers (built-in) > > > ... > > > > For example, if you want to enable the BitTorrent analyzer, you could > write a dynamic-protocol detection signature for it like this: > > > > > # site/bt_dpd.sig > > > signature dpd_bittorrent { > > > ip-proto == tcp > > > payload /\x13BitTorrent protocol.\x00.\x00\x00/ > > > enable "bittorrent" > > > } > > > > Then, in your site/local.bro, you could load this with "@load-sigs > ./dpd.sig". This should be enough to start seeing BitTorrent P2P > connections have the service field of conn.log set to "bittorrent." If you > want to take this a step further, and start writing out a bittorrent.log > file, you could then start handling the BitTorrent events: > https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent > > > > --Vlad > > > > On Tue, May 12, 2015 at 8:28 AM, Ron M. Jenkins < > rjenkins at rmjconsulting.net> wrote: > > Good morning; > > > > I see lots of protocol analyzers in the source, but not after complied and > install. > > > > How do I get all analyzers installed? > > > > > > Thanks! > > > > > > [image: cid:image001.png at 01D08C9A.1F5523B0] > > > > -----Original Message----- > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Doris > Schioberg > Sent: Monday, May 11, 2015 11:31 AM > To: bro at bro.org > Subject: Re: [Bro] P2P Traffic > > > > Hi Ron, > > > > it that what you are looking for: > > > https://www.bro.org/sphinx-git/script-reference/proto-analyzers.html#bro-bittorrent > > > > Doris > > > > On 5/11/15 9:15 AM, Ron M. Jenkins wrote: > > > Good morning; > > > > > > Can Bro detected P2P traffic, specially Bitorrent? > > > > > > > > > Thanks! > > > > > > > > > > > > Ron Jenkins (Owner / Senior Architect) RMJ Consulting, LLC. "Bringing > > > Companies and Solutions Together" > > > 11715 Bricksome Ave STE B-7 > > > Baton Rouge, LA 70816 > > > Toll: 855-448-5214 > > > Direct. 225-448-5214 Ext #101 > > > Fax. 225-448-5324 > > > Cell. 225-931-1632 > > > Email. rjenkins at rmjconsulting.net > > > Web. http://www.rmjconsulting.net > > > > Log Siphon. http://www.logsiphon.com > > > > Linkedin. > > > www.linkedin.com/in/ronmjenkins/ > > > s/> > > > Twitter: > > > www.twitter.com/RMJConsulting > > > > > Facebook: > > > www.facebook.com/rmjcsconsulting > > > ng> RMJ Consulting's Technology Corner. > > > https://www.rmjconsulting.net/main/paper.php > > > > > > > > > > > > > > > _______________________________________________ > > > Bro mailing list > > > bro at bro-ids.org > > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > > > -- > > Doris Schioberg > > Bro Outreach, Training, and Education Coordinator International Computer > Science Institute (ICSI Berkeley) > > Phone: +1 (510) 289-8406 * doris at bro.org > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150512/1cfe4af9/attachment.html From franky.meier.1 at gmx.de Wed May 13 07:46:00 2015 From: franky.meier.1 at gmx.de (Frank Meier) Date: Wed, 13 May 2015 16:46:00 +0200 Subject: [Bro] Extract complete files In-Reply-To: References: Message-ID: <1431528360.4024.0@mail.gmx.net> Hi Albert, it's hard to help without any context, so just some hints: It took me some time to find the -C switch to ignore wrong checksums in bro. Without it the traffic did not reach the extraction layer. Also it's always a good idea to compare bro with other tools. Make sure wireshark does show the complete http session. Franky On Di, Mai 12, 2015 at 7:12 , Albert Zaharovits wrote: > Hello, > > I am experimenting with the Files framework in bro 2.4 beta. I would > like to extract HTTP files, *without* missing_bytes. > Can anyone please help me on this? > > Thanks, > Albert > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150513/71d2ab88/attachment.html From albert.zaharovits at gmail.com Wed May 13 08:12:24 2015 From: albert.zaharovits at gmail.com (Albert Zaharovits) Date: Wed, 13 May 2015 18:12:24 +0300 Subject: [Bro] Extract complete files In-Reply-To: <1431528360.4024.0@mail.gmx.net> References: <1431528360.4024.0@mail.gmx.net> Message-ID: <6ADFEA43-DEA8-47FC-8EF0-A1D8EB91BF9F@gmail.com> Hi Frank, Perhaps I didn?t explain myself properly. I meant extracting only complete files (or removing incomplete ones). There might be file gaps because of improper taping? I attached the Files::ANALYZER_EXTRACT and Files::ANALYZER_SHA256 in the file_sniff event. The event_hash triggers only for complete files, but the file gets extracted anyway. Albert > On 13 May 2015, at 17:46, Frank Meier wrote: > > Hi Albert, > > it's hard to help without any context, so just some hints: It took me some time to find the -C switch to ignore wrong checksums in bro. Without it the traffic did not reach the extraction layer. Also it's always a good idea to compare bro with other tools. Make sure wireshark does show the complete http session. > > > Franky > > On Di, Mai 12, 2015 at 7:12 , Albert Zaharovits wrote: >> Hello, >> >> I am experimenting with the Files framework in bro 2.4 beta. I would like to extract HTTP files, *without* missing_bytes. >> Can anyone please help me on this? >> >> Thanks, >> Albert >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150513/8ee877b4/attachment-0001.html From hosom at battelle.org Wed May 13 08:17:06 2015 From: hosom at battelle.org (Hosom, Stephen M) Date: Wed, 13 May 2015 15:17:06 +0000 Subject: [Bro] Extract complete files In-Reply-To: References: Message-ID: Albert, You have a chicken and egg problem. Specifically, you're not going to be certain of how many bytes are missing at the time you have to determine whether or not you'll be extracting the file. Instead, you'll have to extract all files and then later remove the files that aren't the ones that you want. This is similar to how the issue of 'how do I name the file after the hash' is solved. I have some examples of that here in the plugins directory: https://github.com/hosom/bro-file-extraction/ While it isn't precisely what you want... you'll be able to piece together the hashing examples into removing files from the filesystem that show as having missing bytes. If you're seeing a large number of missing bytes in files consistently, there are likely other problems occurring. Thanks, Stephen -----Original Message----- From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Albert Zaharovits Sent: Tuesday, May 12, 2015 1:13 PM To: bro at bro.org Subject: [Bro] Extract complete files Hello, I am experimenting with the Files framework in bro 2.4 beta. I would like to extract HTTP files, *without* missing_bytes. Can anyone please help me on this? Thanks, Albert _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From albert.zaharovits at gmail.com Wed May 13 08:24:27 2015 From: albert.zaharovits at gmail.com (Albert Zaharovits) Date: Wed, 13 May 2015 18:24:27 +0300 Subject: [Bro] Extract complete files In-Reply-To: References: Message-ID: <6A49F2F6-9403-4844-8B9F-0CF8C16B63DA@gmail.com> Hi Stephen, Your rename-after-hash example is a life saviour! Thanks a bunch, Albert > On 13 May 2015, at 18:17, Hosom, Stephen M wrote: > > Albert, > > You have a chicken and egg problem. Specifically, you're not going to be certain of how many bytes are missing at the time you have to determine whether or not you'll be extracting the file. Instead, you'll have to extract all files and then later remove the files that aren't the ones that you want. This is similar to how the issue of 'how do I name the file after the hash' is solved. > > I have some examples of that here in the plugins directory: https://github.com/hosom/bro-file-extraction/ > > While it isn't precisely what you want... you'll be able to piece together the hashing examples into removing files from the filesystem that show as having missing bytes. > > If you're seeing a large number of missing bytes in files consistently, there are likely other problems occurring. > > Thanks, > > Stephen > > -----Original Message----- > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Albert Zaharovits > Sent: Tuesday, May 12, 2015 1:13 PM > To: bro at bro.org > Subject: [Bro] Extract complete files > > Hello, > > I am experimenting with the Files framework in bro 2.4 beta. I would like to extract HTTP files, *without* missing_bytes. > Can anyone please help me on this? > > Thanks, > Albert > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From dn1nj4 at gmail.com Thu May 14 00:57:18 2015 From: dn1nj4 at gmail.com (Jason) Date: Thu, 14 May 2015 03:57:18 -0400 Subject: [Bro] PPPoE Capture IP Layer Being Stripped Message-ID: On Tue, May 12, 2015 at 12:51 PM, Jason wrote: > > Date: Tue, 12 May 2015 10:04:56 -0600 >> From: James Lay >> Subject: Re: [Bro] PPPoE Capture IP Layer Being Stripped >> To: bro at bro.org >> Message-ID: >> Content-Type: text/plain; charset=US-ASCII; format=flowed >> >> On 2015-05-12 07:43 AM, Jason wrote: >> > Good day all, >> > >> > One of my sites has all PPPoE traffic on the link I'm monitoring. The >> > .log files are all generated correctly, but PCAP files end up with >> > stripped IP layer information. This was easy to reproduce in bro >> > 2.3.1 on Ubuntu by doing: >> > >> > tcpdump -nn -i ethX -w test.pcap >> > bro -r test.pcap -w bro.pcap >> > >> > The tcpdump traffic in test.pcap looks fine, but the bro pcap comes up >> > as Ethernet traffic with an unknown type. >> > >> > Is this a known bug? Or is there perhaps some configuration that >> > needs to be changed in bro support this traffic? >> > >> > Thanks in advance, >> > >> > Jason >> > >> >> I run bro on ppp0, but I don't think I've seen this issue. Have you >> tried having bro listen on the physical interface instead? >> >> James >> >> >> ------------------------------ >> >> I have indeed. Live capture was where the problem was first noticed. I > moved to an offline/tcpdump test as part of my troubleshooting to ensure > nothing else was causing problems (link issues, PF_RING, etc). > Has anyone else run into these problems? Any suggestions? As far as I can tell it's specific to bro. Thanks again, Jason -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150514/6c301b8a/attachment.html From jlay at slave-tothe-box.net Thu May 14 04:06:52 2015 From: jlay at slave-tothe-box.net (James Lay) Date: Thu, 14 May 2015 05:06:52 -0600 Subject: [Bro] PPPoE Capture IP Layer Being Stripped In-Reply-To: References: Message-ID: <1431601612.4282.1.camel@JamesiMac> On Thu, 2015-05-14 at 03:57 -0400, Jason wrote: > On Tue, May 12, 2015 at 12:51 PM, Jason wrote: > > > > Date: Tue, 12 May 2015 10:04:56 -0600 > From: James Lay > Subject: Re: [Bro] PPPoE Capture IP Layer Being > Stripped > To: bro at bro.org > Message-ID: > > Content-Type: text/plain; charset=US-ASCII; > format=flowed > > On 2015-05-12 07:43 AM, Jason wrote: > > Good day all, > > > > One of my sites has all PPPoE traffic on the link > I'm monitoring. The > > .log files are all generated correctly, but PCAP > files end up with > > stripped IP layer information. This was easy to > reproduce in bro > > 2.3.1 on Ubuntu by doing: > > > > tcpdump -nn -i ethX -w test.pcap > > bro -r test.pcap -w bro.pcap > > > > The tcpdump traffic in test.pcap looks fine, but the > bro pcap comes up > > as Ethernet traffic with an unknown type. > > > > Is this a known bug? Or is there perhaps some > configuration that > > needs to be changed in bro support this traffic? > > > > Thanks in advance, > > > > Jason > > > > I run bro on ppp0, but I don't think I've seen this > issue. Have you > tried having bro listen on the physical interface > instead? > > James > > > ------------------------------ > > > I have indeed. Live capture was where the problem was first > noticed. I moved to an offline/tcpdump test as part of my > troubleshooting to ensure nothing else was causing problems > (link issues, PF_RING, etc). > > > > > Has anyone else run into these problems? Any suggestions? As far as > I can tell it's specific to bro. > > > Thanks again, > Jason > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro At this stage I would file a bug report. James -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150514/ad1926d9/attachment.html From john at omernik.com Sat May 16 13:57:03 2015 From: john at omernik.com (John Omernik) Date: Sat, 16 May 2015 15:57:03 -0500 Subject: [Bro] Bro Types Not Following Bro Types Documention Message-ID: I am working doing some automation with Bro, Avro, Kafka and I am a little bit frustrated. (Or I am looking at the wrong documentation, hence my post here, I am very good with being extremely wrong because I am looking at something wrong). Specifically I am looking at the default conn.log. The Type that is specified for some fields such as orig_bytes or resp_bytes is type count Based on the docs I am using here: https://www.bro.org/sphinx/script-reference/builtins.html a count is: count A numeric type representing a 64-bit unsigned integer. A count constant is a string of digits, e.g.1234 or 0. A count can also be written in hexadecimal notation (in which case ?0x? must precede the hex digits), e.g. 0xff or 0xABC123. The count type supports the same operators as the int type. A unary plus or minus applied to acount results in an int. This is well and good, however looking at some of the data in my log I see the character "-" as a value. Based on my reading of a count, that shouldn't exist, a - is not a unsigned integer, nor is it a string of digits whether in base 10 or hexidecimal. Thus my frustration, I'd like to develop some generic bindings to push bro logs into Avro Serialized Kafka messages, but looking at this, I can't even trust the documentation to be accurate? Am I missing something? Is there another documentation reference that more fully represents the data types that would explain why - is a valid integer? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150516/e863196a/attachment.html From dopheide at gmail.com Sat May 16 14:44:48 2015 From: dopheide at gmail.com (Mike Dopheide) Date: Sat, 16 May 2015 16:44:48 -0500 Subject: [Bro] Bro Types Not Following Bro Types Documention In-Reply-To: References: Message-ID: John, The "-" you're seeing in this case isn't meant to be representing a count. The "-" is used in Bro logs to represent a field with missing data. For whatever reason Bro couldn't the bytes sent in this case. You'll see -'s more commonly in other logs, like http.log, ssl.log, etc. -Dop On Sat, May 16, 2015 at 3:57 PM, John Omernik wrote: > I am working doing some automation with Bro, Avro, Kafka and I am a little > bit frustrated. (Or I am looking at the wrong documentation, hence my post > here, I am very good with being extremely wrong because I am looking at > something wrong). > > Specifically I am looking at the default conn.log. The Type that is > specified for some fields such as > orig_bytes or resp_bytes is type count > > Based on the docs I am using here: > > https://www.bro.org/sphinx/script-reference/builtins.html > > a count is: > > count > > A numeric type representing a 64-bit unsigned integer. A count constant > is a string of digits, e.g.1234 or 0. A count can also be written in > hexadecimal notation (in which case ?0x? must precede the hex digits), e.g. > 0xff or 0xABC123. > > The count type supports the same operators as the int > type. > A unary plus or minus applied to acount results in an int. > > > This is well and good, however looking at some of the data in my log I see > the character "-" as a value. Based on my reading of a count, that > shouldn't exist, a - is not a unsigned integer, nor is it a string of > digits whether in base 10 or hexidecimal. > > > Thus my frustration, I'd like to develop some generic bindings to push bro > logs into Avro Serialized Kafka messages, but looking at this, I can't even > trust the documentation to be accurate? Am I missing something? Is there > another documentation reference that more fully represents the data types > that would explain why - is a valid integer? > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150516/5d3658cd/attachment.html From renaud.luca at gmail.com Sun May 17 19:55:38 2015 From: renaud.luca at gmail.com (Luca Renaud) Date: Mon, 18 May 2015 03:55:38 +0100 Subject: [Bro] Bro documentation in pdf format. Message-ID: Is it possible to generate/convert the online bro user manual (sphinx platform) in pdf format?Eventually,for us to have a printed version. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/c098f17f/attachment.html From a.benkhadra at hotmail.com Mon May 18 05:24:45 2015 From: a.benkhadra at hotmail.com (Abdelkarim benkhadra) Date: Mon, 18 May 2015 13:24:45 +0100 Subject: [Bro] Assertion 'field' failed. Message-ID: Hi, i'm writing an analyzer for a protocol, i wrote all files needed and when 'make'ing bro i get an error of: binpac: ..../bro-2.4-beta/aux/binpac/src/pac_expr.cc:225: RecordField* {anonymous}::GetRecordField(const ID*, Env*): Assertion 'field' failed. i assume the error comes from .pac files so what could possibly generate this kind of error? regards, -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/2c7d409b/attachment.html From vlad at grigorescu.org Mon May 18 05:51:37 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Mon, 18 May 2015 07:51:37 -0500 Subject: [Bro] Assertion 'field' failed. In-Reply-To: References: Message-ID: Hi, That's a rather vague error, unfortunately. You can try using gdb to debug and pinpoint which record and field this is referring to. Do you have the code available for review somewhere, or could you share the snippet that you suspect is the issue? --Vlad -------------- next part -------------- Abdelkarim benkhadra writes: > Hi, > > i'm writing an analyzer for a protocol, i wrote all files needed and when 'make'ing bro i get an error of: > binpac: ..../bro-2.4-beta/aux/binpac/src/pac_expr.cc:225: RecordField* {anonymous}::GetRecordField(const ID*, Env*): Assertion 'field' failed. > i assume the error comes from .pac files so what could possibly generate this kind of error? > > regards, > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 800 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/0d44e3c1/attachment.bin From monahbaki at gmail.com Mon May 18 06:02:10 2015 From: monahbaki at gmail.com (Monah Baki) Date: Mon, 18 May 2015 09:02:10 -0400 Subject: [Bro] No http.log and dns.log missing Message-ID: Bro seems to have all the logs except http and dns. If I run a quick tcpdump on my interface for port 80 and 53, I do see event. Anything else I can troubleshoot for? Thanks Monah -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/9056b108/attachment.html From cbakkers at yahoo.de Mon May 18 06:21:11 2015 From: cbakkers at yahoo.de (Yahoo) Date: Mon, 18 May 2015 15:21:11 +0200 Subject: [Bro] No http.log and dns.log missing In-Reply-To: References: Message-ID: <1A20BB80-9F79-4FA0-9C36-CA35E73A9188@yahoo.de> have you checked if your interfaces are running in promiscuous mode? > On 18 May 2015, at 15:02, Monah Baki wrote: > > Bro seems to have all the logs except http and dns. If I run a quick tcpdump on my interface for port 80 and 53, I do see event. > > Anything else I can troubleshoot for? > > > Thanks > Monah > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From monahbaki at gmail.com Mon May 18 06:31:23 2015 From: monahbaki at gmail.com (Monah Baki) Date: Mon, 18 May 2015 09:31:23 -0400 Subject: [Bro] No http.log and dns.log missing In-Reply-To: <1A20BB80-9F79-4FA0-9C36-CA35E73A9188@yahoo.de> References: <1A20BB80-9F79-4FA0-9C36-CA35E73A9188@yahoo.de> Message-ID: netstat -i Kernel Interface table Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg eth0 1500 0 463397 0 10 0 521906 0 0 0 BMRU eth1 1500 0 299482016 0 28 0 8 0 0 0 BMRU eth1 is my listening interface Thanks On Mon, May 18, 2015 at 9:21 AM, Yahoo wrote: > have you checked if your interfaces are running in promiscuous mode? > > > > > On 18 May 2015, at 15:02, Monah Baki wrote: > > > > Bro seems to have all the logs except http and dns. If I run a quick > tcpdump on my interface for port 80 and 53, I do see event. > > > > Anything else I can troubleshoot for? > > > > > > Thanks > > Monah > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/dda4d15c/attachment.html From john at omernik.com Mon May 18 06:37:04 2015 From: john at omernik.com (John Omernik) Date: Mon, 18 May 2015 08:37:04 -0500 Subject: [Bro] Bro Types Not Following Bro Types Documention In-Reply-To: References: Message-ID: (Mike I am putting this on list, I replied only to you) I found https://www.bro.org/sphinx-git/logs/index.html which is helpful in that - represents an unset field. I am still think from a data nerd perspective, having a character that doesn't fit the type to represent something is dangerous, however, I can parse the values and replace programmatically with the information provided, so now I an continue on my merry way. Thanks for the insight. That all said... why put anything in a field (as a default) to represent unset or empty? Are we at risk of evasion? Besides obviously breaking typing, what about when the type actually accepts the unset character... what if the user-agent is - or (empty) couldn't that cause downstream errors? "You can change the logs to log however you want" is likely the answer, and correct I can, but shouldn't we try be logical in our approach so assumptions aren't made on the default material? On Sat, May 16, 2015 at 4:44 PM, Mike Dopheide wrote: > John, > > The "-" you're seeing in this case isn't meant to be representing a > count. The "-" is used in Bro logs to represent a field with missing > data. For whatever reason Bro couldn't the bytes sent in this case. > You'll see -'s more commonly in other logs, like http.log, ssl.log, etc. > > -Dop > > > > On Sat, May 16, 2015 at 3:57 PM, John Omernik wrote: > >> I am working doing some automation with Bro, Avro, Kafka and I am a >> little bit frustrated. (Or I am looking at the wrong documentation, hence >> my post here, I am very good with being extremely wrong because I am >> looking at something wrong). >> >> Specifically I am looking at the default conn.log. The Type that is >> specified for some fields such as >> orig_bytes or resp_bytes is type count >> >> Based on the docs I am using here: >> >> https://www.bro.org/sphinx/script-reference/builtins.html >> >> a count is: >> >> count >> >> A numeric type representing a 64-bit unsigned integer. A count constant >> is a string of digits, e.g.1234 or 0. A count can also be written in >> hexadecimal notation (in which case ?0x? must precede the hex digits), e.g. >> 0xff or 0xABC123. >> >> The count type supports the same operators as the int >> type. >> A unary plus or minus applied to acount results in an int. >> >> >> This is well and good, however looking at some of the data in my log I >> see the character "-" as a value. Based on my reading of a count, that >> shouldn't exist, a - is not a unsigned integer, nor is it a string of >> digits whether in base 10 or hexidecimal. >> >> >> Thus my frustration, I'd like to develop some generic bindings to push >> bro logs into Avro Serialized Kafka messages, but looking at this, I can't >> even trust the documentation to be accurate? Am I missing something? Is >> there another documentation reference that more fully represents the data >> types that would explain why - is a valid integer? >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/e9b45dce/attachment-0001.html From a.benkhadra at hotmail.com Mon May 18 06:37:55 2015 From: a.benkhadra at hotmail.com (Abdelkarim benkhadra) Date: Mon, 18 May 2015 14:37:55 +0100 Subject: [Bro] Assertion 'field' failed. In-Reply-To: References: , Message-ID: i think i found the responsible, i have a record : type myrec = record { //some fields. last_field: bytestring &restofdata; } &length = sizeof(this); with this length specification i get the RecordField* error and when trying &length=20+sizeof(last_field) i get "cannot determine initial buffer length for type myrec" and without &length it passes. so far im not going to specify the &length value and let you know if something comes up. thanks, From: vlad at grigorescu.org To: a.benkhadra at hotmail.com; bro at bro.org Subject: Re: [Bro] Assertion 'field' failed. Date: Mon, 18 May 2015 07:51:37 -0500 Hi, That's a rather vague error, unfortunately. You can try using gdb to debug and pinpoint which record and field this is referring to. Do you have the code available for review somewhere, or could you share the snippet that you suspect is the issue? --Vlad Abdelkarim benkhadra writes: > Hi, > > i'm writing an analyzer for a protocol, i wrote all files needed and when 'make'ing bro i get an error of: > binpac: ..../bro-2.4-beta/aux/binpac/src/pac_expr.cc:225: RecordField* {anonymous}::GetRecordField(const ID*, Env*): Assertion 'field' failed. > i assume the error comes from .pac files so what could possibly generate this kind of error? > > regards, > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/3133bf16/attachment.html From monahbaki at gmail.com Mon May 18 06:46:26 2015 From: monahbaki at gmail.com (Monah Baki) Date: Mon, 18 May 2015 09:46:26 -0400 Subject: [Bro] No http.log and dns.log missing In-Reply-To: References: <1A20BB80-9F79-4FA0-9C36-CA35E73A9188@yahoo.de> Message-ID: Yes it is. eth1 Link encap:Ethernet HWaddr 00:50:56:8b:0f:0a inet6 addr: fe80::250:56ff:fe8b:f0a/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:300657037 errors:0 dropped:28 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:193217271566 (193.2 GB) TX bytes:648 (648.0 B) On Mon, May 18, 2015 at 9:38 AM, Yahoo wrote: > is it set to promiscioud mode? > > > > On 18 May 2015, at 15:31, Monah Baki wrote: > > netstat -i > > Kernel Interface table > Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR > Flg > eth0 1500 0 463397 0 10 0 521906 0 > 0 0 BMRU > eth1 1500 0 299482016 0 28 0 8 0 > 0 0 BMRU > > eth1 is my listening interface > > > Thanks > > On Mon, May 18, 2015 at 9:21 AM, Yahoo wrote: > >> have you checked if your interfaces are running in promiscuous mode? >> >> >> >> > On 18 May 2015, at 15:02, Monah Baki wrote: >> > >> > Bro seems to have all the logs except http and dns. If I run a quick >> tcpdump on my interface for port 80 and 53, I do see event. >> > >> > Anything else I can troubleshoot for? >> > >> > >> > Thanks >> > Monah >> > _______________________________________________ >> > Bro mailing list >> > bro at bro-ids.org >> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/5bedd48f/attachment.html From jbarber at computer.org Mon May 18 07:27:10 2015 From: jbarber at computer.org (Jeff Barber) Date: Mon, 18 May 2015 10:27:10 -0400 Subject: [Bro] pybroker with 'optional' fields Message-ID: I've been playing with sending event data to a peer using the broker interface. I'm able to send records over just fine and my python script can receive and interpret them using the swig-generated wrapper as long as all the fields are present. If I try to send one with optional fields such as pkt_hdr where not all of the fields are present (as is always the case with pkt_hdr), I get various segmentation violations either direct in the swig-generated code or assertion failures in the 'optional' class. Seems like there should be a more intelligent iterator for the record fields in the swig source. I was thinking it would make sense to return a None value in the slot where a non-present optional value goes and then you could just test for that, but I don't know enough about swig to create the iterator. I've tried several combinations of %extend, %pythoncode and so forth, but can't figure out the right magic words. Anybody know the right way to do this? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/bc73a333/attachment.html From johanna at icir.org Mon May 18 07:43:59 2015 From: johanna at icir.org (Johanna Amann) Date: Mon, 18 May 2015 07:43:59 -0700 Subject: [Bro] Bro Types Not Following Bro Types Documention In-Reply-To: References: Message-ID: <20150518144359.GA15642@Beezling.local> On Mon, May 18, 2015 at 08:37:04AM -0500, John Omernik wrote: [...] > That all said... why put anything in a field (as a default) to represent > unset or empty? Are we at risk of evasion? I am not sure what you would should do instead. From a protocol point of view, there is often a huge difference between "an empty string was transferred" and "this was not seen at all". For example, in HTTP a Referrer of "-" means that no referrer header was set at all. "" (the empty string) instead means that it was seen, but empty. Same for sets, there is a difference between the set was not seen at all ("-"), the set was seen but empt ("(empty)") and the set was seen and contains one element with an empty string (""). > Besides obviously breaking typing, what about when the type actually > accepts the unset character... what if the user-agent is - or (empty) > couldn't that cause downstream errors? In that case, the character should be replaced by the escaped version of it (i.e. you should find \x[ascii-code] or similar) in the log-file instead of the -. Hence, it should still be decideable which of the two cases happened. > "You can change the logs to log however you want" is likely the > answer, and correct I can, but shouldn't we try be logical in our approach > so assumptions aren't made on the default material? I hope this helps, Johanna From dopheide at gmail.com Mon May 18 08:00:02 2015 From: dopheide at gmail.com (Mike Dopheide) Date: Mon, 18 May 2015 10:00:02 -0500 Subject: [Bro] Bro Types Not Following Bro Types Documention In-Reply-To: <20150518144359.GA15642@Beezling.local> References: <20150518144359.GA15642@Beezling.local> Message-ID: I'll just add one high level point. It's important to remember that, for a lot of people, the logs are the final output. They must be human readable and easily processed with simple unix command line tools. -Dop On Mon, May 18, 2015 at 9:43 AM, Johanna Amann wrote: > On Mon, May 18, 2015 at 08:37:04AM -0500, John Omernik wrote: > [...] > > That all said... why put anything in a field (as a default) to represent > > unset or empty? Are we at risk of evasion? > > I am not sure what you would should do instead. From a protocol point of > view, there is often a huge difference between "an empty string was > transferred" and "this was not seen at all". For example, in HTTP a > Referrer of "-" means that no referrer header was set at all. "" (the > empty string) instead means that it was seen, but empty. Same for sets, > there is a difference between the set was not seen at all ("-"), the set > was seen but empt ("(empty)") and the set was seen and contains one > element with an empty string (""). > > > Besides obviously breaking typing, what about when the type actually > > accepts the unset character... what if the user-agent is - or (empty) > > couldn't that cause downstream errors? > > In that case, the character should be replaced by the escaped version of > it (i.e. you should find \x[ascii-code] or similar) in the log-file > instead of the -. Hence, it should still be decideable which of the two > cases happened. > > > "You can change the logs to log however you want" is likely the > > answer, and correct I can, but shouldn't we try be logical in our > approach > > so assumptions aren't made on the default material? > > I hope this helps, > Johanna > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/56084049/attachment-0001.html From vlad at grigorescu.org Mon May 18 07:56:39 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Mon, 18 May 2015 09:56:39 -0500 Subject: [Bro] Assertion 'field' failed. In-Reply-To: References: Message-ID: Yeah, that &length looks suspicious to me. &length=sizeof(this) is essentially saying "set the length of the record to the size of the record." I'd try it without the length specified at all, like you said. You might want to review this section of the documentation: https://www.bro.org/sphinx/components/binpac/README.html#buffering --Vlad -------------- next part -------------- Abdelkarim benkhadra writes: > i think i found the responsible, > > i have a record : > > type myrec = record { > > //some fields. > last_field: bytestring &restofdata; > } &length = sizeof(this); > > with this length specification i get the RecordField* error and when trying &length=20+sizeof(last_field) i get "cannot determine initial buffer length for type myrec" and without &length it passes. > so far im not going to specify the &length value and let you know if something comes up. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 800 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/fb6afe48/attachment.bin From vlad at grigorescu.org Mon May 18 08:13:07 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Mon, 18 May 2015 10:13:07 -0500 Subject: [Bro] Bro Types Not Following Bro Types Documention In-Reply-To: <20150518144359.GA15642@Beezling.local> References: <20150518144359.GA15642@Beezling.local> Message-ID: I think that another important point is that this is something that's occuring in the ASCII writer (a bit of a misnomer, really a tab-separated value writer). The fact that Bro has the concept of optional fields means that there's a difference between a field that was set to a non-empty string, a field that was set to an empty string, and a field that was never set. Other output formats (e.g. JSON) have a better way of differentiating between these, but this was the solution developed for TSV output. As with much of Bro, you can redef what exactly is written out in these cases (see: https://www.bro.org/sphinx-git/scripts/base/frameworks/logging/writers/ascii.bro.html#id-LogAscii::empty_field and https://www.bro.org/sphinx-git/scripts/base/frameworks/logging/writers/ascii.bro.html#id-LogAscii::unset_field). As Johanna mentioned, there should always be a 1:1 mapping between the log and the record that's being logged. If you're seeing ambiguity, that's something that we should fix. Fundamentally, a Bro log line should be as clear as possible. --Vlad -------------- next part -------------- Johanna Amann writes: > On Mon, May 18, 2015 at 08:37:04AM -0500, John Omernik wrote: > [...] >> That all said... why put anything in a field (as a default) to represent >> unset or empty? Are we at risk of evasion? > > I am not sure what you would should do instead. From a protocol point of > view, there is often a huge difference between "an empty string was > transferred" and "this was not seen at all". For example, in HTTP a > Referrer of "-" means that no referrer header was set at all. "" (the > empty string) instead means that it was seen, but empty. Same for sets, > there is a difference between the set was not seen at all ("-"), the set > was seen but empt ("(empty)") and the set was seen and contains one > element with an empty string (""). > >> Besides obviously breaking typing, what about when the type actually >> accepts the unset character... what if the user-agent is - or (empty) >> couldn't that cause downstream errors? > > In that case, the character should be replaced by the escaped version of > it (i.e. you should find \x[ascii-code] or similar) in the log-file > instead of the -. Hence, it should still be decideable which of the two > cases happened. > >> "You can change the logs to log however you want" is likely the >> answer, and correct I can, but shouldn't we try be logical in our approach >> so assumptions aren't made on the default material? > > I hope this helps, > Johanna > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 800 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/6681bea8/attachment.bin From john at omernik.com Mon May 18 09:35:15 2015 From: john at omernik.com (John Omernik) Date: Mon, 18 May 2015 11:35:15 -0500 Subject: [Bro] Bro Types Not Following Bro Types Documention In-Reply-To: <20150518144359.GA15642@Beezling.local> References: <20150518144359.GA15642@Beezling.local> Message-ID: Solid point on the difference. Thanks for clarifying. This is a tough problem. One of our systems, has a concept of null vs. empty strings in a final storage, but as pointed out, that makes things difficult from a human readable aspect. (What I mean there is if the referer doesn't exist, the field is NULL, if it does and is empty it's a "") I wonder if like (empty) unset may also be "more" verbose. I know that seems counter intuitive, especially to my point on types, but - may be error prone, especially on string fields (it's more obviously on non-string fields), but what if we went with (unset) instead? at least in that case, if it gets down stream to someone who isn't clear on how Bro is doing things, there is more of a chance that they will understand that it didn't exist vs. just assuming - is the value that was passed. Issues here are obviously backwards compatibility and creating larger log files. On Mon, May 18, 2015 at 9:43 AM, Johanna Amann wrote: > On Mon, May 18, 2015 at 08:37:04AM -0500, John Omernik wrote: > [...] > > That all said... why put anything in a field (as a default) to represent > > unset or empty? Are we at risk of evasion? > > I am not sure what you would should do instead. From a protocol point of > view, there is often a huge difference between "an empty string was > transferred" and "this was not seen at all". For example, in HTTP a > Referrer of "-" means that no referrer header was set at all. "" (the > empty string) instead means that it was seen, but empty. Same for sets, > there is a difference between the set was not seen at all ("-"), the set > was seen but empt ("(empty)") and the set was seen and contains one > element with an empty string (""). > > > Besides obviously breaking typing, what about when the type actually > > accepts the unset character... what if the user-agent is - or (empty) > > couldn't that cause downstream errors? > > In that case, the character should be replaced by the escaped version of > it (i.e. you should find \x[ascii-code] or similar) in the log-file > instead of the -. Hence, it should still be decideable which of the two > cases happened. > > > "You can change the logs to log however you want" is likely the > > answer, and correct I can, but shouldn't we try be logical in our > approach > > so assumptions aren't made on the default material? > > I hope this helps, > Johanna > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/94cb0a30/attachment.html From doris at bro.org Mon May 18 10:06:19 2015 From: doris at bro.org (Doris Schioberg) Date: Mon, 18 May 2015 10:06:19 -0700 Subject: [Bro] Newsletter #5 is ready Message-ID: <555A1C0B.2030603@bro.org> We are happy to announce the 5th issue of the Bro Newsletter. http://blog.bro.org/2015/05/bro-monthly-5.html -- Doris Schioberg Bro Outreach, Training, and Education Coordinator International Computer Science Institute (ICSI Berkeley) Phone: +1 (510) 289-8406 * doris at bro.org From jsiwek at illinois.edu Mon May 18 10:56:06 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Mon, 18 May 2015 17:56:06 +0000 Subject: [Bro] pybroker with 'optional' fields In-Reply-To: References: Message-ID: <62381BC9-BA9E-42AD-BAB3-BC14AAC0FC44@illinois.edu> > On May 18, 2015, at 9:27 AM, Jeff Barber wrote: > > I've been playing with sending event data to a peer using the broker interface. I'm able to send records over just fine and my python script can receive and interpret them using the swig-generated wrapper as long as all the fields are present. > > If I try to send one with optional fields such as pkt_hdr where not all of the fields are present (as is always the case with pkt_hdr), I get various segmentation violations either direct in the swig-generated code or assertion failures in the 'optional' class. > > Seems like there should be a more intelligent iterator for the record fields in the swig source. I was thinking it would make sense to return a None value in the slot where a non-present optional value goes and then you could just test for that, but I don't know enough about swig to create the iterator. I've tried several combinations of %extend, %pythoncode and so forth, but can't figure out the right magic words. > > Anybody know the right way to do this? There?s a brief example of sending/receiving a record with an empty field in tests/test_messages.py. You can call the valid() method on a field to test if there?s data there that you?re allowed to access. If that doesn?t help clarify the issue, can you post some example code? - Jon From seth at icir.org Mon May 18 11:24:21 2015 From: seth at icir.org (Seth Hall) Date: Mon, 18 May 2015 14:24:21 -0400 Subject: [Bro] Bro Digest, Vol 109, Issue 14 In-Reply-To: References: Message-ID: This problem isn?t a huge surprise to me. We haven?t supported the packet-writing feature for several releases (it?s also not explicitly deprecated, we just haven?t given it any attention). At the very least, it isn?t something that we have tests for due to it being complicated and unreliable in some circumstances. .Seth > On May 12, 2015, at 12:51 PM, Jason wrote: > > > Date: Tue, 12 May 2015 10:04:56 -0600 > From: James Lay > Subject: Re: [Bro] PPPoE Capture IP Layer Being Stripped > To: bro at bro.org > Message-ID: > Content-Type: text/plain; charset=US-ASCII; format=flowed > > On 2015-05-12 07:43 AM, Jason wrote: > > Good day all, > > > > One of my sites has all PPPoE traffic on the link I'm monitoring. The > > .log files are all generated correctly, but PCAP files end up with > > stripped IP layer information. This was easy to reproduce in bro > > 2.3.1 on Ubuntu by doing: > > > > tcpdump -nn -i ethX -w test.pcap > > bro -r test.pcap -w bro.pcap > > > > The tcpdump traffic in test.pcap looks fine, but the bro pcap comes up > > as Ethernet traffic with an unknown type. > > > > Is this a known bug? Or is there perhaps some configuration that > > needs to be changed in bro support this traffic? > > > > Thanks in advance, > > > > Jason > > > > I run bro on ppp0, but I don't think I've seen this issue. Have you > tried having bro listen on the physical interface instead? > > James > > > ------------------------------ > > I have indeed. Live capture was where the problem was first noticed. I moved to an offline/tcpdump test as part of my troubleshooting to ensure nothing else was causing problems (link issues, PF_RING, etc). > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/3ae8ec41/attachment-0001.bin From jbarber at computer.org Mon May 18 11:50:00 2015 From: jbarber at computer.org (Jeff Barber) Date: Mon, 18 May 2015 14:50:00 -0400 Subject: [Bro] pybroker with 'optional' fields In-Reply-To: <62381BC9-BA9E-42AD-BAB3-BC14AAC0FC44@illinois.edu> References: <62381BC9-BA9E-42AD-BAB3-BC14AAC0FC44@illinois.edu> Message-ID: Yeah. Using valid() works but it makes the code clumsy and unpythonic. Below is my code. Uncomment the second "sub_recs = ..." line in ppkt() (in place of the first one) to see the issue. With this .bro loaded: const broker_port: port = 9999/tcp &redef; redef BrokerComm::endpoint_name = "events"; export { global jb_packet: event(p: pkt_hdr); } event bro_init() { BrokerComm::enable(); BrokerComm::listen(broker_port, "127.0.0.1"); BrokerComm::auto_event("bro/event/jb_packet", jb_packet); } event new_packet(c:connection, p: pkt_hdr) { event jb_packet(p); } I have this script: #!/usr/bin/env python from select import select import pybroker def get_fields(fields, n_fields): new_fields = [] for n in range(n_fields): f = fields[n] if f.valid(): new_fields.append(f.get()) else: new_fields.append(None) return new_fields def ppkt(p): rec = p.as_record() sub_recs = get_fields(rec.fields(), rec.size()) #===>>> sub_recs = [f.get() for f in fields] print sub_recs def pmsg(msg_type, obj): msg_type = msg_type.as_string() pobj = { "jb_packet": ppkt, }[msg_type] # print "%s: " % msg_type, pobj(obj) def main(): epc = pybroker.endpoint("connector") epc.peer("127.0.0.1", 9999, 1) ocsq = epc.outgoing_connection_status() select([ocsq.fd()], [], []) conns = ocsq.want_pop() for m in conns: print("outgoing connection", m.peer_name, m.status) mql = pybroker.message_queue("bro/event", epc) while True: select([mql.fd()], [], []) msgs = mql.want_pop() for m in msgs: pmsg(*m) main() On Mon, May 18, 2015 at 1:56 PM, Siwek, Jon wrote: > > > On May 18, 2015, at 9:27 AM, Jeff Barber wrote: > > > > I've been playing with sending event data to a peer using the broker > interface. I'm able to send records over just fine and my python script can > receive and interpret them using the swig-generated wrapper as long as all > the fields are present. > > > > If I try to send one with optional fields such as pkt_hdr where not all > of the fields are present (as is always the case with pkt_hdr), I get > various segmentation violations either direct in the swig-generated code or > assertion failures in the 'optional' class. > > > > Seems like there should be a more intelligent iterator for the record > fields in the swig source. I was thinking it would make sense to return a > None value in the slot where a non-present optional value goes and then you > could just test for that, but I don't know enough about swig to create the > iterator. I've tried several combinations of %extend, %pythoncode and so > forth, but can't figure out the right magic words. > > > > Anybody know the right way to do this? > > There?s a brief example of sending/receiving a record with an empty field > in tests/test_messages.py. You can call the valid() method on a field to > test if there?s data there that you?re allowed to access. If that doesn?t > help clarify the issue, can you post some example code? > > - Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/cf817763/attachment.html From dn1nj4 at gmail.com Tue May 19 04:24:39 2015 From: dn1nj4 at gmail.com (Jason) Date: Tue, 19 May 2015 07:24:39 -0400 Subject: [Bro] PPPoE Capture IP Layer Being Stripped Message-ID: Hi Seth, Per James' suggestion I filed bug report BIT-1398. Does "haven't supported" translate to "will not support"? It sounds as though there are some people like me who are actually leveraging this functionality. Thanks in advance, Jason On Mon, May 18, 2015 at 2:24 PM, Seth Hall wrote: > This problem isn?t a huge surprise to me. We haven?t supported the > packet-writing feature for several releases (it?s also not explicitly > deprecated, we just haven?t given it any attention). At the very least, it > isn?t something that we have tests for due to it being complicated and > unreliable in some circumstances. > > .Seth > > > On May 12, 2015, at 12:51 PM, Jason wrote: > > > > > > Date: Tue, 12 May 2015 10:04:56 -0600 > > From: James Lay > > Subject: Re: [Bro] PPPoE Capture IP Layer Being Stripped > > To: bro at bro.org > > Message-ID: > > Content-Type: text/plain; charset=US-ASCII; format=flowed > > > > On 2015-05-12 07:43 AM, Jason wrote: > > > Good day all, > > > > > > One of my sites has all PPPoE traffic on the link I'm monitoring. The > > > .log files are all generated correctly, but PCAP files end up with > > > stripped IP layer information. This was easy to reproduce in bro > > > 2.3.1 on Ubuntu by doing: > > > > > > tcpdump -nn -i ethX -w test.pcap > > > bro -r test.pcap -w bro.pcap > > > > > > The tcpdump traffic in test.pcap looks fine, but the bro pcap comes up > > > as Ethernet traffic with an unknown type. > > > > > > Is this a known bug? Or is there perhaps some configuration that > > > needs to be changed in bro support this traffic? > > > > > > Thanks in advance, > > > > > > Jason > > > > > > > I run bro on ppp0, but I don't think I've seen this issue. Have you > > tried having bro listen on the physical interface instead? > > > > James > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150519/f4f32ccf/attachment.html From anshu.sh123 at gmail.com Tue May 19 08:23:23 2015 From: anshu.sh123 at gmail.com (Anshu Sharma) Date: Tue, 19 May 2015 20:53:23 +0530 Subject: [Bro] About BRO Message-ID: Sir i m working on bro 2.3.2 version i want to know can we detect denial of service attack using bro? if possible can you please provide me some guidance . thanks hoping your early reply -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150519/54c56ef6/attachment.html From jsiwek at illinois.edu Tue May 19 08:45:53 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Tue, 19 May 2015 15:45:53 +0000 Subject: [Bro] pybroker with 'optional' fields In-Reply-To: References: <62381BC9-BA9E-42AD-BAB3-BC14AAC0FC44@illinois.edu> Message-ID: <8BE3E19A-AE78-4759-BE56-9EF828FAB88C@illinois.edu> > On May 18, 2015, at 1:50 PM, Jeff Barber wrote: > > Yeah. Using valid() works but it makes the code clumsy and unpythonic. Below is my code. Uncomment the second "sub_recs = ..." line in ppkt() (in place of the first one) to see the issue. > > def get_fields(fields, n_fields): > new_fields = [] > for n in range(n_fields): > f = fields[n] > if f.valid(): > new_fields.append(f.get()) > else: > new_fields.append(None) > return new_fields > > def ppkt(p): > rec = p.as_record() > sub_recs = get_fields(rec.fields(), rec.size()) > #===>>> sub_recs = [f.get() for f in fields] > print sub_recs Does something like the following work to transform it into the format you want? sub_recs = [f.get() if f.valid() else None for f in fields] - Jon From anthony.kasza at gmail.com Tue May 19 08:57:51 2015 From: anthony.kasza at gmail.com (anthony kasza) Date: Tue, 19 May 2015 08:57:51 -0700 Subject: [Bro] About BRO In-Reply-To: References: Message-ID: Detecting denial of service attacks aren't as clear cut as detecting other attacks e.g. SQL injection. What constitutes a DoS depends on your networks specifics, such as bandwidth. A DoS to your network might not be a DoS to a larger network. This being said, Bro does have the ability to detect common port scan attacks. I believe the detection scripts are built on the sumstats framework. Here's one Seth wrote < https://github.com/sethhall/bro-junk-drawer/blob/master/scan_udp.bro>. I hope that helps. -AK On May 19, 2015 8:44 AM, "Anshu Sharma" wrote: > Sir > i m working on bro 2.3.2 version > i want to know can we detect denial of service attack using bro? > if possible can you please provide me some guidance . > thanks > hoping your early reply > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150519/75d10e34/attachment.html From jbarber at computer.org Tue May 19 10:54:10 2015 From: jbarber at computer.org (Jeff Barber) Date: Tue, 19 May 2015 13:54:10 -0400 Subject: [Bro] pybroker with 'optional' fields In-Reply-To: <8BE3E19A-AE78-4759-BE56-9EF828FAB88C@illinois.edu> References: <62381BC9-BA9E-42AD-BAB3-BC14AAC0FC44@illinois.edu> <8BE3E19A-AE78-4759-BE56-9EF828FAB88C@illinois.edu> Message-ID: Jon, That does seem to work at the top-most level (i.e. pkt_hdr). But then in trying to break apart the next layer (ip_hdr), it doesn't. I get a seg fault in the "vector_of_field" iterator. With: rec = p.as_record() fields = rec.fields() ip, ip6, tcp, udp, icmp = [f.get() if f.valid() else None for f in fields] if ip is not None: fields = ip.as_record().fields() fields = [f for f in fields] This happens in the last line: Program received signal SIGSEGV, Segmentation fault. 0xb76a1386 in broker::util::optional::optional (this=0x846f688, other=...) at /home/jbarber/src/bro/aux/broker/broker/util/optional.hh:84 84 { if ( other.is_valid ) create(other.value); } (gdb) bt #0 0xb76a1386 in broker::util::optional::optional (this=0x846f688, other=...) at /home/jbarber/src/bro/aux/broker/broker/util/optional.hh:84 #1 0xb7704471 in swig::traits_from >::from (val=...) at /home/jbarber/src/bro/build/aux/broker/bindings/python/pybrokerPYTHON_wrap.cxx:3909 #2 0xb770419a in swig::from > (val=...) at /home/jbarber/src/bro/build/aux/broker/bindings/python/pybrokerPYTHON_wrap.cxx:3928 #3 0xb7703cc1 in swig::from_oper >::operator() (this=0x84345a4, v=...) at /home/jbarber/src/bro/build/aux/broker/bindings/python/pybrokerPYTHON_wrap.cxx:4481 #4 0xb77030ec in swig::SwigPyIteratorClosed_T<__gnu_cxx::__normal_iterator*, std::vector, std::allocator > > >, broker::util::optional, swig::from_oper > >::value ( this=0x8434598) at /home/jbarber/src/bro/build/aux/broker/bindings/python/pybrokerPYTHON_wrap.cxx:4549 #5 0xb769f12c in swig::SwigPyIterator::next (this=0x8434598) at /home/jbarber/src/bro/build/aux/broker/bindings/python/pybrokerPYTHON_wrap.cxx:3243 #6 0xb76358b2 in _wrap_SwigPyIterator_next (args=0xb77af38c) at /home/jbarber/src/bro/build/aux/broker/bindings/python/pybrokerPYTHON_wrap.cxx:7044 #7 0x08156a91 in PyEval_EvalFrameEx () ... (gdb) print other $1 = (const broker::util::optional &) @0x0: With the maze of templates plus the swig stuff, it's difficult to follow where things went south. Interestingly, if I stop the script in the python debugger, just before that last line and print some values: (Pdb) fields.size() 2577038276L (Pdb) len(fields) *** OverflowError: long int too large to convert to int (Both of those statements return sane results (5) at the top-level record.) On Tue, May 19, 2015 at 11:45 AM, Siwek, Jon wrote: > > > On May 18, 2015, at 1:50 PM, Jeff Barber wrote: > > > > Yeah. Using valid() works but it makes the code clumsy and unpythonic. > Below is my code. Uncomment the second "sub_recs = ..." line in ppkt() (in > place of the first one) to see the issue. > > > > def get_fields(fields, n_fields): > > new_fields = [] > > for n in range(n_fields): > > f = fields[n] > > if f.valid(): > > new_fields.append(f.get()) > > else: > > new_fields.append(None) > > return new_fields > > > > def ppkt(p): > > rec = p.as_record() > > sub_recs = get_fields(rec.fields(), rec.size()) > > #===>>> sub_recs = [f.get() for f in fields] > > print sub_recs > > Does something like the following work to transform it into the format you > want? > > sub_recs = [f.get() if f.valid() else None for f in fields] > > - Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150519/c122a4b9/attachment-0001.html From seth at icir.org Tue May 19 11:03:50 2015 From: seth at icir.org (Seth Hall) Date: Tue, 19 May 2015 14:03:50 -0400 Subject: [Bro] PPPoE Capture IP Layer Being Stripped In-Reply-To: References: Message-ID: <37AD978D-8C50-400F-905D-C4348042BA1B@icir.org> > On May 19, 2015, at 7:24 AM, Jason wrote: > > Per James' suggestion I filed bug report BIT-1398. Does "haven't supported" translate to "will not support"? It sounds as though there are some people like me who are actually leveraging this functionality. Hah, I wouldn?t say that it translates to ?will not support?. It just hasn?t risen to the level of anyone taking on testing and improving the functionality because I don?t believe it?s used much among the other developers. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150519/89f88e3b/attachment.bin From jbarber at computer.org Tue May 19 13:22:14 2015 From: jbarber at computer.org (Jeff Barber) Date: Tue, 19 May 2015 16:22:14 -0400 Subject: [Bro] packet post-processor plugin Message-ID: Still a bro newbie, so I'm looking for some guidance. I'd like to add a packet post-processor to bro. It'll be written in C++. Essentially I'd like to see every packet that goes through bro, along with its Connection record (or the equivalent) if there is one. Ideally it would be structured it as a plugin. It looks like I could do it by triggering off of events like new_packet (although currently that only triggers for IP packets). However, IIUC, that still has bro packaging all the info up into a RecordVal, then I have to decode it in my plugin event handler. That seems like quite a bit of additional overhead on each packet. It looks like I might also be able to do it as a PktDumper but then I just get the raw packet data and I'd have to go re-parse headers and re-lookup connection info - redoing work that's already been done. What I'd really like is to simply get a call at the end of analysis for every packet, where I'd get passed a pointer to the packet data along with a pointer to the existing Connection record (if any). Maybe there's some obvious way to do this that I've overlooked? Anybody have advice for the best way to go? I'm willing to do work to make this happen, but also would prefer not to fork bro so looking for "right ways". TIA, Jeff -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150519/14df2539/attachment.html From asharma at lbl.gov Tue May 19 13:58:55 2015 From: asharma at lbl.gov (Aashish Sharma) Date: Tue, 19 May 2015 13:58:55 -0700 Subject: [Bro] packet post-processor plugin In-Reply-To: References: Message-ID: <20150519205853.GR16953@yaksha.lbl.gov> Why do you want to do this ? what is that your packet port-processor going to do that you think bro isn't doing for you? > Maybe there's some obvious way to do this that I've overlooked? Yes, use scripting layer for your analysis. If you think a particular protocol parsing is deficient, instead of writing a packet post-processor, might as well write your new protocol parser. Aashish On Tue, May 19, 2015 at 04:22:14PM -0400, Jeff Barber wrote: > > Still a bro newbie, so I'm looking for some guidance. > I'd like to add a packet post-processor to bro. It'll be written in C++. > Essentially I'd like to see every packet that goes through bro, along with > its Connection record (or the equivalent) if there is one. Ideally it would > be structured it as a plugin. > It looks like I could do it by triggering off of events like new_packet > (although currently that only triggers for IP packets). However, IIUC, that > still has bro packaging all the info up into a RecordVal, then I have to > decode it in my plugin event handler. That seems like quite a bit of > additional overhead on each packet. > It looks like I might also be able to do it as a PktDumper but then I just > get the raw packet data and I'd have to go re-parse headers and re-lookup > connection info - redoing work that's already been done. > What I'd really like is to simply get a call at the end of analysis for > every packet, where I'd get passed a pointer to the packet data along with a > pointer to the existing Connection record (if any). Maybe there's some > obvious way to do this that I've overlooked? > Anybody have advice for the best way to go? I'm willing to do work to make > this happen, but also would prefer not to fork bro so looking for "right > ways". > TIA, > Jeff > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Lawrence Berkeley National Laboratory http://go.lbl.gov/pgp-aashish Office: (510)-495-2680 Cell: (510)-612-7971 From life.130815 at gmail.com Tue May 19 23:51:34 2015 From: life.130815 at gmail.com (Mo Jia) Date: Wed, 20 May 2015 14:51:34 +0800 Subject: [Bro] elasticsearch plugin identify the proto? Message-ID: In elasticsearch.cc bool ElasticSearch::DoWrite(int num_fields, const Field* const * fields, Value** vals) We can get the contents, how can I identify the proto of the content? In local logging, it will write to http.log, But in elasticsearch it lost the proto message. From jbarber at computer.org Wed May 20 04:30:34 2015 From: jbarber at computer.org (Jeff Barber) Date: Wed, 20 May 2015 07:30:34 -0400 Subject: [Bro] packet post-processor plugin In-Reply-To: <20150519205853.GR16953@yaksha.lbl.gov> References: <20150519205853.GR16953@yaksha.lbl.gov> Message-ID: On Tue, May 19, 2015 at 4:58 PM, Aashish Sharma wrote: > Why do you want to do this ? what is that your packet port-processor going to do that you think bro isn't doing for you? First, I need to see every packet (including non-IP packets), preferably *after* bro has had a chance to analyze them. Second, I'm intending to interface to another existing system that (among other things) collects network inventory; it's largely orthogonal to protocol parsing (although informed by it which is why I would like to see the Connection data). >> Maybe there's some obvious way to do this that I've overlooked? > > Yes, use scripting layer for your analysis. If you think a particular protocol parsing is deficient, instead of writing a packet post-processor, might as well write your new protocol parser. I'm not adding a protocol parser. If/when I add one of those, I will definitely do it the usual bro way. I think the performance penalty of trying to do what I need to do in bro script would be too high (if it were even possible). The bro documentation warns about the overhead of new_packet, packet_contents events, which would be the closest analog to what I need. It seems clear to me that the need to marshal the various data into the form needed by bro script is the primary source of the overhead and I'd like to avoid that. > > Aashish > > On Tue, May 19, 2015 at 04:22:14PM -0400, Jeff Barber wrote: >> >> Still a bro newbie, so I'm looking for some guidance. >> I'd like to add a packet post-processor to bro. It'll be written in C++. >> Essentially I'd like to see every packet that goes through bro, along with >> its Connection record (or the equivalent) if there is one. Ideally it would >> be structured it as a plugin. >> It looks like I could do it by triggering off of events like new_packet >> (although currently that only triggers for IP packets). However, IIUC, that >> still has bro packaging all the info up into a RecordVal, then I have to >> decode it in my plugin event handler. That seems like quite a bit of >> additional overhead on each packet. >> It looks like I might also be able to do it as a PktDumper but then I just >> get the raw packet data and I'd have to go re-parse headers and re-lookup >> connection info - redoing work that's already been done. >> What I'd really like is to simply get a call at the end of analysis for >> every packet, where I'd get passed a pointer to the packet data along with a >> pointer to the existing Connection record (if any). Maybe there's some >> obvious way to do this that I've overlooked? >> Anybody have advice for the best way to go? I'm willing to do work to make >> this happen, but also would prefer not to fork bro so looking for "right >> ways". >> TIA, >> Jeff > >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > -- > Aashish Sharma (asharma at lbl.gov) > Cyber Security, > Lawrence Berkeley National Laboratory > http://go.lbl.gov/pgp-aashish > Office: (510)-495-2680 Cell: (510)-612-7971 From vlad at grigorescu.org Wed May 20 06:09:19 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Wed, 20 May 2015 08:09:19 -0500 Subject: [Bro] elasticsearch plugin identify the proto? In-Reply-To: References: Message-ID: The name of the log stream is stored as the type field in ElasticSearch. On Wed, May 20, 2015 at 1:51 AM, Mo Jia wrote: > In elasticsearch.cc > > bool ElasticSearch::DoWrite(int num_fields, const Field* const * fields, > Value** vals) > > We can get the contents, how can I identify the proto of the content? > > In local logging, it will write to http.log, But in elasticsearch it > lost the proto message. > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150520/b14d715a/attachment.html From gc355804 at ohio.edu Wed May 20 07:49:44 2015 From: gc355804 at ohio.edu (Gilbert Clark) Date: Wed, 20 May 2015 10:49:44 -0400 Subject: [Bro] packet post-processor plugin In-Reply-To: References: <20150519205853.GR16953@yaksha.lbl.gov> Message-ID: <555C9F08.8040405@ohio.edu> Hi Jeff: Not claiming to be an expert, so Right Way (tm) is relative. This is probably a terrible idea, but until there's a better one ... :) I believe an analyzer plugin can get you access to the raw packets / connection object on a per-packet basis. There's a video that explains how to write analyzer plugins from a previous bro gathering ... can dig up a link if google isn't helpful. Offhand, I think my first attempt at doing this would go something like: * Write the new analyzer to do what I wanted with the packets / connection object * Possibly build a little plugin that would allow me to register new analyzers through script-land (sounds like something that should already exist, but can't remember offhand ...) * Write a connection hook that would use the register new analyzers function to register my new analyzer to handle this connection ] Since we want this analyzer to run last, register my analyzer as a child of all other analyzers currently assigned to this connection ... The challenge, I think, would be interfacing with the external network device from an analyzer. Doing that directly would be an issue ... so, maybe create some kind of ring buffer to interface between: * the main bro thread running the analyzer, and ... * ... some kind of external thread that polled the ring buffer populated by the analyzer The external thread could then shoot gathered information from the local bro instance to the interested network device. Hope something in there is interesting / useful. Cheers, Gilbert Clark On 5/20/2015 7:30 AM, Jeff Barber wrote: > On Tue, May 19, 2015 at 4:58 PM, Aashish Sharma wrote: >> Why do you want to do this ? what is that your packet port-processor going to do that you think bro isn't doing for you? > First, I need to see every packet (including non-IP packets), > preferably *after* bro has had a chance to analyze them. Second, I'm > intending to interface to another existing system that (among other > things) collects network inventory; it's largely orthogonal to > protocol parsing (although informed by it which is why I would like to > see the Connection data). > >>> Maybe there's some obvious way to do this that I've overlooked? >> Yes, use scripting layer for your analysis. If you think a particular protocol parsing is deficient, instead of writing a packet post-processor, might as well write your new protocol parser. > I'm not adding a protocol parser. If/when I add one of those, I will > definitely do it the usual bro way. > > I think the performance penalty of trying to do what I need to do in > bro script would be too high (if it were even possible). The bro > documentation warns about the overhead of new_packet, packet_contents > events, which would be the closest analog to what I need. It seems > clear to me that the need to marshal the various data into the form > needed by bro script is the primary source of the overhead and I'd > like to avoid that. > > >> Aashish >> >> On Tue, May 19, 2015 at 04:22:14PM -0400, Jeff Barber wrote: >>> Still a bro newbie, so I'm looking for some guidance. >>> I'd like to add a packet post-processor to bro. It'll be written in C++. >>> Essentially I'd like to see every packet that goes through bro, along with >>> its Connection record (or the equivalent) if there is one. Ideally it would >>> be structured it as a plugin. >>> It looks like I could do it by triggering off of events like new_packet >>> (although currently that only triggers for IP packets). However, IIUC, that >>> still has bro packaging all the info up into a RecordVal, then I have to >>> decode it in my plugin event handler. That seems like quite a bit of >>> additional overhead on each packet. >>> It looks like I might also be able to do it as a PktDumper but then I just >>> get the raw packet data and I'd have to go re-parse headers and re-lookup >>> connection info - redoing work that's already been done. >>> What I'd really like is to simply get a call at the end of analysis for >>> every packet, where I'd get passed a pointer to the packet data along with a >>> pointer to the existing Connection record (if any). Maybe there's some >>> obvious way to do this that I've overlooked? >>> Anybody have advice for the best way to go? I'm willing to do work to make >>> this happen, but also would prefer not to fork bro so looking for "right >>> ways". >>> TIA, >>> Jeff >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> -- >> Aashish Sharma (asharma at lbl.gov) >> Cyber Security, >> Lawrence Berkeley National Laboratory >> http://go.lbl.gov/pgp-aashish >> Office: (510)-495-2680 Cell: (510)-612-7971 > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From alajal at gmail.com Wed May 20 07:54:52 2015 From: alajal at gmail.com (Mustafa Qasim) Date: Wed, 20 May 2015 19:54:52 +0500 Subject: [Bro] Security Consultant in CA Message-ID: Hi, Is there any security consultant available in CA open to an opportunity to work as a remote hand i.e. visit client site and execute the on-site consultancy requirements? Please approach me off the list. ------ *Mustafa Qasim* GREM, GCFE -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150520/4cb6ab29/attachment.html From anshu.sh123 at gmail.com Wed May 20 08:26:46 2015 From: anshu.sh123 at gmail.com (Anshu Sharma) Date: Wed, 20 May 2015 20:56:46 +0530 Subject: [Bro] BRO port scan attack Message-ID: Sir can you please tell me what are the requirements (i.e hardware as well as software) to detect port scan attack in BRO IDS. Can you tell me how detecting of ports works . Thank you Hoping your early reply -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150520/0eb25fbc/attachment.html From jbarber at computer.org Wed May 20 09:32:05 2015 From: jbarber at computer.org (Jeff Barber) Date: Wed, 20 May 2015 12:32:05 -0400 Subject: [Bro] packet post-processor plugin In-Reply-To: <555C9F08.8040405@ohio.edu> References: <20150519205853.GR16953@yaksha.lbl.gov> <555C9F08.8040405@ohio.edu> Message-ID: Thanks, Gilbert. That's helpful. I have been sort of fumbling around that same basic notion. Not sure though. I'd still need to figure out how to handle the non-IP case; the IP_Hdr is embedded in the analyzer interface. And analyzers (except for the special case of ARP) only get called for IP packets, and they get called via the Connection class (which is only created for IP). On Wed, May 20, 2015 at 10:49 AM, Gilbert Clark wrote: > Hi Jeff: > > Not claiming to be an expert, so Right Way (tm) is relative. This is > probably a terrible idea, but until there's a better one ... :) > > I believe an analyzer plugin can get you access to the raw packets / > connection object on a per-packet basis. There's a video that explains > how to write analyzer plugins from a previous bro gathering ... can dig > up a link if google isn't helpful. > > Offhand, I think my first attempt at doing this would go something like: > > * Write the new analyzer to do what I wanted with the packets / > connection object > * Possibly build a little plugin that would allow me to register new > analyzers through script-land (sounds like something that should > already exist, but can't remember offhand ...) > * Write a connection hook that would use the register new analyzers > function to register my new analyzer to handle this connection > ] Since we want this analyzer to run last, register my analyzer as > a child of all other analyzers currently assigned to this connection ... > > The challenge, I think, would be interfacing with the external network > device from an analyzer. Doing that directly would be an issue ... so, > maybe create some kind of ring buffer to interface between: > > * the main bro thread running the analyzer, and ... > * ... some kind of external thread that polled the ring buffer populated > by the analyzer > > The external thread could then shoot gathered information from the local > bro instance to the interested network device. > > Hope something in there is interesting / useful. > > Cheers, > Gilbert Clark > > On 5/20/2015 7:30 AM, Jeff Barber wrote: >> On Tue, May 19, 2015 at 4:58 PM, Aashish Sharma wrote: >>> Why do you want to do this ? what is that your packet port-processor going to do that you think bro isn't doing for you? >> First, I need to see every packet (including non-IP packets), >> preferably *after* bro has had a chance to analyze them. Second, I'm >> intending to interface to another existing system that (among other >> things) collects network inventory; it's largely orthogonal to >> protocol parsing (although informed by it which is why I would like to >> see the Connection data). >> >>>> Maybe there's some obvious way to do this that I've overlooked? >>> Yes, use scripting layer for your analysis. If you think a particular protocol parsing is deficient, instead of writing a packet post-processor, might as well write your new protocol parser. >> I'm not adding a protocol parser. If/when I add one of those, I will >> definitely do it the usual bro way. >> >> I think the performance penalty of trying to do what I need to do in >> bro script would be too high (if it were even possible). The bro >> documentation warns about the overhead of new_packet, packet_contents >> events, which would be the closest analog to what I need. It seems >> clear to me that the need to marshal the various data into the form >> needed by bro script is the primary source of the overhead and I'd >> like to avoid that. >> >> >>> Aashish >>> >>> On Tue, May 19, 2015 at 04:22:14PM -0400, Jeff Barber wrote: >>>> Still a bro newbie, so I'm looking for some guidance. >>>> I'd like to add a packet post-processor to bro. It'll be written in C++. >>>> Essentially I'd like to see every packet that goes through bro, along with >>>> its Connection record (or the equivalent) if there is one. Ideally it would >>>> be structured it as a plugin. >>>> It looks like I could do it by triggering off of events like new_packet >>>> (although currently that only triggers for IP packets). However, IIUC, that >>>> still has bro packaging all the info up into a RecordVal, then I have to >>>> decode it in my plugin event handler. That seems like quite a bit of >>>> additional overhead on each packet. >>>> It looks like I might also be able to do it as a PktDumper but then I just >>>> get the raw packet data and I'd have to go re-parse headers and re-lookup >>>> connection info - redoing work that's already been done. >>>> What I'd really like is to simply get a call at the end of analysis for >>>> every packet, where I'd get passed a pointer to the packet data along with a >>>> pointer to the existing Connection record (if any). Maybe there's some >>>> obvious way to do this that I've overlooked? >>>> Anybody have advice for the best way to go? I'm willing to do work to make >>>> this happen, but also would prefer not to fork bro so looking for "right >>>> ways". >>>> TIA, >>>> Jeff >>>> _______________________________________________ >>>> Bro mailing list >>>> bro at bro-ids.org >>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> >>> -- >>> Aashish Sharma (asharma at lbl.gov) >>> Cyber Security, >>> Lawrence Berkeley National Laboratory >>> http://go.lbl.gov/pgp-aashish >>> Office: (510)-495-2680 Cell: (510)-612-7971 >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From anshu.sh123 at gmail.com Wed May 20 10:33:00 2015 From: anshu.sh123 at gmail.com (Anshu Sharma) Date: Wed, 20 May 2015 23:03:00 +0530 Subject: [Bro] bro cluster Message-ID: sir/mam i m new at bro ids i want to know how cluster are made in bro and what are the requirements to make cluster. is it possible to made cluster on bro ids version 2.3.2 thank you wishing your early reply -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150520/0608dde1/attachment.html From jsiwek at illinois.edu Wed May 20 11:14:58 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Wed, 20 May 2015 18:14:58 +0000 Subject: [Bro] pybroker with 'optional' fields In-Reply-To: References: <62381BC9-BA9E-42AD-BAB3-BC14AAC0FC44@illinois.edu> <8BE3E19A-AE78-4759-BE56-9EF828FAB88C@illinois.edu> Message-ID: <40FCAFE4-4C02-4C6E-B153-636551903D36@illinois.edu> > On May 19, 2015, at 12:54 PM, Jeff Barber wrote: > > rec = p.as_record() > fields = rec.fields() > ip, ip6, tcp, udp, icmp = [f.get() if f.valid() else None for f in fields] > if ip is not None: > fields = ip.as_record().fields() > fields = [f for f in fields] The problem here was ip.as_record() returns a new object, but then .fields() on it returns a reference to something owned by that object whose reference count is going to drop to zero immediately after the line. So it ends up accessing invalid memory of the object which went out of scope. If you want to workaround the bug, assign the record to a temporary variable for as long as you need to access fields coming from it. E.g.: rec = ip.as_record() # assign to rec in order to keep a reference alive fields = rec.fields() # operate on fields? Or if you want to patch/update the broker source code, the real fix is here: https://github.com/bro/broker/commit/8fc6938017dc15acfb26fa29e6ad0933019781c5 That?s also in the master branch of the bro and broker repositories, but should eventually make it into the final 2.4 Bro release and a 0.3.1 release of Broker as well. - Jon From gc355804 at ohio.edu Wed May 20 11:27:21 2015 From: gc355804 at ohio.edu (Gilbert Clark) Date: Wed, 20 May 2015 14:27:21 -0400 Subject: [Bro] packet post-processor plugin In-Reply-To: References: <20150519205853.GR16953@yaksha.lbl.gov> <555C9F08.8040405@ohio.edu> Message-ID: <555CD209.1060902@ohio.edu> What does handle mean in this context? Cheers, Gilbert On 5/20/2015 12:32 PM, Jeff Barber wrote: > Thanks, Gilbert. That's helpful. > > I have been sort of fumbling around that same basic notion. Not sure > though. I'd still need to figure out how to handle the non-IP case; > the IP_Hdr is embedded in the analyzer interface. And analyzers > (except for the special case of ARP) only get called for IP packets, > and they get called via the Connection class (which is only created > for IP). > > > > On Wed, May 20, 2015 at 10:49 AM, Gilbert Clark wrote: >> Hi Jeff: >> >> Not claiming to be an expert, so Right Way (tm) is relative. This is >> probably a terrible idea, but until there's a better one ... :) >> >> I believe an analyzer plugin can get you access to the raw packets / >> connection object on a per-packet basis. There's a video that explains >> how to write analyzer plugins from a previous bro gathering ... can dig >> up a link if google isn't helpful. >> >> Offhand, I think my first attempt at doing this would go something like: >> >> * Write the new analyzer to do what I wanted with the packets / >> connection object >> * Possibly build a little plugin that would allow me to register new >> analyzers through script-land (sounds like something that should >> already exist, but can't remember offhand ...) >> * Write a connection hook that would use the register new analyzers >> function to register my new analyzer to handle this connection >> ] Since we want this analyzer to run last, register my analyzer as >> a child of all other analyzers currently assigned to this connection ... >> >> The challenge, I think, would be interfacing with the external network >> device from an analyzer. Doing that directly would be an issue ... so, >> maybe create some kind of ring buffer to interface between: >> >> * the main bro thread running the analyzer, and ... >> * ... some kind of external thread that polled the ring buffer populated >> by the analyzer >> >> The external thread could then shoot gathered information from the local >> bro instance to the interested network device. >> >> Hope something in there is interesting / useful. >> >> Cheers, >> Gilbert Clark >> >> On 5/20/2015 7:30 AM, Jeff Barber wrote: >>> On Tue, May 19, 2015 at 4:58 PM, Aashish Sharma wrote: >>>> Why do you want to do this ? what is that your packet port-processor going to do that you think bro isn't doing for you? >>> First, I need to see every packet (including non-IP packets), >>> preferably *after* bro has had a chance to analyze them. Second, I'm >>> intending to interface to another existing system that (among other >>> things) collects network inventory; it's largely orthogonal to >>> protocol parsing (although informed by it which is why I would like to >>> see the Connection data). >>> >>>>> Maybe there's some obvious way to do this that I've overlooked? >>>> Yes, use scripting layer for your analysis. If you think a particular protocol parsing is deficient, instead of writing a packet post-processor, might as well write your new protocol parser. >>> I'm not adding a protocol parser. If/when I add one of those, I will >>> definitely do it the usual bro way. >>> >>> I think the performance penalty of trying to do what I need to do in >>> bro script would be too high (if it were even possible). The bro >>> documentation warns about the overhead of new_packet, packet_contents >>> events, which would be the closest analog to what I need. It seems >>> clear to me that the need to marshal the various data into the form >>> needed by bro script is the primary source of the overhead and I'd >>> like to avoid that. >>> >>> >>>> Aashish >>>> >>>> On Tue, May 19, 2015 at 04:22:14PM -0400, Jeff Barber wrote: >>>>> Still a bro newbie, so I'm looking for some guidance. >>>>> I'd like to add a packet post-processor to bro. It'll be written in C++. >>>>> Essentially I'd like to see every packet that goes through bro, along with >>>>> its Connection record (or the equivalent) if there is one. Ideally it would >>>>> be structured it as a plugin. >>>>> It looks like I could do it by triggering off of events like new_packet >>>>> (although currently that only triggers for IP packets). However, IIUC, that >>>>> still has bro packaging all the info up into a RecordVal, then I have to >>>>> decode it in my plugin event handler. That seems like quite a bit of >>>>> additional overhead on each packet. >>>>> It looks like I might also be able to do it as a PktDumper but then I just >>>>> get the raw packet data and I'd have to go re-parse headers and re-lookup >>>>> connection info - redoing work that's already been done. >>>>> What I'd really like is to simply get a call at the end of analysis for >>>>> every packet, where I'd get passed a pointer to the packet data along with a >>>>> pointer to the existing Connection record (if any). Maybe there's some >>>>> obvious way to do this that I've overlooked? >>>>> Anybody have advice for the best way to go? I'm willing to do work to make >>>>> this happen, but also would prefer not to fork bro so looking for "right >>>>> ways". >>>>> TIA, >>>>> Jeff >>>>> _______________________________________________ >>>>> Bro mailing list >>>>> bro at bro-ids.org >>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>> -- >>>> Aashish Sharma (asharma at lbl.gov) >>>> Cyber Security, >>>> Lawrence Berkeley National Laboratory >>>> http://go.lbl.gov/pgp-aashish >>>> Office: (510)-495-2680 Cell: (510)-612-7971 >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From jdopheid at illinois.edu Wed May 20 13:30:43 2015 From: jdopheid at illinois.edu (Dopheide, Jeannette M) Date: Wed, 20 May 2015 20:30:43 +0000 Subject: [Bro] Open position available at the Bro Project Message-ID: Interested in working for the Bro Project? The Bro Project and the NCSA security team are looking for a new developer. The full job description is posted here: http://www.ncsa.illinois.edu/about/jobs/A1500211 Please direct questions about the position to Adam Slagell's attention: slagell at illinois.edu ------ Jeannette Dopheide Bro Outreach Coordinator National Center for Supercomputing Applications University of Illinois at Urbana-Champaign From anthony.kasza at gmail.com Wed May 20 13:41:31 2015 From: anthony.kasza at gmail.com (anthony kasza) Date: Wed, 20 May 2015 13:41:31 -0700 Subject: [Bro] bro cluster In-Reply-To: References: Message-ID: Anshu, have you read any of Bro's documentation? On May 20, 2015 10:53 AM, "Anshu Sharma" wrote: > sir/mam > i m new at bro ids i want to know how cluster are made in bro and what are > the requirements to make cluster. > is it possible to made cluster on bro ids version 2.3.2 > thank you > wishing your early reply > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150520/30c085a1/attachment.html From jbarber at computer.org Wed May 20 13:55:54 2015 From: jbarber at computer.org (Jeff Barber) Date: Wed, 20 May 2015 16:55:54 -0400 Subject: [Bro] pybroker with 'optional' fields In-Reply-To: <40FCAFE4-4C02-4C6E-B153-636551903D36@illinois.edu> References: <62381BC9-BA9E-42AD-BAB3-BC14AAC0FC44@illinois.edu> <8BE3E19A-AE78-4759-BE56-9EF828FAB88C@illinois.edu> <40FCAFE4-4C02-4C6E-B153-636551903D36@illinois.edu> Message-ID: Awesome. Thanks Jon! On Wed, May 20, 2015 at 2:14 PM, Siwek, Jon wrote: > >> On May 19, 2015, at 12:54 PM, Jeff Barber wrote: >> >> rec = p.as_record() >> fields = rec.fields() >> ip, ip6, tcp, udp, icmp = [f.get() if f.valid() else None for f in fields] >> if ip is not None: >> fields = ip.as_record().fields() >> fields = [f for f in fields] > > The problem here was ip.as_record() returns a new object, but then .fields() on it returns a reference to something owned by that object whose reference count is going to drop to zero immediately after the line. So it ends up accessing invalid memory of the object which went out of scope. If you want to workaround the bug, assign the record to a temporary variable for as long as you need to access fields coming from it. E.g.: > > rec = ip.as_record() # assign to rec in order to keep a reference alive > fields = rec.fields() > # operate on fields? > > Or if you want to patch/update the broker source code, the real fix is here: > > https://github.com/bro/broker/commit/8fc6938017dc15acfb26fa29e6ad0933019781c5 > > That?s also in the master branch of the bro and broker repositories, but should eventually make it into the final 2.4 Bro release and a 0.3.1 release of Broker as well. > > - Jon From jbarber at computer.org Wed May 20 14:26:30 2015 From: jbarber at computer.org (Jeff Barber) Date: Wed, 20 May 2015 17:26:30 -0400 Subject: [Bro] packet post-processor plugin In-Reply-To: <555CD209.1060902@ohio.edu> References: <20150519205853.GR16953@yaksha.lbl.gov> <555C9F08.8040405@ohio.edu> <555CD209.1060902@ohio.edu> Message-ID: > What does handle mean in this context? > A primary goal is just to identify the endpoints represented by the various layers in the packet: mac addresses, vlan tag, layer3 proto, IP addresses, IP proto, TCP/UDP ports, etc. Currently, Bro decodes most of that but then basically discards anything that isn't IP. (Not intending to criticize: it's perfectly sensible considering bro's design goals. Anything that isn't IP is non-routable and hence doesn't typically represent a threat that an IDS system cares about. But for my purposes I still want to see those packets.) On Wed, May 20, 2015 at 2:27 PM, Gilbert Clark wrote: > Cheers, > Gilbert > > On 5/20/2015 12:32 PM, Jeff Barber wrote: >> >> Thanks, Gilbert. That's helpful. >> >> I have been sort of fumbling around that same basic notion. Not sure >> though. I'd still need to figure out how to handle the non-IP case; >> the IP_Hdr is embedded in the analyzer interface. And analyzers >> (except for the special case of ARP) only get called for IP packets, >> and they get called via the Connection class (which is only created >> for IP). >> >> >> >> On Wed, May 20, 2015 at 10:49 AM, Gilbert Clark wrote: >>> >>> Hi Jeff: >>> >>> Not claiming to be an expert, so Right Way (tm) is relative. This is >>> probably a terrible idea, but until there's a better one ... :) >>> >>> I believe an analyzer plugin can get you access to the raw packets / >>> connection object on a per-packet basis. There's a video that explains >>> how to write analyzer plugins from a previous bro gathering ... can dig >>> up a link if google isn't helpful. >>> >>> Offhand, I think my first attempt at doing this would go something like: >>> >>> * Write the new analyzer to do what I wanted with the packets / >>> connection object >>> * Possibly build a little plugin that would allow me to register new >>> analyzers through script-land (sounds like something that should >>> already exist, but can't remember offhand ...) >>> * Write a connection hook that would use the register new analyzers >>> function to register my new analyzer to handle this connection >>> ] Since we want this analyzer to run last, register my analyzer as >>> a child of all other analyzers currently assigned to this connection ... >>> >>> The challenge, I think, would be interfacing with the external network >>> device from an analyzer. Doing that directly would be an issue ... so, >>> maybe create some kind of ring buffer to interface between: >>> >>> * the main bro thread running the analyzer, and ... >>> * ... some kind of external thread that polled the ring buffer populated >>> by the analyzer >>> >>> The external thread could then shoot gathered information from the local >>> bro instance to the interested network device. >>> >>> Hope something in there is interesting / useful. >>> >>> Cheers, >>> Gilbert Clark >>> >>> On 5/20/2015 7:30 AM, Jeff Barber wrote: >>>> >>>> On Tue, May 19, 2015 at 4:58 PM, Aashish Sharma wrote: >>>>> >>>>> Why do you want to do this ? what is that your packet port-processor >>>>> going to do that you think bro isn't doing for you? >>>> >>>> First, I need to see every packet (including non-IP packets), >>>> preferably *after* bro has had a chance to analyze them. Second, I'm >>>> intending to interface to another existing system that (among other >>>> things) collects network inventory; it's largely orthogonal to >>>> protocol parsing (although informed by it which is why I would like to >>>> see the Connection data). >>>> >>>>>> Maybe there's some obvious way to do this that I've overlooked? >>>>> >>>>> Yes, use scripting layer for your analysis. If you think a particular >>>>> protocol parsing is deficient, instead of writing a packet post-processor, >>>>> might as well write your new protocol parser. >>>> >>>> I'm not adding a protocol parser. If/when I add one of those, I will >>>> definitely do it the usual bro way. >>>> >>>> I think the performance penalty of trying to do what I need to do in >>>> bro script would be too high (if it were even possible). The bro >>>> documentation warns about the overhead of new_packet, packet_contents >>>> events, which would be the closest analog to what I need. It seems >>>> clear to me that the need to marshal the various data into the form >>>> needed by bro script is the primary source of the overhead and I'd >>>> like to avoid that. >>>> >>>> >>>>> Aashish >>>>> >>>>> On Tue, May 19, 2015 at 04:22:14PM -0400, Jeff Barber wrote: >>>>>> >>>>>> Still a bro newbie, so I'm looking for some guidance. >>>>>> I'd like to add a packet post-processor to bro. It'll be written >>>>>> in C++. >>>>>> Essentially I'd like to see every packet that goes through bro, >>>>>> along with >>>>>> its Connection record (or the equivalent) if there is one. >>>>>> Ideally it would >>>>>> be structured it as a plugin. >>>>>> It looks like I could do it by triggering off of events like >>>>>> new_packet >>>>>> (although currently that only triggers for IP packets). However, >>>>>> IIUC, that >>>>>> still has bro packaging all the info up into a RecordVal, then I >>>>>> have to >>>>>> decode it in my plugin event handler. That seems like quite a >>>>>> bit of >>>>>> additional overhead on each packet. >>>>>> It looks like I might also be able to do it as a PktDumper but >>>>>> then I just >>>>>> get the raw packet data and I'd have to go re-parse headers and >>>>>> re-lookup >>>>>> connection info - redoing work that's already been done. >>>>>> What I'd really like is to simply get a call at the end of >>>>>> analysis for >>>>>> every packet, where I'd get passed a pointer to the packet data >>>>>> along with a >>>>>> pointer to the existing Connection record (if any). Maybe there's >>>>>> some >>>>>> obvious way to do this that I've overlooked? >>>>>> Anybody have advice for the best way to go? I'm willing to do >>>>>> work to make >>>>>> this happen, but also would prefer not to fork bro so looking for >>>>>> "right >>>>>> ways". >>>>>> TIA, >>>>>> Jeff >>>>>> _______________________________________________ >>>>>> Bro mailing list >>>>>> bro at bro-ids.org >>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>> >>>>> -- >>>>> Aashish Sharma (asharma at lbl.gov) >>>>> Cyber Security, >>>>> Lawrence Berkeley National Laboratory >>>>> http://go.lbl.gov/pgp-aashish >>>>> Office: (510)-495-2680 Cell: (510)-612-7971 >>>> >>>> _______________________________________________ >>>> Bro mailing list >>>> bro at bro-ids.org >>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > From seth at icir.org Wed May 20 18:30:21 2015 From: seth at icir.org (Seth Hall) Date: Wed, 20 May 2015 21:30:21 -0400 Subject: [Bro] packet post-processor plugin In-Reply-To: References: <20150519205853.GR16953@yaksha.lbl.gov> <555C9F08.8040405@ohio.edu> <555CD209.1060902@ohio.edu> Message-ID: > On May 20, 2015, at 5:26 PM, Jeff Barber wrote: > >> What does handle mean in this context? > > A primary goal is just to identify the endpoints represented by the > various layers in the packet: mac addresses, vlan tag, layer3 proto, > IP addresses, IP proto, TCP/UDP ports, etc. Ohhhh, now this whole thread makes sense. There has been some discussion internally and on the bro-dev list lately about how to expose that information to scripts in a way that doesn?t overload Bro. Unfortunately there isn?t a timeline yet on actually implementing what has been discussed. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150520/79cc06c6/attachment.bin From life.130815 at gmail.com Thu May 21 02:59:09 2015 From: life.130815 at gmail.com (Mo Jia) Date: Thu, 21 May 2015 17:59:09 +0800 Subject: [Bro] The right way to disable bro write local file and enable elasticsearch Message-ID: I add redef Log::enable_local_logging = F; the elasticsearch can't work , I think the reason is in this func: WriterFrontend::WriterFrontend(const WriterBackend::WriterInfo& arg_info, EnumVal* arg_stream, EnumVal* arg_writer, bool arg_local, bool arg_remote) { // comment if ( local ) { backend = log_mgr->CreateBackend(this, writer); if ( backend ) backend->Start(); } else backend = 0; } I know the ascii writer and elasticsearch plugin are both like a filter on a stream. I want to know the right way to disable the asscii writer? Is there somethings like a single var I can redef such as enable_ascii_logging , event bro_init() &priority=5 { Log::create_stream(HTTP::LOG, [$columns=Info, $ev=log_http, $path="http"]); Analyzer::register_for_ports(Analyzer::ANALYZER_HTTP, ports); } This may create the stream and default make http.log can be create. From life.130815 at gmail.com Thu May 21 03:19:38 2015 From: life.130815 at gmail.com (Mo Jia) Date: Thu, 21 May 2015 18:19:38 +0800 Subject: [Bro] The right way to disable bro write local file and enable elasticsearch In-Reply-To: References: Message-ID: And it seem that in frameworks/logging/man.bro. Everytime it create stream. function add_default_filter(id: ID) : bool { return add_filter(id, [$name="default"]); } this line auto create a Log::WRITER_ASCII to stream. I am looking for a scripts-method to disable ascii log and enable other writer. ( don't want change any c++ file in bro) Does I need comment this line. If in this way every time I install new bro, I should change it . 2015-05-21 17:59 GMT+08:00 Mo Jia : > I add redef Log::enable_local_logging = F; the elasticsearch can't > work , I think the reason is in this func: > > WriterFrontend::WriterFrontend(const WriterBackend::WriterInfo& > arg_info, EnumVal* arg_stream, EnumVal* arg_writer, bool arg_local, > bool arg_remote) > { > > // comment > > if ( local ) > { > backend = log_mgr->CreateBackend(this, writer); > > if ( backend ) > backend->Start(); > } > > else > backend = 0; > } > > I know the ascii writer and elasticsearch plugin are both like a > filter on a stream. I want to know the right way to disable the asscii > writer? > > Is there somethings like a single var I can redef such as > enable_ascii_logging , > > event bro_init() &priority=5 > { > Log::create_stream(HTTP::LOG, [$columns=Info, $ev=log_http, $path="http"]); > Analyzer::register_for_ports(Analyzer::ANALYZER_HTTP, ports); > } > > This may create the stream and default make http.log can be create. From doris at bro.org Thu May 21 08:27:31 2015 From: doris at bro.org (Doris Schioberg) Date: Thu, 21 May 2015 08:27:31 -0700 Subject: [Bro] Bro documentation in pdf format. In-Reply-To: References: Message-ID: <555DF963.3080602@bro.org> Hi Luca, rst2pdf is one of the standard doc-utils-tools. It is however necessary to change the script to comply with our customized Bro layout. So far I converted single pages to pdf but never the whole documentation. I can give you the adapted script so you can convert and print the parts you need. The problem is that the online version contains long code lines, that exceed a normal page but can be scrolled sidewards online. To bring these lines to pdf is the tricky part which is why we don't have a full pdf version of the documentation so far. When converting to pdf I used an intermediate step via latex. Let me know what you want to do. I can give you a step-by-step how-to for the conversion. Doris On 5/17/15 7:55 PM, Luca Renaud wrote: > Is it possible to generate/convert the online bro user manual (sphinx > platform) in pdf format?Eventually,for us to have a printed version. > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Doris Schioberg Bro Outreach, Training, and Education Coordinator International Computer Science Institute (ICSI Berkeley) Phone: +1 (510) 289-8406 * doris at bro.org From anshu.sh123 at gmail.com Thu May 21 08:48:06 2015 From: anshu.sh123 at gmail.com (Anshu Sharma) Date: Thu, 21 May 2015 21:18:06 +0530 Subject: [Bro] BRO signature Message-ID: Sir/Mam i read the document which is provided in bro.org website regarding signature framework. But the thing which i cant understand is that where i have to place this file i.e i write a simple program with name hello.bro but i don't know in which folder this file to place because in command prompt it always says cannot create a file. Please tell me what can i do and also some information about how to write a signature code . thank you wishing your early reply -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150521/b5ba63ad/attachment.html From dnthayer at illinois.edu Thu May 21 10:00:29 2015 From: dnthayer at illinois.edu (Daniel Thayer) Date: Thu, 21 May 2015 12:00:29 -0500 Subject: [Bro] The right way to disable bro write local file and enable elasticsearch In-Reply-To: References: Message-ID: <555E0F2D.80901@illinois.edu> Have you tried removing the default filter? event bro_init() { Log::remove_filter(HTTP::LOG, "default"); } On 05/21/2015 05:19 AM, Mo Jia wrote: > And it seem that in frameworks/logging/man.bro. Everytime it create stream. > > function add_default_filter(id: ID) : bool > { > return add_filter(id, [$name="default"]); > } > > this line auto create a Log::WRITER_ASCII to stream. > > I am looking for a scripts-method to disable ascii log and enable > other writer. ( don't want change any c++ file in bro) > Does I need comment this line. If in this way every time I install > new bro, I should change it . > > > 2015-05-21 17:59 GMT+08:00 Mo Jia : >> I add redef Log::enable_local_logging = F; the elasticsearch can't >> work , I think the reason is in this func: >> >> WriterFrontend::WriterFrontend(const WriterBackend::WriterInfo& >> arg_info, EnumVal* arg_stream, EnumVal* arg_writer, bool arg_local, >> bool arg_remote) >> { >> >> // comment >> >> if ( local ) >> { >> backend = log_mgr->CreateBackend(this, writer); >> >> if ( backend ) >> backend->Start(); >> } >> >> else >> backend = 0; >> } >> >> I know the ascii writer and elasticsearch plugin are both like a >> filter on a stream. I want to know the right way to disable the asscii >> writer? >> >> Is there somethings like a single var I can redef such as >> enable_ascii_logging , >> >> event bro_init() &priority=5 >> { >> Log::create_stream(HTTP::LOG, [$columns=Info, $ev=log_http, $path="http"]); >> Analyzer::register_for_ports(Analyzer::ANALYZER_HTTP, ports); >> } >> >> This may create the stream and default make http.log can be create. > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > From gc355804 at ohio.edu Thu May 21 10:37:57 2015 From: gc355804 at ohio.edu (Gilbert Clark) Date: Thu, 21 May 2015 13:37:57 -0400 Subject: [Bro] packet post-processor plugin In-Reply-To: References: <20150519205853.GR16953@yaksha.lbl.gov> <555C9F08.8040405@ohio.edu> <555CD209.1060902@ohio.edu> Message-ID: <555E17F5.70200@ohio.edu> It does sound like an analyzer might not completely work after all... One option here might be to duplicate / filter the stuff an analyzer-based solution won't support (e.g. via either a custom packet source or something like [1]) and forward it to a one-off application written to parse / gather information from that specific type of traffic. The advantage I see there is that it might let bro do most of the heavy lifting for the complex stuff, and might also yield faster per-packet performance numbers for the other stuff (since the one-off doubles as a fast-path processor for those specific types of traffic). Once bro adds support for the stuff that's interesting, then look at taking the one-off code and making it part of the bro plugin proper. Just a thought, Gilbert [1] https://github.com/bro/packet-bricks On 5/20/2015 9:30 PM, Seth Hall wrote: >> On May 20, 2015, at 5:26 PM, Jeff Barber wrote: >> >>> What does handle mean in this context? >> A primary goal is just to identify the endpoints represented by the >> various layers in the packet: mac addresses, vlan tag, layer3 proto, >> IP addresses, IP proto, TCP/UDP ports, etc. > Ohhhh, now this whole thread makes sense. There has been some discussion internally and on the bro-dev list lately about how to expose that information to scripts in a way that doesn?t overload Bro. Unfortunately there isn?t a timeline yet on actually implementing what has been discussed. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > From life.130815 at gmail.com Thu May 21 19:59:01 2015 From: life.130815 at gmail.com (Mo Jia) Date: Fri, 22 May 2015 10:59:01 +0800 Subject: [Bro] The right way to disable bro write local file and enable elasticsearch In-Reply-To: <555E0F2D.80901@illinois.edu> References: <555E0F2D.80901@illinois.edu> Message-ID: Thanks . Using this code success get my wish. event bro_init() &priority=-5 { for ( stream_id in Log::active_streams ) { Log::remove_filter(stream_id, "default"); } } by the way, #:/usr/local/bro/logs/current$ ls debug.log packet_filter.log stderr.log stdout.log These logs can't remove by this way. 2015-05-22 1:00 GMT+08:00 Daniel Thayer : > Have you tried removing the default filter? > > event bro_init() > { > Log::remove_filter(HTTP::LOG, "default"); > > } > > > On 05/21/2015 05:19 AM, Mo Jia wrote: >> >> And it seem that in frameworks/logging/man.bro. Everytime it create >> stream. >> >> function add_default_filter(id: ID) : bool >> { >> return add_filter(id, [$name="default"]); >> } >> >> this line auto create a Log::WRITER_ASCII to stream. >> >> I am looking for a scripts-method to disable ascii log and enable >> other writer. ( don't want change any c++ file in bro) >> Does I need comment this line. If in this way every time I install >> new bro, I should change it . >> >> >> 2015-05-21 17:59 GMT+08:00 Mo Jia : >>> >>> I add redef Log::enable_local_logging = F; the elasticsearch can't >>> work , I think the reason is in this func: >>> >>> WriterFrontend::WriterFrontend(const WriterBackend::WriterInfo& >>> arg_info, EnumVal* arg_stream, EnumVal* arg_writer, bool arg_local, >>> bool arg_remote) >>> { >>> >>> // comment >>> >>> if ( local ) >>> { >>> backend = log_mgr->CreateBackend(this, writer); >>> >>> if ( backend ) >>> backend->Start(); >>> } >>> >>> else >>> backend = 0; >>> } >>> >>> I know the ascii writer and elasticsearch plugin are both like a >>> filter on a stream. I want to know the right way to disable the asscii >>> writer? >>> >>> Is there somethings like a single var I can redef such as >>> enable_ascii_logging , >>> >>> event bro_init() &priority=5 >>> { >>> Log::create_stream(HTTP::LOG, [$columns=Info, $ev=log_http, >>> $path="http"]); >>> Analyzer::register_for_ports(Analyzer::ANALYZER_HTTP, ports); >>> } >>> >>> This may create the stream and default make http.log can be create. >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > From anshu.sh123 at gmail.com Thu May 21 23:01:48 2015 From: anshu.sh123 at gmail.com (Anshu Sharma) Date: Fri, 22 May 2015 11:31:48 +0530 Subject: [Bro] BRO signature Message-ID: Sir/Mam I am new at bro i have install 2.3.2 . I want to create a signature framework i have read the document provided on bro website but i cannot understand how to execute it i.e signature my-first-sig { ip-proto == tcp dst-port == 80 payload /.*root/ event "Found root!" } i taken this code from your site but it is not running is it require addition coding please tell me what to do now. thank you waiting for your early reply -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150522/62891874/attachment.html From dnthayer at illinois.edu Fri May 22 08:55:28 2015 From: dnthayer at illinois.edu (Daniel Thayer) Date: Fri, 22 May 2015 10:55:28 -0500 Subject: [Bro] The right way to disable bro write local file and enable elasticsearch In-Reply-To: References: <555E0F2D.80901@illinois.edu> Message-ID: <555F5170.4030403@illinois.edu> On 05/21/2015 09:59 PM, Mo Jia wrote: > Thanks . > > Using this code success get my wish. > > event bro_init() &priority=-5 > { > for ( stream_id in Log::active_streams ) > { > Log::remove_filter(stream_id, "default"); > } > } > > by the way, > #:/usr/local/bro/logs/current$ ls > debug.log packet_filter.log stderr.log stdout.log > > These logs can't remove by this way. The reason why you are seeing the packet_filter.log is because it gets created before you remove its default filter. To prevent the log file from being created, try removing the "&priority=-5" on your bro_init event handler. The reason why you are seeing those other files is because they are not created by the logging framework (e.g., stdout.log/stderr.log are created by broctl). From anthony.kasza at gmail.com Fri May 22 09:34:43 2015 From: anthony.kasza at gmail.com (anthony kasza) Date: Fri, 22 May 2015 09:34:43 -0700 Subject: [Bro] BRO signature In-Reply-To: References: Message-ID: Place your signature in a file named "mysig.sig". Then create another file called "myscript.bro". Within this file, use the @load-sigs directive to load "mysig.sig". When you run Bro be sure to tell Bro to include "myscript.bro". You'll also need to write an event handler for when your signature is matched. See here https://www.bro.org/sphinx-git/frameworks/signatures.html -AK On May 21, 2015 11:23 PM, "Anshu Sharma" wrote: > Sir/Mam > I am new at bro i have install 2.3.2 . I want to create a signature > framework i have read the document provided on bro website but i cannot > understand how to execute it i.e > > signature my-first-sig { > ip-proto == tcp > dst-port == 80 > payload /.*root/ > event "Found root!" > } > > i taken this code from your site but it is not running is it require addition coding > > please tell me what to do now. > > thank you > > waiting for your early reply > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150522/6b8e4959/attachment.html From anshu.sh123 at gmail.com Sun May 24 08:39:10 2015 From: anshu.sh123 at gmail.com (Anshu Sharma) Date: Sun, 24 May 2015 21:09:10 +0530 Subject: [Bro] BRO SIGNATURE Message-ID: Sir/Mam In my previous mail i asked you about the bro signature you told me to create two files i.e mysig.sig and myscript.bro . The second file i downloaded from your site https://www.bro.org/sphinx/_downloads/main16.bro. Then i read the document of signature framework . Now i am totally confused i don't know what to do i cannot understand what changes are to be made in this file and how. If anybody can tell me it is going to be a great help. Please reply this mail. thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150524/21609017/attachment.html From anshu.sh123 at gmail.com Sun May 24 23:26:30 2015 From: anshu.sh123 at gmail.com (Anshu Sharma) Date: Mon, 25 May 2015 11:56:30 +0530 Subject: [Bro] event handler in beo Message-ID: Sir/Mam i need to write an event handler for when my signature is matched . can anyone tell me how to do? thanks From bilal.comsian09 at gmail.com Mon May 25 00:58:35 2015 From: bilal.comsian09 at gmail.com (bilal ahmed) Date: Mon, 25 May 2015 12:58:35 +0500 Subject: [Bro] broccoli python Message-ID: Hi Sir, I have while configuring the broccoli-python ./configure it is ok on make it show the below response make: Nothing to be done for `all'. and on python setup.py install running install running build running build_py running build_ext building '_broccoli_intern' extension x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I../../build/src -I/usr/include/python3.4m -c broccoli_intern_wrap.c -o build/temp.linux-x86_64-3.4/broccoli_intern_wrap.o broccoli_intern_wrap.c:2950:22: fatal error: broccoli.h: No such file or directory #include ^ compilation terminated. I dont know how to solve I have install bro on /nsm/bro path please guide. Regards Bilal -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150525/c3344705/attachment.html From anthony.kasza at gmail.com Mon May 25 08:42:21 2015 From: anthony.kasza at gmail.com (anthony kasza) Date: Mon, 25 May 2015 08:42:21 -0700 Subject: [Bro] broccoli python In-Reply-To: References: Message-ID: It sounds like you need to adjust your environment variables so that header file can be found. -AK On May 25, 2015 4:13 AM, "bilal ahmed" wrote: > Hi Sir, I have while configuring the broccoli-python > ./configure it is ok > on make it show the below response > > make: Nothing to be done for `all'. > > and on python setup.py install > running install > running build > running build_py > running build_ext > building '_broccoli_intern' extension > x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall > -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat > -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I../../build/src > -I/usr/include/python3.4m -c broccoli_intern_wrap.c -o > build/temp.linux-x86_64-3.4/broccoli_intern_wrap.o > broccoli_intern_wrap.c:2950:22: fatal error: broccoli.h: No such file or > directory > #include > ^ > compilation terminated. > > I dont know how to solve > I have install bro on /nsm/bro path please guide. > Regards > Bilal > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150525/c432a581/attachment.html From mlaterma at ucalgary.ca Mon May 25 12:28:32 2015 From: mlaterma at ucalgary.ca (Michel Laterman) Date: Mon, 25 May 2015 19:28:32 +0000 Subject: [Bro] event handler in beo In-Reply-To: References: Message-ID: Hello Anshu, When your signature is matched by a script it raises a signature_match event, as described here: https://www.bro.org/sphinx-git/scripts/base/bif/event.bif.bro.html#id-signature_match For a very simple example of this event matching to a specific signature see: http://try.bro.org/#/trybro/saved/8104 I have just modified the example signature provided in: https://www.bro.org/sphinx-git/frameworks/signatures.html to look for the string "youtube" instead of "root" (anywhere in the payload), this way you can run it on the provided http.pcap file to get a match. Hope that helps, Michel ________________________________________ From: bro-bounces at bro.org on behalf of Anshu Sharma Sent: May 25, 2015 12:26 AM To: bro Subject: [Bro] event handler in beo Sir/Mam i need to write an event handler for when my signature is matched . can anyone tell me how to do? thanks _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From monahbaki at gmail.com Tue May 26 07:16:27 2015 From: monahbaki at gmail.com (Monah Baki) Date: Tue, 26 May 2015 10:16:27 -0400 Subject: [Bro] No http.log and dns.log Message-ID: I have all of the logs except my http.log and dns.log. cat /usr/local/bro/etc/node.cfg [bro] type=standalone host=localhost interface=eth1 eth1 Link encap:Ethernet HWaddr 00:50:56:8b:0f:0a inet6 addr: fe80::250:56ff:fe8b:f0a/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:438596065 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:311886004695 (311.8 GB) TX bytes:648 (648.0 B) Running "tcpdump -i eth1 port 80 or port 53", I do see output. I even ran broctl install broctl restart Still missing http.log and dns.log. Thanks Monah -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150526/95b332be/attachment.html From abhall1 at yahoo.com Tue May 26 16:26:22 2015 From: abhall1 at yahoo.com (Adam Hall) Date: Tue, 26 May 2015 23:26:22 +0000 (UTC) Subject: [Bro] bro2csv Message-ID: <978227049.2507947.1432682782334.JavaMail.yahoo@mail.yahoo.com> I have recently added a little script called bro2csv on Github. The script is intended to run on the default log settings for Bro being tab delimited logs with epoch timestamps. This will allow you to take a file with bro logs and very easily add headers, human readable timestamps,and covert it to CSV format. The script is done purely in Python. I came up with this as many times I would get asked what a particular column was in a log.Sometimes I would even get what does the whole thing mean! So this can save a lot of time if you present your Bro logs to anyone. You will begin a line with a '#' for a comment (#This is a test).You will begin a line with a '@' for a header (@conn).Lastly, you just group your logs however you want them displayed. Just run the command bro2csv -i 'your file' and your done. You can clone this at 'https://github.com/red8383light/BRO2CSV' This will create '/usr/local/bro2csv' and consists of the bro2csv file and your header file. Please check it out if you get sometime.? Feed back is greatly appreciated!I hope to contribute more to the project soon. This is my first addition in Git so if I did something wrong or you have any further suggestions... please let me know. Thanks,Adam "Red8383light" Hall -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150526/0a15ece4/attachment.html From vitologrillo at gmail.com Wed May 27 03:03:15 2015 From: vitologrillo at gmail.com (Vito Logrillo) Date: Wed, 27 May 2015 12:03:15 +0200 Subject: [Bro] How to modify http.log Message-ID: Hi all, i'm trying to modify http.log using the script written below -----script.bro----- redef record HTTP::Info += { host_ip: set[addr] &optional &log; }; event connection_state_remove(c: connection) &priority=5 { local record_flag: bool = F; if (/^[hH][tT][tT][pP]:/ in c$http$uri) { record_flag = T; when (local h = lookup_hostname(c$http$host)) { record_flag = F; print(h); if (|h|>0 && (0.0.0.0 !in h)) { c$http$host_ip = h; Log::write(HTTP::LOG, c$http); } return; } } if (record_flag == T) { return; } } -----end script.bro---- I've added a new field in http.log (host_ip) in order to see the host ip using the function lookup_hostname. The script works well, but the same record is written twice (with and without the host_ip field). I've tried to use a state flag (record_flag) to avoid this, but the result is the same. How can avoid record duplicantion? Thanks, Vito From a.benkhadra at hotmail.com Wed May 27 04:52:13 2015 From: a.benkhadra at hotmail.com (Abdelkarim benkhadra) Date: Wed, 27 May 2015 12:52:13 +0100 Subject: [Bro] initial buffer length Message-ID: Hi, in myproto-protocol.pac, i have: type myproto_PDU(is_orig: bool) = record { header: myproto_Header; //restofdefinition; } &length=header.msgSize; type myproto_Header = record { //some fields; msgSize: uint16; //restoffields; }; msgSize represents the length of the PDU. and when "make"ing bro i get "cannot determine initial buffer length for type myproto_PDU". i tryed some random values like &length=60 and it works fine. i took the modbus analyzer as an exemple to write myproto analyzer but without success. any ideas? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150527/469b3742/attachment.html From jdopheid at illinois.edu Wed May 27 07:22:42 2015 From: jdopheid at illinois.edu (Dopheide, Jeannette M) Date: Wed, 27 May 2015 14:22:42 +0000 Subject: [Bro] BroCon '15: CFP due Friday & other updates Message-ID: Bro Community, BroCon '15 is just around the corner (August 4th -6th). Have you registered yet? https://www.regonline.com/brocon2015 The Call for Presentations is due this Friday, May 29th: https://www.bro.org/community/brocon2015.html#call-forpresentations Not sure if you want to come to BroCon? Check out our working Agenda here: https://www.bro.org/community/brocon2015.html#agenda More updates will be posted to the Agenda once we have approved the submitted proposals. If you're on Twiiter, tag us @Bro_IDS or use #BroCon2015 to connect with other members of the community. Thanks again, and see you in August! The Bro Team ------ Jeannette Dopheide Bro Outreach Coordinator National Center for Supercomputing Applications University of Illinois at Urbana-Champaign From hlin33 at illinois.edu Wed May 27 08:02:53 2015 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Wed, 27 May 2015 10:02:53 -0500 Subject: [Bro] initial buffer length In-Reply-To: References: Message-ID: Hi Abdelkarim, In the myproto_Header, do you have some definitions of records before msgSize (whose length is not explicitly specified)? If that is the case, BinPac cannot decide the initial length in order to locate msgSize. You may refer the DNP3 analyzer that I wrote in a similar way. type Header_Block = record { start: uint16 &check(start == 0x0564); len: uint8; ctrl: uint8; dest_addr: uint16; src_addr: uint16; } &byteorder = littleendian; type DNP3_Request = record { addin_header: Header_Block; ## added by Hui Lin in Bro code .... } &byteorder = bigendian &length= 9 + addin_header.len - 5 - 1; ?Hope this helps. Best, Hui Lin? ?? On Wed, May 27, 2015 at 6:52 AM, Abdelkarim benkhadra < a.benkhadra at hotmail.com> wrote: > Hi, > > in myproto-protocol.pac, i have: > > type myproto_PDU(is_orig: bool) = record { > header: myproto_Header; > //restofdefinition; > } &length=header.msgSize; > > type myproto_Header = record { > //some fields; > msgSize: uint16; > //restoffields; > }; > msgSize represents the length of the PDU. > and when "make"ing bro i get "cannot determine initial buffer length for > type myproto_PDU". > i tryed some random values like &length=60 and it works fine. > i took the modbus analyzer as an exemple to write myproto analyzer but > without success. > any ideas? > -- Hui Lin PhD Candidate, Research Assistant Electrical and Computer Engineering Department University of Illinois at Urbana-Champaign -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150527/4403309d/attachment-0001.html From vlad at grigorescu.org Wed May 27 09:01:29 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Wed, 27 May 2015 11:01:29 -0500 Subject: [Bro] How to modify http.log In-Reply-To: References: Message-ID: Log::write is already being called in the base scripts. You want to add a field to the record and let the base scripts worry about actually logging it out. See policy/protocols/http/var-extraction-cookies.bro ( https://github.com/bro/bro/blob/master/scripts/policy/protocols/http/var-extraction-cookies.bro) as an example. You might not be able to do what you want, though, because lookup_hostname is an asynchronous function. If it doesn't return quickly enough, the log will be written without the field filled in. Another thing to keep in mind is that a large number of asynchronous calls can have a significant performance penalty. c$http, c$http$uri and c$http$host are optional fields[1], so you should check for the presence of those fields with the ?$ operator before accessing them. Finally, the scheme (http://) is not included in the uri field, so I'm not really sure how your if statement is matching. I would replace that if condition with: c?$http && c$http?$host. If the host field is set, you know it's HTTP and that you saw the request. --Vlad [1] - < https://www.bro.org/sphinx-git/scripts/base/protocols/http/main.bro.html#type-HTTP::Info > On Wed, May 27, 2015 at 5:03 AM, Vito Logrillo wrote: > Hi all, > i'm trying to modify http.log using the script written below > > -----script.bro----- > redef record HTTP::Info += { > host_ip: set[addr] &optional &log; > }; > > event connection_state_remove(c: connection) &priority=5 > { > local record_flag: bool = F; > > if (/^[hH][tT][tT][pP]:/ in c$http$uri) > { > > record_flag = T; > > when (local h = lookup_hostname(c$http$host)) > { > record_flag = F; > print(h); > if (|h|>0 && (0.0.0.0 !in h)) > { > c$http$host_ip = h; > Log::write(HTTP::LOG, c$http); > } > return; > } > } > if (record_flag == T) > { > return; > } > } > > -----end script.bro---- > > I've added a new field in http.log (host_ip) in order to see the host > ip using the function lookup_hostname. > The script works well, but the same record is written twice (with and > without the host_ip field). > I've tried to use a state flag (record_flag) to avoid this, but the > result is the same. > How can avoid record duplicantion? > Thanks, > Vito > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150527/4c1d038d/attachment.html From hlin33 at illinois.edu Thu May 28 08:21:51 2015 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Thu, 28 May 2015 10:21:51 -0500 Subject: [Bro] initial buffer length In-Reply-To: References: Message-ID: HI Abdelkarim, Probably you should try to use the basic data type such as using int for "type" and "chunk" instead of using the bytestring (even with the length defined). I remember that the bytestring did not work for my case either. I don't remember the reason in details now, but I remember that it is related to how Binpac compiler is implemented. Best, Hui Lin On Thu, May 28, 2015 at 3:44 AM, Abdelkarim benkhadra < a.benkhadra at hotmail.com> wrote: > hi hui, > thanks for your answer. > > i tryed the DNP3 way still does not work, > > type myproto_Header = record { > type: bytestring &length=3; > chunk: bytestring &length=1, > msgSize: uint16; > }; > > type myproto_req = record { > hdr: myproto_Header; > data: case (hdr.type) of { > //some records; > }; > }&length = hdr.msgSize; > > the length of records in the "case" is specified for each one. > still get the same error. > > ------------------------------ > From: hlin33 at illinois.edu > Date: Wed, 27 May 2015 10:02:53 -0500 > Subject: Re: [Bro] initial buffer length > To: a.benkhadra at hotmail.com > CC: bro at bro.org > > > Hi Abdelkarim, > > In the myproto_Header, do you have some definitions of records before > msgSize (whose length is not explicitly specified)? If that is the case, > BinPac cannot decide the initial length in order to locate msgSize. > > You may refer the DNP3 analyzer that I wrote in a similar way. > > > type Header_Block = record { > start: uint16 &check(start == 0x0564); > len: uint8; > ctrl: uint8; > dest_addr: uint16; > src_addr: uint16; > } &byteorder = littleendian; > > type DNP3_Request = record { > addin_header: Header_Block; ## added by Hui Lin in Bro code > .... > } &byteorder = bigendian > &length= 9 + addin_header.len - 5 - 1; > > > ?Hope this helps. > > Best, > > Hui Lin? > > ?? > > On Wed, May 27, 2015 at 6:52 AM, Abdelkarim benkhadra < > a.benkhadra at hotmail.com> wrote: > > Hi, > > in myproto-protocol.pac, i have: > > type myproto_PDU(is_orig: bool) = record { > header: myproto_Header; > //restofdefinition; > } &length=header.msgSize; > > type myproto_Header = record { > //some fields; > msgSize: uint16; > //restoffields; > }; > msgSize represents the length of the PDU. > and when "make"ing bro i get "cannot determine initial buffer length for > type myproto_PDU". > i tryed some random values like &length=60 and it works fine. > i took the modbus analyzer as an exemple to write myproto analyzer but > without success. > any ideas? > > > > > -- > Hui Lin > PhD Candidate, Research Assistant > Electrical and Computer Engineering Department > University of Illinois at Urbana-Champaign > -- Hui Lin PhD Candidate, Research Assistant Electrical and Computer Engineering Department University of Illinois at Urbana-Champaign -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150528/fa1f5a28/attachment.html From jlay at slave-tothe-box.net Thu May 28 08:47:58 2015 From: jlay at slave-tothe-box.net (James Lay) Date: Thu, 28 May 2015 09:47:58 -0600 Subject: [Bro] Quick PF_RING question Message-ID: <135abb3480ae5e6d737a3719991ade98@localhost> Hey all, So I'm reading up and wanting to try out PF_RING for bro. Is there a way you use this using command line bro? The documentation only talks about clustering using PF_RING, and that's not what I want to do. End goal is to reduce packet loss. Thank you. James From seth at icir.org Thu May 28 12:58:07 2015 From: seth at icir.org (Seth Hall) Date: Thu, 28 May 2015 15:58:07 -0400 Subject: [Bro] Quick PF_RING question In-Reply-To: <135abb3480ae5e6d737a3719991ade98@localhost> References: <135abb3480ae5e6d737a3719991ade98@localhost> Message-ID: > On May 28, 2015, at 11:47 AM, James Lay wrote: > > So I'm reading up and wanting to try out PF_RING for bro. Is there a > way you use this using command line bro? The documentation only talks > about clustering using PF_RING, and that's not what I want to do. End > goal is to reduce packet loss. Thank you. Yeah, just take a look at the pf_ring plugin in broctl. There are just a few environment variables you need to set. .Seht -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150528/9046e718/attachment.bin From jlay at slave-tothe-box.net Thu May 28 13:05:25 2015 From: jlay at slave-tothe-box.net (James Lay) Date: Thu, 28 May 2015 14:05:25 -0600 Subject: [Bro] Quick PF_RING question In-Reply-To: References: <135abb3480ae5e6d737a3719991ade98@localhost> Message-ID: <7cadfd539c039327eb57fa21963b95a2@localhost> On 2015-05-28 01:58 PM, Seth Hall wrote: >> On May 28, 2015, at 11:47 AM, James Lay >> wrote: >> >> So I'm reading up and wanting to try out PF_RING for bro. Is there a >> way you use this using command line bro? The documentation only talks >> about clustering using PF_RING, and that's not what I want to do. End >> goal is to reduce packet loss. Thank you. > > Yeah, just take a look at the pf_ring plugin in broctl. There are > just a few environment variables you need to set. > > .Seht > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ Perfect..thanks for the direction Seth. James From ali at ashemery.com Sat May 30 12:16:07 2015 From: ali at ashemery.com (Ali Hadi) Date: Sat, 30 May 2015 22:16:07 +0300 Subject: [Bro] tx_hosts and rx_hosts in files.log Message-ID: Hi, If you use the PCAP below and analyze it using Bro: https://www.bro.org/static/traces/email.pcap Then when checking the files.log, the tx_hosts is supposed to show the host who transmitted the file, and rx_hosts is for the host who received the file based on Bro's documentation: https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html If you do the following: cat files.log | bro-cut fuid tx_hosts rx_hosts | grep You'll get that the TX Host IP (SrcIP) is 192.168.121.176 and not 192.168.121.179 !!! Is there something I'm doing wrong, or has bro switched their positions in the output? ?Thanks in advance, *Ali* -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150530/26f505d9/attachment.html From usmanshafique68 at hotmail.com Sun May 31 12:06:26 2015 From: usmanshafique68 at hotmail.com (usman shafique) Date: Mon, 1 Jun 2015 00:06:26 +0500 Subject: [Bro] Dynamic protocol Detection In-Reply-To: <7cadfd539c039327eb57fa21963b95a2@localhost> References: <135abb3480ae5e6d737a3719991ade98@localhost>, , <7cadfd539c039327eb57fa21963b95a2@localhost> Message-ID: any idea regarding dynamic protocol detection in bro give me simple script example > Date: Thu, 28 May 2015 14:05:25 -0600 > From: jlay at slave-tothe-box.net > To: bro at bro.org > Subject: Re: [Bro] Quick PF_RING question > > On 2015-05-28 01:58 PM, Seth Hall wrote: > >> On May 28, 2015, at 11:47 AM, James Lay > >> wrote: > >> > >> So I'm reading up and wanting to try out PF_RING for bro. Is there a > >> way you use this using command line bro? The documentation only talks > >> about clustering using PF_RING, and that's not what I want to do. End > >> goal is to reduce packet loss. Thank you. > > > > Yeah, just take a look at the pf_ring plugin in broctl. There are > > just a few environment variables you need to set. > > > > .Seht > > > > -- > > Seth Hall > > International Computer Science Institute > > (Bro) because everyone has a network > > http://www.bro.org/ > > Perfect..thanks for the direction Seth. > > James > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150601/3c005393/attachment.html From vlad at grigorescu.org Sun May 31 14:35:40 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Sun, 31 May 2015 16:35:40 -0500 Subject: [Bro] tx_hosts and rx_hosts in files.log In-Reply-To: References: Message-ID: Thanks for the bug report. Looks like this comes from the assumption made here: https://github.com/bro/bro/blob/master/src/analyzer/protocol/mime/MIME.cc#L1459 --Vlad On Sat, May 30, 2015 at 2:16 PM, Ali Hadi wrote: > Hi, > > If you use the PCAP below and analyze it using Bro: > https://www.bro.org/static/traces/email.pcap > > Then when checking the files.log, the tx_hosts is supposed to show the > host who transmitted the file, and rx_hosts is for the host who received > the file based on Bro's documentation: > https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html > > If you do the following: > cat files.log | bro-cut fuid tx_hosts rx_hosts | grep PDF FILE> > > You'll get that the TX Host IP (SrcIP) is 192.168.121.176 and > not 192.168.121.179 !!! > > Is there something I'm doing wrong, or has bro switched their positions in > the output? > > ?Thanks in advance, > *Ali* > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150531/5134c617/attachment.html