[Bro] sum stats q.
Azoff, Justin S
jazoff at illinois.edu
Thu Nov 29 09:24:04 PST 2018
This is all my fault . Currently trybro limits log output to 200 lines for each file. It shows the first 100 and the last 100. I had always intended on making that more obvious and allowing that '200' parameter to be changed, but forgot all about it. It was mostly done as a performance optimization - the log output can be quite large and the result would either take too long to transfer to the client or the browser would freeze trying to render a table with 20k rows. The good news is that it is already a parameter on the backend, it just needs to be exposed to the api.
If you increase the interval on your script to 500secs that outputs all the records since the total number of rows is just under 200.
If you run it with a local bro binary you should get the output you are expecting as well.
That said.. the script you posted would likely have issues if ran on a cluster. The short time interval combined with the potential for a large number of unique 'keys' in sumstats would cause a large amount of load on the manager. If you're not running it on a cluster on live traffic it should work fine though. If you do want to run that exact analysis on a cluster I can write you a version that uses events directly and would perform a bit better under load.
More information about the Bro