[Bro] [maintenance] what would cause a backlog/erasure in "...logs/current"?
gfaulkner.nsm at gmail.com
Thu Jan 8 10:36:08 PST 2015
I've seen this sort of thing happen when one or more log files get
really large, typically dns.log, conn.log, or http.log. It seems to
partially rotate the logs, perform the initial rename, but not compress
them and do the second renaming (you'll notice that the naming
convention differs from the already moved and compressed logs). You may
also notice you stop getting connection summaries when this happens as
well. I suspect that part of the post-processing never happens or never
Many times I find the logs end up really big due to one or more
misbehaving hosts, such as open DNS resolvers participating in a DDOS or
a compromised host aggressively scanning/attacking something. Best bet
is to manually move the logs that missed rotation(and compress them)
then restart bro. If you restart bro without moving the logs the logs
that didn't fully get processed on prior rotations first I've noticed
those logs tend to simply get deleted. The only work around I've found
is to address the traffic that is causing the logs to explode.
On 1/8/2015 10:24 AM, Glenn Forbes Fleming Larratt wrote:
> My Bro cluster is happily flagging and accumulating data - but:
> 1. The last two hourly cycles left uncompressed logfiles in
> -rw-r--r-- 1 bro bro 73529 Jan 8 11:00 reporter-15-01-08_10.00.00.log
> -rw-r--r-- 1 bro bro 749059 Jan 8 11:00 tunnel-15-01-08_10.00.00.log
> -rw-r--r-- 1 bro bro 2474781 Jan 8 11:00 weird-15-01-08_10.00.00.log
> -rw-r--r-- 1 bro bro 17062559659 Jan 8 10:00 conn-15-01-08_09.00.00.log
> -rw-r--r-- 1 bro bro 2260979370 Jan 8 10:00 files-15-01-08_09.00.00.log
> -rw-r--r-- 1 bro bro 4942559737 Jan 8 10:00 http-15-01-08_09.00.00.log
> : etc.
> 2. No gzip processes were in evidence;
> 3. Figuring it might be the appropriate proverbial kick in the pants, I
> did a "broctl restart", which ran cleanly - and to all appearances,
> *erased* the older uncompressed files in question.
> I now have a hole where the data from 10:00-12:00 today used to be - can
> anyone shed light on what's going on here?
More information about the Bro