[Bro] archive-log process apparently failing

[Daniel Thayer]

On 10/15/2015 01:29 PM, MILLER, BRAD L wrote:
> We are on Bro 2.3.x and have run into a very occasional process that
> appears to indicate the archive-log process fails.  The symptom we see
> is a logjam (the word kind of fit here) of logs staying in the current
> directory and getting larger and larger, with no rotation into gz files
> outside of this directory.  Broctl restart sets it straight again, but
> this issue came up twice now in recent memory.  We tend to lose logs in
> the logjam when this is corrected via broctl restart.
>
> Anything we can do?  Cause?
>
> Brad Miller | Comerica Bank
>

Are you seeing rotated logs? (rotated logs have a timestamp in the 
filename, such as "conn.2015-10-15-14-42-00.log")
Or, are you just seeing the current logs getting larger and larger? 
(such as "conn.log")

If you don't see any rotated logs (and your logs aren't getting 
archived), then you should check if your
log rotation interval is set to a reasonable value (and you must
do "broctl install" and restart Bro if you change your config).

When logs are archived, they are compressed and moved into a
subdirectory named like this:
<PREFIX>/logs/XXXX-XX-XX

If you don't see the logs being archived, then (after doing a
broctl restart) you can check if there are any directories with
names like this:
<PREFIX>/spool/tmp/post-terminate-XXXX-XX-XX-XX-XX-XX-XXXXX
Those directories are where you can find your "lost" log files
(however, if you do "broctl cleanup --all", then broctl will remove
all of those directories without warning).