[Bro] Bro cluster requirements and manager logging backlog bug

Hovsep Levi hovsep.sanjay.levi at gmail.com
Fri Jan 6 16:41:32 PST 2017


Actually file rotation does work but it's prone to fail because of a
timestamp collision.  Each rotated file is named based on the timestamp
when the rotation started.. so they are about 10-20 seconds different in
name.  (ex: x509.22:51:59.. x509.22:52:20.. x509.22:52:30).  I guess the
fix would be to change the filenames relative to each logger, ex:
"logger-1_x509..." or something more clever like merging all logger files
into a single zip file.

A cluster-layout for 2 loggers and 8 loggers is attached.  I don't think
there's anything to fix here based on the comments below.

When I configure 8 loggers only 3 loggers are working.  (logger-3,
logger-4, and logger-8).  I restarted the cluster and this time 5 of the
loggers are working.  (2,3,4,6,8).  Still looking into why this happens.
This problem would affect the Kafka export since each logger would be
exporting.  Restarting the failed loggers didn't fix the log flow.  It
looks like they are associating with the assigned logger correctly after
startup and there's nothing indicative in the worker logs stderr or stdout.


>From logger-1/communication.log after restarting logger-1 post-cluster
startup:

1483746743.134338       logger-1        parent  -       -       -
info    [#10005/10.1.1.2:51512] peer sent class "worker-1-8"
1483746743.134338       logger-1        parent  -       -       -
info    [#10005/10.1.1.2:51512] phase: handshake
1483746743.135891       logger-1        child   -       -       -
info    [#10006/10.1.1.3:17887] accepted clear connection
1483746743.137351       logger-1        parent  -       -       -
info    [#10006/10.1.1.3:17887] added peer
1483746743.137351       logger-1        parent  -       -       -
info    [#10006/10.1.1.3:17887] peer connected
1483746743.137351       logger-1        parent  -       -       -
info    [#10006/10.1.1.3:17887] phase: version
1483746743.137351       logger-1        script  -       -       -
info    connection established
1483746743.139263       logger-1        parent  -       -       -
info    [#10006/10.1.1.3:17887] peer sent class "worker-3-12"
1483746743.139263       logger-1        parent  -       -       -
info    [#10006/10.1.1.3:17887] phase: handshake
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170107/367e83fc/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cluster-layout__2-loggers.bro
Type: application/octet-stream
Size: 49091 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170107/367e83fc/attachment-0001.obj 


More information about the Bro mailing list