[Bro] log streams in a bro cluster
Luis Martin Liras
martin.liras at gmail.com
Thu Jun 9 05:16:01 PDT 2016
A little bit of investigation and I found (with a tcpdump) that the logs
arrive to the manager process BUT they are not stored to disk. Then I
found the following entry at the beginning of the communication.log file :
1465472892.006482 manager parent - - - error
[#10002/192.168.1.10:57322] unserializing event/function
Notice::cluster_notice: write error for creating writer
followed by a lot of errors like:
1465473767.549373 manager parent - - - error
[#10001/192.168.1.10:57322] unserializing event/function
Notice::cluster_notice: write error for log entry
It seems that for any reason, the deserializer writer is not able to
open a writer ad then it's not possible to write the log files.
I found this error in github in RemoteSerializer.cc, in its function
But I still cannot figure out why is this happening.
Tried to change permissions to all log directories to 777, but didn't work.
Any idea why could be happening this error?
On 08/06/16 14:36, Azoff, Justin S wrote:
>> On Jun 8, 2016, at 4:43 AM, Luis Martin Liras <martin.liras at gmail.com> wrote:
>> Hi all,
>> I need some help with the logs generated by a Bro Cluster:
>> I have 5 bro scripts that run in all workers of my cluster
>> infrastructure. All of them work OK, sending notices to the manager and
>> all the staff, but one of them should create a LOG stream (warnings.log)
>> that I can't find anywhere:
>> Log::create_stream(umas::WARN, [$columns=warn_info,$path="warnings"]);
>> If I run my script in a single bro installation, all logs and notices
>> seem to work, but I need it working in a cluster infrastructure.
>> I expected this Log stream to be sent to the 'logs' directory in the
>> manager, but that log file is not there. Only standard log files
>> (dns.log, http.log, stdout.log, etc) are copied to the 'logs' directory.
>> This warnings.log file do not appear either anywhere in the worker, and
>> not error log file is shown, so... I'm lost.
>> I anyone can shed some light into this, I would appreciate it.
> When are you writing to that log? Just creating the log stream doesn't create the file until you do a
> Log::write(umas::WARN, record);
>> The other problem I have is the following: My script should open a
>> config file. In a single machine infrastructure this config file is in
>> the same directory of the scripts, and everything work fine. The file is
>> opened and read. However in a cluster infrastructure the file is not
>> opened in the workers. I find that the file is copied by broctl to the
>> worker BUT it is not read when the bro script is running. Anyone can
>> tell me what I'm doing wrong or where should I locate that file in the
>> Thank you for any help!!
> How are you loading the configuration file?
> You should be using something like
> local config_path = fmt("%s/my-config.something", @DIR);
> otherwise a relative or absolute path may not be what you expect.
More information about the Bro