[Bro] Revisiting log rotate only

[James Lay]

On 2015-01-20 03:17 PM, Daniel Thayer wrote:
> On 01/20/2015 04:13 PM, James Lay wrote:
>> On 2015-01-20 01:04 PM, Daniel Thayer wrote:
>>> On 01/19/2015 07:57 AM, James Lay wrote:
>>>> On Sat, 2015-01-17 at 07:37 -0700, James Lay wrote:
>>>>> Hey all,
>>>>>
>>>>> I posted about this last August here:
>>>>>
>>>>>
>>>>> 
>>>>> http://mailman.icsi.berkeley.edu/pipermail/bro/2014-August/007329.html
>>>>>
>>>>> I also noticed someone have a disappearing log event which I have 
>>>>> seen
>>>>> before  as well here:
>>>>>
>>>>>
>>>>> 
>>>>> http://mailman.icsi.berkeley.edu/pipermail/bro/2015-January/007935.html
>>>>>
>>>>> I documented my process on installing bro on Ubuntu 14.04 using 
>>>>> just
>>>>> log rotation below:
>>>>>
>>>>> sudo apt-get -y install cmake
>>>>> sudo apt-get -y install python-dev
>>>>> sudo apt-get -y install swig
>>>>> cp /usr/local/bro/share/bro/site
>>>>> cp /opt/bin/startbro <- command line bro with long --filter line
>>>>> cp /opt/bin/startbro to /etc/rc.local
>>>>> sudo ln -s /usr/local/bro/bin/bro /usr/local/bin/
>>>>> sudo ln -s /usr/local/bro/bin/bro-cut /usr/local/bin/
>>>>> sudo ln -s /usr/local/bro/bin/broctl /usr/local/bin/
>>>>> sudo ln -s /usr/local/bro/share/broctl/scripts/archive-log
>>>>> /usr/local/bin/
>>>>> sudo ln -s /usr/local/bro/share/broctl/scripts/broctl-config.sh
>>>>> /usr/local/bin/
>>>>> sudo ln -s 
>>>>> /usr/local/bro/share/broctl/scripts/create-link-for-log
>>>>> /usr/local/bin/
>>>>> sudo ln -s /usr/local/bro/share/broctl/scripts/make-archive-name
>>>>> /usr/local/bin/
>>>>> git clone https://github.com/jonschipp/mal-dnssearch.git
>>>>> sudo make install
>>>>>
>>>>> specifics on log rotate only:
>>>>>
>>>>> add the below to local.bro
>>>>> redef Log::default_rotation_interval = 86400 secs;
>>>>> redef Log::default_rotation_postprocessor_cmd = "archive-log";
>>>>> edit the below in broctl.cfg
>>>>> MailTo = jlay at slave-tothe-box.net 
>>>>> <mailto:jlay at slave-tothe-box.net>
>>>>> LogRotationInterval = 86400
>>>>> sudo /usr/local/bro/bin/broctl install
>>>>>
>>>>> Besides the edits to broctl.cfg, file locations are the default. 
>>>>> The
>>>>> above works well usually...it's after a reboot I have found 
>>>>> things go
>>>>> bad.  Usually logs get rotated at midnight and I get an email 
>>>>> with
>>>>> statistics, just what I need.  I rebooted the machine on the 13, 
>>>>> and
>>>>> that's the last email or log rotation I got....this morning I see
>>>>> current has files and my logstash instance has data so I believe 
>>>>> the
>>>>> rotation got..."stuck".  I'm kicking myself for not 
>>>>> heading/tailing
>>>>> the files first, but after issuing a "sudo killall bro", those 
>>>>> file in
>>>>> current vanished, no directory was created, and I received no 
>>>>> email,
>>>>> that data is now gone (no big deal as this is at home).  I 
>>>>> decided to
>>>>> run broctl install again, then start and kill bro one more time. 
>>>>> At
>>>>> that point, I got a new directory with log rotation and an email 
>>>>> with
>>>>> minutes or so of stats.  Please let me know if there's something 
>>>>> I can
>>>>> do on my end to trouble shoot.  Thank you.
>>>>>
>>>>> James
>>>>> _______________________________________________
>>>>> Bro mailing list
>>>>> bro at bro-ids.org  <mailto:bro at bro-ids.org>
>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>>
>>>> Confirming that this method is no longer working.  Heading my 
>>>> connlog
>>>> file I see:
>>>>
>>>> #open 2015-01-19-00-00-05
>>>>
>>>> my /usr/local/bro/logs is completely missing Jan 18th.  From my
>>>> broctl.cfg:
>>>>
>>>> SpoolDir = /usr/local/bro/spool
>>>> LogDir = /usr/local/bro/logs
>>>> LogRotationInterval = 86400
>>>>
>>>>  From my /usr/local/bro/share/bro/site/local.bro:
>>>>
>>>> redef Log::default_rotation_interval = 86400 secs;
>>>> redef Log::default_rotation_postprocessor_cmd = "archive-log";
>>>>
>>>> Anything else I can do to debug this?  Thank you.
>>>>
>>>> James
>>>
>>> Are you using broctl to start and stop Bro?  What does 
>>> /opt/bin/startbro
>>> do?
>>
>> Thanks for looking Daniel.  I am starting this with the below:
>>
>> /usr/local/bro/bin/bro --no-checksums -i eth0 -i ppp0 --filter '( 
>> large
>> filter line here)' local "Site::local_nets += { 192.168.1.0/24 }"
>>
>> I'm not using broctl.  The only small portion that I am is for the 
>> log
>> rotation as outlined in the email thread.  After killing and 
>> starting
>> bro yesterday, this morning at midnight logs got rotated and I got 
>> my
>> report email.  This appears to happen after a complete reboot of the
>> device.  It's very odd.  Thanks again.
>>
>> James
>
> What command do you use to stop (or restart) Bro?

The classic:  sudo killall bro :) when I have to do it manually.  Then 
start with the command line above.  Thanks again.

James