[Bro] BROKER + CLUSTER - stuck (Mike Dopheide)
dnthayer at illinois.edu
Wed Mar 8 14:03:57 PST 2017
When "broctl deploy" hangs, were there any bro processes
running (before you ran "deploy")?
Also, when you ran "stop+check+install+start", which
command hangs? Also, in this case, did you verify that
all bro processes were stopped by the "stop" command?
On 3/8/17 3:52 PM, fatema bannatwala wrote:
> Cool! Thanks Justin for the explanation :)
> Tried using stop first+deploy but hangs and never completes.
> Also tried doing stop+check+install+start, but hangs again.
> Hence, ended up doing install+restart, to bring back the cluster up. :(
> The manager doesn't look overloaded though.
> I think next will try out the check.bro change you suggested at the
> beginning, and see if that helps..
> On Wed, Mar 8, 2017 at 3:56 PM, Azoff, Justin S <jazoff at illinois.edu
> <mailto:jazoff at illinois.edu>> wrote:
> > On Mar 8, 2017, at 3:39 PM, fatema bannatwala <fatema.bannatwala at gmail.com <mailto:fatema.bannatwala at gmail.com>> wrote:
> > Hmm, looks like the manager is also running with low memory:
> > $ free -g
> > total used free shared buff/cache available
> > Mem: 70 9 46 0 14 60
> > Swap: 7 4 3
> That's fine.. you have 46G of ram free.
> > $ top
> > top - 15:30:04 up 47 days, 22:33, 8 users, load average: 1.26, 1.25, 1.37
> > Tasks: 495 total, 2 running, 491 sleeping, 2 stopped, 0 zombie
> > %Cpu(s): 2.9 us, 1.7 sy, 0.4 ni, 94.9 id, 0.1 wa, 0.0 hi, 0.0 si, 0.0 st
> > KiB Mem : 73949688 total, 48927592 free, 9836872 used, 15185224 buff/cache
> > KiB Swap: 8388600 total, 4192868 free, 4195732 used. 63369176 avail Mem
> > Anyways, not going into that rabbit hole :)
> > So the correct sequence to deploy any config changes in a cluster would be:
> > stop -> check -> install -> start
> > I was looking at the cmds available and looks like "restart --clean" would do the trick?
> > or I can just script the above sequence in my restart-bro script :)
> Yeah.. deploy basically automates check+install+restart.
> stop+check+install+start is basically the same as stop+deploy.
> The reason why deploy does check first is because if you
> accidentally broke your configuration, you need to start it again
> with the old configuration, fix the configuration, and then retry -
> effectively wasting a restart.
> Doing check first means that you don't stop bro unless you know the
> new configuration will work.
> The worst thing you can do is stop+install+check . If you do that,
> you can end up with a broken bro installation that needs to be fixed
> before you can start it up again. A lot of people were doing this
> which is why added deploy.
> - Justin Azoff
> Bro mailing list
> bro at bro-ids.org
More information about the Bro