[Bro-Dev] [JIRA] (BIT-1547) broctl sets the same state variables over and over
Justin Azoff (JIRA)
jira at bro-tracker.atlassian.net
Mon Mar 7 07:13:00 PST 2016
[ https://bro-tracker.atlassian.net/browse/BIT-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=24701#comment-24701 ]
Justin Azoff commented on BIT-1547:
-----------------------------------
Before the change: real 1m32.978s
After the change: real 0m6.413s
It set options 1192 times. this is what it sets (count + key, value):
{code}
1 commiting set of configchksum to "8a94f1078550bc5bcfbb78228b673d75"
1 commiting set of confignodechksum to "761eb1e973f7bdbc94120ae522b76db5"
34 commiting set of manager-port to 47760
34 commiting set of manager-port to 47761
34 commiting set of nids-test1a-1-port to 47763
34 commiting set of nids-test1a-2-port to 47764
34 commiting set of nids-test1a-3-port to 47765
34 commiting set of nids-test1a-4-port to 47766
34 commiting set of nids-test1a-5-port to 47767
34 commiting set of nids-test1a-6-port to 47768
34 commiting set of nids-test1a-7-port to 47769
34 commiting set of nids-test1a-8-port to 47770
34 commiting set of nids-test1b-1-port to 47771
34 commiting set of nids-test1b-2-port to 47772
34 commiting set of nids-test1b-3-port to 47773
34 commiting set of nids-test1b-4-port to 47774
34 commiting set of nids-test1b-5-port to 47775
34 commiting set of nids-test1b-6-port to 47776
34 commiting set of nids-test1b-7-port to 47777
34 commiting set of nids-test1b-8-port to 47778
34 commiting set of nids-test2a-1-port to 47779
34 commiting set of nids-test2a-2-port to 47780
34 commiting set of nids-test2a-3-port to 47781
34 commiting set of nids-test2a-4-port to 47782
34 commiting set of nids-test2a-5-port to 47783
34 commiting set of nids-test2a-6-port to 47784
34 commiting set of nids-test2a-7-port to 47785
34 commiting set of nids-test2a-8-port to 47786
34 commiting set of nids-test2b-1-port to 47787
34 commiting set of nids-test2b-2-port to 47788
34 commiting set of nids-test2b-3-port to 47789
34 commiting set of nids-test2b-4-port to 47790
34 commiting set of nids-test2b-5-port to 47791
34 commiting set of nids-test2b-6-port to 47792
34 commiting set of nids-test2b-7-port to 47793
34 commiting set of nids-test2b-8-port to 47794
34 commiting set of proxy-1-port to 47762
{code}
After the change it just does
{code}
1 commiting set of configchksum to "8a94f1078550bc5bcfbb78228b673d75"
1 commiting set of confignodechksum to "761eb1e973f7bdbc94120ae522b76db5"
34 commiting set of manager-port to 47760
34 commiting set of manager-port to 47761
{code}
> broctl sets the same state variables over and over
> --------------------------------------------------
>
> Key: BIT-1547
> URL: https://bro-tracker.atlassian.net/browse/BIT-1547
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: BroControl
> Affects Versions: git/master
> Reporter: Justin Azoff
> Assignee: Daniel Thayer
>
> I happened to notice broctl check on one of our test boxes was slow. traced it to sqlite commits() being very slow. Then noticed that broctl seems to call set_state() with the same key, val over and over again... once for each worker.. so a few thousand sets just to run broctl check.
> Changing set_state to
> {code}
> # Set a dynamic state variable.
> def set_state(self, key, val):
> key = key.lower()
> if self.state.get(key) == val:
> return
> self.state[key] = val
> self.state_store.set(key, val)
> {code}
> Seemed to mostly fix it, aside from this:
> {code}
> Set manager-port to 47760
> Set manager-port to 47761
> Set manager-port to 47760
> Set manager-port to 47761
> Set manager-port to 47760
> Set manager-port to 47761
> Set manager-port to 47760
> Set manager-port to 47761
> Set manager-port to 47760
> Set manager-port to 47761
> Set manager-port to 47760
> Set manager-port to 47761
> {code}
> any idea why that is flipping around like that?
> We should possibly add a way for broctl to update state vars without calling commit where it knows it will be setting a large number of state vars in a loop.
--
This message was sent by Atlassian JIRA
(v7.2.0-OD-03-012#72000)
More information about the bro-dev
mailing list