[Bro] Cluster state synchronization

[anthony kasza]

I think I understand the question now. What does the &synchronize attribute
do for values in a single node cluster? Someone else on the list will have
to jump in on this one...
I'm curious if changing the priority of the event where you add to the
whitelist set would make any difference.

-AK
On Oct 7, 2014 8:56 AM, "Damian Gerow" <damian.gerow at shopify.com> wrote:

> I think I may have mis-spoken when I used the term 'synchronization': our
> cluster is a single-node cluster, with one each of manager, proxy, and
> worker.  So the sets are not tagged with '&synchronized'.
>
> What we're seeing is:
>
> 1. DNS lookup is performed.  Query matches whitelist.  Bro is told to
> place the query response into the 'whitelisted_ips' set.
> 2. Connection is established to the IP address.  Connection is verified
> against the 'whitelisted_ips' set.  No match is found, so a notice is
> raised.
> 3. I confirm that the destination IP exists in the 'whitelisted_ips' set.
>
> I've just done a quick test, and I have confirmed that one DNS name that
> was present in a whitelist took upwards of 10 seconds to show up in the
> whitelisted_ips set.  While this was happening, multiple DNS lookups were
> performed and recorded by Bro in the DNS log.
>
> Why does it take so long for a set to be updated, if that set is not set
> for synchronization?
>
> On Mon, Oct 6, 2014 at 3:54 PM, anthony kasza <anthony.kasza at gmail.com>
> wrote:
>
>> I'm not sure about forcing synchronization. In reply to your question
>> about sleep, you may want to look at scriptland's suspend_processing() and
>> continue_processing().
>>
>> -AK
>> On Oct 6, 2014 11:06 AM, "Damian Gerow" <damian.gerow at shopify.com> wrote:
>>
>>> I'm having some troubles wrapping my head around synchronization of set
>>> values in a cluster.
>>>
>>> We use a relatively simple bro script that correlates sets of
>>> whitelisted/blacklisted DNS names with new connections.  To accomplish
>>> this, we have sets that are just the IP addresses returned by DNS lookups,
>>> which we then use to check against new connections.
>>>
>>> i.e. Host "foo.internal" looks up "blacklist.example.com", and receives
>>> response "10.0.0.1".  Bro then adds IP address "10.0.0.1" to the set named
>>> "blacklisted_ips".  "foo.internal" then proceeds to contact "10.0.0.1" on
>>> TCP/443.  Bro looks up "10.0.0.1" in "blacklisted_ips" and, as there is a
>>> match, raises a notice.
>>>
>>> After migrating from a standalone to a single-node cluster configuration
>>> (manager, proxy, worker), it now appears as though the sets containing IP
>>> addresses are updated after the TCP connection is initialized.  As a
>>> result, our notice log is now growing with entries that should never have
>>> been raised in the first place, and is missing entries that should have
>>> been raised.
>>>
>>> Does this theory make sense?  Is there a way to speed up set
>>> additions/removals, or otherwise force synchronization whenever a
>>> modification is made, before processing any further traffic?
>>> Alternatively, does the Bro scripting language have any concept of a
>>> 'sleep'?
>>>
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20141007/783785ab/attachment.html