[Bro-Dev] Broker data store use case and questions

[Michael Dopheide]

Couple questions.   (Let me know if this isn't appropriate bro-dev content,
but I don't want to cause confusion on the normal bro list.)

First, can Cluster::default_master_node be changed to default to the name
of the current manager node rather than specifying the name as 'manager'?  Easy
to redef to the manager's name, but less easy when you use the same code
base on multiple clusters with different names.

==== stderr.log
fatal error in /usr/local/bro/share/bro/base/frameworks/cluster/./main.bro,
lines 399-400: master node 'manager' for cluster store 'bro/known/certs'
does not exist

Second, when during startup should Bro know that it's persistent stores
exist via Cluster::stores() ?  It appears bro_init may be too soon, but I'm
still playing.  Also, it'd be nice if the persistence of built-in stores
(like known/hosts, known/certs, etc) were redef-able.

Thanks,
-Dop




On Fri, May 11, 2018 at 1:38 PM, Azoff, Justin S <jazoff at illinois.edu>
wrote:

>
> > On May 11, 2018, at 10:13 AM, Jon Siwek <jsiwek at corelight.com> wrote:
> >
> >
> > There's no check against the local cache to first see if the key exists
> > as going down that path leads to race conditions.
>
> What sort of race conditions?
>
> Right now I see a lot of events going around so it seems like there may be
> a bit of overhead in this area.
>
> For example, in about a minute one node in a two node cluster sent 2104
> Software::register events (so likely 4k total)
> In that time, only 7 new entries were logged to software.log.
>
> It's always good to reduce memory usage, but I think especially for things
> like known hosts which are generally kept at LOCAL_HOSTS
> the small amount of memory used for caching already seen hosts saves more
> resources than would be spent sending redundant events around.
> Especially if those events end up queued and buffered in ram anyway.
>
> Things are a bit better off now in that we can use a short lived cache,
> since the cache doesn't need to be the actual data store anymore like the
> old known hosts set was.
>
>
>
> —
> Justin Azoff
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20180511/700d6767/attachment.html