[Bro-Dev] [Bro-Commits] [git/bro] topic/actor-system: First-pass broker-enabled Cluster scripting API + misc. (07ad06b)

[Siwek, Jon]

> On Nov 3, 2017, at 2:13 PM, Jan Grashöfer <jan.grashoefer at gmail.com> wrote:
> 
> Fully agreed! In that case it might be nice if one can define separate 
> special purpose data nodes, e.g. "intel data nodes". But, I am not sure 
> whether this is a good idea as this might lead to complex cluster 
> definitions and poor usability as users need to know a bit about how the 
> underlying mechanisms work.

I had a similar thought, but also not sure if it’s a good idea.  Example node.cfg:

[data-1]
type = data
pools = Intel::pool

[data-2]
type = data
pools = Intel::pool

[data-3]
type = data

[data-4]
type = data

So there would be two pools here: Cluster::data_pool which is already predefined by the cluster framework (and consists of all data nodes that have not been specifically assigned to other pools) and Intel::pool which is defined/registered by the intel framework.  Then there’s some magic that makes broctl set up those nodes so that they will belong to any pools listed in the config file and the cluster framework will manage it from there.  So this gives users more opportunity to customize, but a problem is it’s hard to say whether the default config file will end up doing something sane for all cases or if you end up with script-writers having more complicated installation instructions like “you should definitely change your node.cfg and don’t scale this pool out to more than N data nodes”.

- Jon