[Bro-Dev] Broker::publish API

[Azoff, Justin S]

> On Aug 6, 2018, at 3:50 PM, Robin Sommer <robin at corelight.com> wrote:
> 
>    - Relaying is hardly used.
> 
> 
>    - There's a lot of checks in publishing code of the type "if I am
>      (not) of node type X".

I think these 2 are somewhat related.  Since there weren't higher level things like relaying, in order to relay
a message from one worker to all other workers you had to jump through hoops with worker2manger and
manager2worker events and often lots of @if stuff.

There's also a bunch of places that I think were written standalone first and then updated to work on a cluster in
place resulting in some awkwardness.. like notice/main.bro:

function NOTICE(n: Notice::Info)
    {
    if ( Notice::is_being_suppressed(n) )
        return;

    @if ( Cluster::is_enabled() )
        if ( Cluster::local_node_type() == Cluster::MANAGER )
            Notice::internal_NOTICE(n);
        else
            {
            n$peer_name = n$peer_descr = Cluster::node;
            Broker::publish(Cluster::manager_topic, Notice::cluster_notice, n);
            }
    @else
        Notice::internal_NOTICE(n);
    @endif
    }

event Notice::cluster_notice(n: Notice::Info)
    {
    NOTICE(n);
    }

So on a worker, calling NOTICE publishes a cluster_notice event that then re-calls NOTICE on the manager, 
which then does the right thing.  You end up with a single small function with nested @if/if that works 3 different ways.

But if this was written in a more 'cluster by default' way, it would just look like:

function NOTICE(n: Notice::Info)
    {
    if ( Notice::is_being_suppressed(n) )
        return;

    n$peer_name = n$peer_descr = Cluster::node;
    Broker::publish(Cluster::manager_topic, Notice::cluster_notice, n);
    }

event Notice::cluster_notice(n: Notice::Info)
    {
    if ( Notice::is_being_suppressed(n) )
        return;

    Notice::internal_NOTICE(n);
    }

Which other than the suppression check, has no branching at all.

Broker::publish could possibly be optimized for standalone to raise the event directly if not being ran in a cluster.
The only small downside is on a standalone you'd call is_being_suppressed twice, could always add a @if if you
really wanted, but is_being_suppressed is just a set lookup.

Then this stuff would be a good use for efficient relaying/broadcasting instead of making the manager do all the work:

    Broker::auto_publish(Cluster::worker_topic, Notice::begin_suppression);
    Broker::auto_publish(Cluster::proxy_topic, Notice::begin_suppression);


— 
Justin Azoff