[Bro-Dev] 'async' update and proposal

Robin Sommer robin at icir.org
Fri Jan 26 11:40:03 PST 2018


A while ago we've been discussing adding a new "async" keyword to run
certain functions asynchronously; Bro would proceed with other script
code until they complete. Summary is here:
https://www.bro.org/development/projects/broker-lang-ext.html#asynchronous-executions-without-when

After my original proof-of-concept version, I now have a 2nd-gen
implementation mostly ready that internally unifies the machinery for
"when" and "async". That simplifies the code and, more importantly,
makes the two work together more smoothly. The branch for that work is
topic/robin/sync, I'll clean that up a bit more and would then be
interested in seeing somebody test drive it.

In the meantime I want to propose a slight change to the original
plan. In earlier discussions, we ended up concluding that introducing
a new keyword to trigger the asynchronous behaviour is useful for the
script writer, as it signals that semantics are different for these
calls. Example using the syntax we arrived at:

    event bro_init()
        {
        local x = async lookup_hostname("www.icir.org"); # A
        print "IP of www.icir.org is", x;                # B
        }

Once the DNS request is issued in line A, the event handler will be
suspended until the answer arrives. That means that other event
handlers may execute before line B, i.e., execution order isn't fully
deterministic anymore. The use of "async" is pointing out that
possibility.

However, look at the following example. Let's say we want to outsource
such DNS functionality into a separate framework, like in this toy
example:

    # cat test.bro
    @load my-dns

    event bro_init()
        {
        local x = MyCoolDNSFramework::lookup("www.icir.org"); # A
        print "IP of www.icir.org is", x;                     # B
        }

    # cat my-dns.bro
    module MyCoolDNSFramework;

    export {
        global lookup: function(name: string) : set[addr];
    }

    function lookup(name: string) : set[addr] {
        local addrs = async lookup_hostname(name); # C
        return addrs;                              # D
    }

That example behaves exactly as the 1st: execution may suspend between
lines A and B because the call to MyCoolDNSFramework::lookup()
executes an asynchronous function call internally (it will hold
between C and D). But in this 2nd example that behaviour is not
apparent at the call site in line A.

We could require using "async" in line A as well but that would be
extremely painful: whenever calling some function, one would need to
know whether internally the callee may end up using "async" somewhere
(potentially buried further deep inside its call stack).

I think we should instead just skip the "async" keyword altogether.
Requiring it at some places, but not others, hurts more than it helps
in my opinion. The 1st example would then just go back to look like
this:

      event bro_init()
        {
        local x = lookup_hostname("www.icir.org"); # A
        print "IP of www.icir.org is", x;          # B
        }

This would still behave the same as before: potential suspension
between A and B.

I don't think skipping "async" this would be a big deal for anything,
as the cases where the new behaviour may actually lead to significant
differences should be rare.

Thoughts?

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin


More information about the bro-dev mailing list