[Bro-Dev] 'async' update and proposal

[Robin Sommer]

Jan wrote:

> First of all, this async keyword reminds me of asynchronous programming
> in C#:

Nice, didn't know that.

> For the C# async paradigm, people say that async is like a zombie
> plague as a single asynchronous function can start "infecting" your
> code base by propagating async through the call graph.

That's exactly my point. If we require a keyword to be used for all
asynchronous behavior, it will need to be put in place across the
whole call stack whenever there's even a the slightest chance of
"async" being used somewhere far down inside a framework. I hear you
all on the advantages of making asynchronous behavior explicit, I
fully agree with that. I just don't see it as practical from the
script writer's perspective.

Johanna wrote:

>  And if a user creates a function that in turn calls an asynchronous
>  function, I think we should require that function to be called using
>  async too. Either a user knows that a function uses an asynchronous
>  function, or the script interpreter will raise an error message
>  telling them that the async keyword is required here because an async
>  function is in the call stack.

The problem is that the interpreter cannot determine that statically
(because control flow isn't static), we'd have to resort to runtime
errors -- and that means that code that forgets to use "async" may run
fine for a while until it happens to go down that one branch that
does, e.g., a DNS lookup.

If we required that all the relevant functions (and function
delcarations) get declared as "async", like in C#, then I believe we
could detect errors statcially. But we'd end up having to put that
async declaration into a lot of places just on the chance that
asynchronous behavior could be used somewhere. Consider for example
the plugin functions in NetControl: They'd need to be "async" just so
that someone *could* do DNS lookups in there. Same for hooks: by
definition we don't know what they'll do, so they'll need to be
"async". And that in turn means that NOTICE() for example must become
"async" because it's running a hook. Now everytime we do a NOTICE, we
need to put an "async" in front. And everytime we call a function that
might generate a NOTICE, we'd write "async" there, too.

The point of dependencies/order becoming harder to understand is valid
of course. We already have that challenge with "when" and maybe we
need to find different solutions there to expresse sequentiality
requirements somehow.

Justin wrote:

> event protocol_event_1(...) &priority=1
> event protocol_event_1(...)

> Currently the 2nd event handler is guaranteed to be ran only after the
> first finishes running, right?

Correct, and that's actually something we could ensure even with
"async": we could treat the whole set of all handlers as one block
that gets suspended as a whole if an asynchronous function runs. But
as you point out, that wouldn't solve inter-event dependencies. Per
Jan's mail, one can work around that with custom code, yet it would be
much nicer if we had built-in support for that. Actually, I think one
possible solution has been floating around for a while already: event
*scopes* that express serialization requirements in terms of shared
context. Most common example: serialize all events that are triggered
by the same connection. Originally this came up in the context of
running event handlers concurrently. I believe it would solve this
problem here too: when a function suspends, halt all handlers that
depend on the same context (e.g., same connection). More on that idea
in this paper: http://www.icir.org/robin/papers/ccs14-concurrency.pdf

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin