async-await RFC

[renatoathaydes]

This proposal adds a async/await syntax sugar to Pony in order to make it easier to write asynchronous code via Promises.

[SeanTAllen]

Perhaps Im not finding it. What happens to await as a keyword if called in a function that isn't marked as async? Is that a syntax error?

[SeanTAllen]

The to in translated code (that you say is sugar) still needs to be desugared:

async fun lambdaV(t: T): Promise[V] => ...
async fun lambdaW(v: V): Promise[W] => ...

fun async_fun(): Promise[W] =>
    let promise: Promise[T] = /* obtained from an async call */
    let result: Promise[W] =
        promise.next[V]({(t: T) => lambdaV(t)})
            .next[W]({(v: V) => lambdaW(v)})
    result

An important part of this is "what does async fun lambdaW(v: V): Promise[W]" get desugared to. That isn't clear to me.

[SeanTAllen]

I think this point needs to be elaborated. "presumably" needs to be fleshed out into actual mechanics.

[renatoathaydes]

One may declare an async function and still not use await on anything, or just not ask for an Actor to fulfill the promise, but that would be pointless. If you don't need to have a promise fulfilled by a behaviour, why would you use one? Maybe that should be disallowed? So presumably would become certainly.

[SeanTAllen]

What's the point of async as a keyword then?

[SeanTAllen]

I can only call something marked as async from within an await? Is that the point of the async keyword?

[SeanTAllen]

What happens to the actor that "gives up execution"? Is its stack saved? Does it continue processing messages? If yes, how do you reconcile the stack that was saved with the context of the might have changed (fields for example) due to processing of additional messages.

"give up execution to another actor" is rather vague.

"it simply means to give up execution to another actor, a Promise, until it calls back (or errors, or even never) and executes the remaining of the body of the function (which is really, just a lambda inside the body of the function). This is why async functions are not allowed to return anything but a Promise."

isn't really how Pony works.

I don't think this is purely sugar. With a Promise, I give it a function to execute later when the promise is fulfilled and there is no local state that is saved for me to return to.

Are you suggesting that as part of the sugar, that all state on the stack is captured in a lambda that will be executed when the promise is fulfilled? But, with some sort of partial application?

Imagine for example:

let a: U64 = 1
let b: U64 = 2
let c: U64 = await ... something...
a + b +c

How does the above work?

[SeanTAllen]

From one part of this it might be that I need to do this? It's not clear to me:

let p: Promise[U64] = ... create some promise...
let a: U64 = 1
let b: U64 = 2
let c: U64 = await(a, b) p
a + b + c

[renatoathaydes]

What happens to the actor that "gives up execution"? Is its stack saved? Does it continue processing messages?

I don't understand these questions in light of my proposal.
Let me ask the same questions considering how Pony exists today:

• What happens to an actor that calls next on a Promise, "giving up execution" (because the behaviour ends there) until the lambda it passed into next is called? It its stack saved? Does it continue processing messages?

The answers with my proposal are the same as today.

let a: U64 = 1
let b: U64 = 2
let c: U64 = await ... something...
a + b +c

You need to capture a and b if you need to use them after await (just like today when you create a lambda):

let c: U64 = await(a, b) .... something
a + b + c

Then the whole thing becomes:

let a: U64 = 1
let b: U64 = 2
(some promise).next[T]({(c: U64)(a, b) =>
    a + b + c
})

[SeanTAllen]

What happens to an actor that calls next on a Promise

A promise is an actor. next is an asynchronous message send to it and function execution continues as per normal like it would after any other message send.

[renatoathaydes]

The to in translated code (that you say is sugar) still needs to be desugared:

async fun lambdaV(t: T): Promise[V] => ...
async fun lambdaW(v: V): Promise[W] => ...

fun async_fun(): Promise[W] =>
    let promise: Promise[T] = /* obtained from an async call */
    let result: Promise[W] =
        promise.next[V]({(t: T) => lambdaV(t)})
            .next[W]({(v: V) => lambdaW(v)})
    result

An important part of this is "what does async fun lambdaW(v: V): Promise[W]" get desugared to. That isn't clear to me.

In that case, those functions do not need to be async, they just need to return a Promise. Their contents would be like the one shown - but with an actor actually getting involved in fulfilling the promise - I left that out because that's already how it works today.

[SeanTAllen]

I think it would help make the case for this RFC if you showed in motivation, something that is a pain to do with the existing Promises infrastructure compared to how it would look with the await idea.

[SeanTAllen]

Please show full desugaring. I can't evaluate what desugaring is supposed to be based on this partial desugared example.

[renatoathaydes]

Can't the compiler do that with the above code once you fill in the blanks?

[SeanTAllen]

Please show full desugaring.

[renatoathaydes]

I think it would help make the case for this RFC if you showed in motivation, something that is a pain to do with the existing Promises infrastructure compared to how it would look with the await idea.

I have exactly that on my editor right now :) Will try to re-write it using the proposed syntax and post it here.

[adri326]

As of right now, the lambdas that you give in the Promise::next method have to be sendable.
You already covered how to handle captures. This however breaks the linearity of the written code: a variable that looks to be usable in a part of the function body would actually not be usable.
It would be, in some way, "blocked by the await gatekeeper".

Which is why I am asking you, couldn't the await keyword have a body, in which the lexical scope is different from outside?
Would this vary too much from your conception of what the async/await syntax should be doing/look like?

Instead of doing:

async fun foo(env: Env) =>
  env.out.print("Hey, I'm the env!")
  let meal = await(env) Waiter.serve() // Promise[String]
  env.out.print("Now let's eat our " + meal)

It would look like this:

async fun foo(env: Env) =>
  env.out.print("Hey, I'm the env!")
  await(env) meal from Waiter.serve() then
    env.out.print("Now let's eat our " + meal)
  else
    // on Promise failure
  end

Both would desuggar to this:

fun foo(env: Env) =>
  env.out.print("Hey, I'm the env!")
  Waiter.serve().next[None]({(meal: String)(env) =>
    env.out.print("Now let's eat our " + meal)
  }, {()(env) =>
    // on Promise failure
  })

Which brings me to the next point: how are promise rejections handled?
Should the user be able to handle the error by itself, or will it always propagate the error up the promise chain?
If the user may handle such errors, how could they do it?

[SeanTAllen]

Following along with @adri326. If await is a block (which makes sense) then it would need to have semantics similar to recover in that, in it, you can only access sendable variables from the outer scope.

[SeanTAllen]

As far as I can tell, this is useful for "I am using Promises to do some processing as an actor unto themselves" rather than. I am sending a promise to another actor and it can use the promise to communicate a value back to me. If that is not the case @renatoathaydes, I think it would be wise to show how it can be used in the case of other usages of Promises.

If it is the case, that should be noted in the RFC.

[renatoathaydes]

I like @adri326's proposal very much. It is more consistent with the language and at the same time more powerful as it allows handling a rejected promise (which is normally done by throwing an exception at the point of the await call,which simulates what would happen in a synchronus method - but that option is not available in Pony unless we made all await behave like partial functions, which would not be desirable, I think).
Should I rewrite the RFC to take the better approach into consideration or the procedure is to create another RFC entirely?

[renatoathaydes]

Just would like to ask, wouldn't the new syntax be more consistent with existing constructs if it looked like this?

async fun foo(env: Env) =>
  env.out.print("Hey, I'm the env!")
  await meal = Waiter.serve()(env) then
    env.out.print("Now let's eat our " + meal)
  else
    // on Promise failure
  end

I.e. the captured variables go in the second list of parameters of the call (like in a lambda today) returning a Future, and instead of await meal from ..., await meal = ..., which avoids using yet another keyword and maintains = for all assignments.

[adri326]

Just would like to ask, wouldn't the new syntax be more consistent with existing constructs if it looked like this?

async fun foo(env: Env) =>
  env.out.print("Hey, I'm the env!")
  await meal = Waiter.serve()(env) then
    env.out.print("Now let's eat our " + meal)
  else
    // on Promise failure
  end

I.e. the captured variables go in the second list of parameters of the call (like in a lambda today) returning a Future, and instead of await meal from ..., await meal = ..., which avoids using yet another keyword and maintains = for all assignments.

The await ... (env) then would not work, because the expression could be interpreted as "call Waiter.serve(), then call apply() on the return value"

[SeanTAllen]

From sync:

Adding async/await to Pony is a lot of work that would require some type system changes as well as rewriting a large part of the compiler to support CPS.

That said, our interpretation of this RFC was it's really about making promises easier to use for fork/join type workloads. This we believe can be made easier via a couple means:

1. Pony Pattern(s) that cover different ways to do fork/join type workloads using Pony (both with and without Promises)
2. development of a library to make doing fork/join work distribution easier

If anyone is interested taking on either and needs help, I volunteered to help folks, so please reach out to me on Slack or via email and I can help you with that work.