feat: Null coalesce operator

[Kamirus]

Closes #5144

Reduces verbosity of common expressions like

switch(opt) {
  case (null) { defaultOrTrap(...) };
  case (?v) { v };
};

To just

opt ?? defaultOrTrap(...)

The operator ?? is commonly used in other mainstream languages like TS/JS and C# so it is familiar for both humans and AI.

Follow up: @rvanasa @christoph-dfinity we would need to update the grammar for tools like the formatter, lintoko, vscode etc, yes?

[Kamirus]

Having both QUEST and NULLCOALESCE tokens causes ?? to be lexed as NULLCOALESCE, not double QUEST.
Just adding missing parsing where double ?? is allowed.
This slightly changes the region, but that is a small issue

[crusso]

I think you could actually fix-up the inner sloc

[Kamirus]

yes we could set the correct region for the inner one, I'll change that if we decide on going forward with the ?? operator

[github-actions]

Comparing from f24d95b to 4628776:
In terms of gas, no changes are observed in 5 tests.
In terms of size, no changes are observed in 5 tests.

[crusso]

Since this is a bit like a conditional, we could also consider allowing a block on the rhs (like an else branch).

| e1=exp_bin(B) NULLCOALESCE e2=exp_nest

Maybe a bad idea

[Kamirus]

Oh! Maybe allowing a block on the left? so that

do ? {
} ?? default

Would parse properly without a parenthesis?
Not allowing a block on the RHS would parse records better yes? Without extra brackets

[crusso]

FTR this and the examples below could all be written already using let-else

Suggested change

	let canister = testCanister ?? Prim.trap("null canister");
	let ?canister = testCanister else Prim.trap("null canister");

[crusso]

Looks nice enough to me - lets see what the others think.

Are there any other (commonly used) symbols for ?? that avoid the parser hacks? Perhaps e !/ e(since we already use! indo ? { ... e ! ... }).

It would be nice if we could get notation that works for any binary sum...

[ggreif]

Parser hacks also have a human aspect.

I’d call it ?! because ! is a postfix operator, so not an ambiguity.

(! is “extract” and ? reads as “option” so ?! “optionally extract”)

OTOH @christoph-dfinity has a point in saying that it is a short-circuit operator and should have name like and/or. To be consistent. (But I see consistency being sacrificed on the AI altar).

[alexandru-uta]

Are there any other (commonly used) symbols for ?? that avoid the parser hacks? Perhaps e !/ e(since we already use! indo ? { ... e ! ... }).

Since we try to help an AI write more concise code we should stick to ?? because then we can take advantage of the base model knowing JS/TS.

[Kamirus]

TODO:
Should we allow blocks on the LHS or RHS?

Examples:
RHS without block:

dailyLogs.get(caller) ?? do {
  dailyLogs.add(caller, Map.empty<Nat, DailyRecord>());
  getDailyLogs(caller);
};

RHS without block vs with block

foo.bar(qux) ?? { x = 0; z = null };

vs

foo.bar(qux) ?? { { x = 0; z = null } };

LHS with block:

do ? {
  ...
} ?? default

Does it work?

[christoph-dfinity]

Just writing it up before I forget. Maybe we could disambiguate ?? in the lexer, like we disambiguate >> (but by trailing whitespace, rather than leading). So opt ?? 4 would parse as null-coalesce, but opt ??4 wouldn't.

[Kamirus]

Just writing it up before I forget. Maybe we could disambiguate ?? in the lexer, like we disambiguate >> (but by trailing whitespace, rather than leading). So opt ?? 4 would parse as null-coalesce, but opt ??4 wouldn't.

That's a good idea, I was not aware of the >> disambiguation

[christoph-dfinity]

I was not aware of the >> disambiguation

Right sorry, I should've linked it. It's a bit messy but the logic is here:

motoko/src/mo_frontend/lexer.ml

Lines 49 to 53 in f24d95b

	| Ok Parser.GT
	when opt_is_whitespace (Lib.List.hd_opt (acc @ List.rev !last_trailing))
	&& first (peek ()) = ST.GT ->
	let _, _, end_ = next () in
	(acc, (Parser.SHROP, start, end_))

and a bit more over here:

motoko/src/mo_frontend/lexer.ml

Lines 74 to 78 in f24d95b

	let token =
	match token with
	| Parser.GT when leading_ws () && trailing_ws () -> Parser.GTOP
	| Parser.LT when leading_ws () && trailing_ws () -> Parser.LTOP
	| _ -> token

[Kamirus]

I was not aware of the >> disambiguation

Right sorry, I should've linked it.

Thanks! No worries I would have found it 😄 I meant that I was not aware of that mechanism when I first added the ?? token 😅