New Principle: Use [Exposed=*] to make only purely computational APIs available

[ptomato]

Problem statement

In webidl#526 a new Exposed annotation, [Exposed=*], was introduced. It denotes a fundamental set of interfaces that are intended to be exposed in Window, all Workers, all Worklets, ShadowRealm, as well as any future global scopes.

We've heard concerns (e.g., tc39/proposal-shadowrealm#401) that it's not clear when an interface should belong to this fundamental set and when it shouldn't. We'd like to issue guidance for this in a design principle.

I'm interested in feedback on the proposed guidance below, and am willing to write up the conclusions in a PR if it is generally positive.

Proposed guidance

[Exposed=*] should be applied only to purely computational interfaces. That is, they do not perform I/O and do not affect the state of the user agent or the user's device.

Anything annotated with [SecureContext] should not be exposed everywhere; not all global scopes are secure contexts.

Anything relying on an event loop should not be exposed everywhere; not all global scopes have an event loop.

The [Exposed=*] annotation should also be applied conservatively. If an interface is not that useful without other interfaces that are not exposed everywhere, default to not exposing that interface as well.

Further reading

Directly related, but predates [Exposed=*]: #35

Discussion resulting in the addition of [Exposed=*]: webidl#468

Other relevant reading: #325, #360, #448, tc39/ecma262#1120, WebAudio/web-audio-api#2499, tc39/proposal-shadowrealm#398

[zcorpan]

cc @smaug---- @mgaudet

[annevk]

This sounds reasonable. I'm not sure if it needs to be a principle or could just be advice in the Web IDL standard. It might also be helpful to have some examples so people can evaluate whether they understand the rules correctly.

[ptomato]

As it so happens, I do have some examples to propose.

The TextEncoder interface converts a string to UTF-8 encoded bytes. This is a purely computational interface, generally useful as a JS language facility, so it should be exposed everywhere.

localStorage is not purely computational because it affects the state of the user agent, so it should not be exposed everywhere.

The timeout method of the AbortSignal interface relies on an event loop and should not be exposed everywhere.

The Blob interface is purely computational, but Blob objects are primarily used for, or obtained as a result of, I/O. By the principle of exposing conservatively, Blob should not be exposed everywhere.

[mgaudet]

I support having guidance for this. I think this is a good piece of guidance.

I think Blob, as described, sits right on the line.

[ptomato]

Given the positive feedback so far, I've gone ahead and drafted a text for this in #510.

[annevk]

(Given the File API discussion in WebApps at TPAC Blob should be out for now as some people think it should maybe count towards a website's quota at some point. That would definitely not make it fit for all realms I think.)

[bakkot]

In the spirit of clarifying edge cases, what about

• OffScreenCanvas
• ImageBitmap
• WebGPU
• WebCodecs
  ?

I'm guessing no, but I'm not sure I could articulate exactly what makes OffScreenCanvas or WebCodecs not pure computation (at least the subset of the APIs which doesn't do any network/on-screen rendering stuff).

[ptomato]

I'm less familiar with the above list, but I believe ImageBitmap's primary use is for drawing on a <canvas>, and OffscreenCanvas's primary use is transferring the contents to an ImageBitmap, so I'd say these are out by the principle of exposing conservatively.

WebGPU and WebCodecs I don't know much about, but they also seem like they lose a lot of their usefulness if you don't have canvases and A/V pipelines respectively. I guess WebGPU can also be used for neural networks, but then there's also WebNN.

[bakkot]

It's certainly true that WebCodecs lose a lot of their usefulness without having somewhere to draw, but on the other hand I use ffmpeg a lot without ever displaying content, and something equivalent could be built on WebCodecs even without the ability to display content (I believe, anyway).

WebNN might be broadly available someday, but there's already a lot of ML built on top of WebGPU now. I would guess a majority of WebGPU consumers are actually ML, since for rendering we already have WebGL.

[ptomato]

Assuming we go forward with this guidance, I suppose the maintainers of WebCodecs, WebGPU, and WebNN are now best positioned to decide whether they want to add [Exposed=*].

[padenot]

Web Codecs isn't "purely computational", it can use lots of system calls, hardware facilities, and generally very little of the operations on Web Codecs have any notion of strictly bounded run time. Besides, it uses the event loop heavily, because almost everything is async in the API. OffscreenCanvas, ImageBitmap, WebGPU, WebNN` can use hardware acceleration underneath, same deal (but sometimes sync). It's perfectly plausible and often desirable to decide to expose them in some context, but they cannot be exposed in all contexts.

When thinking if something could be [Exposed=*], a good test is to ask "can I use it in an AudioWorkletGlobalScope without causing too many problems". The answer is almost always no, and almost always the same as [Exposed=*], this is because it's the most restricted context in the Web Platform as things stand today.

Another way to frame this would be: "is this facility I want to expose only doing pure synchronous computation on the CPU, that has an execution duration that is going to depend solely on its arguments".

[bakkot]

All computation involves hardware and is not usually bounded-runtime, so I would still regard Web Codecs as purely computational under the normal meaning of those words. But "pure synchronous computation on the CPU, that has an execution duration that is going to depend solely on its arguments" sounds like a reasonable elaboration of @ptomato's proposed rule to me.

Though that also suggests that perhaps ShadowRealm should be less restrictive than AudioWorkletGlobalScope, because the realtime requirements of audio worklets don't cleanly apply to ShadowRealm - there's no reason, for example, that CompressionStream or WebCrypto couldn't be used in a ShadowRealm, even though those APIs are mostly async. (Exposed=* and Exposed=Nonrealtime, maybe?)

[padenot]

The normal meaning of those words on the web platform is as such: "hardware" means "hardware accelerated". The opposite on the web platform and in general is "in software", meaning, running on the CPU. This is how the words are used for canvas, gl, web codecs, and more. Here we're interested in workloads that are real-time safe-ish, and so we are interested in somewhat of a bounded run time.

As things stand, Web Codecs doesn't tick the boxes for exposure everywhere, simply because it isn't sync and generally allocates (but some folks now would like a real-time safe API, see w3c/webcodecs#19 w3c/media-production-workshop#45 w3c/webrtc-encoded-transform#226).

If an API isn't synchronous, it follows that it can't have a bounded run time (because the control will go back to the event loop or at the very least a microtask checkpoint, and that is unbounded). If an API allocates, it's the same deal: needing to allocate memory can be unbounded (and generally is).

[bakkot]

Right. My point is that "in hardware" vs "in software" was not originally part of the proposed rule, which was just about whether an API was purely computational, which I understand to mean something like "this API does not require any capabilities other than computation and memory to implement" (by contrast to, say, fetch, which requires network access). As you point out, this rule is not sufficient to establish that something is suitable for use in audio worklets, so something stricter is needed for Exposed=*.

If an API allocates, it's the same deal: needing to allocate memory can be unbounded (and generally is).

Hm. TextEncoder is sync, but it allocates memory (in the form of a Uint8Array) to hold the encoded text. However, the buffer (and time taken) is proportional to the size of the input. Does that mean it's not suitable for Exposed=*?

[padenot]

TextEncoder has encodeInto, which is exposable everywhere, since you can use that without making an allocation. The other method allocates.

[bakkot]

OK, cool. If TextEncoder's encode isn't suitable for Exposed=*, that really suggests to me that we need something other than Exposed=*, because I think it would be really unfortunate if you can't use TexEncoder's encode in a ShadowRealm created in a non-worklet context.

Thoughts on something like Exposed=Nonrealtime (name tbd)? And then say that a ShadowRealm gets access to any Exposed=Nonrealtime or Exposed=* API which exists in its creating context, so that a ShadowRealm in a worklet wouldn't get access to APIs which the worklet itself did not have.

[bakkot]

Alternatively, @annevk in this comment suggests maybe we shouldn't be worrying about trying to keep things out of audio worklets? But I don't know if that's currently the consensus opinion.

[annevk]

I guess it shows that tc39/ecma262#1120 is still relevant and we haven't actually figured that out in a way that makes sense to everyone and doesn't create confusion. I don't think audio worklets are actively considered today for new language features (e.g., the new dependencies on the event loop) and they are also not actively considered for web platform features. But the latter is less problematic as there is a way to not expose those in audio worklets.

I'm also not sure if the people developing audio worklets have the ability to maintain a strict boundary as it's easy for new things to sneak in. They would have to be ever vigilant across implementers.