Header-based DirectFromSellerSignals

[caraitto]

Addresses #119

[dmdabbs]

If directFromSellerSignalsHeaderAdSlot matches multiple adSlots from header responses, one of those adSlot responses will be chosen arbitrarily.

Why not LIFO versus arbitrary?

[caraitto]

I could look into switching to LIFO if there's interest -- the current WIP MVP implementation stores an unordered set of responses, but it could possibly be switched to a queue, maybe even in a later version (switching to a non-arbitrary order from an arbitrary order shouldn't be a breaking change).

[JacobGo]

+1 to switching to a LIFO data structure. Ad techs may choose to define the adSlot as an element on the page containing an ad that may refresh the ad over time. It seems unlikely to want the same exact set of signals when a newer response with the same adSlot is available on the page.

Similarly, have you considered how resource management should work in this flow? Once the ad auction is completed, is there any way to dispose the stale signals, or is this an unavoidable source of memory leaks? We plan on iterating on the amount of data transferable within this header and starting small; do you anticipate any issues when approaching O(kilobytes) of JSON?

[caraitto]

OK, I think we can use a LIFO order [0], where a fetch for adSlot "foo" for fetch / seller origin seller.com would overwrite any previous stored signals for the (seller.com, "foo") key tuple.

In the initial version [1], all server responses are stored until the top-level page is unloaded, and there is no way to otherwise remove old responses from memory; each new fetch() with {adAuctionHeaders: true} that captures new headers will result in additional memory consumption, and we also parse potentially all of those responses in arbitrary order, only stopping if a matching adSlot is found.

Currently, the entire set of responses is re-parsed and searched for each top-level, component auction, or non-component auction on the page using the new directFromSellerSignals. So, if there are many responses, or they are large, this will add to the auction latency.

With the new LIFO order, the signals would only be parsed once, and further lookups would just be map lookups (unless more signals have arrived).

Individual responses greater than 1000 bytes are currently discarded [2]. We could consider increasing this, but headers are generally small, and several server implementations and CDNs impose their own limits. Also, the signals get copied from the browser process to worklet processes.

Do folks think these proposed changes would address all your concerns?

[0] Internally in the Chromium implementation, it would technically be a FIFO queue of the JSON string responses -- we'd process all of them during auctions, overwriting the value signals for each (fetch / seller origin, adSlot string) key tuple as we process the list, then empty the processed FIFO -- this would mean that more recent responses overwrite older responses, or LIFO semantics. The spec already works in a similar manner (except that JSON parsing is synchronous in spec language, so the parsing happens in the FETCH spec patch). The reason for processing the JSON during auctions is to avoid complexity around potential races / waiting in auctions until processing has completed if started in the network interceptor.

Note, this means that the memory for old signals wouldn't be freed until an auction is run in this new proposal.

[1] The initial version was cherry-picked into Chrome Beta 117.0.5938.35, but we'll need approval on an I2S to be able to flip the controlling flag on (currently working on getting approval, which requires we complete the spec).

[2] https://source.chromium.org/chromium/chromium/src/+/main:content/browser/interest_group/ad_auction_url_loader_interceptor.cc;l=166;drc=6f4c64436342c818aa41e6a5c55034e74ec9c6b6

[caraitto]

@JacobGo What do you think of the above proposal? I'll probably make the change in a separate PR.

[JacobGo]

Thanks @caraitto, the LIFO order optimization sounds great!

[caraitto]

Here's a PR to change this for the explainer (the spec already matches this behavior): #890

[caraitto]

FYI: the LIFO change will be available in 121.0.6121.0 and newer: https://chromiumdash.appspot.com/commit/b26d4257d640b9abdef6a75eaf7d0eb1a103367d

[JacobGo]

Could you please elaborate on the rationale for introducing directFromSellerSignalsHeaderAdSlot as opposed to reusing directFromSellerSignals? What happens if both are provided? Would it possible to instead infer which one is in use by examining the Content-Type or looking for the appropriate response headers?

[caraitto]

Could you please elaborate on the rationale for introducing directFromSellerSignalsHeaderAdSlot as opposed to reusing directFromSellerSignals? What happens if both are provided? Would it possible to instead infer which one is in use by examining the Content-Type or looking for the appropriate response headers?

Sure -- both directFromSellerSignalsHeaderAdSlot and directFromSellerSignals take a string as a parameter -- an ad-slot for the former, and a prefix for constructing subresource bundle subresource URLs in the latter case.

While it would be possible to try to to parse directFromSellerSignals as a URL, using the subresource bundle version if the string is a valid URL, and treating it as a header ad-slot otherwise, this seems confusing and fragile. Instead, it's easier to just have separate fields.

Changing the type of the directFromSellerSignals auction config parameter (to take an object specifying which type to use in addition to the prefix / ad-slot) would also be a breaking change.

As mentioned in the PR, It is not valid to provide values for both directFromSellerSignalsHeaderAdSlot and directFromSellerSignals in the same component, top-level, or non-component auction -- in this case, runAdAuction() will throw an error explaining the problem, and no auction will run.

Furthermore, different component / top-level auctions belonging to the same top-level may use different forms of directFromSellerSignals, and it's also not required that every seller in overall auction use a form directFromSellerSignals if some of them do.