Skip to content

Add ability to prime URL metrics #1850

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 64 commits into
base: trunk
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
57f9958
Add settings page for Optimization Detective
b1ink0 Jan 23, 2025
71cd706
Add initial admin script and UI elements for URL priming
b1ink0 Jan 27, 2025
da1d275
Implement batch URL priming functionality with REST API endpoint
b1ink0 Jan 27, 2025
bfed250
Include breakpoints in URL processing
b1ink0 Jan 27, 2025
6ec82e3
Add debug mode and better dynamic height generation for breakpoints
b1ink0 Jan 28, 2025
af0172e
Add verification token auth for protecting REST API, Add message pass…
b1ink0 Jan 29, 2025
bff9973
Refactor to improve control flow and add support for pause and resume…
b1ink0 Jan 29, 2025
c155fc1
Move batch URL fetching logic to helper function
b1ink0 Jan 30, 2025
7640ef7
Add support to filter URL whose metrics are already populated
b1ink0 Jan 30, 2025
f05f6c0
Add support to filter breakpoints whose metrics are already populated
b1ink0 Jan 31, 2025
0631daf
Refactor by moving filtering batch URls and computing standard brekpo…
b1ink0 Jan 31, 2025
5d93fdc
Fix sending empty batch URLs caused by filtering the batch URLs
b1ink0 Jan 31, 2025
3118fb3
Save last batch cursor in options table to improve performance
b1ink0 Jan 31, 2025
d8cfa02
Improve verification token handling by using a transient check before…
b1ink0 Feb 3, 2025
8879f60
Add beforeunload event listener to prevent page navigation during pro…
b1ink0 Feb 3, 2025
72b18fd
Add new REST API routes for URL metrics breakpoints and verification …
b1ink0 Feb 3, 2025
f5bba48
Add block editor support for priming URL metrics
b1ink0 Feb 3, 2025
b0422eb
Merge branch 'trunk' into add/url-metrics-priming
b1ink0 Feb 4, 2025
db1fd81
Add missing since and access annotations, Add new JavaScripts to webp…
b1ink0 Feb 4, 2025
63564c4
Remove usage of fetch and Fix URL metrics not priming due to usage of…
b1ink0 Feb 4, 2025
4fd36b6
Merge branch 'trunk' into add/url-metrics-priming
b1ink0 Feb 4, 2025
82a16b5
Fix detect.js getting stuck on 'onLCP' due to IFRAME being offscreen …
b1ink0 Feb 5, 2025
a50f23e
Improve responsiveness of IFRAME in debug mode
b1ink0 Feb 5, 2025
9201c86
Merge branch 'WordPress:trunk' into add/url-metrics-priming
b1ink0 Feb 6, 2025
163d24e
Skip check of already submitted URL metrics
b1ink0 Feb 7, 2025
57d77ea
Merge branch 'trunk' into add/url-metrics-priming
b1ink0 Feb 7, 2025
2bc9f49
Fix `IFRAME` positioning
b1ink0 Feb 7, 2025
d17dede
Add URL priming functionality for classic editor
b1ink0 Feb 7, 2025
97b888f
Refactor to prime URL metrics only when the post is saved
b1ink0 Feb 10, 2025
931861b
Move URL metrics success message to the end of the detection process
b1ink0 Feb 10, 2025
0a7c1f1
Add instructions and important information section to settings page, …
b1ink0 Feb 11, 2025
969f6b5
Pause URL metrics processing when tab visibility chages, Add abort fu…
b1ink0 Feb 11, 2025
c1e245d
Implement task processing with abort functionality and visibility han…
b1ink0 Feb 12, 2025
b7eeef7
Add settings link to plugin action links
b1ink0 Feb 13, 2025
41f5085
Merge branch 'trunk' into add/url-metrics-priming
b1ink0 Feb 14, 2025
42bfb98
Add minification support for classic editor JavaScript file
b1ink0 Feb 14, 2025
7487ae8
Add function to disable admin-based URL priming feature when reached …
b1ink0 Feb 18, 2025
49f9eab
Merge branch 'trunk' into add/url-metrics-priming
b1ink0 Mar 4, 2025
fed3012
Merge branch 'trunk' into add/url-metrics-priming
b1ink0 Mar 14, 2025
591a013
Merge branch 'trunk' into add/url-metrics-priming
b1ink0 Mar 18, 2025
17e24c4
Refactor REST API endpoints to use class
b1ink0 Mar 18, 2025
32f1e9e
Merge branch 'trunk' into add/url-metrics-priming
b1ink0 Mar 31, 2025
53cc462
Refactor large URL metrics processing function into multiple smaller …
b1ink0 Mar 31, 2025
f9fd604
Refactor to avoid creation of new `AbortController` for each task
b1ink0 Apr 1, 2025
8de014a
Add URL metrics priming CLI
b1ink0 Apr 3, 2025
ce69218
Improve task processing with `AbortController` and improve logging
b1ink0 Apr 3, 2025
7908b89
Refactor to use custom event for communication and improve spinner me…
b1ink0 Apr 3, 2025
3df2f36
Refactor to use global varible to pass verification token to page
b1ink0 Apr 3, 2025
14f8be3
Add environment checks
b1ink0 Apr 4, 2025
fdcb05f
Merge branch 'trunk' into add/url-metrics-priming
b1ink0 Apr 11, 2025
a2ebf17
Update js-lint workflow to install npm packages for optimization-dete…
b1ink0 Apr 11, 2025
3a795e9
Improve communication of response of URL Metric sending request
b1ink0 Apr 11, 2025
4f93e14
Refactor to use URL fragment for passing verification token
b1ink0 Apr 17, 2025
ff6ad0f
Add source property to URL Metric schema
b1ink0 Apr 17, 2025
ec5dcd7
Add auto scrolling for URL Metric priming mode
b1ink0 Apr 21, 2025
2550f96
Add function to force URL Metrics compression when in priming mode
b1ink0 Apr 22, 2025
5e64b39
Refactor to make `source` property readonly
b1ink0 Apr 23, 2025
7d42cac
Fix failing test because of new `source` property
b1ink0 Apr 24, 2025
d21ac27
Improve logging for failed tasks, Improve DOC comment of variables
b1ink0 Apr 25, 2025
3b41a12
Standardize functions and improve event handling in priming helper sc…
b1ink0 Apr 25, 2025
d6a07b1
Refactor URL metrics priming API and scripts for consistency and impr…
b1ink0 May 2, 2025
b201dbe
Improve URL metrics priming UI and CLI parameter handling
b1ink0 May 2, 2025
5c7dca6
Merge branch 'trunk' into add/url-metrics-priming
b1ink0 Jun 2, 2025
d38e5ca
Merge branch 'trunk' into add/url-metrics-priming
b1ink0 Jun 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 37 additions & 1 deletion plugins/optimization-detective/detect.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ const consoleLogPrefix = '[Optimization Detective]';

const storageLockTimeSessionKey = 'odStorageLockTime';

let odPrimeUrlMetricsVerificationToken = '';

/**
* Checks whether storage is locked.
*
Expand All @@ -34,6 +36,10 @@ const storageLockTimeSessionKey = 'odStorageLockTime';
* @return {boolean} Whether storage is locked.
*/
function isStorageLocked( currentTime, storageLockTTL ) {
if ( '' !== odPrimeUrlMetricsVerificationToken ) {
return false;
}

if ( storageLockTTL === 0 ) {
return false;
}
Expand Down Expand Up @@ -365,14 +371,29 @@ export default async function detect( {
return;
}

/** @type {HTMLIFrameElement|null} */
const urlPrimeIframeElement = win.parent.document.querySelector(
'iframe#od-prime-url-metrics-iframe'
);
if (
urlPrimeIframeElement &&
urlPrimeIframeElement.dataset.odPrimeUrlMetricsVerificationToken
) {
odPrimeUrlMetricsVerificationToken =
urlPrimeIframeElement.dataset.odPrimeUrlMetricsVerificationToken;
}

// Abort if the client already submitted a URL Metric for this URL and viewport group.
const alreadySubmittedSessionStorageKey =
await getAlreadySubmittedSessionStorageKey(
currentETag,
currentUrl,
urlMetricGroupStatus
);
if ( alreadySubmittedSessionStorageKey in sessionStorage ) {
if (
'' === odPrimeUrlMetricsVerificationToken &&
alreadySubmittedSessionStorageKey in sessionStorage
) {
const previousVisitTime = parseInt(
sessionStorage.getItem( alreadySubmittedSessionStorageKey ),
10
Expand Down Expand Up @@ -663,6 +684,14 @@ export default async function detect( {

// Wait for the page to be hidden.
await new Promise( ( resolve ) => {
if ( '' !== odPrimeUrlMetricsVerificationToken ) {
window.parent.postMessage(
'OD_PRIME_URL_METRICS_REQUEST_SUCCESS',
'*'
);
resolve();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parent and IFRAME communication is handled via postMessage. A message is sent to the parent, and the promise resolves immediately.

If the promise isn't resolved immediately, navigating to a new URL causes the code following the promise to never execute. This is because changing the iframe.src does not trigger events like pagehide, pageswap, or visibilitychange.

// Wait for the page to be hidden.
await new Promise( ( resolve ) => {

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder. Do we even need to post a message here? As soon as the iframe is destroyed won't it automatically cause the URL Metric to be sent, right?

Copy link
Contributor Author

@b1ink0 b1ink0 Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is we need to signal the parent that we can move to next URL or breakpoint using postMessage as the load event can't be used. Check this comment for detailed explanation #1850 (comment) .

Will it makes sense to send the postMessage after the navigator.sendBeacon then?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think it makes sense to send the message after the beacon is sent, definitely.

}

win.addEventListener( 'pagehide', resolve, { once: true } );
win.addEventListener( 'pageswap', resolve, { once: true } );
doc.addEventListener(
Expand Down Expand Up @@ -807,6 +836,13 @@ export default async function detect( {
);
}
url.searchParams.set( 'hmac', urlMetricHMAC );
if ( '' !== odPrimeUrlMetricsVerificationToken ) {
url.searchParams.set(
'prime_url_metrics_verification_token',
odPrimeUrlMetricsVerificationToken
);
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Authentication for REST API

  • WP Nonce Limitation: The default WordPress (WP) nonce does not function correctly when generated for the parent page and then passed to an iframe for REST API requests.

  • Custom Token Authentication: To address this, I have added a custom token-based authentication mechanism. This generates a time-limited token used to authenticate REST API requests made via the iframe.

In #1835 PR, WP nonces are introduced for REST API requests for logged-in users. This may allow us to eliminate the custom token authentication if URL metrics are collected exclusively from logged-in users.


navigator.sendBeacon(
url,
new Blob( [ jsonBody ], {
Expand Down
Loading
Loading