Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid setTimeout in waitOnElement #321

Merged

Conversation

camillobruni
Copy link
Contributor

waitOnElement uses a 50ms timeout, which was historically motivated by FlightJS that used requirejs which had a hard-coded 50ms pause.

We've observed fragile CPU scaling behavior with this approach in Speedometer 2.1. V3.0 is less affected by this since the perf-dashboard is the only workload the regularly needs another cycle to wait (all other workloads are ready after onload has fired).

Easy-change:
Replacing the 50ms with a rAF callback eliminates a potentially very long break when the element isn't ready yet.

Debatable change:
We can directly resolve the promise if the element is there and avoid another rAF callback. If we're using the rAF-based measurement, this should make no difference except reducing the idle time.

Detailed measurements coming up.

@camillobruni camillobruni requested review from julienw and rniwa October 3, 2023 14:10
@camillobruni
Copy link
Contributor Author

Default numbers on my machine:

browser                               Google Chrome     Safari                   Firefox
version                               117.0.5938.132    17.0 (19616.1.27.211.1)  115.0.3
os                                    macos 14.0 arm64  macos 14.0 arm64         macos 14.0 arm64
device                                Mac14,2           Mac14,2                  Mac14,2
cpu                                   Apple M2 8 cores  Apple M2 8 cores         Apple M2 8 cores
runs                                  10                10                       10
failed runs                           0                 0                        0
Charts-chartjs/total                  58.7 ± 4.0%       59.92 ± 1.5%             99.5 ± 1.2%
Charts-observable-plot/total          43.2 ± 3.8%       51.70 ± 1.5%             61.2 ± 1.9%
Editor-CodeMirror/total               22.7 ± 6.5%       32.84 ± 2.2%             28.7 ± 4.1%
Editor-TipTap/total                   87.15 ± 0.83%     455.5 ± 0.41%            96.8 ± 1.1%
NewsSite-Next/total                   80.8 ± 1.7%       98.17 ± 0.99%            86.44 ± 0.77%
NewsSite-Nuxt/total                   62.6 ± 1.8%       92.9 ± 1.4%              81.7 ± 2.0%
Perf-Dashboard/total                  35.7 ± 4.3%       48.81 ± 0.85%            54.44 ± 1.3%
React-Stockcharts-SVG/total           71.3 ± 2.7%       87.82 ± 0.98%            87.6 ± 2.0%
TodoMVC-Angular/total                 28.19 ± 1.8%      38.61 ± 1.9%             46.41 ± 2.1%
TodoMVC-Backbone/total                23.65 ± 2.0%      27.83 ± 1.6%             29.56 ± 1.5%
TodoMVC-JavaScript-ES5/total          59.2 ± 2.7%       47.2 ± 2.6%              53.69 ± 1.1%
TodoMVC-JavaScript-ES6-Webpack/total  35.20 ± 2.2%      35.49 ± 2.4%             44.65 ± 0.72%
TodoMVC-Lit/total                     16.59 ± 1.8%      17.28 ± 3.3%             17.21 ± 1.7%
TodoMVC-Preact/total                  10.26 ± 5.2%      12.58 ± 6.5%             12.15 ± 2.7%
TodoMVC-React-Complex-DOM/total       35.3 ± 4.4%       60.89 ± 0.90%            48.13 ± 1.3%
TodoMVC-React-Redux/total             31.29 ± 2.8%      36.32 ± 2.5%             45.11 ± 0.97%
TodoMVC-React/total                   28.66 ± 2.7%      29.57 ± 1.8%             40.59 ± 1.4%
TodoMVC-Svelte/total                  9.82 ± 1.9%       10.36 ± 4.8%             11.88 ± 3.9%
TodoMVC-Vue/total                     21.83 ± 2.8%      24.33 ± 3.6%             29.27 ± 1.7%
TodoMVC-WebComponents/total           16.91 ± 2.9%      15.69 ± 5.5%             22.18 ± 1.0%
TodoMVC-jQuery/total                  72.5 ± 1.6%       62.3 ± 2.3%              76.4 ± 1.4%
Score                                 29.61 ± 1.7%      23.90 ± 0.56%            23.24 ± 0.84%

With this rAF PR:

browser                               Google Chrome     Safari                   Firefox
version                               117.0.5938.132    17.0 (19616.1.27.211.1)  115.0.3
os                                    macos 14.0 arm64  macos 14.0 arm64         macos 14.0 arm64
device                                Mac14,2           Mac14,2                  Mac14,2
cpu                                   Apple M2 8 cores  Apple M2 8 cores         Apple M2 8 cores
runs                                  10                10                       10
failed runs                           0                 0                        0
Charts-chartjs/total                  56.3 ± 3.1%       60.24 ± 1.1%             98.6 ± 1.0%
Charts-observable-plot/total          41.55 ± 2.3%      52.13 ± 1.5%             59.49 ± 1.4%
Editor-CodeMirror/total               22.04 ± 2.5%      33.06 ± 1.5%             28.8 ± 4.7%
Editor-TipTap/total                   87.34 ± 0.91%     454.3 ± 0.33%            96.5 ± 1.7%
NewsSite-Next/total                   86.5 ± 2.4%       109.5 ± 1.0%             88.8 ± 1.2%
NewsSite-Nuxt/total                   62.9 ± 2.0%       96.69 ± 0.54%            84.7 ± 1.3%
Perf-Dashboard/total                  34.53 ± 2.4%      49.48 ± 1.6%             54.91 ± 1.3%
React-Stockcharts-SVG/total           71.4 ± 3.2%       89.09 ± 1.1%             87.0 ± 2.3%
TodoMVC-Angular/total                 24.84 ± 2.9%      35.75 ± 1.1%             44.76 ± 1.3%
TodoMVC-Backbone/total                18.72 ± 1.3%      26.46 ± 2.7%             28.81 ± 1.6%
TodoMVC-JavaScript-ES5/total          55.8 ± 2.6%       45.2 ± 4.1%              49.06 ± 1.6%
TodoMVC-JavaScript-ES6-Webpack/total  33.4 ± 3.5%       35.4 ± 4.9%              42.37 ± 0.79%
TodoMVC-Lit/total                     14.68 ± 2.8%      15.93 ± 4.7%             16.94 ± 2.2%
TodoMVC-Preact/total                  9.75 ± 3.9%       13.29 ± 6.5%             11.79 ± 2.6%
TodoMVC-React-Complex-DOM/total       34.8 ± 4.7%       66.0 ± 1.6%              49.03 ± 1.7%
TodoMVC-React-Redux/total             30.24 ± 2.1%      38.26 ± 0.80%            45.06 ± 1.3%
TodoMVC-React/total                   26.24 ± 2.4%      30.8 ± 3.4%              40.19 ± 0.59%
TodoMVC-Svelte/total                  8.49 ± 5.4%       10.46 ± 6.3%             11.44 ± 4.0%
TodoMVC-Vue/total                     19.81 ± 2.3%      26.0 ± 4.8%              28.71 ± 2.2%
TodoMVC-WebComponents/total           14.99 ± 3.4%      15.10 ± 3.6%             20.80 ± 1.6%
TodoMVC-jQuery/total                  59.0 ± 2.0%       58.9 ± 1.9%              72.3 ± 1.9%
Score                                 31.68 ± 1.3%      23.74 ± 0.63%            23.683 ± 0.28%

@rniwa
Copy link
Member

rniwa commented Oct 4, 2023

We've observed fragile CPU scaling behavior with this approach in Speedometer 2.1.

Could you clarity what you mean by this?

@camillobruni
Copy link
Contributor Author

Generally we've seen some slow-downs due to CPU throttling that's caused by the initial load. Slower network does have an impact on the first iteration of the benchmark.

And specifically for FlightJS prepare-phase there is a hard-coded 50ms delay, which is enough to clock the CPU down and cause an initial ramp-up delay on the sync phase.

We recently had a bug where Chrome kept the CPU busy with an elaborate noop-loop (repeating posting of posting idle-tasks that did no work). By removing this we caused regressions on the FlightJS score since we no longer kept the CPU warm.

If possible I'd like to not incentivise a "sunspider-mode" where we do some artificial GC work only to keep the CPU warm.

I see various options here:

  • Using rAF seems a very realistic scenario (we don't want jank)
  • Avoid the 50ms timeout if possible (bascially this is happening in S3 for all workloads except the perf-panel)
  • Have a warmup loop (this has been discarded based on past discussions).

@bgrins
Copy link
Contributor

bgrins commented Nov 7, 2023

I like the option

Avoid the 50ms timeout if possible (bascially this is happening in S3 for all workloads except the perf-panel)

That is, take only the second half of this PR which removes the setTimeout call. That seems like a clear and simple improvement, while the first half (which removes the rAF when the element exists) is less clear to me, and also affects much more content.

} else {
setTimeout(resolveIfReady, 50);
}
let callback = resolveIfReady;
Copy link
Contributor

@bgrins bgrins Nov 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nit, but I'd find it simpler to follow if kept the basic structure in place and just replaced the setTimeout with a rAF

const resolveIfReady = () => {
  const element = this.querySelector(selector);
  if (element) {
    window.requestAnimationFrame(() => {
      resolve(element);
    });
  } else {
    window.requestAnimationFrame(resolveIfReady);
  }
};

Alternatively we could refactor this to only look for the element inside the rAF, which could potentially lead to one less rAF if the element does end up existing after the first rAF is done.

const resolveIfReady = () => {
  window.requestAnimationFrame(() => {
    const element = this.querySelector(selector);
    if (element) {
      resolve(element);
    } else {
      resolveIfReady();
    }
  });
};

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no real benefit to delay the lookup at this point, since the only workload where the content isn't there yet is the perf dashboard.
Eventually I'd like to remove the additional rAF delay here alltogether if it's not needed (but I prefer doing this separate to be able to easily measure the perf impact). WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in the call, we can keep the implementation as you have here

@camillobruni
Copy link
Contributor Author

Latest Numbers

browser              Firefox                Safari                 Google Chrome
Score (setTimeout)   23.36 ± 0.90%          20.39 ± 0.54%          21.81 ± 1.2%
Score (rAF)          23.32 ± 1.3%           20.29 ± 0.57%          21.94 ± 0.70%

Default version (setTimeout-based)

label                                 macos.arm64.local_0    macos.arm64.local_1    macos.arm64.local_2
browser                               Firefox                Safari                 Google Chrome
version                               119.0.1                17.1 (19616.2.9.11.7)  119.0.6045.123
os                                    macos 14.1 arm64       macos 14.1 arm64       macos 14.1 arm64
device                                MacBookPro18,2         MacBookPro18,2         MacBookPro18,2
cpu                                   Apple M1 Max 10 cores  Apple M1 Max 10 cores  Apple M1 Max 10 cores
runs                                  5                      5                      5
failed runs                           0                      0                      0
Charts-chartjs/total                  106.6 ± 1.8%           66.9 ± 2.2%            61.9 ± 2.4%
Charts-observable-plot/total          61.8 ± 1.9%            61.1 ± 1.8%            73.5 ± 2.1%
Editor-CodeMirror/total               25.16 ± 3.5%           37.85 ± 2.0%           27.86 ± 3.5%
Editor-TipTap/total                   105.5 ± 3.5%           505.90 ± 0.19%         110.8 ± 1.3%
NewsSite-Next/total                   93.8 ± 1.5%            118.5 ± 1.7%           106.3 ± 1.4%
NewsSite-Nuxt/total                   87.5 ± 1.5%            108.28 ± 0.53%         85.6 ± 2.5%
Perf-Dashboard/total                  51.72 ± 1.1%           59.31 ± 0.54%          43.2 ± 2.9%
React-Stockcharts-SVG/total           94.4 ± 1.3%            106.97 ± 0.82%         80.3 ± 1.3%
TodoMVC-Angular/total                 42.28 ± 1.4%           44.63 ± 1.4%           35.49 ± 1.5%
TodoMVC-Backbone/total                32.07 ± 1.6%           32.78 ± 1.5%           30.21 ± 0.63%
TodoMVC-JavaScript-ES5/total          46.5 ± 2.3%            52.8 ± 2.2%            67.9 ± 1.6%
TodoMVC-JavaScript-ES6-Webpack/total  42.60 ± 0.94%          42.7 ± 2.4%            56.1 ± 3.3%
TodoMVC-Lit/total                     18.27 ± 1.5%           19.62 ± 2.2%           21.638 ± 0.13%
TodoMVC-Preact/total                  11.87 ± 1.6%           15.85 ± 6.1%           17.05 ± 3.1%
TodoMVC-React-Complex-DOM/total       45.92 ± 1.4%           57.75 ± 0.94%          45.8 ± 3.6%
TodoMVC-React-Redux/total             46.20 ± 1.4%           42.75 ± 1.3%           40.63 ± 1.4%
TodoMVC-React/total                   39.80 ± 1.5%           35.80 ± 1.1%           36.21 ± 2.6%
TodoMVC-Svelte/total                  11.66 ± 4.2%           12.96 ± 4.9%           15.92 ± 4.1%
TodoMVC-Vue/total                     27.99 ± 1.3%           29.94 ± 1.5%           32.12 ± 2.3%
TodoMVC-WebComponents/total           21.13 ± 1.5%           18.73 ± 2.0%           28.41 ± 3.1%
TodoMVC-jQuery/total                  77.13 ± 0.94%          76.56 ± 0.78%          119.3 ± 2.0%
Score                                 23.36 ± 0.90%          20.39 ± 0.54%          21.81 ± 1.2%

New version (rAF-based)

label                                 macos.arm64.local_0    macos.arm64.local_1    macos.arm64.local_2
browser                               Firefox                Safari                 Google Chrome
version                               119.0.1                17.1 (19616.2.9.11.7)  119.0.6045.123
os                                    macos 14.1 arm64       macos 14.1 arm64       macos 14.1 arm64
device                                MacBookPro18,2         MacBookPro18,2         MacBookPro18,2
cpu                                   Apple M1 Max 10 cores  Apple M1 Max 10 cores  Apple M1 Max 10 cores
runs                                  5                      5                      5
failed runs                           0                      0                      0
Charts-chartjs/total                  109.0 ± 2.3%           67.13 ± 0.48%          63.0 ± 2.3%
Charts-observable-plot/total          61.7 ± 3.6%            60.15 ± 1.1%           72.4 ± 2.2%
Editor-CodeMirror/total               25.1 ± 4.9%            37.90 ± 2.0%           27.19 ± 3.2%
Editor-TipTap/total                   107.0 ± 2.0%           505.1 ± 0.21%          110.18 ± 0.79%
NewsSite-Next/total                   95.1 ± 1.8%            125.55 ± 0.75%         108.5 ± 1.1%
NewsSite-Nuxt/total                   86.8 ± 2.0%            109.05 ± 0.89%         83.7 ± 3.1%
Perf-Dashboard/total                  52.0 ± 2.2%            60.00 ± 0.77%          41.3 ± 3.1%
React-Stockcharts-SVG/total           94.8 ± 1.4%            107.78 ± 0.92%         80.1 ± 2.0%
TodoMVC-Angular/total                 42.17 ± 2.3%           43.71 ± 0.98%          35.45 ± 2.8%
TodoMVC-Backbone/total                32.23 ± 3.0%           34.02 ± 2.3%           29.79 ± 2.9%
TodoMVC-JavaScript-ES5/total          46.58 ± 1.7%           53.1 ± 2.1%            69.5 ± 1.9%
TodoMVC-JavaScript-ES6-Webpack/total  43.5 ± 2.8%            43.08 ± 0.69%          56.2 ± 3.8%
TodoMVC-Lit/total                     18.05 ± 1.3%           19.66 ± 2.6%           21.66 ± 0.88%
TodoMVC-Preact/total                  11.67 ± 2.0%           15.9 ± 9.0%            16.76 ± 1.2%
TodoMVC-React-Complex-DOM/total       45.48 ± 1.8%           57.48 ± 0.71%          45.33 ± 1.8%
TodoMVC-React-Redux/total             46.05 ± 1.4%           43.34 ± 0.86%          41.12 ± 1.8%
TodoMVC-React/total                   40.22 ± 1.8%           36.03 ± 0.81%          35.95 ± 1.1%
TodoMVC-Svelte/total                  11.10 ± 2.1%           12.61 ± 3.0%           15.68 ± 0.85%
TodoMVC-Vue/total                     28.07 ± 3.0%           29.47 ± 1.6%           31.65 ± 1.1%
TodoMVC-WebComponents/total           21.79 ± 1.1%           19.29 ± 3.6%           28.05 ± 1.3%
TodoMVC-jQuery/total                  78.6 ± 1.5%            75.6 ± 1.9%            119.1 ± 1.1%
Score                                 23.32 ± 1.3%           20.29 ± 0.57%          21.94 ± 0.70%

@camillobruni camillobruni merged commit 208e1ab into WebKit:main Nov 20, 2023
4 checks passed
@camillobruni camillobruni deleted the 2023-10-03_faster_waitOnElement branch November 20, 2023 09:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants