Skip to content

Restore the native file menu tests #6279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 35 commits into from
May 14, 2025
Merged

Restore the native file menu tests #6279

merged 35 commits into from
May 14, 2025

Conversation

jacebrowning
Copy link
Contributor

@jacebrowning jacebrowning commented Apr 11, 2025

Issue

Native file menu tests were flaky

Implementation

  • I updated application state to listen to when the menus are created
  • Cleaned up the e2e logic for waiting for the correct application state
  • Tests are cleaner to read
  • setTimeout next tick because you can navigate to a settings path instantly and the scroll to view code will not run which will fail the E2E test.

Testing


  5 failed
    [chromium] › e2e/playwright/native-file-menu.spec.ts:258:11 › Native file menu › Modeling page › File Role › Modeling.File.Create project @electron 
    [chromium] › e2e/playwright/native-file-menu.spec.ts:1126:11 › Native file menu › Modeling page › Design role › Modeling.Design.Create an additive feature.Revolve @electron 
    [chromium] › e2e/playwright/native-file-menu.spec.ts:1170:11 › Native file menu › Modeling page › Design role › Modeling.Design.Create an additive feature.Loft @electron 
    [chromium] › e2e/playwright/native-file-menu.spec.ts:1126:11 › Native file menu › Modeling page › Design role › Modeling.Design.Create an additive feature.Revolve @electron 
    [chromium] › e2e/playwright/native-file-menu.spec.ts:1148:11 › Native file menu › Modeling page › Design role › Modeling.Design.Create an additive feature.Sweep @electron 
  1 interrupted
    [chromium] › e2e/playwright/native-file-menu.spec.ts:501:11 › Native file menu › Modeling page › File Role › Modeling.File.Preferences.Sign out @electron 
  397 did not run
  307 passed (46.0m)

I ran yarn test:playwright:electron:ubuntu:local --workers 1 --quiet ./e2e/playwright/native-file-menu.spec.ts --repeat-each="10" but then ended it after ~307` tests. The 5 failures came from

  5) [chromium] › e2e/playwright/native-file-menu.spec.ts:1148:11 › Native file menu › Modeling page › Design role › Modeling.Design.Create an additive feature.Sweep @electron 

    Error: Timed out 15000ms waiting for expect(locator).not.toBeDisabled()

    Locator: getByRole('button', { name: 'Start Sketch' }).or(getByRole('button', { name: 'Edit Sketch' }))
    Expected: not disabled
    Received: disabled
    Call log:
      - expect.not.toBeDisabled with timeout 15000ms
      - waiting for getByRole('button', { name: 'Start Sketch' }).or(getByRole('button', { name: 'Edit Sketch' }))
        16 × locator resolved to <button disabled id="sketch" tabindex="-1" name="Start Sketch" data-testid="sketch" aria-description="Start drawing a 2D sketch" class="action-button p-0 m-0 group mono text-xs leading-none flex items-center gap-2 rounded-sm border-solid border border-chalkboard-30 hover:border-chalkboard-40 enabled:dark:border-chalkboard-70 dark:hover:border-chalkboard-60 dark:bg-chalkboard-90/50 text-chalkboard-100 dark:text-chalkboard-10 pr-2 pressed:!text-chalkboard-10 pressed:enabled:hovered:!text-chalkboard-10 !…>…</button>
           - unexpected value "disabled"

Which have nothing to do with the tests I implemented. This means the engine could not connect with 15 seconds.

@jacebrowning jacebrowning self-assigned this Apr 11, 2025
Copy link

qa-wolf bot commented Apr 11, 2025

QA Wolf here! As you write new code it's important that your test coverage is keeping up.
Click here to request test coverage for this PR!

Copy link

vercel bot commented Apr 11, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
modeling-app ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 14, 2025 0:27am

Copy link

codecov bot commented Apr 11, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.97%. Comparing base (22dd4a6) to head (6eaf31e).
Report is 67 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6279      +/-   ##
==========================================
+ Coverage   84.94%   84.97%   +0.02%     
==========================================
  Files         108      108              
  Lines       45568    45568              
==========================================
+ Hits        38708    38720      +12     
+ Misses       6860     6848      -12     
Flag Coverage Δ
rust 84.97% <ø> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…the scroll to view code will not trigger which breaks end to end tests
Copy link
Contributor

@pierremtb pierremtb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LFG

@pierremtb pierremtb merged commit edb4249 into main May 14, 2025
46 checks passed
@pierremtb pierremtb deleted the restore-file-menu-tests branch May 14, 2025 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants