Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snapshot service is not initialized #656

Closed
P-Courteille opened this issue Dec 4, 2024 · 20 comments
Closed

Snapshot service is not initialized #656

P-Courteille opened this issue Dec 4, 2024 · 20 comments

Comments

@P-Courteille
Copy link

Environment (please complete the following information):

  • Node.js version: 22.11.0
  • NPM version: 10.9.1
  • Platform name and version: Android 11 (testing on native android app)
  • WebdriverIO version: 9.4.1
  • @wdio/visual-service version: 6.2.2
  • @wdio/mocha-framework version: 9.2.8
  • @wdio/local-runner version: 9.4.1
  • @wdio/globals version: 9.4.1
  • @wdio/cli version: 9.4.1
  • Appium version: 2.11.5
  • tsx version: 4.19.2

Config of WebdriverIO + @wdio/visual-service

services: [
        // your other services
        [
            "ocr",
            {
                contrast: 0.25,
                imagesFolder: "./onboard/output"
            },
        ],
        [
            "visual",
            {
                baselineFolder: path.join(process.cwd(), "onboard", "baseline"),
                formatImageName: "{tag}-{width}x{height}-{dpr}",
                screenshotPath: path.join(process.cwd(), "onboard", "output", "visual"),
                savePerInstance: true,
                clearRuntimeFolder: true,
            },
        ],
    ],

Describe the bug
After my typescript migration, it no longer work.

  • With "toMatchElementSnapshot"
    (The issue is not explicit.)
[0-0]  takeBase64ElementScreenshot element is not a valid element because of  "Login-logoApp"
[0-0] 2024-12-03T16:12:23.611Z INFO @wdio/visual-service:webdriver-image-comparison:images: awaitedElement =  "Login-logoApp"
[0-0] Error taking an element screenshot with the default `element.takeElementScreenshot(elementId)` method: TypeError: awaitedElement.takeElementScreenshot is not a function  
[0-0]     at takeBase64ElementScreenshot (file:///C:/Users/My%20Name/automatisation/MatatestJS/node_modules/webdriver-image-comparison/dist/methods/images.js:452:41)
[0-0]     at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
[0-0]     at async saveAppElement (file:///C:/Users/My%20Name/automatisation/MatatestJS/node_modules/webdriver-image-comparison/dist/commands/saveAppElement.js:14:25)  
[0-0]     at async checkAppElement (file:///C:/Users/My%20Name/automatisation/MatatestJS/node_modules/webdriver-image-comparison/dist/commands/checkAppElement.js:12:57)  We will retry with a resized screenshot
  • With "toMatchSnapshot":
"status": "broken",
"statusDetails": {
   "message": "Snapshot service is not initialized",
   "trace": "Error: Snapshot service is not initialized\n    at __EXTERNAL_MATCHER_TRAP__ (C:\\Users\\My NAME\\automatisation\\MatatestJS\\node_modules\\expect\\build\\index.js:325:30)\n    at Object.throwingMatcher [as toMatchSnapshot] (C:\\Users\\My NAME\\automatisation\\MatatestJS\\node_modules\\expect\\build\\index.js:326:15)\n    at loginPage.checkLoginPage (C:\\Users\\My NAME\\automatisation\\MatatestJS\\onboard\\pageObjects\\mobile\\loginPage.ts:133:23)\n    at async Context.<anonymous> (C:\\Users\\My NAME\\automatisation\\MatatestJS\\onboard\\specs\\login.ts:71:13)\n    at async Context.executeAsync (C:\\Users\\My NAME\\automatisation\\MatatestJS\\node_modules\\@wdio\\utils\\build\\index.js:1004:20)\n    at async Context.testFrameworkFnWrapper (C:\\Users\\My NAME\\automatisation\\MatatestJS\\node_modules\\@wdio\\utils\\build\\index.js:1075:14)"
                 },

💡It work as expected with saveScreen and checkScreen
Only snapshot service seems to be in issue.

Expected behavior
Snapshot service is initialized correctly, "toMatchSnapshot" and "toMatchElementSnapshot" work as expected

Additional context
I'm using typescript and my tsconfig contain:

"types": [
            "node",
            "@wdio/globals/types", 
            "@wdio/mocha-framework",
            "@wdio/appium-service",
            "@wdio/ocr-service",
            "@wdio/visual-service",
            "i18next"
]

Links:

@wswebcreation
Copy link
Member

hi @P-Courteille

I have some time this weekend to look into it, but can you provide a reproducible example, with the way you've setup your complete config, capabilities and a simple test so I can reproduce it?

@P-Courteille
Copy link
Author

Hi @wswebcreation,
I will try to create this example.
The one I did from scratch doesn't reproduce the behavior so I will compare again with my previous MR and if I don't find anything I will try to create an example by removing all the useless things from my project.

Thank you for your help

@P-Courteille
Copy link
Author

Same result than my issue :
According to my following analysis, the issue started with @wdio/local-runner@9.3.1.


Original dependencies

"devDependencies": {
"@moroo/wdio-slack-reporter": "^8.1.0",
"@types/mocha": "^10.0.10",
"@wdio/allure-reporter": "^9.0.8",
"@wdio/appium-service": "^9.1.0",
"@wdio/cli": "^9.0.9",
"@wdio/local-runner": "^9.1.0",
"@wdio/mocha-framework": "^9.1.0",
"@wdio/ocr-service": "^2.2.1",
"@wdio/visual-service": "^6.2.2",
"canvas": "^3.0.0-rc2",
"i18next": "^23.16.3",
"i18next-sprintf-postprocessor": "^0.2.2",
"mocha-tags-ultra": "^1.0.3",
"tsx": "^4.19.2"
},

Not working target dependencies

"devDependencies": {
"@types/mocha": "^10.0.10",
"@types/node": "^22.10.1",
"@wdio/allure-reporter": "^9.2.14",
"@wdio/cli": "^9.4.1",
"@wdio/globals": "^9.4.1",
"@wdio/local-runner": "^9.4.1",
"@wdio/mocha-framework": "^9.2.8",
"@wdio/ocr-service": "^2.2.1",
"@wdio/visual-service": "^6.2.2",
"canvas": "^3.0.0-rc2",
"i18next": "^24.0.0",
"i18next-sprintf-postprocessor": "^0.2.2",
"mocha-tags-ultra": "^1.0.3",
"tsx": "^4.19.2"
}

Step by step update

➤ FIRST update: $${\color{green}working}$$
npm install @wdio/cli@9.1.0 --save-dev
npm install @wdio/allure-reporter --save-dev

➤ SECOND update: $${\color{green}working}$$
npm install @wdio/cli@9.2.0 --save-dev
npm install @wdio/local-runner@9.2.0 --save-dev
npm install @wdio/mocha-framework@9.2.0 --save-dev

➤ THIRD update: $${\color{green}working}$$
npm install @wdio/cli@9.3.0 --save-dev
npm install @wdio/local-runner@9.3.0 --save-dev
npm install @wdio/mocha-framework@9.2.8 --save-dev

➤ FOURTH update: $${\color{red}not~working}$$
npm install @wdio/cli@9.4.0 --save-dev
npm install @wdio/local-runner@9.4.0 --save-dev

➤ FIRST downgrade: $${\color{red}not~working}$$
npm install @wdio/cli@9.3.1 --save-dev
npm install @wdio/local-runner@9.3.1 --save-dev

➤ SECOND downgrade: $${\color{red}not~working}$$
npm install @wdio/cli@9.3.0 --save-dev
npm install @wdio/local-runner@9.3.1 --save-dev

➤ Return to "THIRD update" : $${\color{green}working}$$
npm install @wdio/cli@9.3.0 --save-dev
npm install @wdio/local-runner@9.3.0 --save-dev

➤ FIFTH update : $${\color{red}not~working}$$
npm install @wdio/cli@9.3.1 --save-dev
npm install @wdio/local-runner@9.3.0 --save-dev

@wswebcreation
Copy link
Member

Awesome! I'll take a look

@P-Courteille
Copy link
Author

Hello,

My example are finally ready.

Package

Example_OCR+visual.zip


Examples

The package is for both:
#657 (OCR loop)
#656 (Visual comparison no longer working)


Scripts

Package scripts are available:

  • npm run Snapshot

    • First test with toMatchElementSnapshot $${\color{red}➤\ Fail\ with\ takeBase64ElementScreenshot\ error\ and\ malformed\ elementId\ error}$$
    • Second with toMatchScreenSnapshot $${\color{red}➤\ Fail\ with\ type\ error\ (look\ like\ path\ error)}$$
  • npm run OCR

    • First test with OCR directly using browser $${\color{red}➤\ Loop\ on\ INFO\ webdriver:\ COMMAND\ ocrGetText(&lt;object&gt;)}$$
    • Second with OCR using my multiremotebrowser $${\color{red}➤\ Loop\ on\ INFO\ webdriver:\ COMMAND\ ocrGetText(&lt;object&gt;)}$$
  • npm run OCR_PO

    • test with OCR with pageObject class $${\color{red}➤\ Loop\ on\ INFO\ webdriver:\ COMMAND\ ocrGetText(&lt;object&gt;)}$$
      (it was in case my implementation was in cause but the issue exist without it so I don't think so)

Notes

  • In this example I use a WebdriverIO.MultiremoteConfig for only one device with name but on my real device, I've one wdio.conf.ts with multiple sets of capabilities with one or multiple device depending of the parameters used.

  • You will maybe need to replace my selector by a selector that exists on your device (if it don't):
    //android.widget.TextView[@content-desc="Contacts"]
    (it was the Contacts icon on the main screen of my phone)

  • I didn't reproduce the issue about "Snapshot service is not initialized" but technically my concerne was only about "toMatchElementSnapshot" and the "toMatchSnapshot" is not interesting in my case.

Thank you @wswebcreation and have a nice day,
Paul

@wswebcreation
Copy link
Member

Hi @P-Courteille

I found the issue for the malformed elementId, it was a stupid mistake. I'm now busy with the other issue regarding the undefined error with the custom matcher. This is related to poor multiremote support in the module for the custom marchers. I know how to solve that

I started working on this. I'll update you there

@wswebcreation
Copy link
Member

Hi @P-Courteille
I've been trying to reproduce the OCR loop issue, but I wasn't able to. The PR now only fixes the snapshot one. Please let me know if you have some other options to test the OCR one, otherwise I'll release a new version of the Visual Service beginning next week

@P-Courteille
Copy link
Author

Hi @wswebcreation,
Thank you for your help on that. Maybe I will be lucky and the next version will also fix the OCR issue.

For now I will stay in version 9.3.0 to avoid the issue because I really need to move on my test production after a week of Typescript migration and issue investigation.


Sometime I've the following message at start-up but i've it when it work or not so I don't think it's the origin of the issue:
[0-0] 'tesseract' n'est pas reconnu en tant que commande interne

PS C:\Users\Paul COURTEILLE\Pictures\Example_OCR+visual> npm run OCR > log.log
(node:34596) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
[0-0] (node:29704) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
[0-0] (Use `node --trace-deprecation ...` to show where the warning was created)
[0-0] 'tesseract' n'est pas reconnu en tant que commande interne
[0-0] ou externe, un programme ex�cutable ou un fichier de commandes.

Another log when it fail:

> matatestjs@0.0.1 OCR
> wdio run ./myProject/configs/wdio.conf.ts --spec OCR.ts


Execution of 1 workers started at 2024-12-09T08:11:27.192Z

2024-12-09T08:11:27.243Z DEBUG @wdio/utils:initializeServices: initialize service "ocr" as NPM package
2024-12-09T08:11:28.457Z DEBUG @wdio/utils:initializeServices: initialize service "visual" as NPM package
2024-12-09T08:11:29.148Z INFO @wdio/visual-service:webdriver-image-comparison: �[33m
##############################
!!CLEARING!!
##############################�[0m
2024-12-09T08:11:29.148Z INFO @wdio/cli:launcher: Run onPrepare hook
2024-12-09T08:11:29.149Z DEBUG @wdio/cli:utils: Finished to run "onPrepare" hook in 0ms
2024-12-09T08:11:29.150Z INFO @wdio/cli:launcher: Run onWorkerStart hook
2024-12-09T08:11:29.150Z DEBUG @wdio/cli:utils: Finished to run "onWorkerStart" hook in 0ms
2024-12-09T08:11:29.152Z INFO @wdio/local-runner: Start worker 0-0 with arg: run ./myProject/configs/wdio.conf.ts --spec OCR.ts
2024-12-09T08:11:29.190Z DEBUG @wdio/local-runner: Send command run to worker with cid "0-0"
[0-0] 2024-12-09T08:11:31.543Z INFO @wdio/local-runner: Run worker command: run
[0-0] 2024-12-09T08:11:31.568Z DEBUG @wdio/runner: init multiremote session
[0-0] 2024-12-09T08:11:31.581Z DEBUG @wdio/utils:initializeServices: initialize service "ocr" as NPM package
[0-0] 2024-12-09T08:11:32.082Z DEBUG @wdio/utils:initializeServices: initialize service "visual" as NPM package
[0-0] 2024-12-09T08:11:32.502Z INFO @wdio/visual-service:webdriver-image-comparison: �[33m
[0-0] ##############################
[0-0] !!CLEARING!!
[0-0] ##############################�[0m
[0-0] RUNNING in MultiRemote - file:///C:/Users/Paul%20COURTEILLE/Pictures/Example_OCR+visual/myProject/specs/OCR.ts
[0-0] 2024-12-09T08:11:33.151Z DEBUG @wdio/runner: init multiremote session
[0-0] 2024-12-09T08:11:33.154Z INFO webdriver: Initiate new session using the WebDriver protocol
[0-0] 2024-12-09T08:11:33.155Z INFO @wdio/utils: Connecting to existing driver at http://127.0.0.1:4723/wd/hub
[0-0] 2024-12-09T08:11:33.157Z INFO webdriver: [POST] http://127.0.0.1:4723/wd/hub/session
[0-0] 2024-12-09T08:11:33.157Z INFO webdriver: DATA {
[0-0]   capabilities: {
[0-0]     alwaysMatch: {
[0-0]       platformName: 'Android',
[0-0]       'appium:automationName': 'UiAutomator2',
[0-0]       'appium:app': '',
[0-0]       'appium:newCommandTimeout': 100,
[0-0]       'appium:systemPort': 8200
[0-0]     },
[0-0]     firstMatch: [ {} ]
[0-0]   }
[0-0] }
[0-0] 2024-12-09T08:11:41.238Z INFO @wdio/ocr-service: Adding commands to Multi Browser: androidDevice
[0-0] 2024-12-09T08:11:41.321Z INFO @wdio/ocr-service: Adding browser command "ocrGetText" to browser object
[0-0] 2024-12-09T08:11:41.321Z INFO @wdio/ocr-service: Adding browser command "ocrGetElementPositionByText" to browser object
[0-0] 2024-12-09T08:11:41.321Z INFO @wdio/ocr-service: Adding browser command "ocrWaitForTextDisplayed" to browser object
[0-0] 2024-12-09T08:11:41.322Z INFO @wdio/ocr-service: Adding browser command "ocrClickOnText" to browser object
[0-0] 2024-12-09T08:11:41.322Z INFO @wdio/ocr-service: Adding browser command "ocrSetValue" to browser object
[0-0] 2024-12-09T08:11:41.322Z INFO @wdio/visual-service: Adding commands to Multi Browser: androidDevice
[0-0] 2024-12-09T08:11:41.324Z INFO webdriver: COMMAND getWindowRect()
[0-0] 2024-12-09T08:11:41.324Z INFO webdriver: [GET] http://127.0.0.1:4723/wd/hub/session/976dd81c-303f-4fd7-982e-2c72ee359d5e/window/rect
[0-0] 2024-12-09T08:11:41.351Z INFO webdriver: RESULT { width: 720, height: 1280, x: 0, y: 0 }
[0-0] 2024-12-09T08:11:41.352Z DEBUG @wdio/utils:shim: Finished to run "afterCommand" hook in 0ms
[0-0] 2024-12-09T08:11:41.352Z DEBUG @wdio/utils:shim: Finished to run "afterCommand" hook in 0ms
[0-0] 2024-12-09T08:11:41.352Z INFO @wdio/visual-service: Adding element command "saveElement" to browser object
[0-0] 2024-12-09T08:11:41.352Z INFO @wdio/visual-service: Adding element command "checkElement" to browser object
[0-0] 2024-12-09T08:11:41.352Z INFO @wdio/visual-service: Adding browser command "saveScreen" to browser object
[0-0] 2024-12-09T08:11:41.352Z INFO @wdio/visual-service: Adding browser command "saveFullPageScreen" to browser object
[0-0] 2024-12-09T08:11:41.352Z INFO @wdio/visual-service: Adding browser command "saveTabbablePage" to browser object
[0-0] 2024-12-09T08:11:41.352Z INFO @wdio/visual-service: Adding browser command "checkScreen" to browser object
[0-0] 2024-12-09T08:11:41.352Z INFO @wdio/visual-service: Adding browser command "checkFullPageScreen" to browser object
[0-0] 2024-12-09T08:11:41.352Z INFO @wdio/visual-service: Adding browser command "checkTabbablePage" to browser object
[0-0] 2024-12-09T08:11:41.352Z INFO @wdio/visual-service: Adding browser command "waitForStorybookComponentToBeLoaded" to browser object
[0-0] 2024-12-09T08:11:41.352Z INFO @wdio/visual-service: Adding browser command "saveScreen" to Multi browser object
[0-0] 2024-12-09T08:11:41.353Z INFO @wdio/visual-service: Adding browser command "saveFullPageScreen" to Multi browser object
[0-0] 2024-12-09T08:11:41.353Z INFO @wdio/visual-service: Adding browser command "saveTabbablePage" to Multi browser object
[0-0] 2024-12-09T08:11:41.353Z INFO @wdio/visual-service: Adding browser command "checkScreen" to Multi browser object
[0-0] 2024-12-09T08:11:41.353Z INFO @wdio/visual-service: Adding browser command "checkFullPageScreen" to Multi browser object
[0-0] 2024-12-09T08:11:41.353Z INFO @wdio/visual-service: Adding browser command "checkTabbablePage" to Multi browser object
[0-0] 2024-12-09T08:11:41.353Z INFO @wdio/visual-service: Adding browser command "waitForStorybookComponentToBeLoaded" to Multi browser object
[0-0] 2024-12-09T08:11:41.353Z INFO @wdio/visual-service: Adding element command "saveElement" to Multi browser object
[0-0] 2024-12-09T08:11:41.353Z INFO @wdio/visual-service: Adding element command "checkElement" to Multi browser object
[0-0] 2024-12-09T08:11:41.353Z DEBUG @wdio/utils:shim: Finished to run "before" hook in 30ms
[0-0] 2024-12-09T08:11:41.362Z DEBUG @wdio/utils:shim: Finished to run "beforeTest" hook in 2ms
[0-0] 2024-12-09T08:11:41.363Z INFO webdriver: COMMAND findElement("xpath", "//android.widget.TextView[@content-desc="Contacts"]")
[0-0] 2024-12-09T08:11:41.364Z INFO webdriver: [POST] http://127.0.0.1:4723/wd/hub/session/976dd81c-303f-4fd7-982e-2c72ee359d5e/element
[0-0] 2024-12-09T08:11:41.364Z INFO webdriver: DATA {
[0-0]   using: 'xpath',
[0-0]   value: '//android.widget.TextView[@content-desc="Contacts"]'
[0-0] }
[0-0] 2024-12-09T08:11:42.023Z INFO webdriver: RESULT {
[0-0]   'element-6066-11e4-a52e-4f735466cecf': '00000000-0000-001e-ffff-ffff0000008b',
[0-0]   ELEMENT: '00000000-0000-001e-ffff-ffff0000008b'
[0-0] }
[0-0] 2024-12-09T08:11:42.023Z DEBUG @wdio/utils:shim: Finished to run "afterCommand" hook in 0ms
[0-0] 2024-12-09T08:11:42.029Z DEBUG @wdio/utils:shim: Finished to run "afterCommand" hook in 0ms
[0-0] 2024-12-09T08:11:42.031Z INFO webdriver: COMMAND ocrGetText(<object>)
[0-0] 2024-12-09T08:11:42.031Z INFO webdriver: COMMAND ocrGetText(<object>)
[0-0] 2024-12-09T08:11:42.031Z INFO webdriver: COMMAND ocrGetText(<object>)
...

@wswebcreation
Copy link
Member

Hi @P-Courteille

I'll check this. This log

[0-0] 'tesseract' n'est pas reconnu en tant que commande interne

means that it can't find a local installed version of Tesseract. It will now use the NodeJS dependency. If you have the option to use a native installed version of Tessaract you will see it will be much faster. See also https://webdriver.io/docs/ocr-testing/more-test-optimization

I'll try to run my tests again with only the NodeJS vesion

@P-Courteille
Copy link
Author

P-Courteille commented Dec 9, 2024

Maybe it will help, this is the list of change I observe in my npm packages between 9.3.0 and 9.3.1:

After test, the issue is with wdio package @wdio/runner/node_modules/webdriverio/index.js

New function getNativeContext and mutliple change on context
image

I will try to found the specific line

@wswebcreation
Copy link
Member

Thanks, most of the changes have been implemented by me 😅 , but as far as I can see this doesn't impact the OCR service

@P-Courteille
Copy link
Author

P-Courteille commented Dec 9, 2024

Those lines are the reason:

driver.instances.forEach(
        (instance) => driver.getInstance(instance).addCommand(name, fn, attachToElement)
 );

The irony is the change is related to another of my ticket about issue with addCommand in multiremote...
(and since this issue, I no longer use addCommand [but service do] and only use browser instantiation in my pageObject)

You are innocent @wswebcreation, it's not your code ^^
webdriverio/webdriverio#13836
https://github.com/webdriverio/webdriverio/pull/13904/files

@wswebcreation
Copy link
Member

Hi @P-Courteille
It's not about being innocent 😉 , but thanks. I've checked if I can find the issue for the OCR loop, but it's (on my machine) also not related to the node version of Tesseract. Please let me know if you have a "better" reproducible example.

I'll go ahead and close this issue and release a new visual service that fixes the multiremote support

@wswebcreation
Copy link
Member

@P-Courteille

I've released a new version if you use 6.2.3 most of the issues should be gone

@P-Courteille
Copy link
Author

On my side the previous example loop in 100% with local-runner 9.3.1 and work as expected with local-runner 9.3.0.
I will try again to reduce the example to the minimum again.

I still don't understand the behavior but it's like the foreach loop to addCommand to all browser from https://github.com/webdriverio/webdriverio/pull/13904/files is in conflict with the code in class WdioOcrService with the same pupose.

@P-Courteille
Copy link
Author

P-Courteille commented Dec 9, 2024

@wswebcreation I think you should remove your #extendMultiremoteBrowser from class WdioOcrService in service.js
Without it, it work because it's already covered by the new for-each loop :)


With For-each without #extendMultiremoteBrowser -> It work
Without For-each and with #extendMultiremoteBrowser -> it work
With For-each and #extendMultiremoteBrowser -> The code enter in loop.

Why ?
Because this loop 'for (const browserName of browserNames)':

  • call for each browser #addCommandsToBrowser
    • that call addCommand
      • but add command itself loop through each browser

So technically, #extendMultiremoteBrowser was a fix for webdriverio/webdriverio#13836 but only in ocr-service scope. When @ccharnkij fixed it in the main project, the conflict started.

@P-Courteille
Copy link
Author

P-Courteille commented Dec 10, 2024

Hi @wswebcreation,
The new version 6.2.3 fix the type issue.

I think the update introduce another issue with the instance in case of mismatch:
image
(not blocking for me, I can create the ticket issue if you want)

@wswebcreation
Copy link
Member

Awesome, can you file a new one?

@P-Courteille
Copy link
Author

P-Courteille commented Dec 10, 2024

@wswebcreation,
The new issue ticket is #679.
If you have time, do not hesitate to confirm my analysis about #657.

Thank you again for your assistance,
Have a nice day,
Paul

@wswebcreation
Copy link
Member

Thanks, a fix for #679 will be released today

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants