Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 88 additions & 6 deletions docs/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ expect.clearSoftFailures();

### Integration with Test Frameworks

The soft assertions feature integrates with WebdriverIO's test runner automatically. By default, it will report all soft assertion failures at the end of each test (Mocha/Jasmine) or step (Cucumber).
The soft assertions feature integrates with WebdriverIO's test runner automatically. By default, it will report all soft assertion failures at the end of each test (Mocha) or step (Cucumber).

To use with WebdriverIO, add the SoftAssertionService to your services list:

Expand All @@ -75,7 +75,7 @@ export const config = {
// ...
services: [
// ...other services
[SoftAssertionService]
[SoftAssertionService, {}]
],
// ...
}
Expand Down Expand Up @@ -113,8 +113,7 @@ This is useful if you want full control over when soft assertions are verified o

### Known limitations

For Jasmine, using `wdio-jasmine-framework` will give a better plug-and-play experiences, else without it, the soft assertion service and custom matchers might not work/be registered correctly.
Moreover, if Jasmine augmentation is used, the soft assertion function are not exposed in the typing, but could still work depending of your configuration. See [this issue](https://github.com/webdriverio/expect-webdriverio/issues/1893) for more details.
The soft assertions service is not supported under Jasmine (e.g. `@wdio/jasmine-framework`) using the global import because Jasmine is already designed to provide similar behavior out of the box.

## Default Options

Expand Down Expand Up @@ -878,7 +877,72 @@ await expect(elem).toHaveElementClass(/Container/i)

## Default Matchers

In addition to the `expect-webdriverio` matchers you can use builtin Jest's [expect](https://jestjs.io/docs/expect) assertions or [expect/expectAsync](https://jasmine.github.io/api/edge/global.html#expect) for Jasmine.
In addition to the WebdriverIO matchers, `expect-webdriverio` also provides basic matchers from Jest's [expect](https://jestjs.io/docs/expect) library.

```ts
describe('Expect matchers', () => {
test('Basic matchers', async () => {
// Equality
expect(2 + 2).toBe(4);
expect({a: 1}).toEqual({a: 1});
expect([1, 2, 3]).toStrictEqual([1, 2, 3]);
expect(2 + 2).not.toBe(5);

// Truthiness
expect(null).toBeNull();
expect(undefined).toBeUndefined();
expect(0).toBeFalsy();
expect(1).toBeTruthy();
expect(NaN).toBeNaN();

// Numbers
expect(4).toBeGreaterThan(3);
expect(4).toBeGreaterThanOrEqual(4);
expect(4).toBeLessThan(5);
expect(4).toBeLessThanOrEqual(4);
expect(0.2 + 0.1).toBeCloseTo(0.3, 5);

// Strings
expect('team').toMatch(/team/);
expect('Christoph').toContain('stop');

// Arrays and iterables
expect([1, 2, 3]).toContain(2);
expect([{a: 1}, {b: 2}]).toContainEqual({a: 1});
expect([1, 2, 3]).toHaveLength(3);

// Objects
expect({a: 1, b: 2}).toHaveProperty('a');
expect({a: {b: 2}}).toHaveProperty('a.b', 2);

// Errors
expect(() => { throw new Error('error!') }).toThrow('error!');
expect(() => { throw new TypeError('wrong type') }).toThrow(TypeError);

// Asymmetric matchers
expect({foo: 'bar', baz: 1}).toEqual(expect.objectContaining({foo: expect.any(String)}));
expect([1, 2, 3]).toEqual(expect.arrayContaining([2]));
expect('abc').toEqual(expect.stringContaining('b'));
expect('abc').toEqual(expect.stringMatching(/b/));
expect(123).toEqual(expect.any(Number));

// Others
expect(new Set([1, 2, 3])).toContain(2);

// .resolves / .rejects (async)
await expect(Promise.resolve(42)).resolves.toBe(42);
await expect(Promise.reject(new Error('fail'))).rejects.toThrow('fail');
});
});
```

### Jasmine

For Jasmine, see the official documentation for [expect/expectAsync](https://jasmine.github.io/api/edge/global.html#expect), [matchers](https://jasmine.github.io/tutorials/your_first_suite#section-Matchers), and [async-matchers](https://jasmine.github.io/api/edge/async-matchers.html).

**Note:**
- With the global import in @wdio/jasmine-framework, only WebdriverIO custom matchers are registered on expectAsync (assigned to global expect), so all matchers are always async, even those that are normally synchronous.
- Default matchers are still available if you import `expect` directly from `expect-webdriverio` instead of using the global.

## Modifiers

Expand All @@ -901,7 +965,7 @@ await expect(element).not.toBeDisplayed({ wait: 0 })
await expect(browser).not.toHaveTitle('some title', { wait: 0 })
```

Note: You can pair `.not` with asymmetric matchers, but to enable the wait-until behavior, `.not` must be used directly on the `expect()` call.
**Note:** You can pair `.not` with asymmetric matchers, but to enable the wait-until behavior, `.not` must be used directly on the `expect()` call.

## Asymmetric Matchers

Expand All @@ -916,3 +980,21 @@ or
```ts
await expect(browser).toHaveTitle(expect.not.stringContaining('some title'))
```

### Jasmine

Even under `@wdio/jasmine-framework`, Jasmine asymmetric matchers do not work with WebdriverIO matchers.

```ts
// DOES NOT work
await expect(browser).toHaveTitle(jasmine.stringContaining('some title'))

// Use expect
await expect(browser).toHaveTitle(expect.stringContaining('some title'))
```

However, when using Jasmine original matchers, both works.
```ts
await expect(url).toEqual(jasmine.stringMatching(/^https:\/\//))
await expect(url).toEqual(expect.stringMatching(/^https:\/\//))
```
11 changes: 8 additions & 3 deletions docs/Framework.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,11 @@ See also this [documentation](https://webdriver.io/docs/assertion/#migrating-fro
### Jasmine
When paired with [Jasmine](https://jasmine.github.io/), [`@wdio/jasmine-framework`](https://www.npmjs.com/package/@wdio/jasmine-framework) is also required to configure it correctly, as it needs to force `expect` to be `expectAsync` and also register the WDIO matchers with `addAsyncMatcher` since `expect-webdriverio` only supports the Jest-style `expect.extend` version.

Jasmine differs from other assertion libraries in two key ways:
1. Jasmine performs soft assertions by default, collecting failures and only failing the test at the end. Because of this, the SoftAssertion service is not needed or supported.
2. Forcing `expectAsync` as `expect` (by `@wdio/jasmine-framework`) makes even basic matchers asynchronous. However, since Jasmine handles all promises at the end of the test, assertions appear to work properly—unlike in other frameworks, where using `await` is mandatory for correct behavior.
- Note: This goes against [this recommendation](https://jasmine.github.io/api/edge/async-matchers) and could cause unexpected issues.

The types `expect-webdriverio/jasmine` are still offered but are subject to removal or being moved into `@wdio/jasmine-framework`. The usage of `expectAsync` is also subject to future removal.

#### Jasmine `expectAsync`
Expand Down Expand Up @@ -173,14 +178,14 @@ Expected in `tsconfig.json`:
```

#### Global `expectAsync` force as `expect`
When the global ambiant is the `expect` of wdio but forced to be `expectAsync` under the hood, like when using `@wdio/jasmine-framework`, then even the basic matchers need to be awaited
When the global ambient `expect` is actually `expectAsync` under the hood (as with `@wdio/jasmine-framework`), it is recommended to `await` even basic matchers, even though Jasmine will handle any un-awaited assertions at the end of the test.

```ts
describe('My tests', async () => {
it('should verify my browser to have the expected url', async () => {
await expect(browser).toHaveUrl('https://example.com')

// Even basic matchers requires expect since they are promises underneath
// Even basic matchers should have `await` since they are promises underneath
await expect(true).toBe(true)
})
})
Expand Down Expand Up @@ -249,4 +254,4 @@ It is recommended to build your project using this approach instead of relying o

### Cucumber

More details to come. In short, when paired with `@wdio/cucumber-framework`, you can use WDIO's expect with Cucumber and even [Gherkin](https://www.npmjs.com/package/@cucumber/gherkin).
More details to come. In short, when paired with `@wdio/cucumber-framework`, you can use WDIO's expect with Cucumber and even [Gherkin](https://www.npmjs.com/package/@cucumber/gherkin).
63 changes: 62 additions & 1 deletion eslint.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ export default wdioEslint.config([
{
ignores: [
'lib',
'**/*/dist'
'**/*/dist',
'playgrounds/**'
]
},
/**
Expand All @@ -27,5 +28,65 @@ export default wdioEslint.config([
'@typescript-eslint/no-require-imports': 'off',
'@typescript-eslint/no-explicit-any': 'off'
}
},
{
files: ['src/**/*.ts'],
plugins: {
local: {
rules: {
'enforce-options-list': {
create(context) {
const fileName = context.filename || context.getFilename()
const isConstantsFile = fileName.endsWith('src/constants.ts')
const definedOptions = new Set()
let listNode = null
const listElements = new Set()

return {
VariableDeclarator(node) {
if (node.id.type === 'Identifier') {
if (node.id.name.includes('DEFAULT_OPTIONS')) {
if (isConstantsFile) {
definedOptions.add(node.id.name)
} else {
context.report({
node: node.id,
message: `Option '${node.id.name}' must be included in 'src/constants.ts#defaultOptionsList', so it can be globally overridden.`
})
}
}
if (isConstantsFile && node.id.name === 'defaultOptionsList' && node.init && node.init.type === 'ArrayExpression') {
listNode = node
node.init.elements.forEach(el => {
if (el && el.type === 'Identifier') {
listElements.add(el.name)
}
})
}
}
},
'Program:exit'() {
if (!listNode) {
return
}

definedOptions.forEach(opt => {
if (!listElements.has(opt)) {
context.report({
node: listNode,
message: `Option '${opt}' must be included in 'defaultOptionsList', so it can be globally overridden in 'src/constants.ts'.`
})
}
})
}
}
}
}
}
}
},
rules: {
'local/enforce-options-list': 'error'
}
}
])
35 changes: 31 additions & 4 deletions jasmine-wdio-expect-async.d.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,37 @@
/// <reference types="./types/expect-webdriverio.d.ts"/>

declare namespace jasmine {

/**
* Async matchers for Jasmine to allow the typing of `expectAsync` with WebDriverIO matchers.
* T is the type of the actual value
* U is the type of the expected value
* Both T,U must stay named as they are to override the default `AsyncMatchers` type from Jasmine.
*
* We force Matchers to return a `Promise<void>` since Jasmine's `expectAsync` expects a promise in all cases (different from Jest)
*/
// eslint-disable-next-line @typescript-eslint/no-unused-vars
interface AsyncMatchers<T, U> extends Omit<ExpectWebdriverIO.Matchers<Promise<void>, T>, 'toMatchSnapshot' | 'toMatchInlineSnapshot'> {
/**
* snapshot matcher
* @param label optional snapshot label
*/
toMatchSnapshot(label?: string): Promise<void>;
/**
* inline snapshot matcher
* @param snapshot snapshot string (autogenerated if not specified)
* @param label optional snapshot label
*/
toMatchInlineSnapshot(snapshot?: string, label?: string): Promise<void>;
}
}

/**
* Overrides the default wdio `expect` for Jasmine case specifically since the `expect` is now completely asynchronous which is not the case under Jest or standalone.
* Overrides the default WDIO expect specifically for Jasmine, since `expectAsync` is forced into `expect`, making all matchers fully asynchronous. This is not the case under Jest or Mocha.
* Using `jasmine.AsyncMatchers` pull on WdioMatchers above but also allow to using Jasmine's built-in matchers and also `withContext` matcher.
*/
declare namespace ExpectWebdriverIO {
interface Expect extends ExpectWebdriverIO.AsymmetricMatchers, ExpectLibInverse<ExpectWebdriverIO.InverseAsymmetricMatchers>, WdioExpect {
interface Expect {
/**
* The `expect` function is used every time you want to test a value.
* You will rarely call `expect` by itself.
Expand All @@ -17,6 +44,6 @@ declare namespace ExpectWebdriverIO {
*
* @param actual The value to apply matchers against.
*/
<T = unknown>(actual: T): ExpectWebdriverIO.MatchersAndInverse<Promise<void>, T>
<T = unknown>(actual: T): jasmine.AsyncMatchers<T, void>
}
}
}
9 changes: 8 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@
"types": "./jasmine.d.ts"
}
],
"./jasmine-wdio-expect-async": [
{
"types": "./jasmine-wdio-expect-async.d.ts"
}
],
"./types": "./types/expect-global.d.ts",
"./expect-global": "./types/expect-global.d.ts"
},
Expand Down Expand Up @@ -65,7 +70,9 @@
"ts:jasmine-async": "cd test-types/jasmine-async && tsc --project ./tsconfig.json",
"checks:all": "npm run build && npm run compile && npm run tsc:root-types && npm run test && npm run ts",
"watch": "npm run compile -- --watch",
"prepare": "husky install"
"prepare": "husky install",
"playgrounds:setup": "for dir in playgrounds/*/; do cd \"$dir\" && npm install && cd ../..; done",
"playgrounds:checks:all": "for dir in playgrounds/*/; do cd \"$dir\" && npm run checks:all && cd ../..; done"
},
"dependencies": {
"@vitest/snapshot": "^4.0.16",
Expand Down
1 change: 1 addition & 0 deletions playgrounds/jasmine/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
node_modules/
28 changes: 28 additions & 0 deletions playgrounds/jasmine/eslint.config.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import tsEslintPlugin from '@typescript-eslint/eslint-plugin';
import tsParser from '@typescript-eslint/parser';

export default {
files: ['**/*.ts', '**/*.js'],
languageOptions: {
parser: tsParser,
parserOptions: {
project: './tsconfig.json',
sourceType: 'module',
ecmaVersion: 2021,
},
globals: {
NodeJS: true,
require: true,
module: true,
__dirname: true,
process: true,
},
},
plugins: {
'@typescript-eslint': tsEslintPlugin,
},
rules: {
...tsEslintPlugin.configs['recommended'].rules,
'@typescript-eslint/no-floating-promises': 'error',
},
};
Loading