Setup E2E testing - Appium for IOS/Android #1099
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| const { getDeviceName, getPlatformVersion } = require('./common'); | ||
| const { config } = require('./wdio.conf'); | ||
| const { join } = require('path'); | ||
|
|
||
| config.capabilities = [ | ||
| { | ||
| // The defaults you need to have in your config | ||
| platformName: 'Android', | ||
| maxInstances: 1, | ||
| // For W3C the appium capabilities need to have an extension prefix | ||
| // This is `appium:` for all Appium Capabilities which can be found here | ||
| // http://appium.io/docs/en/writing-running-appium/caps/ | ||
| 'appium:deviceName': getDeviceName('Android'), | ||
| 'appium:platformVersion': getPlatformVersion('Android'), | ||
| 'appium:automationName': 'UiAutomator2', | ||
| 'appium:app': join(process.cwd(), './apps/em-devapp-3.2.5.apk'), | ||
| }, | ||
| ]; | ||
|
|
||
| exports.config = config; | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| /** | ||
| * get Device Name from script | ||
| * @param platform iOS or Android | ||
| */ | ||
| const getDeviceName = (platform) => { | ||
| const deviceName = process.argv.find((arg) => arg.includes('--deviceName')); | ||
| const defaultDeviceName = platform === 'iOS' ? 'iPhone 13' : 'Pixel 3a API 33'; | ||
| return deviceName ? deviceName.split('=')[1] : defaultDeviceName; | ||
| }; | ||
|
|
||
| /** | ||
| * get Platform Version from script | ||
| * @param platform iOS or Android | ||
| */ | ||
| const getPlatformVersion = (platform) => { | ||
| const platformVersion = process.argv.find((arg) => arg.includes('--platformVersion')); | ||
| const defaultPlatformVersion = platform === 'iOS' ? '15.0' : '13'; | ||
| return platformVersion ? platformVersion.split('=')[1] : defaultPlatformVersion; | ||
| }; | ||
|
Comment on lines
+5
to
+19
There was a problem hiding this comment. I think the way you have it split up currently is really good. The only suggestion I might make is for these functions to be split down even further. Just a suggestion though! I am not too familiar with Appium or e2e testing strategies, and looking at what you've written so far looks great!!! 😊 There was a problem hiding this comment. The flexibility provided by There was a problem hiding this comment. The one suggestion that I might have is to put these versions into a separate setup file, similar to all the other versions that we need in our build https://github.com/e-mission/e-mission-phone/blob/master/setup/export_shared_dep_versions.sh so:
|
||
|
|
||
| module.exports = { getDeviceName, getPlatformVersion }; | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| const { getDeviceName, getPlatformVersion } = require('./common'); | ||
| const { config } = require('./wdio.conf'); | ||
|
|
||
| // Appium capabilities | ||
| config.capabilities = [ | ||
| { | ||
| // The defaults you need to have in your config | ||
| platformName: 'iOS', | ||
| maxInstances: 1, | ||
| // For W3C the appium capabilities need to have an extension prefix | ||
| // This is `appium:` for all Appium Capabilities which can be found here | ||
| // http://appium.io/docs/en/writing-running-appium/caps/ | ||
| 'appium:deviceName': getDeviceName('iOS'), | ||
| 'appium:platformVersion': getPlatformVersion('iOS'), | ||
| 'appium:automationName': 'XCUITest', | ||
| 'appium:app': 'edu.berkeley.eecs.emission.devapp', | ||
|
There was a problem hiding this comment. If we switched from testing the devapp to testing the built 'emission' app, this line would change to |
||
| }, | ||
| ]; | ||
|
|
||
| exports.config = config; | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,279 @@ | ||
| exports.config = { | ||
| // | ||
| // ==================== | ||
| // Runner Configuration | ||
| // ==================== | ||
| // WebdriverIO supports running e2e tests as well as unit and component tests. | ||
| runner: 'local', | ||
| port: 4723, | ||
| // | ||
| // ================== | ||
| // Specify Test Files | ||
| // ================== | ||
| // Define which test specs should run. The pattern is relative to the directory | ||
| // of the configuration file being run. | ||
| // | ||
| // The specs are defined as an array of spec files (optionally using wildcards | ||
| // that will be expanded). The test for each spec file will be run in a separate | ||
| // worker process. In order to have a group of spec files run in the same worker | ||
| // process simply enclose them in an array within the specs array. | ||
| // | ||
| // If you are calling `wdio` from an NPM script (see https://docs.npmjs.com/cli/run-script), | ||
| // then the current working directory is where your `package.json` resides, so `wdio` | ||
| // will be called from there. | ||
| // | ||
| specs: ['../specs/*.js'], | ||
| // Patterns to exclude. | ||
| exclude: [ | ||
| // 'path/to/excluded/files' | ||
| ], | ||
| // | ||
| // ============ | ||
| // Capabilities | ||
| // ============ | ||
| // Define your capabilities here. WebdriverIO can run multiple capabilities at the same | ||
| // time. Depending on the number of capabilities, WebdriverIO launches several test | ||
| // sessions. Within your capabilities you can overwrite the spec and exclude options in | ||
| // order to group specific specs to a specific capability. | ||
| // | ||
| // First, you can define how many instances should be started at the same time. Let's | ||
| // say you have 3 different capabilities (Chrome, Firefox, and Safari) and you have | ||
| // set maxInstances to 1; wdio will spawn 3 processes. Therefore, if you have 10 spec | ||
| // files and you set maxInstances to 10, all spec files will get tested at the same time | ||
| // and 30 processes will get spawned. The property handles how many capabilities | ||
| // from the same test should run tests. | ||
| // | ||
| maxInstances: 10, | ||
| // | ||
| // If you have trouble getting all important capabilities together, check out the | ||
| // Sauce Labs platform configurator - a great tool to configure your capabilities: | ||
| // https://saucelabs.com/platform/platform-configurator | ||
| // | ||
| capabilities: [], | ||
|
|
||
| // | ||
| // =================== | ||
| // Test Configurations | ||
| // =================== | ||
| // Define all options that are relevant for the WebdriverIO instance here | ||
| // | ||
| // Level of logging verbosity: trace | debug | info | warn | error | silent | ||
| logLevel: 'info', | ||
| // | ||
| // Set specific log levels per logger | ||
| // loggers: | ||
| // - webdriver, webdriverio | ||
| // - @wdio/browserstack-service, @wdio/devtools-service, @wdio/sauce-service | ||
| // - @wdio/mocha-framework, @wdio/jasmine-framework | ||
| // - @wdio/local-runner | ||
| // - @wdio/sumologic-reporter | ||
| // - @wdio/cli, @wdio/config, @wdio/utils | ||
| // Level of logging verbosity: trace | debug | info | warn | error | silent | ||
| // logLevels: { | ||
| // webdriver: 'info', | ||
| // '@wdio/appium-service': 'info' | ||
| // }, | ||
| // | ||
| // If you only want to run your tests until a specific amount of tests have failed use | ||
| // bail (default is 0 - don't bail, run all tests). | ||
| bail: 0, | ||
| // | ||
| // Set a base URL in order to shorten url command calls. If your `url` parameter starts | ||
| // with `/`, the base url gets prepended, not including the path portion of your baseUrl. | ||
| // If your `url` parameter starts without a scheme or `/` (like `some/path`), the base url | ||
| // gets prepended directly. | ||
| baseUrl: 'http://localhost', | ||
| // | ||
| // Default timeout for all waitFor* commands. | ||
| waitforTimeout: 10000, | ||
| // | ||
| // Default timeout in milliseconds for request | ||
| // if browser driver or grid doesn't send response | ||
| connectionRetryTimeout: 120000, | ||
| // | ||
| // Default request retries count | ||
| connectionRetryCount: 3, | ||
| // | ||
| // Test runner services | ||
| // Services take over a specific job you don't want to take care of. They enhance | ||
| // your test setup with almost no effort. Unlike plugins, they don't add new | ||
| // commands. Instead, they hook themselves up into the test process. | ||
| services: ['appium'], | ||
|
|
||
| // Framework you want to run your specs with. | ||
| // The following are supported: Mocha, Jasmine, and Cucumber | ||
| // see also: https://webdriver.io/docs/frameworks | ||
| // | ||
| // Make sure you have the wdio adapter package for the specific framework installed | ||
| // before running any tests. | ||
| framework: 'mocha', | ||
|
|
||
| // | ||
| // The number of times to retry the entire specfile when it fails as a whole | ||
| // specFileRetries: 1, | ||
| // | ||
| // Delay in seconds between the spec file retry attempts | ||
| // specFileRetriesDelay: 0, | ||
| // | ||
| // Whether or not retried spec files should be retried immediately or deferred to the end of the queue | ||
| // specFileRetriesDeferred: false, | ||
| // | ||
| // Test reporter for stdout. | ||
| // The only one supported by default is 'dot' | ||
| // see also: https://webdriver.io/docs/dot-reporter | ||
| reporters: ['spec'], | ||
|
|
||
| // Options to be passed to Mocha. | ||
| // See the full list at http://mochajs.org/ | ||
| mochaOpts: { | ||
| ui: 'bdd', | ||
| timeout: 60000, | ||
| }, | ||
|
|
||
| // | ||
| // ===== | ||
| // Hooks | ||
| // ===== | ||
| // WebdriverIO provides several hooks you can use to interfere with the test process in order to enhance | ||
| // it and to build services around it. You can either apply a single function or an array of | ||
| // methods to it. If one of them returns with a promise, WebdriverIO will wait until that promise got | ||
| // resolved to continue. | ||
| /** | ||
| * Gets executed once before all workers get launched. | ||
| * @param {object} config wdio configuration object | ||
| * @param {Array.<Object>} capabilities list of capabilities details | ||
| */ | ||
| // onPrepare: function (config, capabilities) { | ||
| // }, | ||
| /** | ||
| * Gets executed before a worker process is spawned and can be used to initialise specific service | ||
| * for that worker as well as modify runtime environments in an async fashion. | ||
| * @param {string} cid capability id (e.g 0-0) | ||
| * @param {object} caps object containing capabilities for session that will be spawn in the worker | ||
| * @param {object} specs specs to be run in the worker process | ||
| * @param {object} args object that will be merged with the main configuration once worker is initialized | ||
| * @param {object} execArgv list of string arguments passed to the worker process | ||
| */ | ||
| // onWorkerStart: function (cid, caps, specs, args, execArgv) { | ||
| // }, | ||
| /** | ||
| * Gets executed just after a worker process has exited. | ||
| * @param {string} cid capability id (e.g 0-0) | ||
| * @param {number} exitCode 0 - success, 1 - fail | ||
| * @param {object} specs specs to be run in the worker process | ||
| * @param {number} retries number of retries used | ||
| */ | ||
| // onWorkerEnd: function (cid, exitCode, specs, retries) { | ||
| // }, | ||
| /** | ||
| * Gets executed just before initialising the webdriver session and test framework. It allows you | ||
| * to manipulate configurations depending on the capability or spec. | ||
| * @param {object} config wdio configuration object | ||
| * @param {Array.<Object>} capabilities list of capabilities details | ||
| * @param {Array.<String>} specs List of spec file paths that are to be run | ||
| * @param {string} cid worker id (e.g. 0-0) | ||
| */ | ||
| // beforeSession: function (config, capabilities, specs, cid) { | ||
| // }, | ||
| /** | ||
| * Gets executed before test execution begins. At this point you can access to all global | ||
| * variables like `browser`. It is the perfect place to define custom commands. | ||
| * @param {Array.<Object>} capabilities list of capabilities details | ||
| * @param {Array.<String>} specs List of spec file paths that are to be run | ||
| * @param {object} browser instance of created browser/device session | ||
| */ | ||
| // before: function (capabilities, specs) { | ||
| // }, | ||
| /** | ||
| * Runs before a WebdriverIO command gets executed. | ||
| * @param {string} commandName hook command name | ||
| * @param {Array} args arguments that command would receive | ||
| */ | ||
| // beforeCommand: function (commandName, args) { | ||
| // }, | ||
| /** | ||
| * Hook that gets executed before the suite starts | ||
| * @param {object} suite suite details | ||
| */ | ||
| // beforeSuite: function (suite) { | ||
| // }, | ||
| /** | ||
| * Function to be executed before a test (in Mocha/Jasmine) starts. | ||
| */ | ||
| // beforeTest: function (test, context) { | ||
| // }, | ||
| /** | ||
| * Hook that gets executed _before_ a hook within the suite starts (e.g. runs before calling | ||
| * beforeEach in Mocha) | ||
| */ | ||
| // beforeHook: function (test, context, hookName) { | ||
| // }, | ||
| /** | ||
| * Hook that gets executed _after_ a hook within the suite starts (e.g. runs after calling | ||
| * afterEach in Mocha) | ||
| */ | ||
| // afterHook: function (test, context, { error, result, duration, passed, retries }, hookName) { | ||
| // }, | ||
| /** | ||
| * Function to be executed after a test (in Mocha/Jasmine only) | ||
| * @param {object} test test object | ||
| * @param {object} context scope object the test was executed with | ||
| * @param {Error} result.error error object in case the test fails, otherwise `undefined` | ||
| * @param {*} result.result return object of test function | ||
| * @param {number} result.duration duration of test | ||
| * @param {boolean} result.passed true if test has passed, otherwise false | ||
| * @param {object} result.retries information about spec related retries, e.g. `{ attempts: 0, limit: 0 }` | ||
| */ | ||
| // afterTest: function(test, context, { error, result, duration, passed, retries }) { | ||
| // }, | ||
|
|
||
| /** | ||
| * Hook that gets executed after the suite has ended | ||
| * @param {object} suite suite details | ||
| */ | ||
| // afterSuite: function (suite) { | ||
| // }, | ||
| /** | ||
| * Runs after a WebdriverIO command gets executed | ||
| * @param {string} commandName hook command name | ||
| * @param {Array} args arguments that command would receive | ||
| * @param {number} result 0 - command success, 1 - command error | ||
| * @param {object} error error object if any | ||
| */ | ||
| // afterCommand: function (commandName, args, result, error) { | ||
| // }, | ||
| /** | ||
| * Gets executed after all tests are done. You still have access to all global variables from | ||
| * the test. | ||
| * @param {number} result 0 - test pass, 1 - test fail | ||
| * @param {Array.<Object>} capabilities list of capabilities details | ||
| * @param {Array.<String>} specs List of spec file paths that ran | ||
| */ | ||
| // after: function (result, capabilities, specs) { | ||
| // }, | ||
| /** | ||
| * Gets executed right after terminating the webdriver session. | ||
| * @param {object} config wdio configuration object | ||
| * @param {Array.<Object>} capabilities list of capabilities details | ||
| * @param {Array.<String>} specs List of spec file paths that ran | ||
| */ | ||
| // afterSession: function (config, capabilities, specs) { | ||
| // }, | ||
| /** | ||
| * Gets executed after all workers got shut down and the process is about to exit. An error | ||
| * thrown in the onComplete hook will result in the test run failing. | ||
| * @param {object} exitCode 0 - success, 1 - fail | ||
| * @param {object} config wdio configuration object | ||
| * @param {Array.<Object>} capabilities list of capabilities details | ||
| * @param {<Object>} results object containing test results | ||
| */ | ||
| // onComplete: function(exitCode, config, capabilities, results) { | ||
| // }, | ||
| /** | ||
| * Gets executed when a refresh happens. | ||
| * @param {string} oldSessionId session ID of the old session | ||
| * @param {string} newSessionId session ID of the new session | ||
| */ | ||
| // onReload: function(oldSessionId, newSessionId) { | ||
| // } | ||
|
Comment on lines
+133
to
+278
There was a problem hiding this comment. I see that we aren't using hooks now, but it's good to keep this around since we may need them later. |
||
| }; | ||
|
There was a problem hiding this comment. Maybe we could keep tests as |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| const { expect, $ } = require('@wdio/globals'); | ||
|
|
||
| describe('Connect test', () => { | ||
| it('should call app successfully', async () => { | ||
| const selector = driver.isAndroid | ||
| ? await $('android=new UiSelector().className("android.widget.FrameLayout")') | ||
|
There was a problem hiding this comment. Can you explain to me what There was a problem hiding this comment. Yes, $ command is a short and handy way in order to fetch a single element on the page. |
||
| : await $('UIATarget.localTarget().frontMostApp().mainWindow()'); | ||
| expect(selector).toBeDisplayed(); | ||
| }); | ||
| }); | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should discuss whether we want to run e2e tests against the devapp (in the 'serve' configuration) or against the actual e-mission app (after having built it in the 'build' configuration)
I can foresee challenges with trying to run tests through the devapp, and I also think that if we are aiming to test the true behavior of the app, the test target should be as realistic as possible (ie the fully built app)
@shankari What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would also negate the need to copy the devapp apk over to here.
When the app is built, the output apk appears somewhere in
platforms/android/...