Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Behat work again (quick workaround) #110

Open
kabalin opened this issue Feb 21, 2020 · 5 comments
Open

Make Behat work again (quick workaround) #110

kabalin opened this issue Feb 21, 2020 · 5 comments

Comments

@kabalin
Copy link
Contributor

kabalin commented Feb 21, 2020

Most moodle-plugin-ci users are experiencing issues with Behat tests which stopped working at some point last year. While PR #107 is under consideration, this is a quick workaround to make it work again:

language: php

addons:
  postgresql: "9.4"

services:
- mysql
- postgresql
- docker

cache:
  directories:
  - $HOME/.composer/cache
  - $HOME/.npm

php:
- 7.1
- 7.2
- 7.3

env:
  global:
  # Set PROFILE to either chrome or firefox depending on your preference.
  - PROFILE=chrome
  # This prevents starting selenium and php servers using moodle-plugin-ci in-built functionality.
  - MOODLE_START_BEHAT_SERVERS=NO
  matrix:
  - DB=pgsql
  - DB=mysqli

before_install:
- phpenv config-rm xdebug.ini
- nvm install 8.9
- nvm use 8.9
- cd ../..
- composer create-project -n --no-dev --prefer-dist blackboard-open-source/moodle-plugin-ci ci ^2
- export PATH="$(cd ci/bin; pwd):$(cd ci/vendor/bin; pwd):$PATH"

install:
- moodle-plugin-ci install -vvv
# Start Selenuim Standalone server using docker.
- docker run -d -p 127.0.0.1:4444:4444 --net=host -v /dev/shm:/dev/shm -v $HOME/build/moodle:$HOME/build/moodle selenium/standalone-${PROFILE}:3
# Start php built-in server.
- php -S localhost:8000 -t $HOME/build/moodle > /dev/null 2>&1 &
# Wait to make sure docker container is fully running.
- sleep 10

script:
- moodle-plugin-ci phplint
- moodle-plugin-ci phpcpd
- moodle-plugin-ci phpmd
- moodle-plugin-ci codechecker
- moodle-plugin-ci validate
- moodle-plugin-ci savepoints
- moodle-plugin-ci mustache
- moodle-plugin-ci grunt
- moodle-plugin-ci phpdoc
- moodle-plugin-ci phpunit
- moodle-plugin-ci behat --profile $PROFILE

To sum up, we use MOODLE_START_BEHAT_SERVERS=NO to disable built-in selenium and php server initialisation, and then start respective services in install stage after installation step. Selenuim server is initialised using docker image.

Please note, this requires Ubuntu Xenial build environment (currently default one in Travis).

@timhunt
Copy link

timhunt commented Feb 22, 2020

Thanks for this tip @kabalin, which almost worked to get the qtype_stack Behat tests passing again. However, it did not work: https://travis-ci.org/timhunt/moodle-qtype_stack/jobs/653818879

2 errors:

  1. [WebDriver\Exception\CurlExec (-1)]
    599 Webdriver http error: 404, payload :{
    600 "sessionId": "cookie",
    601 "value": {
    602 "error": "invalid session id",
    603 "message": "No active session with ID cookie",
    604 "stacktrace": ""
    605 },
    606 "status": 6
    607 }

  2. 001 Scenario: Restore the STACK demo course. # /home/travis/build/moodle/question/type/stack/tests/behat/restore_demo.feature:15
    689 And I upload "question/type/stack/samplequestions/STACK-demo.mbz" file to "Files" filemanager # /home/travis/build/moodle/question/type/stack/tests/behat/restore_demo.feature:11
    690 Exception: invalid argument: File not found : /home/travis/build/moodle/question/type/stack/samplequestions/STACK-demo.mbz

Have you seen these before? Do you know what they mean?

@kabalin
Copy link
Contributor Author

kabalin commented Feb 24, 2020

Hello @timhunt! No, I did not see similar errors yet. Regarding the first one, I see in the CI log that you run behat test immediately following webserver and senenium container start. I think it could be that behat test is executed before Selenium container is fully ready (as you see it does not fail on the re-run)? I tried adding sleep 10 before leaving install stage (I added this to example above as well), and this seems helped:

https://travis-ci.org/kabalin/moodle-qtype_stack/jobs/653942531

Notice, I also removed some obsolete dependencies in your travis file, we do not need Behat related packages since we run it in container and also Java 9 is not required (we use default one which is version 11, mustache issue associated with this was addressed in #91). You can keep bionic requirement, I removed it only to test if this will resolve your second error.

Regarding second error, not sure why this does not work, worth trying with Firefox driver may be to confirm?

@kabalin
Copy link
Contributor Author

kabalin commented Mar 11, 2020

001 Scenario: Restore the STACK demo course. # /home/travis/build/moodle/question/type/stack/tests/behat/restore_demo.feature:15
689 And I upload "question/type/stack/samplequestions/STACK-demo.mbz" file to "Files" filemanager # /home/travis/build/moodle/question/type/stack/tests/behat/restore_demo.feature:11
690 Exception: invalid argument: File not found : /home/travis/build/moodle/question/type/stack/samplequestions/STACK-demo.mbz

@timhunt after some trial and error this issue has been fixed by mounting Moodle directory inside selenium container, i.e.:

- docker run -d -p 4444:4444 --net=host -v /dev/shm:/dev/shm -v $HOME/build/moodle:$HOME/build/moodle selenium/standalone-chrome:3

Looks like webdriver expects file to be local when upload step is used, so when we are using docker container, it is obviously not there. Here is a successful build of qtype_stack using this change: https://travis-ci.org/github/kabalin/moodle-qtype_stack/jobs/661027012

@timhunt
Copy link

timhunt commented Jun 28, 2020

@kabalin I just wanted to say a huge "thank you" to you for working this out and sharing it. I have been making heavy use of it today, verifying my plugins for Moodle 3.9.

@kabalin
Copy link
Contributor Author

kabalin commented Jun 29, 2020

Not at all @timhunt, glad it was helpful 😃

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants