Support for PHPUnit 10/11 and WebDriver 4

[andronocean]

I'm setting up a new project using wp-browser, and I discovered that there's a conflict between PHPUnit v10 (or higher) and wp-browser.

Part of the trouble comes from the codeception/module-webdriver dependency, for which a version range of "^2.0 || ^3.0" is set.

module-webdriver introduced support for PHPUnit 10 in 4.0, and PHPUnit 11 in 4.0.1.

Given that PHPUnit 9 reached end of bugfix support in February, I'd like to use at least version 10 — especially for new work!

Is there anything special that needs to be done to support the dependency upgrade? I noticed that wp-browser doesn't explicitly declare a specific PHPUnit version dependency, but I'm not nearly familiar enough with the internals to know if that's a guarantee.

(I did a quick test locally, adding wp-browser as a package repository and bumping the dependency line to "codeception/module-webdriver": "^2.0 || ^3.0 || ^4.0",., then requiring PHPUnit 10. My EndToEnd tests using the WPWebDriver module still complete successfully.)

[andronocean]

Well, I just found one issue when trying to use PHPUnit 10 in the generated Integration test suite.

I have a test class SiteConfigWorksTest extends \lucatume\WPBrowser\TestCase\WPTestCase, which I generated with the generate:wpunit command. When I run it I get the following fatal error (paths truncated for clarity):

FATAL ERROR. TESTS NOT FINISHED.
Uncaught ArgumentCountError: Too few arguments to function PHPUnit\Framework\TestCase::__construct(), 0 passed in ..../vendor/lucatume/wp-browser/src/TestCase/WPTestCase.php on line 244 and exactly 1 expected in ..../vendor/phpunit/phpunit/src/Framework/TestCase.php:339
Stack trace:
#0 ..../vendor/lucatume/wp-browser/src/TestCase/WPTestCase.php(244): PHPUnit\Framework\TestCase->__construct()
#1 ..../vendor/lucatume/wp-browser/src/TestCase/WPTestCasePHPUnitMethodsTrait.php(51): lucatume\WPBrowser\TestCase\WPTestCase::getCoreTestCase()
#2 ..../vendor/codeception/codeception/src/Codeception/Subscriber/BeforeAfterTest.php(50): lucatume\WPBrowser\TestCase\WPTestCase::setUpBeforeClass()
#3 ..../vendor/codeception/codeception/src/Codeception/Subscriber/BeforeAfterTest.php(31): Codeception\Subscriber\BeforeAfterTest->executeMethods(Object(Tests\SiteConfigWorksTest), Array)
#4 ..../vendor/symfony/event-dispatcher/EventDispatcher.php(206): Codeception\Subscriber\BeforeAfterTest->beforeClass(Object(Codeception\Event\SuiteEvent), 'suite.before', Object(Symfony\Component\EventDispatcher\EventDispatcher))
#5 ..../vendor/symfony/event-dispatcher/EventDispatcher.php(56): Symfony\Component\EventDispatcher\EventDispatcher->callListeners(Array, 'suite.before', Object(Codeception\Event\SuiteEvent))
#6 ..../vendor/codeception/codeception/src/Codeception/SuiteManager.php(148): Symfony\Component\EventDispatcher\EventDispatcher->dispatch(Object(Codeception\Event\SuiteEvent), 'suite.before')
#7 ..../vendor/codeception/codeception/src/Codeception/Codecept.php(260): Codeception\SuiteManager->run(Object(Codeception\ResultAggregator))
#8 ..../vendor/codeception/codeception/src/Codeception/Codecept.php(216): Codeception\Codecept->runSuite(Array, 'Integration', NULL)
#9 ..../vendor/codeception/codeception/src/Codeception/Command/Run.php(646): Codeception\Codecept->run('Integration')
#10 ..../vendor/codeception/codeception/src/Codeception/Command/Run.php(467): Codeception\Command\Run->runSuites(Array, Array)
#11 ..../vendor/lucatume/wp-browser/src/Command/RunAll.php(28): Codeception\Command\Run->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 ..../vendor/symfony/console/Command/Command.php(279): lucatume\WPBrowser\Command\RunAll->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 ..../vendor/symfony/console/Application.php(1029): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 ..../vendor/symfony/console/Application.php(316): Symfony\Component\Console\Application->doRunCommand(Object(lucatume\WPBrowser\Command\RunAll), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 ..../vendor/symfony/console/Application.php(167): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 ..../vendor/codeception/codeception/src/Codeception/Application.php(112): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 ..../vendor/codeception/codeception/app.php(45): Codeception\Application->run()
#18 ..../vendor/codeception/codeception/app.php(46): {closure}()
#19 ..../vendor/codeception/codeception/codecept(7): require('/Users/andron/C...')
#20 ..../vendor/bin/codecept(119): include('/Users/andron/C...')
#21 {main}
  thrown
in ..../vendor/phpunit/phpunit/src/Framework/TestCase.php:339

The issue appears to be that PHPUnit 9's TestCase class has the following __construct() signature (source):

public function __construct(?string $name = null, array $data = [], $dataName = '')

... which has changed in PHPUnit 10 to

public function __construct(string $name)

(source)

[lucatume]

@andronocean thanks for taking the time to open an issue and report your findings.

I've created an exploratory PR for this (#737), I would appreciate you testing it out and reporting your findings in this issue.

A caveat: the WPTestCase is a wrapper around the PHPUnit-based core test suite; that suite supports, at the most, PHPUnit version 9.5, so this might prove to be a challenge.

Keeping in mind that "end-of-life" for a testing package does not mean danger, what features of PHPUnit 10 or 11 do you want to use that would make this a worthwhile exploration?

[andronocean]

Excellent, thank you! I will try out the PR tomorrow and let you know what I find. I hadn't realized that about the Core test suite, so yeah... that will be interesting 👀

To connect the dots, it looks like this is the Core ticket for 10/11 compatibility: https://core.trac.wordpress.org/ticket/59486. Seems like a big undertaking so I'm not going anticipate a quick fix here.

Honestly the biggest factor pushing me toward PHPUnit 10+ is using attributes for metadata (like designating data providers or coverage) instead of annotations in comment blocks. Especially for new projects that will have long-term support, I don't like the feeling of using a syntax that I know has already been deprecated (though I know there are ways like Rector to automate the migration). Some of the changes around testing exceptions and errors look like very positive developments as well, though I haven't had a chance to try them yet.

[lucatume]

Thanks for your reply, I will look with a more attention to the new PHPUnit features.

Depending on how hard it turns out to be, I'm willing to explore the support for newer PHPUnit versions.
Running ahead of the Core PHPUnit suite is a daring endeavour, but this project is taking on some other challenges in a stride (e.g. supporting test isolation when Codeception decided not to) and I like the challenge.

Let me know when you have updates or findings.

[andronocean]

Update: still early in testing, but so far the PR branch has been working flawlessly. I specified "phpunit/phpunit": "^9.6 || ^10.0" in composer.json, so I'm running 10.5.24. All test suites run successfully; the previous error with the Integration one is gone. I'll keep you posted about anything unusual I encounter.

[lucatume]

Thanks for the updates.

I will come back to this and fix the CI issues for a first, experimental, merge of a PHPUnit version 10 and 11 support in a bit.

All feedback is welcome in the meanwhile.

[github-actions]

This issue is marked stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.