Cannot use `Assert::assertXXX` methods outside the PHPUnit runtime since 11.3.0

[acoulton]

Q	A
PHPUnit version	>= 11.3.0
PHP version	8.3.19
Installation Method	Composer

Summary

Before 11.3.0, it was possible to use the PHPUnit Assert::assertXXX methods outside a PHPUnit execution e.g. as standalone assertion methods.

I am not sure if this was officially recommended / supported by PHPUnit, but it has for example been recommended for 11 years by the Behat project.

I do note that neither the Assert class itself nor the methods have any warnings about not being covered by the BC promise.

However, since #5869, if an assertion fails then instead of the assertion message we get an internal error.

Current behavior

If an assertion fails, we get:

AssertionError: assert(self::$instance instanceof Configuration) in {base}/vendor/phpunit/phpunit/src/TextUI/Configuration/Registry.php:97
Stack trace:
#0 {base}/vendor/phpunit/phpunit/src/TextUI/Configuration/Registry.php(97): assert(false, 'assert(self::$i...')
#1 {base}/vendor/phpunit/phpunit/src/Util/Exporter.php(47): PHPUnit\TextUI\Configuration\Registry::get()
#2 {base}/vendor/phpunit/phpunit/src/Util/Exporter.php(24): PHPUnit\Util\Exporter::exporter()
#3 {base}/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php(133): PHPUnit\Util\Exporter::export(false)
#4 {base}/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php(92): PHPUnit\Framework\Constraint\Constraint->failureDescription(false)
#5 {base}/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php(58): PHPUnit\Framework\Constraint\Constraint->fail(false, '')
#6 {base}/vendor/phpunit/phpunit/src/Framework/Assert.php(2570): PHPUnit\Framework\Constraint\Constraint->evaluate(false, '')
#7 {base}/vendor/phpunit/phpunit/src/Framework/Assert.php(1573): PHPUnit\Framework\Assert::assertThat(false, Object(PHPUnit\Framework\Constraint\IsTrue), '')
#8 {base}/test.php(6): PHPUnit\Framework\Assert::assertTrue(false)
#9 {main}

How to reproduce

Use the following script:

<?php

require_once __DIR__.'/vendor/autoload.php';

try {
  \PHPUnit\Framework\Assert::assertTrue(false);
} catch (Throwable $e) {
  print $e->__toString()."\n";
}

Observe that with PHPUnit >= 11.3.0, we get the error above.

Expected behavior

On PHPUnit <= 11.2.9 (and going back for a very long time) that script outputs:

PHPUnit\Framework\ExpectationFailedException: Failed asserting that false is true. in {base}/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:98
Stack trace:
#0 {base}/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php(51): PHPUnit\Framework\Constraint\Constraint->fail(false, '')
#1 {base}/vendor/phpunit/phpunit/src/Framework/Assert.php(1973): PHPUnit\Framework\Constraint\Constraint->evaluate(false, '')
#2 {base}/vendor/phpunit/phpunit/src/Framework/Assert.php(940): PHPUnit\Framework\Assert::assertThat(false, Object(PHPUnit\Framework\Constraint\IsTrue), '')
#3 {base}/test.php(6): PHPUnit\Framework\Assert::assertTrue(false)
#4 {main}

[sebastianbergmann]

This use case was never supported.

[loru88]

Thank you for your quick answer @sebastianbergmann , it would be a useful use case if we could consider it in the future.
PHPUnit got a pretty powerful library of Asserts, it would be useful to reuse it in Behat or other testing runtime.

Would you have any recommendation against that if one works on it?

[sebastianbergmann]

I do not want to support this, sorry.

[acoulton]

@sebastianbergmann thanks for the quick response - that's disappointing, but not entirely unexpected.

I do think in that case the Assert class should be explicitly marked as only for use within a PHPUnit runtime as currently it appears to be covered by the BC promise and I'm not aware there's ever been anything to say it shouldn't be used separately.

[sebastianbergmann]

It is covered by the BC promise for PHPUnit, but only for use in tests that are written for / run with PHPUnit.

[acoulton]

@sebastianbergmann understood - but my argument is that's not clear, and perhaps could be. I think a user might reasonably expect (and clearly over the years people have expected) that a public static method in a library should be callable from anywhere unless advised otherwise.