Parameters are not validated in local `@MethodSource` factory method

[sbrannen]

Overview

The current implementation of support for local factory method names that accept arguments in @MethodSource does not validate the specified parameters. Rather, the lookup is performed based solely on the names of the parameters. Furthermore, the lookup ignores the parameters if there is only one factory method with the specified name.

This can lead to incorrect configuration being silently ignored or confusing error messages when the specified factory method is overloaded.

Examples

Given the following extensions:

class IntegerResolver implements ParameterResolver {
	@Override
	public boolean supportsParameter(ParameterContext pc, ExtensionContext ec) {
		return pc.getParameter().getType() == int.class;
	}
	@Override
	public Object resolveParameter(ParameterContext pc, ExtensionContext ec) {
		return 2;
	}
}

class IntegerArrayResolver implements ParameterResolver {
	@Override
	public boolean supportsParameter(ParameterContext pc, ExtensionContext ec) {
		return pc.getParameter().getType() == int[].class;
	}
	@Override
	public Object resolveParameter(ParameterContext pc, ExtensionContext ec) {
		return new int[] { 2, 3 };
	}
}

And given the following test class:

@ExtendWith({IntegerResolver.class, IntegerArrayResolver.class})
class MethodSourceTests {

	@ParameterizedTest
	// @MethodSource("example.MethodSourceTests#factory(dog)")
	@MethodSource("factory(dog)")
	void test(String argument) {
		assertTrue(argument.startsWith("2"));
	}

	static Stream<Arguments> factory(int quantities) {
		return Stream.of(arguments(quantities + " apples"), arguments(quantities + " lemons"));
	}

//	static Stream<Arguments> factory(int[] quantities) {
//		return Stream.of(arguments(quantities[0] + " apples"), arguments(quantities[0] + " lemons"));
//	}

}

If we run the test "as is", it will pass because the dog parameter is not validated. It is in fact completely ignored since there are no "competing" overloaded factory methods with the same name.

Whereas, if we attempt to do the same using the fully qualified method name (FQMN) syntax (@MethodSource("example.MethodSourceTests#factory(dog)")), we get the following exception.

org.junit.platform.commons.JUnitException: Failed to load parameter type [dog] for method [factory] in class [example.MethodSourceTests].

Similarly, if we supply a valid parameter type for a factory method that does not exist using the FQMN syntax (@MethodSource("example.MethodSourceTests#factory(java.lang.Integer)")), we get the following exception.

org.junit.platform.commons.JUnitException: Could not find factory method [factory(java.lang.Integer)] in class [example.MethodSourceTests]

In contrast, using the local factory method syntax (@MethodSource("factory(java.lang.Integer)")), the test will pass since the parameters are ignored (as with the dog example).

If we uncomment the factory(int[]) method and run the test, we get the following somewhat confusing error message.

org.junit.platform.commons.PreconditionViolationException: 2 factory methods named [factory(dog)] were found in class [example.MethodSourceTests]: [static java.util.stream.Stream example.MethodSourceTests.factory(int), static java.util.stream.Stream example.MethodSourceTests.factory(int[])]

Related Issues

• Syntax for local @MethodSource factory method does not support canonical array names #3131
• Local qualified @MethodSource factory method name treated differently than FQMN #3266

Deliverables

• Validate factory method parameter types supplied to @MethodSource using the local factory method name syntax.
• Ensure that error messages for configuration errors encountered when using the local factory method name syntax align with the error messages produced for similar configuration errors when using the fully qualified method name syntax.

[sbrannen]

Reopening to document in release notes.