Skip to content

The test-suite LettuceAclIntegrationTests is flaky in nature #3061

Open
@hermya

Description

@hermya

Issue

Test class LettuceAclIntegrationTests can fail non-deterministically based on the order of evaluation of ExecutionConditions.

@EnabledOnRedisAvailable(6382) // Execution condition (1)
@EnabledOnCommand("HELLO") // Execution condition (2)
class LettuceAclIntegrationTests {
    // test-code
}

What's happening?

The @EnabledOnRedisAvailable(6382) checks for the availability of Redis on port 6382. On my environment when Redis is down, I believe it is expected that @EnabledOnRedisAvailable(6382) should skip the tests in this suite. However, Reflection in java doesn't guarantee any order when handling Annotations at class, field or method level. Thus @EnabledOnRedisAvailable(6382) is never guaranteed to be evaluated before @EnabledOnCommand("HELLO").

Effect

@EnabledOnCommand("HELLO") assumes that Redis is available at default port 6379 and tries to execute the tests, resulting in the following error:

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.018 s <<< FAILURE! -- in org.springframework.data.redis.connection.lettuce.LettuceAclIntegrationTests
[ERROR] org.springframework.data.redis.connection.lettuce.LettuceAclIntegrationTests -- Time elapsed: 0.018 s <<< ERROR!
org.junit.jupiter.engine.execution.ConditionEvaluationException: Failed to evaluate condition
[org.springframework.data.redis.test.condition.EnabledOnCommandCondition]: 
Unable to connect to 127.0.0.1/<unresolved>:6379
	# REMOVING UNNECESSARY STACK TRACE
Caused by: io.lettuce.core.RedisConnectionException: Unable to connect to 127.0.0.1/<unresolved>:6379
	at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:63)
	at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:41)
	at io.lettuce.core.AbstractRedisClient.getConnection(AbstractRedisClient.java:354)
	at io.lettuce.core.RedisClient.connect(RedisClient.java:219)
	at io.lettuce.core.RedisClient.connect(RedisClient.java:204)
	at org.springframework.data.redis.test.extension.LettuceExtension$StatefulRedisConnectionSupplier.get(LettuceExtension.java:296)
	at org.springframework.data.redis.test.extension.LettuceExtension$StatefulRedisConnectionSupplier.get(LettuceExtension.java:290)
	at org.springframework.data.redis.test.extension.LettuceExtension.lambda$resolve$0(LettuceExtension.java:133)
	at org.springframework.data.redis.test.extension.LettuceExtension.resolve(LettuceExtension.java:133)
# FAULT LOCATION	at org.springframework.data.redis.test.condition.EnabledOnCommandCondition.lambda$evaluateExecutionCondition$0(EnabledOnCommandCondition.java:59)
# FAULT LOCATION	at org.springframework.data.redis.test.condition.EnabledOnCommandCondition.evaluateExecutionCondition(EnabledOnCommandCondition.java:57)
	... 14 more

(In the above log, please check lines with comments or '#')
This is the same effect as

@EnabledOnCommand("HELLO") // Execution condition (1)
@EnabledOnRedisAvailable(6382) // Execution condition (2)
class LettuceAclIntegrationTests {
    // test-code
}

Possible resolution

The logic inside EnabledOnCommandCondition assumes that Redis connection is available, and hence throws an exception. This exception can be handled so that further logic continues to check the availability of "HELLO" command, which succeeds to skip the test-case. This logic makes it failproof, independent of the order of evaluation of ExecutionConditions.
PR available here #3062

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions