Upgrade to Jedis 5.0 #2612

christophstrobl · 2023-06-15T05:31:31Z

Jedis 5 already released its 2nd beta so we could follow along on current main line till the GA is out.

jxblum · 2023-06-16T02:26:54Z

I just did a quick evaluation on upgrading Spring Data Redis to Jedis 5.0.0-beta2, and it currently leads to the following compilation errors:

[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisInvoker.java:[20,26] error: cannot find symbol
  symbol:   class Queable
  location: package redis.clients.jedis
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisInvoker.java:[1053,40] error: cannot find symbol
  symbol:   class Queable
  location: class JedisInvoker
[ERROR]   reason: class file for javax.annotation.meta.When not found
/Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterStreamCommands.java:[272,50] error: no suitable method found for xpending(byte[],byte[],byte[],byte[],int,byte[])
    method UnifiedJedis.xpending(String,String) is not applicable
      (actual and formal argument lists differ in length)
    method UnifiedJedis.xpending(String,String,XPendingParams) is not applicable
      (actual and formal argument lists differ in length)
    method UnifiedJedis.xpending(byte[],byte[]) is not applicable
      (actual and formal argument lists differ in length)
    method UnifiedJedis.xpending(byte[],byte[],XPendingParams) is not applicable
      (actual and formal argument lists differ in length)
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterHashCommands.java:[163,73] error: incompatible types: List<Entry<byte[],byte[]>> cannot be converted to Map<byte[],byte[]>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterHashCommands.java:[188,73] error: incompatible types: List<Entry<byte[],byte[]>> cannot be converted to Map<byte[],byte[]>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterServerCommands.java:[253,100] error: incompatible types: bad return type in lambda expression
    Map<String,String> cannot be converted to List<String>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterZSetCommands.java:[351,50] error: incompatible types: KeyValue<byte[],Tuple> cannot be converted to List<?>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterZSetCommands.java:[392,50] error: incompatible types: KeyValue<byte[],Tuple> cannot be converted to List<?>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterZSetCommands.java:[821,40] error: incompatible types: List<byte[]> cannot be converted to Set<byte[]>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterZSetCommands.java:[838,77] error: incompatible types: List<Tuple> cannot be converted to Set<Tuple>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterZSetCommands.java:[875,41] error: incompatible types: List<byte[]> cannot be converted to Set<byte[]>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterZSetCommands.java:[892,78] error: incompatible types: List<Tuple> cannot be converted to Set<Tuple>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterZSetCommands.java:[913,58] error: incompatible types: List<Tuple> cannot be converted to Set<Tuple>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterZSetCommands.java:[974,41] error: incompatible types: List<byte[]> cannot be converted to Set<byte[]>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterZSetCommands.java:[991,78] error: incompatible types: List<Tuple> cannot be converted to Set<Tuple>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisClusterZSetCommands.java:[1012,58] error: incompatible types: List<Tuple> cannot be converted to Set<Tuple>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisConnection.java:[327,8] error: cannot find symbol
  symbol:   method quit()
  location: variable jedis of type Jedis
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisHashCommands.java:[125,39] error: cannot find symbol
  symbol:   method entrySet()
  location: variable it of type List<Entry<byte[],byte[]>>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisHashCommands.java:[147,16] error: incompatible types: incompatible parameter types in lambda expression
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisZSetCommands.java:[328,9] error: incompatible types: invalid method reference
    incompatible types: KeyValue<byte[],Tuple> cannot be converted to List<?>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisZSetCommands.java:[359,9] error: incompatible types: invalid method reference
    incompatible types: KeyValue<byte[],Tuple> cannot be converted to List<?>
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisZSetCommands.java:[427,33] error: incompatible types: inference variable R has incompatible bounds
    lower bounds: Set<byte[]>,Object
    lower bounds: List<byte[]>
  where R,T1 are type-variables:
    R extends Object declared in method <R,T1>just(ConnectionFunction1<T1,R>,PipelineFunction1<T1,R>,T1)
    T1 extends Object declared in method <R,T1>just(ConnectionFunction1<T1,R>,PipelineFunction1<T1,R>,T1)
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisZSetCommands.java:[453,33] error: incompatible types: inference variable R has incompatible bounds
    lower bounds: Set<byte[]>,Object
    lower bounds: List<byte[]>
  where R,T1,T2 are type-variables:
    R extends Object declared in method <R,T1,T2>just(ConnectionFunction2<T1,T2,R>,PipelineFunction2<T1,T2,R>,T1,T2)
    T1 extends Object declared in method <R,T1,T2>just(ConnectionFunction2<T1,T2,R>,PipelineFunction2<T1,T2,R>,T1,T2)
    T2 extends Object declared in method <R,T1,T2>just(ConnectionFunction2<T1,T2,R>,PipelineFunction2<T1,T2,R>,T1,T2)
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisZSetCommands.java:[507,33] error: incompatible types: inference variable R has incompatible bounds
    lower bounds: Set<byte[]>,Object
    lower bounds: List<byte[]>
  where R,T1,T2 are type-variables:
    R extends Object declared in method <R,T1,T2>just(ConnectionFunction2<T1,T2,R>,PipelineFunction2<T1,T2,R>,T1,T2)
    T1 extends Object declared in method <R,T1,T2>just(ConnectionFunction2<T1,T2,R>,PipelineFunction2<T1,T2,R>,T1,T2)
    T2 extends Object declared in method <R,T1,T2>just(ConnectionFunction2<T1,T2,R>,PipelineFunction2<T1,T2,R>,T1,T2)
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisConverters.java:[629,5] error: cannot find symbol
  symbol: method addParam(String)
[ERROR] /Users/jblum/pivdev/spring-data-redis/src/main/java/org/springframework/data/redis/connection/jedis/JedisConverters.java:[632,5] error: cannot find symbol
  symbol: method addParam(String)
[INFO] 26 errors 
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  25.974 s
[INFO] Finished at: 2023-06-15T19:18:43-07:00
[INFO] ------------------------------------------------------------------------

vishalsingh2972 · 2023-06-21T06:04:39Z

@christophstrobl can you assign this to me, want to give this a try!

mp911de · 2023-07-11T13:54:37Z

Jedis 5 seems a major rewrite, judging from the amount of breaking changes: https://github.com/redis/jedis/releases/tag/v5.0.0-beta2

lzysuqianqiu · 2023-09-04T02:39:51Z

Jedis 5.0 is GA now ,any progress?

christophstrobl · 2023-09-04T07:18:01Z

@lzysuqianqiu the team is evaluating the impact associated with the quite extensive list of breaking changes.

lzysuqianqiu · 2023-09-04T07:22:52Z

okay,get it

Closes spring-projects#2612

jxblum · 2023-09-19T19:45:52Z

I have adapted SD Redis on the mainline (issue/2612 topic branch) to Jedis 5.0 API changes and all compilation issues have been resolved now.

However, there are still test failures (errors) in the Redis Pipelining and Transactional tests:

[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   JedisClusterConnectionTests.bzPopMaxShouldWorkCorrectly:2309 » RedisSystem Unknown redis exception
[ERROR]   JedisClusterConnectionTests.bzPopMinShouldWorkCorrectly:2282 » RedisSystem Unknown redis exception
[ERROR]   JedisConnectionPipelineIntegrationTests>AbstractConnectionIntegrationTests.testZDiff:2039->AbstractConnectionPipelineIntegrationTests.verifyResults:175->AbstractConnectionPipelineIntegrationTests.getResults:189 » ClassCast class java.util.ArrayList cannot be cast to class java.util.Set (java.util.ArrayList and java.util.Set are in module java.base of loader 'bootstrap')
[ERROR]   JedisConnectionPipelineIntegrationTests>AbstractConnectionIntegrationTests.testZInter:2068->AbstractConnectionPipelineIntegrationTests.verifyResults:175->AbstractConnectionPipelineIntegrationTests.getResults:189 » ClassCast class java.util.ArrayList cannot be cast to class java.util.Set (java.util.ArrayList and java.util.Set are in module java.base of loader 'bootstrap')
[ERROR]   JedisConnectionPipelineIntegrationTests>AbstractConnectionIntegrationTests.testZInterAggWeights:2085->AbstractConnectionPipelineIntegrationTests.verifyResults:175->AbstractConnectionPipelineIntegrationTests.getResults:189 » ClassCast class java.util.ArrayList cannot be cast to class java.util.Set (java.util.ArrayList and java.util.Set are in module java.base of loader 'bootstrap')
[ERROR]   JedisConnectionPipelineIntegrationTests>AbstractConnectionIntegrationTests.testZUnion:2377->AbstractConnectionPipelineIntegrationTests.verifyResults:175->AbstractConnectionPipelineIntegrationTests.getResults:189 » ClassCast class java.util.ArrayList cannot be cast to class java.util.Set (java.util.ArrayList and java.util.Set are in module java.base of loader 'bootstrap')
[ERROR]   JedisConnectionPipelineIntegrationTests>AbstractConnectionIntegrationTests.testZUnionAggWeights:2394->AbstractConnectionPipelineIntegrationTests.verifyResults:175->AbstractConnectionPipelineIntegrationTests.getResults:189 » ClassCast class java.util.ArrayList cannot be cast to class java.util.Set (java.util.ArrayList and java.util.Set are in module java.base of loader 'bootstrap')
[ERROR]   JedisConnectionTransactionIntegrationTests>AbstractConnectionIntegrationTests.testZDiff:2039->AbstractConnectionTransactionIntegrationTests.verifyResults:175->AbstractConnectionTransactionIntegrationTests.getResults:165 » ClassCast class java.util.ArrayList cannot be cast to class java.util.Set (java.util.ArrayList and java.util.Set are in module java.base of loader 'bootstrap')
[ERROR]   JedisConnectionTransactionIntegrationTests>AbstractConnectionIntegrationTests.testZInter:2068->AbstractConnectionTransactionIntegrationTests.verifyResults:175->AbstractConnectionTransactionIntegrationTests.getResults:165 » ClassCast class java.util.ArrayList cannot be cast to class java.util.Set (java.util.ArrayList and java.util.Set are in module java.base of loader 'bootstrap')
[ERROR]   JedisConnectionTransactionIntegrationTests>AbstractConnectionIntegrationTests.testZInterAggWeights:2085->AbstractConnectionTransactionIntegrationTests.verifyResults:175->AbstractConnectionTransactionIntegrationTests.getResults:165 » ClassCast class java.util.ArrayList cannot be cast to class java.util.Set (java.util.ArrayList and java.util.Set are in module java.base of loader 'bootstrap')
[ERROR]   JedisConnectionTransactionIntegrationTests>AbstractConnectionIntegrationTests.testZUnion:2377->AbstractConnectionTransactionIntegrationTests.verifyResults:175->AbstractConnectionTransactionIntegrationTests.getResults:165 » ClassCast class java.util.ArrayList cannot be cast to class java.util.Set (java.util.ArrayList and java.util.Set are in module java.base of loader 'bootstrap')
[ERROR]   JedisConnectionTransactionIntegrationTests>AbstractConnectionIntegrationTests.testZUnionAggWeights:2394->AbstractConnectionTransactionIntegrationTests.verifyResults:175->AbstractConnectionTransactionIntegrationTests.getResults:165 » ClassCast class java.util.ArrayList cannot be cast to class java.util.Set (java.util.ArrayList and java.util.Set are in module java.base of loader 'bootstrap')
[ERROR] org.springframework.data.redis.core.DefaultStreamOperationsIntegrationTests.pendingShouldReadMessageDetails
[INFO]   Run 1: PASS

And:

[ERROR]   Run 10: DefaultStreamOperationsIntegrationTests.pendingShouldReadMessageDetails:409 » RedisSystem Unknown redis exception
[ERROR]   Run 11: DefaultStreamOperationsIntegrationTests.pendingShouldReadMessageDetails:409 » RedisSystem Unknown redis exception
[ERROR]   Run 12: DefaultStreamOperationsIntegrationTests.pendingShouldReadMessageDetails:409 » RedisSystem Unknown redis exception
[ERROR]   Run 13: DefaultStreamOperationsIntegrationTests.pendingShouldReadMessageDetails:409 » RedisSystem Unknown redis exception
[ERROR]   Run 14: DefaultStreamOperationsIntegrationTests.pendingShouldReadMessageDetails:409 » RedisSystem Unknown redis exception
[ERROR]   Run 15: DefaultStreamOperationsIntegrationTests.pendingShouldReadMessageDetails:409 » RedisSystem Unknown redis exception
[ERROR]   Run 16: DefaultStreamOperationsIntegrationTests.pendingShouldReadMessageDetails:409 » RedisSystem Unknown redis exception
[ERROR]   Run 17: DefaultStreamOperationsIntegrationTests.pendingShouldReadMessageDetails:409 » RedisSystem Unknown redis exception
[ERROR]   Run 18: DefaultStreamOperationsIntegrationTests.pendingShouldReadMessageDetails:409 » RedisSystem Unknown redis exception

And, finally:

[ERROR] Tests run: 17174, Failures: 0, Errors: 13, Skipped: 444
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:57 min
[INFO] Finished at: 2023-09-18T11:41:39-07:00
[INFO] ------------------------------------------------------------------------

It seems if I solve the problem with either the SD Redis Pipelining, or the Transactional Integration Tests, I will solve the problem for the other (or be able to solve it in the same way) given the underlying problem is a ClassCastException (due to a change from an (Array)List to a Set).

I am currently trying to track the cause of these errors/failures down.

jxblum · 2023-09-19T20:09:28Z

The SD Redis JedisClusterConnectionTests.bzPopMaxShouldWorkCorrectly() test case method is currently erroring out due to a NullPointerException in the Jedis driver:

org.springframework.data.redis.RedisSystemException: Unknown redis exception

	at org.springframework.data.redis.FallbackExceptionTranslationStrategy.getFallback(FallbackExceptionTranslationStrategy.java:49)
	at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:39)
	at org.springframework.data.redis.connection.jedis.JedisClusterConnection.convertJedisAccessException(JedisClusterConnection.java:635)
	at org.springframework.data.redis.connection.jedis.JedisClusterZSetCommands.convertJedisAccessException(JedisClusterZSetCommands.java:1166)
	at org.springframework.data.redis.connection.jedis.JedisClusterZSetCommands.bZPopMax(JedisClusterZSetCommands.java:397)
	at org.springframework.data.redis.connection.DefaultedRedisConnection.bZPopMax(DefaultedRedisConnection.java:1015)
	at org.springframework.data.redis.connection.jedis.JedisClusterConnectionTests.bzPopMaxShouldWorkCorrectly(JedisClusterConnectionTests.java:2309)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.lang.NullPointerException: Cannot invoke "java.util.List.isEmpty()" because "l" is null
	at redis.clients.jedis.BuilderFactory$37.build(BuilderFactory.java:616)
	at redis.clients.jedis.BuilderFactory$37.build(BuilderFactory.java:611)
	at redis.clients.jedis.Connection.executeCommand(Connection.java:142)
	at redis.clients.jedis.executors.ClusterCommandExecutor.execute(ClusterCommandExecutor.java:139)
	at redis.clients.jedis.executors.ClusterCommandExecutor.executeCommand(ClusterCommandExecutor.java:93)
	at redis.clients.jedis.UnifiedJedis.executeCommand(UnifiedJedis.java:244)
	at redis.clients.jedis.UnifiedJedis.bzpopmax(UnifiedJedis.java:2360)
	at org.springframework.data.redis.connection.jedis.JedisClusterZSetCommands.bZPopMax(JedisClusterZSetCommands.java:395)
	... 5 more

Of course, this could caused by our test setup.

jxblum · 2023-09-19T20:11:34Z

The JedisCusterConnectionTests.bzPopMinShouldWorkCorrectly() test case method is failing for a similar reason:

org.springframework.data.redis.RedisSystemException: Unknown redis exception

	at org.springframework.data.redis.FallbackExceptionTranslationStrategy.getFallback(FallbackExceptionTranslationStrategy.java:49)
	at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:39)
	at org.springframework.data.redis.connection.jedis.JedisClusterConnection.convertJedisAccessException(JedisClusterConnection.java:635)
	at org.springframework.data.redis.connection.jedis.JedisClusterZSetCommands.convertJedisAccessException(JedisClusterZSetCommands.java:1166)
	at org.springframework.data.redis.connection.jedis.JedisClusterZSetCommands.bZPopMin(JedisClusterZSetCommands.java:356)
	at org.springframework.data.redis.connection.DefaultedRedisConnection.bZPopMin(DefaultedRedisConnection.java:994)
	at org.springframework.data.redis.connection.jedis.JedisClusterConnectionTests.bzPopMinShouldWorkCorrectly(JedisClusterConnectionTests.java:2282)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.lang.NullPointerException: Cannot invoke "java.util.List.isEmpty()" because "l" is null
	at redis.clients.jedis.BuilderFactory$37.build(BuilderFactory.java:616)
	at redis.clients.jedis.BuilderFactory$37.build(BuilderFactory.java:611)
	at redis.clients.jedis.Connection.executeCommand(Connection.java:142)
	at redis.clients.jedis.executors.ClusterCommandExecutor.execute(ClusterCommandExecutor.java:139)
	at redis.clients.jedis.executors.ClusterCommandExecutor.executeCommand(ClusterCommandExecutor.java:93)
	at redis.clients.jedis.UnifiedJedis.executeCommand(UnifiedJedis.java:244)
	at redis.clients.jedis.UnifiedJedis.bzpopmin(UnifiedJedis.java:2365)
	at org.springframework.data.redis.connection.jedis.JedisClusterZSetCommands.bZPopMin(JedisClusterZSetCommands.java:354)
	... 5 more

Upon closer inspection, it would simply seem that Jedis 5.0 no longer returns a empty (Array)List (internally) when evaluating the sorted set, and the sorted set contains no data.

Therefore, you cannot attempt to "pop" anything from a sorted set when the set is empty.

NOTE: Previously Jedis 4.x would simply return an empty (Array)List (internally) and ultimately return a null value from the bzpopmax and bzpopmin commands.

This seems like a bug in Jedis 5.0, IMO.

… tests in JedisClusterConnectionTests. Jedis 5.0 changed the bzpopmax and bzpopmin Redis commands to no longer return an empty (Array)List internally when evaluating and popping from an empty sorted set. A NullPointerException will be thrown if either bzpopmax or bzpopmin commands are executd on an empty Redis sorted set in Jedis 5.0 (vs. Jedis 4.x): Caused by: java.lang.NullPointerException: Cannot invoke 'java.util.List.isEmpty()' because l is null: at redis.clients.jedis.BuilderFactory7.build(BuilderFactory.java:616) This seems like a bug in Jedis. It is safe to execute zcard(key) to return the cardinality of (number of elements in) the sorted set before the test executes, ensuring a clean, reliable run. Closes spring-projects#2612

…ests in JedisClusterConnectionTests. Jedis 5.0 changed the bzpopmax and bzpopmin Redis commands to no longer return an empty (Array)List internally when evaluating and popping from an empty sorted set. A NullPointerException will be thrown if either bzpopmax or bzpopmin commands are executd on an empty Redis sorted set in Jedis 5.0 (vs. Jedis 4.x): Caused by: java.lang.NullPointerException: Cannot invoke 'java.util.List.isEmpty()' because l is null: at redis.clients.jedis.BuilderFactory7.build(BuilderFactory.java:616) This seems like a bug in Jedis. It is safe to execute zcard(key) to return the cardinality of (number of elements in) the sorted set before the test executes, ensuring a clean, reliable run. Closes spring-projects#2612

jxblum · 2023-09-20T05:57:44Z

Both the JedisConnectionPipelineIntegrationTests and the JedisConnectionTransactionIntegrationTests: testZDiff(), testZInter(), testZInterAggWeights(), testZUnion() and testZUnionAggWeights() test case methods are failing due to a ClassCastException:

java.lang.ClassCastException: class java.util.ArrayList cannot be cast to class java.util.Set (java.util.ArrayList and java.util.Set are in module java.base of loader 'bootstrap')

	at org.springframework.data.redis.connection.convert.SetConverter.convert(SetConverter.java:34)
	at org.springframework.data.redis.connection.DefaultStringRedisConnection.convertResults(DefaultStringRedisConnection.java:3084)
	at org.springframework.data.redis.connection.DefaultStringRedisConnection.closePipeline(DefaultStringRedisConnection.java:2456)
	at org.springframework.data.redis.connection.AbstractConnectionPipelineIntegrationTests.getResults(AbstractConnectionPipelineIntegrationTests.java:189)
	at org.springframework.data.redis.connection.AbstractConnectionPipelineIntegrationTests.verifyResults(AbstractConnectionPipelineIntegrationTests.java:175)
	at org.springframework.data.redis.connection.AbstractConnectionIntegrationTests.testZDiff(AbstractConnectionIntegrationTests.java:2039)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

Internally, when Jedis returns the "pipelined" (or transactional / "queued") results for the sorted set diff, interaction and union operations, it is returning an ArrayList, not a Set.

mp911de · 2023-09-20T06:37:27Z

Previous versions had that kind of deviation as well. We then typically compensate for such differences via fromMany(…).toList()/.toSet() to be able to make progress.

Refactor and cleanup compiler warnings. Organize source. Closes spring-projects#2612

…ctionIntegrationTests. Closes spring-projects#2612

jxblum · 2023-09-20T16:11:49Z

It is technically more involved than that.

When the Redis Pipeline is (finally) closed, the results are then converted, but are done so in the DefaultStringRedisConnection, which is a shared component between Jedis and Lettuce. Therefore, when you fix Jedis, you break Lettuce and vice versa.

You can see an early solution I was working on yesterday in my cloned, alternative topic branch.

Obviously, this fixes Jedis, but breaks Lettuce! Technically, this re-breaks the non-Pipelined, non-Transactional Jedis tests, too. So, even Jedis is inconsistent in it's return values, where sorted set operations are concerned, at least.

NOTE: As a side note, I have already observed several anomalies with the new Jedis driver, that are BUGS(!), IMO.

I also think part of the problem is that Spring Data Redis itself does not have a enough separation between Jedis and Lettuce, for example, DefaultStringRedisConnection and where Converters get applied. The "intermediate" results returned by the drivers are an "implementation" detail, yet get handled in DefaultStringRedisConnection, which is brittle at best given differences between Jedis and Lettuce in this regard.

I now need to reconsider where the changes need to (can) be applied to appease Jedis and not break Lettuce, which is a sure sign of a design issue in SDR.

Closes spring-projects#2612

…ests in JedisClusterConnectionTests. Jedis 5.0 changed the bzpopmax and bzpopmin Redis commands to no longer return an empty (Array)List internally when evaluating and popping from an empty sorted set. A NullPointerException will be thrown if either bzpopmax or bzpopmin commands are executd on an empty Redis sorted set in Jedis 5.0 (vs. Jedis 4.x): Caused by: java.lang.NullPointerException: Cannot invoke 'java.util.List.isEmpty()' because l is null: at redis.clients.jedis.BuilderFactory7.build(BuilderFactory.java:616) This seems like a bug in Jedis. It is safe to execute zcard(key) to return the cardinality of (number of elements in) the sorted set before the test executes, ensuring a clean, reliable run. Closes spring-projects#2612

Closes spring-projects#2612

…ests in JedisClusterConnectionTests. Jedis 5.0 changed the bzpopmax and bzpopmin Redis commands to no longer return an empty (Array)List internally when evaluating and popping from an empty sorted set. A NullPointerException will be thrown if either bzpopmax or bzpopmin commands are executd on an empty Redis sorted set in Jedis 5.0 (vs. Jedis 4.x): Caused by: java.lang.NullPointerException: Cannot invoke 'java.util.List.isEmpty()' because l is null: at redis.clients.jedis.BuilderFactory7.build(BuilderFactory.java:616) This seems like a bug in Jedis. It is safe to execute zcard(key) to return the cardinality of (number of elements in) the sorted set before the test executes, ensuring a clean, reliable run. Closes spring-projects#2612

jxblum · 2023-09-20T22:34:03Z

Alright!

I have dwindled the Spring Data Redis build issues (compilation and tests) down to a single test case failure involving JedisCluster.

The DefaultStreamOperationsIntegrationTests.pendingShouldReadMessageDetails() test case method is failing for all (9) parameters (combinations) of JedisCluster connections with values [ Strings, Longs, Doubles, bytes, Person, XStream**2, JSON**2 ], due to:

org.springframework.data.redis.RedisSystemException: Unknown redis exception

	at org.springframework.data.redis.FallbackExceptionTranslationStrategy.getFallback(FallbackExceptionTranslationStrategy.java:49)
	at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:39)
	at org.springframework.data.redis.connection.jedis.JedisClusterConnection.convertJedisAccessException(JedisClusterConnection.java:635)
	at org.springframework.data.redis.connection.jedis.JedisClusterStreamCommands.convertJedisAccessException(JedisClusterStreamCommands.java:399)
	at org.springframework.data.redis.connection.jedis.JedisClusterStreamCommands.xPending(JedisClusterStreamCommands.java:294)
	at org.springframework.data.redis.connection.DefaultedRedisConnection.xPending(DefaultedRedisConnection.java:580)
	at org.springframework.data.redis.connection.DefaultStringRedisConnection.xPending(DefaultStringRedisConnection.java:2993)
	at org.springframework.data.redis.connection.RedisStreamCommands.xPending(RedisStreamCommands.java:707)
	at org.springframework.data.redis.core.DefaultStreamOperations.lambda$pending$9(DefaultStreamOperations.java:205)
	at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:406)
	at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:373)
	at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:97)
	at org.springframework.data.redis.core.DefaultStreamOperations.pending(DefaultStreamOperations.java:205)
	at org.springframework.data.redis.core.DefaultStreamOperationsIntegrationTests.pendingShouldReadMessageDetails(DefaultStreamOperationsIntegrationTests.java:410)
....
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
	at redis.clients.jedis.StreamEntryID.<init>(StreamEntryID.java:24)
	at redis.clients.jedis.StreamEntryID.<init>(StreamEntryID.java:19)
	at org.springframework.data.redis.connection.jedis.JedisClusterStreamCommands.xPending(JedisClusterStreamCommands.java:283)
	... 64 more

Closes spring-projects#2612

…ests in JedisClusterConnectionTests. Jedis 5.0 changed the bzpopmax and bzpopmin Redis commands to no longer return an empty (Array)List internally when evaluating and popping from an empty sorted set. A NullPointerException will be thrown if either bzpopmax or bzpopmin commands are executd on an empty Redis sorted set in Jedis 5.0 (vs. Jedis 4.x): Caused by: java.lang.NullPointerException: Cannot invoke 'java.util.List.isEmpty()' because l is null: at redis.clients.jedis.BuilderFactory7.build(BuilderFactory.java:616) This seems like a bug in Jedis. It is safe to execute zcard(key) to return the cardinality of (number of elements in) the sorted set before the test executes, ensuring a clean, reliable run. Closes spring-projects#2612

Closes spring-projects#2612

…ests in JedisClusterConnectionTests. Jedis 5.0 changed the bzpopmax and bzpopmin Redis commands to no longer return an empty (Array)List internally when evaluating and popping from an empty sorted set. A NullPointerException will be thrown if either bzpopmax or bzpopmin commands are executd on an empty Redis sorted set in Jedis 5.0 (vs. Jedis 4.x): Caused by: java.lang.NullPointerException: Cannot invoke 'java.util.List.isEmpty()' because l is null: at redis.clients.jedis.BuilderFactory7.build(BuilderFactory.java:616) This seems like a bug in Jedis. It is safe to execute zcard(key) to return the cardinality of (number of elements in) the sorted set before the test executes, ensuring a clean, reliable run. Closes spring-projects#2612

jxblum · 2023-09-20T23:04:27Z

Spring Data Redis to Jedis 5.0 upgrade complete!

Closes spring-projects#2612

…ests in JedisClusterConnectionTests. Jedis 5.0 changed the bzpopmax and bzpopmin Redis commands to no longer return an empty (Array)List internally when evaluating and popping from an empty sorted set. A NullPointerException will be thrown if either bzpopmax or bzpopmin commands are executd on an empty Redis sorted set in Jedis 5.0 (vs. Jedis 4.x): Caused by: java.lang.NullPointerException: Cannot invoke 'java.util.List.isEmpty()' because l is null: at redis.clients.jedis.BuilderFactory7.build(BuilderFactory.java:616) This seems like a bug in Jedis. It is safe to execute zcard(key) to return the cardinality of (number of elements in) the sorted set before the test executes, ensuring a clean, reliable run. Closes spring-projects#2612

Closes spring-projects#2612

…ests in JedisClusterConnectionTests. Jedis 5.0 changed the bzpopmax and bzpopmin Redis commands to no longer return an empty (Array)List internally when evaluating and popping from an empty sorted set. A NullPointerException will be thrown if either bzpopmax or bzpopmin commands are executd on an empty Redis sorted set in Jedis 5.0 (vs. Jedis 4.x): Caused by: java.lang.NullPointerException: Cannot invoke 'java.util.List.isEmpty()' because l is null: at redis.clients.jedis.BuilderFactory7.build(BuilderFactory.java:616) This seems like a bug in Jedis. It is safe to execute zcard(key) to return the cardinality of (number of elements in) the sorted set before the test executes, ensuring a clean, reliable run. Closes spring-projects#2612

jxblum · 2023-09-26T21:22:45Z

Upgrading to Jedis 5.0.1, which is now available.

Closes spring-projects#2612 Original pull request: spring-projects#2716

christophstrobl added the type: dependency-upgrade A dependency upgrade label Jun 15, 2023

jxblum self-assigned this Sep 14, 2023

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 15, 2023

Adapt to API changes in the Jedis 5.0 driver.

ec121d9

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 15, 2023

Upgrade to Jedis 5.0.

3db8c6d

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 15, 2023

Adapt to API changes in the Jedis 5.0 driver.

cfa00d7

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 16, 2023

Adapt to API changes in the Jedis 5.0 driver.

9001ef7

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 16, 2023

Upgrade to Jedis 5.0.

aa34bf0

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 16, 2023

Adapt to API changes in the Jedis 5.0 driver.

f453309

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 18, 2023

Adapt to API changes in the Jedis 5.0 driver.

4a47976

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 20, 2023

Adapt DefaultStringRedisConnection to API changes in Jedis 5.0.

8255358

Refactor and cleanup compiler warnings. Organize source. Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 20, 2023

Fix JedisConnectionPipelineIntegrationTests and JedisConnectionTransa…

c639531

…ctionIntegrationTests. Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 20, 2023

Adapt to API changes in the Jedis 5.0 driver.

c3e25cc

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 20, 2023

Adapt to API changes in the Jedis 5.0 driver.

0bcae85

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 20, 2023

Adapt to API changes in the Jedis 5.0 driver.

461456b

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 20, 2023

Adapt to API changes in the Jedis 5.0 driver.

c1741a7

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 21, 2023

Upgrade to Jedis 5.0.

5633f07

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 21, 2023

Adapt to API changes in the Jedis 5.0 driver.

1070ee0

Closes spring-projects#2612

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 25, 2023

Adapt to API changes in the Jedis 5.0 driver.

6fe6663

Closes spring-projects#2612

jxblum mentioned this issue Sep 25, 2023

Upgrade to Jedis 5.0 #2716

Closed

jxblum added a commit to jxblum/spring-data-redis that referenced this issue Sep 26, 2023

Upgrade to Jedis 5.0.1.

c79d4cd

Closes spring-projects#2612 Original pull request: spring-projects#2716

christophstrobl closed this as completed in 2075633 Oct 4, 2023

christophstrobl added this to the 3.2 RC1 (2023.1.0) milestone Oct 4, 2023

mp911de mentioned this issue Nov 1, 2023

java.lang.NoClassDefFoundError: redis/clients/jedis/Queable #2758

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to Jedis 5.0 #2612

Upgrade to Jedis 5.0 #2612

christophstrobl commented Jun 15, 2023

jxblum commented Jun 16, 2023 •

edited

Loading

vishalsingh2972 commented Jun 21, 2023

mp911de commented Jul 11, 2023

lzysuqianqiu commented Sep 4, 2023

christophstrobl commented Sep 4, 2023 •

edited

Loading

lzysuqianqiu commented Sep 4, 2023

jxblum commented Sep 19, 2023 •

edited

Loading

jxblum commented Sep 19, 2023 •

edited

Loading

jxblum commented Sep 19, 2023 •

edited

Loading

jxblum commented Sep 20, 2023

mp911de commented Sep 20, 2023

jxblum commented Sep 20, 2023

jxblum commented Sep 20, 2023 •

edited

Loading

jxblum commented Sep 20, 2023

jxblum commented Sep 26, 2023

Upgrade to Jedis 5.0 #2612

Upgrade to Jedis 5.0 #2612

Comments

christophstrobl commented Jun 15, 2023

jxblum commented Jun 16, 2023 • edited Loading

vishalsingh2972 commented Jun 21, 2023

mp911de commented Jul 11, 2023

lzysuqianqiu commented Sep 4, 2023

christophstrobl commented Sep 4, 2023 • edited Loading

lzysuqianqiu commented Sep 4, 2023

jxblum commented Sep 19, 2023 • edited Loading

jxblum commented Sep 19, 2023 • edited Loading

jxblum commented Sep 19, 2023 • edited Loading

jxblum commented Sep 20, 2023

mp911de commented Sep 20, 2023

jxblum commented Sep 20, 2023

jxblum commented Sep 20, 2023 • edited Loading

jxblum commented Sep 20, 2023

jxblum commented Sep 26, 2023

jxblum commented Jun 16, 2023 •

edited

Loading

christophstrobl commented Sep 4, 2023 •

edited

Loading

jxblum commented Sep 19, 2023 •

edited

Loading

jxblum commented Sep 19, 2023 •

edited

Loading

jxblum commented Sep 19, 2023 •

edited

Loading

jxblum commented Sep 20, 2023 •

edited

Loading