Intermittent issues in spec

[kitallis]

Hi, thanks for the library. I wanted to use this primarily because of the queuing semantics that other implementations don't offer. I am now officially running this in production, but I'm seeing a few intermittent errors in specs, which are sometimes rather random and baffling to understand.

Example 1

https://github.com/tramlinehq/tramline/actions/runs/13515882931/job/37764372571?pr=734

What's happening here is that the lock is wrapped in a Result object and that swallows the exception, but the internal exception appears to be in try_to_lock (the stacktrace is separately printed up above). The error here isn't just the one you see, I've seen it fail for a variety of random reasons.

This is the core part of the code that handles locking and external API calls.

This is how I initialize the redlock.

Example 2

I've also seen a couple of errors on my non-production environment like this:

I was wondering if you've seen any of this yourself. Any pointers or insight would really help! Happy to provide more data.

Sidenote: On my cursory glance at the code, I noticed that the multi calls to redis are wrapped over a timeout, the timeout is handled, but I'm curious what happens when the thread times out in between the connection running, is the connection closed cleanly?

Sidenote: Some context on why I'm using this library: tramlinehq/tramline#731

[0exp]

Hi, kitallis!

Thank you for detailed explanation about the problem 👍🏻 . I think the information provided is enough (but i am not 100% sure 😄 ) 🤝🤝 (and I'll look into other failed CI jobs in your repository for more details 👌🏻 )

---

I will look into your issue soon 👌🏻 and will be back soon with some suggestions for a solution

[kitallis]

Ok I got RedisClient::ConnectionError on production as well. Here's the exception: https://tramline.sentry.io/share/issue/9a6c9deac7c94c5f939172a4def28dbf and the other "weird" error https://tramline.sentry.io/share/issue/f0577e0e20024deba1db18194817a1ed

EDIT

Here's another one: https://tramline.sentry.io/share/issue/27ff1ce25ef144ea8594ee2a689090ea

I'll unfortunately have to stop using this library for now, but happy to come back and try any updates.

[0exp]

@kitallis hello! i have investigated problems you reached 🤔 But i need some more information.

---

Could you please provide an example of how you instantiates the redis-client (in the past, before the library swap (update: i found :))) for RQL and how you run your instances of application? (puma/processs count/thread counts/redis pool size) 🤔 I think this problem can happen when the connection pool of redis-client have fewer size than puma thread count 🤔 (not sure, need more information to investigate) OR when something starts to close all threads form elsewhere and this affects our connection pool (need more information too :D to investigate).

And in (https://tramline.sentry.io/share/issue/27ff1ce25ef144ea8594ee2a689090ea / https://tramline.sentry.io/share/issue/f0577e0e20024deba1db18194817a1ed/) do you have any info what key of lock was there? (super strange situation cuz each lock is represented as a hash inside the redis) And how this lock was acquired inside the code (update: found) 🤔 I think it may relates to the redis-client default redis-connection-driver implementation (pure ruby variant) (https://github.com/redis-rb/redis-client/blob/master/lib/redis_client/ruby_connection.rb), and problems around the stream closing, cuz it is super strange that HGETALL returns non-array representation. I will try to setup the test stand for this for further investigation.

---

UPDTE: (error details)

• TypeError: wrong element type String at 0 (expected array) (TypeError) - this happened cuz the HGETALL returns non-array represnetation of lock data, so we trying to extract an element from array by the array index operation but trying to do this on a string object. so we failed. 🤔
• NoMethodError: undefined method 'to_h' for an instance of String (NoMethodError) - the same situation cuz the redis-client returned a string object from HGETALL redis command. so we failed; 🤔

The main question is "why RedisClient's .call("HGETALL", ...) returns non-array/non-hash object" 🤔 (i will investigate)

redlock-rb doesn't have such problems, because it works with locks as with simple values (SET / GET) not with hashes.

---

Could you please setup the hiredis driver for a little time, and run some number of CI builds in order to check if we can caught some errors or not 🤔 I think i have no problems like these cuz i use another IO/Connection driver in my projects 🤔

---

Thank you 🙏🏻