OF-2530: Optimize MUC Message History cache usage #2134
Conversation
When reading or writing cached MUC messages, the entire collection of messages is serialized between cluster nodes. This adds an unacceptable amount of overhead, for every message that is added. In this commit, the singular cached entity (a list of messages), is separated into two parts: - A cached entity that represents a list of message references per chatroom - messages, by reference The purpose of this is to optimize the scenario of adding a message (which is expected to happen more frequently than reading the history). By having to update only a list of references, instead of a list of actual objects, the amount of data that is to be operated on is reduced significantly.
|
I have already applied this change to the 4.7 branch, which helped me generate a distributable artifact that I needed for testing. If this PR is to be changed or rejected, the same needs to be applied to the commit already in the 4.7 branch. |
|
I have reverted this change on the 4.7 branch. The issue at hand was resolved by changing the 'message history' from 'all' to a finite amount. With this change in place, an unexpected (possibly unrelated) message delivery delay got introduced. As this change also introduces a second cache, which depends on faultless operation to avoid leaking orphaned entries (I failed to include a 'cleanup' routine), it might be best to keep this out of the 4.7 branch, and re-evaluate it for 4.8. |
|
@guusdk What's the status of this PR? |
|
I've not looked at it since my last comment. It's something best picked up when there's renewed activity around MUC changes. |
|
Kick back to draft? |
|
Removed from 4.8.0 milestone as beta is imminent and this would need to be in first. |
|
Is this compatible with the new MUC changes? At a guess, yes. But we'd need to solve those outstanding problems. Feels like a good change for vNext. |
When reading or writing cached MUC messages, the entire collection of messages is serialized between cluster nodes. This adds an unacceptable amount of overhead, for every message that is added.
In this commit, the singular cached entity (a list of messages), is separated into two parts:
The purpose of this is to optimize the scenario of adding a message (which is expected to happen more frequently than reading the history). By having to update only a list of references, instead of a list of actual objects, the amount of data that is to be operated on is reduced significantly.