feat: use redis pubsub, transactions, locks

[tituschewxj]

Summary

This PR implements:

• Redis transactions. This allows either the matching to be successful or rolled back if something goes wrong.

• Organize files and functions to be more structured

• Refactors the matching algorithm

  • Removes the mutex in the matching process, because if the operations are atomic, then no locks are necessary.
  • Removes the for loop in the matching process, as we only need to check the queue when a new user joins the queue, starting from the front of the queue. If no new users joins, then there is no point to run the matching algorithm again. This removes the overhead from busy waiting.

• Uses Redis pub/sub to handle the sending of a match result instead of Go channels, to support inter-service communication

  • Removes maps for active connections, instead database is used to check duplicate users.

• Uses Redis lock to ensure only one process modifies the database at any time to ensure correctness of matching algorithm

The server logs have also be slightly tweaked:

2024/10/23 16:25:56 Connected to Redis at the following address: localhost:6379
2024/10/23 16:25:56 Server starting on :8081
2024/10/23 16:25:58 WebSocket connection established for port 49158
2024/10/23 16:25:58 Received match request: {match_request [Algorithms Arrays] [Easy Medium] Timothy} from client port: 49158
2024/10/23 16:25:58 Redis Queue (Before Matchmaking): Timothy
2024/10/23 16:25:59 WebSocket connection established for port 49164
2024/10/23 16:25:59 Received match request: {match_request [Algorithms Graphs] [Medium Hard] Jennie} from client port: 49164
2024/10/23 16:25:59 Redis Queue (After Matchmaking): Timothy
2024/10/23 16:25:59 Redis Queue (Before Matchmaking): Jennie, Timothy
2024/10/23 16:25:59 Users Jennie and Timothy matched on the topic: Algorithms with difficulty: Medium
2024/10/23 16:25:59 Connection closed at port 49164
2024/10/23 16:25:59 Connection closed at port 49158
2024/10/23 16:26:00 Redis Queue (After Matchmaking): 

• Note that the lines: 2024/10/23 16:25:59 Match found for user: Jennie and 2024/10/23 16:25:59 Match found for user: Timothy have been removed as the information is already present elsewhere in the logs.

Background

In Redis, multiple concurrent transactions can occur if they operate on different resources (i.e., different keys), but Redis still processes commands sequentially due to its single-threaded nature.

If multiple clients are interacting with different keys (i.e., different resources), there is no contention, and Redis processes the transactions or commands one after another without conflict.

Redis supports transactions through the use of the MULTI, EXEC, WATCH, and DISCARD commands. Transactions in Redis allow you to queue multiple commands to be executed atomically.

Current Issue

Currently, we are using a process level lock, which does not guarantee atomicity when multiple instance of the matching service is present. Hence the need for a database level atomic operations to be implemented.

[bensohh]

LGTM!

@tituschewxj apart from the comments given I think the rest are good to go. Take a look at the comments, I just wanted to clarify some cases in the implementation:)

[bensohh]

In this case, each Websocket connection would trigger the PerformMatching goroutine, then within the PerformMatching goroutine if they fail to obtain lock would they retry? or just return and end ?

    lock, err := databases.ObtainRedisLock(ctx)
	if err != nil {
		return
	}

Possible case?:
user1 triggers perform matching
user2 and user3 tries to trigger perform matching but both fail to obtain lock so they return but user2 and user3 are a match

[tituschewxj]

If they fail to retrieve the lock, they would retry every 100ms up to 100 times. If they still cannot obtain the lock after this than it just returns and it ends.

func ObtainRedisLock(ctx context.Context) (*redislock.Lock, error) {
	// Retry every 100ms, for up-to 100x
	backoff := redislock.LimitRetry(redislock.LinearBackoff(100*time.Millisecond), 100)

	// Obtain lock with retry
	lock, err := redisLock.Obtain(ctx, matchmakingRedisLock, time.Second, &redislock.Options{
		RetryStrategy: backoff,
	})
	if err == redislock.ErrNotObtained {
		fmt.Println("Could not obtain lock!")
		return nil, err
	} else if err != nil {
		log.Fatalln(err)
		return nil, err
	}
	return lock, err
}

So a possible improvement could be to add errorChan <- err so that an error is returned.

[bensohh]

Links to the previous comment regarding the retry logic if it fails to obtain the lock