Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
9500754
test: increase command test coverage for resp2/resp3 difference analysis
nkaradzhov Mar 31, 2026
151f340
Add RESP2 integration tests for commands with RESP2/RESP3 response di…
nkaradzhov Mar 31, 2026
2428f65
Fix 9 failing tests to use RESP2 response shapes
nkaradzhov Apr 1, 2026
c17f3d5
Export DISCARD command from commands index
nkaradzhov Apr 1, 2026
4524b08
Remove duplicate tests for LCS_IDX, LCS_IDX_WITHMATCHLEN, HGETALL, CO…
nkaradzhov Apr 1, 2026
14d7365
Rename RESP3 test names to RESP2-appropriate names and remove duplica…
nkaradzhov Apr 1, 2026
89e03a2
Remove duplicate XREAD/XREADGROUP tests and rename RESP3 test names
nkaradzhov Apr 1, 2026
877ef80
Revert test changes for commands not in resp-diff.txt
nkaradzhov Apr 1, 2026
914b7f3
Remove duplicate/weak tests, revert unnecessary renames in ZMSCORE/ZS…
nkaradzhov Apr 1, 2026
753bfac
coverage-blitz: add structural RESP2 assertions for commands in resp-…
nkaradzhov Apr 1, 2026
02e5da9
Fix HKEYS and RANDOMKEY tests: use testWithClient instead of testAll …
nkaradzhov Apr 1, 2026
c5e94de
Revert non-test changes: DISCARD export, MRANGE_SELECTED_LABELS trans…
nkaradzhov Apr 1, 2026
ff4b708
WIP: comment out all added testAll tests to isolate cluster CI flakiness
nkaradzhov Apr 1, 2026
6ab2d86
WIP: also comment out added testWithCluster tests
nkaradzhov Apr 1, 2026
46fd3d3
WIP: re-enable testAll tests, keep testWithCluster commented
nkaradzhov Apr 1, 2026
7e12733
Remove CLUSTER_FLUSHSLOTS test (breaks cluster state), uncomment CLUS…
nkaradzhov Apr 1, 2026
3834753
Switch GLOBAL.SERVERS.OPEN to RESP3 across all packages
nkaradzhov Apr 1, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions NORTH_STAR.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# NORTH STAR — RESP2/RESP3 Parity

## Goal
The Redis client must return identical responses regardless of which RESP version is selected by the user.

## Source of truth
`resp-diff.txt` contains all commands where the SERVER differs responses between RESP2 and RESP3. Every one of those commands is a candidate to check.

## Plan

### Phase 1 — Ensure RESP2 test coverage
For each command in `resp-diff.txt`, check its test:
- If the test structurally asserts the RESP2 response shape well enough that it would break if the shape changed → leave it alone.
- If not → improve the test to capture the structure of the RESP2 reply.
- All tests run against RESP2 (`GLOBAL.SERVERS.OPEN`).

### Phase 2 — Flip to RESP3
Switch `GLOBAL.SERVERS.OPEN` to point to RESP3 and run the tests. Any test that breaks means the command's response shape differs between RESP2 and RESP3.

### Phase 3 — Fix broken commands
For each broken command, fix its `transformReply` (or add a RESP3 transform) so that the RESP3 response is transformed to look identical to the RESP2 response.
51 changes: 51 additions & 0 deletions docs/server-test-gaps.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Server Test Gaps Checklist

## TODO

## DONE
- [x] `EXPLAIN` — Implemented: live `client.ft.explainCli()` coverage added to match `EXPLAIN`.
- [x] `SHUTDOWN` — Rejected: terminates Redis server process.
- [x] `HRANDFIELD_COUNT_WITHVALUES` — Implemented: live RESP2/RESP3 tests verify WITHVALUES field/value object mapping.
- [x] `-1` — Implemented: `client.clientGetRedir()` returns `-1` when unset on a live server.
- [x] `EVALSHA` — Implemented: live `scriptLoad` + `evalSha` verifies SHA-based script execution.
- [x] `CLUSTER_MEET` — Rejected: requires extra-node orchestration.
- [x] `CLIENT_CACHING` — Rejected: Meaningful assertions require specific tracking mode setup.
- [x] `ACL_LOAD` — Rejected: Requires ACL file fixture and filesystem side effects that are brittle in CI.
- [x] `ACL_SAVE` — Rejected: Writes ACL state to disk; behavior depends on environment and permissions.
- [x] `ACL_SETUSER` — Implemented: temporary user lifecycle validated with create/update/delete on live server.
- [x] `ACL_USERS` — Implemented: live ACL listing validates default and temporary user presence.
- [x] `ACL_WHOAMI` — Implemented: live server deterministically reports the default authenticated user identity.
- [x] `ASKING` — Rejected: Cluster redirection-context command is hard to assert reliably in server tests.
- [x] `AUTH` — Rejected: covered by connection/auth integration flows.
- [x] `CLUSTER_ADDSLOTS` — Rejected: topology-mutating cluster admin command.
- [x] `CLUSTER_ADDSLOTSRANGE` — Rejected: topology-mutating cluster admin command.
- [x] `CLUSTER_DELSLOTS` — Rejected: topology-mutating cluster admin command.
- [x] `CLUSTER_DELSLOTSRANGE` — Rejected: topology-mutating cluster admin command.
- [x] `CLUSTER_FLUSHSLOTS` — Rejected: destabilizing topology reset operation.
- [x] `CLUSTER_FORGET` — Rejected: mutates cluster membership.
- [x] `CLUSTER_FAILOVER` — Rejected: disruptive, timing-sensitive failover path.
- [x] `CLUSTER_REPLICATE` — Rejected: mutates replication topology.
- [x] `CLUSTER_RESET` — Rejected: destructive cluster reset behavior.
- [x] `CLUSTER_SET-CONFIG-EPOCH` — Rejected: niche cluster-admin path with low ROI.
- [x] `CLUSTER_SETSLOT` — Rejected: slot-state mutation with high flake risk.
- [x] `CLIENT_KILL` — Rejected: Disruptive and flaky because it kills connections.
- [x] `COMMAND` — Implemented: live `client.command()` test validates transformed metadata shape.
- [x] `COMMAND_GETKEYSANDFLAGS` — Implemented: live test validates version-gated transformed key/flags reply.
- [x] `COMMAND_INFO` — Implemented: live mixed lookup validates transformed metadata and nullable missing-command entry.
- [x] `CONFIG_RESETSTAT` — Rejected: low-value smoke for simple `OK` reply.
- [x] `CONFIG_REWRITE` — Rejected: filesystem/config rewrite side effects.
- [x] `DISCARD` — Implemented: live test verifies queued writes are canceled by `client.discard()`.
- [x] `EVALSHA_RO` — Implemented: live `scriptLoad` + `evalShaRo` verifies read-only SHA script execution.
- [x] `FAILOVER` — Rejected: requires replication topology and is disruptive.
- [x] `FUNCTION_KILL` — Rejected: requires long-running function and race-prone kill flow.
- [x] `SCRIPT_KILL` — Rejected: requires long-running script kill race.
- [x] `MIGRATE` — Rejected: requires cross-instance setup (second Redis target).
- [x] `MODULE_LOAD` — Rejected: requires real module artifacts and mutates runtime.
- [x] `MODULE_UNLOAD` — Rejected: depends on loaded module and mutates runtime.
- [x] `MODULE_LIST` — Rejected: module availability varies by environment.
- [x] `READONLY` — Rejected: cluster-routing toggle with low standalone value.
- [x] `READWRITE` — Rejected: cluster-routing toggle with low standalone value.
- [x] `REPLICAOF` — Rejected: mutates replication topology.
- [x] `RESTORE-ASKING` — Rejected: specialized cluster migration context required.
- [x] `SAVE` — Rejected: blocking persistence side effects can destabilize CI.
- [ ] _None yet_
9 changes: 9 additions & 0 deletions packages/bloom/lib/commands/bloom/EXISTS.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,13 @@ describe('BF.EXISTS', () => {
false
);
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.bf.exists with existing item', async client => {
await client.bf.add('key', 'item');

assert.strictEqual(
await client.bf.exists('key', 'item'),
true
);
}, GLOBAL.SERVERS.OPEN);
});
21 changes: 21 additions & 0 deletions packages/bloom/lib/commands/bloom/INFO.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,25 @@ describe('BF.INFO', () => {
assert.equal(typeof reply['Number of items inserted'], 'number');
assert.equal(typeof reply['Expansion rate'], 'number');
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.bf.info - structural shape assertion', async client => {
await client.bf.reserve('key', 0.01, 100);
const reply = await client.bf.info('key');

// Assert the exact RESP2 response structure (object with specific keys)
// This would break if RESP3 returns a different shape
assert.ok(reply !== null && typeof reply === 'object');
assert.ok(!Array.isArray(reply));
assert.ok(!(reply instanceof Map));
assert.ok('Capacity' in reply);
assert.ok('Size' in reply);
assert.ok('Number of filters' in reply);
assert.ok('Number of items inserted' in reply);
assert.ok('Expansion rate' in reply);
assert.equal(reply['Capacity'], 100);
assert.equal(typeof reply['Size'], 'number');
assert.equal(typeof reply['Number of filters'], 'number');
assert.equal(typeof reply['Number of items inserted'], 'number');
assert.equal(typeof reply['Expansion rate'], 'number');
}, GLOBAL.SERVERS.OPEN);
});
11 changes: 11 additions & 0 deletions packages/bloom/lib/commands/bloom/MEXISTS.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,15 @@ describe('BF.MEXISTS', () => {
[false, false]
);
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.bf.mExists with existing items', async client => {
const key = 'mExistsKey';
await client.bf.add(key, 'item1');
await client.bf.add(key, 'item2');

assert.deepEqual(
await client.bf.mExists(key, ['item1', 'item2', 'item3']),
[true, true, false]
);
}, GLOBAL.SERVERS.OPEN);
});
8 changes: 8 additions & 0 deletions packages/bloom/lib/commands/cuckoo/ADDNX.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,12 @@ describe('CF.ADDNX', () => {
true
);
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.cf.addNX returns false when item already exists', async client => {
await client.cf.addNX('key', 'item');
assert.equal(
await client.cf.addNX('key', 'item'),
false
);
}, GLOBAL.SERVERS.OPEN);
});
9 changes: 9 additions & 0 deletions packages/bloom/lib/commands/cuckoo/DEL.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,13 @@ describe('CF.DEL', () => {

assert.equal(reply, false);
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.cf.del with existing item', async client => {
await client.cf.reserve('key', 4);
await client.cf.add('key', 'item');

const reply = await client.cf.del('key', 'item');

assert.equal(reply, true);
}, GLOBAL.SERVERS.OPEN);
});
9 changes: 9 additions & 0 deletions packages/bloom/lib/commands/cuckoo/EXISTS.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,13 @@ describe('CF.EXISTS', () => {
false
);
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.cf.exists with existing item', async client => {
await client.cf.reserve('key', 100);
await client.cf.add('key', 'item');
assert.equal(
await client.cf.exists('key', 'item'),
true
);
}, GLOBAL.SERVERS.OPEN);
});
28 changes: 28 additions & 0 deletions packages/bloom/lib/commands/cuckoo/INFO.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,32 @@ describe('CF.INFO', () => {
assert.equal(typeof reply['Expansion rate'], 'number');
assert.equal(typeof reply['Max iterations'], 'number');
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.cf.info returns object structure', async client => {
await client.cf.reserve('key', 4);
const reply = await client.cf.info('key');

// Structural assertion: response must be a plain object (not an array)
assert.ok(!Array.isArray(reply), 'reply should not be an array');
assert.equal(typeof reply, 'object');

// Assert exact structure with all expected keys
const expectedKeys = [
'Size',
'Number of buckets',
'Number of filters',
'Number of items inserted',
'Number of items deleted',
'Bucket size',
'Expansion rate',
'Max iterations'
];

assert.deepEqual(Object.keys(reply).sort(), expectedKeys.sort());

// Assert all values are numbers
for (const key of expectedKeys) {
assert.equal(typeof reply[key], 'number', `${key} should be a number`);
}
}, GLOBAL.SERVERS.OPEN);
});
15 changes: 15 additions & 0 deletions packages/bloom/lib/commands/t-digest/BYRANK.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,19 @@ describe('TDIGEST.BYRANK', () => {

assert.deepEqual(reply, [NaN]);
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.tDigest.byRank with data', async client => {
await client.tDigest.create('key');
await client.tDigest.add('key', [1, 2, 3, 4, 5]);

const reply = await client.tDigest.byRank('key', [0, 2, 4]);

assert.ok(Array.isArray(reply));
assert.equal(reply.length, 3);
assert.equal(typeof reply[0], 'number');
assert.equal(typeof reply[1], 'number');
assert.equal(typeof reply[2], 'number');
assert.ok(reply[0] <= reply[1]);
assert.ok(reply[1] <= reply[2]);
}, GLOBAL.SERVERS.OPEN);
});
15 changes: 15 additions & 0 deletions packages/bloom/lib/commands/t-digest/BYREVRANK.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,19 @@ describe('TDIGEST.BYREVRANK', () => {

assert.deepEqual(reply, [NaN]);
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.tDigest.byRevRank with data', async client => {
await client.tDigest.create('key');
await client.tDigest.add('key', [1, 2, 3, 4, 5]);

const reply = await client.tDigest.byRevRank('key', [0, 2, 4]);

assert.ok(Array.isArray(reply));
assert.equal(reply.length, 3);
assert.equal(typeof reply[0], 'number');
assert.equal(typeof reply[1], 'number');
assert.equal(typeof reply[2], 'number');
assert.ok(reply[0] >= reply[1]);
assert.ok(reply[1] >= reply[2]);
}, GLOBAL.SERVERS.OPEN);
});
12 changes: 12 additions & 0 deletions packages/bloom/lib/commands/t-digest/CDF.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,16 @@ describe('TDIGEST.CDF', () => {

assert.deepEqual(reply, [NaN]);
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.tDigest.cdf with data', async client => {
await client.tDigest.create('key');
await client.tDigest.add('key', [1, 2, 3, 4, 5]);

const reply = await client.tDigest.cdf('key', [2, 4]);

assert.ok(Array.isArray(reply));
assert.equal(reply.length, 2);
assert.equal(typeof reply[0], 'number');
assert.equal(typeof reply[1], 'number');
}, GLOBAL.SERVERS.OPEN);
});
37 changes: 37 additions & 0 deletions packages/bloom/lib/commands/t-digest/INFO.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,41 @@ describe('TDIGEST.INFO', () => {
assert(typeof reply['Total compressions'], 'number');
assert(typeof reply['Memory usage'], 'number');
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.tDigest.info structural response shape', async client => {
await client.tDigest.create('key', { COMPRESSION: 100 });
await client.tDigest.add('key', [1, 2, 3]);

const reply = await client.tDigest.info('key');

// Assert exact structure to catch RESP2 (Array) vs RESP3 (Map) differences
assert.ok(reply !== null && typeof reply === 'object');
assert.ok(!Array.isArray(reply)); // Should be object, not array
assert.deepEqual(Object.keys(reply).sort(), [
'Capacity',
'Compression',
'Memory usage',
'Merged nodes',
'Merged weight',
'Observations',
'Total compressions',
'Unmerged nodes',
'Unmerged weight'
].sort());

// Verify all values are numbers
assert.strictEqual(typeof reply['Compression'], 'number');
assert.strictEqual(typeof reply['Capacity'], 'number');
assert.strictEqual(typeof reply['Merged nodes'], 'number');
assert.strictEqual(typeof reply['Unmerged nodes'], 'number');
assert.strictEqual(typeof reply['Merged weight'], 'number');
assert.strictEqual(typeof reply['Unmerged weight'], 'number');
assert.strictEqual(typeof reply['Observations'], 'number');
assert.strictEqual(typeof reply['Total compressions'], 'number');
assert.strictEqual(typeof reply['Memory usage'], 'number');

// Verify expected values based on setup
assert.strictEqual(reply['Compression'], 100);
assert.strictEqual(reply['Observations'], 3);
}, GLOBAL.SERVERS.OPEN);
});
8 changes: 8 additions & 0 deletions packages/bloom/lib/commands/t-digest/MAX.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,12 @@ describe('TDIGEST.MAX', () => {

assert.deepEqual(reply, NaN);
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.tDigest.max with data', async client => {
await client.tDigest.create('key');
await client.tDigest.add('key', [1, 2, 3, 4, 5]);
const reply = await client.tDigest.max('key');

assert.equal(reply, 5);
}, GLOBAL.SERVERS.OPEN);
});
10 changes: 10 additions & 0 deletions packages/bloom/lib/commands/t-digest/MIN.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,14 @@ describe('TDIGEST.MIN', () => {

assert.equal(reply, NaN);
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.tDigest.min with data', async client => {
await client.tDigest.create('key');
await client.tDigest.add('key', [1, 2, 3, 4, 5]);

const reply = await client.tDigest.min('key');

assert.equal(typeof reply, 'number');
assert.equal(reply, 1);
}, GLOBAL.SERVERS.OPEN);
});
17 changes: 17 additions & 0 deletions packages/bloom/lib/commands/t-digest/QUANTILE.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,21 @@ describe('TDIGEST.QUANTILE', () => {
[NaN]
);
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.tDigest.quantile with values', async client => {
await client.tDigest.create('key');
await client.tDigest.add('key', [1, 2, 3, 4, 5]);

const reply = await client.tDigest.quantile('key', [0, 0.5, 1]);

assert.ok(Array.isArray(reply));
assert.equal(reply.length, 3);
assert.equal(typeof reply[0], 'number');
assert.equal(typeof reply[1], 'number');
assert.equal(typeof reply[2], 'number');
// Verify approximate quantile values
assert.ok(reply[0] >= 1 && reply[0] <= 1.5); // min
assert.ok(reply[1] >= 2.5 && reply[1] <= 3.5); // median
assert.ok(reply[2] >= 4.5 && reply[2] <= 5); // max
}, GLOBAL.SERVERS.OPEN);
});
11 changes: 11 additions & 0 deletions packages/bloom/lib/commands/t-digest/TRIMMED_MEAN.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,15 @@ describe('TDIGEST.TRIMMED_MEAN', () => {

assert.equal(reply, NaN);
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.tDigest.trimmedMean with data', async client => {
await client.tDigest.create('key');
await client.tDigest.add('key', [1, 2, 3, 4, 5]);

const reply = await client.tDigest.trimmedMean('key', 0.1, 0.9);

assert.equal(typeof reply, 'number');
assert.ok(!isNaN(reply));
assert.ok(reply > 0 && reply < 10);
}, GLOBAL.SERVERS.OPEN);
});
17 changes: 17 additions & 0 deletions packages/bloom/lib/commands/top-k/INFO.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,21 @@ describe('TOPK INFO', () => {
assert.equal(typeof reply.depth, 'number');
assert.equal(typeof reply.decay, 'number');
}, GLOBAL.SERVERS.OPEN);

testUtils.testWithClient('client.topK.info - structural assertion', async client => {
await client.topK.reserve('key', 5);
const reply = await client.topK.info('key');

// Structural assertion to ensure RESP2 array-to-object transformation
assert.ok(reply !== null && typeof reply === 'object' && !Array.isArray(reply));
assert.ok('k' in reply && typeof reply.k === 'number');
assert.ok('width' in reply && typeof reply.width === 'number');
assert.ok('depth' in reply && typeof reply.depth === 'number');
assert.ok('decay' in reply && typeof reply.decay === 'number');

// Verify the structure matches the expected object shape
const expectedKeys = ['k', 'width', 'depth', 'decay'];
const actualKeys = Object.keys(reply).sort();
assert.deepStrictEqual(actualKeys, expectedKeys.sort());
}, GLOBAL.SERVERS.OPEN);
});
1 change: 1 addition & 0 deletions packages/bloom/lib/test-utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ export const GLOBAL = {
OPEN: {
serverArguments: [],
clientOptions: {
RESP: 3,
modules: RedisBloomModules
}
}
Expand Down
Loading
Loading