Skip to content

Conversation

@sushruth
Copy link
Contributor

@sushruth sushruth commented Jan 21, 2026

Summary

Adds complete support for negated string operators and in/!in operators in kql-to-duckdb translator.

Changes

Negated String Operators

  • Add handlers for !contains, !startswith, !endswith, !has in expression translator
  • Translate to NOT LIKE and NOT REGEXP for valid DuckDB SQL

Array Literals & in/!in Operators

  • Add ArrayLiteral expression type to kql-ast
  • Update kql-lezer grammar to parse array literals: (value1, value2, ...)
  • Add CST-to-AST conversion for ArrayLiteral nodes
  • Implement in/!in operator translation to SQL IN/NOT IN

Testing

  • Add comprehensive test suite for string operators (12 tests)
  • All tests passing:
    • ✅ kql-lezer: 110 tests
    • ✅ kql-to-duckdb: 12 tests

Examples

Negated String Operators

Users | where name !startswith "j"
-- Translates to:
SELECT * FROM Users WHERE name NOT LIKE 'j%'

in/!in Operators

Users | where age in (18, 21, 25)
Users | where age !in (18, 21, 25)
-- Translates to:
SELECT * FROM Users WHERE age IN (18, 21, 25)
SELECT * FROM Users WHERE age NOT IN (18, 21, 25)

This PR was created by an AI agent on behalf of @sushruth.

- Add handlers for !contains, !startswith, !endswith, !has
- Translate to NOT LIKE and NOT REGEXP for valid DuckDB SQL
- Add comprehensive tests for all string operators (8 tests passing)
- Update package.json test scripts
- Add ArrayLiteral expression type to kql-ast
- Update kql-lezer grammar to parse array literals (e.g., (1, 2, 3))
- Add CST-to-AST conversion for ArrayLiteral nodes
- Implement in/!in operator translation to SQL IN/NOT IN
- Add tests for in/!in operators with numbers and strings
- All tests passing: kql-lezer (110), kql-to-duckdb (12)
- Remove generated files from git (kql.grammar, parser.ts, parser.terms.ts)
- Add generated files to .gitignore
- Update build script to auto-generate: grammar → parser → TypeScript
- Add fix:parser step to add @ts-nocheck to generated parser
- Update documentation in README.md and CLAUDE.md
- Build process now: generate:grammar → build:parser → fix:parser → tsc
- All tests passing (110 tests)
@sushruth
Copy link
Contributor Author

Update: Grammar Generation Automated

The grammar generation process is now fully automated in the build pipeline:

What Changed

  • Generated files removed from git: kql.grammar, parser.ts, parser.terms.ts are now gitignored
  • Build process automated: bun run build now handles the full pipeline:
    1. generate:grammar - Generate .grammar from TypeScript sources in src/grammar/
    2. build:parser - Generate parser from .grammar using Lezer
    3. fix:parser - Add @ts-nocheck to generated parser
    4. tsc - Compile TypeScript

Developer Workflow

To modify the grammar:

  1. Edit TypeScript sources in src/grammar/ (tokens, rules, plugins)
  2. Run bun run build
  3. Update src/parser/cst-to-ast/ if needed
  4. Run bun test

Documentation updated in README.md and CLAUDE.md.


This comment was created by an AI agent on behalf of @sushruth.

- Check both error.toString() and error.stderr for npm error messages
- Bun's ShellError contains npm output in stderr property
- Add more specific error message for 'You cannot publish over' error
- Improve error logging to show both error and stderr output
- Add ArrayLiteral rule to src/grammar/plugins/rules/expressions.ts
- Import 'many' helper function for grammar generation
- Fixes test failures in kql-to-duckdb that were using old parser
- All tests passing: kql-lezer (110), kql-to-duckdb (12)
- Add kql.grammar, parser.ts, parser.terms.ts to turbo.json outputs
- Ensures generated files are cached and available for tests in CI
- Fixes 'Cannot find module ./parser' errors in CI pipeline
@sushruth sushruth assigned sushruth and Copilot and unassigned sushruth Jan 21, 2026
- Add turbo.json to build task inputs
- Ensures cache is invalidated when Turbo configuration changes
- Prevents stale cache issues in CI when outputs/inputs are modified
- Add --force flag to turbo build command temporarily
- Ensures generated parser files are created in this PR run
- TODO: Remove --force after cache is refreshed
- Remove --force flag from build command (cache is now refreshed)
- Add turbo.json to inputs for test, test:coverage, and lint tasks
- Ensures all tasks invalidate cache when Turbo config changes
- Prevents future stale cache issues across all tasks
@sushruth sushruth merged commit af3cc18 into main Jan 21, 2026
2 checks passed
@sushruth sushruth deleted the fix/negated-string-operators branch January 21, 2026 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants