fix(scoring): add Clojure/ClojureScript AST node weights #179
+16
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Clojure's tree-sitter parser produces node types (list_lit, sym_lit, vec_lit, etc.) that were not present in token_weights.json, causing all .clj/.cljc/.cljs files to score 0 despite being parsed correctly. Adds structural weights: list_lit, vec_lit, map_lit, anon_fn_lit, kwd_lit, sym_lit. Adds leaf weights: sym_name, kwd_name, num_lit, str_lit, nil_lit, sym_ns.
anderdc
requested changes
Feb 10, 2026
Collaborator
anderdc
left a comment
anderdc
left a comment
There was a problem hiding this comment.
adjust structural weights to what I commented
Lower structural weights as requested: list_lit 0.15, vec_lit 0.1, map_lit 0.15, kwd_lit 0.05, sym_lit 0.05.
anderdc
approved these changes
Feb 10, 2026
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
token_weights.json.clj,.cljc, and.cljsfiles scoring 0 token score despite being parsed correctlyRelated Issues
Clojure's tree-sitter parser produces node types (
list_lit,sym_lit,vec_lit, etc.) that are fundamentally different from imperative languages. None of these were present intoken_weights.json, so every Clojure AST node received a weight of 0.0 — causing all Clojure PRs to get 0 base score.For example, penpot/penpot#8301 modifies a
.cljsfile with 66 additions and 32 deletions but scores 0.Clojure AST vs configured weights:
function_definition,call_expressionlist_lit(everything is a list in Lisps)identifiersym_lit/sym_namestringstr_litnumbernum_litAdded weights:
list_lit(0.4),vec_lit(0.2),map_lit(0.3),anon_fn_lit(0.5),kwd_lit(0.15),sym_lit(0.1)sym_name(0.07),kwd_name(0.07),num_lit(0.03),str_lit(0.03),nil_lit(0.1),sym_ns(0.05)Verified with a test diff: score went from 0.00 → 3.61 after the fix.
Type of Change
Testing
Checklist