Basic CommonLisp Treesitter Parsing

[Encryptoid]

Hi all, I was messing around with CommonLisp Treesitter in Lem. I just got the very basics working, reading some lisp code. I will probably have another look in future, but sharing for now in case anyone else wants to try it.

There isn't any official support for CL, both the parser and the grammar are taken from githubs.

Steps

Below are the steps needed to parse a basic lisp string:

Install CL Treesitter Parser

CommonLisp Parser

This will allow you to call any treesitter parser via CommonLisp(c, python, rust, etc.)

(asdf:load-asd (probe-file "<path-to-package>/cl-tree-sitter.asd"))
(ql:quickload :cl-tree-sitter)

Install CL Treesitter Grammar

CommonLisp Grammar

This is the tricky bit. So if you download the above and make it, it will produce a file: libtree-sitter-commonlisp.so. You will have to rename it to tree_sitter_commonlisp.so. I have no idea why this is, and if you don't, it complains that libtree_sitter_commonlisp is missing(not the right name either?). I probably didn't read something.

tree_sitter_commonlisp.so worked for me.

The same is true for the other language I tried, C. The make produced libtree-sitter-c.so, which had to be renamed to tree-sitter-c.so to work with the CL Parser.

You might also need this:

Parse CommonLisp Code

Run this code in Lem to register the language,

;; Exclude .so filetype in string
(cl-tree-sitter:register-language :common-lisp "<path-to-grammar.so dir>/tree_sitter_commonlisp") 
(cl-tree-sitter:parse-string :common-lisp "(list 2 2)")

Then you will get back the treesitter grammar for this. Someone might be interested in trying out certain lisp code in it and understanding the result. Here is the result from the above, you can see the list, and the number literals:

(:SOURCE
 ((0 0) (10 0))
 ((:LIST-LIT
   ((0 0) (10 0))
   (((:VALUE :SYM-LIT)
     ((1 0) (5 0)) NIL)
    ((:VALUE :NUM-LIT) ((6 0) (7 0)) NIL)
    ((:VALUE :NUM-LIT) ((8 0) (9 0)) NIL)))))

[Encryptoid]

I do think this could work for highlighting/indenting/lsp. But whether it's needed is another question. Could be a fun project.

Here is a command for testing, to put the current buffer to treesitter:

(define-command treesitter-current-buffer () ()
  (let* ((lines (uiop:read-file-string (lem/buffer/internal::buffer-%filename (current-buffer))))
         (ts-lines (cl-tree-sitter:parse-string :common-lisp lines)))
    (lem:message "TS LINES :: ~a" ts-lines)))

Another example:
It seems the positioning list is:

((start-col start-line) (end-col) (end-line))

[vindarel]

good stuff, thanks.

2c: you could use buffer-filename

(defun buffer-filename (&optional (buffer (current-buffer)))

[Encryptoid]

Thanks, although I couldn't find any references online, and didn't spot it in the CLOS cookbook, but I assume the % sign is the indicate it's private(to prevent setting mostly), and (buffer-filename) let's you access safely. Cheers for the tip :)