How to run all ekg-test*- in one go?

[qingshuizheng]

This is not related to the ekg project.

I'm quit new to ert test, and run the ekg-test-* tests one by one with M-x ert, which takes a lot of time running all the tests.

Wondering how do you usually run the tests and review the test results? Thanks for any info. 😄

[ahyatt]

M-x ert-run-tests-interactively, then you give a string regexp (that's important): "ekg-test.*". The quotes have to be entered, which I found surprising at first. Hope this helps!

[qingshuizheng]

That works!
It is hard to find out indeed.
Thanks.

[qingshuizheng]

BTW, what about add a command to ekg-test-utils.el?

(defun ekg-run-all-tests ()
  "Run all ekg tests matching regex \"ekg-test-\\.*\"."
  (interactive)
  (ert-run-tests-interactively "ekg-test-.*"))

[qingshuizheng]

Or we use this command below:
With no prefix arg, run all tests in current emacs session;
with prefix arg, run all tests in an async Emacs -Q session.

This is just a raw implementation in my mind, might not have covered all situations.

(defun ekg-run-all-tests (arg)
  "Run all ekg tests.
With no prefix ARG, run all tests in current emacs session;
With prefix ARG, run all tests in an async Emacs -Q session."
  (interactive "P")
  (if arg
      (let* ((path-getter (lambda (lib) (file-name-directory (locate-library lib))))
             (process (start-process
                       "*ekg-test*" "*ekg-test*"
                       (concat invocation-directory invocation-name)
                       "-Q"
                       ;; load path
                       "-L" (funcall path-getter "emacsql")
                       "-L" (funcall path-getter "triples")
                       "-L" (funcall path-getter "ekg")
                       ;; load file
                       "-l" (file-name-sans-extension (locate-library "ekg.el"))
                       ;; eval
                       "--eval" "(progn \
(setq debug-on-error t) \
(require 'ekg-test) \
(ert-run-tests-interactively \"ekg-test-\\.*\"))")))
        (set-process-sentinel process
                              (lambda (proc _)
                                (kill-buffer (process-buffer proc)))))
    (ert-run-tests-interactively "ekg-test-.*")))

Edits: fix errors.

[ahyatt]

Interesting idea, and definitely useful! I'd recommend a few more things: test out ekg-logseq as well, and also byte compile all the files to see if there are any warnings. Ideally, the byte compilation will be removed after the test ends, although we should make sure it doesn't

This should be in a new file, ekg-smoke-test.el, or something like that. It'd be interesting to see how other emacs packages do something similar. I haven't looked into it, myself. Github also offers ways to do this automatically, which I've only briefly glanced at.

Thanks for exploring this area, I've learned a few things from how you did this - the solution looks really nice!

[qingshuizheng]

how other emacs packages do something similar.

I consult-ripgrep through the emacs packages to see how others run their tests, and found magit has this command magit-emacs-Q-command to save a shell command string to kill-ring. So I borrowed his idea and formed the code above.

Code from "magit-base.el":

(defun magit-emacs-Q-command ()
  "Show a shell command that runs an uncustomized Emacs with only Magit loaded.
See info node `(magit)Debugging Tools' for more information."
  (interactive)
  (let ((cmd (mapconcat
              #'shell-quote-argument
              `(,(concat invocation-directory invocation-name)
                "-Q" "--eval" "(setq debug-on-error t)"
                ,@(cl-mapcan
                   (lambda (dir) (list "-L" dir))
                   (delete-dups
                    (cl-mapcan
                     (lambda (lib)
                       (let ((path (locate-library lib)))
                         (cond
                          (path
                           (list (file-name-directory path)))
                          ((not (equal lib "libgit"))
                           (error "Cannot find mandatory dependency %s" lib)))))
                     '(;; Like `LOAD_PATH' in `default.mk'.
                       "compat"
                       "dash"
                       "libgit"
                       "transient"
                       "with-editor"
                       ;; Obviously `magit' itself is needed too.
                       "magit"
                       ;; While these are part of the Magit repository,
                       ;; they are distributed as separate packages.
                       "magit-section"
                       "git-commit"
                       ))))
                ;; Avoid Emacs bug#16406 by using full path.
                "-l" ,(file-name-sans-extension (locate-library "magit")))
              " ")))
    (message "Uncustomized Magit command saved to kill-ring, %s"
             "please run it in a terminal.")
    (kill-new cmd)))

I'd recommend a few more things: test out ekg-logseq as well, ...

As to ekg-logseq, since I don't use it, so... :-(
I've not dived into all the deftest yet, but they are in the bucket list.

byte compile all the files to see if there are any warnings.

There is 2 comp warnings on develop branch (my emacs: GNU Emacs 30.0.50 (Build 20G1116), native-comp):

1. Error: Invalid function #<subr or>

⛔ Warning (comp): /Users/z/Developer/personal/zemacs/lib/ekg/ekg.el: Error: Invalid function #<subr or>`

Have no idea where it comes from, perhaps related to (apply 'or ...) or (funcall 'or ...)? The only related info I could find is the examples in funcall and apply: https://www.gnu.org/software/emacs/manual/html_node/elisp/Calling-Functions.html

2. obsolete function: reduce:

Compiling /Users/z/Developer/personal/zemacs/lib/ekg/ekg-embedding.el...
In ekg-embedding-valid-p:
lib/ekg/ekg-embedding.el:91:32: Warning: ‘reduce’ is an obsolete function (as of 27.1); use ‘cl-reduce’ instead.

Feel free to take the piece of code if you'd like to add them.

Edit: add info.