unit test library

[mvolkmann]

I was unable to find a unit test library that works with Scryer Prolog, so I created my own. It's very basic, but seems reasonable to me. I'd love to get some feedback on it. I am fairly new to Prolog and am definitely open to suggestions on what could be improved, including naming conventions. See https://github.com/mvolkmann/prolog-modules.

The file list_util.pl implements a few predicates that are useful for working with lists.
The file list_util.plt implements unit tests. To run them, enter scry -g test list_util.plt where scry can be replaced with whatever alias you use to run scryer-prolog.
The file unit_test.pl implements predicates for the test library.

The output from running the tests should be "All 7 tests passed".
If you modify the expected value for the last test to be [bar] and run the tests again, you should see this:

user:list_without3 expected [bar] but was []
6 tests passed
1 test failed

[pmoura]

There's always a place for light unit test tools. For a full featured one that supports Scryer Prolog, see:

Overview - https://logtalk.org/tools.html#testing
Full documentation - https://logtalk.org/manuals/devtools/lgtunit.html

[mvolkmann]

Thanks! I want to try lgtunit, but it's not clear to me how I can use it with Scryer Prolog. The examples I see use :- object(...). Is it correct that I cannot use that with Scryer? Is there a page in the documentation that shows how to use lgtunit in a Prolog implementation rather than a Datalog implementation?

[pmoura]

After installing Logtalk, you'll need to start with scryerlgt instead of scryer-prolog to run lgtunit. For example, adapting your tests for the json module:

:- object(json_tests,
	extends(lgtunit)).

	:- use_module(json, [
		json//1
	]).

	test(test_atom, true(Expected == Actual)) :-
		Expected = '"foo"',
		V = foo,
		phrase(json(V), Chars),
		atom_chars(Actual, Chars).
	
	test(test_integer, true(Expected == Actual)) :-
		Expected = "123",
		V = 123,
		phrase(json(V), Actual).
	
	test(test_list, true(Expected == Actual)) :-
		Expected = '["foo","bar","baz"]',
		V = [foo, bar, baz],
		phrase(json(V), Chars),
		atom_chars(Actual, Chars).
	
	test(test_pairs, true(Expected == Actual)) :-
		Expected = '{ "red": "stop", "green": "go", "yellow": "yield" }',
		V = [red-stop, green-go, yellow-yield],
		phrase(json(V), Chars),
		atom_chars(Actual, Chars).
	
	test(test_string, true(Expected == Actual)) :-
		Expected = '"some text"',
		V = "some text",
		phrase(json(V), Chars),
		atom_chars(Actual, Chars).
	
	test(test_structure, true(Expected == Actual)) :-
		Expected = '{"_functor": "a/2", "_args": ["b","c"]}',
		V = a(b, c),
		phrase(json(V), Chars),
		atom_chars(Actual, Chars).

:- end_object.

Assuming the code above saved in a json_tests.lgt file, we can use the following tester.lgt file to drive the tests:

:- initialization((
	set_logtalk_flag(report, warnings),
	logtalk_load(lgtunit(loader)),
	logtalk_load_context(directory, Directory),
	atom_concat(Directory, 'json.pl', File),
	consult(File),
	logtalk_load(json_tests, [hook(lgtunit)]),
	json_tests::run
)).

Assuming that you saved both files in your prolog_libraries directory:

$ scryerlgt
...
?- {tester}.
% 
% tests started at 2023-08-06, 13:46:29
% 
% running tests from object json_tests
% file: /Users/pmoura/Desktop/lgtemp/plt/json_tests.lgt
% 
% test_atom: success (in 0.000000000 seconds)
% test_integer: success (in 0.000000000 seconds)
% test_list: success (in 0.000000000 seconds)
json Pair: Pair = red-stop
json Pair: Key = red
json Pair: Value = stop
!     test_pairs: failure (in 0.000000000 seconds)
!       test goal throws an error but should have succeeded: error(existence_error(procedure,(-)/4),(-)/4)
!       in file /Users/pmoura/Desktop/lgtemp/plt/json_tests.lgt between lines 24-30
% test_string: success (in 0.000000000 seconds)
% test_structure: success (in 0.000000000 seconds)
% 
% 6 tests: 0 skipped, 5 passed, 1 failed (0 flaky)
% completed tests from object json_tests
% 
% no code coverage information collected
% tests ended at 2023-08-06, 13:46:29
% 
   true
;  ... .

[pmoura]

See also https://logtalk.org/blog.html?tag=testing

[mvolkmann]

@pmoura My current lightweight test module requires that I pass it a list of test predicates to execute. I have to prefix each one with user:. See https://github.com/mvolkmann/prolog-modules/blob/8cf4aa2bc0bef2fcd0cad6a03146fd8575e9236d/strings.plt#L79. Do you know a way I could avoid needing to do that or automate adding those namespace qualifiers?

[pmoura]

Given that strings.plt is a plain Prolog file (i.e. it doesn't define a module), its default namespace (when consulted) is already user. To avoid the explicit prefixing of the test predicates, the run_tests/1 would be declared as a meta-predicate. But its argument is not a goal or a closure but a list of closures. That means using:

:- meta_predicate(run_tests(:)).

However, this will not work as you would get a qualified list:

user:[chars_capitalized1,chars_capitalized2,filename_extension1,join1,join2,join3,predicate_namespace_name1,predicate_namespace_name2,repeat1,split1,split2,string_list1,string_list2,string_list3]

which will make the foldl/4 goal fail as it expects a list in the second argument but instead gets a (:)/2 compound term. A ugly hack to solve this problem is:

test :-
  run_tests([
    chars_capitalized1,
    chars_capitalized2,
    filename_extension1,
    join1,
    join2,
    join3,
    predicate_namespace_name1,
    predicate_namespace_name2,
    repeat1,
    split1,
    split2,
    string_list1,
    string_list2,
    string_list3
  ]),
  halt.

run_test(Module, Test, Passed0, Passed) :-
  call(Module:Test, Expected, Actual),
  message(Test, Expected, Actual, Msg),
  length(Msg, Length),
  (Length #= 0 ->
    Passed #= Passed0 + 1
  ; format("~s~n", [Msg]),
    Passed #= Passed0
  ).

:- meta_predicate(run_tests(:)).
run_tests(Module:Tests) :-
  foldl(run_test(Module), Tests, 0, Passed),
  length(Tests, Length),
  (Passed #= Length -> Prefix = "All "; Prefix = ""),
  report_count(Prefix, Passed, "passed"),
  Failed #= Length - Passed,
  (Failed #= 0 ->
    true
  ; report_count("", Failed, "failed")
  ).

This gives:

$ scryer-prolog
?- ['strings.plt'].
   true.
?- test.
14
All 14 tests passed

Note that a main motivation for the meta_predicate/1 directive is to avoid the need for explicit module qualifiers in the first place...

[mvolkmann]

Thank you SO much! I like this approach much better.

[mvolkmann]

@pmoura I updated https://github.com/mvolkmann/prolog-modules/blob/10ca779d9794949ec6e1f3091bffddf503224a50/unit_test.pl#L11 to define an all rule that runs all unit tests in the current directory. It works, but outputs "Warning: overwriting test/0" when consulting each test file except the first. That's because each test file defines a test rule. I thought perhaps I could declare that to be a dynamic rule and retract it after running each test. But I couldn't get that to work. Do you have a suggestion for how I can eliminate those warnings?

[pmoura]

That's just one of the problems you face. What prevents two different test files from defining tests with the same name? That will cause additional warnings. The solution is to not use plain Prolog files for the tests.

[mvolkmann]

Are you suggesting using Logtalk instead of Prolog?

[pmoura]

Not necessarily. You can try to use a Prolog module to encapsulate each tests set.

Btw, it's never Logtalk instead of Prolog but rather Logtalk with Prolog.

[mvolkmann]

Changing each test set (the .plt files) to be Prolog modules did remove those warnings. Thanks!