Printing of useful information when unit test fail

[quekyj]

In this code, I am building a FullAdder. But I realized that the test package in dart didn't return useful information when the test fails. In this case, I want to see the details such as what is the value of a, b, cIn when the test fails.

I know that we can set a breakpoint and kind of debug it with continue / step into (Interactive debug) using the debugger but what I am trying to look for is like when you fail, then all the current variables values are printed on the cmd / log files.

Is there any way to do this?

import 'package:rohd/rohd.dart';
import 'package:test/test.dart';

class FullAdderModule extends Module {
  FullAdderModule(
    Logic a,
    Logic b,
    Logic cIn,
    void Function(
            Logic a, Logic b, Logic cIn, Logic xorAB, Logic sum, Logic cOut)
        faOps,
  ) : super(name: 'full_adder') {
    a = addInput('a', a);
    b = addInput('b', b);
    cIn = addInput('c_in', cIn);

    final sum = addOutput('sum');
    final cOut = addOutput('c_out');

    final xorAB = Logic(name: 'xor_ab');
    faOps(a, b, cIn, xorAB, sum, cOut);
  }
}

class FAResult {
  int sum = 0;
  int cOut = 0;
}

FAResult faTruthTable(int a, int b, int cIn) {
  final res = FAResult();
  if (a + b + cIn == 0) {
    return res
      ..sum = 0
      ..cOut = 0;
  } else if (a + b + cIn == 3) {
    return res
      ..sum = 0 // the correct answer should be 1
      ..cOut = 1;
  } else if (a + b + cIn == 1) {
    return res
      ..sum = 1
      ..cOut = 0;
  } else {
    return res
      ..sum = 0
      ..cOut = 1;
  }
}

void faOps(Logic a, Logic b, Logic cIn, Logic xorAB, Logic sum, Logic cOut) {
  // SUM
  xorAB <= a ^ b;
  sum <= xorAB ^ cIn;

  // C-Out
  cOut <= xorAB & cIn | a & b;
}

void main() async {
  final a = Logic(name: 'a');
  final b = Logic(name: 'b');
  final cIn = Logic(name: 'c_in');

  final xorAB = Logic(name: 'xor_ab');
  final sum = Logic(name: 'sum');
  final cOut = Logic(name: 'cOut');

  faOps(a, b, cIn, xorAB, sum, cOut);

  test('should return true if result sum similar to truth table.', () async {
    for (var i = 0; i <= 1; i++) {
      for (var j = 0; j <= 1; j++) {
        for (var k = 0; k <= 1; k++) {
          a.put(i);
          b.put(j);
          cIn.put(k);

          expect(sum.value.toInt(), faTruthTable(i, j, k).sum);
        }
      }
    }
  });

  final mod = FullAdderModule(a, b, cIn, faOps);
  await mod.build();
}

Current test failure output:

02:38 +0 -1: should return true if result sum similar to truth table. [E]

  Expected: <0>
    Actual: <1>

  package:test_api/src/expect/expect.dart 134:31     fail
  package:test_api/src/expect/expect.dart 129:3      _expect
  package:test_api/src/expect/expect.dart 46:3       expect
  doc/tutorials/chapter_3/test.dart 81:11            main.<fn>
  package:test_api/src/backend/declarer.dart 215:19  Declarer.test.<fn>.<fn>
  ===== asynchronous gap ===========================
  package:test_api/src/backend/declarer.dart 213:7   Declarer.test.<fn>
  ===== asynchronous gap ===========================
  package:test_api/src/backend/invoker.dart 257:7    Invoker._waitForOutstandingCallbacks.<fn>
  
02:38 +0 -1: Some tests failed.

[quekyj]

From what I can think of is using try...catch, setUpAll(), and tearDownAll() to print the variables. But I wonder if there like any better approaches.

setUpAll(() {
    a.put(0);
    b.put(0);
    cIn.put(0);
  });

  tearDownAll(() {
    a.put(0);
    b.put(0);
    cIn.put(0);
  });

  test('should return true if result sum similar to truth table.', () async {
    for (var i = 0; i <= 1; i++) {
      for (var j = 0; j <= 1; j++) {
        for (var k = 0; k <= 1; k++) {
          a.put(i);
          b.put(j);
          cIn.put(k);

          try {
            expect(sum.value.toInt(), faTruthTable(i, j, k).sum);
          } on Exception {
            print('a: ${a.value.toInt()}');
            print('b: ${b.value.toInt()}');
            print('cIn: ${cIn.value.toInt()}');

            rethrow;
          }
        }
      }
    }
  });

[mkorbel1]

You can use the reason named argument in expect to print a more helpful message.

For example:

          final expected = faTruthTable(i, j, k).sum;
          final actual = sum.value.toInt();
          expect(sum.value.toInt(), expected,
              reason: 'For inputs a=$i, b=$j, cIn=$k,'
                  ' expected sum=$expected but found $actual');

Now when the test fails it prints

Expected: <0>
  Actual: <1>
For inputs a=1, b=1, cIn=1, expected sum=0 but found 1