Testing

Overview

Testing is done using flexunit we have used mock4as for mocking and plan to use flexmonkey for UX testing.

Currently our thinking is that Hemlock core will be mainly covered by unit tests (flexunit) and the games themselves by UX tests (flexmonkey).

Running tests

Short answer

rake test:default

This will create all of the tests and then compile HemlockTestRunner.mxml, you will then have a .swf HemlockTestRunner.swf. Rake task will try to run it with `open`, so it should automatically open the swf test file on your Macs.

Longer answer

Tests are run using HemlockTestRunner, this loads in allTests.as which loads a PackageTestSuite.as for each package that you are testing. As these change as you add new tests they are all generated using rake and placed in the “generated_tests” folder.

This should not be checked in (neither should allTests.as) and can be blown away at any time.

Running rake test:default does the following:

• Creates “generated_tests” folder and required sub folders for packages
• Generates test files in “generated_tests”
• Generates test suites for each package that import the tests
• Generates allTests.as that imports each suite
• Compiles HemlockTestRunner
• Opens the swf

Writing new tests

Tests are kept in the “tests” directory and the directory structure should mirror that of the packages in “com” that you are testing. Tests should have the same name as the class they are testing but with “Test” at the end.

Eg.

HemlockContainer.as → HemlockContainerTest.as

Inside these tests you only need to define individual tests as functions, functions you want run as tests should start with “test”.

Eg.

public function testSomething():void{
    assertEquals("a", "a");
}

These belong in a class and import a bunch of files, but this is all done by the rake test procedure. If you need any specific classes import them at the top of this file.

All tests for a package go into the top level dir for the package (ie com.mintidigtal.hemlock), rather than in any further sub dirs. Inside of the package dir, there is one further dir which is “mocks”, this holds all of the mocks for the package.

Eg.

I’m testing com.mintdigital.hemlock, here is the dir structure needed:

- src
   - tests
      - com
         - mintdigital
            Test.as
            - mocks
              Mock.as

Setup & Teardown

You can define setup and teardown methods in your tests like so:

public override function setUp():void {
  // code to be run before every test
}

public override function tearDown():void {
  // code to be run after every test
}

This methods will be run before and after every test in the file.

Assertions

There are couple of basic assertions available with Flex Unit.

- assertEquals()
- assertFalse()
- assertNotNull()
- assertNotUndefined()
- assertNull()
- assertStrictlyEquals()
- assertTrue()
- assertUndefined()

For more info, please, refer to:

http://opensource.adobe.com/svn/opensource/flexunit/tags/0.9/FlexUnitLib/src/flexunit/framework/Assert.as

Mocking

We use mock4as for mocking. Mocking is painfull in as3 due to lack of dynamic language features (comparing to ie Ruby) and requires a lot of code.
We keep all the mock files inside test/com/mintdigital/mocks/ directory and everything in the directory is imported when running the tests.

There are 2 ways of mocking:

Implementing mocked class interface

Our mock inherits from the mock4as’ Mock class. Here’s the example:

package tests.com.mintdigital.toolbox.mocks {
	import org.mock4as.Mock;
	import com.mintdigital.toolbox.ITool
public class MockTool extends Mock implements ITool {

	    public function use():void {

	        record(“use”, target);

	    }

	}

}

So each mock file should include one mock class, with implementation of the interface. Implemented method should call record, to keep track of itself being called.
Then, in a test, we can write the following code:

public function testToolBeingUsed():void {
	var fitter = new Fitter();
	var mock_tool = new MockTool();
	fitter.grab(mock_tool);
	
	mock_tool.expects("use").times(1).willReturn(true);
	
	fitter.use_current_tool();
	
	mock_tool.verify();
	assertNull(mock_tool.errorMessage());
	assertTrue(fitter.sucessfullyUsed(mock_tool));
}

so first we do test set up, then we set the expectations. After that we run the action, verify and check if there are no errors.

KNOWN PROBLEMS/DIFFICULTIES WITH IMPLEMENTING MOCKED CLASS INTERFACE

• this implies ‘programming to interface’ throughout the application code – if the type check is for expected class not definition, this method wil fail miserably. although it’s a good practice to follow, testing the code comes with refactoring and redesigning the code while writing the tests.
• you need to implement ALL the interface methods, and make them compatibile with the interface. While, in ideal world, your tested class could implement various interfaces, each relating to a certain role/interaction, so you can write a separate mock for every role/interaction, in real world usually one will end up with one big interface and, as a result, one big mock implementing all the interface methods.
• you need to include all the packages of course (for argument and return types, any class used, etc)

USE COMPOSITION TO DO PARTIAL MOCKS

The only way to do partial mocks that is known to us so far, is composition: our mock inherits from the mocked class, and has it’s own mock object of a Mock class, to which it delegates all the public methods of Mock class. This requires writing few identical methods in every mock, so we have separated these to another file, which we include in partial mocks.

Here’s the example:

package tests.com.mintdigital.hemlock.mocks {
	import com.mintdigital.toolbox.tool;
	import org.mock4as.Mock;

	public class PartialToolMock extends Tool {
	  /* here we include the mock and delegate methods code */
		include "../../helpers/PartialMockHelper.as"

		public function PartialToolMock() 
		{ 
			mock = new Mock();
	  }
		
		override public function complexMethod(options:Array):void
		{
	    mock.record("complexMethod", options);
	  }
	}
}

And the test for this is similar to previous one:

public function testComplexMethodIsCalledTwiceOnTheActionWithSuccess():void {

	var partial_tool_mock = new PartialToolMock();
	options = ['1','2']
	partial_tool_mock.expects("complexMethod").times(1).withArg(options)
	
	partial_tool_mock.doSomething(options);
	
	partial_tool_mock.verify();
	assertNull(	partial_tool_mock.errorMessage());
}

Partial MOckHelper is always the same, here’s the code:

private var mock:Mock;

public function expects(methodName:String):Mock {
    return mock.expects(methodName); 
} 

public function times(timesInvoked:int):Mock {
    return mock.times(timesInvoked); 
} 

public function verify():void {
    mock.verify(); 
} 

public function success():Boolean {
    return mock.success(); 
} 

public function errorMessage():String {
    return mock.errorMessage(); 
} 

public function withArg(arg:Object):Mock {
	return mock.withArg(arg); 
}

public function withArgs(...args):Mock {
	return mock.withArg(args); 
}

public function willReturn(arg:Object):void {
	mock.willReturn(arg); 
}

KNOWN PROBLEMS/DIFFICULTIES WITH USING COMPOSITION TO DO PARTIAL MOCKS

• Once you define mock, you cannot reuse the same class in the situation, when you need already mocked method not to be mocked (ie to test it’s behavior).

GENERAL COMMENTS ON MOCK4AS

• we altered the code there, to get rid of the warnings caused by lack of type definitions and few other minor changes.
• if you do not define a method on mock but you will expect it, it will not fail. I think it’s design problem with Mock4As, that could be fixed soon I suppose.
• there are few other minor problems with Mock4As, which are hard to track, but should be fixable and before that they shouldn’t affect basic mocking scenarios. Anyway, if you run into anything, it’s worth adding them here or fixing the Mock4As if you feel lucky. I guess we could release Mock4AsFix or contribute to the original source, if we got to the stable stage. The source for Mock4As is very small and quite readable, take a look inside it (org.mock4as). Considering the number of problems, it’s worth spending some time inside the class itself and maybe rewrite it with more features, unit tests etc.

More information on Mock4As:

Mock4As project page