Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(#37): create automated test loader #38

Merged
merged 4 commits into from
Dec 4, 2022
Merged

test(#37): create automated test loader #38

merged 4 commits into from
Dec 4, 2022

Conversation

andymina
Copy link
Owner

@andymina andymina commented Dec 4, 2022

Wow, what a PR. As described in #37, an automated test runner/loader would be excellent as I develop a full-blown testing suite to ensure things continue to work. Here's the general outline of what happened:

Changes

  1. Since seam carving is a computationally heavy operation, 6x6 matrices with values from [0, 255) were randomly generated and their Sobel images were computed with OpenCV (the tests really on the fact that the Sobel operator works properly in OpenCV since the code is taken from their tutorial).
  2. The vertical/horizontal energy map, vertical/horizontal seams, and vertical/horizontal images after removal and insertion were all computed by hand, and this data, along with the data from step 1, was inserted into a spreadsheet. The spreadsheet was exported from Google Sheets and imported into my project.
  3. I wrote a parser to create a neatly-formatted JSON file of all of the relevant testing data from a collection of TSVs. The format of the JSON is described below.

JSON testing data format

The new test data can be found at test/data.json. All matrices, with the exception of the insertion and removal arrays, are uchar one-dimensional arrays of size 36. This array can be parsed as a 6x6 matrix using one of OpenCV's cv::Mat constructors. test/data.json is an array of test objects where a test object has the following structure:

{
	"test_id": int,
	"input": uchar[],
	"energy": {
		"sobel": uchar[],
		"map - vertical": uchar[],
		"map - horizontal": uchar[]
	} ,
	"carver": {
		"find - vertical": Coord[],
		"find - horizontal": Coord[],

		"remove - vertical": uchar[],
		"remove -horizontal": uchar[],

		"insert - vertical": uchar[],
		"insert - horizontal": uchar[]
	}
}

Future testing implications

Adding test cases

As mentioned above, test matrices are 6x6. The data for these matrices aren't too interesting, nor are their seams, but it will do for now. In the future, if I want to expand this size, I can add new sheets to the Google Sheet linked above. However, larger test cases will still need to be generated by hand. I think in the future I will make the test matrices 10x10 to get some more interesting seams.

Using test/data.json

The current way I'm envisioning the new test data being used is as follows:

  1. the test file loads test/data.json
  2. the test file extracts the input and corresponding outputs. For example, if Carver.FindOptimalVerticalSeam() was being tested, then the test file would load all of the carver["find - vertical"] outputs from each test_id.
  3. the test file compares expected outputs from test/data.json against the actual outputs from the function.

This PR took about two weeks 🎉

Closes #37

@andymina andymina added the tests Change something relating to tests label Dec 4, 2022
@andymina andymina added this to the Develop a test suite milestone Dec 4, 2022
@andymina andymina self-assigned this Dec 4, 2022
@andymina andymina removed this from the Develop a test suite milestone Dec 4, 2022
// count the number of seam carving tests
int test_count = 0;
std::string test_data_dir = "test/raw-data/";
for (const auto& entry : fs::directory_iterator(test_data_dir)) {
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

entry.path() would've given me the direct path to the file, but it wouldn't have given me them in numerical order so I decided to do it myself.

@andymina
Copy link
Owner Author

andymina commented Dec 4, 2022

A future improvement of this test loader can use the Google Sheets API to keep itself updated rather than manually exporting tests and accounting for sizes. Issue #39 has been created to keep track of this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tests Change something relating to tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a automated test runner/loader
1 participant