Beta testing

Beta-testing is getting other people to play your game with the purpose of finding bugs and other issues. It is vital that your game gets beta-tested, as there will be things you miss. Other people play in different ways, so will try different things. They will use different nouns, different verbs, different routes, have different ideas about how to solve a puzzle, different ideas about what is a puzzle.

That said, before beta-testing, be sure the game is as good as possible. Sending a game for beta-testing when you know there are errors wastes your time and your testers.

When you are ready for beta-testing, set settings.playMode to "beta" and also set settings.version in settings.js - it is important to track what version of your game is being tested.

There is a great article on beta-testing here, which goes into more detail, including what to do it you are the tester. You might also want to look at this thread on IntFiction.

Transcript

Using a transcript is a great way to beta-test as the author can see exactly how the user got into that situation, and can see all the things the user did that failed. If the user tried fifteen verbs before hitting on the right one to solve the puzzle, you probably need to implement those fifteen verbs, even if they only say it does not work. Not all testers will want to use a transcript, and ultimately it is up to them, but it is great when they do because you can see everything they tried.

In my experience, testers can start to chat to the transcript, and it becomes almost a conversation. This is great; you receive much more feedback this way. I think the testers who do this enjoy the process more too.

In "beta" mode, a transcript is started automatically when the game starts, and a message will be displayed saying this is a beta-version, with the version number. You can change the default message by adding a function to code.js, for example:

  lang.betaTestIntro = function() {
    metamsg("This beta version " + settings.version + )
    io.scriptStart()
  }

When you upload a new version, make sure you have changed the version (hopefully your testers will tell you which version they were looking at, if they are not using the transcript).

If your tester chooses to use the transcript facility, she can add a comment at any point by typing a * or a ; at the start of the line, and then typing whatever she wants. At the end of the session SCRIPT SHOW or Crtl-Enter will display the transcript in a new tab (a link will allow the player to do this if the game has finished, and the text input is no longer there). This can be copy-and-pasted to the author, who can then quickly search through for errors and comments.

Obviously when you release your game, you should set settings.playMode to "play".

Transcript with no text input

These instructions are for beta-testers who are testing a game with no text input. We are going to use the "developer console", as that does allow input to be typed.

Press F12 (or Crtl-Shft-I or Crtl-Shft-J or via a menu) whilst in your browser and viewing the game tab to open the "Developer Tools" - as far as I can tell, this is the same for all browsers; they look a little different, but the same things are there.

The console will appear as a split screen with the web page. It may be at the right, the left or the bottom; there will be an option somewhere allowing you to move it where you want. I prefer to have it to the right. There will be various tabs; "Elements", "Console", and others. If you do not see the tabs, try dragging the divider between the web page and developer tools to make the latter larger.

The "Console" tab is the one you want. There may be text there already - just ignore it.

I want to start a transcript: This should happen automatically if the author set the play mode to "beta", but if it does not, paste this into the developer console:

saveLoad.transcriptStart()

I want to see the transcript: If you get to the end you should see a link to do this, but not or if you stop before reaching the end, paste this into the developer console:

saveLoad.transcriptShow()

This will open up a new tab with a button to click to save the transcript.

I want to add a comment: Any time you want to add a comment, make sure the cursor is in the developer console, then type c(' followed by your comment, followed by '), then press return. Your comment must not include single quotes, and line breaks are not allowed.

c('spelled hat wrong')

If all goes well, you will see a comment in the game text.

NOTE: When beta-testing, the transcript is reset each time you start the game. You MUST save the transcript as soon as you finish play, before closing the tab or browser or navigating elsewhere.

Finding Testers

My personal approach is to get a couple of people on the TextAdventures forum to help, as they tend to be more forgiving, then family members to get an outsider's perspective, then ask for testers on IntFiction.

Be aware that beta-testers can get hard to find as the deadline for IFComp approaches at the end of September.

It is worth telling people the game is essentially just a web page, so there is nothing to install to get it to run, and no communication back to the server whilst playing.

Several Rounds

I suggest breaking beta-testing up into rounds, and introducing new testers at each round. This avoids having six testers finds the same bugs, which is a waste of their time, but also of yours, as you have to go though lists of bugs trying to decide if that one has been resolved or not.

If you have thorough testers, use one per round. Of course, you will probably not know who is thorough until they report back so two at a time may be more realistic...

Remember to update the version each time you realise for testing. The transcript will automatically add the version, so you can then determine whether a tester was playing an older version of your game.

Many Testers

How many testers you use may depend on how many you can find as well as how complex your game is. Arguably you should keep throwing more testers at it until they can find nothing wrong, but that may not be practical. I used fifteen for The House on Highfield Lane.

Credit Your Testers

Remember to credit your testers; you might want them to do it again for another game. If this is for a competition, you may get a lower score if you do not, as people will assume the game is untested.