-
Notifications
You must be signed in to change notification settings - Fork 7
User Testing of Player Integration and Actions
User testing can provide valuable insight into player experiences and game effectiveness. Understandings how, when and why players are frustrated or confused at the game allows developers to create improvements. The player is a key way that the user will interact with the in-game world, thus user testing a variety of player actions ensures users will be able to interact with the world and progress the story.
The following goals were set for participants to cover a range of player interactions and determine a level of success. Participants were not told they had to complete these goals, as it is important to test that the game communicates these aspects effectively to the user.
- Able to move using the WASD keys within 30 seconds of starting the game
- Attempt using SHIFT to sprint after walking for a while, and correctly understand that SHIFT makes the player faster
- Conceptually understands that walking on different types of tiles has a different walkspeed (or block walking), and chooses paths accordingly These first three goals measure how well-integrated (e.g. with the tile system), effective and usable the player's movement actions are
- Quickly understands their first goals in-game, to talk to the questgiver and shovel ship debris This goal is a marker that the game's purpose is either clear or unclear
- Once trying to shovel debris, is able to identify how to shovel within 30 seconds (select item in inventory, click on debris) This goal also measures player integration - this time with tools, inventory and debris. Results from this test can be extrapolated to some extent to other tools and actions, such as tilling the ground, fishing, planting and watering plants, etc. as they all require the player to select an item from their inventory and use the item to interact with the in-game world.
- Conceptually understands completing, redeeming and receiving quests from the questgiver
- After completing a quest and trying to redeem it, is able to redeem the quest within 1 minute
- Is successful in starting the main gameplay loop of farming and completing quests without much confusion and no frustration This is another goal measuring how well the game's purpose and storyline is communicated to the player, and generally will reveal any flaws prior to starting Act II of the game (which will be untested here).
As the player completes all of Act I to achieve the following, it is expected that several other insights into different features such as hostile creatures will also be gained.
The user testing activity was think-aloud, which encourages participants to voice their thoughts when completing the test goals. This is especially useful in providing specific feedback about points in a game that a user experiences a lot of confusion or frustration, as well as identifying common thought patterns and approaches when presented with a task. In this test, the participants were asked to complete Act I of the game (informed when they achieve this goal) from game launch, and vocalise their approaches, thoughts and feelings while doing so. Otherwise, very little instruction was provided to the participants, and help was only given if absolutely required to progress the test and was asked for. Participants' feedback and their context was noted down for later analysis.
User testing was performed by two users, both aged 17. Participant 1 had a decent level of video game proficiency and experience, and Participant 2 was very experienced. Their age of 17 fits neatly within the teenager and young adult demographics that are likely to be part of the main target audience for the game, as well as all participants having video game experience. This ensures that the feedback gained will be useful in improving the game for its target audience, rather than an audience who would otherwise not play the game and may have very different needs and wants.
Analysis of the test results show some strengths in the game as players were able to achieve goals effectively with little to no confusion. Some of these strong points include identifying and using items, including when they are picked up; identify and interact with the inventory, such as rearranging items in the toolbar; taming animals; fertilising soil; understanding walkspeed and tile type effects on player movement; quickly picking up the main gameplay loop; and understanding the plant tips (albeit with purposeful disregard for overwatering).
The two participants experienced some common barriers including struggling to redeem quests; struggling to kill hostiles before dying (not realising they were under attack as they were still on the quest screen); not understanding the oxygen and health bars purpose (assuming they were dead); fishing; being restless while waiting for corn to grow for the first time; being frustrated at the light level at night; and not understanding how to regenerate health.
Additionally, Participant 1 experienced extra barriers: not knowing appropriate keybinds for actions, struggling to read due to font size, having some confusion with quest goals and story beats as they did not enjoy reading the dialogue, needing more time to understand tamed animal indicators (the green ring), not realising that the ship can be interacted with, and not understanding the progress bar.
Important context is that Participant 2 had more video game experience than Participant 1, although both are proficient in video games.
- Able to move using the WASD keys within 10 seconds of starting the game Both participants achieved this successfully.
- Attempt using SHIFT to sprint after walking for a while, and correctly understand that SHIFT makes the player faster Only participant 2 achieved this successfully. Participant 1 achieved this with minor prompting ("Have you tried sprinting?")
- Conceptually understands that walking on different types of tiles has a different walkspeed (or block walking), and chooses paths accordingly Both participants understood this concept thoroughly, utilising different tile types to move faster and avoiding slow tile types. Participant 2 went above and beyond to explore new tile types around the map.
- Quickly understands their first goals in-game, to talk to the questgiver and shovel ship debris Both participants immediately cleared the debris and found the questgiver after. Participant 1 had some struggle finding the interact key E, but Participant 2 had no trouble.
- Once trying to shovel debris, is able to identify how to shovel within 30 seconds (select item in inventory, click on debris) Both participants achieved this with no hesitation.
- Conceptually understands completing, redeeming and receiving quests from the questgiver Both participants understood this concept, but after having trouble redeeming quests they could not understand why they didn't get a new quest.
- After completing a quest and trying to redeem it, is able to redeem the quest within 1 minute Participant 1 required some prompting to redeem a quest, Participant 2 did not require prompting. Both participants spent a while wandering around the map and tending to their farm while they were confused about what to do next.
- Is successful in starting the main gameplay loop of farming and completing quests without much confusion and no frustration Both participants got the hang of the main gameplay loop after the hurdle of redeeming quests. They understood the main goals of the game to be: build the farm as much as possible, complete the quests from the alien.
To address the barriers the participants faced, the following improvements are suggested. The implementation of these suggestions should reduce the difficulties experienced by participants during testing, and aid players in understanding how to interact with the game using the player.
- Adding tooltip popups that can be toggled on and off for new players, such as showing 'E' when in range of the questgiver or ship
- Upon starting a new game, show basic movement control WASD and SHIFT keys, as well as how to select an item
- Either auto-redeeming completed quests, or making the completed quests tab more visible when the player has to redeem a quest (e.g. highlighted and flashing when an unredeemed quest is there)
- Telling the player appropriate fishing locations in the fishing rod description and/or in quest information - or creating a fishing tutorial quest
- Only spawning dragonfly enemies after the player has obtained the gun item
- Making the light level brighter at night time - still dark so it is clear that time has passed, but still navigable
- Making the font size larger, or having accessibility options to change the font size
- Fixing the health bar implementation so the players dies when it reaches 0
- Being more clear on how to redeem health, e.g. on hovering over the health bar, provide instructions
- Being more clear on what the oxygen bar is (similar to health bar suggestion), provide a description and how to increase oxygen on hovering
- Make corn grow at a faster speed while the player is completing their first corn-growing quest to reduce restlessness
However, due to the limitations of these two user tests, further testing is highly encouraged to ensure validity. Think-aloud user tests should be completed by a wider range and larger number of participants, as well as tests where they complete the full game. After the implementation of suggested improvements, further testing should be done to confirm if the improvements were effective in reducing players' difficulty in understanding how to play the game and reduce their frustration. Further testing after making changed would also reveal any new difficulties that may emerge.
Participant 1:
“The title screen is ugly”
“The font is too small”
Navigates using buttons very efficiently
Can walk around instantly
Can select tool and use it to dig debris instantly
“I’m making it look pretty”
“I’m gonna die I’ve got no oxygen”
Keeps trying to click on the NPC
“What do I do?”
“I can’t talk to her!”
Has to be prompted to use E to interact button
Navigation through quest UI is shaky
Can see active quests easily
Gets lost trying to redeem a quest
Understands the concept of tiles having different walkspeeds, chooses to walk on paths
Runs around after being confused on redeeming a completed quest, gets distracted by a cow to 'pet' it using E
After prompting that their task is to redeem a completed quest, is able to successfully navigate and redeem the quest
Is able to till ground immediately
“I like the square because unlike in Animal Crossing you actually know the square you’re digging up”
Continues to struggle to read the text
“Why is he [questgiver] helping me?” (upon asking if they read the intro text, said no)
“How do I know if they’re ready for harvesting?”
Remembers the Q button that they pressed accidentally earlier
“They should have a key guide”
Gets restless while waiting for the corn to grow
Is able to get the hang of the main gameplay loop very quickly
“Oh yeah!!! I can pet the cow” after getting the animal taming quest
“How do I tame it?”
“OH I collected its REDACTED” - in regards to collecting fertiliser
Took a minute, but able to figure out how to tame a cow
“And I got green moo juice”
“Why does it have a ring around it?”
“It’s too dark”
Successfully tames a cow and collects milk with very little struggle
“I can’t see anything”
“Are you supposed to just stop at night? You literally can’t see anything”
Can identify different tile types even in pitch black, “I think it’s like soul sand”
“It’s like what I do in stardew valley” “it’s like a tycoon - more is better” - in regards to making the farm bigger
“I have more seeds so that means I need more dirt”
Upon unlocking the hostile quest, “I don’t wanna kill them!” But is immediately and very proficient at killing them
For the dragonflies, got stuck at top of map
“I can’t kill it”
Had to redeem quest to get the gun item in order to kill dragonfly, help was given at this stage as it was an unexpected bug that dragonflies couldn't be killed with swords when out of reach over the ocean
Did not recognise that the player could sprint, was given prompt “Did you try sprinting?” and immediately pressed shift to sprint
Was very stuck on killing the dragonflies until prompted to complete the current active quest
Was able to see full inventory by pressing I with no problems
With prompting, recognised they had a gun (would have avoided prompting but their crops would have been all gone otherwise)
Was able to use gun very quickly
Did not recognise that “Over Watered” in the plant info was a bad thing
“Did I kill it [the plant]?... They can survive” - in regards to over watering
Read “Make a connection ... with the NPC” and tried to give him milk (presumably applying Stardew Valley knowledge)
After getting the gun, kills a cow accidentally
“Oh no did I kill it? I’m so sad”
“Oh yeah, beef. I feel so bad”
“I didn’t think cows could be killed”
Participant 2: “I’m clicking settings because I usually do that to optimise the game for machine. Yeah that looks pretty good actually. Apply”
“Went to press escape because I thought it’d be a keybind but you have to click Exit”
Started game
Was able to very quickly identify tooltips to select shovel, and start clearing debris
“I like that it highlights your range” - referring to the selected tile highlight
“We’ve got different animations that are based on the direction the tile is away from the player, that’s pretty good”
“Shift to sprint, E to interact…” Was very quick at identifying different keybinds and remembering them
Was proficient at navigating the quest menu, except for redeeming a completed quest
“Oh I can swim! And this is…. Sand..” very quick to identify the effects on walkspeed of different tile types and remembering them, avoiding slow tiles “Oh we’re at 10% oxygen that’s not good”
“What do I do with the bird? I’m guessing I come back because I don’t have an item [for the chicken]”
“I can swim up the waterfall ok” wanted to explore different tile types
After wandering for a bit, was able to redeem a quest after revisiting the questgiver
“Ah you have to collect the reward first”
“I feel like here’s a good area for a plantation.. It’s some some beach some sand, it’s got an amongus guy in view”
Was able to till tiles and plant seeds, as well as redeem the quest very quickly
“I like Jaleel”
“Oh I can water them [the plants], they need water”
“I assume the scythe is for collecting”
“Oh it tells me the growth level, that’s cool” understands the plant tip UI and its purpose
“It’s nice to have a progress bar” understands the concept of quests being part of a main storyline that can be completed
Finds the fast ice, makes proficient use of different tile speeds
“My crops are coming along nicely.. The cosmic cobs.. Also got some random growing speeds, that’s cool”
Gets restless while waiting for the corn to grow
Waters the plants again while knowing they are overwatered
“It’s good for them.. They need to be hydrated”
“Adult that means I can kill it and get seeds and a cosmic cob”
“I have 4 cosmic cobs, it’s beautiful”
Able to recognise the number of items in inventory
“It’s at the ideal water level” continues to make use of plant tips
“Let me guess, it [the shovel] does undo it [tilling the ground]” explores how tools interact with different tiles
“It’s getting night.. There are fireflies” Proficient understanding of the time of day
“Oh it’s very dark. It’s very dark”
“I need light!!”
“Oh you feed the cows the cosmic cobs!!!” “Where are the cows? I can’t see any”
After finding a cow. “Oh he likes it. He gave me poop, oh I mean fertiliser. Cool. Give me more of that please.”
“I got a love heart and there’s a circle around him, that must mean he’s mine”
Very fast at picking up the main gameplay loop, completing quests
“Let’s replant some of this to get the best out of the fertiliser” very proficient understanding of gameplay mechanics and that fertiliser may be a temporary effect without being explained
“I can’t fish yet” Was attempting to fish in the shallow water, assumed it just wasn’t an unlocked skill
After redeeming the hostile quest
“I get a weapon!!! Oh i see the hostile creatures, they’re killing me already”
“Oh I’m at 14% health, I’m at 0% health, this is bad”
Was able to hit dragonfly with sword
“They’re not dying!!! Oh ok yeah they do die, it just takes a while. The mozzies are significantly more tanky than the slime”
“That one’s running away where I can’t get it. I’ll have to bait it, abuse the AI”
“What’s my ship doing? Oh that’s where i get the light” Was able to find and use the Ship AI with no direction
“What button is for inventory? I!”
“Why didn’t you tell me I had the gun?” Unclear when items are added to inventory if not visible on toolbar
“I can’t remember what hammerplant does but I know aloe heals you”
“I’m a master at cosmic cobs, I have 78 seeds, how am I gonna use these all?”
Rearranged inventory to place weapons first, farming equipment second, and seeds last
After forgetting what hammerplant does, pays much more attention to what Jaleel says from then on
Was informed that they completed Act I, wanted to keep playing
“Oh they have different water levels required for different plants, that’s cool”
Proceeds to feed all their cosmic cob to their tamed cow
Seeks out untamed cows to kill “oh I got meat!!”
“How do i use the aloe? Do I put it on me? I don’t know how to use the aloe vera”
“A box? Oh that’s for all my seeds?” Is able to quickly identify and place a chest for extra inventory
“That’s good it moves the UI around”
Is proficient in moving inventory items between chest and player inventory quickly
“I don’t know how to use the milk. I’m guessing that’s for something else, perhaps the space axolotls”
“Oh I have 21% health? When did I get that? Maybe it’s an auto regeneration for each day, since I noticed the night cycle started”
Sees the tractor quest and is very exited
Explores the quests available for Act II
“I forget what I’m doing… Active quests”
“The night darkness is aggressive”
“I can hear things growing but I can’t see them” picked up on sound effects to signify events