User Testing Session 1

The Truth Table – Prototype v1 User Testing Session

Background and Purpose:

The first version (v1) of The Truth Table is a prototype consisting of the first draft designs of a few key pages of the site, including the main page, article view, comments reply, and mobile layout. This iteration is about presenting the general layout and theme of the product. The aim is to receive feedback about the concept itself, while getting new ideas for features potential users would like to see added. A key factor of investigation is finding out whether people would use The Truth Table in the first place. To do so, the session will utilize open ended semi-structured interview questions along with an observation of users as they attempt to complete tasks, describe what they see, and provide feedback.

Testing Process and Agenda:

Description of process:

To begin with, participants are asked pre-testing questions to determine whether they are part of the identified target demographic, as well as providing a general understanding of the backgrounds of our participants with respect to their personal news reading practices. Afterwards, the tester briefly explains what the participants are about to be shown, with a short introduction to The Truth Table and its purpose. Participants are then asked a shorter second series of questions about the concept, and their expectations for it before seeing the prototype. The prototype interaction portion will then begin, with the participants starting at the design of the main page, while being given tasks and questions as they go. Any clicks and navigation are explained by the tester as to what they do, or to provide a live update to bring up the resulting page, if that page is available in the prototype. Following the completion of the interaction phase, participants are asked a final series of interview questions about their experience and final impressions, as the conclusion of the testing session. In total, five participants took part in individual sessions.

Testing Procedure:

Pre-Introduction questions:

1. What is your age?
2. How often do you visit news sites (daily, weekly, seldom, never)?
3. Which news sites and sources do you primarily use?
4. Have you come across news that you read and believed which later turned out to be fake?
5. Have you shared or passed on claims or facts which you did not fact check?
6. When browsing for news, do you tend to do it primarily on your computer/laptop, or mobile phone? Do you do it on the go?

Pre-testing questions:

1. Have you used any similar kinds of forums before such as Reddit?
2. Would you be interested enough to consider using something like The Truth Table? (before having seen the prototype itself)
3. What do you expect you will be able to do on such a site?

Prototype interaction tasks and questions:

1. [Main page version 1] This is the home page view of a user that is already logged in.

a)Could you describe what you can see?

b)Could you try to add a new topic?

c)Could you try to enter one of those topics?

d)[Main page version 2] Here is an alternate design for the front page. What do you think about the differences, and which would you prefer to use and why?

2. [Topic view] You are taken to the following page.

a)Once again, could you describe what you see?

b)Could you try to add an article?

c)Now could you try to add a comment?

3. [Mobile View] Could you describe what you see?

4. [Logo and Slogan Design] Thoughts on the logo?

Closing post-testing questions:

1. What did you like the most?
2. What did you dislike the most?
3. As a user, what features would you like to see added?
4. After testing out the prototype, do you believe you would use The Truth Table in its completed form?

Evaluation:

Results:

The results of the pre-introduction questions revealed that the target audience tends to read the news mainly on a daily basis, but with dedicated time allotted per day, and not multiple times a day. Stated sources of news included new.comau, Gizmodo, ABC, YouTube, Reddit, SBS, ESPN, Sydney Morning Herald, Kotaku, and 9news. Most participants were unsure about whether they've come across news that they've read and believed that turn out to be fake, with multiple stating that they don't check, and thus wouldn't know. In the same vein, all participants admitted that they have passed on claims or facts which they did not fact check. As for browsing the news, results indicated a majority of users browse at home on their PC or laptop, and do not do so when mobile.

The pre-testing questions revealed that the target audience largely uses Reddit, with other sites such as 4Chan and Quora quoted as being forums sites which they use. All users were at least somewhat interested in the The Truth Table, after hearing a brief introduction about it. Some users specifically stated the idea would be of interest for providing multiple views and perspectives about politics. As for expectations of the site's features, anonymous voting and commenting was a commonly brought up topic. Some users were strongly in favor of its addition, while others themselves soon acknowledged potential flaws, as they thought about the idea. Standard features such as contribution of topics were also expected. Terms of note included 'unbiased', 'fact check', 'easier', and 'good start'.

The testing of the home page shows that users were particularly critical of the sidebar, with some users preferring a different location, others wished that the tags were different, citing their personal lack of interest or preference for different tags. There was a divide between users which preferred the addition of the distinction between specific countries' news, while other wanted only a separation between domestic and international news. As for the two proposed designs, more users preferred the look of the borderless buttons. All users were able to quickly complete the tasks of attempting to locate the control to add a new topic, and attempting to enter into a listed topic. One user found the 'Add News' button to be somewhat hidden, and suggested moving it elsewhere.

The testing of the topic page shows that users liked the topic tags such as 'unreliable' and 'reliable'. The task of adding an article encountered no issues with all participants. The task of adding a comment took a little longer on average, and faced some disapproving feedback from users regarding the placement and size. Some users also noted they were unsure what they're commenting on, and would prefer to comment for specific articles. Other noted issues included the lack of a downvote button for topics, headline text size (too small), and 'too much clumped up text'.

The mobile view was relatively easy to use for the participants, with most users being able to identify all possible controls and making correct assumptions about what they would be able to do with them. Suggestions included the removal of The Truth Table logo in order to expand the status bar. Noted issues included a lack of headlines, lack of text, and two users noted that the poor visibility of the buttons and the button text at the bottom.

Responses to the logo were generally positive, with common descriptions including phrases such as 'simple' and 'clean'. Users also liked the blue and white colour scheme. That being said, a few users believed it could be improved, but had no specific suggestions, just the belief that it could have more detail and be 'better'.

As for the closing questions, participants stated that they liked the premise, and liked the simplicity of the site. Location of buttons, flow, and distribution of text size were some key aspects which they disliked. Suggested features for addition included a search bar, their preferred topic filters, an improved comments section, and the aforementioned anonymous commenting/voting. Most users stated that they would be interested in using The Truth Table in it's completed form, but that it would take some work to shore up its shortcomings. The rest of the users stated they would not.

Summary/Decisions:

The team has decided on making numerous changes based on the feedback, particularly the negative observations. Firstly, the question we posed to users regarding the two alternative navigation bar designs lead us to settle on the more minimalistic and cleaner layout, without the borders and button distinctions. As they were often found to be irrelevant to individual users, specific countries filters will no longer appear as direct categories. Regarding the spacing and readability issues encountered throughout the testing sessions, the general text and spacing of the layouts will be revisited for fixes. Basic errors such as the lack of a downvote button will also be patched.