The PublishingLab manual has made its way off a paper and on to the web. In order to see if this format would be useful to new interns, it needed testing. I used the User Checks method taught by Anoushka Scholten because checks are intended to be iterative, real users attempt real tasks with design, there are revisions and a retest. User Checks are also quicker than more traditional lab testing with eye-tracking. Scholten stresses that during qualitative user testing it is necessary to observe and ask to find out why people do the things they do – people don’t always do they things they say they do.
User Checks Procedure:
- Let your user experience the prototype (don’t tell, observe)
- Have the user talk through what they’re doing
- Actively observe (don’t correct, watch for use/misuse)
- Follow up questions
- where do you expect to find x? what do you think that button does? you mentioned this, what did you mean/expect to happen?
In Scholten’s method that there are 3 main roles:
- observer (anyone with post-its)
- moderator* (i.e. product manager)
- tester (UX designer)
Testing
Duration: 20 mins, 3X tests
Participants: observer, user, tester
Documentation: screen capture, post-it notes (observer), feedback notes (tester).
Users: all 20s, 2 male, 1 female, interns, high level English skills (as second language)
*A moderator was not available but I worked as tester with an observer to test the site.
Task Scenario
It is your first week as an intern at the Publishing Lab, you’ve seen the presentation about your team, and you are eager to get started. You need some information about how to start your internship, and the way digital publishing is done.
Visit the PL website to find this info.
User goal: Find out how to set yourself up for this internship.
- Task 1: Go to www.publishinglab.nl/resources/publishinglab-manual and use the PublishingLab Manual to find how you get your HvA id and what you can do with it.
User goal: Find out what the PublishingLab does.
- Task 2: Using the PublishingLab manual look up examples of partners that the PL works with.
User goal: Learn how to create digital and hybrid publications in your internship.
- Task 3: Using the PublishingLab manual, find how the PublishingLab creates publications.
User goal: Understand your role within the internship team.
- Task 4: Using the PublishingLab manual, find the section you’d expect to get this information.
Results
The observations revealed two moments of confusion: task 2 was seen to cause some confusion to the participants, and frustration during part 1 (don’t tell, observe) when the site search function was used and the manual was not found.
User Checks: Observer role, slide from Anoushka Scholten
Feedback
After completing the scenario tasks, I followed up with questions, some related to the reading experience and layout of the site and others to clarify what users expected when they clicked certain areas of the navigation. The feedback was that the navigation terms need more clarity, and that the layout could benefit from a more visual separation of sections.
Layout
“I would rather read this as a pdf… the text is difficult to read with so much hyphenation.”
“It’s not really clear where a section ends.” – participant 1
“I’m more inclined to first scroll and read, get an overview of the content since it’s on one page and then to use the nav for specifics later.” – participant 3
Language / Navigation
“I interpret ‘Studio’ as more about the work, not about feeling, behavior.” – participant 1
“I translated Practical as Praktijk [Dutch] which means what we do, which is why I looked there during Task 2.” – participant 2
“Studio is unclear, could we either use ‘Lab’ (suggested by participant 1) or ‘you in the lab’?”
“Approach is unclear, could it be reworded as ‘how we work’?” – participant 3
Corrections
During part 3 of the process (actively observe), a broken link was discovered in task 2, which caused unnecessary frustration to the first participant.
The wording of task 4 in regard to the word ‘role’ could be interpreted to either apply to behavior or to expected outcomes. It needed to be stated clearer, so I considered the result of participants choosing either the Approach or Studio section as successful.
What next?
Work on the layout and wording of the navigation to improve comprehension, reword task 4 & final retest. Ensure manual is searchable on the PL site.
-
- Before
-
- After