The University of Washington Department of Epidemiology
Web Usability Study
The Problem Space
As a national leader in public health, the UW Department of Epidemiology (UWEpi) provides "critical education and leadership to tackle challenges and [develops] the next generation of leaders and visionaries in public health." To elevate their department's online presence and offerings, the UWEpi Communications team began a comprehensive website redesign.
The Challenge
Over ten weeks, our team of four graduate student researchers conducted a formative usability study to understand students' current behaviors around finding courses and degree-related information and discover the usability issues preventing users from finding this information effectively.
The Nitty Gritty Bits
Project Type: Formative Usability Study
Practices: Web usability testing, usability test moderation, heuristic evaluation
Duration: 10 Weeks
Team: Daniel, Esme, and Rachel
My Role & Responsibilities
As a usability researcher & project manager of the team I :
Communicated with stakeholders to define our project's scope, establish project goals, and manage timeline and logistics for our project.
Identified target users and recruited participants for user research.
Design usability studies in collaboration with the teammates.
Conducted three out the five usability studies with the recruited participant.
Collected and analyzed participant data based on objective & subjective metrics.
Reported the findings back to the stakeholders in written format.
Learn & Distill
Stakeholder Interviews
Why We Did It
I believe every research and design project should start with stakeholder interviews. To kick off this project, we conducted a stakeholder interview with the UWEpi Communication team to understand why conducting a usability study is necessary, define the project's goals, and identify the research areas.
My Role
As the project manager, I was responsible for the initial contact and logistics for our stakeholder interview. Working with my team, we collaboratively created the stakeholder interview questions and distilled our findings to design targeted research areas and define the study goals.
With their input, we scoped our usability testing to current students’ ability to use the website for course registration and understanding their degree requirements. By researching these tasks, we hoped to understand the current usability issues with the UW EPI website and give guidance for improving the included content, information taxonomy, and usability of the site.
After our meeting, we established four primary goals:
Understanding the current ways epidemiology students find information about their degree requirements and course registration requirements.
Understand what type of information students find important in the degree requirements and course registration processes.
Understanding how students currently navigate the UW EPI website to find information about their degree requirements and course registration.
Understanding the technology and resources students use to understand their degree requirements and course registration.
To meet the goals of our project, we sought to investigate three key research areas:
What usability issues prevent participants from discovering information about degree requirements and course registration?
What information do participants seek to understand degree requirements and course information? What sources do they use to gather that information?
How efficiently are participants able to successfully discover the information they seek on the UW EPI website?
Design and Test
With our project goals refined and our research questions defined, we began developing our usability study plan.
Study Recruitment
First, we focused on recruiting participants. Using Google Forms, I developed our participant screening questionnaire. A UWEpi Comm distributed this questionnaire via a voluntary list-serve for promotional news and events related to the Epidemiology Department. We received ten total responses; however, only five matched our target participant demographic. Participants received a $25 gift card for their participation.
Review the screener questionnaire here.
Usability Study Planning & Development
Next, we focused our efforts on developing our usability study plan and subsequent usability tool kit.
Usability Test Design
When deciding which type of study design to conduct, we considered the advantages and disadvantages of between-subject and within-subject designs. In the end, we settle on conducting a within-subject study to reduce the number of participants needed for the usability test and to reduce the potential for random noise.
Next, we designed the tasks our participants would complete during the session. While these aren't the full scenarios given to participants, these are the main actions participants had to take.
Find an Epidemiology class you need to take specific to this track (Global Health) offered in Spring 2020.
Show me how would you find the schedule line number (SLN) and when this course is being offered.
Show me how you would find the course(s) instructor Stanaway teaches and when they are offered.
Show me how you would find a disease/exposure elective available in Autumn.
Deciding on Data
We decided to collect performance and preference data to gain as much insight as possible.
Preference
Satisfaction
Ease of use
Think-a-loud comments
System Usability Scall
Performance
Effectiveness of the website
Task Success and Error rate
Efficiency
Lostness
Test Day Design
We conducted five in-person, moderated task scenario usability tests to collect data about users' ability to find information about their programs and then identify specific courses within the program. Here’s what our testing environment looked like.
What We Learned
With our usability study complete, we turned into video editors and data analysis to understand what we really learned during our usability study.
What We Learned | Study Findings
Research Question One: What usability issues prevent participants from finding information about degree requirements and courses?
At a high level, we found that confusing information architecture and fragmented content prevented participants from identifying information about degree requirements and courses.
1. Confusing Information Architecture
Low discoverability of key resources (MPH Requirement Checklist): 2/5 participants failed to identify track-specific requirements.
Inconsistent and unclear site taxonomy: 5/5 participants expressed frustration with the number of options to explore courses and identify requirements.
Faculty web page organization: 4/5 participants identified a course taught by a specific professor; however, there was a high degree of lostness (.63).
Design of Epi Course Planning Sheet: 5/5
participants felt the information architecture did not reflect content importance.
2. Fragmented Content
Course Content: 5/5 participants expressed frustration with course content (i.e. course description, quarter offering, etc.) being scattered across multiple pages.
Faculty Page: 4/5 participants used the faculty page to identify courses by instructor; however, the faculty page does not specify which courses a faculty member teaches.
“So I know that he teaches EPI 510, which answers the first part, but when that class is offered….I am going to have to go back and find that information, so I’ll go back to the graduate course listings. ”
Research Question Two: What sources of information do participants currently use to understand degree requirements and course information?
We learned that participants use a variety of sources to understand which courses are required, when courses are offered, the number of credits a course is, who is instructing courses, and which electives are available.
1. Key Resource: MPH Degree Requirement Checklist
5/5 participants use this to understand degree requirements and course information.
3/5 participants downloaded the checklist to avoid navigating the website to find degree requirements and course content.
2. Peers, Mentors, & Handbook
3 out of 5 participants use these additional sources of information to understand degree requirements and course information.
“I have my course degree tracker downloaded to my computer… I think I downloaded it because I knew it was hard to find on the website and I didn’t want to click around so I just kind of downloaded all the PDFs I needed”
Research Question Three: How efficiently are participants able to successfully find the information they seek on the UWEpi website?
Our findings indicated that participants could not efficiently find pertaining to course registration and degree requirements information. This is a high severity problem as it highly discourages users from looking for the information that may be vital to registering for courses.
What We Learned | Recommendations
At a high level, we recommended our client make changes to three usability issues.
Information Architecture → make information hierarchies clearer with design changes
Discoverability→ change the name of pages to better guide users to the information listed
Redundancy of Content→ consolidate multiple pages with similar information into one location
A comprehensive, action-driven recommendation report was presented to our client and is being used to inform the decisions during the website redesign process.
Reflect
Learning is the best part of the journey and here’s what this usability study taught me
Moderating is a complex skill that is continuously developing. I found moderating the usability tests to be the most exciting part, but it is a skill I will always be developing. With feedback from team members and observing other moderators, I am ready to take on the challenge!
Define the scope and goals of the project early and stick with it! As projects evolve, they can become more complex and seem to take on a life of their own. By defining our scope and goals early, we were able to stay focused and deliver exactly what our client needed.