STEAMbassadors Case Study
Usability Testing, Heuristic Evaluation
TIMELINE
10 weeks (Jan 2024 - Mar 2024)
ROLE
UX Researcher, Project Lead
SKILLS
UX Research, Cognitive Walkthrough, Heuristic Evaluation, User Interviews,
User Testing, Data Analysis, Personas, Affinity Diagramming
TOOLS
Mural, Google Docs, Google Sheets, Slack
TEAM SIZE
4
How might we improve the application and job search experience of current and potential applicants?
EXPERT EVALUATION
Methodology: Heuristic Evaluation, Cognitive Walkthrough
To begin the examination process, I conducted a heuristic evaluation of the STEAMbassadors experience as a whole, using Jacob Nielsen’s 10 heuristics. Using this set of heuristics, I examined the STEAMbassadors application process with a specific focus on the gig board platform, categorizing each finding and rating each one (qualitatively) by level of severity.
After completing my individual evaluation, I met with 6 other expert evaluators to discuss our usability findings, identify priorities, and ideate potential solutions. Together, we listed our ideas on a whiteboard and prioritized usability issues on a scale of 0-4 for further research. This collaboration allowed for a combination of perspectives and a new set of thoughts.
We found that the majority of usability issues arose during the application and gig search processes.
I then conducted a cognitive walkthrough to imagine the user’s thoughts and actions when interacting with the interface for the first time. I focused on the gig search process in which a user had been accepted into the program and wanted to find a gig that adhered to their availability. I examined this task answering the following questions:
Will the user realistically be trying to do this action?
Is the control for the action visible?
Is there a strong link between the control and the action?
Is the feedback appropriate?
USER RESEARCH
Methodology: User Interviews, Informed Consent, Personas, Affinity Diagramming, Data Analysis
My team and I focused on researching students with an interest in STEAM topics and mentorship. Our goals were to understand the needs, wants, and reasonings of prospective STEAMbassadors.
To ensure we had accurate results and relevant data, we used screener questions catering toward the target audience.
18- 24 year old students
Interest in STEAM
Live in the Chicagoland area
I conducted two in person interviews with questions designed to explore the participants' levels of interest and experiences within the STEAM fields and mentorship. I collected this data by recording audio and taking notes. My team and I were able to conduct nine interviews and utilized this data to create an affinity diagram to discover patterns and themes. We also created dimensions by graphing our interview responses.
RESEARCH FINDINGS
After gathering and organizing our data, we formed insights about the frustrations, goals, and motivations of our users.
By analyzing this qualitative data, we discovered that despite a high interest in STEAM, interest in mentorship was more spread out across the spectrum. Additionally, most users faced interruptions and challenges within the application process that caused feelings of frustration and annoyance towards all applications.
USABILITY TESTING
Methodology: Usability Testing, Data Analysis
My team and I then tested the usability of STEAMbassadors website and Gig Board platform to assess the efficiency and user satisfaction of the process as a whole, but with specific interest in the initial application and applying for gigs. We used data from our interviews, heuristic evaluations, and cognitive walkthroughs to identify specific usability concerns and create the main points of our testing.
Our tasks focused on five key features:
Submitting STEAMbassadors Application
Accessing Gig Board
Searching and applying to Gigs
Checking status of Gig application
Accessing certifications
I facilitated three usability tests and observed a total of three tests while recording audio and taking notes. Our evaluation measures were carefully designed to capture both qualitative and quantitative data.
Through our conducted tests, we successfully identified several common user pain points, shedding light on key areas for improvement within the STEAMbassadors platform. Our research primarily focused on assessing accessibility, design, and user concerns. Here are a few of the main insights derived from our findings:
Usability
Redundancy observed between the application process pages, and emails, as they offered similar functionalities
Design
Most users appreciated the consistent use of a purple theme and simple color contrasts
User Concerns
A notable lack of trust was observed among users, potentially stemming from concerns related to data privacy and security
TESTING FINDINGS
All observations and recommendations were presented task by task using a positive to very severe metric. Overall, our findings suggested changes regarding information architecture, graphic design, and content design.
TAKEAWAYS
Overall, this experience has improved my research skill set and enhanced my communication and collaboration skills. In the future, I hope to utilize this research to design the recommendations and improve the STEAMbassadors experience.
While working on this project, I learned the true importance of user research. Prior to this experience, I was less familiar with the distinction between UX Design and UX Research. However, taking the time to work one on one with users and understand the audience has significantly changed this perspective. I can now value the research necessary for data-driven solutions.
OVERVIEW
STEAMbassadors is a Chicago based community mentorship program that seeks students who have an interest in science, technology, art, or math to teach, mentor, and lead youth STEAM activities. Once accepted into the program, STEAMbassadors can search and apply to gigs that incorporate their interests using the Gig Board platform. My research focused on the perspective of prospective applicants in effort to improve the overall user experience in the application and job search process.
OBJECTIVE
Identify positive findings and usability issues within the STEAMbasssadors platforms to recommend design implementations and improve the experience of future applicants.
RESPONSIBILITES
Creating interview scripts tailored to target audienceFacilitating interviews and usability tests
Analyzing qualitative and quantitative data into valuable insights
Reporting progress during each research phase
Collaborating and communicating with team members