The brief
Why GitHub, and why now.
When my usability studies course handed us a list of clients, I immediately wanted GitHub. I’d been a user for years, and I already had a quiet mental file of small frictions I’d stopped noticing. Now I got to test them instead of just absorb them.
The GitHub team asked us to test the usability of GitHub Projects — their project management tool for organising, prioritising, and tracking work inside GitHub. It’s what you’d reach for instead of Jira or a Kanban board, with the bonus of being tightly integrated with issues and pull requests.
We framed our purpose around two ideas: design debt (small missed features, windows, inconsistencies across views that accumulate during product evolution) and design drift (whether the live product still addressed the needs of its key users).
The study
Twelve sessions, two recruit sources.
5
Researchers
12
Participants
6
From GitHub testers
6
From UW Slack
- Team: 5 graduate researchers from HCDE at the University of Washington
- Participants: 12, split evenly between GitHub’s own testers database and UW Slack channels, selected by self-reported GitHub experience and professional role
- Methods: heuristic evaluation, user interviews, competitive analysis, a ground-up usability test design, remote observation & testing, thematic analysis
What we found
Four patterns, in the shape of friction.
01
Navigation inconsistencies
Users had difficulty moving fluently between different views of the same project, because the interactions and affordances quietly changed each time.
02
Feature discoverability
Different users had different mental models for where features should live in the hierarchy — and the current nav didn't match most of them.
03
Invisible system status
Error states and recovery messages didn't give users enough information to know what had happened or what to do next.
04
Missing bulk actions
Several bulk operations users clearly wanted to perform were either impossible or well hidden. The tool assumed one item at a time.
Impact
A seat at the developer table.
We presented the findings, the specific examples, and the participant quotes directly to the GitHub team, along with prioritised recommendations. The team was enthusiastic about the depth of the research, and we followed up with a deeper working session with the developer team to understand feasibility and trade-offs.
Running a usability study on a live, actively-shipping product teaches a particular kind of humility. You cannot test a frozen snapshot; you are always asking questions about a moving object.
The discipline is less about “finding the answer” and more about documenting a specific moment of friction well enough that the product team can recognise it tomorrow, even after the UI has changed slightly.