UX: A/B Testing
When The University of Alabama in Huntsville’s Student Counseling Center reached out to our office to redesign their website, they had been receiving complaints that their website was difficult to navigate. Their call center was overwhelmed with clients asking for basic information about the center instead of its intended purpose of scheduling and crisis counseling referral.
I was asked to lead them through a process of content auditing, redesign, and user testing. After the content audit, I rebuilt their new site and proposed an A/B test of the new site versus the old with a team of volunteers. I created a series of user personae, trained the volunteer team on their use and a think-aloud protocol to help them verbalize their thought-process for qualitative data collection. Volunteers were given a persona to adopt with a unique goal and were assigned at random one version of the site to navigate in pursuit of their goal, narrating their thought processes as they went. I and an assistant timed each session and kept a log of observations about each user’s experience. We also video recorded each session for later examination and reference.
After the testing I compiled and analyzed the data. Results showed a faster, more efficient, and more positive user experience using the redesigned site. I presented the results to the Counseling Center’s leadership team, who decided to adopt the new site design. Three months after its publication, I followed up with the Center to see how the site was performing. They happily stated that they had been receiving fewer complaints about the site and that call volume was down due to the reorganization and simplification of the site’s content
Think-Aloud A/B Test
The video below shows a Counseling Center volunteer doing a think-aloud protocol. This volunteer was asked to adopt and study a user persona, then attempt to solve a problem using the redesigned website. Pretending to be a university professor who was concerned about a student’s mental health, the volunteer successfully found the resource she needed within about 60 seconds.