Each participant completed all five CUJs to ensure comprehensive coverage of the core app store flows. We implemented partial counterbalancing of task order to reduce learning effects while keeping the session manageable, striking a balance between methodological rigor and participant cognitive load. This approach helped ensure that performance differences were driven by task difficulty or accessibility barriers, rather than by the order in which tasks were attempted.

Before each task, we asked participants to rate their familiarity with the specific task. This allowed us to analyze whether prior experience influenced performance, which could help distinguish between true usability issues and novice learning curves.

During task execution, we used a think-aloud protocol, even though we initially debated whether to exclude it in favor of more accurate time-on-task measurements. We ultimately decided that the rich qualitative insights from hearing participants’ in-the-moment reasoning would outweigh the trade-off in timing precision.

After participants either completed the task or reported that they were finished (even if unsuccessful), the moderator:

• Recorded task completion status and any observable issues.

• Administered post-task questions to capture immediate reflections and perceived difficulty.

Once all five tasks were completed (following the same sequence for each participant), we administered a post-study survey to collect attitudinal feedback on the overall experience, perceived accessibility, and confidence in using the app marketplace independently.