Usability analysis of B2B Applications for a US based Tech company

Challenge

The company develops B2B and B2C software products for various segments that includes cloud computing, web and desktop based office suites, pc games. It was identified that most of the B2B products were not completely accessible for differently abled users with assistive technologies like screen readers, or in high contrast dark themes. (Around 8% of users using their B2B products were differently abled across organizations)

Research Findings

Product release cycle, showing estimated time as 100% and actual time as 120%

Accessibility bugs identified showing, 60 to 65% bugs identified by fast pass tool, 5 to 10% bugs identified by trusted testers, 30 to 35% bugs are unidentified bugs.

The company was testing the software products post development phase. This was increasing the release cycle of the product by 20% as fixing accessibility issues for a developed product was more time consuming. Some of the issues were marked unfixed as the existing code structure could not be modified.
The software products were tested using an auto testing tool called Windows fast pass - Accessibility insights for windows which could capture only 60 - 65% of bugs.
Manual testing performed by Trusted Testers (TT certified experts) could capture only 5 - 10% of additional bugs as they were unable to empathize actual users
The B2B products did not have properly defined channels to get user feedback unlike its counterpart B2C products due to security concerns and lack of wide range of user base.

Problem Statement

"To develop a feasible and easily implementable solution that can efficiently capture, log, track and fix all the accessibility and usability issues present in the B2B products without affecting the product life cycle."

Shift left approach

Step by step process of ideation, design, development, testing done for grade B and grade C highlighting the role of Inclusive and Accessible by Design program in design and code phase.

A program was designed to involve UX designers and PwD (Person with Disability) users trained in web accessibility, during design phase of the product. The users were also given training in using their existing B2B products.
The users were involved in two phases. In first phase, moderated testing is conducted with the help of users with low vision using magnifier on the screen designs created with Figma. Here actual compliance bugs on color contrast, issues with placement of interactive elements are identified by the users. Also, potential bugs that could possibility arise if proper programming practices are not followed, are highlighted.
In addition to persons with disability testing, UX designers perform heuristic evaluation of the screens designed and list down all the usability bugs. Design specs such as tab stops, landmarks, keyboard shortcuts, name - role - values defined for different interactive elements are reviewed and recommendations are provided based on best practices.
The second phase of usability testing is conducted after the product is developed. In this phase, users with no vision using screen readers are involved. They test all the core scenarios and highlight compliance bugs that arise while using different assistive technologies.

The Process

Step 1 Feature onboarding, Step 2 Design spec and PwD review, Step 3 Spec changes, Step 4 Final review

Values delivered

Bug re[port, All the bugs identified are classified under accessibility bugs, usability issues, potential bugs and design feedback. Each bug is structured into observation, user impact and recommendation for the product team on how to fix the bug.

Design spec review, Review of Tab orders, Landmarks, Headings, Name, Role and Value defined for each interactive element is reviewed.

Moderated user feedback recording, Moderated sessions are conducted with the help of PwD users (Users with low vision, no vision, cognitive disability, motor challenge). Each scenario evaluation is recorded and shared.

Impact and Outcome

This program was started in Feb 2020 with remote setup. It was implemented for twenty different products and was identified that almost 95 - 100% of accessibility bugs were getting identified and were fixed without affecting the product's release cycle. Currently, we have successfully completed usability testing of 80+ products with 20+ client appreciations due to the efficiency of the program.

Page updated

Google Sites

Report abuse