Project Overview

Our company was awarded a large scale 4-year federal contract to develop an application to assist users to complete required forms and meet compliancy. Because of the large scope, we scaled the project requirements to be spread through different phases, based on audience groups.

Depending on the user’s situation, completing forms could become overwhelming and complex, and as a result, if incorrect information was submitted, the user could be punished. Therefore, the delicate nature of the project insisted we first focus on these types of users. 

Our solution was two-fold; we created an application with an AR feature to scan forms for the users, and we created a chatbot that would guide users to pertinent information based on their profile answers.  

During the first two phases, our team focused on one small subset of users, and during the next six-month, as UX Lead, our team expanded audience coverage and content for a bigger subset of vulnerable users, specifically for the chatbot.

Challenges

There were several challenges with this type of project. The overwhelming large scope of the project in terms of content, forms, and audience base relied heavily on organized planning and documentation of coverage throughout the project. Many processes were developed to streamline tasks for efficiency in output.

As part of the contract, our team was unable to change the legal language of information provided to the users. Therefore, we had to create solutions to guide users to the correct information through careful design of chatbot conversations.

Another major challenge, due the sensitive nature of the project, our team was unable to have contact with the end user. We relied on the federal organization’s data, such as paperwork submissions and the organization’s experience with end users, in order to identify our target audience. However, we were able to collaborate with a third party contractor to oversee and observe usability testing of the application towards the end of this phase.

Roles and Responsibilities

As UX Lead, I created a project plan with set deliverables to be completed by the end of the six months phase. The plan was broken down into tasks that were tracked on the Jira board. I oversaw and made decisions on the direction and coverage of the project. In addition, I researched content for persona and journey map development and then used these as a foundation for conversational design to be implemented into the chatbot.

The team consisted of two part-time researchers, a 508 tester, a scrum master, and the development team.

Goals and Objectives

Our main goal for this phase was to expand coverage of end users for the chatbot. The main objectives focused on vulnerable end users and included:

• Identify a larger subset of users
• Content and Form Research
• Development of personas with journey maps (i.e. sample persona shown below)
• Conversational design based on research, personas and related content
• Accessibility/508 testing of application
• Usability testing of application

Process

The following graph shows the general process created to meet the goals and objectives for this phase. Within two week sprints, we met with the client four times, performed and discussed researched content, created one persona with a journey map, and designed roughly four conversations for the chatbot based off of the researched content and persona. Once materials were implemented into the chatbot, iterative testing and updates were performed.

Because we did not have access to the end-user, we relied on subject matter experts within the federal organization to provide information and additional data sources. We were able to test the user needs and usability of the app during the third party usability testing sessions towards the end of the phase. The usability testing results were documented and planned for the following phase of the project.  

Testing

Our contract with the federal organization required the application to meet 508 standards as well as their additional requirements. We tested the mobile application using a bluetooth keyboard and using various mobile tools across platforms (VoiceOver, TalkBack, Accessibility Inspector, and Accessibility Scanner). App issues were documented, fixed, and then retested. As a side note, we included a visually impaired individual as a persona to not only make sure that the application met AA WCAG guidelines but also to make sure the quality of the content provided was useful for this type of individual.  

For usability testing, we used a combination of our internal observations and third party results to determine the immediate and future improvements needed for the app. These sessions helped us gain a better understanding of questions users would ask and how they would ask the chatbot.

Deliverables

At the end of the six month phase, we delivered the following to the client:

• 10 personas with journey maps
• 40 chatbot conversations designed
• Usability testing results and intended improvements
• The downloadable and working prototype that included the new conversations

Future Phases

Our personas from this phase act as pinpoints for areas of future expansion for the project. Adding new parameters to expand each persona will cover both breadth and depth of target audience needs. With future phases, our team will continue to develop the app while simultaneously test and make updates to create a robust and useful product.