Fleet App Usability Evaluation and Redesign


Fleet is a mobile app developed by the Baltimore-based software company MasT. Fleet accounts typically are initiated by membership organizations. Fleet clients (membership organizations) can then invite their members to create accounts and use Fleet to maintain an organization calendar and groups. As a full CRM/ERP system, the primary goal is for the organization to collect better data about their membership so as to create and deliver more value toward the organization’s mission, and for member users to have greater control over their own data within the organization.

We differentiate between administrator end users and member end users. Administrator users manage the organization’s calendar, groups, and marketplace and use the app to check members into events and sell merchandise. Member end users are empowered by Fleet to manage their membership status (i.e. check whether active, renew, etc.), view upcoming events and meetings and add to their calendar, view and contact organization groups or committees (where applicable), and make other purchases such as event tickets and organization swag. Fleet calls it, “the whole organization, in your pocket.”

OBJECTIVE: To understand the usability issues in the existing app and to use participatory methods to mock up potential redesigns, refined after A/B usability testing in the lab.
ROLE: Equal collaboration with Manu Peethambar
TIMELINE: One semester (Fall 2018)
TOOLS & SKILLS: Usability analysis, paper prototyping, human-centered design, user research, Figma

Collecting and visualizing data

To turn data into information, we organized important ideas and concepts garnered through in-depth interviews with the client and various types of users, contextual inquiry with users, and heuristic usability evaluation using sticky notes. These categories assisted in structuring participatory design sessions and our first low-fidelity prototypes.

User personas

We developed two personas based on our research:

Ravi (member user)

young professional man sitting in a park and smiling

Ravi is a 29-year-old single man working as an engineer. He is energetic, adventurous and enthusiastic in learning new things. He thinks likes to try out the latest devices and understands how mobile applications work.

He is passionate about his career and was recently promoted to project manager.

Ravi likes to watch movies and TV shows of different genres. He plays guitar and is interested in learning about art. He plans to travel around the world and visit new places.

Ravi is originally from India and his family is an integral part of his life. He spends time planning events like birthdays and anniversaries for family and friends back home to make up for the distance between them.

Ravi has a routine he likes to follow on his working days and tends to enjoy his weekends by hanging out with friends. He tries to have some alone-time each weekend to catch up on things he’s reading. He is looking for volunteer opportunities to help underprivileged kids, but he’s very busy and isn’t sure he can make the time right now. Being independent and spreading kindness is his priority.

Tina (admin user)

A young professional woman working on a laptop with a mobile phone in one hand in a modern open office setting.

Tina is a 35-year-old woman. She got married a couple years ago and does not have children. She is energetic and passionate about her career as an urban planner and was recently promoted to senior management. She wakes up early each morning to take her dog for a walk in the park and to think about her goals for the day.

Typically, Tina heads to the office earlier than her colleagues so that she can dip out for outside meetings when she needs to, which is frequently because she is affiliated with a number of causes and projects. She is often asked to take on leadership positions, which she considers carefully, as she guards her time well.

Tina has volunteered with a community group dedicated to maintaining her local park for several years now and currently serves on the Board of Directors. She attends a few other community meetings each month in order to stay current on her neighborhood and city and gives back using her professional skills whenever she can. It’s a lot of work, and even though she loves it, she has to be careful not to burn out. Sometimes she wishes there was a way to make the smaller tasks easier so she could spend her time on high-impact projects.

She is an active member of her professional organization. Her social life and professional/volunteer life overlap significantly: after-work happy hours with friends double as planning meetings for the park or brainstorming sessions for a potential urban planning contract.

Tina loves traveling, taking weekend trips around the US whenever she can sneak away. She is currently planning a trip to Peru with her sister. She spends whatever other free time she has–maybe an hour or so in the evenings–reading the latest bestselling fiction. Thankfully, her husband loves to cook, and she cherishes the time they get to spend together over dinner, even if it usually isn’t until 8 or 9 PM!

Participatory design (lite) for low-fidelity prototyping

Manu and I each of us worked separately with novice participants to facilitate low-fidelity prototypes based on the high-priority issues identified by heuristic analysis. Participants were provided a range of tools, including paper, index cards, pens, and pencils. Interestingly, each participant came up with similar solutions to our target usability problems. The target usability problems were chosen for their combination of severity and platform independence, thus offering the most benefit to our client, Fleet by MasT. Usability problems explored were:

  1. Login screen – client key text field
  2. Error message state / 0 {items} feedback
  3. Marketplace grouping issue

The images below demonstrate the similar solutions arrived to by participants for usability problem 2 (Error message state / 0 {items} feedback):

These low-fidelity prototypes were cleaned up and made into A and B versions to capture the salient differences between each participant’s suggestions. These refined paper prototypes were tested with other similar user participants and again refined to inform our A/B medium-fidelity prototypes.

Medium fidelity prototypes and usability testing in the lab

Two novice participants met us in the UMBC Interactive Systems Research Center User Studies Lab to test our prototypes as well as the original app. To control for bias, participants tested the versions in different order and also completed the tasks in different order.

Tasks tested (based on the PD session):

  1. Log in
  2. Find out if you belong to any groups, then join a group
  3. Add a t-shirt to your shopping cart

We recorded gaze, time-to-completion OR think-aloud audio (depending on the task), number of errors, and a qualitative debrief session.

Medium fidelity interactive prototype version A
Medium fidelity interactive prototype version A
A prototyped version of the original Fleet by MasT app used to compare against redesign prototypes versions A and B

Findings and final prototype

We found that both prototype versions A and B outperformed the original design in all measurement categories. While the differences between the two redesigns were not as great, version B outperformed version A on a number of measures, most of all time-to-completion.

We did not stop with version B, however. Taking our quantitative and qualitative learnings from the usability lab study, we created a final redesign prototype for Fleet for the three usability problems singled out. We used Material Design by Google to creat the prototype in order to make it more quality and consistent design patterns accessible to a small team with limited resources.

Final redesign prototype integrating the strengths of both versions A and B

We delivered our prototype and report to the client at the end of the semester. Unfortunately, the app is no longer being supported as the company looks for investment. Nevertheless, I learned a lot from the project, especially about the complexity of a relatively straightforward product and the impact that small design decisions can make on usability.