How to execute usability testing for mobile applications?

Intuitiveness — effortless navigation across a mobile application;

Learning curve — how quickly a user can perform some key tasks at the first try;

Efficiency — the speed with which a seasoned user can perform tasks;

Memorability — whether a user is able to easily recollect all details after the first try in order to efficiently use an application next time;

Error frequency and severity — how frequently users make mistakes while using an application, how critical they are, and how effortlessly users can cope with these errors;

Subjective satisfaction — whether users are pleased with an app.

Discovery of real users’ needs, expectations, and behavior

Surely, before kicking off work on a project, you have conducted research to understand if your application is a good idea. Still, it does not matter how prepared you are, there is always something to be omitted or mismatching with your users’ expectations.

Usability testing gives an opportunity to validate your idea using hard data. Understanding what works and what doesn’t work via usability testing gives an opportunity to create the right product roadmaps.

An opportunity to save a lot of time, effort, and money

When usability testing is carried out starting from the project outset, i.e. at the stage of mock-ups, it is possible to detect critical usability issues and tackle UI/UX design correspondingly before the development process is actually started. The thing is that the earlier problems are detected, the less time, effort, and therefore money are needed to fix them.

Moreover, while being executed at the stage of the prototype, usability testing helps reveal useless features. It is important as according to the statistics about 64 % of app features are hardly ever used by real users. So, with usability testing, it is possible to avoid wasting your resources in vain.

Confidence in the success of your mobile application

Usability testing also serves as the major indicator of your app's success in the marketplace as it shows whether real users will find real value in the application. If some red flags are recognized, you can make corresponding improvements in advance, and then hit the market with a jaw-dropping application that can easily win the hearts of numerous users.

High revenue generation and conversion rates

It is well known that by investing in usability testing, you can double your return on investment since users are more likely to keep using an app and recommend it to their friends and family in case they are pleased with the user experience in general.

So you can see that proper and early usability testing makes a great contribution to user satisfaction and app success. Its benefits are happy users, cost cutting, high user acquisition and retention rates, high ROI, investor appeal, and many others.

Remote and in-person testing

Remote usability testing is carried out by participants from their own locations. The main advantage of this approach is that it is possible to engage a considerable number of participants from the most diverse geolocations, who use different mobile devices and network connections. On the flip side, in-person testing is executed at your premises, with a previously prepared environment, and under the inspection of usability specialists or moderators. This testing is more costly and time-incentive, but it provides more insights letting a QA team dive deeper into users’ logic.

Moderated and unmoderated testing

Moderated testing is supervised by trained staff directly or remotely. Moderators introduce a test, answer questions, and pose various follow-up questions. In contrast to it, unmoderated usability testing is executed without any supervision regardless of whether it is remoted or in-person testing.

Moderated testing provides more substantial results due to the close interaction with test participants. Still, moderated testing is more expensive compared to unmoderated one.

While moderated testing is utilized to examine the logic behind real users' behavior, unmoderated testing is suitable for testing extremely specific issues or revealing some behavior patterns.

Exploratory, assessment, and comparative testing

In the course of the exploratory testing, test participants should brainstorm, share their opinion on concepts. It is reasonable to use this approach in the project outset in order to obtain insights helping to define valuable features and introduce new brilliant ideas.

Assessment testing determines the level of user satisfaction, i.e. whether end users can easily settle tasks using an application under test.

When comparative testing is executed participants choose the most preferable option out of several proposed.

Lab usability testing

As you may guess, such testing is held in a specially prepared testing lab. The main advantage of lab usability testing is that it suggests total control over the whole procedure. Testing is executed in a completely controlled environment while test participants perform tasks being observed or supervised by moderators, who take notes and pose questions. This usability testing method is rather expensive due to costs related to equipment procurement, participant recruitment, good moderator employment, etc.

Guerrilla testing

Guerrilla testing is executed by random test participants who are asked to conduct express usability testing taking up to 10 minutes. Most commonly, it is held in public places like coffee shops, shopping malls, petrol stations, etc., and participants receive in return some gift cards, free coffee, or other rewards.

Guerrilla testing is the quickest way to collect a great deal of data validating particular UI elements or app features. Still, it is not appropriate for comprehensive testing or testing having strict requirements to test participants in terms of domain expertise.

Phone interviews

In this case, a moderator gives instructions to participants on task completion and collects feedback via phone. Testing is run in real-world conditions while user behavior can be recorded with the help of special software. A phone interview is a useful and budget-friendly way to execute usability testing covering a wide geographical zone.

Card sorting

Card sorting comes in handy when there is a need to get feedback on an app layout and navigation. Its outcomes show the way in which users naturally structure information. Consequently, it helps to make an application more intuitive and user-friendly. As this method is used, test participants manipulate virtual note cards, categorizing and grouping data in a way that seems to be the most logical according to their point of view.

Session recordings

In this case, special software is leveraged to record users’ actions. Session recording is a good option when there is a need to check the way users interact with UI elements and understand where they bog down, u-turn, or leave an app.

Five-second test

According to this method, test participants have just five seconds to glance at an app screen and then tell which element drew their attention or suggest which audience the app targets. In such a way it is easy to collect plenty of qualitative data on the first impression an application makes on its users.

Observation

As this method is utilized, test participants go through tasks, while moderators do not interfere at all, keeping silent, but observing and evaluating non-verbal cues. Observation can be very helpful, as people often tell things that are not what they think or feel in reality. On the other hand, non-verbal signs can reveal a lot of useful information.

Eye-tracking

In the course of eye-tracking testing, users' eye movements are monitored and analyzed with a special device. Eye-tracking helps to test the app's layout and UI elements, understanding what draws users’ attention on a screen and what makes them distracted. As a result of such testing, efficient heatmaps are created.

Planning

The first step is a traditional one. As well as any other kind of testing, usability testing cannot be properly executed without proper planning. To avoid delays and difficulties, your usability testing plan should cover the following aspects of testing:

Task preparation

As long as the usability testing plan provides information on the app functionality to be checked, QA engineers can define the most appropriate scenarios accurately reflecting what real users should do and prepare tasks on this ground. The tasks should contain minimum steps and should be clear, realistic, and in the right sequence.

Test documentation design

It is important to prepare in advance the whole scope of test documentation. Particular attention should be paid to pre-test and post-test questionnaires that should contain short, clear, and simple questions about a tested application. Receipt forms for minors and adults, a consent form, a digital recording release form, etc. should be taken into consideration as well. By the way, you can find templates of these and other documents on usability.gov.

Recruitment of test participants

To get accurate testing results, it is essential to engage in testing people who have not participated in the app design and development. In a perfect scenario, only real users who embody the target audience must conduct usability testing. In this context, it is worth considering age, gender, marital status, domain expertise, and other criteria. For example, when it comes to mobile game testing it is more reasonable to recruit teenagers who are just mad about gaming. However, when a fintech application should be tested it is better to entrust usability testing to adults or even specialists with corresponding domain knowledge.

Test running

The next step is test running that depending upon your goals, resources, and preferences can be remote or in-person, moderated or unmoderated.

Debrief session

A debrief session is held immediately after test completion with the aim to define and discuss problems arisen during usability testing as well as specify issues requiring further testing.

Analysis and reporting

In the course of testing, a great amount of data is collected, and it should be accurately and carefully analyzed. After a comprehensive analysis, a test report is generated with recommendations on mobile app improvements. As soon as the application is tweaked in a corresponding way, usability testing is executed again and its outcomes are compared with the previous testing results.