Busting the Myths About Agile Development and User Research

The Big Mistake

Nielsen Norman Group, leading pioneers of the UX industry, have conducted serious research into the state of User Experience in agile development and discovered that agile UX processes and development methods, when applied thoughtfully, can result in both improved user experiences and greater business value, as Figure 2 depicts. In comparison to waterfall methods, agile is gaining greater success. UX architects and strategists are implementing agile and Lean User Experience by integrating their user-centered design process into development. On average, 17% of the members of product teams were UX professionals—in comparison with 10% in past years. Thus, the average product team consists of ten people and includes about two UX professionals. An increase in UX investment demonstrates that more organizations now see User Experience as important for business.

Research methods require time and effort to understand both the users’ needs and their behavioral characteristics. While developers are moving into more agile ways of working, many UX designers and researchers feel that the focus on technologies and timelines have constrained the scope of research. But the truth is that designers and researchers have gained huge benefits by working with agile product teams, which align on the product the team is developing faster and, thus, can more quickly meet users’ expectations.

Companies across industries are launching new products and applications by adopting human-centered design. These products and applications produce greater value, while increasing user retention. Apple is a leading company whose mission is to build products that users need, with which they’ll form an emotional connection.

“The challenge is to use the principles of human-centered design to produce positive results, products that enhance lives and add to our pleasure and enjoyment. The goal is to produce a great product, one that is successful, and that customers love. It can be done.”—Don Norman, who coined the term User Experience and is Chairman of the Executive Board of the Interaction Design Foundation

The Cost of Bias to UX Research

Pioneers in software and Industrial design are moving toward agile ways of working to improve their productivity and ensure the quality of their products. The inception of a new product design or digital application should begin with user research. Because agile methods allow only a brief research phase, many UX researchers believe that agile methods are a threat to the human-centered design process. This strong belief has emerged because building a product or digital application that users do not need squanders an organization’s investment in product development.

“Technologists are not noted for learning from the errors of the past. They look forward, not behind, so they repeat the same problems repeatedly.”—Don Norman

Barriers to UX research are often arise as the result of various kinds of human bias. Researchers are passionate about their work and, thus, are unconsciously biased in favor of research. Leading psychologists have identified 108 types of cognitive bias, grouped broadly under information, time, and memory biases. Some of these types of bias affect UX design.

Heuristic principles and the psychology of design have helped both designers and researchers to overcome their biases. However, a few types of bias greatly hamper the user experience of products and digital applications.

“The designer does not begin with some preconceived idea. Rather, the idea is the result of careful study and observations and the design a product of that idea.”—Paul Rand

Cognitive or psychological bias is the human tendency to make decisions or act in an unknowingly irrational way, as Figure 3 shows. There are biases that affect the usability of physical and digital products.

Confirmation Bias

“Ever since I learnt about confirmation bias, I’ve started seeing it everywhere.”—Jon Ronson

Confirmation bias is a cognitive error that occurs when people pursue or analyze information in a way that directly conforms with their existing beliefs or preconceptions. Confirmation bias is the tendency to favor information that supports our beliefs, while ignoring other facts and evidence, and can destroy our ability to empathize and find the truth.

The more time UX researchers spend focusing on their own assumptions about users or a specific design solution during user interviews or focus-group sessions, the stronger their confirmation bias. Confirmation bias also causes UX researchers to ask leading questions of users that validate their assumptions, as Figure 4 depicts. The greater the bias, the greater the tendency of the researcher or designer to look for evidence that says their design works well for the user.

Singularity Bias

“Our technology, our machines, is part of our humanity. We created them to extend ourselves, and that is what is unique about human beings.”—Ray Kurzweil

Human minds have a desire to seek out missing information. Singularity bias occurs when the UX researcher cares disproportionately about an individual user in comparison to a group, as Figure 5 shows. The facilitation of focus-group sessions can provide an example of this bias: the analysis of the information gathered could be unconsciously biased toward the person who participates most.

“The creative workplace is based on a triangle with three vertices—culture, method, and people.”—Perl Zhu

UX researchers are more willing to empathize with a single, identifiable person than a large abstract group. Therefore, the addition of more people to a group doesn’t increase their willingness to help proportionally. On the contrary, our compassion diminishes as more people become involved.

Technological innovation is part of what we do. Therefore, we must not allow the bias of the marketing and sales teams to influence customers or users who are frustrated that their renewal options for managing their software subscription are hidden. Adobe has a provided a very good email option to their customers. When a researcher is gathering data to understand users, their curiosity is high and information passes to them through text messages, images, and videos.

Social proof and the exponential growth of the digital world has opened the gates to user research wider. For some time, marketing and sales have used the principle of social proof extensively.

Social proof is a psychological and social phenomenon in which people copy the actions of others to learn what behavior is appropriate in each situation. The term was coined by Robert Cialdini, Professor of Psychology and Marketing, in his 1984 book Influence: The Psychology of Persuasion.

According to Robert Cialdini, social proof is one of the most powerful concepts in persuading people to take certain actions such as buying your product or service. However, we must use this concept with great mindfulness.

A classic illustration of social proof is the way in which ecommerce Web sites attempt to increase the sales of a product by nudging customers. Statistics from a survey have projected 24.5% growth in ecommerce sales across the globe. The ecommerce user base is increasing. Even though the need for UX researchers, architects, strategists, consultants, and designers is increasing, they must know about human psychology to understand users and improve usability.

Memory Bias, or Shaping

“Every act of perception, is to some degree an act of creation, and every act of memory is to some degree an act of imagination.”—Oliver Sacks

In psychology, memory is defined as the ability of the human brain to encode, store, and review information. Memory is involved in processing vast amounts of information. This information can take many different forms—for example, images, sounds, or meanings. Countless psychological experiments have shown that, on average, the longest sequence that a normal person can recall on the fly contains about seven items. This limit—which the psychologist George A. Miller dubbed the magical number seven when he discovered it in the 1950s—is the typical capacity of the human brain’s working memory, as Figure 6 depicts.

Further research has proven that Miller’s assumption about the capacity of our working memory was overly optimistic. The magical number should actually be three, plus or minus one.

“To accomplish great things, we must not only act, but also dream; not only plan, but also believe.”—Anatole France, poet, journalist, and novelist

Shaping is the process of reinforcing successively closer and closer approximations to a desired terminal behavior. User researchers use the laddering technique of asking questions to probe more deeply and better understand user behaviors. Although agile ways of working have reduced the duration of UX research, this should not create biases that can lead to an inappropriate understanding of users.

Overwhelming Bias

“Users filter out a lot of the information that they receive, even when it could be important.”—William Edmund Hick

The overwhelming bias that we naturally imbibe as a result of our passion about design and creativity does not help us in limiting the number of options that we present to the user. Designers might be overwhelmed by their creativity and try to design a page that entirely solves the user’s problems. Web-site designers who create ecommerce platforms are biased toward framing the user to making a purchase, and this unconsciously biases them toward the use of progressive disclosure. Figure 7 shows why 40 Web sites that have been rated as having poor usability have failed.

Each of us has experienced this behavior. There are many experiences in work and life in which decision making becomes more difficult or takes longer. Hicks law can help researchers and designers to design user-friendly options. In the digital world, users often find it difficult to decide on a purchase in an ecommerce domain that presents a lot of options. Airbnb shares a lot of options, while Google Maps personalizes options, showing the best-rated and cheapest place to stay, as shown in Figure 8, which makes it seem a better option.

The growth of ecommerce is accelerating in the post-pandemic business environment, causing a lifestyle shift for many people across the globe. There is a strong need for UX researchers, architects, strategists, consultants, and designers to understand human psychology so they can understand these users and improve their stores’ usability.

The Empathy Gap

The empathy gap describes our tendency to underestimate the influence of varying mental states on researchers’ behavior, causing them to make decisions that satisfy only their current emotion, feeling, or state of being. Figure 9 illustrates this idea, using social media as an example. When UX researchers empathize with users, this empathy gap results in a higher probability of their neglecting important data when conducting only brief research. Traditional UX research would require more time and effort, enabling UX researchers to bridge the gap by conducting usability testing. Otherwise, the solution we conceptualize might not be useful to the users.

The empathy gap is sometimes also referred to as the hot-cold empathy gap. This is a reference to two kinds of visceral states. When in either a hot or cold mental state, people and researchers fail to acknowledge the temporary nature of their mental state, so they can’t put themselves into the mindset of the other.

Let’s consider a classic case of the empathy gap, in which the researchers clearly underestimated their emotional influence on users’ behaviors. The number of collisions between motorists and pedestrians increased, reducing their safety in the city of Toronto, Canada. A study that the Hospital for Sick Children conducted identified the potential cause of the reduction in pedestrian safety. By examining 1,965 intersections in Toronto where they had installed the countdown signals shown in Figure 10, between January 2000 and December 2009, the study found that collisions between pedestrians and vehicles had increased 26 percent after their installation.

“We looked at the rate of collisions before the countdown timers went up and the rates of collisions after the countdown timers went up and across all of the intersections, the average effect was more collisions … after the timers went up,” said Dr. Andrew Howard, who led the study.

A key finding from the study revealed that a major reason for the increased collisions was as follows: “Countdown timers are designed to provide supplemental information to pedestrians to aid in crossing during the flashing Don’t Walk indication,” the city says on its Web site. “They are not designed to provide motorists with information related to the amount of green time remaining.” This demonstrates cognitive bias and confirmation bias, which are common behavioral traits of people.

Malkovich Bias

When the Malkovich bias takes center stage among UX specialists and technical experts, they make decisions to develop products that real users might not need, as depicted in Figure 11.

Conclusion

Although technology and design might change over time, human psychology changes very little—our desires, emotions, and motivations. As human beings, UX researchers must discover ways of eliminating their bias as they gain experience in the field. Agile methods of product development have increased our awareness of the need for user research. However, most UX researchers feel that they have insufficient time for research. The more UX researchers can consciously eliminate their cognitive bias and choose the right research methods for a project, the more they can engage in logical thinking and quickly complete their research and analysis.

In this article, I have described several important forms of bias. Once we overcome our biases, we can bust the myth that agility in product development reduces opportunities to conduct user research. By implementing an agile approach, we can enable a seamless and effective handshake between User Experience and other crossfunctional teams.

A major advantage of UX research is that it provides huge value to businesses by ensuring that they build products that users need rather products that only the business values. Today, a product achieves success through an amalgamation of users, business, and technical capabilities. 

References

Mind Tools. “Cognitive Bias: How to Make Objective Decisions.” Mind Tools, undated.

Alfred Lua. “The Psychology of Marketing: 18 Ways to Use Social Proof to Boost Your Results.” Buffer, undated.

Robert Cialdini. Influence: The Psychology of Persuasion. Revised edn. New York: Harper Business, December 2006.

Jennifer Junge. “Confirmation Bias in UX.” Nielsen Norman Group, March 13, 2022

Growth Design. “The Psychology of Design: 106 Cognitive Biases & Principles That Affect Your UX.” Growth Design, undated.

Alexander Serjeev. “10 Best Practices for UX Success in Agile Environment.” Hygger, August 15, 2017.

Laura Klein. “Design for Agile: Common Mistakes and How to Avoid Them.” Interaction Design Foundation, undated.