Constructing a User Experience: The Cost-Benefits Compass

Step 3: Estimate the relative proportion of each persona.

Proportions may vary for companies of differing sizes. One way to estimate ratios is to pick a small customer, a medium-sized customer, and a large customer, then estimate the relative proportions of the personas for each of them. These three numbers will help give you a ballpark idea of the distribution. If the numbers indicate a skewed distribution, you may need to adjust the estimates later on in Step 8.

Note—My company constructed this table using information captured during persona interviews and augmented with Internet session data for users of the legacy system that our Service Delivery (Web Hosting) group provided.

Step 4: Estimate usage.

Each persona probably will not work full time with the product, but instead will spend only a portion of his or her time using the system. Persona interview data may suffice for estimating usage rates, though it is often useful to conduct an additional interview with a subject-matter expert (SME) to verify or adjust these estimates.

In the Customer Portal, the time spent using the system varies by persona. We captured session time per user by examining the server logs in the existing system. (Note—There may be multiple sessions per person. We cannot track individuals, so instead rely on user sessions.)

It is generally quite difficult to pin down exact numbers for individuals belonging to a given persona. When in doubt, involve an additional SME to corroborate them.

Step 5: Determine persona salary data.

The best source for salary information is occupational salary data that is freely available on the Web. For personas for whom salary doesn’t make sense, it may be possible to use the national median income.

* US Median Income
** Salary.com data

Step 6: Estimate the total number of users.

If your company licenses its product by workstation, this is as simple as counting the total number of licenses. For a site-licensed product, you can calculate the total number of users by taking the results from Step 3, then multiplying them by the total number of customers.

We sell site licenses for the Customer Portal, so we calculated the total number of users by taking the result from Step 3.

Step 7: Get out your calculator.

You now have all of the information you need to build your UX cost-benefits compass! The formula for the annual recurring cost for your customers to use your product is as follows:

Using this equation, for each persona in your set of personas, you would first multiply the persona’s hours per year by wage per hour and by the number of users corresponding to that persona, then add the resulting costs for all of the personas to get the total cost. The table below shows the calculations for the Customer Portal.

Note the relative cost for each persona: It will cost customers 23 times more to employ content owners than it will to employ designers. One could argue that, for every week of UX work you spend on activities that benefit content owners, you should spend a little less than two hours on UX work that benefits designers. Designers and administrators are both in support roles, so we should be concerned only that they can achieve their goals—not that they can do so with exceptional efficiency.

Even though customers are not themselves paying users of the CP, this persona drives the development of the entire system. When users cannot find the information they need by using Web self-service, they often make costly support calls instead. The high cost of live telephone operators is the single largest reason that customers leverage Web self-service and purchase the CP.

For every 1% increase in efficiency we can achieve for the content owner and developer, our customers will save roughly 4 million dollars. Even though this is only about $3,000 per year for each of our 1,350 customers, it still demonstrates the significant net benefit that UX work can deliver.

Step 8: Look for inconsistencies in the data.

Occasionally, after finishing the preceding steps, some piece of data will appear to be in conflict or disconnected from other pieces of data. As a general rule, you can revise estimated data if your harder data suggests it. You must obviously do this sparingly or else you would risk tainting the entire analysis. The goal here is to remove outliers without disturbing too many data points.

Our experience tells us that Step 4, usage estimation, is the most difficult to do accurately. Often these estimates have a high degree of variance from company to company and even within the same company. The idea behind Step 8 is to try to land within the correct order of magnitude for yearly usage time—Is it tens of hours or hundreds of hours?—rather than attempting to pinpoint time with artificial precision.

Conclusion

While a UX cost-benefits compass lacks complete scientific rigor, having some information is immeasurably better than having no information. Without a clear business direction for UX work, it is simply too tempting for Product Management to drift back into stealing cookies from the feature-list cookie jar. A UX cost-benefits compass can help your organization to get its bearing and point its efforts in the right direction.

If others in your organization question the validity of your analysis, offer to revise the input data, but not your method! Very rarely will the input data change the estimate’s order of magnitude, and you’ll gain buy-in from individuals through listening and responding to their questions. This can often turn resistance into momentum behind the UX cause.

We have explored only the benefits side of the UX cost-benefits compass. Once your organization understands the magnitude of UX benefits, it is a matter of judgment to decide through which specific UX activities you can achieve the desired level of benefit.

Efficiency of use and recurring annual labor costs account for only a fraction of the potential for leveraging cost-benefits justification methods to justify UX work. If you are interested in diving deeper or would like to attempt a more ambitious analysis, I recommend you obtain a copy of the seminal work, Cost-Justifying Usability, by Bias and Mayhew.