Conference Review: Enterprise UX 2018, Part 2

Morning of Day 1: BUILD

BUILD was the theme for Thursday morning’s sessions, which covered pragmatic aspects of designing for the enterprise that included the ideation, sketching, modeling, and prototyping of enterprise processes, products, and strategies.

Bias Toward Action: Building Teams That Build Work

Presenter: Husani Oakley

Husani Oakley, shown in Figure 5, is SVP and Director of Technology at the ad-agency Deutsch. The focus of his talk was on culture, recruiting, and people.

Culture

Husani spoke about culture in the context of “forming teams and building out teams” within an agency. His team “went through a value-identification process, as part of thinking about how we can modify our culture to be much more appropriate for the sort of work that we wanted to be doing. … We did a [four]-step process:

1. Identify your values. I mean really important things like: Who are we as a team and how does who we are ladder up to the larger organization that we’re a part of? What does that mean? What is our output supposed to be? Why do we exist? The answers of those questions are your values.
2. Write them down. Unless you write down the idea, it’s just in your head, [and] it doesn’t actually count.
3. Make them real. Then, you’ve got to identify things that make your team your team—that may not directly relate to the values that you just came up with—and change those things so they reflect the values that you just decided on and agreed upon and are written down. You’re going to make it real. It’s not just talk. It’s not just words on a whiteboard.
4. Hold people accountable. When you implement that, you actually have to hold people accountable. Holding people accountable to the values that you’ve just decided on and created and made real is critical. … You have to be ruthless about holding people accountable. Culture not working? Change it.

“Now, you analyze what you’ve done. You look at the next project that you do after you’ve made these value-based cultural changes? Did they work? Did the project work? … Was the project a success? What’s your definition of success? Your definition of success is driven by [your] core values…. If it didn’t work…, maybe you chose the wrong values…. Sometimes it’s the people that [are] not working. … Do you change the team to work with those people, or do you change the people to work with the team? … It depends on what those values are. But you have to really strongly, objectively consider those things. …

“Our first instinct was to…change process. That change was: there will be user experience, there will be technology in every single brainstorm and every single status meeting and every single brief we are going to make. Sure, with a rule that the right people are in the room at the right time, that’s great…, but that doesn’t change the culture. That’s just setting up a rule from on high—unless people know why that rule has been imposed. …

“We went back…and we analyzed our existing values, and that’s when we made the realization that…creative teams, here at this agency, are valued so much more than noncreative teams. Concepts, at this agency, are valued so much more than the people who make the concepts. Why is that? … It wasn’t appropriate for what we were trying to do. …

“In coming up with your value system, you think about: Why does my team exist? Why do we exist at this larger organization? Why does this larger organization exist? … So we worked on changing the culture. It turns out that one of the reasons that makers and builders weren’t valued as much as the thinkers and the talkers: … We had no personal relationships across teams. … Small adjustments to how people think about the work that they do can actually have a big impact. … Change the context of your team’s relationships with each other.

“Part of culture really is changing process, … as long as that process has those core values at the center. … Otherwise, you end up with process for process’s sake, …[which] doesn’t really help anybody—agile or waterfall or XP or whatever. … Implementing a process works when the process you implement reflects your core values, and your core values are dependent on the answers to the questions: Why am I here? Why is my team here? How do we fit within the larger organization? Once you’re able to do that, you’re really able to form a culture that actually works for you in your unique circumstance. …

Recruiting

“We were able to define our cultural values from scratch. We could define all of the things that we wanted this team to be and do and how we wanted them to think. We could figure that out first, then build a recruiting process and build a team…that reflected those core values. … We’d hit on things during [the] interview process that we knew would ladder up to those core cultural values that we’d already defined.

“Those values are: Getting stuff done is thinking about things in different ways, …breaking the rules of the industry that we were in and doing that in a friendly, open, and honest way—both externally and, even more importantly, internally. …

“ We know what our cultural values are. We know how we think they should be expressed in our team. But how do we find people to actually do that? … How do we ensure that [people] come in having the same values that we do? Or can [they] be taught those values as part of an onboarding process? … We wanted people with bias toward action. How do we find them? … Look for nontraditional sources in nontraditional ways.

“Talent is everywhere. … You can’t find people who [are] going to shake things up…if they have the same cognitive biases as everybody else and if you’re getting them all from the same place as everyone else in your industry. … You’ve already spent the time to define what your values are. Go back to them. Define a process by which, as you’re interviewing people, [you] quantify whether you think they fit your values…on a one to five scale. … At the end of the interviewing process, the talent manager sits with the hiring manager and says, ‘Here [are the] values that…are appropriate for this person and how they ladder up to the values of the larger organization. Here’s how every interviewer scored this person. …

“You should always be looking for talent…that hits your core cultural values. Recruiting is not just…for the hiring manager and…recruiters. It’s everyone’s job. Everyone should always be on the lookout for people who they think can work toward that team’s values. …

People

“We don’t do our work in a vacuum. … You’re building things for a person. There’s a person on…the other side of the screen. … We…put together a fairly formal critiquing process—not just for of our own work, but for competitive Web sites and other apps and software and services…. The full team would get involved. … That really got people to remember the person not just during requirements gathering, but as part of the entire lifecycle.”

Ethics in Tech Education: Designing to Provide Opportunity for All

Presenter: Mariah Hay

Mariah Hay, VP of Product at Pluralsight, who is shown in Figure 6, opened her talk by saying, “I like to break down the thought of human-centered design and what we do in our practice into kind of two things: We are problem finders and we are problem solvers. … These two things together… don’t work without each other. You have to be able to go find to the right problem to solve, then you have to be able to…execute on the solution.”

“With great power comes great responsibility, which is why it is crucial for us to really understand ethics in the context of technology. … Ethics [is] the discipline of dealing with what is good or bad with moral duty and obligation. … From as far back as the 5th century, professional groups have crafted their own ethical codes.” Mariah pointed out that we can learn from the ethical codes of medicine and engineering, but asked, “What happens when it’s not so life or death like in medicine or in engineering? What about when the consequences aren’t so black and white? What if there are some gray areas. …

“What happens when we problem finders and problem solvers end up creating problems for the consumers of our solutions? What then are our ethical responsibilities? … Cambridge Analytica, a data-analytics firm, used personal information harvested from more than 50 million users without their consent in order to build a system that could target voters with personalized content to influence their cultural beliefs and their voting behavior based on their psychological profile. Christopher Wiley helped build the algorithm. The work that Christopher and Cambridge Analytic [did] had far-reaching societal impact. They were manipulating the general public to influence policy, human protections, and legitimate representation.

“If you were asked to design a system like this or if you realized after you had started designing it, what would you do? The problems we are creating as an industry for the general public that we’re serving are originating from a lack of ethical accountability—negligence, disregard, and a lack of empathy. … In the case of Cambridge Analytica, I can’t help but feel that, when you’re designing part of that system, you’re very far removed from these kinds of ideas and concepts of the impact that it could potentially have, which is so large. … That kind of lack of empathy, that apathy toward the impact that you’re having allows people to kind of shrug their shoulders. …

“I’m in technology education, which is a big deal. Education, in general, is how we give people access to improve their lives, to improve the human condition, to provide for their families. …

“The company I work for, Pluralsight, acquired a company called Smarterer, which…had a smart algorithm to be able to, within 20 to 25 questions, assess somebody’s skill level. … We were able to [assess where people were] in their learning journey, where they needed to be, and what they needed to learn next. … The learners like loved it because we were telling them where to start. We were saving them time, and they they didn’t care that we were using their data in order to do this. We were basically harvesting data from them…and they didn’t mind that we kept the data because we needed it to show their progress, benchmark…, and…track their learning.

“But Pluralsight [doesn’t] sell licenses…just directly to users. We also have an enterprise offering, and part of our enterprise offering is surfacing some of the usage data [from users] to the companies that are purchasing these licenses for their employees. This was when stuff started to get tricky…. What should we surface? What do we surface? How do we surface it so we don’t betray…individual learners, but we also meet the needs of our other customers? So it started bringing up all these ethical questions: What would employees be comfortable with us sharing? Would it discourage or encourage them [from taking] an assessment if they knew that there [were] other people looking at this data…? What would their employer do with the information?

“I was really proud of our team for starting to ask these questions because one of our ethical tenets…is: Don’t weaponize your product. Make sure that the solution or experience you’re creating doesn’t have downstream negative impacts to anyone. The team…got to work. making sure that we knew exactly what we were getting into. We do a ton of user research at our company. All of the teams…are trained on qualitative and quantitative. … We did a whole lot of surveys to try to understand the breadth of what people cared about, then dug in to one-on-one interviews with people to understand the why….

“This brings me to the second non-negotiable ethical tenet our team decided we needed to stand behind, which is: Find your blind spots. Be honest and aware of your personal abilities. … The team took a hard look at downstream impact and their own practices. They made sure that, in addition to going and talking to all these humans, they also [asked] other practitioners…: What do you think? Are we covering all this, or do we really understand what’s going on? Do we have sound research methodology and plans for this—[to uncover] these things? They really took a look at how they themselves were doing the research.

“The result of this research…: Lawyers were, in fact, a little worried about us sharing some of this data. … We learned that learners feared being ranked against other people, but wanted conversations about proficiency [scores] with their supervisors. … They were also very concerned about these scores affecting promotions and raises, but they also wanted to showcase their skills. So we decided to share the proficiency level and not the score. …

“Enterprise companies started submitting suggestions for how we could use this new technology to help benefit them. They wanted to…assess candidates for hiring…, tie assessment performance to pay increases…, and require employees to take assessments. They were suggesting that we weaponize our product. …

“But there are other ways we can solve these problems, so…we decided to dig]deeper with the companies and really understand the painpoints…behind those suggestions. Then we could come up with other ways of solving those that didn’t weaponize the product and create problems for our learners. …

“The conscious decision not to take that laundry list of suggestions from the enterprises… and, [instead, to] consider the implications of implementing these things is a perfect example of proactive, accountable ethics.” Mariah described the many potential negative implications of taking a less ethical course.

“The third non-negotiable ethical tenet that our team employed: They took 200% accountability for what they created. They were 100% responsible for themselves and their decisions, and they were 100% responsible for the decisions of the people around them. They didn’t just take feedback at face value. They [took] accountability for truly understanding this feedback and [using] it as a starting point for the next round of work. We went on to use the knowledge of what companies needed to create better solutions that supported both the needs of the organization and the learners’ needs in a very healthy way. We created analytics so leaders could see competency ranges across groups and teams, so the organization could make better and more critical decisions… We also maintained the trust of our learners. …

“Making human-centered design decisions is in the ethical best interests of people. … Ethics are a critical competitive advantage for your company…. Accountability creates trust, openness, respect, and integrity. … Decisions are clearly made and weighted based on that shared understanding of values for customers in all different sectors. Teams can focus on collaborating around the best possible outcomes, and you see increased productivity, effectiveness, happier employees, fewer politics, and overall, a better bottom line for the company. …

“Human-centered ethics pay dividends in both the near-term and long-term success of the company. I challenge [everyone] to follow these principles and cultivate your own ethical awareness. … Ethics are not a given. [This] is something each of us must consciously decide to participate in. … [Ethics] needs to be part of our practice, how we operate, how we find problems, how we decide to solve them. … We powerful problem finders and problem solvers must make ethical decisions, not be problem creators, because with great power comes great responsibility.”

Lives on the Line: The Stakes of UX at the Scale of Government

Presenter: Marina Martin

Marina Martin, shown in Figure 7, was formerly Chief Technical Officer (CTO) of the US Department of Veterans Affairs (VA). She began her talk by describing the many challenges of delivering healthcare to veterans, who must enroll at each of the 154 VA hospitals individually. The fact that each of these hospitals was using a slightly different version of the VA’s electronic health record complicated data entry for users—at least it would have if veterans actually used the application, but they didn’t, so the VA wasn’t able to take advantage of the software’s many efficiencies. “We were told that veterans don’t use the Internet.”

A few years ago, “my team and I decided to go talk to an actual veteran and figure out what’s going on.” A team at the White House catalogs all letters that people write to the President by topic, “so a member of my team…went to this office…to find veterans who were trying to get healthcare from the VA who had written to President Obama. She found one…and sat with him to see what the experience was like.”

Marina shared a video showing this veteran’s experience when trying to apply for VA healthcare online. He didn’t remember his login information. Navigation was difficult, and he often found it hard to find VA forms on the Web site, including the enrollment form. He had to enter the same data multiple times on various forms. Too often, error messages appeared and obstructed his progress—for example, asking him to update his PDF viewer. He found the application’s terminology hard to understand, so finally bailed and called the VA’s 800 number to enroll. This user had tried and failed to apply online twelve times—even though he ultimately qualified for VA healthcare.

“This…was the viral video that finally changed things because this [user could] use the Internet. The problem was that [the application] didn’t work, so [we pitched] a new user experience for veterans at VA…called Vets.gov, which we launched on Veterans Day four years ago. … Every experience with the VA [was] basically like this…broken in its own way, and it [was] confusing. … We had three different registries of VA Web sites and, [on] those three registries…, we had between 564 and 1,565 Web sites for the VA. We had at least 12 user names and passwords. [Plus] all of the services and forms that weren’t available online at all.

“So this led us to try Vets.gov, which was about how do we make VA work for veterans and listen to veterans and get their perspective.” Marina showed another video with the same user trying to apply for VA healthcare using the Vets.gov prototype. The moderator asked this user for his first impressions. “What do you notice first?” He said, “This is definitely easier to read. It’s a larger font. It’s very…straightforward…to enter your name and general information. … I have not had that level of ease with a Web site for the government.” Marina told us, “I am very proud to announce that as, of yesterday, we have instantly enrolled 369,000 veterans in healthcare with this form.

“My team…continues…to go experience by experience with veterans, putting them first. … Every experience at VA [was] about as bad as the [old] healthcare application.” Next, Marina described the challenges of the appeal process, which is appallingly daunting. Her team redesigned the online form for applying for an appeal, including key information that veterans needed to make decisions that were in their best interests. They took on the redesign of other transactions on Vets.gov, one by one.

Next, Marina offered us some broadly applicable, tactical advice.

“Tactic #1: Lead with Your Users.”

“Going around complaining about the [user experience] is fundamentally ineffective.” Sharing videos of users struggling with an application were “an unstoppable force that led us to fundamentally make a lot of other transformations at the VA.”

“Tactic #2: Look Between the Silos.”

There is “opportunity between the silos. Often,…when you try to go at [an organization’s silos] sort of head on, it has a lot of defense mechanisms. The older the silo is, the more [it’s] looking for you. It doesn’t want to let you in. But something I discovered—admittedly by accident—at the VA, [which has] consistently worked for me since then, is that there’s a lot of opportunity in between the silos.”

To illustrate this tactic, Marina described how her team solved the issue of “unprocessed disability claims” for veterans, whose “disability [had to have been] caused by something in [their] military service.” She “talked to lots of real veterans” about the process, “sat in on [their] disability exams, then…in the office that processed the paperwork….” Marina discovered that the disability reports doctors wrote after examining veterans were reviewed by people “with no medical training,” who had to review a doctor’s detailed report in free text, then select one of just three “very specific legal choices.”

Marina told us, “First thing I did was get all the people in one room, and I had [each] demonstrate their half of the process to one another. Mind’s blown!” We discovered “some space in between the silos. So we built basically a TurboTax for disability exams. It asks the doctor exactly the questions that were needed [by] law and no other questions. Then, it [automatically] gave the [approvers] exactly the [information] they needed…, [so] we were able to do instant disability claims. … If I had gone at one side or the other side, they [would have said], ‘I can’t pivot like that.’ But because we were between the silos, nobody…saw us coming until we had a full-fledged solution that translated between the two.”

“Tactic #3: Use the Bureaucracy Against Itself.”

Marina illustrated this tactic by telling us another story about how, when she “came on as CTO of the VA, we were not allowed to use the cloud. So everything was within server farms. We were paying hundreds of millions of dollars a year for this, and people would come in and [say just] put it in the cloud…without anybody noticing.” Marina declared, “You can’t duct tape your way through fundamental transformational change.”

Her people suggested that she retrain the 16,000 people in VA IT by sending them through a short “Amazon Web Services course….” Marina asserted, “That approach fundamentally is wrong. … You can’t take someone who has spent 20 years in a career being the expert of their field, taking pride in…serving their [country’s] veterans by making sure that that server farm is running, [then] just say go learn this other skill really quick…. They are going to fight back. …

“The bigger an organization gets, the more processes [it has] in place.” Never proceed as though a “process doesn’t apply to [you]. … Instead, we wrote a great narrative, then we just copied and pasted it into every…piece of security paperwork. … Our Inspector General would not let us use the cloud because you cannot put the cloud in an evidence bag. … They have to conduct investigations. They were used to doing it by walking up to a particular server, picking it up, and putting it in their evidence bag. … So we spent a lot of time with them, showing how they could get their job done even better with logical access to the cloud—if they have kind of a superuser access to all of our logs—that they could search. …

“We did everything that the bureaucracy needed us to do, so now the bureaucracy sees the cloud as part of itself. Now, when you want to launch something new, you can just put it in the cloud. It took two and a half years to get there, but I don’t think that there’s any other way to do it.”

These highlights from Marina’s talk don’t convey her tongue-in-cheek delivery, which often elicited laughter from her audience. She actually made what might otherwise have been a dry topic interesting—even amusing.

BUILD Discussion

The moderator of this discussion, Eduardo Ortiz, Cofounder of PRTNRS, kicked things off by saying to Marina, “Your talk was sobering. … What we do matters, and it can sometimes mean someone’s life or death.” Then he asked a really important question: How can we help put a process in place in the enterprise to prevent [UX problems]? It doesn’t matter whether we are creating an application for sending messages to collecting data to providing healthcare, …the team has this structure in place to just actually care and be protected.”

Marina Martin: “I think it’s really about creating that culture of putting the user first and also definitely following Mariah’s point about not weaponizing what you’re building. You guys are all leaders so you can set an example, repeatedly—even if it’s not in the bureaucracy’s paperwork. Go out and actually talk to your users and think about that broader scale. … You actually have like a lot more ways that you are impacting people than you might realize at first.”

Husani Oakley: “Yeah, I’d second that. … Who of us realized that playing around with the Facebook API years ago would have ended up with Cambridge Analytica? You didn’t necessarily think of a direct line between what you’re making and a potential issue, which means all of us need to take that responsibility.”

Mariah Hay: “One thing that really gives me heart is, over the past decade, people that work in design and technology and creation have really started to be taken seriously, and it’s not just the MBAs running the show based on how much money we’re gonna make as a company. So we need to step up and use that voice. … It was interesting listening to Lisa earlier because she’s talking about things in the context of how you create that within a company, from an organizational standpoint. I have been thinking so much about it being the individual. I think it can start anywhere, but just making sure that it starts somewhere I think is crucial to us all moving forward as an industry.”

Eduardo: “We’ll take an audience question.”

Janaki Kumar: “My question is to Marina. You talked about turning the bureaucracy onto yourself, and you’d mentioned that it took you two and a half years. If you were to do this all over again, how would you go about it? Do you think there is a shortcut?”

Marina: I don’t think there’s a shortcut, and that’s a story I try to emphasize again and again. VA has 330,000 employees and 16,000 people in IT, so two and a half years—I have actually reflected on this a lot and I’ve spent time with the team—if anything we would have stopped resisting earlier and understood that the only way through it was through it. … I literally could have brought President Obama to a table and had him sign paperwork and that wouldn’t actually have changed what happened. I heard that a lot from people. They were like, ‘Just get somebody to sign off on it,’ and that’s just not the way to make change. It’s so tempting, and it seems like it should work, but it doesn’t. So I I wish I had—and maybe somebody has a secret shortcut—but I think it really did take as long as it took.”

Eduardo: “And that’s just really almost just a veneer, if you get someone to sign off on that, but it doesn’t create lasting change, which ideally is what a framework like Mariah was talking to would help create. We have one more…

Woman in audience: “So I’m wondering if our panelists have any advice on the long game. We’re talking about making meaningful change in big and complicated organizations, helping our employees and designers understand the ramifications and implications of the work that they’re doing on a daily basis. That’s much more about the long game. Advice on how to keep focused on the right things and the meaningful things and how to stay motivated.”

Mariah: “I have worked for a whole lot of companies—public/private, small/large, education/medicine, now EdTech, and I feel like the cultural permission to be able to even start talking about these things comes from the top, so you have to bring in leaders that are willing to create that kind of environment for the employees. Managing up is so much longer a road, and it will only happen in pockets. But, if you want that true proliferation and you’re serious as an org, you need a leadership team that can start planting those seeds and watering that message. Only then do I think you start to have that long-term cultural change to support those narratives.”

Husani: “Agreed, and it’s the small things also. How do you add it? How do you have conversations about this sort of work on a daily basis? How does your interaction with each other and with other stakeholders in your organization—how do those bits change? So that 20-second conversation may actually end up leading up to something much bigger, longer term, but it starts with that 20-second conversation.”

Eduardo: “Husani, you actually said, …you ended [your talk]with almost like a call to action: to help us all understand that just sticking with the status quo is not fine. We are not fine. But if we flip that from having leadership that is encouraging you to actually do the right thing to an organization that maybe hasn’t questioned themselves—like I’m sure that the Facebooks of the world never thought our platform is going to be weaponized, our platform is going to be driving change in a way that no one will ever foresee. How do you ensure that employees actually start asking those questions that maybe management is not asking because they maybe focused on other things.”

Husani: “Someone has to be the person to ask the question first and—whether that’s an employee, whether that’s someone at the top, whether that’s someone from the outside whispering that question into someone’s ear who’s on the inside—someone has to be the person to ask the question. So if my call to action meant anything, at the very least, it’s asked the question in your organization. We’re not necessarily all built for hopping into the streets. I’m small and frail, but I ask questions. It’s about being the person to open one’s mouth.

Eduardo: “Do you have any other audience questions?”

Lada Gorlenko: “So once you become the person who asked the question, you ultimately become that perpetual pain in the arse because no one wants to be delegated and be that person if you’re in the room. At some point, your reputation precedes you, and you become that…perpetual pain the arse. How do you manage that—being the pain in the arse that we are—with being someone who is being seen as a positive person rather than the negative person—and a lot of ‘Why do we need to be so difficult every single time?’ That’s a question I get every day when I ask my question.”

Eduardo: “I can tell you, from personal experience, that I am known as that pain in the arse wherever I walk in. But the one thing that I learned was to create coalitions and to work with partners. Almost all of us have something that we want to get done. As makers, builders, leaders, we have to learn to read the room. We have to learn about those that we are working with and understand what it is that drives them. Once you understand what it is that drives them, you need to figure out how that correlates with the wider or the larger question, which is: How do we ensure that we do the right thing? If you’re able to present it in such a way that it helps them meet their goal and it helps your goal be furthered, all of a sudden you’re not going into a room by yourself, but you’re going into a room with a team.

I saw this in government where, often, I would not even be in the room—and I’m sure you saw it as well Marina—and the questions that you would ask—Have we talked to the users? Have we done any research?—all of a sudden were being asked by someone…. They would ask, where are we going to do that? How [can] we talk to users? Do we know how users are going to actually use this? That is a huge win.”

Mariah: “I know personally as a hiring manager of product teams, you need those pain-in-the-ass people like you that speak up and stand up and have an opinion about things. Your whole team can’t [consist] of them, but if you don’t have at least a few, you’re not doing something right. Either you’re creating a culture where people feel like they can’t speak up and [are] just toeing the line—and that’s very dangerous—or you’re being really biased in your hiring—or maybe you’re not attracting people that are effective. So I actually see people speaking up and having those conversations over and over as signs of a healthy team, and I would I hope that other teams feel similarly about them.”

Husani: “I think it’s being strategic about when you speak up and how you speak up. I’m totally…that pain-in-the-ass guy. …It’s reading the room. It’s the right tool for the right job. It took me a while to learn in my career, okay, so slamming my hand on the table and shouting is not necessarily the right way to go. Although sometimes it is. Most of the time it isn’t, so how can you figure that out? It’s a fun challenge.”

Marina: “I have a personal policy: I stab people in the chest, which is to say I will not record people using your app, then play it for the president before I sat down with you and you’ve seen it and had a chance to remedy it first. In fact, that you are seeing that video may reflect how that team responded to it. But many other teams, go to them directly, give that feedback, don’t don’t let them be blindsided.”

Eduardo: “I think Robin had a question?”

Robin, in the audience: “Marina, in the story that you told about the space between the silos, who paid for that project?”

Marina: “We built it in a way that didn’t cost any money, and then, actually, because—talk about bureaucracy against itself—I helped this other project save $27 million, and after I did that, I said, ‘Can I please have $400,000?’ and they said, ‘Sure so then [it] just got bigger and bigger. Then, I paid for the pilot with that savings from that other project.”

Eduardo: “One more question.”

A man in the audience: “There is growing evidence that design thinking is reaching a barrier in organizations—that, in fact, it will soon reach a point of diminishing returns. My question to you is this: Can you focus your work on the role of the middle manager? Because it seems to be the case in each of your presentations that the middle management becomes the key obstacle that must be overcome now. It’s no longer the corporate C suite. Can you comment on that? Middle management is the barrier. It becomes both the opportunity and the barrier. Middle managers are resistant to the kind of changes that design thinking brings.”

Mariah: “I think it depends on the organization. That’s a broad brushstroke for all organizations. I’ve encountered it to be so, and I’ve encountered it not to be so. I haven’t personally looked at a study that kind of has those numbers in it, so that’s hard to speak to. I know that earlier today I was in a taxi and the fellow riders in the taxi were kind of commenting around design thinking as a term being kind of misunderstood, used and abused, or passé. The reality is, my hope is, no matter what we call it, is we’re doing the right thing for the people that we’re serving…, and doing it with…and learning a set of skills to apply critical thinking to problem solving. You can call that whatever you want. If you have middle managers that are creating a barrier and being resistant, the leadership of that company that governs those middle managers, they have an ethical responsibility to either to remove them and remedy that and, if they’re not, they’re not doing their job.”

Marina: “I think it’s about understanding the risk framework of [those] middle manager—like what is driving them, literally, what’s in their position description? If your design thinking requires them to stick their neck , you do the work of getting that position description rewritten. That’s the sort of stuff you have to change—their risk framework—so their best interests are aligned with what you need to do.”

Eduardo: “That is absolutely accurate. It cannot be stressed enough how an organization needs to be responsible for its personnel. As we know, an organization is not an organization on its own. It’s a group of people that have decided that they are going to do something. So, at the end of the day, it comes down to people actually caring about one another, people stressing…doing the right thing, and leveraging the tools that are that are in place or bringing in new tools in order to address the problems that are at hand. So [if] is design thinking the the problem, remove the label. [If] it is a lack of tools that they have, bring new tools. It doesn’t matter what it is. At the end of the day, it’s just about actually caring and doing the right thing.”

Husani: “I think sometimes we we find a way to think about a problem or to design a process and then we latch on to that and that’s suddenly the only answer. It’s ‘Your org isn’t agile?’ Then, you’re doing everything wrong. You’re not doing design thinking, then you’re wrong.’ No! People they’re individuals, and there are different ways to do things. We like tools, and we should remember that we’re using the tools for a reason and maybe focus more on the reason and less the tool.”

Mariah said to Marina: “I love your point about change the job description. Very rarely have I found somebody that I work with that is difficult just for the sake of being difficult. There’s usually communication in the way, ulterior motives, other incentives that are working against it, which is why I kind of go back to like we need to really take a look at that like you said, and it’s the leaders above them that have to do the hard work and understand those motives and incentives or disincentives to be able to overcome whatever barriers and accomplish what you’re trying to implement—whether it’s design thinking or anything else.”

Eduardo: “Okay, parting shots.”

Husani: Make things and make them well. Care about the things that you make.”

Mariah: “Stand up. Really understand what you stand for before you show up to do anything.”

Marina: “Go out. There’s ways that you can serve veterans or foster kids whatever it may be. Take your skills—even if it’s not at your day job—and find a way…”

Eduardo: “Remember that technology is easy; people are not. Dealing with people takes a lot out of everyone. Most importantly, remember that it is your job, because no one else is coming. It is up to us.”

Afternoon of Day 1: COMMUNICATE

COMMUNICATE was the theme for Thursday afternoon, whose sessions focused on communicating our insights about the needs of users, the value of UX practices and the unique skills we bring to the enterprise, and finding common language with our partners in other disciplines.

Communicating the ROI of UX Within a Large Enterprise and Out on the Streets

Presenter: JD Buckley

JD Buckley, Principal, UX Strategy & Research, at ADP Innovation Center, who is shown in Figure 8, gave a great talk about communicating the ROI (Return On Investment) of User Experience.

I first heard JD speak on this important topic in September 2017, at a Better UX event at UserZoom’s headquarters in San Jose: “Crafting Experiences in the Enterprise.” As a result, I invited her to write about this topic for UXmatters. We’ve published these two articles:

• Measuring the ROI for UX in an Enterprise Organization, Part 1
• Measuring the ROI of UX in an Enterprise Organization, Part 2

These articles convey similar information to her talk. I’ll provide heavily pictorial highlights from what was a very pithy, information-rich presentation.

JD acknowledged, “Concerns regarding disruption due to globalization and technology have encouraged more and more organizations to embrace user-centered research and design methods.” Figure 9 depicts some of the benefits of User Experience that have accrued to major corporations. “After years of fighting to be included in strategic-boardroom, decision-making, many UX professionals are now finding themselves seated at the grown-up table.”

“Along with that hard-won seat, many executives are often asking UX teams to quantify the value and return on investment of their efforts,” JD continued. “While
calculating a UX ROI can seem challenging in consumer products and services, it can be almost daunting in enterprise organizations,” as Figure 10 makes clear. “A completely fair question is: can a UX ROI be calculated and, if it can, what would it take? Could it help you communicate your value and your team’s value to the organization?”

JD related the stories of UX teams in two different organizations and their efforts to develop a UX ROI:

1. ADP
2. Design Matters

ADP’s Story

In this large enterprise organization, the UX team has been “incubating a UX ROI development method, said JD. “About five years ago, executive leadership at ADP decided to follow the example of other large, established enterprise organizations [such as] IBM, GE, Honeywell, and even JPL. They decided to start investing in innovation centers across the US and hiring more than 300 UX professionals. They embrace the belief that user-centered design was an essential component of the innovation equation.” Figure 11 depicts the central role of user-centered design.

The intense organizational focus on and support for User Experience at ADP let them think more strategically. JD told us, “We realized if we wanted to even begin to demonstrate our team’s impact on the organization, we were going to have to undertake a significant initiative prior to the redesign release. We were going to have to benchmark our users’ current experience. … It’s [difficult] to know where you’re going if you don’t know where you’ve been. [This] holds true when you’re trying to measure the value of a UX team. We couldn’t even begin to attempt to demonstrate our team’s impact on the organization unless we could answer the question: compared to what?”

The team found little information about exactly what UX metric would show the best connection to a company’s KPIs (Key Performance Indicators)—especially when considering the complexities of enterprise organizations—so they began to devise their own UX measurement plan, which is shown in Figure 12. They decided to take “a mixed-method approach, collecting both qualitative and quantitative data and to explain our impact with numerical data—which we felt can sometimes be easier to understand than qualitative data—especially when we were talking to executives.”

In determining how to execute their UX measurement plan, they realized that, before they could do a benchmark study, they’d first have to identify their primary users’ highest priority activities, or top tasks. Then, their benchmark study established a baseline for the existing user experience by identifying users’ attitudes, behaviors, and processes when trying to perform those tasks. Figure 13 shows top tasks and their benchmarks. They collected the same metrics for their redesign so they could quantify the difference between the metrics for the existing experience and those for the redesign. Finally, they connected these UX metrics to company KPIs.

JD’s team defined a framework for building a UX ROI that is shown in Figure 14.

JD noted, “When you’re thinking of [UX] metrics, it’s best to start with a broad array than to get too specific too soon.” Figure 15 shows the Heart metrics they initially used.

JD and her team concluded: “By comparing shifts in UX metrics, between our benchmark and subsequent redesign releases, it’s possible to determine which aspects of our users’ attitudes, satisfaction, and behaviors have the strongest correlation to specific company KPIs and, therefore, offer the largest return on investment for our UX team’s efforts,” as Figure 16 shows. (View an enlarged version of Figure 16.)

“Even if teams can surpass all the challenges inherent in taking a metric-driven approach to connecting metrics to UX efforts, the final hurdle of connecting UX metrics to company KPIs can be really, really tough and somewhat elusive.” said JD. Her team partnered with a new CFO (Chief Financial Officer) whose goal was to align “revenue metrics across the various siloed business units [and helped them to focus] on
exactly the right company KPIs,” as Figure 17 depicts.

JD said, “We knew that all of our measurement efforts were pointless if we couldn’t make them actionable. So many…product roadmaps…are never-ending lists of features and functionality—disjointed and disconnected; none making a significant improvement on users most important tasks or end-to-end tasks or activities.

“So a key part of our process was a series of mini-workshops [in which] we worked with our product managers to influence their future roadmaps, as shown in Figure 18. “In these sessions, we help them look for gaps in their current roadmap offerings and try to tie them back to the most important users,” as well as their top tasks. Working together, we [identified] their assumptions about users’ goals and the problems they thought the roadmap items were trying to solve for the user. [We] also look at their business success metrics for those particular features. We [also] shared some insightful data with our product managers that we’d gathered while collaborating with our Web analytics partners. We had been working with them to develop the critical paths and user flows based on information from our benchmark research.”

“In an effort to continuously evolve our UX ROI initiative, we’ve begun to investigate a multidimensional model [that] is built upon the foundation of the Service Profit Chain,” shown in Figure 19. “With this model, we examine the influence of various metrics across the organization—including UX metrics—to look for their correlations to company KPIs such as revenue growth and profitability.”

JD’s team has completed two studies comparing redesigns to their original benchmarks. For the most part, their metrics are trending in the hoped-for direction, as shown in Figure 20. But, for the metrics that didn’t, they’ve been able to determine what’s causing issues, then address those parts of the experience. They’re collecting metrics on a biannual basis to determine whether they’re making measurable improvements in the user experience.

Design Matters

JD’s team “started to wonder how flexible this framework might be” and whether it could work in another context. So JD applied the approach within the Art Center College of Design’s social design program, Design Matters.

The project team used the same basic UX ROI Framework in a completely different
context to communicate the UX ROI of a digital experience for an anti-gun–violence initiative called Where’s Daryl? Figure 21 shows the minor adjustments they made to the framework for this different context.

Peace Is Waged with Sticky Notes: Mapping Real-World Experiences

Presenter: Jim Kalbach

Jim Kalbach, Head of Customer Success at MURAL, who is shown in Figure 22, spoke about his experience interviewing a man who “formed what became the largest neo-Nazi group in America.”

“When you’re doing user research, …doing interviews, …you should ask the participants about themselves, get them talking about themselves, [and] build rapport,” said Jim. “That’s really good advice and that’s what I usually do. And that’s what I did this time.”

[This man] practiced hate and…hurt people because they were different [from] him,” said Jim, who initially felt fear in listening to him. “But [then] he began to…tell me about his life of change—how he transformed from a life of hate to a life of inclusion and peace and love.” Jim is a very effective speaker and dramatically conveyed the emotions he experienced during this interview.

In 2016, Jim published a book titled Mapping Experiences—“all about journeys and blueprints and diagrams.” A CVE (Countering Violent Extremism) nongovernmental organization (NGO) in Abu Dhabi, called Hedayah, creates counter-narratives to those of hate groups. A man who works for Hedayah contacted Jim because he wanted Jim to “lead a workshop to map the experience of former violent extremists,” or formers. Someone had suggested to him that, if he really wanted to understand formers who have left hate groups, he should map their experiences. So Jim met six more former violent extremists, as well as nine people from the State Department and some other NGOs when he conducted that workshop. They were looking for patterns, commonalities across the formers’ experiences.

The underlying reason for conducting this three-day workshop was to leverage these formers’ understanding of their commonalities to get more formers to work for CVEs. “They have a very valuable perspective.” They can understand what terrorists or other violent extremists are thinking. The workshop participants mapped their experience together, using the techniques that we use in our enterprises and applying them in a completely different situation. Jim said, “I wanted to prove that my book is not about software design. … I thought it wasn’t. In fact, I wrote it explicitly to not be only about software design. So I wanted to prove to myself that what we do can be applied to other situations. … The answer is: yes, we can. We did clustering and design-thinking techniques…How might we?, and prioritization exercises. The mapping exercise was a springboard into a larger discussion to understand their experience.”

In preparation for the workshop, Jim created the stakeholder map shown in Figure 23, with formers at the center. He also mapped out some types of formers and types of CVEs. From all of his research, Jim created the draft experience map shown in Figure 24. “It had three phases: radicalization, disengagement, and integration. I started to map…on the left because that’s where you start to map. I was overwhelmed with the amount of information there and, at some
point, I said I think I’m doing this wrong. … I started off on the wrong path. I was mapping the wrong experience. They didn’t want to know about how people get radicalized…. … They didn’t want to know about disengagement—becoming deradicalized. … [They] just [wanted] to know about this third phase here—this integration part and, within that, [they specifically] wanted to know…: How do formers, after they get out, decide to get involved?
”

Jim said, “I came up with a hypothesis diagram…based on my my investigation,” which is shown in Figure 25. “I didn’t want to go in there with a completely blank wall and do this from scratch, so I had a hypothesis about what that journey could be, and that’s how I got started.”

“So research, stakeholder mapping, experience mapping. These are all things that we have in our toolbox. … Those activities…prepared me for this mission. But it was more than…just the techniques. It was also…a sensibility that we, as designers, have—first around empathy, listening. I did a lot of listening. … Listening and being able to absorb what I was hearing. … As designers, we’re really good at observing human behavior. We observe the outside world. That’s one of the first parts of design.

“But also finding patterns. We’re expert pattern finders as well. We observe the outside world and then we create models of that outside world—personas, journey maps, interaction models—the main points that help us do things like scale design. You can’t scale design unless you find those patterns.

“This sensibility allowed me to listen to the conversation and [recognize] that’s a main point. I’d better write that down because we’re going to need that later. I knew that, if you [get] enough main points, you can cluster them and come to a deeper conclusion. So that sensibility guided me through this.

“Then, finally, telling stories, narratives, communicating. The exercise was really one of communication. How do we get the…stories of the formers? How do we get that connected with people from the State Department and NGOs to tell that story? It was basically facilitating a narrative.

“So those skills that you have, the sensibilities that you’re learning in your enterprise organizations to make sense of the problems that you’re confronted with are the same sensibilities that I used for this effort.” Figure 26 shows Jim’s deliverable: “A Former’s Journey to Involvement in CVE—from guilt to atonement.”

“Visualizations—diagrams like this that I talked in my about in my book—are really about encapsulating our understanding, our empathy of the outside world, encapsulating that in a model we can communicate back to our organizations.

“[It’]s a measure of success…when I hear my stakeholders, my team, using the language that I use. What is experience? The tools and the topics that we’ve been talking about speak to a broader human experience that we can understand.”

Innovate with Purpose

Presenter: Janaki Kumar

Janaki Kumar, who is shown in Figure 27, was formerly VP, Design Evangelist, in the Office of the Chief Design Officer at SAP Labs. She spoke about the important work that remains for people to do in the wake of artificial intelligence (AI) and automation.

“We don’t have to look very far in popular media to see predictions of AI and automation taking over our jobs,“ said Janaki. “How many of you…believe that, through AI and automation, robots will take our jobs and we, as humans, will have much less to do in the future? … Elon Musk has been very articulate on this topic, and he…says, ‘There is a pretty good chance that we will end up with universal basic income due to AI. People will have time to do other things, more interesting things, more complex things. Certainly we’ll have more leisure time.’

“But…there is still so much work to be done—from dealing with climate change, protecting wildlife, rebuilding infrastructure, feeding the world, ending disease, resettling refugees, educating the next generation, retraining this generation, and much, much more. So how can we responsibly say that there is no more work to be done?

“The real question here is: What is the relationship between work and employment and what is the role of business in society? Milton Friedman, a Nobel prize-winning economist, gave us a definition in 1970. He said, ‘The social responsibility of business is to increase profits.’ Friedman’s definition has guided our thinking for many years, in a dualistic mindset: for profit/not-for-profit; public/private; individual versus society. But this is changing. Larry Fink, the CEO of Blackrock, one of the largest investment companies in the world, [which] manages over 6 trillion…dollars worth of investment, said earlier this year, ‘Society is demanding more from companies, both public and private. To prosper over time, every company must not only deliver financial performance but also show how it is making a positive contribution to society. Now, this is a fundamental change from the current status quo.

“This purpose-driven mindset combined with the rise in AI and automation implies there are big changes ahead in the enterprise. When the role of business changes, this will mean the mission and the metrics by which we measure the success of business will need to change. It will need to take into account ways in which the company is contributing to society. When the mission changes, the work that needs to be done within an organization will need to change as well. Human tasks will become more complex [and] multidimensional—especially since things that are simple and can be codified and automated will be automatically automated.

“ Finally, the skills that an enterprise worker needs to have will fundamentally change. We all have to hone our skills to deal with complexity—to match the complexity of the tasks at hand. … When this purpose-driven, but complex and multidimensional future arrives, the enterprise designers have an amazing chance of success. So, in this talk, I’m going to share with you four superpowers that we have developed through the course of our hard work as enterprise designers that will enable us to be the superheroes of this future.”

“I’m going to…tell you a story of how I used my work experience in a completely different context. A few years ago, I went on vacation to India, and I met my childhood friend Dr. Vinu Arum, [who] is a medical doctor [and also has] a PhD in public health from Harvard. She runs a nongovernmental organization (NGO) in rural South India. She and her team work on…children’s preschool education and nutrition, women’s empowerment and entrepreneurship, youth employment, and community health. … I shared with her the work that I’m doing on business innovation and design, and she asked me, ‘Could you apply the same things to her NGO?’ At first, I was a little bit overwhelmed. The topics that she’s working on are so vast and complex. … I decided to give it a try—to use the exact same process that I use for business for the NGO—the design-thinking process of discover, design, and deliver.”

“ I went on site visits. I visited preschools. I observed the children. I talked to the teachers. I talked to the parents. Then, I went to some women-led businesses and talked to them about their challenges and how the NGO could help them. Then I very carefully organized the workshop. I organized the agenda, the materials, and the space, according to all the four focus areas—community health, women’s entrepreneurship, youth employment, and preschool education and nutrition.

“Then I invited a cross-functional team of not only NGO staff but…the business leaders, …doctors, and other stakeholders. I invited end users—the parents, teachers, and women entrepreneurs—into the same workshop. Then I taught them the basics of design thinking, focusing more on the mindset of divergent and convergent thinking. We learned interviewing skills, problem definition, brainstorming, and finally, prototyping.

“I’m happy to report that the workshop was a tremendous success from their perspective. The ideas that they generated out of this workshop energized the NGO. It also engaged the community because they felt that they co-created these solutions with the community, and the NGO was able to find funding and build the community health center, which was one of their goals going into the workshop.

“Reflecting back on this experience and some other social-impact experience that I’ve had subsequently, I’m struck by the easy transferability of our work, as enterprise designers, to the world of social impact. I truly believe that experienced enterprise designers…through the years of our hard work develop four superpowers that enable us to innovate with purpose:

1. Systems thinking—Enterprise designers are trained to think about the interconnection of systems and multiple sides of the platform. We have to think about the data creators and the data consumers. We have to think about the requesters and the approvers. We have to think about the supply and the demand. We have to think about access rights and value chains. Many of these links are invisible and yet they need to be considered, and we are trained to consider them in order to create sustainable systems. This is the same in social impact. It is important to think about the delicate balance of the systems when you’re designing for social impact. Especially when you’re creating solutions for vulnerable populations, you have to think about the sustainability of your solution from day one. Otherwise, you create a dependency that you cannot follow through. … So when designing for social impact, systems thinking is critical to build trust and to create sustainable success.
2. Silo-busting—As enterprise designers, we encounter silos frequently. Silos are kind of inevitable in large organizations. They are usually created for efficiency, and they end up standing in the way of our effectiveness. But that doesn’t stop us enterprise designers. We have to find a way to make it work across these silos. When we assemble a cross-functional team, we find out: Who are the decision makers? Who are the influencers? Who are my allies? Sometimes we translate across the different silos in order to get our job done, and sometimes we share credit with the silo leaders to build trust and help the silos weaken over time. We do what it takes to work across these silos to get our job done. My work in the NGO taught me that silos exist in small organizations as well. The staff assigned to the various initiatives work really, really hard, but they work independently, so they have very little visibility into the work of their colleagues. So the way I organized the workshop, with all four topics represented in the same room, was the first step in breaking the silo. However, there needed to be more done. Once during the brainstorming session, I noticed that some of the table members were whispering their ideas to their other table team members, so the other teams wouldn’t listen to their ideas. … I said, ‘I give you full permission to go eavesdrop on your neighbor.’ That changed their dynamic completely. They started visiting each other’s table to find out: What was their problem? What are their solutions? Then, …the final presentation encapsulated the interconnection between the different topics beautifully. … One of the NGO staff said later on, ‘For the first time, I now understand the challenges that my colleagues face and how I can help them.
3. Fighting for the underdog—Enterprise designers are relentless in fighting for the underdog. The products that we create are paid for by IT and management and used by enterprise workers. Sometimes the purchasing decisions are made based on features…, not on the experience. So, as enterprise design leaders, we have to make the case for experience and ease of use. We may use business terms such as higher productivity, lower errors, higher adoption, lower training costs, and higher capacity for innovation. We have to make sure we articulate it in ways that will resonate with the people who are buying buying it. [In] the same way, when you’re [doing] social-impact work, there’s usually a philanthropic organization that needs to provide funding for your work. So a successful NGO understands the needs of the community that they want to serve and what needs to be done there, but they have to articulate the message in such a way that it matches the goals and the KPIs of the philanthropic organization. [There are] similarities in the way a successful NGO operates compared to how a successful enterprise design leader operates.
4. Over-communication—We in enterprises are very familiar with this phenomenon: we are inundated by emails and messages and meetings, yet as leaders, we know that not everyone gets the message. George Bernard Shaw says, ‘The biggest mistake in communication is the illusion that it has happened.’” Janaki had us play a fun game in which we clapped out the rhythm of a favorite song to a partner who tried to guess the song. She told us that people don’t usually guess the songs. The person who is clapping the song hears the song in his head, but this person listening to the clapping doesn’t. “So this is the problem with communication: the more passionate you are about a topic or song, [the more] you hear [it] loudly in your own your head, and the rest of the rest of us just hear the clapping. You have to repeat your message multiple times. You have to be patient with the listener and repeat yourself until your message gets heard. This is extremely important when you’re working in a social impact setting. Many NGOs make the mistake of deprioritizing communication, and they want to have their work speak for itself. But when you’re working on social impact, there is an element of social change, and when you want to be a successful change agent, get ready to repeat your message multiple times, in multiple ways.

“To summarize, when the robots do take over all the tasks that can be automated, the tasks that will be left for us humans to do will be strategic, multifaceted, creative, and complex. This, combined with the change in the role of business—to expand beyond profits to people and planet—means this is an extremely great opportunity for enterprise designers to shine. The four superpowers of systems thinking, silo-busting, fighting for the underdog, and over-communication that we have developed through years of hard work in the enterprise are finally going to make us the superheroes of the future.”

I found Janaki’s presentation and her playfulness delightful.

COMMUNICATE Discussion

Margot Dear, Senior Director of User Experience at ADP, was the moderator of this discussion. She said, “Leading up to this conference, we were able to meet a few times to discuss this theme and how we were going to weave these presentations and these stories together. During these discussions we were looking at the theme of communication and how we could extend it beyond our nine-to-five jobs…. But enterprise companies are complex, they’re political, there are things that we have to deal with every day when we go to the job. The User Experience professional can bring skills or even superpowers to help with a sense making or meaning making inside and outside of work.

“I have a quote from the writer E.M. Forrester. He said, ‘How can I know what I think until I see what I say,’ So people are cognitively lazy. When we find an answer to a question, we stop searching. For example, the song in the song in your head. No alternatives are evaluated, not even knowing the half of it. I’m sure many of us here in this room today have experienced that at work.

“To the speakers, how can you apply your UX communication tools when you’re up against a team or a situation at work that might ignore all of the research, all of the work that you’ve done? How do you want to handle that?”

Janaki Kumar: Ignore the research. I think it’s important to get the buy-in up front. I think that’s the the best way. If they ignore the research, ultimately, it’s a business decision, and my go-to with that, if I truly believe that there’s value in the research, I would try to understand what it is. I really see that as a learning opportunity. Usually, the stakeholders that I work with are well intentioned. They want the best for the product. If they’re truly ignoring the research, I want to understand why.”

Jim Kalbach: I mentioned in my talk about sensemaking in general, through the visualizations and the models that we create. But I always say it’s not about the noun, it’s about the verb. So to answer your question: we have to engage and involve people. So, from my standpoint, it’s not about the map, it’s about the mapping. In my story, I didn’t even know if they were going to play along at all, but I set up the workshop so we could look and understand. There was not that much research, but a little bit, but that just became a springboard into the sensemaking that happened after that. It was the conversations that were the real thing. The diagram’s in the report, but that’s not the meat. It’s the activities that we facilitate as well.”

JD Buckley: “I agree with both of you. There are two tactics that seem to work: … People tend to be less critical of something that they have ownership of. So what we’re great at as UX professionals is alignment, collaborative work. Once people contribute something, they’re more inclined to somehow embrace some of it. So you start there with getting them involved in the solution and continuously look for ways to speak to people in their language. If it’s metrics, if you show a connection to quantitative elements. If you find out what’s valuable to a particular executive at a particular point in time, try to make the connection to that. Try to make it more meaningful to that executive. It will happen slowly. I wouldn’t expect it to be transformational. So that’s another part. People talked today about patience, in the last theme, and it does take a while and some fortitude. But it’s a lot about continuously looking for language that resonates with whoever your executive stakeholder is.”

Margot: “And sometimes it’s learning what that language is the hard way. Do we want to take some questions from the audience?”

Man in the audience: I have a question [for] JD. Can you talk a little bit more about the statistics behind the ROI? I don’t think you were showing causation—you said correlation—but did you have to show… What was the strength of the correlation? Were you just showing that things were going up when the UX people were doing certain things, or were you actually quantifying how strong the correlation was?”

JD: There were a couple metrics we were looking at. We gathered a lot. We looked for success and failure. We also looked at satisfaction—from the previous [version] to the redesign. We looked at that has a baseline. What was their level of satisfaction? We used NPS. We used SUS. We uses success and failure. We also did a kind of subset from the SUPR-Q, in terms of credibility and trust. So we baselined from the before the redesign and then looked for, in terms of a five-point Likert scale, a statistically significant difference between those metrics before the redesign and then after. We looked at success and failure rates, and we had specific tasks from our top tasks that we knew were the most important activities for those users. So we looked at how well they could perform them after the redesign.

“We also, in terms of recruiting, looked for a particular audience, and we collected demographic metrics, in terms of what payroll systems they use, age, so we gathered a lot of data. It was really a very challenging project and continues to be because we are looking for statistically significant metrics and correlations over time, running regression analysis over time.”

A woman in the audience: “One of the things I’m encountering is we’re on the transformation train, and we’re about a year in and resilience is something we are needing some inspiration around. So I plan to take these stories back. But I’d like to know if there [are] some things that you do—because you know, after year one, your transformation is old hat, and people are not interested in what you’re communicating. How do you bring freshness and keep yourself energized in the workplace? Because, out of the workplace, it seems it’s easier. But when you hit that wall of can’t yet again, do you have any stories or tricks that you would give to people?”

Janaki: “I would love to share that with you. As part of our work, I was able to work with many companies on their transformation journeys. One of the tricks I would offer is: create a storyboard or journey map of the ideal experience,d then use that as inspiration. When [we] create a storyboard, we’ve taken every frame of the storyboard to drill into what are the operational elements, what are the design elements and technology elements that you need …in order to achieve the storyboard. Make that really visible and, if there’s a hallway, a blank hallway in your office, just put that storyboard up and kind of say what you’re going to do and check it off. So that way, you can keep the people still remembering what that ideal vision is.

“One of the tricks we’ve done is map the current, as-is journey map. So you do that, and then you tell people, ‘Imagine you had a magic wand. Now, create an ideal storyboard. Or another technique is go interview people and say what was the ideal shopping experience, for example, and then use that as the inspiration. You can see the delta between the current and what you want. That usually motivates people very much.”

Margot: “I want to add to that because the user experience team in ADP is three years in. I mean UX has been around for a while, but nothing the way it is right now. We’re three years into our transformation, so I’m seeing kind of this dip, which is interesting because I hadn’t really thought about it before. I think part of what UX has to do is kind of rally around together and become the cheerleaders again. Because, I remember, when the team was first designed and developed and implemented, we were the shiny new thing. We were invited to every meeting. We spoke up. We challenged things. We were we were pushing back. We were not moving buttons around on a page, we were talking about full customer journeys—getting into their heads, service design—and we may have pissed off a couple of people. Maybe we don’t always get invited to meetings anymore. There’s just not enough of us.

“But I feel like you have to keep the fight because you have to keep looking back at where you started to realize how you’re getting there, and it’s slow. What was the example this morning? Two and a half years to get the document signed in the cloud. That’s pretty fast at enterprise. But I think it’s important to kind of, at the beginning of your year, to bring your team together and say, ‘These are the objectives that we set out to do at the beginning of the year. This is what we did—like amazing!’ Keep that faith and then move the needle a little bit. But you just probably have to regroup every once in a while to keep sane. It’s hard. I empathize with that one.”

Another woman in the audience: “After the past few presentations, I can’t help but feel very connected to this idea of using my design skillsets for social impact. Sometimes I think, at work, in enterprise design—sometimes that connection can maybe not be there in your day-to-day tasks. Do you have any recommendations or resources to how I might be able to be of use in social impact, besides just, ‘Oh, I’ll sign up and volunteer at like a local nonprofit.’ It might not be like, ‘Yeah, let’s immediately throw you in here for design workshops.’ So how could I get more involved with that?”

Jim: “It’s really tough. My story, my project, found me. I wasn’t going out and looking for that. I literally fell into that. But I’ve been thinking about this topic and, when I’ve given this talk in the past, people asked me that exact question. One thing that I realized, working with an NGO in a completely different field, is they’re not set up—logistically, mentally, or otherwise—to actually absorb the potential benefit that we can offer them. So running a Design Thinking session—why? What are you going to do with the outcome? How? I think there’s a mismatch. I think what we can do—and I don’t want to aggrandize this in any kind of distorted way—but I think we can offer more than we do. But there needs to be a mechanism to help those organizations like the interface and input there, and it’s not there. And I don’t know how to repeat it—at least what I did or some of the stories up here as well. I don’t know how you found the gun-violence thing, [JD].”

JD: Because I teach at Art Center College of Design, it was an opportunity that I reached out and asked because I already taught at the school. It was kind of like ‘Whoo, this is awesome! It wasn’t something that I was expecting. I reached out said, ‘I’m available,’ and they happened to reach back, so it was kind of a lucky accident. So it’s a very good question, and I don’t have a really efficient, well-informed answer, but it’s a really important question to ask.”

Janaki: “I can answer. I can share some perspective on this [from] my daughter’s graduation. Most convocation speeches at the very end say, ‘Find out who you are. What is your passion? What do you want to do?’ That’s the typical advice that people give. This person who gave the speech there gave a very interesting speech. He said go find the problems you want to solve. He told the high school students, it’s not just about what you know, what your skills are. Go out into the world and go see which problems really make you feel like you want to you want to solve them. Maybe then you can say, ‘I can I offer you some help to solve this problem.’ So that’s one way.

“In your own local community, do you see any problems? Don’t expect people are going to come to you and say, ‘Oh my god! We’ve been waiting for you to do this’. Usually, there may be some resistance. There may be skepticism. Every single design-thinking workshop, usually there’s skepticism. Then, finally, they really become vocal advocates. So that’s one aspect.

“I think what JD is doing with mentoring young students is an extremely great idea. We, as experienced enterprise designers, have a story to tell. We have developed certain skillsets that are worth teaching young students as they come out of school. So mentoring is a great way. Maybe there are girls-who-code programs you can help. Those are great ideas. Another thing to just throw in there.

“SAP has been really good about doing social sabbaticals, so I was able to sponsor one of my team members to go to Greece, …when they were resettling refugees from Syria. He went in [with] a group of other people. What he came back and reported to me was that he was able to use design thinking there automatically. He was able to add value from day one and bring people [along], whereas all the systems analysts and the front-end coders and the back-end coders, it took them a lot of warm-up time to start to contribute. But the skillsets that we have, we can go right from day one and start asking the right questions. Try to bring people together. So we do have a superpower that we should utilize for good.”

JD: “Just as a thought… At most companies—a fairly large company or a large enterprise—there’s oftentimes some internal support to do some community work. Is there a way you could potentially have [your] company pay you for a couple of days of doing [something]? Find an idea or a cause that you’re interested in and see if you can suggest some of your design thinking and interaction design and experience design strategizing solutions to that cause. That’s another potential way to approach it.”

Jim: “You might have to be willing to work for free or for nothing. I think, if you find something that you’re passionate about though, it [would be] something that you might want to spend your free time on. For the project that I did, I did not get paid for that. It was all pro bono.”

Janaki: “My hope is that, in the future, there will be space in organizations. For the sake of narrative, I kept this around one story, but there are lots of other stories that influenced my thinking as well. I am considered an influencer for Mercedes-Benz. A board member of Mercedes brings together certain groups of people to discuss future topics. One of them that they’re really concerned about is once robots take over much of the automation on the factory floor. What are they going to do with the employees? This is a real question and, in companies that are located in Europe, they cannot just fire people. They have their responsibility, So one of the ideas that was pitched by the board member and was discussed seriously was maybe we should take on certain topics. There are these UN goals. Maybe we should pledge to solve some of these goals and get serious and collect data on how we are measuring against them. Employees can pick the problems that they want to solve, and they are helping the company as much as they are helping themselves and the world.”

A woman from the audience: I’m super inspired by your stories. We’re really early in our transformation, too, and I’ve been very impressed with the work the ADP has done. I’m curious to know, actually, from all of you, given all that you’ve been able to do and share with us today, are there any things you would have done differently now?”

JD: “So many! What am I doing?!”

Margot: “That’s a tough question.”

Woman, continuing: “Maybe just a couple of things that are top of mind.”

Jim: “Stay level-headed.”

Margot: “Every day, I go home and I think, Did that just happen?”

Jim: “It does take a lot of patience. Then, you hit thresholds—particularly in large organizations—and you hit thresholds of frustration, of tolerance of yourself, but you have to stick with it. It’s brick by brick. I love the idea of kind of mapping it out because sometimes you forget about the progress that you’re making, but you are actually doing really, really good things, and you just have to remind yourself.”

Margot: “What I was trying to allude to is, maybe in a way, we kind of do a baseline every year, because when we’re setting our goals, we say these are the things that we want to achieve. Like we talk about, we’ve got to deliver this feature and this product, but what are the things that we want to do as a team. Then, review that every so often— to help you keep sane. But I like this idea of mapping this out because I think about my career and I feel like I’ve had the same conversations over and over again, in different companies, with different and different people. I think, Is that ever getting better or ever improving? Obviously, it is because we have conferences like this. They didn’t have these ten years ago.”

Jim: “Celebrate the wins.”

Margot: “Yeah, so we’ll celebrate later at Battle Decks. Every day, I think I could improve”

JD: I think the biggest thing is patience. I’m not a patient human being. So really working on patience. I have a lot of fortitude, but I can be impatient. Sometimes when you’re passionate about a solution, you get frustrated because you can see it and other people can’t. ”

Margot: “It’s like the song and the clapping.”

JD: “I think really just finding a way to communicate my passion in a way that’s digestible for other people. I think that’s the biggest lesson I keep trying to remind myself of.”

Janaki: “Another thing I would add is giving permission to yourself and your team. So one of the practices that I’ve had is lighthouse projects. Sometimes, as UX designers, when you’re doing projects that are internal for confidentiality reasons, you cannot share the entire process. It is it is good if you can take a social-impact project, but then you get permission from the organization to share the entire process—the photographs and everything. Then it’s a value add to your organization as well as to the NGO. So trying to find creative win-win situations will allow you to…exercise different parts of your muscles. For example, as an enterprise designer, you may get pigeonholed into just being a visual designer and you really are trying to get into research. Things like that. You can also do that, then bring those skillsets in and showcase your skillsets in that setting.”

Margot: “Sometimes just sleep on it before you send an email.”

Highlights from the Enterprise Storytelling Sessions

Session coordinator: Dan Willis, UX Consultant at Cranky Productions

Dan Willis was the curator for the storytelling sessions and introduced the storytellers. Here a few useful snippets from their stories.

Kit Unger: “When I interviewed for head of User Experience at Smartsheet four years ago, the founder of the company, Brent Fry, asked me, ‘What’s your North Star for design?’ I was a little bit nervous and…just started rambling on and on about design-thinking methodology. I talked about empathy for the user, framing the problem, ideating, prototyping, testing, and on and on…. Brent listened patiently for a couple of minutes and then…said, ‘That sounds like your process. What’s your guidance system?’

“So I thought for a minute, then I started talking about the importance of usability principles—things like visibility of system status, match between the system and the real world, recognition over recall, and several of Jakob Nielsen’s other heuristics. I think I got through about seven of the ten before Brent stopped taking notes, laughed, and said, ‘I didn’t quite follow all of that, but I am marking that as a very good answer. I got the job. …

“I didn’t think much about him saying, “I didn’t quite follow all of that’ until much later, after I was building out the UX practice. I was trying to get the designers to adopt these principles and to think about them when they were making design decisions or evaluating the product, but no one was adopting these principles that I was trying to instill. When I would talk about them to explain some of my own design decisions, they would ask things like, ‘What does recognition over recall mean again or what’s the difference between user control and flexibility?

“This really puzzled me…, then it hit me…: UXers tend to make things sound a lot more complicated than they really are. We say things like contextual inquiry when we mean watch somebody work in their own environment or data visualization when we mean infographic or heuristic analysis when we mean usability audit. We even complicate our titles. Are we UX designers, UI designers, UX/UI designers, product designers, interaction designers, information architects, UX architects? … The way we talk about what we do is completely opposite from what we do, which is to simplify things.

“Still these principles are really important to me, and I believe in them. I wanted my team to take them seriously because when you look under the hood at what…we do, it’s all about cognitive science. Design principles are based on human psychology, and [their] use…makes our designs better.

“So I decided it was time to apply some UX principles to the UX principles—to, hopefully, demystify them. I gathered all of the words of wisdom I could find from Susan Weinschenk, Don Norman, Jakob Nielsen, and a few other gurus in our field, and I put them in one big card sort. I asked my team to group these into no more than ten categories, label them, and describe them, and as a team, we went over the results together. We…debated until we arrived at our very own set of eight Smartsheet UX principles. It didn’t take long before all of the designers had them pinned up at their desks, and they were using them daily to make design decisions and to explain them in a way that everyone on their teams could understand. Even the product managers and the engineers started using them to argue for better UX decisions.

“I can’t say that these principles eliminated all second-guessing of UX designers. The arguments still happened, but at least these arguments are no longer based on opinion. That’s the great thing about principles: …they’re based on a lot of research and they work. So the answer to the question ‘What’s my North Star for design?’ is actually a lot simpler than what I told Brent Frye in my interview four years ago. My North Star for design is simply to design things that are truly valuable to people, that improve their lives in some way. The principles just helped me do that. … It doesn’t matter what set of principles you follow, whether you adopt one of the gurus or you arrive at your own through a lot of work and research. What matters is that you have some principles and you use them to do right by the people you are designing for.”

Kim Gausepohl: “I didn’t go over the rationale with the developers. I didn’t talk about the savings or the positive UX implications. I went straight to the design. Why didn’t I do that? That’s so basic. … All I had to do was provide the rationale that I had provided to all the other stakeholders. …

“We spend a lot of design effort designing for those initial touchpoints because our people just naturally tend to remember the beginning and end of experiences a lot better than all those things in the middle. If we can design those memorable experiences to be as as awesome as possible, that’s what our customers will talk about when they talk to their friends and colleagues about us. This helps us get more customers by word of mouth. …

Jesse Livingston: “I’m frustrated with the way we talk about enterprise users. In my experience, the focus in enterprise is all on the tools people need and the tasks that they do to just get their jobs done. In my consumer days, by contrast, we focused more on human emotion—how someone feels about a product or a brand. So I asked myself: Isn’t enterprise deeper than tools and tasks? Aren’t there deep human emotions behind those tasks that are just as important in enterprise as they are in consumer?

“Let me tell you a story of how shifting our focus from tools and tasks to human emotion totally transformed the direction of a product [for which] you would never expect emotion to matter: a data analytics product. I work as a user researcher for LinkedIn’s Talent Solutions products, …[which] help companies hire more effectively. I was brought on specifically to launch LinkedIn Talent Insight, …our first self-service data-analytics product for talent professionals. It gives them access to LinkedIn data to help inform their talent strategy. … We were laser focused on all the nitty-gritty product requirements… but after a few feature-focused user-research studies, something big was missing. … We didn’t understand the emotions that were driving our customers.

“So we took a big step back and just listened to them without any agenda…, and that’s when all the human emotion behind the pie charts and the tables started to emerge. … We were hearing this deep anxiety that our customers were expressing when they have to present data to their hiring managers because they’re under this constant scrutiny when they do. … When we…really listened to them…, we didn’t hear anything about custom dashboards or homepage content. We heard them asking the questions they knew their hiring managers would ask them: … Where does your data come from? How is that number calculated? Are you sure this data is reliable?

“This insight into the emotions of our customers was the clue that we needed to move forward, so we built a product framework that would help arm our customers with data that they can trust and understand [and] present…to their hiring managers with confidence. …First and foremost, to get that confidence, our customers need to feel that the data is accurate, that is trustworthy, so the first pillar is accuracy, We started making significant investments into the rigor and the methodologies behind our data so our customers could just trust that it’s rock-solid. But accuracy isn’t enough. They also have to understand the data so they can defend it, so the next pillar is transparency. We started building in explanatory ToolTips to show the underlying calculations behind the data, so they wouldn’t be caught off guard, not knowing how to explain the way the data was calculated. But neither accuracy nor transparency really matter if the data isn’t relevant, so the third pillar is…relevance. …

“They also need to highly manicure and control the data they’re presenting so it doesn’t generate unnecessary questions or create confusion, so the last pillar is control. We built some really granular controls into the product so our customers can manipulate the data at the most minute level possible. …

“In spite of the differences between consumer and enterprise, at the end of the day, people are just people. If we want to build products that engender trust and confidence, we have to…listen to their emotions.”

Brianna Koch: “I am absolutely not shy about admitting that I battle impostor syndrome all the time. … Despite your best intentions, the way you’re comforting your new designers really might not be that comforting. … I was given the opportunity to mentor a new, junior designer on our team. At first, I really did not want to do it because the impostor in me was saying, What skills or experience do I have to offer this person? But…I realized…maybe…I’m the best person to mentor him because it really wasn’t that long ago that I was overcoming the same challenges that he was about to…face….

“I watched people tell him, ‘You’re great! You were hired for a reason. We believe in you, and you have nothing to worry about.’ … Watching this happen to him made me remember how I felt when I first started this job. … Put yourself in the shoes of somebody new. You’re excited, but you’re insecure, and when you don’t know something, it can be embarrassing. It can be overwhelming. … Sometimes…you think that you’re doing it all wrong and you’re worried. … So the next time I met with my mentee, I told [him], ‘If you ever have a moment where you don’t you feel like you can do this job and you’re in over your head, it’s totally normal. It’s not true, but it’s totally normal. We can talk about it if you feel this way’. … I’ve never seen somebody look both so shocked and relieved at the same time. …

“Recently, I’ve learned about the four stages of competence that people go through when learning a new skill. The first stage is unconscious incompetence—you don’t know what you don’t know. … You quickly move on to conscious incompetence—you know what you don’t know and you know that you don’t know a whole lot. … There’s a lot going on that you don’t know about. … Being consciously incompetent is not a bad thing. The first step to learning is realizing that you have a deficit in knowledge, but this can be really frustrating. This is a phase in learning when people are prone to making mistakes—which is crucial for learning, but…scary and uncomfortable. We need to learn how we can reassure people in ways that affirm their feelings, but let them know that it gets better and how we can help them.

“Eventually, you learn and you grow and you move on to conscious competence—where you’ve learned the skills that you didn’t have before. [Then] eventually unconscious competence—[when] the knowledge and skills become second nature. …

“Hopefully, my coworker can navigate this period of feeling consciously incompetent, but comforted by the fact that being aware of what you don’t know is a good thing. …

Chris Chandler: “I want to share the secrets and the story of how I learned to love estimating [projects]. … The best and most accurate way to…make an estimate is to base it on something you’ve already done. … The second best approach to producing an accurate estimate is enumeration, which is a synonym for tedium. My team broke down a very complicated vision into what we called seven mythic cycles. Why mythic cycles? An epic is larger than a story, but what’s larger than an epic [is] a mythic cycle. We produced an incredibly complicated spreadsheet where we enumerated every single screen that we thought we might need for every single one of these mythic cycles and…included time for the process for getting those screens produced. …

“Executives…all seem to have two basic beliefs: Somebody, somewhere had a magical way to produce an accurate estimate for a project like this and…their slow…, internal team [does] not know the magical incantation. …

“I promised I was going to tell you the secrets to learning to love estimating, and I’m sorry to say I lied. The best that I can hope for is to give you a couple of ideas to think about that should make estimating a little bit less onerous:

1. It’s important to realize that asking how accurate an estimate is is the wrong question. Estimates are not prophecies. Estimates are guesses. What you really need to be doing instead is thinking about what’s the riskiest part of this project? What’s the most complex? What’s the thing we know the least about? [Then], focus your time and estimating on that part. Or better yet, think about things you can do or make that will help reduce the uncertainty and risk of that part.
2. It’s important to revisit your estimates. It’s very easy to fall into the pattern of doing all that hard work of estimating at the beginning of the project, then never revisiting that estimate. As Eisenhower said, ‘Plans are useless, but planning is essential.’ … t’s important to go back and update your estimates as you learn more. I know this is challenging for people, …but it’s really important. …
3. The real key…: estimates are not commitments. An estimate is a guess, …and the funny thing about a guess…is the farther out the guess, the more complicated, more complex, more risk involved, the less likely your guess is to be correct. So ironically, what that means is: the bigger and more complicated and more stress everybody else is feeling about the estimate that has to get done, the less stress you should feel because you’re not going to be right anyway. … Every time you give an estimate someone is going to mistake that estimate for a commitment. The secret to not hating estimates is to never fall for that.”

Jana Sedivy: “Those of us who work in enterprise software are accustomed to big projects and complex products. But my story is about little things, simple things, on a big, complex project. … Here’s something about engineers that I think we in the UX world don’t often fully appreciate: When it comes to releasing a product, the buck ultimately stops with them, not us. … Their main priorities are get it done on time and make sure it works. Everything else is second to that. So it’s understandable…for them to push back on our recommendations. Every time we make a [recommendation], they need to make that calculation: How long is this going to take? Is it going to break anything?

“We were coming into this project with a very healthy respect and admiration…for the craft and effort it takes to engineer extremely complex systems. The problem was that respect was not reciprocated. … When I was studying HCI (Human-Computer Interaction) in school, I was led to believe that my job was going to be all about customer insights and making the lives of our users better. But over time, I’ve come to realize that’s just a small part of my job. Most of the time, most of my job is trying to influence people who don’t understand what we do, think what we do is common sense, and often don’t even want us at the table.

“So how did we bridge that gulf? How did we earn their respect and build trust? I have five rules that I like to follow, but I’m kind of embarrassed to list them out because, honestly, they’re so simple and basic. On the other hand, simple and basic is often overlooked and undervalued, so here goes:

• Rule 1—Show don’t tell. You can’t just tell people, ‘Listen to me because I’m an expert. Even if you are, you have to demonstrate it every day.
• Rule 2—Admit when you’re wrong. Admitting you’re wrong is the most efficient way I know of a building trust. It makes the other person feel good because they’re right and you’re wrong. It also shows that your ego is not going to get in the way of building a good product. So the next time you stand your ground, they’re more willing to give you the benefit of the doubt.
• Rule 3—Details matter. Warren Buffett once said, ‘It takes 20 years to build a reputation and five minutes to destroy it.’ So we have to be super-meticulous about all of our deliverables. I can tell you, …when we did drop the ball or get a bit sloppy, it was a real setback in our relationship.
• Rule 4—Talk often whenever you’re in a challenging relationship. The temptation is always to avoid it, then before you know it, you’re just talking over email and Slack, and that just makes the problem worse. Building relationships takes interaction. There’s just no other way.
• Rule 5—Pick your battles. I have to admit I’m not proud of this rule because I don’t feel that it serves the user best when we compromise because we’re just so tired of arguing. But really how many meetings do you want to sit in arguing about the button placement when you know you need to address much more impactful issues. So I do think it’s inevitable.

“So that’s my boring secret: Doing simple, basic things over time consistently gives big results. It took time, but in the end, we did win them over, we did build trust, and most importantly, we built a great product.”

Conclusion

In Part 3 of my Enterprise UX 2018 review, I’ll cover the sessions from Day 2 of the main conference, which took place on Friday, June 15. The themes for Friday were INVEST and SCALE.

Discount for UXmatters Readers—To attend Rosenfeld Media’s Enterprise Experience 2019 conference and workshops, register using the discount code UXMATTERS, and you’ll get 5% off the price of your conference ticket.