The X-Interviews | The Future of Research
Interview #11 - Alfonso de la Nuez on Research for Business Impact
Alfonso de la Nuez on Research for Business Impact
Today on The X-Mentor it’s Alfonso de la Nuez on The Future of Research.
Alfonso is an entrepreneur, advisor, investor, co-founder and formerly CEO of UserZoom, a UX Insights Management company founded in 2007 that helps businesses gather and manage the insights they need to design exceptional digital experiences. UserZoom was acquired by Thoma Bravo and was merged with UserTesting in 2023 and is positioned to become a billion-dollar user experience, insights, customer experience, digital experience insights leader.
Alfonso is also an author and featured speaker at industry conferences, where he is inspiring the next generation of CX & UX professionals to create innovative products.
In this X-Interview, Alfonso and I will be discussing:
The X-Mentor: Welcome to the X-Mentor, Alfonso. Thank you very much for being with us today!
Alfonso: Thank you for inviting me, Greg! My honor.
The X-Mentor: It's been a crazy year, 2023! One of the things that really stood out most was how industry and roles are shifting. It reminds me of what you were doing with UserZoom back in 2007, basically changing the way User Research gets done. We’re going to talk about UserZoom’s innovation and impact. But first, I'd like to reflect on what's happened in 2023 with the merger of UserZoom with UserTesting and talk about how these two platforms are going to take it to the next level.
The UserZoom & UserTesting Merger
Alfonso: Absolutely. I think we both started at the same time, maybe less than a year in difference between UserTesting and UserZoom founding time. We both started with a very similar value proposition to address a problem or ideal scenario. Which is, doing usability testing in a physical lab one-on-one, was a difficult thing to do. It was a great thing to do, but difficult. It was time consuming, not efficient and limited to geographic location. It was expensive. We had a little bit of a different approach to business in general. But essentially, we were both in this world of remote testing, online testing, or cloud-based testing as I like to call it. You know, a lab in the cloud so to speak, where you have automation and scalability. I always talked about scalability of research and testing, but I think in that sense there were a lot of similarities.
However, through the years UserTesting very much focused on qualitative research and a lot of the one-on-one interviews that you would do in the lab. While UserZoom focused a little bit more on trying to quantify usability and quantify experience. I know that sounded crazy at the time and a lot of people criticized us.
Today, when you think about business and business impact, which we'll talk about later in the interview, quantifying the impact is important, right?
So, the two companies had a similar focus on what usability testing and UX research in the cloud would be. Then UserTesting focused on the qualitative and in order to get fast qualitative research insights you needed a panel. UserTesting started building a panel of people that were ready to participate. UserZoom partnered with panel companies, and we focused much more on building software that would allow you to do multiple types of studies.
So over time, we were both successful and we grew, and we raised capital. UserZoom was a profitable business for a while. And then in 2014, UserTesting raised capital from Accel Partners. Accel is a top-notch VC, a type of investor that wants to focus on fast growth and building very big companies. UserZoom, by contrast, was raised from a private equity firm, Sunstone Partners, a different type of investor that wanted to focus more on profitability. And so that's where I think the two companies went separate ways in terms of how to do business. But they still focused on the same value proposition.
A few years later, you can see that we've built similar companies, we have a ton of the same customers. So, the same ICP (Ideal Customer Profile), the same type of people that buy from UserZoom also buy from UserTesting. Once we get to 2022, UserTesting has gone public and grown to about $200m in revenue, while UserZoom remained private at $100 million, and both companies are really leading the space.
Thoma Bravo, a top Private Equity investor, is the company that ended up acquiring both UserZoom and UserTesting and merging us in Q1 2023 to create a category leader. That's a very typical play for an investor like this. Which is, they find companies that have similar value propositions. The customer is going to love that they don't have to deal with two companies, and they can just deal with one company. So, the market loves it. And then what they do is they add operational expertise, operational excellence to make sure that we're not double investing in things or we're not overlapping. And in a period of one to two years, the company will become one company, is profitable and is growing healthy.
That’s what basically has happened with UseZoom and UserTesting. This past year, 2023, was the year of the merger and like other mergers, it's been a rather challenging year. It's been a year where you have to put two cultures and two products together, but at the same time it's also a difficult year in the market economy. All companies, all startups and scaleups have had a rough year. So, it's been a rough year all around.
However, I do have a ton of faith and hope for 2024 and 2025 and beyond where you'll have a billion-dollar user experience, insights, customer experience, digital experience insights leader dominating the category. Then investing a ton in AI and automation to make the researchers lives easier and help them design great digital experiences.
The X-Mentor: How difficult was it in those early days? UserZoom was trying to convince Researchers to conduct their research outside of a physical usability lab.
What was it like trying to convince Researchers to change their behavior?
“In those early years, we went through hell and back honestly when it came to convincing UXResearchers.”
In those early years, we went through hell and back honestly when it came to convincing UXResearchers. I remember very well that we had two to three hurdles to go through. Maybe even more. For instance, what is UX? Because we are talking 2007, so I know that obviously you know Jacob Nielsen, Jared Spool and a lot of others who have been doing this for decades. But when it comes to the mass-market, we were telling people why UX was important in the 1st place.
And I remember that in 2007, Steve Jobs said here's a great experience when he introduced his iPhone. And here's a poor experience. He didn't say it that way exactly, but I imagined that.
[Laughs]
But he made design ‘an experience’ and put the importance of great design and UX on the map, so to speak. So, from a timing perspective, it took a while for that to take off.
The first hurdle we had to go through was, “What is UX and why is it important?”
When we talk about ICP, (Ideal Customer Profiles), we go to the Researchers and the [Product] teams. Not the Design teams, by the way, because design wasn't even involved in this back in the day. But with the Researchers we had to say, OK, so you know that UX is important. We don't have to convince you because you're doing UX research. But how about online and remote to complement your work in the lab? Having been in a lab for seven years prior to UserZoom, we never ever said don't be in the lab, don't do lab research. Instead, we said do lab research whenever you can, but complement it with online because it's going to help you tremendously in speed, efficiency, and quantification – validating your lab research more broadly.
The second hurdle was, “What is Usability testing?” Our value proposition was to say, “Don't just do lab, go remote.” People looked at us like, wait a minute, you can't ‘touch the user’. And that's literally what they said. “I cannot do research if I can't touch the person, if I’m not there in the same room.”
We went through years of that, Greg. We still prevailed because the value was there over time. We just showed the value was there.
The third hurdle was that everything was now cloud based and so now we have to convince security teams, legal, compliance, all of that stuff.
Because even if we had a user that was happy to go beyond the lab or even sometimes full remote and do research, we still must tell the purchasing departments, the legal departments, the security departments that their data was going to be secure in a cloud, just like Google's, just like Amazon Web Services, etc. UserZoom used Amazon Web Services, by the way. That took literally sometimes 50% of the whole purchasing or sales cycle, if not more.
Why am I explaining this to you? It’s because I think 2023 has been tough, but trust me, we had very, very tough years back then.
[Laughs]
Because we had to convince about the concept of UX. Then the remote UX testing or research and then the cloud. We had all three of those hurdles to jump across.
Research Innovation
The X-Mentor: The reason I'm bringing up the difficulty of change is to give our readers some perspective on “The Four Frictions” that operate against innovation and change as described by Loran Nordgren and David Schonthal in their book The Human Element:
Inertia – Does the idea represent a radical break or tweak from the status quo?
Effort – How difficult is it to implement change?
Emotion – When people feel threatened by change.
Reactance – When people feel pressure to change.
UserZoom encountered a number of hurdles while on your journey. Those hurdles are various levels of The Four Frictions, right. And it was the diligent and persistent effort to change the way User Research was conducted, to solve a real problem, that led to UserZoom’s success. Underneath those hurdles were The Four Frictions that were pushing back against your innovation efforts. But ultimately, change did indeed happen because it just had to. The problem clearly needed to be solved.
That’s what true innovation looks like, right?
Alfonso: I consider UserZoom awesome innovators. I don't like to brag, but of course I like to take credit for certain things. I have three US patents right here on my wall, issued to me and the company co-founders. We definitely innovated in this space, no question about it. Because it took us years and we found a way to get real value, using technology, taking huge chances, etc. Now for me, innovation needs to be linked to value. Because you can innovate and do some really cool things, but are they really delivering true value to the business, to the users, etcetera, right? And I think those are two different things that you must consider. Sometimes you talk about innovation, but there's still no focus on value. Is this innovation really valuable? And so, I think we did both with UserZoom.
With AI, I have to be honest, you know, I do respect “the revolution” or the evolution. I'm still not 100% certain whether it's an evolution or a revolution, and a lot of people say that AI is going to change everything. I also heard about cryptocurrency, web3, and meta world. I’ve heard a lot in the last 20 years or 25 years since I’ve been in business, since I went to school 27 years ago. I have to say it goes back to my comment on innovation and value. There's no question that AI is incredibly game changing in many ways. How much change can humanity support? How can we create some sort of UI and UX so that we can actually make use of that AI.
I see a whole lot of other examples of AI and it's like, so what? OK, great! But how do we make that into something that is going to be mainstream? Something that is going to be used heavily and massively. With some use cases, it’s clear, But I still have a ton of questions. And for me, what I do know for sure is that parts of the journey of the research process can greatly benefit from AI technologies, which reminds me of back in the day when we were breaking down the process of doing research in the lab.
We started with the design and the strategy of the research. Why should we do the research in the first place? Then designing the right script. Then finding participants. Then meeting them. Then build something that is connected to their computers. Then collecting the data. Then actually making sense of the data through analysis. Then presenting it. I mean, there's a ton of steps that you can break the process down into. And there's no question that with AI you can automate many of those stages and make it so that you reduce the time. And ultimately, just make it a little more embedded, which is what we always tried to do. We embed research in the product design and development process and make it scalable, so that it doesn’t have to be seen as a pause or break where you need to figure things out. People don't like that, you know.
If it's embedded and that's the way you do business, then it doesn’t feel like research. It just feels like the design process.
The X-Mentor: On that notion of making research embedded, I remember something called “Send-A-Smile” that Microsoft Office used to ‘embed’ in our Beta products to collect user feedback. For example, when Outlook shipped with a new feature called “Conversation View,” our Beta users could click on the Send-a-Smile button and share feedback whenever they encountered something worth commenting on. Usually, it was a complaint, which is how our Send-a-Smile button got the nickname “Send-a-Frown.”
[Laughs]
Still, it was a useful feature because it captured the context of the task the user was performing, and it gave us a clear picture of the problem that needed to be fixed. However, as Office Researchers we literally had to write documentation for Developers to put “hooks in the code” to be able to quantify and create a visualization of what was happening across the Office Suite of applications.
When you're talking about embedding research you’re talking about when you're designing the product as part of the engine that runs the platform, right?
Alfonso: That's right. What we did with UserZoom was automation of the research process, essentially. You know, we used the cloud and software tools to make the process more efficient, etc. But the ultimate decision was that we automate and make it scalable for you. And scalable means that you can do it faster, you can do it more often, and it doesn't cost you excessive time or money. We used software to automate a lot of stuff. We saw the results from a study in a matter of hours or days, versus weeks or months. I felt like it was magic, right? And I felt it was like AI (artificial intelligence), which wasn't really called artificial intelligence. But I remember having a conversation with our investors and our board, maybe in 2017, about the vision and where we could be going. Because as a business we were moving past the $30 million to $50 million mark in those years. We’re seeing growth and value and saying, hey, there’s a business here. And we were wondering whether we were going to be a $100 million business, or we were going to get bought for $100 million. You know two different things. And I remember thinking, Qualtrics was a great example for us to follow, right? They were executing so well on the strategy in the so-called Experience Management space now. I always thought, yeah, of course this could be a really big business.
But how we do it is by becoming an AI company eventually. And I remember the faces of our investors looking at me like, “OK, what the hell is this founder talking about?
None of us were aware that this “generative” type of AI was coming like it did this last year. But at that time, we knew that we could use software to automate and break down processes or even reduce human intervention in many of these research stages.
Let me give you an example. To me, one of the biggest barriers, and you alluded to it earlier with the Send-a-Smile button, right. One of the biggest barriers that you'll see in the UX Research or UX Design space when it comes to actually making research part of the process is that a lot of the people are coming in new. Especially the ones that are coming in new in the last few years, they don't know what type of research to do. What type of study to do, right? Normally you have to go through school to really get the necessary training. Depending on the goal and what it is that you're trying to learn, you need to apply a different research method. Whether it's quantitative or qualitative, or whether it's a contextual inquiry or whether it's a survey, or whether it's a usability test, right? There are many different methods to choose from, right. So, I thought that the biggest hurdle for us to scale even more was to get people to ‘push the right button’ in terms of what type of research study to do and what questions to ask when you're designing that study.
UX Research & AI
The X-Mentor: Yes, spot on, you nailed it! Generally, where an educated and experienced Researcher would start is by understanding what questions need to be answered for the business. Why else conduct research, right? By far, this is one of the biggest pushbacks that people had with remote unmoderated tests or even moderated tests. The promise was that you can get feedback rapidly. But what good is rapidly gathered feedback if it’s not answering the right questions? It’s useless, at best. And we see Product Managers, who are not formally trained in Research, use these tools to conduct “research” without the benefit of well-formed research questions.
As you correctly pointed out Alfonso, in recent years there’s been a glut of newcomers to UX/Research that lack an understanding of research methodologies and how to form good research questions. On my side, I’ve also observed newcomers to UX lacking the skillset that’s essential to steer stakeholders toward the right research questions to be answering for the business. So, it’s UX Researcher’s “business understanding” that’s inadequate and needs to be improved.
Then there’s the newcomers that have mastered the tools, but not the domain. This is where we must be mindful about the use of AI because it’s also a tool. I’ve literally seen cases where UXResearch have mastered their toolset (e.g., UserTesting, Respondent, User Interviews, Jira Boards or other agile tools, etc.) but have failed to form a good research question and/or select the appropriate research method for answering that question.
And it’s not just UX Research doing this because you’ll also see newcomer UXDesigners who have mastered their toolset (e.g., Figma, Photoshop, etc.) yet have failed to fully master the most impactful aspects of their Design discipline (e.g., Strategy, Scope, Structure, etc.), the stuff that creates value early in the design process. Let me just say it, Good Design is not about mastering tools. I have skills with many chef tools, but by no means does that make me a Master Chef. Trust me, you don’t want me cooking when you’re paying for an exceptional culinary experience!
The tools are where many think the Design process starts. But it’s not.
You first must THINK.
So, the question for a Researcher then becomes, what research method(s) will best answer the right research questions for the business? Today, when we’re talking about The Business Impact of Design, we’re talking about capturing a certain type of data that will give you key leading indicators of how well your experiences are performing. E.g., Emotional metrics with economic indicators tied to Customer Journeys.
First, THINK: The framework for research methods includes qualitative research, quantitative research, attitudinal research, and behavioral research. And inside each one of those quadrants is anywhere from 7 to 15 different research methodologies.
Then ANSWER: Which method or which combination of methods should I use to be able to get my research question answered in a way that will have value and impact for the business?
Alfonso: Absolutely. From the famous Christian Rohrer’s popular framework for understanding user research methods. Christian, of course, a good friend of mine and part of the Advisory Board at UserZoom for many years. I 100% agree with you.
This is why I brought it up because it’s a great opportunity for AI to help. Maybe by asking a few questions you can kind of lead the researcher to the right question and offer up the best methodology to answer that question. I'm saying maybe there's an application there for AI.
We at UserZoom chose purposely not to let people go out and run with this “Ferrari” [UserZoom] in the street to go grocery shopping, because you know what? You're gonna crash. I’m talking generally speaking, Greg. What we wanted to do was to say let's make sure you write the right research, and you get quality insights at the end of the day. Whether it's to write the right script, select the right participants, or analyze the data. We came in with a solutions package that included our research expertise, our expert, a workshop up front, a training session, all those things to make sure that they picked the right methodology. We always felt at UserZoom, that research needed a craft, it needed training and it needed to be done correctly. Bad data is worse than no data.
“Bad data is worse than no data.”
So, we really wanted to make sure that our users were prepared. And ultimately, running the right studies. But I think that it's something that is harder to scale, right, because that takes time and effort. So, that's one of the areas where I feel like AI, if it's really that intelligent, an AI should be able to say, “oh, hold on, hold on a second. This is not the right type of study. Or not the right question for this type of goal. Well, for this type of need that you have, ALERT!” AI could prompt you by saying, “You should consider this other option.” You know what I mean? Maybe that would be an amazing use case for AI-enabled Research.
The X-Mentor: What we’ve been learning from our AI Godfather(s) and their disciples is that today's architectures for Gen-AI are about to be completely replaced with new architectures. There are several good reasons for that change, but one interesting reason is because today’s LLM architectures cannot support reasoning or planning. So, there are key pieces that are missing that need to be in place so that we can get to that level of intelligence that I think you're talking about.
Alfonso: Yep, I'm very carefully saying this because maybe in a couple years, you know, somebody's going to come back and say you said that this wasn't big enough and look and laugh at me. But what I see right now, and maybe I'm saying this from a user experience perspective because I’ve been in this industry for so long, that I ultimately have faith in what's going to provide a better UX. Not something that is just popping up amazing animals by putting some stroke on the keyboard.
“If NextGen-AI is able to have that type of intelligence where it's able to tell you, “Whoa, whoa, that's not the right research approach. Try this one.” I would be incredibly surprised.”
For now, perhaps what it can do is help us build a report. In an easy way. It's kind of like what Google already does with some sort of Google Cloud service or even Microsoft right with the [Copilot] PowerPoints. I mean a lot of those things are amazing what you can do with software. And hopefully, AI will help us with that as well.
The X-Mentor: Yeah, I think you're right. Google’s Bard, as well as Microsoft’s Copilot and ChatGPT has the capability to produce that kind of output.
Yet, when it does get to the point where it can reason and plan, because NextGen-AI basically understands the contextual world view, it can give us not only a rational direction, but also the why behind going in one [right] direction, and not go in a different [wrong] direction.
Research for Business Impact
The X-Mentor: Alfonso, I'd like to pivot and talk about how we can convert research into something that produces results for the business and drives business outcomes.
What are some of the questions that you're expecting we should be better able to answer that will fundamentally drive business outcomes?
What are some of the questions that researchers should be asking?
Alfonso: I always look at the research and design process for digital products as a cycle, right? And so, if I can say it on record because everybody knows this, this is part of why UserTesting as a brand is a little limiting. OK, follow me on this one. I've said this openly, so we can discuss it here, OK. They all know. See for me, testing is only a part or a fraction of what a researcher does. To be a researcher, that's way more than just testing or user testing.
You asked me about what kind of questions we should be asking. The first one, I'll tell you, comes from the entrepreneur in me. We had this major Revelation two to three years in, where we had to really understand who our ICP was. Who is our end user? Who is our target? What do they need? What are their pain points? What are the gaps that they're finding in the market? What are their daily lives? How do they have to cope with their bosses and teams? How do they collaborate with others?
Testing comes way later. First, you need to really dig and do discovery research on who is that persona. I always thought that everything starts with that persona and then you got to just continue. But think about it. It sounds pretty trivial, right? It sounds pretty normal, especially to us in the experience research space. How many companies do that?
I'm helping a ton of startups right now in their business. Do you think that they really, and when I say really, I mean have done research, have they actually spent time talking to users, you know the CEO and founder, have they actually been doing that themselves? I can tell you not many, not many.
The X-Mentor: I know just what you're talking about here. And as we’ve heard from other guests here on The X-Mentor, “Executives have a completely twisted view of what their customer experience is.” Meaning, someone else is doing the work for the Executive. Or as you point out, the work is not being done at all. The question is, how many of them do it well, or with rigor?
Alfonso: It's even worse.
The X-Mentor: Some Researchers go and talk to their sales team to get a little bit of an understanding of what they're encountering in the field and that's good. I think that collaboration should be happening a lot more. But it's a piece of information. So, it's those researchers that can get that piece of information, then they can get some information from marketing. They can get some information from other types of research, such as ethnography and front-end research, you know, that helps you understand the problem space and the technological advancements in the market better. All of these things are pieces of information that are hard to coalesce into a more holistic understanding.
Alfonso: So, I will tell you this was my experience. It's really hard to get everyone to collaborate and focus on that one or two or three personas and really have answers to who they are, what they need across the process, across the business operations.
Do you think finance thinks about this? They don't think about this. Of course, they don't. They're thinking about numbers, right. OK, then you have engineers, do you think engineers really think about the end users? They're thinking about code, and they're thinking about, you know, how to be efficient with what they do. Do you think marketing really thinks about those? They do. They start with it, but then in their daily lives, they get lost in all these campaigns and messaging and branding and ads and things like that. And on and on and on it goes, right. Then there’s the C-Level Execs, buried in work, trying to lead their teams and execute their strategies. The board, the investors. How many times do you think investors sit on a board and start talking about who are the target personas we’re after? What do they need? And what are the solutions we're building for them? No. It's often all about business performance, financials, growth and EBITA, etc., right.
So, my experience is that I think the researchers, the people that are providing answers to many of the of the questions that you have across all the different departments of the company, but hopefully you bring those questions up to the board level or up to the C-Level so that it's actually discussed up there versus just down here in a tactical level. You know, “oh, they'll figure out how to do it in the design team.” No. Ultimately, I feel like that should be a culture of customer centricity, curiosity, and empathy in the company to answer persona-related questions.
To get questions constantly answered or constantly researched and answered by a team of people that provide insights. That’s how you impact business.
The X-Mentor: Just this morning, for example, Alfonso, I read a post by a researcher. I won't mention the companies that they have worked for, but to say the least, they have worked with some very formidable companies. And what they proclaimed is there's this engagement problem that we have between the researchers and the product managers. They claim these two groups rarely communicate or collaborate well enough together for the research to fully have impact and reach its fullest potential. And what I seem to be hearing from you sounds like you would be saying to this Researcher, you’re focusing in at the wrong level. Is that right?
Alfonso: No, not necessarily. You know, I think the researcher must do certain jobs. And you know, sometimes they're very tactical, sometimes highly strategic, right? It may just be that we need to test the specific UI or something, right. And maybe that's not going to get up to the to the Board level or whatever. That’s fine.
In order to get A Seat at The Table, as I read in one of your X-Interviews, I just believe that the researchers and especially the leaders of the research team who I believe needs to absolutely be having direct conversation, direct connection to C-Level. For instance, spend quality time understanding the connection between business and the insights that you're collecting, right.
“It's very difficult to link insights and research findings to actual business impact.”
I'm not suggesting that you can actually link and say, hey, if you do, you know more studies you're going to get better retention, right? It's very difficult to link insights and research findings to actual business impact. Like in SaaS, it’s retention. NPS from users. Growth, it's very hard. However, I do believe that you should have the conversation around the business context.
For instance, Netflix. I spoke to the Head of Design of Netflix not long ago, like less than a year ago. He informed me about the decision the company took back in 2022 (I’m not sure it’s still the same today), the strategic decision that had been made at Netflix of having an independent Insights Team. Have a dedicated user research and analytics team be absolutely independent, and not reporting to Design, Product, or Engineering. If I understood what he said correctly, they were reporting directly to a COO, but I’m not 100% certain, to be honest. What I do know is that the goal was to task a specific team with the very tough objective of providing answers to questions. Right. If you look at most companies out there, you'll have a research team under product, or under design. But hardly ever having the visibility up to the very top. And I believe that that's a strategic decision that you need to make as a CEO: this is a company that is going to be curious, constantly asking questions. It's going to have curiosity as part of the culture.
The X-Mentor: One of those things I just wanted to touch on before we get too far away from it. You mentioned a couple of things, Retention and NPS. And one of the problems that I've encountered, and I've reported in my 2023 Year in Review is this gap between the metrics that executives tend to focus on, which are lagging metrics, which don't capture any details about human experience.
Alfonso: That's right.
The X-Mentor: And then the leading metrics and of course I reported on Journey Economics and Experiential metrics and how emotions influence business outcomes.
I want to get your input on this because one of the things that I've seen as it relates to NPS, and in fact you may have read this book too, Winning On Purpose by Fred Reichel, who created NPS. Fred has a new metric called Earned Growth Rate. It’s about the NRR (Net Revenue Retention) you get after you’ve spent your cost on gaining new customers. But NRR and Earned Growth Rate is still a lagging metric that is not measuring human experiences that drive economic behaviors. Like NPS, it's still more of a proxy metric.
What are your thoughts on that notion of being able to get the C-Suite to pay more attention to the leading Experiential metrics which are proven to drive economic behaviors?
Alfonso: First, I completely agree. I did mention NPS earlier because I know that it's a popular metric in the C-Suite. But by no means am I happy or comfortable with what it does by itself. It's obviously very vague.
The way I will answer your question is by being honest, I have to sell what we created at UserZoom which is now part of UserTesting and has been adopted as part of the merger.
We came up with something called the QX score. Essentially, we saw NPS as asking a question after an experience, asking a [customer/user] to rank 1-to-10 is extremely vague, right? Extremely vague. It may have some meaning, but it is very vague. So, we said let's add Task Completion Rate. Let's ask a question after a task, a specific interaction, an experience [customer/user] has been had with a very specific moment or a specific task or journey.
For instance, booking a flight. What we try to do with the QX score, which by the way was developed and designed by our bunch of PHD's in the company, it's a combination of the “Behavioral” and the “Attitudinal” to go beyond the number, which is an NPS promoter score. The number in this case with QX score is a combination of multiple metrics of different types of questions. In this case it includes also attitudes or behavior with the task completion, which is the essence of usability testing, right?
So, you'll find the QX score takes NPS to another level where there's a little bit more of the “why” behind the way the score is what it is, and it gives you a little more color. To me, the QX score has a place and we offered it to companies to use on a quarterly basis, but it doesn't substitute the actual research you have to do specifically to go deeper into why or how things happen. I've always been a big fan of complementing or having what we call the triangulation of data and using multiple methods to come up with more complete answers.
There's no easy way out because NPS is easy, it's one number. It's also easy to say as an executive, “look at our NPS” or E-NPS as well with employees. It’s easy to digest, so to speak.
The usability of NPS is what makes it popular, but I think that it goes back to the VP or Chief Insights Officer, whoever is in charge of the insights, to educate the C-Level Executive on why it's important to have a full picture, a lot more information related to experience, to the quality of an experience, than just a simple NPS survey.
The X-Mentor: Right! Reflecting on The Four Frictions, C-Level Executives are human too. They are also subject to the same forces that promote the status quo just like all of us humans. We all resist change, especially being forced to change by others (e.g., being “Educated” by Experience experts). So, it’s not surprising that “Leadership people are looking for key metrics that they identify with…because they know they can rely on them.”
Becoming Business Savvy
The X-Mentor: What do Design & Research need to be thinking about in terms of becoming more business savvy in their practice?
Alfonso: Yeah. Because design is so important to business, especially the leaders in the Design process, I think the entire team in the design process, but certainly all members of the design team should be aware of the business strategy of the company. What is it that we're trying to do? Where are we going? What's the mission and vision? Like the strategy, right and the plan for the company? What are the most important metrics or KPIs of the company? If we are a SaaS business and we are at $5,000,000 growing at 100% and being funded by a reputable Silicon Valley VC, that's a very different business than $100 million business that's growing at 10% in a specific vertical right. The strategy, the mission, the goals, why we're doing what we're doing, why we're investing more in this than that.
I think those are all things that as an entrepreneur, I've noticed how it's important to be transparent. As a designer, I would be very curious. Again, like a researcher, I would be very curious and ask businesspeople, OK, why are we doing what we're doing?
A lot of times this happened to me at UserZoom and I loved it. I loved it when the team came to me and started asking me not just why I started the business, but also what are we trying to accomplish?
I loved it when people in the product design, research, and development came over and asked that, and I would just go on and on and on, you know, explaining the details. I think that's something that needs to happen more often.
However, they also need those things to happen the other way. Businesspeople need to understand that Design has a huge impact on the business, the quality of the design is going to have a huge impact. So, Businesspeople need to go in the other direction and teach and share and openly discuss business metrics and why we're doing what we're doing to those design and development teams. I think it goes both ways.
“Businesspeople need to understand that Design has a huge impact on the business.”
The X-Mentor: Do designers need to understand financials? Or the business model? Or planning and budgeting?
What are the boundaries or the sweet spot, so that we don't get in someone else's lane?
Alfonso: If you went to a restaurant or a bar and you had the executives sitting here and the designers sitting over there, would they be talking? Most likely not. Right? These people here are talking about golf or, you know, retention rates. And these guys over there are talking about, you know, the design patterns, things like that. It's only normal, Greg, it's only normal, right. But at some point, and I would say frequent, like you know All-Hands, you know constant AMAs (Ask Me Anything). You know that's where I feel like the C-Level Executives need to make it a culture of curiosity and exchange and educate each other. I think even if you don't get along or you wouldn't share time together at the bar. But the fact is that if they do, they will collaborate better. They will do things for the right reason.
I can't tell you how many times we had to reemphasize why retention was so important. And why we needed research to tell us to give us answers on net retention versus growth retention. We just repeatedly mentioned this in the development and the design. Not everybody got it or not everybody was listening. Or maybe we're not doing a good job explaining it, you know. But I think that is something that is key in a digital experience company.
The X-Mentor: In QBRs (Quarterly Business Reviews) you typically don't get into Experience Outcome conversations at QBR's right?
E.g., The why behind our numbers. Experience outcomes for products, specific interactions in a journey. Or something like that. We don’t talk about how our designs either eased friction or created friction. We’re not able to show causation. Perhaps maybe we can, but we don’t in a QBR. Right?
Alfonso: No. In general, transparency is not easy. It's harder to tell people sometimes bad news. It’s why I focus so much on culture, Greg.
“Because at the end of the day, it all starts with the CEO and the board and the management team and the culture they're trying to build.”
Because the end user has so much more power today, because they have an ability to do everything on their phone digitally, I would argue that a successful digital experience company needs to have a much better culture of collaboration.
You can't have a successful digital experience company if you don't have great collaboration and culture in the company. Because digital forces you to have a lot more collaboration and connection between the different teams. You can't rely on the retail store out there that has the salespeople. No, now everything is interconnected, and the powers have shifted. So, it feels to me like the C-Level Executives need to have a lot of emphasis on culture. And what I mean by culture again is collaboration and openness and transparency and all that stuff that is part of the culture to succeed.
The X-Mentor: This was fantastic, Alfonso! Thank you so much for being with us on the X-Mentor today. I’m sure our readers have enjoyed hearing from you and learning more about your journey at UserZoom.
Alfonso:
Absolutely. My pleasure, Greg!
ABOUT THE AUTHOR(S)
Alfonso de la Nuez is the Co-Founder and former CEO of UserZoom, which was acquired by Thoma Bravo and merged with UserTesting in 2023.
Greg Parrott is The X-Mentor and publisher of The X-Interviews.