REVISITING ROI

Several months ago, after reading an article about an effort to assess the ROI of research and determine how to value it, my partner and I had similar reactions of disappointment because it seemed that industry leadership was continuing to miss an important point about ROI, and was simply rehashing ground covered very thoroughly 20 years ago.

At that time, I wrote an article in which I stated, somewhat provocatively, that focusing on ROI is a canard (an attractive, alluring waste of time). Instead, I suggested that insights professionals (both Client and Supplier side) must narrow their focus to delivering value via “impact” – on every project. In other words, if we aim to be consultants to our clients and deliver a coherent POV, rather than limiting our role to being expert information providers who deliver insights, ROI will become a non-issue.

Since that article, I have had the opportunity to present my ideas more fully about the futility and misguided focus of trying to measure research ROI. In a nutshell, I have argued that:

– insights are subjective – the impact of insight(s) CANNOT be meaningfully measured quantitatively;

– contrary to some assertions, such measurement is NOT required by management; management is highly interested in “impact” but does not require precise ROI;

– the effort to measure ROI is not just futile, but wrong, to such an extent that the effort will backfire; the effort is itself a demonstration that you do not understand how to make an impact;

– impact must be evaluated by the receiver of insights, rather than the producer; and

– because impact is “obvious” to the observer, any evaluation process must be kept very simple.

I have continued to advance the premise that all research projects, to be successful, must be designed to provide a POV on at least one specific, focused business issue. And that to do this, one must understand the business CONTEXT behind the questions being asked, because even doing a great job of answering questions is no guarantee of being perceived as making an impact.  

The Conversations Continue

At recent industry conferences I have seen several examples of presentations on similar topics, and I’ve had discussions with organization leaders about the challenges of changing the ROI focus. For example:

– A presentation on the value of “storytelling” boiled down to the idea that the best stories were ones that presented a POV on a topic. I wanted to tell the presenter that he was definitely correct, however, he was missing the point that this isn’t just the ‘best’ approach but has to be the primary focus. He implied, as many still believe, that by simply organizing information well, you can get to a compelling story, without realizing that it is the existence of the POV itself that is the necessary organizing force/ingredient that leads to effective structure and stories. Great stories do not “emerge” – they are the result of clear focus.

– A panel discussion about how to ‘get to the table’ and gain more recognition for value-add in an organization touched on the idea of focusing on impact, with many good suggestions about thoroughness of communication, customization of deliverables for different audiences, as well as the need for clarity and buy-in before beginning projects. All absolutely correct, but again, missing the point that one must first have clarity on the desired impact and reason for this study or analysis being conducted in order to achieve these goals and buy-in.

– Two separate conversations with leaders in Fortune 100 organizations who indicated that they were trying to implement in their organizations the approaches I’m advocating. One company had made “impact” the focus of success and was in the process of changing its evaluation systems to reflect this focus. The other leader indicated that she was doing the same but was dealing with internal pushback on how to define responsibility. It is no small shift to change from measuring people on project management and quality of insights delivered (which they can fully control) to impact, which is harder.

– Discussions with one of our leading partners, who I would say is the furthest along at taking a Consultative approach to the Insights function, led to agreement that part of the challenge to implementation is the difficulty of gaining agreement on even the meaning of some simple terms; that concepts such as “objectives” and “consulting” don’t have commonly accepted definitions. In research we all too easily conflate clarity of purpose with clarity of objectives or clarity of questions to be asked. And, due to shortages of time, it is common even for those who are trained and come from deep experiences in Consulting to undervalue the need to get clarity up front rather than just trusting in their ability to ‘sort it out’ at the end.

Revisiting My POV

The missing piece, I think, continues to be less about the agreement on a need to make an impact, and more about the definition of success. Our industry does not yet accept that what we are currently doing and delivering is NOT satisfying our role; there remains a strong undercurrent of belief that “…if only management understood…”.  ROI is attractive because it seems to offer the clarity of measurement that is the heart of what we are good at.

A good friend suggested that this is because there are some key barriers to achieving a consultative role and gaining understanding of business context.

– Our clients don’t always know the answers: It is very possible (perhaps even common) that the person who comes to ask a question may not, themselves, fully understand the context behind the question, but are just told to “see if you can find out ___.”

– We CAN often “get away” with less: For many tactical studies and issues, the answer to the question is already close to a POV – e.g. Which of these two messages is best? Which of these flavors is next best to add to a product line?

For tactical issues, the incremental value of understanding business context and what decisions are being made is still there, but it is incrementally less than for a more strategic study.

But when the goal is to get to the table, we have to remember that increasing our perceived impact is about changing how we are perceived. Thus, the questions we ask about business context engage our clients in a more productive discussion and reposition us as partners rather than information gatherers.  Every time we fail to take this step and seek clarity of purpose, even on a simple, tactical project, we degrade the perception of us in the organization.

This raises additional barriers:

– Being effectively assertive – something called “Red team” thinking – is not easy: Challenging someone’s questions, surfacing assumptions, and being a good devil’s advocate is difficult. How and when to do this requires experience; especially the first time you work with someone.

– Researchers are not marketers; zebras are not horses: There are cultural biases that limit researchers – many of us went into research because we really like working with data, finding answers, and providing insights.  That makes us different animals from most of the marketers we work with, and certainly from C-suite types and sales teams that will make the decisions informed by the insights we provide.

There IS a secret sauce to consulting effectively on the way to delivering business impact, and despite the barriers, it can be taught / learned. But consulting is not as easy as telling a story or answering questions. In my experience, getting clarity on the purpose of a project up front requires many people to get out of their comfort zone, so if we expect people to try to do it, we have to make it important; It has to be at the core of how we evaluate ourselves.

With the right training and mindset, delivering business impact is easy. It requires thoughtful partnering with clients, helping them ask the right questions to get the right focus. And then leveraging that focus to choose the right methodology and ask the right questions of the right target, leading to a well-thought-out, concise POV that directly addresses the business issue at hand. Insights designed, and results delivered in this manner provide clearly perceived value and inspires client confidence.