As a researcher by trade, I’ve worked on a variety of projects – from the intensely rigorous and strategic pieces with clear business value, to those that functioned a bit more like a tick-box exercise.
Contrary to what is generally believed, the primary difference between the two may not be the quality of the research firm or consultant involved. Whilst it is certainly true that there are mediocre or even poor researchers, there is a bigger problem that many businesses fail to acknowledge, and that is that they don’t know how to procure good research.
So let’s talk about how most research gets procured, and how small changes can improve the process.
Step One – you realize there is a problem or an opportunity, something you don’t know and need to, or something you need to evaluate and measure. So far, so good.
Step Two – you decide to hire an external firm or consultant because you believe a fresh set of eyes, or frankly, an extra set of hands, is necessary to get the job done.
You speak to a few colleagues to gather questions they might have, and you put together a rough pitch that covers a lot of ground but reflects – generally – the kinds of things you think are probably applicable to the business.
However, it’s not always clear how the questions you have identified are linked to core business processes, problems, or learning objectives, nor is it clear how the data would be used by key stakeholders. You are taking a scattershot approach when you should be looking at causal chains, attribution, and contribution, etc.
Step Three – you put together a budget request. Which finance immediately cuts in half. You don’t necessarily amend the terms of reference to reflect the available budget, but rather hope it’s still feasible.
Step Four– you put out a call for proposals, or you approach a few firms and ask for bids. You look for a firm who can cover all of the work you’ve asked them to, for the money you have available. You send them a contract that copy-pastes the terms of reference.
In just four steps you have virtually – and unintentionally – guaranteed that no matter how great your research firm or consultant is, they cannot give you a quality product.
Let me explain.
Everything was going fine in Step One, but in Step Two, when you realized there was a value in bringing someone else in, you looked to those most likely to share your general thoughts on the matter, rather than looking to your firm or consultant for their input.
You have thus limited the ability of an external firm or consultant to influence what you should be asking, and instead task them only with figuring out how to ask it. As a result, you miss out on the depth of expertise available to you.
If you are lucky, your colleagues may know something about research and they have the time and the interest in helping you to shape these questions. More likely than not, though, you end up with a list of research objectives and questions that isn’t tied strategically to your actual business needs.
As a result, the research results you get back will likely be somewhat vague, or you might even find they don’t tell you anything you didn’t already know. The data isn’t useful because its not targeted, and you research firm or consultant isn’t particularly well-placed to help you make heads or tails of how best to make sense of it.
In Step Three you diminished the value of your research by decreasing the budget but not the scope of work. Good research takes time and budgets need to reflect that time or you will signal to research firms and consultants that what you want is a general overview rather than a deep dive.
For instance, I generally sit with my client for the better half of a day, interviewing them about what they think they need. I ask hard questions, I poke the soft spots. I take the time to speak to other people in the group, the industry, or the target demographic, and review documents provided, all to determine the evaluability of the thing my client thinks they want to know.
I will then tell them what can and cannot be answered, with what degree of confidence – and not just statistical confidence – and how the answers I provide can actually be used.
If your budget only reflects the collection of data, you aren’t making best use of your researchers’ expertise, and you are going to get a report that reflects your ability to design research – not there.
If you aren’t a researcher by trade, or if you are too busy doing your actual job to put considerable time and thinking into research questions, you are doing yourself a disservice if you don’t request that the firm or researcher you select review the research questions and learning objectives in advance of any data collection. With a limited budget, however, very few if any quality firms or consultants will be able (or willing) to do this.
Now we come to Step Four, in which these earlier problems compound. If you pick a firm that shows they can provide you with answers to the questions you asked them to, or you pick a firm that looks like good ‘value for money’, you may be missing out. A good firm or consultant will show you they have methodical and thematic expertise, but they will tell you that they want to get to know you and your business before they commit to any hard or fast approach. They want to fix your research before they do your research.
To do that, however, they need time and money. You should be looking for a firm or consultant that tells you what good research costs. They will probably go above your budget – this is not always a bad sign. Indeed, it can be a very good sign because it means they value getting your research right.
They know that research done well takes time and money, but they also know it can and does have a critically important impact on your business. They will want to reconfigure the Terms of Reference so that they are feasible, and so they know they can deliver something of quality, before they ever sign a contract. They aren’t being difficult – they want to get it right.
Good research is as much about the design as it is about the collection and analysis of data. If I’m honest with you, the second half of the equation is the cheap and easy part. So if you are consistently getting less than stellar research from your service providers, its time to take a long hard look at the design processes that got you there.
Melyn McKay is a socio-cultural anthropologist with the advisory firm, Monticello LLP.