Are you using marketing case studies wrong?

Are You Using Marketing Case Studies Wrong?

I love research, and while I can all too easily get on my academic soapbox, I think there is a lot of value in research done by average Joes, Joesephines, and businesses.

I acknowledge that not everyone has the resources or particular “types” of intelligence to complete a higher education degree – and quite frankly not everyone wants to. There’s nothing wrong with learning mostly on your own, even online.

However, as a former psychology student and current master’s candidate for a research degree, I see not just flawed research performed by over-eager marketers, but a whole lot of flawed interpretation and discussion of research.

Case studies are a great tool of exploration, understanding, and example, but they are often weighed far too heavily – enabling poor business decisions.

Here are 3 ways you should avoid using case studies, and a couple of ways you should use case studies.

3. DON’T use a case study to prove your point

Don't use case studies as proof

You can conduct the most in-depth, elaborate study with thousands of participants and tens of thousands of data points, and you still will never prove your point.

You can use data and insights from tens or hundreds or thousands of research studies, and you still will not prove your point.

When you are talking about research, eliminate all iterations of the word “proof” from your vocabulary.

Research provides evidence, but never proves anything with 100% certainty. No matter how much math you use or how fancy it is or how fancy your charts and graphs look, it is impossible to “prove” something. And if you declare in any research paper or blog post reviewing research that something is “proven,” us academics will barf a little in our mouths and silently close the browser tab.

Like any study type, case studies can’t prove anything – but it’s not just statistical significance that inhibits this.

In other types of studies (surveys, observational, experimental, etc.), there are typically at least a couple dozen participants – if not hundreds. With significant findings from an appropriately sized, randomized, and representative sample, we can make an informed assumption that the results would be similar if testing methods were applied to others in the target population (even if the “population” is referring to businesses and not individual people).

Case studies differ in that their sample size is 1 which can hardly be representative of a population, creating an issue of low external validity.

If you’re appealing to your boss for support using a specific tactic, or researching for your next in-depth blog post, be sure to avoid “proving” your point with a case study.

2. DON’T use a case study to create your marketing strategy

Don't copy tactics from case studies

Don’t base your entire marketing strategy (or even a substantial part of it) on a case study.

Even if the case study focuses on a company in your industry or a company with a similar audience, it does not mean that you will have the exact same outcome.

It’s easy to get caught up in dreamland when you come across a super cool case study using practices that promise to explode your web traffic by 500% in 14 days, but there are so many variables involved that can’t be replicated.

For example, even if your website is in the exact same niche as the subject of the case study, there may be dozens of differences including:

  • Age of the domain
  • Size and authority of the website
  • Number and quality of backlinks to the site prior to implementing the new practices
  • Perception of the brand
  • Social presence of the brand
  • Current competition in the SERPs
  • Team members working on the project
  • Existing relationships with other companies, site owners, and editors

Case studies are often used as selling tools for expensive software and services, so if you are considering investing hundreds or thousands of dollars don’t let a case study be the determining factor in the decision.

Consider tactics (or software) used in case studies that have yielded awesome results; but only cautiously after considering both the unique circumstances of the highlighted subject and your own unique situation.

1. DON’T use case studies to generalize

Ultimately all of these “don’ts” come down to avoiding generalization when you discuss, use, or apply case studies.

Don't generalize with case studies

If in a case study a company exploded their traffic by 500% in 14 days by using Pinterest, does that make it safe to say that Pinterest is a good channel for your business?

No.

Does it make it safe to say that Pinterest is a good channel for businesses?

No.

Generalizing results of a case study is a quick way to make yourself look like a fool.

Again, no two businesses or websites are exactly the same. Because case studies focus on a single participant, it is impossible to know how others will react under similar conditions.

Most research is based on a sample that is the closest representation of the general population as possible (or a specific subset of a population).

If you are sending out a survey studying “millennials in the U.S.”, you would strive to send it to a group that has similar representation of age, gender, race, and income as the total population of millennials in the U.S.

We often see the opposite of this in the selection of cases for case studies. Hypothetically a “typical case” could be selected for a case study, but it’s uncommon to find them inspiring – because the results are likely to be typical.

Case studies are chosen because they are extreme or deviant

More commonly we see the use of “extreme” or “deviant” cases, which are selected because the results are not typical.

These cases are selected specifically because they are unique, so it is unlikely that the same methods would yield the same results for a different subject.

Instead of generalizing and “proving” your point with them or applying the concepts learned directly to your own strategy, here are a couple of ways you should be using them:

How to Use Marketing Case Studies for Your Business

DO use case studies to inspire discussion

If you read a case study that seems relevant to your business, one of the most valuable things you can do is use it to inspire discussion.

Bring it up in your next meeting and see what input your coworkers have.

Create a discussion thread online to see what insights others gained from it.

Discuss case studies

Discussing the case study as it may relate to your own business will expose you to different understandings, perspectives, and explanations for the results, and help you better understand where your company is similar or different.

DO use case studies to inspire further research and investigation

Eisenhardt describes case studies as ““useful in early stages of research on a topic or when a fresh perspective is needed”.

Whether you find a case study interesting because it reaffirms your beliefs or contradicts them, they provide a great opportunity to inspire further research. Ideas of further research may include:

  • Seeking out related articles and studies online
  • Surveying your own audience
  • Running a controlled test using one of the tactics or methods used in the study

It’s all too easy to get caught up in dreamland when you read an awesome case study. Keep in mind the subject was likely selected because it was an extreme example, not a typical case. The tactics may very well apply to your business, but it’s vital to slow down and think about context before making a big business decision.

Comments

comments