Experimentation Essentials Series for Lifecycle Marketers (Part 2/5)
Skill #2: Experimenting with a clear intention. Knowing where to test but not knowing what learnings are you aiming for can lead you into experimentation chaos.
This is a short series of 5 essential skills that you need - to get better at experimentation and unlock the value it offers through lifecycle marketing (but also any other domain really). Here’s the article on Skill 1: Knowing where to experiment in case you haven’t checked it out.
Once you’ve identified “where to play” with your experimentation efforts. The next steps is to have clear intentions about the learnings you are looking to get out of your experiments.
Is it do anything to move the metric?
That’s not learning, that’s an attempt of increased activity.
We need a clear hypothesis. We need to align what our efforts are aimed towards - not metrics, but how do we beleive we can move the metric. What works, what doesn’t work?
I’ve been in many such projects over the years and unfortunately no one has the luxury to go through experimentation in such a step by step way. I acknowledge that. But try to go through these posts to try and refine your approach when you are dealing with this part of the process.
Some recent examples of projects I’ve helped clients with through experimentation:
Our app store reviews are a small % of our users. Can we leverage CRM to increase the volume, while keeping a high average rating?
Our Week 0 CRM initiatives haven’t been optimised for months now - can you help us with a fresh pair of eyes to improve this funnel?
Our CRM team currently can’t get out of BAU work - can you help us zoom out and develop an experimentation roadmap?
These are problems in which “where to play” is clear. We know what we are looking to optimise & improve. We typically know there’s room to improve but just are out of ideas.
Usually, we aren’t out of ideas - we lack intentions. We don’t have a clear hypothesis.
4- step process to get to a clear hypothesis & experiment ideas
Let’s imagine we are looking at improving our existing Week 0 comms setup.
These are the first set of questions we need to look into (either individually or as a team):
What is the problem we are trying to solve or the question you are trying to answer for your users? It’s important that this problem statement is based on an insight. Insight could be qualitative, quantitative or a mix. But it should be something we have identified or believe to be a problem.
For instance: We saw that 10% of the users who land on the subscription screen during the onboarding, stopped using the app. Could it be that these users perceive that the app is only usable after paying?