Retention: The Foundation Of All Your Metrics

John Koetsier: Are benchmarks bogus? Is measurement meaningless? Hello and welcome to Retention Masterclass. My name is John Koetsier.

Peggy Anne Salz: And I’m Peggy Anne Salz, and of course, one of your co-hosts for today’s show. Brian Balfour, CEO of Reforge

John Koetsier: Most marketers look for benchmarks and they treat measurement like gold. And it’s understandable, right? We want to know how we’re doing relative to others.

Peggy Anne Salz: Yeah, but you know, when you think about it, that’s very true, that has huge challenges, but a benchmark at the end of the day is only an average, John. And really, who wants to be average, right? That’s what I’m hearing out there and that’s why I’m so excited about this show, because that’s what I’m hearing, benchmarks are bogus. But no matter how you cut it, whatever you measure, is your activation, your engagement, it’s going to be different depending on your app category and your company, depending on where you compare it to someone else. So, a lot of reasons to deep dive into what retention benchmarks really mean, and that’s what we’re going to do today. So here we are.

John Koetsier: Yeah, absolutely. And we’re super pumped to bring you an all-star today. He was the VP of Growth at Hubspot. And of course, everybody knows about HubSpot which is absolutely massive and the growth curve there has been incredible, I think they now have almost 80,000 customers, which is just unreal. He’s been a founder multiple places, an entrepreneur in residence, and Peggy, you’ve got more about him.

Peggy Anne Salz: Well yeah, he’s the founder and CEO of Reforge, which is like the place, right? It’s like pick up a pew, join in and get your course there, because it’s probably the best place on earth for marketers to learn about marketing. So welcome to Retention Masterclass, Brian Balfour! Great to have you here today.

Brian Balfour: Yeah, thanks for having me. I appreciate all the kind words.

John Koetsier: Oh, it’s a big pleasure. And of course we had somebody from MasterClass.com last time, now we’ve got Reforge. It’s amazing. The best of the best! It’s wonderful. You’ve said, and recently said actually, that retention determines category leaders. Can you talk about that a little bit?

Brian Balfour: Yeah. So this was part of the research we did as part of a retention engagement course for Reforge and trying to establish why we kind of intuitively knew why it was so important. But if you actually look at any category among SaaS companies, B2C social, anything, you’ll always find that the companies and the products that are the category leaders in their space are always the ones with the highest retention.

And so the real question is why, right? Like, is that just a random coincidence, or what it is. But it isn’t random, right? Like when you actually kind of drill down and understand retention engagement kind of at its foundational level and what it does, what you find is what we call it, kind of the center of the growth engine. So if you think about growth, it’s a system of really three parts: acquisition, retention, and monetization. And those three don’t work in silos, they work all together as one system. So if you change one part of the system the other part has effects on the other part of the system, and so you need to kind of understand all of these relationships.

Well, it just so happens that retention is the thing that kind of moves everything else, right? So retention moves acquisition. So as you increase retention, it typically flows through to your various acquisition mechanisms. If your primary acquisition mechanism is virality, then increased retention is going to increase your kind of viral users that you’re getting in two different ways. You’re going to have a larger percentage of a cohort of users that are inviting other users, but the longer that they retain they will also have more invitation touchpoints.

So you can think about this in Dropbox where like the longer I use Dropbox, right, the more people I probably invite to Dropbox because the more files I share and all that kind of stuff. But this works for any type of acquisition mechanism, whether it’s virality, like just paid acquisition, sales, content, anything, right? But the second thing is that you typically, as you increase retention also increase kind of the monetization part of your ecosystem as well, right? Like the longer that I retain a user, the more money that I’m going to make from them.

And also the larger percentage of a cohort of users that I retain, the more money I’m going to get out of that cohort. And so that also, like the more money I make out of a cohort flows back through to acquisition because that means I’m extending my LTV. I have more to spend on CAC. The more I have to spend on CAC, the more I can acquire, the more that I acquire, the more that I retain, right? Like you can see how this all starts to work together, but it does really start at the foundation of retention. If you don’t have that thing, you don’t necessarily have essentially the pieces of your system connected in a way that are working together in a healthy way.

John Koetsier: It’s kind of funny actually, just to ping in on that for half a moment. I’m pretty sure there’s some mobile games, hyper-casual games, or maybe a little not so hyper-casual, but pretty casual, that I’ve been in for a long time and have not purchased, that are like doing their best to kick me out. Hahaha, it almost feels that way sometimes. It’s the opposite of retention because, you know, this guy is not spending any money here, get rid of him.

Peggy Anne Salz: Probably short-sighted though, you would imagine.

Brian Balfour: Yeah, it can happen. I mean, look, I think any type of, any healthy product or marketing team has a clear understanding of exactly who they’re targeting, and not just who they’re targeting today, but who you’re trying to target next. This is a concept that we talk a lot about in Reforge with one of our current EIRs, Bengaly Kaba, he’s the former head of growth at Instagram and Instacart, right.

But you have to have a very clear definition of who you’re targeting now and who you’re targeting next, and those are the people that your teams should be working on. And so like in that case, look, they might just not be talking to you and that’s okay, right? That is a fundamentally okay thing.

And we went through this, you know, in the lore of HubSpot, the history of HubSpot that they tell you as you come in as a new employee, they talk about a few different pivotal points in the company. And one of the pivotal points in the company was getting extreme clarity in like a few years on exactly who they were targeting and who they weren’t targeting. And they ended up going through a big phase where they fired essentially a bunch of customers and, you know, ignored them. And so look, maybe you’re on the firing side, who knows.

John Koetsier: It’s possible.

Peggy Anne Salz: That is fascinating. I am fascinated, I had no idea. But when you think about it, it makes such sense that retention is at the core of what determines a category leader. You know, it’s at the anatomy of high performance in high retention. I think that’s a fascinating thought that I’d like to pull in another thought that you’ve put out there many times, Brian, and just want to understand it.

You know, it’s one thing to say that this is important to look at as being part of that anatomy, as I said, but the benchmarks, the benchmarks, who wants to be average? Benchmarks are useless! That could be something you could put on a tee shirt. What’s the reasoning behind that? What’s the thinking behind that? You’re disrupting, Brian, what does it mean?

Brian Balfour: I have so many frustrations with benchmarks. I think we could probably just do a like anti-benchmark podcast episode if we really wanted to. So, but I will attempt to constrain myself and talk about like the highlights. And so, I think to start off, you have to really kind of understand what is the question that the person or the team is trying to answer with the benchmark, and start there. But in most of my experience the question that they’re trying to answer is some form of like, are we doing a bad, good, or a great job?

And the problem is that question, and the answer to that question, depends on some very like first principled thinking about who your product is, what your growth model is, what your audience is, what your monetization model is, what your category is. Just like all of these different things. And the problem is that the way that benchmarks are typically collected and presented get rid of all of those nuances, right? They don’t actually collect those nuances in the same way, because it’s really hard, right? Like when it’s really hard to capture all of those nuances and categorize it in the best way possible, and so what you end up with is when you look at benchmarks, what you’re seeing is essentially — I don’t know what the right analogy here is, it’s like a giant game of telephone or watered down metrics without all of these nuances.

And as a result, it ends up actually not really answering your question. Or even worse, you think you’ve answered the question, but you actually haven’t, and so that is a bad position for a team.

John Koetsier: Yes.

Brian Balfour: A really bad position for a team to be operating from. There’s all sorts of other problems, right? Most benchmarks are based on averages, they don’t clean out — some of them clean out outliers, but the whole point of being a venture backed company is to be an outlier, and I’m like, so why are you cleaning out the outliers, right? Like that is what you should actually be benchmarking against. And so there’s all of these frustrations with it, but at the end of the day, nine times out of ten the benchmark doesn’t actually help you understand the question in a very healthy way.

And so the best things to possibly do in my recommendation, is rather than relying on those numbers, is try to find a small group of other professionals where you share a lot of attributes or adjacencies. Like I don’t know, maybe you’re both Freemium SaaS companies and you’re relatively priced in the same way, but you’re targeting different audiences. Something like that, right? You have to have like some percentage of an overlap and you can kind of share numbers and kind of what you’re seeing.

Find a small group of people that you can do that with and you’re probably going to gather much more relevant information from that.

Obviously the perfect scenario is to know all of the numbers of your competitors and who’s doing best, right? And also to have all of those numbers being measured in exactly the same way. Sorry, that’s the other thing I skipped, and I think we’ll talk about, is that all of these metrics, all of these numbers in every single company that I’ve looked at, tend to be measured and defined in fundamentally different ways. I’ve named the same thing, but the stuff that’s going on underneath the surface of how they are getting to those numbers is fundamentally different. So anyways, all of these components the factor is just like, you know, in Reforge we get these questions all the time. It’s like a battle that I feel like I’m constantly fighting. And what I would really encourage people to think from is truly like first principles of what we encourage is like, if you’re trying to answer that question: are my numbers bad, good or great?

Especially in retention, what I’d like to do is say, is just run through a thought exercise and say, okay, who’s your target market? What if you acquired 100% of your target market today and you multiplied that by your long term retention rates now, times whatever your monetization model. What would that get you in terms of yearly revenue business? And a lot of times when people do that math, it doesn’t add up to be a huge business. And so right there and then, you know that you have bad retention numbers, and then you can start to play with those numbers to really understand. And there’s an equation here of like how much of a percentage of your target market can you realistically capture over time and all that kind of stuff. Kind of more bottoms up from that perspective is typically a much better approach than kind of just like looking at these numbers out there.

Sorry, last thing on this, and then I will cut myself off. There are some exceptions to this, there are some people who gather some pretty good numbers and provide some of this context. And there’s a SaaS survey done by David Skok from Matrix and Pacific Crest. I’m getting emails about it right now. I know they’re collecting this year’s data, so that’s interesting.

Peggy Anne Salz: Mm-hmm.

Brian Balfour: ProfitWell does a pretty decent job on revenue retention, metric numbers. They have so much data that they’re able to segment it for you and give you some of that context and nuance.

So there are some exceptions out there, but in most cases than not, I get pretty frustrated with the conversations around benchmarks.

Okay, I’m done. I’m done, I’ll stop myself.

Peggy Anne Salz: I just want to ping in on one thing. I think it’s amazing, John, we have the goodie bag here right now, right? We have not only that benchmarks are useless and why, but we have a couple data providers that we can actually look at, you know, and read and consider in the future. I mean, that’s a plus right there, you could almost end this show right there.

But I did want to think about one other thing though, when you were talking — I mean, I would just be curious because I was sitting here, it’s the first time I’m hearing this, Brian, right? And I’m almost like ready to be hyperventilating when you say these numbers are useless and how we look at them, and outliers and all the rest. I’m just curious what the response is from people.

Do you find that people are looking at you and saying, ‘Yes, I’m following Brian’ or are they saying. ‘Oh, my goodness. I have to go back because now I need to go in a quiet, dark place and rethink my entire model,’ because it is a shocker.

Brian Balfour: Well, I mean, essentially both are happening, right?

Peggy Anne Salz: Okay.

Brian Balfour: We actually do in Reforge get quite a few people who’ve come through it, and it’s kind of like opening their eyes essentially to what “great” actually looks like. And so, yeah, that’s a really hard view for you to get when you’re kind of heads down working like all in one basket. And so, we do get that.

We do get people who are like, ‘Great, let me start thinking about this from first principles and foundational perspective,’ but it’s hard to get away from the benchmarks for a couple of reasons. One is that unfortunately, executives still demand it.

John Koetsier: They do.

Brian Balfour: And it’s so frustrating, it’s like, a lot of times they’re looking for that external credibility to tell them what “good” and “great” kind of looks like, versus kind of helping go through a lot of that first principles thinking. Also professionals are just incredibly, like their time is incredibly constrained and demanded, and look, thinking through this from a foundational perspective is more time consuming. It is harder.

I understand the friction there, but that friction is worth it when you consider that the potential costs and downsides of drawing the wrong conclusion from benchmarks could actually lead you in a bad direction, and you don’t even know you’re going in it.
You’re kind of like looking at fool’s gold, right? You don’t even know you’re going in a bad direction until everything hits a brick wall. And so, that, I think is like, that is kind of like the downside of it.

So we kind of see a little bit of all of it and I don’t, if your excuse is, ‘I don’t want to put in the hard work,’ that is a very bad excuse. If your excuse is like, ‘Hey, I’ve done this anyways and my executives are still demanding it,’ then it’s just a hard thing to fight with. I wish we could educate everybody on how to fundamentally think and influence and change everybody’s mind. We’re working on it, but that I think will take a while.

John Koetsier: It’s funny, and I’m going to ask both of you for help here, because I think we need a new term, because as you were originally talking about this and you were talking about the danger of thinking you know, right?
I mean, I was thinking about known knowns, right? And you’ve got known unknowns. You’ve also got unknown unknowns, right, which are business risks that are probably significant.

But then you’ve got these things that you think you know, you think are known knowns, but are actually unknown. And those might be the most dangerous of all.

Brian Balfour: Definitely the most dangerous, 100% the most dangerous.

John Koetsier: Yeah, yeah. In terms of measurement, you’ve delved into it a little bit, but what have you seen in terms of differences in measurement, whether it’s CAC, whether it’s LTV, whether it’s different things.

Are you mostly talking about across different verticals they measure differently, or even within the same vertical you have vastly different ways of measuring the standard kind of categories?

Brian Balfour: Oh, it’s definitely way different even in the same category. So there was this blog post at one point, I have struggled to find it since, I really hope I can find it — but it was an awesome blog post done by some PE firm a number of years ago, looking at all the public SaaS companies at the time and their equation for LTV, and every single company there was something different in their equation, across 20 companies.

And I was like, oh boy. And so, but on the retention side, while we’re on this topic, right, let’s just walk through a branch here, like somebody will say, ‘This is what our retention is’ and they’ll spit out some percentage number. And I’m like, okay, well first, is that user retention or is that dollar retention, right? So that’s the first branch of the tree. Now we can go down the user retention, and I’m like, well, are you talking about a user, a team, or a company? Because in a SaaS vertical those three things are very different. And then I’m like, are you talking about retention on the product, the feature, a use case, right? So all of a sudden, like, I don’t even know what the number of permutations is that I just kind of described right there, but it’s all different. And now we’re not even getting to the permutations of like, okay, once you define all of those things, then there’s all sorts of different permutations on the time frequency, the action that is indicating retention, as well as a number of other factors.

So there you have to be very specific in what you’re defining. And we also didn’t even talk about time period. When people say ‘I have like 80% user retention on the product and the action that indicates this is XYZ,’ that’s still not enough. I’m like, okay, well, is that 80% over 6 months, 12 months, 18 months, right? And so a lot of these things have to be contextualized with basically the time period. So they’re all different. You look at every single company and to some extent they should be different. Because once again, what a retention metric should fundamentally at its core indicate that you built a habit with a user around them using your product to solve a problem. That’s what it should indicate, and so your product, your problem, your solution, right? Your audience, like all those things are different and you can also slice and dice it in a million different ways.

And as a result, it’s just like a retention number from one company does not equal a retention number at another company.

Peggy Anne Salz: So it’s very much an output. It’s very much what I produce because I’ve done something that keeps my user coming back, keeps my user loyal, keeps my user, you know, we won’t even go down to is it the feature or the product? We’ll just say that they’re loyal. Okay, Brian, just to keep it there, but I’m just curious as well about how I engineer that output.

Is there some way that I can be doing this, some best practice to share, because it is something within my control. I do have some levers to push and pull, right, Brian. So there is a level of it is the numbers, very, very much, but I do have some control over that. What is that, that I can do? What can I engineer?

Brian Balfour: Yeah. So this is a fundamental message that we say at Reforge, retention is fundamentally an output. The three core inputs into retention are: activation, engagement and resurrection, sometimes called a reactivation. So I can improve usage retention, which flows through to improving revenue retention in most cases by focusing on one of those inputs. Now those inputs break down into a bunch of other inputs specific to each of their categories. And so knowing all of these inputs and exactly kind of which area you’re focused on, is incredibly important.

But typically over time as a company grows, you basically need teams focused on each one of these areas constantly, because your audience is always changing. Therefore, like the product experiences that produce these inputs fundamentally need to be constantly changing as well. Those are the three primary inputs, but there’s a whole other input which is essentially what we call at Reforge “layering on use cases.” And so retention is fundamentally rooted in the use case of your product. What is the problem that your product solves and who is it for? What are their alternatives? Why do they choose you over the alternative? Like, what is the natural frequency of that problem? That’s kind of what makes it the core of a use case.

And you can break all these products down into their use cases, and those use cases determine what good retention looks like. And so what you’ll see is most products, what they start off with is they start off by targeting one use case, and you’ve got to nail activation engagement and reactivation for that single use case, and that basically determines a certain threshold of retention among your users. But what smart companies do over time is that they layer on use cases over time to basically build up the frequency of potential use for their product. So in the marketplace environment, this typically looks like layering on categories, right?

So if you think about Thumbtack, right, as they go into a market, they might start with I don’t know, roofers, or — sorry, Thumbtack is a local service marketplace here in the U.S. for your international uses. I actually don’t know if they’re international or not, but I might start off with a roofer. Well, I only need a roofer once every, hopefully, like multiple years or something like that, and so as a result, if I only have roofers as a category, right, I’m only going to use Thumbtack once every few years. But you see a company like Thumbtack, they layer on a bunch of other use cases and specifically how they sequence those use cases to more frequent use cases. Like, let’s say home cleaning, which is something I might need monthly or every other month, or lawn mowing or something like during the summer, which I also need, right. Like all of a sudden now all of those use cases essentially add up to a much more frequent use of Thumbtack overall.

I think you’ve also kind of seen this brilliantly in a couple other spaces. So, I think one is the meditation app Calm, led by an amazing leader there, her name is Dun Wang. Their team has executed incredibly well. Not only have they nailed the improvements around activation engagement and reactivation for their core use case of meditation, but they were one of the first people to layer on the use case of helping you sleep.

Now you think about that, right?

And so, and those two use cases are fundamentally different. On meditation, I typically as a user am building a habit. That is a very hard thing for a product to do. Sleep, I already have a habit ingrained. I sleep every night. I have to sleep, right? Not only that, for those who have trouble sleeping like myself, the problem is real, right? Like the problem is, the pain point is very real, and so they’re layering on a use case where they’re not building a habit, they’re tapping into a natural habit, and that habit is pretty frequent, it’s daily. And so combining those two together, those adjacencies together, ends up actually increasing retention as a whole. So …

John Koetsier: That’s genius! That’s absolute genius.

Peggy Anne Salz: Yeah.

John Koetsier: I have a Muse, and so, you know, the Muse is supposed to help you meditate and stuff like that. And honestly, I might pull it out of the box every three months or something like that, because meditation doesn’t ring, right? There’s nothing that pings and says, ‘Hey meditate now.’ There’s no burning platform for me to get there most of the time, right.

But yeah, for sleep, I gotta do that every night, just like you said, and there’s a reason for me to want to “sleep successfully” or have a good sleep. So that’s a nice habit builder.

Brian Balfour: Mm-hmm, yeah, exactly. And so these are some of the nuances around retention, right? Like we just talked about a couple of them, which is around like, are you tapping into an existing habit versus building on, trying to build a new one. But there’s some other aspects of this, right. If we would just want to carry this through as an example, the reward I get from meditating is not as visceral as the reward from a great night’s sleep. Like if I have a bad night’s sleep, I hate to say it, but I am not a nice person, okay … I just, I am a miserable person. And so the reward there is just so strong and so visceral. Versus meditation, it’s like, I know I feel good afterwards, right.

And I know that feel-good will add up over doing many meditations over a very long period of time. I rationally know that, but it’s still such a hard thing for me to ingrain, right. And so, anyways, like even tying this back to the metrics, this is why your metrics always start with understanding these aspects of your use case in your strategy. Metrics without understanding those two things are useless, and even worse, as you were talking about, can point you in the wrong directions.

John Koetsier: And that’s interesting, right? Because that’s something that Peggy and I were talking about earlier, I mean, can you engineer in retention or is it part of the core product use?

And you’ve kind of said in some way it’s kind of both, if I understand. Because you know, obviously there was an additional feature added to Calm, right, which is engineering but has become a core part of the product. And they’re related as well, correct?

Brian Balfour: Yes, they’re definitely related. And it’s definitely both. There are ways to engineer it., there are aspects that you can engineer and there are aspects that come naturally with the use case of your product. We say this across all of Reforge, even like if you look at your growth model in its entirety, right?

Every growth model has some natural strengths and some natural weaknesses. And in other words, there are things that come very easy to the product, and there are things that the team ends up working on over and over and over again. So, you know, if I think about, let’s think about some examples here to kind of accentuate this, but, I mean, you just take a social product, like Facebook. Acquisition — super easy, right? I can get somebody over the acquisition wall very, very easily. It’s free, most of the time
I’m coming from a referral from a friend, which is like a very high intent type of thing.

But the fact that it’s like so frictionless, it’s incredibly hard to basically retain somebody on a product like that. Now Facebook has done a masterful job, but just think about how many other kinds of free social products you’ve probably tried in your lifetime.
For every Facebook, you probably have 20 or more others that you didn’t maintain the habit on. So looking at it in that context, right, you can see how hard it is for a product like that.

And that’s where Facebook has certainly, over time, put most of their work around activation and engagement and to a certain extent — to a detriment, which we won’t talk about, we won’t get into that, it’s a whole rabbit hole at this moment — but other products, right? There’s these whole category of products called “set & forget” products where if you think about something like — ah, let me think — a lot of developer tools kind of fit in this category, like API products and stuff where if we think about like a Twilio or something like that, where there’s all of this set up I have to do, or Stripe is a great example too, right?

John Koetsier: Yeah.

Brian Balfour: There’s all of this setup I have to do, but if I get over that setup hump and get that thing integrated, then the product just continuously delivers value to me. And so, the friction, the headwind there, the weakness of those growth models is getting people over that initial acquisition hump. Whereas if I do that, the retention comes more easily.

So once again, there’s strengths and weaknesses in all aspects of our growth model and understanding those at an intuitive level helps you understand where do I need to place my efforts of where I need to like really reinforce and where I need to really engineer this stuff, and where I need to bring this stuff to life, versus where are the other areas that I can probably, you know, just kind of let the fire burn because the fire isn’t that big.

Peggy Anne Salz: So where do you look for inspiration when you’re trying to shape this? We talked, okay, it’s part product, part core loop — we won’t go there, it’s ‘depends,’ right? You’re talking about sort of set it, forget it. I was thinking of my experience with Zapier now that I now will, you know …

Brian Balfour: It’s gone.

Peggy Anne Salz: Oh, it’s part of my life forever now, it’s not going anywhere. So I get that, but, you know, to look for examples and to talk about good examples, you talked about for example, Calm. Okay, but are there other examples or should we even, I’ll throw it out to you, Brian, should we even be looking at these for our role models for the ideas of how to do this?

Or should we really be completely breaking some rules because retention is so personal, so different, depending on how we’re looking at, how we’re measuring it. Maybe we don’t even look at those. So where do you see the good examples or do we seriously just say, ‘Nope, break the rules, go your way.’

Brian Balfour: Look, there’s value to both. There’s definitely patterns in all of this in strategies, and that’s what we teach at Reforge, like, if that didn’t exist, then I don’t think — if everything was custom, I don’t think Reforge would have a business, right? It’s that people don’t understand the depth of these patterns and understand, you know, when you get into kind of copy paste mode, whether that’s like on the marketing side, acquisition and tactics and all that kind of stuff, or it’s on the retention and the product side.

Those copy-paste modes kind of just rarely work unless you really understand the deep nuances and stuff, and how it kind of fits into your overall ecosystem. But usually like I recommend paying it, once again, kind of going back to this analogy that there are things that come natural to a product and there are things that don’t come as naturally, and you have to essentially work for them. It’s more important to identify the companies where they had to work for that element of their growth model and they did an amazing job.

And so the reason I bring up Calm as an example, is because Calm used to be a fraction of the size of Headspace, but they have just executed on this element of their business so well that they are now bigger than Headspace in a very short period of time, and have become the category leader. And once again, that has all to do with the leader Dun Wang there and the team, so all the credit goes to her.

And she’s come and spoken at Reforge multiple times, so thankful for all that. But I think leaders like that and teams and stuff like that deserve to be recognized because they were definitely in a category where they had to work for this and they had to really figure it out. And they blended both the foundational elements of like, thinking about foundational product and use cases and as well as like some of the engineered optimization stuff at the same time. So it depends on the category that you’re in, of where you, of elements that you’ve had to work for and none.

So, trying to think of other examples, a lot of people kind of look at companies like Zoom, as an example, where they’re like, ‘Oh man, amazing, that dollar retention, like 140%’ or whatever their number is, right? Like, ‘that’s amazing that I want that.’ But if you look at their growth motion and their growth model, you would expect to see that number, right? ‘Cause they start as a single, their cohorts start as like a single person in the company and the product is very naturally viral, so if you didn’t see that type of expansion…

John Koetsier: Yes.

Brian Balfour: I would be very concerned, right? Like very, very concerned. And so certainly they’ve done an amazing job with the product and stuff, but that is something that is more natural to their growth motion.

You compare that to that HubSpot, right? Where the model with HubSpot did not have that kind of natural sort of land-with-one-person-expand motion. And therefore, as a result, our model was to capture up front, you know, the average ACV was like $9k, it was a year long contract, and it was a capture and maintain motion, right? Like, so we’d capture it and then we had to really work at maintaining that revenue over time through an amazing customer success team and additional product elements and things of that nature.

And so, HubSpot’s net revenue retention numbers I think are also, they kind of creep up and down past the 100% number, but what you don’t see looking at that number is how much work has gone in to achieve that type of healthy number as a result of it. And there’s a lot of reasons why that was more of a headwind, right? We sold into a VP of marketing. They were usually solo marketing teams, so if that VP of marketing left then the tools tended to get ripped out. And so there was a bunch of work that had to go into making that part of our growth model healthy.

John Koetsier: And that’s super interesting as an example as well, because you have aspects of kind of both sides there, you have a bit of the set and forget, because you’ve got the platform aspect where it connects with a lot of APIs and stuff like that, but you’ve also got a lot of that daily habit stuff going on with that particular model. Very interesting when you can combine those in a product.

Brian Balfour: Yeah, and look, like I think every growth model has its flaws. The pursuit of perfection is one that I have to constantly keep myself away from, especially when I think about Reforge, right, because I spend all my days thinking about and talking about these amazing category leader companies and what they did to get there and all that kind of stuff.

And a lot of times we kind of want to fit our business, you know, we’re like, oh, this is amazing from this business, and amazing from this business, amazing from this business, and sometimes I’m like, I don’t have all of those elements and so I feel crappy about myself. And so I have to remind myself that no business has like all of those elements perfectly combined. It’s about identifying the things that you really have to work at and just kind of building progressively over time. And, you know, all of these types of things.

John Koetsier: I wanted to ask you about people who come into your courses. I’m guessing a lot of them are pretty high-end marketers, pretty high-end brands in a lot of cases. What are one or two things that you find that they still maybe understand least about retention?

Brian Balfour: Well, it depends, right? So for Reforge, we specifically build our programs like any product, we build for a specific target audience. We can strain that target audience and the reason we do that is because if we try to build for everybody, it’s going to be impactful for no one. I think this is actually a huge problem in education today, especially professional education and especially in our general business schools, is that the topics and the courses that they teach are so general, they try to apply it to so many different types of businesses, that at the end of the day they’re just so watered down that it’s very hard for people to apply.

So we very specifically build for product managers, marketers, people working on growth teams that have at least three years of experience. They are in a company typically of at least 30 people, typically a little bit larger. They are post product market fit, they are specifically working on a product in a company, right. They’re not just kind of generally advising, they’re kind of like in the trenches. A number of other factors as well. And so you have to know that context specifically for the things I’m about to say, but I think look, it would take a lifetime to become an expert in all of these areas, an absolute, absolute life…

John Koetsier: And you know your knowledge will be toast anyways, because it’s all changing.

Peggy Anne Salz: Yeah.

Brian Balfour: Yeah, exactly. I don’t think you could even do it in a lifetime. And so this actually, what you see in a lot of companies is that we work from problem to problem… like we kind of shift from problem to problem as we kind of go through this.
And so what at least we see a lot is like some very experienced people that get put on a problem, but they’ve never worked on that problem before and they can either kind of learn it the hard way by trial and error, or they can kind of learn how the people who’ve done it before have done it. And so we see a lot of the same things that we’ve just been talking about, like how to properly measure this, how to probably identify your use cases, how those use cases inform your activation engagement and reactivation levers.

Once you inform that strategy, like what are all the different levers that you can pull on? And all of these things do evolve and they do change over time, especially as new categories of products emerge, we can think about how all these elements kind of combine in a different way. So, and not only that, but we get a lot of people who are like, hey, I’ve heard a lot of bits and pieces of this, or I’ve been exposed to a lot of bits and pieces of this, but I now understand it in such a cohesive way that I’m much more able to clearly communicate, influence, get to the right answer more quickly, all of those elements as well. And so that’s kind of like the second value of Reforge as well, but it’s a lot of the core foundational things.

I think what people are surprised at Reforge is you hear these things, but hearing them is not knowing it. And when we show the ten layers of depth that we kind of just, we walked through on the retention metric, right. Just like, imagine there are those layers of depth for every single concept that you could potentially imagine — use cases, cohorts, manufactured in environment loops, reactivation, psychological levers, customer states, like all of these things. And so the amount of depth there is like, as we kind of really go, we really try to drill down to those deep details and then build people back up once they kind of understand like those layers of depth from it.

John Koetsier: It’s a bootcamp, tear them down, build them back up.

Brian Balfour: Haha, I don’t want to say that. I mean, yeah, like I said before, the purpose is not to tear them down.

John Koetsier: I get it.

Brian Balfour: But I think once again, we try to open everybody’s eyes to the amount of depth and what “great” kind of really looks like.

John Koetsier: Well, the reality is, I mean, you’re obviously not trying to tear anybody down, they’re your customers and they’re your colleagues in a lot of sense, but thinking you know is a perfect barrier to learning, right?

Brian Balfour: Right, yeah.

John Koetsier: And so when we think we know something then it’s really, really hard to teach us that thing. So I totally get it.

Brian Balfour: There’s this blog post that my CTO, Matt Greenberg and I have been talking about writing and I really want to write it, which we don’t have an exact title for it, but it’s something to the effect of like, “I was stupid.”

It is just a series of stories, like most of the things if I look back at my time at HubSpot or my past companies where I was like super frustrated with something, this team didn’t get it, or they didn’t agree with my idea, or this executive … like all those types of situations, I now with hindsight realize that they were not the ones that were wrong. I was wrong and I just didn’t have the right context, the right communication, the like whatever it was, there’s a million different elements. And I’ve talked to so many other leaders about this that have come through Reforge and we all have these stories.

We all have these stories. But it’s so hard to recognize it in the context of things. And I wish I could go back and redo those moments again in my career because I think would lead to just like not experiencing a bunch of pain and frustration, but yeah, there’s, I’ve only gotten that with kind of a lot of hindsight looking at a lot of different examples and seeing a lot of similar situations.

And so I wish it was something that, yeah, like that perspective is I think a really important one for those that grow the most.

Peggy Anne Salz: I have to ask, so I’m just curious, when you say you made a mistake or you — not made a mistake, where you were stupid or had a preconceived notion — is that because you yourself were looking at the data, like you said at the beginning, we tend to sort of simplify, we throw the outliers out, right, because that doesn’t fit the model, and we look at the data and we sort of construct our data-driven reality at some level. I know a lot of marketers who do that. I’m just curious, was that at the core of that at the moment?

Brian Balfour: It depends, right. I think the core of it can stem from a lot of different things. I think specifically, at my time at HubSpot, I’ve actually written about this, where I was brought in to help establish and grow new product categories for the company and they wanted to do it in more of a product growth motion versus a sales and marketing lead motion.

And so, like, I kind of, I over assumed. I kind of knew what that would take in terms of what the monetization model would need to look like, what the product would need to look like, how it would need to target an individual, be low-priced, have some virality in it. Like just all of these details that I had in my head that I assumed others had in their head. And there were details that they had in their head that I think they assumed that I had in my head. And so as a result, right, it led us at the early days of these new products that I led it in a certain product direction, a certain audience direction that we eventually ended up course correcting because we kind of realized that there were these gaps in our heads about what kind of the company’s strategy was and some very nuances there and all of the nuances I knew in my head around what it would take to get to this model. You know, but there was a lot of frustration, a lot of thinking along that pathway of like, ‘I’m right. Why do these people not understand this?’

And now in hindsight, I’m like, I was equal to more of the problem there than they were. And so look, there’s all these types of situations and a lot of times it’s just, yeah, there’s some kind of like unspoken truth or some misalignment around a metric, or a piece of the company strategy, or some important context, or the target audience, or whatever it could be that’s leading to those deltas. So it’s just kind of like, if you just generally take the assumption that you are working with really smart people, right, then the likely answer to these types of frustrations aren’t that you’re right and they’re wrong, it’s just somebody is missing a piece of information, right? And so I don’t know, that’s been my experience, but we’ll write this easy framework, but it’s definitely it.

John Koetsier: That will be super interesting to read. And I know that if I was going to write a similar blog post, I know there’s two or three decisions I’ve made previously that would save literally hundreds of thousands of dollars if I could go back…

Peggy Anne Salz: Oh yeah.

John Koetsier: …with the knowledge I have right now and change them. We’re coming to an end here, but you’ve written recently about COVID-19 and of course that’s been a massive upheaval in a lot of business categories. And some categories have seen a massive influx of new customers.

You’ve talked about new habits as a result of shutdowns and different things happening because of COVID and coronavirus. And you’ve talked about marginal audience, identifying and keeping your marginal audience. Can you talk a little bit about that?
What is that?

Brian Balfour: Yeah, so the credit on this concept really goes to Bangaly Kaba, the former head of growth of Instagram, and VP of growth at Instacart. I’d been talking to him, he’s an EIR at Reforge right now, he leads a couple of our programs, we’ve been talking a lot about it, but essentially the concept of a marginal audience, or what we’ve kind of been renaming to like the “adjacent user” is essentially, it’s that user that is on like the edges of your current users, the next user that you want to acquire.

So there’s a bunch of tricks around this, right? Like personas tend to focus on who your current audience is. Product teams tend to fall in the trap of building for themselves or the power users, but if you really want to grow, what you actually need to do is identify who that next user is, who is that person on the edge that is showing intent of like visiting and signing up, or trying to use the product, but is like failing for some reason or another. And how do you start to unblock them? Why are they struggling and how do you help them start to unstruggle?

And so, the marginal user is important in the context of COVID specifically for those that I would say that we call “experiencing tailwinds,” right? Like they’re benefiting, for some reason they’re benefiting from the new behavior change. And the reason it’s important is most people who are experiencing tailwinds are just getting, seeing like massive growth.

John Koetsier: Yes.

Brian Balfour: Right? And there’s this real question though is like, as kind of hopefully at some point we resolve COVID and some behaviors go back to “normal” or they kind of enter a “new normal” whatever you want to call it, we believe that the people who try to retain everybody will not do as well as others.

And the reason is that those experiencing tailwinds have probably seen a massive influx of customers and users that the product just wasn’t initially built for, like they weren’t targeting that use case. And so there’s tons of examples out there, right? Like, especially Zoom has been the most popular one to mention.

John Koetsier: The poster child.

Brian Balfour: Yeah, the poster child of like, hey, you’ve got all of these college professors that are now forced to use Zoom, but they really hate it and the moment that they can probably go back into the classroom safely, it’s very unlikely that they’re going to use Zoom. So therefore, trying to focus on retaining those people is probably just like wasted efforts and trying to focus your firepower on retaining people that you’ve already built a core use case for and their habit has just been accelerated is fine, but the highest ROI effort is kind of on this adjacent user, these adjacent use cases where they’re kind of like one degree out from your core use case, and they’re starting to use the product in times of COVID because like COVID has basically given them a push over the friction that they’re experiencing in adopting the product.

But you kind of want to take those users and try to lock in their habit and really find, for Zoom, the example we give is there’s probably a bunch of companies that were not remote before, that are now forced to go remote and they’re likely going to stay fully remote or some hybrid therefore after, right? Like those are the set of companies that would be probably an amazing set to make sure that you’ve locked in their behavior no matter what goes after. But that college professor that just wants to get back to the lecture hall or something like that, probably not the highest use of your efforts. And so, my guess is on the topic of retention, a lot of these companies that are experiencing tailwinds, we will see retention decay and degrade, at some point.

Peggy Anne Salz: Mm-hmm.

Brian Balfour: It’s going to be interesting to see how markets kind of respond to that because it’s just impossible for these companies to retain 100% of the new usage that has come in, because they’re coming in and using the product for things that you’ve just never intended to build for. And to build for that new use case is like maybe like two, three, four hops away from where you are now. Like, it just doesn’t make sense, right, you need to progressively build versus just try to like hop to a new island essentially.

Peggy Anne Salz: So it’s fascinating, the position of retention in this discussion, you know, it’s like you have to think about the next user and you have to build retention. You have to build loops. You have to pick out whether you’re going to hop onto existing habits, create new habits. Now, what I do a lot of is interview app marketers who are just really beginning to think about this and it’s interesting because there it’s like, oh, acquisition sits here, retention is over there, and it’s very sort of disjointed.

There’s a disconnect there that’s fascinating in and of itself, which is probably a nightmare for you. But in just a brief description of where retention should sit in an organization because I’m just beginning to see cross-functional teams –it’s getting really exciting, you know, it’s like a petri dish of leaving them out alone and seeing what will happen. But there has to be a place in the org chart for retention. What is it? What should it be? Where’s the fit?

Brian Balfour: Yeah. Almost in any org it’s going to be in, it’s going to stem in product and it should be formed from a cross-functional team, so PM, engineer, design, depending on your growth model, you might have marketers on that team. You might have even customer success folks on that team, but it fundamentally has to stem from product. I fundamentally believe that there are very few meaningful problems that you can solve without product and engineering support. And, in fact, if you get into a mode where there isn’t that support you tend to end up just starting to throw bodies at it. And when you just start throwing bodies at it, you don’t, you’re not building any leverage in your business whatsoever, right? So leverage comes from building like technology software, ingraining it within your product.

But these teams fundamentally need to work together, at least at HubSpot in the early days when we were focused on our new model, our monetization team was kind of a PM in Eng, a designer, and the sales guy’s name’s Mike Pici, he actually runs the growth team at HubSpot now, the product growth team. And it was really important to have that sales person on the team because they were in the trenches kind of figuring out like the new sales pitch, the new model. He was testing things like super quickly and then we took a lot of those things and we started to productize it, build technology around it to gain efficiency and leverage.

And so when you silo the teams off in different ways, you basically, yeah, you end up in this way where you’re not only not solving the problem effectively, but you’re not building leverage over time. So that doesn’t mean marketing or customer success or those other things — I think the reversed egocentric view of this is that those functions don’t matter, and that is 100% not true either, right. But if you basically put them in a silo and like tie their hands behind their back, like what’s the point. It’s extremely ineffective and I guarantee you you’re going to lose the best people in those functions over time because they don’t want to work in that environment. I wouldn’t want to work in that environment either.

John Koetsier: Yeah, exactly. Well, Brian, it has been a wonderful, almost hour that we’ve been chatting with you.

Peggy Anne Salz: Yeah.

John Koetsier: Very insightful. I want to thank you so much for being on the show.

Brian Balfour: Yeah. Thank you for having me. It was a great discussion, and if people want to reach out to me more than welcome to at my personal blog, brianbalfour.com. Reforge.com, if you want to join us for any of the programs. I occasionally tweet
@bbalfour. I’ve been posting more on LinkedIn. So feel free to connect with me.

John Koetsier: Wonderful. Well, thank you so much.

Peggy Anne Salz: It has been amazing, may I say, really, I have to — it’s been amazing, Brian, it’s been like an emotional roller coaster with you, like an intellectual roller coaster with you. I started out like, oh, preconceived notions, throw those out, you know? Oh, you think you know something, no John told me I know nothing, you know? So I had a really exciting day and evening with both of you. I think it’s the most memorable in the series. Brian, I can’t thank you enough for being so incredibly frank and honest about that. So…

Brian Balfour: Awesome.

Peggy Anne Salz: Yes, I had to say that John.

Brian Balfour: Yep, we’ll take it.

John Koetsier: Haha, it’s all good.

Peggy Anne Salz: Had to do it, had to do it.

Brian Balfour: Yeah, I appreciate it, and like I said, yeah, thanks for having me. And if you’re out there and you have a question, feel free to reach out.

John Koetsier: Wonderful. Well, thank you again, Brian. And for everybody else, I’ll try the close again. Thanks for joining us on Retention Masterclass. It’s been a real pleasure to put this one together. Whatever platform you’re on, hey, like subscribe, share,
comment, all the above. And if you love this podcast, rate it and review it, that’d be a massive help.

Peggy Anne Salz: And of course, until next time, keep well, stay safe. This is Peggy Anne Salz signing off with Retention Masterclass.

John Koetsier: And I’m John Koetsier, have a great day.