Image of Stuart Hall, Co-founder and CEO at Appbot

Your App Reviews Are Pure Gold—So Start Mining Them For Insights

Your app users are sharing why they love or hate your app freely in app reviews. Understanding what they say—and what it really means—is pure old when it comes to fixing bugs, adding features or just showing you care when it counts. But when you get hundreds, even thousands, of review a day, you can’t sift through it all on your own. Our host Peggy Anne Salz catches up with Stuart Hall, co-founder, and CEO at Appbot, a company that has harnessed AI and machine learning to help companies like BMW, PayPal and Twitter understand customer feedback and insights contained in app reviews. In practice, the Appbot tool aggregates app store reviews for all countries on iTunes, Google Play, Amazon and Windows stores. Stuart, who has a long track record as a developer, blogger, and entrepreneur, also discusses his early day in the App Economy and offers advice you can follow to maintain an amazing app.

Transcript

Hello and welcome to Mobile Presence.  I’m your host, Peggy Anne Salz, with Mobile Groove, where I plan, produce and promote content that allows my clients to reach performance goals and scale growth.  And of course, here at Mobile Presence, it’s all about growth.  We talk to app marketers, we look at tools and technology, we look at hot topics, we look at everything you need to know to make your app into a sustainable business and that’s why we’re just going to step back for a moment to one of the megatrends I’ve heard for year after year actually and that is the importance of mining your app reviews.  That is a goldmine of information, it’s people telling you voluntarily, you’re not even paying them, they’re telling you what they like, love and hate about your app. 

There’s a lot that you can glean from that and that’s why my guest today is Stuart Hall, he’s Co-Founder and CEO of Appbot, a technology, a tool, a solution that does exactly this.  So, Stuart, first of all, great to have you here on Mobile Presence.

Thanks for having me, it’s great to be on.

So, for listeners, of course, you’re coming to us from Australia.  What’s the story here, Stuart, you’re obviously in Australia, Appbot, globally I would imagine a global tool I can get anywhere in the world but what got you into mobile and what keeps you in Australia?

That’s a great question, yes.  So, we are based in Australia, a very small percentage of our customers are here, the majority through the US and Europe but what keeps us in Australia?  I think, you know, the current climate means you can do this sort of business from anywhere, the only challenge is the time zones and taking calls with customers early in the morning or late at night, otherwise the internet’s opened up the opportunity to work from anywhere.  We love it here, we live in Perth, the weather’s great, the beaches are great, the lifestyle’s great, so we want to stay here as long as we can.

I hear you there and maybe we can do a live broadcast at some point, take it on the road, I’ll be down there.  So Appbot is your baby, you’re there, this is a great place, you have great climate and you’ve got maybe a great climate to innovate but what got you into mobile because I was looking at your background in advance of the show, I always do that, looking and seeing that you’ve been there since the early days of the app store, developer, blogger, entrepreneur, very, very into AI machine learning maybe even before it was cool to do so.

Yes, that’s right.  So when the app store kind of got announced, it got me pretty excited, I was a developer for a long time and I made some simple little apps that got some small traction and they did alright.  Then kind of around 2010 is when it changed for me.  I met up with a guy called Dave McKinney, we looked to do music discovery app called Discovr, it was basically a visual pandora, so you’d type in your favorite artist and then it would kind of pop out with a whole bunch of similar artists and you could listen to their music, watch their videos, read their bios and so on.  And people really loved it, we went on over about two years and we had about 4 million downloads, it was a number one app all around the world but in the end, we just never really managed to monetize it and we got crushed by the streaming services when services like Spotify started to come out.

But, through that journey, kind of the process of learning to read our user feedback and iterate based on that was really the learning out of that.

Of course, that is priceless, I mean, I’ve been talking about it, as I recall it was early on in sort of ASO which is now app marketing optimization – it’s everything, it’s not just about the app on-store, it’s in-store, off-store, everywhere.  And the whole idea was that this is a goldmine of information and people were saying – they did it just on spreadsheets, it was not easy to do.  It was like, “Okay, I’m seeing this adjective to describe my app, this many times, so it’s a great keyword, it’s also something I need to fix” – if it says, you know, sort of like crashes too many times, you’re getting a message”.  As I said, that was a spreadsheet thing.  Now, you’ve got actually Appbot which is a tool that does this – I guess the question for me is what is it focused on?  Is it reviews for keywords, is it reviews for feedback, is it all of the above?

Yes, definitely all of the above.  So we focus on, we do sentiment analysis and we also do clustering so we take the – we do all sorts of customer feedback but app reviews is one of the main use cases, so we take the free text and we analyze it with our AI and we can group it into different buckets, so we know which ones are talking about UX, which ones are support tickets, which ones – who’s a satisfied user, who’s an unsatisfied user so we can bucket it up and send it to the right department in different companies and then analyze the sentiment on there.

So you can have a look at a release and easily find out if there’s a particular problem with a page that has a recurring theme, people are talking over and over again how the sentiment trends over time, you can dig into specific countries.  We had a customer recently at a really big app that probably most of your listeners would know and they found by digging in by country and looking at sentiment, there was really negative sentiment around the language and what they’d actually done is shipped the wrong localization to that country but they had no idea about it until they really drilled down and found out about that negative sentiment.

That’s really interesting because these are things that you’re really not going to notice about your app but it’s great, it’s out there in the wild and users are telling you what they think and this allows you to listen.  I mentioned before the spreadsheets, it’s probably why people didn’t use this, this was a lot of heavy lifting.  What actually am I getting in this tool?  Am I getting something that can come back into my all singing, all dancing dashboard that lets me actually execute on this?  Because this needs to be part of my strategy, how’s it actually feeding back into that?

Yes, so we have our own dashboards and people use that in all different ways.  A lot of the big app developers have insights teams so the people looking at this user feedback and trying to surface out the insights, there are teams of people in these large apps where it’s their entire jobs.  They used to be doing it with spreadsheets like you mentioned, now they use Appbot to automate that process.  A lot of people do embed our dashboards within their dashboards internally or they live in Appbot day to day, whereas that’s their specific role, so…

And is this something that has – it’s globally available, you mentioned, for example, the major client who figured out that they had the wrong localization.  How many languages is Appbot actually able to read as far as app store reviews go?

Yes, we do all the languages on the App Store and Google Play so the machine learning’s built to learn by language so we can spot all the different languages which is one of the major advantages.  We also do translations so you can take an app review from a different language and convert it back to English or your native language.

And you also do Amazon, Windows Stores I’m reading here, all of them.  I mean, is it just I can choose because there are quite a number now of alternative app stores and getting to be more consistently growing offering a lot of opportunity to app developers who are saying, OK, I don’t want to do just iTunes, Google Play, do the others – what does that require on their end or on your end to take care of that?

Yes, so depending on the different stores, some of them are public, some of them you have to authenticate against, so you just go in and you add your app and then you’re away, it’s quite a simple process and we’ve just moved from just focusing on app reviews – a lot of our customers were telling us they wanted to be able to do the same sort of analysis on their other feedback so a lot of the app developers do, you know, entry and exit surveys or MPS surveys and things like that so they’re gathering feedback in lots of other ways rather than app reviews, so we’ve just started branching out to be able to analyze any sort of free text like that.

So, even if I have like a survey at the end, how did you like my app – I see this more and more because they want to get you in that moment where it’s relevant, you’ve just done something so it’s like “How did we do?”  So what is this, this can work with any feedback tool or do you have to make that survey with them, how does that work?

Yes, so we have a bunch of sources, we work within or out of the box or you can upload a csv file with the text in it, we also connect via Zapier which supports over a thousand of different sources, so almost anything you can get into Appbot.  So I think recently there was an app called Superhuman, I’m not sure if you’ve heard about them, they wrote a really good blog post about using their feedback to find product-market fit and they’re really focusing on asking the question of how would you feel if you could no longer use the product and then they focused in on two things – what do people really love about the product and what holds people back from loving the product.  And then by using techniques via spreadsheets like you mentioned, they managed to keep iterating until they found product-market fit and they just raised a whole heap of money, so this process that they found is really what we’re trying to automate.

And that’s always an outcome that particularly my listeners will be interested in, how is that, and that is also incidentally a great segue because product is the new marketing, I’m going to so many conferences, speaking out there, hearing people say it’s great, you have to do your ASO, you have to do your app marketing, optimize your UA, do everything you need to do but if the app isn’t any good, you’re wasting your time and that’s where I believe Appbot and what you offer can answer a lot of questions.  And I’ve got plenty of questions but right now we have to go to a break so listeners, don’t go away, we’ll be right back with Stuart Hall, Co-Founder, and CEO of Appbot.

And we’re back.  Welcome back to Mobile Presence.  I’m your host, Peggy Anne Salz with Mobile Groove and we have today Stuart Hall, Co-Founder, and CEO of Appbot and Stuart right before the break we were talking about, you know, the wealth of information available when you literally mine the reviews but it’s not just information that will help you optimize for ASO keywords – that’s important but it’s very exciting to think about the opportunity to understand what users want without having to pay a focus group.  They’re telling you I’d like to have this bug fixed but I’d also like to have this feature fixed – how to make that product marketing fit, that product fit.  Maybe you can give me an example of this in action.  I mean, you know, maybe you have a case study to share of a company that said: “Yes, I’ll open it up to crowdsourcing, in a way, get this information and reviews and make an awesome app as a result”.

Yes, the one that jumps straight to mind is a large customer that they did a massive redesign of their profile page within their app and they were telling us about how they were trying to track how people felt about that redesign.  So they used Appbot to use our custom topics which allows you to put in some keywords to really narrow down the feedback you’re getting around that particular feature or page, so in this case they narrowed to everything that was talking about that profile page, and then they had a look at the sentiment around the profile page before they released and then after they released and then they used that to pull out the key themes – it was quite a negative feedback once it changed because it was quite a large change and people often are a bit hesitant to change so they really used that to dig in and find out what people were complaining about and they found a number of features that they’d lost and regressed on that people really used and loved.  So it allowed them to iterate on that and really get that sentiment back up and people liking that page again.

What are some of the surprises because sometimes it might be like that one customer that you, unfortunately, can’t name but I can imagine that it’s one of a large customer because I’m looking at a list of your customers – you’ve got quite a number of top-ranking apps, also companies, Pinterest, PayPal, Disney, Verizon, Twitter, Rovia, Microsoft, just to name a few.  These are companies that don’t play, they might experiment a little bit but it’s always got a business reason behind it.  What might be an example of where this feedback really just, you know, really was mind-blowing and told them either to scrap an app or really try something completely different?

Yes, we have a really large customer that has hundreds and hundreds of apps in the app store.  What they do is they make games and then the games fade off or they become irrelevant, they chop them from the app store, so they use us to analyze the sentiment and the trends around how those games are going in time and then chop them off once they start to decline.

Well, that’s an interesting use I hadn’t thought of but it makes sense because if you have a family of apps, you want to know which ones do I really, really need to support, which ones do I need to take out or which ones can I ignore and that’s something that sentiment and reviews will tell you.  If people aren’t thrilled and you have resources, put it into another app.  Correct?

Yes, that’s correct.  Sentiment and trends are really important.  We have 35% of the top charting apps as customers so it shows you how important these large customers think about app reviews and improving their products.

What about the – I mean, obviously you know it succeeds when you see that you have success but are there like signs of how to approach this because you’re just picking up in the reviews what people are saying but that doesn’t tell you necessarily when you’re succeeding because then you would see that by that not being in the reviews.  I mean, is there a way to measure this or to approach this in a more pragmatic way?

Yes, I mean it’s really using the sentiment, so your aim is – as I mentioned before, I think the main aim of any product is to try and get customers to tell other people about your app.  So, as you’re trending the sentiment up and up to more and more positive, you’re finding those people are starting to share the app and tell people about the app.  So, that’s the real thing that you try to measure and pushing that feedback more and more positive over time.

And that feedback is something you can also maybe direct if you understand how these reviews are sort of being grouped or seeing them in a different way that you can actually say, “Okay, now I know which ones to reply to or to manage those replies”.  I mean, one part of it is just the one-way conversation, user to app developer but how does it help in making a two-way conversation?

Yes, sure.  We enable people to reply on the App Store as well so that’s still something that people are working out, how to have those conversations especially in a public forum when it’s a negative app review and you’re trying to extract information out of there but you’re doing it on a public forum.  That’s really in its infancy and people are still trying to work that out but we do enable the replies and we offer some guidance around maybe redirecting them out to hopefully contact support and walk through the issue with them there.

But, yes, the two-way conversation is really interesting, and we have a really large customer that was telling us the other day that they get so much of this sort of feedback, they just can’t even read it let alone reply to it.  They have buckets and buckets of data that comes in around feature requests and people’s comments, but it just goes into this bucket that no-one has time to look at.  So, there is so much to look at that that’s where we’re trying to enable people and really look at huge amounts of data on a trend level.

So, is this also a bit of literally my eyes and ears that AI here, in Appbot is also not just telling me about sentiment but also maybe perhaps helping me answer, figure out the answers or is this still manual?  When you say you’re advising or giving your clients some idea of how they need to be replying to reviews, is that some inspiration or some automation?

Around the replying, it’s just inspiration but we do surface, we group by the most common words and things like that and spikes in negative sentiment, so we just surface those to the top so that that’s all automatic.  I think one thing it really does solve is a lot of the time this information doesn’t make it to the top of the tree, it sits with the person doing the support or reading the app reviews and then it’s a bit of Chinese whispers going up the chain but with a tool like Appbot, the product manager or the CEO of a company can see the real truth straight from the source.

You have a way of describing this in one of your blogs I was reading, Feedback Driven Growth – I really like that, that’s yours so I can’t take it but it’s a great way to look at a different type of growth.  You know, when we talk about growth, we talk about growth hacking, we talk about doing something in programmatic, this is feedback driven growth.  Maybe you can explain what that really is because I have a feeling that that’s going to be something, we’re going to be seeing a lot more of because it seems to be more lasting or at least more about loyalty and we all want to go from acquisition-only to retention.

Yes, definitely.  I mean, I think it’s a lot of what we’ve been talking about so far and I think we read about on the tech blogs about all these companies that grow, they launch and they’re huge straightaway but in reality I think most successful tech companies don’t actually have that hockey stick growth, they have linear growth, they kind of launch really quietly and then they have very few customers in the early days and they constantly think they won’t make it.  But the companies that do make it I think they really focus in on those first few customers and they try and delight them as much as they can and then use these techniques to gather the feedback, analyze it, pull out the really key things, find out why people are abandoning your app or not using your app or saying how it could be improved and just continually iterating until they find that product market fit.

So the feedback-driven growth is really just that process of going round and round, gathering feedback, tagging it, analyzing it, iterating and then starting again.

So, Stuart, the business value is very clear here and that’s great because some of our listeners will be listening and saying “Yes, I get it, I understand that there is value in app store reviews and Appbot is a tool that’s going to aggregate that and bubble that up for me”.  What is exactly your business model around this?  How do you at Appbot make money?

Sure, so we’re a SAS company with recurring monthly costs so our off-shelf plans are between $50 and $200 a month and depending on how many users you want or the specific features, then we also have enterprise plans that are specific above that.  But, yes, it’s a monthly recurring cost.

And I mean you have a number of very, very large companies but is this something also for the indies of interest?

Yes, definitely.  We have a plan at $49 a month and we have a lot of indie developers that are using that, they know how important that reviews are and the content coming out of them and I’ve been an indie developer and I know how busy you are and how many things you’re trying to juggle, so a tool that can automate this for you is extremely beneficial.

I was going to say and you as an indie developer, maybe this even came out of your own experience where you said, “Man if I only knew what they were thinking about the app, I could have done something differently or fixed it earlier”.  Was that the inspiration here?

Yes, definitely, I had an app called “7 Minute Workout” that I made – I made it as an experiment really and I wanted to blog the experience of listening to customers and iterating and I grew that over 18 months and blogged the whole experience and grew it to 2.3 million downloads and eventually got acquired and that really fleshed out the process.

And it was part of that – I’m wondering, is that still out there to check out there, Stuart?

Yes, it is.

It’d be interesting to see that experience documented, day one, day two of my app development adventure.

Yes, for sure.  It originally was a 5-part blog post but it’s been put together in one blog post called “The App Store Experiment”, it’s on Medium, it did quite well, it was back in the – when it was released, it was the third most read article on Medium for the month and it’s had almost a million views, I think.  So, it was really successful.

So something still relevant, something we should still all check out.  I think I will, I think you gave me that link, I’m definitely going to check that out.  And listeners, we do have to go to a break for the moment but when we get back, we’re going to be talking more about the nuts and bolts of what you can do and what you should and shouldn’t do, maybe is better put, dos and don’ts of getting more out of your reviews that will pay off for your app.  So don’t go away, we’ll be right back after the break.

And we’re back to Mobile Presence.  I’m your host, Peggy Anne Salz with Mobile Groove and we have today Stuart Hall, Co-Founder, and CEO of Appbot.  And Stuart, it’s great speaking with you about your experiences with reviews, what we get out of reviews, why we need to pay attention to reviews and you are giving advice to your clients increasingly on how to answer reviews.  I mean, to kick it off, what’s like the weirdest or oddest or just the most surprising review that you’ve had to deal with on behalf of a client?

Yes, I mean, I think personally one of the funniest reviews I ever got was one guy talking about how he always remembers he find my app because he was eating the greatest sandwich of his life while he was eating it.  You do get some really funny app reviews, you know, there are people out there with some great senses of humor.

Absolutely.  And I’m just wondering, so they can be odd, they can be a little unexpected which is all the more reason to understand how you’re going to answer them, and that’s what you’re giving some advice on.  So, just at a high level, Stuart, a couple of dos and don’ts.  What should I never do when I’m answering a review?  I’m getting some bad feedback, what don’t I do?

I think the key thing to remember with app reviews compared to other support is your reply is going to be public on the App Store so your potential future customers are going to come and land there and potentially see your response, so your response there reflects on you quite a lot.  So I would say, you see a lot of app reviews that are just like, then the reply is “You’re completely wrong, you’re an idiot” and things like that.  Definitely never do that, one, you should never talk to your customers like that but, two, other customers in the future see that, they’re not going to download your app.

I think the real key for me is to try and be empathetic to the customer, you know, reiterate their problem, thank them for leaving the review and give them some sort of hope that you’re going to fix their issue in the future or thank them for their great review and tell them how much it helps, they’re really like the key things I tell people.

So do you make promises, do you say “Yes, we’re going to fix it by X” or “We’re going to add that feature by X”?  It makes you sound as if you’re responsive, you’re certainly empathetic but can you also potentially shoot yourself in the foot by promising what you can’t deliver?

Yes, I think you’ve got to be very careful about that, not promising what you can’t deliver but you definitely can promise either you’re going to bring it up with the team or take it to the product managers or things like that.  Customers really respond to knowing that you’ve listened to them and you’re going to take that feedback and pass it on.

Speaking of feedback, is there a rule for often – because I can think of a number of features I’d really love to see on some apps but I might be the lone wolf out here asking for it.  Do you have a rule of thumb about how often you need to see a request for something or companies should see a request or something show up in what Appbot is surfacing before they actually act on it because features aren’t cheap?

No, definitely not.  I think it’s not necessarily a number but the things that come to the top of the list, classify everything and then certain things always bubble to the top of the list that becomes the most common things.  So, I think if one person is asking for something, you know, you’re definitely not going to build that but most companies have their process around taking the customer feedback and looking at the most request things and then finding out a lot of the time those replies will actually be the solution but it’s really looking at that solution and going “What’s the actual pain point this customer is having and how can we solve that in the best possible way for our app?”  So, yes, I think that’s the real key.

Okay.  And so you’ve got a lot of firsthand experience, you’ve got a lot that you can also share, I’m sure our listeners will want to keep up with you – obviously you’ve been writing on Medium, you might have some blogs you want to share or other things – how can they catch up with you, stay up to date with what you’re writing, what you’re doing over at Appbot, what’s the best way?

Sure, the two best ways are I write over on our Medium publication which is stories.appbot.co or they can follow me on Twitter, @stuartkhall is my Twitter handle and then I’m sharing a lot of my thoughts on that.

Okay, and we’ll have both of those in our show notes, also a link to your Medium, I’ll have to check that out more often, Stuart.  Great to have you on Mobile Presence.  And, listeners, if you want to keep up with me throughout the week or find out more about how you can be a guest or sponsor on Mobile Presence, then you can email me, peggy@mobilegroove.com, Mobile Groove is where you can find my portfolio of content marketing and app marketing services. 

And that, my friends, is a wrap of yet another episode of Mobile Presence.  Of course, you can check out this and all earlier episodes of our show by going to webmasterradio.fm or you can find our shows on iTunes, Stitcher, Spreaker, Spotify and iheartRadio simply by searching Mobile Presence.  So until next time, remember, every minute is mobile, so make every minute count.  We’ll see you soon.

Leave a Comment

Your email address will not be published. Required fields are marked *