If it’s user acquisition in 2018, it’s data-driven by default and tech-driven by design. Our host Peggy Anne Salz from MobileGroove catches up with Matt Sadofsky–Senior Manager, Growth Marketing at Tilting Point, a new generation games partner for top independent development studios–to discuss the approaches marketers must know, and the personal talents and traits they should cultivate, to achieve success. Matt candidly discusses where UA is going and how his passion for data and thinking outside-the-box makes him a better marketer, allowing him to reach the KPIs that have earned him the title of Mobile Hero as chosen by Liftoff, a full-service mobile app marketing and retargeting platform.
Transcript
Hello and welcome to Mobile Presence. I’m your host, Peggy Anne Salz, Content Marketing Strategist at Mobile Groove, where I plan, produce and promote content that allows my clients to reach their performance goals and scale growth. And of course growth is what it’s about here because this is our mini-series at Mobile Presence where we take the inside track, we look at what UA experts, app marketing ninjas are doing, how they’re doing it, how they’re moving the needle on growth. It’s success that has earned them the title of Mobile Hero as chosen by Liftoff, a full-service mobile app marketing, and retargeting platform, and we have every other week a Mobile Hero here.
So, this week, we’re going to take a different approach, we’re going to look at what the buzzwords are – AI, machine learning, it’s what you’re hearing, it’s what I’m hearing at conferences, it’s also the topic of my own webinar coming up soon on Venturebeat – marketers want to know what is it, how can they use AI, what can they automate and how? What’s the potential uplift and if it’s even, you know, worth all the effort – so these are all great questions, I don’t have the answers but that’s why we have our guest today – he’s Matt Sadofsky, Senior Manager Growth Marketing at Tilting Point. Matt, great to have you here.
Yes, thanks for having me Peggy.
So this is your zone, you like AI, you’re very into sort of tech-driven UA and we’re going to talk about that but first let’s talk about what you do over at Tilting Point. Now, I’m reading that Tilting Point defines itself as a new generation games partner for independent development studios. So, that sounds very exciting, sounds very cutting age but what is that? What do you mean by new generation, what’s that approach you’re taking.
Yes, so Tilting Point’s kind of in an interesting point right now where we’re both servicing independent development studios, kind of build their games out from early stage by providing user acquisition, custom tech data, data science, tech engineering, game design, kind of full service publishing and then we’re also working with larger studios, still some indies, some not so indie, on running user acquisition for them. And they’re choosing our custom UA tech, UA funding and UA managers that we have in-house here.
And you yourself over there, Senior Manager, Growth Marketing, and I’m reading about some of your accomplishments here in the time that you’ve been there – managing a 5 person growth marketing team, seeing a portfolio of apps go from 5 to 15, watching UA monthly spend increase 5 fold. All huge accomplishments, probably what earned you in part the title Mobile Hero, but let’s backtrack. I mean, what is your background, what has allowed you to chalk up those wins?
Yes. So, kind of coming out of college, I graduated with a marketing and management in global business degree, so I had a good sense of kind the psychology behind marketing, marketing techniques but I’d always been a huge data nerd. You know, I make a spreadsheet for everything – I want to sleep, I make a spreadsheet. So kind of I was figuring OK, well how do I bring my love of marketing and my data nerd together and kind of stumbled into a marketing analyst position at a company called Snap Interactive, and that was my first exposure to user acquisition marketing.
Sounds like you really covered the bases because that’s what it’s all about, isn’t it? I mean, how do you get to the data side of marketing which is where everybody wants to go these days? Well, it’s basically that, you had the foresight in a way to combine your inner nerd, nurture your inner nerd and bring that through marketing is what it sounds like, just mashing those two up. Is that what it is?
That really is what it is. You know, when it comes down to it, user acquisition, you know, that’s just a fancy word for advertising and advertising back in the day, you’d think Mad Men, you’d think spreadsheets, not spreadsheets, you’d think posters and cigars and backroom coming up with concepts and that’s kind of gone. It’s now a data-driven position, it’s more of a data science than it is a marketing role.
I like the way you’re comparing those because it is true, you think about Mad Men, but you’re right, it is advertising, it’s just advertising the way we do it. Since we’re not in that back room with those cigars and having that great time, which it would be awesome if we were in the industry, I’m just curious, Matt, what does your daily job look like?
Yes, so right now, it’s kind of split between three different sections. One is working with our creative team to make sure that we have top-notch creative, we’re testing a large amount of things across our portfolio of games, our portfolio of vendors that we work with and making sure that we have good data on creatives performing best. The second is building out custom tech, so that’s making sure that hey, if there’s something that a computer could be doing better than a human, well, let’s have the computer do it. So working with our data science and engineering teams to build tech out.
And lastly, being kind of person to person, you’re in negotiations with vendors, working with clients that we have, you know, kind of selling the service. So, that’s kind of the three areas where I focus my day.
So, I mean, you have your skill set because it’s part of basically what’s in your DNA that got you where you are but if we’re talking about growth marketers overall, you know, thinking about how you spend your day, it’s between data science and thinking and thinking about the future, where we’re going to build out, what computers can do better than people and vice versa, and then also the service side. All of that put together – is there any specific sort of skill set a growth marketer needs these days based on that routine?
From a hard skill standpoint, you absolutely need to have a good understanding of data analysis, you should be able to calculate an LTV projection from a dataset. You should be able to know how to do a pivot table in Excel and look at data. You should be able to know how to read dashboards from attribution partners. But on the soft skill side, just kind of in general, you need to have really good critical thinking skills because maybe you are doing a really good job optimizing your Facebook campaign but what’s that next layer of optimization that could push it even farther? Maybe it is that custom tech that you can build out that sits on top of Facebook and maybe looks at things that you would have never thought of like LTV by device, by country, by language. You can get super granular with the data and having that critical thinking skill to want to explore those avenues is huge.
How do you, on the other hand, Matt, how do you sort of stop your inner passion for this and say OK, there has to be a cut off line somewhere because the reason I bring this up is I was talking to a growth marketer really about AB testing can get into ad infinitum, into infinity, it can go on and on – you have to have a cut off point somewhere because he was talking about Oh well, the captions and the language and the colours and this is that, and if you really did this seriously, you could do this only. So, what would you suggest are the areas where you might say OK, that’s good but let’s just stop right there?
For me, it depends on where the app that we’re working on is currently at. We have apps that are kind of in the earlier stage and at that point, I don’t want to have a conversation on whether or not the light green button or the dark green button is performing better. I want to have a conversation about does this concept work better than this concept? So, I think some people try to jump too quickly into the nuances of what can be different in one ad versus another but sometimes you’ll find that really weird concepts can work for a game.
You know, maybe you take your game and you reduce it down to a stick figure story of what’s going on, maybe you do something where it’s a TGI trailer that has that and a mix of gameplay, maybe you just do a gameplay trailer. These type of learnings are more important early on than is dark green better than light green.
So, really what you’re saying is to understand your app, your audience, know that your concept flies, you can worry about the details, so to speak, later on, correct?
Exactly.
So, what about, you know, something gets you up in the morning – is it about just hacking the growth and getting the most out of platforms or what is it that makes you excited and passionate about what you’re doing?
Yes, I think it’s finding new ways to use data to make informed decisions. Sometimes I’ll have an idea, like the end of the day on a Friday, about Oh my god, we could totally build out this tech that could optimize this campaign in a new way, or maybe we could look out by breaking out our analysis by looking at, you know, service provider like AT&T and Verizon, and then I’ll get so excited about it that I’ll probably start specing it out on a Friday night and go to bed at two in the morning.
So, you know, finding new ways to kind of growth hack with data gets me super excited.
I’m hoping you’ll lift the lid on maybe just a few of those, Matt, if we’re lucky, but right now, listeners, we have to go to a break, so don’t go away, we’ll be right back with Matt Sadofsky, Growth Marketer at Tilting Point.
Hello, and we’re back. Welcome back to Mobile Presence. I’m your host, Peggy Anne Salz with Mobile Groove and we have today, Senior Manager, Growth Marketing at Tilting Point, Matt Sadofsky. And Matt, right before the break, we were talking about basically how you probably don’t get an awful lot of sleep because you get inspired trying to do different things with data. Now, I won’t say this is going where no man has gone before but it does sound exciting – can you take me a little bit on the journey of how you do something completely different with data and get some amazing results?
Yes, for sure. So, the way that I kind of think about it is at scale, humans are innately going to fall within the law of averages so if you can have a good idea on how one of your apps is going to perform within, let’s say, Iron Source or an App Leven, on a single publisher, as you get enough data on all these publishers that you’re running your ads in, you should be able to find similar publishers and start to utilize kind of machine learning to lump together similar apps to kind of get this law of averages and make a projection on the performance of those apps.
And that’s super useful when you’re thinking about kind of when you’re first launching a new campaign on one of these networks, kind of taking those learnings and applying that and especially using tech like machine learning to kind of aid that. So, you know, kind of diving into these kinds of new ways of looking at data I think is really the way to go.
So, a little bit like bell curve meets benchmark in a way?
Yes, in a way and we’re working on ways of even kind of getting that down to a more minute level, like, you know, instead of looking at it on a publisher level, we should be looking at it on a user level which is much harder to predict but there’s no reason why we shouldn’t be able to at least get an indication on whether or not an individual user would fall as a quality or non quality user.
So, where do you see, you know, it’s a broad question but I would love to hear what you have to say – where do you see the future of UA?
So the future of UA is going to be programmatic. I think you’re going to have Facebook and Google and probably some other kind of proprietary social networks like Snapchat, Instagram out there, but then I think the rest of the network is really going to fall into kind of these open exchange programmatic RTB marketplaces and those are so data-driven. I just found out that those bids on RTB take place in typically less than 100 milliseconds and often less than 10 milliseconds. So, you need to be able to predict whether or not an impression is likely to become a paying user in 10 milliseconds.
That’s definitely not human, that’s not a human doing that, that’s really tech-driven UA, absolutely.
So, there’s a lot to be done on the tech side and I think we’re going to see bigger budgets being managed by smaller teams and have more tech do the heavy lifting.
So, how are you preparing in your job for this shift? I mean, for you it’s probably not so much but maybe how you manage your team?
Yes, so the way that we’ve kind of been thinking about our team right now, because I think as we’ve been growing, we were small enough that kind of everybody could work across every channel. We’re now starting to get to that point where we have so many clients and so many people on our growth team that we’re starting to develop expertise, we’re going to have someone be a paid social expert, someone be a search expert, network expert and so on. And not only are they going to be an expert in optimizing those channels but they’re also going to become the experts in making tech recommendations.
Someone who sits there all day optimizing an App Level campaign should be able to recognize Hey, here are things that we could be doing better if we implemented machine learning on top of this.
OK.
You know, kind of really making sure that we have people, like I mentioned earlier, that have really strong critical thinking skills and aren’t just going to get comfortable with doing their spreadsheet analysis of a campaign.
So they’re going to say to themselves, I’m looking at this and I see that this is a routine or that this is something – it’s either a routine or it’s just something that needs to be done so quickly that humans can’t do it or can’t do it that well and pinpoint those areas where this is where we need machine learning, this is where we need AI, this is where we could offload this onto the machines. Is that right?
Yes, absolutely. You know, for example, on Facebook, we created a campaign creation tool, we used to have an intern sit there all day and create Facebook campaigns for us and we were like, Why are we subjecting somebody to doing this? So we built out a campaign creation tool and now we can create hundreds of campaigns in a couple of minutes. So, you know, not only is it machine learning but also just things that reduce redundancies and user acquisition as well.
So, this is what you do in your job but also the great thing I’m excited about Tilting Point is you open up these capabilities to the companies you’re working with as well, when you’re helping them grow their app or you’re helping them reach whatever benchmark that needs to be for them. So, what can you tell me about the capabilities you’re building out at Tilting Point to do this?
Yes, so so far what we’ve built out is a really strong machine learning algorithm which helps predict LTV at a sub-pub level, so that can be an ad set on Facebook, a publisher within an SDK network campaign or a country within a Google AdWords campaign. And we’ve also built out a lot of tech to help us create campaigns, optimise creatives, stuff like that but kind of what 2018 and 2019 is going to look like for us is really going to be focused on getting user level predictions, starting to look into building tech for real-time bidders, optimising the in-game experience on the user level, really pushing things to a more granular data level than we’ve been at in the last year or so.
So we had the opportunity to catch up a few times and I recall I was looking through my notes from our earlier conversations, also AI for image recognition – tell me about that.
Yes, so that’s something we’ve been looking at for a couple of months now and it kind of comes back to what we were talking about earlier with how minute do you want to go in your AB testing? And if you can remove the human guessing to what makes one ad work better than another, that just makes everything so much more efficient. If you’re running 10 ads and an AI can identify that your 3 best performing ads, they all had in common the color purple, that each had 2 people in it, and everything was shifted to the left side of the ad, that’s going to be something that can be much more effectively done by a computer than a human trying to say why did this ad do better than this ad.
I’m speechless because I’m thinking how much sense that makes. I’ve been doing some interviews with people about AB testing and they’re saying well, you need to do these tests, you do them across countries, you do them across stores and then you need to figure out the key components that they had in common and you’re telling me exactly that. AI can tell you that – machine learning can say Hey, these ads that were all pegging the needle, well, here’s why – and then you can decide from those component parts, you know, what are you going to put in other ads.
So, this is something you’re thinking about, this is extremely exciting and very breakthrough. Is this something we’re going to be seeing out there soon, because I’m sure listeners listening in are saying Hey, if that’s a capability they have over at Tilting Point and if I have a good game doing really well, you know, not just someone out of a garage, I want to make my way over there. So, what can we look for?
Yes, so we have – I’ve sent a list of 15 new UA tools that I’d like to have my tech team build in 2018, that is one of them. There’s also that kind of granular level user quality score projection, there’s new machine learning leveling for other networks that we haven’t yet partnered with, kind of API-based machine learning. So there are quite a few things that we’re working on. I’d like to achieve them all in 2018, we’ll see if we can get there.
Just thinking off the top of my head, you know, that also then creates a different perhaps skill, even potentially for you, Matt. How do we interact, create, co-create, think, brainstorm with AI for your job? Have you given any thought to what that might entail?
Yes, so I think the UA manager of the future is going to be more focused on feeding the right data into your AI or your machine learning system rather than having to do that analysis yourself. So, for example, with real-time bidders, you can adjust your bid based off of, you know, things on a high level like time of day but it goes as granular as the lap long of the user. So, there’s an endless amount of data that you could use to optimize campaigns and as we go into this more granular tech-heavy data UA future, you’re really going to need people who can come in and say Hey, we have this large amount of data, this is what we can do with it, rather than here’s my campaign performance so far, here’s how I can improve it.
And currently, what are the sources of sort of not the data, the data is of course in your system and what you’re seeing but there’s external tools and data and reports that you’re looking at the benchmarks that help you do your job – do you have any thoughts on what they are for you, maybe because listeners listening in might want to know, OK, when I’m getting all the data for the holistic view of everything, what else do I need to keep in mind, what else do I need to be reading and watching?
Yes, so things that we look for, you know, depending on how new a campaign is, obviously when we first launch a campaign, we’re looking at things like GO, gender, age. But once we get past that stage, then we start diving into things like what device has the highest LTV, what network where they running on Verizon, AT&T, things like that, what languages were those people speaking, what other apps do they tend to engage with, even going down to OK, so this user installed and they’re time between their first session and their second session was this long, so maybe that means that this user is going to have a higher likelihood of being a payer.
So, it just keeps on getting more and more granular from when we launch a campaign to when we try to be fully optimized.
Well, I can see, Matt, speaking with you gives such a really valuable inside look at the data we need to be dealing with and what’s coming down the road next. I mean, it’s really interesting what you’re talking about AI and those capabilities, but right now, we have to go to a break so listeners, don’t go away, we’ll be right back and talking more about AI and the future of UA. So don’t go away.
Hello, and welcome back, we’re back. Welcome to Mobile Presence, I’m your host, Peggy Anne Salz with Mobile Groove and of course, we have today, Matt Sadofsky, he is Senior Manager, Growth Marketing at Tilting Point. Matt, right before the break, we were talking really an exciting deep dive – I’m thrilled, I can’t wait till your team starts to work through your wish list of cool AI things to build for UA – hope you’ll keep me posted on that but in the meantime, let’s just talk about, you know, at one point it’s AI, then it’s the human skill to give AI what it needs to do its job best. But, how do you reduce error in UA whether it’s human or machine learning or otherwise – do you have some thoughts on basically how to make certain it’s more wins than let’s say misses?
Yes, absolutely. For me, I think the most important thing when it comes to more of something that’s not necessarily a skill but it’s being passionate about what you’re doing. If you aren’t passionate about data, if you aren’t passionate about what you’re marketing, if you’re not passionate about user acquisition, I think you’re going to be prone to getting comfortable and getting comfortable definitely leads to error.
So, you know, if you are super passionate about data, go for user acquisition, if you’re not so sure about it, don’t just let yourself fall into it because when you’re talking about spending hundreds of thousands of dollars, millions of dollars a month, you really have to make sure that you’re staying on top of that.
So, I think on the human side, kind of on the soft skill and a personality trait, you’ve got to make sure you’re passionate about it. And then the other side, it’s really making sure that you have all of your data lined up and staying organized. You know, make sure your reporting is clean and your database is clean. If you have data that’s muddy and that’s not communicated well to people and people are making decisions based on incorrect data, you’re going to have errors.
So, you know be passionate about what you’re working on and make sure that you have organized, clean data and then from there just, you know, explore and see what you can do to kind of push the boundaries of using data to make informed user acquisition decisions.
So, how did you – I’m just curious because I talk to a lot of UA managers and I’m talking about, OK, how did you order your data, how do you keep it clean, what do you do when what you want to be looking at goes beyond the confines of a spreadsheet because it’s a 2D sort of thing that you’re doing and there’s a lot beyond that in size and scope – other questions like how often do you want to look at a report, how often do you need that fresh data? Obviously a lot of questions, not always answers that you can give but I would love to hear your take.
Yes, so I think that you should be looking at data as much as you can. It doesn’t mean that every hour you should be refreshing your data and checking see how a campaign’s going but making sure that you’re working with your attribution partner on any discrepancies you see, making sure that you’re working with your tech team to make sure that your database is set up properly and then always checking that data against verified sources. So check your revenue data against the ITunes Connect Store and GooglePlay, check your in-app data maybe on your attribution partner against the actual log events coming into the app developer and then once you have that good data, and you feel comfortable with it, be bold with your choices, be bold with your decision making – push the boundaries.
If you test something and it doesn’t work, you could always pull it back but if you don’t test it, you don’t know where you’re going to get your optimizations.
What about you personally, Matt, obviously I can tell by listening and speaking with you, you know, you are into this, this is your zone, this is your craft but how are you going to sort of maintain this passion, this level of – just for lack of a better word, I’ll call it excellence, I think it is? It’s sort of like you don’t want to rest on your laurels but you don’t want to push too hard so sort of like those stretch goals, how are you going to do that and what’s next for you?
So, I like to think six months ahead, you know, what I’m doing today is so that my team will be ready for six months from now and then six months from now, I should already be working on what they’re going to need six months ahead of that.
I wish I could work with you, Matt!
So, yes, it’s staying on top of the future and if you get complacent, if you get comfortable and you say what we’re doing now is great, then you’re probably not going to be doing well in six months from now because the UA space is rapidly changing, it’s rapidly going towards programmatic RTB, data-heavy, data science and that excites me, it keeps me going and I love kind of innovating and pushing tech and user acquisition.
Well, you’ve got me and absolutely I want to check in with you in about another six months because then I’ll have a view into what? 2019, that will be awesome. In the meantime, how do our listeners stay in touch with you? I mean, I’ve read a couple of blogs of yours so I know you’re out there, so how do we stay in touch with you and how do we also keep up on what you’re doing in the blogs?
Yes, absolutely. So, the kind of two places right now where you can keep in touch with me, LinkedIn, you know, feel free to connect on there, I do like I post some blog posts, actually working on a new one right now called “Growth Marketing is Data Science”, and also Liftoff Mobile Heroes have a blog post which is going live with them next week about kind of the skills it takes to build a career in user acquisition, hopefully doing some more blog posts and articles with them in the future too.
Absolutely. And of course, you know, we can see your Heroes page over there, a dedicated page over at Heroes, heroes.liftoff.io, so that would be where your blogs are, that’s where we can find out more about you and connect with you. So, thanks so much, Matt.
Yes, thank you, Peggy, it was a pleasure.
And that, my friends, is a wrap, and of course over the next weeks we’ll see more Mobile Heroes with more amazing companies so do stay tuned and if you want to keep up with me throughout the week or find out more about how you can be a guest or sponsor on Mobile Presence, then you can email me, peggy@mobilegroove.com, that is Mobile Groove where you can also find my own personal portfolio of content marketing and app marketing services. And of course you can be sure to check out this and all other earlier episodes of our show by going to webmasterradio.fm or you can find our shows on iTunes, Stitcher, Spreaker and iheartRadio simply by searching Mobile Presence. So until next time, remember, every minute is mobile, so make every minute count. We’ll see you soon.