App marketing powered by machine learning and AI allows app marketers to identify the events, audiences and ad creatives that can take their campaigns to the next level. Google and Facebook may have been the pioneers, but their dominance is challenged by managed DSPs (demand-side platforms) that are fighting smart with approaches that harness algorithms and AI to help app marketers make better buys and achieve higher returns. How can app marketers teach and train the algorithms to get the best results? Our host Peggy Anne Salz catches up with Misha Syrotiuk, Head of Ad Networks and Programmatic Buying at Huuuge Games, a leader in the social casino games market, to provide you a crash course in DSPs (what they are and what they offer) and some tips on how to train DSP “black-box” algorithms to help you net high-value users and payers. Misha, a Mobile Hero recognized for his app marketing expertise, also shares a checklist of what to look for (and avoid) when choosing the right DSP for your growth campaigns.
We have this series here basically every two weeks so glad you’re checking in and be sure to check all the other ones out because they are indeed amazing Mobile Heroes. And today we’re going to look at AI, marketing automation – we know it’s hot, we know it is necessary to app marketing, it is necessary to understand it but of course the technology is only ever going to be as smart as the teacher is. So how do we train it, how do we train algorithms, how do we train AI to help you do your job better? And while we’re at it, you know, how do we get the most out of AI within demand-side platforms, DSPs, all this and more, we’re going to catch up with our guest today is Misha Syrotiuk, he is Head of Ad Networks and Programmatic Buying at Huuuge Games. Misha, first of all, great to have you here, how are you doing today?
I’m doing great, how are you doing, Peggy?
It’s good, it’s good. We’re in the same time zone, you are based in where?
I am based in Warsaw, Poland, so just two hours’ flight probably from where you’re based.
Exactly, and we’re not far from most of our guests, most of our listeners in the States. Let’s start off with yourself, I told them what they’ve got to sit tight for, it’s going to be all about AI which is hot, huge, like Huuuge Games, actually, and they have to hang in there and listen because this is something that you have to know to do your job better but tell me about you yourself, Misha. You’re a Mobile Hero, you’re also Head of Ad Networks Programmatic Buying, Huuuge Games. Tell me about yourself and Huuuge Games.
Yes, so I started in the mobile marketing actually in 2013, so it’s about six years ago and at that time I started in the Polish m-commerce and the digital audio book app, and I remember still the time when I was buying Facebook ads for one, two cents in different markets – those times are long gone. So three years ago, my friend, he was hired in Huuuge Games, it was hot growing Polish gaming start-up and he asked if I want to join and of course I wanted so I’ve been with Huuuge Games ever since managing a team of four people that does UA in programmatic and DSP for our two biggest titles, Huuuge Casino and Billionaire Casino, and as of today I’m focusing probably more than 50% of my time on specifically programmatic buying and we have big plans for 2019 and 2020.
I mean, there’s a lot on your plate which is why I have you on the show, we have to learn from people like you but on a personal level, I’m just curious, you know, you’re in a career that moves so fast, all of UA managers actually, but how do you personally do it? Is there something you can tell us or share with us how you run your daily routine, what gets you through it faster, what helps you think faster because programmatic moves really fast. Maybe it’s, I don’t know, a Pomodoro that you’re setting, 25 minutes for full focus – what can you share?
Yes, so I think I don’t have any specific rule or routine, I am motivated by two things. So, first is people and second is hard work, so actually I try to combine and mix these two components so I come to work, do all the daily routine emails, meetings, but then I spend the majority of my time actually with internal teams so first of all my own team on one to one meetings, with creative team, the IT team trying to find out where else we can buy smarter, but a big part of my day and my time is actually talking to our partners so we work with a lot of external third party partners who would either do managed DSPs for himself or with our MMP partners or with people like you and trying to find out what’s new in the industry and to be sure we implement it as soon as possible.
I like what you say about buying smarter, that’s a big part of what you do and probably why you even focus on really getting the most out of the DSPs because we know what they are and we have some idea but I have to say you’re one of the few I’ve spoken with who really talks about, okay, we have to get demand-side platforms, we have to get them up to speed in AI. First of all for our listeners who are not quite clear about DSPs and the whole sort of where that fits in – I mean, maybe you can tell me why focus on getting the most out of your DSP in the first place with also the idea of like who they are.
Yes, of course. So, if you are working on the advertisement side of the company, then probably you can do the majority of your user acquisition with just the duopoly, so Facebook and Google. But when you are scaling and when you want to buy certain specific markets, there are a few reasons why you actually should focus beyond Facebook and Google. First of all, not all users are on Facebook and Google, in some countries, for example Japan and Korea, Facebook is not that popular as other ad services. Second, for traditionally Facebook and Google, they will mostly focus in on their own inventory and a lot of other gaming companies specifically would be selling the ad spaces through SDK or the video networks, so in today’s world actually programmatic DSPs can buy those inventories in a smarter way.
Third reason and it’s actually quite big reason for us is that DSP offers advanced bidding possibilities so they don’t really beat per publisher, they beat per device, so they know so much more about the user because they’ve seen this user maybe engaging with their ads in the past so they can categorise whether or not this user is available but also there are a few more reasons. For example, in our vertical, so we do social casino games, not in all the countries Google is available for us – some examples of those countries are Poland and Russia where we just cannot buy Google inventory for social casino games. So, if we want to scale in those countries, we actually have to work with more partners and DSP is a big bucket for us.
And in many ways it’s the same thing that we’re hearing – you said the duopoly, so Google, Facebook – they have AI, they have machine learning but so do DSPs. Is it, before I ask you exactly how to teach the algorithm of DSP, is it really markedly different, is it a very different approach, is their AI, I guess the question is, up to speed?
Yes, so well it’s quite complex question actually. I would divide DSP into two big categories. There are self-managed DSPs and managed DSPs. So if you work with managed DSPs, they usually are big companies like for example Liftoff who would require quite big minimal budget to start with, so they would usually work with a big company specifically in the gaming space but not only, and they would actually make sure that you reach your KPIs. So they have their own beta, they have their own algorithm that is optimising activities for you.
However, if you want to move in 2019/2020 with self-service DSPs, then you actually have to have a combination of a creative team and AI team to make sure that you understand how to best beat on your potential user.
And do you have a mix, a team to cover this or are you focused more on the managed DSP?
So, we focus on the managed DSP as for now, we have I would say ambition to take it in-house one day. However, we do know that it’s almost impossible not because managed DSPs are, you know, ruling the world but more that’s why they are good at it – they have been doing this for the past years, maybe six, seven years, and they have a lot of other advertisers, not only Huuuge Games, but they have first of all, all of our direct competitors, they know all the knowhow, so by the time we will figure out how to do things that they are doing, it will take us another few years.
However, we will keep on trying and probably one day we will take some of their activities in-house but I cannot say if that will be this year or next year or maybe in five years from now.
I’m just curious, when you’re working with a DSP, is it you need perhaps a different team or a larger team or different skills in your team or is it really just UA as usual?
I would say you don’t need to have different team, you need to have different understanding and you need to optimise towards different KPIs. So, first of all, our KPIs are the same for all of our partners, we look at the ROI in different windows, so day 7, 30 etc. But if you let’s say manage campaigns on Facebook, you can probably and pause and pause campaigns a few times a day or at least a few times a week and upload creatives, upload new targeting very similar with some of the other partners. For example, with SDK networks where you beat per publisher but if you work with a managed DSP, you cannot do very drastic moves because it’s all about teaching algorithms, right? So, if algorithm is trying to find the best users for us, we will do 300 changes per day, you will be confused, you will not even know what kind of users we’re looking for or to acquire. So, it’s more about the availability and the amount of budget that you have spare to spend that it will allow you to actually optimise and to play around with.
I mean, we’re going to be talking about, you know, what to look for in a DSP and also how to teach the algorithm – we’ll touch upon that in a moment but I’m just wondering background here, Misha, is this something that you’ve been doing for a while, is this a passion, is this something for you as a growth marketer where you’re saying, Hey, I decided early on I was going to crack the code on this because this is where some hidden value is?
Yes, so probably yes, I think in order to answer this question, we’ll have to look a few years in the past when Huuuge Casino as a company and where I was working at was just a newcomer to the social casino world and social casino is a quite competitive market – it’s quite, there are very few players who are leading it and we were, let’s say, latecomers, so first of all we came in 2015 with new games and as an innovative company we actually were very eager to implement all available strategies that are available and at that time, like, programmatic DSP on the mobile was just scaling up and we didn’t think very long, we just start implementing it and a few of the partners that we started working three years ago, we’re still working at a very big scale.
So while some of our competitors take a while to think what their strategy should be, we just keep doing it, keep trying and keep testing new ideas.
And that’s why I’m very eager to hear what you found out because obviously tried, tested, you’re passionate about your work, you hear that Misha. We do have to go to a break but listeners don’t go away because when we get back, we will be talking about how you go about teaching DSP algorithms to get the most bang for your buck, so don’t go away, we’ll be right back.
And we’re back to Mobile Presence. I’m your host, Peggy Anne Salz with Mobile Groove and our guest today is Misha Syrotiuk, he is Head of Ad Networks and Programmatic Buying at Huuuge Games. Misha, before the break, we were talking about why you need to interact with DSPs, the benefits, it’s not a duopoly, never was really but now definitely not, and there was a lot of reason to work with DSPs, there’s a lot of reason to teach the algorithm to give you better results. How do you go about that? I mean, is there some sort of, I wouldn’t say what you need to do and what you don’t need to do, but how do you start, I guess is the question?
Yes, so I would probably focus on an example, so as I mentioned already in the previous questions, we optimise the words ROI, so ROI comes after we already acquire the player, the player becomes a payer and the payer pays money in the form of in-app purchases. So, it takes a while, it takes a few days, weeks, sometimes even months for the player to become a payer. So in our example, when we start with a brand new DSP or we start promoting the new game, we actually tell DSP our CPI goal because it’s easier for them to convert impression of that ad to install than after a few weeks or a few days of collecting data, DSP already see what’s the conversion like, what’s the conversion from first of all impression to install and then from install to payer? And after a few weeks, in an ideal world, DSP can shift the algorithm from optimised into words install, to words optimised into payer itself.
There are of course a few examples where it’s a little bit accelerated but on the average, this is how it looks like. But then there are a few exceptions. For example, there are some countries where there are just not enough payers, such a country could be Austria where it’s a good country for us to target but there are just not enough payers for DSP to even start learning. So what we try doing there, we try optimising into words proxy event. As an example, it could be let’s say you’ve defined that 17% of your payers reaching level X at the game, so instead of actually optimising towards payer himself, we can start optimising in towards this event, then we’ll be as a proxy towards our main goal.
I mean, that’s a smart thing to do because you’re saying, okay, this event is characteristic of someone who’s going to pay in my app, so you don’t say, you know, this particular demographic or what have you, you optimise to the event because the event is going to net you people who behave in that way. That’s what I’m hearing here, correct?
Yes, yes, exactly.
So, picking up the events, that’s something I go to conferences a lot and people are like “pick out the right post install event” and it sounds like it would be logical but do you have some tips for that, because it’s not always straightforward, it seems to be like pick the event that makes the most money sort of but maybe not always. What are the rules there?
Yes, so the event I think a lot of games have events in the game so the event could be, let’s say, measured in absolutely number, could be level 50, level 100, and that is probably the easiest way to do. We have to work here with a BIT. Now what we also had an idea to test and we tested it would be not only just the event itself, let’s say it’s level 50, but also how fast the user will reach that event because if user plays the game and reaches level 50 in two months, that’s a different kind of user that reaches level 50 in two days, right? So, the user who will reach level 50 in two days most likely will be highly engaged, will probably have to do in-app purchases in order to be progressing that fast. So, you can combine actually these two metrics, you can work with the product team and create an event, it could be let’s say level 50 within two days and ask your DSP to optimise towards that.
It’s quite, let’s say unique way to do it and you probably want to try it out on the big markets not like Austria where you probably will not have many signals, but one of the examples of this advanced level optimisation would be this approach.
And I’m just curious, how does the communication or what is the ideal communication on the DSP because you’re teaching the algorithm and you’re setting up what you need so the algorithm learns, but you also want a feedback loop, you also want to say, “Hey, DSP, Liftoff, whomever, you’re doing a great job, this is working, we’re all optimising to the right outcomes”. What’s the best way to do that? Do you like email all of them or Skype them, how do they know they’re doing well so then they are on the right track and continue to do well, or is there absolutely no feedback loop, it’s just learning by doing, you don’t need to tell anyone, keep that algorithm on that track I guess is the question.
Yes, I guess in an ideal world, we would not have to tell each other what to do because everything would be super-transparent, we would see cost and all the margins and all the ROI data and AI will optimise. But we’re not...
We’re not there yet...
Right, yes, we’re not there yet, we probably will go there one day but since we’re not there yet and we like to talk to each other, we do write email, we Skype a lot, but we don’t take a drastic action. So, as mentioned in the previous conversation, that with Facebook campaign optimisation, we can just turn on off campaigns on a daily basis. With a managed DSPs, we probably don’t want to do that on a daily basis, not even weekly basis, if it doesn’t work one or two weeks, we probably would lower daily cap, we probably would lower weekly cap or raise it or do some creative changes or targeting changes but we wouldn’t do drastic changes.
There is actually one example that I want to share is that ideally managed DSP should optimise towards your ROI from day one and we had an example with one of our partners that we started working on UA for our new game where we saw an example of three publishers that delivered over 200 installs over the period of 7 days and had zero ROI. In ideal world, again, this shouldn’t happen because the algorithm should shut down delivery on those publishers and even you should not optimise on publisher level – you should not blacklist this publisher at all but since it was a brand new game and the algorithm was still learning, we wanted to insert a little bit of manual input so we asked our partner to actually blacklist those publishers specifically for this period of time after already a few months of working together, we would not actually request such an action.
I mean, that is, I can understand that but there are also some times where, as you said, you don’t want to blacklist the publisher really, correct? You want to learn from the data, you want to sort of let it go but I guess in this case, what was the lesson from this for you?
Yes, so the lesson from this was that the algorithm was optimising towards CPI, towards the cheapest cost per install because it was the first signal that it would get. So if the creative is converted well, so the conversion rate is high, the CPI is lower, theoretically it works towards the good direction, well, theoretically, but practically it wasn’t. So we actually wanted to help the algorithm to understand that cheap CPI or cheap cost per install is not the only thing we are looking at, we actually are not even looking at that number that much, we are looking at the ROI. But of course the cheaper user acquisition costs you will have, the more likely your ROI will be higher.
Yes, it’s always that case of, you know, you get what you pay for, there was a day, and you were there in the industry as well early on, it was like, “Yes, CPI, CPI is cheap, life is good”. And now we know that’s not the case and that’s why we have to do this in the first place. So, do you see that teaching DSPs is going to be sort of like your ongoing activity, I guess it’s one of those things where the smarter they get, the better it gets, the better you get and then teach them some more. I don’t see that there’s really an end to what you’ve made from your hobby to your work.
Yes, I think, well, first of all, teaching, yes, is part one of the job but I think as 2019/2020 will be going on, there will be other things we will be doing. So not just teaching DSPs but also working on the new creative formats and integrating the new ads inventory and going into new spaces that are not available for us yet.
So, lots of reasons to bring you back more than once this year for sure, Misha, as you experiment and progress. Right now we do have to go to a break, but when we get back, listeners, we’ll be talking about what you need to look for in a DSP and what you need to avoid. So, lots of green flags, lots of red flags, you need to know and follow, so don’t go away, we’ll be right back.
And we’re back to Mobile Presence. I’m your host, Peggy Anne Salz with Mobile Groove and of course we have today Misha Syrotiuk, he is Head of Ad Networks and Programmatic Buying at Huuuge Games. Misha, it’s been a great show so far and we’ve got to wrap it up with more of your wisdom about DSPs. You’ve obviously been working at this, you’ve been crafting an approach that gets you the most out of your effort but how do you know at the first moment, you know, what to watch for in a DSP because they’re not all the same and there are some things to avoid. So, I guess the question, a green flag, what tells you I need to look at this DSP, or what a marketer needs to look for?
and there are some things to avoid. So, I guess the question, a green flag, what tells you I need to look at this DSP, or what a marketer needs to look for?
Yes, yes, So, I think before even you start working with DSP, it’s a good question of where you even – where do you look at the full list of the partners you should work with? So, we work with probably 5 managed DSPs at once as of today and we probably will test a few more, so on average we test one or two DSP per quarter, it’s quite a lot, and why would we even do that? There are a lot of reasons but one of them is actually we are moving and we want to work with companies who are also new and who are moving with us in the same direction.
So, one of the ways to actually find out whom you should work with is to check AppsFlyer Index that tests the MMP we are working with and actually the recent one was just in the middle of March. So, probably with the majority of those names, we’ve worked or are working or will be working and then once you actually do work or once you are considering whether you should work with them, you should probably focus on a few of the green and the red areas.
So, first of all, maybe ask them, not maybe, but ask them actually for the realtime transparency so speaking about on all level publishers, creatives, exchanges and ideally in the realtime when you can use the macro so if you’re tracking for different mention actually transparency, then number two because DSP buys on impression level, the very important metric here is actually how well your creative perform, therefore both in-house and external creatives teams should be aligned whether it’s playable or dynamic end cart of static or video, so the creative should be really catchy and I guess the third green side of this story would be to check both short and long-term results on ROI. So, for our games specifically, they are quite aligned but there are certain games or certain DSPs where maybe short-term ROI would actually be lower than you expect but on the long-term, let’s say day 60 or day 90, would actually back up for you.
And speaking about the red flags...
I want to talk about your green flags for a moment, because that’s a very good point... the AppsFlyer Index, we had AppsFlyer here on the show, I read the Index, just came out again, a good source of names, also ones below the radar which is very cool, it’s not always the usual suspects, so that’s all the green flags. So let’s end with one big red flag that our listeners need to watch and avoid.
So, I would say the big red flag is that do not tolerate fraud behaviour, we haven’t talked much about fraud and fraud is everywhere, even on Facebook and Google and as well on programmatic DSP. Fortunately, DSPs are not trying to sell you fraud but they are buying these sometimes from ad exchanges which didn’t have very well sophisticated anti-fraud systems, therefore if you do spot it, if your BI systems spot it, it exists, do not tolerate it, act fast and ask for a refund or open negotiation with your partner, but make sure you watch out for this as well.
That’s a very good point and also because a lot of people say, “Oh, I can tolerate a certain amount of fraud”, or “I don’t really want to get into the hassle of asking for a refund” so good point there, Misha, you do have to open up, you do have to say “I will not tolerate this” and do something about it. All the reason, since we’ve run out of time, Misha, all the more reason to have you back again at some point, maybe just telling us how you fight fraud at Huuuge Games. But in the meantime, how can our listeners keep up to date with you, maybe you’re out there doing things in social media, a blog, a website where they can keep up with you. Of course, they can also check you out on your Mobile Heroes page and we’ll get to that in a moment, but where else can they do that?
Yes, so I'm trying to be present on the few industry events per year, the majority of them in Europe but some of them are in US as well. But speaking about the digital connections, so my email is email@example.com so I will reply to all the emails and LinkedIn is one of the best methods to also get in touch.
Excellent, and we include those of course in the show notes with every episode. And of course as I said, we can check out your page or any of the other Mobile Heroes in the series-dedicated web page over at heroes.liftoff.io. And if you want to keep up with me throughout the week or find out more about how you can be a guest or sponsor on Mobile Presence, then you can email me, firstname.lastname@example.org, Mobile Groove is also where you can find my portfolio of content marketing services.
And that, my friends, is a wrap of yet another episode of Mobile Presence. Time does fly, enjoying it so much, so be sure to check out this and all earlier episodes of our show by going to webmasterradio.fm or you can find our shows on iTunes, Stitcher, Spreaker, Spotify and iheartRadio simply by searching Mobile Presence. So until next time remember, every minute is mobile, so make every minute count. We’ll see you soon.