Advertising & Brand

Learning AR Storytelling From USA Today Network To Bring Audiences Deeper Into Your Content

48 min read

AR is going mainstream, enabling brands and marketers to tell amazing and immersive stories, so why wait to test the waters? We learn how to choose and tell stories with the help AR from Ray Soto, Director of Emerging Tech for the USA TODAY NETWORK, part of Gannett Co, Inc. and the largest local-to-national media organization in the U.S. His primary role within the organization is to revolutionize storytelling with emerging technologies, such as AR and VR, and we learn how he did just that bringing audiences deeper into apps and experiences including the launch of a Falcon 9 rocket from Kennedy Space Center and an investigative podcast revealing the inner workings of America’s cities.

Transcript

Hello and welcome to Mobile Presence.  I’m your host, Peggy Anne Salz, from Mobile Groove, where I plan, produce and promote content that allows my clients to reach performance goals and scale growth.  And growth is of course, it’s about effective mobile marketing but it’s also more and more about amazing storytelling and, you know, in essence the pros at storytelling are the media companies and the apps, and so if you want to understand how to engage your audiences with content, we’re going to go to the source, we’re going to deep dive into examples that you can draw from as you seek to use AR, that is augmented reality, to bring you audiences deeper into your app or your content experience.

And as I said, we are going to the source and I’m excited to welcome Ray Soto, Director of Emerging Technology for USA Today Network.  Ray, great to have you on Mobile Presence today.

Hi, Peggy, it’s great to connect again.

Absolutely, I mean, I have to say full disclosure, we already did an interview, I already did an article, I’ll include those in the show notes for this show as well where we talked about how you are leveraging AR to elevate the storytelling experience, I mean, probably quite literally, Ray, because if I think about it, the fantastic app content experience we’re talking about is launching a Falcon 9 rocket, so maybe you want to tell me about that particular experience.

Yes, absolutely.  You know, kind of to lead off with a 321 Launch app, building a Falcon 9 and launching it was – started off with a really basic idea and at the time when our space experts, Florida Today, had reached out to us, they kind of pitched an idea of creating a companion app for folks who were at Cape Canaveral, you know, watching the rockets lift off from Kennedy Space Centre and we were interested in seeing if we could explore opportunities to make it seem as if users were actually in control of these rockets, as if they were Mission Control launching, stage operations, even landing them.

But where the opportunities for us really came through is we wanted to move beyond the gimmick and understood how augmented reality can create a story aspect to it.  We recognized quickly that there was an opportunity for users to feel as if, you know, not only that they were in control but that they have a better understanding of the small nuances, terms that might be said during a broadcast, not really understanding what that might be and through AR and through interactivity and that sense of control, we’re able to create something that’s fairly unique.

So, the experience within the 321 Launch app is fairly unique.  While the feature of it is a live AR broadcast, we wanted to incorporate something that users felt as if they can engage with a content during non-launch days so we created a small segment in there, an interactive story that allowed users to build the Falcon 9, drag it onto the actual launch pad and go through the entire sequence from liftoff back to landing.  But what’s great about that experience is we incorporated content annotations to learn a little bit more about some of the, let’s say the launch facilities themselves.  Such as the water towers, we didn’t know this until we were actually down at NASA that those water towers aren’t actually to cool the concrete from the heat but more just to dampen the sound because the vibrations are so strong it could actually crack the foundations of these massive facilities.

So, just those small types of aspects of being able to interact and learn a little bit more and there’s quite a bit more within there but that’s just one small example.  But the main feature of this app, and one thing that we’re really excited about that we were able to create was a live launch experience and space fans, they’re not unique to just Florida, it’s a global audience, everyone’s really excited about launching the space missions and just following them along.

So, through AR, what we’ve done is for users who have the app, they receive a notification saying the launch is about to happen in let’s say an hour, half an hour, and kind of leading up to just a few minutes before launch, users can log in through the app and see a virtual 3D object of a Falcon 9 just sitting idle on the launch pad, on your desktop, and we’ll have audio that’s filtered in so you can really understand exactly where it’s at within the launch sequence and you’ll be able to see the rocket launch in realtime in perfect sequence with the actual rocket that’s taking off at Kennedy Space Centre.  And it’s pretty powerful to kind of check that out.

What’s also really interesting about the experience too is our space experts at Florida Today, they are also broadcasting a live scribble feed so folks can tune in and ask questions and they’ll answer them in realtime, so you really feel as if you’re watching this thing, it’s your own personal experience and it was something that was fairly unique and something we’re able to pull off which was fun considering it was our first time working in augmented reality.

I mean, that is really cool because it’s about bringing the experience to life, you know, engaging – I love the idea also that you can engage before but also after, not just before or during but after the launched to keep that experience going.  I want to get into some of the ideas that you have in mind when you’re choosing a story, when you’re constructing the storytelling but I’m just curious first of all how the audience reacted?  I’m into this topic so you’ve got me, I’m a convert but what was the engagement of the audience or what did they enjoy about it, what was the feedback there?

The audience reaction to this was actually much stronger than we had anticipated.  The 321 Launch app was mostly just meant to be a pilot, a test of what we can do in AR to see what audiences would engage with, how long they would engage with the app and really just kind of figure out how we can fit it into our workflows as applied to the storytelling.  And as soon as we’d released the app and fortunately we were featured by Apple in the App Store, it exceeded all our expectations from downloads numbers to retention rates to engagement lengths.  We just didn’t expect it and what that told us immediately is that there an audience that is not just interested in the content but there are audiences that are tech-savvy and understand the opportunities and kind of the unique nature of AR through this type of content, storytelling most specifically, and the live component was pretty unique as well.

But, folks were blasting out on Twitter and on Facebook, “Hey, check this out” and we saw some really strong numbers which surprised us, so much that it actually crashed the app just because we weren’t anticipating it so our first live launch as four days after the app was published and as soon as it went live, we started to see a nice little spike in audiences engaging and all of a sudden it hit a really strong spike and none of the live features were working so we were scrambling like crazy trying to fix it so while there was that bit of excitement, it was at the same time, “Okay, there is an audience that’s out there” and we started to notice kind of strong numbers through each of the live launches but also a nice steady stream of users who would still go back into the app to experience the non-live interactive as well.

I mean, that’s one of the reasons I have you here, Ray, is because your example proves you do it right and you can hit the right metrics.  I mean, you’ve got long session lengths, you’ve got long session times, rather, you’ve got strong retention rates, it tells us that if you approach this properly, the audience is there.  So, I guess the first point is AR, it’s not just the early days we’re hearing about it, it’s really ready for maybe not primetime but definitely for other companies out there, app developers, marketers, to think about seriously integrating this into a strategy to engage audiences.  What would you say – AR – are we ready?

I’d say we’re ready and now is the time to get into it.  What I really like about where this platform is currently at – so, it’s been great about where the functionality is now is all the tools are available, whether it’s working through unity, there’s a lot of APIs that are available.  The process to actually start experimenting and really understanding the opportunities are available to anybody who’s really interested.  So, getting in now is an opportunity for folks to really figure things out and how it applies to possibly their business or their brand and in our case how we applied to storytelling, but essentially what it does is the resources are available and there’s a community out there that’s really excited about this.

I think of where we’re at with augmented reality now is similar to where virtual reality was about five years ago, its experiments and lots of excitement around it but what’s great about the platform now is the fact that with native integration into phones, users are ready for this, they understand it and they’re really looking forward and experimenting – not experimenting but let’s say experiencing as much as they can.

And also to your point, the audience is there now, you have a lucky point in your favor, you have a lot of people from video games on your team and you yourself, you’re a passionata, you love it as well, I feel it when I interview you both times.  Just in a nutshell, can a company do this or does a company need to specifically tap video games programmers for this opportunity?

I don’t think that companies need to have a team of former video game developers, it’s definitely helpful because all the tools are very similar to what the games industry has been using for years.  I mentioned earlier, the resources are available and the tools are intuitive enough to at least get you started and through 3D modelling, let’s say for example that is helpful as well but I don’t think that a company or a team really needs to be starting to build up a game development studio just to support this, just get in early, experiment – it is helpful but I don’t think it’s required.

Okay, so there we have it, we have a good set-up here to go into break, we’re hearing that AR is ready, tools are ready, audience is eager and ready so everything spelling opportunity for AR so don’t go away, listeners, when we get back, we’ll be talking with Ray about now how can you really get involved and map out an AR strategy to bring your stories to life.  So, don’t go away, we’ll be right back.

And we’re back to Mobile Presence.  I’m your host, Peggy Anne Salz with Mobile Groove and we have today Ray Soto, Director of Emerging Technology for USA Today Network.  And Ray, right before the break, we established AR is ready for primetime, get in there, but it’s also about making the match between your story and the audience and avoiding gimmicks, I would imagine.  Is that what you would say should be top of mind when you’re looking at this, just trying to get away from the wow factor or should we embrace the wow factor?

You know, we have to move away from the wow factor.  Audiences do recognise that while this might be a fun little experiment, let’s say very technology forward, that unless there’s a story and something really engages the user, they’re not going to return this and they can see right through it so it is a challenge but we have to move beyond the gimmick when it comes to AR.

So, I’m not quite certain what the story would be, you always think it has to be something really cool, you know, watching the launch of a Falcon 9 rocket, getting involved in that, that’s pretty cool.  You’ve also done one Life on the USSR Eisenhower so you get the feeling it has to be something that you experience.  Could you take stories that maybe don’t have that much experience and moving parts to them to AR?  Is there a rule, this one goes, this one doesn’t?

Yes, we follow several different rules and this is what I encourage folks to consider if they’re thinking about building out an emerging tech story through AR.  Really good examples for us would be we created an AR interactive with a podcast team to promote The City which is a new series that came out I’d say back in September and we were having discussions with the podcast team – when you consider that is very much an audio-first experience, we wanted to answer the question that folks would still have after listening to the entire series and that is what does this place look like.  To provide some specifics around it without getting too in depth, the story that we were looking to tell was provide visuals of football fields long, you know, dump in a neighbourhood just south of Chicago, but that dump affected the communities, whether it’s the schools, the neighborhoods and the homes, the businesses, cracked foundations, higher rates of asthma.

You know, it’s a very powerful story and it’s not what you might think of when it comes to AR or even VR, but there was an opportunity to really show users the proximity, the scale of this mountain of rubble and through interactivity, we kind of guided users through that story so they have an understanding of the impact of this dump within the community.  And we leveraged it not as a standalone story but as a teaser for the podcast so it was to get users in, have them understand what the story is and guide them to the podcast, so that’s just one small example beyond the wow factor and beyond the gimmick of what you could do with virtual reality and AR.

Speaking to let’s say VR specifically, we worked on a piece a couple of years ago as part of The Wall feature story project and through VR and the conversations we were having with the editorial team, again, it’s just answering those questions of what can we provide that is unique to the story that makes best use of the platform and in VR’s case it’s a sense of presence, being immersed within an environment.  And for The Wall, it was the story of the impact of Trump’s proposed water wall from coast to coast and we wanted to get users into those locations, we rounded up four unique locations along the border and through there we were able to allow the user to feel as if they’re standing within there, they feel immersed through the spatial audio, they’re being guided within through one of the reporters speaking to the impact that this particular spot will have should the wall be built, whether it was through micro crossings to let’s say the species and migration routes and eminent domain.  It’s again not what you might think of when it comes to virtual reality but it’s really thinking through the platform and what its strengths are to enhance the story experience.

And that’s really the approach that we take as well for every story that we move forward with and it’s what I encourage folks to really consider before they even get into it.  Start to concept out ideas, start to really think through your story and how you might be able to leverage the platforms to inform a user but at the same time make them feel as if they were either part of the story but more importantly walk away having learned something they would want to possibly reengage with it.

I mean, that’s inspiring and I like the way you just say it’s a sense of presence and ironically you augment a podcast with augmented reality so I think it challenges us to think outside the box, to push the envelope.  Let’s say that because it’s Mobile Presence, we’ve got a lot of marketers, a lot of app marketers, people out there – we’ve talked about what you need to have as a mindset to use AR to really bring a content to life, give a sense of presence.  What about the team and the talent, we said you don’t necessarily need video game developers but what kind of teams should they be assembling to put this together from ideation to production?  Is there something special in that mix, something you can share?

I would highly recommend for anyone who’s looking to build a team, to start creating these interactive experiences, a UX team, design, interaction – that is common across most visual types of platforms but it’s really, really important for VR and AR.  You have to have someone who really understands the entire user flow, some really good examples there is unless you engage the user from the onset, they might not go into the story itself and we had a couple of examples that we’re released that just didn’t work out well because we didn’t think through UX and onboarding more specifically.  Users really didn’t understand what it is that we were doing, they didn’t understand what the point of the YAR to support this or even just have to start the experience.

You need to have folks who really understand how to build that user flow, design is pretty common across most platforms, there are some nuances here and there but you really have to have someone from the UX perspective to kind of guide that conversation and can mold the story, someone who really understands interactive storytelling as well is nice to have.

When you combine some of those in the visuals, I think things start to get crafted around there but really it’s just think through creative brainstorming, what is it that we do – build out the entire userflow, experiment and then from there start to build out your prototype and see what works and what doesn’t work.

So my recommendation would be, think through building a team similar to let’s say interactive web experience, start to build out the UX, start to build out the story, start to build out from there the entire flow to really understand are we going in the right direction, are we telling the story that we’re looking to tell and I think that’s really all you need to kind of get started.

That’s really helpful advice and to your point about the onboarding, that’s a big one because just even with just ordinary app experiences, you know, just cut and dried, no AR, no nothing, just the real how to get through the app to make an order or how to get to the app to engage with a game.  Onboarding is critical.  Anything you can give in the way of advice regarding onboarding for this because in a way you’re saying, yes, this is how you use it but I guess you’re also trying to say this is why you want to use it.

Yes, you really have to think through not just the core demographic of very tech-savvy folks who already understand AR but you have to think bigger.  The opportunities to engage a new audience and how can you create something that is as intuitive and as seamless as possible?

Now, with AR, there is a bit of a challenge, well, it is a fairly new platform, there are still nuances around let’s say surface detection, let’s say functionality when it comes to how to interact with objects.  You really have to think though how to guide a user on the onset more specifically on the onboarding where you’re not overwhelming them and I know it’s kind of just very high-level thoughts around how to approach it but if you start thinking through how to make this as easy as possible and really internal testing, get feedback on there, really understand are we heading in the right direction, does this make sense – you’re really setting yourselves up for success there.

The onboarding side of things is very, very difficult because at the same time you don’t want to surprise users as well.  If they’re going into an experience, in our case a story that we’re building with an expectation and we either don’t meet those expectations through the onboarding sequence or at the same time we lose them through that onboarding, we’ve failed.  And we had a couple of instances where things did not work out for us in the past but we’ve learned through that considerably, we’ve been building VR and AR – VR for about four years or so and AR for about a year – we stumbled a bit, we’re growing as the technology evolved but it’s through that onboarding sequence, through testing and really just kind of building that out so users have an understanding of what this is and what their engaging with.

You also said make certain that it resonates with the user which is really smart, that’s what you always have to do – except, in apps that’s like a soft launch period and you have a chance to sort of fly or fail.  I don’t know, focus groups, what are you suggesting here because yes, you do need feedback or maybe it’s something just to ask your friends.  Some people say you can ask a lot of friends and if they like it then you’ve got a good chance.  How do you go about collecting the feedback that’s going to tell you whether or not you need to really bring this experience to life and launch it?

Focus groups are very, very helpful, it’s honest feedback.  While I always appreciate feedback from friends, they’re telling me what I want to hear.

It’s going to be biased, they’re going to say what a great job you did.

But through focus groups, it’s putting together a series of prototypes.  Maybe it’s just two and testing kind of AB testing from there just to make sure you’re heading in the right direction.  At the same time, you’re going to get that unbiased feedback that’ll really guide development for the story or the app that you might be looking to build.  I cannot recommend that enough.

Also, while you’re building, internal feedback from let’s say someone in your office is also great.  Most of the stuff we’re working on has to move fairly rapidly so with 321 there was an opportunity to do some focus testing there.  For the stories that we’re building, we’re learning as we’re going along and we accept failures which is fine but if you have the opportunity to do focus group testing, absolutely go that way, you’re going to get that unbiased feedback that will really set you up for success.

Well, I’m enjoying this a lot, Ray, especially because you’re bringing what some people are still looking at as the next big thing, one of the 2019 trends, you’re making it understandable, the feeling that we can actually embrace it for storytelling going forward, which is exactly the idea here.  So, listeners, don’t go away because when we get back we’re going to talk about some top tips that you can do, a little bit of a crash course in how you can get started and make 2019 the year you bring your stories to life.

Hey, and we are back to Mobile Presence.  I’m your host, Peggy Anne Salz with Mobile Groove and we have today Ray Soto, Director of Emerging Technology for USA Today Network.  Ray, you’ve been there, done that, you’ve got not just one AR experience but a whole line-up of them and you’ve talked about some others so you’ve seen the accomplishments, what AR can do to elevate storytelling.  You’ve probably also seen what it can do to achieve basically the exact opposite and just be so gimmicky that storytelling suffers as a result.  What would you say based on your experience should you avoid?

Yes, definitely have quite a few experiences to share in which we’ve stumbled building out stories.  So, when it comes to VR and AR, what I’d recommend is just because you have the technology, you have the capabilities, don’t just throw everything together in the sense of trying to use it because you can to try to tell a story and more specifically using, let’s say, the Eisenhower VR experience we released just a few years ago – that was a kitchen sink project for us.  We understood that we could create 3D visuals, we understood that we could do strong 360 videos, interaction in UX but it fell flat and while visually the content was very thrilling, the user experience that I was speaking to earlier just wasn’t working.  We really needed to think through a way of creating a seamless experience that users felt as if they were in control and we didn’t do that.

If someone were to take a look at Eisenhower, it was more of an in and out experience where you have a hub and you go to a story and you go back to the hub.  It’s very confusing, it didn’t leverage the platform that it should be intended for.

So we learned from there and incorporated those changes into The Wall project, so we treated The Wall project more along the lines of a book into a series of chapters so users could feel as if, hey, maybe they don’t want to start at chapter one, they could start at chapter three – while there was a narrative flow, they were in complete control.

Now, with what we’ve learned through those two projects really guided us for AR – we understood that users need to feel as if they’re in control, we can’t be the gatekeepers of a story in the way they would want to explore the story.  So, in AR, with the stories that we’ve released over the past year, 321 was a great success for us when you consider the interactive story, we’re guiding the user but what we noticed through testing as well and the feedback we received, they were engaged, there were bits that users felt as if they were trying to skip past, let’s say for example users just wanted to launch the rocket, they didn’t want to go through the entire sequence of building it, dragging it out onto the launch pad.

So, through there and what we ended up learning, we made some adjustments here and there through the stories we released towards the end of the year and we really used two words to really guide everything we built.  We wanted to create an experience that would allow users to explore and discover and I can’t stress that enough.  Users really do appreciate through these technologies, VR and AR, as if they have complete control – they want to experience the story in a way that they want to and that’s where we’ve had to figure it out, it took us a few years and that’s what I would recommend folks to really consider when they’re building out their own AR experiences.

And we have the benefit now of your few years of experimentation, you know, the trial and error because I hadn’t thought of it that user control was so important but that is true because you have an experience and maybe people want to just dip in and out.  I mean, they can do that with on-demand streaming, they can do whatever they want whenever they want and then you have this big chunky AR experience which says no, you’re going to start here, you’re going to build the rocket, you’re going to drag it out on the launch pad – no, they might just want to launch it as you said and you have to build that in there and let them do that.  So, in a way, you enable experiences but you don’t dictate them.

Right, right.  And more recently we just released an AR experience featuring Washington Capitals hockey player and we had three 3D models within there built through Photogrammetry but someone says we forgot out best practices for this particular user experience – we didn’t provide a way for users to skip to the second, to skip to the third piece, to really get the entire story.  We were treating it almost as if there was a beginning, a middle and an end but the stories that we were telling within the experience allowed for more flexibility and that particular project was a bit of a challenge considering we only had about a week to develop the experience.

We hit all of our metrics that we were looking to gain but again from a story experience perspective from the usability side of things, we didn’t have that built to allow the users as has been mentioned before to explore and discover.

Well, great tips especially based on your practical experience, that’s always valuable.  I know that you’re out there speaking, Ray, I’m hoping sometime to hear you because I find it very engaging, I have to say, but how would our listeners stay in touch with you?  Are you writing, blogging out there elsewhere if they have questions or just want to find out more about the great apps that you’re producing over there at USA Today?

Yes, first and foremost, I highly recommend folks to download the USA Today app, we release AR experiences fairly frequently, we’ve got another one lined up that’ll be released towards the end of February that we’re really, really excited about.  But if folks are interested in following my work as part of this team, follow me on Twitter, @raysototech – follow me on LinkedIn, feel free to reach out to me and ask questions, I’m more than happy to answer any.

And that app, I mean, because there are many app sort of versions, not versions, but you want to be very clear about what you want to download so what is it called?  Is it just the USA Today app in iTunes and Google Play or does it have a special version for this experience?

It’s the USA Today app in the App Store and Google Play, yes.

Got it, okay, just wanted to be clear on that.  So, great, Ray, absolutely thank you for being on Mobile Presence and thank you, listeners, for listening in.  If you want to keep up with me throughout the week or find out more about how you can be a guest or sponsor on Mobile Presence, then you can email me, peggy@mobilegroove.com, Mobile Groove is also where you can find my portfolio of content marketing and app marketing services. 

And of course you can check out this and all earlier episodes of our show by going to webmasterradio.fm or you can find our shows on iTunes, Stitcher, Spreaker, Spotify and iheartRadio simply by searching Mobile Presence.  So friends until next time remember, every minute is mobile, so make every minute count.  We’ll see you soon.