Virtual and augmented reality are two of the most exciting technological developments of the last 20 years. Already widely used as a training tool in the military, healthcare, and professional sports, VR has many potential uses in a consumer-facing industry such as retail. Augmented reality is already starting up with fun extras, like augmented packaging, to practical uses, such as wayfinding through a store or checking to see how an item of furniture might look in your living room. As exciting as these technologies are, a lot of us don’t really have a great understanding of the full implications.
In this episode, we speak with Brian Banton, a designer whose work in virtual and augmented reality has been put to good use at Facebook, is going to talk us through what’s actually possible and how to plan to get there.
Transcript
Melinda: Hi, I’m Melinda, and you’re listening to Think Retail.
Virtual and augmented reality are two of the most exciting technological developments of the last 20 years. Already widely used as a training tool in the military, healthcare, and professional sports, VR has many potential uses in a consumer-facing industry such as retail. Augmented reality is already starting up with fun extras, like augmented packaging, to practical uses, such as wayfinding through a store or checking to see how an item of furniture might look in your living room. As exciting as these technologies are, a lot of us don’t really have a great understanding of the full implications.
Today my guest, Brian Banton, a designer whose work in virtual and augmented reality has been put to good use at Facebook, is going to talk us through what’s actually possible and how to plan to get there.
Brian, thank you so much for joining us, today.
Brian: Thank you for having me. I’m excited to be here.
Melinda: Awesome. Can you start us off by telling us a little bit about you, your journey as a designer, and how you ended up at Facebook?
Brian: Yeah, for sure. I mean, I started out studying traditional graphic design. I did a bachelor’s degree and a master’s degree in design here in Toronto. And after I graduated, I worked at a couple of studios, like, traditional graphic design studios where I’d work with clients in the education sector and the art sector. And it was really fun. I really enjoyed that part of my career, but I felt like Toronto, as a market for me at that time, was a little bit small. I was really excited about the idea of going to New York because it was, like, this big, cool design center. And I was lucky enough to land at Pentagram, which is a big branding firm. And they have offices all over the world.
So I was in the New York office. And, I was there, it was a really exciting time in my life and I worked there for actually only about a year because I realized that the scope of the work, as exciting as it was to work at Pentagram, the scope of the work was a little bit too narrow for me. It was essentially, like, design a logo, put it on a business card. And I felt like I wanted to branch out and do different kinds of work. So, I decided to take a leap into what’s called product design, which is like working with digital products like websites and mobile experiences. And I applied to a company called Frog, which was very well known for having designed the first Apple computers back in the ‘80s.
And their approach to design was quite different from what I had experienced before, they very much emphasized human-centered design. And so these processes were quite different. It was a really kind of big learning curve for me. But what was super interesting was I got to work with a variety of different types of people, strategists, UX designers, design technologists. And actually the first project I worked on was…it was an industrial design project for a digital jukebox. So this jukebox would go into bars, and it would have a digital interface and people could pick songs that they liked. But what was really interesting about it is that when we presented it, we presented it to the client in VR. And that was the first time at Frog that anyone had ever done anything like that. And it turned out to be a super successful presentation. And the client was, very, kind of, overwhelmed by the experience of being able to engage with this product we have designed in VR before we actually built it.
So that got me really interested in VR. And while I was at Frog, I started dabbling in augmented reality as well. And after about two and a half years of working at Frog, I just kind of randomly got recruited by Oculus. And I thought, like, this is an amazing opportunity. I love kind of exploring, you know, making things, making products, making experiences in a variety of different kind of media. And I was already excited about VR, so you know, I jumped at the opportunity to take that job.
Melinda: Great. So, I thought maybe it’d be fun to start off with something a little bit on the fun side. Can you tell me your favorite application of either AR or VR to date by a brand?
Brian: That’s a tough one for me. So I think brands, as I understand, like, kind of retail brands, I don’t think they have fully leveraged the technology yet. And I think that the most compelling experiences so far have been in the fields that you kind of mentioned at the top, like training, healthcare, sports, military, etc. But there are some, like, I think IKEA, their SPACE10 Innovation Lab is working on some really cool stuff. I think you’ve mentioned this, where you can drop furniture in your house to see what it looks like. And they’ve actually just released…I’m not sure if it’s in public yet, but they’re working on a newer version where you could actually, kind of, fill your whole house with furniture. I think the current version, you can just drop one piece of furniture. But now you can design your entire space with IKEA furniture in augmented reality. And I think that’s, like, super cool.
I think gaming has, like, a lot of cool things, like Half-Life: Alyx was, like, a watershed moment in AAA VR gaming. And that’s a super exciting experience in VR. There’s this app called Acute Art, which is a…it’s like a curated AR app for engaging with art. And there’s a particular artist, KAWS, that just sold an NTF for, like, millions of dollars or something. But he also has, like, an AR experience on Acute Art. And I really enjoyed that personally. It’s cute. It’s not, like, kind of earth-shattering, but it’s a fun experience.
Melinda: Right. I think in North America, especially, we’re a little behind the curve, because there have been some interesting things done in China, especially, but in North America, definitely the consumer facing applications are definitely not realized yet.
Brian: Yes, for sure. And I’ve also been in a bubble working in the Bay Area for a while. So I’ve been I’m mostly exposed to, like, what you can do in there. But I haven’t actually seen a ton of consumer-facing experiences outside of, you know, what I deal with day to day.
Melinda: So if we were to just, sort of, talk practically speaking and understanding that it’s not anywhere near where it could be, what are the most common consumer-facing uses that we’re currently seeing?
Brian: I think the most obvious one is the, like, face filters on Instagram and Snapchat. People engage with that, like, every day, and I’m not sure if people refer to it or understand it as augmented reality, but, like, when you kind of take a picture of yourself with a face filter on Instagram or Snapchat, that’s augmented reality. And that’s probably, like, the most…it’s almost ubiquitous experience of augmented reality. I think gaming is the big thing in virtual reality. Like, the kind of most dominant experience in virtual realities is playing games at 카지노사이트. But there’s also, like, home and fashion-related experiences. Like I mentioned before, there’s IKEA’s AR experience.
I just moved from the Bay Area back to the Toronto area. I kind of discovered in the move was that there’s VR experiences for real estate like the one on This Site. So, I was looking for a house in the city, and right now I live in Oshawa, and it’s an hour from the city, and it’s COVID, and you can’t visit houses that are on the market anyways. But I discovered that I can actually go into a house in virtual reality and look inside the house and see what it’s like, kind of feel the experience of being there without actually going there. I think it was a surprising but also super compelling, interesting use case for VR, for me.
And in terms of fashion, it’s a big thing with shoes right now. So there’s an app called Wanna Kicks. There’s others where you can actually try on sneakers, like Nike sneakers, and just kind of point your phone at your feet. And the sneakers kind of augment onto your feet. And you get a sense of what those shoes would look like if you were wearing them, which I think is kind of fun.
And then in addition to that, there’s lots of media stuff. So there’s, like, The New York Times creates content in augmented reality. So you can kind of click out of their stories. And if you like augmented reality experiences on your phone, I think they’re partnered with Spark AR at Facebook. And so they kind of augment their stories through these augmented reality experiences. And they also have VR experiences, where they have reporters embedded in, say, another continent, and they’re walking through a town that might be war-torn. And they’ll have 360 videos, so you can kind of be immersed in that experience.
Melinda: So if we were to talk about some of the misconceptions that you face when you’re talking about AR and VR to clients, or challenges that are maybe preventing us from getting to that kind of experience that’s possible, what are some of those common challenges that you see?
Brian: I think there’s a couple things. I think on the one side, especially with, I would say, with virtual reality, there’s kind of an aversion, I think, people have. I was working in the office down at Facebook campus in Menlo Park, and a friend of mine from Toronto came down and I was, like, kind of showing her what I did at work. And I put on the VR headset, and she was like, “You look like such a nerd right now.” And for me, it was weird because I’ve been immersed in this bubble where everyone’s always wearing headsets so it’s just, like, it’s normal, like, everyone just wears a headset. It doesn’t look weird. But she was like, “Oh, you look like such a nerd right now.” But then she tried on the headset and had the experience and she was like, “Wow, this is amazing.” So I think there’s like a little bit of a roadblock there, where people have an idea that it’s this nerdy thing that…you know, and you have this bulky headset on, it might look a little bit uncomfortable. And that’s something that might prevent people from trying the experience. But I find every time someone puts on a headset, they have…I mean, especially with the more modern headsets, they have a really compelling experience.
The other side of it is, like, kind of the opposite. There’s an aversion to it on one side, but on the other side there’s people that are really excited about it. And they think that it can just like do everything, and we’re kind of, like, early stages of AR/VR. And even when I started at Facebook, like, I’d be like, “Oh, we should have live streaming 360, 3D video so that my mom can see her grandkids in real-time.” And the engineers kind of rolled their eyes because it probably wasn’t the first time someone suggested something like that, that technologically wasn’t possible. It’s just, like, the kind of computer performance to do that on a headset, do kind of, like, really immersive 3D, 360 stuff wasn’t available yet.
And, I mean, at this point it’s starting to emerge. But people think, like, “Oh, we can, you know, we can just do everything.” And they want it to happen now. You know, we’re at a certain point right now where we can do some things, but not everything. And I’m working with a kind of institution of higher learning, and they’re talking about how to integrate VR into their education. And, you know, they’re suggesting, “Let’s have classes in VR,” which is a kind of interesting idea, but it’s just totally not feasible right now. And, you know, during the pandemic, when I was working with my colleagues at Facebook, we would kind of have meetings in VR, but they didn’t last long. Because it’s just, like, it’s not there yet, in terms of the experience,
Melinda: Right. That’s exactly what I was going to ask you, is, like, how big is that gap? But also, how quickly do you think that gap is going to close?
Brian: I think there’s a couple of things. Depending on what you do or what you want to do, the gap is bigger or smaller. And, like, once you’re familiar with what the constraints are, there’s a lot to work with. I think there’s two things in that gap, there’s the technology and then there’s, like, the user experience. In terms of the technology, I think, you know, the form factors of the devices, they’re still quite bulky. Most VR headsets, you have to connect to a computer, which is expensive. And then you’re kind of tethered to this computer, which is not necessarily ideal. There are the mobile versions of the headsets, like the Quest and the Quest 2, which are kind of great experiences. But they’re not perfect yet. The headsets are still a little bit uncomfortable. The compute power is maybe not at the place it should be.
So there are kind of technological developments that needs to be made. And then on the kind of user experience side, you know, one of the problems, or I wouldn’t say problems but challenges that we were facing at Facebook is that we’re creating unprecedented experiences. There’s not a lot of information on how people can navigate through a space in VR, or, like, kind of transport from one world into another world. Like, in theory and in movies, it’s fine, but actually making that experience, like an intuitive kind of, delightful experience is something that we’re continuously working on. So there’s a kind of a slight gap there. And, like, AR glasses are not really a thing yet.
And I think when AR glasses become a thing, then I think a lot of the tech companies think that we’re not going to have phones anymore. We’re going to just have AR glasses. But that’s, like, 10 years down the road, I would say. So I think that we’re making iterative improvements over the next decade. And I think a lot of the tech firms are betting that in, 10 years, maybe AR glasses will kind of take over, and phones won’t be a thing anymore. But I think they’re also, like, kind of baking in some kind of, you know, unforeseen development. There’s lots of R&D going on. And you can never predict when there’s going to be, like, this kind of quantum leap. But I think a lot of people are betting within the next 10 years.
Melinda: Right. So right now, if you’ve got some sort of virtual or augmented reality experience in-store that is enough on its own, it doesn’t even matter how good it is, it will draw people there. But as this becomes more common, the bar is going to rise. So if we’re just talking about, you know, something in the physical environment that is sort of a one-off experience, or even online, so if we’re at, like, a very, very sort of basic level right now, what would be the next step?
Brian: Well, I think it depends on how you look at the experience. So I’ve never experienced this, but Selfridges in London has an in-store AR experience. I saw the case study of it online. I think there was a firm, either based in New York or London, that did it. And it looks really cool. Just, like, for me, as a designer, I look at them, like, “Well, that’s really cool.” I’d love to kind of experience that thing. But, again, not having experienced it, only seeing the case study, I don’t know what, like, value it offers a customer, other than it looks really cool. And so I think the approach to leveraging AR and VR, I think, is to think about, like, what’s the value proposition? And what people problems are we trying to solve? That’s really going to open up use cases for having AR experiences in stores.
I think we’re at a point where there’s enough technology to kind of create those experiences. But we really got to think about, like, what value are we offering consumers. And that could be a number of things, it could be something really utilitarian, like if you go into a shoe store, and, you know, it’s a busy place, you know, there’s not enough kind of store clerks to help every customer that’s in the store, can the user just go up to the shoe that they like and have some kind of augmented experience that gives them all the information they need about the shoe? Like, this is for long-distance runners versus, like, medium-distance runners, or the sole has, like, arch support that you need, but all that is an augmented experience.
So you don’t need to wait for someone serving you to help you, which is kind of very utilitarian. But there’s also, you know, more emotive, like, storytelling use cases, where maybe someone really cares about the environment and they want to see that the shoe that they’re going to buy has a net-zero carbon footprint. And so if you have this augmented experience that kind of shows you the journey of how the shoe was created and, like, how it got to the store, you know, that might be a really innovative kind of experience for someone and kind of make them feel really good about buying the shoe. Those are just off the top of my head. My point really is, like, I think in terms of the technology, I think it’s…that’s not so much the problem of, like, creating a great in-store experience. I think it’s just really, like, what do we want to communicate? How do we want to help the consumer make the decisions that they need to make?
Melinda: Yeah, I think right now, people really don’t have a sense of how to integrate it based on a consumer need. It’s more of, like, this exciting thing that they’re using as a way to draw people. I think robots have really been…it’s a similar thing where, you know, it’s this exciting new thing, we’ve got Pepper the Robot, but it hasn’t really solved a consumer need. So after you’ve seen it once or twice, you’re kind of over it.
Brian: Yeah, for sure.
Melinda: Okay, so are there any unexpected uses that you would want to draw retailers’ attention to?
Brian: I mean, the experience that I had that was unexpected, which I mentioned above was being able to enter a space, like I’m looking for a house, and I can’t visit the house, because it’s either too far away or we’re in a pandemic. And being able to explore that house in VR, for me, it was a little surprising in a delightful way. I think the IKEA stuff is kind of cool. I mean, it’s not so surprising. It seems like such an obvious use case, but I think that for a retailer to be able to provide people with an experience to see the product without having to have it physically there, I think, is such a great use case.
And it relates back to the experience I had when I was at Frog, and we’re presenting this industrial design object to the client. It was really expensive to build a prototype of that. So we did it in VR and it was very cheap and very portable. We were in New York and the client came to New York, but then we also went to Montreal, because they’re based in Montreal, and we were able to give the company, other kind of executives at the company the same VR experience. So when you can give a customer an experience in their own place, or kind of, like, remotely, and it’s of a fidelity that really communicates what, as a brand, you want to communicate, I think that’s a really great opportunity for retailers.
Melinda: Interesting that you mentioned prototypes, because, in retail, the building of prototypes, when you’re redesigning a store, has really fallen to the wayside simply because it is so time-consuming and expensive. And we actually built a virtual prototype for a client to overcome this, one of our Chinese clients. Because, you know, for that exact reason, the cost of building the prototype and testing it with customers is just so exorbitant. But you could build a virtual prototype and have customers experience that and give you feedback and then make some changes based on the feedback that you get using a virtual prototype, which is just a fraction of the cost and the time. So interesting that you brought that up.
Brian: Yeah, totally. I mean, that’s not something that was on my train of thought right now, but it’s totally legitimate. It’s a really great use case. It’s, like, prototyping a space, especially in terms of building a retail experience or environment. Yeah, for sure, way cheaper, and the experience in VR is, like, the fidelity is just so great. Bang for your buck, you’re not going to find a better way to do that.
Melinda: Absolutely.
Brian: Another experience that is not strictly VR, but it could translate into VR is Balenciaga did…I believe they did their fall/winter 2021 campaign, kind of, fashion shows in a video game, and they put it online. The reason I bring it up is it’s, like, a video game. But the way that they built it, they used a game engine. It could easily translate into a VR experience. And I think, like, being able…for brands to be able to have their runway shows in VR, so that it’s accessible to anyone around the world, it’s another great way to connect with potential customers.
Melinda: Yeah, I’m really interested in how video games have started to become this alternative space. And it’s not really technically virtual or augmented reality, but it is a virtual environment where brands have been dabbling, and there have been, you know, music concerts and different types of events being held in Minecraft and Fortnite. And, you know, it is an interesting way for brands to engage, especially during a pandemic, but then also to be able to reach a wider audience.
Brian: Yeah, totally. I mean, we can call video games and AR, VR, like, simulations. And I think that Facebook, Oculus Quest 2 have sold far beyond what the expectations were of the company. And it’s becoming to the point where it’s, like, they’re trying to create a self-sustaining environment for VR. And I think as it becomes more mainstream, these kind of game experiences will translate into VR experiences. I personally think it’s inevitable.
Melinda: Yeah, I absolutely agree, especially as the cost for the technology decreases, everybody will want it. Can you just tell us a little bit about what, more specifically, what you were doing at Facebook?
Brian: Yeah, so I’ve kind of mentioned Facebook and Oculus a few times. I was a designer working on the Oculus VR platform. And I was there for about two and a half years…just over two and a half years. And I worked on the Oculus Quest. And I worked on the teams that launched the Oculus Quest and the Oculus Quest 2. And a lot of the work that I was doing was to try to help build the infrastructure of this new kind of experience for people. So, like I mentioned before, there’s not a lot of precedents or, like, kind of design patterns that exist for VR. So I was part of the team that was trying to think of, like, how are people going to use this? How can we bring value to users? And then how can we build an infrastructure that other people can create experiences, and put them on VR? And so a lot of work in terms of ergonomics, like design thinking, thinking about how interaction, like social interaction works, and even productivity.
Melinda: That sounds really, really exciting. But I understand that you’re moving on to a new opportunity. Can you tell us about that?
Brian: Yeah. So I moved back to the Toronto area this year. And I left Facebook, and I’m launching a studio, and it’s going to be called Plan of Record. And we’re going to work with brands and agencies to help leverage emerging technologies, such as AR and VR, to create compelling and delightful experiences for users and consumers.
Melinda: Super exciting. So if you could give our listeners your top three pieces of advice about how to approach or even just think about virtual and augmented reality, what would they be?
Brian: I think, you know, kind of what we talked about a little bit earlier. Think about it in terms of people’s problems, like, think about how can we leverage these technologies to solve people’s problems, to help people have better experiences, to help people be more informed, and be able to make better decisions about the stuff that they want to buy? I think starting now, there’s a great opportunity to start defining how, like, the AR/VR experience is going to fit into our lives and into our retail experience. Because there isn’t a dominant kind of experience with that right now. So the space is kind of ripe to define that experience. You know, it’s about connection. It’s about how we connect with people and, you know, connect to the things around us, and technology helps facilitate those connections.
When I first started at Facebook, Chris Cox, he’s head of product, he kind of greets all the newbies, as we’re called. And he gave this really great speech about the trajectory of connection, and how, like, over time, with technology, we are able to connect with more people faster to the point where it’s in real-time and at a higher fidelity. And AR/VR kind of facilitate that movement towards being able to connect with many people doing it in real-time, and definitely adding kind of a level of higher fidelity.
Melinda: Perfect. Well, thanks so much for chatting with us today.
Brian: Thank you for having me.
Melinda: If you want to understand what’s possible, look for inspiration in other fields, like gaming, museums, art gallery, and sports. Technology is quickly advancing. And while we can’t easily achieve best-in-class ideas yet, thinking ahead will allow you to act quickly when your dream plans do become achievable.
While we’re waiting, simpler applications–such as using a QR code on packaging to add storytelling or product information–is a relatively simple way to start experimenting with this kind of technology. And if you want to take it a step further, an in-store virtual experience, say, for a flagship, or allowing people to remotely shop the store through virtual reality, those kinds of things are possible today. Consumers are always looking for the next big thing, and when this technology becomes accessible, it will be a game-changer.
Thanks for listening to Think Retail.
About
Brian Banton is a multidisciplinary designer who has worked for the likes of SID LEE, Pentagram, Frog and Facebook. He is currently running his own agency, Plan of Record.
Think Retail is a podcast where top designers, strategists, thought leaders and business people discuss what’s coming next. For more information, email info@sld.com.
