March 23, 2023
Automotive companies have been on the cutting edge of XR adoption for nearly a decade now. As an industry, automotive is a perfect example of how AR/VR/MR (XR) can transform an entire sector. Here are the many ways car brands are using immersive technologies throughout the vehicle lifecycle:
*This is not an exhaustive list but rather highlights of the XR journeys of the following car brands.
Fiat
Back in 2014, Fiat created a VR experience for the 500x using both CGI and real footage (seems almost primitive now). Fiat took the experience on the road, offering consumers an Oculus Rift or Google Cardboard to virtually sit inside the car controlled by a magician named Dynamo.
Two years later, Fiat unveiled a prototype AR car sales app built by Accenture. The mobile app provided a glimpse into the future of car dealerships, allowing users to walk around, look inside, and configure a full-scale virtual car.
By 2020, Fiat was using VR for collaborative factory design, enabling engineers to manipulate the layout of a virtual plant in real time before it’s built. Teams could, for instance, move equipment around the plant and simulate new manufacturing processes. Fiat cited improved communication, reduced costs, and increased quality and safety among other benefits.
In December 2022, Fiat, Microsoft, and Touchcast launched the Fiat Metaverse Store accessible via VR headset, mobile device, or computer. “Inside” the virtual store, customers can experience, configure, and even purchase the 500 La Prima by Bocelli, as well as direct questions to an artificial Product Genius powered by popular AI tool ChatGPT. (Hard questions are still referred to a human.)
Ford
In 2015, Ford used VR along with infrared cameras and motion capture software to identify unergonomic assembly procedures. After piloting HoloLens in 2016 to reduce the number of clay models needed to fine tune designs, Ford made XR a regular part of the vehicle design process.
In the Ford Immersive Vehicle Environment Lab (FiVE), mixed reality helped improve communication between different disciplines and speed up concepting. In a dark room with ceiling-mounted motion capture cameras, a military-grade HMD, and a car seat, 88 people were able to evaluate a vehicle together at the same time.
In 2019, Ford tested Gravity Sketch’s Co-Creation feature for collaborating in the same virtual design space. Wearing VR headsets, designers could be thousands of miles apart and walk around the same 3D model, making changes in real time with hand-tracking controllers. Results included reduced travel, more human-centric designs, and accelerated vehicle development.
Ford worked with Bosch in 2020 to train technicians to service the Mustang Mach-E in VR without seeing a physical car; and when COVID hit, globally dispersed Ford designers took home their HTC VIVE headsets to continue working on team projects.
More recently, Ford’s Technical Assistance Center (TAC) has been using TeamViewer’s Frontline solution on RealWear headsets to provide remote support to dealership technicians. The American automaker has also forayed into AR marketing with a mobile AR experience to view the F-150 Lightning in your driveway and a widely viewed Ford Maverick TikTok campaign.
*Watch this case study by Michael Smith, Design Technology R&D Manager at Ford, from the 2022 Augmented Enterprise Summit
BMW
In 2014, BMW tested the original Google Glass for quality control of pre-series vehicles in one of its U.S. plants. The luxury automaker went on to embrace AR and VR in product development, production, training, marketing, and more.
By 2020, BMW was using VR for workstation planning, allowing planners in construction, plant engineering, logistics, and assembly to simulate new production areas and processes in a virtual plant before implementing.
BMW uses AR to provide training for engine assembly and other processes, inspect parts, and even assist drivers: Using computer vision and vehicle sensor data, the AR system for the BMW iX adds information like lane recommendations and parking rates over live video on the Central Information Display.
On the marketing side, BMW created an AR app to place and customize a virtual BMW i4 or iX in your driveway and an Nreal-powered showroom with a fully modeled BMW iX1 electric vehicle. More recently, BMW put VR headsets on customers, allowing them to test drive a physical BMW M2 through a futuristic virtual city. And at CES in January, the company debuted i Vision Dee, a concept that would turn your windshield into an AR display.
Perhaps most impressive is BMW’s use of NVIDIA’s Omniverse platform to simulate every aspect of its manufacturing operations. Just this week, BMW announced it's expanding its use of Omniverse for building industrial metaverse apps across its global production network. Among the benefits of digital twins, BMW reports a 30% reduction in production planning time.
Porsche
In 2017-2018, Porsche introduced “Tech Live Look” at U.S. dealerships. The program connects dealership technicians to remote experts via AR glasses, cutting service resolution times by up to 40%. Use of the solution tripled in 2020 when COVID made it impossible for Field Technical Experts to visit dealerships in person.
Porsche released a mobile AR app in 2021 letting users see “under the skin” of the new Taycan all-electric sports car and got into the metaverse spirit with the Porsche Vision Gran Turismo, a concept car designed solely for the virtual world. More recently, the brand worked with NVIDIA, Autodesk, Lenovo, and Varjo to bring the Mission R electric car, or rather its digital twin, to life.
*Watch this 2018 case study by Doug House, Manager for Technical Support at Porsche Cars North America, to learn more about how Tech Live Look came to be.
Audi
In 2018, five employees from different departments developed a low-code VR SDK with which Audi trainers could create their own VR training courses. Over eight months, the SDK was used to create 20+ courses for German Audi plants, including a pick-by-vision logistics course.
On the AR side, Audi adopted HoloLens 2 to plan complex logistics processes. The LayAR solution makes use of existing CAD data to help planners place virtual shelves, containers, etc. in a real factory. The luxury automaker has also used AR to present new products at various events.
In 2021, Audi used VR to plan the entire production of the e-tron GT at its Neckarsulm plant, meaning no physical prototypes. VR is now critical to decision making throughout the design and development of Audi vehicles, enabling remote collaboration, saving material resources, and getting Audi cars on the road faster.
In the last few years, Audi has focused on in-vehicle XR, debuting a heads-up AR display in the Q4 E-Tron in 2021, integrating the Holoride VR entertainment system in certain vehicle models in 2022, and just recently revealing a concept SUV that uses AR glasses in lieu of a screen display.
Nissan
In 2019, Nissan adopted HaptX Gloves to bring touch and input into the vehicle design process. Designers can touch and interact with 3D models via haptic feedback, further reducing the need for physical prototypes.
In 2021, the company employed HoloLens 2 at its Tochigi Plant for on-the-job assembly training via augmented work instructions. Nissan is also interested in improving safety and showing products with XR: In 2017, the automaker flirted with a Star Wars AR dealership experience. Flash forward to 2022: The company debuted its Tokyo gallery Nissan Crossing in VRChat, a virtual dealership where customers can explore vehicles and even test drive the Nissan Sakura in an island setting.
On the horizon is Nissan’s Invisible-to-Visible or I2V technology. Slated for debut in 2025, I2V uses sensors to display heads-up safety information to drivers in AR.
Toyota
Toyota has been using VR to evaluate vehicle ergonomics and validate designs faster, as well as HoloLens 2 for immersive training and on-the-job work instructions: Toyota technicians wear headsets to view wire harness diagrams, connect with remote experts, follow inspection workflows, and learn new car features.
The Japanese automaker also built virtual workspaces where employees in certain departments (ex. technical development, human resources) can participate in meetings and events as avatars.
Just last month, Toyota partnered with Yahoo to launch a mobile AR experience – part marketing campaign, part consumer research – allowing shoppers to view, customize, and test drive the 2023 Crown. The Japanese automaker had previously experimented with WebAR product launches and showrooms for both the Toyota and Lexus brands.
Mercedes-Benz
Last but not really last – there are over 60 automotive brands around the world, many (if not all) of which are exploring and/or using immersive tech – we have Mercedes-Benz.
In 2018, the iconic luxury automaker used HoloLens to show off potential applications to management, product, branding, and sales employees. That same year, the company launched the Mercedes cAR AR app (a sales configuration tool), rolled out in-car AR navigation, and replaced the traditional owner’s manual with an AR app for the 2018 E-Class and S-Class.
Mercedes went on to use HoloLens to connect diagnostic technicians at dealerships with remote specialists for faster service and maintenance. In the same vein, Mercedes’ Formula 1 team considered using TeamViewer’s AR platform to speed up problem solving at grand prix events.
Like BMW, Mercedes-Benz is pursuing AI-enabled digital twins, partnering with NVIDIA to digitize the entire automotive lifecycle as well as the ownership and automated driving experience. At CES last month, the company demonstrated how Omniverse can help to plan and operate its manufacturing facilities. Mercedes is also using the NVIDIA DRIVE Sim platform built on Omniverse to develop and test autonomous vehicles systems.
*Watch this 2021 case study about team collaboration in AR by Mercedes AR Specialist and Agile Coach Robert Dyhringer.
Main image source: Vecteezy
