Digging for Digital Transformation Using AR/VR & Wearables: The Sustainable Future of Mining

Despite advances in technology, mining operations today are fundamentally the same as they were half a century ago. Faced with increasing demand, diminished ore grades, less accessible deposits, and public pressure to be more environmentally and socially responsible, mining companies must develop new techniques by adopting emerging technologies to evolve their industry.


Trends and Pain Points in Mining

Bad Reputation

Mining has a less than stellar reputation when it comes to social and environmental impact. Despite technical advances and modern equipment, the mining industry as a whole has increased water consumption and is trailed by a legacy of poorly rehabilitated mines that have left behind chronic environmental problems like acid drainage. Mining companies can improve their image and build trust if they increase supply chain transparency and implement environmentally-sound practices that can stand up to regulatory pressure and the scrutiny of an increasingly aware consumer market. Firms must push innovation and R&D to find solutions (advancing techniques like biomining) to reduce their environmental footprint and mitigate the risk of large-scale incidents.

Material Resources

As the global population rises, so too does demand for minerals and metals. The depletion of near surface resources has pushed mining companies to look for lower-grade ores at much greater depths and to consider mine development in less politically stable areas. Extensive investment is required prior to mineral extraction from difficult-to-access deposits, and though there are innovative mining methods like block caving, high profitability must be assured before capital-intensive projects can proceed for the industry is highly risk-averse. Facing tight profit margins and buffeted by social and regulatory pressure, mining companies that streamline operations and develop new methods of mineral and metal extraction and processing will be able to meet demand and control costs.

Siloed Operations

The operations of a typical mining company are geographically dispersed. Valuable information is lost due to these operational silos, obstructing a company’s ability to coordinate and collaborate. Individual mines often operate with a high degree of independence and varied corporate structures limit centralized management, making it difficult to introduce disruptive technologies across an organization.

Safety & Labor Scarcity

Mines are busy, noisy and dangerous working environments. Workplace injuries are underreported globally and deaths not uncommon. Entering a mine can expose a miner to dust, gases, explosions, high heat, flooding, falling rocks, and cave-ins. Mines with the highest safety standards are not immune to these risks, but with proper precautions and investment in safety solutions and training, safety can be significantly improved. Compounding issues, the mining industry is suffering from a labor shortage. A culture of innovation, a renewed focus on safety, and the implementation of new technologies is key to recruitment and training of new workers.


Current State of Mining Technology

The mining industry is one of the least digitized in the world, with leadership that up until recent years hesitated to invest in any tech if a quantifiable, short-term return could not be guaranteed. Today, proven technologies that have been successfully implemented in other industries do not present the same level of risk. Drones and robots are being successfully introduced to mining operations to eliminate dangerous and monotonous jobs, and companies are investing in mine connectivity like leaky feeders or LoRA technologies to extend a signal deep underground. Some forward-thinking companies like Rio Tinto have begun to pursue digital transformation on a grand scale. Part of Rio Tinto’s ‘Mine of the Future’ program, for example, involves a massive investment to automate a mine’s supply chain from pit to port including an extensive rail network.


Potential for AR/VR and Wearables

In addition to drones and robots, technologies like augmented and virtual reality and wearable devices will optimize the productivity and safety of the mining workforce. Immersive and wearable technologies, whether worn within the mine or in a control center a continent away, can help users interact with remote colleagues and visualize and analyze data generated from sensors deep below the surface of the earth. Wearables can enhance real-time visibility into a mine’s operations, allowing for more effective and informed decision making; while simulating mine environments and interacting with asset data in AR/VR have a wide range of training and other applications.

Applications of Immersive and Wearable Tech in Mining

Exploration of New Mine Sites

Today, drones and UAVs are routinely used to study an area’s geology, producing 3D maps for general inspection. Drones themselves can even be operated via smart glasses (ex. Epson). The data gathered above and below ground forms the basis for digital models in virtual or mixed reality that can be used to perform safety inspections and maintenance assessments, for planning construction and environmental mitigation efforts, and to monitor inventory.

Before drones, workers typically performed surveying tasks by mounting high scaffolds, exposing themselves to great risk. Moreover, the information wasn’t always accurate. Drone mapping is cheaper, faster and more precise; and the information gathered – when put into AR/VR – allows for intuitive visualization and comprehension of the results of exploration, development drilling, geological models, and topography studies at scale. AR/VR also make for better remote collaboration and understanding among stakeholders such as surveyors, mining engineers and equipment operators, which speeds up decision making.

Few discoveries make it beyond feasibility studies to become an actual mining site, so it’s important to keep costs down and build an accurate model in a short period of time to get a comprehensive picture of the potential mine. A lot of time and money can be saved by not having to visit a mine site on foot, which eliminates risks associated with traversing difficult terrain in addition to travel expenses.

Development and Planning

AR/VR is a powerful visualization tool, making data easily accessible, engaging and meaningful to potential investors and other stakeholders. High-fidelity imaging of geological information, mine plans, geolocated borehole data, etc. can be modeled in AR/VR for easy, interactive analysis. Immersive simulations can also be used to show local community members the footprint of a planned mine throughout its development and operation, and how mine closure and post-mine closure activities will affect the area.

AR/VR, increasingly used by construction contractors to plan mines and discover design flaws before production begins, produces interactive 3D models that can be used throughout the life of a mine and integrated with an operation’s other digital assets for maintenance, training, etc. A full digital twin of a mine – uniting all mine assets via spatial data and other real-time information – allows for live monitoring and management of its vehicles, ore deposits, human workers, and machines; however, the use of digital twin technology requires a high level of digitization of the entire mining operation.

Safety

Future advances in automation may largely remove humans from the dangers of the most hazardous mines, but today’s miners are still at risk and require the most effective tools available for communication, health, and safety. Workers entering a mine today can be equipped with a range of wearable sensors and sensor-embedded protective equipment (PPE) that track their health and environment. Any device brought into a mine must be highly durable and able to perform in hazardous, wet environments as per industry regulations. Wearables might alert workers via sound, light or vibration to issues such as exposure to dangerous gases, seismic anomalies, and proximity to moving or malfunctioning equipment or vehicles. Currently in use are sensor-enabled safety helmets (ex. Jannetec), vests (Lightflex), shirts (Mitsufuji) and wristbands (Fatigue Science). These keep workers connected and alert to danger within and without their bodies, and in most cases can communicate with equipment and vehicles on site.

Wearable devices that track biometric information embedded with RFID technology can track a worker’s location, even detect falls and physical distress, which is key for lone workers. Sensors can track assets and people in real time, generating data that can be later analyzed to improve operations and the mine site itself. AR smart glasses (with appropriate safety ratings, of course) present another means of notifying workers about safety threats and even providing heads-up, hands-free safety protocols and directions.

There are now early-warning drowsiness detection systems like Optalert and other wearables designed to monitor a mining vehicle operator’s alertness in order to reduce fatigue-related incidents. AR glasses can eliminate an operator’s blind spots and minimize peripheral distractions like the complicated control panels inside the vehicle. Should an accident occur, someone wearing AR glasses could livestream the situation to an expert or supervisor, helping to treat the fallen worker before first responders arrive. AR/VR can also be used to train workers for hazardous environments, allowing them to gain experience without assuming the risk of practicing in a live environment.

Operations

Proper servicing and maintenance of mining equipment and vehicles can help avoid potentially catastrophic mechanical breakdowns in a mine. Of course, this is difficult with a shortage of highly-trained workers, but new fleets of connected mining machinery provide real-time diagnostic data allowing for predictive maintenance. A worker wearing AR glasses, even without a clear understanding of standard operating procedures or familiarity with the piece of equipment in question, can perform maintenance and repair with the assistance of a remote expert or vendor, remaining heads-up and hands-free the whole time. This reduces reliance on key personnel without impairing equipment output.

Mine suppliers like Caterpillar and Atlas Copco now market their machines and vehicles with VR training simulations and use the same tech to provide AR assistance for maintenance and repair. Miners can practice tasks in VR, tasks like performing an inspection on a Haulpak vehicle in a Mobile Maintenance Repair Workshop or performing a 3D scan of a physical pump for visualization; and then perform the same tasks in real life with prompts in a pair of smart glasses. Better maintenance, repair and overhaul practices with the aid of AR/VR will result in less equipment downtime, higher productivity, lower maintenance costs and, most importantly, improved safety for human operators.

Training

Virtual reality is an incredibly effective and efficient training tool especially for industrial workers because it allows trainees to gain experience without visiting a mine in person. Restrictive permit policies at some mines mean that employees can’t enter the mine without training. VR is the closest thing to doing the job in real life, and research from Stanford University and other institutions has found that learners recall more when using virtual teaching methods than with traditional methods. When it comes to high-risk tasks and hazard awareness, there’s no way to simulate a realistic mine rescue situation other than in VR. In VR, the user can be burned, fall from a height or even be electrocuted without real consequences. The medium also offers measurable data to assess a user’s performance. For instance, in a virtual inspection of a mine, the trainer can observe not only the user’s movements but also her gaze to see what draws her attention first.


Conclusion

The ability to walk, climb and interact in an environment using AR/VR will make for easier discovery and better planning of mines, faster innovation and greater productivity, increased safety and higher quality, all of which can improve industry recruitment. The changing nature of mining, including increasing digitization and automation, should draw a new generation of workers—tech-savvy individuals traditionally attracted to more high-profile industries as well as talent that hadn’t considered mining because they didn’t want to work underground or in remote areas. Pushing into frontier mining areas and planning new mines with new extraction and processing techniques (with a lighter human touch) will further the incorporation of new technologies; allowing miners to face less challenging working conditions and making mining as a whole a more sophisticated sector. Who knows? Tech companies that rely on mined materials to build their products might even begin their own mining operations in the future.

 

 

The Enterprise Wearable Technology Summit (EWTS) is an annual conference dedicated to the use of wearable technology for business and industrial applications. As the leading event for enterprise wearables, EWTS is where enterprises go to innovate with the latest in wearable tech, including heads-up displays, AR/VR/MR, body- and wrist-worn devices, and even exoskeletons. The 6th annual EWTS will be held September 17-19, 2019 in Dallas, TX. More details, including agenda and confirmed speakers, available on the conference website.

 

 

 

In-flight VR, Smart Bands at the Resort, and AR Glasses for Tourists

Today, “every business is a tech business” and in every industry consumers’ digital customer service expectations are growing. A decade after the U.S. travel and hospitality industry emerged from the 2008 recession; industry players, including airlines, airports, cruises, hotels, and other travel brands, are feeling the heat to compete and earn the loyalty of a new customer base via emerging technologies.

Trends and Pain Points in Travel and Hospitality   

Shift in Target Demographics

Though Gen Y overtook Baby Boomers as America’s largest living generation in 2016, the demographic with the most purchasing power around the world today is millennials, and they don’t vacation like their parents. Travel brands need to both court and cater to millennials, who prefer to spend their money on experiences (like immersing themselves in another culture) over material objects and are more spontaneous and comfortable with tech than previous generations.

Competition

First it was online travel agents like Expedia and Priceline; then came Airbnb and VRBO—OTAs and the sharing economy have rocked the travel industry, altering distribution channels, taking business away from traditional industry players, and forcing airlines and hoteliers to compete online to win back customers. According to ADI, approximately 60% of all travel reservations are now made online despite Loyalty Rewards Programs for travelers who book directly through the airline or hotel. Another consequence of OTAs and millennials’ spontaneity is that the window between booking a ticket and boarding a flight is getting smaller, putting strain on travel and hospitality operations.

Heightened Consumer Expectations

We live in an experience economy, where it’s becoming critical for businesses to have customized offerings and personalized services. Millennials want to do something new and memorable on each trip but they also want personalized experiences and don’t mind sharing their data to receive customized travel recommendations. In a time when a single data breach can destroy a brand, travel companies must walk a fine line between capturing enough data to personalize services and respecting guests’ privacy and security. In addition to personalization, today’s consumers consider sustainability and wellness in their travel choices, expecting hotels to “go green” and have state-of-the-art fitness centers, healthy food and beverage options, even yoga classes.

Labor Gap

 Within the leisure and hospitality sector, there are an estimated one million job openings in the U.S. alone. As companies struggle to attract and retain the right talent to fill the experience void, reduced immigration is impacting the supply of transient and hourly workers that have come to make up a large portion of the hospitality workforce. Moreover, recruitment for new job roles needed to incorporate the latest tech into the travel experience is proving difficult and high turnover is discouraging investment in skills development for new and existing employees.

A Testing Ground for New Tech

Historically, the travel industry has been quick to adopt new tech: In the late 1940s, before most consumers had a television set at home, hotels began to install TVs in the guest rooms. Travel companies were also among the first to leverage the World Wide Web to increase sales, with the first hotel websites launching in 1994; and one of the very first use cases for Google Glass came from Virgin Airlines in 2014. But the challenges above call for real implementations and dramatic digital transformation.

Applications for Immersive and Wearable Tech in Hospitality

Virtual booking

“Try-before-you-buy” shopping apps have become an early hit for augmented and virtual reality, especially for big-ticket items like furniture and real estate. Travel, too, is expensive and consumers need a lot of information before deciding to book. Virtual reality presents the ideal medium for selling an experience, giving travelers insight that no amount of text on a website or any number of customer reviews can match by allowing them to essentially preview their trip – from their seat on the plane to the view from their hotel and local attractions – before committing.

In 2017, Amadeus unveiled the first VR booking experience in which users shop for travel in a virtual world. Users can search for flights, review cabins, compare hotel prices, and book rooms all through a VR headset. And while you might think that as VR gets more and more immersive it will replace travel altogether, current research has found that visiting a destination in VR actually makes one more inclined to visit the real place. If VR hits critical mass at $199 per headset over the next few years, VR travel planning and booking may very well be one of the killer apps for the technology.

Marketing

Hospitality brands spend a lot on marketing. AR/VR is becoming a major differentiator in this area, as hotels themselves adopt the technology as a selling tool. Hundreds of hotels now offer virtual tours. For instance, Atlantis Dubai offers a virtual tour on its website so guests can explore the hotel’s luxury rooms and on-site experiences like swimming with dolphins from the comfort of home. Once on the website, consumers are more likely to book directly through the hotel, as well. In 2017, Marriot launched a VR tour of its meeting rooms, allowing corporate clients and event planners to virtually walk through its function areas from anywhere. During an on-site tour, one might even digitally augment the space to get a more realistic feel for a venue’s suitability. Palladium also uses VR, not to inform prospective guests but instead to educate travel agents about its properties. Palladium salespeople go around giving agents VR headset-enabled virtual tours so they can better sell the chain’s hotels to customers. Some hotels even offer on-site AR/VR experiences, usually smartphone-enabled, that both entertain guests and enlist them in the brand’s marketing efforts via social media sharing.   

Operations

There are a lot of moving parts in the travel and hospitality industry, requiring staff to be in constant communication in order to provide seamless customer service around the clock. Management and staff have traditionally kept in contact via two-way radios, a method prone to lost connections and poor audio quality. Looking for a better way to communicate, Viceroy Hotels turned to wearables: At the Viceroy L’Ermitage in Beverly Hills, hotel staff piloted Samsung Gear S3 smartwatches to manage guest requests and resolve incidents more efficiently than they could with a walkie-talkie or phone. The pilot showed response times going down from 3-4 minutes to just 60 seconds; the solution was also less intrusive, sending silent vibration alerts to the staff members best placed to serve a guest’s need. Houston’s Hotel Alessandra also uses Samsung smartwatches for fast and discrete communication among employees, improving the experience for both guests and staff.

Entertainment & Tour Guide

VR headsets are popping up in airport lounges, on flights, and in hotel rooms alongside other amenities. Qantas, for one, has experimented with providing virtual experiences and games on high-quality VR headsets to first-class passengers; and in 2015, Marriott launched its “VRoom Service,” whereby guests can order a Samsung Gear VR headset delivered to their room—a step up from streaming services and on-demand movies. The headsets come preloaded with “virtual postcards” that not only entertain but also sell users on new destinations (where they can stay in a Marriott hotel, of course).

Others are using mobile AR apps and VR headsets for guest engagement. For example, Holiday Inn created an AR app allowing guests to view virtual celebrities in the hotel through their smartphones; while at London hotel One Aldwych, a whiskey cocktail called The Origin comes with a VR headset showing how and where the whiskey was made—a truly unique cultural experience made possible by VR. Hotels and travel brands are also developing custom AR tour guide apps, like a mobile concierge that provides real-time, heads-up navigation and personalized recommendations for loyalty program members, and enhances sightseeing with digital information overlaid on the landmark itself. The Hub Hotel from Premier Inn in the UK does this with special maps on the walls of every room, which, when viewed through a smartphone, display information about local places of interest—an unexpected, value-added feature for the hotels’ guests.

Airlines and hotels can also adopt augmented reality smart glasses to enable flight attendants and hotel staff to personalize customer service, using facial recognition to greet guests by name and tapping into a customer resource management system, social media and other data sources to bring up information relevant to individual passengers.

Convenience

AR certainly provides convenience by supporting guests and passengers in their native language, showing them directions, etc. Below the neck, IoT (Internet of Things) wearables provide convenience, as well. Case in point: Disney’s MagicBand, one of the earliest and most successful (bespoke) wearable devices in the travel sector, widely used today in Disney theme parks as an all-purpose means of payment, admission and keyless entry for resort guests. In 2017, Carnival announced its Ocean Medallion, a small, waterproof device that can be worn or carried, enabling cruisegoers to embark the ship, enter their staterooms, shop, and make reservations. The Medallion works with Carnival’s Ocean Compass app, which displays personalized recommendations for every passenger with the help of 7,000 sensors installed throughout the ship. Likewise, Meliá Hotels has begun offering waterproof, Bluetooth-enabled smart wristbands by Oracle, which, in addition to serving as a payment method on the Spanish resort of Megaluf, also work at nearby participating merchants like Starbucks.

Training 

Compared to traditional teaching methods, immersive simulations have proven more effective for quick learning and retention of knowledge, which is why major corporations around the world are using AR/VR to train new employees and retrain core staff for new roles. In travel and hospitality, immersive tech can help prepare employees for exceptional scenarios that are hard (or undesirable) to train for in real life like diffusing an angry guest. Need to walk a team through new green housekeeping measures or alterations to the menu? Use VR.

In 2016, Best Western partnered with Mursion to develop a series of VR simulations for front-desk staff to practice interpersonal skills. According to the hotelier, the 60-minute virtual guest interaction training sessions contributed to a noticeable boost in guest satisfaction. Recently, luxury cruise line Seabourn worked with Pixvana to create a VR training solution to help wait staff quickly memorize the dining room’s 105 tables and 12 serving stations. Hilton has used VR with its corporate staff to build appreciation and empathy for the chain’s employees, having higher-ups virtually take part in routine operational tasks like cleaning a guest room and arranging a room service tray.

Conclusion

The convenience of wearables is appealing not just to millennials but to most modern consumers, as are enhanced experiences of physical spaces enabled by augmented and virtual reality. VR will surely become a popular way of shopping for hotels and AR a natural addition to sightseeing and other aspects of the travel experience (on-demand, in-context information). Early adopters in the travel industry are poised to define the competition, providing experiences to guests they cannot get at home, attracting new workers with brand new tech for training and carrying out daily tasks, empowering staff to provide superior, personalized customer service, and easily preparing employees for the roll out of new sustainability and wellness features.

*Learn more about emerging tech in the Travel & Hospitality industry at EWTS 2019: Hear from Blaire Bhojwani of Hilton Hotels, Andy Kozak of JetBlue, Jayson Maxwell of Six Flags, and more.

 

The Enterprise Wearable Technology Summit (EWTS) is an annual conference dedicated to the use of wearable technology for business and industrial applications. As the leading event for enterprise wearables, EWTS is where enterprises go to innovate with the latest in wearable tech, including heads-up displays, AR/VR/MR, body- and wrist-worn devices, and even exoskeletons. The 6th annual EWTS will be held September 17-19, 2019 in Dallas, TX. More details, including agenda and confirmed speakers, available on the conference website.

The Price of Drugs: Exploring New Realities in Pharma

Drug discovery, development, testing, approval, manufacturing, marketing, sales, and distribution—the pharma industry is defined by a number of processes – some years-long and all highly sensitive – that are coming under increasing strain due to rising demand, changing regulations, pricing pressures, the rise of personalized medicine, and the explosion of available data from new wearable devices. In response, large, mainline biotech firms like Pfizer and Novartis, smaller CMOs (contract manufacturing organizations), equipment manufacturers and others involved in the highly fragmented pharmaceutical sector are looking to emerging technologies to improve efficiency, speed up research and production, widen margins, and guarantee quality and safety.

What is takes to develop a drug

In the past, drugs were discovered either by isolating the active ingredient from a traditional remedy or completely randomly; today, molecular biology or biochemistry is used to manipulate the metabolic pathways related to a disease, with major pharma companies increasingly outsourcing this research to universities and biotech companies. Once a compound (potential drug) is identified, it costs an estimated $1.3 billion to develop it and over a decade to gain approval and begin commercial production. In most nations, only a small fraction of potential drugs is ultimately approved by government authorities, and only a fraction of those ever provide a return on investment.

FDA approval comes only after heavy investment in pre-clinical studies and human trials, which help to determine correct formulation and dosing, as well as safety and effectiveness. Drugs can fail part-way through development and capital can dry up, forcing a company to discontinue testing. While new patented drugs are potentially the most profitable, the time to market is very long. Needless to say, the pharmaceutical business is high-risk, low reward.

Drug production

Pharma is one of the most wasteful industries, losing billions each year in manufacturing costs alone. Production has been plagued by inefficient communication, inaccurate reporting, and poor efficiency and reliability. This is as much the result of the fragmented, globalized, and extremely risk-averse nature of the pharmaceutical industry as the increasing complexity of drugs, stringent standards, and lack of financial incentive. Moreover, the equipment itself is difficult to operate, requiring trained specialists to use and maintain, and many engineers still use long, paper-based procedures. The combination of complex equipment and high stakes make pharma ripe for digital disruption.

Disruptive trends in Pharma in 2019

 As in other industries, data is becoming one of the most valuable assets for pharma companies, but data has to be analyzed and delivered to real people in order to drive smarter, faster (real-time) decision making. The potential is great: Applying machine learning to aggregate data sets from all stages of drug production and distribution, including such sources as new wearable devices, smart machines, track-and-trace initiatives, etc. can help pharmaceutical firms meet regulatory scrutiny, reduce human errors, speed up time to market for new drugs, and even better market products. Though pharma is significantly behind other advanced manufacturing sectors in adopting new tech, a number of trends coming to the fore in 2019 are expected to force the industry’s hand.

Strained manufacturing operations

Biologics are large molecule drugs made from living organisms; used to treat diseases like cancer and autoimmune disorders; produced through complex, carefully monitored manufacturing processes (1,000+ steps); and given through injection or infusion (vaccines, gene therapy, etc.). Though the large majority of drugs on the market are small molecules, biologics are on the rise, requiring expensive manufacturing infrastructure. Drug manufacturing has also been impacted by serialization: Introduced to help combat counterfeit drugs, serialization – whereby each saleable unit of a prescription product is given a unique serial number – slows down packaging and requires manufacturers to update their equipment, software and training. On the other hand, serialization generates loads of data that could provide efficiency-boosting insights via advanced analytics.

Rising demands

Tight government price controls, supply disruptions created by natural disasters, job cuts and other factors have created a shortage of generic drugs and medical staples. There is also great demand for oncological and immune-suppressant drugs and therapies driving increased use of HPAPIs (high potency active pharmaceutical ingredients) in drug manufacturing. As the pipeline of major blockbuster drugs winds down, HPAPIs are becoming a more attractive market; and as a result, more pharma manufacturers are investing in upgrading existing facilities to meet their specialized containment requirements and protect employees. Pharma is becoming a tougher market in general, with politicians, health insurers and consumers calling for pharma companies to reduce exorbitant drug prices while also maintaining standards and production efficiency—a tall order.

Immersive wearable tech in pharma

If you can’t raise prices, then you need to cut costs elsewhere. For pharma companies, this means spending less time and money on R&D and going to market faster. As the drug pipeline shifts to meet demand for personalized medicine (targeted biologics), pharma companies are feeling the pressure to revamp their product lines, factories, and processes to become more streamlined and cost-efficient.

AR/VR

For drug discovery

R&D spending in pharma has been rising parallel to the growing complexity of drug development, leading forward-thinking companies to explore AR/VR as a tool for discovering new drugs faster (and therefore cheaper). If VR-trained surgeons are able to complete procedures faster than non-VR trained surgeons, it follows that pharma researchers would innovate faster with VR than they currently can using computer graphics (CAD) and static models of molecules made of wooden balls and wires. Indeed, whether in the classroom or the lab, virtual reality is proving effective for visualizing and conveying difficult concepts while augmented reality can put interactive complex molecules into the scientist’s real-world environment.

Wearing a VR headset, drug developers can step inside a molecule or compound to see how it responds to different stimuli and quickly simulate complex drug interactions. Wearing AR smart glasses or a mixed reality headset, researchers can manipulate molecules and chemical structures in space – folding, knotting, and changing the shape of the molecules right before their eyes – and tweak a drug’s chemical makeup so it bonds to the protein in question, altering its function to the desired effect. AR/VR decreases the number of errors in the years-long process of drug discovery, which is essentially one of trial and error, by helping “drug hunters” iterate and improve (get to the right shape) faster. As a result, companies are able to develop better drugs with fewer side effects. Immersive tech can also improve collaboration among researchers around the world, eliminating barriers like distance and language by allowing two or more scientists to walk through the same chemical structure together from separate locations.

For manufacturing

Training and education

In other manufacturing sectors, augmented and virtual reality are allowing new workers to learn on the job without making mistakes as well as safely practice operating equipment before using a real machine. Likewise, AR/VR can significantly improve training outcomes for pharmaceutical workers. In addition to “practice runs” on complex pharmaceutical manufacturing equipment even before entering a facility; a process engineer wearing safety smart glasses can learn on the job while still meeting high levels of control and quality by accessing step-by-step instructions and other multimedia support for troubleshooting and repairing a machine right in her field of view or connecting via livestream to a remote expert for guidance and support. Operators and scientists can also use VR to learn the proper principles of aseptic technique and the proper procedures for different laboratory and production environments (ex. the specialized containment and personal protection requirements for HPAPIs). Beyond production, AR/VR can help explain new treatments to doctors and patients, and train nurses to administer a new drug or therapy.

Heads-up, hands-free information and documentation

 In manufacturing in general, data from connected machines is unlocking the ability to perform predictive maintenance, saving manufacturers millions of dollars in downtime; so a systems engineer wearing smart glasses in a pharmaceutical plant could receive real-time, heads-up and hands-free notifications about, say, a location that will soon need replenishment or an instrument that’s predicted to fail, allowing him to catch and address issues in advance, thereby improving efficiency, speeding up production, and lowering costs. Anywhere along the production cycle, digital information can be beamed in this way to augment an engineer’s view and intuitively show him or her what to do. For instance, an engineer could use smart glasses to scan the QR code on a piece of equipment, automatically bringing up work instructions or an interactive diagram tailored to that machine. Engineers could access batch records heads-up and hands-free and record values and videos via voice command, never needing to take their hands or attention away from a process. This is also an easy and effective method for audit readiness.

Remote support

All of this instant, hands-free access to information – presented heads-up and in context – is designed to enable users to work faster and more accurately, but it’s not just the challenges of visualizing complex drugs and the use of incorrect, out-of-date paper procedures, manuals, and documentation that slow down time to market; the need to fly in specialists to a pharmaceutical facility when something goes wrong is another contributor to what has become a years-long, complicated, error-prone and unrewarding process. Immediate ROI and time saved can be had from adopting AR glasses for remote support, especially when users need vendor advice. With augmented reality software, the expert can even draw on the user’s display to highlight specific buttons or connections and drop 3D arrows into her real-world environment in the facility.

Conclusion

The possibilities for AR/VR in the pharmaceutical sector are great and desperately needed. Pharma companies should be taking cues from other advanced manufacturing sectors, which are already seeing results in training, efficiency, quality insurance, and safety through the use of AR glasses and VR headsets. Of course, pharma is a sensitive industry, and new devices open up new opportunities for hackers to gain patient data and secret drug research. Any investments in emerging technologies must be accompanied by investments in cybersecurity.

 

The Enterprise Wearable Technology Summit (EWTS) is an annual conference dedicated to the use of wearable technology for business and industrial applications. As the leading event for enterprise wearables, EWTS is where enterprises go to innovate with the latest in wearable tech, including heads-up displays, AR/VR/MR, body- and wrist-worn devices, and even exoskeletons. The 6th annual EWTS will be held September 17-19, 2019 in Dallas, TX. More details, including agenda and early confirmed speakers, to come on the conference website.

 

VR: Getting People to Take Safety Training Seriously, Retail Merchandising, and More

Virtual reality is making inroads in the retail industry, at the same time as more and more companies are realizing the powerful potential of VR for training. Here are some of the most recent use cases of VR in enterprise, all of which, when it comes down to it, are customer-facing applications of the technology. The following is evidence of digital disruption ramping up beyond work instructions and collaborative design, to directly impact the products and services that consumers experience everyday:


ADT

In a recent mailer campaign to convey the importance of professional alarm monitoring services, ADT sent out Google Cardboard-like VR headsets that put recipients into a simulated life-threatening situation. Noting a lack of awareness among homeowners around what actually happens during a fire, ADT worked with Harte Hanks to create a VR experience accessible on YouTube that would drive home the potential side effects of carbon monoxide, the physical obstacles that can prevent you from escaping, and other elements of a house fire.

Placing a phone in the viewer sent by ADT, you find yourself in a bedroom filled with smoke. You’re immersed in a mother’s fear and disorientation as she’s awoken by a call from ADT and goes to find her daughter, who refuses to leave without her dog. When the two go downstairs, you see an inferno coming from the kitchen, and then the house loses power. The experience is raw and definitely more powerful than a pamphlet, but at the end of the day it is a marketing campaign to not only change the batteries in your smoke detector every year but also buy ADT’s services. 


Walmart

The retailer is expanding its use of VR beyond employee training to the customer experience, making novel use of its 4,000+ physical stores. Walmart subsidiary Spatial& and DreamWorks created a VR experience based on the new “How to Train Your Dragon” movie that is now touring 40 Walmart store parking lots via 50-foot tractor trailers. In this way, Walmart is able to play host to exclusive VR experiences (featuring VR-powered chairs) and give the masses access to VR overnight—a technology that’s still too expensive for most and that Spatial& views as the future of retail marketing.

By working VR into stores, Walmart and other retailers can cut back on large displays and market products in interactive ways. For instance, shoppers might use a VR headset to put a tent together to test out camping gear or try stowing a stroller in an overhead airplane bin before buying. This is essentially “try before you buy” but inside the store itself. VR can even bring products to life, for example by enabling shoppers to virtually visit the vineyard that produces Walmart wine.


Royal Mail

Mailman vs. dog: It’s a classic TV trope that Royal Mail says prevents postal workers from taking anti-dog attack training seriously. In addition to the cartoon vision of the dog chasing the mailman, the group environment in which this training is traditionally held hurts its effectiveness, as trainees don’t want to be seen taking it seriously. Over the years, Royal Mail has tried videos, brochures, slogans and posters to reduce the number of dog attacks on its employees; now it’s turning to VR.

With around 150,000 postal workers delivering to some 30 million addresses, Royal Mail sees around 2,275 dog attacks per year. Injured employees are unable to finish their routes, which severely impacts customer experience. Looking for a way to isolate the training and eliminate the “banter culture” around dog attacks, Royal Mail began using VR: Now, trainees use smartphones inserted into headsets to experience potential dog attack scenarios, select different actions and receive feedback on their choices. The result: Many units haven’t reported a single dog attack since adopting the VR training system in November 2018. The training was also inexpensive for Royal Mail to pilot nationally.


Kellogg

Accenture Extended Reality, Qualcomm and Kellogg recently teamed up to pilot an eye-tracking VR headset for retail merchandising. The idea is to enable companies to do market research faster, cheaper and on a larger scale. Accenture developed the solution based on a Qualcomm VR reference design headset powered by the Qualcomm Snapdragon 845, using eye-tracking tech from Tobii, eye-tracking data analytics from Cognitive3D, and mixed reality software from InContext Solutions. By tracking where a user looks while moving through a full-scale virtual store, walking down virtual aisles, picking up products and placing items into his cart, retailers can determine the best way to stock shelves.

Brands spend a lot of time, money and effort figuring out optimal product placement but are usually limited in the data they’re able to collect (online surveys and in-home user tests can only go so far). Eye-tracking in VR provides richer and more accurate behavioral data than traditional testing. You can observe what users are looking at, for how long, in a realistic shopping scenario; and expand testing to more geographically dispersed participants (mobile VR). In fact, the VR eye-tracking solution has led to insights that directly contradicted some of Kellogg’s prior assumptions, resulting in an 18% increase in the brand’s sales during testing.

 

The Enterprise Wearable Technology Summit (EWTS) is an annual conference dedicated to the use of wearable technology for business and industrial applications. As the leading event for enterprise wearables, EWTS is where enterprises go to innovate with the latest in wearable tech, including heads-up displays, AR/VR/MR, body- and wrist-worn devices, and even exoskeletons. The 6th annual EWTS will be held September 17-19, 2019 in Dallas, TX. More details, including agenda and early confirmed speakers, to come on the conference website.

Using AR/VR for Assurance in Insurance

I recently watched a Netflix documentary about the Fyre Festival. Two things from the story really stuck with me: 1) Festival owner Billy McFarland failed to get festival insurance; and 2) He couldn’t (or wouldn’t) listen to reason, as multiple people told him it would be impossible to pull off such an ambitious festival in under six months. At one point, someone tried to show Billy – using a map spread out on a table – that the island venue could not accommodate the number of festivalgoers and luxury villas that had already sold. While watching, I thought about Virtual Reality, not because it’s my job but because immersive technologies might have prevented the disaster that Fyre Festival turned out to be. What if those around Billy had used VR to snap him out of his delusions? Or what if Billy had tried to get festival insurance? Might an insurance agent have used VR to “preview” the festival and ultimately denied coverage? Perhaps that would have convinced Billy to cancel the event.

The insurance industry is, in fact, exploring virtual as well as augmented reality for a number of applications, including risk assessment, accident recreation, remote claims handling, and customer education. AR/VR may also be a solution to the insurance sector’s labor concerns and the answer to rising customer experience expectations.

State of the Insurance Industry

Insurance companies are not exempt from digital disruption or the need to create a more flexible and even virtual workforce for the digital age. As some manual and traditional industry tasks become automated, insurers will need to both recruit and upgrade their talent at a time when the labor market is incredibly tight. This is especially true for the tech, data science and actuarial labor pool (Deloitte). Furthermore, employees impacted by automation as well as Baby Boomers with irreplaceable institutional knowledge will need to be repurposed, which means retraining and leveraging cutting-edge technology to facilitate remote expert mentoring of new workers.

The traditional insurer-insured relationship can be boiled down to a monthly bill or claims submission when something goes wrong; but today’s insurance customers – many of whom are millennials – want more: More convenience and more personalization in the insurance buying and claims processes. Consumers want more control over their coverage through digital channels; they want insurers to leverage advanced sensors and analytics for tracking trends and results that will lower their payments (as in auto and homeowner’s insurance), and they want more innovative and hybrid types of coverage. These and other new expectations are clashing with the long-established culture of the insurance industry, pressuring companies to look for technology that appeals to a new generation of adults seeking insurance.

Applications for AR/VR in Insurance with Real-life Use Cases

Though the insurance sector is usually slow to adopt new technology, augmented and virtual reality are beginning to show up in the ways insurers market and provide their services. Insurance companies are exploring AR/VR as part of marketing strategies, for educating clients, to estimate damage, for employee training, and more:


Customer-facing Applications:

Insurance is a large and valuable market; and with new players offering fast, efficient, digital services, it’s also a fiercely competitive one. Traditional insurers are turning to technology – both the enabler and accelerator of digital transformation – to stay relevant to a changing customer base:

Explaining Insurance Plans

AR/VR can make the complex process of buying insurance easier by simulating real-life situations to showcase the value of various life, health and other coverage plans. Far more powerful than a brochure, website or salesperson, immersive simulations can drive home the need to save for retirement, simplify pension planning, etc.

Consumer Education / Risk Mitigation

In a similar vein, AR/VR can be used to warn clients about dangers and help them prevent the need to file a claim. By allowing insurers to demonstrate both common and exceptional risks in a virtual, risk-free environment, immersive simulations can improve the safety practices of different types of policyholders. For instance, doctors could use VR to practice on a new machine before using it with real patients, employees could learn to identify workplace risks, and homeowners could learn to prevent floods and fires.

Insurers are also toying with VR incident management and training programs that would give customers a fairer rate (ex. virtual driving tests for auto insurance). After successfully completing such a program, the customer would send her results to her insurance agent, verifying her enrollment and qualifying her for discounts (reduced premiums).

Marketing and Customer Engagement

With the ubiquity of AR-capable smartphones, companies today are increasingly incorporating AR into their brand apps and other marketing strategies. Insurers are no exception: AR experiences and VR simulations that create awareness about the importance of buying different types of insurance are part of new marketing and customer engagement plans. In general, insurers are looking to attract and retain new and existing customers by providing informational and entertaining content. This represents a significant move away from the usually distant or aloof position of an insurance company vis-à-vis its clients.

Customer Service

One way to improve the customer experience is to increase an organization’s operational efficiency; for instance, faster order picking in a warehouse leads to faster delivery and higher customer satisfaction. Another way is to focus on those times the customer directly interacts with the business. In insurance, these times are when a customer purchases a coverage plan, files a claim, or contacts support.

In addition to helping consumers understand insurance plans, AR/VR can provide real-time guidance to policyholders on how to fill out claim forms, resolve billing issues, and more. Some insurers are experimenting with virtual customer service (like a virtual support center) and enabling policyholders to interact with adjusters and begin documenting damage in real time through AR. Whether it’s through an individual’s mobile camera or, one day, smart glasses, adjusters can be “on the scene” with the policyholder, reviewing the damages, even taking exact measurements; allowing for faster and more accurate documentation of loss and faster case resolution.


Employee-facing or Operational Applications:

The game of insurance is about risk avoidance, the goal being to convert consumers and businesses into policyholders while driving down claims. AR/VR can be an effective tool for reaching these goals, not just through customer education but also by improving employee performance, making insurance workers shrewder and more efficient:

(Ongoing) Risk Assessment

AR/VR open a number of new capabilities for risk assessors to reduce cost and loss ratios. As mentioned above, auto insurers are considering administering virtual driving tests to determine whether someone is a safe driver before insuring them. VR is also being used to model risk: Assessors can navigate a building before it’s built, thereby improving insurance estimates, and better judge the safety of, say, a warehouse by simulating potential accidents within and evaluating the locations of exit doors and stairs. During risk inspections, assessors could use smart glasses to instantly document and record notes hands-free, and to connect with remote experts who might point out weak spots by augmenting the user’s field of view.

The Internet of Things (ex. smart automobiles, smart homes, etc.) is huge for insurance, enabling predictive analysis and preemptive actions that should reduce the number of high-frequency, low-impact claims. This paves the way for innovative insurance models, like plans that trigger based upon forecasts of loss as opposed to an actual event. Insurers might also use the wealth of data from IoT technologies along with statisticians to visualize and analyze complex data sets in a virtual setting.

Damage Estimation

Most early use cases of immersive tech in insurance come from the property and casualty side of the industry. This is because AR/VR present the ideal tool for safely recreating real-life disasters and estimating repair costs. Through the use of digital building plans and real-time sensor information overlaid on top of a damaged building, AR glasses-wearing agents can carefully review the damage on-site, doing things like seeing behind walls to determine the location of gas lines and other critical or hazardous objects.

Claims adjusters can overlay images of a building’s pre-loss condition for comparison, document damaged areas hands-free (useful for later VR accident simulations) and confer with remote experts. This makes it possible to more precisely estimate damage and process claims quicker, which, of course, pleases customers. AR glasses also allow for remote damage assessments, where an adjuster shares the view of a colleague at the incident site (wearing smart glasses) or looks through the customer’s mobile device to assess the damage without physically being there.

Remote Guidance and Employee Training

Accenture has found that 85% of insurance executives are interested in leveraging AR/VR solutions to bridge the physical and informational distance between newer and experienced employees and between agents and customers. This is especially key in the training of claims processors, who have one of the most important jobs in the industry (investigating claims). As studies show that people learn and retain information better when it’s presented in context over their real-world view, insurance employees should be able to train faster and more effectively “by doing” whether in a virtual environment or via AR-powered remote guidance on the job.

Indeed, leading insurers are finding AR/VR great for training agents at a lower cost, giving them virtual experience that raises their confidence and the accuracy of their work. Immersive training programs can also help insurance agencies prepare employees to work in specific sectors (ex. auto insurance reps learning about engine repair; home insurance reps learning about maintenance lifecycles), so they can make more informed decisions and offer policy-specific recommendations to clients. Remote technical experts might also provide a second pair of eyes, training agents in real time using AR.

Visual Claims and the Claims Process

Alluded to above is the potential for AR/VR to enhance and speed up claims processing by unlocking new methods for evaluating claims and detecting fraud in the field. With AR, multiple agents are no longer required to visit the claim site; just one employee equipped with smart glasses can go, while experts look on, inspecting damages and calculating losses remotely from the office. The time and money saved leads to greater employee efficiency and higher customer satisfaction. Customers themselves can serve in this role using an AR-enabled mobile device or perhaps smart glasses received upon purchasing a policy.

Policyholders are becoming fans of visual insurance claims, which promise more efficient claims processing and quicker payment. AR-powered video solutions can expedite claim settlements by enabling remote inspections at the First Notice of Loss and reducing adjustors’ time in the field (thereby lowering overhead). Customers can show a contact center agent the cause and extent of, say, a car crash, through a live video connection; giving the agent immediate, real-time access to information, including valuable pieces of temporary information like road conditions, vehicle position, skid marks, etc. This significantly shortens the claims process, eliminating not only the usual site visit but also any lengthy back-and-forth communication between agent and customer. The result: More accurate appraisals and faster resolution time.


Conclusion:

The transition from old industry methods to new ways of working with augmented reality will produce a more efficient and cost-effective insurance marketplace, transforming the ways agents interact with customers, enforce policies, and assess claims. Moreover, business and personal use of AR/VR technologies will open new categories of risk exposure leading to entirely new types of insurance.

 

The Enterprise Wearable Technology Summit (EWTS) is an annual conference dedicated to the use of wearable technology for business and industrial applications. As the leading event for enterprise wearables, EWTS is where enterprises go to innovate with the latest in wearable tech, including heads-up displays, AR/VR/MR, body- and wrist-worn devices, and even exoskeletons. The 6th annual EWTS will be held September 17-19, 2019 in Dallas, TX. More details, including agenda and early confirmed speakers, to come on the conference website.

Insider Secrets to Adopting Wearables

Watch this throwback 2016 expert panel led by Upskill’s Brian Ballard, in which enterprise end users from Jacobs Engineering, Powerstream Inc., the AES Corporation, and Intel share their secrets to adopting wearables. Some key insights include referring to the people closest to the problem (i.e. the workers), getting them involved early in the process and allowing them to opt in; creating a partnership between the business and IT sides of your organization; and talking to the standards bodies for your industry from the get-go. In addition, don’t underestimate the impact on your company’s infrastructure, as content and information management are key challenges in this space, especially when it comes to AR.

Challenges of Enterprise Wearables, AR and VR: A Changing Landscape, Budget, Battery, and More

In this largely Q&A-driven panel discussion from last month’s EWTS 2018, Tacit’s Todd Boyd and members of the audience question IT leaders from Worthington Industries, HB Fuller, Ford, JetBlue and The American Bureau of Shipping (ABS) on the cultural and technical challenges of adopting wearable technologies. Some of the challenges addressed include keeping people engaged, dealing with opponents and a constantly changing hardware landscape, budget and financing, battery life and back-end system integration. Watch now:

 

The Enterprise Wearable Technology Summit (EWTS) is an annual conference dedicated to the use of wearable technology for business and industrial applications. As the leading event for enterprise wearables, EWTS is where enterprises go to innovate with the latest in wearable tech, including heads-up displays, AR/VR/MR, body- and wrist-worn devices, and even exoskeletons. The 6th annual EWTS will be held September 17-19, 2019 in Dallas, TX. More details, including agenda and early confirmed speakers, to come on the conference website.


Augmented World Expo (AWE), the world’s #1 AR+VR conference and expo, returns to Santa Clara, CA May 29-31, 2019. Join us for the biggest AWE yet and help celebrate the show’s 10th Anniversary! Apply to speak and/or exhibit at AWE 2019on the event website.

Let Your Customers and Workers Choose the Right XR Use Case for You

Here’s a common misconception: The more robotics and Artificial Intelligence (AI) advance, the more expendable human beings become in the workplace.

Although Forrester Research predicts automation will displace 24.7 million jobs by 2027, it’s irrational to fear that robots will ultimately replace all human workers. For as robotics and AI improve, so do technologies for empowering human workers. I’m talking about wearable technologies like augmented and virtual reality headsets as well as wearable robotics (exoskeletons) that enable humans to work longer, quickly train for new jobs, and perform in sync with automation. You could even argue that as automation progresses, human workers will become more indispensable to enterprises—while robots may assume the dangerous and repetitive aspects of work, unmanned technology won’t be able to address every productivity issue or match distinctly human capabilities like human dexterity and imagination.

When it comes to embracing disruptive technology, successful organizations take a “user is king” approach, finding out pain points in the business directly from the source, i.e. workers or customers who are expected to use or benefit from the technology. Whether it’s getting a group together for a brainstorming session, including members of the workforce in the proof of concept stage, or simply encouraging a company culture where employees feel comfortable sharing their ideas with leadership; there is no one better than the user herself to determine where and how to digitally transform.


“Treat employees like they make a difference, and they will.” – SAS CEO Jim Goodnight


Two companies have gone beyond merely asking for user input: KLM Royal Dutch Airlines established a physical hub to foster workers’ original ideas for using emerging technologies; while Lowe’s went directly to the customer, applying “young” immersive tech to age-old home improvement shopping challenges. Essentially, KLM and Lowe’s are letting their employees and customers come up with the use cases in which they’re investing.

KLM Royal Dutch Airlines

In 2016 at its Amsterdam Airport Schiphol East base, KLM Royal Dutch Airlines opened its Digital Studio, a creative space where workers from all areas of the airline’s business are encouraged to come and innovate. Here, employees can put forward ideas on how to use digital technologies like AR and blockchain in their work, and see their ideas fast-tracked into development and then, hopefully, into practice.

The Digital Studio, which currently has room for 200 workers, is based upon Dave West’s Scrum Studio concept of an environment where high-performing teams, physically separated from the main business, can fast-track projects. It’s very hard to change large legacy companies like KLM from within: The larger the organization, the higher the chances of disruptive technologies ending up in pilot purgatory and innovation suffocating in red tape between divisions and levels of management.

Though most of the current projects at KLM’s Digital Studio are still in the experimental stage, a handful have turned into practice. The studio has embraced KLM employees of all different backgrounds and roles, who may not have otherwise had the opportunity to take their transformative ideas further. Take Chris Koomen, who was stationed in KLM’s engineering and maintenance division: Chris had an idea for using VR, so he joined the Digital Studio and has been a part of integrating VR for training aircraft crew. Another idea pitched by a KLM mechanic involves using AR in aircraft and engine maintenance.

Every four weeks, the Digital Studio hosts a demo of what it’s working on to interested observers. The lesson here is don’t hide emerging tech in a lab unless you’re going to let the user in. Show employees what’s out there, give them resources, and let those who perform the job every day tell you how to transform the business.


“The customer experience is the next competitive battleground.” – Jerry Gregoire, former VP & CIO of Dell


Lowe’s

Despite the impression one might get from HGTV, building things is not easy for the non-professional. Planning a home improvement project, shopping for building materials, executing the project…what’s most difficult for the average consumer, even a hardcore DIY-er, is visualizing the final product. But it seems a solution has finally appeared in the form of XR (AR, VR, MR), and all the major home improvement brands recognize the potential. There are now apps for virtually measuring your surroundings and picturing all kinds of design options and home products in your real space. And it’s not just the Lowe’s and Home Depots of the world—architects and engineers have seized upon VR to help clients visualize new structures, real estate agents are giving virtual home tours, and even Gulfstream Aerospace employs XR so its clients know exactly what their custom jets will look like when delivered.

Lowe’s has been conspicuously innovative in making the benefits of XR available to its customers. For the last four years, powerful new immersive technology design and shopping tools have been brewing in Lowe’s Innovation Labs. Josh Shabtai, Director of the Labs Productions and Operations, says he looks at those problems that keep resurfacing. Since the introduction of Holoroom How-To in 2014, Lowe’s Innovation Labs has rolled out an impressive suite of mobile apps / pilot projects to gauge customers’ comfort level with XR, including Lowe’s Vision, In-Store Navigation, and View in Your Space.

Lowe’s is trying to solve the classic pain points of home improvement shopping by giving customers the ability to see with the eyes of a contractor or interior designer, determine whether products fit in their space, virtually tile a bathroom, operate a power tool, and more. By focusing on customer problems, Lowe’s has made some of the strongest cases for consumer AR and VR to date. The retailer’s steady flow of practical immersive experiences even landed it at the top of a list of most innovative companies in AR/VR by Fast Company!


With each employee-generated idea, KLM not only gains a potentially transformative technology solution but also primes its workers for the change to digital—there’s no need to convince employees to use solutions they helped conceive of. And with each application, Lowe’s refines the XR tools that future consumers will use to visualize spaces and learn new skills; ideally positioning itself to scale when the time comes, build customer loyalty and future-proof its business from online competition.

 

The Enterprise Wearable Technology Summit (EWTS) is an annual conference dedicated to the use of wearable technology for business and industrial applications. As the leading event for enterprise wearables, EWTS is where enterprises go to innovate with the latest in wearable tech, including heads-up displays, AR/VR/MR, body- and wrist-worn devices, and even exoskeletons. The 5th annual EWTS will be held October 9-10, 2018 at The Fairmont in Austin, TX. For more details, please visit the conference website.


Augmented World Expo (AWE), the world’s #1 AR+VR conference and expo, comes to Munich, Germany on October 18-19, 2018. CXOs, designers, developers, futurists, analysts, investors and top press will gather at the MOC Exhibition Center to learn, inspire, partner and experience first-hand the most exciting industry of our times. Tickets now available at www.aweeu.com.

 

Image source: Lowe’s via Road to VR

Making Your Next Flight Safer and Smoother with Wearable AR+VR

From building the actual plane to the in-flight experience, wearable XR (AR, VR, MR) devices have a role to play in multiple professions within the commercial aviation industry. Employees whose jobs affect every aspect of one’s trip, including aircraft maintenance workers and flight crew can make use of wearable XR technologies to ensure the end goal: A safe and satisfied traveler. Find out how XR might be used on the ground and in the air when you go on your next business trip or vacation:


On the Ground: AR for Assembly

Both Airbus and Boeing employ augmented reality (AR) glasses in the aircraft assembly process. Airbus workers follow plans directly in their field of view, superimposed on the plane’s interior during cabin installation. They use the same solution to check the accuracy and quality of their work (image recognition technology and artificial intelligence at work); while Boeing employees use smart glasses to view a heads-up, hands-free roadmap for wire harness assembly over their real-world view. In each case, AR functions to form a stronger connection for the user between textual or diagrammatic instructions and the real working environment.

Using AR glasses with software by Upskill helped Boeing save tens of millions of dollars, but it’s not all about money: By helping employees work faster without error, aircraft manufacturers can deliver defect-free planes to customers quicker. Airlines and other buyers thus receive faster-built, higher quality aircraft and parts that breakdown less often. Aircraft and parts engineers can also use AR and VR devices to collaborate on new designs from anywhere in the world, sharing and testing ideas and even simulating the assembly or installation process to foresee issues. New XR platforms are only making this collaboration easier.


VR for Training

After assembly comes maintenance: It can take up to eight years to train and license an aviation maintenance professional. This includes aircraft OEM mechanics and airline technicians who perform safety checks, prepare aircraft components for flight, make repairs, and more. While accessing real aviation equipment for hands-on training is costly and difficult, in VR trainees can practice skills in a realistic, accident-proof immersive environment with virtual parts and tools. For instance, a mechanic wearing a VR headset could walk inside an engine and examine its parts as well as simulate different repair scenarios. With advanced audio and haptics (like a haptic suit), the trainee could even hear the noise and feel the motion of the engine, better preparing him for the real thing.

A recent study at the University of Maryland found that people actually learn and retain information better through immersive experiences compared to using a computer or tablet. Enterprises are also finding VR to be superior to reading a manual, watching videos, or taking a lecture-style class. While not an example of full immersion, Japan Airlines used Microsoft’s HoloLens to improve training for its engine mechanics—in place of physical hangouts, trainees learned all the engine components by working on a virtual engine in mixed reality.

Learning by doing with AR is effective and cost-saving for training, as well. Aviation maintenance workers can learn on the job without risk of error by using heads-up, hands-free smart glasses to view fool-proof text and visual aids over their work. The technology can even validate each step of an inspection or repair to prevent errors. Static instructions can become interactive, with virtual arrows and labels appearing on top of real-life aircraft equipment, showing the user where parts and tools should go. The result: Faster training without sacrificing accuracy or quality = quicker maintenance, fewer flight delays, and happier travelers.

Once the engine has been overhauled, the plane is ready for service. Expensive and logistically challenging, pilot training is another opportunity for VR. In recent years, the burden of paying for flight school has fallen onto pilots themselves. The $60,000-$80,000 price tag explains why flight school enrollment has fallen in the U.S., leading to a growing shortage of trained pilots not all that unlike the troubling shortage of skilled workers in other industries. CAE forecasts that over 255,000 pilots will be needed in the global commercial aviation industry by 2027, yet less than half that number has even begun training. Some carriers and manufacturers are making efforts by sponsoring aspiring aviators or expanding their flight training services, but the cost and time is still too great.

For industries with large, complex and expensive equipment like aviation, VR offers the closest thing to hands-on training. Virtual reality, capable of simulating almost every aspect of flying, feels more real than many current flight simulators (essentially stripped airplane cockpits with screens for windows) and is adaptable to all kinds of scenarios. Rookie pilots can walk around the cockpit, interact with the plane’s controls, and even practice an emergency landing, with tactile feedback to increase the sense of realness and help build muscle memory. VR is already finding its way into flight training programs: Airbus, for one, has been able to reduce training time and train more people in limited space using VR to supplement training in real aircraft; while Future Visual created a simulation for Oculus which takes pilot students through the entire pre-flight process. And VR isn’t just for ground crew and pilots; cabin crew and even airport staff training could incorporate immersive tech, as well.


In the Air: AR for Guidance

The length of runway required for a standard aircraft to get off the ground can be calculated, but what if there are unexpected failures? What if the engines aren’t working to full capacity or the takeoff field is wet? Will the aircraft still reach the required speed for takeoff? According to Boeing, 13% of fatal aircraft accidents occur during takeoff. In fact, pilot errors, not maintenance failures, are responsible for the vast majority of all aviation accidents. This isn’t surprising considering it’s largely left to the pilot’s subjective opinion to determine a response when something goes wrong.

The problem lies in how information is presented to the pilot inside the cockpit. It’s hard to focus on flying when you have to read and quickly analyze the text on a bunch of small instruments and screens all around you. AR technology can display this information in a more intuitive format. For instance, with smart glasses, information like pre-flight checklists, step-by-step instructions, current weather and air traffic information, even a 3D graphic of the takeoff path can appear overlaid in a pilot’s vision before takeoff. Aero Glass actually has a solution that displays flight path and instrument data to small airline pilots wearing smart glasses. The same cockpit information a pilot might get using physical controls and touch screens can be retrieved instead by voice command; and when a snap decision needs to be made during a flight, AI technology can pick out the most relevant information to display to the pilot.


XR in Flight Service?

The benefits of integrating AR glasses and VR headsets into aircraft assembly and technician training are tangible today, but at this point airlines have merely proposed ideas for using XR in the air without seriously investing. This is probably due to the consumer-facing nature of the in-flight experience. Providing flight attendants with smart glasses to interact with passengers or offering VR headsets as in-flight entertainment are not critical use cases like the need to quickly train thousands of new pilots. Moreover, the timeline for mainstream consumer use of AR and VR is still unclear.

XR hasn’t yet transformed the experience of flying, but some airlines are considering it. Air New Zealand, for example, had its crew members try out HoloLens to expedite and provide more tailored customer service during the flight. To cater to individual passengers, flight attendants might access their flight details (to help make connections), food allergies (to personalize meals), even their emotional state (facial recognition tech). Air France trialed VR headsets for in-flight, immersive entertainment; and though not in the air Lufthansa has used VR to sell upgrades to premium class right at the gate. Who knows? Maybe one day those safety instructions in your seat pocket will be replaced by a virtual reality video. In the meantime, rest assured that XR technologies are improving aviation operations behind the scenes, from the hangar to the cockpit.

 

The Enterprise Wearable Technology Summit (EWTS) is an annual conference dedicated to the use of wearable technology for business and industrial applications. As the leading event for enterprise wearables, EWTS is where enterprises go to innovate with the latest in wearable tech, including heads-up displays, AR/VR/MR, body- and wrist-worn devices, and even exoskeletons. The 5th annual EWTS will be held October 9-10, 2018 at The Fairmont in Austin, TX. For more details, please visit the conference website.


Augmented World Expo (AWE), the world’s #1 AR+VR conference and expo, comes to Munich, Germany on October 18-19, 2018. CXOs, designers, developers, futurists, analysts, investors and top press will gather at the MOC Exhibition Center to learn, inspire, partner and experience first-hand the most exciting industry of our times. Apply to exhibit, submit a talk proposal and buy Super Early Bird tickets now at www.aweeu.com.

Everything Enterprise XR Announced at AWE USA 2018

The scope of the Augmented World Expo is large to say the least—six tracks, a huge expo divided into pavilions, a Playground of entertaining immersive experiences, workshops, and more. As opposed to EWTS’ enterprise focus, AWE truly gathers everyone interested in defining and progressing the future of XR in every aspect of life; and BrainXchange was happy to partner with the show’s producers to help plan the industry event.

There were many announcements at the 9th AWE and some really cool tech on the expo floor (mixed reality backpack, anyone?) For our followers interested in the business and industrial applications of wearable XR technologies, we’ve separated enterprise from consumer in recapping the major developments (yet still beta in many cases) that came out of last week’s event:


Kopin

One of the most anticipated announcements was for the Kopin Golden-i Infinity: A compact and lightweight, gesture- and voice-controlled smart screen that attaches magnetically to turn any pair of suitable eyewear into an AR display. The Golden-i is powered by an Android or Windows mobile device – thereby offloading the heavy lifting – and can connect to apps using a USB-C cable. It’s intended for enterprise use and will arrive by the third quarter of this year at a price of around $899.


Qualcomm

Qualcomm revealed the Snapdragon XR1 Platform, the first chip specially made for standalone XR devices. The new processor features special optimizations for better interactivity, power consumption and thermal efficiency; and could potentially reduce the cost of entry for new AR/VR hardware developers. Qualcomm also released a reference design that has already influenced forthcoming standalone devices from VIVE, Meta, Vuzix and Picoare.


Vuzix

In addition to taking the stage alongside Qualcomm to reveal the new Snapdragon XR1, Vuzix announced a partnership with Plessey Semiconductor and a shipping date of June 1st for the Blade AR Smart Glasses. Both partnerships will affect Vuzix’s next-gen smart glasses (expected in 2019) by increasing processing power and upgrading the display engine. During his keynote presentation, Lance Anderson also called on developers to help augmented reality move forward by creating practical and entertaining apps for the Vuzix Blade, the first fashion-friendly smart glasses for both work and play.


RealWear

AWE attendees were introduced to the HMT-1Z1, the first commercially available, ruggedized head-mounted AR computer certified for use in potentially explosive work environments (ATEX Zone 1 and C1/D1). The intrinsically safe wearable computer presents no ignition risk, allowing all workers to go hands-free and take advantage of the efficiency benefits of the HMD, and will ship on June 15th.


eSight

SPEX, a new division of eSight Corporation, showcased its first AR headset platform offering “breakthrough enhanced vision” in commercial, industrial and medical scenarios that require precision vision. The lightweight HMD has no release date as of yet but has been described as comfortable, providing an augmented view of the world without obstructing the user’s natural vision.


Atheer

Atheer announced the latest version of its AR platform, which includes secure group collaboration so that multiple remote experts can provide live video guidance and support across the supply chain (think of manufacturers with multiple suppliers). The company also widened the range of business processes supported by the Atheer AR Workflow Engine to include dynamic warehouse pick lists, contextual task guidance, checklists, link workflows, surveys, and note-taking for seamless process documentation.


Epson

Epson released the Moverio AR SDK for its line of Moverio Smart Glasses, which adds new capabilities like 3D object tracking using CAD data and 2D image tracking to the former SDK. The update enables the creation of 3D content for Moverio glasses and can detect various objects from 3D CAD files (no need for QR codes or other markers) as well as track multiple 2D images on a 3D plane. Epson is accepting applications for beta testers to help identify bugs.


Kaaya Tech

Kaaya Tech’s HoloSuit, a motion capture suit featuring haptic feedback for full immersion, was on showcase at AWE. The MoCap suit with haptic tech comes in two models, a basic one with 26 sensors and a higher-end version with 36 sensors. As opposed to games and entertainment, Kaaya Tech sees its technology being used in physical training simulations for industrial jobs, factory line work and the operation of heavy machinery.


ODG

ODG demonstrated a working model of an AR oxygen mask it has been developing with FedEx. The mask, named SAVED for Smoke Assured Vision Enhanced Display, has a heads-up AR display to help pilots make a safe landing despite smoke filling up the plane. In the near future, ODG plans to offer the technology to civil and commercial aircraft manufacturers and pilots as well as the military.


ScopeAR

ScopeAR debuted a new AR platform offering real-time remote assistance and augmented reality smart instructions. The all-in-one solution combines Scope AR’s video calling app Remote AR and the AR content creation library WorkLink to enable increased levels of collaboration and guidance.


Toshiba

At AWE, Toshiba demonstrated its dynaEdge AR Smart Glasses with two new applications resulting from recently-announced partnerships with Applied Computer Services (ACS) and Ubimax. ACS’ Timer Pro Storyboard software for video training and the Ubimax Frontline application suite are now both available on the dynaEdge.


Meta

AWE attendees got a live, on-stage demo of the Meta Viewer, the first software application for the Meta 2 headset that lets users view 3D CAD models in AR. Currently in beta state, the app will save time and reduce costs in the product development process—everyone in the development chain (designers, salespeople, etc.) will be able to use Meta Viewer to collaborate and interact with 3D designs without having any special technical skills.


RE’FLEKT 

The company has added Sync – “the first software solution to automatically create edge-based tracking from CAD data” – to REFLEKT ONE, its suite of AR/MR app development tools. Sync is designed to further simplify the transformation of existing technical documentation and CAD data into AR/MR manuals and enterprise applications. With Sync, RE’FLEKT claims AR apps for maintenance, training and operations can be built completely in-house. Companies can save time and money and do not have to share their proprietary CAD and other data with a third party.

 

Image source: Wareable

 

The Enterprise Wearable Technology Summit (EWTS) is an annual conference dedicated to the use of wearable technology for business and industrial applications. As the leading event for enterprise wearables, EWTS is where enterprises go to innovate with the latest in wearable tech, including heads-up displays, AR/VR/MR, body- and wrist-worn devices, and even exoskeletons. The 5th annual EWTS will be held October 9-10, 2018 at The Fairmont in Austin, TX. For more details, please visit the conference website or download the EWTS 2018 Brochure.


Augmented World Expo (AWE), the world’s #1 AR+VR conference and expo, comes to Munich, Germany on October 18-19, 2018. CXOs, designers, developers, futurists, analysts, investors and top press will gather at the MOC Exhibition Center to learn, inspire, partner and experience first-hand the most exciting industry of our times. Apply to exhibit, submit a talk proposal and buy Super Early Bird tickets now at www.aweeu.com.