April 29, 2016
Written by Special Guest Blogger Joakim Elvander, Senior Partner Engineer and Developer Advocate, Sony Mobile Communications
We have a gap. It’s actually quite a huge gap.
The gap I’m speaking of is between traditional industries dealing in machinery, maintenance and so on, and the new technology and software that may help them. Many of them don’t have their own software development departments. This means that they have to hire consultants whenever there’s integration work to be done. With the digitalization of the industry this becomes a real problem if they cannot buy off-the-shelf solutions that immediately fit their business.
At Sony, we have chosen to work mostly with partners on developing solutions for SmartEyeglass for the industry, but we have also made sure to understand the needs ourselves so that we may help out and guide companies wanting to use or re-sell smart glasses.
The common traits we have found for the industries we are talking with, are that they are in most cases interested in Augmented Reality, and not Mixed Reality or Virtual Reality. The workers and personnel intended for the glasses need to have both hands free most of the time. They also need to see what they are doing. This is where our product has found a nice fit; they are lightweight and comfortable, have >85% see-through transparency, and are capable of clear and to-the-point information in the wearer’s field of view as well as augmenting reality. As the glasses can easily be fit with prescription lenses as well as shades when needed, they can be tailored to individuals yet still transferred between colleagues easily.
1. Remote Guidance
Many businesses, large and small, have a need of being able to do maintenance or repairs with the help of an expert or instructions. Even hospitals and ambulance services are experiencing the problem of having to make use of a few experts (doctors with specializations) with responsibility over a large geographical area, and several hospitals and many clients and colleagues to help and advise.
The return on investment can be huge for certain businesses: An airline, for one, can save millions of dollars if they can get a grounded plane fixed and in the air within a few hours instead of having to wait until an expert is flown 12 hours to the remote location. The ROI can also be on a more humanistic level: Utilities maintenance organizations may keep their experienced yet less able-bodied workers in the back office helping the crew that are out in the field.
How can this be achieved then? *I will expand on the design of a Remote Guidance application in my talk at EWTS East, with details on how we at Sony, together with one of our customers, thought about how to go about this.*
But the most common requirement is that the back-office should see a video feed of what the wearer is seeing, and that they should be able to talk to them. The communication can then be further enhanced depending on the needs of the customer and the business they’re in. The back office may, for example, send short text messages, which are more suitable than oral communication when it comes to such things as part numbers, serial numbers, and any other information that is very exact.
Another really cool thing is to be able to use the AR capabilities of the glasses, and for example point out things in the glass wearer’s reality by marking them with just the help of the video feed. The marker stays in place around the marked object or place as the wearer turns his head. This can be even further enhanced by providing more detailed information on the marker, like which tool to use at that spot, how many turns a knob should be adjusted, etc.
Features aside, we have found that it’s worthwhile to discuss the customer’s expectations of the solution. Many customers start out with a sci-fi idea of how they want things to work–e.g. full documentation in front of your eyes, or CAD drawings aligned and fitted with reality, and so on. It is certainly possible to do some of these things (not all) if you are wearing extremely advanced glasses and are prepared to invest heavily in extremely advanced software. But what is then often forgotten is once again that the wearer needs to see what he is doing above all else, or he will take off the glasses to get the work done. The conclusion we have made is that in designing remote guidance solutions, less is more.
There are enough tricky parts even if you keep things simple. A stand-out problem today is security and networking. The firewalls, VPNs and other configurations of security of each company make it quite hard to create general solutions for getting a live video feed to a back office. This part of the solution should not be underestimated.
There are also cases where the requirements are much simpler. Just being able to record on video how a maintenance task was performed is good enough for many companies. This doesn’t really fall into the category of Remote Guidance but it is related. It is, however, questionable whether glasses are really needed in that case, when there are excellent wearable cameras out there more suitable for such a case if that is indeed the sole requirement.
2. Parts-picking and related check-list-based tasks
AREA recently published an article citing a study it was found that workers in logistics performed 45% faster using smart glasses and code scanners, compared to paper-based checklists. They also reduced the error rates to 1%. Allegedly, the non-smart-glass users made ten times more errors.
Since the same study also concluded that on average order pickings constitute 20% of the total logistics cost, and nearly 55% of the total warehousing cost, there is a lot of money to be saved here.
As a bonus, the study proved that mental effort was less for the workers using smart glasses than for those using pen and paper, under the same stress level (heart rate variability).
Even though this study only concentrated on direct measurable factors when it comes to the benefits of using this kind of solution, it’s not hard to imagine some secondary (less quantifiable) benefits, as well. Customers that get erroneous shipments may not become returning customers, and the brand value may take a hit. On top of this, the cost of handling faulty shipments increases with the error rate in the parts-picking stage.
It is possible to create solutions that are almost completely hands-free if the glass wearer is using a finger scanner instead of the larger ones. The glasses can provide contextual information to the wearer and automatically check things off the list as they get scanned, reducing any other interaction with the glasses.
There are related possibilities of making use of similar solutions in the modern industry. The advent of digitalization, as well as the Internet of Things, creates an overflow of information in the factory. In order for this information to become useful, it needs to be processed, filtered, and put in context for the people who need to act on it. This is where SmartEyeglass on the factory floor can shine. Using context such as who the wearer is, where he is on the floor, his expertise and skill set and so on, he can get tailored information on what needs to be done, right in front of his eyes.
3. Businesses needing live captioning and similar for their customers
There are numerous other categories of businesses that may benefit from using SmartEyeglass. But for this article, the last type of business I’d like to mention is the type where a customer of some kind is going to be the wearer of the glasses, as opposed to the previous cases where the workforce is the intended user.
We have been working with Opera Touch, for example, where the audience may choose to wear a pair of glasses during the performance in order to have live subtitles and even the sheet music right beneath the action on stage. The benefit of course is that the audience doesn’t have to shift their focus from the stage to a display underneath, above, or to the side of the stage to read the translation.
Verbavoice is another company that uses SmartEyeglass for live translation when you are talking to someone speaking a language you don’t understand. If both participants are wearing the glasses, it’s possible to have a two-way conversation in this manner, without either one knowing the language of the other. You simply get the translation as subtitles.
There are many possibilities with this kind of technology. It may, for instance, also benefit the deaf in dealing with their surroundings, by having live transcription of spoken communication in front of their eyes. The entertainment industry and museums are looking into this particular use of smart glasses, in order to be able to cater to the hearing impaired in nations and states where it is required by law.
Whatever the case in this category, it is understandable that many of them turn to Sony, since high transparency and comfort are essential when customers are the wearers of SmartEyeglass. And yet it is, in fact, becoming clear from talking to industries in need of other solutions that many businesses are actually looking for the same qualities.
Don’t miss Joakim Elvander‘s case study “Using High-Transparency See-Through Augmented Reality for Remote Guidance” on Day 1 of EWTS East. While we can learn so much from real enterprise end users of wearables, including how companies are innovating with the latest in wearable tech and the challenges they are facing; insight from speakers like Joakim can reveal even more about the implementation process, including the realities of integration and best-practice advice when it comes to adopting wearables in the workplace.
