Using XR to See Underground: Interview with Arcadis’ Allison Yanites

Before EWTS 2019 went down last month, I had the chance to interview one of the event’s thought leaders. Check out our interview with Allison Yanites, Immersive Technology Lead at Arcadis, the global natural and built asset consulting firm.

Emily (BrainXchange): To begin, could you provide our readers with a little background on yourself and what you do at Arcadis? Also, when did you first encounter AR/VR?

Allison: I am the Immersive Technology Lead at Arcadis North America. I am currently working to find different ways that augmented reality, virtual reality and other related technologies can improve customer experience, health and safety, and quality of life. Before this role at Arcadis, I worked as a geologist on environmental remediation projects: understanding subsurface conditions such as layers of soil and rock, if any groundwater or soil contamination is present, and if impacts are static or still moving below ground.A big piece of that work was creating 3D visualizations of subsurface data to help our clients and stakeholders better understand the full picture of what is happening below ground and help determine the next steps to clean up any contamination.

A few years ago, our team developed a mixed reality visualization of one of these environmental sites, where our stakeholders could see and interact with a holographic image of the groundwater contamination of the site. That was my first real experience with immersive technology as an industry application, and it was a gamechanger for me. Working with our digital team at Arcadis, I wanted to look beyond just holographic visualizations of environmental models and see how much we can do with AR/VR across all of the types of programs Arcadis is involved with, how we can use immersive and 360 technology for design, engineering, project and management services across all markets.


E: So, you really start at the beginning of a project, with touring a site? 

A: It depends. On some projects, a lot of data has already been collected, such as sites that have been monitored for decades; on other projects we are collecting data in an area for the first time. Either way, we are taking a large collection of data and trying to understand the complex geological and chemical patterns underground, and ultimately, determine the best ways to remove chemical impacts at the site.


E: Can you speak a little more about Arcadis’ business and its customers (who they are)?

A: Arcadis is a natural and built asset consulting firm. We work in partnership with our clients to deliver sustainable outcomes throughout the lifecycle of their natural and built assets. We have 27,000 people in over 70 countries with expertise in design, consulting, engineering, project and management services, and we work with a wide range of markets and industries, including oil and gas, manufacturing, transportation, infrastructure and municipal water.

At Arcadis, our mission is to improve quality of life in the communities we serve. Whether that is by ensuring the environmental health of communities or reducing the amount of time people spend in traffic, we develop our solutions with our client’s end-users in mind. To design the most impactful solutions, Arcadis has committed to digitally transforming our business at every level of our organization. That includes training our staff on new digital capabilities, using cutting-edge technologies and then applying our subject matter expertise. We then use these tools and skills to better understand, and address, our client’s needs.


E: How is Arcadis using augmented and virtual reality? What pain points does AR/VR address?

A: Arcadis is using augmented and virtual reality in different ways across a variety of projects. Our immersive technology practice includes on-site visualization with different types of headsets, 360-degree photos, video and virtual tours, and remote assistance with AR capabilities. Generally, immersive technology is addressing four main pain points. The first is increased safety — for example, we can share access to difficult-to-reach sites with 360-degree imagery or livestream video, and bring additional staff or clients to the site virtually. Ultimately, we must keep people safe while still collecting as much data as possible. The second is speed of decision making for example, using AR to overlay a 3D design over an active construction site helps quickly identify any differences between the plan and the current project status. The third is cost reduction — for example, we can now virtually connect project teams and clients to remote sites. This reduces travel and helps reduce the costs associated with delayed communication or unplanned rework. And the fourth is enhancing stakeholder communication and collaboration — for example, virtual 360-degree site tours and remote assistance are virtually bringing staff, clients, and stakeholders to the site where they can participate in discussions about site conditions or questions on certain issues. AR/VR visualizations also greatly improve our communication of design plans or subsurface data visualization.


E: I imagine there are a lot of new demands for the built environment, especially with climate change. Do you think that AR/VR are unleashing more creativity, enabling designers to do things they’ve never done before?

A: Absolutely. There is a lot of power in using AR/VR to understand how the environment is changing, and how to prepare communities and businesses accordingly. AR and VR visualizations can communicate designs to stakeholders that address sustainability needs or flood and storm resilience. AR/VR technology also gives designers the flexibility to share their designs with stakeholders more clearly and effectively, with a greater level of detail, than ever before. When you use AR/VR to see first-hand how a flood level impacts homes and businesses, it takes on a greater urgency than it may have before. We are also using AR/VR technology for training situations, and many training scenarios are relevant to our changing environment and being prepared for the future.


E: How have customers received the technology? Was it easy for them to use? Have any clients adopted AR/VR for themselves?

A: We have had success applying immersive technology services, and it’s exciting to see this technology expand and scale in our industry. At the same time, we are continually working to apply the right technologies for the right projects, and find new ways to solve problems for clients. These technologies are a moving target; they evolve so quickly. It seems like every few weeks there is a new product, software/hardware capability, or integration that opens new opportunities for how AR/VR can be applied. In addition to gaining traction and adoption with the services and capabilities we have established, we are constantly evaluating how we can solve emerging client challenges with new and immersive technology.


E: What was piloting like? Was there an actual test period and were there any major challenges to using the technology at Arcadis?

A: Several years ago, we started with a few different pilots and tested different AR glasses, VR headsets, 360-degree cameras and various software programs to develop content. Each of the solutions or services that we have explored has been rigorously tested, and if appropriate is then developed internally or in partnership with our clients. We are still doing pilots because the space is evolving. With one particular workflow there might be an update in either the hardware or the software that offers a new opportunity, so we’ll go in and test that. The pilots are really tied to the problems we can solve and the solutions we can bring to our clients, working with them to customize what we do with these different tools.


E: Where does the content come from?

A: So far, we have developed everything on our own. We use plugins and software to create content, but the content is coming from our own project locations and 3D designs, like wastewater treatment plant designs, environmental remediation sites or highway infrastructure designs. We already work in those spaces so we have the data sets, which we can use to create the AR/VR visualizations. Through our FieldNow™ program, we have also committed to collecting data 100 percent digitally, which means we can now apply this technology to more projects than ever before.


E: How do you measure the ROI of AR/VR at Arcadis?

A: ROI varies from project to project, but does generally come back to the four KPIs: Increased safety, speed of decision making, cost reduction, and enhanced stakeholder communication and collaboration.


E: How has Arcadis handled the security part of adoption?

A: Arcadis takes data security very seriously. Our group works with our IT department to thoroughly vet each technology against industry security standards. Additionally, our use of each of these technologies is also typically evaluated by our clients to make sure it is compliant with their security protocols. Security is always a leading factor in any new technology we adopt.


E: Are there any applications you’re hoping to test in the future at Arcadis?

A: We are constantly evaluating what we can do to exceed our client’s changing expectations. As new applications and technologies become more accessible, we want to make sure we are equipped to address both traditional and emerging client challenges.

Beyond finding new ways to integrate software platforms, we are starting to leverage the internet of things and wearable technologies more frequently. As a large company that is involved in many different industries, Arcadis uses a lot of different software programs. For each software program (3D design visualization, data analytics, program management system, etc.), we develop unique workflows to create AR/VR and 360 visualizations and/or integrate with a program management system. We are always looking for new software products or software updates that make it easier to integrate AR/VR into our daily routines.


E: With sensors in the environment and wearables, I assume you’re gathering new kinds of information for these models?

A: Absolutely. We are using sensor data, which provides real-time results that can be fed into our data analytics platforms and visualized in different ways. We are also excited about platforms that can house data and be updated in a seamless way, so a whole project team across the globe has access to one central data set.


E: What are your greatest hopes for this technology?

A: As immersive technology becomes more mainstream and awareness keeps spreading about its value for industry, it is exciting to see how many ways immersive technology is adopted and applied. This technology is still so new, I am excited to follow its evolution and see what will be possible in five, 10 and even 30 years. My hope is that as the technology starts to deliver more and more value to businesses, we also see increasingly creative ways to improve quality of life in communities around the world.

AR/VR and Wearables: Shaping the Future of Construction

Bechtel, Caterpillar, CDM Smith, MacDonald-Miller Facility Solutions, Martin Brothers Construction, Brady Services, Rogers-O’Brien Construction… These are just a few of the engineering and construction players who recognize the potential for wearables, AR and VR to increase efficiency and productivity, facilitate better communication and collaboration, and improve safety on the job site.

When we last checked in on the construction industry, the difficulty of implementing wearables posed a great challenge. The devices weren’t robust enough, IT departments were unprepared, etc. Solution providers have since worked to make their products more suitable for construction environments, and companies are increasingly developing PoCs, setting up pilots, and thinking ahead. The construction industry is in need of disruption by wearable technology for a number of reasons:

Poor Safety

Safety is a top priority on every construction project and yet working in construction – as a laborer, equipment operator, steel worker, carpenter – is still one of the most dangerous occupations in the world. The most workplace accidents? Construction. One of the highest rates of fatal injuries? Construction. OSHA has even dubbed a nickname, the Fatal Four, for the leading causes of construction worker deaths: Falls, electrocutions, getting struck by an object, and getting caught in/between objects. Sadly, the number of deaths in this industry has been steadily increasing, but emerging technologies present new opportunities to make the construction site safer.

Rising Demand, Expectations and Costs vs. Shortage of Skilled Labor

Infrastructure and housing needs are growing globally; so too is the scale and complexity of construction projects as older infrastructure needs replacing and climate change challenges our communities. On top of demand, material and labor costs are rising; and customer expectations as well as the nature of contracting are changing: Customers want high-performing (sustainable) buildings, shorter timeframes, and less financial risk. E&C firms are being asked to adopt green construction practices and assume costs as contracting shifts toward public-private partnerships and lump-sum, turnkey contracts. Competition for bids is increasing, as well, as clients demand unbundled contracts in order to cherry-pick among subcontractor offers.

All of the above might be manageable if not for a serious labor and managerial shortage. Employers are struggling to staff job sites and to stay on schedule for projects. Several factors are contributing to this problem:

  • The industry laid off 40% of its workforce (2.3M jobs) during the last recession. Many of those workers did not return.
  • There is less emphasis today on the trades and technical training in schools.
  • Aging workers are leaving the job faster than they can be replaced.

And so, firms are forced to spend more money for slimmer profits; raising wages to recruit workers, shelling out healthcare and workers’ compensation costs, and paying higher prices for supplies.

Simply put, construction companies need to find and train skilled labor fast. They need to be more productive with less if, as the McKinsey Global Institute estimates, the world will have to spend $57 trillion on infrastructure by 2030.

Uncertain Times

In the U.S. and among America’s trade partners, there is a feeling of uncertainty under the current administration. Will existing trade agreements stay in place? Might possible trade conflicts raise material prices? Will new immigration policies further weaken the construction labor force? Will there be steady federal funding of infrastructure projects? To say the least, E&C firms may have to quickly adjust to the whims of the White House.

Industry Trends vs. Old Ways 

Although one of the world economy’s largest sectors, Construction is one of the least “digitized” and least productive industries. It’s also wasteful in terms of time, labor and materials; and very much stuck in its ways, including being slow to adopt new technology and underinvesting in IT and R&D.

A typical large-scale construction project takes 20% longer to finish than scheduled and comes in 80% over budget. This productivity problem is, of course, a result of the labor shortage but the nature of construction itself also impairs productivity: Construction projects are highly fragmented and unpredictable, with multiple parties and work crews involved and remote worksites that are constantly changing as building progresses. It doesn’t help that the industry still relies mainly on paper drawings, blueprints, orders, logs and reports. And then there’s the lost-in-translation factor: Blueprints are flat but structures are three-dimensional—a discrepancy that makes communication and collaboration difficult. With the architect of the building, the owner and the various contractors working off of different versions of reality; errors, order changes, delays and rework are inevitable.


Construction projects today are not being optimally coordinated, and the labor force is not being sufficiently developed or protected from harm; but there is tremendous value to be gained if only the construction industry would digitally transform. Emerging technologies including wearable devices are the answer to many of the sector’s ails; to realizing a more connected jobsite and enabling digital collaboration, and becoming more productive, timely, efficient and safe. Read on to learn how wearables are being applied in construction:

Top Applications:

Safety and Efficiency

Most construction project managers can’t tell you the number of workers on site or where those workers are located at any given time; and yet knowing where your assets are, including people, equipment and inventory, is half the battle in avoiding inefficiencies and keeping workers safe in construction. Eliminating paper from workflows (ex. taking building plans out of workers’ hands and putting them directly in their field of view) would address another major source of inefficiency.

The construction jobsite is a hectic place, with multiple contractors and their crews working at the same time, so inefficiencies can go unnoticed; but strategic use of robust sensors could make a big difference. Some companies are already turning to IoT solutions like drone surveying and automation to improve site operations. Real-time visibility into jobsite operations is the ultimate goal, giving project managers the ability to make intelligent decisions based upon real-time information. After all, how can you build smart buildings if the building site itself isn’t smart?

Real-time information comes from tracking workers (GPS and EHS sensors,) tracking inventory (RFID tags,) and monitoring equipment health and repair status. All this data yields actionable insight through advanced analytics—insight that can be pushed to the right person at the right time via wearable devices to optimize the jobsite.

Sensors in a variety of form factors can track workers’ biometrics and surroundings (ex. heart rate, fatigue, gas exposure) along with their location and movement (ex. proximity to danger zones, falls, bad form.) To this end, a number of companies are making construction safety clothing and gear “smart” with embedded sensor technology. The data can be relayed from the jobsite to determine unsafe working conditions and notify workers when dangerous levels are reached or there’s potential for injury. In addition to displaying glanceable safety info, wearables can also have a panic button feature enabling the user to call for help.

Tracking workers keeps them safe. Live location and status information also improves efficiency: Knowing where crews, materials and equipment are and how these assets flow through the work site, knowing when inventory stocks are running low, and knowing when equipment is on the verge of malfunctioning improves project coordination and jobsite organization.

A change in ergonomics would further boost safety and efficiency. Exoskeletons are a rather extreme example that will nevertheless empower construction workers to manage physically demanding tasks with less risk of injury. More practical today are smart Augmented Reality glasses: Constantly looking down at paper plans or BIM models on tablets is dangerous. –> Wearing smart glasses (or an AR helmet,) a person can look at a BIM model overlaid onto the built environment while remaining heads-up and hands-free. Conducting inspections with a checklist in hand, holding up a camera to take photos, and taking written notes is slow and not very accurate. –> Smart glasses allow for hands-free documentation of building progress (including voice memos,) the ability to annotate and update blueprints in the field, even view automated dashboards from live field data. AR glasses, and even more so Virtual Reality headsets, can also be used for faster, safer and more effective training on heavy equipment; and to visualize where assets should go in planning staging areas, supply deliveries and equipment storage.

Of course, taking the data collected from different devices on the jobsite and turning it into quality, actionable information that gives a manager or worker greater context or situational awareness is not easy. Additionally, employee tracking can be a sticky issue due to privacy concerns. And deploying digital solutions at scale, across construction sites that are geographically dispersed and essentially “shared” among firms of varying size and sophistication, is also problematic. None of this is going to get any easier, however, so waiting doesn’t put a company in a better position down the road. The time to overcome these challenges is now.

Design Visualization, Communication, and Productivity

The rise of Augmented and Virtual Reality signals a new era in design visualization. Better visualization capabilities improve communication, speed up decision making, shorten the project lifecycle, and reduce material costs. Thus, companies are developing BIM- and CAD-based design and construction solutions for AR/VR platforms.

Buildings today are usually planned out on flat screens and pieces of paper, yet 2D drawings and scale models of an architect’s 3D vision are hard for project owners to interpret, and not a foolproof guide for contractors either. The leap from 2D blueprints or even 3D models on a computer screen to the real built environment is great enough to cause misinterpretations and errors requiring changes to be made after construction is already underway; but what if you could virtually walk through a design or interact with digital content in the physical environment? That is the power of AR/VR in E&C.

Virtual Reality is useful for design conception and group design review. Multiple users wearing VR headsets, regardless of their location, can interact within the same virtual environment, identifying issues and making necessary changes before building begins. VR can also be used by project owners and builders to visualize the final space or structure. During construction, workers can use AR glasses to view schematics and detailed specs like electricity floor plans overlaid on top of the building site for guidance, helping them work faster and avoid mistakes.

AR and VR also enhance communication, as does instant documentation and sharing of information enabled by hands-free wearables. Disagreements, mistakes and delays arise on a construction project when not every stakeholder is able to envision the design and provide feedback; information sharing is slow and imprecise; and different contractors use different platforms for design and project planning. Everyone involved, from the architect/engineer to the end user, needs to understand the design; and contractors need to be able to communicate in a standardized way, providing updates on building progress, cost and schedule.

Smart glasses present a quick and accurate way to capture, view, update and convey jobsite information. Visual information is universally understood—smart glasses can take photos and record video via voice command; and that data can be integrated into live building models and workflows, and shared with off-site decision makers and remote experts in real time to more effectively explain and respond to issues.

AR/VR in the design process and to share ideas increases productivity, with virtual reality having a bigger impact on the design side (facilitating group design, expediting design review) and augmented reality helping out on the construction site. Smart glasses help construction go smoother and faster in several ways:

  • Augmented work instructions and other (real-time) task-based information in one’s FOV: On-the-ground crew can access information while working with their hands, wearing gloves or using tools.
  • Direct, hands-free connection to remote experts and supervisors: Live photo/video/audio streaming allows for faster issue resolution and decision making; and with expert oversight lesser-skilled workers can be as productive as experienced ones.
  • AR inspections: In addition to smart glasses-enabled remote inspections, managers and contractors can wear AR glasses to view a 3D model of a building over the as-built environment, comparing the two as they walk through the site.

Despite the high implementation costs, leveraging AR/VR in construction can have tremendous ROI. The industry actually has somewhat of a head start in taking advantage of new realities thanks to advancements in visualization software like BIM and CAD; but to adopt new solutions capable of converting those platforms to VR simulations, E&C firms will have to increase IT spending.

Training and Recruiting  

Construction companies could use VR headsets to design better, make the jobsite more connected with a variety of sensors, and give AR glasses to workers to reduce errors; but that won’t be enough to make up for the fact that the skilled labor workforce is shrinking rapidly. Along with attracting and training new talent, E&C firms must harness the knowledge of industry veterans and even prolong their careers if possible.  

Augmented and Virtual Reality present new paradigms for training employees to perform complex procedures and physical tasks, including emergency safety procedures, operating a crane, etc. It has been shown that experiential learning is more effective than using written or video material to transmit skills. Virtual Reality immersion is the next best thing to hands-on learning in the field, and a more ideal method for preparing workers for real-life situations on the jobsite than the static, standard training most construction workers receive today. And new tech like AR/VR is attractive to Millennial workers who are able to transition skills learned in virtual reality to the real world more easily.

Thanks to smart glasses, new workers can also train on the job without risking the integrity of the project (i.e. without making rookie mistakes and hurting quality.) Following step-by-step instructions in a heads-up display and receiving real-time remote guidance from older, more experienced workers—that’s learning by doing, and it accomplishes training while also contributing to building progress.

Exoskeletons might help extend the physical careers of aging workers, but smart glasses offer a more realistic, immediate proposition for preserving these employees. By wearing smart glasses to record their work from a first-person point of view, older workers can capture their knowledge for the next generation, creating training material for new recruits to reference on the jobsite using the same devices. And, of course, the most experienced workers can continue to provide remote support, viewing the jobsite through another’s eyes (or smart glasses) once the physical demands of the job become too much. So instead of a mass exodus of talent, smart glass technology could help transfer the value of older skilled workers from the construction site to a central planning office.


Much of this technology is expensive, especially the Augmented Reality helmet and exoskeletons, and challenging to implement given the rugged conditions and remote locations of most construction sites. But consider the productivity and safety gains, weighing these against the rising costs of workplace injuries; need to grow and empower the workforce to balance out the loss of retiring workers; new design possibilities, etc. From sensors embedded in familiar PPE to strength- (and safety-) enhancing robotic suits, perhaps the most attractive potential benefit of wearable technologies in construction is the cost savings. Better trained workers have fewer accidents; more informed workers make fewer errors; a connected jobsite eliminates inefficiencies; and wearable-enhanced communication prevents common hiccups. The result is less injury-related costs, fewer expensive building delays and rework, projects completed on time and within budget, more satisfied customers, and more profit. That’s the bottom line.

 

The Fall Enterprise Wearable Technology Summit 2017 taking place October 18-19, 2017 in Boston, MA is the leading event for wearable technology in enterprise. It is also the only true enterprise event in the wearables space, with the speakers and audience members hailing from top enterprise organizations across the industry spectrum. Consisting of real-world case studies, engaging workshops, and expert-led panel discussions on such topics as enterprise applications for Augmented and Virtual Reality, head-mounted displays, and body-worn devices, plus key challenges, best practices, and more; EWTS is the best opportunity for you to hear and learn from those organizations who have successfully utilized wearables in their operations. 

photo credit: dmitryzhkov 2_DSC8644 via photopin (license)

3 Cool Use Cases of HoloLens in Enterprise

New realities – Augmented, Virtual and Mixed – open up tremendous opportunities for visualization, communication and training in the enterprise. Applications for these technologies may not be as exciting as in Gaming and Entertainment, but they are changing the way we work and will ultimately revolutionize a number of industries.

Recognizing this, Microsoft has been marketing its HoloLens Mixed Reality headset to the enterprise sector. It’s not just that the price point puts the technology out of reach for consumers, or that HoloLens could potentially endow companies with capabilities solving many of the limitations of current technologies—HoloLens is paving the way for the future of work, even allowing industries to take a major leap into that future. Read on for some of the latest and most exciting use cases of HoloLens in enterprise:

CDM Smith

This Boston-based construction firm specializing in large-scale infrastructure jobs believes Augmented Reality will help improve project collaboration and prevent costly mistakes. To that end, CDM Smith has been working with Object Theory, whose software solution attempts to “enable better conversations” by making sure everyone involved in a building project is on the same page, looking at the same thing.

With its ability to overlay holograms onto the real world, AR could prove to be a powerful tool for construction. CDM Smith’s innovation group imagines several potential uses for the HoloLens, including allowing construction crew to view a 3D model on top of the actual building site, in order to see where walls, equipment and materials should go. Essentially, HoloLens will allow construction workers to envision what something will look like before they build it.

In another scenario, HoloLens could help in determining whether a piece of heavy machinery will work in a given room before paying for a crane to place it there. This is somewhat hard to imagine for us non-construction folk but the idea is that before spending time and money putting materials or equipment in place for use, you could visualize aspects of the building execution to prevent setbacks and rework.

But the company is really interested in using HoloLens to close the gap between flat blueprints and 3D structures, and to speed up collaboration. It is hard to communicate three-dimensional ideas with flat screens and pieces of paper. Inevitably there are errors, omissions and order changes due to ineffective communication. With HoloLens, the architects, engineers and on-the-ground crew can discuss and visualize a potential problem by viewing the same 3D model, and quickly come up with solutions together.

*Hear CDM Smith’s Scott Aldridge share his insight on wearable tech at EWTS 2017

ThyssenKrupp Elevator

Urban space is becoming increasingly limited; hence the need to squeeze in as much living and office space as possible in our already packed cities by building upwards. ThyssenKrupp builds and maintains elevators around the world. New Yorkers alone travel up and down the city’s infamous skyscrapers using 71,000 elevators, half of which are more than 20 years old. To keep these in working order, ThyssenKrupp is looking to HoloLens.

A holographic training guide on the HoloLens could teach the company’s elevator repairmen how to work on different elevators and parts. With the headset, they could reference tutorials in Augmented Reality while working on a broken elevator, as well as call a supervisor or part manufacturer for remote guidance on a fix or installation. This will prove to be tremendously rewarding, as a repairman can wait days for assistance from a far-away component company, which means days of elevator downtime in a world where 1 billion people ride elevators each day.

A technician armed with HoloLens could respond to a call better prepared than ever before: Hypothetically, he could view his calls, access all kinds of relevant data on the elevator in question (ex. maintenance history, 3D pictures, safety alerts)—all hands-free, which is a novel capability for these workers.

To start, ThyssenKrupp plans to use Skype on HoloLens to enable on-site collaboration; with the larger goal of building out an entire suite of training tools to empower its 20,000+ engineers. The company sees HoloLens as the future of the elevator repair industry, the key to making its workers more efficient, skilled and protected; to increasing the availability of elevators and escalators; and to serving its customers better. HoloLens is also a “perfect complement” to ThyssenKrupp’s advanced predictive maintenance service—a great example of IoT or the marrying of different technologies.

*Learn more about this use case at EWTS 2017, where ThyssenKrupp’s Thomas Felis will be presenting a case study

Newport News Shipbuilding

Newport News believes shipbuilding will become a digital business, and devices like the HoloLens will enable the company to position itself at the forefront of this inevitable transition. NNS is among the first wave of enterprises to partake in Microsoft’s exclusive HoloLens developer program. Right now, its AR team is demoing the mixed reality headset, developing prototype applications for the technology.

NNS sees great potential in the way HoloLens overlays information onto the real world—upon putting on the headset, its workers will be able to access whatever information and tools they need to do their jobs by simply air tapping or clicking in space. This kind of dynamic information accessed hands-free will free workers from having to reference physical drawings and manuals, making their lives easier if nothing else.

Again, the company is currently “playing around” with HoloLens, considering scenarios in which the device would improve its operations, as in reducing the amount of gear a worker needs to carry up a ladder to just a heads-up display. The possibilities are limitless for shipbuilding – and manufacturing in general – and NNS sees a future in which HoloLens will offer many different “work packages and apps.”

*Dan McDonald of Newport News Shipbuilding will shed more light on NNS’ experimentation with HoloLens at EWTS 2017

 

What a world of difference a holographic set of instructions will make for deskless workers—holograms they can view on top of the real world, on top of a real structure, a real room, a real machine. HoloLens offers this and more, including the ability to manipulate holograms in real space, say, in planning a project, and to share that kind of view with others. An individual worker using the headset to get a job done quicker, with less room for error; a group of professionals using the same device to collaborate more effectively, with less room for miscommunication…The applications that HoloLens unlocks for industry may not have the same wow factor as a game that augments your living room into an alien planet, but for the enterprises above and so many others the device represents the future of work.

Top Use Cases of Augmented and Virtual Reality in Architecture, Engineering and Design

In our last post, we talked about some of the opportunities that new realities – Augmented Reality, Virtual Reality, and also Mixed Reality – present to the AEC Industry, specifically to the design side of the industry (as opposed to construction.) With these technologies, one can view and manipulate virtual elements in real space or immerse him/herself in a digital recreation of a real-world environment. This can help both in designing a building and in enabling others to understand the design.

The user could be the architect in the initial design stage, or the owner/customer in the pitch or approval stage; a group of architects, engineers and designers working together on a single project, or the construction team responsible for turning a building plan into actual architecture—all putting on a headset or heads-up display to visualize and create. The three use cases that follow are prime examples of this, of AR, VR and MR being used to envision new buildings and streamline the design process.

 

TreeHouse

TreeHouse is a Texas-based home improvement startup offering eco-friendly, smart home solutions. For its second massive retail location in Dallas, TreeHouse CEO Jason Ballard wanted a store with zero annual energy costs—a completely sustainable store where consumers could buy sustainability solutions for their own homes. Now, the sustainability initiatives achieved at this, the company’s second big box store, are impressive. Just as impressive, however, is the use of Virtual Reality in the architectural design process that led to the groundbreaking for the store. In fact, VR was so valuable in this use case that TreeHouse intends to employ the technology in every step of future store planning. Let’s see why:

First thing, Ballard hired architecture firm Lake Flato along with a team of designers and technologists charged with the task of creating a virtual reality model of the new store based upon Lake Flato’s design. The team “hacked together” a system combining Unity (a video game development platform), SketchUp (a design program), and an Oculus Rift VR headset. This system would allow Ballard and the Lake Flato architects to virtually walk through the store, trying different design options and configurations and spotting problems before they became real-life headaches.

Big box retail stores typically use a lot of energy; in this project, the design team started with the idea of zero energy and then worked to reverse engineer the means to achieve it. Sustainable building requires conserving as much energy as possible and using renewable energy for any remaining power needs. Since the greatest “energy hogs” in a large store are usually lighting and air conditioning, the team knew they had to create something “bright and cool.” They used virtual reality to realize the key design decision, which was a saw-tooth roof line. Without going into too much detail (you can read more about the design specifics here), the roof design had three major effects: 1) lighting the space, 2) reducing solar heat gain, and 3) maximizing solar energy production; with the added bonus of looking really cool.

Using virtual reality technology in the design process on this project turned out to be both a critical energy- and cost-saving innovation. In one example, VR saved TreeHouse $50,000, by allowing the design team to try out an element of the original design – an elaborate staircase meant to evoke a tree trunk – and see that it would be a “great eyesore.” The staircase was subsequently scrapped for something simpler and cheaper before it was ever built.

As TreeHouse plans its third store, Ballard wants to create a full-time VR position at the company. Not only does he want to keep using virtual reality to design new stores but he also sees potential for the technology to help scout out store locations and design product displays in existing stores. He wants live models of all TreeHouse locations so that he can try out displays without traveling or resorting to trial-and-error, both of which are costly and environmentally-unfriendly. And, of course, Ballard wants to pass the benefits of VR along to TreeHouse’s customers, helping them to design sustainable homes in virtual reality for free.

 

TEG Architects and Thorntons

TEG is an Indiana-based architecture firm, and its client Thorntons is a Kentucky gas station and convenience store chain. TEG has been using Virtual Reality – specifically Samsung Gear VR headsets – to help bring clients like Thorntons into the design process by putting them into buildings that haven’t been built yet, or rather into 360-degree renderings of those buildings.

Before VR and the like, architects could show clients 2D drawings and 3D printouts to help them understand a space and give input on different design elements. In the end, however, a lot was left up to the customer’s imagination. Well, it’s pretty hard to imagine a 100,000-square-foot space (like the one in this use case), but having a 360-degree view of the architect’s design in virtual reality makes communication between architect and client easier.

Not only did VR give Thorntons better insight into and confidence in TEG’s design, but the technology actually revealed quirks of the design that might have been lost on paper, even to the design team. One aspect of a building that can get lost between 2D and 3D are sight lines. For example, in the Thorntons project, a monumental staircase – as originally designed – would have disrupted the sight line from the front to the back of the building; the architects were able to catch this blunder in virtual reality, and make the decision to narrow the staircase.

While the customer was thoroughly impressed by VR (so much so that Thorntons invested in four Samsung Gear VR headsets to use and experiment with internally); TEG found that incorporating the new technology required reformulating existing workflows. So in addition to the obvious technical challenges, adopting VR as a decision-making tool in architecture can necessitate accelerating the design process, or making certain design decisions much earlier on in the project lifecycle in order to create an effective 360-degree view. Another challenge – but one that TEG believes will be resolved as VR becomes more widely used in the industry – was to keep careful track of all design changes, ensuring that any alterations made in the design software appeared in the VR model and vice versa.

The TEG/Thorntons case goes to show that new realities can be a great tool for both architects and clients to make better-informed decisions, making for less unhappy and costly surprises all around.

 

Architect Greg Lynn and the Packard Plant Project

Greg Lynn is the owner of Greg Lynn FORM, a professor at the UCLA School of the Arts and Architecture, and the architect chosen to represent the U.S. at the 2016 Venice Biennale. For that event, Lynn was assigned the project of revitalizing the Packard Plant, an abandoned car factory in Detroit occupying 3.5 million square feet of space. Greg knew the project would stretch the imagination and that he had to be forward-thinking in his commission; so in perhaps the best-known use case of AR/MR in architecture, he decided to use the Microsoft HoloLens mixed reality headset along with Trimble’s building information modeling (BIM) software to develop his design for a new plant.

To begin, Lynn created a standard design model of the abandoned Packard Plant using the Trimble platform. He then used HoloLens to immerse himself in a holographic representation of the factory at scale. With the technology, Lynn was able to virtually navigate the space at all stages of the design – from the initial state in which he found the plant through each proposed design change – visualizing the project as he was designing it and without having to leave his Venice Beach office.

From the get-go, therefore, Lynn immediately understood the scale of the space he was working with because he could enter the Packard Plant in augmented reality and look around. He could also put on the HoloLens to compare the sizes of various structures and get a clear sense of how much space a given structure would take up, which enabled him to develop and perfect the proportions and individual features of his design without trying out different configurations in multiple 2D or 3D (scale) drawings and models. Even more, the technology allowed Lynn to model dynamic components of his design like vehicles and human beings, and make adjustments accounting for how traffic would flow in and around the factory.

Overall, AR/MR provided Lynn with the “perspective and foresight” to make design decisions months earlier in the design process than is normally possible, saving him in time and stress in addition to money and rework. We’ve also mentioned that new realities present an effective means of communicating one’s design to others via a shared experience, and Lynn did in fact incorporate the Microsoft HoloLens into his presentation for the Biennale.

Greg Lynn has been pretty vocal about AR, VR and MR in terms of the future of designing buildings as well as the future of building things. HoloLens personally helped him to both conceptualize and showcase his work, but the architect also sees potential benefits in construction project delivery and communication. In his opinion, new realities solve the greatest problem all architects encounter, which is getting a project from the screen to physical space.

“I’ve spent my whole life trying to get things from geometry into the physical world. HoloLens is going to bridge that gap between two-dimensional and three-dimensional and physical space—and that’s architecture.”

Augmented and Virtual Reality for Architecture, Engineering and Design

 

What is the potential for Augmented Reality and Virtual Reality in the AEC industry? How might viewing virtual objects integrated into one’s physical environment or immersing oneself into a virtual world benefit the AEC sector? In this article, we will focus specifically on the use of augmented and virtual reality technology on head-mounted displays by architects, engineers and designers in the building design process.

There is potential for both AR and VR in all stages of bringing a construction or engineering project to life; pretty much every enterprise involved in a large project could utilize these technologies to improve their own working methods and also in communicating with one another. In the actual building phase, for example, AR could be used to help project managers view plans and schematics overlaid on top of real structures, to allow workers to view step-by-step instructions for how to install something, and even to train future operators of a building. But before construction even gets going – in the initial design process – AR and VR could change the way architects, engineers and designers conceive of, collaborate on, and revise designs.

First thing, let’s talk about what goes into designing a building, including current visualization tools; and let’s define the different “realities,” i.e. AR, VR and also MR (Mixed Reality). Architects and engineers “trade in” the creation and manipulation of the real world, of real environments, real structures; so, essentially, they have to dream in three dimensions and then translate that vision into a two-dimensional representation which is then translated again into a real, three-dimensional space.

The process really begins, however, with collecting information, with visiting a site (or multiple sites) and documenting existing conditions. This information, as well as client demands and requirements, is put into consideration as the architect or engineer begins to brainstorm and develop a preliminary design. In addition to the property or site itself, the designer must think about how the final building will be used and experienced, including how people and objects will move through the space and what materials it will be composed of. Next come graphics, illustrations, plans, diagrams, elevations, even small scale models—lots of paper and lots of time using complex software go into getting the designer’s ideas into a format that can be shared and presented for input and feedback.

For a typical medium-sized or major commercial commission, the design is rarely a one-architect deal. Designing a building is an increasingly collaborative process, so while the initial creative idea usually belongs to one designer, the final design is a team effort (“it takes a village.”) The core design team might consist of an architect, a few engineers (structural, mechanical, services, fire), and specialist designers (landscape, interior, acoustic); and they might be supported by various experts and advisors like an urban planner, a sustainability consultant, and an expert in health and safety. Contributions may also be made by contractors and suppliers. These individuals come together, discuss options and restraints, and revise the original design until a final one is agreed upon. Much of the work undertaken – the multiple design possibilities expressed in a sequence of technical drawings and models – is rejected or aborted in the process.

Visualization technologies like computer-aided design (CAD) and building information modeling (BIM) help architects to plan projects and communicate their ideas; but they’re not always successful in doing so. For one, the software is highly complex; and the digital drawings and models produced are still confined to a two-dimensional screen, which makes it hard – for collaborators and clients as well as the architects themselves – to get a real, accurate sense of how a design will look, function, and take up space in the real world. CAD and BIM have certainly technologically enabled architects and engineers but the reality is that designers are still viewing blueprints on computer screens as well as paper; they’re still using pictures and drawings and plans, and it’s difficult to conceive, revise and execute a project based upon static renderings.

The drawbacks of those current technologies make the process of design reviews rather lengthy and expensive; and inevitably lead to issues down the road, during construction, that cost dearly. The problem lies in using 2D documents and 3D models – both digital and physical scale models – to simulate form and space, understand spatial relationships, and capture the experiential qualities of a building, which must impact the design. The existing tools just aren’t optimal for expressing the architect’s vision or tweaking a design. There’s a lot of room for unanticipated design flaws that will have to be corrected once building has already commenced—errors arising from the architect’s own oversight as well as the client and construction team not being able to clearly imagine the design. On the bright side, new technologies – AR, VR and MR – can help at all stages of a building project, from conception to revision to execution, improving both the individual and group design processes.

Wearable tech, AR/VR, IoT—they’re all about eliminating inefficiencies, bridging knowledge gaps, and streamlining processes in a business. An architect’s work is no different. There are inefficiencies in the design process: Multiple iterations of a design, miscommunications between architect and client or between design team and contractor, trial and error = inefficient. But new realities enable architects, engineers and designers to better and more easily visualize ideas and make quicker, more informed decisions, avoiding costly scenarios like customer dissatisfaction and tear downs/rework during construction.

AR, VR and MR are all related:

  • Augmented reality is additive, overlaying digital content onto the real world. The user is aware of and can still interact with his environment. Devices include the Sony SmartEyeglass, Recon Jet, Epson Moverio BT-300, Vuzix M300 and, of course, Google Glass.
  • Virtual reality is immersive, creating a computer-generated environment that replaces the real world. The user interacts solely within this virtual world. Devices include the Oculus Rift, Samsung Gear VR, and HTC Vive.
  • AR and MR are used somewhat interchangedly, but they are different. Some describe mixed reality as a kind of hybrid between the other two technologies. MR superimposes convincing holographic images onto reality; the holograms are integrated into the user’s environment, can be manipulated, and are even responsive to the real world. Devices: Microsoft HoloLens and Magic Leap.

So what do these technologies allow designers to do? Architects can actually use AR/VR to design buildings, not just to better convey their ideas or collaborate with others but to create and make design decisions. Virtual reality allows a user to virtually inhabit a space in three dimensions; this virtual space can be based upon – even identical to – a real-world environment. So, for instance, at the outset of the design process an architect could go out and capture a physical environment (the property he is designing for) and recreate it in VR, and then design within that virtual space with infinite “room” to experiment and test out different design concepts—all from the comfort of his or her office or studio. The architect could also take a design developed on an architectural platform like Autodesk and produce it in VR, allowing him – and others members of the design team – to virtually inhabit and manipulate a building that does not yet exist.

That is a much more powerful (and effective) means to visualize one’s ideas and evaluate design possibilities. An architect can translate a CAD model into an interactive walk through: Instead of viewing blueprints of 3D models on a 2D screen, he can put on, say, an Oculus Rift to virtually experience an architectural plan; achieving a far better sense of scale, form and space, of the physical limitations of a space, of how someone will move through a building, of how a particular design component will look or function, and the flaws in a design that might not otherwise be realized without multiple scale models or even until construction had gotten underway. In addition, the owner/customer and the construction team will be able to more comprehensively understand the building design before it is executed thanks to virtual reality technology.

Augmented reality and mixed reality will also help enhance and speed up the design process at the outset of a project, with the trickle-down effect of minimizing delays at the construction stage due to design errors and changes. AR and MR present their own unique opportunities to view and manipulate digital representations or facsimiles of physical realities, but these representations are not immersive. Apple CEO Tim Cook recently compared AR and VR, saying that augmented reality “gives the capability for both of us to sit and be very present talking to each other, but also have other things visually for both of us to see;” virtual reality, on the other hand, “sort of encloses and immerses the person into an experience.” So while VR is great for solo design, AR and MR are perhaps better for group design and collaboration.

So, for instance, mixed reality can be used to simulate a meeting or collaborative space, allowing the various professionals who make up the design team on a complex building project to work together in real time, viewing and interacting with the same virtual model – or rather with holograms superimposed on a physical model or integrated into a physical space (like the building site) – via a HoloLens headset; and they don’t have to be in the same room to do so. Essentially, with AR/VR/MR, the process of design reviews can be completely virtualized–all existing visualization tools and the 3D models they create can be “pushed” beyond the 2D screen into virtual environments or projections on physical models that designers can interact with(in) in real time.

In conclusion, the ability to immerse people into virtual worlds – specifically into digital simulations of proposed buildings – will be a game changer for the AEC industry. New realities – augmented, virtual, mixed – offer a new, far superior level of real-world scale, proportion and perspective over current tools. These technologies will empower architects, engineers and designers to become more innovative by freeing them from the limitations of 3D models in 2D formats and bringing their 3D dreams to life.

Stay tuned for our next post, in which we will share some of the top use cases of Augmented and Virtual Reality for architecture, engineering, and design.

 

About EWTS 2017:

The 3rd annual Enterprise Wearable Technology Summit 2017 taking place May 10-12, 2017 in San Diego, California is the leading event for wearable technology in enterprise. It is also the only true enterprise event in the wearables space, with the speakers and audience members hailing from top enterprise organizations across the industry spectrum. Consisting of real-world case studies, engaging workshops, and expert-led panel discussions on such topics as enterprise applications for Augmented and Virtual Reality, head-mounted displays, and body-worn devices, plus key challenges, best practices, and more; EWTS is the best opportunity for you to hear and learn from those organizations who have successfully utilized wearables in their operations. 

Join the Enterprise Wearable Technology Community (LinkedIn Group)