A damaged reactor building in a nuclear power plant does not sound like the world's easiest place to perform maintenance and renovation work. The risk factors of the environment and safety critical nature and complexity of the systems in question set high demands on the work to be performed.
What if the maintenance staff could practice in advance all the maintenance work required and in an environment almost entirely identical with the reactor building itself? Or if it would be possible one way or the other to have detailed maintenance instructions for each device to be repaired in real-time in the maintenance personnel’s line of vision whilst they are on site?
Nowadays, performing maintenance work is exciting and meaningful thanks to virtual background systems. This is what the concepts of Virtual Reality (VR) and Augmented Reality (AR) really mean: digital information and elements produced with computer graphics are added to the surrounding reality, which the user will view via see-through displays. The video below provides an excellent demonstration of what this is all about.
Technical solutions to make virtual and augmented reality a part of modern environmental environment already exist. Several different models of virtual glasses, helmets, gloves etc. are available.
For example, Google glasses, the Oculus Rift 2 headset or Epson Moverio smart glasses enable the achievement of virtual reality at a reasonable cost with prices being in the region of 500–1,000 euros. Increasingly, mobile devices familiar to consumers are also utilised, such as smart phones, tablet PCs and smart watches. Intelligent software combines augmented reality information on video with the help of, for example, geo-location, image recognition and object sensoring.
The computing power of devices has increased at an incredible pace; they are increasingly networking while their price level has decreased.
This combination, added to the fact that an increasing amount of data is available and possible to link with other data in other systems, has boosted development in the utilisation of augmented reality both in industry and consumer markets.
3D modelling of devices, machines, systems and even entire factories is possible with advanced tools and development environments. Models are becoming increasingly realistic in appearance when drawn with photographically detailed graphics. It is also possible to abstract parts of the devices, clarify them, make them transparent and show invisible data in their colours, such as temperature or the calculated remaining life cycle of a component.
Virtual reality is used increasingly in factory environments. The assembly of NASA spacecraft is tested in a 3D simulation laboratory, where the virtual world is entered through avatars and problems are located before the actual construction stage of the craft. Solving problems in advance brings large savings and may save lives. It is the vision of Airbus to model the aircraft factory of the future in its entirety and at all its stages. Production optimisation and efficient planning of work phases are among the factors that drive investment in this.
In Finland, VTT, the Technical Research Centre of Finland, conducts extensive research on this subject matter. VTT is focusing especially in the applications of Mixed Reality (MR). The concept of mixed reality can be seen as a sliding continuum of lightly augmented reality towards a more complete immersion in virtual reality. For example, VTT studies the interaction between human beings and machines, design of user interfaces in virtual environments, simulation of human-machine interaction and product management.
3D modelling and user interfaces
Virtual environments require an entirely new kind of perspective in order to bring a realistic and easy-to-use user experience in the user interface visible to the user. Now we are moving from place to place by walking and moving our eyes, not by using a mouse. 3D modelling and real-time graphics set additional challenges to software designers. Bad quality picture or an unrealistic user experience may water down the whole idea and make the user feel nauseous in which case the added value enabled by technology in the user experience cannot be fully utilised.
There are advanced tools such as Unity3D for designing 3D user interfaces and models. In addition to Unity3D enabling the production of great games and 3D user interfaces, one of its big advantages is the possibility to publish the finished work on almost any platform: PC, tablet PCs, mobile devices, mobile phones and browsers.
The most recent version, 5.0, of Unity3D will be published at the end of 2014 and it will include several improvements, such as WebGL and HTML5 support and increasing number of platforms where it will work directly in the browser, improved modelling of shadowing/lighting, 64 bit development environment and other improvements.
Unity3D Integrated Development Environment
Virtual simulators for using machinery and equipment used for teaching and training purposes are the best contemporary examples of 3D modelling. For example, forestry and mining machinery simulators with genuine controls habituate the drivers to correct steering movements and how to use the machine smoothly before taking control of actual machinery.
Maintenance business and augmented reality
One large application area where augmented reality is brought in strongly by even large players is the machine and equipment maintenance business. BMW has already created an application for mechanics that shows how the car model in question is serviced, with detailed instructions conveyed through virtual glasses.
The display of the repair shop's diagnostic reader no longer displays the bar code read from the device's data transfer channel, but the mechanic will see directly the diagnostic context which is a more illustrative way of showing where the problem lies in relation to its environment. This will make it easier to understand the causes behind the fault. This truly illustrative example shows how augmented reality can be used to help a mechanic to service a pump.
It is also a part of augmented reality that the device or machine can illustrate in one way or other where the fault is located to the user/mechanic. Examples of this include paper jam user interfaces in printers and illustrative pictures of the fault location provided to the user.
“Service subject” can also be a human being; virtual surgery is already used to help the work of surgeons. Patient data, such as X-rays, 3D models on intestines, treatment history, etc. are continuously available and visible when needed in the virtual glasses of the surgeon during operation. The surgeons may also communicate with their colleagues who can provide assistance in difficult situations during the operation. Moreover, they can practise operations in a virtual operating theatre in advance like the drivers of forestry and mining machines.
Fox TV is currently broadcasting a series entitled Intelligence. In the show, a microchip has been embedded in the brain of the main character, a top agent in CyberCom, agovernment agency. With the microchip, the agent can access data networks and make extremely rapid calculations to combine the data received with the real world while creating a virtual reality 3D model. He can “enter” the 3D virtual model he has created and move there while looking for clues to solve crimes.
In fact, a microchip in the brain is not even science fiction; Cochlear ear implants and cameras and eye implants linked with the visual system are current examples of it. Israeli scientists have managed to manufacture an electronic cerebellum for a rat and have returned the ability to move to a rat suffering from brain damage. An electronic hippocampus, which stores memories in the human brain, is going to be tested in humans next year. When we think about the above-mentioned brain implants, we are not far from the case of the agent in the TV show.
If microchips implanted in the brain sound like a bit too big a leap forward and impractically large virtual glasses too cumbersome to use, then why not use contact lenses with implemented augmented reality technology? It is possible to get the same functionality integrated in a much smaller and inconspicuous space.
Advantages and benefits of augmented reality
The technology and data required already exist to introduce augmented reality in an industrial environment. The potential uses are almost limitless. Benefits gained in industrial applications in particular include:
- Cost savings in designing and implementing the project first in virtual form
- Speeding up the operational processes of servicing and maintenance and decreasing costs and time used in work phases
- Performance of demanding maintenance duties in a controlled manner stage by stage without any comprehensive special competence
- Advance modelling and testing of safety-critical systems
- Mapping and virtual testing of various alternatives before production
- Training to use a system or a piece of equipment
- Cooperation of product development, sales, management and customers through visualising things