WIMI Hologram Academy: Research and Implementation of Electromechanical Operation Simulation System Based on Virtual Reality
HONG KONG, July 14, 2022 (GLOBE NEWSWIRE) — WIMI Hologram Academy, working in partnership with the Holographic Science Innovation Center, has written a new technical article describing their research and implementation of electromechanical operation simulation system based on virtual reality. This article follows below:
Virtual reality-based electromechanical simulation technology, through the establishment of a three-dimensional simulation structure of virtual reality scene and connected with virtual reality equipment such as posture capture systems, and head-mounted displays, can observe the operating conditions of equipment from different viewpoints, relative position relationships, simulated follow-up tracking monitoring and live-action roaming, and a variety of interactive operations. It realizes multi-person collaborative operation of electromechanical systems and greatly facilitates operators familiar with actual electromechanical equipment. Scientists from WIMI Hologram Academy of WIMI Hologram Cloud Inc.(NASDAQ: WIMI) studied the research and implementation of virtual reality-based electromechanical operation simulation systems.
Virtual reality-based electromechanical system simulation mainly utilizes virtual reality technology, intelligent wearable technology and distributed computer simulation technology in order to achieve full-process interactive simulation analysis and immersive real-world simulation operation simulation of the system. The system constructs virtual models in the actual size of real scenes. In the design and development stage, the system can be used to verify whether the equipment structure, layout, and control process are reasonable, and then provide a design basis for the adaptation study of the equipment. The system supports multi-person collaborative operation, which can provide important technical support for operators to be fully familiar with the function, structure and composition of the actual equipment, and to quickly master equipment operation, maintenance, repair and emergency safety handling methods, and is an important equipment as an auxiliary training and simulation operation training after the equipment manufacturing is completed.
1.Hardware system design scheme
Motion capture devices, data grips and data gloves are included in the posture capture control equipment.
2. Software solutions
2.1 Overall software structure
The main functions of the software are: driving of 3D models, simulation of actual working conditions, switching of scene views, remote communication, capturing equipment data collection and character model driving, etc. The software includes operation simulation software, human-computer interaction interface software, model calculation and interface control software, and control simulation software. The HCI interface is mainly responsible for acquiring character movement, hand joint, and posture information. It also drives the character model and renders the scene to the corresponding display device in real time. The operation simulation software is responsible for handling the simulation, driving the 3D model and simulating the actual working conditions. The model calculation and interface control software is responsible for data analysis, processing, and transmission to the corresponding receiver. The control simulation software is able to receive the commands entered by the operator at the control panel and control the entire flow of the 3D scene and model accordingly, while correctly displaying the status parameters in the system.
2.2 Operation simulation software
2.2.1 The overall structure of operation simulation software
The virtual reality-based electromechanical system simulation is built on the Unity3D platform, supplemented by scripting programs written in C#. The system enables model driving, calculation of physical effects, animation rendering, human-computer interface control, communication, and motion response. The interface control software receives input commands and motion control parameters in real time and sends them to the digital simulation calculation program. The digital simulation calculation program processes the input commands and motion control parameters, calls its corresponding 3D model and digital model, and realizes the simulation physical effects such as model collision, articulation, and force with the assistance of the physics engine, calculates the parameters of the model such as speed and direction in real time, and sends these physical state parameters to the virtual reality simulation subsystem. The virtual reality simulation subsystem renders and displays the 3D model according to the corresponding state parameters of each model, and sends the model feedback parameters to the digital simulation calculation program, which sends the state information (position, velocity, etc.) to the communication interface control server software after processing by the digital simulation calculation program.
2.2.2 3D and digital models
The virtual reality-based electromechanical system simulation has 3D models and digital models of each subsystem inside. The 3D model is built according to the 1:1 scale of the physical model, and the appearance is consistent with the physical model. The digital model is based on the 3D model and adds the corresponding physical characteristics, such as rigid body, joint, collision body and material. The digital simulation calculation program calls different 3D models and corresponding digital models according to different control instructions, and initializes the state values and flag bits of the digital models in the initial stage. After the system starts, the control flag bits are scanned frame by frame according to the frame rate interval, and the operating conditions of the equipment are updated in the virtual reality simulation subsystem, and physical calculations are performed according to the fixed physical time interval, and the physical state of the equipment is refreshed.
2.3 Human-computer interaction interface
A virtual reality scene with a 3D simulated structure is created and connected with human-computer interaction devices such as posture capture systems, data gloves, head-mounted displays and data handles. Engineers can immerse themselves into the virtual reality scene, realize interactive operation of the devices and interact immersively with the virtual reality scene of the 3D simulated structure. The posture capture system is mainly used to capture the motion process of the human torso, hands and feet and other major joints. Based on the posture capture system, the walking and movement of the person in the simulated equipment scene is realized. The character movement and posture data are collected through the relevant capture software and communicated with the professional software through the transmission control protocol / inter-network protocol. The character movement and posture data are corrected in the professional software to derive the position and rotation information of the joints, which drive the human model.
The data glove can accurately capture the motion process of the human hand and finger joints, enabling more delicate operations in simulated device scenarios. The finger bending angle is obtained through sensors on the glove, and a communication program is written in C# to communicate with specialized local software. The data correction is then completed in the software and the character hand motion is driven in real time. The data handle enables motion control of the virtual device and virtual human body, mainly for operation verification, spatial and layout analysis, etc. The data handle has a special interface in the professional software, and the control program can be written to define the function of the handle buttons to achieve a wide range of movement of the character and switching of the viewpoint. The head-mounted display is connected to the software through an official development kit, and dual cameras are set in the head-mounted display to simulate binocular vision of the human eye, and the corresponding parameters are set in the relevant software to complete immersive human-computer interaction.
Through posture capture devices, gloves and head-mounted displays, operators can use virtual characters to complete interactions in virtual scenes. The implementation of interaction technology is mainly realized through collision detection, trigger collision detection, and logic processing algorithms. During the actual operation of human-computer interaction, the phenomenon of characters crossing virtual objects in virtual scenes can occur due to the conflict between the physical engine and external character capture. Considering the reasonableness and reliability of the actual process, the following processing will be carried out. First, when the character in the virtual scene is about to cross the object, an illegal operation prompt will appear on the screen, and the prompting process is not in line with reality. Secondly, when the character has crossed the object in the virtual scene, the character capture system will stop working and the character model will be stationary. Finally, when the character in the virtual scene is about to leave the object, the character capture can be resumed and the character model can be driven when the distance is judged to be in accordance with the conditions.
3.Conclusion
The design scheme of this system realizes interactive simulation analysis and immersive real-world simulation operation simulation of the whole process of electromechanical system. In the product design and development stage, the system can be used to verify whether the equipment structure, layout and control process are reasonable, and provide design basis for equipment adaptability study. At the same time to help operators familiar with equipment functions, equipment structure and composition. It provides important technical support for operators to quickly master equipment operation, maintenance, repair and emergency safety handling methods, and can also be used to assist training and simulation operation training after equipment manufacturing is completed.
Founded in August 2020, WIMI Hologram Academy is dedicated to holographic AI vision exploration and researches basic science and innovative technologies, driven by human vision. The Holographic Science Innovation Center, in partnership with WIMI Hologram Academy, is committed to exploring the unknown technology of holographic AI vision, attracting, gathering, and integrating relevant global resources and superior forces, promoting comprehensive innovation with scientific and technological innovation as the core, and carrying out basic science and innovative technology research.
Contacts
Holographic Science Innovation Center
Email: pr@holo-science. com
