Augmented reality for assembly guidance using a virtual interactive tool

The application of augmented reality (AR) technology for assembly guidance is a novel approach in the traditional manufacturing domain. In this paper, we propose an AR approach for assembly guidance using a virtual interactive tool that is intuitive and easy to use. The virtual interactive tool, termed the virtual interaction panel (VirIP), is an easy-to-use tool that can be used to interactively control AR systems. The VirIP is composed of virtual buttons, which have meaningful assembly information that can be activated by an interaction pen during the assembly process. The interaction pen can be any general pen-like object with a certain colour distribution. It is tracked using a restricted coulomb energy (RCE) network in real-time and used to trigger the relevant buttons in the VirIPs for assembly guidance. Meanwhile, a visual assembly tree structure (VATS) is used for information management and assembly instructions retrieval in this AR environment. VATS is a hierarchical tree structure that can be easily maintained via a visual interface. It can be directly integrated into the AR system or it can alternatively act as an independent central control station on a remote computer to control the data flow of the assembly information. This paper describes a typical scenario for assembly guidance using VirIP and VATS. The main characteristic of the proposed AR system is the intuitive way in which an assembly operator can easily step through a pre-defined assembly plan/sequence without the need of any sensor schemes or markers attached on the assembly components. Several experiments were conducted to validate the performance of the proposed AR-based method using a monitor and a head-mounted display. The results show that the AR-based method can provide an efficient way for assembly guidance.