As progress in automation continues at breakneck speed, discussions about whether robots would ever replace humans in industrial settings have become mainstream. While robots can automatically perform repetitive tasks, they still rely heavily on human input for more precise pick-and-place operations. Such robot-human teamwork is exemplified by a vision-based control method called “visual servoing”, where a user guides a robot’s limb and gripper in performing delicate manipulation tasks. The technique uses feedback information extracted from a vision sensor to control the motion of a robot.
Traditional visual servoing techniques require humans to manually select features to use as feedback for the movement, making the process labour-intensive and limiting the robots to operate only in environments they have been pre-programmed to perform in.
Fortunately, algorithms called deep neural networks could eliminate human input, enabling robots to program themselves. However, such algorithms require vast amounts of real-life training data and only work on limited functions, making them unprepared to fully take over from human controllers.
To address the functional and data limitations of current systems, En Yen Puang, a Research Engineer at A*STAR’s Institute for Infocomm Research (I2R), together with colleagues Keng Peng Tee and Wei Jing developed a novel keypoint-based visual servoing framework called KOVIS.
KOVIS stands out from other visual servoing frameworks because of how easy, fast and labour-free it is when deployed on new tasks. Compared to its predecessors, KOVIS boasts a higher efficiency, using a deep auto-encoder algorithm to learn and encode visual features without any human input.”
– En Yen Puang, Research Engineer at A*STAR’s I2R
Uniquely, the system represents objects as key points based only on essential geometric information. This feature makes KOVIS quicker and easier to deploy in new environments as it eliminates the appearance variations of an object in different settings. Consequently, KOVIS can quickly pick up the simple ‘peg-and-hole’ relationship between the robot’s gripper and the target object.
Another major benefit of KOVIS is how little real-life training data the system needs to perform these tasks, with the framework trained entirely using synthetic images captured in simulated conditions. While these objects may appear different in reality, KOVIS works around this by generalizing what it learns so that it can recognise the same objects in different environments.
Despite being trained on only synthetic data, the system successfully performed real-world manipulation tasks like gripping a mug by its handle and inserting a peg and screw into a designated hole with a remarkable 90% success rate.
Currently, KOVIS is being used in Collab-AI, a research project that aims to advance developments for safer and more efficient human-robot collaboration in production and manufacturing settings. In the future, the researchers hope to expand the applications of KOVIS from simple manipulation tasks to precise routine manoeuvres such as the docking and landing of mobile robots.
As reported by OpenGov Asia, A*STAR’s research and development (R&D) efforts have enabled the country to address emerging challenges and seed capabilities in critical areas. But while R&D is an unquestionably valuable resource, capturing value through commercialising those research efforts is equally important.
A*STAR’s commercialisation initiatives have since resulted in a steady stream of landmark patents and successful collaborations, all of which have set Singapore on the path towards being an innovation-driven, knowledge-based economy, rich in job and research opportunities alike.
Since the genesis of its R&D journey decades ago, Singapore has firmly established a vibrant research and innovation ecosystem. Through the Enterprise division, A*STAR will build upon those successes and deliver even more value through science and technology.
