GWANGJU, South Korea, April 20, 2022 /PRNewswire/ — When AI systems encounter scenes where objects are not fully visible, they must make estimates based only on the visible parts of the objects. This partial information leads to detection errors, and large training data are required to correctly recognize such scenes. Now, researchers at the Gwangju Institute of Science and Technology have developed a framework that allows robotic vision to successfully detect such objects the same way we perceive them.
Robotic vision has come a long way, reaching a level of sophistication with applications in complex and demanding tasks, such as autonomous driving and object manipulation. However, it still struggles to identify individual objects in cluttered scenes where some objects are partially or completely hidden behind others. Typically, when dealing with such scenes, robotic vision systems are trained to identify the occulted object based only on its visible parts. But such training requires large data sets of objects and can be quite tedious.
Associate Professor Kyoobin Lee and Ph.D. Student Seunghyeok Returning from Gwangju Institute of Science and Technology (GIST) in Korea, they encountered this problem when developing an artificial intelligence system to identify and sort objects in cluttered scenes.” We expect a robot to recognize and manipulate objects it has never encountered before or been trained to recognize. In reality, however, we have to manually collect and label the data one by one, because the generalization of deep neural networks highly depends on the quality and quantity of the training dataset.“, says Mr. Back.
In a new study accepted at the 2022 IEEE International Conference on Robotics and Automation, a research team led by Prof. Lee and Mr. Back developed a model called “modeless instance segmentation of invisible objects” (UOAIS) to detect occulted objects in cluttered scenes. To train the model to identify the geometry of the object, they developed a database containing 45,000 photorealistic synthetic images containing depth information. With this (limited) training data, the model was able to detect a variety of occluded objects. When it encounters a cluttered scene, it first selects the object of interest, then determines if the object is occluded by segmenting the object into a “visible mask” and a “amodal mask.”
The researchers were excited about the results. “Previous methods were either limited to detecting only specific types of objects, or to detecting only visible regions without explicitly reasoning about occluded areas. On the other hand, our method can infer the hidden regions of occluded objects like a human vision system. This helps reduce data collection efforts while improving performance in a complex environmentcommented Mr. Back.
To allow “occlusion reasoning” into their system, the researchers introduced a “Hierarchical Occlusion Modeling (HOM) scheme, which assigned a hierarchy to the combination of multiple extracted features and their order of prediction. By testing their model against three benchmarks, they validated the effectiveness of the HOM scheme, which reached the state of performance art.
The researchers are optimistic about the future prospects of their method. “Perceiving invisible objects in a cluttered environment is essential for modeless robotic manipulation. Our UOAIS method could serve as a reference on this front,said Mr. Back.
This certainly looks like a giant leap for robotic vision!
Title of the original article: Segmentation of modeless instances of invisible objects via hierarchical occlusion modeling
Log: IEEE International Conference on Robotics and Automation
About Gwangju Institute of Science and Technology (GIST)
please visit: http://www.gist.ac.kr/.
Alone Hye Kim
82 62 715 6253
SOURCE Gwangju Institute of Science and Technology