Generating Synthetic Datasets to Enhance Robotic Perception in Food Pick-and-Place Tasks

Obada, Al Aama

WEKO3

lat lon distance

[[sub_check.contents]]

[[sub_radio.contents]]

Field does not validate

[[sub_attr.contents]]　

インデックスツリー

アイテム

フードピックアンドプレースタスクにおけるロボティック認識向上のための合成データセットの生成

https://doi.org/10.18997/0002000942

名前 / ファイル	ライセンス	アクション
sei_k_483.pdf (4.5 MB)

アイテムタイプ

学位論文 = Thesis or Dissertation(1)

公開日

2024-09-03

資源タイプ

資源タイプ識別子

http://purl.org/coar/resource_type/c_db06

資源タイプ

doctoral thesis

タイトル

Generating Synthetic Datasets to Enhance Robotic Perception in Food Pick-and-Place Tasks

言語

タイトル

フードピックアンドプレースタスクにおけるロボティック認識向上のための合成データセットの生成

言語

eng

著者

Obada, Al Aama

抄録

内容記述タイプ

Abstract

内容記述

The main purpose of this research is to enhance robot vision to improve robot performance in food classification, detection, and pick and place into the lunch box.
The scale performance of neural networks can be improved by training them with large datasets. However, constructing a large-scale food dataset requires significant time and effort. This thesis introduces a Cycle Generative Adversarial Network (Cycle-GAN) as a solution to generate a large pseudo-realistic food dataset. This approach utilizes a large number of simulated images and a small number of real images, minimizing the need for traditional techniques of real data collections. To capture real RGB-D images of various food samples, an RGB-D camera and a turntable were employed from two different angles. Simulated images of 3D food models were generated using 3D modeling software and a 3D scanner, following the same configuration as the captured real images. Subsequently, a VGG-16 model was trained and evaluated using the generated RGB dataset. The results demonstrate that Cycle-GAN is an effective tool for generating pseudo realistic images, significantly reducing the efforts required for real image acquisition.
Although the proposed method successfully generates a large food scale dataset, a trained Cycle-GAN suffers from low-resolution output images due to the resolution of the input images. This limitation might contribute to the low accuracy of the neural network. Furthermore, more diverse feature variations of the food are necessary to enhance the performance of Cycle-GAN. However, providing additional variation images poses a time-consuming challenge. Moreover, the manual annotation of the dataset remains ineffective, emphasizing the need for an efficient dataset generation method with automatic annotation. Improving the accuracy of object detection is a crucial aspect to address.
To implement object detection and a pick-and-place system, the Robot Operating System (ROS) was utilized due to its system integration flexibility. The RGB images obtained from the RGB-D camera were employed for the object detection task by feeding them into the trained You Only Look At CoefficienTs (YOLACT) neural network. A PyBullet simulator software was used to generate simulated images based on 3D food models. Furthermore, to generate a larger dataset, data augmentation steps such as flipping, zooming in and out, brightening, changing contrast and coloring effects were applied to the output of simulator simulated images. YOLACT neural network was trained using the generated simulated dataset images.
For the robot demonstration, a pick-and-place test was conducted using different food object classes (real objects). The YOLACT neural network accurately recognized the objects, and the center of these objects was determined for subsequent object grasping points. Then, robot's joint values were then calculated to perform the object pick-and-place task. The results indicate that the YOLACT neural network quickly and successfully detects objects with an accuracy of up to 94.56%. This demonstrates the robot's proficiency in detecting and grasping objects, achieving high accuracy and successful pick-up rates across various object classes. In addition, our proposed YOLACT neural network succeeded in detecting the items in a group at a time.

内容記述タイプ

TableOfContents

内容記述

1 INTRODUCTION| 2 LITERATURE REVIEW| 3 RESEARCH METHODOLOGY| 4 EXPERIMENTAL RESULTS AND DISCUSSION| 5 CONCLUSIONS AND FUTURE WORK

備考

内容記述タイプ

Other

内容記述

九州⼯業⼤学博⼠学位論⽂学位記番号：生工博甲第483号学位授与年⽉⽇: 令和6年3⽉25⽇

学位授与番号

甲第483号

学位名

博士（工学）

学位授与年月日

2024-03-25

学位授与機関

学位授与機関識別子Scheme

kakenhi

学位授与機関識別子

17104

学位授与機関名

九州工業大学

言語

学位授与年度

内容記述タイプ

Other

内容記述

令和5年度

出版タイプ

VoR

出版タイプResource

http://purl.org/coar/version/c_970fb48d4fbd8a85

アクセス権

open access

アクセス権URI

http://purl.org/coar/access_right/c_abf2

ID登録

10.18997/0002000942

ID登録タイプ

JaLC

戻る

views

See details

	Views

Versions

Ver.1

2024-09-03 02:43:30.426714

Show All versions

Cite as

Other

エクスポート

OAI-PMH

JPCOAR 2.0
JPCOAR 1.0
DublinCore
DDI

Other Formats

インデックスリンク

インデックスツリー

アイテム

フードピックアンドプレースタスクにおけるロボティック認識向上のための合成データセットの生成

× Obada, Al Aama

Versions

Share

Cite as

Other

エクスポート

コミュニティ

メニューを最小化

インデックスリンク

インデックスツリー

アイテム

フードピックアンドプレースタスクにおけるロボティック認識向上のための合成データセットの生成

× Obada, Al Aama

Versions

Share

Cite as

Other

エクスポート

コミュニティ