This repository contains serves the purpose of an appendix enclosed to master thesis:
title: Robotic Object Manipulation in Virtual and Physical Environment
author: Ondrej Svec
supervisor: Tomas Jochman
keywords: robotic manipulation, industrial metaverse, digital twin, 6D pose estimation
abstract:
Robotic manipulation using cameras is a critical advancement
in the automation industry, enhancing the flexibility and efficiency
of robotic systems. This thesis investigates the integration
of camera-based robotic manipulation within virtual and physical
environments, focusing on the capabilities of NVIDIA Omniverse
to improve simulation accuracy. The study begins by choosing
and implementing a 6D pose estimation algorithm using a static RGB
camera. A robotic cell is designed and tested in both
virtual and physical settings, with the virtual environment created
using Process Simulate and KUKA.OfficeLite, and subsequently refined
in NVIDIA Omniverse for high-fidelity rendering.
A major component of the thesis involves developing tools within NVIDIA
Omniverse: a dataset generation tool for machine learning model training,
and an interactive application serving as a Digital Model. The proposed
modular architecture includes a high-level controller and a unified
interface, ensuring scalability. Experimental results demonstrate
that synthetic datasets can be used to effectively train object
detection models. The consistency of performance across
virtual and physical domains validates the digital model's authenticity.
Recommendations for enhancing setup performance include using a higher
zoom camera lens or redesigning the cell with a robot-mounted camera.
Faculty of Electrical Engineering, Czech Technical Unversity in Prague
2024
All code and data developed and created in this thesis is stored in a private repository for the purpose of protecting a know-how created in CIIRC Laboratory Testbed for Industry 4.0. For access to this repository, feel free to reach out to ondrej.svec@cvut.cz, tomas.jochman@cvut.cz or pavel.burget@cvut.cz.
