Week 07/07/2024 - 13/07/2024

This week, I continued writing literature summary for the following papers:

1. A Mobile Robotic Arm Grasping System with Autonomous Navigation and Object Detection
2. Robotic Arm Grasping and Placing Using Edge Visual Detection System

Title: A Mobile Robotic Arm Grasping System with Autonomous Navigation and Object Detection

Link: https://ieeexplore.ieee.org/abstract/document/9624636

Objective/Aim:

Develop a mobile grasping system for robotic arm.

Method:

First, the YOLOv4 algorithm is introduced to identify and locate the individual objects. Secondly, the output frame can be used as an input of the GrabCut algorithm to segment the objects from the background for calculating the capture orientation. 

Thirdly, the conversion between the coordinate systems is conducted to acquire the positions of objects, and the obtained information is sent to the robotic arm to complete the grasping task on the Robot Operating System (ROS).

Process flow:

Robot performs SLAM through lidar to reach the desired position.

System will turn on the camera to identify and locate the objects, and compute the pose of each object.

User enters the object category to be captured, and the system will make a judgment.

Computer System/Software:

The system is implemented in Ubuntu16.04 on a Lenovo 2.6 GHz processor RTX 2060 with 16 GB RAM and 8 GB video memory

Setup:

The robot is equipped with a Hokuyo lidar, RealSense D435 camera.

Evaluation:

Experiment is done on 7 objects of different colour and shapes placed in the scene at different positions and angles. 3 measurements are obtained: grasping times, successful times and success rate.

Conclusion: 

The results show that the proposed method can complete the robot’s navigation to the target position to realize the grasping function of the specified object.

Future works/considerations: 

Increase the accuracy of the system by improving the moving object extraction algorithm.

Title: Robotic Arm Grasping and Placing Using Edge Visual Detection System

Link: https://ieeexplore.ieee.org/document/5899202

Objective/Aim:

Aims to develop the object visional detection system that can be applied to the robotic arm grasping and placing.

Method:

Accurately measure the relative distance between the object and robot arm using the edge detection algorithm with a camera device.

Uses image processing and shape matching to identify object

Placement of object is done using the same image processing method.

Computer System/Software:

Borland C++ Builder 6 programming was used for the image processing

Setup:

The robotic arm in the study uses seven RX-64 servomotors and line moving rail.

The webcam with the resolution of 320 × 240(pixel) is installed at the upper location of arm clip.

Evaluation:

Grabbing experiments using four kinds of color (red, yellow, green, and blue) ball for

each one, each two, and each three, respectively, indoor and the hall under different environment.

Conclusion:

Accomplished three tasks, i.e., “find the ball”, “clip the ball” and “put the ball”. 

More importantly, this system is a really streamlined and inexpensive system.

Future works/considerations:

Focus on finding, grasping and placement of objects in different shapes and colors.

Found two more papers to read. These papers focus more on placement:

1. Predicting Stable Configurations for Semantic Placement of Novel Objects (https://arxiv.org/abs/2108.12062)
2. ReorientBot: Learning Object Reorientation for Specific-Posed Placement (https://arxiv.org/abs/2202.11092)

10/7/2024 Today, we did an elevator pitch at the workshop, explaining what the objective and goals of my project are. We also listed down the hardware that we will need for our projects. Tentatively, mine are:

1. UR10

2. RGBD Camera

3. Raspberry Pi

We also talked about the Gantt chart today. This is the Gantt chart I came up with.

Gantt Chart

Rough sketch of how the setup would be:

11/7/2024

Installed the SDK file for the Intel RealSense D435 RGBD camera. 

However, encountered issues with it:

Upon searching for solutions, similar issues seem to indicate that the above error is due to long term storage:

https://github.com/IntelRealSense/librealsense/issues/10708

The camera was left on for a while as per suggestion on the forum, but the most I could get out from it was:

Will need to ask for another camera to test.

Meanwhile, discovered helpful videos for object detection and depth mapping.

12/7/2024

Found a video that would probably be helpful for the placement part:

13/7/2024

Title: ReorientBot: Learning Object Reorientation for Specific-Posed Placement

Link: https://ieeexplore.ieee.org/abstract/document/9811881

Objective/Aim:

To present a robotic system that can rearrange objects to a specific goal state, including reorientation and regrasping for final placement. 

Method:

Systems runs detection, post estimation and motion planning to rearrange objects. 

i) 6D pose estimation and volumetric reconstruction 

ii) Motion waypoint selection that pairs start and end waypoints via learned filtering

iii) Trajectory generation by motion planning using the selected waypoints.

Computer System/Software:

ROS, PyTorch to implement the learned models. PyBullet as physics engine to simulate behaviour of objects. 

Setup:

Franka Emika Panda robot, Realsense D435 mounted on robotic arm.

Two types of suction gripper:

1. I-shape

2. L-shape

Evaluation:

Done in both simulation and real-world. 6 large/medium sized objects were used in both simulation and real-world experiments: drill, cracker box, sugar box, mustard bottle, pitcher and detergent. 

Conclusion:

The authors’ system improves in both efficiency and success rate and has shown capable, dynamic reorientation for significant rotation and precise placement in various target configurations. 

Future works/considerations:

Possible combinations of learning models with traditional motion planning.