Skip to content

(1) 3D scan object by Baxter. (2) Label objects automatically by depth camera and (3) train Yolo. (4) [TODO; NOT DONE YET!!!] Finally, detect object and fit 3D model to know the 6D pose.

Notifications You must be signed in to change notification settings

felixchenfy/Detect-Object-and-6D-Pose

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 

Repository files navigation

Winter Project at Northwestern University

Feiyu Chen
2019. Mar 22th

The goal of the project is to locate the 6D pose of an object from the RGB-D image. This task is divided into 4 components. I've completed the first 3 components as shown in following paragraphs.

1. Scann object's 3D model

Scan objects 3D point cloud model by Baxter robot and depth camera.

See this Github:
https://github.com/felixchenfy/3D-Scanner-by-Baxter

A figure and a GIF of scanning a bottle is shown below.

2. Collect data for training Yolo

Use a depth camera to detect objects on table and get their mask in 2D image. The image and mask are used for training Yolo, which is a deep learning framework for object detection.

See this Github:
https://github.com/felixchenfy/Mask-Objects-from-RGBD

A figure of the image, point cloud, and the detected object and mask is shown below.

3. Train Yolo

The masked objects obtained in the last step were put onto different background images with random locations, scales, rotations, and shears to complete the data augmentation step. Then, these generated images are used for training Yolo.

See this Github:
https://github.com/felixchenfy/Data-Augment-and-Train-Yolo

Four generated images are shown below.

The final result of Yolo is shown below. Video are at here: bottle, meter.

Bottle Meter

4. Locate object 6D pose

(TODO)

Since I've get the object's 3D model and its 2D pos in RGB image, I can locate the object's 6D pose by fitting the 3D model onto the RGB-D image.

About

(1) 3D scan object by Baxter. (2) Label objects automatically by depth camera and (3) train Yolo. (4) [TODO; NOT DONE YET!!!] Finally, detect object and fit 3D model to know the 6D pose.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published