Skip to content

Implemented calibration and computer-vision algorithms for an existing multi-camera system to capture the motion and shape sense of neurosurgical tendon-driven continuum robots under the Continuum Robotics Laboratory and University of Toronto Robotics Institute.

License

Notifications You must be signed in to change notification settings

YasPHP/Continuum-Robotics

Repository files navigation

Continuum Robotics Laboratory (CRL)

CRL logo

What are Continuum Robots?

Continuum robots, which are biologically inspired and organic compliant structures, differ fundamentally from traditional robots, which rely on a rigid joint-link composition. Their appearance is evocative of animals and organs such as trunks, tongues, worms, and snakes. Composed of flexible, elastic, or soft materials, continuum robots can perform complex bending motions and appear with curvilinear shapes. However, their softness and deformability raise major challenges, including the continuous recognition of the manipulator’s actual shape.

Research Purpose

At the Continuum Robotics Laboratory at University of Toronto, we create innovative continuum robot designs with novel features. To test our continuum robots and validate our models, algorithms, and controllers, we perform experiments on the bench-top and observe the robot's performance using different sensors. The aim of this research project is to implement calibration and computer-vision algorithms for an existing multi-camera system to capture the motion and sense the shape of tendon-driven continuum robots.

Research Poster

Undergraduate Research Symposium Presentation Video:

Project Outcomes

The following functionalities have to be implemented using Python or C++ using OpenCV and Qt:

  • Run, save, and load extrinsic and intrinsic camera calibration for a set of three cameras
  • Capture images and movies from up to three cameras simultaneously
  • Extraction a depth map from the images
  • Segmentation of tendon-driven continuum robot features (e.g. spacer disks)
  • Determining the pose of the continuum robot's tip and its shape
  • Building a GUI encompassing the above

Research Documentation

Get Started

Clone this repository and install the other dependencies with pip:

git clone https://github.com/YasPHP/CRL.git
cd CRL
pip install -U -r requirements.txt

Multi-Camera System (Stereovision Camera Pairs)

  • Note: to escape each python script's steps, either press 'x' on the popped up window each time and then the 'esc' key twice when the detected ArUco marker screen pops up at the end (as there is no 'x'/minimize/maximize screen option for those last two outputted images).
  • You know you will have reached the end of each program, when the terminal outputs the TDCR's final ArUco Marker Coordinates matrix!

Run Camera 1 & 2 Program

python cam1and2.py

Run Camera 1 & 3 Program

python cam1and3.py

Run Camera 2 & 3 Program

python cam2and3.py

Multi-n-Camera System Extension

Run Individual methods

python test_functions.py

Initialize n Cameras

python calibrate.py

ArUco Markers

  • Print out these ArUco Markers for future TDCR detection tests
  • Generate your own ArUco markers here

Credited Tutorial Sources

About

Implemented calibration and computer-vision algorithms for an existing multi-camera system to capture the motion and shape sense of neurosurgical tendon-driven continuum robots under the Continuum Robotics Laboratory and University of Toronto Robotics Institute.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published