Skip to content

Commit

Permalink
Merge pull request #896 from jnader/feat_demo_multiple_realsense
Browse files Browse the repository at this point in the history
New tutorial for image grabbing from multiple realsense devices
  • Loading branch information
fspindle committed Feb 24, 2021
2 parents b5f270a + 6101aed commit cc7f645
Show file tree
Hide file tree
Showing 5 changed files with 175 additions and 13 deletions.
30 changes: 30 additions & 0 deletions doc/tutorial/image/tutorial-grabber.dox
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,36 @@ To grab a sequence of images, you may rather use:
./tutorial-grabber-realsense --seqname I%04d.pgm --record 0
\endcode

\subsection tutorial-grabber-multiple-realsense-use How to acquire images from 2 Realsense devices

Once tutorial-grabber-multiple-realsense.cpp is built, the binary usage is given by:
\code
./tutorial-grabber-multiple-realsense --help
\endcode
To acquire images from 2 T265 devices with serial numbers 11622110511 and 11622110433 respectively, run:
\code
./tutorial-grabber-multiple-realsense --T265 11622110511 --T265 11622110433
\endcode
To acquire images from 1 T265 device (Serial Number:11622110511) and 1 D435 device (Serial Number: 752112070408), you may use:
\code
./tutorial-grabber-multiple-realsense --T265 11622110511 --D435 752112070408
\endcode

\note There is getRealSense2Info.cpp in `example/device/framegrabber` folder that could be used to get the device serial number.
\verbatim
$ ./getRealSense2Info
RealSense characteristics:
Intel RealSense T265 11622110409 0.2.0.951
Device info:
Name : Intel RealSense T265
Serial Number : 11622110409
Firmware Version : 0.2.0.951
Physical Port : 2-3-2
Product Id : 0B37
Usb Type Descriptor : 3.1
Product Line : T200
\endverbatim

\section grabber-camera-opencv Frame grabbing using OpenCV

It is also possible to grab images using OpenCV. You may find an example in tutorial-grabber-opencv.cpp.
Expand Down
25 changes: 14 additions & 11 deletions modules/sensor/include/visp3/sensor/vpRealSense2.h
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ int main()
If you want to change the default stream parameters, refer to the
librealsense2 `rs2::config` documentation. The following code allows to
capture the color stream in 1920x1080:
capture the color stream in 1920x1080 at 30 Hz:
\code
#include <visp3/gui/vpDisplayGDI.h>
Expand Down Expand Up @@ -213,7 +213,7 @@ int main() {
}
\endcode
This example shows how to get depth stream aligned on color stream:
This other example shows how to get depth stream aligned on color stream:
\code
#include <visp3/core/vpImageConvert.h>
#include <visp3/gui/vpDisplayGDI.h>
Expand Down Expand Up @@ -258,7 +258,8 @@ int main() {
References to `rs2::pipeline_profile` and `rs2::pipeline` can be retrieved
with (`rs.open() must be called before`):
\code
rs2::pipeline_profile& profile = rs.getPipelineProfile(); rs2::pipeline& pipeline = rs.getPipeline();
rs2::pipeline_profile& profile = rs.getPipelineProfile();
rs2::pipeline& pipeline = rs.getPipeline();
\endcode
Information about the sensor can be printed with:
Expand All @@ -268,17 +269,19 @@ rs2::pipeline_profile& profile = rs.getPipelineProfile(); rs2::pipeline& pipelin
int main() {
vpRealSense2 rs;
rs.open();
std::cout << "RealSense sensor characteristics: \n" << rs << std::endl;
std::cout << "RealSense sensor characteristics: \n" << rs << std::endl;
return 0;
}
\endcode
\note This class has been tested with the Intel RealSense SR300
(Firmware: 3.21.0.0) using librealsense (API version: 2.8.3). Refer to the
librealsense2 documentation or [API how
to](https://github.com/IntelRealSense/librealsense/wiki/API-How-To) for
additional information.
It is also possible to use several RealSense sensors at the same time. In that case, you need to create a
vpRealSense2 object for each device and use vpRealSense2::enable_device(const std::string &serial_number)
to select the device explicitly by its serial number. An example is provided
in tutorial-grabber-multiple-realsense.cpp.
\note Additional information can be found in the
[librealsense wiki](https://github.com/IntelRealSense/librealsense/wiki/).
*/
class VISP_EXPORT vpRealSense2
{
Expand Down Expand Up @@ -358,8 +361,8 @@ class VISP_EXPORT vpRealSense2

vpHomogeneousMatrix getTransformation(const rs2_stream &from, const rs2_stream &to, int from_index = -1) const;

void open(const rs2::config &cfg = rs2::config());
void open(const rs2::config &cfg, std::function<void(rs2::frame)> &callback);
bool open(const rs2::config &cfg = rs2::config());
bool open(const rs2::config &cfg, std::function<void(rs2::frame)> &callback);

friend VISP_EXPORT std::ostream &operator<<(std::ostream &os, const vpRealSense2 &rs);

Expand Down
6 changes: 4 additions & 2 deletions modules/sensor/src/rgb-depth/realsense/vpRealSense2.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1349,7 +1349,7 @@ void vpRealSense2::getIMUData(vpColVector *imu_acc, vpColVector *imu_vel, double
/*!
Open access to the RealSense device and start the streaming.
*/
void vpRealSense2::open(const rs2::config &cfg)
bool vpRealSense2::open(const rs2::config &cfg)
{
if (m_init) {
close();
Expand All @@ -1373,14 +1373,15 @@ void vpRealSense2::open(const rs2::config &cfg)
}

m_init = true;
return m_init;
}

/*!
Open access to the RealSense device and start the streaming.
\param cfg : A rs2::config with requested filters on the pipeline configuration. By default no filters are applied.
\param callback : Stream callback, can be any callable object accepting rs2::frame. The callback is invoked immediately once a frame is ready.
*/
void vpRealSense2::open(const rs2::config &cfg, std::function<void(rs2::frame)> &callback)
bool vpRealSense2::open(const rs2::config &cfg, std::function<void(rs2::frame)> &callback)
{
if (m_init) {
close();
Expand All @@ -1404,6 +1405,7 @@ void vpRealSense2::open(const rs2::config &cfg, std::function<void(rs2::frame)>
}

m_init = true;
return m_init;
}

/*!
Expand Down
1 change: 1 addition & 0 deletions tutorial/grabber/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ set(tutorial_cpp
tutorial-grabber-CMU1394.cpp
tutorial-grabber-flycapture.cpp
tutorial-grabber-ids-ueye.cpp
tutorial-grabber-multiple-realsense.cpp
tutorial-grabber-opencv.cpp
tutorial-grabber-opencv-threaded.cpp
tutorial-grabber-realsense.cpp
Expand Down
126 changes: 126 additions & 0 deletions tutorial/grabber/tutorial-grabber-multiple-realsense.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
/*! \example tutorial-grabber-multiple-realsense.cpp */
#include <visp3/core/vpImage.h>
#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/sensor/vpRealSense.h>
#include <visp3/sensor/vpRealSense2.h>

/*!
Grab images from multiple Intel realsense cameras
*/

int main(int argc, char **argv)
{
#if defined(VISP_HAVE_REALSENSE) || defined(VISP_HAVE_REALSENSE2) && (VISP_CXX_STANDARD >= VISP_CXX_STANDARD_11)
std::vector<std::pair<std::string, std::string> > type_serial_nb;
std::vector<bool> cam_found;

for (int i = 0; i < argc; i++) {
if (std::string(argv[i]) == "--T265") {
type_serial_nb.push_back(std::make_pair("T265", std::string(argv[i + 1])));
}
else if (std::string(argv[i]) == "--D435") {
type_serial_nb.push_back(std::make_pair("D435", std::string(argv[i + 1])));
}
else if (std::string(argv[i]) == "--SR300") {
type_serial_nb.push_back(std::make_pair("SR300", std::string(argv[i + 1])));
}
else if (std::string(argv[i]) == "--help" || std::string(argv[i]) == "-h") {
std::cout << "\nUsage: " << argv[0]
<< " [--T265 <serial number>] [--D435 <serial number>] [--SR300 <serial number>]\n"
<< "\nExample to use 2 T265 cameras:\n"
<< " " << argv[0] << " --T265 11622110511 --T265 11622110433 \n"
<< "\nExample to use 1 T265 and 1 D435 cameras:\n"
<< " " << argv[0] << " --T265 11622110511 --D435 752112077408 \n"
<< "\nExample to use 2 T265 and 1 D435 cameras:\n"
<< " " << argv[0] << " --T265 11622110511 --T265 11622110433 --D435 752112070408 \n"
<< std::endl;
return 0;
}
}

rs2::config T265_cfg, D435_cfg;
vpRealSense2 g[type_serial_nb.size()];
vpImage<unsigned char> I[type_serial_nb.size()];

#ifdef VISP_HAVE_X11
vpDisplayX d[type_serial_nb.size()];
#elif defined(VISP_HAVE_GDI)
vpDisplayGDI d[type_serial_nb.size()];
#elif defined(VISP_HAVE_OPENCV)
vpDisplayOpenCV d[type_serial_nb.size()];
#else
std::cout << "No image viewer is available..." << std::endl;
#endif

bool clicked = false;

for (size_t i = 0; i < type_serial_nb.size(); i++) {
std::cout << "Opening " << type_serial_nb[i].first << " with ID: " << type_serial_nb[i].second << "." << std::endl;
if (type_serial_nb[i].first == "T265") { // T265.
T265_cfg.enable_device(type_serial_nb[i].second);
T265_cfg.enable_stream(RS2_STREAM_FISHEYE, 1, RS2_FORMAT_Y8);
T265_cfg.enable_stream(RS2_STREAM_FISHEYE, 2, RS2_FORMAT_Y8);
cam_found.push_back(g[i].open(T265_cfg));
if(!cam_found.back()) {
std::cout << "Device with ID: " << type_serial_nb[i].second << " not found." << std::endl;
}
}
else { // D435 or SR300
D435_cfg.enable_device(type_serial_nb[i].second);
D435_cfg.disable_stream(RS2_STREAM_DEPTH);
D435_cfg.disable_stream(RS2_STREAM_INFRARED);
D435_cfg.enable_stream(RS2_STREAM_COLOR, 640, 480, RS2_FORMAT_RGBA8, 30);
cam_found.push_back(g[i].open(D435_cfg));
if(!cam_found.back()) {
std::cout << "Device with ID: " << type_serial_nb[i].second << " not found." << std::endl;
}
}
}

while (true) {
for (size_t i = 0; i < type_serial_nb.size(); i++) {
if (cam_found[i]) {
if (type_serial_nb[i].first == "T265") { // T265.
g[i].acquire(&I[i], NULL, NULL);
if (!d[i].isInitialised()) {
d[i].init(I[i], 100*i, 100*i, "T265 left image");
}
}

else { // D435.
g[i].acquire(I[i]);

if (!d[i].isInitialised()) {
d[i].init(I[i], 100*i, 100*i, type_serial_nb[i].first.c_str());
}
}

vpDisplay::display(I[i]);
vpDisplay::flush(I[i]);
clicked = vpDisplay::getClick(I[i], false);
}

if(clicked) {
break;
}
}

if(clicked) {
break;
}

}
#else
(void) argc;
(void) argv;
#if !(defined(VISP_HAVE_REALSENSE) || defined(VISP_HAVE_REALSENSE2))
std::cout << "Install librealsense version > 2.31.0, configure and build ViSP again to use this example" << std::endl;
#endif
#if (VISP_CXX_STANDARD < VISP_CXX_STANDARD_11)
std::cout << "This turorial should be built with c++11 support" << std::endl;
#endif
#endif
return EXIT_SUCCESS;
}

0 comments on commit cc7f645

Please sign in to comment.