Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make finding corner neighbor symmetrical in ChessBoardDetector::findQuadNeighbors #25427

Open
wants to merge 4 commits into
base: 4.x
Choose a base branch
from

Conversation

MaximSmolskiy
Copy link
Contributor

Pull Request Readiness Checklist

The basic idea of finding pair of corners neighbors is to find best candidate for first corner and check if first corner quite good candidate for its best candidate. And we test first corner for its best candidate less than best candidate for first corner.

Idea of changes is to make finding corner neighbor symmetrical - find best candidate for first corner, find best candidate for second corner and match them as pair iff they are both best candidates for each other.

Additional advantage - it simplifies code and removes some code duplication.

I tested this PR with benchmark

python3 objdetect_benchmark.py --configuration=generate_run --board_x=7 --path=res_chessboard --synthetic_object=chessboard

There are minor changes in results

cell_img_size = 100 (default)

before

                                 category  detected chessboard  total detected chessboard  total chessboard  average detected error chessboard
                          _none_none_blur             1.000000                        360               360                           0.630345
                    _none_none_gaussNoise             0.833333                        300               360                           0.623405
                          _none_none_none             1.000000                        360               360                           0.631517
                    _none_none_strongBlur             1.000000                        360               360                           0.630316
                   _none_undistorted_blur             1.000000                        360               360                           0.671232
             _none_undistorted_gaussNoise             1.000000                        360               360                           0.672619
                   _none_undistorted_none             1.000000                        360               360                           0.673669
             _none_undistorted_strongBlur             1.000000                        360               360                           0.671257
                   _perspective_none_blur             1.000000                       1080              1080                           0.588694
             _perspective_none_gaussNoise             0.805556                        870              1080                           0.599312
                   _perspective_none_none             1.000000                       1080              1080                           0.591063
             _perspective_none_strongBlur             1.000000                       1080              1080                           0.588604
            _perspective_undistorted_blur             1.000000                       1080              1080                           0.622081
      _perspective_undistorted_gaussNoise             1.000000                       1080              1080                           0.625704
            _perspective_undistorted_none             1.000000                       1080              1080                           0.624191
      _perspective_undistorted_strongBlur             1.000000                       1080              1080                           0.621618
             _strongPerspective_none_blur             1.000000                        360               360                           0.482934
       _strongPerspective_none_gaussNoise             0.166667                         60               360                           0.391551
             _strongPerspective_none_none             1.000000                        360               360                           0.480290
       _strongPerspective_none_strongBlur             0.333333                        120               360                           0.469080
      _strongPerspective_undistorted_blur             1.000000                        360               360                           0.503458
_strongPerspective_undistorted_gaussNoise             0.250000                         90               360                           0.448713
      _strongPerspective_undistorted_none             1.000000                        360               360                           0.504412
_strongPerspective_undistorted_strongBlur             0.166667                         60               360                           0.473791
                                      all             0.904167                      13020             14400                           0.600512
Total detected time:  139.65614900000008 sec

after

                                 category  detected chessboard  total detected chessboard  total chessboard  average detected error chessboard
                          _none_none_blur             1.000000                        360               360                           0.630345
                    _none_none_gaussNoise             0.750000                        270               360                           0.636279
                          _none_none_none             1.000000                        360               360                           0.631517
                    _none_none_strongBlur             1.000000                        360               360                           0.630316
                   _none_undistorted_blur             1.000000                        360               360                           0.671232
             _none_undistorted_gaussNoise             1.000000                        360               360                           0.672619
                   _none_undistorted_none             1.000000                        360               360                           0.673669
             _none_undistorted_strongBlur             1.000000                        360               360                           0.671257
                   _perspective_none_blur             1.000000                       1080              1080                           0.588694
             _perspective_none_gaussNoise             0.888889                        960              1080                           0.594106
                   _perspective_none_none             1.000000                       1080              1080                           0.591064
             _perspective_none_strongBlur             1.000000                       1080              1080                           0.588604
            _perspective_undistorted_blur             1.000000                       1080              1080                           0.622081
      _perspective_undistorted_gaussNoise             1.000000                       1080              1080                           0.625703
            _perspective_undistorted_none             1.000000                       1080              1080                           0.624191
      _perspective_undistorted_strongBlur             1.000000                       1080              1080                           0.621618
             _strongPerspective_none_blur             1.000000                        360               360                           0.482934
       _strongPerspective_none_gaussNoise             0.166667                         60               360                           0.391551
             _strongPerspective_none_none             1.000000                        360               360                           0.480290
       _strongPerspective_none_strongBlur             0.333333                        120               360                           0.469080
      _strongPerspective_undistorted_blur             1.000000                        360               360                           0.503458
_strongPerspective_undistorted_gaussNoise             0.333333                        120               360                           0.422259
      _strongPerspective_undistorted_none             1.000000                        360               360                           0.504412
_strongPerspective_undistorted_strongBlur             0.166667                         60               360                           0.473791
                                      all             0.910417                      13110             14400                           0.599746
Total detected time:  142.40333700000005 sec

----------------------------------------------------------------------------------------------------------------------------------------------

cell_img_size = 10

before

                                 category  detected chessboard  total detected chessboard  total chessboard  average detected error chessboard
                          _none_none_blur             0.991667                        357               360                           4.905091
                    _none_none_gaussNoise             0.750000                        270               360                           5.215633
                          _none_none_none             1.000000                        360               360                           4.943304
                    _none_none_strongBlur             0.916667                        330               360                           3.806217
                   _none_undistorted_blur             0.994444                        358               360                           5.220915
             _none_undistorted_gaussNoise             0.997222                        359               360                           4.542443
                   _none_undistorted_none             0.997222                        359               360                           4.340208
             _none_undistorted_strongBlur             0.161111                         58               360                           5.024331
                   _perspective_none_blur             0.629630                        680              1080                           4.825401
             _perspective_none_gaussNoise             0.966667                       1044              1080                           3.895425
                   _perspective_none_none             0.971296                       1049              1080                           3.920378
             _perspective_none_strongBlur             0.000000                          0              1080                                NaN
            _perspective_undistorted_blur             0.583333                        630              1080                           4.594335
      _perspective_undistorted_gaussNoise             0.999074                       1079              1080                           3.553195
            _perspective_undistorted_none             0.750000                        810              1080                           3.604110
      _perspective_undistorted_strongBlur             0.000000                          0              1080                                NaN
             _strongPerspective_none_blur             0.000000                          0               360                                NaN
       _strongPerspective_none_gaussNoise             0.000000                          0               360                                NaN
             _strongPerspective_none_none             0.083333                         30               360                           2.382460
       _strongPerspective_none_strongBlur             0.000000                          0               360                                NaN
      _strongPerspective_undistorted_blur             0.000000                          0               360                                NaN
_strongPerspective_undistorted_gaussNoise             0.000000                          0               360                                NaN
      _strongPerspective_undistorted_none             0.000000                          0               360                                NaN
_strongPerspective_undistorted_strongBlur             0.000000                          0               360                                NaN
                                      all             0.539792                       7773             14400                           4.209964
Total detected time:  2.6968930000000015 sec

after

                                 category  detected chessboard  total detected chessboard  total chessboard  average detected error chessboard
                          _none_none_blur             0.991667                        357               360                           4.905091
                    _none_none_gaussNoise             0.750000                        270               360                           5.215633
                          _none_none_none             1.000000                        360               360                           4.943304
                    _none_none_strongBlur             0.916667                        330               360                           3.806217
                   _none_undistorted_blur             0.994444                        358               360                           5.220915
             _none_undistorted_gaussNoise             0.997222                        359               360                           4.542443
                   _none_undistorted_none             0.997222                        359               360                           4.340208
             _none_undistorted_strongBlur             0.161111                         58               360                           5.024331
                   _perspective_none_blur             0.629630                        680              1080                           4.825401
             _perspective_none_gaussNoise             0.966667                       1044              1080                           3.895425
                   _perspective_none_none             0.999074                       1079              1080                           3.865684
             _perspective_none_strongBlur             0.000000                          0              1080                                NaN
            _perspective_undistorted_blur             0.583333                        630              1080                           4.594335
      _perspective_undistorted_gaussNoise             0.999074                       1079              1080                           3.553195
            _perspective_undistorted_none             0.750000                        810              1080                           3.604110
      _perspective_undistorted_strongBlur             0.000000                          0              1080                                NaN
             _strongPerspective_none_blur             0.000000                          0               360                                NaN
       _strongPerspective_none_gaussNoise             0.000000                          0               360                                NaN
             _strongPerspective_none_none             0.000000                          0               360                                NaN
       _strongPerspective_none_strongBlur             0.000000                          0               360                                NaN
      _strongPerspective_undistorted_blur             0.000000                          0               360                                NaN
_strongPerspective_undistorted_gaussNoise             0.000000                          0               360                                NaN
      _strongPerspective_undistorted_none             0.000000                          0               360                                NaN
_strongPerspective_undistorted_strongBlur             0.000000                          0               360                                NaN
                                      all             0.539792                       7773             14400                           4.208308
Total detected time:  2.7706419999999983 sec

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

int& closest_corner_idx,
cv::Point2f& closest_corner_pt)
{
if (cur_quad.neighbors[i])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition should be moved outside the function. i is not used inside the function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -471,6 +471,110 @@ static void icvBinarizationHistogramBased(Mat & img)
}
}

static bool findCornerNeighbor(
cv::AutoBuffer<ChessBoardQuad>& all_quads,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose it should be just ChessBoardQuad* all_quads. AutoBuffer details are not needed in the function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

const std::vector<Point2f>& all_quads_pts,
flann::GenericIndex<flann::L2_Simple<float>>& all_quads_pts_index,
const int idx,
const ChessBoardQuad& cur_quad,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cur_quad is all_quads[idx], I do not think that it's needed in the function parameters.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

const float radius,
std::vector<int>& neighbors_indices,
std::vector<float>& neighbors_dists,
const float thresh_scale,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to set default value for thresh_scale or even make it local constant. It's constant equal to 1.0 on caller side.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is also used outside function

float radius = cur_quad.edge_len * thresh_scale + 1;

So, do you propose to set it twice with same value?

std::vector<float>& neighbors_dists,
const float thresh_scale,
int& closest_quad_idx,
ChessBoardQuad*& closest_quad,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

closest_quad= all_quads[closest_quad_idx]. No need to pass it with parameters.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines 484 to 485
std::vector<int>& neighbors_indices,
std::vector<float>& neighbors_dists,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The values are not used outside the function. I propose to make it local.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Their lengths are equal to all_corners_count which can be quite large. So, it's better to avoid such frequent allocations on every function call

@asmorkalov asmorkalov self-assigned this Apr 24, 2024
@asmorkalov asmorkalov removed this from the 4.10.0 milestone May 2, 2024
@MaximSmolskiy
Copy link
Contributor Author

@asmorkalov @AleksandrPanov gentle ping

@opencv-alalek opencv-alalek added this to the 4.10.0 milestone May 13, 2024
@MaximSmolskiy
Copy link
Contributor Author

@asmorkalov @AleksandrPanov gentle ping

@MaximSmolskiy
Copy link
Contributor Author

@asmorkalov @AleksandrPanov ping

@AleksandrPanov AleksandrPanov modified the milestones: 4.10.0, 4.11.0 May 31, 2024
@MaximSmolskiy
Copy link
Contributor Author

@asmorkalov @AleksandrPanov ping

@AleksandrPanov
Copy link
Contributor

@MaximSmolskiy, it looks like there are too many parameters in findCornerNeighbor():

static bool findCornerNeighbor(
    ChessBoardQuad* all_quads,
    const std::vector<Point2f>& all_quads_pts,
    flann::GenericIndex<flann::L2_Simple<float>>& all_quads_pts_index,
    const int idx,
    const cv::Point2f& pt,
    float& min_dist,
    const float radius,
    std::vector<int>& neighbors_indices,
    std::vector<float>& neighbors_dists,
    int& closest_quad_idx,
    int& closest_corner_idx,
    cv::Point2f& closest_corner_pt)

I added the NeighborsFinder struct and simplified the API with it:

bool ChessBoardDetector::NeighborsFinder::findCornerNeighbor(
    const int idx,
    const cv::Point2f& pt,
    float& min_dist,
    const float radius,
    int& closest_quad_idx,
    int& closest_corner_idx,
    cv::Point2f& closest_corner_pt)

Do you have any comments or suggestions?

Otherwise, I like it.

@AleksandrPanov AleksandrPanov force-pushed the make-finding-corner-neighbor-symmetrical-in-ChessBoardDetector-findQuadNeighbors branch from 31217ac to e705bae Compare June 4, 2024 07:47
@AleksandrPanov AleksandrPanov force-pushed the make-finding-corner-neighbor-symmetrical-in-ChessBoardDetector-findQuadNeighbors branch from e705bae to e064683 Compare June 4, 2024 08:48
@MaximSmolskiy
Copy link
Contributor Author

@AleksandrPanov Thanks for refactoring! I don't have any suggestions

modules/calib3d/src/calibinit.cpp Outdated Show resolved Hide resolved
modules/calib3d/src/calibinit.cpp Outdated Show resolved Hide resolved
modules/calib3d/src/calibinit.cpp Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants