Application of RANSAC algorithm in slam feature matching

#include <iostream>  
#include "opencv2/opencv.hpp"  
#include "opencv2/core/core.hpp"  
#include "opencv2/features2d/features2d.hpp"  
#include "opencv2/highgui/highgui.hpp" 

using namespace cv;
using namespace std;
int main(int argc, char** argv)
    //Read picture
    Mat img_1 = imread("1.png");     
    Mat img_2 = imread("2.png");   
    if (img_1.empty() || img_2.empty())
        cout << "Can't open the picture!\n";
        return 0;
    vector<KeyPoint> img_1_keypoints, img_2_keypoints;
    Mat img_1_descriptors, img_2_descriptors;
    Ptr<ORB> detector=ORB::create();     //Feature points are extracted by ORB algorithm 
    //Detect FAST corner position
    detector->detect(img_1, img_1_keypoints);
    detector->detect(img_2, img_2_keypoints);
    //Calculate the BRIEF descriptor according to the corner position
    detector->compute(img_1, img_1_keypoints, img_1_descriptors);
    detector->compute(img_2, img_2_keypoints, img_2_descriptors);
    //Hamming distance as similarity measure
    BFMatcher matcher(NORM_HAMMING, true);   
    vector<DMatch> matches; //Note that the type here is DMatch
    matcher.match(img_1_descriptors, img_2_descriptors, matches);
    Mat match_img;
    drawMatches(img_1, img_1_keypoints, img_2, img_2_keypoints, matches, match_img);
    imshow("All matches", match_img);

    //Optimized by RANSAC algorithm
    //Save matching sequence number  
    vector<int> queryIdxs(matches.size()), trainIdxs(matches.size());
    for (size_t i = 0; i < matches.size(); i++)
        queryIdxs[i] = matches[i].queryIdx;
        trainIdxs[i] = matches[i].trainIdx;

    Mat homography_matrix;   //Transformation matrix  

    vector<Point2f> points1; KeyPoint::convert(img_1_keypoints, points1, queryIdxs);
    vector<Point2f> points2; KeyPoint::convert(img_2_keypoints, points2, trainIdxs);
    int ransacReprojThreshold = 5;  //Reject threshold  

    homography_matrix = findHomography(Mat(points1), Mat(points2), CV_RANSAC, ransacReprojThreshold);
    vector<char> matchesMask(matches.size(), 0);
    Mat points1t;
    perspectiveTransform(Mat(points1), points1t, homography_matrix);
    for (size_t i = 0; i < points1.size(); i++)  //Save 'local point'  
        if (norm(points2[i] -<Point2f>((int)i, 0)) <= ransacReprojThreshold) //Mark internal points  
            matchesMask[i] = 1;
    Mat match_img2;   //After filtering out "external points"  
    drawMatches(img_1, img_1_keypoints, img_2, img_2_keypoints, matches, match_img2, Scalar(0, 0, 255), Scalar::all(-1), matchesMask);
    imshow("Optimized matching", match_img2);

    return 0;

   RANdom SAmple Consensus algorithm (RANSAC algorithm for short) estimates the parameters of the mathematical model from a group of observed data containing outliers in an iterative manner. It is widely used in the field of computer vision and mathematics, such as line fitting, plane fitting, calculating the transformation matrix between images or point clouds, calculating the basic matrix and so on.

   basic idea of RANSAC algorithm: randomly select a group of local points from the data set (the number should ensure that all parameters of the model can be solved) and calculate a set of model parameters. Use the obtained model to test all other data points. If the error of a point is within the set error threshold, it is determined as an internal point, otherwise it is an external point. Only the model with the largest number of internal points so far is retained and recorded as the best model. After repeating steps 1 and 2 for a sufficient number of times (i.e. reaching the preset number of iterations), the local point corresponding to the best model is used to finally solve the model parameters. Finally, the model can be evaluated by estimating the error rate between the local point and the model.

Record some functions that appear in the above code.

1, BFMatcher matching for C++ OpenCV feature extraction

Brute Force matching is a common method of opencv two-dimensional feature point matching. BFMatcher always tries all possible matches, so that it can always find the best match. This is also the original meaning of Brute Force.

2, OpenCV  drawMatches() function:

void drawMatches(Mat img1, vector<KeyPoint> keypoints1,
                 Mat img2, vector<KeyPoint> keypoints2,
                 vector<DMatch> matches, 
                 Mat outImg) //want keypoints1[i] = keypoints2[matches[i]]

Note that the matching is the type vector < dmatch >. Here is  DMatch  Constructor:

 DMatch(int queryIdx,int trainIdx,float distance)

If you call the match function in order:

  match (descriptor_for_keypoints1,descriptor_for_keypoints2,matches)

Then queryIdx refers to keypoints1 and trainIdx refers to keypoints2

3, keypiont::convert

When detecting diagonal points in opencv, you need to convert between vector < keypoint > and vector < point2f >

The opencv version comes with related conversion functions

1. KeyPoint to point2f

 CV_WRAP static void convert(const std::vector<KeyPoint>& keypoints,
                                CV_OUT std::vector<Point2f>& points2f,
                                const std::vector<int>& keypointIndexes=std::vector<int>());

2. point2f to KeyPoint

CV_WRAP static void convert(const std::vector<Point2f>& points2f,
                                CV_OUT std::vector<KeyPoint>& keypoints,
                                float size=1, float response=1, int octave=0, int class_id=-1);

4, Findgeometry

Mat cv::findHomography ( InputArray srcPoints,
InputArray dstPoints,
int method = 0,   
double ransacReprojThreshold = 3,
OutputArray mask = noArray(),
const int maxIters = 2000,
const double confidence = 0.995 
//srcPoints the coordinate matrix of the midpoint of the source plane, which can be cv_ Type 32fc2, or vector < point2f >
//dstPoints the coordinate matrix of the midpoint of the target plane, which can be cv_ Type 32fc2, or vector < point2f >
/*method       The method used to calculate the homography matrix. Different methods correspond to different parameters, as follows:
0 - General method using all points
RANSAC - RANSAC-Robust algorithm based on RANSAC
LMEDS - Minimum median robust algorithm
RHO - PROSAC-Robust algorithm based on PROSAC*/
 The maximum allowable re projection error threshold for treating point pairs as interior points (for RANSAC and RHO methods only). If
 Then the point is considered to be an outer point (i.e. a wrong matching point pair). If srcPoints and dstPoints are in pixels, this parameter is usually set in the range of 1 to 10.*/
 Optional output mask matrix, usually set by robust algorithm (RANSAC or LMEDS). Note that the input mask matrix does not need to be set.
maxIters RANSAC The maximum number of iterations of the algorithm. The default value is 2000.
confidence Credibility value, ranging from 0 to 1*/

5, Perspective transformation

void perspectiveTransform(InputArray src, OutputArray dst, InputArray m)
// Coordinates, coordinates, transformation matrix

Note that the input of src and dst is not the image, but the coordinates corresponding to the image.

Tags: Python OpenCV Algorithm

Posted on Sat, 04 Sep 2021 01:23:35 -0400 by wcl99