Opencv note 3 image smoothing

1, Convolution definition and matrix form

The commonly used smoothing algorithms are Gaussian filter and mean filter based on two-dimensional discrete convolution. And median filter based on statistical method.

Suppose there are two matrices

  Rotate K 180 degrees (I didn't figure out why)

  Then, according to each element of I, from left to right, from top to bottom,

A new matrix M is obtained  

Then we can say that matrix K is a convolution kernel (convolution mask), and M is the full convolution of I with respect to K. This understanding is very simple. However, we generally use the same convolution in the process of image pixel processing, that is, the height and width of the convoluted image are the same as those of the original image, as follows:  

For the same and full convolution, the values at the boundary of Matrix I lack complete neighborhood values, so we need to do special processing in these areas during convolution operation. The method is to expand the boundary, as follows

The effect is as follows. It is obvious that the expanded image is larger than the previous one


  The code is shown as follows:

	int  top = (int)src.rows * 0.05;  // The amount of expansion at the top of the edge extension
	int  bottom = (int)src.cols * 0.05;   // Expansion of the lower part of the edge extension
	int  left = (int)src.rows * 0.05;
	int  right = (int)src.cols * 0.05;
	RNG rad(12345); // random number
	int  borderType = BORDER_DEFAULT;
	int  c = 0;
	while (true)

		c = waitKey(500);
		if ((char)c == 27)   // esc 
		if ((char)c == 'r')
			borderType = BORDER_REPLICATE;
		else if ((char)c == 'w')
			borderType = BORDER_WRAP;
		else if ((char)c == 'c')
			borderType = BORDER_CONSTANT;
		else if ((char)c == 'a')
			borderType = BORDER_REFLECT;
		}else if ((char)c == 'b')
			borderType = BORDER_REFLECT_101;
			borderType = BORDER_DEFAULT;
		Scalar  color = Scalar(rad.uniform(0, 255), rad.uniform(0, 255), rad.uniform(0, 255));
		copyMakeBorder(src, dest, top, bottom, left, right, borderType, color);
		imshow("Edge expansion", dest);

Add some convolution operation functions:

Mat kernel = (Mat_<int>(3, 3)<< 0,-1,0,-1,5,-1,0,-1,0); // Convolution kernel
void filter2D( 
 InputArray src,
 OutputArray dst,
 int ddepth, //Bit depth of the image. If it is - 1, it indicates that it is consistent with the input image
 InputArray kernel,
 Point anchor=Point(-1,-1), //The reference point (anchor) of the kernel. Its default value is (- 1, - 1), indicating that it is located in the center of the kernel
 double delta=0,
 int borderType=BORDER_DEFAULT //Pixel outward approximation method. The default value is BORDER_DEFAULT is to calculate all boundaries. This is the above-mentioned edge pixel expansion methods

2, Gaussian blur

  Gaussian filter is a low-pass filter, which can filter out the high-frequency components in the image and retain the low-frequency components. Therefore, the image will become blurred after Gaussian blur, which can suppress Gaussian noise (positive distribution).

Gauss blur principle: to blur a picture, generally, we take each pixel as the center and take a 3 * 3 area to calculate the average value as the pixel value of the pixel. Is it really reasonable if each image is so reasonable? The image is continuous. If the area you choose is large enough, the pixels closer to the center are closer to the pixel value of the changed point, The farther the pixels are, the greater the difference is. If the mean value is calculated, some features of the image may be changed. So we thought of weighting different positions of the mask area. The closer the weight is, the greater the antisense is. Therefore, the weight distribution mode of Zhengtai distribution is introduced.  

Gaussian function
Gaussian function is defined as follows

Where a, b and c are the corresponding parameters. The Gaussian function is a bell curve. Parameter a controls the amplitude of the function, parameter b controls the horizontal position of the bell curve, and parameter c reflects the width of the bell curve.


 Generation of Gaussian template

Assuming that the coordinates of the center point are (0,0), the coordinates of the 8 closest points to it produce a weight matrix as follows

In order to calculate the weight matrix, you need to set σ Value of. assume σ= 1.5, the weight matrix with fuzzy radius of 1 is as follows:  

  We can calculate the above data through matlab class. The sum of the probability of Zhengtai distribution is 1, but we roughly calculate that the sum of the probability of 9 points is less than 0.5. In this case, we need to normalize, because if the sum of this probability is less than 1 or greater than 1, it is equivalent to changing the gray value of the pixel. If it is less than 1, it will become dark and if it is greater than 1, it will become bright. Sum of each value / 9 values (weight normalization).

  Next, we'll discuss it σ The significance and selection of value, σ It represents the standard deviation and represents the degree of dispersion of data.

 σ The larger the, the wider the graph, the smaller the peak, and the smoother the graph transformation. Therefore, the values of each element of the generated template are not different, which is similar to the average template;  

σ The smaller the, the narrower the graph, the larger the peak, and the more violent the graph transformation, so the values of each element of the generated template vary greatly

Gaussian filter calculation:


With Gaussian template, it is weighted calculation: suppose a region covered by a 3 * 3 mask

Then add up the calculated values to replace the original center value. The approximate value is 24.98 ~ ~ 25.

Optimization: separation characteristics of Gaussian function:

Direct two-dimensional Gaussian blur is inefficient. In fact, Gaussian blur can also calculate two independent one-dimensional spaces on two-dimensional images, which is called linear separability:

  As can be seen from the above, we can first calculate the Gaussian blur in one dimension, that is, the one-dimensional blur G(x), and then perform the Gaussian blur G(y) - "G(x)*G(y) in the other direction for G(x). Since then, all the relevant theories of Gaussian blur have been completed, and the next is the code implementation:

Mean filtering: the average of the gray values of all points in a region is used as the gray value of this point
void blur(InputArray src, OutputArray dst, Size ksize, Point anchor=Point(-1,-1), int borderType=BORDER_DEFAULT )
src – Input picture
dst – Output picture
ksize – Fuzzy kernel size
anchor – Anchor, default is(-1,-1),That is, the anchor point is in the center of the kernel.
borderType – The mode used to determine the image boundary.

GaussianBlur(InputArray src, OutputArray dst, Size ksize, double sigmaX, double sigmaY=0, int borderType=BORDER_DEFAULT)
src – Input pictures, only CV_8U, CV_16U, CV_16S, CV_32F or CV_64F.
dst – Output picture
ksize – Gaussian kernel size. ksize.width and ksize.height Different numbers are allowed, but they must be positive odd numbers. Or equal to 0, determined by the parameter sigma Take the opportunity to decide.
sigmaX – Gaussian kernel in X Standard deviation of direction,
sigmaY – Gaussian kernel in Y The standard deviation of the direction can be understood as sigmaX Compensation for
borderType – The mode used to determine the image boundary.
The most useful filter (Although not the fastest).  Gaussian filtering is to convolute each pixel of the input array with the Gaussian kernel, and take the convolution sum as the output pixel value.

Median smoothing: median filtering uses each pixel of the image as a neighborhood (A square area centered on the current pixel)Replace with the median of the pixel 
void medianBlur(InputArray src, OutputArray dst, int ksize)
src – It supports 1, 3 and 4 channels of image input( CV_8U,CV_16U, CV_32F)
dst – Output picture
ksize – Linear diameter size can only be an odd number greater than 1

void bilateralFilter(InputArray src, OutputArray dst, int d, double sigmaColor, double sigmaSpace, intborderType=BORDER_DEFAULT )
src – The source must be an 8-bit or floating-point number, 1 or 3-channel picture.
dst – Output picture
d – The neighborhood diameter of each pixel used in the filtering process. If this is a non integer, this value is determined by sigmaSpace decision.
sigmaColor – The standard variance of color space. The larger the value, the farther the color will be mixed into the neighborhood, so that the larger color segment will obtain the same color.
sigmaSpace – The annotation variance of coordinate space. The larger the value, the farther the pixels will affect each other, so that the larger area can obtain the same color with similar colors. When d>0,d The neighborhood size is specified and is consistent with sigmaSpace Nothing. Otherwise, d Proportional to sigmaSpace. 


#If 1 / / implement Gaussian blur

void  GaussianFilter(const Mat& src, Mat& dst, int ksize, double segma);
void  GaussianFilter(const Mat& src, Mat& dst, int ksize, double segma) 
	if (ksize<1|| ksize>21) {
		ksize = 3;//When ksize is very large and exceeds a certain value, the peak of the Gaussian function will almost disappear, which is almost a mean value, so it is meaningless
	CV_Assert(src.channels() || src.channels() == 3); // Only single channel or three channel images are processed
	//1. Define a two-dimensional array --- Gaussian template
	double ** GaussianTemplate = new double *[ksize];
	int  radius = ksize / 2;
	// Calculate the center position of this template 
	int x=0, y=0;
	for (size_t i = 0; i < ksize; i++)
		GaussianTemplate[i] = new double [ksize];

	// Since then, a ksize *ksize matrix has been formed. The following is to calculate the weight of each position
	double sum = 0;//Used to sum the weights of ksize *ksize

	for (int k = 0; k < ksize; k++)
		x = pow(k - radius,2);
		for (int j = 0; j < ksize; j++)
			y = pow(j - radius, 2);
			// Calculate Gaussian function
			double g = exp(-(x + y) / (2 * segma * segma));
			sum += g;
			GaussianTemplate[k][j] = g;

	//2. Normalization
	for (int i = 0; i < ksize; i++)

		for (int j = 0; j < ksize; j++)
			GaussianTemplate[i][j] /= sum;
			cout << GaussianTemplate[i][j] << " ";
		cout << endl;

	// Gaussian template has been completed and can be used safely

	// 3. Start mask
	 // Edge supplement first
	copyMakeBorder(src, dst, radius, radius, radius, radius, BorderTypes::BORDER_REFLECT);
	// Because src expands the boundary, the width+radius of dst will also become larger
	int rows = dst.rows - radius; 
	int cols = dst.cols - radius;
	// Find the same convolution
	for (int i = radius; i < rows; i++)
		for (int j = radius; j < cols; j++)
			// Only single channel and three channel are calculated here
			double sum_value[3] = {0};
			// [i][j] represents the coordinates of the most central point of the template, which is also the xcenter and ycenter mentioned above
			for (int u = -radius; u <= radius; u++)
				for (int v = -radius; v <= radius ; v++)
					if (src.channels()==1) 
						double  gray_value = GaussianTemplate[radius+u][radius+v] *<uchar>(i+u,j+v);
						sum_value[0] += gray_value;
					else if (src.channels()==3)
						Vec3b bgr=<Vec3b>(i + u, j + v);
						double  k = GaussianTemplate[radius + u][radius + v] ;
						sum_value[0] += k * bgr[0];
						sum_value[1] += k * bgr[1];
						sum_value[2] += k * bgr[2];

			if (src.channels() == 1) // single channel
			{<uchar>(i, j) = static_cast<uchar>(sum_value[0]);
			else  if (src.channels() == 3) 
				Vec3b bgr = { static_cast<uchar>(sum_value[0]), static_cast<uchar>(sum_value[1]), static_cast<uchar>(sum_value[2]) };<Vec3b>(i, j) = bgr;
	// Release template array
	for (int i = 0; i < ksize; i++)
		delete[] GaussianTemplate[i];
	delete[] GaussianTemplate;

	printf("The calculation has been completed");
Mat src, dst, bimage, desg, Gaussian_shuangbian, mind, My_Gaussian;
const char* win_gaussian_myself = "Custom Gaussian blur";
int  ksize = 3;
int  max_ksize = 20;
int  segma = 1;
int   segma_max = 5;
void callbackMyGaussianfilter(int,void*);
void callbackMyGaussianfilter(int, void*) 
	int size = 2 * ksize + 1;
	GaussianFilter(src, My_Gaussian, size, segma);
	imshow(win_gaussian_myself, My_Gaussian);
int main(int args, char* arg)
	// point  

	src = imread("C:\\Users\\19473\\Desktop\\opencv_images\\88.jpg");
	if (!
		printf("could not  load  image....\n");
	imshow("Original drawing", src);

	printf("src.channels() %d\n", src.channels());

	// Gaussian  fiter 
	GaussianBlur(src, desg, Size(3, 3), 0, 9);
	imshow("OPencv GaussianBlur ", desg);

	printf("Gaussian blur realized by ourselves-------------------------\n");
	namedWindow(win_gaussian_myself, CV_WINDOW_AUTOSIZE);
	My_Gaussian = Mat::zeros(src.size(), src.type());

	//imshow("custom Gaussian Blur", My_Gaussian);
	createTrackbar("size", win_gaussian_myself, &ksize, max_ksize,callbackMyGaussianfilter);
	createTrackbar("segma", win_gaussian_myself, &segma, segma_max, callbackMyGaussianfilter);

	return -1;

#If 0 / / image fuzzy mean filtering Gaussian filtering
int main(int args, char* arg)
	// point  
	Mat src, bimage,desg, Gaussian_shuangbian,mind;
	bimage = imread("C:\\Users\\19473\\Desktop\\opencv_images\\88.jpg");
	if (!
		printf("could not  load  image....\n");
	imshow("Original drawing", bimage);

	// Mean filtering
	blur( bimage, src,Size(9,9),Point(-1,-1));
	imshow("Mean filtering", src);

	// Median filtering algorithm --- remove salt and pepper noise
	medianBlur(bimage, mind,3);
	imshow("median filtering ", mind);

	// Edge preserving algorithm bilateral edge preserving algorithm -- skin grinding algorithm
	bilateralFilter(bimage, Gaussian_shuangbian,3,80,3);
	Mat   result;
	Mat kernel = (Mat_<int>(3, 3)<< 0,-1,0,-1,5,-1,0,-1,0);
	filter2D(Gaussian_shuangbian, result,-1, kernel,Point(-1,-1),0);
	imshow(" Bilateral reserved edge", result);
	return -1;

  Summary: as long as you understand Gaussian filtering, everything else is easy to understand

Next step: principle and code implementation of edge operator

Tags: OpenCV AI Computer Vision

Posted on Fri, 03 Dec 2021 16:17:55 -0500 by cinos11