1, A rough understanding of expansion and corrosion
Because expansion and corrosion belong to the category of morphological filtering, there must be a structural element first. The structure element is the filter matrix, which has many types, including x-shape, rectangle, cross shape, diamond, etc. cv2 has the default getstructurelelement() method to obtain rectangular and cross shaped structure elements. Of course, it can also be customized through numpy.array() in numpy library. The following is a code example:
A. cv2 internal method obtains rectangular filter and cross filter respectively by default:
The first parameter is the type, and the second parameter is the kernel size ksize, that is, the filter size, preferably odd (3, 5, 7)
kernel_rect=cv2.getStructuringElement(cv2.MORPH_RECT,(5,5))，
kernel_cross=cv2.getStructuringElement(cv2.MORPH_CROSS,(5,5))
B. numpy.array() custom
# x-shaped filter structure element
kernel_x=np.array([[1,0,0,0,1],[0,1,0,1,0],[0,0,1,0,0],[0,1,0,1,0],[1,0,0,0,1]],dtype=np.uint8)
# Rhombic filter structure element
kernel_diamond = np.array([[0,0,1,0,0],[0,1,1,1,0],[1,1,1,1,1],[0,1,1,1,0],[0,0,1,0,0]],dtype=np.uint8)
After the structural elements are obtained, the expansion or corrosion operation is carried out. The so-called expansion is to traverse the structural elements obtained above and the gray image of the original image one by one to obtain the maximum value in the region. Therefore, the effect image after expansion is that there are more bright areas and less dark areas in the original image, which is conducive to the extraction of background. If you expand continuously for many times, you will eventually get an all white picture with the size of the original image of 255 pixels. Corrosion is the non operation of expansion, which obtains the minimum value in the area. In the original figure, there are more dark areas and less bright areas, which is conducive to the extraction of foreground. If you expand continuously for many times, you will eventually get a black picture with the size of the original image, which is all 0 pixels. Here is a binary image to demonstrate:
Original drawing:
After expansion:
After continuous expansion:
Corrosion:
Continuous corrosion:
Encapsulate the function of corrosion and expansion with functions:
Note: the way variable represents the structure type. In order to combine the two methods of obtaining structure elements, the flag variable is set. 1 represents the CV2 default method of obtaining structure elements (rectangle and cross shape can be obtained respectively), and 0 represents the custom kernel type and size
def dilate(image,way,flag): #Get structure element 1. Get getStructurinElement directly if(flag): kernel=cv2.getStructuringElement(way,(5,5)) #expand result=cv2.dilate(image,kernel) #expand else: result=cv2.dilate(image,way) return result def erode(image,way,flag): if(flag): kernel=cv2.getStructuringElement(way,(5,5)) #corrosion result=cv2.erode(image,kernel) #corrosion else: result=cv2.erode(image,way) return result src=cv2.imread('building.jpg') src=cv2.cvtColor(src,cv2.COLOR_BGR2GRAY) #The kernel size of custom morphological filter uses np.array #x-shaped filter structure element kernel_x=np.array([[1,0,0,0,1],[0,1,0,1,0],[0,0,1,0,0],[0,1,0,1,0],[1,0,0,0,1]],dtype=np.uint8) #Rhombic filter structure element kernel_diamond = np.array([[0,0,1,0,0],[0,1,1,1,0],[1,1,1,1,1],[0,1,1,1,0],[0,0,1,0,0]],dtype=np.uint8) #Rhombic corrosion after cross expansion src1=dilate(src,cv2.MORPH_CROSS,1) src1=erode(src1,kernel_diamond,0) cv2.imshow('result',src1) cv2.waitKey(-1)2, Transform expansion and corrosion, find corners and draw
The general idea is that the result src1 is obtained by rhombic corrosion after the cross expansion result, and the result src2 is obtained by rectangular corrosion after the x-shaped expansion result on the original drawing. The difference between the two is the calculated corner point.
Why? Rough understanding: the cross expansion corner only expands in the horizontal and vertical directions, but the edge remains unchanged. After the rhombic corrosion, the corner on the angle of 135 degrees or 45 degrees shrinks, but the edge remains unchanged. x-shaped expansion, the corner and edge expand in the x-shaped direction, and shrink in all directions when it comes to rectangular corrosion, so the result is that the corner is restored and the edge corrosion is more severe The difference between the two is the corner. (because the edge corrodes badly, after subtraction, it is equivalent to expanding a little, so that the final edge does not change.)
Encapsulate the function of finding corners with morphology and drawing the found corners with functions (code implementation):
Note: step1. The function to realize expansion and corrosion has been defined above
Step 2. Decrse() returns the difference between the two, that is, the result variable result of the corner is found, but sometimes not all the corner points found meet the expectations, so it needs to be thresholded. By slicing, all pixel values greater than 35 in the result are assigned to 255, and all values less than or equal to 35 are assigned to 0.
step3. Draw the found corner points on the original drawing and use cv2.cirle() to draw the garden. You can set the display. In this paper, only the pixel points greater than 120 in the found result will be displayed. In particular, when drawing the garden, (x,y) should be reversed, and the coordinates in the result are stored as (i,j), but when drawing the garden, it should be changed to (j,i)
The open and close operations will be explained below. Don't worry
def decrse(src1,src2): return src1.astype(np.int32)-src2.astype(np.int32) def Print_corr(src,diff): kernel=np.ones((5,5),dtype=np.uint8) src=cv2.morphologyEx(src,cv2.MORPH_CLOSE,kernel)#Close operation, expand first and then corrode, and fill the darker part after expansion #Finally, the dark part of the image is darker and smoother cv2.imshow('CLOSE',src) cv2.waitKey(100) for i in range(diff.shape[0]): for j in range(diff.shape[1]): if diff[i,j]>100: img=cv2.circle(src,(j,i),5,255) return img #Rhombic corrosion after cross expansion src1=dilate(src,cv2.MORPH_CROSS,1) src1=erode(src1,kernel_diamond,0) #Rectangular corrosion after x-shaped expansion src2=dilate(img_ori,kernel_x,0) src2=erode(src2,cv2.MORPH_RECT,1) result=decrse(src1,src2) result[result>=35]=255 result[result<35]=0 result_draw=Print_corr(src,result) cv2.imwrite('dilate_erode.jpg',result_draw) src=result_draw cv2.imshow('result',src) cv2.waitKey(-1)The renderings are as follows:
3, Open operation, close operation, top cap, bottom cap
3.1 open operation
The opening operation is to corrode first and then expand. The effect obtained after corroding first is that the dark part in the original drawing will highlight darker, but the dark part will also have less dark parts (relatively speaking) , and then expand. For this part that is not very black, its expansion will not be so exaggerated, and the expanded surrounding images are relatively smooth, which can well remove the noise. You can feel the effect first:
Note: the left is the gray scale image of the original image, and the right is the result of the open operation. Take the fence as an example, it is already black, and it is darker after corrosion. Then the red circle in the figure is not very black, or even white, but it does not deform very much during expansion, but this part of the area is smoother. Therefore, from the side, the open operation has a good denoising effect.
3.2 closed operation
The closed operation is to expand first and then corrode. The effect obtained after expansion is that the brighter parts in the original drawing will be highlighted, but the brighter parts will also have less bright parts (relatively speaking) , and then corrode. For this part that is not very bright, the corrosion will shrink this part and fill the bright cracks in this part, so as to better extract the foreground. (similar to the P picture of Meitu XiuXiu, the portrait is the foreground.
Remember that closed operation is similar to removing acne, filling holes, and finally realizing corrosion. You won't remember who corrodes first, who expands first, and then how to judge whether it is open or closed operation.
The renderings are as follows:
Note: on the left is the gray-scale image of the original image, and on the right is the result of closed operation. Take circle 1 in the figure as an example, it is white, and it becomes whiter after expansion. Then, the red circle 2 in the figure is not very white, or even dark, but it does not shrink very much during corrosion, but this partition is filled. Therefore, from the side, closed operation has a good effect of filling small gaps.
three point three Top cap
Top hat = original image - open operation, which is simply to extract brighter areas than those in the original image.
Bottom cap = original image - close operation, which is simply to extract darker areas than the original image.
Code implementation: 1. Get structure elements
2. Directly call tophat and blackhat of cv2
import cv2 import numpy as np kernel=np.ones((3,3),dtype=np.uint8) img=cv2.imread('building.jpg') img=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) black_hat=cv2.morphologyEx(img,cv2.MORPH_BLACKHAT,kernel) top_hat=cv2.morphologyEx(img,cv2.MORPH_TOPHAT,kernel) cv2.imshow('black_hat',black_hat) cv2.imshow('top_hat',top_hat) key=cv2.waitKey() cv2.destroyAllWindows()
The renderings are as follows:
4, Harris detects and draws corners
4.1 principle
A fixed window slides in any direction on the image. compare before sliding and after sliding, the gray value in the window changes. If the change is large, there are corners in the window
4.2 code implementation
step1. Get structure element
Step 2. Call the cornerHarris() method to detect corners
step3. Detection corner normalization cv2.normalize (equivalent to thresholding result)
step4. Draw corners (circle) in the original drawing
def harris_checkPoint(image): #image=cv2.cvtColor(image,cv2.COLOR_GRAY2BGR) blocksize=2 apertureSize=3 k=0.04 kernel=np.ones((5,5),dtype=np.uint8) image=cv2.morphologyEx(src,cv2.MORPH_OPEN,kernel)#Open operation first corrodes and then expands to remove noise #The effect image is a part of the background image, which is brighter and smoother cv2.imshow('OPEN',image) cv2.waitKey(100) #gray=cv2.cvtColor(image,cv2.COLOR_BGR2GRAY) dst=cv2.cornerHarris(image,blocksize,apertureSize,k) dst_norm=np.empty(dst.shape,dtype=np.float32) #Result normalization cv2.normalize(dst,dst_norm,alpha=0,beta=255,norm_type=cv2.NORM_MINMAX) #Draw results for i in range(dst_norm.shape[0]): for j in range(dst_norm.shape[1]): if int(dst_norm[i,j])>120: cv2.circle(image,(j,i),2,(255,255,255),2) return image
The effect is shown in the figure:
That's enough for today. This is a little more, mainly to understand the application of corrosion and expansion. If you don't want to understand the concept, you can skip this chapter and go to the next chapter. The content of the next chapter will directly attach the complete code to facilitate the operation and see the results. For today's summary and edge detection, please also pay attention to my next chapter. The code word is not easy, please click 666 three times, ha ha!!!