In depth learning of Hisilicon SDK and API from official routines

1, Basic principles of image coding and compression

Since a large number of excellent articles have talked about this on the Internet, I won't write here. The content repetition is meaningless!
reference resources: http://blog.csdn.net/newchenxf/article/details/51693753

Compression and decompression is the process of mathematical operation.

2, Interpretation of image coding in MPP manual


	Code stream(DataRate)It refers to the data flow used by video files in unit time, also known as code rate, which is in video coding
 The most important part of picture quality control. At the same resolution, the larger the code stream of the video file, the smaller the compression ratio and the lower the picture quality
 The higher the amount.
   Uplink bandwidth is the bandwidth of local information uploaded to the network. Uplink rate refers to the rate at which the user's computer sends information to the network
 Data transfer rate, e.g FTP When uploading files to the Internet, the "uplink rate" affects the upload speed.

   Downlink bandwidth is the bandwidth for downloading information from the network. Downlink rate refers to the data when the user's computer downloads information from the network
 Transmission rate, such as from FTP When files on the server are downloaded to the user's computer, the "downlink rate" affects the downlink speed.

	Note: the bandwidth of the monitoring point is the minimum bandwidth required for uplink(The monitoring point uploads the video information to the monitoring center);monitor
 The bandwidth of the center is the minimum bandwidth required for downlink(Download the video information of the monitoring point to the monitoring center);Example: Telecom 2 Mbps
 of ADSL Broadband. Theoretically, its uplink bandwidth is 512 kbps=64kb/s,Its downlink bandwidth is 2 Mbps=256kb/s  

BP: the most basic, high transmission rate, but poor video quality
MP: mainstream, transmission speed and video DC are relatively OK

Detailed explanation: https://blog.csdn.net/u014470361/article/details/88713266

   the typical coding process includes the reception of input image, occlusion and coverage of image content, image coding, and output of code stream.

  VENC module consists of encoding channel sub module (VENC) and encoding protocol sub module (H.264/H.265/JPEG/MJPEG). The channel supports receiving YUV format image input. The supported formats are semi planar YUV 4:2:0 or semi planar YUV 4:2:2, where H.264/H.265 only supports semi planar YUV 4:2:0, and JPEG/MJPEG supports semi planar YUV 4:2:0 or semi planar YUV 4:2:2. In addition, the Hi3518EV200 can support single component input (there is only Y component, i.e. black-and-white image). The channel module receives the external original image data, and does not care which external module the image data comes from.

After the channel receives the image, compare the image size with the encoded channel size:
	If the input image is larger than the encoding channel size, VENC Will be called according to the size of the encoding channel VGS yes
 The source image is reduced, and then the reduced image is encoded.
	If the input image is smaller than the encoding channel size, VENC Discard the source image. VENC Enlarged input map is not supported
 Like coding.
	If the input image is equivalent to the size of the encoding channel, VENC Directly accept the source image and encode it.

explain:
1, The frame rate control of the channel is not turned on by default, and the user needs to call the interface setting. RC(Rate controller) It also has frame rate control
 Control function. Recommended use RC Frame rate control, so as not to cause too much impact on code rate control.

2, about Hi3518EV200 of H.264 Coding, when the input image format is switched from non single component to single component
 The quantization error is predicted between frames and the next one is encoded I There will be chromaticity residue in the image before the frame. The customer is advised to switch between single points
 Call interface when measuring HI_MPI_VENC_ResetChn Perform channel reset.

  the REGION module supports occlusion and superposition of image content. After completing the video area management, the image is sent to the specific protocol type coding channel to complete the video coding and output the code stream.

   the code rate controller controls the coding code rate. From the perspective of Informatics, the lower the compression ratio of the image, the higher the quality of the compressed image; The higher the image compression ratio, the lower the quality of the compressed image. For the real scene with changing scene, the image quality is stable and the coding rate will fluctuate; The coding rate is stable and the image quality will fluctuate. Taking H.264 coding as an example, generally, the lower the image Qp, the better the image quality and the higher the bit rate; The higher the image Qp, the worse the image quality and the lower the bit rate. Rate control is for continuous coded code streams, so the JPEG protocol coding channel does not include rate control function.

Qp reference: http://blog.csdn.net/u013354805/article/details/51988171

  the rate controller provides three rate control modes for H.264\H.265\MJPEG protocol coding channel CBR, VBR and FIXQP to adjust the image quality and rate. Hi3518EV200 does not support H.265 coding, so it does not support H.265 type rate control.

CBR (Constant Bit Rate) fixed bit rate.

VBR (Variable Bit Rate) Variable Bit Rate allows the coding rate to fluctuate within the code rate statistics time, so as to ensure the stable quality of the coded image.

Fix Qp fixed QP value. In the code rate statistics time, the QP values of all macroblocks of the encoded image are the same. Using the image QP value set by the user, the QP values of I frame and P frame can be set respectively.

For more details, please read the documentation:
Hi3518E V200R001C01SPC030.software\board\document_cn\HiMPP IPC V2.0 media processing software development reference.pdf

3, Interpretation of source code of venc module in sample

    /******************************************
     step 5: start stream venc
    ******************************************/
    /*** HD1080P **/
    printf("\t c) cbr.\n");
    printf("\t v) vbr.\n");
    printf("\t f) fixQp\n");
    printf("please input choose rc mode!\n");
    c = (char)getchar();
    switch(c)
    {
        case 'c':
            enRcMode = SAMPLE_RC_CBR;
            break;
        case 'v':
            enRcMode = SAMPLE_RC_VBR;
            break;
        case 'f':
            enRcMode = SAMPLE_RC_FIXQP;
            break;
        default:
            printf("rc mode! is invaild!\n");
            goto END_VENC_1080P_CLASSIC_4;
    }

	/*** enSize[0] **/
	if(s32ChnNum >= 1)
	{
		VpssGrp = 0;
	    VpssChn = 0;
	    VencChn = 0;
	    s32Ret = SAMPLE_COMM_VENC_Start(VencChn, enPayLoad[0],\
	                                   gs_enNorm, enSize[0], enRcMode,u32Profile);
	    if (HI_SUCCESS != s32Ret)
	    {
	        SAMPLE_PRT("Start Venc failed!\n");
	        goto END_VENC_1080P_CLASSIC_5;
	    }

	    s32Ret = SAMPLE_COMM_VENC_BindVpss(VencChn, VpssGrp, VpssChn);
	    if (HI_SUCCESS != s32Ret)
	    {
	        SAMPLE_PRT("Start Venc failed!\n");
	        goto END_VENC_1080P_CLASSIC_5;
	    }
	}
SAMPLE_COMM_VENC_Start
	SAMPLE_COMM_SYS_GetPicSize //Get the parameters of the picture
	HI_MPI_VENC_CreateChn     Create encoding channel
	HI_MPI_VENC_StartRecvPic Open the encoding channel to receive the input image.
SAMPLE_COMM_VENC_BindVpss
	HI_MPI_SYS_Bind Data source to data receiver binding interface

4, Encoded stream file output and column summary

    /******************************************
     step 6: stream venc process -- get stream, then save it to file. 
    ******************************************/
    s32Ret = SAMPLE_COMM_VENC_StartGetStream(s32ChnNum);
    if (HI_SUCCESS != s32Ret)
    {
        SAMPLE_PRT("Start Venc failed!\n");
        goto END_VENC_1080P_CLASSIC_5;
    }

    printf("please press twice ENTER to exit this sample\n");
    getchar();
    getchar();

    /******************************************
     step 7: exit process
    ******************************************/
    SAMPLE_COMM_VENC_StopGetStream();    
/******************************************************************************
* funciton : start get venc stream process thread
******************************************************************************/
HI_S32 SAMPLE_COMM_VENC_StartGetStream(HI_S32 s32Cnt)
{
    gs_stPara.bThreadStart = HI_TRUE;
    gs_stPara.s32Cnt = s32Cnt;

    return pthread_create(&gs_VencPid, 0, SAMPLE_COMM_VENC_GetVencStreamProc, (HI_VOID*)&gs_stPara);
}
/******************************************************************************
* funciton : get stream from each channels and save them
******************************************************************************/
HI_VOID* SAMPLE_COMM_VENC_GetVencStreamProc(HI_VOID *p)
{
//I deleted part of the function, leaving only the main part for sorting out ideas

    /******************************************
     step 1:  check & prepare save-file & venc-fd
    ******************************************/
    for (i = 0; i < s32ChnTotal; i++)
    {
        /* decide the stream file name, and open file to save stream */
        VencChn = i;
        s32Ret = HI_MPI_VENC_GetChnAttr(VencChn, &stVencChnAttr);
        enPayLoadType[i] = stVencChnAttr.stVeAttr.enType;

        s32Ret = SAMPLE_COMM_VENC_GetFilePostfix(enPayLoadType[i], szFilePostfix);

        /* Set Venc Fd. */
        VencFd[i] = HI_MPI_VENC_GetFd(i);//Open a blank file, get the file descriptor, and then
        if (VencFd[i] < 0)//Write, that is, write the video stream file
        {
            SAMPLE_PRT("HI_MPI_VENC_GetFd failed with %#x!\n", 
                   VencFd[i]);
            return NULL;
        }
        if (maxfd <= VencFd[i])
        {
            maxfd = VencFd[i];
        }
    }

    /******************************************
     step 2:  Start to get streams of each channel.
    ******************************************/
    while (HI_TRUE == pstPara->bThreadStart)//Start getting video streams from each channel
    {//When the enter key is pressed twice, the variable pstpara - > bthreadstart is assigned to false
        FD_ZERO(&read_fds);
        for (i = 0; i < s32ChnTotal; i++)
        {
            FD_SET(VencFd[i], &read_fds);
        }

        TimeoutVal.tv_sec  = 2;
        TimeoutVal.tv_usec = 0;
        s32Ret = select(maxfd + 1, &read_fds, NULL, NULL, &TimeoutVal);
        if (s32Ret < 0)
        {
            SAMPLE_PRT("select failed!\n");
            break;
        }
        else if (s32Ret == 0)
        {
            SAMPLE_PRT("get venc stream time out, exit thread\n");
            continue;
        }
        else
        {
            for (i = 0; i < s32ChnTotal; i++)
            {
                if (FD_ISSET(VencFd[i], &read_fds))
                {
                    /*******************************************************
                     step 2.1 : query how many packs in one-frame stream.
                    *******************************************************/
                    memset(&stStream, 0, sizeof(stStream));
                    s32Ret = HI_MPI_VENC_Query(i, &stStat);//Query coding channel status
					
					/*******************************************************
					step 2.2 :suggest to check both u32CurPacks and u32LeftStreamFrames at the same time,for example:
					 if(0 == stStat.u32CurPacks || 0 == stStat.u32LeftStreamFrames)
					 {
						SAMPLE_PRT("NOTE: Current  frame is NULL!\n");
						continue;
					 }
					 In the encoding channel state structure, u32CurPacks represents the number of bitstream packets of the current frame.
					Calling hi_ MPI_ VENC_ Make sure u32CurPacks is greater than 0 before getstream

					In the encoding channel state structure, u32LeftStreamFrames represents the code stream buffer 
					The number of frames remaining in the. Before resetting the channel, you can query whether there is still a code stream of the image
					It is necessary to determine the reset time to prevent the possible code stream from being cleaned out during reset
					*******************************************************/
					if(0 == stStat.u32CurPacks)
					{
						  SAMPLE_PRT("NOTE: Current  frame is NULL!\n");
						  continue;
					}
                    /*******************************************************
                     step 2.3 : malloc corresponding number of pack nodes.
Code stream structure VENC_STREAM_S consists of four parts:
− Bitstream packet information pointer pstPack
 Point to a group of vencs_ PACK_ S, which is allocated by the caller. If it is obtained by package
 If yes, the space is not less than the size of sizeof (VENC_PACK_S); If acquired by frame, this
 Space not less than N ×  sizeof (VENC_PACK_S), where N represents the size in the current frame
 The number of packages can be obtained through the query interface after select.
− Number of bitstream packets u32PackCount
 When entered, this value specifies Venc in pstPack_ PACK_ Number of S. When getting by package,
u32PackCount Must not be less than 1; When acquiring by frame, u32PackCount must not be less than the current frame
 Number of packages. After the function call is successful, u32PackCount returns the number of packages actually filled with pstPack
 Number.
− Serial number u32Seq
 Frame sequence number when acquiring by frame; When obtained by package, it is the package serial number.
− Code stream characteristic information. The data type is a consortium, which contains the code stream characteristic information corresponding to different coding protocols
stH264Info/ stJpegInfo/ stMpeg                
                    *******************************************************/
                    stStream.pstPack = (VENC_PACK_S*)malloc(sizeof(VENC_PACK_S) * stStat.u32CurPacks);

                    /*******************************************************
                     step 2.4 : call mpi to get one-frame stream
                    *******************************************************/
                    stStream.u32PackCount = stStat.u32CurPacks;
                    //Get encoded code stream
                    s32Ret = HI_MPI_VENC_GetStream(i, &stStream, HI_TRUE);
                    /*******************************************************
                     step 2.5 : save frame to file,
                    *******************************************************/
                    s32Ret = SAMPLE_COMM_VENC_SaveStream(enPayLoadType[i], pFile[i], &stStream);
                    /*******************************************************
                     step 2.6 : release stream
                    *******************************************************/
                    //Free the bitstream cache.
                    s32Ret = HI_MPI_VENC_ReleaseStream(i, &stStream);
                    /*******************************************************
                     step 2.7 : free pack nodes
                    *******************************************************/
                    free(stStream.pstPack);
                    stStream.pstPack = NULL;
                }
            }
        }
    }

    /*******************************************************
    * step 3 : close save-file
    *******************************************************/
    for (i = 0; i < s32ChnTotal; i++)
    {
        fclose(pFile[i]);//Close open files
    }

    return NULL;
}
HI_S32 SAMPLE_COMM_VENC_StopGetStream()
{
    if (HI_TRUE == gs_stPara.bThreadStart)
    {
        gs_stPara.bThreadStart = HI_FALSE;//Used to stop the While loop in the function above
        pthread_join(gs_VencPid, 0);
    }
    return HI_SUCCESS;
}
SAMPLE_COMM_VENC_SaveStream
	SAMPLE_COMM_VENC_SaveH264
		        fwrite
       			fflush //flush() forces the data in the buffer to be written back to the file specified by the parameter stream

If you want to understand more function details and some processes and parameter settings of video codec, please carefully read the data manual mentioned in our previous articles this season.

Note: most of this material is compiled from the course notes of Mr. Zhu's Internet of things lecture hall, refers to the development manual provided in Huawei Hisilicon SDK, and quotes the contents of some other people's blogs. If there is infringement, please contact to delete it! The level is limited. If there are errors, you are welcome to communicate in the comment area.

Tags: Linux ARM

Posted on Tue, 30 Nov 2021 08:37:34 -0500 by reeferd