Making dynamic pictures of the project

1. Background of dynamic picture production

With the development of wechat, more and more people like to fight with pictures and use dynamic pictures. If you can explain with a picture, you don't need to force with words. I've made an executable program, which can be used to make dynamic pictures.
First of all, making motion pictures is divided into two parts: picture making and video making.
So before making a motion picture, let's get to know the tools.

2. Tool introduction

2.1 win32 Application

2.1.1 introduction
A Win32 application program can be divided into two parts: program code and UI resources. The two parts are finally integrated into a complete exe executable program with rc. The so-called UI resource refers to the function menu, dialog box appearance, program icon, cursor shape and other things. The programmer describes them in a. rc resource description document. After reading the description of rc document, rc compiler makes all rc resources into a. res document, and then combines it with program code.
Note: the entry point of Win32 program is WinMain. The four parameters of WinMain are passed by the operating system. main is the entry point of console program.
2.1.2 general steps of Win32
1. Design and create a class - improve the structure of the serial port class - the name of the window class (unique) + the response function (callback function) that provides the window message
2. Register window class - the name of window class, and provide window procedure processing function
3. Create windowcreatewindow()
4. Display windowshowwindow ()
5. Update windowupdatewindow()
6. Entering the message loop while (Getmassage) is equivalent to a dead loop
7. Message response: in the user-defined window process processing function, the user intercepts and responds to the messages he needs to process, and uses the default message response function DefWindowProc() to process the messages he doesn't care about
2.1.3 message mechanism of Win32
**Message introduction**
Message: a data structure set in the system:

typedef struct MSG {     HWND hwnd;
//hwnd is the handle of the window, which determines which window procedure function processes the message     
 UINT message;      //Message is a message constant that represents the type of message      
 WPARAM wParam;     //32-bit additional information. The specific content depends on the message type      
 LPARAM lParam;     //32-bit additional information. The specific content depends on the message type      
 DWORD time;       //Time is the time when the message was sent      
 POINT pt;         //Where the mouse is when the message is sent 
 } 

From the above code, we can see that the message type is actually a variable of UINT type. The range of message value defined by the system is 0x00000x03 ", and the range of user-defined message value is 0x0400-0x07".
Message type
Message types include system defined message and user-defined message.
The system defined messages are divided into window messages, command messages and control notification messages.
Window messages: messages related to the internal operation of the window
Command message: when the user selects a command item from the menu, presses a shortcut key, clicks a button on the toolbar or clicks a control, the WM COMMAND message will be sent. Through the members of wParam and lParam in the message structure, the source of the message can be clearly known.
Control notification message: to send more information to the parent window, Microsoft has defined a new WM notification message to extend the WM COMMAND message. M ﹣ notify message still uses MSG message structure, but at this time wParam is the control ID and lParam is an NMHDR pointer. Different controls can expand NMHDR according to rules, so the amount of information transmitted by WM ﹣ notify message can be quite large.
Message queue
Message queue is a queue used to store messages. A global system message queue is maintained in the system, and a UI thread message queue is also maintained for each UI thread:
System message queue: the message queue maintained by the operating system to store messages generated by the system, such as mouse click movement, keyboard press messages, etc. User message queue: a message queue belonging to each application (thread), maintained by the application.
When there is a message in the system message queue, the system will distribute it to the corresponding UI thread message queue of the application according to the UI thread (hWnd: window handle) to which the message belongs.
2.1.4 defects of Win32
The efficiency of using Win32 to develop graphical interface on windows is very low, but the traditional MFC interface library has the following defects:
1. unsightly
2. The interface details are not handled well
3. Low development efficiency
4. Large volume of generating program
5. The use of HOOK technology in MFC interface beautification library may cause system instability or other errors

2.2 Duilib Library

2.2.1 introduction
Duilib library is a powerful and lightweight interface development tool, which can completely separate user interface and processing logic, and greatly improve the development efficiency of user interface. Provide WYSIWYG development tool UIDesigner. Duilib interface library can be widely used in Internet client, tool software client, management system client, multimedia client (such as KTV, touch screen), car computer system, gps system and mobile phone client software, etc., and duilib is only a set of UI Library Based on Win32, not a WIN32 program after using duilib.
2.2.2 Duilib interface library framework
The Library Directory of Duilib mainly contains four directories:
Control: UI classes corresponding to each control of Duilib, such as: button (CButton UI), edit box (CEdit UI), drop-down box (CComboBoxUI), etc.
Layout: UI classes corresponding to various layouts of Duilib, such as horizontal layout UI and vertical layout UI.
Core: the core operation of Duilib Library: window correlation (Win32 process encapsulation - CWindowWnd), window parsing (CMarkup), etc.
Utils: some types of Duilib encapsulation: CDUIString, CPoint, compression, etc.
2.2.3 environmental construction of duilib
Before using the Duilib interface library in a project, you must compile the Duilib library. After compiling, you only need to include the static library and the directory of the Duilib source file in the project, and the generated dll in the project directory. How to do this
1. Copy the Duilib source file and the generated lib folder to the project directory

2. Configure the environment

3. Copy the dll file in the bin directory under the duilib master folder to the exe file of the project after compilation

4. Add a new header file

#include  "UIlib.h"
using namespace DuiLib;//Header file containing 1duilib has namespace
#pragma comment(lib,"DuiLib_ud.lib")//Import library directory

2.2.4 creating a simple window

/INotifyUI:duilib Self defined class-->abstract class
class CDuiFramWnd : public CWindowWnd, public INotifyUI
{
public:
	// The pure virtual function of CWindowWnd class must return the class name of the window defined by the user, which needs to be used when registering the window
	//Returns the name of the window class
	virtual LPCTSTR GetWindowClassName() const
	{
		return _T("DuiFramWnd");//_T macro replace convert format to duelib format
	}
	// uMsg: message ID obtained --- distinguish what type of messages are captured
	virtual LRESULT HandleMessage(UINT uMsg, WPARAM wParam, LPARAM lParam)//Subclass if you need to process system messages, you need to override
	{
		if (WM_CREATE == uMsg)
		{
			m_PaintManager.Init(m_hWnd);

			CDialogBuilder builder;    
			// duilib.xml needs to be placed in the exe directory    
			CControlUI* pRoot = builder.Create(_T("duilib.xml"), (UINT)0, NULL, &m_PaintManager); 
			m_PaintManager.AttachDialog(pRoot);   
			m_PaintManager.AddNotifier(this);     
				return 0;
		}
		//Remove title block
		else if (uMsg == WM_NCACTIVATE)        
		{ 
			if(!::IsIconic(m_hWnd))           
			{
				return (wParam == 0) ? TRUE : FALSE; 
			}
		}
		else if (uMsg == WM_NCCALCSIZE)       
		{
			return 0; 
		}
		else if (uMsg == WM_NCPAINT)
		{
			return 0; 
		}

		//Intercept painting related messages
		LRESULT lRes = 0;
		if (m_PaintManager.MessageHandler(uMsg, wParam, lParam, lRes))
		{
			return lRes;
		}
		//Other news
		//__super:: refers to the
		return __super::HandleMessage(uMsg, wParam, lParam);
	}
	virtual void Notify(TNotifyUI& msg)  //If you need to intercept messages maintained by duilib, you only need to override Notify in the subclass
	{
		//Response button click message
		if (msg.sType == _T("click"))       
		{
			MessageBox(m_hWnd, _T("Hello World"), _T("DuiFramWnd"), IDOK);//Pop up test
		}
	}

private:
	CPaintManagerUI m_PaintManager;
};

int APIENTRY _tWinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPTSTR lpCmdLine, int
	nCmdShow)
{
	CPaintManagerUI::SetInstance(hInstance); 
	// Set the default path of the resource (set here to be in the same directory as exe)
	CPaintManagerUI::SetResourcePath(CPaintManagerUI::GetInstancePath()); 
	CDuiFramWnd framWnd;
	// Cashier is an explicit name in the upper right corner of the window
	// UI > wndstyle > frame: a duilib encapsulated macro, which represents the window, has a title bar, maximizes and minimizes, and closes functions
	// Ws? Ex? Window: the window style of Win32 with a border
	framWnd.Create(NULL, _T("Cashier"), UI_WNDSTYLE_FRAME, WS_EX_WINDOWEDGE);
	//Display window, activate message loop
	framWnd.ShowModal();
	return 0;
}

2.3 ffmpeg

2.3.1 introduction
Ffmpeg is not only a piece of audio and video codec tool, but also a group of audio and video codec development kits, which provides a rich interface for developers to call audio and video processing. In ffmpeg, "FF" refers to "Fast Forward", while mpeg is a dynamic image expert group. It provides a complete solution for recording, converting, and streaming audio and video. It contains a very advanced audio / video codec library libavcodec. In order to ensure high portability and codec quality, many codecs in libavcodec are developed from scratch.

3. Implementation principle

3.1 generating with pictures

3.2 video generation

3.3 interface
The interface implementation is drawn by using UI Designer

4. Function realization

4.1 interface implementation
The implementation of the interface is mainly applied to the duilib library. The main way of making the interface is XML + UI engine + win32 framework. The window is rewritten by XML. Then duilib parses the XML and creates the window successfully.
For Duilib, there is a visual tool for interface design: Designer. Open the above XML file with the interface layout device provided by Duilib: the interface layout device can be used to layout the interface quickly and conveniently, and then save it as XML, and the Duilib program can be parsed.

Note: there is a bug in Duilib's interface layout, which may crash accidentally. You need to save the modified content in time.
Then the implementation of the interface is drawn through the above tools.
4.2 using cmd to send commands to ffmpeg

void SendMessage(CDuiString& strCMD)
	{
		// 1. Initialize the structure    
		SHELLEXECUTEINFO strSEInfo; 
		memset(&strSEInfo, 0, sizeof(SHELLEXECUTEINFO));   
		strSEInfo.cbSize = sizeof(SHELLEXECUTEINFO);   
		strSEInfo.fMask = SEE_MASK_NOCLOSEPROCESS;  //Mask  
		strSEInfo.lpFile = _T("C:\\WINDOWS\\system32\\cmd.exe"); //Command tool location
		strSEInfo.lpParameters = strCMD;  // order of the day   
		strSEInfo.nShow = SW_HIDE;  // Console window hidden

		// 2. Send command to cmd  
		ShellExecuteEx(&strSEInfo); //Call the command line window to execute the user command. In this function, a new process will be created to call the command line window to execute the command
		//Waiting command response completed
		//WaitForSingleObject(strSEInfo.hProcess,INFINITE);
		MessageBox(m_hWnd, _T("Command response complete"), _T("GIF"), IDOK);

	}

4.3 respond to the buttons on the interface
1. Load button
Steps:
Define and initialize the OPENFILENAME struct variable ofn
Call the GetOpenFileName function with the address of ofn as a parameter, and the open file dialog box pops up
Find the file location in the pop-up dialog box and extract the file location from the lpstrFile parameter of ofn
After detecting the acquired path, it is explicitly displayed in the edit box.
Code implementation:

void LoadFile()
	{
		OPENFILENAME ofn;
		memset(&ofn, 0, sizeof(OPENFILENAME));

		TCHAR strPath[MAX_PATH] = { 0 };
		ofn.lStructSize = sizeof(OPENFILENAME);
		ofn.lpstrFile = strPath;
		ofn.nMaxFile = sizeof(strPath);
		ofn.lpstrFilter = _T("All(*.*)\0*.*\0mkv(*.mkv)\0 *.mkv\0");
		ofn.Flags = OFN_PATHMUSTEXIST | OFN_FILEMUSTEXIST;

		if (GetOpenFileName(&ofn))
		{
			//Set the path of the file to edit
			 ((CEditUI*)m_PaintManager.FindControl(_T("edit_path")))->SetText(strPath);


		}
	}

2. Intercept button
The intercept button sends commands to ffmpeg through cmd to intercept the required video.
Command resolution:
ffmpeg -i input.mkv -vcodec copy -acodec copy -ss 00:40:07 -to 00:40:28 11.mkv -y
-i input
11.mkv needs to intercept the path of video
-vcode copy -acode copy indicates the encoding format of the video and audio to be used, and copy indicates the original copy
-ss 00:40:07 for start time-
to 00:40:28 means the end time, i.e. 21 seconds in total-
y: Represents that if the file exists in the directory, the source file will be overwritten
Code implementation:

void CutView()
	{
		CDuiString strPath = CPaintManagerUI::GetInstancePath();//Access path
		strPath += _T("ffmpeg\\");
		CDuiString strViewPath = ((CEditUI*)m_PaintManager.FindControl(_T("edit_path")))->GetText();
		//1. Construction command
		//ffmpeg -i input.mkv -vcodec copy -acodec copy -ss 00:36:55 -to 00:37:07 11.mkv -y
		CDuiString strCMD;
		strCMD += _T("/c ");//\c parameter for command execution
		strCMD += strPath;
		strCMD += _T("ffmpeg -i ");

		//The path of the video, priority to load in the interface
		if (!strViewPath.IsEmpty())
		{
			strCMD += strViewPath;
		}
		else
		{
			strCMD += strPath;//Video path
			strCMD += _T("input.mkv");
		}
		
		strCMD += _T(" -vcodec copy -acodec copy -ss ");
		//Get start time and end time
		CDuiString strStartTime = ((CEditUI*)m_PaintManager.FindControl(_T("edit_start")))->GetText();
		if (!IsValidTime(strStartTime))
		{
			MessageBox(NULL, _T("Wrong starting time"), _T("GIF"), IDOK);
			return;
		}
		CDuiString strEndTime = ((CEditUI*)m_PaintManager.FindControl(_T("edit_end")))->GetText();
		if (!IsValidTime(strEndTime))
		{
			MessageBox(NULL, _T("Wrong termination time"), _T("GIF"), IDOK);
			return;
		}
		strCMD += strStartTime;
		strCMD += _T(" -to ");
		strCMD += strEndTime;
		strCMD += _T(" ");

		//Path to output file
		strCMD += strPath;
		strCMD += _T("11.mkv -y");

		//2. Send command to cmd
		SendMessage(strCMD);

	}

3. Extract SRT button
Extracting SRT is the extraction of subtitles, which can be divided into many types:
(1) Hard subtitle: overlay subtitles on the video screen. Because this kind of subtitle is integrated with the video picture, it has the best compatibility. As long as the video can be played, the subtitle can be displayed. For current mobile phones and MP4 players, only this type of subtitle is supported. The disadvantage is that the subtitle occupies the video picture, destroys the video content, and cannot be cancelled or edited.
(2) Subtitle: make subtitles into a single file. Subtitle files have multiple formats. The advantage of this kind of subtitle is that it does not damage the video picture, can change the subtitle language at any time as needed, and can edit the subtitle content at any time. The disadvantage is that the playback is more complex, which needs the support of the corresponding subtitle playing tools.
(3) Soft subtitle: in some way, the external subtitle and video are packed together. When downloading and copying, only one file needs to be copied. For example, VOB files in DVD, high-definition video packaging formats MKV, TS, AVI, etc. This type of file can generally encapsulate multiple subtitle files at the same time. It is very convenient to select the required subtitle through the player when playing. When necessary, subtitles can also be separated for editing, modification or replacement.
What I am doing here is to add subtitles, and extract the subtitle command: ffmpeg -i 11.mkv input.srt -y
Code implementation:

void GetSRTFile()
	{
		CDuiString strCMD;
		strCMD += _T("/c cd ");
		strCMD += CPaintManagerUI::GetInstancePath() + _T("ffmpeg");
		strCMD += _T(" & ");
		// ffmpeg -i 11.mkv input.srt -y
		strCMD += _T("ffmpeg -i 11.mkv input.srt -y");

		SendMessage(strCMD);
	}

4. Write STR button
This button is to modify the subtitle in the edit box. We need to write the modified subtitle back into the SRT file. There is a problem of subtitle format conversion here. The WideCharToMultiByte function needs to be used for the conversion.
Code implementation:

void WriteSRT()
	{
		CDuiString strPath = CPaintManagerUI::GetInstancePath();//Access path
		strPath += _T("ffmpeg\\input.srt");

		std::ofstream fOut(strPath.GetData());
		//1 get text from list
		CListUI* pList = (CListUI*)m_PaintManager.FindControl(_T("list_srt"));
		int szCount  = pList->GetCount();
		for (int i = 0; i < szCount; i++)
		{
			CListTextElementUI* pListItem = (CListTextElementUI*)pList->GetItemAt(i);

			//Serial number
			CDuiString strNo;
			strNo.Format(_T("%d"), i + 1);
			//time axis
			CDuiString strTime = pListItem->GetText(0);
			//text
			CDuiString strWord = pListItem->GetText(1);
			//2. Write the acquired content back to srt
			string strNewLine = Unicode2ANST(_T("\n"));
			//Write line number
			//unicode 2anst: convert unicode to ANSI, otherwise garbled code will appear
			string itemNo = Unicode2ANST(strNo);
			fOut.write(itemNo.c_str(),itemNo.size());
			fOut.write(strNewLine.c_str(), strNewLine.size());
			//Convert timeline
			string itemTime = Unicode2ANST(strTime);
			fOut.write(itemTime.c_str(), itemTime.size());
			fOut.write(strNewLine.c_str(), strNewLine.size());
			//Writing text
			string itemWord = Unicode2ANST(strWord);
			fOut.write(itemWord.c_str(), itemWord.size());
			fOut.write(strNewLine.c_str(), strNewLine.size());
			//There is a line break between each caption
			fOut.write(strNewLine.c_str(), strNewLine.size());
		}
		fOut.close();
	}

5. Extract video button
Order:
ffmpeg -i 11.mkv -vcodec copy -an -sn 22.mkv -y
-an: cancel audio
-sn: cancel subtitle
Code implementation:

	void GenerateView()
	{
		//ffmpeg -i 11.mkv -vcodec copy -an -sn 22.mkv -y
		CDuiString strCMD;
		strCMD += _T("/c cd ");
		strCMD += CPaintManagerUI::GetInstancePath() + _T("ffmpeg");
		strCMD += _T(" & ");
		strCMD += _T("ffmpeg -i 11.mkv -vcodec copy -an -sn 22.mkv -y");
		//2. Send command to cmd
		SendMessage(strCMD);
	}

6. Burn button
Command: ffmpeg -i 22.mkv -vf subtitles=input.srt 33.mkv -y
Code implementation:

void BornSRT2View()
	{
		//ffmpeg -i 22.mkv -vf subtitles=input.srt 33.mkv -y
		CDuiString strCMD;
		strCMD += _T("/c cd ");
		strCMD += CPaintManagerUI::GetInstancePath() + _T("ffmpeg");
		strCMD += _T(" & ");
		
		strCMD += _T("ffmpeg -i 22.mkv -vf subtitles=input.srt 33.mkv -y");
		//2. Send command to cmd
		SendMessage(strCMD);
	}

7. Generate gif button
Order:
To generate a gif using a picture:
Ffmpeg - R 3 - I. \ pictrue% d.jpg output.gif - Y - R control frame number
Code implementation:

void GenerateGifWithPic()
	{
		// fmpeg -r 3 -i .\Pictrue\%d.jpg output.gif -y
		CDuiString strCMD;
		strCMD += _T("/c cd ");
		strCMD += CPaintManagerUI::GetInstancePath() + _T("ffmpeg");
		strCMD += _T(" & ");
		
		strCMD += _T("fmpeg -r 3 -i .\Pictrue\%d.jpg output.gif -y");
		//2. Send command to cmd
		SendMessage(strCMD);
	}

Generate gif using video:
ffmpeg -i 33.mkv -vf scale=iw/2:ih/2 -f gif output.gif -y
Code implementation:

//Generate dynamic graph
	void GernerateGifWithView()
	{
		// ffmpeg -i 33.mkv -vf scale=iw/2:ih/2 -f gif output.gif -y
		CDuiString strCMD;
		strCMD += _T("/c cd ");
		strCMD += CPaintManagerUI::GetInstancePath() + _T("ffmpeg");
		strCMD += _T(" & ");
		strCMD += _T("ffmpeg -i 33.mkv -vf scale=iw/2:ih/2 -f gif output.gif -y");
		//2. Send command to cmd
		SendMessage(strCMD);
	}

5. Result display

5.1 generate gif with pictures:
Use video to generate gif: (the video here is a video with external subtitles)
This can't be uploaded because the memory is too large, but the reason has been explained above.

6. Problems encountered

1. Didn't send the command to the console in the program?
Solution: found that \ c did not add
2. The Designer library is easy to crash. Once it was written in half, it crashed
There's no way to solve it. You can only draw and save it at the same time
3. When extracting the SRT file, you find that the text is garbled?
Solution: convert UFT-8 to Unicode format and use MultiByteToWideChar function
4. Be careful when writing the project. In the process of writing the file path, write a blank space less, which leads to compilation failure.

Published 31 original articles, won praise 2, visited 769
Private letter follow

Tags: xml codec Windows Mobile

Posted on Sat, 07 Mar 2020 09:07:02 -0500 by SleepyP