catalogue
- Breakpoint debugging of ffplay
- (unpacking part) analysis of common structures and their relationships
- data
- harvest
If you want to do a good job, you must first sharpen your tools. Breakpoint debugging is very important for us to sort out the process and troubleshoot problems. You can debug ffmpeg and conduct convenient debugging and Analysis on ide s such as XCode, VS code and QT. In this article, taking XCode as an example, we first introduce the breakpoint debugging of ffplay and analyze it with ffmpeg 4.4.
1, Breakpoint debugging of ffplay
First download and compile ffmpeg. For details, please refer to Audio and video development journey (33) - cross compile FFmpeg(3.x and 4.x) used by android The difference is that we are not cross compiling this time, but compiling, installing and debugging on the Mac.
./configure --enable-static --disable-shared --enable-debug --disable-doc --disable-x86asm --enable-nonfree --enable-libvpx --enable-gpl --enable-opengl --enable-libx264 --enable-libx265 --enable-libvmaf make -j8 sudo make install
After the compilation is successful, we will see several important executable files ffmpeg_g, ffprobe_g and ffplay_g, which will be used in the next operation and debugging. How to configure and debug ffmpeg source code under Xcode? Please refer to: https://www.jianshu.com/p/27a90b113413
We analyze the structure used in the read_thread process of ffplay at the break point of the main function of ffplay.c.
Open media streaming
VideoState *stream_open(const char *filename,const AVInputFormat *iformat)
Structure involved: AVInputFormat
Start readthread to start reading
is->read_tid = SDL_CreateThread(read_thread, "read_thread", is);
Allocate AVFormatContext memory
AVFormatContext ic = avformat_alloc_context();
Open streaming media file
int avformat_open_input(AVFormatContext **ps, const char *filename, const AVInputFormat *fmt, AVDictionary **options)
Structures involved: AVFormatContext, AVInputFormat, AVDictionary
Get stream information
int avformat_find_stream_info(AVFormatContext *ic, AVDictionary **options)
Structure involved: AVStream AVCodecParameters AVRational
Loop reading frame data
for (;;) { ... int av_read_frame(AVFormatContext *s, AVPacket *pkt) ... }
Structures involved: AVFormatContext, AVPacket, etc
The unpacking process is here first. It can be seen that if you want to learn the source code of ffplay, you must first understand the main process and the key structures involved in the process. In the next section, we will analyze these structures in detail.
3, (unpacking part) common structures and their relationship analysis common structures and their relationship analysis
3.1 common structures and their relationships
FFMPEG There are many structures in. The most critical structures can be divided into the following categories: a) Solution protocol( http,rtsp,rtmp,mms) AVIOContext,URLProtocol,URLContext It mainly stores the type and status of the protocol used by video and audio. URLProtocol Store the packaging format used for input video and audio. Each protocol corresponds to one URLProtocol Structure. (Note: FFMPEG The file is also used as a protocol“ file") b) Unpacking( flv,avi,rmvb,mp4) AVFormatContext It mainly stores the information contained in the video and audio packaging format; AVInputFormat Stores the packaging format used by the input video and audio. Each video and audio packaging format corresponds to one AVInputFormat Structure. c) Decode( h264,mpeg2,aac,mp3) each AVStream Store a video/Relevant data of audio stream; each AVStream Corresponding to one AVCodecContext,Store this video/Relevant data of audio stream using decoding method; each AVCodecContext Corresponding to one in AVCodec,Include this video/The decoder corresponding to the audio. Each decoder corresponds to one AVCodec Structure. d) Save data For video, each structure usually stores one frame; for audio, there may be several frames Data before decoding: AVPacket Decoded data: AVFrame Quoted from: https://blog.csdn.net/leixiaohua1020/article/details/11693997
Their relationship is as follows:

Picture from: The relationship between the most critical structures in FFMPEG
3.2. AVFormatContext This structure is defined in libavformat/Avformat.h. It is a data structure that runs through the whole process and is used as a parameter in many functions. The functions of several main variables are as follows:
struct AVInputFormat *iformat: Encapsulation format of input data struct AVOutputFormat *oformat: Encapsulation format of output data AVIOContext *pb: Cache of input data unsigned int nb_streams: Number of video and audio streams AVStream **streams: Video audio stream char filename[1024]: file name int64_t duration: Duration in microseconds us,Convert to seconds (divide by 1000000) int bit_rate: Bit rate (unit) bps,Convert to kbps (you need to divide by 1000) AVDictionary *metadata: metadata
3.3 AVInputFormat This structure is also defined in libavformat/Avformat.h. It is the main variable of the unpacker object. Its functions are as follows
const char *name: Name of the format const char *mime_type: mime Type such as video/avc video/hevc audio/aac etc. And a series of function pointers int (*read_probe)(const AVProbeData *); int (*read_packet)(struct AVFormatContext *, AVPacket *pkt); int (*read_close)(struct AVFormatContext *); int (*read_seek)(struct AVFormatContext *, int stream_index, int64_t timestamp, int flags); int (*read_play)(struct AVFormatContext *); int (*read_pause)(struct AVFormatContext *); int (*read_seek2)(struct AVFormatContext *s, int stream_index, int64_t min_ts, int64_t ts, int64_t max_ts, int flags);
3.4 AVStream Each AVStream stores relevant data of a video / audio stream; it is a stream object separated by the unpacker, that is, the product of unpacking, which is saved in AVFormatcontext.
The structure is also defined in libavformat/Avformat.h. The main variables are as follows:
int index; Stream index int id; flow id void *priv_data; Stream data AVRational time_base; Time base, through which you can PTS,DTS Into real time; PTS*time_base=Real time int64_t duration: Flow length AVRational sample_aspect_ratio; sampling rate AVRational avg_frame_rate: Frame rate AVCodecContext *codec: Point to this video/Audio streaming AVCodecContext(They are one-to-one correspondence)
AVStream is not only the output of the unpacking link, but also the input of the decoding link. Each AVStream corresponds to an AVCodecContext, which stores the relevant data of the decoding method used by the video / audio stream; each AVCodecContext corresponds to an AVCodec, including the decoder corresponding to the video / audio. Each decoder corresponds to an AVCodec structure. The data structure of the decoding part is analyzed and studied in the next article.
3.5 AVPacket A structure that stores information related to compressed encoded data, and stores data before decoding after understanding encapsulation, as well as PTS, DTS, Duration, streamId and other information The structure definition is located in libavcodec/Packet.h. The main variables are as follows:
uint8_t *data; about H.264 Come on. One AVPacket of data Usually corresponds to one NAL. int size: data Size of int64_t pts: presentation time stamp int64_t dts: Decoding timestamp AVPacketSideData *side_data;Additional information
3, Information
Android audio and video development - Chapter 8 Xcode debugging ffmpeg source code (XV) The relationship between the most critical structures in FFMPEG FFMPEG structure analysis: AVFormatContext FFMPEG structure analysis: AVStream FFMPEG structure analysis: AVPacket
4, Harvest
Through the study and practice of this article, we have learned
- How to debug ffmpeg at breakpoint under Xcode and analyze the unpacking process of ffplay
- Understand the relationship between common structures: de protocol, de encapsulation, decoding corresponding structures and the relationship between them
- Understand the main variables and functions of several key structures related to unpacking. AVFormatContext, AVInputFormat, AVStream
Thank you for reading The next section we analyze the common structures of ffmpeg decoding part is welcome to pay attention to the official account of "audio and video development tour" to learn and grow together. Welcome to communicate