Audio and video series 5: rtmp streaming of ffmpeg, Alibaba cloud forwarding and vlc streaming

title: audio and video series 5: rtmp streaming of ffmpeg Alibaba cloud forwarding vlc streaming


tags: [audio and video programming]

date: 2021/11/30

< div align = 'right' > Author: Hackett < / div >

<div align ='right'> WeChat official account: overtime apes </div>

In the first two articles Alicloud server builds Nginx+rtmp streaming server In, we have configured to build Alibaba cloud's rtmp streaming service. We use PC software OBS to push streaming to Alibaba cloud server. Next, we use Raytheon's simplest ffmpeg based streaming device to push streaming in rtmp mode, Alibaba cloud server to forward streaming and VLC to pull streaming.

Link address: The simplest stream pusher based on FFmpeg (taking pushing RTMP as an example)

1, Introduction to RTMP

RTMP is Real Time Messaging Protocol agreement )Acronym for. The protocol is based on TCP and is a protocol family, including RTMP basic protocol and many variants such as RTMPT/RTMPS/RTMPE. RTMP is a network protocol designed for real-time data communication. It is mainly used for audio, video and data communication between Flash/AIR platform and streaming media / interactive server supporting RTMP protocol. Software supporting this protocol includes Adobe Media Server/Ultrant Media Server/red5, etc. Like HTTP, RTMP belongs to the application layer of TCP/IP four layer model-- Baidu Encyclopedia

The role of RTMP Streamer in streaming media system can be shown in the figure below. First, send the video data in the form of RTMP to the streaming media Server (such as FMS, Red5, Wowza, etc.), and then the client (generally Flash Player) can watch the real-time streaming by accessing the streaming media Server.

2, Program flow chart

The packaging format adopted by RTMP is flv. Therefore, when specifying the output streaming media, you need to specify its packaging format as "flv". Similarly, other streaming media protocols also need to specify their encapsulation format. For example, when using UDP to push streaming media, you can specify its packaging format as "mpegts".

av_register_all(): register all codecs of FFmpeg.

avformat_network_init(): initializes the Network.

avformat_open_input(): input.

avformat_find_stream_info(): find flow information.

avformat_alloc_output_context2(): allocate the output AVFormatContext.

avcodec_copy_context(): Copy the settings of AVCodecContext.

avformat_write_header(): Write file header.

av_read_frame(): Get an AVPacket (Get an AVPacket).

av_rescale_q_rnd(): Convert PTS/DTS

av_interleaved_write_frame(): write the end of the file.

3, Code

#include <iostream>

extern "C" {//Include C header file
#include "libavformat/avformat.h" 
#include "libavutil/time.h"

int main(int argc, char* argv[]) {
    AVOutputFormat* ofmt = NULL;
    //The input corresponds to an AVFormatContext, and the output corresponds to an AVFormatContext
    //(Input AVFormatContext and Output AVFormatContext)
    AVFormatContext* ifmt_ctx = NULL, * ofmt_ctx = NULL;
    AVPacket pkt;
    const char* in_filename, * out_filename;
    int ret, i;
    int videoindex = -1;
    int frame_index = 0;
    int64_t start_time = 0;
    //in_filename  = "";
    //in_filename  = "cuc_ieschool.mkv";
    //in_filename  = "cuc_ieschool.ts";
    //in_filename  = "cuc_ieschool.mp4";
    //in_filename  = "cuc_ieschool.h264";
    in_filename = "./ouput_1min.flv";//Input file URL
    //in_filename  = "shanghai03_p.h264";

    //out_filename = "rtmp://localhost/publishlive/livestream ";// Output URL [RTMP]
    out_filename = "rtmp://Alibaba cloud server IP:1935/live "; / / Output URL [RTMP]
    //out_filename = "rtp:// "; / / Output URL [UDP]

    //Register all FFmpeg codecs
    if ((ret = avformat_open_input(&ifmt_ctx, in_filename, 0, 0)) < 0) {
        printf("Could not open input file.");
        goto end;
    if ((ret = avformat_find_stream_info(ifmt_ctx, 0)) < 0) {
        printf("Failed to retrieve input stream information");
        goto end;

    for (i = 0; i < ifmt_ctx->nb_streams; i++)
        if (ifmt_ctx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO) {
            videoindex = i;

    av_dump_format(ifmt_ctx, 0, in_filename, 0);


    avformat_alloc_output_context2(&ofmt_ctx, NULL, "flv", out_filename); //RTMP
    //avformat_alloc_output_context2(&ofmt_ctx, NULL, "mpegts", out_filename);//UDP

    if (!ofmt_ctx) {
        printf("Could not create output context\n");
        ret = AVERROR_UNKNOWN;
        goto end;
    ofmt = ofmt_ctx->oformat;
    for (i = 0; i < ifmt_ctx->nb_streams; i++) {
        //Create output AVStream according to input AVStream
        AVStream* in_stream = ifmt_ctx->streams[i];
        AVStream* out_stream = avformat_new_stream(ofmt_ctx, in_stream->codec->codec);
        if (!out_stream) {
            printf("Failed allocating output stream\n");
            ret = AVERROR_UNKNOWN;
            goto end;
        //Copy the settings of AVCodecContext
        ret = avcodec_copy_context(out_stream->codec, in_stream->codec);
        if (ret < 0) {
            printf("Failed to copy context from input to output stream codec context\n");
            goto end;
        out_stream->codec->codec_tag = 0;
        if (ofmt_ctx->oformat->flags & AVFMT_GLOBALHEADER)
            out_stream->codec->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
    //Dump Format
    av_dump_format(ofmt_ctx, 0, out_filename, 1);
    //Open output URL
    if (!(ofmt->flags & AVFMT_NOFILE)) {
        ret = avio_open(&ofmt_ctx->pb, out_filename, AVIO_FLAG_WRITE);
        if (ret < 0) {
            printf("Could not open output URL '%s'", out_filename);
            goto end;
    //Write file header
    ret = avformat_write_header(ofmt_ctx, NULL);
    if (ret < 0) {
        printf("Error occurred when opening output URL\n");
        goto end;

    start_time = av_gettime();
    while (1) {
        AVStream* in_stream, * out_stream;
        //Get an AVPacket (get an AVPacket)
        ret = av_read_frame(ifmt_ctx, &pkt);
        if (ret < 0)
        //FIX: No PTS (Example: Raw H.264)
        //Simple Write PTS
        if (pkt.pts == AV_NOPTS_VALUE) {
            //Write PTS
            AVRational time_base1 = ifmt_ctx->streams[videoindex]->time_base;
            //Duration between 2 frames (us)
            int64_t calc_duration = (double)AV_TIME_BASE / av_q2d(ifmt_ctx->streams[videoindex]->r_frame_rate);
            pkt.pts = (double)(frame_index * calc_duration) / (double)(av_q2d(time_base1) * AV_TIME_BASE);
            pkt.dts = pkt.pts;
            pkt.duration = (double)calc_duration / (double)(av_q2d(time_base1) * AV_TIME_BASE);
        if (pkt.stream_index == videoindex) {
            AVRational time_base = ifmt_ctx->streams[videoindex]->time_base;
            AVRational time_base_q = { 1,AV_TIME_BASE };
            int64_t pts_time = av_rescale_q(pkt.dts, time_base, time_base_q);
            int64_t now_time = av_gettime() - start_time;
            if (pts_time > now_time)
                av_usleep(pts_time - now_time);


        in_stream = ifmt_ctx->streams[pkt.stream_index];
        out_stream = ofmt_ctx->streams[pkt.stream_index];
        /* copy packet */
        //Convert PTS/DTS
        pkt.pts = av_rescale_q_rnd(pkt.pts, in_stream->time_base, out_stream->time_base, (AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
        pkt.dts = av_rescale_q_rnd(pkt.dts, in_stream->time_base, out_stream->time_base, (AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
        pkt.duration = av_rescale_q(pkt.duration, in_stream->time_base, out_stream->time_base);
        pkt.pos = -1;
        //Print to Screen
        if (pkt.stream_index == videoindex) {
            printf("Send %8d video frames to output URL\n", frame_index);
        //ret = av_write_frame(ofmt_ctx, &pkt);
        ret = av_interleaved_write_frame(ofmt_ctx, &pkt);

        if (ret < 0) {
            printf("Error muxing packet\n");


    //Write file tracker
    /* close output */
    if (ofmt_ctx && !(ofmt->flags & AVFMT_NOFILE))
    if (ret < 0 && ret != AVERROR_EOF) {
        printf("Error occurred.\n");
        return -1;
    return 0;

4, Operation results

If you think the article is good, you can give a "three links". The article is synchronized to the following personal WeChat official account [overtime ape].

I'm hackett. I'll see you next time

Tags: Alibaba Cloud RTMP vlc

Posted on Tue, 30 Nov 2021 03:07:00 -0500 by bstruss