title: ffmpeg encoding, Opus sound in the webm container does not work
body: <p>I'm trying to encode audio and video to a webm file with VP8 and Opus encoding. It almost works. (I use FFmpeg 3.3.2)</p>
  
  <p>I can make a only video webm file and play it in VLC, FFPlay and upload it to YouTube (and all works). If I add Opus sound to the file, it still works in VLC but <strong>not</strong> in FFPlay or on youtube, on youtube the sound becomes just "ticks".</p>
  
  <p>I have the same problem if I encode only Opus audio to the webm file; it only works in VLC. But if I encode only Opus audio to a <strong>ogg container</strong> it works everywhere, and I can even use FFmpeg to combine the ogg file with a video only webm file and produce a fully working webm file with audio and video.</p>
  
  <p>So it seems to me that only when I use my code to encode Opus into a webm container, it just wont work in most players and YouTube. I need it to work in  youtube.</p>
  
  <p>Here is the code for the opus to webm only encoding (you can toggle ogg/webm with the define): <a href="https://pastebin.com/jyQ4s3tB" rel="nofollow noreferrer">https://pastebin.com/jyQ4s3tB</a></p>
  
  <pre class="lang-cpp prettyprint-override"><code>#include &lt;algorithm&gt;
  #include &lt;iterator&gt;
  
  extern "C"
  {
  
  //#define OGG
  
  #include "libavcodec/avcodec.h"
  #include "libavdevice/avdevice.h"
  #include "libavfilter/avfilter.h"
  #include "libavformat/avformat.h"
  #include "libavutil/avutil.h"
  #include "libavutil/imgutils.h"
  #include "libswscale/swscale.h"
  #include "libswresample/swresample.h"
  
      enum InfoCodes
      {
          ENCODED_VIDEO,
          ENCODED_AUDIO,
          ENCODED_AUDIO_AND_VIDEO,
          NOT_ENOUGH_AUDIO_DATA,
      };
  
      enum ErrorCodes
      {
          RES_NOT_MUL_OF_TWO = -1,
          ERROR_FINDING_VID_CODEC = -2,
          ERROR_CONTEXT_CREATION = -3,
          ERROR_CONTEXT_ALLOCATING = -4,
          ERROR_OPENING_VID_CODEC = -5,
          ERROR_OPENING_FILE = -6,
          ERROR_ALLOCATING_FRAME = -7,
          ERROR_ALLOCATING_PIC_BUF = -8,
          ERROR_ENCODING_FRAME_SEND = -9,
          ERROR_ENCODING_FRAME_RECEIVE = -10,
          ERROR_FINDING_AUD_CODEC = -11,
          ERROR_OPENING_AUD_CODEC = -12,
          ERROR_INIT_RESMPL_CONTEXT = -13,
          ERROR_ENCODING_SAMPLES_SEND = -14,
          ERROR_ENCODING_SAMPLES_RECEIVE = -15,
          ERROR_WRITING_HEADER = -16,
          ERROR_INIT_AUDIO_RESPAMLER = -17,
      };
  
      AVCodecID aud_codec_comp_id = AV_CODEC_ID_OPUS;
      AVSampleFormat sample_fmt_comp = AV_SAMPLE_FMT_FLT;
  
      AVCodecID aud_codec_id;
      AVSampleFormat sample_fmt;
  
  #ifndef OGG
      char* compressed_cont = "webm";
  #endif
  #ifdef OGG
      char* compressed_cont = "ogg";
  #endif
  
      AVCodec *aud_codec = NULL;
      AVCodecContext *aud_codec_context = NULL;
      AVFormatContext *outctx;
      AVStream *audio_st;
      AVFrame *aud_frame;
      SwrContext *audio_swr_ctx;
  
      int vid_frame_counter, aud_frame_counter;
      int vid_width, vid_height;
  
      char* concat(const char *s1, const char *s2)
      {
          char *result = (char*)malloc(strlen(s1) + strlen(s2) + 1);
  
          strcpy(result, s1);
          strcat(result, s2);
  
          return result;
      }
  
      int setup_audio_codec()
      {
          aud_codec_id = aud_codec_comp_id;
          sample_fmt = sample_fmt_comp;
  
          // Fixup audio codec
          if (aud_codec == NULL)
          {
              aud_codec = avcodec_find_encoder(aud_codec_id);
              avcodec_register(aud_codec);
          }
  
          if (!aud_codec)
              return ERROR_FINDING_AUD_CODEC;
  
          return 0;
      }
  
      int initialize_audio_stream(AVFormatContext *local_outctx, int sample_rate, int per_frame_audio_samples, int audio_bitrate)
      {
          aud_codec_context = avcodec_alloc_context3(aud_codec);
          if (!aud_codec_context)
              return ERROR_CONTEXT_CREATION;
  
          aud_codec_context-&gt;bit_rate = audio_bitrate;
          aud_codec_context-&gt;sample_rate = sample_rate;
          aud_codec_context-&gt;sample_fmt = sample_fmt;
          aud_codec_context-&gt;channel_layout = AV_CH_LAYOUT_STEREO;
          aud_codec_context-&gt;channels = av_get_channel_layout_nb_channels(aud_codec_context-&gt;channel_layout);
          //aud_codec_context-&gt;profile = FF_PROFILE_AAC_MAIN;
  
          aud_codec_context-&gt;codec = aud_codec;
          aud_codec_context-&gt;codec_id = aud_codec_id;
  
          AVRational time_base;
          time_base.num = per_frame_audio_samples;
          time_base.den = aud_codec_context-&gt;sample_rate;
          aud_codec_context-&gt;time_base = time_base;
  
          int ret = avcodec_open2(aud_codec_context, aud_codec, NULL);
  
          if (ret &lt; 0)
              return ERROR_OPENING_AUD_CODEC;
  
          local_outctx-&gt;audio_codec = aud_codec;
          local_outctx-&gt;audio_codec_id = aud_codec_id;
  
          audio_st = avformat_new_stream(local_outctx, aud_codec);
  
          audio_st-&gt;codecpar-&gt;bit_rate = aud_codec_context-&gt;bit_rate;
          audio_st-&gt;codecpar-&gt;sample_rate = aud_codec_context-&gt;sample_rate;
          audio_st-&gt;codecpar-&gt;channels = aud_codec_context-&gt;channels;
          audio_st-&gt;codecpar-&gt;channel_layout = aud_codec_context-&gt;channel_layout;
          audio_st-&gt;codecpar-&gt;codec_id = aud_codec_context-&gt;codec_id;
          audio_st-&gt;codecpar-&gt;codec_type = AVMEDIA_TYPE_AUDIO;
          audio_st-&gt;codecpar-&gt;format = aud_codec_context-&gt;sample_fmt;
          audio_st-&gt;codecpar-&gt;frame_size = aud_codec_context-&gt;frame_size;
          audio_st-&gt;codecpar-&gt;block_align = aud_codec_context-&gt;block_align;
          audio_st-&gt;codecpar-&gt;initial_padding = aud_codec_context-&gt;initial_padding;
          audio_st-&gt;codecpar-&gt;extradata = aud_codec_context-&gt;extradata;
          audio_st-&gt;codecpar-&gt;extradata_size = aud_codec_context-&gt;extradata_size;
  
          aud_frame = av_frame_alloc();
          aud_frame-&gt;nb_samples = aud_codec_context-&gt;frame_size;
          aud_frame-&gt;format = aud_codec_context-&gt;sample_fmt;
          aud_frame-&gt;channel_layout = aud_codec_context-&gt;channel_layout;
          aud_frame-&gt;sample_rate = aud_codec_context-&gt;sample_rate;
  
          int buffer_size;
          if (aud_codec_context-&gt;frame_size == 0)
          {
              buffer_size = per_frame_audio_samples * 2 * 4;
              aud_frame-&gt;nb_samples = per_frame_audio_samples;
          }
          else
          {
              buffer_size = av_samples_get_buffer_size(NULL, aud_codec_context-&gt;channels, aud_codec_context-&gt;frame_size,
                  aud_codec_context-&gt;sample_fmt, 0);
          }
  
          if (av_sample_fmt_is_planar(sample_fmt))
              ret = av_frame_get_buffer(aud_frame, buffer_size / 2);
          else
              ret = av_frame_get_buffer(aud_frame, buffer_size);
  
          if (!aud_frame || ret &lt; 0)
              return ERROR_ALLOCATING_FRAME;
  
          aud_frame_counter = 0;
  
          return 0;
      }
  
      int initialize_audio_only_encoding(int sample_rate, int per_frame_audio_samples, int audio_bitrate, const char *filename)
      {
          int ret;
  
          avcodec_register_all();
          av_register_all();
  
          outctx = avformat_alloc_context();
  
          char* with_dot = concat(filename, ".");
          char* full_filename = concat(with_dot, compressed_cont);
  
          ret = avformat_alloc_output_context2(&amp;outctx, NULL, compressed_cont, full_filename);
  
          free(with_dot);
  
          if (ret &lt; 0)
          {
              free(full_filename);
              return ERROR_CONTEXT_CREATION;
          }
  
          ret = setup_audio_codec();
          if (ret &lt; 0)
              return ret;
  
          // Setup Audio
          ret = initialize_audio_stream(outctx, sample_rate, per_frame_audio_samples, audio_bitrate);
          if (ret &lt; 0)
              return ret;
  
          av_dump_format(outctx, 0, full_filename, 1);
  
          if (!(outctx-&gt;oformat-&gt;flags &amp; AVFMT_NOFILE))
          {
              if (avio_open(&amp;outctx-&gt;pb, full_filename, AVIO_FLAG_WRITE) &lt; 0)
              {
                  free(full_filename);
                  return ERROR_OPENING_FILE;
              }
          }
  
          free(full_filename);
  
          ret = avformat_write_header(outctx, NULL);
          if (ret &lt; 0)
              return ERROR_WRITING_HEADER;
  
          return 0;
      }
  
      int write_interleaved_audio_frame(float_t *aud_sample)
      {
          int ret;
  
          aud_frame-&gt;data[0] = (uint8_t*)aud_sample;
          aud_frame-&gt;extended_data[0] = (uint8_t*)aud_sample;
  
          aud_frame-&gt;pts = aud_frame_counter++;
  
          ret = avcodec_send_frame(aud_codec_context, aud_frame);
  
          AVPacket pkt;
          av_init_packet(&amp;pkt);
          pkt.data = NULL;
          pkt.size = 0;
  
          while (true)
          {
              ret = avcodec_receive_packet(aud_codec_context, &amp;pkt);
              if (!ret)
              {
                  av_packet_rescale_ts(&amp;pkt, aud_codec_context-&gt;time_base, audio_st-&gt;time_base);
  
                  pkt.stream_index = audio_st-&gt;index;
  
                  av_interleaved_write_frame(outctx, &amp;pkt);
  
                  av_packet_unref(&amp;pkt);
              }
              if (ret == AVERROR(EAGAIN))
                  break;
              else if (ret &lt; 0)
                  return ERROR_ENCODING_SAMPLES_RECEIVE;
              else
                  break;
          }
  
          return ENCODED_AUDIO;
      }
  
      int write_audio_frame(float_t *aud_sample)
      {
          int ret;
          aud_frame-&gt;data[0] = (uint8_t*)aud_sample;
          aud_frame-&gt;extended_data[0] = (uint8_t*)aud_sample;
  
          aud_frame-&gt;pts = aud_frame_counter++;
  
          ret = avcodec_send_frame(aud_codec_context, aud_frame);
          if (ret &lt; 0)
              return ERROR_ENCODING_FRAME_SEND;
  
          AVPacket pkt;
          av_init_packet(&amp;pkt);
          pkt.data = NULL;
          pkt.size = 0;
  
          fflush(stdout);
  
          while (true)
          {
              ret = avcodec_receive_packet(aud_codec_context, &amp;pkt);
              if (!ret)
                  if (pkt.pts != AV_NOPTS_VALUE)
                      pkt.pts = av_rescale_q(pkt.pts, aud_codec_context-&gt;time_base, audio_st-&gt;time_base);
              if (pkt.dts != AV_NOPTS_VALUE)
                  pkt.dts = av_rescale_q(pkt.dts, aud_codec_context-&gt;time_base, audio_st-&gt;time_base);
              {
  
                  av_write_frame(outctx, &amp;pkt);
                  av_packet_unref(&amp;pkt);
              }
              if (ret == AVERROR(EAGAIN))
                  break;
              else if (ret &lt; 0)
                  return ERROR_ENCODING_FRAME_RECEIVE;
              else
                  break;
          }
  
          return ENCODED_AUDIO;
      }
  
      int finish_audio_encoding()
      {
          AVPacket pkt;
          av_init_packet(&amp;pkt);
          pkt.data = NULL;
          pkt.size = 0;
  
          fflush(stdout);
  
          int ret = avcodec_send_frame(aud_codec_context, NULL);
          if (ret &lt; 0)
              return ERROR_ENCODING_FRAME_SEND;
  
          while (true)
          {
              ret = avcodec_receive_packet(aud_codec_context, &amp;pkt);
              if (!ret)
              {
                  if (pkt.pts != AV_NOPTS_VALUE)
                      pkt.pts = av_rescale_q(pkt.pts, aud_codec_context-&gt;time_base, audio_st-&gt;time_base);
                  if (pkt.dts != AV_NOPTS_VALUE)
                      pkt.dts = av_rescale_q(pkt.dts, aud_codec_context-&gt;time_base, audio_st-&gt;time_base);
  
                  av_write_frame(outctx, &amp;pkt);
                  av_packet_unref(&amp;pkt);
              }
              if (ret == -AVERROR(AVERROR_EOF))
                  break;
              else if (ret &lt; 0)
                  return ERROR_ENCODING_FRAME_RECEIVE;
          }
  
          av_write_trailer(outctx);
  
          return 0;
      }
  
      void cleanup()
      {
          if (aud_frame)
          {
              av_frame_free(&amp;aud_frame);
          }
          if (outctx)
          {
              for (int i = 0; i &lt; outctx-&gt;nb_streams; i++)
                  av_freep(&amp;outctx-&gt;streams[i]);
  
              avio_close(outctx-&gt;pb);
              av_free(outctx);
          }
  
          if (aud_codec_context)
          {
              avcodec_close(aud_codec_context);
              av_free(aud_codec_context);
          }
      }
  
      void fill_samples(float_t *dst, int nb_samples, int nb_channels, int sample_rate, float_t *t)
      {
          int i, j;
          float_t tincr = 1.0 / sample_rate;
          const float_t c = 2 * M_PI * 440.0;
  
          for (i = 0; i &lt; nb_samples; i++) {
              *dst = sin(c * *t);
              for (j = 1; j &lt; nb_channels; j++)
                  dst[j] = dst[0];
              dst += nb_channels;
              *t += tincr;
          }
      }
  
      int main()
      {
          int sec = 5;
          int frame_rate = 30;
          float t = 0, tincr = 0, tincr2 = 0;
  
          int src_samples_linesize;
          int src_nb_samples = 960;
          int src_channels = 2;
          int sample_rate = 48000;
  
          uint8_t **src_data = NULL;
  
          int ret;
  
          initialize_audio_only_encoding(48000, src_nb_samples, 192000, "sound_FLT_960");
  
          ret = av_samples_alloc_array_and_samples(&amp;src_data, &amp;src_samples_linesize, src_channels,
              src_nb_samples, AV_SAMPLE_FMT_FLT, 0);
  
          for (size_t i = 0; i &lt; frame_rate * sec; i++)
          {
                  fill_samples((float *)src_data[0], src_nb_samples, src_channels, sample_rate, &amp;t);
                  write_interleaved_audio_frame((float *)src_data[0]);
          }
  
          finish_audio_encoding();
  
          cleanup();
  
          return 0;
      }
  }
  </code></pre>
  
  <p>And some of the files:</p>
  
  <p>The webm audio file that does <strong>not</strong> work (only in VLC):
  <a href="https://drive.google.com/file/d/0B16rIXjPXJCqcU5HVllIYW1iODg/view?usp=sharing" rel="nofollow noreferrer">https://drive.google.com/file/d/0B16rIXjPXJCqcU5HVllIYW1iODg/view?usp=sharing</a></p>
  
  <p>The ogg audio file that works:
  <a href="https://drive.google.com/file/d/0B16rIXjPXJCqMUZhbW0tTDFjT1E/view?usp=sharing" rel="nofollow noreferrer">https://drive.google.com/file/d/0B16rIXjPXJCqMUZhbW0tTDFjT1E/view?usp=sharing</a></p>
  
  <p>Video and Audio file that only works in VLC: <a href="https://drive.google.com/file/d/0B16rIXjPXJCqX3pEN3B0QVlrekU/view?usp=sharing" rel="nofollow noreferrer">https://drive.google.com/file/d/0B16rIXjPXJCqX3pEN3B0QVlrekU/view?usp=sharing</a></p>
  
  <p>If a play the ogg file in FFPlay it says "aq= 30kb", but if I play the webm audio only file i get "aq= 0kb". So that does not seem right either.</p>
  
  <p>Any idea? Thanks in advance!</p>
  
  <p><strong>Edit:</strong> So I made it work by just encoding both VP8 and Opus into the ogg container and then simply renaming it to .webm and uploading it YouTube. I did not actually know ogg could have video inside of it. I do not really know what how it affects the encoding and stuff... I can upload the original ogg file with video and it also works on YouTube. But the whole reason I went for webm was the licensing it has (<a href="https://www.webmproject.org/license/" rel="nofollow noreferrer">https://www.webmproject.org/license/</a>)... So I'm a bit confused now.</p>
  
  <p>I need to read up on what exactly a "container" means in the context and what just changing the extension means.</p>
  
  <p>Any comments to shed some light on this appreciated!</p>