title: Youtube-dl download script debug
body: <p>Dumping very bad idea of updating this post. New place for this script:
  <a href="https://gist.github.com/Wogol/66e9936b6d49cc5fecca59eaeca1ca2e" rel="nofollow noreferrer">https://gist.github.com/Wogol/66e9936b6d49cc5fecca59eaeca1ca2e</a></p>
  <hr />
  <p>Im trying to create a .command macOS script (Should also work under GNU/Linux) that use Youtube-dl that is simple to use. I have fixed so it downloads description, thumbnail, subtitles, json, creates folder structure and also saves video ID, uploader and upload date.</p>
  <hr />
  <p><strong>ISSUES WITH THE MAIN SCRIPT:</strong></p>
  <ul>
  <li><p><strong>FIXED (13th august)</strong> Problem I struggle with is the option of Audio &amp; Video or Audio only. For some reason only audio works in the script. The download script for the video&amp;audio dont work but if I paste that same command line (&quot;The DEBUG output line&quot;) in a terminal window it works. Scratching my head.
  Youtube-dl gives me this message:
  ERROR: requested format not available</p>
  </li>
  <li><p><strong>FIXED (31th august)</strong> Get max resolution of video working. Have found information to force mp4 or max resolution but not combined them.</p>
  </li>
  </ul>
  <hr />
  <p><strong>ISSUES WITH INFORMATION FILE:</strong></p>
  <p>Also creating a info file with title, channel name, release date, description. Im now struggling with getting video information from .json and youtube-dl to be exported into the info.txt file.</p>
  <ul>
  <li><strong>FIXED (5th september)</strong> textfile=&quot;&quot;$folder&quot;info.txt&quot; not working. Gives this error: (There I want to add youtube-dl folder.</li>
  </ul>
  <pre><code>    ytdl.command: line 104: ~/Downloads/ytdl/dog_vids/info.txt: No such file or directory
  </code></pre>
  <ul>
  <li><strong>FIXED (5th september)</strong> Find youtube-dl folder and get it to work with grep.
  Something like this:</li>
  </ul>
  <pre><code>   youtube-dl --simulate --SHOW_THE_OUTPUT_PATH -o $folder'/%(title)s/%(title)s - (%(id)s) - %(uploader)s - %(upload_date)s.%(ext)s' https://www.youtube.com/watch?v=dQw4w9WgXcQ
  </code></pre>
  <ul>
  <li><p><strong>FIXED (5th september)</strong> With grep command i named the json file &quot;*.json&quot; because there will only be one per directory but I dont like that solution. (Could be answered with point above)</p>
  </li>
  <li><p><strong>FIXED (5th september)</strong> How to make so grep dont grab &quot;? It now adds them before and after everything.</p>
  </li>
  <li><p><strong>FIXED (5th september)</strong> How to get the tags information from json file? Tags look like this:</p>
  </li>
  </ul>
  <pre><code>    &quot;tags&quot;: [&quot;music&quot;, &quot;video&quot;, &quot;classic&quot;]
  </code></pre>
  <ul>
  <li><strong>FIXED (5th september)</strong> Run the creation of info file part of the script in the background of downloading the video?</li>
  </ul>
  <hr />
  <p><em><strong>CURRENT VERSION TRYING TO GET IT WORKING</strong></em></p>
  <p>(12 august)</p>
  <pre><code>    textfile=&quot;&quot;$folder&quot;info.txt&quot;
      
      echo TITLE &gt;&gt; ~/Downloads/ytdl/dog_vids/info.txt
      youtube-dl -e $url &gt;&gt; ~/Downloads/ytdl/dog_vids/info.txt
      echo \ &gt;&gt; ~/Downloads/ytdl/dog_vids/info.txt
      
      echo CHANNEL &gt;&gt; $textfile
      echo \ &gt;&gt; $textfile
      
      echo CHANNEL URL &gt;&gt; $textfile
      echo \ &gt;&gt; $textfile
      
      echo UPLOAD DATE &gt;&gt; $textfile
      echo \ &gt;&gt; $textfile
      
      echo URL &gt;&gt; $textfile
      echo $url &gt;&gt; $textfile
      echo \ &gt;&gt; $textfile
      
      echo TAGS &gt;&gt; $textfile
      echo \ &gt;&gt; $textfile
      
      echo DESCRIPTION &gt;&gt; $textfile
      youtube-dl --get-description $url &gt;&gt; $textfile
  </code></pre>
  <hr />
  <p><em><strong>EXPERIMENT FUTURE VERSION - EXTRACTING INFORMATION FROM JSON FILE</strong></em></p>
  <p>This isnt a working script. Showing how I want it with $textfile, $ytdlfolder and $jsonfile.</p>
  <pre><code>    url=https://www.youtube.com/watch?v=dQw4w9WgXcQ
      
      textfile=&quot;&quot;$folder&quot;&quot;$YOUTUBE-DL_PATH&quot;info.txt&quot;
      ytdlfolder=&quot;$folder&quot;&quot;$YOUTUBE-DL_PATH&quot;
      jsonfile=&quot;$folder&quot;&quot;$YOUTUBE-DL_JSON-FILE&quot;
  
      Echo TITLE &gt;&gt; $textfile
      grep -o '&quot;title&quot;: *&quot;[^&quot;]*&quot;' $jsonfile | grep -o '&quot;[^&quot;]*&quot;$' &gt;&gt; $textfile
      Echo \ &gt;&gt; $textfile
      
      Echo CHANNEL &gt;&gt; $textfile
      grep -o '&quot;uploader&quot;: *&quot;[^&quot;]*&quot;' $jsonfile | grep -o '&quot;[^&quot;]*&quot;$' &gt;&gt; $textfile
      Echo \ &gt;&gt; $textfile
      
      Echo CHANNEL URL &gt;&gt; $textfile
      grep -o '&quot;uploader_url&quot;: *&quot;[^&quot;]*&quot;' *.json | grep -o '&quot;[^&quot;]*&quot;$' &gt;&gt; $textfile
      Echo \ &gt;&gt; $textfile
      
      Echo UPLOAD DATE &gt;&gt; $textfile
      grep -o '&quot;upload_date&quot;: *&quot;[^&quot;]*&quot;' *.json | grep -o '&quot;[^&quot;]*&quot;$' &gt;&gt; $textfile
      Echo \ &gt;&gt; $textfile
      
      Echo TAGS &gt;&gt; $textfile
      grep -o '&quot;tags&quot;: *&quot;[^&quot;]*&quot;' *.json | grep -o '&quot;[^&quot;]*&quot;$' &gt;&gt; $textfile
      Echo \ &gt;&gt; $textfile
      
      echo URL &gt;&gt; $textfile
      echo $url &gt;&gt; $textfile
      echo \ &gt;&gt; $textfile
      
      Echo DESCRIPTION &gt;&gt; $textfile
      youtube-dl --get-description $url &gt;&gt; $textfile
  </code></pre>
  <hr />
  <p><em><strong>THE SCRIPT:</strong></em></p>
  <p>12 august.</p>
  <ul>
  <li>Moved url to the top so when user paste the url they get the videos title back. This so the user know they got the right video.</li>
  <li>Added max resolution 1920x1080. (Do not work)</li>
  </ul>
  <p>13 august.</p>
  <ul>
  <li>Downloading Audio &amp; Video works.</li>
  </ul>
  <p>31 august.</p>
  <ul>
  <li>Fixed force mp4 and max heigh of 1080.</li>
  </ul>
  <p>5 september.</p>
  <ul>
  <li>Finally working script. Read more about it here (Or scroll down):
  <a href="https://stackoverflow.com/questions/63336809/youtube-dl-download-script-debug/63753752#63753752">Youtube-dl download script debug</a></li>
  </ul>
  <p>2020-09-17</p>
  <ul>
  <li>Folders can now have spaces in them.</li>
  </ul>
  <p><strong>2020-09-22</strong></p>
  <ul>
  <li>Select menus is now one column.</li>
  <li>Minor fixes.</li>
  <li>Now all the bugs is fixed. Issues left is only optimizations.</li>
  </ul>
  <pre><code>    #! /bin/bash
      ################################################################################
      # Script Name: Youtube-dl Easy Download Script
      # Description: Easy to use script to download YouTube videos with a couple of
      #              options.
      #
      # What this script do:
      #   - Downloads video in MP4 with highest quality and max resolution 1920x1080.
      #   - Downloads thumbnail and subtitles.
      #   - Gives user option where to download the video and video or only audio.
      #   - Creates a folder with same name as video title and puts all files there.
      #   - Creates a .txt file with information about the video.
      #
      #
      # Author:      Wogol - Stackoverflow.com, Github.com
      # License:     The GNU General Public License v3.0 - GNU GPL-3
      #
      # 
      # Big thanks to the people at youtube-dl GitHub and Stack Overflow. Without
      # their help this would never ever been possible for me.
      #
      # Special thanks to:
      #              Reino @ Stack Overflow
      #
      # #####
      #
      # Software required:                 youtube-dl, xidel, printf
      #
      # macOS:       1. Install Homebrew:  https://brew.sh
      #              2. Terminal command:  brew install youtube-dl xidel
      #
      # Linux:       Depends on package manager your distribution use.
      #
      # #####
      #
      # Version history:
      # 2020-09-22
      #   - Select menus is now one column.
      #   - Minor fixes.
      #   - Now all the bugs is fixed. Issues left is only optimizations.
      #
      # 2020-09-17
      #   - Folders can now have spaces in them.
      #
      # 2020-09-05
      #   - First working version.
      #
      # #####
      #
      # Issues left:
      #   - In the beginning there is a confirmation that show the title of the
      #     video so user know they got the correct video. It takes youtube-dl a
      #     couple of seconds. To speed up the script it is DISABLED by default.
      #
      #   - Have found out that the script dont need xidel to get json information
      #     but youtube-dl can get it. Dont know how to use youtube-dl --dump-json
      #     to get the same result.
      #
      #   - To get the path to the .txt file script use youtube-dl. This gives the
      #     script a pause for a few seconds. Best would get to get the path some how
      #     without connecting to YouTube again but use the output from youtube-dl
      #     some how. ... or run it in the background when video is downloading.
      #
      ################################################################################
  
  
  
      clear
  
  
      
      # - WELCOME MESSAGE -
  
      echo
  
      COLUMNS=$(tput cols)
      title=&quot;-= Youtube-dl Easy Download Script =-&quot; 
      printf &quot;%*s\n&quot; $(((${#title}+$COLUMNS)/2)) &quot;$title&quot;
  
  
  
      # - PASTE URL -
  
      echo -e &quot;\n*** - Paste URL address and hit RETURN. Example:\nhttps://www.youtube.com/watch?v=dQw4w9WgXcQ --OR-- https://youtu.be/dQw4w9WgXcQ\n&quot;
  
      read url
  
  
  
      # - VIDEO TITLE -
  
      # So users know they have the correct URL.
  
      #echo -e &quot;\nThe video is: (This takes 3-4 seconds, or more ...)&quot;
      #youtube-dl -e $url
      #echo
  
  
  
      # - DOWNLOAD LOCATION -
  
      
      # DIRECTORY MUST END WITH SLASH: /
  
  
      echo -e &quot;\n\n*** - Choose download folder:\n&quot;
      
      COLUMNS=0
      PS3='Choose: '
      select directory in &quot;~/Downloads/ytdl/Rick Astley/&quot; &quot;~/Downloads/ytdl/Never Gonna Give You Up/&quot; &quot;~/Downloads/ytdl/Other Rick Videos/&quot; ; do
      echo -e &quot;\nOption $REPLY selected. Download directory is:\n $directory&quot;
  
  
  
      # - AUDIO/VIDEO SETTINGS -
  
      echo -e &quot;\n\n*** - Choose download settings:\n&quot;
  
      COLUMNS=0
      PS3='Choose: '
      options=(&quot;Audio &amp; Video&quot; &quot;Audio only&quot;)
      select settingsopt in &quot;${options[@]}&quot;
          do
      case $settingsopt in
  
      &quot;Audio &amp; Video&quot;)
      av=&quot;-f bestvideo[ext=mp4][height&lt;=1080]+bestaudio[ext=m4a]/best[ext=mp4]/best --merge-output-format mp4&quot;
  
      ;;
  
      &quot;Audio only&quot;)
      av=&quot;-f bestaudio[ext=m4a]/bestaudio&quot;
  
      ;;
  
      esac
  
      echo -e &quot;\nOption $REPLY selected:\n $settingsopt&quot;
  
  
  
      # - THE DOWNLOAD SCRIPT -
  
      echo -e &quot;\n\n*** - Starting download:\n&quot;
      youtube-dl $av --write-thumbnail --all-subs --restrict-filenames -o &quot;$directory%(title)s/%(title)s.%(ext)s&quot; $url
  
  
  
      # - INFORMATION FILE -
  
      textfile=$(youtube-dl --get-filename --restrict-filenames -o &quot;$directory%(title)s/%(title)s.txt&quot; $url)
  
      xidel -s &quot;$url&quot; -e '
      let $json:=json(
          //script/extract(.,&quot;ytplayer.config = (.+?\});&quot;,1)[.]
        )/args,
        $a:=json($json/player_response)/videoDetails,
        $b:=json($json/player_response)/microformat
      return (
      &quot;- TITLE -&quot;,
      $a/title,&quot;&quot;,
      &quot;- CHANNEL -&quot;,
      $a/author,&quot;&quot;,
      &quot;- CHANNEL URL -&quot;,
      $b//ownerProfileUrl,&quot;&quot;,
      &quot;- UPLOAD DATE -&quot;,
      $b//publishDate,&quot;&quot;,
      &quot;- URL -&quot;,
      $json/loaderUrl,&quot;&quot;,
      &quot;- TAGS -&quot;,
      $a/keywords,&quot;&quot;,
      &quot;- DESCRIPTION -&quot;,
      $a/shortDescription
      )
      ' --printed-json-format=compact &gt;&gt; &quot;$textfile&quot;
  
  
  
      # - THE END -
      
      echo
      COLUMNS=$(tput cols) 
      ending=&quot;Download Complete!&quot;
      printf &quot;%*s\n\n&quot; $(((${#ending}+$COLUMNS)/2)) &quot;$ending&quot;
      
      exit
      
      done
      done
  </code></pre>