Shell command or series of commands similar to the Automator action "Get Link URLs from Webpages"
Here's a script to get you started:

#!/bin/bash

urls="/path/to/Input_URLs_List.txt"
output="/path/to/Output_Link_URLs.txt"

n="$(awk 'END {print NR}' "$urls")"
echo "Processing $n URLs..."

cat "$urls" | \
while read url; do
    lynx -dump -listonly "$url" &gt;&gt; "$output"
    sleep 5
    n=$((n-1))
    echo "$n URLs left to process..."
done


This will dump all of the links into a file that you can further process based on what you're looking for.  Additional code could be added to filter and process the output however without knowing what it is you need you'll have to work on it or ask some additional questions.



To clean up the output, use the following as an example:

Using "https://www.google.com" as one of the URLs the output would look like:

$ lynx -dump -listonly "https://www.google.com"


References

    1. https://www.google.com/imghp?hl=en&amp;tab=wi
    2. https://maps.google.com/maps?hl=en&amp;tab=wl
    3. https://play.google.com/?hl=en&amp;tab=w8
    4. https://www.youtube.com/?tab=w1


I've truncated the output, there's actually 19 Link URLs.

To have the output just be a list of URLs, no numbers or whitespace, etc., use awk either in conjunction with lynx or afterwards.

$ lynx -dump -listonly "https://www.google.com" | awk '/:/{print $2}'
https://www.google.com/imghp?hl=en&amp;tab=wi
https://maps.google.com/maps?hl=en&amp;tab=wl
https://play.google.com/?hl=en&amp;tab=w8
https://www.youtube.com/?tab=w1


So if you want the output file to be just the Link URLs, change the lynx command line to:

lynx -dump -listonly "$url" | awk '/:/{print $2}' &gt;&gt; "$output"


You can always process the contents of the output file later in the script or afterwards to get it down to the really wanted Link URLs and use a different search parameter in awk, e.g., I used ":" to both eliminate the blank lines in the lynx output and to show an example of how it can be filtered.  In this example only Link URLs get redirected into the output file because only lines containing a : get output by awk, as all URLs should have a colon in them. The {print $2}, simplified in this explanation, removes everything to the left of the actual Link URL. 



Here's an updated script which sorts and removes duplicate Link URLs:

#!/bin/bash

urls="/path/to/Input_URLs_List.txt"
output="/path/to/Output_Link_URLs.txt"

n="$(awk 'END {print NR}' "$urls")"
echo "Processing $n URLs..."

cat "$urls" | \
while read url; do
    lynx -dump -listonly "$url" | awk '/:/{print $2}' | sort | uniq &gt;&gt; "$output"
    sleep 5
    n=$((n-1))
    [[ $n -gt 0 ]] &amp;&amp;  echo "$n URLs left to process..." || echo "Processing Completed!"
done




Update to capture stderr output from lynx to a file:

To capture stderr output from lynx to a file, redirect stderr to a disk file, e.g., 2&gt;&gt;"$file" added after "$url", e.g.:

lynx -dump -listonly "$url" 2&gt;&gt;"$file" &gt;&gt; "$output"


Add errlog="/path/to/Lynx_Errors.txt" under output="/path/to/Output_Link_URLs.txt" and then change the lynx command line to, e.g.:

lynx -dump -listonly "$url" 2&gt;&gt;"$errlog" &gt;&gt; "$output"


Or:

lynx -dump -listonly "$url" 2&gt;&gt;"$errlog" | awk '/:/{print $2}' | sort | uniq &gt;&gt; "$output"


Example:

#!/bin/bash

urls="/path/to/Input_URLs_List.txt"
output="/path/to/Output_Link_URLs.txt"
errlog="/path/to/Lynx_Errors.txt"

n="$(awk 'END {print NR}' "$urls")"
echo "Processing $n URLs..."

cat "$urls" | \
while read url; do
    lynx -dump -listonly "$url" 2&gt;&gt;"$errlog" | awk '/:/{print $2}' | sort | uniq &gt;&gt; "$output"
    sleep 5
    n=$((n-1))
    [[ $n -gt 0 ]] &amp;&amp;  echo "$n URLs left to process..." || echo "Processing Completed!"
done