Modified A-Z grouping
When looping over the letters in the range, you should be able to use the keys filter to check if the current letter is in the entriesByGroup array. If it is, you can link it or add a class, etc.
{% set letterArray = 'A'..'Z' %}
{% set entries = craft.entries.section('myEntries').orderBy('title asc').all() %}
{% set entriesByGroup = entries|group('title[:1]') %}

{% for letter in letterArray %}
   {% if letter in entriesByGroup|keys %}
      {# this letter has at least one entry, anchor link, add a class #}
   {% endif %}
{% endfor %}
 
{% for firstLetter, entriesInGroup in entriesByGroup %}
   {# your alphabetical entry listing %}
{% endfor %}

I do something similar here on the CraftQuest Archive page but with a slightly different approach (here's a video on it).