Revised: Counting subnumbers divisible by a given prime
It is a bit weird to have two special cases and start with the case that it is neither of those. I would re-arrange your cases to
if p == 2:
    ...
elif p == 5:
    ...
else:
    ...

Unless you did this as a performance improvement, because a random prime is much more likely to be neither 2 nor 5, but in that case you should have some measurements to show that this is not a premature optimization and probably add a comment in the code.

If you iterate over something and need both the index and the value, use enumerate:
if p == 2:
    ans = sum(i for i, digit in enumerate(number, 1)
              if digit in {'0', '2', '4', '6', '8'})
elif p == 5:
    ans = sum(i for i, digit in enumerate(number, 1)
              if digit in {'0', '5'})

Note that I used the second (optional) argument of enumerate to start counting at one and I used a set instead of a tuple, because it has faster membership test (not a lot faster for small collections as this, but measurably faster). I also put it all into a generator expression and used sum.

Python has an official style-guide, PEP8, which recommends using lower_case for variables, so your counter should be called c, or better yet, something readable like digit_count. You can also factor out the 0 case to avoid having to check for it up to nine times:
else:
    slices = (int(number[i:]) % p for i in range(len(number)))
    digit_count = Counter(slices)

    # get the count of zeroes, with a default value of zero
    zeroes = digit_count.pop(0, 0)  
    ans = (zeroes * (zeroes + 1)) // 2    
    ans += sum((count * (count - 1)) // 2
               for count in digit_counts.values())