Are counters (noun) physical things?
Yes, counters should be represented by physical objects.


  122.1. A counter is a marker placed on an object or player that modifies its characteristics and/or interacts with a rule, ability, or effect. [...]


There has been official products for use as counters. Punch cards including -1/-1 counters and brick counters were provided alongside the Amonkhet block (Apr 2017). To help support the keyword counters it introduced, the Ikoria set (Apr 2020) provided punch cards with +1/+1 counters and keyword counters.



You aren't required to use these. The objects used as counter can be anything. Dice and flat-bottomed glass beads meant to be used in flower vases are popular. You can get the latter at Michaels or Jo-Ann, among others. Some gaming stores carry them as well.



I've seen people use pennies, but they will scratch your cards if the cards aren't protected. (Anything could, but beads and dice have rounder edges.)