What do I need to know about buying a VCR for digitising old VHS tapes?
so now I'm wondering if my tapes are marked as PAL SECAM, do I need a PAL VCR?

Yes, VCRs partially deconstruct the signal to encode it on the tape, and so are specific to a particular analogue video format.
In practical terms, if you try to play a PAL tape in an NTSC VCR connected to a PAL TV, you'll probably get a distorted black-and-white image, or perhaps no usable signal at all.

I've seen a few local listings on eBay for NTSC VCRs, and confusingly I even saw one that said &quot;NTSC Playback on PAL TV&quot;

It was very common to import North American tapes back in the day.  These VCRs can convert an NTSC format tape to PAL output for European TV sets.  The reverse also existed (though quite a bit rarer) in North America.

I see some are listed as 4-head, 6-head, Super Drive, etc.? Do any of these matter for my purposes?

4 head machines have a slight advantage in quality, particularly with playing ELP/LP tapes.  The extra 2 heads in 6 head machines are only for recording.  And &quot;Super Drive&quot; just appears to be a Panasonic marketing term, as far as I can tell.

Is this true?

Sort of.  VCRs are temperamental machines, and it's all analogue.  During their heyday, the quality of video you'd get from units varied enormously.  A quality S-VHS machine recording to quality tape on a well-calibrated machine was almost as nice as 480p digital is, really.  But I think we're just about old enough to remember what VHS usually looked like.

Will I need to shell out for a more expensive S-VHS VCR to avoid sacrificing quality?

This would only really be necessary if the original footage was recorded as S-VHS.  If so, then yes.  Some later VHS machines can play S-VHS tapes with reduced quality.  Still, play S-VHS tapes on S-VHS machines.
S-VHS equipment was high-end, and so generally better quality than the cheap VCRs that proliferated in the 90s and 00s.  So you will often find a preference for S-VHS machines.  But a good quality S-VHS and VHS deck should play a VHS tape back equivalently well.
Practically speaking, for high quality, you'd want to play back on a good deck that has been calibrated and had its heads cleaned and belts replaced, etc.  That would mean a machine that's been maintained or refurbished, not something that's been sitting on a damp shelf for 20 years.
With all that said, a few years ago I transferred a relative's home movies with a cheap 20 year old VCR and it's about what one would expect for 1980s camcorder footage.  I fed it The Great Mouse Detective first though, in case it might have been a tape-eater.