What can I replace an 8-spline handle with?
If the antique handles are not prohibitively expensive and you don't want to update, you may need to accept they are the correct part and use them.  We don't take shopping questions, so I haven't attempted an exhaustive search for you, but I would note that some sites have search tools based on stem shape or number of splines and you should attempt to check out at the least the major manufacturers in your area and the brand of your parts if you can identify it, possibly including a search of whether that company was absorbed into another.
Be thorough in your searching.  I recently had to identify a very cheap (sub $100) 10-15 year old shower set to find a replacement valve, and found that the manufacturer's website is designed with a front end to look like a tool to help you find replacement parts for existing product, but for products old enough to require replacement parts, they just redirect you to look at the surface fixtures (handles, escutcheon, spout, etc. of their high end ($500-$1200) products from years too recent to require replacement parts.  I found the correct, common, cheap replacement valve by identifying the valve assembly part number by manual visual comparison of replacement parts on google images.  Once I knew the relevant part numbers, finding the referenced valve was easy, but it was obvious the manufacturer would prefer replacement of the entire assembly, preferably with an expensive model.
That said, you could indeed use a dremel to grind a flat spot or wedge on the existing stem that would fit well with a suitably sized set screw handle.  It is even likely that if you did this carefully it would remain compatible with 8 spline handles.
Goal is to produce a keyed mechanical shape that holds the handle in place based on the position of the set screw rather than the force it is done up with.
Oh and if your issue with the antique parts is the way they look and you're looking at saving hundreds of dollars by not tearing open the wall and replacing the valve assembly, you could even contact some local machinists and see what they'd charge.  Prototyping is expensive, but not that expensive, especially if you can provide the design.  You haven't provided any example pictures, but if you have artistic skills and tools you could also try your hand at making one from a suitable mating piece and a variety of well documented hobby manufacture techniques.