Stepping-down DC voltage (at low current) in the presence of an audio circuit
Yes, a linear regulator won't be the most efficient for this application. It will, however, be the least noisy, especially if you select a regulator designed for low noise such as the LP2985-50. And at 100mA there will only be 700mW loss, which may be considered acceptable for the application.