Establishing ground truth in a particle filter experiment using compass and gyro
The easy answer is  to use a surveyed direction.

A map, your position, and a landmark out a window, a rifle scope and there you go.

Worst case you need to get a couple or three points surveyed.

In geodesy, setting up observatories, testing ranging gear, all the same problems.

Then you use the correct magnetic variation for your location. ( you'll end up knowing your countries geodesy service pretty well by the end of this).

If you just need some known points to calibrate from, then...
It's relatively simple to get 0, 90, 180 and 270 degrees from that with cheap hardware.
A rotation stage, even an industrial one for a milling machine will give almost any given angle.
(Magnetic compensation for metal equipment may be a pain. Just a plastic 360 degree protractor and some wood will do for all but very fine work.) [I have seen it used in compass calibration before.... if you need more accuracy, make a bigger protractor on a huge bit of cardboard.]

We used to used computer-driven rotation stages to calibrate gravity sensors.
The people we bought IMU's from used a robot arm to test the IMU.

For compasses, a cardboard compass rose and wooden/plastic pivots.

If you need very fine variation, I pity you. Maybe use a coil set.