Should I switch the transfomer input or output?
I would aim to switch the high voltage. A 500 VA transformer will waste power even when unloaded.

No-load losses are caused by the magnetizing current needed to energize the core of the transformer, and do not vary according to the loading on the transformer. They are constant and occur 24 hours a day, 365 days a year, regardless of the load, hence the term no-load losses. They can be categorized into five components: hysteresis losses in the core laminations, eddy current losses in the core laminations, I 2R losses due to no-load current, stray eddy current losses in core clamps, bolts and other core components, and dielectric losses. Hysteresis losses and eddy current losses contribute over 99% of the no-load losses, while stray eddy current, dielectric losses, and I 2R losses due to no-load current are small and consequently often neglected. Thinner lamination of the core steel reduces eddy current losses.
The biggest contributor to no-load losses is hysteresis losses. Hysteresis losses come from the molecules in the core laminations resisting being magnetized and demagnetized by the alternating magnetic field. This resistance by the molecules causes friction that results in heat. The Greek word, hysteresis, means &quot;to lag&quot; and refers to the fact that the magnetic flux lags behind the magnetic force. Choice of size and type of core material reduces hysteresis losses.
Source: Copper.org.

For example, if the 500 VA transformer no-load losses are 5% (and I have no idea if that's reasonable) then continuous power waste is 25 W. There are 8760 hours in a year so energy loss is 8760 h x 0.025 kW = 219 kWh*. You can save energy in all the off hours by switching the primary.
Do the right thing!
Reference:

Transformer Losses and Efficiency.
Carroll-Meynell.

* To generate 219 kWh of electrical energy with a 40% efficient oil or coal-powered generating station would use about 50 kg of fuel (based on 10 kWh per kilo of oil).