Does Xの時 need a に particle to indicate when something happened?


I know that absolute time references such as 九時 have に after them to indicate
when something happened, and relative time references and parts of the day do
not.

But what about "time references" like 子供の時 (childhood) or 若い時 (lit: young
time), do they have に after them to indicate when something happened?

Ex: 若い時に運動しました vs 若い時、運動しました (I exercised when I was young)

