Why is 于 needed after 不屑 (but sometimes not)?
不屑于 is a set phrase.
Check out《现代汉语规范词典》's distinctions:

不屑
1 动 不值得
不屑一切 | 不屑与之争辩
2 形 形容轻视的样子
一副不屑的神情。

and

不屑于
动 认为不值得做或不值得理睬
他很自负, 好像不屑于做这种具体工作。

《规范》defines the two slightly differently while treating 不屑于 as a separate independent word and not a construct.