Is there a neutral way of saying "deliberately"?
'故意' implying someone intend to do some bad things. If you want to say someone intend to do things specially. You can use '特意'. This word do not imply any bad or good intention. It only emphasize 'specially'. But in most of case, Chinese love to use it in positive senarios, like me.

为了让学生听得明白，我们老师特意把语速放慢。
在下班的路上，他特意绕路去医院看望母亲。