Are phrases of the form 我[something]去 informal or slang (or just plain incorrect)?
It's a correct usage of 我...去. It's usually used when asked "你干什么去？" 

It would be like:

A: "你干什么去？"

B: "我跑步去"；“我上厕所去”；“我睡觉去”；“我换衣服去”；“我做礼拜去” （more clear than 我礼拜去）

When asked with "你（要）去干什么？"， one usually uses 我（要）去...

It would be like:

A: "你（要）去干什么？"

B: "我（要）去跑步"；“我（要）去上厕所”；“我（要）去睡觉”；“我（要）去换衣服”；“我（要）去做礼拜”

Those colloquial terms are commonly used in daily dialogues.