What are the rules for nested sentences?
Sentence as S
There may not be such a thing. In English you might want to say That I came out of the closet surprised them all. While in Chinese you can have

我出柜惊了他们所有人。

Which is normally interpreted as I came out of the closet (and) surprised them all. Not quite the same.
Sentence as O
Plenty.

你知道我出柜了。 You know I've come out of the closet.

And yes, you can have many layers:

你知道他们听说我出柜了。 You know they've heard that I came out of the closet.

But use these in moderation. Long sentences are much more frowned upon in Chinese than they are in English. Keep the nested sentences short.
Sentence as t and p
Not sure if this is what you want, but here you go:

我在他睡觉时出柜了。 I (while he is sleeping) came out of the closet.
我在他睡觉的地方出柜了。 I (at the place where he slept) came out of the closet.

But let's just talk about verb modifiers in general, OK?

我以迅雷不及掩耳之势出柜了。 I (as fast as a lightning strike at close, and you don't even have time to cover your ears) came out of the closet.

It is hell of a long phrase in English. Normally you would consider this unidiomatic in Chinese, but here we have only six (or nine, depends on how you count) Chinese characters and it is an idiom from classical Chinese. so the sentence come out just fine.
Sentence as V
I put this at last because I've run out of closet related examples.
There are some idioms that look like sentences but can function as verbs:

你别王婆卖瓜了。

This directly translates to You stop old-lady-Wang-sells-watermelon-ing, which is to say stop bragging about your goods.
But generally it is very rare. If you create one yourself instead of using idioms, other people may have a hard time understanding.
How to be idiomatic
The English language has this great right branching ability; clauses or nested sentences are like its second nature. The Chinese language can do a bit of right branching as well, but it is not very good at it. So in most cases if you have an English sentence that has a clause, you want to change its style while you are translating.
For example, you say

I have a bird that sings beautifully.

But in Chinese you want to say

我有一只唱歌很好听的鸟。

Notice how you get rid of the clause and make it a simple noun modifier and stick it to the left of the object with “的”. Or you can say

我有一只鸟，（它）唱歌很好听。

Again, you get rid of the clause by making it to two sentences.