When does protein folding begin?
When does protein folding begin?

With reference to time you have asked, it can be after the translation has occurred (called Translational protein folding) or while translation is still occuring (called Co-Translational protein folding). Here is the link to an article for basic understanding of co-translational protein folding.

There is a lot of debate on the true time. Experimentalists have amassed extensive evidence over the past four decades that proteins appear to fold during production by the ribosome. Protein structure prediction methods, however, do not incorporate this property of folding (reference). 

If you are looking at particularly some organisms it is advised to refer to the research articles related to them.The primary mechanism is the interplay between several non-covalent interactions such as hydrophobic effect and hydrogen bonding in a cooperative manner (they also follow basic rules of thermodynamics) which leads to a marginally stable tertiary structure.

When do secondary and tertiary structures begin forming?

Ofcourse, after Primary structure forms.The alpha helices/beta sheets then fold in a cooperative manner to form a tertiary structure.

One point to note is that, slight variations in these cooperative act causes misfoldingng of proteins which may cause certain diseases.