Does food at room temperature spoil faster if refridgerated beforehand?
The short answer is no, that is a myth. Food spoils because of chemical breakdown and microorganism activity, both of these are slowed considerably by refrigeration. When you take food out of the refrigerator and it warms up these chemical and microbial processes speed up again, but they don't work faster because the food was previously refrigerated.
Some foods shouldn't be refrigerated, like bananas and avocados, but that's a different question.