Stereoelectronic effects in torquoselectivity of 4π electrocyclic opening
The torquoselectivity of the above reactions is dictated by stereoelectronic effects, which (in this case) are significant enough to take precedence over steric effects. In general, electron-donating substituents (e.g. OMe, OTMS, ...) prefer to move outwards, and electron-accepting substituents (e.g. CHO) prefer to move inwards.

These can be understood by considering the frontier orbitals in the transition state of the conrotatory opening, as explained by the Houk group in earlier publications1,2 (the reactions given above were, in fact, carried out in order to provide experimental verification of their theoretical predictions). A more accurate orbital correlation diagram for the reaction is given below (modified from ref 2 to include diagrams of the relevant MOs):



A brief explanation is as follows. As the reaction proceeds, the C–C σ bond in the reactant is stretched, and consequently the σ* orbital is initially strongly destabilised (its energy increases rapidly). At a certain point near the transition state, though, it becomes sufficiently close in energy to the π* orbital such that orbital mixing can occur; this mixing leads to stabilisation and eventual "evolution" into the HOMO of butadiene.

Therefore, in the transition state, the HOMO is mostly σ-like, and the LUMO σ*-like; this was also verified computationally (details in ref 2). These frontier orbitals are of the correct symmetry to interact with pi-type orbitals of substituents. For donor substituents such as OMe, these orbitals are filled (lone pair on oxygen); for acceptor substituents such as CHO, these orbitals are empty (low-lying C=O π*).

The dashed orbitals in the following diagram represent the positions of these orbitals in the TS for both cases of outward and inward rotation. In the case of outward rotation, the substituent orbital can only interact with the lobe on C–3 which is in close spatial proximity; however, in the case of inward rotation, the substituent orbital can interact with the lobes on both C–3 and C–4.



In general, the interaction of a filled orbital and an unfilled orbital leads to stabilisation, whereas the interaction of two filled orbitals leads to destabilisation. The torquoselectivity is determined by maximisation of the former and minimisation of the latter. If the interaction with the lobe of C–4 leads to destabilisation, then inward rotation will be disfavoured, and vice versa.

For a donor substituent:


Inward rotation decreases the stabilising interaction between the donor orbital and the LUMO of the TS, as the lobe on C–4 is of the opposite phase as the lobe on the donor orbital, which leads to poorer overlap between the two orbitals.
Inward rotation increases the destabilising interaction between the donor orbital and the HOMO of the TS, as the lobe on C–4 is of the same phase as the lobe on the donor orbital, which leads to stronger overlap.




Based on these reasons, donor substituents generally prefer to rotate outwards. In the case of OH, this preference was calculated to be a whopping 16.4 kcal/mol.3

For acceptor substituents, the interaction with the LUMO is not relevant as the interaction of two unfilled orbitals does not lead to any stabilisation. However, inward rotation maximises the stabilising interaction with the HOMO, since this is enhanced by overlap with the lobe on C–4. Hence, acceptor substituents generally prefer to rotate inwards. This preference, however, is markedly smaller than in the case of donor substituents - perhaps because there is only one factor that biases the course of the reaction (as opposed to two in the case of donor substituents). Experimentally there are fewer examples of torquoselectivity with electron-accepting substituents, and for the second reaction above, the preference for the inward rotation of the formyl group was calculated to be a much more modest 4.5 kcal/mol.4

Interestingly enough, for the cyano group (-CN), a preference of 4.3 kcal/mol was calculated - for outward rotation.4 It turns out that the donor π orbitals are more important than the acceptor π* orbitals, which are relatively high in energy. The formyl group only rotates inwards because it has an exceptionally low-lying π* orbital!



References

1. Kirmse, W.; Rondan, N. G.; Houk, K. N. Stereoselective substituent effects on conrotatory electrocyclic reactions of cyclobutenes. J. Am. Chem. Soc. 1984, 106 (25), 7989–7991. DOI: 10.1021/ja00337a067.

2. Rondan, N. G.; Houk, K. N. Theory of stereoselection in conrotatory electrocyclic reactions of substituted cyclobutenes. J. Am. Chem. Soc. 1985, 107 (7), 2099–2111. DOI: 10.1021/ja00293a046.

3. Houk, K. N.; Spellmeyer, D. C.; Jefford, C. W.; Rimbault, C. G.; Wang, Y.; Miller, R. D. Electronic control of the stereoselectivities of electrocyclic reactions of cyclobutenes against incredible steric odds. J. Org. Chem. 1988, 53 (9), 2125–2127. DOI: 10.1021/jo00244a058.

4. Rudolf, K.; Spellmeyer, D. C.; Houk, K. N. Prediction and experimental verification of the stereoselective electrocyclization of 3-formylcyclobutene. J. Org. Chem. 1987, 52 (16), 3708–3710. DOI: 10.1021/jo00392a047.