Why do I sometimes “cream” and “beat until smooth”, and sometimes not?
For the same reason that sauce recipes will sometimes direct you to cut the tomatoes, sometimes to puree them, sometimes to deseed them, etc. You are not only making a cake every time, you are making different kinds of cake, and for each kind, there is a method which will produce the desired type of cake.
It will probably be more useful to stop thinking that 'the basic rule of thumb is to “cream”'. There is no basic rule of thumb. What is needed is to mix the ingredients in a way such as to provide the desired final texture in the cake. There are perhaps 6-7 methods which can be used, and the choice of method is always an integral part of the recipe, to the point where Michael Ruhlmann considers sponge cake and pound cake to be two different cakes, made with exactly the same ingredients at exactly the same ratios, but with two different methods (the creaming and the egg-foaming method).
As a very brief description of the specific methods of creaming and all-at-once, creaming is generally used for cakes with a finer final texture that is softer, with uniform small bubbles. Festive decorated cakes and tortes are frequently made with layers using this technique. The all-in-once method, also called the muffin method, is more commonly used for everyday cakes. It gives you a somewhat irregular texture with some tooth, requires a chemical leavener, and is more tolerant of adding extra ingredients, such as fruit or vegetables in sweet loaves like banana bread, zucchini bread, etc.