Grain refinement in lath martensite and bainite structures, which is important for strengthening and toughening, are discussed in various aspects. Strain accommodation plays important roles to determine final crystal sizes of bainitic ferrite (BF) and martensite. There is strong variant selection of BF by natures of the austenite grain boundary where it nucleates. For small undercooling, such variant selection leads to coarse bainite block and packet sizes. More BF variants are formed by increasing undercooling, which leads to nucleation of BF variants of less potency, and by increasing strength of materials, which results in more self-accommodation of transformation strain due to suppression of plastic accommodation. In lath martensite, there seems to be similar variant selection at austenite grain boundaries. However, packet/block sizes in lath martensite decreases with an increase in carbon content because of more extensive self-accommodation due to lower transformation temperatures than bainite.