LM338 with pass transistors
Short and useless answer: you only need Ohm's law and maybe \$Ib = Ic/beta\$ for the base current of the transistors.

A bit less useless answer:

Instead of concentrating on which formulas you need, first ask yourself: what happens ? Then it will become clear what formulas to apply.

R1 is across base-emitter of Q1 so I would expect a voltage of 0.7 V (roughly) across R1. Current ? The current at which Q2 - Q5 need to "kick-in", say 100 mA: 0.7/0.1 = 7 ohms. Ha ! I only used Ohm's law :-)

R9, R10 see how these are a voltage divider setting the voltage at the bases of all those 2N3055. The higher that voltage, the more voltage drop you get arcross the 4 0.1 ohm resistors. So determine how much voltage you want across these: take the maximum current, divide by 4 (4 resistors). Then calculate the voltage across them. Add about 0.8 - 1 Volt for the VBEs of Q2-Q5 (look in the datasheet what VBE will be at the current they will conduct).

Now you know the voltage across R10. Output voltage is 13.8 V, input is 24 V so roughly 10.2 V across VCE of Q1 and R9 and R10. We know voltage across R10 so leaves us voltage across VCE of Q1 and R9. I would split that so that Q1 has a couple of volts to work with. Q1 and R9 also need to supply the base currents for Q2-Q5 which can be significant (2N3055 have a low beta).

Draw it on a piece of paper and start annotationg some values. You don't have to be 100% correct the first time. Just fill in values to see what works.