How is the resistor value for boost conversion calculated?
You are free to choose \$R_\mathrm{BOT}\$ such that
(a) you can find close-enough standard resistor values for both \$R_\mathrm{BOT}\$  and \$R_\mathrm{TOP}\$, and
(b) \$R_\mathrm{BOT}\$ and \$R_\mathrm{TOP}\$ are the right order of magnitude.
About the order of magnitude, cf. the explanation on section 5.2:

There are some potential issues with higher value resistors. For small surface mount resistors, environment contamination can create leakage paths that significantly change the resistor divider ratio and modify the output voltage tolerance.
Smaller feedback resistor values will increase the current drained from the battery by a few μA, but will result in good regulation over the entire temperature range and environment conditions.

So if you pick 1 K or 10 K resistors instead of 100 K resistors you will have a bit more current drain* but it will otherwise work fine. If you pick 1 M resistors or more, you may have issues with noise and stability.
They mention contaminants: moisture and dirt on the PCB can create parasitic resistors of some 10s or 100s of MOhms. You want your divider network to be lower than that so that it &quot;wins&quot; over these parasitic current paths.
* estimate it as \$I_\mathrm{drain}=\frac{V_\mathrm{out}}{R_\mathrm{BOT}+R_\mathrm{TOP}}\$