In chemistry what do they mean by disorder when in the context of entropy?
What you have said about entropy is generally correct. For a given energy it is related to the number of accessible states the system can be in. If a system has 100 energetically accessible configurations then we would say it has more entropy than a system with only 10. You can call this disorder because you have no knowledge of which of the 100 (or 10) states the system is actually in.

That said, entropy has a very well define mathematical meaning. I'll start with the punchline and then breakdown what it means. Entropy ($S$) is equal to the natural log of the multiplicity function ($W$, or $g$ sometimes) times boltzmann's constant ($k_b$):
\begin{equation}
S = k_b \ln(W)
\end{equation}

The multiplicity function $W$ is the number of states in the system of a given energy. So what does this mean? Let's imagine we have a magnet with three unpaired electrons which can either be pointed up or down. (e.g. $\uparrow \uparrow \uparrow $,  $\uparrow \downarrow \uparrow$, etc. ) Now let's say that there's a magnetic field applied so that up and down have different energies. Each $\uparrow$ adds $\epsilon$ to the energy while each $\downarrow$ subtracts $\epsilon$ from  the energy. This system can now exist in states ranging from +3$\epsilon$ to -3$\epsilon$.

Let's look at the E = +3$\epsilon$ and -3$\epsilon$ states first. In both cases, W($\pm 3\epsilon$) =1 becasue there is only one configuration for each energy, $\uparrow \uparrow \uparrow $ and $\downarrow \downarrow \downarrow$  respectively. Thus, using the formula, S = $k_b \ln(1) = 0$. Thinking of entropy as disorder, this makes sense. If the energy is $\pm3\epsilon$ then we know exactly which state the system is in and there is no chance whatsoever. 

Now let's look at the case of E = $\pm \epsilon$. Here, we have multiple options for each state. In the E = +$\epsilon$ there are three possible states: $\uparrow \uparrow \downarrow$, $\uparrow \downarrow \uparrow$, and $\downarrow \uparrow \uparrow$. Similarly, for the E = -$\epsilon$ case there are also three states: $\uparrow \downarrow \downarrow$, $\downarrow \uparrow \downarrow$, and $\downarrow \downarrow \uparrow$. Thus, $W(\pm \epsilon) = 3$, and the entropy is $S = k_b \ln(3)$

$k_b$ is called The Boltzmann Constant and is just a scaling constant. It doesn't contain any physics and exists only because we happen to use different units for temperature and energy. In SI it's equal to ~$1.38×10^{−23}$ J/K, but you can equivalently measure temperature in energy units and then $k_b =1$ (with no units). 

The physical insight is that entropy entirely a consequence system being allowed in multiple states.