Changing units in Fourier filtering
Short answer: your conclusion seems ok.

Concept of units make sense only in the world of physics. Mathematical objects such as DFT trasforms do not have any kind of units but just their values, quasi exceptions being the planar angle meausure of radian and volumetric angle mesaure of the steradian. However, when a variable represents a physical quantity (as almost always is the case) then it's customary to associate units with those mathematical variables as well.

Let's do a 1D analysis of those units, from which the 2D version follows straightforwardly. Discrete-time variables are indexed sequences such as $x[n]$ or $X[k]$ which may represent samples of a continuous-time signal $x(t)$ and the DFT of $x[n]$ respectively.

We can assume that a sequence $x[n]$ bears a unit which is the unit of the associated physical variable $x(t)$ from which $x[n]$ was sampled. So if $x(t)$ was a voltage in units of Volt,   then the unit of $x[n]$ will be Volt too.

The index $n$ does not have any units which can be seen by the following uniform sampling relation :

$$ t_n = n T_s $$ 

where $t_n$ is the sampling time with a unit of second (s), and $T_s$ is the sampling period with a unit of second (s) too; hence the index number $n$ is unitless. However, some like to call $n$ with a unit of samples. Yet this is not only unnecessary but also as meaningless as associating a unit with room numbers in a hotel. Nevertheless it's customary to do so and I do also use it.

The 1D-DFT $X[k]$ associated with a given sequence $x[n]$ is defined as follows:

$$ X[k] = \sum_{n=0}^{N-1} x[n] e^{ j \frac{2\pi}{N} k n} $$

The complex exponential function (as well as other trigonometric functions such as sine, cosine, tangent) produce unitless outputs as pure numbers. The trigonometric functions expects angles in units of radians as their input arguments. 

Therefore, the unit of the DFT $X[k]$ will be that of the sequence $x[n]$.

The unit of the DFT frequency bin index $k$ is found from the unit of the complex exponential argument :

$$ \frac{2 \pi }{N} k n = \theta $$

In this expression, the sample index $n$ and the total sample count $N$ (or frame length as some would like to call) have no units (or they both have the unit of samples, but it will cancel neverthelesss) and the angles $\theta$ and $2\pi$ have the unit of radians hence the DFT frequency index $k$ is unitless too.

How about calling the frequency variable $k$ as cycles per frame length ? Ok, it is also preferred and some may find it informative to call $k$ as the number of cycles a sine wave fits into the given frame length of analysis.

With the same reasoning, a 2D intensity image sample sequence $f[n,m]$ will have units of intensity and the spatial index variables $m,n$ are either unitless or given a samples unit. And for the 2D-DFT $X[u,v]$, the frequency index variables $u,v$ are unitless. Nevertheless, again some prefer calling them in the units of cycles per image frame length and you can also do it.

Now coming to the units of the expression

$$ \sqrt{ u^2 + v^2 } &lt; R ,$$ 

the variable $R$ is either unitless as explained above or can be associated with the unit of cycles per frame length as some would prefer. From your expanded questions, it became clear that you wanted to change the scale of this actually unitless parameter from cycles per frame length to cycles per mm.

Now, to find out the continuous-space frequency in cycles per meter associated with a given DFT frequency bin index $u$ or $v$ , you would actually perform the following conversion :

$$ f_u =  \frac{u}{H} F_s $$

where $u$ is the unitless DFT frequency bin index count, $H$ is the number of samples (frame length in samples or image height in your example) and $F_s$ is the sampling frequency in cycles per meter (I assume). In your project you set $f_s = H$ (assuming $H = \min(H,W)$) and you give a pixel-size parameter of $\gamma$ in mm/pixel units. Then this $\gamma$ is the inverse of sampling frequency $F_s$ in samples per mm. And the above formula becomes :

$$ f_u =  \frac{u}{f_s} F_s = \frac{u}{f_s} \frac{1}{\gamma} = \frac{u}{f_s \gamma} $$

Finally since the ideal circular lowpass filter bandwidth radius parameter $R$ is a same kind of variable with DFT frqeuency bin index variable $u$ or $v$, therefore its conversion from cycles per frame length to cycles per mm will be done using the same formula:

$$ r =  \frac{R}{f_s} F_s = \frac{R}{f_s} \frac{1}{\gamma} = \frac{R}{f_s \gamma} $$

where $r$ has the units of cycles per mm.