Using DFT Circular convolution property
Yes, that conclusion would be valid. However, as you suspected, there are some limitations. Consider the case where the system's frequency response $Y[k]$ contains one or more zeros. In that case, corresponding frequency bin in your estimate of the input signal $X[k]$ would diverge to infinity (because of the division by zero).

What you're really trying to perform by doing this sort of calculation is deconvolution, which is a complex topic in itself, but suffice it to say that deconvolution is often difficult; you illustrated one of the reasons why. This naive approach of just dividing in the frequency domain is known as a zero-forcing type of algorithm.