Does undecidability ever imply unmeasurability and make a notion of probability ill-defined? Not sure precisely how to ask this question, but I want to understand if it is meaningful to ask about probabilities when aspects of the definition might be undecidable.  My curiosity extends to potential implications for definitions of entropy in physical systems.  Let me try to get a little more precise about the question, and maybe someone can point me in the direction of papers or textbooks that address the topic, or has an answer themselves.
For example, imagine I have a dynamical system which can be mapped to a universal Turing machine, so there are some input settings where I can map the outputs to Halting vs not.  Now some inputs are chosen at random, and I want to know the probability that the machine halts or not.  Does the fact that knowing if the problem halts on the input or not is undecidable imply that the a probability measure is ill-defined in this case?  Are there any implications for probability of input or output in such a case?
The space of possible inputs of a Turing machine is countable, so all its subsets are measurable. Same for outputs.
However, it's possible that this probability is not computable. In particular, you may want to look up Chaitin's constant. It isn't exactly what you are talking about, but closely related.