Theoretical justification of "halting problem avoidance"
It is absolutely theoretically justified. 

First realize that a loop is just a form of recursion: do the loop body, then either stop or do the loop body again with different variable values. 

System F is a lambda calculus (programming language) with no recursion built in, and it is known to be strongly normalization. That is well typed every computation in this system halts. It's also powerful enough to compute basically every function you can think of, including the infamous Ackerman function. 

In this system, you can use Church numerals to simulate the finite looping you mention in your question. 

If you take a programming language without loops, you can model it in System F, which will give you a guarantee that all programs in this system halt. 

It's implicit that if you remove loops, you also remove GOTO, which can be used to build loops.