If an abstract machine can simulate itself, does that make it Turing complete?
No, it's not. I know two major classes of techniques for avoiding inconsistency/Turing completeness.


The first line of attack is to set up the system so that syntax can be arithmetized, but Godel's fixed point theorem doesn't go through. Dan Willard has worked extensively on this and given consistent self-verifying logical systems. The trick is to eliminate the multiplication and addition function symbols, and replace them with divisibility and subtraction. This gives you enough horsepower to represent syntax arithmetically, but the fixed point theorem doesn't go through since multiplication is not provably total. 

See Dan Willard. Self-Verifying Axiom Systems, the Incompleteness Theorem and Related Reflection Principles. Journal of Symbolic Logic 66 (2001) pp. 536-596. 
The second line of attack allows more use of fixed points, but to set things up so that syntax doesn't arithmetize. The prettiest systems for this are (IMO) based on variants of linear logic. For example, in Kazushige Terui's Light Affine Set Theory, even the full unrestricted set comprehension principle is sound, but since the ambient logic of the set theory is linear (and hence contraction is not allowed), Russell's paradox is not derivable. 

The intuitive reason that arithmetization fails is that the light linear function space $A \multimap B$ is set up so that all its inhabitants are polynomial time. As a result, the light linear version of the Peano axioms can't prove exponentiation total (since exponentiation of unary numbers takes exponential time), and so there is no longer an isomorphism between the natural numbers and bit strings. 

Kazushige Terui. Light affine set theory: A naive set theory of polynomial time. Studia Logica, Vol. 77, No. 1, pp. 9-40, 2004.

I think this paper is more accessible after reading the following paper of Yves Lafont:

Y. Lafont, Soft Linear Logic and Polynomial Time, Theoretical Computer Science 318 (special issue on Implicit Computational Complexity) p. 163-180, Elsevier (2004)

Terui's set theory is very expressive, but it's hard to compare with traditional set theories, since proof-theoretic ordinals are not a good tool for comparing very weak systems. For example, Terui's set theory obviously cannot prove exponentiation total, and hence its proof-theoretic strength cannot even reach up to $\omega$. Complexity classes are probably better -- it is complete for polytime (it can prove every polytime function total, but no more). 

I tend to think of these kinds of systems as proofs-of-concept for the idea that complexity theory can serve as a foundation for certain kinds of ultrafinitism.