The question title doesn't fully capture my question, so I will clarify:

Assume we have the structure of natural numbers: $$(\mathbb N, +,0,1)$$

We know from theorems in mathematical logic, that we cannot use first order predicate logic to fully characterize this structure with a set of axioms. Yet second order logic is incomplete (not all true statements can be proven). 

As far as I understand it, in axiomatic set theory we use first order logic, which is complete, but we quantify over a domain that contains both sets and urelements, thereby allowing us to state things in first order logic that would normally require second order logic (I am not saying that this is the purpose of axiomatic set-theory, but merely that it is true). 

But to simplify, we don't need to study the domain of all mathematical objects as set theory does. We can also formulate the structure: 

$$(\mathbb N,\mathcal P(\mathbb N), +,0,1,\in)$$

and then use first order logic on this structure. **I assume that the standard proof calculus for this is complete, as it has been shown to be complete for first order logic?**

If so, then my question is: 

**What is the difference between** 
$$
\begin{align}
\text{applying second-order logic to } \quad &(\mathbb N, +,0,1) &\quad \text{ and}\\
\text{applying first-order logic to } \quad &(\mathbb N,\mathcal P(\mathbb N), +,0,1,\in)&
\end{align}
$$

* **Is there a difference?**
* **How do we make sense of first-order-logic-completeness, and second-order-logic-incompleteness, if there is no difference?**