Understanding the definition of reduction
The definition you're quoting is very abstract, but the concepts you're trying to understand are pretty intuitive. A problem $A$ is NP-hard if you can solve any problem in NP using it. This means that any $B \in NP$ can be reduced to $A$, i.e. there is some polytime function $f$ such that $x \in B$ iff $f(x) \in A$; so you can test whether $x \in B$ by computing $f(x)$ and test whether the latter is in $A$.

A problem is NP-complete if it is both in NP-hard and in NP. This means that it is hardest among problems in NP. A problem can be NP-hard without being in NP, for example the halting problem.

You're mentioning sets of problems as belonging to NP, but that's a typing error: members of NP are problems, in the guise of subsets of the set of natural numbers (or of the set of finite binary strings, which is the same). The subset specifies the set of inputs for which the problem has the answer YES.