What exactly is the semantic difference between category and set?
In brief, set theory is about membership while category theory is about structure-preserving transformations.

Set theory is only about membership (i.e. being an element) and what can be expressed in terms of that (e.g. being a subset). It does not concern itself with any other properties of elements or sets.

Category theory is a way to talk about how mathematical structures of a given type1 can be transformed into one another2 by functions that preserve some aspect of their structure; it provides a uniform language for speaking of a great range of types1 of mathematical structure (groups, automata, vector spaces, sets, topological spaces, … and even categories!) and the mappings within those types1. Although it formalises the properties of mappings between structures (really: between the sets on which the structure is imposed), it only deals with abstract properties of maps and structures, calling them morphisms (or arrows) and objects; the elements of such structured sets are not the concern of category theory, and nor are the structures on those sets. You ask “what is it a theory of”; it is a theory of structure-preserving mappings of mathematical objects of an arbitrary type1.

The theory of Abstract categories3, however, as just stated, totally ignores the sets, operations, relations and axioms specifying the structure of the objects in question, and just provides a language in which to talk about how mappings that do preserve some such structure behave: without knowing what structure is preserved, we know that the combination of two such maps also preserves structure. For that reason, the axioms of category theory require that there be an associative composition law on morphisms and, similarly, that there be an identity morphism from each object to itself. But it does not assume that morphisms actually are functions between sets, just that they behave like them.


  To be worked out: Concrete categories model the idea of adding structure to the objects of a ‘base category’; when this is $\mathsf{Set}$ we can have the situation where we add structure like a group operation to a set. In this case one may have more to say about how structure is added in terms of the specific base category.


As for the implications of your formulations, saying that “$G$ is a group”, that “$G$ is an element of the set of groups” (actually a proper class) or that “$G$ is (an object) in $\mathsf{Grp}$” (or a “$\mathsf{Grp}$-object”) mean the same thing logically, but talking about the category suggests you are interested in group homomorphisms (the morphisms in $\mathsf{Grp}$) and perhaps in what they have in common with other morphisms. On the other hand, saying $G$ is a group might suggest you are interested in the structure of the group (its multiplication operation) itself or perhaps in how the group acts on some other mathematical object. You would be unlikely to talk about $G$ belonging to the set of groups, though you could easily write $ G ∈ S $ for some particular set $ S $ of groups you are interested in.

See also


https://math.stackexchange.com/questions/tagged/category-theory?sort=votes


In particular https://math.stackexchange.com/questions/272875/concrete-category-and-abstract-category#1774821

Abstract and Concrete Categories: The Joy of Cats by Adámek, Herrlich and Strecker
https://en.wikipedia.org/wiki/Category_theory


https://en.wikipedia.org/wiki/Concrete_category



1 Here and passim I do not refer to type in the sense of type theory, but rather a set of properties required of the mathematical objects/structures, i.e. a set of axioms they satisfy. Normally these describe the behaviour of some operations or relations on elements of the sets considered to carry the structure, though in the case of sets themselves ($\mathsf{Set}$) there is no structure beyond the sets themselves. In any case, as said above, category theory ignores the details of this structure.

2 I should perhaps say into all or part of one another: one allows the homomorphism from $ \mathbb Z $ (integers) into $ \mathbb Q $ (rationals) given by $ n \mapsto \frac n 2 $ . 

3 Without qualification, ‘category’ normally means ‘abstract category’, introduced, as far as I can see, in 1945 and developed in the 1960’s while Concrete categories seem to appear in the 1970’s.