Syntax and formal grammar of a formal language
Syntax is a very general term that encompasses the description and
properties of physical representation of languages, i.e. of sets of
sentences intended to convey ideas, concepts, meanings, etc.  It is to be
contrasted with semantics that is concerned with what is being
conveyed.

Syntax is often thought of as structured sets of strings of symbols on
a given alphabet (sentences), but that is the simpler form. Many syntaxes have a
more complex structure, such as spoken natural language, signed
language, graphical languages, musical scores, etc.

Syntax could be thought of as just a way to describe the set of
legitimate representations, but it is usually expected to also
associate structure with these representations. This structure is the
scaffolding that is used to associate meaning (semantics) with a
syntactic construct.

Grammars are a specific way to define these sets of sentences. There
are other ways, such as automata (e.g. FA, PDA, TM), or string set
expressions (regular expressions). Some types of grammars
(Context-free in particular, but not the only one) are the preferred
medium for expressing syntax as they are complex enough to provide
structure allowing to express semantics with enough perspicuity, while
remaining simple enough to be intellectually tractable.

Regular grammar, for example, do not provide enough structure and
expressive power for general syntactic structure, but are often
adequate for the structure of simpler semantics constituants
(identifiers, numbers, morphological  strcture of natural languages).

On the other hand, while context-sensitive languages can define more
complex sets of sentences, the associated structure is usually too
complex for deriving semantics from it in a simple enough way.

Context-free grammar are convenient because they associate a tree
structure with sentences. This tree structure can be seen as an
expression in a formal algebra, from which meaning can be derived by a
convenient and well understoof mathematical tool: the homomorphism.

There are other ways of associating a tree structure with a sentence,
for example by considering derivation trees with the structure (see
mildly context-sensitive formalisms).

One could also define the syntax with automata, but this is much more
computational and less perspicuous.

Denotational vs operational aspects

The preference for grammars over automata may also be related to the
preference for denotational definitions over operational definition.
Many grammars may be seen as more denotational as they may be read as
equational definitions of syntactic structures. Though it may sometimes be
disputed, denotational definitions are perceived as closer to the
essence and mathematical properties of the defined entity, as giving a
more perspicuous understanding, more appropriate for mathematical analysis, while operational semantics are more
encumbered with technical computational or algorithmic details on the
use of these entities.

Hence, it is common practice to prefer denotational definitions as
reference, including for syntax. Operational tools (such as parsers)
are then derived for practical computational work, and have to be
proved equivalent.

Meta-language issues

Note that, when I say that a grammar "may be read as equational
definitions of syntactic structures", I assume that the grammar itself
is a sentence in a formal language (the language for writing grammar)
and that this language may be assigned one or several semantics by
appropriate means. Indeed, there is at least one well known
standardized language for writing context-free grammars which is the
Backus-Naur Form (BNF).

Actually, different semantics can often be associated with the same
syntax, though they will often be in some mathematical relation,
possibly isomorphic. This is in particular the topic of what is called
"abstract interpretations" and 
"non standard interpretations" of languages.

Remark on the question

When answering the question, I did take the expression "formal
language" in a general sense of language formally defined, as often
used in mathematical context, with syntax and semantics. But the
discussion is very much the same in the case of natural language.

Now it is possible that the question was intended only for the
consideration of formal languages as simple sets of strings, as in
formal language theory.

I would tend to think that this should not change much the answer,
since even formal language theory often sees more in its formal
languages than simple set of strings. This is typically the case when
it makes a difference between strong and weak equivalence of grammars.
Furthermore, a grammar is usually only one possible way to define the
language and/or give it structure.