Is associativity number for caches per row?
In a direct mapped cache, each unit of memory is tied to a specific cache block (a.k.a cache line). Some bits from the address are taken, and they determine the number of the cache block. If there is a cache miss, that cache block is evacuated and replaced with the block being accessed. This is a cheap cache to implement.

In a fully associative cache, memory is not tied to a specific cache block. When there is a cache miss, the entire cache is searched for a good block to evacuate and replace. This is the "deluxe" cache type.

Set association is a cost/performance tradeoff between the two. A given address in memory is not eligible to be planted anywhere in the entire cache, but is numerically tied to a specific set: some bits from an address determine the set number that it maps to. In that set, there is more than one cache line which can be selected for replacement according to a strategy like least recently used.

Direct mapping is like set association, where the set size is 1: there are as many sets as cache blocks. Full association is like set association, where there is one big set which contains all the cache blocks.

Mapping to a set is direct: bits from the address identify the set.  Mapping within the set is associative: some other bits from the address constitute a tag, and the set must be searched to find the entry with that tag.

A cache with associativity 4 and 10 sets would have 40 blocks. However, in practice, 10 sets won't occur because it isn't a power of two. The number of sets will be some power of two, indexed by a range of bits from the address. Associativity is expressed as "N-way".  If there are four cache lines in a set, it is "four-way associative".  Four separate ranges of memory that clash to the same set can all be cached and accessed without evacuation.