Why are many silver salts insoluble?
This is just an unproven guess on my part.  Whether compounds are soluble or not depends on 1) how strong the bonding (including lattice) forces are that hold the compound together (e.g. how much energy do we have to put in to liberate the ions, and 2) how much energy is recovered through solvation of the resulting ions.  I don't see anything odd about the ionic radius of the silver ion so I'm guessing that the energy of solvation of the resultant ions would be fairly normal; so it is probably not factor "2" that makes most silver salts insoluble.  Here is a link to an article that suggests that bonding in silver compounds may have a covalent component, it says, for silver salts "the structure has some contribution from covalent bonding. This appears to be bonding the structure together more tightly, as in a giant covalent structure."  The words "more tightly" catch my eye.  This seems to suggest that factor "1" may be in play in that silver salts may have increased bond\lattice energies.  If true, this would help explain the insolubility of many silver salts.  Finally I'll add that some of the soluble silver salts ($\ce{AgF, AgNO3, Ag(CH3CO2)}$) seem to involve anions that would be expected to be strongly hydrated.  The extra energy generated through solvation would further assist in solubilizing these silver salts.