DIY 18650 charge pack questions
1) 35 A is a bogus rating for 18650 batteries. This is a "pulse rating", which in practice does not tell you anything of interest. If you buy "high discharge" batteries from a respected brand, they can deliver 20 A or even slightly more of continuous current, but the real continuous rating of the cheap knockoff batteries (and the many counterfeits out there) is anyone's guess. Don't trust the sales pitch.

However, for your scenario this is not a problem, because:

2) The batteries will not output more current than is drawn from them. If your boost module draws 4 A, then that is what the batteries will deliver.

3) As long as your solar panels don't exceed 5 V on the output, feeding the output into a USB socket should work.