How do I build a solar charger for my phone / other devices?
I think it's unlikely you'll be able to do better than the cheap systems off eBay without spending a huge amount of time. Those systems are relatively rugged, weatherproof and have fairly smart charging systems. Building something equivalent means spending a lot of time finding cheap parts and making the bits you can't buy. If you have a 3D printer that will make it easier, but it will still take time.

I found a ~$40 one that folds and has a 10Wh battery that seemed like the best of the lot on eBay but don't have the link handy. Will edit the link in here tonight.

Given that $40 budget for a DIY system you'll need to start with some scrounged parts. If you can find a LiIon USB battery that will make your life a lot easier - all you need to do is add some scrounged solar panels. But I suggest trying to find all the small solar panels you can to start with. Test them to find which ones still work and what their output is. Work out how to stitch them together to get a single source (both mechanically and electrically). 

Then spend most of your money on a smart LiIon charger and voltage regulator to feed 5.1V out the USB port you'll use to get power out of the device. I suggest a battery inside it to buffer the erratic solar input as some smart devices don't like that (my phone, for one). Even a smallish battery will do, a couple of Wh, as long as it can handle the max power you get out of your panels.

I would probably go with either a small 12V SLA or ideally scrounge a complete LiIon booster battery system that has a working charge/protection circuit. Then add a 5V regulated output you're good to go.