LM317 or DC to DC Buck converter for solar charger
A linear regulator works by wasting exactly so much input power, that the output is at the right voltage. If you go from 12V to 5V, it means that you will be wasting over half of your power, always! Apart from the fact that it is very inefficient, there is another problem here - if you are really getting the rated 80W out of your solar panel, it means you will have to dissipate more than 40W in your LM317. This is on par with desktop computers, and there is a reason those have big heatsinks with fans. 
Even if you made some very fancy cooling system to get rid of this heat, most LM317s in a TO220 package are rated to 1.5A. 80W at 12V is about 6.5A of current - so your LM317 would be damaged no matter what. In fact, I don't know of any linear converter that can handle that much current (There are however linear regulator controllers and such that can do it, but they need big external transistors to do it).

A switching regulator however moves around charge or current, and by doing so, it will be more efficient. Switching regulators can get more than 95% efficiency (given it's a good regulator, working at it's optimal point, ...).

If budget isn't too much of a problem, I would suggest looking at pre-built DC/DC convertor modules. They aren't too expensive, and they are a lot easier to use - building a switching converter is a bit more complicated as they often require external components (switching elements, capacitors, inductors). On top of that, things like placement of components, imperfections in the components and their effects become quite important. 

A lot of the prebuilt modules are also protected against overload and short circuits.

I would also like to point out that there are likely DC/DC converters designed for solar panels. These can make sure that your solar panels are always operated in their optimal working point.