Will I need a heatsink for this voltage regulator
I watched your video.  Parker, can I tell you that the regulator's heat sink is the least of your problems.  I'm not trying to sass you, but having read through your question and subsequent comments, I'd advise you against this project.  

Some of the people on this site are hopeless amateurs like me, but some are professional electrical engineers.  I'm sure that there will be a general consensus that lithium ion batteries are dangerous.  They can be very dangerous.  They are very absolutely not like normal batteries. The reason your laptop hasn't caught fire is that there are at least three levels of independent protection to stop it bursting into flame.  And it's been professionally designed by many geeks.  I'm not kidding you.

Ripping open the laptop cell carrier with a large blunt instrument risks perforating the cell wall.  There may be current protection in the cell carrier that you'll be chucking away.  And lithium batteries have to be carefully charged with a specialist circuit.  You'll be missing all these protections.  And running anything off a solar panel is tricky as the generated voltage and current can be all over the place.  

If your skill level only covers the basics, a dual solar /usb lithium ion power pack using old recovered laptop batteries is going to be way too complicated for you.  Sorry.  Can I suggest that you watch my video, and google "Are laptop batteries dangerous".  

https://www.youtube.com/watch?v=pizFsY0yjss

You have been warned :-)

If you continue, just get one 3" square and that should keep the heat down.  And don't use a normal fire extinguisher, it might not work.  You'll need a specialist class D type appropriate for metal fires.