Algorithm for packing various shapes inside of a rectangle
As pointed out by the other answer, this is an example of bin packing, a type of problem that is NP-complete.

Skiena Section 17.9 reports:


  Fortunately, relatively simple heuristics tend to work well on most
  bin-packing problems.


The same source provides the following heuristic recommendation:


  Analytical and empirical results suggest that first-fit decreasing is
  the best heuristic. Sort the objects in decreasing order of size, so
  that the biggest object is first and the smallest last. Insert each
  object one by one into the first bin that has room for it. If no bin
  has room, we must start another bin. In the case of one-dimensional
  bin packing, this can never require more than $22\%$ more bins than
  necessary and usually does much better. First-fit decreasing has an
  intuitive appeal to it, for we pack the bulky objects first and hope
  that little objects can fill up the cracks.
  
  First-fit decreasing is easily implemented in $O(n \ \text{lg} n + bn)$
  time, where $b \leq min(n,m)$ is the number of bins actually used.
  Simply do a linear sweep through the bins on each insertion. A faster
  $O(n \ \text{lg} n)$ implementation is possible by using a binary tree
  to keep track of the space remaining in each bin. We can fiddle with
  the insertion order in such a scheme to deal with problem-specific
  constraints.


I recommend reading the original material in its entirety.