Must people tell an AI which algorithm it should use? Can an AI learn algorithms by itself?
There are problems we for which we don't have a known, optimal, deterministic algorithm. By and large we use heuristics to "solve" those problems.  A closely related idea is that of satisficing where we seek out answers that are "good enough" for immediate purposes.

Likewise, machines can also use heuristics, whether they are programmed in explicitly or, presumably, learned.  Within the range of ways that a machine can use heuristics, there are meta heuristics and hyper heuristics.

Going beyond that, there are other ways that machines an learn "algorithms" or "rules" for solving problems.  One are that I'm particularly interested in is known as rule induction.  

This is all an area of open and active research BTW... so if you're interested in exploring any of these approaches, you'll probably find a lot of ground to cover.