Why can't amylase digest glycogen?
Amylase can’t digest glycogen because of its inability to attack the branching (1→6) linkages. 

Perhaps, another very important reason is controlling the rate of glycogen metabolism through glycogen phosphorylase. Just like any other biological system, regulation of metabolic substrates and/products is crucial to maintaining the balance (homeostasis) so to prevent excess glucose production from glycogen metabolism or to little according the needs of the organism. 



Tl;dr

There are  actually three forms of amylases:


α-amylase (an endoglycosidase, which can hydrolyze a glycosidic linkage anywhere along the chain to produce glucose and maltose).
β-amylase (an exoglycosidase that cleaves from the nonreducing end of the polymer).
gamma amylase


Now starch consists of two main components amylose and amylopectin;   


Amylose is a linear polymer of several thousand glucose residues linked by (1→4)
bonds:





Amylopectin consists mainly of (1→4)-linked glucose residues but is a branched molecule with (1→6) branch points every 24 to 30 glucose residues on average.
The primary structure of glycogen resembles that of amylopectin(not amylose), but glycogen is more highly branched, with branch points occurring every 8 to 14 glucose residues:




Digestion

The digestion of starch, the main carbohydrate source in the human diet, begins in the mouth. Saliva contains α-amylase, which randomly hydrolyzes all the (1→4) glycosidic bonds of starch except its outermost bonds and those next to branches. 
On the other hand glycogen being a highly branched molecule, it is evident that amylase won’t be a good enzyme to digest it due to physical structure constraints.
 Substrate specificity of the enzymes ensure that the appropriate substrate which is highly optimized for the active site will be broken down. 

Although glycogen posseses parts similar to starch's amlyopectin its structure is not optimized to be broken down by amylases. Note: even these amylases only partially break down starch to different oligosaccharides