What determines whether a reaction using ATP produces ADP or AMP?
Summary
It is the nature of the biochemical reaction that determines whether a reaction of ATP involves hydrolysis of the β- or γ- phosphoanhydride bond. If a part of the ATP molecule is incorporated into one of the products, the choice of bond to be cleaved emerges from the chemistry of the reaction. If the hydrolysis (with a negative free energy change, ΔG) is being coupled to a reaction with a positive ΔG, hydrolysis of single (γ-) phosphoanhydride bond usually suffices, with the production of ADP and Pi. If the free-energy of hydrolysis of the phosphoanhydride bond is being ‘transferred’ to another molecule in order to ‘activate’ it for a specialized group-transfer role (often in the synthesis of macromolecules) two bonds are often broken: the first (β-) transfers the free energy with the production of pyrophosphate, whereas the second (now in the pyrophosphate) is hydrolysed non-productively (free energy lost as heat) to ensure that the overall reaction is irreversible.
Energy considerations in reactions involving ATP
I have discussed some of this at greater length in an answer to another question, however it is important to clarify this at the outset. In considering biological reactions, it is usual to adopt a thermodynamic approach in which the thermodynamical (Gibbs) Free Energy change (ΔG) of the reaction is considered. This is because, to quote from Berg et al.:

A reaction can occur spontaneously only if ΔG is negative.
A system is at equilibrium and no net change can take place if ΔG is zero.
A reaction cannot occur spontaneously if ΔG is positive. An input of
free energy is required to drive such a reaction.

The literature provides values of the standard free energy changes, ΔGo, for reactions, i.e. the values obtained under conditions in which the concentration of all reactants and products are set at the same concentration. The actual free energy change, ΔG, depends on the concentration of reactants and products (mass action effects).
The hydrolysis of the β- and γ- phosphoanhydride bonds of ATP both have a high negative values of ΔGo of ca. –45 and –30 kJ per mol, respectively†.
Use of ATP to drive unfavourable reactions
In many cases the hydrolysis of ATP is coupled to an energetically unfavourable reaction (one with a +ve ΔGo) so that the coupled reaction has overall negative free energy change and hence the energetically favourable reaction can take place:
A → B                    ΔG =  20 kJ/mol (unfavourable)
ATP → ADP + Pi           ΔG = –30 kJ/mol (favourable)
A + ATP → B + ADP + Pi   ΔG = –10 kJ/mol (favourable)

In this hypothetical example, 20 kJ/mol of the free energy of hydrolysis of ATP is used to convert A to B, the other 10 kJ/mol is lost as heat, but makes the reaction essentially irreversible. The energy of hydrolysis of only a single phosphoanhydride bond is sufficient for this, and is achieved by the hydrolysis of the γ bond with the production of ADP. (Hydrolysis of the β bond would ‘loose’ the second bond in pyrophosphate, where its hydrolysis cannot be used productively.) An example is the conversion of pyruvate to oxaloacetate in gluconeogenesis, in which energy is required to form a carbon–carbon bond:
CH3COCOO– + HCO3– + ATP → COO–CH2COCOO– + ADP + Pi
Incorporation of ATP into products
In some cases there is not only hydrolysis of a phosphoanhydride bond, but a chemical component of ATP is incorporated into one of the products. In this case the actual chemical reaction determines which bond is hydrolysed.
In the first type of reaction a phosphate group is incorporated into the product, in which case this will obviously be the γ-phosphate and ADP will be generated. A simple example of this is the hexokinase reaction:
Glucose + ATP  → Glucose 6-P + ADP
The reaction is essentially irreversible as the energy of the hexose phosphate bond is much less than that of the phosphoanhydride bond.
In the second type of reaction the AMP component of ATP (or similar molecules) is incorporated into one of the products, so that the other product must be pyrophosphate. The obvious example is the RNA polymerase reaction, which for our purposes can be represented as:
------ribose-3′OH + ATP → ------ribose--3′O-PO2-O-5′-Adenosine + PPi
ATP and ‘activation’ reactions: rationale for pyrophosphate
In certain reactions of ATP the energy of a  phosphoanhydride bond is being used to created a bond of similar free energy of hydrolysis, which can itself be used subsequently to activate other reactions. In this case the reaction may involve little of no free energy change, so that it would appear to be easily reversible. In many such cases the β phosphoanhydride bond is cleaved, generating pyrophosphate. The rationale for this is that the pyrophosphate is then converted to phosphate in a reaction catalysed by pyrophosphatases, which has a high negative DG and is hence essentially irreversible because the free energy of hydrolysis of the phosphoanhydride bond in pyrophosphate is lost as heat.
A + ATP → B + AMP + PPi     ΔG = 0 kJ/mol (reversible)
PPi → 2 Pi                  ΔG = –33 kJ/mol (irreversible)

The examples often given to explain this are the nucleic acid polymerase reactions, but, although this applies also to them, the nature of their reaction necessitates the generation of pyrophosphate. To some extent this is the case with most other activation reactions, but it may be clearer to consider one in which the component of ATP is incorporated as a temporary intermediate, rather than as a component of the end product. Such a reaction is amino acid activation for protein synthesis:
amino acid + ATP → aminoacyl-AMP + PPi
aminoacyl-AMP + tRNA → aminoacyl-tRNA + AMP
The ‘activation’ is the formation of bond with a high free energy of hydrolysis in the aminoacyl-AMP from the β phosphoanhydride bond. This is then used to drive the reaction forming a bond between the amino acid and the tRNA. The generation of pyrophosphate ensures the effective irreversibility of this key step in protein biosynthesis.
†Possible role of differences of free energy of hydrolysis
The fact that the standard free energy of hydrolysis of the β phosphoanhydride bond is greater than that of the γ phosphoanhydride bond may also be relevant in some cases, as @user1136 has argued in his answer to this question.