While it may stop the heartburn, the consequences of suddenly stopping some medications can be worse than the heartburn. Medications that may cause heartburn include: 1  Anti-anxiety medications, such as Valium or Ativan. 2  Some antibiotics, such as tetracycline.  Anticholinergics.