Are all mutagens carcinogens?
Short answer
All mutagens are potential carcinogens unless the mutagen is highly specific to a site. As noted in the question, carcinogens need not be mutagenic.

HPV causes oncogenic transformation of a cell because of certain proteins that it expresses. HPV is considered a carcinogen by the IARC. Some retroviruses are oncogenic: they might carry an oncogene or insert randomly near an endogenous proto-oncogene and cause oncogenic transformation; this phenomenon is called insertional mutagenesis (Coffin JM, Hughes SH, Varmus HE, editors. Retroviruses). They can also integrate within a gene and knock it out. Many of these viruses are considered carcinogens by IARC.
Hepatitis viruses (B &amp; C) can also induce carcinogenesis but insertional mutagenesis is not the sole mechanism (Cougot et al., 2005; Fung et al., 2009; Lemon &amp; McGivern, 2012). Hepatitis viruses are considered carcinogens by IARC.
Ethanol in alcoholic beverages is considered carcinogenic by IARC. There are many substances listed as carcinogens by IARC but they are not mutagenic. The general mechanism of carcinogenesis by these substances may be prolonged inflammation or ROS generation.
Loss of function of the tumour suppressor gene p53 leads to cancerous progression (Chiche et al., 2016; Venkatachalam et al. ISBN:978-1-59259-100-8). So an siRNA that targets p53 should be called a carcinogen. It would however not cause any mutations and is hence not a mutagen.
If you consider the tools developed for genome editing, such as ZFN, CRISPR-Cas and TALEN as mutagens then they are not carcinogenic. However the term mutagen is not used for these molecules. Mutagen almost always refers to a molecule that causes random mutagenesis thereby making it a potential carcinogen.