Despite the centrality of harm to crime and criminalization and increasing interest in harm as a basis for crime-control policy, there has been little systematic reflection within criminology on criminal harms or their identification, evaluation, and comparison. In this paper, we review the literature on the harms of crime and related concepts, i.e., the perceived seriousness and cost of crime, impact of criminal victimization, and drug-related harm. Each of these related bodies of work suggests either a reason, by way of inadequacy, or a means, by way of insight or analytical method, to advance a harm-based approach. We then identify substantial challenges in assessing the harms of crime and conclude that, despite these challenges, a systematic empirically-based assessment of the harms of criminal activities can serve important roles in policy analysis.