Word based perplexity from char-rnn model
Actually, there is a formula which can easily convert character based PPL and word based PPL. 

$PPL = 2^{(BPC*Nc/Nw)}$ 

where $BPC$ is character based $PPL$, $Nc$ and $Nw$ are the number of characters and words in a test set, respectively.

The formula is not completely fair, but it at least offers a way to comparing them. The following are some reference.

[1] Hwang K, Sung W. Character-Level Language Modeling with Hierarchical Recurrent Neural Networks[J]. 2016.

[2] Graves A. Generating Sequences With Recurrent Neural Networks[J]. Computer Science, 2013.

[3] T. Mikolov, I. Sutskever, A. Deoras, H. Le, S. Kombrink, and J. Cernocky.Subword language modeling with neural networks. Technical report, Un-published Manuscript, 2012.