Title: Fast Generation for Convolutional Autoregressive Models,
Abstract: Convolutional autoregressive models have recently demonstrated
state-of-the-art performance on a number of generation tasks. While fast,
parallel training methods have been crucial for their success, generation is
typically implemented in a naïve fashion where redundant computations are
unnecessarily repeated. This results in slow generation, making such models
infeasible for production environments. In this work, we describe a method to
speed up generation in convolutional autoregressive models. The key idea is to
cache hidden states to avoid redundant computation. We apply our fast
generation method to the Wavenet and PixelCNN++ models and achieve up to
$21\times$ and $183\times$ speedups respectively.