Are batch iteration and epochs different in reinforcement learning compared to supervised learning?
Your initial statements are correct.

Epoch is one-single pass over the full training set. 

Most people use the word Batch as the number of samples used for one update of the weights. (The back-propagation process calculates the gradients for every single sample in the batch, but the weights update is performed a single time for the mean of the gradients over the batch).

However, originally the word batch was used to mean a training process where the batch size is equal to the overall number of samples in the training set (meaning that the weights update is performed only once per epoch). What we now refer to as "batch", was originally named "mini-batch".

Over the years, the word batch became synonymous with mini-batch (batch training is very rare these days). I think that this is the source to your confusion.