How can Image Caption work?
The Standard Image Captioning Pipeline is to train the model in a single batch(or mini-batch) i.e. get the features from the CNN Image encoder and then feed that into an RNN decoder (features + Real Captions) to produce output captions for the Image.
The training loop in PyTorch would look something like this:
# zero the parameter gradients
decoder.zero_grad()
encoder.zero_grad()
        
# Forward pass
features = encoder(image)
outputs = decoder(features, captions)
        
# Compute the Loss
loss = criterion(outputs.view(-1, vocab_size), 
                         captions.view(-1))
        
# Backward pass.
loss.backward()
        
# Update the parameters in the optimizer.
optimizer.step()

I'd suggest you go through the paper Show and Tell: A Neural Image Caption Generator.
I also made this Kaggle Kernel implementing the paper from scratch. Should help clear up any other doubts.