What does 1024 by 3 model mean?
At 3:30 in video, he mentions this switching. What he is referring to by 'switch around' is that he is specifying the size of the LSTM being used. The 1024 refers to number of cells (or size) of the LSTM, and the 3 is the number of layers. 

In the tutorial page being referenced I the snippet he is referencing is:

python translate.py
  --data_dir [your_data_directory] --train_dir [checkpoints_directory]
  --size=256 --num_layers=2 --steps_per_checkpoint=50


For details on why LSTMs are useful for this task, check out the paper.