Can we use transformers for audio classification tasks? Since transformers are good at processing sequential data, can we also use them for audio classification problems (same as RNNs)?
Yes, Transformers can be used to work with audio data, such as audio processing (audio classification, speaker identification, etc) (Audio ALBERT), speech-to-text (Streaming Automatic Speech Recognition with the Transformer Model), and text-to-speech (Neural Speech Synthesis with Transformer Network).