Is there a way to classify sounds
There are probably many different approaches to doing what you are talking about, but the most common would probably be using long short term memory recurrent neural networks to operate on the data through time. You would be able to train your neural network to do the things that you want to do such as classify the speaker, and classify noises into different categories. However, to train neural networks like these, it requires a lot of training data as does any other deep neural network. So, if you want to apply these kinds of neural networks to practical problems you will need a lot of training data. Or, you could just use a smaller simpler neural network.