Signal processing for audio and speech
In line with a previous similar question here are my suggestions:

There are so many nice books but I believe you should first have a look at the science of sound from Rossing for getting the most broad view on the subject.

Then you can look at the following books , each dealing with a particular dimension of the problem:

1- Acoustics_BERANEK

2- Elements of Acoustical Engineering_OLSON

3- Music, Physics and Engineering_OLSON

4- Art of Digital Audio_WATKINSON

5- Audio Engineering_SINCLAIR

6- Digital Audio Effects_ZÖLZER

7- Digital Audio Signal Processing_ZÖLZER

8- Fundamentals of Acoustic Signal Processing_TOHYAMA

9- Audio Signal Procesing and Coding_SPANIAS

10- Audio Programming Book_BOULANGER

11- PsychoAcoustics_ZWICKER

12- Acoustics and PsychoAcoustics_HOWARD

EDIT:
I'm sorry that I've forgotten to mention about books on speech processing, so here you can find a list:

1- Speech and Audio Signal Processing_GOLD

2- Speech and Hearing for Communication_FLETCHER

3- Discrete Time Processing of Speech Signals_PROAKIS

4- Fundamentals of Speech Recognition_RABINER

5- Introduction to Digital Speech Processing_RABINER