Turning a conference proceedings into a book chapter. Is it self plagiarism?
You have a few misconceptions. Having an ISBN is no guarantee of legitimacy, honesty, or quality as anyone can purchase such. In bulk they are very cheap. I currently have several unassigned ISBNs myself, intended for future work. 

Second, if your paper has been submitted to a conference and appears in the proceedings then it is "published" and so subsequent uses are not original. 

However, self plagiarism only occurs if you reuse some work without citing it. If you still hold copyright, as is likely, you can reuse the thoughts and many of the actual words as long as you are careful. Cite the original conference work as appropriate and formally quote from it as necessary. Perhaps you gave up copyright already, but I think that is unlikely in this case and you would know anyway if you had signed a transfer. And it isn't the ISBN on the original that makes it self plagiarism. It is copying without citing. 

Cite your own work the way you would that of another author. 

The main purpose of avoiding self plagiarism is to permit a future reader to trace the full context of the ideas in the new work by examining the old with its full context, including all references you cited there. Future researchers want a full record. Ordinary plagiarism includes this as well as other concerns, of course. 

But some of what the publishers seem to be saying seems a bit of a scam. I can't say without more information, but it has that feel. Legitimate publishers also have "page charges". Try to assess the legitimacy of the publisher using other information before you agree.