Is it plagiarism to copy from the textbook and referencing properly but not using quotation marks?
Plagiarism is not a matter of quotation marks. The question is whether, when reading your homework, it is clear that the excerpts are not your own words but from some place else. You could also use an indentation for a paragraph, or put it all in italics. Quite often this is a matter of style guides or the preferences of the respective department or whoever is grading your work.

However, if you simply referenced a book, chapter or anything else at the beginning of a section and then started copy and pasting, at worst changing the order of paragraphs and mixing in your own words to connect passages, I'd have serious problems with that. Even if not reading your homework cover to cover (but let's say when opening a random page) it needs to be clear what is copied and what not and where copied passages are from. So yes, depending on how you have formatted and phrased the text, the omission of quotation marks may well be interpreted as plagiarism.

By the way, copying several paragraphs from any source will most likely not get you the best grade. Quote where necessary but focus on your own thougths and words and use the quotes to get your overall point across.