Is it illegal to cite books that were illegally downloaded or accessed from a library?
The illegal act of course is the download, never the citation. @Federico Poloni answered that correctly. 

However:


  If they aren't available in the university library, if you can't really afford the books


besides the possibilities @StasK mentioned:


In many countries there exist inter-library catalogues (e.g. GVK) that tell you which libraries have the book you're looking for. Once you know that (or your local librarian found out for you), you can ask your local librarian to either get the whole book via inter-library loan, or
to order a partial copy (e.g. the chapter or the exact pages you need).
There are also commercial document delivery services (see for an example: Subito). Which offer tracking down documents and again either loan of the book or partial copies.
Even the linked commercial service charges only 9 € per book inside Germany for non-commercial customers and 25 € worldwide for commercial customers but excluding UK and USA (I don't know why). 
If you're talking about a thesis: university libraries usually have a copy at least of all PhD theses done at that university, for Bachelor or Master theses you'd often have to ask the institute (or supervisor) where the work was done.
In Germany, the national library has a copy of each book published in Germany or in German (or about Germany) from 1913 on. There are also field-specific large libraries, as for example the  TIB Hannover for technical literature.


So, are you sure "can't really afford" is a valid argument?

(Whether you nevertheless download pirate copies, or ask your neighbour to let you have a look into her book, or buy it despite the fact that the neigbour at the other desk owns it as well, stays entirely your own choice.)



If I have to get a book by inter-library loan, I try to make a copy of the vital chapter. This is legal here in Germany (single copies for personal use, reseach or teaching of not too large part of a book).