how does word variety depend on total words? I want to compare the word variety of several books. But some are short, while others are long. So how can I correct for the fact that longer books will generally contain a larger number of unique words. I tried simply dividing the number of unique words in each book by the number of total words in each book. But I think that was overdoing it, because now the shorter books appear to have the most variety. What is the best approach to this?
The simple answer is to use a sample of equal size from every book, or even better to randomly extract several samples of equal size from every book and then use the mean across samples.


  I tried simply dividing the number of unique words in each book by the number of total words in each book


This is known as the type/token ratio, the simplest way to measure lexical density. I think it makes perfect sense in the case you describe, as far as I know usually it's not too biased.