title: missing attribute in lda2vec module in notebook
body: <p>A jupyter notebook cannot import dirichlet_likelihood.py from lda2vec.<br>
  This py file exists in github for the current lda2vec.</p>
  
  <p>I installed the module and opened the workbook, then attempted to run it.  I suspect there is a very simple reason for my problem.</p>
  
  <p>The notebook is 
  <a href="https://github.com/cemoody/lda2vec/blob/master/examples/twenty_newsgroups/lda2vec/lda2vec.ipynb" rel="nofollow noreferrer">https://github.com/cemoody/lda2vec/blob/master/examples/twenty_newsgroups/lda2vec/lda2vec.ipynb</a></p>
  
  <p>When I try the below at the python command-line (in the current environment) it does not give the below error and instead it wanted keras, which I installed.  On the command line it says it cannot import preprocess.</p>
  
  <pre class="lang-sh prettyprint-override"><code>uname -a
  Linux ubuntu 4.18.0-15-generic #16~18.04.1-Ubuntu SMP Thu Feb 7 14:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
  sudo apt-get install python3-venv
  python3.6 -m venv .env  
  source .env/bin/activate
  pip install --upgrade pip
  pip install jupyter
  pip install lda2vec
  </code></pre>
  
  <pre class="lang-py prettyprint-override"><code>from lda2vec import preprocess, Corpus
  </code></pre>
  
  <pre><code>---------------------------------------------------------------------------
  AttributeError                            Traceback (most recent call last)
  &lt;ipython-input-4-2b87256bea6b&gt; in &lt;module&gt;
  ----&gt; 1 from lda2vec import preprocess, Corpus
        2 import matplotlib.pyplot as plt
        3 import numpy as np
        4 get_ipython().run_line_magic('matplotlib', 'inline')
        5 
  
  ~/.env/lib/python3.6/site-packages/lda2vec/__init__.py in &lt;module&gt;
  ----&gt; 1 import lda2vec.dirichlet_likelihood as dirichlet_likelihood
        2 import lda2vec.embedding_mixture as embedding_mixture
        3 from lda2vec.Lda2vec import Lda2vec as model
        4 import lda2vec.word_embedding as word_embedding
        5 import lda2vec.nlppipe as nlppipe
  
  AttributeError: module 'lda2vec' has no attribute 'dirichlet_likelihood'
  </code></pre>
  
  <pre class="lang-sh prettyprint-override"><code>python
  from lda2vec import preprocess, Corpus
  </code></pre>
  
  <pre><code>Using TensorFlow backend.
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  ImportError: cannot import name 'preprocess'
  </code></pre>
  
  <p>EDIT</p>
  
  <p>I got it work by doing these things
  - Get ubuntu-18.04.2-desktop-amd64.iso
  - Have BIOS virtualisation settings changed for hyper-V
  - Make a VM in VMWare
  - Increase memory to 3G
  - Give it 40G of disk</p>
  
  <p>Then in a terminal</p>
  
  <pre class="lang-sh prettyprint-override"><code>sudo apt install python2.7
  sudo apt install python-pip
  pip install virtualenv
  Mkdir 2.7env
  Cd 2.7env
  python2.7 -m venv .env
  python2.7 -m virtualenv .env
  source .env/bin/activate
  pip install --upgrade pip
  pip install jupyter
  pip install -U spacy
  python -m spacy download en
  pip install wheel nltk gensim pyLDAvis lda2vec
  sudo apt install git
  git clone https://github.com/cemoody/lda2vec.git
  cp ~/lda2vec/build/lib.linux-x86_64-2.7/lda2vec/corpus.py ~/2.7env/.env/lib/python2.7/site-packages/lda2vec/Corpus.py
  cp ~/lda2vec/build/lib.linux-x86_64-2.7/lda2vec/preprocess.py ~/2.7env/.env/lib/python2.7/site-packages/lda2vec/preprocess.py
  python -m pip install ipykernel
  python -m ipykernel install --user
  python lda2vec/examples/twenty_newsgroups/lda2vec/lda2vec_run.py
  cd lda2vec/examples/twenty_newsgroups/lda2vec/
  jupyter notebook
  change the kernel to 2
  </code></pre>
  
  <p>In firefox open lda2vec.ipynb</p>
  
  <p>As above, I am now stuck trying to get it to re-create the twenty_newsgroups npz file so I can eventually provide my own content.  In case anyone out there understands this better, I suspect trying to run this script on VM with little RAM might be the problem but the error is reported as</p>
  
  <pre><code>(.env) craig@ubuntu:~/whcjimmy/lda2vec/examples/twenty_newsgroups/data$ python preprocess.py 
  Traceback (most recent call last):
    File "preprocess.py", line 31, in &lt;module&gt;
      n_threads=4)
    File "/home/craig/whcjimmy/.env/lib/python3.6/site-packages/lda2vec-0.1-py3.6.egg/lda2vec/preprocess.py", line 104, in tokenize
      vocab = {v: nlp.vocab[v].lower_ for v in uniques if v != skip}
    File "/home/craig/whcjimmy/.env/lib/python3.6/site-packages/lda2vec-0.1-py3.6.egg/lda2vec/preprocess.py", line 104, in &lt;dictcomp&gt;
      vocab = {v: nlp.vocab[v].lower_ for v in uniques if v != skip}
    File "vocab.pyx", line 242, in spacy.vocab.Vocab.__getitem__
    File "lexeme.pyx", line 44, in spacy.lexeme.Lexeme.__init__
    File "vocab.pyx", line 157, in spacy.vocab.Vocab.get_by_orth
    File "strings.pyx", line 138, in spacy.strings.StringStore.__getitem__
  KeyError: "[E018] Can't retrieve string for hash '9243420536193520'. This usually refers to an issue with the `Vocab` or `StringStore`."
  </code></pre>