title: Concatenating a dask dataframe and a pandas dataframe
body: <p>I have a dask dataframe (<code>df</code>) with around 250 million rows (from a 10Gb CSV file). I have another pandas dataframe (<code>ndf</code>) of 25,000 rows. I would like to add the first column of pandas dataframe to the dask dataframe by repeating every item 10,000 times each. </p>
  
  <p>Here's the code that I tried. I have reduced the problem to a smaller size.</p>
  
  <pre><code>import dask.dataframe as dd
  import pandas as pd
  import numpy as np
  
  pd.DataFrame(np.random.rand(25000, 2)).to_csv("tempfile.csv")
  df = dd.read_csv("tempfile.csv")
  ndf = pd.DataFrame(np.random.randint(1000, 3500, size=2500))
  df['Node'] = np.repeat(ndf[0], 10)
  </code></pre>
  
  <p>With this code, I end up with an error.</p>
  
  <blockquote>
    <p>ValueError: Not all divisions are known, can't align partitions. Please use <code>set_index</code> to set the index.</p>
  </blockquote>
  
  <p>I can perform a <code>reset_index()</code> followed by a <code>set_index()</code> to make <code>df.known_divisions</code> <code>True</code> for the dask dataframe. But it is a time consuming operation. Is there a better faster way to do what I am trying to do? Can I do this using pandas itself?</p>
  
  <p>The end goal is to find rows from <code>ndf</code> where any of the corresponding rows from <code>df</code> matches some criteria.</p>