title: Numpy hstack - "ValueError: all the input arrays must have same number of dimensions" - but they do
body: <p>I am trying to join two numpy arrays. In one I have a set of columns/features after running TF-IDF on a single column of text. In the other I have one column/feature which is an integer. So I read in a column of train and test data, run TF-IDF on this, and then I want to add another integer column because I think this will help my classifier learn more accurately how it should behave. </p>
  
  <p>Unfortunately, I am getting the error in the title when I try and run <code>hstack</code> to add this single column to my other numpy array.</p>
  
  <p>Here is my code : </p>
  
  <pre><code>  #reading in test/train data for TF-IDF
    traindata = list(np.array(p.read_csv('FinalCSVFin.csv', delimiter=";"))[:,2])
    testdata = list(np.array(p.read_csv('FinalTestCSVFin.csv', delimiter=";"))[:,2])
  
    #reading in labels for training
    y = np.array(p.read_csv('FinalCSVFin.csv', delimiter=";"))[:,-2]
  
    #reading in single integer column to join
    AlexaTrainData = p.read_csv('FinalCSVFin.csv', delimiter=";")[["alexarank"]]
    AlexaTestData = p.read_csv('FinalTestCSVFin.csv', delimiter=";")[["alexarank"]]
    AllAlexaAndGoogleInfo = AlexaTestData.append(AlexaTrainData)
  
    tfv = TfidfVectorizer(min_df=3,  max_features=None, strip_accents='unicode',  
          analyzer='word',token_pattern=r'\w{1,}',ngram_range=(1, 2), use_idf=1,smooth_idf=1,sublinear_tf=1) #tf-idf object
    rd = lm.LogisticRegression(penalty='l2', dual=True, tol=0.0001, 
                               C=1, fit_intercept=True, intercept_scaling=1.0, 
                               class_weight=None, random_state=None) #Classifier
    X_all = traindata + testdata #adding test and train data to put into tf-idf
    lentrain = len(traindata) #find length of train data
    tfv.fit(X_all) #fit tf-idf on all our text
    X_all = tfv.transform(X_all) #transform it
    X = X_all[:lentrain] #reduce to size of training set
    AllAlexaAndGoogleInfo = AllAlexaAndGoogleInfo[:lentrain] #reduce to size of training set
    X_test = X_all[lentrain:] #reduce to size of training set
  
    #printing debug info, output below : 
    print "X.shape =&gt; " + str(X.shape)
    print "AllAlexaAndGoogleInfo.shape =&gt; " + str(AllAlexaAndGoogleInfo.shape)
    print "X_all.shape =&gt; " + str(X_all.shape)
  
    #line we get error on
    X = np.hstack((X, AllAlexaAndGoogleInfo))
  </code></pre>
  
  <p>Below is the output and error message :</p>
  
  <pre><code>X.shape =&gt; (7395, 238377)
  AllAlexaAndGoogleInfo.shape =&gt; (7395, 1)
  X_all.shape =&gt; (10566, 238377)
  
  
  
  ---------------------------------------------------------------------------
  ValueError                                Traceback (most recent call last)
  &lt;ipython-input-12-2b310887b5e4&gt; in &lt;module&gt;()
       31 print "X_all.shape =&gt; " + str(X_all.shape)
       32 #X = np.column_stack((X, AllAlexaAndGoogleInfo))
  ---&gt; 33 X = np.hstack((X, AllAlexaAndGoogleInfo))
       34 sc = preprocessing.StandardScaler().fit(X)
       35 X = sc.transform(X)
  
  C:\Users\Simon\Anaconda\lib\site-packages\numpy\core\shape_base.pyc in hstack(tup)
      271     # As a special case, dimension 0 of 1-dimensional arrays is "horizontal"
      272     if arrs[0].ndim == 1:
  --&gt; 273         return _nx.concatenate(arrs, 0)
      274     else:
      275         return _nx.concatenate(arrs, 1)
  
  ValueError: all the input arrays must have same number of dimensions
  </code></pre>
  
  <p>What is causing my problem here? How can I fix this? As far as I can see I should be able to join these columns? What have I misunderstood?</p>
  
  <p>Thank you.</p>
  
  <p>Edit : </p>
  
  <p>Using the method in the answer below gets the following error : </p>
  
  <pre><code>---------------------------------------------------------------------------
  ValueError                                Traceback (most recent call last)
  &lt;ipython-input-16-640ef6dd335d&gt; in &lt;module&gt;()
  ---&gt; 36 X = np.column_stack((X, AllAlexaAndGoogleInfo))
       37 sc = preprocessing.StandardScaler().fit(X)
       38 X = sc.transform(X)
  
  C:\Users\Simon\Anaconda\lib\site-packages\numpy\lib\shape_base.pyc in column_stack(tup)
      294             arr = array(arr,copy=False,subok=True,ndmin=2).T
      295         arrays.append(arr)
  --&gt; 296     return _nx.concatenate(arrays,1)
      297 
      298 def dstack(tup):
  
  ValueError: all the input array dimensions except for the concatenation axis must match exactly
  </code></pre>
  
  <p>Interestingly, I tried to print the <code>dtype</code> of X and this worked fine : </p>
  
  <pre><code>X.dtype =&gt; float64
  </code></pre>
  
  <p>However, trying to print the dtype of <code>AllAlexaAndGoogleInfo</code> like so : </p>
  
  <pre><code>print "AllAlexaAndGoogleInfo.dtype =&gt; " + str(AllAlexaAndGoogleInfo.dtype) 
  </code></pre>
  
  <p>produces :</p>
  
  <pre><code>'DataFrame' object has no attribute 'dtype'
  </code></pre>