title: python xarray indexing/slicing very slow
body: <p>I'm currently processing some ocean model outputs. At each time step, it has 42*1800*3600 grid points.</p>
  
  <p>I found that the bottelneck in my program is the slicing, and calling xarray_built in method to extract the values. And what's more interesting, same syntax sometimes require a vastly differnt amount of time.</p>
  
  <pre><code>ds = xarray.open_dataset(filename, decode_times=False)
  vvel0=ds.VVEL.sel(lat=slice(-60,-20),lon=slice(0,40))/100    #in CCSM output, unit is cm/s convert to m/s
  uvel0=ds.UVEL.sel(lat=slice(-60,-20),lon=slice(0,40))/100   ## why the speed is that different? now it's regional!!
  temp0=ds.TEMP.sel(lat=slice(-60,-20),lon=slice(0,40)) #de
  </code></pre>
  
  <p>Take this for example, reading a VVEL and UVEL took ~4sec, while reading in TEMP only needed ~6ms. Without slicing, VVEL and UVEL took ~1sec, and TEMP needed 120 nanosecond.</p>
  
  <p>I always thought that, when I only input part of the full array, I need less memory, and therefore less time. It turned out, that XARRAY loads in the full array and any extra slicing takes more time. But, could somebody please explain why is reading different variables from the same netcdf file takes that different of time?</p>
  
  <p>The program is designed to extract a stepwise section, and calculate the cross-sectional heat transport, so I need to pick out either UVEL or VVEL, times that by TEMP along the section. So, it may seems that, loading in TEMP that fast is good, isn't it? </p>
  
  <p>Unfortunately, that's not the case. When I loop through about ~250 grid points along the prescribed section... </p>
  
  <pre><code># Calculate VT flux orthogonal to the chosen grid cells, which is the heat transport across GOODHOPE line
  vtflux=[]
  utflux=[]
  vap = vtflux.append
  uap = utflux.append
  #for i in range(idx_north,idx_south+1):
  for i in range(10):
      yidx=gh_yidx[i]
      xidx=gh_xidx[i]
      lon_next=ds_lon[i+1].values
      lon_current=ds_lon[i].values
      lat_next=ds_lat[i+1].values
      lat_current=ds_lat[i].values
      tt=np.squeeze(temp[:,yidx,xidx].values)  #&lt;&lt; calling values is slow
      if (lon_next&lt;lon_current) and (lat_next==lat_current):   # The condition is incorrect
          dxlon=Re*np.cos(lat_current*np.pi/180.)*0.1*np.pi/180.
          vv=np.squeeze(vvel[:,yidx,xidx].values)  
          vt=vv*tt
          vtdxdz=np.dot(vt[~np.isnan(vt)],layerdp[0:len(vt[~np.isnan(vt)])])*dxlon
          vap(vtdxdz)
          #del  vtdxdz
      elif (lon_next==lon_current) and (lat_next&lt;lat_current):
          #ut=np.array(uvel[:,gh_yidx[i],gh_xidx[i]].squeeze().values*temp[:,gh_yidx[i],gh_xidx[i]].squeeze().values) # slow
          uu=np.squeeze(uvel[:,yidx,xidx]).values  # slow
          ut=uu*tt
          utdxdz=np.dot(ut[~np.isnan(ut)],layerdp[0:len(ut[~np.isnan(ut)])])*dxlat
          uap(utdxdz) #m/s*degC*m*m ## looks fine, something wrong with the sign
          #del utdxdz
  total_trans=(np.nansum(vtflux)-np.nansum(utflux))*3996*1026/1e15
  </code></pre>
  
  <p>Especially this line:</p>
  
  <pre><code>tt=np.squeeze(temp[:,yidx,xidx].values)
  </code></pre>
  
  <p>It takes ~3.65 Sec, but now it has to be repeated for ~250 times. If I remove <code>.values</code>, then this time reduces to ~4ms. But I need to time the <code>tt</code> to <code>vt</code>, so I have to extract the values. What's weird, is that the similar expression, <code>vv=np.squeeze(vvel[:,yidx,xidx].values)</code> requires much less time, only about ~1.3ms.</p>
  
  <hr>
  
  <p>To summarize my questions:</p>
  
  <ol>
  <li>Why loading in different variables from the same netcdf file takes different amount of time? </li>
  <li>Is there a more efficient way to pick out a single column in a multidimensional array? (not necessary the xarray structure, also numpy.ndarray)</li>
  <li>Why does extracting values from Xarray structures need different amount of time, for the exact same syntax?</li>
  </ol>
  
  <p>Thank you!</p>