title: How to download nasa satellite OPeNDAP data using python
body: <p>I have tried requests, pydap, urllib, and netcdf4 and keep either getting redirect errors or permission errors when trying to download the following NASA data:</p>
  
  <p>GLDAS_NOAH025SUBP_3H: GLDAS Noah Land Surface Model L4 3 Hourly 0.25 x 0.25 degree Subsetted V001 (<a href="http://disc.sci.gsfc.nasa.gov/uui/datasets/GLDAS_NOAH025SUBP_3H_V001/summary?keywords=Hydrology" rel="nofollow">http://disc.sci.gsfc.nasa.gov/uui/datasets/GLDAS_NOAH025SUBP_3H_V001/summary?keywords=Hydrology</a>)</p>
  
  <p>I am attempting to download about 50k files, here is an example of one, which works when pasted into google chrome browser (if you have proper username and password): </p>
  
  <p><a href="http://hydro1.gesdisc.eosdis.nasa.gov/daac-bin/OTF/HTTP_services.cgi?FILENAME=%2Fdata%2FGLDAS_V1%2FGLDAS_NOAH025SUBP_3H%2F2016%2F244%2FGLDAS_NOAH025SUBP_3H.A2016244.2100.001.2016256190725.grb&amp;FORMAT=TmV0Q0RGLw&amp;BBOX=-11.95%2C28.86%2C-0.62%2C40.81&amp;LABEL=GLDAS_NOAH025SUBP_3H.A2016244.2100.001.2016286201048.pss.nc&amp;SHORTNAME=GLDAS_NOAH025SUBP_3H&amp;SERVICE=SUBSET_GRIB&amp;VERSION=1.02&amp;LAYERS=AAAB&amp;DATASET_VERSION=001" rel="nofollow">http://hydro1.gesdisc.eosdis.nasa.gov/daac-bin/OTF/HTTP_services.cgi?FILENAME=%2Fdata%2FGLDAS_V1%2FGLDAS_NOAH025SUBP_3H%2F2016%2F244%2FGLDAS_NOAH025SUBP_3H.A2016244.2100.001.2016256190725.grb&amp;FORMAT=TmV0Q0RGLw&amp;BBOX=-11.95%2C28.86%2C-0.62%2C40.81&amp;LABEL=GLDAS_NOAH025SUBP_3H.A2016244.2100.001.2016286201048.pss.nc&amp;SHORTNAME=GLDAS_NOAH025SUBP_3H&amp;SERVICE=SUBSET_GRIB&amp;VERSION=1.02&amp;LAYERS=AAAB&amp;DATASET_VERSION=001</a></p>
  
  <p>Anyone have any experience getting OPeNDAP NASA data from the web using python? I am happy to provide more information if desired. </p>
  
  <p>Here is the requests attempt which gives 401 error:</p>
  
  <pre><code>import requests
  
  def httpdownload():
      '''loop through each line in the text file and open url'''
      httpfile = open(pathlist[0]+"NASAdownloadSample.txt", "r")
      for line in httpfile:
          print line 
          outname = line[-134:-122]+".hdf"
          print outname 
          username = ""
          password = "*"
          r = requests.get(line, auth=("username", "password"), stream=True)
          print r.text
          print r.status_code
          with open(pathlist[0]+outname, 'wb') as out:
               out.write(r.content)
          print outname, "finished" # keep track of progress
  </code></pre>
  
  <p>And here is the pydap example which gives redirect error:</p>
  
  <pre><code>import install_cas_client
  from pydap.client import open_url
  
  def httpdownload():
      '''loop through each line in the text file and open url'''
      username = ""
      password = ""
      httpfile = open(pathlist[0]+"NASAdownloadSample.txt", "r")
      fileone = httpfile.readline()
      filetot = fileone[:7]+username+":"+password+"@"+fileone[7:]
      print filetot
      dataset = open_url(filetot)
  </code></pre>