How to convert a set of osm files to shape files using ogr2ogr in python
Question: I strongly believe that this question is already asked but I can't find the
answer so I am placing it before you. I am having a problem while running the
script to convert osm files to shp files. The script is reading all the osm
files but just creating one shp file of the first osm file at the end instead
of converting all the osm files. I am providing the code I used below. So
please kindly help me in resolving me this.

* * *
    
    
    from xml.dom import minidom
    import os, sys
    import xml.etree.ElementTree as ET
    
    ### ruta a gdal-data C:\Program Files (x86)\PostgreSQL\9.4\gdal-data
    
    path = r"C:\Users\Administrator\Desktop\CHECKING\T2"
    
    systemOutput = 'Shp'
    
    
    print ("\n#### Execute python NY_osm2shapes")
    print ("#### MONITORING CITIES")
    print ("#### Conversor osm to shapes")
    print ("#### OSM Path: " + path)
    
    print "#### "
    """
    Modify
    Win: C:/Program Files/GDAL/gdal-data/osmconfig.ini
    Linux: /usr/share/gdal/1.11/osmconfig.ini
    report_all_ways=yes #activate lines without tag
    attributes=landuse, plots #inside [lines]
    attributes=landuse, plots #inside [multipolygons]
    """
    
    ### Check if path from argv
    try:
        if len(sys.argv) >= 2:
           print("#### Path from argv: ", sys.argv[1])
           path = sys.argv[1]
        else:
           print "#### Path set to", path
           sys.exit()
    except:
          pass
    
    
    #### Ogr config
    print "\n#### Process: osm to shapes"
    ogrOutputType = ''      #-f "Esri Shapefile"'
    ogrProjection = ''      # -t_srs EPSG:4326' #+ epsg
    ogrProjectionA = ''     #-a_srs EPSG:3827'
    ogrProjectionIn = ''    #-s_srs EPSG:3827' #-t_srs EPSG:4326
    
    ogrConfigType = ' --config OSM_USE_CUSTOM_INDEXING NO'
    
    ogr2ogr = 'ogr2ogr %s %s %s %s %s %s -overwrite %s %s %s  %s layer %s'
    
    ### Process
    for l in os.walk(path):
        archivos = l[2]
        ruta = l[0]
    
    for a in archivos:
        if a.endswith(".osm"):
            osmFile = os.path.join(ruta, a)
            folder = os.path.join(ruta, systemOutput)
            shapeFile = a[:-4]
            ogrFileOutput = " -nln " + shapeFile
            print "Archivo Shape: ", shapeFile,
            layerType = shapeFile[-1]
    
    
            if layerType=="0":
                print "\t TIPO 0: Circles"
                ogrSelectLayer = "lines"
                ogrLcoType = ' -lco SHPT=ARC'
                ogrSelect = ' -select ID_string'
    
            elif layerType == "1":
                print "\t TIPO 1: Blocks"
                ogrSelectLayer = "lines"
                ogrLcoType = ' -lco SHPT=ARC'
                ogrSelect = ' -select Land_use'
    
            elif layerType == "2":
                print "\t TIPO 2: Plots"
                ogrSelectLayer = "lines"
                ogrLcoType = ' -lco SHPT=ARC'
                ogrSelect = ' -select Plot'
    
            elif layerType == "3":
                print "\t TIPO 3: Medians"
                ogrSelectLayer = "lines"
                ogrLcoType = ' -lco SHPT=ARC'
                ogrSelect = ' -select ID_string'
    
    
            else:
                print "ELSE ERROR*"
    
    
            systemOutput = ogr2ogr % (ogrOutputType, folder, osmFile, ogrProjectionA, ogrProjectionIn, ogrProjection, ogrFileOutput, ogrLcoType, ogrConfigType, ogrSelect, ogrSelectLayer)
    
            #print ("Fichero: ", osmFile, shapeFile, layerType, ogrSelectLayer)
    
            os.system(systemOutput)
    print "End process"
    


Answer: The way you used `os.walk` returns in `archivos` all `osm` files in the last
`ruta` of the tree structure traversed. That is possibly (at least part of)
your problem, or it may be so in the future.

You have to use `os.walk` differently:

    
    
    import os, re
    
    ext_regx = '\.osm$'
    archivos = []
    
    for ruta, dirs, archs in os.walk( path ) :
        for arch in archs :
            if re.search( ext_regx, arch ) :
                archivos.append( os.path.join( ruta, arch ) )
    
    for osmFile in archivos :
        print( osmFile )
        ...
    

Now if the code inside the `for` loop does not do what you mean to, that is
another issue. I suggest you:

  1. Add `print( systemOutput )` to check that each command executed is what you intend it to be.
  2. Check that the files and dirs refered to in that command are correct.

PS: each item in `archivos` will already contain the dir part, so you have to
`split` the folder part, instead of `join`ing.

PS2: you might need to use double backslashes for dirs. Also, bear in mind
[`os.sep`](http://docs.python.org/2/library/os.html#os.sep).

