title: How to iterate over all keys of a json with nested dictionaries and lists?
body: <p>I need to modify the following json file, test.json:</p>
  
  <pre><code>{
    "install": {
      "site": {
        "acls": {
          "dns": {
            "authorized_ports": ["53:tcp", "53:udp"]
          }
        },
        "network": {
          "clusters": {
            "__ip_range_1__": {
              "dhcpstart": "__ip__",
              "dhcpend": "__ip__",
              "adminip": "__ip__"
            },
            "__ip_range_2__": {
              "dhcpstart": "__ip__",
              "dhcpend": "__ip__",
              "adminip": "__ip__"
            }
          }
        }
      }
    }
  }
  </code></pre>
  
  <p>The above is abbreviated, there are many more entries in the original file.  I have several of these files per site and thus <code>__ip_range_x__</code> is different in each file and so is every IP.  I need to add entries to each <code>__ip_range_x__</code> element.  The new entry is a dictionary of dictionaries (<code>interface_config</code> below) stored in mod.json:</p>
  
  <pre><code>{
    "path": "{install}{site}{network}{clusters}{*}",
    "install" :  {
      "site": {
        "network": {
          "clusters": {
            "__iprange": {
              "interface_config": {
                "framesize": "1500",
                "framesize_vm": "1500"
              }
            }
          }
        }
      }
    }
  }
  </code></pre>
  
  <p>I also need to add other entries in different sections of the original json file.</p>
  
  <p>Right now, I'm just trying to iterate over all the elements in test.json.  Eventually I want to build a path to each element in test.json and match it with path from mod.json to modify test.json.  However, I can't get all the elements printed in the original file.  My current code:</p>
  
  <pre><code>import json
  import pprint
  
  def traverse(d, path=None):
      if path is None:
          path = []
      for item,val  in d.iteritems():
          if isinstance(item, dict):
              for k,v in item.iteritems():
                  print k
                  traverse(v)
          elif isinstance(item, list):
              for j in item:
                  (traverse(j))
          else:
              print item
          if isinstance(val, dict):
              for k,v in val.iteritems():
                  print k
                  traverse(v)
          elif isinstance(val, list):
              for j in val:
                  (traverse(j))
  with open("test.json", "r") as jf:
      data = json.load(jf)
  traverse(data)
  </code></pre>
  
  <p>The output of the above is:</p>
  
  <pre class="lang-none prettyprint-override"><code>$ ./now.py
  install
  site
  acls
  dns
  authorized_ports
  Traceback (most recent call last):
    File "./now.py", line 51, in &lt;module&gt;
      traverse(data)
    File "./now.py", line 23, in traverse
      traverse(v)
    File "./now.py", line 23, in traverse
      traverse(v)
    File "./now.py", line 26, in traverse
      (traverse(j))
    File "./now.py", line 9, in traverse
      for item,val  in d.iteritems():
  AttributeError: 'unicode' object has no attribute 'iteritems'
  </code></pre>
  
  <p>I know my first <code>iteritems</code> call is in the wrong place but I just can't get my mind around recursion...  Any pointers appreciated.  Btw, I'm on Python 2.</p>
  
  <p><strong>EDIT</strong></p>
  
  <p>The actual json I'm trying to process is more complex than what was listed above.  Here is an edited version:</p>
  
  <pre><code>{
    "install": {
      "site": {
        "acls": {
          "dns": {
            "authorized_ports": ["53:tcp", "53:udp"]
          }
        },
        "network": {
          "clusters": {
            "__ip_range_1__": {
              "dhcpstart": "__ip__",
              "dhcpend": "__ip__",
              "adminip": "__ip__"
            },
            "__ip_range_2__": {
              "dhcpstart": "__ip__",
              "dhcpend": "__ip__",
              "adminip": "__ip__"
            }
          }
        }
      }
    }
    "config": {
      "ippool": [
        {
          "pool_name": "/ippool1",
          "pool_description": "IP Pool1",
          "ranges": [["__ip__", "__ip__"]]
        },
        {
          "pool_name": "/ippool2",
          "pool_description": "IP Pool2",
          "ranges": [["__ip__", "__ip__"]]
        }
      ],
      "storage": [
        {
          "account": "/root",
          "credentials": {
            "account": "admin",
            "service": "storage",
            "user": "admin",
            "password": "pass"
          }
        }
      ]
  
    }
  }
  </code></pre>
  
  <p>I modified Paul Panzer's answer to include lists as follows:</p>
  
  <pre><code>def traverse(d, path=[]):
      for k, v  in d.iteritems():
          yield path + [k], v
          if isinstance(v, dict):
              for k,v in traverse(v, path + [k]):
                  yield k,v
          elif isinstance(v, list):
              for k in v:
                  traverse(k, path + [])
  </code></pre>
  
  <p>However the above does not print elements inside the ippool and storage lists.  Once a list of dictionaries is encountered, it is not traversed for some reason.</p>