JSON sub for loop produces KeyError, but key exists
Question: I'm trying to add the JSON output below into a dictionary, to be saved into a
SQL database.

    
    
    {'Parkirisca': [
        {    
        'ID_Parkirisca': 2,
        'zasedenost': {
                       'Cas': '2016-10-08 13:17:00',
                       'Cas_timestamp': 1475925420,
                       'ID_ParkiriscaNC': 9,
                       'P_kratkotrajniki': 350
                      }
        }
    ]}
    

I am currently using the following code to add the value to a dictionary:

    
    
    import scraperwiki
    import json
    import requests
    import datetime
    import time
    from pprint import pprint
    
    html = requests.get("http://opendata.si/promet/parkirisca/lpt/")
    
    data = json.loads(html.text)
    
    for carpark in data['Parkirisca']:
    
        zas = carpark['zasedenost']
        free_spaces = zas.get('P_kratkotrajniki')
        last_updated = zas.get('Cas_timestamp')
    
        parking_type = carpark.get('ID_Parkirisca')
        if parking_type == "Avtomatizirano":
            is_automatic = "Yes"
        else:
            is_automatic = "No"
    
        scraped = datetime.datetime.fromtimestamp(time.time()).strftime('%Y-%m-%d %H:%M:%S')
    
        savetodb = {
        'scraped': scraped,
        'id': carpark.get("ID_Parkirisca"),
        'total_spaces': carpark.get("St_mest"),
        'free_spaces': free_spaces,
        'last_updated': last_updated,
        'is_automatic': is_automatic,
        'lon': carpark.get("KoordinataX_wgs"), 
        'lat': carpark.get("KoordinataY_wgs")
        }
    
        unique_keys = ['id']
    
        pprint savetodb
    

However when I run this, it gets stuck at `for zas in carpark["zasedenost"]`
and outputs the following error:

    
    
    Traceback (most recent call last):
    File "./code/scraper", line 17, in <module>
        for zas in carpark["zasedenost"]:
    KeyError: 'zasedenost'
    

I've been led to believe that `zas` is in fact now a string, rather than a
dictionary, but I'm new to Python and JSON, so don't know what to search for
to get a solution. I've also searched here on Stack Overflow for `KeyErrror
when key exist` questions, but they didn't help, and I believe that this might
be due to the fact that's a sub for loop.

Update: Now, when I swapped the double quotes for single quotes, I get the
following error:

    
    
    Traceback (most recent call last):
    File "./code/scraper", line 17, in <module>
        free_spaces = zas.get('P_kratkotrajniki')
    AttributeError: 'unicode' object has no attribute 'get'
    


Answer: I fixed up your code:

  1. Added required imports.
  2. Fixed the `pprint savetodb` line which isn't valid Python.
  3. Didn't try to iterate over `carpark['zasedenost']`.

I then added another `pprint` statement in the `for` loop to see what's in
`carpark` when the `KeyError` occurs. From there, the error is clear. (Not all
the elements in the array in your JSON contain the `'zasedenost'` key.)

Here's the code I used:

    
    
    import datetime
    import json
    from pprint import pprint
    import time
    
    import requests
    
    html = requests.get("http://opendata.si/promet/parkirisca/lpt/")
    
    data = json.loads(html.text)
    
    for carpark in data['Parkirisca']:
        pprint(carpark)
        zas = carpark['zasedenost']
        free_spaces = zas.get('P_kratkotrajniki')
        last_updated = zas.get('Cas_timestamp')
    
        parking_type = carpark.get('ID_Parkirisca')
        if parking_type == "Avtomatizirano":
            is_automatic = "Yes"
        else:
            is_automatic = "No"
    
        scraped = datetime.datetime.fromtimestamp(time.time()).strftime('%Y-%m-%d %H:%M:%S')
    
        savetodb = {
            'scraped': scraped,
            'id': carpark.get("ID_Parkirisca"),
            'total_spaces': carpark.get("St_mest"),
            'free_spaces': free_spaces,
            'last_updated': last_updated,
            'is_automatic': is_automatic,
            'lon': carpark.get("KoordinataX_wgs"), 
            'lat': carpark.get("KoordinataY_wgs")
        }
    
        unique_keys = ['id']
    
        pprint(savetodb)
    

And here's the output on the iteration where the `KeyError` occurs:

    
    
    {u'A_St_Mest': None,
     u'Cena_dan_Eur': None,
     u'Cena_mesecna_Eur': None,
     u'Cena_splosno': None,
     u'Cena_ura_Eur': None,
     u'ID_Parkirisca': 7,
     u'ID_ParkiriscaNC': 72,
     u'Ime': u'P+R Studenec',
     u'Invalidi_St_mest': 9,
     u'KoordinataX': 466947,
     u'KoordinataX_wgs': 14.567929171694901,
     u'KoordinataY': 101247,
     u'KoordinataY_wgs': 46.05457609543313,
     u'Opis': u'2,40 \u20ac /dan',
     u'St_mest': 187,
     u'Tip_parkirisca': None,
     u'U_delovnik': u'24 ur (ponedeljek - petek)',
     u'U_sobota': None,
     u'U_splosno': None,
     u'Upravljalec': u'JP LPT d.o.o.'}
    Traceback (most recent call last):
      File "test.py", line 14, in <module>
        zas = carpark['zasedenost']
    KeyError: 'zasedenost'
    

As you can see, the error is quite accurate. There's no key `'zasedenost'` in
the dictionary. If you look through your JSON, you'll see that's true for a
number of the elements in that array.

I'd suggest a fix, but I don't know what you want to do in the case where this
dictionary key is absent. Perhaps you want something like this:

    
    
    zas = carpark.get('zasedenost')
    if zas is not None:
        free_spaces = zas.get('P_kratkotrajniki')
        last_updated = zas.get('Cas_timestamp')
    else:
        free_spaces = None
        last_updated = None
    

