title: Clean fix to KeyError NoneType while parsing JSON
body: <p>I'm pretty noob and using some borrowed python code for a very simple JSON parse of tweets I pulled with tweepy. I've tried solving myself but seem to be creating more problems.</p>
  
  <p>Just a few lines in the .txt file cause problems. On the one hand, the file contains this periodically: </p>
  
  <blockquote>
    <p>{"limit":{"track":3,"timestamp_ms":"1487481419833"}}</p>
  </blockquote>
  
  <p>On the other hand, for some tweets the tweet fields are all there but seems to not be in JSON format (e.g. Sublime doesn't recognize the syntax as such).</p>
  
  <p>The result is the below code gives me a KeyError 'text'</p>
  
  <pre><code>tweets['text'] = map(lambda tweet: tweet['text'], tweets_data)
  </code></pre>
  
  <p>So I fix that by changing it to:</p>
  
  <pre><code>tweets['text'] = map(lambda tweet: tweet.get('text', None), tweets_data)
  </code></pre>
  
  <p>However, now I get a different error later in my code when use:</p>
  
  <pre><code>def word_in_text(word, text):
  word = word.lower()
  text = text.lower()
  match = re.search(word, text)
  if match:
      return True
  return False
  </code></pre>
  
  <p>I get the error: </p>
  
  <blockquote>
    <p>AttributeError: 'NoneType' object has no attribute 'lower'</p>
  </blockquote>
  
  <p>I figure there must be a cleaner, simpler fix to all this. Any suggestions? Below is the full code (minus some charts I'm printing and few more columns I created in the dataFrame).</p>
  
  <p>Many thanks, and apologies if I haven't got the proper posting etiquette.</p>
  
  <pre><code>import json
  import pandas as pd
  import matplotlib.pyplot as plt
  import re
  
  tweets_data_path = '[...trump_tweets.txt]'
  
  tweets_data = []
  tweets_file = open(tweets_data_path, "r")
  for line in tweets_file:
  try:
      tweet = json.loads(line)
      tweets_data.append(tweet)
  except:
      continue
  
  print "Number of tweets included is " + str(len(tweets_data))
  
  tweets = pd.DataFrame()
  
  tweets['text'] = map(lambda tweet: tweet.get('text', None), tweets_data)
  tweets['lang'] = map(lambda tweet: tweet.get('lang', None), tweets_data)
  tweets['country'] = map(lambda tweet: tweet.get('place', None), tweets_data)
  
  def word_in_text(word, text):
  word = word.lower()
  text = text.lower()
  match = re.search(word, text)
  if match:
      return True
  return False
  
  search_term_1 = 'America'
  search_term_2 = 'POTUS'
  search_term_3 = 'ban'
  
  tweets[search_term_1] = tweets['text'].apply(lambda tweet: word_in_text(search_term_1, tweet))
  tweets[search_term_2] = tweets['text'].apply(lambda tweet: word_in_text(search_term_2, tweet))
  tweets[search_term_3] = tweets['text'].apply(lambda tweet: word_in_text(search_term_3, tweet))
  
  print "tweets mentioning search_term_1: " + str(tweets[search_term_1].value_counts()[True])
  print "tweets mentioning search_term_2: " + str(tweets[search_term_2].value_counts()[True])
  print "tweets mentioning search_term_3: " + str(tweets[search_term_3].value_counts()[True])
  </code></pre>