Extracting required Variables from Event Log file using Python
Question: [![enter image description
here](http://i.stack.imgur.com/zSGWk.png)](http://i.stack.imgur.com/zSGWk.png)

sample first row of event log file ,here i have successfully extracted
evrything apart from last key value pair which is attribute-

    
    
    {"event_type":"ActionClicked","event_timestamp":1451583172592,"arrival_timestamp":1451608731845,"event_version":"3.0",
      "application":{"app_id":"7ffa58dab3c646cea642e961ff8a8070","cognito_identity_pool_id":"us-east-1:
        4d9cf803-0487-44ec-be27-1e160d15df74","package_name":"com.think.vito","sdk":{"name":"aws-sdk-android","version":"2.2.2"}
        ,"title":"Vito","version_name":"1.0.2.1","version_code":"3"},"client":{"client_id":"438b152e-5b7c-4e99-9216-831fc15b0c07",
          "cognito_id":"us-east-1:448efb89-f382-4975-a1a1-dd8a79e1dd0c"},"device":{"locale":{"code":"en_GB","country":"GB",
            "language":"en"},"make":"samsung","model":"GT-S5312","platform":{"name":"ANDROID","version":"4.1.2"}},
      "session":{"session_id":"c15b0c07-20151231-173052586","start_timestamp":1451583052586},"attributes":{"OfferID":"20186",
        "Category":"40000","CustomerID":"304"},"metrics":{}}
    

Hello Every One ,I am trying to extract the content from Event log file as
shown in attached image .As to requirement i have to fetch `customer ID`,
`offer id`, `category` these are important variable i need to extract from the
this event log file .this is csv formatted file. i tryed with regular
expression but it is't working because you can observe format of every column
is different. As you see first row has `category` `customer id` `offer id` and
second row is totally blank in this case regular expression wont work apart
from this we have to consider we have to consider all possible condition, we
has 14000 sample.in Event log file ...#Jason # Parsing #Python #Pandas


Answer: **Edit**

The data, after your edit, now appears to be JSON data. You can still use
`literal_eval` as below, or you could use the
[`json`](https://docs.python.org/3/library/json.html#module-json) module:

    
    
    import json
    
    with open('event.log') as events:
        for line in events:
            event = json.loads(line)
            # process event dictionary
    

To access the `CustomerID`, `OfferID`, `Category` etc. you need to access the
nested dictionary associated with the key `'attributes'` in the `event`
dictionary:

    
    
    print(event['attributes']['CustomerID'])
    print(event['attributes']['OfferID'])
    print(event['attributes']['Category'])
    

If it is the case that some keys could be missing use `dict.get()` instead:

    
    
    print(event['attributes'].get('CustomerID'))
    print(event['attributes'].get('OfferID'))
    print(event['attributes'].get('Category'))
    

Now you will get `None` if the key is missing.

You can extend this principle to access other items with the dictionary.

If I understand your question you also want to create a CSV file containing
the extracted fields. You use the extracted values with `csv.DictWriter` like
this:

    
    
    import csv
    
    with open('event.log') as events, open('output.csv', 'w') as csv_file:
        fields = ['CustomerID', 'OfferID', 'Category']
        writer = csv.DictWriter(csv_file, fields)
        writer.writeheader()
        for line in events:
            event = json.loads(line)
            writer.writerow(event['attributes'])
    

`DictWriter` will simply leave fields empty when the dictionary is missing
keys.

* * *

**Original answer** The data is not in CSV format, it appears to contain
Python dictionary strings. These can be parsed into Python dictionaries using
[`ast.literal_eval()`](https://docs.python.org/3/library/ast.html#ast.literal_eval):

    
    
    from ast import literal_eval
    
    with open('event.log') as events:
        for line in events:
            event = literal_eval(line)
            # process event dictionary
    

