Python 2.7 - Find the number of web server hits
Question: I am trying to compute the number of hits to a web server per calendar month
(Dec, Jan, Feb, ..) by Year.

I am very new to Python so I don't even know where to begin. I suppose you
have to use some string split or regexp.

I am given a log file with the following output:

    
    
    [31/Dec/1994:23:55:08 -0700] "GET 45.html HTTP/1.0" 200 5489  
    remote - - [31/Dec/1994:23:56:55 -0700] "GET 2195.ps HTTP/1.0" 200 522318 
    remote - - [31/Dec/1994:23:59:37 -0700] "GET 957.ps HTTP/1.0" 200 122146 
    remote - - [01/Jan/1995:00:31:54 -0700] "GET index.html HTTP/1.0" 200 2797 
    remote - - [01/Jan/1995:00:31:58 -0700] "GET 2.gif HTTP/1.0" 200 2555
    remote - - [01/Jan/1995:00:32:33 -0700] "GET 3.gif HTTP/1.0" 200 36403
    remote - - [01/Jan/1995:01:39:21 -0700] "GET 20.html HTTP/1.0" 200 378
    local - - [01/Jan/1995:01:47:41 -0700] "GET index.html HTTP/1.0" 200 2797
    local - - [01/Jan/1995:01:47:49 -0700] "GET 39.html HTTP/1.0" 200 669
    local - - 
    


Answer: Splitting is more easy (IMO) than applying the regular expressions. You can
extract these values with the following function by passing in a log line:

    
    
    def year_month(s):
         try:
             discard, month, rest = s.split('/', 2)  # split twice on '/' 
         except ValueError:
             return None  # for last line without /
         year = rest.split(':', 1)[0]
         year = int(year)  # optional
         return year, month
    

You can use the returned tuple from this function as a key to some dictionary:

    
    
    d = {}
    for line in open('yourfile'):
        ym = year_month(line)
        if ym is None:
             continue
        x = d.setdefault(ym, [0])
        x[0] += 1
    for ym in sorted(d):
        print('Year: {0}, Month: {1}, Hits: {2}'.format(ym[0], ym[1], d[ym][0]))
    

If you want to use regular expressions you can replace the `year_month()` part
with:

    
    
    import re
    
    # see https://docs.python.org/2/howto/regex.html
    year_month_pattern = re.compile(r"""
    .*/
    (?P<month>.*)
    /
    (?P<year>\d*)
    :.*
    """, re.VERBOSE)
    
    def year_month(s):
        res = year_month_pattern.match(s)
        if res is None:
            return None
        try:
            return int(res.group('year')), res.group('month')
        except ValueError:
            pass
    

