Manipulate string data
Question: I'm new to python and trying to create a script to modify the output of a JS
file to match what is required to send data to an API. The JS file is being
read via urllib2.

    
    
    def getPage():
        url = "http://url:port/min_day.js"
        req = urllib2.Request(url)
        response = urllib2.urlopen(req)
        return response.read()
    
    # JS Data
    # m[mi++]="19.12.12 09:30:00|1964;2121;3440;293;60"
    # m[mi++]="19.12.12 09:25:00|1911;2060;3277;293;59"
    
    # Required format for API
    # addbatchstatus.jsp?data=20121219,09:25,3277.0,1911,-1,-1,59.0,293.0;20121219,09:30,3440.0,1964,-1,-1,60.0,293.0
    

As a breakdown (Required values are bold)

m[mi++]="**19.12.12** **09:30:00** |**1964** ;2121;**3440** ;**293** ;**60** "

and need to add values of -1,-1 into the string

I've managed to get the date into the correct format and replace characters
and line breaks to make the output look as such, but I have a feeling I'm
heading down the wrong track if I need to be able to reorder this string
values. Although it looks like the order is in reverse in regards to time as
well.

    
    
    20121219,09:30:00,1964,2121,3440,293,60;20121219,09:25:00,1911,2060,3277,293,59
    

Any help would be greatly appreciated! I'm thinking along the lines of regex
might be what I need.


Answer: Here's a Regex pattern to strip out the bits you don't want

> `m\[mi\+\+\]="(?P<day>\d{2})\.(?P<month>\d{2})\.(?P<year>\d{2})
> (?P<time>[\d:]{8})\|(?P<v1>\d+);(?P<v2>\d+);(?P<v3>\d+);(?P<v4>\d+);(?P<v5>\d+).+`

and replace with

> `20\P<year>\P<month>\P<day>,\P<time>,\P<v3>,\P<v1>,-1,-1,\P<v5>,\P<v4>`

This pattern assumes that the characters before the date are constant. You can
replace `m\[mi\+\+\]="` with `[^\d]+` if you want more general handling of
that bit.

So to put this in practice in python:

    
    
    import re
    
    def getPage():
        url = "http://url:port/min_day.js"
        req = urllib2.Request(url)
        response = urllib2.urlopen(req)
        return response.read()    
    
    def repl(match):
        return '20%s%s%s,%s,%s,%s,-1,-1,%s,%s'%(match.group('year'),
                                                match.group('month'),
                                                match.group('day'),
                                                match.group('time'),
                                                match.group('v3'),
                                                match.group('v1'),
                                                match.group('v5'),
                                                match.group('v4'))
    
    pattern = re.compile(r'm\[mi\+\+\]="(?P<day>\d{2})\.(?P<month>\d{2})\.(?P<year>\d{2}) (?P<time>[\d:]{8})\|(?P<v1>\d+);(?P<v2>\d+);(?P<v3>\d+);(?P<v4>\d+);(?P<v5>\d+).+')
    
    data = [re.sub(pattern, repl, line).split(',') for line in getPage().split('\n')]
    
    # If you want to sort your data
    data = sorted(data, key=lambda x:x[0], reverse=True)
    
    # If you want to write your data back to a formatted string
    new_string = ';'.join(','.join(x) for x in data)
    
    # If you want to write it back to file
    with open('new/file.txt', 'w') as f:
        f.write(new_string)
    

Hope that helps!

