Converting timezone-aware date string to UTC and back in Python
Question: I'm parsing the National Weather Service alerts feed into a web application.
I'd like to purge the alerts when they hit their expiration time. I'd also
like to display the expiration time in the local time format for the
geographic area they pertain to.

The alerts cover the whole US, so I think the best approach is to store and
compare the times in UTC timestamps. The expiration time arrives in the feed
as a string like this: `2011-09-09T22:12:00-04:00`.

I'm using the _Labix dateutils_ package to parse the string in a timezone-
aware manner:

    
    
    >>> from dateutil.parser import parse
    >>> d = parse("2011-09-18T15:52:00-04:00")
    >>> d
    datetime.datetime(2011, 9, 18, 15, 52, tzinfo=tzoffset(None, -14400))
    

I'm also able to capture the UTC offset in hours:

    
    
    >>> offset_hours = (d.utcoffset().days * 86400 + d.utcoffset().seconds) / 3600
    >>> offset_hours
    -4
    

Using the `datetime.utctimetuple()` and `time.mktime()` methods, I'm able to
convert the parsed date to a UTC timestamp:

    
    
    >>> import time
    >>> expiration_utc_ts = time.mktime(d.utctimetuple())
    >>> expiration_utc_ts
    1316393520.0
    

At this point, I feel pretty good that I'm able to convert the raw strings
into a timestamp representing the expiration time in UTC. I'm able to compare
the current time as a UTC timestamp to the expiration and determine if it
needs to be purged:

    
    
    >>> now_utc_ts = time.mktime(time.gmtime())
    >>> now_utc_ts
    1316398744.0
    >>> now_utc_ts >= expiration_tc_ts
    True
    

The difficulty I'm having is trying to convert my stored UTC timestamp back to
the original localized format. I have the offset hours stored from the
original conversion and a string I parsed to store the timezone label:

    
    
    >>> print offset_hours
    -4
    >>> print timezone
    EDT
    

**I'd like to convert the UTC timestamp back to a locally formatted time, but
converting it back to a`datetime` doesn't seem to be working:**

    
    
    >>> import datetime
    >>> datetime.datetime.fromtimestamp(expiration_utc_ts) + datetime.timedelta(hours=offset_hours)
    datetime.datetime(2011, 9, 18, 16, 52) # The hour is 16 but it should be 15
    

It looks like it's off by an hour. I'm not sure where the error was
introduced? I put together another test and got similar results:

    
    
    >>> # Running this at 21:29pm EDT
    >>> utc_now = datetime.datetime.utcnow()
    >>> utc_now_ts = time.mktime(right_now.utctimetuple())
    >>> datetime.datetime.fromtimestamp(utc_now_ts)
    datetime.datetime(2011, 9, 18, 22, 29, 47) # Off by 1 hour
    

Can someone help me find my mistake? I'm not sure if it's a daylight savings
issue? I came across some stuff that leads me to believe it might be trying to
localize my dates and times but at this point I'm pretty stumped. I was hoping
to do all of these calculations/comparisons in a timezone-agnostic manner.


Answer: The problem is that Daylight Savings time is being applied twice.

A trivial example:

    
    
    >>> time_tuple = datetime(2011,3,13,2,1,1).utctimetuple()
    time.struct_time(tm_year=2011, tm_mon=3, tm_mday=13, tm_hour=2, tm_min=1, tm_sec=1, tm_wday=6, tm_yday=72, tm_isdst=0)
    >>> datetime.fromtimestamp(time.mktime(time_tuple))
    datetime.datetime(2011, 3, 13, 3, 1, 1)
    

I am fairly certain that the fault lies within `time.mktime()`. As it says in
its [documentation](http://docs.python.org/library/time.html#time.mktime):

> This is the inverse function of localtime(). Its argument is the struct_time
> or full 9-tuple (since the dst flag is needed; use -1 as the dst flag if it
> is unknown) which expresses the time in local time, not UTC. It returns a
> floating point number, for compatibility with time(). If the input value
> cannot be represented as a valid time, either OverflowError or ValueError
> will be raised (which depends on whether the invalid value is caught by
> Python or the underlying C libraries). The earliest date for which it can
> generate a time is platform-dependent.

When you pass a time tuple to `time.mktime()`, it expects a flag on whether
the time is _in daylight savings time_ or not. As you can see above,
`utctimetuple()` returns a tuple with that flag marked `0`, as it says it will
do in its [documentation](http://docs.python.org/library/datetime.html):

> If datetime instance d is naive, this is the same as d.timetuple() except
> that tm_isdst is forced to 0 regardless of what d.dst() returns. DST is
> never in effect for a UTC time.
>
> If d is aware, d is normalized to UTC time, by subtracting d.utcoffset(),
> and a time.struct_time for the normalized time is returned. tm_isdst is
> forced to 0. Note that the result’s tm_year member may be MINYEAR-1 or
> MAXYEAR+1, if d.year was MINYEAR or MAXYEAR and UTC adjustment spills over a
> year boundary.

Since you have told `time.mktime()` that your time is not DST, and its job is
to convert all times into local time, and it is currently daylight savings
time in your area, it **adds an hour** to make it daylight savings time. Hence
the result.

* * *

While I don't have the post handy, I came across a method a couple of days ago
to convert timezone-aware datetimes into naive ones in your local time. This
might work much better for your application than what you are currently doing
(uses the excellent [pytz](http://pytz.sourceforge.net/) module):

    
    
    import pytz
    def convert_to_local_time(dt_aware):
        tz = pytz.timezone('America/Los_Angeles') # Replace this with your time zone string
        dt_my_tz = dt_aware.astimezone(tz)
        dt_naive = dt_my_tz.replace(tzinfo=None)
        return dt_naive
    

Replace 'America/LosAngeles' with your own timezone string, which you can find
somewhere in `pytz.all_timezones`.

