How to deal with timezone adjusted "epoch" timestamps in Python
Today I discovered that we have a system that returns "epoch" timestamps, but adjusted for Pacific time. This means that depending on whether daylight savings time is in effect, these timestamps are 7 or 8 hours ahead when interpreted by most tools. These are horribly difficult to deal with as unix timestamps are assumed to be in UTC time. I spent a good deal of time banging my head against Python's datetime and pytz modules (as well as the wall). With some help from John Hopkins I found a solution:
In the following example we'll convert the Pacific "epoch" timestamp 1383000394 to a proper epoch timestamp (which is 1382975194).
First, we need a tzinfo object for Pacific time:
>>> import pytz
>>> pacific_time = pytz.timezone("America/Los_Angeles")
Next, we need to get the initial timestamp into a datetime object to work with it. Note that it's important to use utcfromtimestamp() here otherwise you'll get a localized datetime object - which will only be useful if the machine you run this on is in Pacific time:
>>> from datetime import datetime
>>> dt = datetime.utcfromtimestamp(1383000394)
>>> dt
datetime.datetime(2013, 10, 28, 22, 46, 34)
It gets a little weird from here. We need to subtract the Pacific offset from the datetime object in order to get it into an actual UTC time. To do that we can force the datetime object in UTC time and then use its built-in astimezone() method to do the conversion. I think this still leaves a 7 or 8 hour window whenever DST starts or ends where this conversion is an hour off - but it's good enough for my usage:
>>> dt = dt.replace(tzinfo=pytz.utc)
>>> dt
datetime.datetime(2013, 10, 28, 22, 46, 34, tzinfo=
Now we have a datetime object with the correct time, but claiming to be in Pacific. We can fix that by replacing the tzinfo again:
>>> dt = dt.replace(tzinfo=pytz.utc)
>>> dt
datetime.datetime(2013, 10, 28, 15, 46, 34, tzinfo=
The only thing left to do now is convert to epoch time!
>>> import calendar
>>> calendar.timegm(dt.utctimetuple())
1382975194
Voila, the timestamp we were looking for!
Much credit to John Hopkins for his code that taught me how to use datetime.replace() and astimezone(). No credit at all goes to Python's datetime module, which is sorely in need of an overhaul.