Python and time zones: Fighting the beast
Time zones is not a funny subject. In an effort of finishing the time zone issues in Plone4ArtistsCalendar before a 1.1 release, I’ve now spent more than two days just on this. And I’m not sure I’m done yet. It’s getting rather annoying. If you want to know more about the time zone issue, read on:
The time zone issue is of course made up of several problems.
1. Python has no native timezone implementation.
The datetime library supports timezones, but doesn’t provide implementations. Of course, there are implementations around already, but although I know of two implementations, none of them perfect. One is a part of Gustavo Niemeyers dateutil, and the second is pytz. (There is some sort of basic timezone support in zope.app.datetimeutils too, but I haven’t found it at all useful). Since I use dateutil otherwise for it’s recurrence module, it was a natural start. However, I found some bugs (which Gustavo says he will fix in the next release). But then I ran into the next problem:
2. Posix has GMT backwards.
Dateutil assumes Posix broken GMT handling. I.e, if you ask for GMT+2, you will get a time zone that is GMT minus two hours (“That’s logic!” said Tweedledee). Only Posix does this afaik, but because of this loads of Unix software thinks it’s the right thing to do, since Posix time strings does this. In fact, most Unix implementations now come with two sets of Etc/XXX zone-definitions, one called posix/Etc and one called right/Etc. However, although dateutil is happy to use posix/Etc/GMT+2, it doesn’t want to use right/Etc/GMT+2 for some reason. And also, I don’t think we can’t count on these files existing on all Unix distributions.
Why is this a problem? Well, mainly because one of the things I need to do is to convert between Zopes DateTime object and pythons datetime objects. And of course, DateTime objects use GMT all the time. Correctly. So DateTime thinks GMT+2 is actually two hours more than GMT, can you imagine. So I need to have a set of timezones, including GMTxx that does the right thing. And dateutil doesn’t give me that.
Is pytz better? Not at all, it also gets GMT+2 backwards, and refuses to use either posix/Etc/GMT+2 or right/Etc/GMT+2. But Zope 2.11 is poised to use pytz as timezone implementation, so code already exists for pytz that sets up a bunch of correct GMT timezones. So I copied that code from Zope trunk and used it. That was that problem solved (or so I thought). But the next problem I encountered was that…
3. Time zone names are not unique.
When you tell your unix in which time zone you are, you do this typically by some UI software where you choose things like “US/Eastern” or “Europe/Paris” that will set things up for you. I’m not sure how most software actually does this, but the general effect is that there is a file called /etc/localtime that contains all the information, and an API to get that information from the OS. However, one very crucial piece of information is missing in that file: The actual location you have configured your computer to use. It contains all other possible information, but not that. This is probably the idiots at Posix that need to be spanked again. The effect is that when you are for example in Australia/Sydney, Python does not know that. All it knows is that you are in a time zone called EST, and the offset. However, there are three time zones this can be with three offsets. The Australian Eastern Standard Time, the US Eastern Standard Time and the Brazilian Eastern Standard Time (although this is usually called BRT, thankfully). Which one is it? Well, we can figure that out by going trough all the timezones, and comparing the name and the offsets. This way we at least will end up in the same time zone. So people in Australia will in fact get a UTC offset of -11 and not -5. Problem solved? Nope.
4. Locations with the same time zone may have different daylight saving.
Figuring out which EST we are in is fine if you want to figure out your current UTC offset. However, that’s not all there is to time zones. No, in calendaring you want to be able to calculate your local time for an event that happens in half a year. Is this a problem? Oh, yeah. For example if you are in Brazil/Eastern this has BRT as a standard time zone and BRST as the daylight saving time zone. But there are MANY locations that has those. Including, for example, America/Araguaina. But while Brazil observes daylight saving, Araguaina does not. So, you will get incorrect offsets for all converted times during Brazilian daylight saving.
This is however only a problem for the local time zone, as you don’t have the name for it. If you want to convert to another time zone, it can be assumed that you know to where you want to convert it. And dateutil has a clever solution there: It can wrap zoneinfo files. This means that to get the local timezone, you just wrap /etc/localtime, and you will be all right. That way you don’t have to know the name of the local zone. So, out goes pytz, and back comes dateutil. It means I have to create a bunch of GMTxx zones and also not use dateutil.tz.gettz, but my own gettz that first looks in my extra GMT-zones, but that isn’t too much work. But it does gives us a new problem:
5. Some times happen twice.
The night you switch your clock back one hour, a whole hour happens twice. There is no way to know of 1:30 that night is the first 1:30 or the second 1:30. This is written about a bit more in the pytz documentation. The problem comes from that datetime stores it’s data in the local time format. A better solution could have been to store it in UTC, but the problem with that is that there would have to make a conversion each time you use the datetime, which would slow things down. pytz solves this somehow, but it does mean you have to tell ambiguous times whether they are in DST or not. I can’t find any support for this in dateutil, although I might have missed it. And also, it has to be admitted, that this is a rather minor issue. Just don’t put meetings during a daylight saving switch, OK?😉
Although, OK. Not all problems are solved. One last problem remains:
6. What about windows!?
I have no idea if this works on Windows or not. People are welcome to try.