[tz] Filling the gaps ...

Tue Dec 5 14:25:52 UTC 2017

On 05/12/17 10:52, Guy Harris wrote:
> On Dec 5, 2017, at 1:31 AM, Lester Caine <lester at lsces.co.uk> wrote:
> 
>> The main problem that everybody seems to be skirting around is that the
>> tz database has NO interest in exactly where someone is located
> 
> It's obviously not the tzdb's job to determine their location, so presumably you mean "the tz database has NO interest in providing information to determine, for a given longitude and latitude, the tzdb region containing that longitude and latitude".

Actually my English was probably exactly what I meant. tzdb IS purely a
database of time change rules. While some locations seem to expect their
own personal set of rules, the reality is that many locations simply
share one set of rules.

>> Even tzdist which was the latest
>> attempt to plug a major distribution hole has no remit on supplying the
>> correct data for a particular location!
> 
> I.e., they don't offer, for example, a service where you send it a longitude/latitude pair and it returns data for the tzdb region containing that location.

The geographic lookup was certainly part of the discussion, but deemed
out of scope for RFC7808. I can't remember now what happened about the
geolocate proposals :( 4.1.8 seems to be the 'coverall' for bits that
did not fully make the RFC ...

>> Some other data source is really needed to serve up the CURRENT set of
>> rules for a location, and that is probably only currently possible using
>> geonames.org which uses the current timezone_id and it's own simplified
>> timeZone.txt table! What is missing is any historic (or ongoing) changes
>> to that information, such as a location that was using one tz rule set
>> but moved to another.
> 
> That would happen only if the tzdb region containing that location were to split, with the location being in the newly-created tzdb region (rather than remaining in the old region).
> 
> In that case, I'm not sure why the tzdb region it was in before the split is relevant; before the split, the old region and the new region have the same rules, as they weren't yet separate regions, and after the split, the rules of the old region aren't relevant to the location.

So you are not interested in maintaining accurate local time
normalization for say the 2nd world war. As I said for many people it is
irrelevant, which is why RFC7808 has the option ONLY to provide a short
version of the database. The problem comes when someone accesses data
that HAS been normalised and expects a tz look up to provide the then
current local time. Event X happened an hour before event Y because the
correct DST offset is missing in your version of tz data.

>> In addition any pre-1970 material can not be
>> identified.
> 
> That's because, as has been noted in the past, having separate tzdb regions for locations that used different rules prior to 1970 but that didn't use different rules in 1970 or afterwards isn't a high priority for the tzdb.  That's a separate issue unrelated to the "mapping locations to the tzdb region in which they're located" issue.

Pre-1970 data IS all about the location and the start 'event' is the
move from solar time to some agreed standard time. I accept that this is
'unrelated' to tzdb but it IS a major gap in the standardisation of what
we are using.

>> tzdist has the remit and tools to fill one of the holes that tz on it's
>> own can not and that is historic changes TO the rule sets.
> 
> If you're thinking of
> 
> 	https://tools.ietf.org/html/rfc7808#section-3.10
> 
> that doesn't seem to provide anything that access to a complete archive of tzdb releases would.

tzdb is one possible publisher of tz data and as such follows the
monolithic model. When a copy of that data is downloaded it will
identify the version that has been download. Because of the 'committee'
nature of defining RFC's there is as much unwritten as is written! and a
user needs to know what version of data they are looking at. The 'get'
clause that is actually used needs to be able to 'get' the version of
tzdist data that matches the version it is to be used with. This may
indeed be an alternate publisher even but there is no guarantee that the
CURRENT set of rules for a timezone actually matches the set required.

In the case of notification of short term changes to offset, the
'changedsince' value only ensure that a current set of rules have
changed, not the rules that were in place when a calender of events was
created.

> And that's an issue separate from the "mapping a location to a tzdb region" issue.

Not saying otherwise ...

>> The whole point of tzdist was to sort current day problems, so if a
>> mobile device with a stored calendar lands in a new location where there
>> has been an 'unscheduled' change of tz rules that change CAN be
>> propagated quickly without needing a full tz update but that will only
>> work if someone is actually running the primary tzdist source. Moving
>> forward, that primary tzdist source needs to maintain a full historic
>> record prior to 1970
> 
> ...assuming they wish to provide services for people processing pre-1970 local times.  Perhaps they might not wish to, disappointing though that may be for those who want to process pre-1970 local times.

The rules for a 'Root Provider' on RFC7808 is that it has to provide a
full set of tz data. It can't simply provided a truncated set of data.

>> Merging geonames.org with openstreetmap.org one gets graphic displays of
>> the area that timezones CURRENTLY cover, but one can not establish if
>> the same offsets applied back in the 1960's when DST local times are
>> well documented. And both databases throw away the historic data to
>> those mappings along with a whole load of other historic material. While
>> the vast majority of users probably could not care less, the annoyance
>> here is that the material existed so it's not as if we are asking for
>> extra work, just a mechanism that remembers that say in 2014 the Crimea
>> moved from one set of tz rules to another.
> 
> ...and what the tzdb did in response was to add some additional lines to the entry for the *existing* entry for the Europe/Simferopol tzdb region.  I don't know whether any tzdb regions changed their boundaries; if not, then, from the point of view of the tzdb, no location changed which tzdb region it was in, the *existing* rules for a tzdb region changed by adding new transitions.
> 
> That change isn't a correction to historical tzdb entries, of course, it's a consequence of the annexation of Crimea by the Russian Federation, so presumably by "it's not as if we are asking for extra work, just a mechanism that remembers that say in 2014 the Crimea moved from one set of tz rules to another", you mean "a mechanism that remembers that new entries were added to the Europe/Simferopol rules in 2014 can also be used to remember that Southeast Nowheresville's UTC offset in 1905 was corrected from 1:30 to 1:47".

This is about having a mechanism that can at the very least say "The
normalised event time I have may now be wrong". If I've created a
timetable for an international conference that has events across several
timezones, if one event site moves in time one needs to know? It would
be nice if the system also said "event X's local time has change to
XX:XX" but any decision on does the event's UTC stay fixed or does the
other sites timetable need to be adjusted to compensate IS outside the
process of updating that timezone data.

> BTW, if there's any mechanism in tzdist to request particular versions of the tzdb, or to enumerate all of the versions that have ever existed, I'm not seeing it anywhere in RFC 7808, so either I missed it or it was added subsequently.

The problem is when a majority of contributors are ONLY concerned with a
current set of working offsets some things are very difficult to get
accepted. Does ANY RFC actually nail down all of the rules? In an ideal
world tzdist would only distribute tzdb data and updates, but that was
not acceptable to all, so the multiple publishers arrived. They are
going to define different timezone id's, hence the need when publishing
an event calendar to tell your users just who's id's you are using and
hence who's version of the data ... and track changes form that ...

Is this why nobody has picked up the baton and started actually using
tzdist. tzdb has a lot of limitations, but at least it IS a single set
of data!

-- 
Lester Caine - G8HFL
-----------------------------
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk