[tz] Dealing with Pre-1970 Data

[Guy Harris]

On Sep 1, 2013, at 2:43 PM, Lester Caine <lester at lsces.co.uk> wrote:

> Guy Harris wrote:
>>> Following on from that, indicating that some amendment from this base is in place and can be trusted would be useful.
>> To what sort of amendments are you referring?  LMT is LMT, i.e. "1 hour for every 15 degrees of longitude from the prime meridian, and similar treatment of minutes and seconds of longitude".  There's nothing to amend there.
> 
> The amendments are the rest of the content of the tz database other than the base LMT time.

What do you mean by "base LMT time"?  The LMT time is not a base from which the subsequent entries are derived; the subsequent entries are derived, ultimately, from what historical records we can find of *standardized* time in various regions.

In, for example, the Zone entry for America/New_York:

Zone America/New_York	-4:56:02 -	LMT	1883 Nov 18 12:03:58
			-5:00	US	E%sT	1920
			-5:00	NYC	E%sT	1942
			-5:00	US	E%sT	1946
			-5:00	NYC	E%sT	1967
			-5:00	US	E%sT

The lines that begin "-5:00" are not "based on LMT" in some sense that makes the formula used to calculate LMT an issue for them, they're based on the US railroads having adopted a proposal for standard time.  Yes, the standardized offsets for the various time zones were based on LMT for some location within the zone, but the offsets were in one-hour units, so it's not as if the precise longitude of that location, or details of how the calculation was done (or the programming language you do it in or the compiler or interpreter you use for that language), are in any way relevant.

> Whether correct or otherwise. But based on Paul's last comment it would seem that it's time simply to start again gathering all of the facts again and assume nothing in the database is right? :)

It certainly doesn't seem that way to *me*.  For the US, for example, 40 Stat. 450:

	http://www.webexhibits.org/daylightsaving/usstat.html

and subsequent laws establishing standardized time in the US are pretty clear.  That statute says, for example:

	The standard time of the first zone shall be based on the mean astronomical time of the seventy-fifth degree of longitude west from Greenwich

and 75 degrees of longitude west of Greenwich, at 1 hour for every 15 degrees, is 75/15 = 5 hours.

> The original question was 'do we need to start a second database with pre 1970 history'

Given that the current database has some pre-1970 history, I sincerely hope the original question wasn't phrased in exactly that fashion; if it was phrased that way, it's an invalid question that cannot be answered with "yes" or "no", just as, for example, asking a person who has never married "are you still married?" is asking them an invalid question that cannot be answered with "yes" or "no".

Perhaps "do we need to start a second database for which we're willing to split a tzdb zone if we discover that, prior to 1970, its history had different parts of the zone having different *standardized time* offsets from GMT/UTC?" is a better question.

> my feeling currently that this should be undertaken simply because there is obviously no interest in managing the material within the current framework? Since nobody seems to have any confidence in what is being served up prior to 1970,

People have considerably less confidence in what's being served up prior to 1970 than they have in what's served up starting in 1970; the confidence decreases the further back you go in time.  (We serve nothing useful up prior to the establishment of standardized time in a given location.)

> then should any transitions be included at all?

Yes, if we have a solid historical reference (e.g., the 1918 Standard Time Act, the Act For the repeal of the daylight-saving law:

	http://books.google.com/books?id=by8PAAAAYAAJ&pg=PA280#v=onepage&q&f=false

which explains the entries that start in 1920, and various subsequent laws turning daylight savings time on and off for the US).

If we have only "Shanks says", maybe not; as Paul explained, sometimes his claims have been proven wrong by research.

> A single LMT based offset just to make the data stable

If, for America/New_York, "LMT-based" means "based on the mean astronomical time of the seventy-fifth degree of longitude west from Greenwich", yes.  If it means anything based on LMT at any other longitude, no; -5:00 is better than anything not a multiple of one hour, unless a solid historical reference can be found to support it.

> and if we want historic information then basically we are already on our own anyway.

You've *always been* on your own for sufficiently old historical information.

> ALL I am trying to establish is if there is any demand to rectify the situation that you cut and pasted into you post on mass. What the current situation is has been quite clearly stated. That people do not want to LOOSE what history IS currently displayed has ALSO been established. So to MY mind, the next logical step is to re-assess the evidence and as a minimum give some level of confidence to that data. If that has to be outside of the existing database then so be it, and as a starting point, establishing how confident the LMT times are is essential.

If somebody has a lack of confidence about the LMT time of, for example, the seventy-fifth degree of longitude west of Greenwich, I'd *really* like to hear their reasons for it.  If they have no such lack of confidence, the *only* questions that remain about, for example, America/New_York are:

	Did its standard-time offset from GMT/UTC ever change to a value other than -5:00?

	What happened to daylight savings time in various locations within that zone?

The answer to the first question is almost certainly "no", and nobody's put forth any citations to indicate that there should be any lack of confidence about that.

The *second* question is the important one for most of the world, and *that's* where the bulk of the lack of confidence in the data base is.

> From you quoting ...
>> However, the database is not designed for and does not suffice for
>> applications requiring accurate handling of all past times everywhere,
>> as it would take far too much effort and guesswork to record all
>> details of pre-1970 civil timekeeping.
> 
> Unfortunately that effort IS required for some applications of the data

Then perhaps people who really need that level of accuracy should maintain their own database.

> so when Paul says
>> ... and for pre-1970 data most of the commentary will
>> boil down to "see Shanks", which has been demonstrated
>> to be unreliable when we've checked it.  There is little
>> reason to trust that data.  Shanks gives no sources --
>> zero -- for any of his data.
> 
> Then those of us who are looking to the past have to go back to basics and re-assess, something which I have been doing myself in the light of recent developments and now have the facts but no where to archive them :(

I think that changes to the pre-1970 entries that don't cause new tzids to be created (and for which there's more direct evidence than just a citation of Shanks) would be accepted in the database.

I suspect that changes to the pre-1970 entries that *do* cause new tzids to be created might *not* be accepted into the database; people who really need that information should perhaps, as per the above, maintain their own database.