[tz] Adding a "test" region?

[Brian Inglis]

On 2018-01-24 15:24, Paul G wrote:
> On 01/24/2018 04:55 PM, Paul.Koning at dell.com wrote:
>> No, please do NOT do that.  Anyone who depends on an automated or semi-automated system to track real changes will get completely messed up by such a tactic.  Let the real database contain real data.  A test database is fine; a test zone in the real data is not.
> 
> Do you have examples of this? I'm not really sure what problem it would cause to have well-namespaced fictional zones in the real database. What kind of automated processes would be affected by there being some zones that don't represent real zones? I could imagine there being some applications that don't know about these test zones that erroneously display them to users, but why would users set their time zone to `TestZone/Test_Monthly_DST` or something? Doesn't seem hugely risky to me, but I would not be surprised to find that there are some applications where this could be a deal-breaker.

See Hawai'i Missile Alert technical articles or search Risks list for "test in
production" or "test data in production database" articles.

How much do you want to further delay downstream distribution of changes?
As someone else mentioned about Debian, the current release on most major
distributions is 2017c, often on ...-update or even ...-security repos to get
changes distributed!
Package maintainers and QA have enough work to do to get stable releases thru
their processes without introducing deliberate errors that get a release put on
the back burner until the *volunteers* have time to deal with broken packages,
or release a package that breaks downstream applications before it gets
detected, complained about, and reverted.

Distros are already pretty selective about what they package from a release, as
looking at the selection of files available in a few distros' tzdata binary and
source packages will show.
The non-standard (non-existent) structure of the release probably requires
package scripts to at least move each file into a directory structure acceptable
to the distro build and packaging tools, add build and packaging control files,
and/or patch distributed files to conform to their tools and processes, then get
the builds and packages to a state where they can pass their QA processes to
become officially released binary and source packages.
As a result, additions to a release are likely to be ignored until that breaks a
process or fails QA e.g. moving version from Makefile to a separate file.

QA processes are likely to involve regression testing all downstream packages
dependent on tzdata, so any change that causes a downstream package issue can
halt tzdata packaging until the issue can be reported, investigated, diagnosed,
fixed, tested, and either resolved or the test bypassed for minor issues.
Unfortunately, not all (no?) libraries use the tzcode reference implementation,
as they also have to support i18n and l14n facilities, so QA will presumably run
libc, coreutils, and other package tests across multiple locales, increasing the
possibility of regressions.

> That said, one of the problems this would be designed to solve is that apparently a lot of people are already *not* testing against the master branch or doing their own tests, since we hear about a lot of problems only *after* new releases come out. Having the default be "you don't get the test zones" would at least partially defeat the purpose of such an initiative.

As some major distributors have previously mentioned, their orgs have policies
that they can only use official releases from official sources, and do not have
either the authorization or resources to test against the experimental github
releases.
The result is that most significant complaints are made by major distributors
only after official releases.
Some users perform their own QA against the experimental github repo and they
contribute reports and patches that will avoid some problems when an official
release is generated.

>> Of course it would be possible for the tools that track changes to ignore the test entry, but that's an incompatible change.  And that also only works if the test zone(s) have rigorously defined name patterns fixed for all time.
> 
> I assume it's fine to have rigorously defined name patterns. I was thinking of something designed from the beginning to be filtered out (it could, for example, *only* generate zones in TestZones/, and/or there can be a "testzones" file that lists all the fictional zones).
> 
> Since it seems like Paul is already leaning towards moving to a 2-track
release ("main" release and "compatibility" release that is more stable with
respect to some of these features), it would make sense to distribute the test
zones *only* as part of the "main" release (and whether the `TestZone/`
region/folder is generated at all can be opt-in in `zic`, since this is mostly
intended for downstream *compilers*, not for the much more stable compiled
binaries). Presumably anyone not ready to handle the test zones can opt-out
easily enough by using the compatibility releases.

The official release must then remain the main stable compatibility release for
consumption by the major downstream distributors and users.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada