From eggert at cs.ucla.edu Thu Jun 1 00:08:54 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Wed, 31 May 2023 17:08:54 -0700 Subject: [tz] Tzdb build issue In-Reply-To: References: Message-ID: On 2023-05-31 13:14, Guy Harris via tz wrote: > I was able to build on Ubuntu 14.10 with GCC 4.9.1 by adding -DPORT_TO_C89 to the end of the "CFLAGS=" line in Makefile before running make; it didn't build unless I did that, as GCC 4.9.1 didn't recognize "restrict" as a keyword. says the 'restrict' keyword is supported by GCC 2.95 (1999) and later. GCC 4 and earlier, though, need an option like '-std=gnu11' to enable 'restrict', as these GCC versions default to C89 and 'restrict' is incompatible with C89. Come to think of it, it's probably better to use -std=gnu11 than -DPORT_TO_C89 on Naveen's platform. Something like this, say: make -j6 zic=zic cc=arm-linux-gnueabi-gcc \ ZDUMPDIR=/usr/sbin ZFLAGS="-b slim" \ CFLAGS="-std=gnu11 -Os -g -pipe" I installed the attached commentary patch into TZDB, to try to help others in Naveen's shoes. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Prefer-std-gnu11-to-DPORT_TO_C89.patch Type: text/x-patch Size: 1053 bytes Desc: not available URL: From Brian.Inglis at Shaw.ca Thu Jun 1 00:39:58 2023 From: Brian.Inglis at Shaw.ca (Brian Inglis) Date: Wed, 31 May 2023 18:39:58 -0600 Subject: [tz] [PROPOSED] Update tzselect comment In-Reply-To: <20230531180324.52614-1-eggert@cs.ucla.edu> References: <20230531180324.52614-1-eggert@cs.ucla.edu> Message-ID: Announced 2019-09-17: https://blogs.oracle.com/solaris/post/great-news-about-extended-support-for-oracle-solaris-10-os Note: Sustaining Support is still available after 2025-01. On 2023-05-31 12:03, Paul Eggert via tz wrote: > * tzselect.ksh: Update comment, as Oracle has extended the > end-of-life date for Solaris 10. > --- > tzselect.ksh | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tzselect.ksh b/tzselect.ksh > index 9a91acfc..98774c03 100644 > --- a/tzselect.ksh > +++ b/tzselect.ksh > @@ -21,7 +21,7 @@ REPORT_BUGS_TO=tz at iana.org > # MirBSD Korn Shell > # > # For portability to Solaris 10 /bin/sh (supported by Oracle through > -# January 2024) this script avoids some POSIX features and common > +# January 2025) this script avoids some POSIX features and common > # extensions, such as $(...) (which works sometimes but not others), > # $((...)), ! CMD, ${#ID}, ${ID##PAT}, ${ID%%PAT}, and $10. -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien ? ajouter not when there is no more to add mais lorsqu'il n'y a plus rien ? retirer but when there is no more to cut -- Antoine de Saint-Exup?ry From gharris at sonic.net Thu Jun 1 08:55:01 2023 From: gharris at sonic.net (Guy Harris) Date: Thu, 1 Jun 2023 01:55:01 -0700 Subject: [tz] Tzdb build issue In-Reply-To: References: Message-ID: <491475F1-69BC-4FFA-9EF4-E8A3C49C1342@sonic.net> On Jun 1, 2023, at 1:12 AM, Naveen kumar wrote: > I added the -std=gnu11 options also it's shows same error. So you added it to the CFLAGS= line in the Makefile and it still complains about "restrict"? If so, what happens if you add "-std=gnu99" instead? Or you ran make CFLAGS="-std=gnu11" ... and it still complains about "restrict"? If so, what happens if you use "-std=gnu99" instead? From navinagaraj18 at gmail.com Thu Jun 1 08:12:13 2023 From: navinagaraj18 at gmail.com (Naveen kumar) Date: Thu, 1 Jun 2023 13:42:13 +0530 Subject: [tz] Tzdb build issue In-Reply-To: References: Message-ID: Hi , I added the -std=gnu11 options also it's shows same error. On Thu, 1 Jun, 2023, 5:38 am Paul Eggert, wrote: > On 2023-05-31 13:14, Guy Harris via tz wrote: > > I was able to build on Ubuntu 14.10 with GCC 4.9.1 by adding > -DPORT_TO_C89 to the end of the "CFLAGS=" line in Makefile before running > make; it didn't build unless I did that, as GCC 4.9.1 didn't recognize > "restrict" as a keyword. > > says the 'restrict' keyword is > supported by GCC 2.95 (1999) and later. GCC 4 and earlier, though, need > an option like '-std=gnu11' to enable 'restrict', as these GCC versions > default to C89 and 'restrict' is incompatible with C89. > > Come to think of it, it's probably better to use -std=gnu11 than > -DPORT_TO_C89 on Naveen's platform. Something like this, say: > > make -j6 zic=zic cc=arm-linux-gnueabi-gcc \ > ZDUMPDIR=/usr/sbin ZFLAGS="-b slim" \ > CFLAGS="-std=gnu11 -Os -g -pipe" > > I installed the attached commentary patch into TZDB, to try to help > others in Naveen's shoes. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From navinagaraj18 at gmail.com Thu Jun 1 10:31:26 2023 From: navinagaraj18 at gmail.com (Naveen kumar) Date: Thu, 1 Jun 2023 16:01:26 +0530 Subject: [tz] Tzdb build issue In-Reply-To: <491475F1-69BC-4FFA-9EF4-E8A3C49C1342@sonic.net> References: <491475F1-69BC-4FFA-9EF4-E8A3C49C1342@sonic.net> Message-ID: Hi Team, Now it's working fine with c11 standard. Thanks & Regards, Naveen. On Thu, 1 Jun, 2023, 2:25 pm Guy Harris, wrote: > On Jun 1, 2023, at 1:12 AM, Naveen kumar wrote: > > > I added the -std=gnu11 options also it's shows same error. > > So you added it to the CFLAGS= line in the Makefile and it still complains > about "restrict"? If so, what happens if you add "-std=gnu99" instead? > > Or you ran > > make CFLAGS="-std=gnu11" ... > > and it still complains about "restrict"? If so, what happens if you use > "-std=gnu99" instead? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From m.17768 at gmail.com Fri Jun 2 13:04:36 2023 From: m.17768 at gmail.com (Mohammad Ahmadian) Date: Fri, 2 Jun 2023 16:34:36 +0330 Subject: [tz] Iran Time Zone is Incorrect in Your Database Message-ID: Hello, Thanks for your service. This year, the Iran government decided to cancel the daylight time zone. So they did not change the time zone to 04:30. It is now 03:30. I updated my tablet via this application to the latest version of your database successfully: https://github.com/meefik/tzupdater But the time zone is wrongly 04:30 again. Please change your database. Let me know what you think and do. Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From eggert at cs.ucla.edu Fri Jun 2 22:12:24 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Fri, 2 Jun 2023 15:12:24 -0700 Subject: [tz] Iran Time Zone is Incorrect in Your Database In-Reply-To: References: Message-ID: <616f55e1-99a0-9af9-fad7-06b381a643e8@cs.ucla.edu> On 6/2/23 06:04, Mohammad Ahmadian via tz wrote: > This year, the Iran government decided to cancel the daylight time zone. So > they did not change the time zone to 04:30. It is now 03:30. I updated my > tablet via this application to the latest version of your database > successfully:https://github.com/meefik/tzupdater Yes, last year we were informed about that change for Iran, and the change appeared in tzdb 2022b, released 2022-08-10. See: https://mm.icann.org/pipermail/tz-announce/2022-August/000071.html I wasn't aware of meefik/tzupdater until today, and I don't know how it works or what's going wrong with your update. You might ask its author for help. From eggert at cs.ucla.edu Tue Jun 20 06:08:54 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Mon, 19 Jun 2023 23:08:54 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: Message-ID: On 2023-06-15 10:03, enh wrote: > any thoughts on either renaming `struct state` (i went with `struct > __timezone_t` to match the typedef to timezone_t for the pointer) or having > a knob for that (i added a `#define state __timezone_t` in the `#if > NETBSD_INSPIRED` in private.h)? why do i care? because tzcode pulls in the > system , which is actually fine for the tzcode code, but breaks the > OpenBSD wcsftime(). We could rename 'struct state', but I'm a bit lost as to what the problem is, as I don't see mention of 'state' or 'timezone_t' in openbsd/lib/libc/time/wcsftime.c. Although there is a 'struct state' in openbsd/lib/libc/time/localtime.c I assume you're replacing that with tzcode's localtime.c somehow. User apps (which tzcode is, by default) shouldn't define symbols like __timezone_t because these symbols are reserved to the system. We could easily rename 'struct state' to 'struct timezone_struct' so that you could compile with -Dtimezone_struct=__timezone_t, though I'm still curious about why 'struct state' causes problems. I'll cc this to the tz mailing list to see whether other people there have insight into this issue. For those new to the discussion, the original thread is archived here: https://www.openwall.com/lists/libc-coord/2023/06/14/1 From webfirmaet at gmail.com Sat Jun 17 18:20:53 2023 From: webfirmaet at gmail.com (Anders Rosendahl) Date: Sat, 17 Jun 2023 20:20:53 +0200 Subject: [tz] Error in Scandinavian tz data Message-ID: Hello! I'm developing an application using the JavaScript Intl API (which is relying on the IANA tz db) for converting between local time and UTC. I started receiving error reports from users about wrong calculations (particularly from Denmark in the 1940's), and I discovered that in some cases daylight saving time was applied in a way not in accordance to historical facts. I investigated the IANA tz database files and I discovered that a change has happened in version 2022b (in the file "europe"). Compared to the earlier version (2022a), 4 zones seem to have their rules removed and instead linked to Europe/Berlin: Link Europe/Berlin Arctic/Longyearbyen Link Europe/Berlin Europe/Copenhagen Link Europe/Berlin Europe/Oslo Link Europe/Berlin Europe/Stockholm This, evidently, is not correct, and will result in erroneous results for these zones. Best wishes, Anders Rosendahl -------------- next part -------------- An HTML attachment was scrubbed... URL: From enh at google.com Tue Jun 20 21:00:29 2023 From: enh at google.com (enh) Date: Tue, 20 Jun 2023 14:00:29 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: Message-ID: On Mon, Jun 19, 2023 at 11:08?PM Paul Eggert wrote: > On 2023-06-15 10:03, enh wrote: > > any thoughts on either renaming `struct state` (i went with `struct > > __timezone_t` to match the typedef to timezone_t for the pointer) or > having > > a knob for that (i added a `#define state __timezone_t` in the `#if > > NETBSD_INSPIRED` in private.h)? why do i care? because tzcode pulls in > the > > system , which is actually fine for the tzcode code, but breaks > the > > OpenBSD wcsftime(). > > We could rename 'struct state', but I'm a bit lost as to what the > problem is, as I don't see mention of 'state' or 'timezone_t' in > openbsd/lib/libc/time/wcsftime.c. Although there is a 'struct state' in > openbsd/lib/libc/time/localtime.c I assume you're replacing that with > tzcode's localtime.c somehow. > it's actually the openbsd strptime.c that's unhappy: ``` In file included from bionic/libc/tzcode/strptime.c:40: bionic/libc/tzcode/private.h:790:23: error: typedef redefinition with different types ('struct state *' vs 'struct __timezone_t *') typedef struct state *timezone_t; ^ bionic/libc/include/time.h:46:30: note: previous definition is here typedef struct __timezone_t* timezone_t; ^ 1 error generated. ``` the trick in private.h assumes that it's #included _before_ , but that's not true in that file. (since we already have local changes in that file for gnu extensions that openbsd wasn't interested in, i've just reordered the #includes.) User apps (which tzcode is, by default) shouldn't define symbols like > __timezone_t because these symbols are reserved to the system. We could > easily rename 'struct state' to 'struct timezone_struct' so that you > could compile with -Dtimezone_struct=__timezone_t, though I'm still > curious about why 'struct state' causes problems. > true; since tzcode _is_ the implementation for 3.5+ billion devices, i forget that others might not consider it "the implementation" :-) > I'll cc this to the tz mailing list to see whether other people there > have insight into this issue. For those new to the discussion, the > original thread is archived here: > > https://www.openwall.com/lists/libc-coord/2023/06/14/1 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From PaulGBoulder at AIM.com Wed Jun 21 02:16:23 2023 From: PaulGBoulder at AIM.com (Paul Gilmartin) Date: Tue, 20 Jun 2023 20:16:23 -0600 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: Message-ID: <306f3fe4-1152-2fb6-608d-1994d746dcf1@AIM.com> ( the Archives of this thread seem incomplete.) On 6/20/23 15:00:29, enh via tz wrote: > ... > We could rename 'struct state', but I'm a bit lost as to what the > problem is, as I don't see mention of 'state' or 'timezone_t' in > openbsd/lib/libc/time/wcsftime.c. Although there is a 'struct state' in > openbsd/lib/libc/time/localtime.c I assume you're replacing that with > tzcode's localtime.c somehow. (Replying mostly to the Subject:) A thread-safe localtime() seems impractical. localtime() depends on the environment variable TZ. If concurrent threads set different values of TZ the results will be unpredictable. The same applies to localtime_r(). -- gil From gharris at sonic.net Wed Jun 21 02:52:25 2023 From: gharris at sonic.net (Guy Harris) Date: Tue, 20 Jun 2023 19:52:25 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <306f3fe4-1152-2fb6-608d-1994d746dcf1@AIM.com> References: <306f3fe4-1152-2fb6-608d-1994d746dcf1@AIM.com> Message-ID: <2B79184C-2633-43DD-B5DC-DF4572D2B326@sonic.net> On Jun 20, 2023, at 7:16 PM, Paul Gilmartin via tz wrote: > ( the Archives of this thread seem incomplete.) tz mailing list archives, or lib-coord: https://www.openwall.com/lists/libc-coord/2020/01/30/1 archives? The former are incomplete because the thread didn't start on the tz mailing list; the first message, to libc-coord, was linked from Paul Eggert's message to tz in this thread: https://www.openwall.com/lists/libc-coord/2023/06/14/1 > On 6/20/23 15:00:29, enh via tz wrote: >> ... >> We could rename 'struct state', but I'm a bit lost as to what the >> problem is, as I don't see mention of 'state' or 'timezone_t' in >> openbsd/lib/libc/time/wcsftime.c. Although there is a 'struct state' in >> openbsd/lib/libc/time/localtime.c I assume you're replacing that with >> tzcode's localtime.c somehow. > > (Replying mostly to the Subject:) > A thread-safe localtime() seems impractical. localtime() depends > on the environment variable TZ. If concurrent threads set > different values of TZ the results will be unpredictable. The > same applies to localtime_r(). If concurrent thread set different values of TZ, perhaps the concurrent threads should use a reentrant localtime_z() routine, possibly with a different name, as has been proposed in a number of places, including in this list back in 2001: http://mm.icann.org/pipermail/tz/2001-June/011636.html That requires making such a routine common on multiple platforms, of course.... And, in fact, in the first message in the thread, the author said is anyone aware of any standardization work in this area? i know netbsd has tzalloc()/localtime_rz()/tzfree() (and tzcode has implementations), but they're the only ones who've shipped anything, right? and there's no in-progress work on any alternative? So the proposal there is for localtime_rz(), taking a "time zone" structure, or pointer to same, provided by tzalloc() and freed by jtzfree(), rather than something that looks at TZ. (Perhaps the title was a bit misleading, and should have said "lcoaltime() equivalent".) From gharris at sonic.net Wed Jun 21 06:27:58 2023 From: gharris at sonic.net (Guy Harris) Date: Tue, 20 Jun 2023 23:27:58 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: Message-ID: <53EB31E0-4AFB-4089-BFF8-27D27455431E@sonic.net> On Jun 20, 2023, at 2:00 PM, enh via tz wrote: > it's actually the openbsd strptime.c that's unhappy: > ``` > In file included from bionic/libc/tzcode/strptime.c:40: I see the string "bionic" rather than "openbsd" there, so that appears to be *Bionic's* strptime.c that's unhappy. Is there a requirement that Bionic's strptime.c be unchanged from OpenBSD's? (BTW, is there some software using Markdown involved here? the ``` won't change the formatting to fixed format in plain-text or HTML email; in the former, what you see is pretty much what you get, and in the latter, you'd need whatever tags ``` turns into.) > (since we already have local changes in that file for gnu extensions that openbsd wasn't interested in, i've just reordered the #includes.) Exactly. From gharris at sonic.net Wed Jun 21 07:04:47 2023 From: gharris at sonic.net (Guy Harris) Date: Wed, 21 Jun 2023 00:04:47 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <53EB31E0-4AFB-4089-BFF8-27D27455431E@sonic.net> References: <53EB31E0-4AFB-4089-BFF8-27D27455431E@sonic.net> Message-ID: <177D39A7-F50F-4DD1-84D1-244E7888D439@sonic.net> On Jun 20, 2023, at 11:27 PM, Guy Harris via tz wrote: > On Jun 20, 2023, at 2:00 PM, enh via tz wrote: > >> it's actually the openbsd strptime.c that's unhappy: >> ``` >> In file included from bionic/libc/tzcode/strptime.c:40: > > I see the string "bionic" rather than "openbsd" there, so that appears to be *Bionic's* strptime.c that's unhappy. Is there a requirement that Bionic's strptime.c be unchanged from OpenBSD's? In fact, the openbsd strptime.c, as of CVS revision 1.32: https://cvsweb.openbsd.org/src/lib/libc/time/strptime.c?rev=1.31&content-type=text/x-cvsweb-markup does *not* contain the line typedef struct state *timezone_t; so it appears that it's the *Bionic* strptime.c that's unhappy. From gharris at sonic.net Wed Jun 21 07:54:37 2023 From: gharris at sonic.net (Guy Harris) Date: Wed, 21 Jun 2023 00:54:37 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: Message-ID: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> From an earlier message of yours: https://www.openwall.com/lists/libc-coord/2023/06/15/2 sent only to libc-coord: > writing some basic unit tests for these functions went fine, though it was > sad that i couldn't use `std::unique_ptr > seoul{tzalloc("Asia/Seoul"), tzfree}` because the size of timezone_t isn't > known. i'd be tempted to further mangle this so i have a public struct > containing a large-enough char[], but i worry about just how large to make > it to be safe from future tzcode changes. (and on the other hand, do i > think that tzalloc() is going to be used frequently enough to warrant the > effect? probably not?) Is there some particular reason to make timezone_t an API/ABI structure, rather than a pointer to an opaque structure? Presumably if it's a structure, "tzalloc()" really means "given a tzid, fill in a structure as appropriate for that tzid, and return that structure", not "allocate a structure, fill it in as appropriate for the tzid, and return a pointer to that structure", and "tzfree()" means "free up anything pointed to by the structure whose value was passed to it" rather than "free everything pointed to by the structure pointed to by the argument and then free the structure". If you want to do that, perhaps names that don't imply that they allocate and free a timezone_t should be chosen. From enh at google.com Wed Jun 21 19:44:26 2023 From: enh at google.com (enh) Date: Wed, 21 Jun 2023 12:44:26 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <177D39A7-F50F-4DD1-84D1-244E7888D439@sonic.net> References: <53EB31E0-4AFB-4089-BFF8-27D27455431E@sonic.net> <177D39A7-F50F-4DD1-84D1-244E7888D439@sonic.net> Message-ID: On Wed, Jun 21, 2023 at 12:05?AM Guy Harris wrote: > On Jun 20, 2023, at 11:27 PM, Guy Harris via tz wrote: > > > On Jun 20, 2023, at 2:00 PM, enh via tz wrote: > > > >> it's actually the openbsd strptime.c that's unhappy: > >> ``` > >> In file included from bionic/libc/tzcode/strptime.c:40: > > > > I see the string "bionic" rather than "openbsd" there, so that appears > to be *Bionic's* strptime.c that's unhappy. Is there a requirement that > Bionic's strptime.c be unchanged from OpenBSD's? > > In fact, the openbsd strptime.c, as of CVS revision 1.32: > > > https://cvsweb.openbsd.org/src/lib/libc/time/strptime.c?rev=1.31&content-type=text/x-cvsweb-markup > > does *not* contain the line > > typedef struct state *timezone_t; > > so it appears that it's the *Bionic* strptime.c that's unhappy. (this is answered on the other thread: it's tz's "private.h" that contains that. i'm afraid without reading _both_ threads, the tz@ thread is going to be confusing.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From enh at google.com Wed Jun 21 19:50:09 2023 From: enh at google.com (enh) Date: Wed, 21 Jun 2023 12:50:09 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> Message-ID: On Wed, Jun 21, 2023 at 12:54?AM Guy Harris wrote: > From an earlier message of yours: > > https://www.openwall.com/lists/libc-coord/2023/06/15/2 > > sent only to libc-coord: > > > writing some basic unit tests for these functions went fine, though it > was > > sad that i couldn't use `std::unique_ptr > > seoul{tzalloc("Asia/Seoul"), tzfree}` because the size of timezone_t > isn't > > known. i'd be tempted to further mangle this so i have a public struct > > containing a large-enough char[], but i worry about just how large to > make > > it to be safe from future tzcode changes. (and on the other hand, do i > > think that tzalloc() is going to be used frequently enough to warrant the > > effect? probably not?) > > Is there some particular reason to make timezone_t an API/ABI structure, > rather than a pointer to an opaque structure? > no, my preference would be for timezone_t to be more like DIR... typedef struct DIR DIR; means you need to write the * in signatures, but that std::unique_ptr-style safe usage "just works" without having to talk about the underlying struct or use std::remove_pointer. > Presumably if it's a structure, "tzalloc()" really means "given a tzid, > fill in a structure as appropriate for that tzid, and return that > structure", not "allocate a structure, fill it in as appropriate for the > tzid, and return a pointer to that structure", and "tzfree()" means "free > up anything pointed to by the structure whose value was passed to it" > rather than "free everything pointed to by the structure pointed to by the > argument and then free the structure". If you want to do that, perhaps > names that don't imply that they allocate and free a timezone_t should be > chosen. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gharris at sonic.net Wed Jun 21 20:08:40 2023 From: gharris at sonic.net (Guy Harris) Date: Wed, 21 Jun 2023 13:08:40 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> Message-ID: <607AA65B-ECDF-473B-8B4E-8A99B56B65F1@sonic.net> On Jun 21, 2023, at 12:50 PM, enh wrote: > On Wed, Jun 21, 2023 at 12:54?AM Guy Harris wrote: > >> Is there some particular reason to make timezone_t an API/ABI structure, rather than a pointer to an opaque structure? > > no, my preference would be for timezone_t to be more like DIR... > > typedef struct DIR DIR; Like DIR, which, at least in macOS (and thus probably in most *BSDs), is fully defined in , or like pcap_t, which is "defined" as an opaque structure in and only defined fully in a header internal to libpcap? (And, yes, pcap_t's contents are subject to change over time, have been changed over time, and will continue to change over time. > means you need to write the * in signatures, but that std::unique_ptr-style safe usage "just works" without having to talk about the underlying struct or use std::remove_pointer. Does that work if the structure is opaque outside the routines that process tz files? > Presumably if it's a structure, "tzalloc()" really means "given a tzid, fill in a structure as appropriate for that tzid, and return that structure", not "allocate a structure, fill it in as appropriate for the tzid, and return a pointer to that structure", and "tzfree()" means "free up anything pointed to by the structure whose value was passed to it" rather than "free everything pointed to by the structure pointed to by the argument and then free the structure". If you want to do that, perhaps names that don't imply that they allocate and free a timezone_t should be chosen. I.e., it would be timezone_t *tzalloc(const char *tzid); and void tzfree(timezone_t *); From enh at google.com Wed Jun 21 20:17:57 2023 From: enh at google.com (enh) Date: Wed, 21 Jun 2023 13:17:57 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <607AA65B-ECDF-473B-8B4E-8A99B56B65F1@sonic.net> References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> <607AA65B-ECDF-473B-8B4E-8A99B56B65F1@sonic.net> Message-ID: On Wed, Jun 21, 2023 at 1:08?PM Guy Harris wrote: > On Jun 21, 2023, at 12:50 PM, enh wrote: > > > On Wed, Jun 21, 2023 at 12:54?AM Guy Harris wrote: > > > >> Is there some particular reason to make timezone_t an API/ABI > structure, rather than a pointer to an opaque structure? > > > > no, my preference would be for timezone_t to be more like DIR... > > > > typedef struct DIR DIR; > > Like DIR, which, at least in macOS (and thus probably in most *BSDs), is > fully defined in , or like pcap_t, which is "defined" as an opaque > structure in and only defined fully in a header internal to > libpcap? (And, yes, pcap_t's contents are subject to change over time, > have been changed over time, and will continue to change over time. > NetBSD exposes its `struct _dirdesc` but FreeBSD/OpenBSD don't (though they use the same name for the private struct). somewhat unusually, bionic didn't start from any of the BSD's , and uses exact the line i showed above, with the definition only in dirent.cpp itself. > > means you need to write the * in signatures, but that > std::unique_ptr-style safe usage "just works" without having to talk about > the underlying struct or use std::remove_pointer. > > Does that work if the structure is opaque outside the routines that > process tz files? > > > Presumably if it's a structure, "tzalloc()" really means "given a tzid, > fill in a structure as appropriate for that tzid, and return that > structure", not "allocate a structure, fill it in as appropriate for the > tzid, and return a pointer to that structure", and "tzfree()" means "free > up anything pointed to by the structure whose value was passed to it" > rather than "free everything pointed to by the structure pointed to by the > argument and then free the structure". If you want to do that, perhaps > names that don't imply that they allocate and free a timezone_t should be > chosen. > > I.e., it would be > > timezone_t *tzalloc(const char *tzid); > > and > > void tzfree(timezone_t *); exactly, just like opendir() and closedir(). -------------- next part -------------- An HTML attachment was scrubbed... URL: From gharris at sonic.net Wed Jun 21 20:33:20 2023 From: gharris at sonic.net (Guy Harris) Date: Wed, 21 Jun 2023 13:33:20 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> <607AA65B-ECDF-473B-8B4E-8A99B56B65F1@sonic.net> Message-ID: <9A3B3756-7B05-4D9A-BA01-EEA6D8B4E42E@sonic.net> On Jun 21, 2023, at 1:17 PM, enh wrote: > NetBSD exposes its `struct _dirdesc` but FreeBSD/OpenBSD don't (though they use the same name for the private struct). Then I vote for doing as FreeBSD and OpenBSD and CupertinoBSD and systems using glibc do with DIR, rather than as NetBSD does with DIR, and have whatever the handle for a tzdb region is called be a pointer to an opaque structure. From eggert at cs.ucla.edu Wed Jun 21 21:18:26 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Wed, 21 Jun 2023 14:18:26 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> Message-ID: <6b8b6f43-8c53-42ad-a7e5-187b05997950@cs.ucla.edu> On 2023-06-21 12:50, enh wrote: > my preference would be for timezone_t to be more like DIR... > > typedef struct DIR DIR; Although that would also work, that's not how NetBSD did things when it introduced timezone_t in NetBSD 6.0 (2012), and tzcode copied this part of NetBSD in tzdb 2014g. I doubt whether we should change the meaning of timezone_t at this point, as it'd be too much backward-compatibility hassle. We could use a different name, though, for the struct. How about if we change this line in tzcode private.h: typedef struct state *timezone_t; to the following? typedef struct tm_timezone *timezone_t; and change all other instances of "struct state" to "struct tm_timezone"? This would let users write "struct tm_timezone *" and "timezone_t" interchangeably. The name tm_timezone, like timezone_t, is reserved by POSIX when you include [1], so if the implementation uses this name it won't break any conforming programs. [1]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_02_02 From enh at google.com Wed Jun 21 21:41:52 2023 From: enh at google.com (enh) Date: Wed, 21 Jun 2023 14:41:52 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <9A3B3756-7B05-4D9A-BA01-EEA6D8B4E42E@sonic.net> References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> <607AA65B-ECDF-473B-8B4E-8A99B56B65F1@sonic.net> <9A3B3756-7B05-4D9A-BA01-EEA6D8B4E42E@sonic.net> Message-ID: On Wed, Jun 21, 2023 at 1:33?PM Guy Harris wrote: > On Jun 21, 2023, at 1:17 PM, enh wrote: > > > NetBSD exposes its `struct _dirdesc` but FreeBSD/OpenBSD don't (though > they use the same name for the private struct). > > Then I vote for doing as FreeBSD and OpenBSD and CupertinoBSD and systems > using glibc do with DIR, rather than as NetBSD does with DIR, and have > whatever the handle for a tzdb region is called be a pointer to an opaque > structure. > you were right first time about macOS --- it's in the NetBSD camp (well, it inlines the struct definition in the typedef, so effectively the same). i haven't looked at glibc, but musl is in the FreeBSD/OpenBSD/MountainViewBsdOnLinux camp, so i assume glibc is too :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From enh at google.com Wed Jun 21 21:41:55 2023 From: enh at google.com (enh) Date: Wed, 21 Jun 2023 14:41:55 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <6b8b6f43-8c53-42ad-a7e5-187b05997950@cs.ucla.edu> References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> <6b8b6f43-8c53-42ad-a7e5-187b05997950@cs.ucla.edu> Message-ID: On Wed, Jun 21, 2023 at 2:18?PM Paul Eggert wrote: > On 2023-06-21 12:50, enh wrote: > > my preference would be for timezone_t to be more like DIR... > > > > typedef struct DIR DIR; > > Although that would also work, that's not how NetBSD did things when it > introduced timezone_t in NetBSD 6.0 (2012), and tzcode copied this part > of NetBSD in tzdb 2014g. I doubt whether we should change the meaning of > timezone_t at this point, as it'd be too much backward-compatibility > hassle. > > We could use a different name, though, for the struct. How about if we > change this line in tzcode private.h: > > typedef struct state *timezone_t; > > to the following? > > typedef struct tm_timezone *timezone_t; > > and change all other instances of "struct state" to "struct tm_timezone"? > > This would let users write "struct tm_timezone *" and "timezone_t" > interchangeably. The name tm_timezone, like timezone_t, is reserved by > POSIX when you include [1], so if the implementation > uses this name it won't break any conforming programs. > (i just checked and NetBSD does have `typedef struct __state *timezone_t;` in its . my objection to that is that the compiler will actually use the underlying name in error messages, which is less than helpful if the names aren't close. so having s/state/tm_timezone/ seems like a step forward even just in that regard!) up to you (especially since my #define in "private.h" is unlikely to be much of a maintenance burden), but "i'd use this" (if only to get rid of my #define)... > [1]: > > https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_02_02 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From magfr at lysator.liu.se Tue Jun 27 07:19:49 2023 From: magfr at lysator.liu.se (Magnus Fromreide) Date: Tue, 27 Jun 2023 09:19:49 +0200 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: Message-ID: On Sat, Jun 17, 2023 at 08:20:53PM +0200, Anders Rosendahl via tz wrote: > Hello! > > I'm developing an application using the JavaScript Intl API (which is > relying on the IANA tz db) for converting between local time and UTC. > > I started receiving error reports from users about wrong calculations > (particularly from Denmark in the 1940's), and I discovered that in some > cases daylight saving time was applied in a way not in accordance to > historical facts. > > I investigated the IANA tz database files and I discovered that a change > has happened in version 2022b (in the file "europe"). Compared to the > earlier version (2022a), 4 zones seem to have their rules removed and > instead linked to Europe/Berlin: > > Link Europe/Berlin Arctic/Longyearbyen > Link Europe/Berlin Europe/Copenhagen > Link Europe/Berlin Europe/Oslo > Link Europe/Berlin Europe/Stockholm > > This, evidently, is not correct, and will result in erroneous results for > these zones. Oh dear. This horse has been beaten to death and then resurrected for another turn, the trenches are very deep. Basically, IANA tz only cares about dates after 1970-01-01 and for that reason the maintainer have started merging zones that don't differ after that date but does still provide the data but it won't be included in the default builds (which is what everyone gets from their updates) Other people have, just like you, said that this is a bad move and this have lead to a fork of the project but I do not remember where that is located, this fork is used in at least some of the BSDs. My personal opinion is that the default IANA build should say nothing about dates before 1970-01-01 - it should, in particular, not claim that changes for some random place (like Berlin) applies to some other place (like Copenhagen). /MF From gharris at sonic.net Tue Jun 27 07:30:26 2023 From: gharris at sonic.net (Guy Harris) Date: Tue, 27 Jun 2023 00:30:26 -0700 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: Message-ID: <5C993909-780C-47C7-867F-9FF4A762BF08@sonic.net> On Jun 27, 2023, at 12:19 AM, Magnus Fromreide via tz wrote: > My personal opinion is that the default IANA build should say nothing about > dates before 1970-01-01 - it should, in particular, not claim that changes > for some random place (like Berlin) applies to some other place (like > Copenhagen). And, presumably, it shouldn't make any other claims about Copenhagen before 1970-01-0, either. Or about Berlin, or about New York City, or any of the other cities chosen for tzdb region names, for that matter. (Not that Continent/City should be read as only making claims about that city.) (At this point, I wish we could get away with doing as Microsoft do, and just return -1 for all attempts to call localtime() for dates prior to the UNIX Epoch.) From scolebourne at joda.org Tue Jun 27 08:35:20 2023 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 27 Jun 2023 09:35:20 +0100 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: Message-ID: On Tue, 27 Jun 2023 at 08:20, Magnus Fromreide via tz wrote: > On Sat, Jun 17, 2023 at 08:20:53PM +0200, Anders Rosendahl via tz wrote: > Other people have, just like you, said that this is a bad move and this have > lead to a fork of the project but I do not remember where that is located, > this fork is used in at least some of the BSDs. https://github.com/JodaOrg/global-tz I don't want to maintain the fork, but I have no choice given the current situation at IANA tzdb is simply broken as the OP notes. > My personal opinion is that the default IANA build should say nothing about > dates before 1970-01-01 - it should, in particular, not claim that changes > for some random place (like Berlin) applies to some other place (like > Copenhagen). Indeed, this would be an acceptable and equitable outcome. It isn't ideal, as TZDB actually holds the data for pre-1970, but at least the pre-1970 data would be equally wrong everywhere, rather than favouring some places over others (which is my primary objection). On Tue, 27 Jun 2023 at 08:31, Guy Harris via tz wrote: > (At this point, I wish we could get away with doing as Microsoft do, and just return -1 for all attempts to call localtime() for dates prior to the UNIX Epoch.) The simplest approach would be to determine a rule, eg. - take the standard offset that applied on most days (modal average) for the 5 years from 1970 to 1975 - and use that pre-1970. - or, take the LMT of the city and round to the nearest hour So long as the answer is reasonable for most cases, it would be fine. Stephen From eggert at cs.ucla.edu Tue Jun 27 09:30:58 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Tue, 27 Jun 2023 02:30:58 -0700 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: Message-ID: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> On 2023-06-27 01:35, Stephen Colebourne via tz wrote: > The simplest approach would be to determine a rule, eg. > - take the standard offset that applied on most days (modal average) > for the 5 years from 1970 to 1975 - and use that pre-1970. > - or, take the LMT of the city and round to the nearest hour > So long as the answer is reasonable for most cases, it would be fine. This would take the existing TZif files (admittedly problematic, as you say) and make them worse, as they'd become wrong for every location, even the location that names the Zone. Surely it would be better to discard the pre-1970 data - then users would be on notice that it's missing. And there's a standard way to do that, documented in the Makefile: use 'make ZFLAGS=-r at 0'. Perhaps this option should be documented more prominently. It's not clear that -r at 0 should be the Makefile default, though, as that could well cause more trouble than it would cure. For example, it would cause the following behavior: $ export TZ=Europe/Copenhagen $ date -r 1; date -r 0; date -r -1 Thu Jan 1 01:00:01 CET 1970 Thu Jan 1 01:00:00 CET 1970 Wed Dec 31 23:59:59 -00 1969 and the UT offset zero and abbreviation -00 of pre-1970 timestamps would likely give many users pause. That being said, in installations not needing pre-1970 timestamps, -z at 0 is a clear win. Getting back to Anders's original question, one can get behavior closer to what he asked with something like this: make PACKRATDATA=backzone PACKRATLIST=zone.tab install although I don't recommend this for ordinary applications. One should treat the result with a grain of salt as it's likely wrong for Copenhagen and it's certainly a bit wrong for Aarhus etc. This is a hazard of trying to push tzdata further than it can reliably bear. From scolebourne at joda.org Tue Jun 27 11:09:12 2023 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 27 Jun 2023 12:09:12 +0100 Subject: [tz] Error in Scandinavian tz data In-Reply-To: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> Message-ID: On Tue, 27 Jun 2023 at 10:31, Paul Eggert wrote: > On 2023-06-27 01:35, Stephen Colebourne via tz wrote: > > The simplest approach would be to determine a rule, eg. > > - take the standard offset that applied on most days (modal average) > > for the 5 years from 1970 to 1975 - and use that pre-1970. > > - or, take the LMT of the city and round to the nearest hour > > So long as the answer is reasonable for most cases, it would be fine. > > This would take the existing TZif files (admittedly problematic, as you > say) and make them worse, as they'd become wrong for every location, > even the location that names the Zone. Being slightly wrong everywhere is a much better outcome than what we have today. > Surely it would be better to discard the pre-1970 data - then users > would be on notice that it's missing. And there's a standard way to do > that, documented in the Makefile: use 'make ZFLAGS=-r at 0'. Perhaps this > option should be documented more prominently. > > It's not clear that -r at 0 should be the Makefile default, though, as that > could well cause more trouble than it would cure. For example, it would > cause the following behavior: > > $ export TZ=Europe/Copenhagen > $ date -r 1; date -r 0; date -r -1 > Thu Jan 1 01:00:01 CET 1970 > Thu Jan 1 01:00:00 CET 1970 > Wed Dec 31 23:59:59 -00 1969 > > and the UT offset zero and abbreviation -00 of pre-1970 timestamps would > likely give many users pause. That being said, in installations not > needing pre-1970 timestamps, -z at 0 is a clear win. In most cases, end users do not pick what options to install with. They just get what they are given by their packager. Since the packager cannot know whether the end user wants pre-1970 data or not, a sensible packager will err on the side of providing more data, not less, and thus want to include pre-1970 data. 'make ZFLAGS=-r at 0' is of no interest to packagers precisely because it is obviously wrong. ie. choosing UTC does not make people think, it just means the make option is not used. It is simply not good enough to be a viable choice. It is also not close enough to any values that have previously been placed in long-term storage. Hopefully all this explains why "Surely it would be better to discard the pre-1970 data then users would be on notice that it's missing" (ZFLAGS=-r at 0) isn't a viable route forward. The rule-based truncation I outlined above is a compromise position suitable for use as the default in the makefile. I believe it meets the needs of packagers who would hopefully accept it. It provides truncated data pre-1970, but in a way that is not completely unreasonable: * It is good enough for most use cases except those that really care about historical detail * It avoids the weird per-second LMT offsets in the far past that often confuse end users * It is close enough to what end users have in long-term storage to not cause migration issues Just returning UTC does not meet these goals. > it's likely wrong for Copenhagen I think you do a disservice to TZDB's many authors here. I'd argue that the data for Copenhagen is likely to be entirely correct, as it has had many eyes on it for many years. Beyond that, I think there is also a need to recognise that TZDB's pre-1970 data is the de facto truth for large parts of the world. Most end users don't care about the accuracy, just that someone has made an effort to record it. I still believe that reinstating the data would be by far the best outcome, but a rules-based truncation approach would be a viable alternative if pre-1970 data is not something to be maintained. Stephen From paw at boel-it.dk Tue Jun 27 11:13:10 2023 From: paw at boel-it.dk (Paw Boel Nielsen) Date: Tue, 27 Jun 2023 11:13:10 +0000 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> Message-ID: > Beyond that, I think there is also a need to recognise that TZDB's > pre-1970 data is the de facto truth for large parts of the world. > Most end users don't care about the accuracy, just that someone > has made an effort to record it. I still believe that reinstating the > data would be by far the best outcome, but a rules-based truncation > approach would be a viable alternative if pre-1970 data is not > something to be maintained. Hear hear --------------------------- DISCLAIMER: The information contained in this electronic message and in any attachments to this message is intended only for the person or entity to which this electronic message is addressed. If you are not the intended recipient, you are hereby notified that any distribution, copying, review, retransmission, dissemination or other use of this electronic transmission or the information contained in it is strictly prohibited. -----Original Message----- From: tz On Behalf Of Stephen Colebourne via tz Sent: Tuesday, June 27, 2023 1:09 PM To: Time zone mailing list Subject: Re: [tz] Error in Scandinavian tz data On Tue, 27 Jun 2023 at 10:31, Paul Eggert wrote: > On 2023-06-27 01:35, Stephen Colebourne via tz wrote: > > The simplest approach would be to determine a rule, eg. > > - take the standard offset that applied on most days (modal > > average) for the 5 years from 1970 to 1975 - and use that pre-1970. > > - or, take the LMT of the city and round to the nearest hour So long > > as the answer is reasonable for most cases, it would be fine. > > This would take the existing TZif files (admittedly problematic, as > you > say) and make them worse, as they'd become wrong for every location, > even the location that names the Zone. Being slightly wrong everywhere is a much better outcome than what we have today. > Surely it would be better to discard the pre-1970 data - then users > would be on notice that it's missing. And there's a standard way to do > that, documented in the Makefile: use 'make ZFLAGS=-r at 0'. Perhaps this > option should be documented more prominently. > > It's not clear that -r at 0 should be the Makefile default, though, as > that could well cause more trouble than it would cure. For example, it > would cause the following behavior: > > $ export TZ=Europe/Copenhagen > $ date -r 1; date -r 0; date -r -1 > Thu Jan 1 01:00:01 CET 1970 > Thu Jan 1 01:00:00 CET 1970 > Wed Dec 31 23:59:59 -00 1969 > > and the UT offset zero and abbreviation -00 of pre-1970 timestamps > would likely give many users pause. That being said, in installations > not needing pre-1970 timestamps, -z at 0 is a clear win. In most cases, end users do not pick what options to install with. They just get what they are given by their packager. Since the packager cannot know whether the end user wants pre-1970 data or not, a sensible packager will err on the side of providing more data, not less, and thus want to include pre-1970 data. 'make ZFLAGS=-r at 0' is of no interest to packagers precisely because it is obviously wrong. ie. choosing UTC does not make people think, it just means the make option is not used. It is simply not good enough to be a viable choice. It is also not close enough to any values that have previously been placed in long-term storage. Hopefully all this explains why "Surely it would be better to discard the pre-1970 data then users would be on notice that it's missing" (ZFLAGS=-r at 0) isn't a viable route forward. The rule-based truncation I outlined above is a compromise position suitable for use as the default in the makefile. I believe it meets the needs of packagers who would hopefully accept it. It provides truncated data pre-1970, but in a way that is not completely unreasonable: * It is good enough for most use cases except those that really care about historical detail * It avoids the weird per-second LMT offsets in the far past that often confuse end users * It is close enough to what end users have in long-term storage to not cause migration issues Just returning UTC does not meet these goals. > it's likely wrong for Copenhagen I think you do a disservice to TZDB's many authors here. I'd argue that the data for Copenhagen is likely to be entirely correct, as it has had many eyes on it for many years. Beyond that, I think there is also a need to recognise that TZDB's pre-1970 data is the de facto truth for large parts of the world. Most end users don't care about the accuracy, just that someone has made an effort to record it. I still believe that reinstating the data would be by far the best outcome, but a rules-based truncation approach would be a viable alternative if pre-1970 data is not something to be maintained. Stephen From eggert at cs.ucla.edu Wed Jun 28 00:25:59 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Tue, 27 Jun 2023 17:25:59 -0700 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> Message-ID: <7e10f040-3255-7b70-f0a1-2c610847eabb@cs.ucla.edu> On 2023-06-27 04:09, Stephen Colebourne via tz wrote: > 'make ZFLAGS=-r at 0' is of no interest to packagers precisely because it > is obviously wrong. Then I must not be understanding "the default IANA build should say nothing about dates before 1970-01-01", a proposal that you said was acceptable. I thought "make ZFLAGS=-r at 0" would implement that proposal, as it builds TZif files that say nothing about timestamps before 1970. But since you're saying "make ZFLAGS=-r at 0" is of no interest, it seems the proposal is about something else. If so, it would be helpful to know what the proposal is. > Just returning UTC does not meet these goals. "make ZFLAGS=-r at 0" does not cause tzdb localtime to just return UTC, as localtime also returns an error indication indicating that the actual offset is unknown. This error indication is in the form of the time zone abbreviation "-00". > the data for Copenhagen is likely to be entirely correct I doubt that, as some of the data are sourced only from Shanks, which has proven to be unreliable. > there is also a need to recognise that TZDB's > pre-1970 data is the de facto truth That would not be accurate advertising. TZDB is only TZDB. It has never been "the de facto truth", unless "truth" means only "act like TZDB Release X". For many years TZDB's files have said that it is "by no means authoritative", and for many years TZDB installations have varied in minor ways from one platform to another. From scolebourne at joda.org Wed Jun 28 08:17:44 2023 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 28 Jun 2023 09:17:44 +0100 Subject: [tz] Error in Scandinavian tz data In-Reply-To: <7e10f040-3255-7b70-f0a1-2c610847eabb@cs.ucla.edu> References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> <7e10f040-3255-7b70-f0a1-2c610847eabb@cs.ucla.edu> Message-ID: On Wed, 28 Jun 2023 at 01:26, Paul Eggert wrote: > Then I must not be understanding "the default IANA build should say > nothing about dates before 1970-01-01", a proposal that you said was > acceptable. Re-reading the thread, I see where your confusion comes from. What I was agreeing with was the concept of removing attempts to be accurate pre-1970. I then pointed out why being completely inaccurate (all UTC) is not viable as the default. My proposal was: > The simplest approach would be to determine a rule, eg. > - take the standard offset that applied on most days (modal average) for the 5 years from 1970 to 1975 - and use that pre-1970. > - or, take the LMT of the city and round to the nearest hour > So long as the answer is reasonable for most cases, it would be fine. ie. I believe that if tzdb provided a fixed offset pre-1970, but one linked to LMT or common practice in that location, it would be a suitable approach for most packagers. I reckon to the nearest hour would be sufficient. Stephen From doug at ewellic.org Wed Jun 28 16:52:39 2023 From: doug at ewellic.org (Doug Ewell) Date: Wed, 28 Jun 2023 16:52:39 +0000 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> <7e10f040-3255-7b70-f0a1-2c610847eabb@cs.ucla.edu> Message-ID: Stephen Colebourne wrote: > Re-reading the thread, I see where your confusion comes from. What I > was agreeing with was the concept of removing attempts to be accurate > pre-1970. I then pointed out why being completely inaccurate (all UTC) > is not viable as the default. I think Paul noted already that ?-00? is not the same as UTC: >> "make ZFLAGS=-r at 0" does not cause tzdb localtime to just return UTC, >> as localtime also returns an error indication indicating that the >> actual offset is unknown. This error indication is in the form of the >> time zone abbreviation "-00". -- Doug Ewell, CC, ALB | Lakewood, CO, US | ewellic.org From eggert at cs.ucla.edu Wed Jun 28 17:27:56 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Wed, 28 Jun 2023 10:27:56 -0700 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> <7e10f040-3255-7b70-f0a1-2c610847eabb@cs.ucla.edu> Message-ID: On 2023-06-28 01:17, Stephen Colebourne via tz wrote: > I believe that if tzdb provided a fixed offset pre-1970, but one > linked to LMT or common practice in that location For timestamps before 1970 we could approximate LMT along with a time zone abbreviation saying that the UT offset is approximate. Something equivalent to this, say: Zone America/New_York -5:00 - c-05 1970 Jan 1 0u -5:00 US E%sT Zone Asia/Tokyo 9:00 - c+09 1970 Jan 1 0u 9:00 Japan J%sT The idea would be that a time zone abbreviation "c+NN" stands for "circa +NN", where the exact offset is unspecified. Software that cares whether an offset is approximate could look in the abbreviation for leading "c+" and "c-" (along with looking for "-00", which would continue to have its current meaning). POSIX allows "c+" and "c-" at the start of time zone abbreviations so this would pass POSIX muster. This sort of thing could be generated automatically from the current data. From scolebourne at joda.org Wed Jun 28 19:49:50 2023 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 28 Jun 2023 20:49:50 +0100 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> <7e10f040-3255-7b70-f0a1-2c610847eabb@cs.ucla.edu> Message-ID: On Wed, 28 Jun 2023 at 18:28, Paul Eggert wrote: > On 2023-06-28 01:17, Stephen Colebourne via tz wrote: > > I believe that if tzdb provided a fixed offset pre-1970, but one > > linked to LMT or common practice in that location > > For timestamps before 1970 we could approximate LMT along with a time > zone abbreviation saying that the UT offset is approximate. Something > equivalent to this, say: > > Zone America/New_York -5:00 - c-05 1970 Jan 1 0u > -5:00 US E%sT > > Zone Asia/Tokyo 9:00 - c+09 1970 Jan 1 0u > 9:00 Japan J%sT > > The idea would be that a time zone abbreviation "c+NN" stands for "circa > +NN", where the exact offset is unspecified. Software that cares whether > an offset is approximate could look in the abbreviation for leading "c+" > and "c-" (along with looking for "-00", which would continue to have its > current meaning). POSIX allows "c+" and "c-" at the start of time zone > abbreviations so this would pass POSIX muster. Or the name could be something consistent across the globe, like "LMT", such as "SET" (Solar estimated time). It would probably be easier to have one consistent name (like LMT) than a prefix. I think before progressing this further it would need other packagers to chime in and comment on whether they feel it would be acceptable as the default setting for tzdb pre-1970. Stephen From eggert at cs.ucla.edu Thu Jun 1 00:08:54 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Wed, 31 May 2023 17:08:54 -0700 Subject: [tz] Tzdb build issue In-Reply-To: References: Message-ID: On 2023-05-31 13:14, Guy Harris via tz wrote: > I was able to build on Ubuntu 14.10 with GCC 4.9.1 by adding -DPORT_TO_C89 to the end of the "CFLAGS=" line in Makefile before running make; it didn't build unless I did that, as GCC 4.9.1 didn't recognize "restrict" as a keyword. says the 'restrict' keyword is supported by GCC 2.95 (1999) and later. GCC 4 and earlier, though, need an option like '-std=gnu11' to enable 'restrict', as these GCC versions default to C89 and 'restrict' is incompatible with C89. Come to think of it, it's probably better to use -std=gnu11 than -DPORT_TO_C89 on Naveen's platform. Something like this, say: make -j6 zic=zic cc=arm-linux-gnueabi-gcc \ ZDUMPDIR=/usr/sbin ZFLAGS="-b slim" \ CFLAGS="-std=gnu11 -Os -g -pipe" I installed the attached commentary patch into TZDB, to try to help others in Naveen's shoes. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Prefer-std-gnu11-to-DPORT_TO_C89.patch Type: text/x-patch Size: 1053 bytes Desc: not available URL: From Brian.Inglis at Shaw.ca Thu Jun 1 00:39:58 2023 From: Brian.Inglis at Shaw.ca (Brian Inglis) Date: Wed, 31 May 2023 18:39:58 -0600 Subject: [tz] [PROPOSED] Update tzselect comment In-Reply-To: <20230531180324.52614-1-eggert@cs.ucla.edu> References: <20230531180324.52614-1-eggert@cs.ucla.edu> Message-ID: Announced 2019-09-17: https://blogs.oracle.com/solaris/post/great-news-about-extended-support-for-oracle-solaris-10-os Note: Sustaining Support is still available after 2025-01. On 2023-05-31 12:03, Paul Eggert via tz wrote: > * tzselect.ksh: Update comment, as Oracle has extended the > end-of-life date for Solaris 10. > --- > tzselect.ksh | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tzselect.ksh b/tzselect.ksh > index 9a91acfc..98774c03 100644 > --- a/tzselect.ksh > +++ b/tzselect.ksh > @@ -21,7 +21,7 @@ REPORT_BUGS_TO=tz at iana.org > # MirBSD Korn Shell > # > # For portability to Solaris 10 /bin/sh (supported by Oracle through > -# January 2024) this script avoids some POSIX features and common > +# January 2025) this script avoids some POSIX features and common > # extensions, such as $(...) (which works sometimes but not others), > # $((...)), ! CMD, ${#ID}, ${ID##PAT}, ${ID%%PAT}, and $10. -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien ? ajouter not when there is no more to add mais lorsqu'il n'y a plus rien ? retirer but when there is no more to cut -- Antoine de Saint-Exup?ry From gharris at sonic.net Thu Jun 1 08:55:01 2023 From: gharris at sonic.net (Guy Harris) Date: Thu, 1 Jun 2023 01:55:01 -0700 Subject: [tz] Tzdb build issue In-Reply-To: References: Message-ID: <491475F1-69BC-4FFA-9EF4-E8A3C49C1342@sonic.net> On Jun 1, 2023, at 1:12 AM, Naveen kumar wrote: > I added the -std=gnu11 options also it's shows same error. So you added it to the CFLAGS= line in the Makefile and it still complains about "restrict"? If so, what happens if you add "-std=gnu99" instead? Or you ran make CFLAGS="-std=gnu11" ... and it still complains about "restrict"? If so, what happens if you use "-std=gnu99" instead? From navinagaraj18 at gmail.com Thu Jun 1 08:12:13 2023 From: navinagaraj18 at gmail.com (Naveen kumar) Date: Thu, 1 Jun 2023 13:42:13 +0530 Subject: [tz] Tzdb build issue In-Reply-To: References: Message-ID: Hi , I added the -std=gnu11 options also it's shows same error. On Thu, 1 Jun, 2023, 5:38 am Paul Eggert, wrote: > On 2023-05-31 13:14, Guy Harris via tz wrote: > > I was able to build on Ubuntu 14.10 with GCC 4.9.1 by adding > -DPORT_TO_C89 to the end of the "CFLAGS=" line in Makefile before running > make; it didn't build unless I did that, as GCC 4.9.1 didn't recognize > "restrict" as a keyword. > > says the 'restrict' keyword is > supported by GCC 2.95 (1999) and later. GCC 4 and earlier, though, need > an option like '-std=gnu11' to enable 'restrict', as these GCC versions > default to C89 and 'restrict' is incompatible with C89. > > Come to think of it, it's probably better to use -std=gnu11 than > -DPORT_TO_C89 on Naveen's platform. Something like this, say: > > make -j6 zic=zic cc=arm-linux-gnueabi-gcc \ > ZDUMPDIR=/usr/sbin ZFLAGS="-b slim" \ > CFLAGS="-std=gnu11 -Os -g -pipe" > > I installed the attached commentary patch into TZDB, to try to help > others in Naveen's shoes. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From navinagaraj18 at gmail.com Thu Jun 1 10:31:26 2023 From: navinagaraj18 at gmail.com (Naveen kumar) Date: Thu, 1 Jun 2023 16:01:26 +0530 Subject: [tz] Tzdb build issue In-Reply-To: <491475F1-69BC-4FFA-9EF4-E8A3C49C1342@sonic.net> References: <491475F1-69BC-4FFA-9EF4-E8A3C49C1342@sonic.net> Message-ID: Hi Team, Now it's working fine with c11 standard. Thanks & Regards, Naveen. On Thu, 1 Jun, 2023, 2:25 pm Guy Harris, wrote: > On Jun 1, 2023, at 1:12 AM, Naveen kumar wrote: > > > I added the -std=gnu11 options also it's shows same error. > > So you added it to the CFLAGS= line in the Makefile and it still complains > about "restrict"? If so, what happens if you add "-std=gnu99" instead? > > Or you ran > > make CFLAGS="-std=gnu11" ... > > and it still complains about "restrict"? If so, what happens if you use > "-std=gnu99" instead? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From m.17768 at gmail.com Fri Jun 2 13:04:36 2023 From: m.17768 at gmail.com (Mohammad Ahmadian) Date: Fri, 2 Jun 2023 16:34:36 +0330 Subject: [tz] Iran Time Zone is Incorrect in Your Database Message-ID: Hello, Thanks for your service. This year, the Iran government decided to cancel the daylight time zone. So they did not change the time zone to 04:30. It is now 03:30. I updated my tablet via this application to the latest version of your database successfully: https://github.com/meefik/tzupdater But the time zone is wrongly 04:30 again. Please change your database. Let me know what you think and do. Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From eggert at cs.ucla.edu Fri Jun 2 22:12:24 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Fri, 2 Jun 2023 15:12:24 -0700 Subject: [tz] Iran Time Zone is Incorrect in Your Database In-Reply-To: References: Message-ID: <616f55e1-99a0-9af9-fad7-06b381a643e8@cs.ucla.edu> On 6/2/23 06:04, Mohammad Ahmadian via tz wrote: > This year, the Iran government decided to cancel the daylight time zone. So > they did not change the time zone to 04:30. It is now 03:30. I updated my > tablet via this application to the latest version of your database > successfully:https://github.com/meefik/tzupdater Yes, last year we were informed about that change for Iran, and the change appeared in tzdb 2022b, released 2022-08-10. See: https://mm.icann.org/pipermail/tz-announce/2022-August/000071.html I wasn't aware of meefik/tzupdater until today, and I don't know how it works or what's going wrong with your update. You might ask its author for help. From eggert at cs.ucla.edu Tue Jun 20 06:08:54 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Mon, 19 Jun 2023 23:08:54 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: Message-ID: On 2023-06-15 10:03, enh wrote: > any thoughts on either renaming `struct state` (i went with `struct > __timezone_t` to match the typedef to timezone_t for the pointer) or having > a knob for that (i added a `#define state __timezone_t` in the `#if > NETBSD_INSPIRED` in private.h)? why do i care? because tzcode pulls in the > system , which is actually fine for the tzcode code, but breaks the > OpenBSD wcsftime(). We could rename 'struct state', but I'm a bit lost as to what the problem is, as I don't see mention of 'state' or 'timezone_t' in openbsd/lib/libc/time/wcsftime.c. Although there is a 'struct state' in openbsd/lib/libc/time/localtime.c I assume you're replacing that with tzcode's localtime.c somehow. User apps (which tzcode is, by default) shouldn't define symbols like __timezone_t because these symbols are reserved to the system. We could easily rename 'struct state' to 'struct timezone_struct' so that you could compile with -Dtimezone_struct=__timezone_t, though I'm still curious about why 'struct state' causes problems. I'll cc this to the tz mailing list to see whether other people there have insight into this issue. For those new to the discussion, the original thread is archived here: https://www.openwall.com/lists/libc-coord/2023/06/14/1 From webfirmaet at gmail.com Sat Jun 17 18:20:53 2023 From: webfirmaet at gmail.com (Anders Rosendahl) Date: Sat, 17 Jun 2023 20:20:53 +0200 Subject: [tz] Error in Scandinavian tz data Message-ID: Hello! I'm developing an application using the JavaScript Intl API (which is relying on the IANA tz db) for converting between local time and UTC. I started receiving error reports from users about wrong calculations (particularly from Denmark in the 1940's), and I discovered that in some cases daylight saving time was applied in a way not in accordance to historical facts. I investigated the IANA tz database files and I discovered that a change has happened in version 2022b (in the file "europe"). Compared to the earlier version (2022a), 4 zones seem to have their rules removed and instead linked to Europe/Berlin: Link Europe/Berlin Arctic/Longyearbyen Link Europe/Berlin Europe/Copenhagen Link Europe/Berlin Europe/Oslo Link Europe/Berlin Europe/Stockholm This, evidently, is not correct, and will result in erroneous results for these zones. Best wishes, Anders Rosendahl -------------- next part -------------- An HTML attachment was scrubbed... URL: From enh at google.com Tue Jun 20 21:00:29 2023 From: enh at google.com (enh) Date: Tue, 20 Jun 2023 14:00:29 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: Message-ID: On Mon, Jun 19, 2023 at 11:08?PM Paul Eggert wrote: > On 2023-06-15 10:03, enh wrote: > > any thoughts on either renaming `struct state` (i went with `struct > > __timezone_t` to match the typedef to timezone_t for the pointer) or > having > > a knob for that (i added a `#define state __timezone_t` in the `#if > > NETBSD_INSPIRED` in private.h)? why do i care? because tzcode pulls in > the > > system , which is actually fine for the tzcode code, but breaks > the > > OpenBSD wcsftime(). > > We could rename 'struct state', but I'm a bit lost as to what the > problem is, as I don't see mention of 'state' or 'timezone_t' in > openbsd/lib/libc/time/wcsftime.c. Although there is a 'struct state' in > openbsd/lib/libc/time/localtime.c I assume you're replacing that with > tzcode's localtime.c somehow. > it's actually the openbsd strptime.c that's unhappy: ``` In file included from bionic/libc/tzcode/strptime.c:40: bionic/libc/tzcode/private.h:790:23: error: typedef redefinition with different types ('struct state *' vs 'struct __timezone_t *') typedef struct state *timezone_t; ^ bionic/libc/include/time.h:46:30: note: previous definition is here typedef struct __timezone_t* timezone_t; ^ 1 error generated. ``` the trick in private.h assumes that it's #included _before_ , but that's not true in that file. (since we already have local changes in that file for gnu extensions that openbsd wasn't interested in, i've just reordered the #includes.) User apps (which tzcode is, by default) shouldn't define symbols like > __timezone_t because these symbols are reserved to the system. We could > easily rename 'struct state' to 'struct timezone_struct' so that you > could compile with -Dtimezone_struct=__timezone_t, though I'm still > curious about why 'struct state' causes problems. > true; since tzcode _is_ the implementation for 3.5+ billion devices, i forget that others might not consider it "the implementation" :-) > I'll cc this to the tz mailing list to see whether other people there > have insight into this issue. For those new to the discussion, the > original thread is archived here: > > https://www.openwall.com/lists/libc-coord/2023/06/14/1 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From PaulGBoulder at AIM.com Wed Jun 21 02:16:23 2023 From: PaulGBoulder at AIM.com (Paul Gilmartin) Date: Tue, 20 Jun 2023 20:16:23 -0600 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: Message-ID: <306f3fe4-1152-2fb6-608d-1994d746dcf1@AIM.com> ( the Archives of this thread seem incomplete.) On 6/20/23 15:00:29, enh via tz wrote: > ... > We could rename 'struct state', but I'm a bit lost as to what the > problem is, as I don't see mention of 'state' or 'timezone_t' in > openbsd/lib/libc/time/wcsftime.c. Although there is a 'struct state' in > openbsd/lib/libc/time/localtime.c I assume you're replacing that with > tzcode's localtime.c somehow. (Replying mostly to the Subject:) A thread-safe localtime() seems impractical. localtime() depends on the environment variable TZ. If concurrent threads set different values of TZ the results will be unpredictable. The same applies to localtime_r(). -- gil From gharris at sonic.net Wed Jun 21 02:52:25 2023 From: gharris at sonic.net (Guy Harris) Date: Tue, 20 Jun 2023 19:52:25 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <306f3fe4-1152-2fb6-608d-1994d746dcf1@AIM.com> References: <306f3fe4-1152-2fb6-608d-1994d746dcf1@AIM.com> Message-ID: <2B79184C-2633-43DD-B5DC-DF4572D2B326@sonic.net> On Jun 20, 2023, at 7:16 PM, Paul Gilmartin via tz wrote: > ( the Archives of this thread seem incomplete.) tz mailing list archives, or lib-coord: https://www.openwall.com/lists/libc-coord/2020/01/30/1 archives? The former are incomplete because the thread didn't start on the tz mailing list; the first message, to libc-coord, was linked from Paul Eggert's message to tz in this thread: https://www.openwall.com/lists/libc-coord/2023/06/14/1 > On 6/20/23 15:00:29, enh via tz wrote: >> ... >> We could rename 'struct state', but I'm a bit lost as to what the >> problem is, as I don't see mention of 'state' or 'timezone_t' in >> openbsd/lib/libc/time/wcsftime.c. Although there is a 'struct state' in >> openbsd/lib/libc/time/localtime.c I assume you're replacing that with >> tzcode's localtime.c somehow. > > (Replying mostly to the Subject:) > A thread-safe localtime() seems impractical. localtime() depends > on the environment variable TZ. If concurrent threads set > different values of TZ the results will be unpredictable. The > same applies to localtime_r(). If concurrent thread set different values of TZ, perhaps the concurrent threads should use a reentrant localtime_z() routine, possibly with a different name, as has been proposed in a number of places, including in this list back in 2001: http://mm.icann.org/pipermail/tz/2001-June/011636.html That requires making such a routine common on multiple platforms, of course.... And, in fact, in the first message in the thread, the author said is anyone aware of any standardization work in this area? i know netbsd has tzalloc()/localtime_rz()/tzfree() (and tzcode has implementations), but they're the only ones who've shipped anything, right? and there's no in-progress work on any alternative? So the proposal there is for localtime_rz(), taking a "time zone" structure, or pointer to same, provided by tzalloc() and freed by jtzfree(), rather than something that looks at TZ. (Perhaps the title was a bit misleading, and should have said "lcoaltime() equivalent".) From gharris at sonic.net Wed Jun 21 06:27:58 2023 From: gharris at sonic.net (Guy Harris) Date: Tue, 20 Jun 2023 23:27:58 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: Message-ID: <53EB31E0-4AFB-4089-BFF8-27D27455431E@sonic.net> On Jun 20, 2023, at 2:00 PM, enh via tz wrote: > it's actually the openbsd strptime.c that's unhappy: > ``` > In file included from bionic/libc/tzcode/strptime.c:40: I see the string "bionic" rather than "openbsd" there, so that appears to be *Bionic's* strptime.c that's unhappy. Is there a requirement that Bionic's strptime.c be unchanged from OpenBSD's? (BTW, is there some software using Markdown involved here? the ``` won't change the formatting to fixed format in plain-text or HTML email; in the former, what you see is pretty much what you get, and in the latter, you'd need whatever tags ``` turns into.) > (since we already have local changes in that file for gnu extensions that openbsd wasn't interested in, i've just reordered the #includes.) Exactly. From gharris at sonic.net Wed Jun 21 07:04:47 2023 From: gharris at sonic.net (Guy Harris) Date: Wed, 21 Jun 2023 00:04:47 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <53EB31E0-4AFB-4089-BFF8-27D27455431E@sonic.net> References: <53EB31E0-4AFB-4089-BFF8-27D27455431E@sonic.net> Message-ID: <177D39A7-F50F-4DD1-84D1-244E7888D439@sonic.net> On Jun 20, 2023, at 11:27 PM, Guy Harris via tz wrote: > On Jun 20, 2023, at 2:00 PM, enh via tz wrote: > >> it's actually the openbsd strptime.c that's unhappy: >> ``` >> In file included from bionic/libc/tzcode/strptime.c:40: > > I see the string "bionic" rather than "openbsd" there, so that appears to be *Bionic's* strptime.c that's unhappy. Is there a requirement that Bionic's strptime.c be unchanged from OpenBSD's? In fact, the openbsd strptime.c, as of CVS revision 1.32: https://cvsweb.openbsd.org/src/lib/libc/time/strptime.c?rev=1.31&content-type=text/x-cvsweb-markup does *not* contain the line typedef struct state *timezone_t; so it appears that it's the *Bionic* strptime.c that's unhappy. From gharris at sonic.net Wed Jun 21 07:54:37 2023 From: gharris at sonic.net (Guy Harris) Date: Wed, 21 Jun 2023 00:54:37 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: Message-ID: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> From an earlier message of yours: https://www.openwall.com/lists/libc-coord/2023/06/15/2 sent only to libc-coord: > writing some basic unit tests for these functions went fine, though it was > sad that i couldn't use `std::unique_ptr > seoul{tzalloc("Asia/Seoul"), tzfree}` because the size of timezone_t isn't > known. i'd be tempted to further mangle this so i have a public struct > containing a large-enough char[], but i worry about just how large to make > it to be safe from future tzcode changes. (and on the other hand, do i > think that tzalloc() is going to be used frequently enough to warrant the > effect? probably not?) Is there some particular reason to make timezone_t an API/ABI structure, rather than a pointer to an opaque structure? Presumably if it's a structure, "tzalloc()" really means "given a tzid, fill in a structure as appropriate for that tzid, and return that structure", not "allocate a structure, fill it in as appropriate for the tzid, and return a pointer to that structure", and "tzfree()" means "free up anything pointed to by the structure whose value was passed to it" rather than "free everything pointed to by the structure pointed to by the argument and then free the structure". If you want to do that, perhaps names that don't imply that they allocate and free a timezone_t should be chosen. From enh at google.com Wed Jun 21 19:44:26 2023 From: enh at google.com (enh) Date: Wed, 21 Jun 2023 12:44:26 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <177D39A7-F50F-4DD1-84D1-244E7888D439@sonic.net> References: <53EB31E0-4AFB-4089-BFF8-27D27455431E@sonic.net> <177D39A7-F50F-4DD1-84D1-244E7888D439@sonic.net> Message-ID: On Wed, Jun 21, 2023 at 12:05?AM Guy Harris wrote: > On Jun 20, 2023, at 11:27 PM, Guy Harris via tz wrote: > > > On Jun 20, 2023, at 2:00 PM, enh via tz wrote: > > > >> it's actually the openbsd strptime.c that's unhappy: > >> ``` > >> In file included from bionic/libc/tzcode/strptime.c:40: > > > > I see the string "bionic" rather than "openbsd" there, so that appears > to be *Bionic's* strptime.c that's unhappy. Is there a requirement that > Bionic's strptime.c be unchanged from OpenBSD's? > > In fact, the openbsd strptime.c, as of CVS revision 1.32: > > > https://cvsweb.openbsd.org/src/lib/libc/time/strptime.c?rev=1.31&content-type=text/x-cvsweb-markup > > does *not* contain the line > > typedef struct state *timezone_t; > > so it appears that it's the *Bionic* strptime.c that's unhappy. (this is answered on the other thread: it's tz's "private.h" that contains that. i'm afraid without reading _both_ threads, the tz@ thread is going to be confusing.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From enh at google.com Wed Jun 21 19:50:09 2023 From: enh at google.com (enh) Date: Wed, 21 Jun 2023 12:50:09 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> Message-ID: On Wed, Jun 21, 2023 at 12:54?AM Guy Harris wrote: > From an earlier message of yours: > > https://www.openwall.com/lists/libc-coord/2023/06/15/2 > > sent only to libc-coord: > > > writing some basic unit tests for these functions went fine, though it > was > > sad that i couldn't use `std::unique_ptr > > seoul{tzalloc("Asia/Seoul"), tzfree}` because the size of timezone_t > isn't > > known. i'd be tempted to further mangle this so i have a public struct > > containing a large-enough char[], but i worry about just how large to > make > > it to be safe from future tzcode changes. (and on the other hand, do i > > think that tzalloc() is going to be used frequently enough to warrant the > > effect? probably not?) > > Is there some particular reason to make timezone_t an API/ABI structure, > rather than a pointer to an opaque structure? > no, my preference would be for timezone_t to be more like DIR... typedef struct DIR DIR; means you need to write the * in signatures, but that std::unique_ptr-style safe usage "just works" without having to talk about the underlying struct or use std::remove_pointer. > Presumably if it's a structure, "tzalloc()" really means "given a tzid, > fill in a structure as appropriate for that tzid, and return that > structure", not "allocate a structure, fill it in as appropriate for the > tzid, and return a pointer to that structure", and "tzfree()" means "free > up anything pointed to by the structure whose value was passed to it" > rather than "free everything pointed to by the structure pointed to by the > argument and then free the structure". If you want to do that, perhaps > names that don't imply that they allocate and free a timezone_t should be > chosen. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gharris at sonic.net Wed Jun 21 20:08:40 2023 From: gharris at sonic.net (Guy Harris) Date: Wed, 21 Jun 2023 13:08:40 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> Message-ID: <607AA65B-ECDF-473B-8B4E-8A99B56B65F1@sonic.net> On Jun 21, 2023, at 12:50 PM, enh wrote: > On Wed, Jun 21, 2023 at 12:54?AM Guy Harris wrote: > >> Is there some particular reason to make timezone_t an API/ABI structure, rather than a pointer to an opaque structure? > > no, my preference would be for timezone_t to be more like DIR... > > typedef struct DIR DIR; Like DIR, which, at least in macOS (and thus probably in most *BSDs), is fully defined in , or like pcap_t, which is "defined" as an opaque structure in and only defined fully in a header internal to libpcap? (And, yes, pcap_t's contents are subject to change over time, have been changed over time, and will continue to change over time. > means you need to write the * in signatures, but that std::unique_ptr-style safe usage "just works" without having to talk about the underlying struct or use std::remove_pointer. Does that work if the structure is opaque outside the routines that process tz files? > Presumably if it's a structure, "tzalloc()" really means "given a tzid, fill in a structure as appropriate for that tzid, and return that structure", not "allocate a structure, fill it in as appropriate for the tzid, and return a pointer to that structure", and "tzfree()" means "free up anything pointed to by the structure whose value was passed to it" rather than "free everything pointed to by the structure pointed to by the argument and then free the structure". If you want to do that, perhaps names that don't imply that they allocate and free a timezone_t should be chosen. I.e., it would be timezone_t *tzalloc(const char *tzid); and void tzfree(timezone_t *); From enh at google.com Wed Jun 21 20:17:57 2023 From: enh at google.com (enh) Date: Wed, 21 Jun 2023 13:17:57 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <607AA65B-ECDF-473B-8B4E-8A99B56B65F1@sonic.net> References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> <607AA65B-ECDF-473B-8B4E-8A99B56B65F1@sonic.net> Message-ID: On Wed, Jun 21, 2023 at 1:08?PM Guy Harris wrote: > On Jun 21, 2023, at 12:50 PM, enh wrote: > > > On Wed, Jun 21, 2023 at 12:54?AM Guy Harris wrote: > > > >> Is there some particular reason to make timezone_t an API/ABI > structure, rather than a pointer to an opaque structure? > > > > no, my preference would be for timezone_t to be more like DIR... > > > > typedef struct DIR DIR; > > Like DIR, which, at least in macOS (and thus probably in most *BSDs), is > fully defined in , or like pcap_t, which is "defined" as an opaque > structure in and only defined fully in a header internal to > libpcap? (And, yes, pcap_t's contents are subject to change over time, > have been changed over time, and will continue to change over time. > NetBSD exposes its `struct _dirdesc` but FreeBSD/OpenBSD don't (though they use the same name for the private struct). somewhat unusually, bionic didn't start from any of the BSD's , and uses exact the line i showed above, with the definition only in dirent.cpp itself. > > means you need to write the * in signatures, but that > std::unique_ptr-style safe usage "just works" without having to talk about > the underlying struct or use std::remove_pointer. > > Does that work if the structure is opaque outside the routines that > process tz files? > > > Presumably if it's a structure, "tzalloc()" really means "given a tzid, > fill in a structure as appropriate for that tzid, and return that > structure", not "allocate a structure, fill it in as appropriate for the > tzid, and return a pointer to that structure", and "tzfree()" means "free > up anything pointed to by the structure whose value was passed to it" > rather than "free everything pointed to by the structure pointed to by the > argument and then free the structure". If you want to do that, perhaps > names that don't imply that they allocate and free a timezone_t should be > chosen. > > I.e., it would be > > timezone_t *tzalloc(const char *tzid); > > and > > void tzfree(timezone_t *); exactly, just like opendir() and closedir(). -------------- next part -------------- An HTML attachment was scrubbed... URL: From gharris at sonic.net Wed Jun 21 20:33:20 2023 From: gharris at sonic.net (Guy Harris) Date: Wed, 21 Jun 2023 13:33:20 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> <607AA65B-ECDF-473B-8B4E-8A99B56B65F1@sonic.net> Message-ID: <9A3B3756-7B05-4D9A-BA01-EEA6D8B4E42E@sonic.net> On Jun 21, 2023, at 1:17 PM, enh wrote: > NetBSD exposes its `struct _dirdesc` but FreeBSD/OpenBSD don't (though they use the same name for the private struct). Then I vote for doing as FreeBSD and OpenBSD and CupertinoBSD and systems using glibc do with DIR, rather than as NetBSD does with DIR, and have whatever the handle for a tzdb region is called be a pointer to an opaque structure. From eggert at cs.ucla.edu Wed Jun 21 21:18:26 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Wed, 21 Jun 2023 14:18:26 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> Message-ID: <6b8b6f43-8c53-42ad-a7e5-187b05997950@cs.ucla.edu> On 2023-06-21 12:50, enh wrote: > my preference would be for timezone_t to be more like DIR... > > typedef struct DIR DIR; Although that would also work, that's not how NetBSD did things when it introduced timezone_t in NetBSD 6.0 (2012), and tzcode copied this part of NetBSD in tzdb 2014g. I doubt whether we should change the meaning of timezone_t at this point, as it'd be too much backward-compatibility hassle. We could use a different name, though, for the struct. How about if we change this line in tzcode private.h: typedef struct state *timezone_t; to the following? typedef struct tm_timezone *timezone_t; and change all other instances of "struct state" to "struct tm_timezone"? This would let users write "struct tm_timezone *" and "timezone_t" interchangeably. The name tm_timezone, like timezone_t, is reserved by POSIX when you include [1], so if the implementation uses this name it won't break any conforming programs. [1]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_02_02 From enh at google.com Wed Jun 21 21:41:52 2023 From: enh at google.com (enh) Date: Wed, 21 Jun 2023 14:41:52 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <9A3B3756-7B05-4D9A-BA01-EEA6D8B4E42E@sonic.net> References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> <607AA65B-ECDF-473B-8B4E-8A99B56B65F1@sonic.net> <9A3B3756-7B05-4D9A-BA01-EEA6D8B4E42E@sonic.net> Message-ID: On Wed, Jun 21, 2023 at 1:33?PM Guy Harris wrote: > On Jun 21, 2023, at 1:17 PM, enh wrote: > > > NetBSD exposes its `struct _dirdesc` but FreeBSD/OpenBSD don't (though > they use the same name for the private struct). > > Then I vote for doing as FreeBSD and OpenBSD and CupertinoBSD and systems > using glibc do with DIR, rather than as NetBSD does with DIR, and have > whatever the handle for a tzdb region is called be a pointer to an opaque > structure. > you were right first time about macOS --- it's in the NetBSD camp (well, it inlines the struct definition in the typedef, so effectively the same). i haven't looked at glibc, but musl is in the FreeBSD/OpenBSD/MountainViewBsdOnLinux camp, so i assume glibc is too :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From enh at google.com Wed Jun 21 21:41:55 2023 From: enh at google.com (enh) Date: Wed, 21 Jun 2023 14:41:55 -0700 Subject: [tz] [libc-coord] thread-safe localtime() for an arbitrary timezone In-Reply-To: <6b8b6f43-8c53-42ad-a7e5-187b05997950@cs.ucla.edu> References: <2D1CCCE8-0F37-48B7-8820-4D3DA76342A1@sonic.net> <6b8b6f43-8c53-42ad-a7e5-187b05997950@cs.ucla.edu> Message-ID: On Wed, Jun 21, 2023 at 2:18?PM Paul Eggert wrote: > On 2023-06-21 12:50, enh wrote: > > my preference would be for timezone_t to be more like DIR... > > > > typedef struct DIR DIR; > > Although that would also work, that's not how NetBSD did things when it > introduced timezone_t in NetBSD 6.0 (2012), and tzcode copied this part > of NetBSD in tzdb 2014g. I doubt whether we should change the meaning of > timezone_t at this point, as it'd be too much backward-compatibility > hassle. > > We could use a different name, though, for the struct. How about if we > change this line in tzcode private.h: > > typedef struct state *timezone_t; > > to the following? > > typedef struct tm_timezone *timezone_t; > > and change all other instances of "struct state" to "struct tm_timezone"? > > This would let users write "struct tm_timezone *" and "timezone_t" > interchangeably. The name tm_timezone, like timezone_t, is reserved by > POSIX when you include [1], so if the implementation > uses this name it won't break any conforming programs. > (i just checked and NetBSD does have `typedef struct __state *timezone_t;` in its . my objection to that is that the compiler will actually use the underlying name in error messages, which is less than helpful if the names aren't close. so having s/state/tm_timezone/ seems like a step forward even just in that regard!) up to you (especially since my #define in "private.h" is unlikely to be much of a maintenance burden), but "i'd use this" (if only to get rid of my #define)... > [1]: > > https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_02_02 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From magfr at lysator.liu.se Tue Jun 27 07:19:49 2023 From: magfr at lysator.liu.se (Magnus Fromreide) Date: Tue, 27 Jun 2023 09:19:49 +0200 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: Message-ID: On Sat, Jun 17, 2023 at 08:20:53PM +0200, Anders Rosendahl via tz wrote: > Hello! > > I'm developing an application using the JavaScript Intl API (which is > relying on the IANA tz db) for converting between local time and UTC. > > I started receiving error reports from users about wrong calculations > (particularly from Denmark in the 1940's), and I discovered that in some > cases daylight saving time was applied in a way not in accordance to > historical facts. > > I investigated the IANA tz database files and I discovered that a change > has happened in version 2022b (in the file "europe"). Compared to the > earlier version (2022a), 4 zones seem to have their rules removed and > instead linked to Europe/Berlin: > > Link Europe/Berlin Arctic/Longyearbyen > Link Europe/Berlin Europe/Copenhagen > Link Europe/Berlin Europe/Oslo > Link Europe/Berlin Europe/Stockholm > > This, evidently, is not correct, and will result in erroneous results for > these zones. Oh dear. This horse has been beaten to death and then resurrected for another turn, the trenches are very deep. Basically, IANA tz only cares about dates after 1970-01-01 and for that reason the maintainer have started merging zones that don't differ after that date but does still provide the data but it won't be included in the default builds (which is what everyone gets from their updates) Other people have, just like you, said that this is a bad move and this have lead to a fork of the project but I do not remember where that is located, this fork is used in at least some of the BSDs. My personal opinion is that the default IANA build should say nothing about dates before 1970-01-01 - it should, in particular, not claim that changes for some random place (like Berlin) applies to some other place (like Copenhagen). /MF From gharris at sonic.net Tue Jun 27 07:30:26 2023 From: gharris at sonic.net (Guy Harris) Date: Tue, 27 Jun 2023 00:30:26 -0700 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: Message-ID: <5C993909-780C-47C7-867F-9FF4A762BF08@sonic.net> On Jun 27, 2023, at 12:19 AM, Magnus Fromreide via tz wrote: > My personal opinion is that the default IANA build should say nothing about > dates before 1970-01-01 - it should, in particular, not claim that changes > for some random place (like Berlin) applies to some other place (like > Copenhagen). And, presumably, it shouldn't make any other claims about Copenhagen before 1970-01-0, either. Or about Berlin, or about New York City, or any of the other cities chosen for tzdb region names, for that matter. (Not that Continent/City should be read as only making claims about that city.) (At this point, I wish we could get away with doing as Microsoft do, and just return -1 for all attempts to call localtime() for dates prior to the UNIX Epoch.) From scolebourne at joda.org Tue Jun 27 08:35:20 2023 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 27 Jun 2023 09:35:20 +0100 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: Message-ID: On Tue, 27 Jun 2023 at 08:20, Magnus Fromreide via tz wrote: > On Sat, Jun 17, 2023 at 08:20:53PM +0200, Anders Rosendahl via tz wrote: > Other people have, just like you, said that this is a bad move and this have > lead to a fork of the project but I do not remember where that is located, > this fork is used in at least some of the BSDs. https://github.com/JodaOrg/global-tz I don't want to maintain the fork, but I have no choice given the current situation at IANA tzdb is simply broken as the OP notes. > My personal opinion is that the default IANA build should say nothing about > dates before 1970-01-01 - it should, in particular, not claim that changes > for some random place (like Berlin) applies to some other place (like > Copenhagen). Indeed, this would be an acceptable and equitable outcome. It isn't ideal, as TZDB actually holds the data for pre-1970, but at least the pre-1970 data would be equally wrong everywhere, rather than favouring some places over others (which is my primary objection). On Tue, 27 Jun 2023 at 08:31, Guy Harris via tz wrote: > (At this point, I wish we could get away with doing as Microsoft do, and just return -1 for all attempts to call localtime() for dates prior to the UNIX Epoch.) The simplest approach would be to determine a rule, eg. - take the standard offset that applied on most days (modal average) for the 5 years from 1970 to 1975 - and use that pre-1970. - or, take the LMT of the city and round to the nearest hour So long as the answer is reasonable for most cases, it would be fine. Stephen From eggert at cs.ucla.edu Tue Jun 27 09:30:58 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Tue, 27 Jun 2023 02:30:58 -0700 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: Message-ID: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> On 2023-06-27 01:35, Stephen Colebourne via tz wrote: > The simplest approach would be to determine a rule, eg. > - take the standard offset that applied on most days (modal average) > for the 5 years from 1970 to 1975 - and use that pre-1970. > - or, take the LMT of the city and round to the nearest hour > So long as the answer is reasonable for most cases, it would be fine. This would take the existing TZif files (admittedly problematic, as you say) and make them worse, as they'd become wrong for every location, even the location that names the Zone. Surely it would be better to discard the pre-1970 data - then users would be on notice that it's missing. And there's a standard way to do that, documented in the Makefile: use 'make ZFLAGS=-r at 0'. Perhaps this option should be documented more prominently. It's not clear that -r at 0 should be the Makefile default, though, as that could well cause more trouble than it would cure. For example, it would cause the following behavior: $ export TZ=Europe/Copenhagen $ date -r 1; date -r 0; date -r -1 Thu Jan 1 01:00:01 CET 1970 Thu Jan 1 01:00:00 CET 1970 Wed Dec 31 23:59:59 -00 1969 and the UT offset zero and abbreviation -00 of pre-1970 timestamps would likely give many users pause. That being said, in installations not needing pre-1970 timestamps, -z at 0 is a clear win. Getting back to Anders's original question, one can get behavior closer to what he asked with something like this: make PACKRATDATA=backzone PACKRATLIST=zone.tab install although I don't recommend this for ordinary applications. One should treat the result with a grain of salt as it's likely wrong for Copenhagen and it's certainly a bit wrong for Aarhus etc. This is a hazard of trying to push tzdata further than it can reliably bear. From scolebourne at joda.org Tue Jun 27 11:09:12 2023 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 27 Jun 2023 12:09:12 +0100 Subject: [tz] Error in Scandinavian tz data In-Reply-To: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> Message-ID: On Tue, 27 Jun 2023 at 10:31, Paul Eggert wrote: > On 2023-06-27 01:35, Stephen Colebourne via tz wrote: > > The simplest approach would be to determine a rule, eg. > > - take the standard offset that applied on most days (modal average) > > for the 5 years from 1970 to 1975 - and use that pre-1970. > > - or, take the LMT of the city and round to the nearest hour > > So long as the answer is reasonable for most cases, it would be fine. > > This would take the existing TZif files (admittedly problematic, as you > say) and make them worse, as they'd become wrong for every location, > even the location that names the Zone. Being slightly wrong everywhere is a much better outcome than what we have today. > Surely it would be better to discard the pre-1970 data - then users > would be on notice that it's missing. And there's a standard way to do > that, documented in the Makefile: use 'make ZFLAGS=-r at 0'. Perhaps this > option should be documented more prominently. > > It's not clear that -r at 0 should be the Makefile default, though, as that > could well cause more trouble than it would cure. For example, it would > cause the following behavior: > > $ export TZ=Europe/Copenhagen > $ date -r 1; date -r 0; date -r -1 > Thu Jan 1 01:00:01 CET 1970 > Thu Jan 1 01:00:00 CET 1970 > Wed Dec 31 23:59:59 -00 1969 > > and the UT offset zero and abbreviation -00 of pre-1970 timestamps would > likely give many users pause. That being said, in installations not > needing pre-1970 timestamps, -z at 0 is a clear win. In most cases, end users do not pick what options to install with. They just get what they are given by their packager. Since the packager cannot know whether the end user wants pre-1970 data or not, a sensible packager will err on the side of providing more data, not less, and thus want to include pre-1970 data. 'make ZFLAGS=-r at 0' is of no interest to packagers precisely because it is obviously wrong. ie. choosing UTC does not make people think, it just means the make option is not used. It is simply not good enough to be a viable choice. It is also not close enough to any values that have previously been placed in long-term storage. Hopefully all this explains why "Surely it would be better to discard the pre-1970 data then users would be on notice that it's missing" (ZFLAGS=-r at 0) isn't a viable route forward. The rule-based truncation I outlined above is a compromise position suitable for use as the default in the makefile. I believe it meets the needs of packagers who would hopefully accept it. It provides truncated data pre-1970, but in a way that is not completely unreasonable: * It is good enough for most use cases except those that really care about historical detail * It avoids the weird per-second LMT offsets in the far past that often confuse end users * It is close enough to what end users have in long-term storage to not cause migration issues Just returning UTC does not meet these goals. > it's likely wrong for Copenhagen I think you do a disservice to TZDB's many authors here. I'd argue that the data for Copenhagen is likely to be entirely correct, as it has had many eyes on it for many years. Beyond that, I think there is also a need to recognise that TZDB's pre-1970 data is the de facto truth for large parts of the world. Most end users don't care about the accuracy, just that someone has made an effort to record it. I still believe that reinstating the data would be by far the best outcome, but a rules-based truncation approach would be a viable alternative if pre-1970 data is not something to be maintained. Stephen From paw at boel-it.dk Tue Jun 27 11:13:10 2023 From: paw at boel-it.dk (Paw Boel Nielsen) Date: Tue, 27 Jun 2023 11:13:10 +0000 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> Message-ID: > Beyond that, I think there is also a need to recognise that TZDB's > pre-1970 data is the de facto truth for large parts of the world. > Most end users don't care about the accuracy, just that someone > has made an effort to record it. I still believe that reinstating the > data would be by far the best outcome, but a rules-based truncation > approach would be a viable alternative if pre-1970 data is not > something to be maintained. Hear hear --------------------------- DISCLAIMER: The information contained in this electronic message and in any attachments to this message is intended only for the person or entity to which this electronic message is addressed. If you are not the intended recipient, you are hereby notified that any distribution, copying, review, retransmission, dissemination or other use of this electronic transmission or the information contained in it is strictly prohibited. -----Original Message----- From: tz On Behalf Of Stephen Colebourne via tz Sent: Tuesday, June 27, 2023 1:09 PM To: Time zone mailing list Subject: Re: [tz] Error in Scandinavian tz data On Tue, 27 Jun 2023 at 10:31, Paul Eggert wrote: > On 2023-06-27 01:35, Stephen Colebourne via tz wrote: > > The simplest approach would be to determine a rule, eg. > > - take the standard offset that applied on most days (modal > > average) for the 5 years from 1970 to 1975 - and use that pre-1970. > > - or, take the LMT of the city and round to the nearest hour So long > > as the answer is reasonable for most cases, it would be fine. > > This would take the existing TZif files (admittedly problematic, as > you > say) and make them worse, as they'd become wrong for every location, > even the location that names the Zone. Being slightly wrong everywhere is a much better outcome than what we have today. > Surely it would be better to discard the pre-1970 data - then users > would be on notice that it's missing. And there's a standard way to do > that, documented in the Makefile: use 'make ZFLAGS=-r at 0'. Perhaps this > option should be documented more prominently. > > It's not clear that -r at 0 should be the Makefile default, though, as > that could well cause more trouble than it would cure. For example, it > would cause the following behavior: > > $ export TZ=Europe/Copenhagen > $ date -r 1; date -r 0; date -r -1 > Thu Jan 1 01:00:01 CET 1970 > Thu Jan 1 01:00:00 CET 1970 > Wed Dec 31 23:59:59 -00 1969 > > and the UT offset zero and abbreviation -00 of pre-1970 timestamps > would likely give many users pause. That being said, in installations > not needing pre-1970 timestamps, -z at 0 is a clear win. In most cases, end users do not pick what options to install with. They just get what they are given by their packager. Since the packager cannot know whether the end user wants pre-1970 data or not, a sensible packager will err on the side of providing more data, not less, and thus want to include pre-1970 data. 'make ZFLAGS=-r at 0' is of no interest to packagers precisely because it is obviously wrong. ie. choosing UTC does not make people think, it just means the make option is not used. It is simply not good enough to be a viable choice. It is also not close enough to any values that have previously been placed in long-term storage. Hopefully all this explains why "Surely it would be better to discard the pre-1970 data then users would be on notice that it's missing" (ZFLAGS=-r at 0) isn't a viable route forward. The rule-based truncation I outlined above is a compromise position suitable for use as the default in the makefile. I believe it meets the needs of packagers who would hopefully accept it. It provides truncated data pre-1970, but in a way that is not completely unreasonable: * It is good enough for most use cases except those that really care about historical detail * It avoids the weird per-second LMT offsets in the far past that often confuse end users * It is close enough to what end users have in long-term storage to not cause migration issues Just returning UTC does not meet these goals. > it's likely wrong for Copenhagen I think you do a disservice to TZDB's many authors here. I'd argue that the data for Copenhagen is likely to be entirely correct, as it has had many eyes on it for many years. Beyond that, I think there is also a need to recognise that TZDB's pre-1970 data is the de facto truth for large parts of the world. Most end users don't care about the accuracy, just that someone has made an effort to record it. I still believe that reinstating the data would be by far the best outcome, but a rules-based truncation approach would be a viable alternative if pre-1970 data is not something to be maintained. Stephen From eggert at cs.ucla.edu Wed Jun 28 00:25:59 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Tue, 27 Jun 2023 17:25:59 -0700 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> Message-ID: <7e10f040-3255-7b70-f0a1-2c610847eabb@cs.ucla.edu> On 2023-06-27 04:09, Stephen Colebourne via tz wrote: > 'make ZFLAGS=-r at 0' is of no interest to packagers precisely because it > is obviously wrong. Then I must not be understanding "the default IANA build should say nothing about dates before 1970-01-01", a proposal that you said was acceptable. I thought "make ZFLAGS=-r at 0" would implement that proposal, as it builds TZif files that say nothing about timestamps before 1970. But since you're saying "make ZFLAGS=-r at 0" is of no interest, it seems the proposal is about something else. If so, it would be helpful to know what the proposal is. > Just returning UTC does not meet these goals. "make ZFLAGS=-r at 0" does not cause tzdb localtime to just return UTC, as localtime also returns an error indication indicating that the actual offset is unknown. This error indication is in the form of the time zone abbreviation "-00". > the data for Copenhagen is likely to be entirely correct I doubt that, as some of the data are sourced only from Shanks, which has proven to be unreliable. > there is also a need to recognise that TZDB's > pre-1970 data is the de facto truth That would not be accurate advertising. TZDB is only TZDB. It has never been "the de facto truth", unless "truth" means only "act like TZDB Release X". For many years TZDB's files have said that it is "by no means authoritative", and for many years TZDB installations have varied in minor ways from one platform to another. From scolebourne at joda.org Wed Jun 28 08:17:44 2023 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 28 Jun 2023 09:17:44 +0100 Subject: [tz] Error in Scandinavian tz data In-Reply-To: <7e10f040-3255-7b70-f0a1-2c610847eabb@cs.ucla.edu> References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> <7e10f040-3255-7b70-f0a1-2c610847eabb@cs.ucla.edu> Message-ID: On Wed, 28 Jun 2023 at 01:26, Paul Eggert wrote: > Then I must not be understanding "the default IANA build should say > nothing about dates before 1970-01-01", a proposal that you said was > acceptable. Re-reading the thread, I see where your confusion comes from. What I was agreeing with was the concept of removing attempts to be accurate pre-1970. I then pointed out why being completely inaccurate (all UTC) is not viable as the default. My proposal was: > The simplest approach would be to determine a rule, eg. > - take the standard offset that applied on most days (modal average) for the 5 years from 1970 to 1975 - and use that pre-1970. > - or, take the LMT of the city and round to the nearest hour > So long as the answer is reasonable for most cases, it would be fine. ie. I believe that if tzdb provided a fixed offset pre-1970, but one linked to LMT or common practice in that location, it would be a suitable approach for most packagers. I reckon to the nearest hour would be sufficient. Stephen From doug at ewellic.org Wed Jun 28 16:52:39 2023 From: doug at ewellic.org (Doug Ewell) Date: Wed, 28 Jun 2023 16:52:39 +0000 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> <7e10f040-3255-7b70-f0a1-2c610847eabb@cs.ucla.edu> Message-ID: Stephen Colebourne wrote: > Re-reading the thread, I see where your confusion comes from. What I > was agreeing with was the concept of removing attempts to be accurate > pre-1970. I then pointed out why being completely inaccurate (all UTC) > is not viable as the default. I think Paul noted already that ?-00? is not the same as UTC: >> "make ZFLAGS=-r at 0" does not cause tzdb localtime to just return UTC, >> as localtime also returns an error indication indicating that the >> actual offset is unknown. This error indication is in the form of the >> time zone abbreviation "-00". -- Doug Ewell, CC, ALB | Lakewood, CO, US | ewellic.org From eggert at cs.ucla.edu Wed Jun 28 17:27:56 2023 From: eggert at cs.ucla.edu (Paul Eggert) Date: Wed, 28 Jun 2023 10:27:56 -0700 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> <7e10f040-3255-7b70-f0a1-2c610847eabb@cs.ucla.edu> Message-ID: On 2023-06-28 01:17, Stephen Colebourne via tz wrote: > I believe that if tzdb provided a fixed offset pre-1970, but one > linked to LMT or common practice in that location For timestamps before 1970 we could approximate LMT along with a time zone abbreviation saying that the UT offset is approximate. Something equivalent to this, say: Zone America/New_York -5:00 - c-05 1970 Jan 1 0u -5:00 US E%sT Zone Asia/Tokyo 9:00 - c+09 1970 Jan 1 0u 9:00 Japan J%sT The idea would be that a time zone abbreviation "c+NN" stands for "circa +NN", where the exact offset is unspecified. Software that cares whether an offset is approximate could look in the abbreviation for leading "c+" and "c-" (along with looking for "-00", which would continue to have its current meaning). POSIX allows "c+" and "c-" at the start of time zone abbreviations so this would pass POSIX muster. This sort of thing could be generated automatically from the current data. From scolebourne at joda.org Wed Jun 28 19:49:50 2023 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 28 Jun 2023 20:49:50 +0100 Subject: [tz] Error in Scandinavian tz data In-Reply-To: References: <9d3d96d5-d375-6aae-e6bb-65e507ae0637@cs.ucla.edu> <7e10f040-3255-7b70-f0a1-2c610847eabb@cs.ucla.edu> Message-ID: On Wed, 28 Jun 2023 at 18:28, Paul Eggert wrote: > On 2023-06-28 01:17, Stephen Colebourne via tz wrote: > > I believe that if tzdb provided a fixed offset pre-1970, but one > > linked to LMT or common practice in that location > > For timestamps before 1970 we could approximate LMT along with a time > zone abbreviation saying that the UT offset is approximate. Something > equivalent to this, say: > > Zone America/New_York -5:00 - c-05 1970 Jan 1 0u > -5:00 US E%sT > > Zone Asia/Tokyo 9:00 - c+09 1970 Jan 1 0u > 9:00 Japan J%sT > > The idea would be that a time zone abbreviation "c+NN" stands for "circa > +NN", where the exact offset is unspecified. Software that cares whether > an offset is approximate could look in the abbreviation for leading "c+" > and "c-" (along with looking for "-00", which would continue to have its > current meaning). POSIX allows "c+" and "c-" at the start of time zone > abbreviations so this would pass POSIX muster. Or the name could be something consistent across the globe, like "LMT", such as "SET" (Solar estimated time). It would probably be easier to have one consistent name (like LMT) than a prefix. I think before progressing this further it would need other packagers to chime in and comment on whether they feel it would be acceptable as the default setting for tzdb pre-1970. Stephen