[RSSAC Caucus] Revisiting RSSAC002v3

[Paul Hoffman]

On Aug 19, 2019, at 8:23 AM, Ray Bellis <ray at isc.org> wrote:
> 
> On 19/08/2019 16:03, Paul Hoffman wrote:
> 
>> - v3 uses the term "latency" in a way that that is very different
>> from that which the Metrics Work Party is. In the v3 document,
>> "latency" is the delay in publication of a new root zone, whereas in
>> the Metrics WP document, "latency" is the amount of time it takes for
>> a response to a DNS query to be received. The latter is the much more
>> common usage of "latency" in the DNS world. The Metrics WP document
>> uses "staleness" for the concept from v3. I propose that v4 start
>> using "staleness" to match the more common usage in the DNS world.
>> Note that the metric doesn't need to be renamed: it remains
>> "load-time", which is different than either "latency" or
>> "staleness".
> 
> I don't like "staleness" either.
> 
> "stale" is already overloaded by the "serve-stale" model, where in that context "stale" really means "this zone is mouldy, use at your own risk" rather than "this zone isn't *quite* as fresh as it should be".
> 
> (in other words, a zone is not IMHO "stale" until its expiry timer has elapsed)
> 
> Why not just qualify latency as "publication latency" ?

This is possible if the Metrics WP also changes "latency" to "response latency" and "staleness" to "publication latency". The two documents would then align, and the community would more likely understand that there are two types of latency.

>> - To make it clearer that root zone size is not being measured by the
>> RSOs, I propose moving that entire discussion to its own section.
> 
> That's fine with me.
> 
>> - A second root zone metrics that might be measured is "number of
>> records changed". It will always be at least 1 (for the SOA), but it
>> might be useful to know how many root zones are just changing the SOA
>> versus changing actual data.
> 
> This is problematic.  There is no readily available in-server metric
> and/or log that contains this information.

Creating the metric is trivial. In fact, there is already a Twitter account that exists pretty much for it.

>> - The DNS community would probably like to watch the adoption rate
>> for QNAME minimization. To measure that, I propose a new RSO
>> measurement: number of queries that are single-label requests that
>> result in NOERROR responses. The latter part would eliminate effects
>> of things like Chrome asking for more names. I recognize that this
>> idea might not be popular for RSSAC, but it would be one of the best
>> ways for the DNS community to measure uptake of the protocol.
> 
> I'm also against this.  It pre-supposes that either an on-the-wire DNS
> monitoring system or in-server metrics to measure this are deployed.

Agree with that pre-supposition.

> So long as wire-sniffing systems remain viable a long term view of this can be gleaned from DITL data, even if it is only sampled once a year.
> 
> On a more general note, my take on RSSAC 002 (and I said so vociferously during the v3 discussion) is that it was not intended for answering arbitrary research questions.  That just becomes a never-ending list of "wouldn't it be nice if..." discussions.

That view seems reasonable to me, but then it would argue for the removal of the root zone size metric as well.

--Paul Hoffman