[NCAP-Discuss] ICANN OCTO responses to request for technical expertise

[Thomas, Matthew]

NCAP DG,

I wanted to share with you the responses received from ICANN OCTO for a request to provide some technical expertise around any known technical challenges or problems with the proposed delegation strategies related to the passive measurement system (e.g., Ad measurements).

Matt Thomas

---
ICANN OCTO first response below:
---

Thank you for your request for OCTO to preview the NCAP delegation proposal. As you discuss in the proposal, all the configurations would represent a change from the current behavior (the root sending authoritative NXDOMAIN replies). Changing this behavior could potentially cause disruption to some systems, so the benefit of this new behavior has to be weighed against the risk of disruption. We note that some in the DNS-OARC technical community have brought up similar issues in their Mattermost discussion system.

However, the NCAP delegation proposal says nothing concrete about the expected benefits. The introduction talks about “name collision assessment purposes” and “collision measurement”, but doesn't say what benefit will come from them. Based on the text of the proposal and also based on further context we are aware of from following NCAP, the purpose of this proposal is to collect additional data that will later be analyzed for possible value. However, we have yet to see a description that goes into specifics on the planned use for the data. Our opinion is therefore that since specific benefits arising from gathering additional data have not been specified, the possible risk of disruption from the changed behavior is not justified.

Further, OCTO notes that the measurements taken from such delegations are more easily gamed than measurements taken from captures at the root servers. (We do recognize the difficulty involved in capturing data at the root because of the large number of instances.) That is, someone who wants to sway the outcome of the measurements will be more able to do so if the TLDs in question are delegated to a small number of authoritative name servers. The proposal from NCAP is silent on the measurement problems that the proposal will suffer from and how these problems might be addressed.

We would like to take this opportunity to express our concern that this proposal represents a departure from controlled interruption, a technique that has worked successfully for over 1200 new delegations in the past decade. Proposals for divergence from controlled interruption that could cause new risks of disruption need to be justified with specific benefits.

----
Below is a follow up response from ICANN OCTO after sending some additional context about the measurement proposal:
----

After reviewing the ad measurement proposal for additional context, we stand by our original response. Our position is that any change to root zone operational behavior first requires an analysis to determine if the risk is worth the intended benefits. The proposal would result in additional data being collected without specifying how the data will be interpreted or even what the benefits of having the additional data would be. As a result, the proposal does not supply enough information to perform a risk analysis, so the risk of any changed behavior is not justified.

Specifically, the ad measurement proposal describes a new measurement of what percentage of ads referencing a new TLD string cause DNS queries to be received at the TLD’s authoritative servers, but does not describe how results are to be interpreted. The proposal calls for a risk assessment based on both passive data collection and ad-based measurement data, but does not describe the specific criteria for the risk assessment. On the contrary, the proposal specifically states that the criteria need to be developed.

We repeat the conclusion from our initial response, which we believe is still valid, if not even more so in light of the additional ad measurement proposal: both passive data collection and ad measurement represent a departure from controlled interruption, a technique that has worked successfully for over 1200 new delegations in the past decade. Proposals for divergence from controlled interruption that could cause new risks of disruption need to be justified with specific benefits.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mm.icann.org/pipermail/ncap-discuss/attachments/20220801/ff61fdcb/attachment.html>