[NCAP-Discuss] [Ext] Re: An Approach to Measuring Name Collisions Using Online Advertisement

Thu Jun 9 00:47:53 UTC 2022

Hi Jeff,
I don't know if it's up to the DG to evaluate the ad tool, but I like Geoff Huston's suggestion that we run a "test case" on an undelegated string.   My best guess is that the SSAC and/or the Technical Review Team would need to do the final evaluation of that tool, but I defer to the Co-Chairs.

In my opinion, the problem we are solving is that the risk assessment used in the 2012 round was not systematic.  Perhaps the best evidence of this is that strings which arguably carry more collision risk than .mail were delegated and .mail was not.  Another problem we are trying to solve is the bad experience of some organizations as documented in Casey's research.  And another problem we are trying to solve is we have no idea what sort of harm may have occurred to consumers.  No one at ICANN marketed this topic to end users of concern to ALAC (and others) and asked them to file reports on their experiences.

What is different now is we are called upon to develop a systematic process for risk assessment.   That comes with a "MUST" in the Sub Pro Final Report Recommendations.  If you want the Board to lockup and defer going forward on the next round, your approach makes sense.  If you want the next round to move forward, there needs to be consistency between the answers to the  questions the Board put to the SSAC and the "MUST" part of the Sub Pro Recommendations.

Another point of doing this risk assessment in a systematic manner is that if the string is characterized as what the Board refers to in its questions to the SSAC as  a "Collision String" (e.g. a high risk string like maybe .corp or .home), then the Applicant can elect not to proceed to evaluation (and will hopefully get its application money refunded promptly).  Having this option be systematic would be different from the 2012 round and the advice is codified and sits before the Board in the form of Sub Pro Implementation Guidance 29.5.  (As a reminder, the word "should" in the Sub Pro Final Report means that it's expected to be done unless there is a documented compelling reason not to do it.)

I think the last thing we want as a community is for the Board to come back with "What is your documented compelling reason for not developing the test described in Implementation Guidance 29.5?"  That would just result in more delay.  Or from the ICANN Board -  "Well, given the questions we posed to the SSAC and the Advice provided by the SSAC, the GAC and the ALAC, we really can't move forward with the next round until you provide us with the risk assessment tool set out in the Sub Pro Final Report as a "MUST" because "We, the Board, don't make policy and so the community has to go back to the drawing board."  In short, the community has to learn to resolve its differences before they get to the Board level, so I'm happy you are participating in good faith toward that end.  Said another way, community participants are hopefully mature enough to resolve these difference without fighting in front of the "parents" and expecting them to resolve the fights.

I'm very interested in the observation you made that everything in the Passive Collision Assessment section Matt explained is "nothing new" (though that terminology is a bit derogatory, as is "science experiment". )  I guess that means that Interisle and JAS already know how to supply that PCA analysis to the Technical Review Team and could do that for a fee pursuant to an awarded RFP.  Is that correct?  At least they could offer to perform that service if it doesn't involve the ad measurement piece?  (As I understand it, the purpose of PCA is to identify very  high risk strings that likely should not move forward.)

Thank you,
Anne

Anne E. Aikman-Scalese

Of Counsel

AAikman at lewisroca.com<mailto:AAikman at lewisroca.com>

D. 520.629.4428

[cid:image003.png at 01D87B5C.0E334B80]

From: NCAP-Discuss <ncap-discuss-bounces at icann.org> On Behalf Of Jeff Schmidt via NCAP-Discuss
Sent: Wednesday, June 8, 2022 11:17 AM
To: Steve Sheng <steve.sheng at icann.org>; ncap-discuss at icann.org
Subject: Re: [NCAP-Discuss] [Ext] Re: An Approach to Measuring Name Collisions Using Online Advertisement

[EXTERNAL]
________________________________

While leveraging the Google ad network to collect data is clever, we have absolutely no idea what sorts of data it will generate and if the data is at all useful for the purpose. It's truly a science experiment. Not that I'm opposed to science projects or evaluating new approaches, but this is something that needs to be production ready and defensible if NCAP is going to recommend it.

We know a lot about the other collision metrics we've discussed: they've been used for decades or longer, published, peer-reviewed, and are back-test-able. Throwing new, unproven, un-back-test-able Google ad network data into the mix at this late date, with no history, guidelines, thresholds, etc. - as a gatekeeper in the application process - seems a recipe for disaster. We can't even discuss thresholds or criteria in advance because we have no idea what the data will look like! And it's not testable until after the first delegation so we have to recommend it first then hope for the best. We have no baseline and nothing to compare it to. What is "good" and what is "bad?"

Also, the Google ad network approach will only tell us about a subset of unknown representation of networks/infrastructure where Google ad processing browsers are attached. False negatives and false positives (result is not representative of local network segment/infrastructure) may occur in situations where the DNS resolution infrastructure is unrelated to the local network segment (people using GoogleDNS, CloudflareDNS, OpenDNS, on a VPN, etc) - all increasingly common. Additionally, Google ad networks are blocked in not insignificant networks globally.

Also, I believe this approach relies on a unique relationship between Google and a researcher (allowing JavaScript network libraries to run in ads that are typically blocked) and this approach may not be directly transferrable to other ad networks (where those unique relationships may not exist). Unfortunately, if limited to Google (itself a large Registry Operator), perceptions of conflicts of interest and unequal access to data may become an issue.

I have my doubts if this adds anything to our understanding of collisions; certainly one wonders if the proverbial juice is worth the squeeze. See page 25 of our final report:

"Even though all of our HTTP honeypot pages contained the overt request to contact
us, JAS received not a single notification. Reviewing our HTTP logs, less than 8% of
DNS resolutions ultimately led to the retrieval of one of our HTTP honeypot pages.
Reviewing the HTTP logs further, less than 12% of those 8% reported an HTTP
user-agent that could be considered a user-facing application (i.e. a Browser)."

Net-net: The vast majority of the things requesting colliding DNS names weren't human-facing/browser-y things. The things that would be running Google ads.

Finally, this approach is ripe for gaming and will be gamed if it is a gatekeeper in a future application round. All the approaches to ad-fraud / click-fraud might be used to steer this type of study any way an unscrupulous actor desires. The folks with experience running ad network research have never had to deal with adversaries actively working against them seeking to manipulate the results.

This is a solution in search of a problem, IMHO. What specific problems are we solving and is a science experiment justified?

Jeff

From: Steve Sheng <steve.sheng at icann.org<mailto:steve.sheng at icann.org>>
Date: Friday, April 29, 2022 at 9:32 AM
To: Jeff Schmidt <jschmidt at jasadvisors.com<mailto:jschmidt at jasadvisors.com>>, ncap-discuss at icann.org<mailto:ncap-discuss at icann.org> <ncap-discuss at icann.org<mailto:ncap-discuss at icann.org>>
Subject: Re: [Ext] Re: [NCAP-Discuss] An Approach to Measuring Name Collisions Using Online Advertisement

You don't often get email from steve.sheng at icann.org<mailto:steve.sheng at icann.org>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>

Hi Jeff,

  This is not SSAC recommendation or an official SSAC work product, but a thought piece generated by some SSAC members.

  I hope this clarifies, let me know if you have further questions.

Best
Steve

From: Jeff Schmidt <jschmidt at jasadvisors.com<mailto:jschmidt at jasadvisors.com>>
Date: Thursday, April 28, 2022 at 6:08 PM
To: Steve Sheng <steve.sheng at icann.org<mailto:steve.sheng at icann.org>>, "ncap-discuss at icann.org<mailto:ncap-discuss at icann.org>" <ncap-discuss at icann.org<mailto:ncap-discuss at icann.org>>
Subject: [Ext] Re: [NCAP-Discuss] An Approach to Measuring Name Collisions Using Online Advertisement

For clarity, is this an "SSAC Recommendation" or any sort of official SSAC work-product, or is it a thought piece generated by people that happen to also be SSAC Members?

Thx,
Jeff

From: NCAP-Discuss <ncap-discuss-bounces at icann.org<mailto:ncap-discuss-bounces at icann.org>> on behalf of Steve Sheng <steve.sheng at icann.org<mailto:steve.sheng at icann.org>>
Date: Thursday, April 28, 2022 at 4:57 PM
To: ncap-discuss at icann.org<mailto:ncap-discuss at icann.org> <ncap-discuss at icann.org<mailto:ncap-discuss at icann.org>>
Subject: [NCAP-Discuss] An Approach to Measuring Name Collisions Using Online Advertisement

You don't often get email from steve.sheng at icann.org<mailto:steve.sheng at icann.org>. Learn why this is important [aka.ms]<https://urldefense.com/v3/__https:/aka.ms/LearnAboutSenderIdentification__;!!PtGJab4!p3DXeMB6HOS0UqNysHwCmwPhWuQvaDpsGkjieh_W0vvXPhgDbXClUE1yeuD1-om5PzXKFbw$>

Dear NCAP Discussion Group,

   The SSAC NCAP WP, a group of SSAC members who are actively following and participating in the NCAP work, have developed a proposal to measure name collisions using advertisement-based measurement. It will complement the passive collision analysis as discussed in the discussion group by providing more direct data on the collision rates for a given candidate TLD string as well as  variance in collision rates between countries and between individual networks. The data may also help mitigation of name collisions.

   Please see the attached proposal for your consideration.

Best
Steve Sheng
On behalf of SSAC NCAP WP

________________________________

This message and any attachments are intended only for the use of the individual or entity to which they are addressed. If the reader of this message or an attachment is not the intended recipient or the employee or agent responsible for delivering the message or attachment to the intended recipient you are hereby notified that any dissemination, distribution or copying of this message or any attachment is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the sender. The information transmitted in this message and any attachments may be privileged, is intended only for the personal and confidential use of the intended recipients, and is covered by the Electronic Communications Privacy Act, 18 U.S.C. ?2510-2521.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mm.icann.org/pipermail/ncap-discuss/attachments/20220609/ad883522/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 2031 bytes
Desc: image003.png
URL: <https://mm.icann.org/pipermail/ncap-discuss/attachments/20220609/ad883522/image003-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 217 bytes
Desc: image004.png
URL: <https://mm.icann.org/pipermail/ncap-discuss/attachments/20220609/ad883522/image004-0001.png>