[NCAP-Discuss] NCAP notes: 22 February 2023

Thu Mar 2 18:42:52 UTC 2023

Jim:

> Are users likely to see those “breadcrumbs” and thus actually use them?
> No.

One of our areas of disagreement is that (IMO) you overestimate the “notification effectiveness” of the honeypot approach and underestimate the notification effectiveness of CI. Because very little of the colliding traffic is actually browser-y, and all modern browsers will prefer https, it is extremely, extremely unlikely that *any* end users in normal course will actually see the honeypot content. Both Casey and I have written about this ad nauseum. We are therefore left to communicate with administrators through logs.

> When I define user I’m imagining an actual physical person.
> The question, in my mind, is whether or not that actual physical
> person is going to understand what is happening when a name
> collision happens and thus be able to take direct action to resolve
> the “problem” they are having.

I agree with you, “Mabel in Accounting” is not the intended audience for the notification. The intended audience is some IT guy or gal that has a chance of understanding the problem and fixing it. That’s how I define “user.”

Those are the very people that search google, stackoverflow.com, serverfault.com, etc, and are most likely to benefit from the tens of thousands of references therein. These are the people that start looking at logs when things break – thus the “interrupt and communicate through logs” approach.

If you drill into the “google 127.0.53.53” experience – as this group did in NCAP Study 1, and as JAS did in our “Phase 2 report” – we see evidence that this does indeed happen. Those people are getting both the notification and the information they need. There are many, many, many more examples of this desired experience than the microscopic sample of people that filled-out the ICANN form claiming otherwise. (Notwithstanding the fact that they probably found ICANN and the form by Google!)

Again, I’m all for notification. Notification is a Very Good Thing. We all want that. The disagreement is about whether the *effectiveness* of the notification may be improved. Given DNS has no end user signaling channel, we’re all being creative to layer on these sorts of comms in “clever” ways. It will be a risk-reward balancing act.

> that are certainly no worse than CI and absolutely improve its effectiveness
> in response to the changed DNS infrastructure of today. You may have a
> different opinion.

I strongly object to blanket statements like this. “Absolutely improve its effectiveness” is objectively false. Hoping real hard that it’s an improvement - without facts or operational experience - does not equate to such a disingenuous and completely unsupported blanket statement. I don’t know how we’re still saying things like this as a group – if you don’t like or believe the JAS, Verisign, and Casey commentary on this. . . I’m not sure how to help.

CI is an imperfect solution to a problem for which there are no perfect solutions. Try as we may, we have yet to identify an objectively superior solution to CI.

Slide 1 of my comparison deck (re-attached here, first shared with NCAP in Feb 2022 and largely ignored) would be good content for the NCAP report. Also see Casey’s 2/16/23 email to the list, which is clear and well-reasoned. In my opinion, this group needs to debate the *facts* of the attached and Casey’s recent 16 Feb email. Upon agreement of baseline facts, we can consider paths forward. In the current state where factions are operating from different facts, consensus on the path forward is impossible.

Jeff

From: James Galvin <galvin at elistx.com>
Date: Thursday, March 2, 2023 at 10:37 AM
To: Jeff Schmidt <jschmidt at jasadvisors.com>
Cc: Casey Deccio <casey at deccio.net>, ncap-discuss at icann.org <ncap-discuss at icann.org>
Subject: Re: [NCAP-Discuss] NCAP notes: 22 February 2023

Two comments.

First, any editorial suggestions you have for how to make the point are most welcome.

Second, I think we’re talking past each other and in fact we are in violent agreement. I surmise that the problem is how we are each defining “user notification”.

When I define user I’m imagining an actual physical person. The question, in my mind, is whether or not that actual physical person is going to understand what is happening when a name collision happens and thus be able to take direct action to resolve the “problem” they are having.

So, is controlled interruption (CI) disruptive? Yes.

Is CI impactful to users? Yes.

Does CI leave helpful “breadcrumbs” for users? Yes.

Can users follow those “breadcrumbs” and get help with their issue? Yes.

Are users likely to see those “breadcrumbs” and thus actually use them? No.

In my opinion, it is self-evident that the answer to the last question is “no”. You may have a different opinion.

In my opinion, based on 5 decades of experience with technology and people who know nothing about technology except maybe how to use it, if you don’t actually put instructions in front of people you get very little reaction. I don’t think it takes any form of academic research to accept and believe that is true, i.e., it is self-evident. CI does not put instructions in front of people. It leaves breadcrumbs that may or may not be seen and followed. In contrast, the proposed enhancements seek to put actual instructions in front of users in a few select cases. The proposed enhancements will still leave breadcrumbs, like CI, in many cases, but in a few select cases they have a higher probability of getting instructions to users.

CI is not effective at user notification because it does not directly reach the user. It leaves “breadcrumbs”, which you could make the case are a form of user notification. However, if user is a physical person, then CI is simply not effective. You may have a different opinion.

In conclusion, CI is certainly a choice for ACA. This study is simply proposing enhancements, adding features to the particular CI implementation choice made in 2012 (which personally I believe were great choices in 2012), that are certainly no worse than CI and absolutely improve its effectiveness in response to the changed DNS infrastructure of today. You may have a different opinion.

Please propose text for the study final report.

Jim

On 1 Mar 2023, at 14:34, Jeff Schmidt wrote:
Googling “127.0.53.53” yields 14,600+ results right now:

+ 70+ threads from stackoverflow.com (common IT forum)
+ 162+ threads from serverfault.com (another common IT forum)
+ Microsoft’s technet forum
+ Google’s chromium forum
+ ICANN’s own information

Compare this to the truly microscopic sample of research conducted based on reports to ICANN.

Any statement that CI is somehow “not effective for user notification” is simply wrong and not supported quantitatively or qualitatively.

Sure we all want things to be “better” but these blanket statements are unhelpful, factually wrong, and disingenuous.

Jeff

From: NCAP-Discuss <ncap-discuss-bounces at icann.org> On Behalf Of Casey Deccio
Sent: Wednesday, March 1, 2023 1:14 PM
To: James Galvin <galvin at elistx.com>
Cc: ncap-discuss at icann.org
Subject: Re: [NCAP-Discuss] NCAP notes: 22 February 2023

Jim,

Part of our disagreement is that we are understanding terms very differently.

To me, "user notification" is ambiguous.  It might refer to either or both of the following: disruption, which falls under the "effective alerting coverage" category (i.e., "something is wrong"); or root cause identification, which leads to the user to know what caused the issue.  This is why the root causes analysis uses both "disruption" and "root cause identification".

Additionally, the finding quoted below has been quoted in isolation.  Please see other findings for more context to support this.  For example: "Usage of private DNS suffixes colliding with newly delegated TLDs has decreased over time."  The analysis shows that private use of DNS suffixes has decreased over time, and the only mechanism that has been deployed thus far is controlled interruption.  I would not therefore call that "ineffective."

Precision in language is important, particularly as we are categorically evaluating past and future methods.

Casey

On Mar 1, 2023, at 11:41 AM, James Galvin <galvin at elistx.com<mailto:galvin at elistx.com>> wrote:

These notes do not capture an important point that I made during the meeting.

These notes say:
Finding E.a: Controlled Interruption is not effective for user notification: Casey believes this is a misrepresentation and noted the root cause analysis data supports his statement on this.

The notes do not capture the point I made:
This finding is an accurate reflection supported by the Root Cause Analysis research.

It would seem that Casey and I are drawing different conclusions from the report. However, in partial support of my support for the Study 2 Finding I submit the following Root Cause Analysis Finding (page 47):
Controlled interruption is effective at disruption, but not at root cause identification.
Controlled interruption has shown to be good at disruption, but not at helping affected users identify the cause of the problem—at least not in the way that was intended. Evidences. Of the survey respondents that indicated that they used of TLDs, 70% reported having experienced technical issues related to their suffix. Of those, 43% experienced the problems within days of delegation of the TLD. Over two-thirds (71%) of organizations experiencing technical problems indicated that they knew that the issues were related to TLD delegation before the problem was resolved. It appears that most of the ineffectiveness was due to the controlled interruption IP address not even being observed, which occurred in 71% of cases, according to the survey. However, when the controlled interruption IP address was observed, the success rate in identifying ICANN and controlled interruption as the cause was between 50% and 76%, according to the survey results and the Web search results analysis, respectively.

I also made the point during the meeting that it seemed to me the issue being debated was simply the wording of the Study 2 Finding, not its factual assertion. I said then and I’m repeating now that I welcome suggestions for how to restate the Study 2 Finding.

Thanks,
Jim

On 23 Feb 2023, at 6:02, Jennifer Bryce wrote:
Dear NCAP Discussion Group members,

The notes from the 22 February NCAP Discussion Group meeting are attached and will be posted to the meeting record on the wiki: https://community.icann.org/x/4YTKDQ.

The recording can be directly accessed here: https://icann.zoom.us/rec/play/sbcwTpyP5v0lmz_OTQFZOsWgFe056FPBSFGAo0pkQjKZ3WEmrCnLCmvgqEwLbFLTiQ619ysohCmhR35l.oNOIneK13LjTRyFh?continueMode=true&_x_zm_rtaid=69f-25JgT9uQd9f13bYzDw.1677121850114.64d0c334f967482c6f176722421c11dd&_x_zm_rhtaid=110

One action item was captured:

Action item: Discussion Group members should review Section 4: Findings<https://docs.google.com/document/d/1H89gyy31dGVXQjn304PEGv85EKIpHUAR3vPm76YQMcc/edit> and make comments in the document to highlight the areas where they have concerns. This will aid discussion next week.

Best,
Jennifer
--
Jennifer Bryce
Project Manager, Office of the Chief Technology Officer (OCTO)
Internet Corporation for Assigned Names and Numbers (ICANN)

Skype: jennifer.bryce.icann
Email: jennifer.bryce at icann.org<mailto:jennifer.bryce at icann.org>
_______________________________________________
NCAP-Discuss mailing list
NCAP-Discuss at icann.org<mailto:NCAP-Discuss at icann.org>
https://mm.icann.org/mailman/listinfo/ncap-discuss
_______________________________________________
By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.
_______________________________________________
NCAP-Discuss mailing list
NCAP-Discuss at icann.org<mailto:NCAP-Discuss at icann.org>
https://mm.icann.org/mailman/listinfo/ncap-discuss

_______________________________________________
By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mm.icann.org/pipermail/ncap-discuss/attachments/20230302/5c9862c0/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Honeypot cost-benefit[97].pptx
Type: application/vnd.openxmlformats-officedocument.presentationml.presentation
Size: 58283 bytes
Desc: Honeypot cost-benefit[97].pptx
URL: <https://mm.icann.org/pipermail/ncap-discuss/attachments/20230302/5c9862c0/Honeypotcost-benefit97-0001.pptx>