[gnso-rpm-wg] Critique of INTA survey

[George Kirikos]

P.S. There are roughly 8 hours to go until our scheduled call. I would
invite Lori and/or INTA to simply withdraw the paper from this PDP
(and the other ICANN group to which it was presented), since
ultimately it is not a scientifically valid study. Any conclusions
from it are indefensible.

It would bring more credibility to INTA to withdraw it, in my opinion,
recognizing it as deeply flawed now, rather than to attempt to defend
it for 90 minutes tonight, and ultimately see it abandoned/ignored by
the PDP. As a group, we're always seeking efficiencies --- withdrawing
this paper and giving everyone back their Wednesday night appears to
me to be "low hanging fruit" in that regard.

The sooner it's withdrawn, the more time folks will have to make
arrangements to enjoy their Wednesday evening.

Sincerely,

George Kirikos
416-588-0269
http://www.leap.com/


On Wed, Aug 30, 2017 at 2:13 PM, George Kirikos <icann at leap.com> wrote:
> Hi Kurt,
>
> Thanks for mostly agreeing with my analysis. However:
>
> On Wed, Aug 30, 2017 at 1:47 PM, Kurt Pritz <kurt at kjpritz.com> wrote:
>> There was one conclusion I could draw. It states that UDRP and Sunrise were
>> the favored rights protection mechanisms, used to a major or moderate extent
>> by 67% and 64% of the respondents respectively. The next most utilized RPMs
>> were Trademark Claims and URS (by 36% and 27% respectively). To me this says
>> that, to those who are in-the-know, Sunrise is a highly-valued RPM and,
>> therefore, should be continued. (Sorry, George) (see slides 15 and 51)
>
> The first part of your conclusion is correct (obviously anyone who
> personally benefits from "front of the line" privileges is going to
> value it), but the second part (therefore, that it should be
> continued) is NOT correct. As a PDP, our job is to weigh the benefits
> against the costs of policy choices amongst ALL stakeholders, not just
> ones receiving benefits.
>
> Thus, if that was "the one conclusion (you) could draw", and it's now
> debunked, then we're left with the truth, that no conclusions can be
> drawn from it --- it's for entertainment value only, i.e. it's an
> advocacy piece, marketing fluff, not a scientifically-valid survey
> that would endure any serious peer review from those in the field of
> statistics.
>
> To be clear, I tried to keep yesterday's email as short as possible
> (remember, it was a response to a very long document), and didn't
> point out every flaw with the survey. To point out another, note that
> on page 6 it notes that 67% of responses were from USA and Canada.
> However, INTA's own website states that:
>
> https://www.inta.org/Membership/Pages/Membership.aspx
>
> "63% of our member organizations are outside of North America."
>
> This further reinforces my point that it was an unrepresentative
> sample. As we know from election polling, the survey companies make
> adjustments in weighting to attempt to compensate for the
> unrepresentative samples (e.g. if too many men were sampled relative
> to the known proportion, they'd lower the weights accordingly, etc.).
> No attempts were made to do this (nor could they credibly have done
> so, given the small sample size, and lack of randomness).
>
> This is a classic case of "If you torture the data long enough, it
> will confess to anything."
>
> Sincerely,
>
> George Kirikos
> 416-588-0269
> http://www.leap.com/