[gnso-rpm-wg] Critique of INTA survey

[Dorrain, Kristine]

Hi George,

As I recall, no one has offered this survey into evidence as conclusive proof of anything.  Both subteams (sunrise and claims) called for the WG to review as many data sources as possible, including mere anecdotes. Both subteams were well aware of the limited data available for scrutiny.  I think your (and Kurt's) observations are meaningful, in that they will spark a relevant discussion, but that does not mean the WG should ignore the user stories.  Thirty-three companies took significant time and resources to provide their feedback and it's incumbent upon us to at least listen. 

This WG, though the co-chairs, have asked Lori to present the survey.  I think our responsibility here is to listen respectfully and determine what we may be able to use or build on.

Thanks,

Kristine



-----Original Message-----
From: gnso-rpm-wg-bounces at icann.org [mailto:gnso-rpm-wg-bounces at icann.org] On Behalf Of George Kirikos
Sent: Wednesday, August 30, 2017 1:35 PM
To: gnso-rpm-wg <gnso-rpm-wg at icann.org>
Subject: Re: [gnso-rpm-wg] Critique of INTA survey

Hi Theo,

On Wed, Aug 30, 2017 at 4:23 PM, theo geurts <gtheo at xs4all.nl> wrote:
> From my perspective, there is this survey that George questions due to 
> the low amount of participants.

It wasn't just the small sample size. It's also the fact that it's a non-random, self-selected and entirely unrepresentative sample.

For example, if you wanted to determine the average height of adults in the USA, and survey 3000 current or former NBA players, that would be a large sample size. However, it would fail the other part, namely being non-random, entirely unrepresentative of the entire population of adults, etc.

The combination of both defects in the INTA study (small sample size, and also non-random) makes the study essentially worthless for saying anything about the larger population it purports to represent.

Lori talked about "Everything we do is a learning." The only learning that will happen tonight, unfortunately, is a lesson in how not to a proper survey. That might be of some value, in helping to design the future surveys (e.g. making them much shorter, to encourage greater number of responses; doing proper randomization, like the other study I mentioned in my first email of this thread, investing more money, etc.). Nielsen obviously knows how to do things right -- they're a highly respectable organization. Unfortunately in this instance, what they produced can't withstand scrutiny.

Sincerely,

George Kirikos
416-588-0269
http://www.leap.com/
_______________________________________________
gnso-rpm-wg mailing list
gnso-rpm-wg at icann.org
https://mm.icann.org/mailman/listinfo/gnso-rpm-wg