[gnso-rpm-wg] Critique of INTA survey
icann at leap.com
Thu Aug 31 04:23:22 UTC 2017
On Wed, Aug 30, 2017 at 10:54 PM, Greg Shatan <gregshatanipc at gmail.com> wrote:
> Random samples are preferable as a starting point, but even there there are
> various types of biases. One is "non-response bias" -- that members of the
> selected sample don't respond to the survey. [George seems to ignore the
> distinction between the initial sample and the responses, but that wouldn't
> serve his purposes....]
The report made not attempt to adjust for any of those biases, like
self-selection bias, or that it was unrepresentative of the larger
(1000+ members) group of INTA members.
I didn't "ignore" anything. To anticipate every argument in the first
email I sent, and make more in depth would have distracted from the
key points. They'd have been lost in a 50 page email.
> This is an important point -- the initial sample is really the 1000+ members
> of INTA, which is in turn a subset of all trademark owners. The group of
> respondents is a subset of that sample.
How is that an "important point"? We have 33 observations. Is that in
dispute? We know those 33 are unrepresentative of the 1000+ members
of INTA. And we know from the metrics of INTA that they're
unrepresentative of all TM holders (i.e. INTA's membership is skewed
large, and the 33 were skewed even larger).
What attempts were made to "correct" (if it's even possible) for that?
None. It's a lost cause, because there just wasn't enough data to
begin with to make those assertions about the larger groups in a
statistically robust manner.
If ICA or EFF had submitted 33 self-selected survey responses that
were skewed in a similar manner, and tried to make assertions about a
large population, I'd be equally hard on them, as I've shown in the
past through my balanced criticisms. My focus is on the pure math,
regardless of where it came from or what the results said.
If/when Lori/Nielsen ever comes back to us with the margins of error
that were requested in the other thread, those will help demonstrate
things further (although Kurt already took an initial stab at things
with his calculations; they'd be different for each slide, though).
More information about the gnso-rpm-wg