[gnso-rpm-wg] Misleading Slides (was Re: URS Practitioners Survey Results and Presentation for ICANN62)

[George Kirikos]

Hi folks,

In preparation for today's call, I reviewed the slides, and compared
them with the "raw" results in the 2nd PDF. It's important to note
that on the agenda page on the wiki
(https://62.schedule.icann.org/meetings/699479) only the summary
presentation is linked to, and not the raw results (which we received
by email).

The presentation is highly misleading (by omission), in particular for
page 6, where it purports to neutrally summarize all the respondents
to the survey. But, the actual survey question it's reporting was
(from page 1 of the raw results) "In how many URS proceedings have you
been involved as Complainant or its representative?" The related
question (from page 2) "2. In how many URS proceedings have you been
involved as Respondent or its representative?" was very clear that
there was not a single response from practitioners who've represented
Respondents."

By omitting the actual questions from the survey, the presentation
makes it appear (to someone who has not done a deep dive into the raw
results) that this was a balanced survey of practitioners who
represent both complainants and respondents to URS disputes. That
could not be further from the truth, and so the results must be taken
with a grain of salt. This was the equivalent of doing a survey of
criminal attorneys, with all the responses from criminal prosecutors
and without any responses from criminal defense lawyers.

Furthermore, if we go to the raw questions, we had explicitly
contemplated receiving input from those who'd only been involved in 1
or 2 cases (i.e. the "answer choices" for the first two questions
were: 1 to 2, 3 to 5, 5 to 10, 10 or more, none). Yet, somehow in the
implementation of the survey, it only "selected URS practitioners  who
had handled 5 cases or more" (page 5 of the presentation). This
further skewed the results, and was inconsistent with the intent of
the survey's questions.

By the way, the survey claims to have "34 practitioners" on the "final
list", but page 5 actually lists 36 in the right hand column. There's
no explanation for that discrepancy.

In summary, this appears to be yet another unrepresentative small
sample, that is ultimately destined for the dust bin (along with the
previously discredited INTA study).

ICANN staff should add the PDF with the raw results to the agenda
page, lest anyone be misled by this presentation.

Sincerely,

George Kirikos
416-588-0269
http://www.leap.com/


On Thu, Jun 21, 2018 at 11:27 AM, Julie Hedlund <julie.hedlund at icann.org> wrote:
> Dear RPM PDP Working Group members,
>
>
>
> On behalf of the URS Practitioners Sub Team, please see the attached
> documents for your reference for the presentation that the Sub Team will
> provide to the Working Group at ICANN62 during the following session:
>
>
>
> Wednesday, 27 June, 10:30-12:00 (Panama City UTC-5): RPM PDP WG Meeting
> Session 1: 1) Presentation from URS Practitioners Sub Team re: Survey
> Results; 2) Update from TMCH Data Sub Team; 3) Update from URS Documents Sub
> Team
>
> see: https://62.schedule.icann.org/meetings/699479
>
>
>
> The attachments are the results of the responses to the URS practitioners
> survey, and the slides for the Sub Team’s presentation.
>
>
>
> Best regards,
>
> Mary, Ariel, Berry, and Julie
>
>
> _______________________________________________
> gnso-rpm-wg mailing list
> gnso-rpm-wg at icann.org
> https://mm.icann.org/mailman/listinfo/gnso-rpm-wg