<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Never trust a study that you have not tweaked/interpreted towards
the desired results yourself...<br>
</p>
<br>
<div class="moz-cite-prefix">Am 30.08.2017 um 19:47 schrieb Kurt
Pritz:<br>
</div>
<blockquote type="cite"
cite="mid:E91DBE29-7024-41BA-8B30-275B62FB059D@kjpritz.com">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
Hi Everyone:
<div class=""><br class="">
</div>
<div class="">With the exception of the hyperbole at the outset
and some slight garbling on confidence level definitions, I
largely agree with George.
<div class=""><br class="">
</div>
<div class="">The extremely low response rate and (more
importantly) lack of randomness in the sample of responders
essentially prohibits one from drawing any conclusions about
the population as a whole. IF the selection was random, then
the survey margin of error would be about +/- 18% with a 95%
confidence level. This would mean we are 95% sure the margin
of error is 18% or less and there is a 5% chance the margin of
error is greater. </div>
<div class=""><br class="">
</div>
<div class="">However, “random" means that we picked the 33
members that responded essentially out of a hat filled with
the names of the 6600 members. But that is not what happened.
The survey probably received responses from their most
DNS-savvy members - those that found the purpose of the survey
interesting or where the questions seemed more
straight-forward. This significantly skews the results.
Georges email demonstrates this in more detail.</div>
<div class=""><br class="">
</div>
<div class="">I don’t think the survey deals with the skewed
data set fairly or honestly. The survey characterizes the
findings as traits of the entire membership rather than as
traits of the population that responded. This can’t be
defended.</div>
<div class=""><br class="">
</div>
<div class="">For instance, I don’t think it is correct to say:
"Vast majority (97%) of <b class=""><i class="">members</i></b>
registered domain names in past 24 months, with 9 in 10
registering new TLDs,“ and "9 in 10 <b class=""><i class="">members</i></b>
have registered new TLD domains in the past two years in the
Sunrise Period." I think new TLD owners would be very
surprised (and happy) to hear this. It would be accurate to
say that "97% of the <b class=""><i class="">respondents</i></b>
registered….” (See slides 9 and 11)</div>
<div class=""><br class="">
</div>
<div class="">More harmful to the credibility of the study are
statements such as: </div>
<ul class="">
<li class="">3 in 4 members (76%) have incurred costs for
internet monitoring of trademarks in the past 2 years, with
more than half (57%) of the members spending $10k or more.
(see slide 12)</li>
<li class="">On average, INTA members spend $150,000 per year
on defensive actions (see slides 10 & 27)</li>
</ul>
These are the types of quotes that find themselves into print
and become believed. (“INTA members spend $150K each in
defensive efforts, a ~$1 billion cost to industry!”) As George
noted, as a rule, larger companies responded and so it can not
established by this survey that each of the remaining (smaller)
INTA members average $150,000 per year in defensive spend.
<div class=""><br class="">
</div>
<div class="">There is another interesting facet to the asserted
$150K / year spend rate. One company spent $5.2MM. Assuming
this $5.2MM spend was over a two-year period, that means that
the other 32 respondents averaged (33 x $292K — $5.2MM) / (32
x 2) = $69,000 / year. So except for one outlier, the per
year spend by the brand owners that chose to answer the study
is half of what the study states. Why didn’t the study make
this clear? (see slide 10). </div>
<div class=""><br class="">
</div>
<div class="">I am not sure of the purpose of the study but
there are uses that can be made of it: </div>
<div class=""><br class="">
</div>
<div class="">There was one conclusion I could draw. It states
that UDRP and Sunrise were the favored rights protection
mechanisms, used to a major or moderate extent by 67% and 64%
of the respondents respectively. The next most utilized RPMs
were Trademark Claims and URS (by 36% and 27% respectively).
To me this says that, to those who are in-the-know, Sunrise is
a highly-valued RPM and, therefore, should be continued.
(Sorry, George) (see slides 15 and 51) </div>
<div class=""><br class="">
</div>
<div class="">Also, the study makes one fact clear that we have
already supposed: that business are not aware of new gTLDs and
domain utility in general. There are several data sets that
point to this. Rather than education efforts that identify
costs and target abuse prevention and mitigation only, Brand
education could also describe the benefits of domains as
strategic tools, that provide greater access to products and
indicia of reliability to brands’ customers.</div>
<div class=""><br class="">
</div>
<div class="">I know this was way pedantic. Sorry. I can’t be on
the call as it is at 4AM my time but I’d be pleased to respond
to comments or questions. </div>
<div class=""><br class="">
</div>
<div class="">Best regards,</div>
<div class=""><br class="">
</div>
<div class="">Kurt</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Aug 29, 2017, at 10:05 PM, George
Kirikos <<a href="mailto:icann@leap.com" class=""
moz-do-not-send="true">icann@leap.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">Hi folks,<br class="">
<br class="">
I'm not sure how many have had a chance to read
the INTA materials for<br class="">
tomorrow's call yet, or have any background in
statistics, but the<br class="">
survey has truly deep and fatal flaws, making any
conclusions drawn<br class="">
from it entirely unreliable and non-robust.<br
class="">
<br class="">
I could write 50 pages on this (I've read the
report three times now,<br class="">
in horror), but I'll keep it relatively brief (and
make these<br class="">
statements in advance of the call, so that Lori or
INTA/Nielsen have a<br class="">
chance to rebut).<br class="">
<br class="">
The entire basis of statistical inference is that
one can make<br class="">
statements about an entire population with a
certain level of<br class="">
confidence using only data from a subset of that
population (i.e. the<br class="">
sample in question). Prerequisites are that (a)
the sample be random,<br class="">
and (b) the sample be of sufficient size. INTA's
study fails on both<br class="">
counts (self-selected and unrepresentative sample,
and a mere 33<br class="">
responses).<br class="">
<br class="">
INTA claims to represent 7,000 organizations as
members:<br class="">
<br class="">
<a
href="https://www.inta.org/About/Pages/Overview.aspx"
class="" moz-do-not-send="true">https://www.inta.org/About/Pages/Overview.aspx</a><br
class="">
<br class="">
While they acknowledge on page 5 of the slides the
small sample size<br class="">
and suggest "some caution", alarm bells should be
ringing regarding<br class="">
that small sample size. Page 6 then demonstrates
how unrepresentative<br class="">
and non-random that sample is, with 52% of the 33
respondents having<br class="">
total revenue exceeding $5 billion/year, and a
whopping 77% (27%+52%)<br class="">
having revenues exceeding $1 billion. This is
hardly representative of<br class="">
typical TM owners. Similarly, 39% of this sample
had 25,000 or more<br class="">
employees, and 78% (39%+39%) had 5,000 or more
employees.<br class="">
<br class="">
All throughout the report, the slides say "INTA
members" (i.e. wrongly<br class="">
attempting to extrapolate and assert a truth about
the entire<br class="">
population, rather than limiting the statements to
be applicable only<br class="">
to the sample of 33 respondents).<br class="">
<br class="">
Basic sanity checks were not done with those<br
class="">
extrapolations/inferences. On page 25, the report
asserts that "more<br class="">
than 4 in 10 members have applied to operate a new
TLD"?<br class="">
<br class="">
45% of 7000 members implies 3,150 INTA members
applied for new gTLDs.<br class="">
That's not correct. The total applications by
everyone was 1930 -- see<br class="">
<a class="moz-txt-link-freetext" href="https://newgtlds.icann.org/en/program-status/statistics">https://newgtlds.icann.org/en/program-status/statistics</a>, and the<br
class="">
number by brand owners is a subset of that total
(664 according to<br class="">
<a class="moz-txt-link-freetext" href="https://icannwiki.org/Brand_TLD">https://icannwiki.org/Brand_TLD</a> and that will be a
bit high, due to<br class="">
multiple applications). If one extrapolated that
to the entire<br class="">
universe of trademark holders (i.e. including
non-INTA members),<br class="">
millions of TM owners, it would be even more
obvious how<br class="">
unrepresentative and non-random the data in this
sample is relative to<br class="">
a "typical" TM holder. This sample is highly
skewed to the largest of<br class="">
the large organizations who happened to
self-select a response to this<br class="">
survey.<br class="">
<br class="">
All throughout the report, important data on
confidence intervals is<br class="">
missing, obscuring the fact that the level of
confidence is extremely<br class="">
low (and the margin of error is high) due to the
small sample size.<br class="">
[confidence intervals are statements like "+/- 5%,
19 times out of 20]<br class="">
<br class="">
There are actually calculators that let one know
how big a sample<br class="">
should be, in order to have a certain level of
confidence and/or a<br class="">
margin of error.<br class="">
<br class="">
e.g. see:
<a class="moz-txt-link-freetext" href="https://www.surveymonkey.com/mp/sample-size-calculator/">https://www.surveymonkey.com/mp/sample-size-calculator/</a><br
class="">
<br class="">
For a population size of 7000 members (INTA's
total membership) and a<br class="">
95% confidence level, with a huge 10% margin of
error, you'd still<br class="">
need 95 survey responses. Yet, there were only 33
responses. This is<br class="">
particularly important to be kept in mind for
charts with percentages<br class="">
(pp. 17 and beyond), where the margin of error,
even if sampled<br class="">
properly, would be enormous. Furthermore, those
would have had to<br class="">
have been RANDOMLY sampled responses to be proper,
which we know isn't<br class="">
the case. If you wanted smaller margins of error,
say +/- 5%, you need<br class="">
an even larger sample size (in this case, 365).
Another useful<br class="">
calculator is at:<br class="">
<br class="">
<a class="moz-txt-link-freetext" href="https://www.surveymonkey.com/mp/margin-of-error-calculator/">https://www.surveymonkey.com/mp/margin-of-error-calculator/</a><br class="">
<br class="">
ICANN has done surveys, by Nielsen even, that
didn't suffer from these<br class="">
deficiencies, e.g. see:<br class="">
<br class="">
<a class="moz-txt-link-freetext" href="https://newgtlds.icann.org/en/reviews/cct/registrant-survey-faqs-25sep15-en">https://newgtlds.icann.org/en/reviews/cct/registrant-survey-faqs-25sep15-en</a><br
class="">
<br class="">
A key takeaway from that work was "Due to a low
response rate to<br class="">
emailed invitations to complete the survey, ICANN
then worked with<br class="">
Domain Tools to procure a larger sample of WHOIS
records." They took<br class="">
greater care in that study to have *randomized*
samples, too, along<br class="">
with the larger sample size.<br class="">
<br class="">
While it is somewhat interesting to have a glimpse
into brand<br class="">
protection of some of the largest companies,
ultimately this study is<br class="">
not robust.<br class="">
<br class="">
In summary, any conclusions from this INTA study
really need to be<br class="">
taken with a grain of salt, due to the small
sample size, combined<br class="">
with the non-random and unrepresentative sample
itself. Indeed, many<br class="">
of the conclusions need to be read as the
*opposite* of what the study<br class="">
suggests (i.e. if defensive costs are $150K/year
for companies with $5<br class="">
billion+ in revenues, that's a drop in the bucket,
and would be much,<br class="">
much smaller for a "typical" TM owner). To correct
these deficiencies,<br class="">
future surveys need to be random (easily done,
e.g. random sample the<br class="">
USPTO database or other national registries) and
have a much larger<br class="">
sample size. Understandably, that costs money, but
that's what it<br class="">
takes to do things properly.<br class="">
<br class="">
Sincerely,<br class="">
<br class="">
George Kirikos<br class="">
416-588-0269<br class="">
<a class="moz-txt-link-freetext" href="http://www.leap.com/">http://www.leap.com/</a><br class="">
<br class="">
<br class="">
On Mon, Aug 28, 2017 at 4:02 PM, Mary Wong
<a class="moz-txt-link-rfc2396E" href="mailto:mary.wong@icann.org"><mary.wong@icann.org></a> wrote:<br class="">
<blockquote type="cite" class="">Dear all,<br
class="">
<br class="">
<br class="">
<br class="">
The proposed agenda for our next Working Group
call, scheduled for 0300 UTC<br class="">
on Thursday 31 August, is as follows:<br
class="">
<br class="">
<br class="">
<br class="">
Roll call (via Adobe Connect and phone bridge
only); updates to Statements<br class="">
of Interest<br class="">
Review and discuss results of INTA Cost Impact
Survey<br class="">
Next steps/next meeting<br class="">
<br class="">
<br class="">
<br class="">
For Agenda Item #2, please review the survey
results here:<br class="">
<a class="moz-txt-link-freetext" href="https://community.icann.org/download/attachments/61606864/INTA%20Cost%20Impact%20Report%20revised%204-13-17%20v2.1.pdf?version=1&modificationDate=1500376749000&api=v2">https://community.icann.org/download/attachments/61606864/INTA%20Cost%20Impact%20Report%20revised%204-13-17%20v2.1.pdf?version=1&modificationDate=1500376749000&api=v2</a><br
class="">
<br class="">
<br class="">
<br class="">
Lori Schulman of INTA, and a member of this
Working Group, also did a<br class="">
presentation of the results to the Competition,
Consumer Protection &<br class="">
Consumer Trust (CCT) Review Team recently that
may be helpful to review:<br class="">
<a class="moz-txt-link-freetext" href="https://community.icann.org/download/attachments/61606864/ICANN%20New%20gTLD%20Survey%20Update%2010May%20Final.pdf?version=1&modificationDate=1501098808000&api=v2">https://community.icann.org/download/attachments/61606864/ICANN%20New%20gTLD%20Survey%20Update%2010May%20Final.pdf?version=1&modificationDate=1501098808000&api=v2</a>.<br
class="">
We are hopeful that Lori will be able to join us
for this call, to<br class="">
facilitate our review and discussion.<br
class="">
<br class="">
<br class="">
<br class="">
Thanks and cheers<br class="">
<br class="">
Mary<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
_______________________________________________<br
class="">
gnso-rpm-wg mailing list<br class="">
<a class="moz-txt-link-abbreviated" href="mailto:gnso-rpm-wg@icann.org">gnso-rpm-wg@icann.org</a><br class="">
<a class="moz-txt-link-freetext" href="https://mm.icann.org/mailman/listinfo/gnso-rpm-wg">https://mm.icann.org/mailman/listinfo/gnso-rpm-wg</a><br class="">
</blockquote>
_______________________________________________<br
class="">
gnso-rpm-wg mailing list<br class="">
<a class="moz-txt-link-abbreviated" href="mailto:gnso-rpm-wg@icann.org">gnso-rpm-wg@icann.org</a><br class="">
<a class="moz-txt-link-freetext" href="https://mm.icann.org/mailman/listinfo/gnso-rpm-wg">https://mm.icann.org/mailman/listinfo/gnso-rpm-wg</a><br
class="">
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
gnso-rpm-wg mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gnso-rpm-wg@icann.org">gnso-rpm-wg@icann.org</a>
<a class="moz-txt-link-freetext" href="https://mm.icann.org/mailman/listinfo/gnso-rpm-wg">https://mm.icann.org/mailman/listinfo/gnso-rpm-wg</a></pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Bei weiteren Fragen stehen wir Ihnen gerne zur Verfügung.
Mit freundlichen Grüßen,
Volker A. Greimann
- Rechtsabteilung -
Key-Systems GmbH
Im Oberen Werk 1
66386 St. Ingbert
Tel.: +49 (0) 6894 - 9396 901
Fax.: +49 (0) 6894 - 9396 851
Email: <a class="moz-txt-link-abbreviated" href="mailto:vgreimann@key-systems.net">vgreimann@key-systems.net</a>
Web: <a class="moz-txt-link-abbreviated" href="http://www.key-systems.net">www.key-systems.net</a> / <a class="moz-txt-link-abbreviated" href="http://www.RRPproxy.net">www.RRPproxy.net</a>
<a class="moz-txt-link-abbreviated" href="http://www.domaindiscount24.com">www.domaindiscount24.com</a> / <a class="moz-txt-link-abbreviated" href="http://www.BrandShelter.com">www.BrandShelter.com</a>
Folgen Sie uns bei Twitter oder werden Sie unser Fan bei Facebook:
<a class="moz-txt-link-abbreviated" href="http://www.facebook.com/KeySystems">www.facebook.com/KeySystems</a>
<a class="moz-txt-link-abbreviated" href="http://www.twitter.com/key_systems">www.twitter.com/key_systems</a>
Geschäftsführer: Alexander Siffrin
Handelsregister Nr.: HR B 18835 - Saarbruecken
Umsatzsteuer ID.: DE211006534
Member of the KEYDRIVE GROUP
<a class="moz-txt-link-abbreviated" href="http://www.keydrive.lu">www.keydrive.lu</a>
Der Inhalt dieser Nachricht ist vertraulich und nur für den angegebenen Empfänger bestimmt. Jede Form der Kenntnisgabe, Veröffentlichung oder Weitergabe an Dritte durch den Empfänger ist unzulässig. Sollte diese Nachricht nicht für Sie bestimmt sein, so bitten wir Sie, sich mit uns per E-Mail oder telefonisch in Verbindung zu setzen.
--------------------------------------------
Should you have any further questions, please do not hesitate to contact us.
Best regards,
Volker A. Greimann
- legal department -
Key-Systems GmbH
Im Oberen Werk 1
66386 St. Ingbert
Tel.: +49 (0) 6894 - 9396 901
Fax.: +49 (0) 6894 - 9396 851
Email: <a class="moz-txt-link-abbreviated" href="mailto:vgreimann@key-systems.net">vgreimann@key-systems.net</a>
Web: <a class="moz-txt-link-abbreviated" href="http://www.key-systems.net">www.key-systems.net</a> / <a class="moz-txt-link-abbreviated" href="http://www.RRPproxy.net">www.RRPproxy.net</a>
<a class="moz-txt-link-abbreviated" href="http://www.domaindiscount24.com">www.domaindiscount24.com</a> / <a class="moz-txt-link-abbreviated" href="http://www.BrandShelter.com">www.BrandShelter.com</a>
Follow us on Twitter or join our fan community on Facebook and stay updated:
<a class="moz-txt-link-abbreviated" href="http://www.facebook.com/KeySystems">www.facebook.com/KeySystems</a>
<a class="moz-txt-link-abbreviated" href="http://www.twitter.com/key_systems">www.twitter.com/key_systems</a>
CEO: Alexander Siffrin
Registration No.: HR B 18835 - Saarbruecken
V.A.T. ID.: DE211006534
Member of the KEYDRIVE GROUP
<a class="moz-txt-link-abbreviated" href="http://www.keydrive.lu">www.keydrive.lu</a>
This e-mail and its attachments is intended only for the person to whom it is addressed. Furthermore it is not permitted to publish any content of this email. You must not use, disclose, copy, print or rely on this e-mail. If an addressing or transmission error has misdirected this e-mail, kindly notify the author by replying to this e-mail or contacting us by telephone.
</pre>
</body>
</html>