[gnso-rpm-wg] TMCH data on abandonment

[George Kirikos]

Hi folks,

On Fri, Jun 9, 2017 at 10:56 AM, Kurt Pritz <kurt at kjpritz.com> wrote:
> We can ask for a study, but I am pretty darned sure the information we want
> is not available. We have no data on this issue to guide the discussion.

There actual *is* a source of data that can help determine whether
there is a significant difference caused by the TM Claims notice, in
particular, the monthly ICANN Registry reports:

https://www.icann.org/resources/pages/registry-reports

Let's pick out an example report, for .xyz in Feb 2017:

https://www.icann.org/resources/pages/xyz-2014-06-19-en

There's a transactions report:

https://www.icann.org/sites/default/files/mrr/xyz/xyz-transactions-201702-en.csv

which has columns which include not just "net adds" (new domains
registered), but more importantly "attempted-adds".

One can take the ratio of adds vs attempted adds (across all
registrars, or filtering for some that might exclude outliers) and use
that as a proxy for a "conversion rate", and compare that over time,
and across multiple TLDs.

If folks are familiar with the concept of "event studies":

https://en.wikipedia.org/wiki/Event_study
http://www.investopedia.com/terms/e/eventstudy.asp
http://www.bauer.uh.edu/rsusmel/phd/lecture%206.pdf
http://web.mit.edu/doncram/www/eventstudy.html

used in empirical research, the "event" is essentially the ending of
the claims period (i.e. if there's an impact, we should see a
statistical difference in the above ratio in the period from 0-3
months after GA when the Claims Notices are shown to registrants, and
in the period 4 months+ after GA when the Claims Notices are NOT shown
to registrants. We have time series data, and a cross-section of TLDs
(akin to different "stocks", when those are financial event studies).

One would have to make some minor adjustments to take into account
that the end of the claims notices doesn't fall "neatly" at the exact
end of a month. And of course the first month of the GA (where there's
much more activity after a launch) might be expected to be different
from months 2 and 3, in terms of those averages. But, probably months
2 and 3 should be "normal" for a claims notice effect, and months 4
and beyond "normal" for a non-claims notice effect. One would also
need to take into account possible effects from GA "anniversary dates"
(i.e. GA+12 months, +24 months, +36 months), where data might be
skewed somewhat by large amounts of expiring domain names ("drop
catching" might impact "attempted adds", if they're automated
inquiries).

So, I do believe we do have a valuable and valid source of data *now*,
namely running an event study with the above data and methodology,
comparing adds to attempted adds using each of the registry's public
historical data provided to ICANN in the monthly reports. It can even
be compared against .com/net/org/biz/info etc., which have no claims
notice at all, to see if there are statistical differences.

It's a lot of grunt work (I won't be doing it), but something a
graduate statistics/econometrics student would be capable of doing
easily. All the data is already in electronic form, too (although, one
would need to get exact "GA" dates from the registry operators or
ICANN, or IBM/Deloitte, to calculate GA+3 months properly).

It's not as "direct" as getting the data directly from registrars (I'm
not sure why they'd not share that data; they can either do it
anonymously, or just share the "change", without sharing their actual
cart abandonment data), but it should suffice, to help solve the
question *now*, in this PDP, rather than waiting another 10 or 20
years.

Sincerely,

George Kirikos
416-588-0269
http://www.leap.com/