[SLE Team] FW: [CWG-Stewardship] Marc's presentation and report with some background

[Paul M Kane - CWG]

Trang

Many thanks for the email reminder.

I sent out the presentation some weeks ago to the members directly (I was
experiencing mailing list issues so direct guaranteed circulation) and there is
interest in having a call before or meeting at ICANN Marrakech to discuss.
Fundamentally, despite being offered access to the raw data mid December, then
12 Jan, then end Jan.  

Many of the Working Group will be in Marrakech and I found our Dublin meeting
with Akram very useful so would it be possible to arrange a similar meeting
(same ICANN staff list) in Marrakech.

I believe Jay, Patricio, Elaine and the two Jeffs will be present.  I will not
be in Marrakech but it would be good for you to hear from other members of the
Working Group. 

With regard to the substantive (now annulled by ICANN) on the 4th Feb a
consultant gave a presentation to say that it was too complicated and the raw
IANA transaction data would not be made available to us.  

I think the WG members would have preferred to have seen the raw data and come
to that conclusion that the data was not fit for the intended purpose
themselves. If your consultant is controllable because you have constrained him
via NDA, I am sure many if not all of the volunteer WG members would have
entered into an appropriate NDA which would have saved ICANN money by not having
pay external consultants as well as build some good PR too.

Kind regards

Paul.




Quoting Trang Nguyen <trang.nguyen at icann.org>:

> Dear members of the DT-A,
> 
> The below email and attached information were recently circulated to the
> CWG mail list and we want to make sure that you have this information as
> well.
> 
> This information provides background and results of the data parsing work
> that was recently completed.
> 
> If you have any questions after reviewing this information, please let us
> know and we¹d be happy to provide a response.
> 
> Please note that the next CWG call is scheduled for 1600 UTC on February
> 25. ICANN staff will be providing an update on the next phase of work: SLE
> metric collection and SLA setting. We encourage you to join the call and
> participate in that discussion.
> 
> Thank you,
> 
> Trang Nguyen
> ICANN
> 
> 
> 
> -----Original Message-----
> From: <cwg-stewardship-bounces at icann.org> on behalf of David Conrad
> <david.conrad at icann.org>
> Date: Thursday, February 11, 2016 at 8:48 AM
> To: CWG Mailing List <cwg-stewardship at icann.org>
> Subject: [CWG-Stewardship] Marc's presentation and report with
> some	background
> 
> >Hi,
> >
> >On the 76th CWG call on February 4th
> >(https://community.icann.org/x/nJBlAw), Marc Blanchet of Viagenie gave a
> >presentation on the work ICANN contracted them to perform that attempted
> >to derive approximations of the SLEs using the current RZMS and RT logs
> >and databases.  Attached is the PowerPoint deck that Marc used for his
> >presentation as well as the final report detailing his analyses. I am
> >circulating these reports for your review.
> >
> > 
> >Based on the questions raised during the CWG call, it seems there might
> >have been some confusion around why that work was undertaken and how it
> >relates to the implementation of the SLEs. I am providing some background
> >here to help remove any confusion. My apologies in advance for the length
> >of this message, however I believe it important for there to be clarity
> >on this issue.
> >
> >1. Before the SLEs were developed, ICANN staff informed the DT-A that the
> >current RZMS and RT systems collect performance metrics as directed by
> >NTIA and our own internal requirements. The DT-A felt these metrics were
> >insufficient to ensure IANA performance met community requirements and
> >that new metrics would be necessary. ICANN staff informed the DT-A that
> >changes to performance metrics would require code changes to the RZMS.
> > 
> >
> >2. After the SLEs were developed, ICANN staff informed the CWG that due
> >to the number of simultaneous demands placed upon ICANN to safely
> >and securely modify systems and processes to meet transition
> >requirements, ICANN staff estimated the code changes to RZMS would be
> >completed by the end of March 2016. Some in the CWG suggested ICANN add
> >additional staff to the RZMS development team in an attempt to deploy the
> >code sooner. ICANN informed the CWG that doing so would more likely
> >result in development taking longer since any new developers would need
> >to become familiar with the existing code base and this familiarization
> >would necessarily involve interruptions to the existing development team,
> >delaying their efforts.
> > 
> >
> >3. During ICANN 54, ICANN staff were informed of a new requirement that
> >SLE data must be collected for a period of 6 months before SLAs could be
> >set. ICANN staff was further informed that the March 2016 timeframe was
> >unacceptable since a 6-month data collection requirement would lead to
> >insufficient time being available to incorporate the SLE-derived SLAs
> >into the ICANN-PTI contract. A request was made to ICANN to make
> >available RZMS and RT raw data so that SLEs could be extracted from data
> >collected by the current RZMS/RT systems. Due to the confidential nature
> >of the root zone change request data, which includes email discussions
> >between ICANN staff and the requesters in which potentially business
> >proprietary details of registry operation are disclosed, ICANN staff
> >informed the CWG chairs and DT-A that releasing the raw RZMS/RT data
> >would be a violation of existing IANA policy and thus, would not be
> >possible, particularly to any organization that competes in the domain
> >name space. ICANN staff also again explained that the RZMS/RT systems do
> >not currently collect data the way the SLEs were defined and the primary
> >task for ICANN was to modify RZMS in order to collect the new SLEs.
> > 
> >
> >4. As a compromise to try to address the new 6-month data collection
> >requirement, ICANN staff contracted with Viagenie as an independent and
> >neutral third-party to explore whether the data collected by the current
> >RZMS/RT system could be used to "seed" some portion of the SLEs, thereby
> >reducing the data collection time requirement.
> > 
> >
> >5. Viagenie completed their work and presented a summary on the Feb 4th
> >CWG call, confirming that: (a) The current RZMS/RT systems do not collect
> >data in accordance with the newly defined SLEs; (b) The heuristics
> >developed provided approximations for most metrics, but some
> >approximations were less conclusive; and (c) The RZMS/RT tool is a
> >complex system that frequently relies on email interactions for
> >progressing request state.
> > 
> >
> >6. In parallel to the work ICANN contracted Viagenie to perform, ICANN
> >has continued to pursue modifying the RZMS system to collect data in the
> >way that the SLEs were defined. ICANN now expects that the RZMS
> >modifications will be completed at the end of February, a month ahead of
> >schedule. Data collection for the new SLEs can thus begin in March.
> >Assuming everything goes well, this would allow for sufficient time for
> >SLE data collection without delay of the transition in order to meet the
> >6-month SLE data collection requirement.
> > 
> >
> >The main approach to implementing the SLEs had always been to make code
> >changes to the RZMS so that it can capture processing time data that the
> >DT-A defined for all future change request processing. ICANN is on-track
> >to have this work completed by the end of this month, February.
> > 
> >
> >If the CWG would like any additional clarification around the work
> >performed by Viagenie, the RZMS development work, or other aspects of the
> >technical implementation of the transition, please let ICANN staff know
> >and we will be happy to provide clarification.
> > 
> >
> >Regards,
> >-drc
> >ICANN CTO
> > 
> >
> 
>