[CWG-Stewardship] FW: Marc's presentation and report with some background

[Paul M Kane - CWG]

Thanks David

Great news the RZMS modifications will be completed at the end of February so
data collection can start.

Sorry for delay - there is a gremlin somewhere on the CWG mailing list....  I
receive direct emails to my but not those distributed via the CWG mailing list
at the moment... 

Best

Paul

Quoting Grace Abuhamad <grace.abuhamad at icann.org>:

> Hi Paul ‹ Forwarding this to you in case you did not receive through
> mailing list. Please respond to the mailing list so that the rest of the
> group and the original sender are included. There have been no replies to
> date. I will do my best to forward you any replies to the thread
> considering your issue with receiving email from the mailing list.
> 
> ‹Grace
> 
> On 2/11/16, 11:48 AM, "cwg-stewardship-bounces at icann.org on behalf of
> David Conrad" <cwg-stewardship-bounces at icann.org on behalf of
> david.conrad at icann.org> wrote:
> 
> >Hi,
> >
> >On the 76th CWG call on February 4th
> >(https://community.icann.org/x/nJBlAw), Marc Blanchet of Viagenie gave a
> >presentation on the work ICANN contracted them to perform that attempted
> >to derive approximations of the SLEs using the current RZMS and RT logs
> >and databases.  Attached is the PowerPoint deck that Marc used for his
> >presentation as well as the final report detailing his analyses. I am
> >circulating these reports for your review.
> >
> > 
> >Based on the questions raised during the CWG call, it seems there might
> >have been some confusion around why that work was undertaken and how it
> >relates to the implementation of the SLEs. I am providing some background
> >here to help remove any confusion. My apologies in advance for the length
> >of this message, however I believe it important for there to be clarity
> >on this issue.
> >
> >1. Before the SLEs were developed, ICANN staff informed the DT-A that the
> >current RZMS and RT systems collect performance metrics as directed by
> >NTIA and our own internal requirements. The DT-A felt these metrics were
> >insufficient to ensure IANA performance met community requirements and
> >that new metrics would be necessary. ICANN staff informed the DT-A that
> >changes to performance metrics would require code changes to the RZMS.
> > 
> >
> >2. After the SLEs were developed, ICANN staff informed the CWG that due
> >to the number of simultaneous demands placed upon ICANN to safely
> >and securely modify systems and processes to meet transition
> >requirements, ICANN staff estimated the code changes to RZMS would be
> >completed by the end of March 2016. Some in the CWG suggested ICANN add
> >additional staff to the RZMS development team in an attempt to deploy the
> >code sooner. ICANN informed the CWG that doing so would more likely
> >result in development taking longer since any new developers would need
> >to become familiar with the existing code base and this familiarization
> >would necessarily involve interruptions to the existing development team,
> >delaying their efforts.
> > 
> >
> >3. During ICANN 54, ICANN staff were informed of a new requirement that
> >SLE data must be collected for a period of 6 months before SLAs could be
> >set. ICANN staff was further informed that the March 2016 timeframe was
> >unacceptable since a 6-month data collection requirement would lead to
> >insufficient time being available to incorporate the SLE-derived SLAs
> >into the ICANN-PTI contract. A request was made to ICANN to make
> >available RZMS and RT raw data so that SLEs could be extracted from data
> >collected by the current RZMS/RT systems. Due to the confidential nature
> >of the root zone change request data, which includes email discussions
> >between ICANN staff and the requesters in which potentially business
> >proprietary details of registry operation are disclosed, ICANN staff
> >informed the CWG chairs and DT-A that releasing the raw RZMS/RT data
> >would be a violation of existing IANA policy and thus, would not be
> >possible, particularly to any organization that competes in the domain
> >name space. ICANN staff also again explained that the RZMS/RT systems do
> >not currently collect data the way the SLEs were defined and the primary
> >task for ICANN was to modify RZMS in order to collect the new SLEs.
> > 
> >
> >4. As a compromise to try to address the new 6-month data collection
> >requirement, ICANN staff contracted with Viagenie as an independent and
> >neutral third-party to explore whether the data collected by the current
> >RZMS/RT system could be used to "seed" some portion of the SLEs, thereby
> >reducing the data collection time requirement.
> > 
> >
> >5. Viagenie completed their work and presented a summary on the Feb 4th
> >CWG call, confirming that: (a) The current RZMS/RT systems do not collect
> >data in accordance with the newly defined SLEs; (b) The heuristics
> >developed provided approximations for most metrics, but some
> >approximations were less conclusive; and (c) The RZMS/RT tool is a
> >complex system that frequently relies on email interactions for
> >progressing request state.
> > 
> >
> >6. In parallel to the work ICANN contracted Viagenie to perform, ICANN
> >has continued to pursue modifying the RZMS system to collect data in the
> >way that the SLEs were defined. ICANN now expects that the RZMS
> >modifications will be completed at the end of February, a month ahead of
> >schedule. Data collection for the new SLEs can thus begin in March.
> >Assuming everything goes well, this would allow for sufficient time for
> >SLE data collection without delay of the transition in order to meet the
> >6-month SLE data collection requirement.
> > 
> >
> >The main approach to implementing the SLEs had always been to make code
> >changes to the RZMS so that it can capture processing time data that the
> >DT-A defined for all future change request processing. ICANN is on-track
> >to have this work completed by the end of this month, February.
> > 
> >
> >If the CWG would like any additional clarification around the work
> >performed by Viagenie, the RZMS development work, or other aspects of the
> >technical implementation of the transition, please let ICANN staff know
> >and we will be happy to provide clarification.
> > 
> >
> >Regards,
> >-drc
> >ICANN CTO
> > 
> >
> 
>