<div dir="ltr">Well done. Thanks for your work.</div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jun 23, 2015 at 5:48 AM, Jay Daley <span dir="ltr"><<a href="mailto:jay@nzrs.net.nz" target="_blank">jay@nzrs.net.nz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Paul<br>
<br>
Thanks.<br>
<br>
I’ve been asked if we can post these to the CWG list now - is that something you are happy to do?<br>
<span class="HOEnZb"><font color="#888888"><br>
Jay<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
> On 23/06/2015, at 8:36 am, Paul M Kane - CWG <<a href="mailto:paul.kane-cwg@icb.co.uk">paul.kane-cwg@icb.co.uk</a>> wrote:<br>
><br>
> Thanks Jay for doing the Q&A.<br>
><br>
> Excellent.<br>
><br>
> Best<br>
><br>
> Paul<br>
><br>
> Quoting Jay Daley <<a href="mailto:jay@nzrs.net.nz">jay@nzrs.net.nz</a>>:<br>
><br>
>> I had some feedback on my initial Q&As so I’ve updated them below. The<br>
>> changes are about a) The IANA system development; b) The plan for more data<br>
>> capture to inform the performance targets; c) the next steps. Now that<br>
>> I’ve had some feedback, I was planning to share these later today with a<br>
>> few people who are involved in CWG-Stewardship and who are concerned about<br>
>> our progress - please let me know if this is an issue.<br>
>><br>
>> Jay<br>
>><br>
>><br>
>> Q: Why has there been so little open discussion by DT-A on the mailing list?<br>
>> It all looks very secretive?<br>
>><br>
>> A: We began work some time before a DT-A mailing list was established and by<br>
>> the time it was, a lot of work on drafting had already taken place. Soon<br>
>> after the list was established the issue of how IANA would be involved was<br>
>> resolved with the full engagement from the IANA staff, and it was felt that<br>
>> direct conversations with IANA were the best way to take advantage of that<br>
>> rather than slow conversations on the list.<br>
>><br>
>> In order to engage fully IANA had to get assemble a series of documents that<br>
>> detail their internal processes and then get permission from the NTIA to<br>
>> release them. Altogether this process took a long time.<br>
>><br>
>><br>
>> Q: What involvement has ICANN/IANA had and how committed are they to this?<br>
>><br>
>> A: Initially it appeared to us that ICANN was only going to engage at a<br>
>> senior executive level to insist that the SLA stays unchanged from that in<br>
>> place currently. This was not acceptable to the DT-A team as we regard the<br>
>> current NTIA approved SLA as well below the minimum SLA required for a<br>
>> service of such global importance. Fortunately IANA staff are now involved<br>
>> and they are fully committed to a proper SLA that serves the need of<br>
>> customers and drives internal improvements.<br>
>><br>
>> It is useful to note that an SLE (Service Level Expectation) is generally set<br>
>> by customers alone, whereas with this full engagement from IANA it is<br>
>> possible for us to agree an SLA (Service Level Agreement) which is equally<br>
>> supported by both customers and the providers.<br>
>><br>
>><br>
>> Q: Does the DT-A team have an agenda here, such as forcing through the<br>
>> automation of IANA?<br>
>><br>
>> A: No. The team has taken the view from the outset that it is not our role<br>
>> to change the processes in any way. IANA have already started a process of<br>
>> automation of some elements of their service, as presented at various ICANN<br>
>> meetings, and we are trying to keep up with that but we are not introducing<br>
>> any agenda of our own.<br>
>><br>
>><br>
>> Q: What is the general plan for the SLA?<br>
>><br>
>> A: Conceptually it breaks down into three parts:<br>
>><br>
>> 1. A full service definition with each service broken down into constituent<br>
>> processes and stages within those processes that require individual<br>
>> measurement.<br>
>> 2. An initial set of performance targets.<br>
>> 3. An initial set of breach thresholds - a percentage of requests that must<br>
>> meet the performance target or IANA is in breach of the SLA.<br>
>><br>
>> A set of principles has been developed to guide the work on these three<br>
>> parts.<br>
>><br>
>><br>
>> Q: What is being measured?<br>
>><br>
>> A: Both time to complete a request (broken down by stage) and the accuracy<br>
>> of the work completed. For some processes, such as nameserver changes, the<br>
>> accuracy needs to be 100% but for others a lower standard is acceptable.<br>
>><br>
>><br>
>> Q: Is all that measurement necessary?<br>
>><br>
>> A: It is the clear consensus view of DT-A that without this level of<br>
>> measurement it is not possible to define an SLA nor is it possible to ensure<br>
>> that an SLA meets the needs of the customer. The breakdown into stages of<br>
>> processes has been chosen to reflect the different responsibilities for those<br>
>> stages and the different nature of those stages. For example, if IANA ask<br>
>> for more information about a root change then the clock needs to stop and the<br>
>> responsibility for progressing that request transfers from IANA to the<br>
>> requestor.<br>
>><br>
>> This level of measurement is vital to resolve, once and for all, two<br>
>> different perceptions that people have of IANA performance:<br>
>> - that IANA operates a two-tier SLA with contracted TLDs receiving a faster<br>
>> service than non-contracted TLDs while both are still within the overall<br>
>> SLA.<br>
>> - that IANA is the primary cause of delays. By specifying stages where the<br>
>> clock stops when responsibility transfers, this ensures an accurate<br>
>> measurement of IANA performance.<br>
>><br>
>><br>
>> Q: How were the initial performance targets set?<br>
>><br>
>> A: It is important to note that no targets have yet been set. Some targets<br>
>> were initially proposed based on analysis of previous IANA performance (the<br>
>> full data set used for the analysis is available) but we have agreed with<br>
>> IANA that we need more data and are hoping for 2-3 months of data capture<br>
>> from IANA.<br>
>><br>
>> The key takeaway is that the performance targets in current NTIA SLA are far<br>
>> looser than we as customers believe they should be and IANA as the operator<br>
>> clearly recognises this because it is over-performing by so much.<br>
>><br>
>><br>
>> Q: Has work been done to ensure that these targets match the importance of<br>
>> the processes and are not arbitrary? Isn’t there a problem that if we use<br>
>> current performance then IANA needs to establish a base line even higher to<br>
>> ensure that they never breach those targets?<br>
>><br>
>> A: The answers are yes the work has been done and no there isn’t a<br>
>> problem. These issues are addressed by the use of breach levels, which work<br>
>> in conjunction with the performance targets. For example a performance<br>
>> target might be set for a specific type of customers request at 5 days based<br>
>> on evidential analysis, but the breach threshold might be set at 90%, which<br>
>> means that only 90% of requests of this type need to meet the 5 day target.<br>
>> This breach threshold serves a dual purpose:<br>
>><br>
>> - It provides headroom for IANA above the target by allowing them to ignore<br>
>> the worst performing requests.<br>
>> - This level of headroom given reflects the customer priority of this type<br>
>> of request.<br>
>><br>
>><br>
>> Q: Once these targets are set are they fixed in stone?<br>
>><br>
>> A: No, these are an initial set of targets and it is expected that they will<br>
>> develop over time in two ways:<br>
>><br>
>> - ongoing process improvement by IANA leading to improvements in<br>
>> performance<br>
>> - better consultation with customers to understand our priorities better,<br>
>> leading to adjustments in targets to reflect that<br>
>><br>
>><br>
>> Q: How important is it that these elements are in place before transition?<br>
>><br>
>> A: The view of the DT-A team is that a full SLE/SLA needs to be in place by<br>
>> the start of the transition however IANA have noted that there are some<br>
>> pre-requisites that must be met before they can implement it. These are:<br>
>><br>
>> 1. The current IT system needs to have features added to extract the<br>
>> required measurements. This development can only begin once there is<br>
>> certainty that this transition is going ahead on this basis and IANA can only<br>
>> turn those changes on after transition given the nature of their agreement<br>
>> with the NTIA. It should also be remembered that IANA will need to make a<br>
>> case for development resources to the ICANN executive team as IANA has no<br>
>> development resources of its own.<br>
>> 2. IANA staff will need time to adapt to the changes in their working<br>
>> practices to implement this measurement.<br>
>><br>
>><br>
>> Q: When will the DT-A be finished?<br>
>><br>
>> A: We are close to completing the SLA except the actual performance targets.<br>
>> These will be added later after a data collection period and discussion with<br>
>> IANA.<br>
>><br>
>> --<br>
>> Jay Daley<br>
>> Chief Executive<br>
>> NZRS Ltd<br>
>> desk: +64 4 931 6977<br>
>> mobile: +64 21 678840<br>
>> linkedin: <a href="http://www.linkedin.com/in/jaydaley" rel="noreferrer" target="_blank">www.linkedin.com/in/jaydaley</a><br>
>><br>
>> _______________________________________________<br>
>> dt1 mailing list<br>
>> <a href="mailto:dt1@icann.org">dt1@icann.org</a><br>
>> <a href="https://mm.icann.org/mailman/listinfo/dt1" rel="noreferrer" target="_blank">https://mm.icann.org/mailman/listinfo/dt1</a><br>
<br>
<br>
--<br>
Jay Daley<br>
Chief Executive<br>
NZRS Ltd<br>
desk: <a href="tel:%2B64%204%20931%206977" value="+6449316977">+64 4 931 6977</a><br>
mobile: <a href="tel:%2B64%2021%20678840" value="+6421678840">+64 21 678840</a><br>
linkedin: <a href="http://www.linkedin.com/in/jaydaley" rel="noreferrer" target="_blank">www.linkedin.com/in/jaydaley</a><br>
<br>
_______________________________________________<br>
dt1 mailing list<br>
<a href="mailto:dt1@icann.org">dt1@icann.org</a><br>
<a href="https://mm.icann.org/mailman/listinfo/dt1" rel="noreferrer" target="_blank">https://mm.icann.org/mailman/listinfo/dt1</a><br>
</div></div></blockquote></div><br></div>