[DTA - SLE] Questions and Answers - Revised
Jay Daley
jay at nzrs.net.nz
Tue Jun 23 12:48:14 UTC 2015
Paul
Thanks.
I’ve been asked if we can post these to the CWG list now - is that something you are happy to do?
Jay
> On 23/06/2015, at 8:36 am, Paul M Kane - CWG <paul.kane-cwg at icb.co.uk> wrote:
>
> Thanks Jay for doing the Q&A.
>
> Excellent.
>
> Best
>
> Paul
>
> Quoting Jay Daley <jay at nzrs.net.nz>:
>
>> I had some feedback on my initial Q&As so Iâve updated them below. The
>> changes are about a) The IANA system development; b) The plan for more data
>> capture to inform the performance targets; c) the next steps. Now that
>> Iâve had some feedback, I was planning to share these later today with a
>> few people who are involved in CWG-Stewardship and who are concerned about
>> our progress - please let me know if this is an issue.
>>
>> Jay
>>
>>
>> Q: Why has there been so little open discussion by DT-A on the mailing list?
>> It all looks very secretive?
>>
>> A: We began work some time before a DT-A mailing list was established and by
>> the time it was, a lot of work on drafting had already taken place. Soon
>> after the list was established the issue of how IANA would be involved was
>> resolved with the full engagement from the IANA staff, and it was felt that
>> direct conversations with IANA were the best way to take advantage of that
>> rather than slow conversations on the list.
>>
>> In order to engage fully IANA had to get assemble a series of documents that
>> detail their internal processes and then get permission from the NTIA to
>> release them. Altogether this process took a long time.
>>
>>
>> Q: What involvement has ICANN/IANA had and how committed are they to this?
>>
>> A: Initially it appeared to us that ICANN was only going to engage at a
>> senior executive level to insist that the SLA stays unchanged from that in
>> place currently. This was not acceptable to the DT-A team as we regard the
>> current NTIA approved SLA as well below the minimum SLA required for a
>> service of such global importance. Fortunately IANA staff are now involved
>> and they are fully committed to a proper SLA that serves the need of
>> customers and drives internal improvements.
>>
>> It is useful to note that an SLE (Service Level Expectation) is generally set
>> by customers alone, whereas with this full engagement from IANA it is
>> possible for us to agree an SLA (Service Level Agreement) which is equally
>> supported by both customers and the providers.
>>
>>
>> Q: Does the DT-A team have an agenda here, such as forcing through the
>> automation of IANA?
>>
>> A: No. The team has taken the view from the outset that it is not our role
>> to change the processes in any way. IANA have already started a process of
>> automation of some elements of their service, as presented at various ICANN
>> meetings, and we are trying to keep up with that but we are not introducing
>> any agenda of our own.
>>
>>
>> Q: What is the general plan for the SLA?
>>
>> A: Conceptually it breaks down into three parts:
>>
>> 1. A full service definition with each service broken down into constituent
>> processes and stages within those processes that require individual
>> measurement.
>> 2. An initial set of performance targets.
>> 3. An initial set of breach thresholds - a percentage of requests that must
>> meet the performance target or IANA is in breach of the SLA.
>>
>> A set of principles has been developed to guide the work on these three
>> parts.
>>
>>
>> Q: What is being measured?
>>
>> A: Both time to complete a request (broken down by stage) and the accuracy
>> of the work completed. For some processes, such as nameserver changes, the
>> accuracy needs to be 100% but for others a lower standard is acceptable.
>>
>>
>> Q: Is all that measurement necessary?
>>
>> A: It is the clear consensus view of DT-A that without this level of
>> measurement it is not possible to define an SLA nor is it possible to ensure
>> that an SLA meets the needs of the customer. The breakdown into stages of
>> processes has been chosen to reflect the different responsibilities for those
>> stages and the different nature of those stages. For example, if IANA ask
>> for more information about a root change then the clock needs to stop and the
>> responsibility for progressing that request transfers from IANA to the
>> requestor.
>>
>> This level of measurement is vital to resolve, once and for all, two
>> different perceptions that people have of IANA performance:
>> - that IANA operates a two-tier SLA with contracted TLDs receiving a faster
>> service than non-contracted TLDs while both are still within the overall
>> SLA.
>> - that IANA is the primary cause of delays. By specifying stages where the
>> clock stops when responsibility transfers, this ensures an accurate
>> measurement of IANA performance.
>>
>>
>> Q: How were the initial performance targets set?
>>
>> A: It is important to note that no targets have yet been set. Some targets
>> were initially proposed based on analysis of previous IANA performance (the
>> full data set used for the analysis is available) but we have agreed with
>> IANA that we need more data and are hoping for 2-3 months of data capture
>> from IANA.
>>
>> The key takeaway is that the performance targets in current NTIA SLA are far
>> looser than we as customers believe they should be and IANA as the operator
>> clearly recognises this because it is over-performing by so much.
>>
>>
>> Q: Has work been done to ensure that these targets match the importance of
>> the processes and are not arbitrary? Isnât there a problem that if we use
>> current performance then IANA needs to establish a base line even higher to
>> ensure that they never breach those targets?
>>
>> A: The answers are yes the work has been done and no there isnât a
>> problem. These issues are addressed by the use of breach levels, which work
>> in conjunction with the performance targets. For example a performance
>> target might be set for a specific type of customers request at 5 days based
>> on evidential analysis, but the breach threshold might be set at 90%, which
>> means that only 90% of requests of this type need to meet the 5 day target.
>> This breach threshold serves a dual purpose:
>>
>> - It provides headroom for IANA above the target by allowing them to ignore
>> the worst performing requests.
>> - This level of headroom given reflects the customer priority of this type
>> of request.
>>
>>
>> Q: Once these targets are set are they fixed in stone?
>>
>> A: No, these are an initial set of targets and it is expected that they will
>> develop over time in two ways:
>>
>> - ongoing process improvement by IANA leading to improvements in
>> performance
>> - better consultation with customers to understand our priorities better,
>> leading to adjustments in targets to reflect that
>>
>>
>> Q: How important is it that these elements are in place before transition?
>>
>> A: The view of the DT-A team is that a full SLE/SLA needs to be in place by
>> the start of the transition however IANA have noted that there are some
>> pre-requisites that must be met before they can implement it. These are:
>>
>> 1. The current IT system needs to have features added to extract the
>> required measurements. This development can only begin once there is
>> certainty that this transition is going ahead on this basis and IANA can only
>> turn those changes on after transition given the nature of their agreement
>> with the NTIA. It should also be remembered that IANA will need to make a
>> case for development resources to the ICANN executive team as IANA has no
>> development resources of its own.
>> 2. IANA staff will need time to adapt to the changes in their working
>> practices to implement this measurement.
>>
>>
>> Q: When will the DT-A be finished?
>>
>> A: We are close to completing the SLA except the actual performance targets.
>> These will be added later after a data collection period and discussion with
>> IANA.
>>
>> --
>> Jay Daley
>> Chief Executive
>> NZRS Ltd
>> desk: +64 4 931 6977
>> mobile: +64 21 678840
>> linkedin: www.linkedin.com/in/jaydaley
>>
>> _______________________________________________
>> dt1 mailing list
>> dt1 at icann.org
>> https://mm.icann.org/mailman/listinfo/dt1
--
Jay Daley
Chief Executive
NZRS Ltd
desk: +64 4 931 6977
mobile: +64 21 678840
linkedin: www.linkedin.com/in/jaydaley
More information about the dt1
mailing list