[DTA - SLE] Questions and Answers - Revised

Jay Daley jay at nzrs.net.nz
Tue Jun 23 12:48:14 UTC 2015



I’ve been asked if we can post these to the CWG list now - is that something you are happy to do?


> On 23/06/2015, at 8:36 am, Paul M Kane - CWG <paul.kane-cwg at icb.co.uk> wrote:
> Thanks Jay for doing the Q&A.
> Excellent.
> Best
> Paul
> Quoting Jay Daley <jay at nzrs.net.nz>:
>> I had some feedback on my initial Q&As so I’ve updated them below.  The
>> changes are about a) The IANA system development; b) The plan for more data
>> capture to inform the performance targets; c) the next steps.  Now that
>> I’ve had some feedback, I was planning to share these later today with a
>> few people who are involved in CWG-Stewardship and who are concerned about
>> our progress - please let me know if this is an issue.
>> Jay
>> Q:  Why has there been so little open discussion by DT-A on the mailing list?
>> It all looks very secretive?
>> A:  We began work some time before a DT-A mailing list was established and by
>> the time it was, a lot of work on drafting had already taken place.  Soon
>> after the list was established the issue of how IANA would be involved was
>> resolved with the full engagement from the IANA staff, and it was felt that
>> direct conversations with IANA were the best way to take advantage of that
>> rather than slow conversations on the list. 
>> In order to engage fully IANA had to get assemble a series of documents that
>> detail their internal processes and then get permission from the NTIA to
>> release them.  Altogether this process took a long time.
>> Q:  What involvement has ICANN/IANA had and how committed are they to this?
>> A:  Initially it appeared to us that ICANN was only going to engage at a
>> senior executive level to insist that the SLA stays unchanged from that in
>> place currently.  This was not acceptable to the DT-A team as we regard the
>> current NTIA approved SLA as well below the minimum SLA required for a
>> service of such global importance.  Fortunately IANA staff are now involved
>> and they are fully committed to a proper SLA that serves the need of
>> customers and drives internal improvements.  
>> It is useful to note that an SLE (Service Level Expectation) is generally set
>> by customers alone, whereas with this full engagement from IANA it is
>> possible for us to agree an SLA (Service Level Agreement) which is equally
>> supported by both customers and the providers.
>> Q:  Does the DT-A team have an agenda here, such as forcing through the
>> automation of IANA?
>> A:  No. The team has taken the view from the outset that it is not our role
>> to change the processes in any way.  IANA have already started a process of
>> automation of some elements of their service, as presented at various ICANN
>> meetings, and we are trying to keep up with that but we are not introducing
>> any agenda of our own.
>> Q:  What is the general plan for the SLA?
>> A:  Conceptually it breaks down into three parts:
>> 1.  A full service definition with each service broken down into constituent
>> processes and stages within those processes that require individual
>> measurement.
>> 2.  An initial set of performance targets.
>> 3.  An initial set of breach thresholds - a percentage of requests that must
>> meet the performance target or IANA is in breach of the SLA.
>> A set of principles has been developed to guide the work on these three
>> parts.
>> Q:  What is being measured?
>> A:  Both time to complete a request (broken down by stage) and the accuracy
>> of the work completed.  For some processes, such as nameserver changes, the
>> accuracy needs to be 100% but for others a lower standard is acceptable. 
>> Q:  Is all that measurement necessary?
>> A:  It is the clear consensus view of DT-A that without this level of
>> measurement it is not possible to define an SLA nor is it possible to ensure
>> that an SLA meets the needs of the customer.  The breakdown into stages of
>> processes has been chosen to reflect the different responsibilities for those
>> stages and the different nature of those stages.  For example, if IANA ask
>> for more information about a root change then the clock needs to stop and the
>> responsibility for progressing that request transfers from IANA to the
>> requestor.  
>> This level of measurement is vital to resolve, once and for all, two
>> different perceptions that people have of IANA performance:
>> -  that IANA operates a two-tier SLA with contracted TLDs receiving a faster
>> service than non-contracted TLDs while both are still within the overall
>> SLA.
>> -  that IANA is the primary cause of delays.  By specifying stages where the
>> clock stops when responsibility transfers, this ensures an accurate
>> measurement of IANA performance.
>> Q:  How were the initial performance targets set?  
>> A:  It is important to note that no targets have yet been set.  Some targets
>> were initially proposed based on analysis of previous IANA performance (the
>> full data set used for the analysis is available) but we have agreed with
>> IANA that we need more data and are hoping for 2-3 months of data capture
>> from IANA.
>> The key takeaway is that the performance targets in current NTIA SLA are far
>> looser than we as customers believe they should be and IANA as the operator
>> clearly recognises this because it is over-performing by so much.
>> Q:  Has work been done to ensure that these targets match the importance of
>> the processes and are not arbitrary?  Isn’t there a problem that if we use
>> current performance then IANA needs to establish a base line even higher to
>> ensure that they never breach those targets?
>> A:  The answers are yes the work has been done and no there isn’t a
>> problem.  These issues are addressed by the use of breach levels, which work
>> in conjunction with the performance targets.  For example a performance
>> target might be set for a specific type of customers request at 5 days based
>> on evidential analysis, but the breach threshold might be set at 90%, which
>> means that only 90% of requests of this type need to meet the 5 day target. 
>> This breach threshold serves a dual purpose:
>> -  It provides headroom for IANA above the target by allowing them to ignore
>> the worst performing requests.
>> -  This level of headroom given reflects the customer priority of this type
>> of request.
>> Q:  Once these targets are set are they fixed in stone?
>> A:  No, these are an initial set of targets and it is expected that they will
>> develop over time in two ways:
>> -  ongoing process improvement by IANA leading to improvements in
>> performance
>> -  better consultation with customers to understand our priorities better,
>> leading to adjustments in targets to reflect that
>> Q:  How important is it that these elements are in place before transition?
>> A:  The view of the DT-A team is that a full SLE/SLA needs to be in place by
>> the start of the transition however IANA have noted that there are some
>> pre-requisites that must be met before they can implement it.  These are:
>> 1.  The current IT system needs to have features added to extract the
>> required measurements.   This development can only begin once there is
>> certainty that this transition is going ahead on this basis and IANA can only
>> turn those changes on after transition given the nature of their agreement
>> with the NTIA.  It should also be remembered that IANA will need to make a
>> case for development resources to the ICANN executive team as IANA has no
>> development resources of its own.
>> 2.  IANA staff will need time to adapt to the changes in their working
>> practices to implement this measurement.
>> Q:  When will the DT-A be finished?
>> A:  We are close to completing the SLA except the actual performance targets.
>> These will be added later after a data collection period and discussion with
>> IANA.  
>> -- 
>> Jay Daley
>> Chief Executive
>> NZRS Ltd
>> desk: +64 4 931 6977
>> mobile: +64 21 678840
>> linkedin: www.linkedin.com/in/jaydaley
>> _______________________________________________
>> dt1 mailing list
>> dt1 at icann.org
>> https://mm.icann.org/mailman/listinfo/dt1

Jay Daley
Chief Executive
desk: +64 4 931 6977
mobile: +64 21 678840
linkedin: www.linkedin.com/in/jaydaley

More information about the dt1 mailing list