[DTA - SLE] Questions and Answers

Mon Jun 22 17:33:09 UTC 2015

Jay,

Many thanks for your very detailed report, it helps us be better prepared
for tomorrow's ccNSO session. I agree with your answers.

Patricio

On Mon, Jun 22, 2015 at 12:29 PM, Jay Daley <jay at nzrs.net.nz> wrote:

> Yesterday I spent a lot of time talking to people in the ccTLD community
> about the SLE work.  They had some concerns that it took me a while to
> clear up.  I thought it would be useful for me to share the questions I
> received and the answers I gave.  It would be useful if you could comment
> as to whether I have captured the views of the group here or not and
> whether this Q&A can be shared wider than this list.
>
>
> Q:  Why has there been so little open discussion by DT-A on the mailing
> list?  It all looks very secretive?
>
> A:  We began work some time before a DT-A mailing list was established and
> by the time it was, a lot of work on drafting had already taken place.
> Soon after the list was established the issue of how IANA would be involved
> was resolved with the full engagement from the IANA staff, and it was felt
> that direct conversations with IANA were the best way to take advantage of
> that rather than slow conversations on the list.
>
> In order to engage fully IANA had to get assemble a series of documents
> that detail their internal processes and then get permission from the NTIA
> to release them.  Altogether this process took a long time.
>
>
> Q:  What involvement has ICANN/IANA had and how committed are they to this?
>
> A:  Initially it appeared to us that ICANN was only going to engage at a
> senior executive level to insist that the SLA stays unchanged from that in
> place currently.  This was not acceptable to the DT-A team as we regard the
> current NTIA approved SLA as well below the minimum SLA required for a
> service of such global importance.  Fortunately IANA staff are now involved
> and they are fully committed to a proper SLA that serves the need of
> customers and drives internal improvements.
>
> It is useful to note that an SLE (Service Level Expectation) is generally
> set by customers alone, whereas with this full engagement from IANA it is
> possible for us to agree an SLA (Service Level Agreement) which is equally
> supported by both customers and the providers.
>
>
> Q:  Does the DT-A team have an agenda here, such as forcing through the
> automation of IANA?
>
> A:  No. The team has taken the view from the outset that it is not our
> role to change the processes in any way.  IANA have already started a
> process of automation of some elements of their service, as presented at
> various ICANN meetings, and we are trying to keep up with that but we are
> not introducing any agenda of our own.
>
>
> Q:  What is the general plan for the SLA?
>
> A:  Conceptually it breaks down into three parts:
>
> 1.  A full service definition with each service broken down into
> constituent processes and stages within those processes that require
> individual measurement.
> 2.  An initial set of performance targets.
> 3.  An initial set of breach thresholds - a percentage of requests that
> must meet the performance target or IANA is in breach of the SLA.
>
> A set of principles has been developed to guide the work on these three
> parts.
>
>
> Q:  What is being measured?
>
> A:  Both time to complete a request (broken down by stage) and the
> accuracy of the work completed.  For some processes, such as nameserver
> changes, the accuracy needs to be 100% but for others a lower standard is
> acceptable.
>
>
> Q:  Is all that measurement necessary?
>
> A:  It is the clear consensus view of DT-A that without this level of
> measurement it is not possible to define an SLA nor is it possible to
> ensure that an SLA meets the needs of the customer.  The breakdown into
> stages of processes has been chosen to reflect the different
> responsibilities for those stages and the different nature of those
> stages.  For example, if IANA ask for more information about a root change
> then the clock needs to stop and the responsibility for progressing that
> request transfers from IANA to the requestor.
>
> This level of measurement is vital to resolve, once and for all, two
> different perceptions that people have of IANA performance:
> -  that IANA operates a two-tier SLA with contracted TLDs receiving a
> faster service than non-contracted TLDs while both are still within the
> overall SLA.
> -  that IANA is the primary cause of delays.  By specifying stages where
> the clock stops when responsibility transfers, this ensures an accurate
> measurement of IANA performance.
>
>
> Q:  How were the initial performance targets set?
>
> A:  It is important to note that no targets have yet been set.  Some
> targets have been proposed and these are still under discussion.  These
> proposed targets were set based on analysis of previous IANA performance.
> The full data set used for the analysis is available.
>
> The key takeaway is that the performance targets in current NTIA SLA are
> far looser than we as customers believe they should be and IANA as the
> operator clearly recognises this because it is over-performing by so much.
>
>
> Q:  Has work been done to ensure that these targets match the importance
> of the processes and are not arbitrary?  Isn’t there a problem that if we
> use current performance then IANA needs to establish a base line even
> higher to ensure that they never breach those targets?
>
> A:  The answers are yes the work has been done and no there isn’t a
> problem.  These issues are addressed by the use of breach levels, which
> work in conjunction with the performance targets.  For example a
> performance target might be set for a specific type of customers request at
> 5 days based on evidential analysis, but the breach threshold might be set
> at 90%, which means that only 90% of requests of this type need to meet the
> 5 day target.  This breach threshold serves a dual purpose:
>
> -  It provides headroom for IANA above the target by allowing them to
> ignore the worst performing requests.
> -  This level of headroom given reflects the customer priority of this
> type of request.
>
>
> Q:  Once these targets are set are they fixed in stone?
>
> A:  No, these are an initial set of targets and it is expected that they
> will develop over time in two ways:
>
> -  ongoing process improvement by IANA leading to improvements in
> performance
> -  better consultation with customers to understand our priorities better,
> leading to adjustments in targets to reflect that
>
>
> Q:  How important is it that these elements are in place before transition?
>
> A:  The view of the DT-A team is that a full SLE/SLA needs to be in place
> by the start of the transition however IANA have noted that there are some
> pre-requisites that must be met before they can implement it.  These are:
>
> 1.  New IT system in place that carries out the required level of
> measurement.  They will only be able to begin development once there is
> certainty that this transition is going ahead on this basis and can only
> turn those changes on after transition given the nature of their agreement
> with the NTIA.  It should also be remembered that IANA will need to make a
> case for development resources to the ICANN executive team as IANA has no
> development resources of its own.
> 2.  IANA staff will need time to adapt to the changes in their working
> practices to implement this measurement.
>
>
> Q:  When will the DT-A be finished?
>
> A:  We can’t answer that just yet as we are still negotiating with IANA
> and considerable effort is going into this from both sides.
>
>
> Jay
>
> --
> Jay Daley
> Chief Executive
> NZRS Ltd
> desk: +64 4 931 6977
> mobile: +64 21 678840
> linkedin: www.linkedin.com/in/jaydaley
>
> _______________________________________________
> dt1 mailing list
> dt1 at icann.org
> https://mm.icann.org/mailman/listinfo/dt1
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mm.icann.org/pipermail/dt1/attachments/20150622/1967a26b/attachment-0001.html>