[SLE Team] [DTA - SLE] Questions and Answers - Revised

[Elaine Pruis]

Well done. Thanks for your work.

On Tue, Jun 23, 2015 at 5:48 AM, Jay Daley <jay at nzrs.net.nz> wrote:

> Paul
>
> Thanks.
>
> I’ve been asked if we can post these to the CWG list now - is that
> something you are happy to do?
>
> Jay
>
> > On 23/06/2015, at 8:36 am, Paul M Kane - CWG <paul.kane-cwg at icb.co.uk>
> wrote:
> >
> > Thanks Jay for doing the Q&A.
> >
> > Excellent.
> >
> > Best
> >
> > Paul
> >
> > Quoting Jay Daley <jay at nzrs.net.nz>:
> >
> >> I had some feedback on my initial Q&As so I’ve updated them below.
> The
> >> changes are about a) The IANA system development; b) The plan for more
> data
> >> capture to inform the performance targets; c) the next steps.  Now that
> >> I’ve had some feedback, I was planning to share these later today
> with a
> >> few people who are involved in CWG-Stewardship and who are concerned
> about
> >> our progress - please let me know if this is an issue.
> >>
> >> Jay
> >>
> >>
> >> Q:  Why has there been so little open discussion by DT-A on the mailing
> list?
> >> It all looks very secretive?
> >>
> >> A:  We began work some time before a DT-A mailing list was established
> and by
> >> the time it was, a lot of work on drafting had already taken place.
> Soon
> >> after the list was established the issue of how IANA would be involved
> was
> >> resolved with the full engagement from the IANA staff, and it was felt
> that
> >> direct conversations with IANA were the best way to take advantage of
> that
> >> rather than slow conversations on the list.
> >>
> >> In order to engage fully IANA had to get assemble a series of documents
> that
> >> detail their internal processes and then get permission from the NTIA to
> >> release them.  Altogether this process took a long time.
> >>
> >>
> >> Q:  What involvement has ICANN/IANA had and how committed are they to
> this?
> >>
> >> A:  Initially it appeared to us that ICANN was only going to engage at a
> >> senior executive level to insist that the SLA stays unchanged from that
> in
> >> place currently.  This was not acceptable to the DT-A team as we regard
> the
> >> current NTIA approved SLA as well below the minimum SLA required for a
> >> service of such global importance.  Fortunately IANA staff are now
> involved
> >> and they are fully committed to a proper SLA that serves the need of
> >> customers and drives internal improvements.
> >>
> >> It is useful to note that an SLE (Service Level Expectation) is
> generally set
> >> by customers alone, whereas with this full engagement from IANA it is
> >> possible for us to agree an SLA (Service Level Agreement) which is
> equally
> >> supported by both customers and the providers.
> >>
> >>
> >> Q:  Does the DT-A team have an agenda here, such as forcing through the
> >> automation of IANA?
> >>
> >> A:  No. The team has taken the view from the outset that it is not our
> role
> >> to change the processes in any way.  IANA have already started a
> process of
> >> automation of some elements of their service, as presented at various
> ICANN
> >> meetings, and we are trying to keep up with that but we are not
> introducing
> >> any agenda of our own.
> >>
> >>
> >> Q:  What is the general plan for the SLA?
> >>
> >> A:  Conceptually it breaks down into three parts:
> >>
> >> 1.  A full service definition with each service broken down into
> constituent
> >> processes and stages within those processes that require individual
> >> measurement.
> >> 2.  An initial set of performance targets.
> >> 3.  An initial set of breach thresholds - a percentage of requests that
> must
> >> meet the performance target or IANA is in breach of the SLA.
> >>
> >> A set of principles has been developed to guide the work on these three
> >> parts.
> >>
> >>
> >> Q:  What is being measured?
> >>
> >> A:  Both time to complete a request (broken down by stage) and the
> accuracy
> >> of the work completed.  For some processes, such as nameserver changes,
> the
> >> accuracy needs to be 100% but for others a lower standard is acceptable.
> >>
> >>
> >> Q:  Is all that measurement necessary?
> >>
> >> A:  It is the clear consensus view of DT-A that without this level of
> >> measurement it is not possible to define an SLA nor is it possible to
> ensure
> >> that an SLA meets the needs of the customer.  The breakdown into stages
> of
> >> processes has been chosen to reflect the different responsibilities for
> those
> >> stages and the different nature of those stages.  For example, if IANA
> ask
> >> for more information about a root change then the clock needs to stop
> and the
> >> responsibility for progressing that request transfers from IANA to the
> >> requestor.
> >>
> >> This level of measurement is vital to resolve, once and for all, two
> >> different perceptions that people have of IANA performance:
> >> -  that IANA operates a two-tier SLA with contracted TLDs receiving a
> faster
> >> service than non-contracted TLDs while both are still within the overall
> >> SLA.
> >> -  that IANA is the primary cause of delays.  By specifying stages
> where the
> >> clock stops when responsibility transfers, this ensures an accurate
> >> measurement of IANA performance.
> >>
> >>
> >> Q:  How were the initial performance targets set?
> >>
> >> A:  It is important to note that no targets have yet been set.  Some
> targets
> >> were initially proposed based on analysis of previous IANA performance
> (the
> >> full data set used for the analysis is available) but we have agreed
> with
> >> IANA that we need more data and are hoping for 2-3 months of data
> capture
> >> from IANA.
> >>
> >> The key takeaway is that the performance targets in current NTIA SLA
> are far
> >> looser than we as customers believe they should be and IANA as the
> operator
> >> clearly recognises this because it is over-performing by so much.
> >>
> >>
> >> Q:  Has work been done to ensure that these targets match the
> importance of
> >> the processes and are not arbitrary?  Isn’t there a problem that if
> we use
> >> current performance then IANA needs to establish a base line even
> higher to
> >> ensure that they never breach those targets?
> >>
> >> A:  The answers are yes the work has been done and no there isn’t a
> >> problem.  These issues are addressed by the use of breach levels, which
> work
> >> in conjunction with the performance targets.  For example a performance
> >> target might be set for a specific type of customers request at 5 days
> based
> >> on evidential analysis, but the breach threshold might be set at 90%,
> which
> >> means that only 90% of requests of this type need to meet the 5 day
> target.
> >> This breach threshold serves a dual purpose:
> >>
> >> -  It provides headroom for IANA above the target by allowing them to
> ignore
> >> the worst performing requests.
> >> -  This level of headroom given reflects the customer priority of this
> type
> >> of request.
> >>
> >>
> >> Q:  Once these targets are set are they fixed in stone?
> >>
> >> A:  No, these are an initial set of targets and it is expected that
> they will
> >> develop over time in two ways:
> >>
> >> -  ongoing process improvement by IANA leading to improvements in
> >> performance
> >> -  better consultation with customers to understand our priorities
> better,
> >> leading to adjustments in targets to reflect that
> >>
> >>
> >> Q:  How important is it that these elements are in place before
> transition?
> >>
> >> A:  The view of the DT-A team is that a full SLE/SLA needs to be in
> place by
> >> the start of the transition however IANA have noted that there are some
> >> pre-requisites that must be met before they can implement it.  These
> are:
> >>
> >> 1.  The current IT system needs to have features added to extract the
> >> required measurements.   This development can only begin once there is
> >> certainty that this transition is going ahead on this basis and IANA
> can only
> >> turn those changes on after transition given the nature of their
> agreement
> >> with the NTIA.  It should also be remembered that IANA will need to
> make a
> >> case for development resources to the ICANN executive team as IANA has
> no
> >> development resources of its own.
> >> 2.  IANA staff will need time to adapt to the changes in their working
> >> practices to implement this measurement.
> >>
> >>
> >> Q:  When will the DT-A be finished?
> >>
> >> A:  We are close to completing the SLA except the actual performance
> targets.
> >> These will be added later after a data collection period and discussion
> with
> >> IANA.
> >>
> >> --
> >> Jay Daley
> >> Chief Executive
> >> NZRS Ltd
> >> desk: +64 4 931 6977
> >> mobile: +64 21 678840
> >> linkedin: www.linkedin.com/in/jaydaley
> >>
> >> _______________________________________________
> >> dt1 mailing list
> >> dt1 at icann.org
> >> https://mm.icann.org/mailman/listinfo/dt1
>
>
> --
> Jay Daley
> Chief Executive
> NZRS Ltd
> desk: +64 4 931 6977
> mobile: +64 21 678840
> linkedin: www.linkedin.com/in/jaydaley
>
> _______________________________________________
> dt1 mailing list
> dt1 at icann.org
> https://mm.icann.org/mailman/listinfo/dt1
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mm.icann.org/pipermail/dt1/attachments/20150628/46f604c8/attachment.html>