[SLE Team] SLEWG Meeting | 18:00 UTC | 17 August 2015 - this one!

[Kim Davies]

Hi Paul,

(changed cc to go to the WG mailing list instead of a long list of folks)

> Number 11 – “Informational Reporting, changed wording on B3 to make it clear that non IANA processing time is not being reported”
> We all agreed (including ICANN/IANA) that IANA will post the corresponding times for each step in process on their dashboard.
> Adam has asked the DTA for guidance as to what time should be used (real-time, average time etc)?
> In my opinion, we should have them post the actual time of the each process step alongside the SLE. This way one can see both the real time performance vs. the SLE for each change request.
> 
> It seems a small point but it just needs clarifying.Welcome your comments…..

This is a new issue raised on the call today that I’d figured was an implementation detail that didn't necessarily need to be nailed down yet. Bear in mind the key focus is to identify the right things ICANN is to measure, so that once finalised we can invest the time and resources to developing the necessary instrumentation for our systems, and bring back sufficient data to support a discussion on setting what the actual service level commitments should be.

That said my initial thoughts are that the “real time dashboard” would likely show statistics for a sliding window of the last “X” days (e.g. last 30 days) for all the different metrics, and then have some kind of colour indication when the given metric is outside of bounds. Given the SLEs are expected to be defined as pass/fail (breach or no breach), we have a natural definition for what “green” and “red” should be. If what is sought is a traffic light system, another implementation detail is what should be orange/amber/yellow. A more complex implementation would allow customer configurable timespans, maybe that is an idea for later development.

That all being said, the colour coding on the dashboard would be an general indicator only. The real reporting against the SLEs will be monthly reports that must be produced and delivered in the subsequent month. This would be the definitive accounting of performance against the SLE for the purposes of assessing against the contract, reporting and discussing with the CSC etc. Because the dashboard’s measurement window is constantly changing it will rarely, if ever, exactly align to the calendar month worth of measurements contained in the monthly reports.

> The more contentious issue is:
> 
> Number 15 – “Cumulative Time vs. Step Time” - IMHO there is has been confusion in what is meant here by the term "cumulative". As a consequence I would like to distinguish between summation time taken for each SLE process step and the aggregated time for the change request.
> 
> ICANN/IANA have argued there is little point in double counting, ie capturing the time for each SLE process step, then a for each change request adding up each step to have a total overall processing time SLE.
> From my perspective the two should equate to the same and conducting an SLE for the aggregated time is futile as it is almost double counting as the total should be the sum of the earlier (measured) component process steps.

I believe I am aligned with your thoughts.

kim