[Ssr2-review] Please prepare for the plenary call on Thursday

[ALAIN AINA]

Hello,

> On 1 Nov 2018, at 23:39, Russ Housley <housley at vigilsec.com> wrote:
> 
> 
>> On Nov 1, 2018, at 4:36 AM, ALAIN AINA <aalain at trstech.net> wrote:
>> 
>> I still think the methodology we used for the assessment of the implementation of the 28 recommendations is not  very suitable. We ended  the following conclusions:
>> 
>> 1- most of the recommendations are vague and not measurable and sometimes not implementable 
>> 2- we have not seen evidence of implementation  and  effect, and so the recommendation has not been fully implemented, effect can’t be measured, etc...
>> 
>> on 1) we will have at some point,  to  review the board actions on the 28 recommendations and  respond  to certain aspects in our assessment. 
>> 
>> https://www.icann.org/resources/board-material/resolutions-2012-10-18-en#1.e
>> 
>> On 2) we have  not agreed on all documents or evidences required  for our assessment. We shall also factor in the fact that the organisation has evolved over the years. The recommendations were issued on 2012, and implemented throughout  up to 2017  and we are reviewing  the  impact in 2017/2018.
>> 
>> the SSR framework and the implementation of some of the recommendations  have influenced the Strategic plan and Operation plan and the 5 years operating plan regularly updated, have KPIs for programs  to y19 and y20 which are SSR related.
>> 
>> 
>> Most of the impact of the implementation  in my opinion  are to be investigated in strategic objectives and their implementation through activities portfolio throughout the years. We  shall measure the impact from the  KPI and deliverable in the OP, Annual report and  also  how SSR inputs and influences the strategic/ operation plannings and  the prioritisation of the activities and projects. Engage with Staff, board and any other affected parties or constituencies by the recommendations.
>> 
>> 
>> On the specifics. 
>> 
>> For example we claimed that definition of security  in the registry agreement differs from the one  published in  the SSR framework, added to the ICANN glossary and supposed to be used in all materials.
>> 
>> But, section 7.3  of the registry agreement does not define “security”,  or “stability” but define  what “effect  on them mean or refer to"
>> 
>> ======
>> 
>> (a)                          For the purposes of this Agreement, an effect on “Security” shall mean
>> 
>> (1) the unauthorized disclosure, alteration, insertion or destruction of registry data, or (2) the unauthorized access to or disclosure of information or resources on the Internet by systems operating in accordance with all applicable standards.
>> 
>> 
>> (b)                         For purposes of this Agreement, an effect on “Stability” shall refer to
>> 
>> (1) lack of compliance with applicable relevant standards that are authoritative and published by a well-established and recognized Internet standards body, such as the
>> 
>> =======
>> So using this example to justify a breach in the implementation of  the recommendation is not appropriate in my opinion.
> 
> Alain:
> 
> The Bylaws say:
> 
>       (iv) The SSR Review Team shall also assess the extent to which
>   prior SSR Review recommendations have been implemented and the extent
>   to which implementation of such recommendations has resulted in the
>   intended effect.
> 
> I think we have done what the Bylaws require.

We are in full agreement on the task list  as per the bylaws, especially on the SSR1 recommendations.  I assumed it is still work in progress as you asked for comments/discussions. 

> 
> Some actions were taken to implement each of the recommendations.  The write-up summarizes those actions.


We concluded that some recommendations  were not fully implemented, while staff concluded that they were all fully implemented.

This would call for  reviewing board/staff understanding of the recommendations, the implementation plan and  actions taken to implement to eventually determine where the gap is.

> 
> We made an assessment regarding whether those actions "resulted in the intended effect.”

my main concern in this thread is about how this assessment was made. It  needs a more structured approach than going through  responses to some questions to staff  and  through  some random documentation.


> 
> In some cases, we have follow-on recommendations.
> 
> We agreed that the wording of many SSR1 recommendations were not measurable,


SSR1 like SSR2 is an “review” and i would say a ”high level review”, from which recommendations may not be directly measurable, as they affect strategic thinking/planning  and operations of the organisation and/or the ecosystem.


This is my last communication of the topic and as Boban suggested, would be good to hear what the team’s thoughts are about these points.

Thanks

—Alain


> and we agreed to two things:
> 1) Include some text regarding the situation.  Your words above seem like a good strawman.
> 2) When we write our recommendations, we will make sure that they are measurable.
> 
> Russ
>