[gnso-improvem-impl-sc] Draft v2-WG Self-Assessment Questionnaire

Tue Jun 18 20:06:37 UTC 2013

Dear Ken,

Thank you for the excellent efforts.  In following your various email
exchanges with Committee members I noted (from below) the following: 

3. The PDP is a complex, multi-step process in its own right and probably
should be evaluated exhaustively, but separately, via survey or other
method. 

On the last point, we were looking for a set of questions that a WG might
address collectively vs. individually. Perhaps a separate questionnaire
could be developed that would delve deeply into the PDP methodology itself
and could be administered to all or a random sample of appropriate WGs. 

Most of us on this list will agree with you that PDPs are complex.  For my
part, I would like us to keep our eye on the ball vis-¨¤-vis establishing
an assessment questionnaire that is more one-size-fits-all as opposed to
different questionnaires for different purposes.  Better to try to compare
apples to apples as best we can. 

PDP methodology is pushing the limits, in my view, in terms of what we are
looking for now.

My two cents (as a member, rather than Chair).

Thanks,

RA

Ron Andruff

RNA Partners

 <http://www.rnapartners.com> www.rnapartners.com 

From: owner-gnso-improvem-impl-sc at icann.org
[mailto:owner-gnso-improvem-impl-sc at icann.org] On Behalf Of Ken Bour
Sent: Tuesday, June 18, 2013 15:05
To: 'Mike O'Connor'; gnso-improvem-impl-sc at icann.org
Cc: 'Hoggarth, Robert (ICANN)'
Subject: RE: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment
Questionnaire

Mikey:

Based upon the way I have designed the WG Self-Assessment, the PROCESS
dimension is intended to refer to the WG’s internal operations (norms,
logistics, decision-making, etc.). In that framework, I would consider the
PDP to be an INPUT to the WG (imposed methodology) and, based upon your
earlier comments, I attempted to cover it generically in the 1st question of
Section II. If a WG member found that the PDP (or any other
requirement/constraint) was detrimental to the team’s ability to accomplish
its mission, he/she could address it in Section II of the questionnaire. 

I would recommend that the WG Self-Assessment instrument not be a means for
evaluating the PDP (per se) for three reasons: 

1)      The WG Guidelines and Charter Template (source documents) do not
specifically integrate the PDP within the WG process scope; rather, the PDP
is specified as part of the ICANN Bylaws. 

2)      Not all WGs deal with PDP issues, which would mean that any such
questions would have to be skipped for some percentage of respondents. The
WG Guidelines mention the PDP, but only as an example of the type of
methodology that may be imposed upon a WG chartered to address a domain name
policy issue. 

3)      The PDP is a complex, multi-step process in its own right and
probably should be evaluated exhaustively, but separately, via survey or
other method. 

On the last point, we were looking for a set of questions that a WG might
address collectively vs. individually. Perhaps a separate questionnaire
could be developed that would delve deeply into the PDP methodology itself
and could be administered to all or a random sample of appropriate WGs. 

Ken

From: owner-gnso-improvem-impl-sc at icann.org
<mailto:owner-gnso-improvem-impl-sc at icann.org>
[mailto:owner-gnso-improvem-impl-sc at icann.org] On Behalf Of Mike O'Connor
Sent: Friday, June 14, 2013 11:12 AM
To: gnso-improvem-impl-sc at icann.org <mailto:gnso-improvem-impl-sc at icann.org>

Cc: Hoggarth, Robert (ICANN); Ken Bour
Subject: Re: [gnso-improvem-impl-sc] Draft v2-WG Self-Assessment
Questionnaire

hi Ken,

see?  sometimes i'm not so lightning fast in my replies.

this is looking really good.  i've got just a little bit i'd like to see us
work into the "Processes" section.  is there a way to get a question or two
in there that gives participants a chance to talk about the PDP process?
we've got decision-making methodology (consensus) in there already, maybe
this question/LO is closely related to that?  looking back at the projects
that the SCI is working on (method to suspend/end a WG is a good example), i
think some feedback on the *structure* of the PDP would be helpful.

for example, we have a lot of comment-periods (and subsequent reviews of
those comments) built into the PDP right now.  Marika laid all those end to
end one time and came up with a minimum time to get through a PDP that's
pretty long.  it would be nice to start getting feedback from WG
participants as to whether they felt that those were all needed and whether
they were helpful to the work of the WG.  i'm seeing a transition in the way
that WG's review those comments.  at first, the review felt like a burden
that we had to get through because it was a requirement imposed on us by the
PD.  more recently those comment-reviews have been a really good source of
discussion-points and preliminary language that we've woven into initial and
final reports.  

if we stick with that example, there's a process-definition dimension (have
we got the right number of comment-periods?) and a process-effectiveness
dimension (did the WG effectively make use of those comments in doing their
analysis?).  both are important.  we don't want to change a good process
that's being badly carried out, in that case we want to improve the
effectiveness of the WG participants.  we DO want to review a bad process
even though the WG has implemented it well if that badly-defined process is
hurting effectiveness and timeliness.  

one reason i picked that particular example is because there's a lot of
pressure to become "more agile" in the PDP right now.  what this often turns
into is a shorthand for "do it faster!"  and, since the comment/review
cycles are fixed/required that sometimes means that the only place to
shorten the process is the deliberative portion of the WG's work.  it would
be nice to be able to get some data from participants that might give an
early indication that something needs to change there.  that analysis would
also be helpful in the "let's not break/bypass the bottom-up process" and
"policy vs implementation" conversations that are going on at the moment.

thanks Ken -- really like where this is headed.

mikey

On Jun 12, 2013, at 10:42 AM, Ken Bour <ken.bour at verizon.net
<mailto:ken.bour at verizon.net> > wrote:

SCI Members:

I have been thinking about the topics raised by Mikey and Avri in their
first impressions of the Draft Self-Assessment Questionnaire. Something was
definitely missing from the original formulation, which led to an “Aha!”
moment spurring me to create a second design which, I hope, addresses a few
of the comments expressed thus far.

I have reconstituted the questionnaire (still five sections but renamed),
reorganized some of the original questions, and added a few new ones. In
order not to lose track of the first iteration, I made a completely new page
and will henceforth house all questionnaire versions under a new heading:
Questionnaire Drafts/Versions ( <https://community.icann.org/x/ai5-Ag>
https://community.icann.org/x/ai5-Ag).

In this new Draft v2 iteration ( <https://community.icann.org/x/bC5-Ag>
https://community.icann.org/x/bC5-Ag), I am attempting to take into
consideration the dimension raised by Mikey that was only partially
accommodated in Draft v1. To evaluate any dynamic system, we could subdivide
it into three basic or core components: Inputs → Processes → Outputs. In
the first version, I captured many of the processes, the outputs, but only a
few of the inputs, namely, team member representativeness, tools, and
outside experts. I did not ask about the other critical inputs that impinge
upon the success of a WG, e.g., its charter/mission (including time or other
constraints) and team memberexpertise. That led to a reconceptualization of
the external resources questions into three buckets: administrative,
technical, and human.

This Draft v2 also shifts the rating scale to 7 points and adds “Background
Contributor” to the Role list as suggested by Avri.

I thank Mikey and Avri for their ideas/suggestions and look forward to
additional feedback from the team…

Ken

PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com
<http://www.haven2.com> , HANDLE: OConnorStP (ID for Twitter, Facebook,
LinkedIn, etc.) 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mm.icann.org/pipermail/gnso-improvem-impl-sc/attachments/20130618/991edb98/attachment.html>