FW: [gnso-improvem-impl-sc] Draft WG Self-Assessment Design, Objectives, and Questionnaire

Tue Jun 11 00:37:36 UTC 2013

Mikey just noticed that I replied only to him on this email and not to the
entire group. Remedying that inadvertent error now.

From: Ken Bour [mailto:ken.bour at verizon.net] 
Sent: Monday, June 10, 2013 7:30 PM
To: 'Mike O'Connor'
Subject: RE: [gnso-improvem-impl-sc] Draft WG Self-Assessment Design,
Objectives, and Questionnaire

Mikey:

Thanks for another blazingly fast response.

RE: PDP Framework

You mentioned this item several times in your initial comments, but I'll
address it once here. This is an interesting question as to whether the PDP
framework itself is so intricately linked to the WG process that it could,
on its own merits, influence the overall effectiveness of the WG. If I were
to rephrase your inquiry, would this be close: "If the PDP process is
cumbersome and difficult to follow, a WG's Mission could be challenged (if
not inhibited) from the outset. If we are intending to evaluate a WG's
effectiveness, one of the issues may be the very subject/content (and
attendant processes) it is required to follow, e.g., the PDP framework." If
I'm near the mark, I had not thought about that linkage until you raised it.
Most of the WGs I have supported were not PDP-related, so it escaped my
attention. I will be interested to see what others have to add on this
matter. Another option would be to develop a separate survey-like instrument
to evaluate the effectiveness of the PDP Framework itself although, as you
point out, it is the WG construct that is asked to navigate those waters.
Hmmm. 

RE: Individual vs. Collective

For consistency (analytical, historical, design), I think we should have one
defined process vs. having each WG roll its own. That design constraint
would not preclude identifying a couple of overarching questions that we
want each WG to address every time (via separate format). Furthermore, it is
not inconceivable to have a couple of questions that are addressed both
collectively and individually. Then, we could compare both sets of answers
to see whether the WG answered similarly or differently from the individuals
that made up the team (very intriguing and reminiscent of the Abilene
Paradox puzzle). 

RE: Demographics

I agree that private vs. public is complex and there are pros/cons to be
weighed. I tend to favor the collection of personal identifying info for the
reasons provided, but safeguarding that info in order to promote honesty and
candor (still requires trust). Even in a largely volunteer organization,
there are some people who will not be comfortable offering what may be
perceived as negative commentary unless they can do so under the cover of
anonymity. I think it might be cybernetics that advises: in dynamic systems,
we learn primarily from error and, if failures go unreported and
undiscovered, then those improvement opportunities are forever lost. 

RE: Size, Length, Complexity

Hopefully, providing several open-ended text boxes will be generally
sufficient for those who have a "lot to say." I'm not sure that any online
survey instrument could be developed that would accommodate the exhaustive
and insightful analysis you authored on your experience with the Fast Flux
WG. From my recollection, that was and probably will always be remembered as
a ONE-OF-A-KIND! J 

RE: Learning Objectives

You are exactly correct. I started by extracting LOs from the WG Guidelines
and Charter material; then, I built the questionnaire. Once I got into it, I
began seeing that I could fold a couple of topics together (similar content)
and ended up with fewer questions. 

Thanks for the input! I am energized by the responses thus far and am
looking forward to continuing this process. 

Ken

From: owner-gnso-improvem-impl-sc at icann.org
[mailto:owner-gnso-improvem-impl-sc at icann.org] On Behalf Of Mike O'Connor
Sent: Monday, June 10, 2013 5:27 PM
To: Ken Bour
Cc: gnso-improvem-impl-sc at icann.org; Hoggarth, Robert (ICANN)
Subject: Re: [gnso-improvem-impl-sc] Draft WG Self-Assessment Design,
Objectives, and Questionnaire

hi Ken,

i really like the work you've done.  i've sprinkled reactions inline, but
overall this is a great start.

thanks,

mikey

On Jun 10, 2013, at 3:43 PM, "Ken Bour" <ken.bour at verizon.net> wrote:

SCI Team Members:

Since our last teleconference, I have been working on the development of a
WG Self-Assessment instrument. Rather than inundate the SCI email list with
multiple drafts and iterations during the incubation period, I thought it
would be more helpful to post my work products to the ICANN Community Wiki.
I have now reached a point where I think the material is ready for your
review.

In particular, you will find the following pages under the GNSO tab
(category) and Working Group Resources space. Below is the hierarchy
including links:

WG Self-Assessments:  <https://community.icann.org/x/2Cp-Ag>
https://community.icann.org/x/2Cp-Ag

really good introduction.  on second reading i realized that one thing that
might need expanding in the "lower-level" chunks of work is the LOs and
questions that try to tease out the effectiveness of the formally-documented
processes (the newly-changed PDP, developed by all the WGs that came before
us).  not a lot more, but just a bit.  i think one of the reactions i had to
the first questionnaire was that it was too focused on the formal-process
stuff.  so there's some balancing to be found.  i could certainly live with
the balance you've struck in this first pass -- but if there was a graceful
way to add just a little more review of the PDP framework, i'd like that
too.  not too much though.  we're going for "just right."  :-)

.         Design Considerations:  <https://community.icann.org/x/USt-Ag>
https://community.icann.org/x/USt-Ag

individual vs collective -- i like where you're going in the highlighted box
-- some kind of hybrid between the two, maybe up to the WG.  if they get
really stuck arriving at consensus, just let the discussion serve as
documentation rather than trying for consensus.  i agree, it would be a
shame to run a working group through another tough consensus process at the
very end.  especially tough if they weren't at consensus on their findings
and were reopening old wounds.

demographic information -- all good -- wisecracks about the NSA
notwithstanding.  although another approach to consider might be to have the
information be public.  i don't know -- tradeoffs.  something to puzzle
about.

size, complexity, length -- yep, this is where i came into the discussion --
the previous questionnaire was really long/hard to fill out.  but it would
be interesting to see if there's a way to build in "optional" sections if
people had a lot to say.  my post-mortem on my experience chairing the Fast
Flux WG took several days to write.  if somebody's got a will to do
something like that, we should provide a way.

methodology -- no strong opinions either way here.  

.         Learning Objectives:  <https://community.icann.org/x/hit-Ag>
https://community.icann.org/x/hit-Ag

first pass -- WOW!  i like these a lot.  there may be more, but this is a
really good start.  good for: starting off the WG to set norms and
expectations, periodically referring back to "mid flight" to offer
course-corrections to the process as its's going, and reflection on
opportunities for improvement at the end.

since we're a PDP-process focused group, i'm wondering if there's a way to
work a couple of LOs in about the framework itself.  the goal being to see
if there WG members noticed flaws in the process that got in their way.  a
good source of ideas might be the questionnaire i was so cranky about.  i
was mostly cranky because it was hard to answer, but maybe there's a way to
rework some of those into LO's that fit in here?

.         Questionnaire (Draft v1):  <https://community.icann.org/x/EC1-Ag>
https://community.icann.org/x/EC1-Ag

this looks like it flows pretty directly out of the LO's -- so maybe we
focus on those first?  i really like how the number of LOs is longer than
the number of questions -- this looks like a questionnaire that could be
filled out pretty quickly.  and maybe it's just distributed as a
fill-in-blanks PDF or spreadsheet (to make scraping the answers easier)?  

same reactions on the "policy-process" part being a little light, perhaps.
not a bunch more questions, but maybe one or two for those reflective types
who want a place to note something in the PDP that got in the WG's way?

If you want to follow my developmental logic path, I suggest starting at the
top and proceeding in the sequence shown. If you want to skip to the actual
draft questionnaire, it is contained in the bottom linked page. I included a
few notes which you should bear in mind before perusing the actual
questions.

At this stage, I would like to receive your feedback before proceeding
further to be sure that I am on a productive track. You will certainly
notice that this instrument is no longer focused on the WG Guidelines
documentation (the original survey scope); instead, on the effectiveness of
the WG's operations, norms, logistics, decision-making, and outputs. I
included my rationale for that shift at the top of the
<https://community.icann.org/x/USt-Ag> Design Considerations page.

Feel free to comment on the email list or add comments to the individual
Wiki pages. I will attend to both sources over the next week or so as you
have time to review and evaluate my recommended approach to the WG
Self-Assessment.

Regards,

Ken Bour

PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE:
OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.) 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mm.icann.org/pipermail/gnso-improvem-impl-sc/attachments/20130610/886712d0/attachment.html>