[rssac-caucus] Opening RSSAC-002 for revision

[Wessels, Duane]

> On Oct 31, 2015, at 12:42 PM, Bruce E. Crabill <bcrabill at umd.edu> wrote:
> 
> On Oct 21, 2015, at 12:52 AM, Wessels, Duane <dwessels at verisign.com> wrote:
>> 
>> RSSAC has decided to open the RSSAC-002 "Measurements of the Root Server System"
>> document for revision and is asking the Caucus to produce an update.  Based on
>> implementation experience, three minor problems have been identified in the current
>> document.

Hi Bruce,


> 
> The following are notes from D-Root on RSSAC002. Some of the comments may have already mentioned by others...
> 
> - load-time:
>    - In a system with a large number of sites,there may be a large variance in the time it takes to load specific sites. Should the metric reflect worst case, average, or what? This should be specified so that meaningful comparisons may be be made.

I think the document is pretty clear about this already:

    Due to the nature of operating anycast DNS clouds there may be multiple
    steps in the distribution of the Root Zone, dependent on the internal
    distribution processes at each root-server operator.  Additionally the
    availability of anycasted instances may present as anomalies in
    measurements and investigators are therefore forewarned.  Therefore
    measurements may only represent the average for 95% of the
    operationally-active instances of a root server per root zone serial.


>    - The examples in the document should not include the quotes in the keys.

I see that that the sample YAML for load-time and zone-size do have single quotes
around the keys.  In my test the parser doesn't complain about this, but I'm happy
to suggest they be removed.


> - zone-size:
>    - It would be nice if metric was based on something other than a specific implementation's zone traffic compressed message size. There appears to be differing opinions if that size includes the TCP 2-byte DNS message length prefix. A suggested alternative would be the sum of the uncompressed RR's in the answer set of the AXFR/IXFR response that implements the zone transfer.

As we discussed in the in-person caucus meeting yesterday, I think this
will be addressed separately from some of these easier errata and simple
clarifications.

>    - The examples in the document should not include the quotes in the keys.
> - traffic-volume:
>    - Correct YAML syntax in example.
> - traffic-sizes:
>    - Why is the '0-15' range not included? They do appear to exist.

My guess is that the sample YAML output is from an actual, but short, data
collection during which no '0-15' messages were observed.  If you think it would
be helpful to include something like: 

   0-15: 1

in the sample YAML, I am happy to support that.

>    - The TCP message size should not include the 2-byte length prefix, since that is really a transport header and not part of the DNS message itself.

It was the consensus of the in-person caucus meeting that the 2-byte length prefix should be excluded.


>    - Buckets with zero counts should be allowed to be ignored and not written to the output YAML file, as is done with the 'rcode-volume' metric.

I agree.  This is probably implied but could be made explicit.


> - unique-sources:
>    - Correct example indenting.
>    - Question the usefulness of this metric. It appears to involve more work than it is worth.

We talked about this during the caucus meeting as well.  My opinion is that while it is somewhat
tricky, it is still doable and I think the data is useful and interesting.


> - General:
>    - Many metrics are written from the viewpoint that they represent a single server. The reality is that all RSOs have multiple sites and servers at each site and the document does not take into account about how metrics are to be merged to form a single value representing all the available values. The method should be specified.

Personally I don't see the need for such specifications, but I'd like to hear others' opinions.  


>    - Individual RSOs may wish to make available additional metrics, or alternate versions of the existing metrics. Should we specify a mechanism to support the definition of such metrics?
> 

That sounds to me like something that would warrant a work party.

DW