[RSSAC Caucus] [Ext] 48 HOUR LAST CALL : RSSAC002v4

[Andrew McConachie]

On Feb 25, 2020, at 15:32, John Heidemann <johnh at isi.edu<mailto:johnh at isi.edu>> wrote:

On Tue, 25 Feb 2020 09:32:43 +0000, Andrew McConachie wrote:


On Feb 24, 2020, at 21:10, John Heidemann <johnh at isi.edu<mailto:johnh at isi.edu>> wrote:

On Mon, 24 Feb 2020 10:45:37 +0000, Andrew McConachie wrote:


On Feb 21, 2020, at 23:52, Ray Bellis <ray at isc.org<mailto:ray at isc.org>> wrote:

On 21/02/2020 22:33, Paul Hoffman wrote:

Just to be clear: are you saying that you can only produce figures
for Section 3.5 (number of sources seen) for a subset of your nodes,
or that you can only produce figures for a subset of your nodes for
*all* the measurements? If the latter, that's going to mess up the
evaluation of the measurements for the whole RSS. Even if it is just
the former, that will make the measurements in Section 3.5 be not
comparable to the other sections.

Only for the unique sources measurement.


Hi Ray,

I changed section 6.6 to mark the ‘unique-sources’ metric as optional. This is reflected in the changelog as well.

This change means that if the 'unique-sources' metric is produced by an RSO they must provide values for both 'num-sources-ipv4' and 'num-sources-ipv6-aggregate’. So produce values for both or produce values for neither.

I would like to hear from other RSO’s if this approach is OK or if a different approach is needed.

Ray asked the important question: is this metric useful.

Wearing my "researcher" hat: yes, we have found number of unique-sources
in RSSAC data useful to indicate when a RSO is subject to an attack with
spoofed sources.


But I can understand the challenge in computing this metric.  It is by
far the most stressful (least compressible).


Rather than just declaring the metric optional, what about declaring the
precision of the answer variable?  Perhaps adding a
"fraction-of-traffic-for-sources" field, which for some RSOs would be 1
and others might be less than 1?  There is value in a num-sources even
if it's based on a 10% sample of traffic, or 10% of anycast sites, and
a sampled value might be easier for some RSOs to compute.

(And yes, I recognize that sampling could be non-uniform.)

 -John

Hi John,

I think the idea of a percent-instances-sampled makes a lot of sense, I’m just concerned about adding it this late in the process of RSSAC002v4. Is it OK to postpone this suggestion until RSSAC002v5?

I can imagine a fair amount of discussion surrounding how exactly to measure precision given the different operating environments.


That is understandable.  I'm certainly not insisting it go in.

(Although making unique-sources optional is also a major change.)

It is considering the diff between the document circulated at the beginning of this thread and the current draft RSSAC002v4. But it’s not a major change between the draft RSSAC002v4 and RSSAC002v3. num-sources-* have been optional since RSSAC002v1.

I’m still interested in hearing from other Caucus members what they think about the draft RSSAC002v4 and if any other changes are needed.
<https://docs.google.com/document/d/1na7VpPYfo4VEBfQMHwix6I2Puohvc1hvwRh0aW1cmFE/edit?usp=sharing>

Please let me know by the end of this week if there are any more items to discuss.

Thanks,
Andrew

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mm.icann.org/pipermail/rssac-caucus/attachments/20200225/9154527a/attachment.html>