[RSSAC Caucus] [Ext] 48 HOUR LAST CALL : RSSAC002v4

[John Heidemann]

On Tue, 25 Feb 2020 09:32:43 +0000, Andrew McConachie wrote: 
>
>
>> On Feb 24, 2020, at 21:10, John Heidemann <johnh at isi.edu> wrote:
>> 
>> On Mon, 24 Feb 2020 10:45:37 +0000, Andrew McConachie wrote: 
>>> 
>>> 
>>>> On Feb 21, 2020, at 23:52, Ray Bellis <ray at isc.org> wrote:
>>>> 
>>>> On 21/02/2020 22:33, Paul Hoffman wrote:
>>>> 
>>>>> Just to be clear: are you saying that you can only produce figures
>>>>> for Section 3.5 (number of sources seen) for a subset of your nodes,
>>>>> or that you can only produce figures for a subset of your nodes for
>>>>> *all* the measurements? If the latter, that's going to mess up the
>>>>> evaluation of the measurements for the whole RSS. Even if it is just
>>>>> the former, that will make the measurements in Section 3.5 be not
>>>>> comparable to the other sections.
>>>> 
>>>> Only for the unique sources measurement.
>>> 
>>> 
>>> Hi Ray,
>>> 
>>> I changed section 6.6 to mark the ‘unique-sources’ metric as optional. This is reflected in the changelog as well.
>>> 
>>> This change means that if the 'unique-sources' metric is produced by an RSO they must provide values for both 'num-sources-ipv4' and 'num-sources-ipv6-aggregate’. So produce values for both or produce values for neither.
>>> 
>>> I would like to hear from other RSO’s if this approach is OK or if a different approach is needed.
>> 
>> Ray asked the important question: is this metric useful.
>> 
>> Wearing my "researcher" hat: yes, we have found number of unique-sources
>> in RSSAC data useful to indicate when a RSO is subject to an attack with
>> spoofed sources.
>> 
>> 
>> But I can understand the challenge in computing this metric.  It is by
>> far the most stressful (least compressible).
>> 
>> 
>> Rather than just declaring the metric optional, what about declaring the
>> precision of the answer variable?  Perhaps adding a
>> "fraction-of-traffic-for-sources" field, which for some RSOs would be 1
>> and others might be less than 1?  There is value in a num-sources even
>> if it's based on a 10% sample of traffic, or 10% of anycast sites, and
>> a sampled value might be easier for some RSOs to compute.
>> 
>> (And yes, I recognize that sampling could be non-uniform.)
>> 
>>   -John
>> 
>Hi John,
>
>I think the idea of a percent-instances-sampled makes a lot of sense, I’m just concerned about adding it this late in the process of RSSAC002v4. Is it OK to postpone this suggestion until RSSAC002v5?
>
>I can imagine a fair amount of discussion surrounding how exactly to measure precision given the different operating environments. 
>

That is understandable.  I'm certainly not insisting it go in.

(Although making unique-sources optional is also a major change.)

   -John