Subject: [Ppm] Invalid inputs

From: Max Franke <[email-masked]>
Date: Wed, 30 Mar 2022 16:43:35 +0200

Hi PPM,

as I understand it, the aggregators only include valid inputs in their reports to the collector. The validity is determined by wether or not the data is in a specific range, e.g. between 1 and 30 seconds or between 1 and 2 meters. My question is this: Is there is a way for the collector to learn how many inputs are valid and how many are invalid? What once might have been considered invalid or bogus could eventually (or suddenly) become the norm. If the collector has no way of learning that there is an increase in invalid inputs such changes might get missed and the gathered data becomes less meaningful. Additionally, the collector wouldn’t know if the boundaries that have been set from the get-go are to restrictive and already exclude a lot of results (just consider the fact that there are still websites that require a last name to be at least 3 characters…). 

Best,
Max

---

From: Tim Geoghegan <[email-masked]>
Date: Wed, 30 Mar 2022 09:26:37 -0700

Hi Max,

I think this is a good point. I've previously argued (though I can't find
an email thread or issue anywhere) that aggregators should tell collectors
how many report shares were included in an aggregate share, which besides
making it easy to compute averages serves as a crude checksum on the
aggregate shares that can prevent a collector from accumulating garbage
into their data set. But also including either the total number of inputs
considered or rejected would help participants to detect and remedy
problems (for instance, a sudden spike in rejected inputs might indicate a
bug in the client).

For some context and precedent, we did do something like this in the
Exposure Notification Private Analytics system (ENPA, [1]) (ENPA predates
PPM but it does use Prio). When ENPA aggregators transmit a sum part to the
collector-analog in that system, they include a list of identifiers of
inputs that were rejected.

Tim

[1]:
https://covid19-static.cdn-apple.com/applications/covid19/current/static/contact-tracing/pdf/ENPA_White_Paper.pdf

On Wed, Mar 30, 2022 at 7:43 AM Max Franke <[email-masked]>
wrote:

> Hi PPM,
>
> as I understand it, the aggregators only include valid inputs in their
> reports to the collector. The validity is determined by wether or not the
> data is in a specific range, e.g. between 1 and 30 seconds or between 1 and
> 2 meters. My question is this: Is there is a way for the collector to learn
> how many inputs are valid and how many are invalid? What once might have
> been considered invalid or bogus could eventually (or suddenly) become the
> norm. If the collector has no way of learning that there is an increase in
> invalid inputs such changes might get missed and the gathered data becomes
> less meaningful. Additionally, the collector wouldn’t know if the
> boundaries that have been set from the get-go are to restrictive and
> already exclude a lot of results (just consider the fact that there are
> still websites that require a last name to be at least 3 characters…).
>
> Best,
> Max
> --
> Ppm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ppm
>

---

From: Christopher Patton <[email-masked]>
Date: Wed, 30 Mar 2022 18:29:21 -0700

Hi Max,

I agree this would be a useful feature! One wrinkle is that there are a few
different ways that a report can fail to be aggregated, and the failure
reporting mechanism might want to account for these:
1. The report has been replayed
2. The batch to which the report belongs has already been collected
3. One of the aggregators could not decrypt its share
4. VDAF preparation failure, due to the input being invalid

Unfortunately, the complete set of failure modes isn't all that clear right
now. Hopefully it'll be more clear once we've fleshed out the aggregation
flow a bit more. (See [1] for work-in-progress.) For now, I think it's
worth noting this feature in the "Operational Requirements" section of the
draft. Would you care to send a PR.

Best,
Chris P.

[1] https://github.com/abetterinternet/ppm-specification/pull/179.

---

From: Tim Geoghegan <[email-masked]>
Date: Thu, 31 Mar 2022 16:01:35 -0700

I knew we had discussed this before: issue #57 [1] tracks a mechanism for
reporting errors from aggregators to the collector. Discussion here in
email, there on GitHub, or in a PR as Chris P. suggested is very welcome!

Tim

[1] https://github.com/abetterinternet/ppm-specification/issues/57

On Wed, Mar 30, 2022 at 6:29 PM Christopher Patton <cpatton=
[email-masked]> wrote:

> Hi Max,
>
> I agree this would be a useful feature! One wrinkle is that there are a
> few different ways that a report can fail to be aggregated, and the failure
> Unfortunately, the complete set of failure modes isn't all that clear
> right now. Hopefully it'll be more clear once we've fleshed out the
> aggregation flow a bit more. (See [1] for work-in-progress.) For now, I
> think it's worth noting this feature in the "Operational Requirements"
> section of the draft. Would you care to send a PR.
>
> Best,
> Chris P.
>
>