Subject: [Ppm] Making long-running requests asynchronous

From: Tim Geoghegan <[email-masked]>
Date: Mon, 24 Jan 2022 18:30:04 -0800

Hello ppm,

The design of PPM has thus far assumed that the interactions between
participants will be synchronous HTTP transactions. We've suspected for a
while that this probably won't work in realistic deployments for all
messages[1]. In particular, servicing a Collect request[2] will require the
leader and helper to verify and aggregate every report falling into the
relevant batch interval, which could take quite a lot of distributed
computation by the aggregators, and for VDAFs like `poplar1`, none of that
work can begin until the collector issues the CollectReq. So we wouldn't
want the collector and leader to keep connections open for minutes (maybe
even hours) waiting for an HTTP response from the leader.

To deal with this, I think we will make at least `CollectReq/CollectResp`
into an asynchronous interaction. Roughly, I envision that the Collector
would send a CollectReq to the leader's `collect` endpoint as it does
today, but instead of blocking until it gets an HTTP 200 with a CollectResp
in the body, it would immediately get an HTTP 202 Accepted response whose
body contains a unique identifier (a UUID should work) for the collect job.
The collector would subsequently poll `[leader]/collect/<job identifier>`
at some interval, which would not return HTTP 200 OK until the aggregation
is complete and the CollectResp is ready.

Another option would be for the Collector to listen on some HTTP endpoint
at which it could be notified of completion of collect jobs. The downside
is that this would require collectors to operate a publicly available HTTP
server that listens for requests (I don't think they need to do this yet)
and we would need a way for the leader to authenticate to the collector (to
prevent network attackers from sending collectors bogus aggregates).

I'd love to hear thoughts on what the established best practices for
long-running tasks in HTTP APIs are.

Another problem to chew on is the aggregate portion of the protocol. I feel
certain that collect needs to be asynchronous, but it's not obvious that
the same is true for the aggregation sub-protocol executed between leader
and helper[3]. The reason is that each message in that interaction
references far far fewer reports than any collect message, and the leader
should always be able to carve up execution of the aggregate protocol into
individual jobs that reference few enough reports so that synchronous
handling of aggregate messages should be reasonable. What kind of
heuristics usually inform whether or not to make an API asynchronous?
Should we make these messages async just in case it ends up being useful in
the future?

Thanks,
Tim Geoghegan
ISRG

[1] https://github.com/abetterinternet/ppm-specification/issues/111
[2]
https://abetterinternet.github.io/ppm-specification/draft-gpew-priv-ppm.html#name-collecting-results
[3]
https://abetterinternet.github.io/ppm-specification/draft-gpew-priv-ppm.html#name-verifying-and-aggregating-r

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 25 Jan 2022 14:29:53 +1100

Hi Tim,

> long-running tasks in HTTP APIs are.

Maybe this might help you: https://httpwg.org/specs/rfc7240.html#respond-async  (URIs, not UUIDs)

You might also find this informative: https://www.rfc-editor.org/rfc/rfc8820.html

You are probably familiar with how ACME does asynchronous processing, which is largely OK.

> What kind of heuristics usually inform whether or not to 
> make an API asynchronous? Should we make these messages async just in 
> case it ends up being useful in the future?

This probably comes down to preference at some level.  However, asynchronous interactions are a fair bit more complex than synchronous ones, so avoiding that complexity is good.  Keep in mind that modern HTTP versions aren't stalled if there are a handful of long-running requests, so unless you are talking a fairly significant time, it might be cleaner and easier to just have the client wait on hold.

Asynchronous requests do come with some benefits to go with the complexity. For instance, robustness can be improved if the initial request creates resources that can be referred to later on a different connection without having to rely on the connection remaining viable.

RFC 7240 does suggest that you might offer both, but I would only go that far if you thought that implementations would consistently use both options.