Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to get the response after upload is complete #2312

Open
kazuho opened this issue Nov 11, 2022 · 11 comments · May be fixed by #2744
Open

how to get the response after upload is complete #2312

kazuho opened this issue Nov 11, 2022 · 11 comments · May be fixed by #2744

Comments

@kazuho
Copy link
Contributor

kazuho commented Nov 11, 2022

IIUC, with #2292, we have two resources. One being the resource to which the transaction is started (A), and another to which the chunks are sent (B).

Assuming that responses that we get for resource B is that for resource B, how do we obtain the response for resource A when the upload is complete?

To give an example, consider the following case:

  • Client starts a POST to /foo, and server sends a 1xx response specifying a transaction resource /transaction/1234.
  • The clients uploads the final chunk to /transaction/1234, and gets a 500 response.

Does this 500 mean that the upload of the final chunk failed? Or is it a response for the original request (for resource A)?

@kazuho
Copy link
Contributor Author

kazuho commented Nov 11, 2022

IMHO, one way of fixing this ambiguity issue would be to define a method to commit the transaction, that would target the original URI (A), even though chunks can be uploaded through a different URI (B).

@guoye-zhang
Copy link
Contributor

This is a really good point. Currently, responses to Upload Appending Procedure include the upload offset. We could say something like "if the offset is equal to the size of the upload, treat this response as the final response and don't retry".

@Acconut
Copy link
Member

Acconut commented Feb 15, 2023

"if the offset is equal to the size of the upload, treat this response as the final response and don't retry".

I like this idea, yes. However, what should the client do if it does not receive the response for the last Upload Appending Procedure? Should we add a mechanism where the client can retry fetching the response for the last Upload Appending Procedure?

@guoye-zhang
Copy link
Contributor

If they don't receive the response, performing an offset retrieving and attempting a resumption (even if it's a 0-length resumption at the end) should be sufficient I think.

@Acconut
Copy link
Member

Acconut commented Feb 15, 2023

Sounds good. Shall we add this to the current draft?

@guoye-zhang
Copy link
Contributor

Sure

@kazuho
Copy link
Contributor Author

kazuho commented Mar 28, 2023

If they don't receive the response, performing an offset retrieving and attempting a resumption (even if it's a 0-length resumption at the end) should be sufficient I think.

👍 this sounds good to me. Thank you for the efforts.

The flip side of the resolution is that, once the server receives the entire upload, the server is expected to generate a response and cache it for certain amount of time, because the client might fail to receive the response and re-issue a PATCH request that is possibly zero-length.

As the PATCH request (or the original POST / PUT request that initiates the upload) is not idempotent, a response for a Resumable Upload request cannot be resumed (like we do for file downloads). But that's probably okay, most HTTP responses for Resumable Upload is going to be tiny.

@Acconut
Copy link
Member

Acconut commented Mar 30, 2023

The flip side of the resolution is that, once the server receives the entire upload, the server is expected to generate a response and cache it for certain amount of time

That is correct, but probably not a big issue in practice. The servers already have to create a resource for storing the upload state and data, so including additional response data has likely little overhead. Even more, if the server is not interested in returning a response, they can be hardcoded to return an empty response. So o don't think this is too problematic.

@MikeBishop
Copy link
Contributor

MikeBishop commented Nov 13, 2023

The server could also include the location of the response, either:

  • as a Content-Location when the client uploads the final chunk, alongside actually sending the response
  • as a Location when the client queries the status of a completed upload resource

Once the client has the URL of the response, even if it's large and interrupted, it can perform traditional resumable downloads on it for as long as the server is willing to keep it around.

@Acconut
Copy link
Member

Acconut commented Nov 24, 2023

I agree with the feedback from Prague that Content-Location would be a great fit to solve this problem. The server can sends a URL using Content-Location during the upload. The client can then use this URL to query a potential result of the upload process. I will open a PR to add a note explaining this possibility.

@smatsson
Copy link

I have updated the tusdotnet POC to add support for Content-Location. For now it will call user code each time the Content-Location is requested and the user code can determine if it wants to provide a CL or just ignore the call (resulting in no header returned to the client).

Some thoughts on Content-Location that are probably not really thought through:

  • Can it change once set? Can it differ between HEAD/POST/PATCH responses?
  • Can the header be returned directly when the upload is created or can it only be returned once the upload is complete?
  • The URI pointed to by Content-Location - is it allowed to return different data during processing? Based on the below section I would say yes.

"Otherwise, such a Content-Location indicates that this payload is
a representation reporting on the requested action's status and
that the same report is available (for future access with GET) at
the given URI. For example, a purchase transaction made via a
POST request might include a receipt document as the payload of
the 200 (OK) response; the Content-Location field-value provides
an identifier for retrieving a copy of that same receipt in the
future." - https://datatracker.ietf.org/doc/html/rfc7231#section-3.1.4.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.

5 participants