How to get list of failed parts on s3 multipart upload - go

sometimes after completing the multipart upload request with s3 client, I get this error:
InvalidPart: One or more of the specified parts could not be found. The part may not have been uploaded, or the specified entity tag may not match the part's entity tag.\n\tstatus code: 400, request id: 15BACD159D8498B7
there is no error on any s3client.UploadPart(partInput) calls, so the parts are uploaded well, and my completedParts slice is complete and all of the s3.CompletedPart are in it.
How can I identify those "One or more" parts that s3 can't find and re-upload them?

Related

Which status code to use when an operation has already been done?

Users will be able to share images in my project. But let's say that user1 has already shared the image to user2, but user1 tries to share the picture again to user2. What would be the appropriate status code for "already shared"?
409 is the right status code. It means the item already exists.
In your case "image already shared"
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/409
I do not believe there a standard convention for this scenario. Status code 409 would highlight the fact the request was 1. Received and understood and 2. There was a server side conflict with processing and storing the data. Some other options would be 400, 403, and 422, but those could mean a variety of other issues with the request that could be misinterpreted.

Which http status codes to use when processing http post?

I have a HTML form, which I submit via http post.
There are two cases:
Case 1: The data is valid and data on the server will be updated accordingly
Case 2: The data is invalid and the http response contains an error message for the user.
Which http status codes should be used for each case?
I use htmx to submit the form. This means I don't need to use the POST/Redirect/GET pattern.
This question is not about JSON-APIs.
The complete list of HTTP response codes published by the Mozilla Foundation is pretty comprehensive and easy-to-read, so I'd recommend always consulting it as a guide. For the generic use-cases mentioned by you, there are a couple of different codes you can return - depending on what happens with the data on the server, and what you want to happen in the user's browser.
CASE 1: data is valid, data on server is updated
Base on your short description, the different status codes that might be applicable are:
200 (OK): you are updating an existing record on your own server - eg., the user is submitting a form which updates their existing contact information on your website - and information was received, and the record updated successfully. The response would usually contain a copy of the updated record.
201 (Created): you are not updating an existing record, but rather, creating a new record on your server - eg., your user is adding a new phone number to their contact details, which is saved in your database as a separate 'phone' record. The response should contain a copy of the newly created record.
205 (Reset Content): the same as 200, but implies that the browser view needs to be refreshed. This is useful when the record that is being updated has values that are dynamically calculated by the server, and which might change automatically depending on the values you're submitting. For example, if you have a user adding extra information to their online profile, which might grant them special status, badges and privileges on the website. This means, that if the user is viewing their profile information, that information will need to be updated with the new 'status' automatically granted by the server. The 205 response body will normally be empty, which means that to update the browser view your response-handling code will need to:
do further ajax requests and update the relevant part(s) of your
interface with new information from the server, or
redirect the user to a new URL, or
reload the entire page.
If working with HTMX, a 200 or 201 response would include the actual html snippet of the record that you want updated on the page - and HTMX will replace it automatically for you when it receives the response. With a 205 response, you could send an HX-Trigger response header that would call a custom event on the interface elements that need to update themselves - see the examples in the docs.
CASE 2: data is invalid, data on server is not updated
The status code that needs to be returned in case of an error varies depending on what caused the error. Errors that the server believes are the responsibility of the client - such as 'sending invalid data' - have a status code in the 4XX range. Some of the common errors in that range include 404 ('Not Found'), 403 ('Forbidden'), and 'Unauthorised' (401).
In the case of a client sending data that the server cannot process because it is 'not valid' - either because the request itself is malformed, or because the data doesn't pass some business validation logic - the current advice is to return status 400 (Bad Request).
Many years ago, some people believed that the status code 400 should only be used to indicate a malformed request (syntactical error) - not to indicate a failure in business validation logic (semantic error). There was a lot of debate, and temporarily a new status code (422) was created, that was supposed to cover semantic errors, exclusively. In 2014, however, the official definition of the status 400 code was changed to allow for the inclusion of both syntactical and semantical errors - which rendered status 422 essentially unnecessary.
You can find lots of discussions and explanations online about the differences between 400 and 422, and some people still argue passionately about this to this day. In practice, however, the 400 code is all you'll need - and you can include a response body in it that explains in detail, if needed, the cause of the error.
Note that when working with HTMX, a response with a 400 code should trigger an htmx:responseError event automatically. You can trap that event, for example, to update your form interface elements, in case of data validation errors caught by the server.
Well, 200 OK and 201 Created are the best for successful result.
For invalid data I would return 422 Unprocessable Entity, because the headers are correct, but body is not (though parseable by the server). The caveat is some HTTP clients won't handle 422 properly and in this case you have to use 400 Bad Request, however, the most of the modern clients will be fine.
You have said it is not about JSON APIs, but how will you meet this type of requirement - it is not clear whether this is relevant for your scenario???
SERVER DRIVEN BEHAVIOUR
I cannot see how a client could ever decide an HTTP status code based on input data. How would the client deal with these examples?
The call is not authenticated (cookie or token) - an API would return 401 - this tells the UI to perform a retry action.
The call is not authorized - an API would return 403 or 404 and the UI would present an error display.
The data is malformed or invalid according to domain specific checks - an API would return 400 and tell the UI what is wrong so that it can perform actions.
Something went wrong in server processing, eg data cannot be saved because database is down.
MY THOUGHTS
htmx looks interesting but a key requirement before using it would be ensuring that htmx can read server side error responses and use values returned. Maybe there is an elegant way to do this ...
Maybe I am just paranoid :). But it is worth being careful when choosing technologies that there are no blocking issues. Lack of error handlng control would be a blocking issue in most systems.
I'm using htmx 1.8 with asp.net core 6.0.
This works for me.
controller:
//server side validation failed
Response.StatusCode = 422;
return PartialView("Core", product);
client side javascript:
document.body.addEventListener('htmx:beforeOnLoad', function (evt) {
if (evt.detail.xhr.status === 422) {
//
// allow 422 responses to swap as we are using this as a signal that
// a form was submitted with bad data and want to rerender with the
// error messages
//
evt.detail.shouldSwap = true;
evt.detail.isError = false;
}
});
200 OK or 201 Created are the best choice for a successful POST request.
However, for invald data, you can pass 415 Unsupported Media Type

FineUploader.js - How to access S3 policy document signature?

Purpose: In order to confirm with the server that the image was uploaded to the S3 bucket successfully, I would like to send the policy document signature back to the server. My policy document includes a key randomly generated server-side (this way a malicious user can't overwrite another user's uploads by deliberately using a duplicate key), and a corresponding row is temporarily created in the database marked as "pending". If the correct policy document signature does not come back to the server before the expiry, that means that the upload may have been abandoned, so the server may try to delete any file with that key from the bucket and then delete the temporary database row.
Question: I intend to include the policy document signature in uploadSuccess.params. To accomplish this, how can the policy document signature be accessed?
Possibly relevant snippet of source code, but I don't know how to "reach" responseJson and use it (assuming this is where the signature is contained):
function handleSignatureReceived(id, xhrOrXdr, isError) {
var responseJson = xhrOrXdr.responseText,
pendingSignatureData = pendingSignatures[id],
promise = pendingSignatureData.promise,
signatureConstructor = pendingSignatureData.signatureConstructor,
errorMessage, response;
The response passed to your onComplete callback handler is the response from s3 when the upload completes, as expected. There is no good reason to track the exact signature. If you want to know if the upload failed or succeeded, check for the success property in the response.

Error in uploading file on skydrive in wp7

I am trying to upload some recorded audio files on sky drive after uploading some file i am getting an exception
"The application request limit has been reached".
how to handle this exception.
please suggest some solution.
Thank you
After refering the link
[1]: http://msdn.microsoft.com/en-us/library/windowsphone/develop/microsoft.phone.backgroundtransfer.backgroundtransferservice%28v=vs.105%29.aspx i found that background transfer service is capable of holding only 25 open requests at a time therefore it gives such exception.
To handle the above exception i used following code.
List<BackgroundTransferRequest> list = BackgroundTransferService.Requests.ToList();
foreach (BackgroundTransferRequest item in list)
{
if (item.TransferStatus == TransferStatus.Completed)
{
BackgroundTransferService.Remove(item);
}
}
Apply this code when your request get completed. and if the request count reaches to 25 request stop sending more request until the previous are completed.

Ruby Mechanize Zlib::BufError

Not sure why I'm getting this error now with the Mechanize gem - been using it for a while now with no issues.
My script will randomly stop and throw the following error:
/Users/username/.rvm/gems/ruby-1.9.3-p194/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:798:in `rescue in response_content_encoding': error handling content-encoding gzip: buffer error (Zlib::BufError) (Mechanize::Error)
Any ideas?
It's possible that you're hitting a URL that points to a load-balancer. One of the hosts behind that load-balancer is mis-configured, or perhaps it's configured differently than its peers, and is returning a gzipped version of the content, where others aren't. I've seen that problem in the past.
I've also see situations where the server said it was returning gzipped content, but sent it uncompressed. Or it could be sending zipped, not gzipped. The combinations are many.
The fix is to be sure your code is capable of sensing whether the returned content is compressed. Make sure you're sending the correct acceptable-content HTTP headers for your code to their server too. You have to program defensively and look at the actual content you get back, and then branch to do the right decompression, then pass that on for parsing.
I was able to get around this by setting the request headers like the following:
mechanize.request_headers = { "Accept-Encoding" => "" }

Resources