I'm trying to write an API in Sinatra that accepts a temporary CSV file as a parameter. I want to raise an exception if the filetype isn't text/csv or if the csv doesn't have an email column, and I wanted the confirmation page to simply display the error message. I imagined it to look something like this:
if params[:recipients_file]
raise ArgumentError, 'Invalid file. Make sure it is of type text/csv.' unless params[:recipients_file][:type] == "text/csv"
recipients_csv = CSV.parse(params[:recipients_file][:tempfile].read, {headers: true})
raise ArgumentError, 'Invalid CSV. Make sure it has an "email" column' unless recipients_csv.headers.include?('email')
recipients += recipients_csv.map {|recipient| recipient["email"]}
end
However, any time one of those conditions isn't met, I get really ugly error messages like NoMethodErrors etc. I just want the API to stop execution and to return the error message on the confirmation page. How do I do this?
You should define an error block:
error do
env['sinatra.error'].message
end
See http://www.sinatrarb.com/intro.html#Error for more details, including how to set up different error handlers for different exception types, HTTP status codes, etc.
Related
I have an import that takes data from a foreign API and updates a Django model. The code to do this passes the existing model instance and the incoming data to the serializer class to get an updated instance to save.
For this update I don't want to fail the validation, or the update, if some of the incoming data is wrong, but I would like to notify the errors (warnings) so they can be logged outside of the serializer.
Is there a DRF way of doing this, or any way of doing this?
To be clear: I still want the serializer to validate and raise for other less complex fields, but not the special fields that will need additional processing.
I'm currently over-riding to_internal_value in the serializer so that I can deal with the strangely formed data and record error messages before I let the incoming data through to super().to_internal_value(): https://www.django-rest-framework.org/api-guide/serializers/#overriding-serialization-and-deserialization-behavior
My work-around is to add a warnings attribute to the serializer to save the messages in. The to_internal_value over-ride can then selectively remove bad fields from the incoming data and record warning messages in the warnings list.
Update: I think this is going to work but I'm still wondering if there's a better Django built-in approach.
I'm going to answer my own question because I have worked out my intended solution was more reasonable than I thought. If I get a better answer I'll select it instead.
To be able to raise on some invalid fields but not others requires the incoming serialization to be intercepted to avoid the standard behaviour, which is why to_internal_value must be overridden.
To be able to pass back any warnings without raising an exception requires some persistent attribute somewhere, logically on the Serializer object I suppose, because we can't rely on the processing not raising somewhere down the line once we pass execution back to Django ModelSerializer.
This all seems quite solid in retrospect, so I guess I was just being cautious because I hadn't gone this route before. I'd really appreciate anyone raising any gotchas.
Django already has the errors OrderedDict that it uses in the validation exception to formalise per-field error messages. The trouble is, this can't be accessed until is_valid() is called, so another attribute is needed.
Here is a simplified example implementation.
This will work just like the Django ModelSerializer, except the warnings attribute can be referenced after calling is_valid() for reporting back what happened:
class AnObjectSerializer(serializers.ModelSerializer):
"""
Serialize normal fields using Django ModelSerializer magic to raise when invalid.
Intercept incoming crazy foreign fields and record warnings when invalid without raise.
"""
CRAZY_FIELDS = ('foreign_omicron', 'foreign_delta', 'foreign_crazy')
warnings = {} # Some field failures will not invalidate but are reported here
def to_internal_value(self, data):
"""Try to add incoming foreign crazy fields or warn if not."""
foo = data['foo']
for field in self.CRAZY_FIELDS:
value = data.pop(field, None) or {}
value = value.get('foobar') or '' # Dig crazy field out of the incoming blob
value = value.lower() # Normalize crazy field somewhat
# Do not raise serializers.ValidationError to allow AnObject load to continue
# But report errors back to caller for logging in warnings list
if value:
try:
some_external_validation(value)
except serializers.ValidationError:
self.warnings[field] = f'{foo}: foobar "{value}" has a problem'
value = ''
data[field] = value
validated_data = super().to_internal_value(data)
return validated_data
class Meta:
model = AnObject
fields = (
'foo', 'bar', 'baz', 'qux',
'foreign_omicron', 'foreign_delta', 'foreign_crazy',
)
If this is a one time import only you may:
You can iterate over each object , validate it using serializer manually. If the data is valid you may update else you keep a log file with the exceptional objects that failed validation.
A serializer wont do what you are asking, you need to catch the error or you need to modify the validation function for the fields to pass, if you dont want it to fail. he standard purpose of serializer that it will raise error on invalid data.
I'd like my decoration actions to be able to raise ValidationErrors in certain cases. How should I implement that in a simple way, or at least to return a 400 error with error json?
Ok, I figured it out: I should raise an APIException, which encloses the detail message in json.
I'm developing a RESTful web application in Ruby with Sinatra. It should support CRUD operations, and to respond to Read requests I have the following function that formats the data according to what the request specified:
def handleResponse(data, haml_path, haml_locals)
case true
when request.accept.include?("application/json") #JSON requested
return data.to_json
when request.accept.include?("text/html") #HTML requested
return haml(haml_path.to_sym, :locals => haml_locals, :layout => !request.xhr?)
else # Unknown/unsupported type requested
return 406 # Not acceptable
end
end
Only I don't know what is best to do in the else statement. The main problem is that browsers and jQuery AJAX will accept */*, so technically a 406 error is not really the best idea. But: what do I send? I could do data.to_s which is meaningless. I could send what HAML returns, but they didn't ask for text/html and I would rather notify them of that somehow.
Secondly, supposing the 406 code is the right way to go, how do I format the response to be valid according to the W3 spec?
Unless it was a HEAD request, the response SHOULD include an entity containing a list of available entity characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. Depending upon the format and the capabilities of the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.
It looks like you're trying to do a clearing-house method for all the data types you could return, but that can be confusing for the user of the API. Instead, they should know that a particular URL will always return the same data type.
For my in-house REST APIs, I create certain URLs that return HTML for documentation, and others that return JSON for data. If the user crosses the streams, they'll do it during their development phase and they'll get some data they didn't expect and will fix it.
If I had to use something like you're writing, and they can't handle 'application/json' and can't handle 'text/html', I'd return 'text/plain' and send data.to_s and let them sort out the mess. JSON and HTML are pretty well established standards now.
Here's the doc for Setting Sinatra response headers.
I am making an external request and using HTTPARTY for the JSON file and then am parsing it.
BUT should the request fail (the file no longer exists or is a bad uri) how can I handle errors so I could still display the page?
Not sure of how best to protect the application from this point of failure and I have not done much in error handling.
def api_fetch(url)
JSON.parse HTTParty.get(url).response.body
end
api_fetch('http://example.com/data.json')['test']
Please help
The below should work. It will check if the method returns nil when you call it.
def api_fetch(url)
begin
JSON.parse HTTParty.get(url).response.body
rescue
nil
end
end
Based on the documentation, %s will be replaced by field name when setting error messages.
If you include %s in your error string, it will be replaced with the
"human" name you used for your field when you set your rules.
Is there a way to have multiple %s in the string and define them differently?
This is the line in the form validation library that creates the error message:
// Build the error message
$message = sprintf($line, $this->_translate_fieldname($row['label']), $param);
$line: The error message template, for example "The % field is invalid"
$this->_translate_fieldname($row['label']): The field's label
$param: The parameter passed the the validation function, if any, max_length[5] would pass "5"
So that's the extent of what the form validation class does for you. If you need more flexibility you'll have to prepare the error message yourself <-- might be useful beforehand with the extra variables already populated.
It might be interesting to extend the form validation class via core/MY_form_validation.php and work this functionality in. Unfortunately this line is contained in the main function _execute which is very big and messy, and from looking at it now, you'd need to overload some other functions as well. I started to write an example but it actually looks like a chore. You might be better off using:
$this->form_validation->set_message('rule_name', 'Your custom message here');
More on setting error messages: http://ellislab.com/codeigniter/user_guide/libraries/form_validation.html#settingerrors