I am using Joi 10.6.0 in a Typescript project to validate the payloads on a series of requests. I am having the weirdest issue while validating a dynamic key in the payload:
validate: {
payload: Joi.object()
.pattern(/^id_[\s\S]*$/i, Joi.string().required())
.keys({
state: Joi.string().optional(),
presentation: Joi.string().optional(),
// etc
The idea is that I want the payload to include at least one key starting with id_. Alas, the required property on that pattern key seems to be totally ignored.
The regex matches the key correctly, and if I put other types of validation on the value, like Joi.number() the value is validated accordingly, but if I use string().required() and the value for id_ is not in the payload, the validation, bafflingly, still passes. In short, any validation seems to be applied, bar required.
I am sure I am doing something stupid but I am stumped -- I use Joi every day and I've never had such issues.
Related
I have an import that takes data from a foreign API and updates a Django model. The code to do this passes the existing model instance and the incoming data to the serializer class to get an updated instance to save.
For this update I don't want to fail the validation, or the update, if some of the incoming data is wrong, but I would like to notify the errors (warnings) so they can be logged outside of the serializer.
Is there a DRF way of doing this, or any way of doing this?
To be clear: I still want the serializer to validate and raise for other less complex fields, but not the special fields that will need additional processing.
I'm currently over-riding to_internal_value in the serializer so that I can deal with the strangely formed data and record error messages before I let the incoming data through to super().to_internal_value(): https://www.django-rest-framework.org/api-guide/serializers/#overriding-serialization-and-deserialization-behavior
My work-around is to add a warnings attribute to the serializer to save the messages in. The to_internal_value over-ride can then selectively remove bad fields from the incoming data and record warning messages in the warnings list.
Update: I think this is going to work but I'm still wondering if there's a better Django built-in approach.
I'm going to answer my own question because I have worked out my intended solution was more reasonable than I thought. If I get a better answer I'll select it instead.
To be able to raise on some invalid fields but not others requires the incoming serialization to be intercepted to avoid the standard behaviour, which is why to_internal_value must be overridden.
To be able to pass back any warnings without raising an exception requires some persistent attribute somewhere, logically on the Serializer object I suppose, because we can't rely on the processing not raising somewhere down the line once we pass execution back to Django ModelSerializer.
This all seems quite solid in retrospect, so I guess I was just being cautious because I hadn't gone this route before. I'd really appreciate anyone raising any gotchas.
Django already has the errors OrderedDict that it uses in the validation exception to formalise per-field error messages. The trouble is, this can't be accessed until is_valid() is called, so another attribute is needed.
Here is a simplified example implementation.
This will work just like the Django ModelSerializer, except the warnings attribute can be referenced after calling is_valid() for reporting back what happened:
class AnObjectSerializer(serializers.ModelSerializer):
"""
Serialize normal fields using Django ModelSerializer magic to raise when invalid.
Intercept incoming crazy foreign fields and record warnings when invalid without raise.
"""
CRAZY_FIELDS = ('foreign_omicron', 'foreign_delta', 'foreign_crazy')
warnings = {} # Some field failures will not invalidate but are reported here
def to_internal_value(self, data):
"""Try to add incoming foreign crazy fields or warn if not."""
foo = data['foo']
for field in self.CRAZY_FIELDS:
value = data.pop(field, None) or {}
value = value.get('foobar') or '' # Dig crazy field out of the incoming blob
value = value.lower() # Normalize crazy field somewhat
# Do not raise serializers.ValidationError to allow AnObject load to continue
# But report errors back to caller for logging in warnings list
if value:
try:
some_external_validation(value)
except serializers.ValidationError:
self.warnings[field] = f'{foo}: foobar "{value}" has a problem'
value = ''
data[field] = value
validated_data = super().to_internal_value(data)
return validated_data
class Meta:
model = AnObject
fields = (
'foo', 'bar', 'baz', 'qux',
'foreign_omicron', 'foreign_delta', 'foreign_crazy',
)
If this is a one time import only you may:
You can iterate over each object , validate it using serializer manually. If the data is valid you may update else you keep a log file with the exceptional objects that failed validation.
A serializer wont do what you are asking, you need to catch the error or you need to modify the validation function for the fields to pass, if you dont want it to fail. he standard purpose of serializer that it will raise error on invalid data.
Here is my script:-
if (!isset($request->security_token))
{
// Provide security token
$error = TRUE;
$err_message[] = Lang::get('caption_validation_error.ser_valid_security_token.value');
$err_validation[] = Lang::get('caption_validation_error.ser_valid_security_token.value').' [security_token]';
}
It means, if a param is not "sent", then the validation will be triggered. However, if a param with "blank" value is sent, it must not trigger the validation.
However, when I am hitting the api through POSTMAN app, the security_token with blank value enters the !isset validation.
What am I doing wrong?
This is not that obvious why it's like this because it seems quite strange at first glance. However if you know a bit more about Laravel and you look at app/Http/Kernel.php file you will see in there:
\Illuminate\Foundation\Http\Middleware\ConvertEmptyStringsToNull::class,
which as you can find out from name converts empty strings to null. So in your case when you are sending empty parameter it's considered as null so !isset($null) will be true.
So you have 2 options:
remove this middleware from array - however it will affect the whole application so it might not be the best way if you are not sure about it
Assuming you want to trigger this validation only if parameter is not sent at all instead of
if (!isset($request->security_token))
you can use for example
if (!$request->has('security_token'))
Obviously it's not exactly the same - if you now sent this token and set it to null it won't be still executed but I believe when you now know what's happening with this empty string you can now adjust it exactly to your needs.
I'm defining swagger definitions for an API and I came across this use case.
The request and the response object model definitions look exactly the same.
However, there is one field in the object which returns more enum values during the get operation but restricts to minimal enum values for the put operation. Is it possible to reference different enum values for the same field conditionally thus avoiding duplicate definitions. I do not want to recreate the entire model definitions for request and response for the sake of overcoming this limitation.
Here is my example,
definitions:
EntryRequest:
properties:
entries:
$ref: '#/definitions/EntityResponse/properties/entries'
EntryResponse:
properties:
entries:
type: array
items:
$ref: '#/definitions/Entry'
Entry:
properties:
entryStatus:
type: string
enum:
- ENABLE
- DISABLE
- IN_PROGRESS
In the above, there are two things that I'm worried about.
1) For EntryRequest, the API accepts only ENABLE/DISABLE for PUT operation while the API returns all of them in the GET operation. I would like to create a reference to the entryStatus property conditionally. Is it possible?
2) Also, everything except the entryStatus is same for both the EntryRequest and EntryResponse object model. I do not want to duplicate that as well for the sake of representing the differention of entryStatus field.
Is there a way to do it?
EDIT:
As I learn more on this, I get a feeling that this is more an yaml related query. But I would like to have it here to see if anyone has faced a similar situation and how they have handled it. Or to see if I get any recommendations on how to handle this.
For tagging different enums to the same field, I think I can do like this,
RequestEntryStatus:
type: string
enum: &requestStatus
- ENABLE
- DISABLE
ResponseEntryStatus:
type: string
enum: &responseStatus
- ENABLE
- DISABLE
- IN_PROGRESS
Entry:
properties:
entryStatus: *requestStatus
But still this would enforce me to create duplicates of request and response objects with different mapping to the entryStatus field. I would like to know if there is a better way to handle this.
The request and the response object model definitions look exactly the same. However, there is one field in the object which returns more enum values during the get operation but restricts to minimal enum values for the put operation. Is it possible to reference different enum values for the same field conditionally thus avoiding duplicate definitions. I do not want to recreate the entire model definitions for request and response for the sake of overcoming this limitation.
No, it's not possible.
If you want to avoid duplication, you can use a single model with all the enum values, but document verbally in the description that certain values are only used in the responses but not the requests.
I'm trying to find a good way to do field validation in a WebObjects app. If I have a text field and I tie a number formatter to it, it seems that the default behavior is to parse out the number IF the user enters in a valid number, or, if the user enters an invalid number, it seems to just ignore the value entered by the user. I can't do the validation in a save method or an action method because WO will have already ignored the non-number input by the time it reaches the action method. Is there a standard/recommended way, in a WebObjects app, of validating user input such that the user can be alerted of invalid input, rather than just ignoring the invalid input?
This page: http://en.wikibooks.org/wiki/WebObjects/EOF/Using_EOF/Validation claims that WO and EOF have "an incredible array of validation mechanisms" and even hints that there is a built-in way to prevent the user from entering inappropriate data, but I haven't been able to find any documentation or examples of how to do that (if there is, in fact, a built-in way). Coming up with a custom javascript validator to prevent inappropriate data seems like it would be a nightmare - finding a way to make the JS recognize and handle all of the same edge cases that the backend formatters/parsers handle. It would be nice if WO really did have a built-in way to propagate the formatter edge cases over to JS validation.
The above link also says there is a validationFailedWithException method in WOComponent that gets called "when an EO or formatter failed validation during an assignment", but how can I make a formatter fail validation in the non-number example case above? I've tried having the formatter throw an exception in the parse method if a non-number is entered, but that exception doesn't get passed to the validationFailedWithException method. Does anyone know how I can trigger an exception in a formatter that will trigger a call to validationFailedWithException()? And is that even the best/recommended way? Does anyone know of a better way?
I'm pretty sure, that validationFailedWithException is getting called for every formatting error. You should receive there an NSValidationException that wraps a ParseException. The method is usually called on the component containing the binding. It may get skipped on caret (^) bindings.
All the standard number formatter already throw a ParseException (see Format.parse(String)).
The validation handling in WebObjects can get quite complex, it really depends on your needs. But it was designed without JavaScript or Ajax in mind. Newer approaches in Wonder may incorporate the client side, but I have no experience with it.
The normal validation sequence is:
if needed convert the input into the target type with a formatter
call a validateAttributeName method on the target object, where AttributeName is the attribute name to receive the value
When something fails in this sequence validationFailedWithException is called.
While saving an enterprise object "validateFor..." is called on the objects. An exception at this point has to be caught in your action method.
So you have two points to handle validation errors. The "syntactical" errors have to be handled in validationFailedWithException. After this point you have valid inputs. You may manually further check those or greater object structures in your action method or in validateFor... (e.g. validateForSave).
I have an e-mail field which is -not- required in my form validation. It should be able to be left blank. However, when I use the "valid_email" parameter of the form validation's set_rules, it still gives an error message that the e-mail is not valid when it's not supposed to check this if the field has not been filled out.
Rule Reference
Checking the reference on this matter tells us the following regarding the valid email rule:
Returns FALSE if the form element does not contain a valid email address.
This would be true of an empty field, as well as a field with bad values.
Trimming
I notice in the examples provided by CodeIgniter that emails are usually not only required, and required to be valid emails, but are also trimmed. This may result in a different outcome.
$this->form_validation->set_rules('email', 'Email', 'trim|required|valid_email');
During the validation process, the following is considered:
// If the field is blank, but NOT required, no further tests are necessary
if ( ! in_array('required', $rules) AND is_null($postdata))
It may be the case that the contents of your email field aren't exactly null, and are therefore raising flags with the valid_email requirement.
Possible Related Bugs
Three months prior to the date of this answer there was discussion on bitbucket regarding this very topic. The discussion can be viewed at https://bitbucket.org/ellislab/codeigniter-reactor/issue/117/input-fields-are-automatically-required.
It's stated that using array-syntax (see below) in the markup results in similar errors even when the required rule is not set:
<input name="user[email]" />
Further discussion, and patches, are available here, http://codeigniter.com/forums/viewthread/159243. One suggest patch that seems to solve the issue is to replace the is_null() call with empty() in the aforementioned code:
So the following:
if ( ! in_array('required', $rules) AND is_null($postdata))
Becomes:
if ( ! in_array('required', $rules) AND empty($postdata))
according to https://codeigniter.com/user_guide/libraries/validation.html?highlight=validation#id28
use permit_empty this
Allows the field to receive an empty array, empty string, null or
false
so your code looks like this:
$this->form_validation->set_rules('email', 'Email', 'permit_empty|valid_email');
You just have to appreciate that '' IS not a valid e-mail address. If you don't want to validate some postdata and don't care if it's empty, you shouldn't set a rule on it, like so:
if($this->input->post('item'))
{
$this->form_validation->set_rules('item', 'Item number', 'trim|alpha_numeric|max_length[30]');
}
In this case, if there is nothing submitted for 'item', no rule is added, so the rest of the data would go on to validation stage etc. as normal.