Dealing with unique constraints that should be replaced in django_rest_framework - django-rest-framework

tl;dr: How can I ignore (turn off) a unique constraint in django_rest_framework Create calls with a ListCreateAPIView, because I'm going to deal with it manually in the perform_create method?
Im using a third party library django-push-notifications. It has a nice model for APNSDevice (apple push notification service device) that has a unique constraint on a registration_id field.
My problem is that sometimes I want to manually delete old values in the table that have the registration ID, so that I can insert a new value. I'd like to use this serializer:
class APNSDeviceSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = APNSDevice
fields = ('name', 'active', 'device_id', 'registration_id')
along with this code for PUT
class MyAppleDevices(generics.ListCreateAPIView):
permission_classes = (permissions.IsAuthenticated,)
serializer_class = APNSDeviceSerializer
model = APNSDevice
def get_queryset(self):
return APNSDevice.objects.filter(user = self.request.user)
def perform_create(self, serializer):
print "Looking for old devices with registration id "+str(self.request.registration_id)
oldDevices = APNSDevice.objects.filter(registration_id = self.request.registration_id)
for oldDevice in oldDevices:
oldDevice.delete()
apnsDevice = serializer.save(user=self.request.user)
In other words, I'm trying to manually delete other entries that have the unique constraint in this particular PUT, so that I can insert the new one without violating the unique constraint. The problem is the validator runs before the perform_create method is called, and I can't figure out how to turn off the validator's unique constraint. I tried adding this to the Serializer
def get_validation_exclusions(self, instance = None):
exclusions = super(APNSDeviceSerializer, self).get_validation_exclusions(instance)
return exclusions + ['registration_id']
but it doesn't help so obviously I have no clue even though I've been pouring through the documentation and Stack Overflow posts. Any help appreciated, thanks. I suppose as a last resort I could remove the unique constraint from the model, but it is a valid constraint so I'd rather leave it in.

I found this question because I had this exact problem with that exact library. You can get around it by subclassing the serializer and manually overriding the field definition:
class APNSDeviceSerializerWithNonUniqueRegistrationId(APNSDeviceSerializer):
registration_id = serializers.CharField(min_length=64, max_length=64)
class Meta(APNSDeviceSerializer.Meta):
fields = ("name", "registration_id", "device_id", "active", "date_created")
Then, if you're using django-push-notifications, you'll also need to override the ViewSet that uses that serializer:
class APNSDeviceAuthorizedViewSetWithNonUniqueRegistrationId(AuthorizedMixin, APNSDeviceViewSet):
"""
The out of the box viewset/serializer combo require the registration ID to be unique and won't
allow setting a registration ID to a new user (which is useful if we have potentially more than
one account on a device.)
"""
serializer_class = APNSDeviceSerializerWithNonUniqueRegistrationId
def perform_create(self, serializer):
if self.request.user.is_authenticated():
try:
existing_registration = APNSDevice.objects.get(
registration_id=serializer.validated_data['registration_id'])
existing_registration.delete()
except APNSDevice.DoesNotExist:
pass
serializer.save(user=self.request.user)
return super(DeviceViewSetMixin, self).perform_create(serializer)

Related

django rest framework: SlugRelatedField options- limit by user

Note: I'm new to django rest framework. First project with it. My model has a foreign key which in the serializer I link by a SlugRelatedField. This related model is a small list of options that can be selected however what is available for selection depends on the user (or more specifically the user group).
I have found this question that says how to get the user into a serilaizer. But this doesn't seem to help as the field definition is static right?
Removing the irrelevant parts I have:
class MyModelSerializer(serializers.ModelSerializer):
sequence = serializers.SlugRelatedField(
many=False,
read_only=False,
slug_field='prefix',
queryset=Sequence.objects.filter(active=True,
sequence_groups__sequence_group_id__in=SequenceGroup.objects.filter(users=serializers.CurrentUserDefault()))
)
This query works as I also use it in a normal form. When I start the dev. server I get an exception:
TypeError: int() argument must be a string, a bytes-like object or a number, not 'CurrentUserDefault'
So my question is how can I get the current user into the queryset of the SlugRelatedField?
It's funny how after hours of trying, writing down the question leads one to a working solution.
Add request to sterilizer context in ModelViewSet
Simply add below method to the ModelViewSet:
def get_serializer_context(self):
return {"request": self.request}
Adjust the queryset of the SlugRelatedField in the constructor of the Serializer
def __init__(self, *args, **kwargs):
super(MyModelSerializer, self).__init__(*args, **kwargs)
# superuser can choose from all sequences, normal users can only choose from
# active sequences he is assigned to
request = self.context.get("request")
if request and hasattr(request, "user"):
sequence = self.fields['sequence']
if request.user.is_superuser:
sequence.queryset = Sequence.objects.all()
else:
sequence.queryset = Sequence.objects.filter(active=True,
sequence_groups__sequence_group_id__in=SequenceGroup.objects.filter(users=request.user))
In my case the admin should be able to select any of the available options hence the extra condition.

Django REST Framework (DRF): ModelSerializer does not validate models on serialization

I want to ask how to use Django REST Framework (DRF) ModelSerializers correctly for serializing from model.
I have Django model with two required fields:
class Book(models.Model):
title = models.CharField()
desc = models.CharField()
I have DRF ModelSerializer:
class BookSerializer(serializers.ModelSerializer):
class Meta:
model = Book
fields = ['title', 'desc']
I can deseralize and validate incoming request using:
serializer = BookSerializer(data=request.data)
serializer.is_valid(raise_exception=True)
But how to serialize and send response? DRF allows me to break contact built using ModelSerializer. If I forgot to set one of mandatory Book fields, it will still still pass through BookSerializer!
invalid_book = Book(title="Foo") # but forgotten to set "desc"
serializer = BookSerializer(instance=invalid_book)
serializer.data # it contains book without required "desc"
Serialized created using instance parameter throws error if I try is_validate().
Why ModelSerializer can validate incoming data, but cannot outgoing?
Validation is only performed when deserializing. As per the documentation:
Serializers also provide deserialization, allowing parsed data to be converted back into complex types, after first validating the incoming data.
That makes sense (edit: in the way Django Rest Framework seems to be construed). Because it isn't the 'role' of the Serializer to make sure that your complex data such as querysets and model instances (eg. your Book instance) that you are going to serialize is construed 'legitimately', thus they also don't validate while serializing.
So if you would save the instance like invalid_book.save(), Django would throw an error because of the missing field.
Edit
After a comment about being 'a point of view' and thus being opiniated I want to stress and make clear that this seems to be the way that Django Rest Framework (DRF) is construed. After digging deeper on SO I link this answer in support.
Also if you read the documentation of DRF, it is somewhat implied that serialization and validation are two separate concepts.
Furthermore, analyzing serializers.py makes clear that validation is only run when calling is_valid() and the validation is only run on the provided data flag. In fact, it can't even be run when only an instance is provided:
def __init__(self, instance=None, data=empty, **kwargs):
self.instance = instance
if data is not empty:
self.initial_data = data
self.partial = kwargs.pop('partial', False)
self._context = kwargs.pop('context', {})
kwargs.pop('many', None)
super().__init__(**kwargs)
...
def is_valid(self, raise_exception=False):
assert hasattr(self, 'initial_data'), (
'Cannot call `.is_valid()` as no `data=` keyword argument was '
'passed when instantiating the serializer instance.'
)
if not hasattr(self, '_validated_data'):
try:
self._validated_data = self.run_validation(self.initial_data)
except ValidationError as exc:
self._validated_data = {}
self._errors = exc.detail
else:
self._errors = {}
if self._errors and raise_exception:
raise ValidationError(self.errors)
return not bool(self._errors)
You are under a very wrong assumption. A serializer (not de-serializer) does one thing. Convert an Object to JSON. Here, you are creating an object Book(name='sad book'). This is just a regular Python Object. Django Serializers will attempt to serialize any Object that is passed to it.
What you might be wondering is the field is required in Model but why doesn't the serializer validate? Because of the way DRF handles serialization. I will show some excerpts from DRF Source code.
This is how the data property is calulated.
class BaseSerializer():
...
...
#property
def data(self):
if hasattr(self, 'initial_data') and not hasattr(self, '_validated_data'):
msg = (
'When a serializer is passed a `data` keyword argument you '
'must call `.is_valid()` before attempting to access the '
'serialized `.data` representation.\n'
'You should either call `.is_valid()` first, '
'or access `.initial_data` instead.'
)
raise AssertionError(msg)
if not hasattr(self, '_data'):
if self.instance is not None and not getattr(self, '_errors', None):
# THIS IS WHERE WE GO. THE to_representation() CAN BE FOUND IN THE IMPLEMENTATION
# OF ModelSerializer() which inherits from this class BaseSerializer
self._data = self.to_representation(self.instance)
elif hasattr(self, '_validated_data') and not getattr(self, '_errors', None):
self._data = self.to_representation(self.validated_data)
else:
self._data = self.get_initial()
return self._data
What happens in ModelSerializer.to_representation() ?
class ModelSerializer(BaseSerializer):
...
...
def to_representation(self, instance):
"""
Object instance -> Dict of primitive datatypes.
"""
ret = OrderedDict()
fields = self._readable_fields
for field in fields:
try:
attribute = field.get_attribute(instance)
except SkipField:
continue
# We skip `to_representation` for `None` values so that fields do
# not have to explicitly deal with that case.
#
# For related fields with `use_pk_only_optimization` we need to
# resolve the pk value.
check_for_none = attribute.pk if isinstance(attribute, PKOnlyObject) else attribute
if check_for_none is None:
ret[field.field_name] = None
else:
ret[field.field_name] = field.to_representation(attribute)
return ret
As you can see, in this case the serializer only maps the fields from the Object being passed. So, there is no validation during serialization. For more info, check the source code of DRF. It's pretty easy if you use Pycharm Pro.

django-REST: Nested relationships vs PrimaryKeyRelatedField

Is it better to use nested relationships or PrimaryKeyRelated field if you have lots of data?
I have a model with deep relationships.
For simplicity I did not add the colums.
Model:
Usecase:
User creates 1 Workoutplan with 2 Workouts and 3 WorkoutExercises.
User creates 6 Sets for each WorkoutExercise/Exercise.
User starts workout > new FinishedWorkout is created
User does first exercise and enters the used weights > new FinishedWorkoutExercise with FinishedSet is created
Question:
I want to track the progression for each workoutplan > workout > exercise.
So with time the user may have finished dozens of workouts therefore hundreds if sets are already in the database.
If I now use nested Relationships I may load a lot of data I don't need.
But if I use PrimaryKeyRelatedFields I have to load all the data I need separately which means more effort in my frontend.
Which method is preferred in such a situation?
Edit:
If I use PrimaryKeyRelatedFields how do I distinguish if e.g. Workouts in Workoutplan is an array with primary keys or an array with the loaded objects?
If you use PrimaryKeyRelatedField, you'll have a big overload to request the the necessary data in frontend
In your case, I would create specific serializers with the fields you want (using Meta.fields attribute). So, you won't load unecessary data and the frontend won't need to request more data from backend.
I can write a sample code, if you need more details.
I'll get to the question regarding serializers in a second, but first of all and for clarification. What is the purpose of having duplicate models as Workout/Finished Workout, Set/Finished Set,...?
Why not...
class Workout(models.Model):
#...stuff...
finished = models.DateTimeField(null=True, blank=True)
#...more stuff...
Then you can just set a finished date on a workout when it's done.
Now, regarding the question. I would suggest you think about user interactions. What parts of the front-end are you trying to populate? How is the data related and how would the user access it?
You should think about what parameters you're querying DRF with. You can send a date and expect workouts finished on a specific day:
// This example is done in Angular, but you get the point...
var date= {
'day':'24',
'month':'10',
'year':'2015'
};
API.finishedWorkout.query(date).$promise
.then(function(workouts){
//...workouts is an array of workout objects...
});
Viewset...
class FinishedWorkoutViewset(viewsets.GenericAPIView,mixins.ListModelMixin):
serializer_class = FinishedWorkOutSerializer
queryset = Workout.objects.all()
def list(self, request):
user = self.request.user
day = self.data['day'];
month = self.data['month'];
year = self.data['year'];
queryset = self.filter_queryset(self.get_queryset().filter(finished__date=datetime.date(year,month,day)).filter(user=user))
page = self.paginate_queryset(queryset)
serializer = self.get_serializer(queryset, many=True)
return response.Response(serializer.data)
And then your FinishedWorkoutSerializer can just have whatever fields you want for that specific type of query.
This leaves you with a bunch of very specific URLs, which isn't all that great, but you can use specific serializers for those interactions and you're also open to dynamically changing the filter, depending on what paramaters are in self.data.
There is also a chance that you may want to filter differently depending what method is being called, say you want to list only active exercises, but if a user queries a specific exercise, you want him to have access to it (note that the Exercise object should have a models.BooleanField attribute called "active").
class ExerciseViewset(viewsets.GenericViewSet, mixins.RetrieveModelMixin, mixins.ListModelMixin):
serializer_class = ExerciseSerializer
queryset = Exercise.objects.all()
def list(self, request):
queryset = self.filter_queryset(self.get_queryset().filter(active=True))
page = self.paginate_queryset(queryset)
serializer = self.get_serializer(queryset, many=True)
return response.Response(serializer.data)
Now you have different objects show up on the same URL, depending on the action. It's a bit closer to what you need, but you're still using the same serializer, so if you need a huge nested object on retrieve(), you're also gonna get a bunch of them when you list().
In order to keep lists short and details nested, you need to use different serializers.
Let's say you want to only send exercises' pk and name attributes when they are listed, but whenever an exercise is queried, you wan't to send along all related "Set" objects ordered inside an array of "WorkoutSets"...
# Taken from an SO answer on an old question...
class MultiSerializerViewSet(viewsets.GenericViewSet):
serializers = {
'default': None,
}
def get_serializer_class(self):
return self.serializers.get(self.action, self.serializers['default'])
class ExerciseViewset(MultiSerializerViewSet, mixins.RetrieveModelMixin, mixins.ListModelMixin):
queryset = Exercise.objects.all()
serializers = {
'default': SimpleExerciseSerializer,
'retrieve': DetailedExerciseSerializer
}
Then your serializers.py could look a bit like...
#------------------Exercise
#--------------------------Simple List
class SimpleExerciseSerializer(serializers.ModelSerializer):
class Meta:
model Exercise
fields = ('pk','name')
#--------------------------Detailed Retrieve
class ExerciseWorkoutExerciseSetSerializer(serializers.ModelSerializer):
class Meta:
model Set
fields = ('pk','name','description')
class ExerciseWorkoutExerciseSerializer(serializers.ModelSerializer):
set_set = ExerciseWorkoutExerciseSetSerializer(many=True)
class Meta:
model WorkoutExercise
fields = ('pk','set_set')
class DetailedExerciseSerializer(serializers.ModelSerializer):
workoutExercise_set = exerciseWorkoutExerciseSerializer(many=True)
class Meta:
model Exercise
fields = ('pk','name','workoutExercise_set')
I'm just throwing around use cases and attributes that probably make no sense in your model, but I hope this is helpfull.
P.S.; Check out how Java I got in the end there :p "ExcerciseServiceExcersiceBeanWorkoutFactoryFactoryFactory"

Adding filtering changes ordering

I have a ModelViewSet that I want to add filtering to. My simple model looks like
class Article(models.Model):
date = = models.DateField()
language = models.CharField(max_length=10)
class Meta:
ordering = ['-date']
And the ModelViewSet (read only):
class ArticleViewSet(viewsets.ReadOnlyModelViewSet):
queryset = Article.objects.all()
serializer_class = ArticleSerializer
Articles on the API are now ordered by date descending as I would expect. Now I wich to allow filtering on language. I've set the filter backend to DjangoFilterBackend in settings.py. My updated ModelViewSet now looks like:
class ArticleViewSet(viewsets.ReadOnlyModelViewSet):
queryset = Article.objects.all()
serializer_class = ArticleSerializer
filter_fields = ['language']
This changes the ordering to language ASC. Adding order_by('-date') to queryset does not change anything. Adding ordering = ('-date', ) does not change anything. => How do I specify both filtering and ordering (or simply use default ordering while allowing filtering)?
EDIT:
Current functionality seems to come from AutoFilterSet created in Rest Framework by default:
https://github.com/tomchristie/django-rest-framework/blob/822eb39599b248c68573c3095639a831ab6df99a/rest_framework/filters.py#L53
... where order_by=True and the handing of this in django-filter get_ordering_field here: https://github.com/alex/django-filter/blob/d88b98dd2b70551deb9c128b209fcf783b325acc/django_filters/filterset.py#L325
=> Seems I have to create a FilterSet class:
class LanguageFilter(django_filters.FilterSet):
class Meta:
model = Article
fields = ['language']
order_by = model()._meta.ordering
class ArticleViewSet(viewsets.ReadOnlyModelViewSet):
queryset = Article.objects.all()
serializer_class = ArticleSerializer
filter_class = LanguageFilter
Does this look correct? Seems a bit "much"/verbose to retain default ordering.
Rather than implementing your own FilterSet, you can instead just add an OrderingFilter, specifying an ordering = ['-date'] or better: ordering = Article._meta.ordering on your view, to restore the lost (default) ordering. This would also allow your users to use an ordering query parameter to override your default ordering of results.
Note that this issue has been resolved in master... https://github.com/tomchristie/django-rest-framework/pull/1836 and is due to be released in version 2.4.3.
Good question.
Is ok to apply an ordering filter in conjuction with a Django-Filter but I think is not right that a Filter Backend applies a reorder function.
In my case I have to cache my random queryset and so i can't use Django-Filter anymore, even if I'm not filtering at the page's first asyncronous call.

Exposing "virtual" field in a tastypie view?

I want to create a view using tastypie to expose certain objects of the same type, but with the following two three twists:
I need to get the objects using three separate queries;
I need to add a field which doesn't exist in the underlying model, and the value of that field depends on which of the queries it came from; and
The data will be per-user (so I need to hook in to one of the methods that gets a request).
I'm not clear on how to hook into the tastypie lifecycle to accomplish this. The recommended way for adding a "virtual" field is in the dehydrate method, which only knows about the bundle it's operating on.
Even worse, there's no official way to join querysets.
My problem would go away if I could get tastypie to accept something other than a queryset. In that case I could pass it a list of subclasses of my object, with the additional field added.
I'm open to any other sensible solution.
Edit: Added twist 3 - per-user data.
In the last version you should override the dehydrate method, e.g.
def dehydrate(self, bundle):
bundle.data['full_name'] = bundle.obj.get_full_name()
return bundle
Stumbled over similar problem here. In my case, items in the list could be "checked" by user.
When an item is retrieved by AJAX, its checked status is returned with the resource as a normal field.
When an item is saved to the server, "checked" field from the resource is stored in user's session.
First I thought hydrate() and dehydrate() methods to be the best match for this job, but turned out there are problems with accessing request object in these. So I went with alter_data_to_serialize() and obj_update(). I think there's no need to override obj_create(), since item can't be checked when it's first created, I think.
Here is the code, but note that it hasn't been properly tested yet.
class ItemResource(ModelResource):
def get_object_checked_status(self, obj, request):
if hasattr(request, 'session'):
session = request.session
session_data = session.get(get_item_session_key(obj), dict())
return session_data.get('checked', False)
return False
def save_object_checked_status(self, obj, data, request):
if hasattr(request, 'session'):
session_key = get_item_session_key(obj)
session_data = request.session.get(session_key, dict())
session_data['checked'] = data.pop('checked', False)
request.session[session_key] = session_data
# Overridden methods
def alter_detail_data_to_serialize(self, request, bundle):
# object > resource
bundle.data['checked'] = \
self.get_object_checked_status(bundle.obj, request)
return bundle
def alter_list_data_to_serialize(self, request, to_be_serialized):
# objects > resource
for bundle in to_be_serialized['objects']:
bundle.data['checked'] = \
self.get_object_checked_status(bundle.obj, request)
return to_be_serialized
def obj_update(self, bundle, request=None, **kwargs):
# resource > object
save_object_checked_status(bundle.obj, bundle.data, request)
return super(ItemResource, self)\
.obj_update(bundle, request, **kwargs)
def get_item_session_key(obj): return 'item-%s' % obj.id
OK, so this is my solution. Code is below.
Points to note:
The work is basically all done in obj_get_list. That's where I run my queries, having access to the request.
I can return a list from obj_get_list.
I would probably have to override all of the other obj_* methods corresponding to the other operations (like obj_get, obj_create, etc) if I wanted them to be available.
Because I don't have a queryset in Meta, I need to provide an object_class to tell tastypie's introspection what fields to offer.
To expose my "virtual" attribute (which I create in obj_get_list), I need to add a field declaration for it.
I've commented out the filters and authorisation limits because I don't need them right now. I'd need to implement them myself if I needed them.
Code:
from tastypie.resources import ModelResource
from tastypie import fields
from models import *
import logging
logger = logging.getLogger(__name__)
class CompanyResource(ModelResource):
role = fields.CharField(attribute='role')
class Meta:
allowed_methods = ['get']
resource_name = 'companies'
object_class = CompanyUK
# should probably have some sort of authentication here quite soon
#filters does nothing. If it matters, hook them up
def obj_get_list(self, request=None, **kwargs):
# filters = {}
# if hasattr(request, 'GET'):
# # Grab a mutable copy.
# filters = request.GET.copy()
# # Update with the provided kwargs.
# filters.update(kwargs)
# applicable_filters = self.build_filters(filters=filters)
try:
#base_object_list = self.get_object_list(request).filter(**applicable_filters)
def add_role(role):
def add_role_company(link):
company = link.company
company.role = role
return company
return add_role_company
director_of = map(add_role('director'), DirectorsIndividual.objects.filter(individual__user=request.user))
member_of = map(add_role('member'), MembersIndividual.objects.filter(individual__user=request.user))
manager_of = map(add_role('manager'), CompanyManager.objects.filter(user=request.user))
base_object_list = director_of + member_of + manager_of
return base_object_list #self.apply_authorization_limits(request, base_object_list)
except ValueError, e:
raise BadRequest("Invalid resource lookup data provided (mismatched type).")
You can do something like this (not tested):
def alter_list_data_to_serialize(self, request, data):
for index, row in enumerate(data['objects']):
foo = Foo.objects.filter(baz=row.data['foo']).values()
bar = Bar.objects.all().values()
data['objects'][index].data['virtual_field'] = bar
return data

Resources