I have a table with 3500 entries and foreign keys.
I'm using client-sided processing datatables with: django-rest-framework and Ajax.
It takes up to 10 seconds to load.
Is there a way to show the first 10 entries (first page results) - for the user not to think that my website is broken, because it's taking too long - while the rest of the entries loads in the background?
Also looking for optimizations for the load speed.
Thank you for your time.
EDIT:
models.py
class Bibrest51(models.Model):
cadastro_id = models.AutoField(primary_key=True)
autor = models.CharField(db_column='Autor', max_length=255, blank=True, null=True)
tema = models.ForeignKey('BibTema', models.DO_NOTHING, blank=True, null=True)
tipo = models.ForeignKey('Tipo', models.DO_NOTHING, blank=True, null=True)
class Meta:
managed = False
db_table = 'bibrest51'
#property
def bib_tipo_nome(self):
return self.tipo.tipo_nome
class BibTema(models.Model):
tema_id = models.AutoField(primary_key=True)
tema_nome = models.CharField(max_length=150, blank=True, null=True)
class Meta:
managed = False
db_table = 'tema'
def __str__(self):
return self.tema_nome
serializers.py
class TemaSerializer(serializers.ModelSerializer):
class Meta:
model = BibTema
fields = '__all__'
depth = 1
class TipoSerializer(serializers.ModelSerializer):
class Meta:
model = Tipo
fields = '__all__'
depth= 1
class BibSerializer(serializers.ModelSerializer):
temas = TemaSerializer(read_only=True)
tipos = TipoSerializer(read_only=True)
class Meta:
model = Bibrest51
fields = (
'autor', 'ano', 'titulo', 'referencia','tipos','temas'
)
For optimization you can use:
Nested relationship serializing
Pagination System
Caching output
Using select-related and prefetch-related
Is there a way to show the first 10 entries (first page results) - for the user not to think that my website is broken, because it's taking too long - while the rest of the entries loads in the background?
You can implement this with pagination. Request the first 10 records and if the total number of records is more than 10, then request the rest in the background
UPDATED
For serializer try this:
class TemaSerializer(serializers.ModelSerializer):
class Meta:
model = BibTema
fields = '__all__'
# depth= 1 # delete string
class TipoSerializer(serializers.ModelSerializer):
class Meta:
model = Tipo
fields = '__all__'
# depth= 1 # delete string
class BibSerializer(serializers.ModelSerializer):
temas = TemaSerializer(read_only=True, sourse='tema')
tipos = TipoSerializer(read_only=True, sourse='tipo')
class Meta:
model = Bibrest51
# There are fewer fields in the code you provided,
# so I rely on you for this.
fields = (
'autor', 'ano', 'titulo', 'referencia','tipos','temas'
)
Related
SCENARIO
I have a many-to-many relationship between two models:
Supplier
class Supplier(models.Model):
class Meta:
unique_together = ['supplier_no', 'supplier_name']
ordering = ['supplier_name']
supplier_no = models.IntegerField(blank=False, null=False)
supplier_name = models.CharField(max_length=180)
...
updated = models.DateTimeField(auto_now=True, blank=True)
updated_by = models.ForeignKey(UsaUser, on_delete=models.CASCADE, blank=True, null=True,
related_name='supplierUpdatedByUser')
def __str__(self):
return self.supplier_name
Plant
class Plant(models.Model):
class Meta:
unique_together = ['plant_no', 'plant_name']
ordering = ['plant_no']
plant_no = models.IntegerField(unique=True)
plant_name = models.CharField(max_length=180, unique=True)
...
suppliers = models.ManyToManyField(Supplier, related_name='plants')
updated = models.DateTimeField(auto_now=True, blank=True)
updated_by = models.ForeignKey(UsaUser, on_delete=models.CASCADE, blank=True, null=True,
related_name='plantUpdatedByUser')
def __str__(self):
return self.plant_name
Simply put: a supplier can be active in any number of plants, and a plant can have any number of suppliers.
And here is the SupplierSerializer with the create method in question:
class SupplierSerializer(serializers.ModelSerializer):
plants = PlantSerializer(many=True) # serializes entire supplier objects instead of just returning the pk
class Meta:
model = Supplier
fields = ['id', 'supplier_no', 'supplier_name', ... 'plants']
def create(self, validated_data):
# Add the UsaUser as a blameable field, which will be passed in the 'context' object to the serializer.
validated_data.update({"updated_by": self.context['request'].user})
assoc_plants = validated_data.pop('plants') # remove the many-to-many association from the data before saving
supplier = Supplier.objects.create(**validated_data)
# Now add in the associated plants
for plant in assoc_plants:
supplier.plants.add(plant)
return supplier
...
PROBLEM:
When creating a Supplier, I get the following 400 response:
{"plants":[{"non_field_errors":["Invalid data. Expected a dictionary, but got int."]},{"non_field_errors":["Invalid data. Expected a dictionary, but got int."]}]
If I remove the following line from my serializer:
plants = PlantSerializer(many=True)
the problem is resolved. However, I want full plant objects to be returned to my front end, not just the plant ids.
I thought maybe I needed to return the full Plant object since the error says it's looking for a dictionary, but then I get another error:
{"plants":[{"plant_no":["plant with this plant no already exists."],"plant_name":["plant with this plant name already exists."]}]}
Sample Request Payload
{"supplier_no":"5052","supplier_name":"MySupplier","is_active":true,"plants":[1,10]}
^^ 1 and 10 are the pks of the Plants
When passing a whole Plant object:
{"supplier_no":"54564","supplier_name":"MySuppleir","is_active":true,"plants":[{"id":1,"plant_no":1,"plant_name":"AutoPlant1","is_active":true...}]}
I think you want to use the plants that are already created. Then you can use some extra fields.
class SupplierSerializer(serializers.ModelSerializer):
plants = PlantSerializer(many=True, read_only = True) # set it as read_only
plant_ids = serializers.ListField(
child = serializers.IntegerField(),
write_only = True
)
class Meta:
model = Supplier
fields = [..., 'plant_ids'] # add plant_ids
def create(self, validated_data):
plant_ids = validated_data.pop('plant_ids')
supplier = Supplier.objects.create(**validated_data)
supplier.plants.set(plant_ids)
return supplier
I am trying to implement a feature to my backend and allow the owner of private "Group" to add other users by their usernames instead of ID's and allow them to add their images to FileField only once after they were added to the model. The code I have so far:
models.py
class Group(models.Model):
group_name = models.CharField(max_length=255)
group_text = models.TextField(max_length=360, blank=True)
owner = models.ForeignKey(
settings.AUTH_USER_MODEL, related_name='owner_user', on_delete=models.SET(get_deleted_user), default=1)
created_on = models.DateTimeField(auto_now_add=True, null=True)
shared_to = models.ManyToManyField(UserProfile, blank=True, related_name='shared_to_user', null=True)
def __str__(self):
return self.group_name
def save(self, *args, **kwargs):
super(Group, self).save(*args, **kwargs)
class GroupImage(models.Model):
group_file = models.FileField(blank=True, null=True,
upload_to='media/covers/%Y/%m/%D/')
gallery_group = models.ForeignKey(Group, related_name='images', on_delete=models.CASCADE)
serializers.py
class GroupImageSerializer(serializers.ModelSerializer):
class Meta:
model = models.GroupImage
fields = ('group_file', )
class SharedToSerializer(serializers.ModelSerializer):
class Meta:
model = models.Group
fields = ('shared_to', )
class GroupSerializer(serializers.ModelSerializer):
images = GroupImageSerializer(many=True, read_only=True)
person = SharedToSerializer(many=True, read_only=True)
class Meta:
model = models.Group
fields = ('id', 'group_name', 'group_text', 'person', 'images')
def create(self, validated_data):
images_data = self.context.get('view').request.FILES
owner_id = self.context['request'].user.id
gallery_group = models.Group.objects.create(group_name=validated_data.get('group_name', 'no-
group_name'), group_text=validated_data.get('group_text'), owner_id=1)
for image_data in images_data.values():
models.GroupImage.objects.create(gallery_group=gallery_group,
group_file=image_data)
return gallery_group
views.py
class GroupCreateAPIView(generics.CreateAPIView):
queryset = models.Group.objects.all()
serializer_class = serializers.GroupSerializer
permission_classes = [AllowAny]
So if your only requirement is how to add users by their username and not their id. You should use SlugRelatedField. I also feel your serializer naming convention is quite confusing. Below is the serializer for Group model that can add users to a group.
class GroupSerializer(Serializer):
... other fields here
shared_to = models.SlugRelatedField(queryset = UserProfile.objects.all(), many=True, slug_field="username", allow_empty=True)
So first checkout SlugRelatedField. This basically is used to map to objects using a specific field of that object(username in this case). You will then get all the UserProfile instances in the shared_to field of the validated_data
property of the serializer which you can fetch in create method and add to you group. And then in the file upload api for your group you can check whether this user belongs to the group or not for permission checking.
I have two models in a 'many to many' relationship Video and Genre. I want to represent them in my API using the Django Rest Framework and with Hyperlinking. I have tried using the the nested relationships solution that I have found over the internet but it brings me an error for the list viewss eg genres/ and videos/ as described below.
models.py
class VideoGenre(models.Model):
video = models.ForeignKey(Video, on_delete=models.CASCADE, related_name='genres')
genres = models.ForeignKey(Genre, on_delete=models.CASCADE, related_name='videos')
class Meta:
unique_together = ['video', 'genre']
class Video(models.Model):
genres = models.ManyToManyField(Genre, related_name="videos", blank=True, through='VideoGenre')
title = serializers.CharField(default=serializers.CurrentUserDefault())
class Genre(models.Model):
name = models.CharField(max_length=20)
descr = models.CharField(max_length=255)
serializers.py
class GenreSerializer(serializers.HyperlinkedModelSerializer):
videos = serializers.HyperlinkedIdentityField(many=True, view_name="video-detail")
class Meta:
model = ku_api_models.Genre
fields = "__all__"
class VideoSerializer(serializers.HyperlinkedModelSerializer):
genres = GenreSerializer(source='genre_set', many=True, read_only=True)
class Meta:
model = ku_api_models.Video
fields = "__all__"
class VideoGenreSerializer(serializers.HyperlinkedModelSerializer):
video = serializers.ReadOnlyField(source='video.id')
genre = serializers.ReadOnlyField(source='genre.id')
class Meta:
model = ku_api_models.VideoGenre
fields = "__all__"
unique_together = ['video', 'genre']
views.py
class VideoDetailView(RetrieveUpdateDestroyAPIView):
parser_classes = (MultiPartParser,)
serializer_class = VideoSerializer
queryset = Video.objects.all()
def perform_update(self, serializer):
serializer.save(owner=self.request.user)
class VideoListView(ListCreateAPIView):
parser_classes = (MultiPartParser,)
serializer_class = VideoSerializer
queryset = Video.objects.all()
def perform_create(self, serializer):
serializer.save(owner=self.request.user)
class GenreListView(ListCreateAPIView):
serializer_class = GenreSerializer
queryset = Genre.objects.all()
class GenreDetailtView(RetrieveUpdateDestroyAPIView):
serializer_class = GenreSerializer
queryset = Genre.objects.all()
urls.py
path('videos/', views.VideoListView.as_view(), name='video-list'),
path('videos/<int:pk>', views.VideoDetailView.as_view(), name='video-detail'),
path('genres/', views.GenreListView.as_view(), name="genre-list"),
path('genres/<int:pk>', views.GenreDetailtView.as_view(), name="genre-detail"),
I am getting below exception on the browsable API renderer
Exception Type: ProgrammingError at /api/genres/
Exception Value: relation "ku_api_videogenre" does not exist
LINE 1: ..._video"."owner_id" FROM "ku_api_video" INNER JOIN "ku_api_vi...
I have looked into these below links as referred above
django rest framework serializing many to many field
many to many relationship through an intermidiate model
serializing many to many field
... and tried them but getting the same/almost the same, only different is the relation name that "does not exist"
What am I missing? How to make it right? Thanks
I have found a way to this:
Changes will be in the serializers.py
serializers.py
class VideoSerializer(serializers.HyperlinkedModelSerializer):
genres = serializers.HyperlinkedRelatedField(many=True, read_only=False, view_name='genre-detail', queryset=ku_api_models.Genre.objects.all())
class Meta:
model = ku_api_models.Video
#fields = ('url', 'id', 'datetime_added', 'title', 'video_file', 'owner')
fields = "__all__"
def create(self, validated_data):
genres = validated_data.pop('genres', [])
video_instance = ku_api_models.Video.objects.create(**validated_data)
video_instance.genres.set(genres)
return video_instance
def update(self, instance, validated_data):
genres = validated_data.pop('genres', [])
instance.genres.set(genres)
instance.save()
return instance
class VideoGenreSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = ku_api_models.VideoGenre
fields = "__all__"
class GenreSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = ku_api_models.Genre
fields = "__all__"
Source: Docs: intermediary-manytomany
I am making a POST api using DRF. In that api, I need only few fields(name, size, customer_name, customer_address), but don't require this fields(status, ordered_time) because these fields I want to save these fields in run time as status='open' and ordered_time=DateTimeField.now()
views.py
class PizzaOrderCustomerView(APIView):
def post(self, request):
orders = request.data.get('orders')
# Create an article from the above data
serializer = ArticleSerializer(data=orders)
if serializer.is_valid(raise_exception=True):
article_saved = serializer.save()
return Response({"success": "Article '{}' created successfully".format(article_saved.name)})
models.py
class PizzaOrder(models.Model):
name = models.CharField(max_length=120)
size = models.CharField(max_length=10, choices=SIZE_CHOICE, default='MEDIUM')
customer_name = models.CharField(max_length=120)
customer_address = models.TextField()
ordered_time = models.DateTimeField(default=timezone.now, editable=False)
status = models.CharField(max_length=20, default='open', editable=False)
serializers.py
class OrderSerializer(serializers.ModelSerializer):
class Meta:
model = PizzaOrder
# fields = '__all__'
read_only_fields = ('status',)
But when I try to create an order, it needed status and ordered_time also. But it should save at the time of creating order automatically.
Suggest a good way to do it.
from rest_framework import viewsets, mixins
class PizzaViewsets(viewsets.ViewSet, mixins.CreateModelMixin):
model = PizzaOrder
serializer_class = OrderSerializer
queryset = model.objects.all(
serializer, it is always good practise to mention all fields instead of
all
class OrderSerializer(serializers.ModelSerializer):
class Meta:
model = PizzaOrder
fields = ('status','ordered_time','name', 'size', 'customer_name', 'customer_address',)
read_only_fields = ('status','ordered_time',)
I have the following models:
class Contact(models.Model):
class Meta:
managed = False
db_table = 'contact'
class ContactPhone(models.Model):
contact = models.ForeignKey(Contact, on_delete = models.CASCADE)
number = models.CharField(max_length = 45)
class Meta:
managed = False
db_table = 'contact_phone'
Also, I have the following serializers:
class ContactSerializer(serializers.ModelSerializer):
server_id = serializers.IntegerField(source='id', read_only=True)
class Meta:
model = Contact
fields = '__all__'
class ContactPhoneSerializer(serializers.ModelSerializer):
class Meta:
model = ContactPhone
fields = '__all__'
Now, I have a view that insert phone numbers for an existing contact.
The input is a json that looks like this:
data = {'contact_id': 12322,
'phones':[{'number': '89120000001'}]}
The view:
def insert_contact_phone(request):
for record in request.data['phones']:
data['contact_id'] = request.data['contact_id']
serializer = ContactPhoneSerializer(data = data)
if serializer.is_valid():
serializer.save()
I end up with the following error:
RelatedObjectDoesNotExist at /contacts/edit ContactPhone has no
contact.
What am I doing wrong?
If you specify __all__ for the fields in your ContactPhoneSerializer, it does not include contact_id.
So the contact_id taken from the json input is not serialized. It is basically ignored and when you try to save and create new ContactPhone - it fails, because it does not have contact's foreign key correctly set.
But simply adding contact_id to the serializer's fields won't solve your problem.
In your view, i recommend you to set the contact instead:
data['contact'] = request.data['contact_id']
and pass this to the ContactPhoneSerializer.