How to optimize the Search Filter which returns paginated result but the intermediate table can return millions for records but actual results is few - spring-boot

Example
I have 3 entities Users, Address & County
The User is having one to one Relation with Address Entity and Country has one to many Relation with Address. I want to display users details with filter parameter on country code.
The one of the possible flow is I will fetch the corresponding Object of Country from Country code. Then we find Country in all in address entity. Then we will be getting list of addresses of that country code. Then we will find all the address corresponding with users using in Query 'in' SQL. But the issue arise when the data set is very large.
Lets say we a Country Code IN corresponds to 1 Million Address then I will 1st fetch the all 1 Million records then map it with the Users. Then I am performing pagination. It will be very slow.
I am thinking of an idea to paginate the address entity from the response somehow ?
Please help

Related

Salesforce flow Datatable multiple resource

In my flow I am getting accounts filtered with Billing city with get records. I have two get records elements to filter data on two different cities. I want to show selected rows from both the tables in third table. I tried Assignment element and formula element but both are not available for datatable resources.

Pivot two objects of same table in laravel

Assume the following two tables (stripped down to this question, assume there is more data on both):
user
id
user_similarity
user_1_id
user_2_id
similarity
I'd like to use user_similarity as a pivot table to construct a list of users that are similar to a given user.
To avoid storing duplicate data on the pivot table, I'd only like one record per user pair, with the lower user ID being stored in the first, and the higher ID in the second column.
Is it possible to set this up without duplicate data records and only a single eloquent relation?

Data structure to show postal code on selecting city

I have a table with three fields-
Postal code
City
Suburb
Each row has a unique combination of these three columns. On the front-end I have an address form where I want to show the list of postal codes when the user is typing the city name. Something like this-
The table has around 30,000 entries so I can't hit the database everytime the user adds an alphabet. I want to store the entire table in a data structure when the form page loads and then retrieve the relevant rows everytime an alphabet is added in the city box.
What is the data structure that I should use to save the entire table in the JVM when the page loads?
Why not a Hash Map with keys as city names, and values as a string concatenating all the three fields (which is ultimately what has to be sent to the browser)
This map can be pre populated at startup

Search/retrieve by a large OR query clause with Solr or Elasticsearch

I have a search database of car models: "Nissan Gtr", "Huynday Elantra", "Honda Accord", etc...
Now I also have a user list and the types of cars they like
user1 likes: carId:1234, carId:5678 etc...
Given user 1 I would like to return all the cars he likes, it can be 0 to even hundreads.
What the best way to model this in Solr or potentially another "nosql" system that can help with this problem.
I'm using Solr but I have the opportunity to use another system if I can and if it makes sense.
EDIT:
Solr solution is to slow for Join (Maybe we can try nested). And the current MySQL solution which uses join tables has over 2 billion rows.
so, you just want to store a mapping between User->Cars, and retrieve the cars based on the user...sounds very simple:
Your docs are Users: contain id (indexed), etc fields
one of the field is 'carsliked', multivalued, which contains the set of car ids he likes
you have details about each care in a different collection for example.
given a user id, you retrieve the 'carsliked' field, and get the car details with a cross collection join
You could also use nested object to store each liked car (with all the info about it) inside each user, but is a bit more complex. As a plus, you don't need the join on the query.
Solr would allow you many more things, for example, given a car, which users do like it? Elasticsearch will work exactly the same way (and probably many other tools, given how simple your use case seems).

How to store different type and number of fields in one database table?

Hello everybody I'm making a "Bulletin board", like this: http://stena.kg/ad/post, I'm using Laravel 5.0, and don't know how to store different fields in database table, for example if I choose "Cars" category I should to fill Mark, Model, Fuel (etc fields for cars category), If I choose Flats category I should fill fields like Area, Number of rooms etc...How to organize all of this? I tried some ideas but nothing helped me(
Try to save data as json in table. Parse json format to string and save it in db, but it will cause many problems in future, so not recommend that solution. I recommend to store data in separate tabels, each one for category. For optimise process it is possible to create catregory table, and category_item table with fields like name, description and so on. Different category demands sp=ecific fields, so best solution is to create table per category.

Resources