Having the query:
$events = $this->entityManager->createQueryBuilder()
->select('e')
->from('AppBundle:Event', 'e')
->where('e.member = :member')
->andWhere('e.isCancelled = false') // this one makes the query extremely slow
->andWhere('DATE(e.startsAt) = :date')
->setParameter('member', $this->member)
->setParameter('date', $this->date->format('Y-m-d'))
->setMaxResults(1)
->getQuery()
->useQueryCache(true)
->getResult()
;
Member is ManyToOne relation, startsAt and isCancelled are indexed.
Now, total rows of AppBundle:Events is 500 000. Total rows belonging to Member is only 271. Total rows of isCancelled = false is about 450 000.
So without e.isCancelled = false query runs within few milliseconds but adding that condition extends the query to 4.5 minute (localhost, mysql 8.0).
My guess is that MySQL is seeking isCancelled in all rows of Events, instead of assigned to Member only. How can I improve this query, to force Doctrine to fetch those non-cancelled events in Member result.
Related
I am trying to fetch some result from a table in a particular order.
I have a table, with few thousands questions. Each question have a category_id.
My task is, collect 100 questions from this table, in random order. Where first 30 questions will be from category_id =1, next 30 questions from category_id =2, and last 40 question from category_id=3.
My current solution is:
Question::inRandomOrder()->where('category_id',1)->limit(30)->get()
Question::inRandomOrder()->where('category_id',2)->limit(30)->get()
Question::inRandomOrder()->where('category_id',3)->limit(40)->get()
My question is, can I get same result with only one query?
With unions you could do the following.
$questions = Question::inRandomOrder()->where('category_id',1)->limit(30)
->union(Question::inRandomOrder()->where('category_id',2)->limit(30))
->union(Question::inRandomOrder()->where('category_id',3)->limit(40))
->get();
Technically a single query, but I'd rather have 3 queries.
you can use union
$q1 = Question::inRandomOrder()->where('category_id', 1)->limit(30);
$q2 = Question::inRandomOrder()->where('category_id', 2)->limit(30);
$value = Question::inRandomOrder()->where('category_id', 3)->limit(40)->union($q1)->union($q2)->get();
I have a builder query that pulls info depending on info entered. The main tables are hotels and rates. The user can sort the results with price high to low and price low to high. I need to put all results that return 0 ( some rate date ranges have place holders with 0 as the price ) or hotels that dont have rates at the end of the results no matter what. How can I handle making those both show up at the end regardless of the sorting. I have the below as where I am now but this doesnt handle things if it doesnt exist only if its 0 but it also doesnt handle putting the results at the end.
One more thing, Im using pagination or else I would just do a foreach and reorder them.
$builder->select('hotels.id','hotels.main_image','hotels.sleeps','hotels.bathrooms','hotels.destination_id','hotels.hotel_name','hotels.slug','hotels.summary','hotels.sleeps','hotels.bedrooms',DB::raw('CASE WHEN max(rates.nightly_rate) = 0 THEN NULL ELSE max(rates.nightly_rate) END AS price_high'),DB::raw('CASE WHEN min(rates.nightly_rate) = 0 THEN NULL ELSE min(rates.nightly_rate) END AS price_low')
->leftJoin('rates', 'hotels.id', '=', 'rates.hotels_id');
It is possible to enforce records where nightly_rate is 0 or null to be ordered last in order by clause.
A clause like rates.nightly_rates = 0 or rates.nightly_rate IS NULL ASC will ensure that matched records where this condition is FALSE are ordered first before records where it is TRUE.
$builder
->
//...
->orderByRaw('rates.nightly_rate = 0 OR rates.nightly_rate IS NULL ASC');
I'm trying to query a large amount of data (40K records), and planning to query much larger datasets in the future. Eloquent seems to take a very long time to load this data. I wondered if there is a faster way to process this data. I'm attempting to look at the validity of my data, and hence checking to see if all fields are null.
I've used regular Eloquent calls. I don't think chunking the data is appropriate as I'm not planning on modifying the data in any way. I debated whether running a job every so often and calling the results of this job might be a better approach.
$journal = Journal::where('issn', $this->issn)->first();
$collection = $journal->outputs;
$collectionUnique = $collection->unique('doi');
$collectionDupes = $collection->diff($collectionUnique);
dd('Total Articles '.$this->getTotal(), 'Total Articles '.count($collection));
Just use Query Builders !
Why we should use Query Builders for lots of records instead of Eloquent ?!
Here is the reason :
Query Builder is so faster than Eloquent :
Comparison (Eloquent vs Query Builder ) :
To insert 1000 rows for a simple table Eloquent takes 1.2 seconds and in that case DB facades take only 800 mili seconds(ms).
Another comparison :
Eloquent ORM average response time
Joins | Average (ms)
------+-------------
1 | 162,2
3 | 1002,7
4 | 1540,0
Result of select operation average response time for Eloquent ORM
Raw SQL average response time
Joins | Average (ms)
------+-------------
1 | 116,4
3 | 130,6
4 | 155,2
Result of select operation average response time for Raw SQL
For more information : Laravel Eloquent vs Query Builder
Edited :
Your code should be :
$journal = DB::table('journals')->where('issn', $this->issn)->first();
And Then For using Collection ( Simple way ) :
$journal = Collection::make($journal); //For use Collection Methods
$collection = $journal->get("outputs");//Changed
$collectionUnique = $collection->unique('doi');
$collectionDupes = $collection->diff($collectionUnique);
dd('Total Articles '.$this->getTotal(), 'Total Articles '.count($collection));
Best Performance :
Use queries and Query Builder instead of collections . Because operations in SQL often is faster .
Please compare time for your last code and this code and please let me know in comments :)
Why are the two statement below behaving differentlY? The first returns 3 the second returns 1 for $progress. I thought that the first aggregates on the DB and the second on the server. So they should still return the same value.
$progress = $this->user->userActivities()->select('date')
->groupBy('date')
->get()->count();
$progress = $this->user->userActivities()->select('date')
->groupBy('date')
->count();
->get()->count() will load Eloquent model objects into memory and then will count those.
->count() will use DB aggregate function, so it will definitely be more efficient:
select count(*) as aggregate ...
It's quite late answer but I had faced the same issue. These two examples are returning two different counts because of groupBy you have used.
$progress = $this->user->userActivities()->select('date')
->groupBy('date')
->get()->count();
This takes the rows first, then count the row numbers and returns the count.
$progress = $this->user->userActivities()->select('date')
->groupBy('date')
->count();
Whereas this counts the rows by its group in the DB and returns the first group's row record count only from the list.
the solution I used instead of get()->count() for my case is like
$progress = $this->user->userActivities()
->distinct('date')
->count('date');
Use ->get()->count() is wrong. I used ->get()->count() in my web app and when my database records have more than 80000 records I get error 500 in my app.
The first counts the number of returned records, the second counts the records and returns the number. If you are using a paging of 10, for example, then the first will yield 10 and the second the actual number if the number of mathcing elements is greater than 10.
$count = Model::all()->count();
return view('home',compact('count'));
this will function for laravel 6;
Both of them return the same result. But as said for preventing to crash or decrease the speed of response it's better to use count() instead of get()->count()
When I use the following Linq query in LinqPad I get 25 results returned:
var result = (from l in LandlordPreferences
where l.Name == "Wants Student" && l.IsSelected == true
join t in Tenants on l.IsSelected equals t.IsStudent
select new { Tenant = t});
result.Dump();
When I add .Distinct() to the end I only get 5 results returned, so, I'm guessing I'm getting 5 instances of each result when the above is used.
I'm new to Linq, so I'm wondering if this is because of a poorly built query? Or is this the way Linq always behaves? Surely not - if I returned 500 rows with .Distinct(), does that mean without it there's 2,500 returned? Would this compromise performance?
It's a poorly built query.
You are joining LandlordPreferences with Tenants on a boolean value instead of a foreign key.
So, most likely, you have 5 selected land lords and 5 tenants that are students. Each student will be returned for each land lord: 5 x 5 = 25. This is a cartesian product and has nothing to do with LINQ. A similar query in SQL would behave the same.
If you would add the land lord to your result (select new { Tenant = t, Landlord = l }), you would see that no two results are actually the same.
If you can't fix the query somehow, Distinct is your only option.