I have a complex SELECT with several time consuming calculated fields. I only want the first 100 rows, however, I also need the total rows found. This means executing the query once with LIMIT and once without.
I would like to be able to build two SELECT's simultaneously with CodeIgniter's QueryBuilder and eliminate the time consuming calculated fields on the second query.
I would like to know if there is a good way to do this with QB or should I just build the SELECT without QB.
UPDATE: Apparently I have confused people. I need to do something like this:
To get the records to display:
SELECT id, [ several really time intensive calculated fields ]
FROM table
WHERE [some complicated criteria]
LIMIT 100;
To get the total records I would like to re-use the built query or build a 2nd one simultaneously but without the limit and with only the id field:
SELECT id
FROM table
WHERE [some complicated criteria];
Another option; I have learned that if I put the time intensive calculated fields in a sub SELECT, Postgres will not execute it until the row is actually retrieved.
For now I am manually building two queries but I'd like to do it the correct QueryBuilder way.
There is no way to run multiple query in one statement in codeigniter. But you may do as following
$query1 = $this->db->query('SELECT * FROM table_name LIMIT 10');
$query2 = $this->db->query('SELECT * FROM table_name');
But what I do is make a method and call based on parameter like
public function get_rows($table, $num_rows = FALSE){
$this->db->select('*');
$this->db->from($table);
if($num_rows === FALSE){
$this->db->limit(10);
}
$query = $this->db->get();
if($num_rows){
return $query->num_rows();
}
return $query->result_array();
}
Now you may call it as your requirements
//call for only count rows (without limit)
$this->model_name->get_rows('table_name', TRUE);
//call with limit for result rows
$this->model_name->get_rows('table_name');
with count you can do this with query builder I think
look at this code, you may want something like that
if($limit != null){
$this->db->limit($limit, $offset);
}
if($order_by != null){
$this->db->order_by($order_by, $sort);
}
$query = $this->db->get('courses');
$courses['rows'] = $query->result();
$courses['num_rows'] = $this->db->count_all('courses');
return $courses;
Related
Suppose I have a user table with columns id, name, age
With a normal get query User::get(), I get all results with those column's value.
But instead of just getting the id, name, age columns, is it possible if I add another column, perhaps a column with an alias of future_age with a value of age + 1.
The above result could be achieved in SQL like so:
SELECT res.*, (SELECT sum(res.age+1) from users where id = res.id) as future_age from users as res
I've thought about looping through the result and creating new key. but I think this would make the query execution time slow when the data is lengthy.
Is it possible to create the future_age key/column (I don't know the term hehe) directly on the query?
Currently, this is my query:
$user = User::get();
$new_res = []
if($user->count() > 0) {
foreach ($user as $u) {
$u['future_age'] = $u->age + 1
$new_res[] = $u;
}
}
The query works tho, but I don't think this is good if I have a large set of data.
The code below is what I have to get all the duplicated products (by title) and group them together. It works perfectly fine. However, I so many records in my Products table and getting all of them causes a performance issue. Is there a way this could be optimised to avoid getting all records and group them in one query? Thank you.
$products = Product::all();
$groupsOfProducts = $products->groupBy('title');
$duplicatedProductsGrouped = [];
foreach($groupsOfProducts as $productGroup) {
$productIsDuplicated = $productGroup->count() > 1;
if($productIsDuplicated) {
$duplicatedProductsGrouped[] = $productGroup;
}
}
var_dump($duplicatedProductsGrouped);
You can use having in the group by:
Product::groupBy('title')->having(DB::raw('count(*)'), ">", "1")->select('title')->get()
And you will get the titles of the duplicates, then you can query the database with those titles
EDIT:
Please also try and see if this is faster
Product::getQuery()->whereIn('title', array_column( DB::select('select title from products group by title having count(*) > 1'), 'title'))->get();
with this line you will get ONLY the products that has a duplicate title, and so your Collection groupby should be faster to aggregate the records by the title
Let your database do the work. When you call Product::all(), you're getting every single record, then making PHP do the rest. Change your query to something like the following:
Product::selectRaw("title, COUNT(*) AS count")->groupBy("title")->get();
The result will be a Collection of Product instances with a title and count attribute, which you can access and determine duplicated ones:
$products = Product::selectRaw("title, COUNT(*) AS count")->groupBy("title")->get();
$duplicatedProducts = collect([]);
foreach($products AS $product){
if($product->count > 1){
$duplicatedProducts->push($product);
}
}
dd($duplicatedProducts);
I am using the CodeIgniter database class to generate results and then use the pagination class to navigation through them. I have a model that retrieves member results from the table. I would like to calculate the total number of rows from the query so I can pass this to the pagination class. The count_all db helper function won't suffice because that doesn't take in account any "where" or "join" clauses that I include.
If this is my query:
$this->db->select('m.user_id AS id, m.email_address, m.display_name, m.status, UNIX_TIMESTAMP(m.join_date) AS join_date,
l.listing_id, COUNT(l.member_id) AS total_listings,
g.group_id AS group_id, g.title AS group_title')
->from('users AS m')
->join('listings AS l', 'm.user_id = l.member_id', 'left')
->join('groups AS g', 'm.group_id = g.group_id', 'left')
->group_by('m.user_id');
How can I continue to use this query if I wanted to do something like this:
if($query_total = $this->db->get()){
$this->total_results = $query_total->num_rows();
}
$this->db->limit($limit, $offset);
if($query_members = $this->db->get()){
return $query_members->result_array();
}
Update: In other words, I want to run a query with the get() method without clearing it from the query builder when it's done running so I can use the top part of the query later.
You can get it by using Active Record Caching (the last category on that page).
$this->db->start_cache(); // start caching
$this->db->select('m.user_id AS id, m.email_address, m.display_name, m.status, UNIX_TIMESTAMP(m.join_date) AS join_date,
l.listing_id, COUNT(l.member_id) AS total_listings,
g.group_id AS group_id, g.title AS group_title')
->from('users AS m')
->join('listings AS l', 'm.user_id = l.member_id', 'left')
->join('groups AS g', 'm.group_id = g.group_id', 'left')
->group_by('m.user_id');
$this->db->stop_cache(); // stop caching
Then you can use that query as much as you want.
if($query_total = $this->db->get()){
$this->total_results = $query_total->num_rows();
}
$this->db->limit($limit, $offset);
if($query_members = $this->db->get()){
return $query_members->result_array();
}
You can also flush the cache when you don't want that cached query anymore.
$this->db->flush_cache();
Hope it will be useful for you.
You are not passing any table name in $this->db->get() function.
Thank you
It's possible to make one query to get total, sold & unsold in laravel eloquent?
$total_apple = Item::whereName('Apple')->count();
$sold_apple = Item::whereName('Apple')->whereStatus(2)->count();
$unsold_apple = Item::whereName('Apple')->whereStatus(1)->count();
Yes you can totally do that. You can use filter method on collection object returned by your Eloquent query.
$apples = Item::whereName('Apple')->get();
$soldApples = $apples->filter(function ($apple){
return $apple->status == 2;
});
$unsoldApples = $apples->filter(function ($apple){
return $apple->status == 1;
});
$soldApples and $unsoldApples contains the object of the items. You can then just use count($soldApples) and count($unsoldApples) to get their count.
filter method is against the collection object so there is no sql overhead.
There is no need run multiple queries or even fetch the entire results and use collection methods to loop through. Just use raw queries.
$apples = Item::whereName('Apple')
->selectRaw('COUNT(*) as total_apples,
SUM(status=2) as sold_apples,
SUM(status=1) as unsold_apples')
->first();
echo $apples->total_apples; // Outputs total apples
echo $apples->unsold_apples; // Outputs the unsold apples
echo $apples->sold_apples; // Outputs the sold apples
Since you are only doing simple counts though, you can use the query builder as well.
I would get all the items in one collection, then run the where statement on that collection. This should trigger a single Query.
$apples = Item::whereName('Apple')->get(); // This goes against SQL
$total_apple = $apples->count(); //This runs on the Collection object not SQL
$sold_apple = $apples->whereStatus(2)->count();
$unsold_apple = $apples->whereStatus(1)->count();
I'm using codeigniter and the pagination class. This is such a basic question, but I need to make sure I'm not missing something. In order to get the config items necessary to paginate results getting them from a MySQL database it's basically necessary to run the query twice is that right?
In other words, you have to run the query to determine the total number of records before you can paginate. So I'm doing it like:
Do this query to get number of results
$this->db->where('something', $something);
$query = $this->db->get('the_table_name');
$num_rows = $query->num_rows();
Then I'll have to do it again to get the results with the limit and offset. Something like:
$this->db->where('something', $something);
$this->db->limit($limit, $offset);
$query = $this->db->get('the_table_name');
if($query->num_rows()){
foreach($query->result_array() as $row){
## get the results here
}
}
I just wonder if I'm actually doing this right in that the query always needs to be run twice? The queries I'm using are much more complex than what is shown above.
Unfortunately, in order to paginate you must know how many elements you are breaking up into pages.
You could always cache the result for the total number of elements if it is too computationally expensive.
Yeah, you have to run two queries, but $this->db->count_all('table_name'); is one & line much cleaner.
Pagination requires reading a record set twice:
Once to read the whole set so that it can count the total number records
Then to read a window of records to display
Here's an example I used for a project. The 'banner' table has a list of banners, which I want to show on a paginated screen:
Using a public class property to store the total records (public $total_records)
Using a private function to build the query (that is common for both activities). The parameter ($isCount) we pass to this function reduces the amount of data the query generate, because for the row count we only need one field but when we read the data window we need all required fields.
The get_list() function first calls the database to find the total and stores it in $total_records and then reads a data window to return to the caller.
Remember we cannot access $total_records without first calling the get_list() method !
class Banner_model extends CI_Model {
public $total_records; //holds total records for get_list()
public function get_list($count = 10, $start = 0) {
$this->build_query();
$query = $this->db->get();
$result = $query->result();
$this->total_records = count($result); //store the count
$this->build_query();
$this->db->limit($count, $start);
$query = $this->db->get();
$result = $query->result();
return $result;
}
private function build_query($isCount = FALSE) {
$this->db->select('*, b.id as banner_id, b.status as banner_status');
if ($isCount) {
$this->db->select('b.id');
}
$this->db->from('banner b');
$this->db->join('company c', 'c.id = b.company_id');
$this->db->order_by("b.id", "desc"); //latest ones first
}
And now from the controller we call:
$data['banner_list'] = $this->banner_model->get_list();
$config['total_rows'] = $this->banner_model->total_records;
Things get complicated when you start using JOINs, like in my example where you want to show banners from a particular company! You may read my blog post on this issue further:
http://www.azmeer.info/pagination-hitting-the-database-twise/