Eloquent - Detect multiple column duplicate data - laravel

I have a table that has 3 column id, sub_id, name. That is a pretty big table and there are some duplicates.
What is the best way to detect the duplicates so that I can remove them?
I tried this but it returns everything (I guess thinking ids are making them non-unique)
$collection = \App\MyModel::all();
$colUnique = $collection->unique(['name', 'sub_id']);
$dupes = $collection->diff($colUnique);
I want to get the models that has same name and sub_id.
id sub_id name
1 2 John
2 2 John <- duplicate
3 2 Robin <- unique

My best bet would be DB::Query.
Step 1: Fetch data by group by
$uniqueData = DB::table('TABLE_NAME')
->groupBy(['sub_id', 'name'])
->select('id')
->toArray();
Step 2: Delete duplicate record.
$noOfDeletedRecords = DB::table('TABLE_NAME')
->whereNotIn($uniqueData)
->delete();
Benefits:
1. Only 2 Queries
2. Better performance over collection.

You can utilize Collection.groupBy method.
$collection = \App\MyModel::all();
$collection
// Group models by sub_id and name
->groupBy(function ($item) { return $item->sub_id.'_'.$item->name; })
// Filter to remove non-duplicates
->filter(function ($arr) { return $arr->count()>1; })
// Process duplicates groups
->each(function ($arr) {
$arr
// Sort by id (so first item will be original)
->sortBy('id')
// Remove first (original) item from dupes collection
->splice(1)
// Remove duplicated models from DB
->each(function ($model) {
$model->delete();
});
})

Related

get all rows in single query with multiple id

I have a asset_request table with fields id and request_id.
I want to select multiple rows with specific ids.
$ids = $request->ids // 5,6
I want to select only rows with ids of 5 and 6 in request table
$ids = $request->ids;
$asset_request = asset_request::whereIn('id',array($ids))->first(); //gets only 6th row.
I need to get all rows matching the given ids.
To clarify after a chat discussion with the Op:
The Op was passing back a string request, therefore, the Op needed to change the following:
$id = $request->id;
$ids = str_split(str_replace(',', '', $id));
$asset_request = asset_request::whereIn('id', $ids)->get();
First you are calling the first method which will return only the first row matched.
You need to call get method to get all rows matched.
Secondly if you are sending ids as a comma separated string you need to convert it to array using explode.
$ids = $request->ids;
$asset_requst = asset_request::whereIn('id', explode(",", $ids))->get();
DB::table('asset_request')
->whereIn('id', (array) $request->ids)
->get();
or
TableModel::whereIn('id', (array) $request->ids)->get();

How to order by value when i get results in array?

I get all tags and its look like this :
"testeng" => 0
"testeng1" => 5
So what i want to do is order this by value so that first one is this with value 5. Any suggestion how can i do this?
$tags = ATags::with('articles')->whereHas('language',function($query) use($current_language_id){
$query->where('id','=',$current_language_id);
})->get();
$count_tag = [];
foreach($tags as $tag){
$count_tag[$tag->name] = $tag->articles->count();
}
You can use arsort() as:
arsort($count_tag)
arsort — Sort an array in reverse order and maintain index association
OR
You can use withCount to sort the result directly in query as:
$tags = ATags::with('articles')
->whereHas('language',function($query) use($current_language_id){
$query->where('id','=',$current_language_id);
})
->withCount('articles')
->orderBy('articles_count', 'desc')
->take(5)
->get();
From the docs
If you want to count the number of results from a relationship without
actually loading them you may use the withCount method, which will
place a {relation}_count column on your resulting models.
You can use Collection's sortBy or sortByDesc & count methods to sort your results like this:
$tags = ATags::with('articles')->whereHas('language',function($query) use($current_language_id) {
$query->where('id','=',$current_language_id);
})
->get()
->sortByDesc(function($tag) { // <---- sorting it via article's count
return $tag->articles->count();
})
->take(5); // <----- fetch largest 5 from collection
You can pass a closure method to sortBy or sortByDesc collection's methods to manipulate your results.

Using whereIn method to return multiple results for the same array value but at different index

$pidArray contains product ID's, some of those product ID's can be the same. I.E: 34 34 56 77 99 34. As is, it appears the whereIn method does not return results for a productId it has already found in $pidArray, even if it has a different index.
$productDataForOrder = Product::whereIn('id', $pidArray)->get(['id','price']);
$totalAmount = $productDataForOrder->sum('price');
$productDataForOrder now contains product data, but only for unique ProductID's in $pidarray. So when sum function is run, the sum is wrong as it does not take into account the price for multiple instances of the same productID.
The following code also does not return objects for every product ID in the array which are the same. So if $pidArray contains three identical product ID's, the query will only return a collection with one object, instead of three.
$query = Product::select();
foreach ($pidArray as $id)
{
$query->orWhere('id', '=', $id);
}
$productDataForOrder = $query->get(['id','price']);
$totalAmount = $productDataForOrder->sum('price');
You're not going to be able to get duplicate data the way that you're trying. SQL is returning the rows that match your where clause. It is not going to return duplicate rows just because your where clause has duplicate ids.
It may help to think of it this way:
select * from products where id in (1, 1)
is the same as
select * from products where (id = 1) or (id = 1)
There is only one record in the table that satisfies the condition, so that is all you're going to get.
You're going to have to do some extra processing in PHP to get your price. You can do something like:
// First, get the prices. Then, loop over the ids and total up the
// prices for each id.
// lists returns a Collection of key => value pairs.
// First parameter (price) is the value.
// Second parameter (id) is the key.
$prices = Product::whereIn('id', $pidArray)->lists('price', 'id');
// I used array_walk, but you could use a plain foreach instead.
// Or, if $pidArray is actually a Collection, you could use
// $pidArray->each(function ...)
$total = 0;
array_walk($pidArray, function($value) use (&$total, $prices) {
$total += $prices->get($value, 0);
});
echo $total;
The whereIn method only limits the results to the values in the given array. From the docs:
The whereIn method verifies that a given column's value is contained within the given array
Id make a query variable and loop through the array adding to the query variable in each pass. Something like this:
$query = Product::select();
foreach ($pidArray as $id)
{
$query->where('id', '=', $id);
}
$query->get(['id','price']);
Here is a code that would work for your use case expanding on #patricus
You first fetch an array of key as id and value as price from the products table
$prices = Product::whereIn('id', $pidArray)->lists('price', 'id');
$totalPrice = collect([$pidArray])->reduce(function($result, $id) use ($prices) {
return $result += $prices[$id];
}, 0);

laravel 4 relations - how display a top-5 ranking of records voted by users

I am creating a system of newsfeed, and as you can easily guess, it is beyond my skills.
Please be kind to put me on the right track or provide something I can go on with.
I have several hundred events (model name is Event1, table 'events')
I also have a pivot table in which users can assign any event's importance (values 0,1,2,3)
The relevant columns of the pivot table user_attitudes (Model Userattitude) are
id, item_type, item_id, importance, attitude, creator_id
An example three record are:
456 - event - 678 - 2 - 4
457 - event - 690 - 3 - 15
458 - event - 690 - 1 - 4
459 - participant - 45 - 1 - 4
Plain English: Total aggregated importance of the event #690 is '4', while the event #678 is '2'.
Therefore in my ranking the event #690 should be listed as first.
Just to see the bigger pic: the user #4 also rated participant # 45 as importance = 1.
The table services many models - the above example include two - just to give a better image of what I have.
WHAT I NEED:
I wish to print a ranking of top 5 events (and later other models). I wish to be able to use two methods of calculating the total score:
by counting the actual value (0,1,2,3)
by counting any value above 0 as 1 point.
I want to generate views which filter events by this criteria:
at least one user set the importance to '0' (I need it to flag an event as untrustworthy)
events which has not been rated yet
reverse of the above - events which are rated by at least one user
events listed by number of users who assigned any importance to it
This is easy, but still I have no idea how to make it happen. The same filters as the above #2, but related to a particular user decisions:
list 5 or 10 events (random or newest) which has not yet been rated by the user
maybe something like this would be an answer:
$q->where('creator_id', '=', Auth::user()->id);
Relevant code:
As I don't really grasp the merged relations, I might fail to show everything needed to provide help - ask for more code in comments.
Models:
Event1 (table 'events'):
public function importances()
{
return $this->morphMany('Userattitude', 'item');
}
public function user_importance($user)
{
return $this->morphMany('Userattitude', 'item')->where('creator_id', ($user ? $user->id : NULL))->first();
}
User: (table 'users' - standard user table)
public function importances()
{
return $this->hasMany('Userattitude', 'creator_id');
}
In model Userattitude (different from User, table name 'user_attitudes')
public function events()
{
return $this->morphTo('item')->where('item_type', 'event');
}
public function event()
{
return $this->belongsTo ('Event1', 'item_id');
}
PROBLEMS IN REPLY TO #lucas answer:
PROBLEM 1.
table name 'items' keeps me confused as in my project 'items' are events (model Event1), the participants (model Entity) and other objects.
Can we stick to my naming until I get hold of the knowledge you are providing?
it also contains column named attitudes, which is used for blacklisting particular items.
For instance, an item of type 'entity' (possible participant of multiple events) can be voted by user two-wise:
- by importance set by an user (we are doing this now, values available to use are 0,1,2,3)
- by attitude of an user toward (possible value (-1, 0, 1)
Such solution allows me to compute karma of each item. For instance -1 x 3 = -3 (worst possible karma value), while 1 x 2 = 2 (medium positive karma).
In consequence I am unable to use queries with the users method. It is still too confusing to me, sorry. We diverted too far from my original mental image.
Consider this query:
$events = Event1::has('users', '<', 1)->get();
If in Event1 I declare
public function users()
{
return $this->morphToMany('User', 'item', null, null, 'creator_id');
}
Note: User is the standard users table, where username, password and email are stored
I get this error:
[2014-12-28 05:02:48] production.ERROR: FATAL DATABASE ERROR: 500 = SQLSTATE[42S02]: Base table or view not found: 1146 Table 'niepoz_niepozwalam.items' doesn't exist (SQL: select * from `Events` where (select count(*) from `users` inner join `items` on `users`.`id` = `items`.`creator_id` where `items`.`item_id` = `Events`.`id` and `items`.`item_type` = Event1) >= 1) [] []
if I change the method definition to
public function users()
{
return $this->morphToMany('Userattitude', 'item', null, null, 'creator_id');
}
Note: Userattitude is model (table name is 'user_attitudes') where i store user judgments. This table contains columns 'importance' and 'attitude'.
I get the same error.
If I change the method to
public function users()
{
return $this->morphToMany('User', 'Userattitudes', null, null, 'creator_id');
}
I get this:
[2014-12-28 05:08:28] production.ERROR: FATAL DATABASE ERROR: 500 = SQLSTATE[42S22]: Column not found: 1054 Unknown column 'user_attitudes.Userattitudes_id' in 'where clause' (SQL: select * from Events where (select count(*) from users inner join user_attitudes on users.id = user_attitudes.creator_id where user_attitudes.Userattitudes_id = Events.id and user_attitudes.Userattitudes_type = Event1) >= 1) [] []
Possible solution:
the 'user_attitudes' table alias with name 'items'.
I could create a view with the required name.
I did it, but now the query produces no results.
PROBLEM 2
should I rename creator_id into user_id - or keep both columns and keep duplicated information in them? The creator_id follows conventions and I use it to create records... how to resolve this dillema?
PROBLEM 3.
As far as I understand, if I want to get a USER-RELATED list of top-5 events,
I need to ad another line to the code, which narrows search scope to records created by a particular logged in user:
Auth::user()->id)
The code would look like this:
All with importance 0
$events = Event1::whereHas('users', function($q){
$q->where('importance', 0);
$q->where('creator_id', '=', Auth::user()->id);
})->get();
right?
PROBLEM 5:
Ok, I am now able to output a query like these:
$rank_entities = Entity::leftJoin('user_attitudes', function($q){
$q->on('entity_id', '=', 'entities.id');
$q->where('item_type', '=', 'entity');
})
->selectRaw('entities.*, SUM(user_attitudes.importance) AS importance')
->groupBy('entities.id')
->orderBy('importance', 'desc')
->take(6)
->get();
and in the foreach loop I can display the total importance count with this code:
{{$e->importance or '-'}}
But How I could display count of an alternative query: SUM of values from another column, named attitude, which can be computed in this SEPARATE query:
In other words, in my #foreach loop I need to display both $e->importance and a computed SUM(user_attitudes.attitude) AS karma, which for now can be received with this query:
$rank_entities = Entity::leftJoin('userattitudes', function($q){
$q->on('entity_id', '=', 'entities.id');
$q->where('item_type', '=', 'entity');
})
->selectRaw('entities.*, SUM(userattitudes.karma) AS karma')
->groupBy('entities.id')
->orderBy('karma', 'desc')
->take(5)
->get();
My solution would be to create some extra columns in the 'entities' table:
- karma_negative
- karma_positive
to store/update total amount of votes each time someone is voting.
First, let's talk about the setup. I wasn't entirely sure how and if your's works but I created this on my testing instance and it worked, so I recommend you change yours accordingly:
Database
events
That's a simple one (and you probably already have it like this
id (primary key)
name (or something like that)
etc
users
I'm not sure if in your example that is Userattitude but I don't think so...
id (primary key)
email (?)
etc
items
This is the important one. The pivot table. The name can be different but to keep it simple and follow conventions it should be the plural of the polymorphic relation (in your case item => items)
id (actually not even necessary, but I left it in there)
item_type
item_id
importance
creator_id (consider changing that to user_id. This would simplify the relationship declaration)
Models
I think you have to read the docs again. You had several weird relations declared. Here's how I did it:
Event1
By default Laravel uses the classname (get_class($object)) as value for the ..._type column in the database. To change that you need to define $morphClass in your models.
class Event1 extends Eloquent {
protected $table = 'events';
protected $morphClass = 'event';
public function users()
{
return $this->morphToMany('User', 'item', null, null, 'creator_id');
}
}
User
class User extends Eloquent implements UserInterface, RemindableInterface {
// ... default laravel stuff ...
public function events(){
return $this->morphedByMany('Event1', 'item', null, null, 'creator_id');
}
}
Queries
Alright now we can get started. First one additional information. I used Eloquent relations whenever possible. In all the queries a join() is made it would be slower to use relations because certain things (like counting or calculating the maximum) would have to be done in PHP after the query. And MySQL does a pretty good job (also performance wise) at those things.
Top 5 by total value
$events = Event1::leftJoin('items', function($q){
$q->on('item_id', '=', 'events.id');
$q->where('item_type', '=', 'event');
})
->selectRaw('events.*, SUM(items.importance) AS importance')
->groupBy('events.id')
->orderBy('importance', 'desc')
->take(5)
->get();
Top 5 by number of votes over 0
$events = Event1::leftJoin('items', function($q){
$q->on('item_id', '=', 'events.id');
$q->where('item_type', '=', 'event');
$q->where('importance', '>', 0);
})
->selectRaw('events.*, COUNT(items.id) AS importance')
->groupBy('events.id')
->orderBy('importance', 'desc')
->take(5)
->get();
All with importance 0
$events = Event1::whereHas('users', function($q){
$q->where('importance', 0);
})->get();
All without any votes
$events = Event1::has('users', '<', 1)->get();
All with 1+ votes
$events = Event1::has('users')->get();
All ordered by number of votes
$events = Event1::leftJoin('items', function($q){
$q->on('item_id', '=', 'events.id');
$q->where('item_type', '=', 'event');
})
->selectRaw('events.*, COUNT(items.id) AS count')
->groupBy('events.id')
->orderBy('count', 'desc')
->get();
Newest 5 without votes
If you are using Eloquents timestamps created_at:
$events = Event1::has('users', '<', 1)->latest()->take(5)->get();
If you're not (order by greatest id):
$events = Event1::has('users', '<', 1)->latest('id')->take(5)->get();
Random 5 without votes
$events = Event1::has('users', '<', 1)->orderByRaw('RAND()')->take(5)->get();
I did not add any explanations to the queries on purpose. If you want to know more about something specific or need help, please write a comment
PROBLEM 4: SOLVED! (credit to #lukasgeiter)
If you wish to display a ranking of items and limit the results to a particular tag defined in a pivot table, this is the solution:
$events = Event1 (table name = 'events')
For example, the tag would be war: defined in table
eventtags
Event nature are defined as
id = '1' is name = 'wars'
id = '2' is name = 'conflicts'
pivot table, which assigns multiple tags:
event_eventtags they are defined as id = '4'
Example records for event_eventtags:
id - event_id - eventtag_id
1 - 45 - 1
2 - 45 - 2
Plain English: the Event1 #45 is tagged as war(#1) and conflict(#2)
Now in order to print a list of 10 wars you should define your query in this way:
$events= Entity::join('event_eventtags', function($q){
$q->on('entity_id', '=', 'entities.id');
$q->where('entitycapacitytypes_id', '=', 1);
})->leftJoin('user_attitudes', function($q){
$q->on('item_id', '=', 'entities.id');
$q->where('item_type', '=', 'entity');
})
->selectRaw('entities.*, SUM(user_attitudes.importance) AS importance')
->groupBy('entities.id')
->orderBy('importance', 'desc')
->take(10)
->get();
The user_attitudes is part of voting system described in the original question. You can remove it and sort events by another method.

Codeigniter - How to get values as array to use in next select with where_not_in

I have three tables; user, car and user_x_car. user_x_car holds users who own car; user_id and car_id are stored. I want to get users who don't own a car as follows:
$car_owner = $this->db->select()->from('user_x_car')->get()->result();
for ($i = 0; $i < count($car_owners); $i++)
$car_owner_id[$i] = $car_owner[$i]->user_id;
$non_car_owner = $this->db->select()->from('user')->where_not_in('id', $car_owner_id)->get()->result();
I get what I want, however, is there any way to bypass the for loop in the middle which creates and array of id's selected in the first select. Is there any way to get array of selected user_ids directly?
you can do it by two queries like
first one get all ids from user_x_car table
$temp1=array();
$temp=$this->db->distinct()->select('user_id')->get('user_x_car')->result_array();
then from user table fetch those users who have no cars
foreach($temp as $each)
{
array_push($temp1,$each['user_id']);
}
$rs=$this->db->where_not_in('id',$temp1)->get('user');
if($rs->num_rows()>0)
{
$data=$rs->result_array();
print_r($data);die;
}
$data will print all users who have no car. Please let me know if you face any problem.
function get_unread_notifications_ids()
{
//get unread notifications ids
$this->db->select('GROUP_CONCAT(fknotification_id) as alll');
$this->db->from("te_notification_status_tbl");
$this->db->where('read_status',0);
$ids=$this->db->get()->row();
return $idss=str_replace(",","','",$ids->alll);
}
and second function like this:
function get_unviewed_photos_events(){
$idss = $this->get_unread_notifications_ids();
$this->db->select('img.*',False);
$this->db->from("te_notifications_tbl notif");
$this->db->join('te_images_tbl img','img.id=notif.reference_id','LEFT OUTER');
$this->db->where("notif.id IN('".$idss."')");
$rslt = $this->db->get()->result_array();
return $rslt;
}
Query
$non_car_owner = $this->db->query('SELECT user.*
FROM user LEFT JOIN user_x_car ON user_x_car.id=user.id
WHERE table2.id IS NULL')->result();
Here users who are not on the table user_x_car
foreach($non_car_owner as $user){
echo $user->user_id;
}

Resources