Calculated fields in database - Laravel - laravel

What is the best practice to store calculated fields in database.
For example, lets say a table has fields height, weight, bmi
A user enters height weight values and bmi field is automatically filled. How to achieve this with a form.
Formula for bmi
$bmi = weight / (height * height)
Tried the following
Profile Model
protected $table = 'profiles';
protected $fillable = ['user_id', 'weight', 'height', 'dob', 'age', 'bmi'];
public function user(){
return $this->belongsTo(User::class, 'user_id');
}
protected static function boot() {
parent::boot();
static::saving(function($model){
$model->bmi = $model->weight / ($model->height * $model->height);
$model->age = (date('Y') - date('Y',strtotime($model->dob)));
});
}
Profile Controller
public function store(Request $request)
{
$profile = new Profile();
$profile->weight = $request->get('weight');
$profile->height = $request->get('height');
$profile->dob = $request->get('dob');
$profile->age;
$profile->bmi;
$profile->save();
return back()->with('success', 'Your profile has been updated.');
}
But im receiving an error
Illuminate \ Database \ QueryException (42S22)
SQLSTATE[42S22]: Column not found: 1054 Unknown column 'weight' in 'field list' (SQL: insert into `users` (`weight`, `height`, `dob`, `bmi`, `age`, `created_by`, `updated_by`, `updated_at`, `created_at`) values (60, 175, 1988-04-03, 0.0019591836734694, 30, 2, 2, 2018-03-08 20:06:02, 2018-03-08 20:06:02))

You could do this in the boot method of the model:
protected static function boot() {
parent::boot();
static::saving(function($model){
$model->bmi = $model->weight / ($model->height * $model->height);
});
}

What is the best practice to store calculated fields in database.
In general, don't. It's redundant - your database already has all the information needed to compute it, so why store redundant data?
Instead, put an accessor on the model:
public function getBmiAttribute() {
return ($this->weight / ($this->height * $this->height));
}
You can then do $this->bmi to get the computed value.
If you must have it in the database, use an Eloquent event. You'd probably want to hook into the saving event.

As of Laravel 5.3 & MySQL 5.7, you can use Column Modifiers virtualAs and storedAs to create a VIRTUAL (evaluated when rows are read) or STORED (evaluated and stored when rows are inserted or updated) generated column for MySQL.
I've made use of virtualAs in the example below...
Schema::create('results', function (Blueprint $table) {
$table->bigIncrements('id');
...
$table->time('predicted_time')->nullable()->default(NULL);
$table->time('handicap')->nullable()->default(NULL);
$table->time('finish_time')->nullable()->default(NULL);
$table->time('actual_time')->virtualAs('TIMEDIFF(finish_time, handicap)');
$table->time('offset')->virtualAs('TIMEDIFF(actual_time, predicted_time)');
...
});

What is the best practice to store calculated fields in database.
It depends on your use case. If you're using a relational database, and your use case does not involve big data (in terms of volume, variety or velocity), the best practice is to not store calculated fields and calculate them on the fly.
If you're using a noSQL database (such as MongoDB, which is supported by Laravel) or Hadoop, the best practice is to store calculated fields to avoid computational time.
In A Nutshell
It's a tradeoff between time complexity and space/storage complexity.
For big data / noSQL systems, store calculated fields especially if
they are computationally complex. For a relational database, calculate
them on the fly and keep your data non-redundant
Laravel Solution for RDBMS:
Use accessors like so:
public function getBmiAttribute($value)
{
return ($this->weight / ($this->height * $this->height));
}
Laravel Solution for NoSql
Use model events like so:
protected static function boot() {
parent::boot();
static::saving(function($model){
$model->bmi = $model->weight / ($model->height * $model->height);
});
}

If your DBMS supports computed columns (aka generated columns), you might consider utilizing them.
Highlights:
Computed columns don't persist the data to the database (or if they do technically they'll be managed by the DBMS), so you're not duplicating any data or making anything redundant
The calculation is then available outside of the application's code base for anything to use. For example, if there is a need to develop reporting with raw SQL queries then the calculation will be available there.
Eloquent (Laravel's default ORM) will pick up the column without you necessarily needing to define it anywhere.
You could execute code to create it during a migration. For example, I had a use case where I wanted to simplify determining if something was currently published and I did it like this:
Schema::table('article', function (Blueprint $table) {
$table->dateTime('published_at')->nullable();
$table->dateTime('unpublished_at')->nullable();
});
$sql = "ALTER TABLE article ADD is_published AS CAST(CASE WHEN GETUTCDATE() BETWEEN ISNULL(published_at, '2050-01-01') AND ISNULL(unpublished_at, '2050-01-01') THEN 1 ELSE 0 END AS BIT);";
DB::connection()->getPdo()->exec($sql);
Then after retrieving a record from the database I can determine if it's published by simply checking...
if ($article->is_published) ...
And if I want to query it from the database I can easily do...
SELECT * FROM article WHERE is_published = 1
Note, this syntax is for T-SQL since the RDBMS for my project is MS SQL Server. However, other popular DBMS have similar features.
MS SQL Server: Computed Columns
ALTER TABLE dbo.Products ADD RetailValue AS (QtyAvailable * UnitPrice * 1.35);
MySQL: Generated Columns
ALTER TABLE t1 ADD COLUMN c2 INT GENERATED ALWAYS AS (c1 + 1) STORED;
Oracle: Virtual Columns
ALTER TABLE emp2 ADD (income AS (salary + (salary*commission_pct)));
PostgreSQL doesn't seem to support it (yet), but this SO answer provides a decent workaround.

another way to make this
making a computed column in the migration
this working with all DB:-
$table->computed ('tax','price * 0.27');
but this working with MySQL DB:-
$table->integer('tax')->virtualAs('price * 0.27')->nullable();
$table->integer('discount')->storedAs('price - 100')->nullable();

Related

Autoincrement column with conditions

I would like to implement an order column (that will be used for determining hierarchy for the entities, both in backend and frontend). Is there a nice way to do this on the migration level?
So that when a new model is saved, the highest number of the order column will be found - but only for the models that share shop_id. So if I add models like this:
// First model should get order = 1
$priceExceptionA = new PriceException();
$priceExceptionA->shop_id = 1;
$priceExceptionA->amount = 100;
$priceExceptionA->save();
// Second model should get order = 2
$priceExceptionB = new PriceException();
$priceExceptionB->shop_id = 1;
$priceExceptionB->amount = 100;
$priceExceptionB->save();
I would like to find a way to implement this behavior already through the migration:
public function up()
{
Schema::create('price_exceptions', function (Blueprint $table) {
$table->id();
$table->integer('shop_id')->unsigned();
$table->integer('amount')->nullable();
// Find the highest order value for rows with the same shop_id and increment this by one
$table->integer('order')->default(////)
$table->timestamps();
$table->softDeletes();
$table->foreign('shop_id')->references('id')->on('shops')->onDelete('cascade');
});
}
I do not believe SQL can what you want to do with the default functionality.
Instead there is a simple solution using model events, that secures all models will get these order values. The creating events, is called before the model is saved to the database and therefor all models will get the order set.
public static function boot()
{
static::creating(function(PriceException $priceException)
{
$priceException->order = ($priceException->shop->priceExceptions()->max('order') ?? 0) + 1;
});
}
This can create race conditions a way to combat this is to use a lock to secure you don't get duplicate order numbering. If you work on a load balanced setup the cache should be a shared redis server.
static::creating(function(PriceException $priceException)
{
Cache::lock($priceException->shop_id)->get(function () use ($priceException) {
$priceException->order = ($priceException->shop->priceExceptions()->max('order') ?? 0) + 1;
});
});

Laravel - eager loading filtering related model on different database instance

i'm trying to implement filtering as defined here.
This works (to filter model A from controller A), however controller A/model A has a relation to model B, which is what I want to filter, as well as a 3rd relationship to model C from model B.
Model A is hosted on DB instance 1, Model B is hosted on DB instance 2 and are totally separated.
These relationships, without any filters, work fine.
Trying to mess around, I tried something like the below, which clearly does not work, however will hopefully illustrate what i'm trying to do. This filter gets applied to model A
protected function sn($sn)
{
$s= Bid::where('crm', function ($query) {
$query->where('sNumber', '=', 'SN512345');
})->get();
return $s;
}
SQLSTATE[42S22]: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Invalid column name 'crm'. (SQL: select * from [bids] where [crm] = (select * where [sNumber] = SN512345))
Bid is model A/controller A, CRM is model B which is the one I want to filter.
I thought about having numerous different functions in the model to filter, however I don't know if this was the best solution and I thought it was better to get it all out into another class.
I tried the below, which does not work as it applies the query to DB1.
$s= Bid::with('crm')->whereHas('crm', function ($query) {
$query->where('sNumber', '=', 'SN512345');
})->get();
[SQL Server]Invalid object name 'Opportunity'. (SQL: select * from [bids] where exists (select * from [Opportunity] where [bids].[crm_ref] = [Opportunity].[sNumber] and [sNumber] = SN512345))
Is there a way to implement this in some coherent and reusable way? I was thinking something along the lines of load Bid::, load CRM:: with applied filters, then append CRM:: to Bid:: somehow in the normal way that Eloquent would do this.
Thanks.
EDIT:
I am using the following filter in BidFilter.php
protected function sn($sn)
{
$users = DB::connection('sqlsrv_crm')->table('OpportunityBase')->select('*')->where('new_SalesNumber', '=', $sn)->get();
return $users;
}
And this filters the result set, as I can see in the debug bar:
debug bar queries
However this is also loading the normal unfiltered eager loaded CRM relationship. How can I switch to the filtered CRM results instead of the default unfiltered?
BidController index method:
public function index(BidFilter $filters)
{
$bids = $this->getBids($filters);
return view('public.bids.index', compact('bids'));
}
BidFilter
public function index(BidFilter $filters)
{
$bids = $this->getBids($filters);
return view('public.bids.index', compact('bids'));
}

Get column names in Laravel as ordered on the Table

I am using the following method to get the column names of the database tables of my Laravel (v5.6.24) project. I am using mysql and it was working fine as expected. But from this week the columns names are showing as ordered by name. Previously it was showing names as in order as the actual table.
How can i get the column names in same order as the table?
/*
* Get Table Column Names
*/
public function getTableColumns()
{
return $this->getConnection()->getSchemaBuilder()->getColumnListing($this->getTable());
}
Might help someone: this is a slightly modified version of #Jonas Staudenmeir answer - but tweaked to be reusable. Just feed it the table name and it will spit out an array of the table fields.
private function getTableColumns($table_name)
{
return DB::select(
(new \Illuminate\Database\Schema\Grammars\MySqlGrammar)->compileColumnListing()
.' order by ordinal_position',
[env('DB_DATABASE'), $table_name]
);
}
You'll have to order the columns by ordinal_position:
public function getTableColumns()
{
return $this->getConnection()->select(
(new \Illuminate\Database\Schema\Grammars\MySqlGrammar)->compileColumnListing()
.' order by ordinal_position',
[$this->getConnection()->getDatabaseName(), $this->getTable()]
);
}

How to GROUP and SUM a pivot table column in Eloquent relationship?

In Laravel 4; I have model Project and Part, they have a many-to-many relationship with a pivot table project_part. The pivot table has a column count which contains the number of a part ID used on a project, e.g.:
id project_id part_id count
24 6 230 3
Here the project_id 6, is using 3 pieces of part_id 230.
One part may be listed multiple times for the same project, e.g.:
id project_id part_id count
24 6 230 3
92 6 230 1
When I show a parts list for my project I do not want to show part_id twice, so i group the results.
My Projects model has this:
public function parts()
{
return $this->belongsToMany('Part', 'project_part', 'project_id', 'part_id')
->withPivot('count')
->withTimestamps()
->groupBy('pivot_part_id')
}
But of course my count value is not correct, and here comes my problem: How do I get the sum of all grouped parts for a project?
Meaning that my parts list for project_id 6 should look like:
part_id count
230 4
I would really like to have it in the Projects-Parts relationship so I can eager load it.
I can not wrap my head around how to do this without getting the N+1 problem, any insight is appreciated.
Update: As a temporary work-around I have created a presenter method to get the total part count in a project. But this is giving me the N+1 issue.
public function sumPart($project_id)
{
$parts = DB::table('project_part')
->where('project_id', $project_id)
->where('part_id', $this->id)
->sum('count');
return $parts;
}
Try to sum in Collection,
$project->parts->sum('pivot.count');
This is best way I found. It's clean (easy to read) and able to re-use all of your scope, ordering and relation attribute caching in parts many-to-many defination.
#hebron No N+1 problem for this solution if you use with('parts') to eager load. Because $project->parts (without funtion call) is a cached attribute, return a instance of Collection with all your data. And sum('pivot.count') is a method of Collection which contains pure funcional helpers (not relative to database, like underscore in js world).
Full example:
Definition of relation parts:
class Project extends Model
{
public function parts()
{
return $this->belongsToMany('Part', 'project_part', 'project_id', 'part_id')
->withPivot('count')
->withTimestamps();
}
}
When you use it (note that eager load is important to avoid N+1 problem),
App\Project::with('parts')->get()->each(function ($project) {
dump($project->parts->sum('pivot.count'));
});
Or you can define the sum function in Project.php,
class Project extends Model
{
...
/**
* Get parts count.
*
* #return integer
*/
public function partsCount()
{
return $this->parts->sum('pivot.count');
}
}
If you want to avoid with('parts') on caller side (eager load parts by default), you can add a $with attribute
class Project extends Model
{
/**
* The relations to eager load on every query.
*
* #var array
*/
protected $with = ['parts'];
...
}
From the code source:
We need to alias all of the pivot columns with the "pivot_" prefix so we can easily extract them out of the models and put them into the pivot relationships when they are retrieved and hydrated into the models.
So you can do the same with select method
public function parts()
{
return $this->belongsToMany('Part', 'project_part', 'project_id', 'part_id')
->selectRaw('parts.*, sum(project_part.count) as pivot_count')
->withTimestamps()
->groupBy('project_part.pivot_part_id')
}
The best way that you can use is:
$project->parts->sum('pivot.count');
I faced the same problem, but this solved my issue.

Creating a Many-to-many relationship in Laravel with additional data

I have in my database a pivot table that stores extra information. It has 2 foreign keys, and an additional field. Here's what it looks like:
EventTeam
int event_id (fk)
int team_id (fk)
boolean home
The intent here is that an Event may have many teams (in fact, it must have at least 2, but that's not a database constraint), and a team may participate in many events. However, for each event-team relationship, I want to also track whether the team is considered the home team for that event.
How do I define my model with this in mind? Do I have an EventTeam model at all, or do I define a belongsToMany relationship in both the Team and Event models? If I need a separate model, what relationships do I define in it? If I don't, how do I add the boolean field to the pivot table that gets used? I really have no idea how to do this.
You dont need a EventTeam model per se, but it could come in handy for seeders or if you are going to attach models to your EventTeam connection anywhere else in your app. This should work:
Event model:
public function teams()
{
return $this->belongsToMany('Team');
}
Team model:
public function events()
{
return $this->belongsToMany('Event');
}
For the extra boolean you can use ->withPivot().
$this->belongsToMany('Event')->withPivot('is_home');
See http://laravel.com/docs/eloquent#working-with-pivot-tables for more info.
Updated answers:
1) I would put it in both models so you can access the pivot data from both sides without a problem.
2) It should be to column name indeed.
3) Like i said its not really needed for you in this situation, but you could do this:
EventTeam model:
public function event()
{
return $this->belongsTo('Event');
}
public function team()
{
return $this->belongsTo('Team');
}
Add withPivot('home') on your relations definitions, then you can access it like this:
$team->events->first()->pivot->home; // 0/1
$event->teams->first()->pivot->home; // 0/1
first is just an example of getting single related model here.
Now, next thing is adding that value to the relation:
$team = Team::find($id);
$event = Event::find($eventId);
$team->events()->attach($event, ['home' => 1]);
// or
$team->events()->attach($eventId, ['home' => 1]);
// or using sync
$event->teams()->sync([1,5,15], ['home' => 0]);
Another thing is querying that field:
// load first team and related events, that the team hosts
$team = Team::with(['events'=>function ($q) {
$q->wherePivot('home', 1);
}])->first();
// load only teams that are hosts for any event
$hostTeams = Team::whereHas('events', function ($q) {
// wherePivot won't work here!
$q->where('event_team.home', 1);
})->get();
and so on.

Resources