I am using maatwebsite/excel package to import a CSV file. I cannot able to limit the imported rows. Like I need the first "n" number of rows from the uploaded file. I saw an interface
WithLimit
But when I am using that with Limit(). It throws some errors, like some table values which I am trying to insert is null. I think it is using the limit for columns. Are there any workarounds to fix the problem?
My controller
class FinancialTransController extends Controller
{
public function store(Request $request)
{
Excel::import(new FinancialTrans, request()->file('myfile'));
return 'success';
}
}
This is my import class
class FinancialTrans implements ToModel,WithHeadingRow
{
/**
* #param array $row
*
* #return \Illuminate\Database\Eloquent\Model|null
*/
public function model(array $row)
{
return new FinancialTran([
'module_id' =>$row['sr'],
'tran_id'=>$row['sr'],
'amount'=>$row['sr'],
'crdr'=>$row['sr'],
'tran_date'=>$row['date'],
'acad_year'=>$row['academic_year'],
'entry_mode'=>$row['sr'],
'voucher_number'=>$row['voucher_no'],
'br_id'=>$row['sr'],
]);
}
public function headingRow(): int
{
return 6;
}
}
WithHeadingRow loads all previous rows. use interface WithStartRow instead.
use Maatwebsite\Excel\Concerns\ToModel;
use Maatwebsite\Excel\Concerns\WithStartRow;
use Maatwebsite\Excel\Concerns\WithLimit;
class FinancialTrans implements ToModel, WithLimit, WithStartRow
{
protected $startRow;
public function __construct(int $startRow)
{
$this->startRow = $startRow;
}
...
public function startRow(): int
{
return $this->startRow;
}
public function limit(): int
{
return 10;
}
If you look to the code in maatwebsite/excel, you can see it is an interface with has a limit function, this one you has to implement. If you are using an IDE, it will often suggest to implement these methods.
use Maatwebsite\Excel\Concerns\WithLimit;
class FinancialTrans implements ToModel,WithHeadingRow, WithLimit
{
public function limit(): int
{
return 100; // only take 100 rows
}
...
}
Related
Hi I have a large csv file with 130.000 rows
I use laravel excel 3.1 and lavaravel 5.8
Import class:
<?php
namespace App\Imports;
use App\UsoSuelo;
use Maatwebsite\Excel\Concerns\ToModel;
use Maatwebsite\Excel\Concerns\WithHeadingRow;
use Maatwebsite\Excel\Concerns\WithChunkReading;
use Maatwebsite\Excel\Concerns\WithBatchInserts;
class UsoSueloImport implements ToModel, WithHeadingRow, WithChunkReading, WithBatchInserts
{
/**
* #param array $row
*
* #return \Illuminate\Database\Eloquent\Model|null
*/
public function model(array $row)
{
return new UsoSuelo([
'cod_pais' => $row['cod_pais'],
'cod_fundo' => $row['cod_fundo'],
'nom_fundo' => $row['nom_fundo'],
]);
}
public function batchSize(): int
{
return 1000;
}
public function chunkSize(): int
{
return 1000;
}
}
And I use a trait class from my controller:
trait storeTrait{
public function storeUsoSuelo($archivo) {
Excel::import(new UsoSueloImport,$archivo);
}
public function storeFundo($archivo) {
Excel::import(new FundosImport,$archivo);
}
public function storeFundoGrilla($archivo) {
Excel::import(new FundosGrillasImport,$archivo);
}
public function storeCuadrante($archivo) {
Excel::import(new CuadrantesImport,$archivo);
}
}
This is my ImportController
class ImportController extends Controller
{
use storeTrait {
storeUsoSuelo as storeUsoSuelos;
storeFundo as storeFundos;
storeFundoGrilla as storeFundoGrillas;
storeCuadrante as storeCuadrantes;
}
public function store(Request $request)
{
$usoSuelo = 'uso_suelo.csv';
$this->storeUsoSuelos($usoSuelo);
$cuadrante = 'cuadrantes.csv';
$this->storeCuadrantes($cuadrante);
$fundo = 'mv_qav_fundos.csv';
$this->storeFundos($fundo);
$fundoGrilla = 'fundos_grilla.csv';
$this->storeFundoGrillas($fundoGrilla);
}
}
I have done tests and my code works with a csv of less than 100 rows
but when I try with the 130,000 rows it takes too long, and I end up getting the following error:
"Maximum execution time of 60 seconds exceeded"
And after 1 minute only 4000 rows have been inserted in the database (postgresql)
I put these 2 lines in my controller, at the beginning of the script:
ini_set ('max_execution_time', 3600);
ini_set ('memory_limit', '2048M');
With this I solved it, I also changed the chunk from 1000 to 5000, but it still takes too long, at least 5 minutes
Have you tried this package league/csv from The League of Extraordinary Packages.
If you only need support for CSV that is, and not xlsx or other excel types.
From what I read this package might give you a better performance, this comment from one of the Maatwebsite team members (the package you are currently using) also confirms this.
I have a morph relationship, where the subject could have multiple relationships. Their existence depends on morphed model. I need to retrieve all the related models (whereHas() doesn't solve the problem) and I want their relationships being loaded if they exists on particular model (with() won't work, because the relationship doesn't always exist).
Is there anything else (built-in) that I can use to fluently solve this scenario, or hacking is the only way around it?
<?php
...
class Post extends Model
{
/**
* Get all of the post's comments.
*/
public function comments()
{
return $this->morphMany('App\Comment', 'commentable');
}
/**
* This relationship is available for Post model only
*/
public function relationA()
{
// return $this->hasMany(...);
}
}
class Video extends Model
{
/**
* Get all of the video's comments.
*/
public function comments()
{
return $this->morphMany('App\Comment', 'commentable');
}
/**
* This relationship is available for Video model only
*/
public function relationB()
{
// return $this->hasMany(...);
}
}
class Comment extends Model
{
/**
* Get all of the owning commentable models.
*/
public function commentable()
{
return $this->morphTo();
}
public static function feed()
{
self::with('commentable')
->withIfExists(['commentable.relationA', 'commentable.relationB'])
// ...
->get();
}
public function scopeWithIfExists($query, $relation)
{
// There is no way to implement such a scope
// in order to reduce umber of queries by eager loading relations
// as we don't know of what type the subject is
// without looking it up in database
}
}
Check out at Query Scopes.
With that you can create a scope to load a relation if it exists, for example:
User::withRelationIfExists('cars')->where(...)
For example: (code not tested)
public function scopeWithRelationIfExists($query, $relation)
{
if (! method_exists(get_class(), $relation)) {
return;
}
return $query->with($relation);
}
Below is the one of the model. I would like to delete a Telco entry only if no other model is referencing it? What is the best method?
namespace App;
use Illuminate\Database\Eloquent\Model;
class Telco extends Model
{
public function operators()
{
return $this->hasMany('App\Operator');
}
public function packages()
{
return $this->hasMany('App\Package');
}
public function topups()
{
return $this->hasMany('App\Topup');
}
public function users()
{
return $this->morphMany('App\User', 'owner');
}
public function subscribers()
{
return $this->hasManyThrough('App\Subscriber', 'App\Operator');
}
}
You can use deleting model event and check if there any related records before deletion and prevent deletion if any exists.
In your Telco model
protected static function boot()
{
parent::boot();
static::deleting(function($telco) {
$relationMethods = ['operators', 'packages', 'topups', 'users'];
foreach ($relationMethods as $relationMethod) {
if ($telco->$relationMethod()->count() > 0) {
return false;
}
}
});
}
$relationships = array('operators', 'packages', 'topups', 'users', 'subscribers');
$telco = Telco::find($id);
$should_delete = true;
foreach($relationships as $r) {
if ($telco->$r->isNotEmpty()) {
$should_delete = false;
break;
}
}
if ($should_delete == true) {
$telco->delete();
}
Well, I know this is ugly, but I think it should work. If you prefer to un-ugly this, just call every relationship attributes and check whether it returns an empty collection (meaning there is no relationship)
If all relationships are empty, then delete!
After seeing the answers here, I don't feel copy pasting the static function boot to every models that need it. So I make a trait called SecureDelete. I put #chanafdo's foreach, inside a public function in SecureDelete.
This way, I can reuse it to models that need it.
SecureDelete.php
trait SecureDelete
{
/**
* Delete only when there is no reference to other models.
*
* #param array $relations
* #return response
*/
public function secureDelete(String ...$relations)
{
$hasRelation = false;
foreach ($relations as $relation) {
if ($this->$relation()->withTrashed()->count()) {
$hasRelation = true;
break;
}
}
if ($hasRelation) {
$this->delete();
} else {
$this->forceDelete();
}
}
}
Add use SecureDelete to the model that needs it.
use Illuminate\Database\Eloquent\Model;
use App\Traits\SecureDelete;
class Telco extends Model
{
use SecureDelete;
public function operators()
{
return $this->hasMany('App\Operator');
}
// other eloquent relationships (packages, topups, etc)
}
TelcoController.php
public function destroy(Telco $telco)
{
return $telco->secureDelete('operators', 'packages', 'topups');
}
In addition, instead of Trait, you can also make a custom model e.g BaseModel.php that extends Illuminate\Database\Eloquent\Model, put the function secureDelete there, and change your models to extends BaseModel.
I have a class called SubjectData:
class SubjectData extends Model
{
protected $table = 'subject_datas';
protected $fillable = ['firstname','lastname','birthdate','birthcity','months'];
protected $dates = ['birthdate'];
public function setBirthdateAttribute($date)
{
// So we can add the time, not just he php date
$this->attributes['birthdate'] = Carbon::createFromFormat('d/m/Y', $date);
}
public function anamnesis() {
return $this->belongsTo('App\Anamnesis');
}
}
And I have a class called Anamnesis:
class Anamnesis extends Model
{
public function meetingTest() {
return $this->belongsTo('App\MeetingTest');
}
public function subject() {
return $this->belongsTo('App\Subject','subject_id','id');
}
public function subjectData() {
return $this->hasOne('App\SubjectData');
}
public function scholarHistory() {
return $this->hasOne('App\ScholarHistory');
}
public function familyHistory() {
return $this->hasOne('App\FamilyHistory');
}
public function psicodiagnosis() {
return $this->hasOne('App\Psicodiagnosis');
}
}
The store function of the SubjectController class works like this:
public function store(CreateSubjectRequest $request)
{
$input = $request->all();
// Let's generate the anamnesis of the subject
$anamnesis = Anamnesis::create();
$anamnesis->save();
$newSubjectData = $this->saveSubjectData($input);
$anamnesis->subjectData()->save($newSubjectData);
......
......
}
where the function called is:
public function saveSubjectData($input)
{
$subjectData['firstname'] = $input['firstname'];
$subjectData['lastname'] = $input['lastname'];
$subjectData['birthcity'] = $input['birthcity'];
$subjectData['birthdate'] = $input['birthdate'];
return SubjectData::create($subjectData);
}
The problem is with the "birthdate" property.
If i check the value of $newSubjectData (dd($newSubjectdata)) after the call $this->saveSubjectData($input) the value of the birthdate is exactly the one i set on the calendar in the frontside (and also in the db the value is correct)
If I put the dd($anamnesis->subjectData) after the call $anamnesis->subjectData()->save($newSubjectData) the result is the "today" date and also in the DB the value is not the one I set but the date of today.
I can't find the error
P.S. The calendar is inside a Vue template
I think the problem is that, the date must be an instance of Carbon or is properly formatted according to your database table. Try the following inside your saveSubjectData() method
$subjectData['birthdate'] = Carbon\Carbon::parse($input['birthdate']
I write down the answer but i thank John Aldrin that guided me in the right direction.
You have to put the timestamp('birthdate') AFTER the default timestamps of the migration table (so at the end of the migration table)
I don't know why. If someone knows please explain !
Mode Position.php has following code.
class Position extends \Eloquent {
public static function get_position_by_url($url) {
return Position::where('url','=',$url)->where('status','=','0')->with('message')->get();
}
public function message() {
return $this->hasMany('Message');
}
}
The above code works fine, What I need is to get data by condition in messages table, like following:
class Position extends \Eloquent {
public static function get_position_by_url($url,$screenSize) {
return Position::where('url','=',$url)->where('status','=','0')->with('message')->where('screensize','=',$screensize)->get();
}
public function message() {
return $this->hasMany('Message');
}
}
But the both conditions are applied to positions table, which I want to apply second condition to messages table.
You can filter eager loaded models by passing a closure:
return Position::where('url','=',$url)->where('status','=','0')->with(['message' => function($q) use ($screensize){
$q->where('screensize','=',$screensize);
}])->get();
Like documented here under Eager Loading Constraints