I have seen this tutorial online https://www.itsolutionstuff.com/post/laravel-automatic-daily-database-backup-tutorialexample.html on how to backup database daily for mysql. How to do this in postgreSQL?
MySQL
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
use Carbon\Carbon;
class DatabaseBackUp extends Command
{
/**
* The name and signature of the console command.
*
* #var string
*/
protected $signature = 'database:backup';
/**
* The console command description.
*
* #var string
*/
protected $description = 'Command description';
/**
* Create a new command instance.
*
* #return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* #return int
*/
public function handle()
{
$filename = "backup-" . Carbon::now()->format('Y-m-d') . ".gz";
$command = "mysqldump --user=" . env('DB_USERNAME') ." --password=" . env('DB_PASSWORD') . " --host=" . env('DB_HOST') . " " . env('DB_DATABASE') . " | gzip > " . storage_path() . "/app/backup/" . $filename;
$returnVar = NULL;
$output = NULL;
exec($command, $output, $returnVar);
}
}
This backup package is your friend ;)
Related
I am trying to set up a scheduler for weekly database backup in laravel. I have created a command and filled out some data like the command itself and description, and registered it in the console kernel as well. The issue is that the file never gets created and/or stored in storage.
This is the part of the code where is the command:
public function handle()
{
Log::info('Database backup completed.');
$filename = 'mysite' . Carbon::now()->format('Y-m-d') . ".gz";
$command = "mysqldump --user=" . env('DB_USERNAME') ." --password=" . env('DB_PASSWORD') . " --host=" . env('DB_HOST') . " " . env('DB_DATABASE') . " | gzip > " . storage_path() . "/app/backup/" . $filename;
$returnVar = NULL;
$output = NULL;
exec($command, $output, $returnVar);
}
This is the kernel part:
/**
* The Artisan commands provided by your application.
*
* #var array
*/
protected $commands = [
OtherCron::class,
DatabaseBackupCron::class,
];
/**
* Define the application's command schedule.
*
* #param Schedule $schedule
* #return void
*/
protected function schedule(Schedule $schedule)
{
$schedule->command('othercron')->dailyAt('00:00');
$schedule->command('database-backup:cron')->everyFiveMinutes();
}
Note: I have used "->everyFiveMinutes()" just for testing purposes :)
You could use a ready-made lib for that:
github.com/spresnac/laravel-artisan-database-helper
and then just call the command in the scheduler as you like ;)
You can also set the full path to your mysqldump binary, if it's not in your path ;)
I tried your code, it is working for me, the only difference is in Kernel, I wrote like this.
protected $commands = [
'App\Console\Commands\DatabaseBackUp'
];
/**
* Define the application's command schedule.
*
* #param \Illuminate\Console\Scheduling\Schedule $schedule
* #return void
*/
protected function schedule(Schedule $schedule)
{
//some other commands
$schedule->command('database:backup')->weekly();
}
now the moment I run php artisan database:backup a file with the extension .zp created in the storage folder
php artisan make:command DatabaseBackUp
Use this command make databasebackup file.
app/Console/Commands/DatabaseBackUp.php
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
use Carbon\Carbon;
class DatabaseBackUp extends Command
{
/**
* The name and signature of the console command.
*
* #var string
*/
protected $signature = 'database:backup';
/**
* The console command description.
*
* #var string
*/
protected $description = 'Command description';
/**
* Create a new command instance.
*
* #return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* #return int
*/
public function handle()
{
$filename = "backup-" . Carbon::now()->format('Y-m-d') . ".gz";
$command = "mysqldump --user=" . env('DB_USERNAME') ." --password=" . env('DB_PASSWORD') . " --host=" . env('DB_HOST') . " " . env('DB_DATABASE') . " | gzip > " . storage_path() . "/app/backup/" . $filename;
$returnVar = NULL;
$output = NULL;
exec($command, $output, $returnVar);
}
}
In this step, we need to create "backup" folder in your storage folder. you must have to create "backup" on following path:
storage/app/backup
Now, in this step, we need to schedule our created command. so let's update kernel file as like bellow:
app/Console/Kernel.php
<?php
namespace App\Console;
use Illuminate\Console\Scheduling\Schedule;
use Illuminate\Foundation\Console\Kernel as ConsoleKernel;
class Kernel extends ConsoleKernel
{
/**
* The Artisan commands provided by your application.
*
* #var array
*/
protected $commands = [
'App\Console\Commands\DatabaseBackUp'
];
/**
* Define the application's command schedule.
*
* #param \Illuminate\Console\Scheduling\Schedule $schedule
* #return void
*/
protected function schedule(Schedule $schedule)
{
$schedule->command('database:backup')->daily();
}
/**
* Register the commands for the application.
*
* #return void
*/
protected function commands()
{
$this->load(__DIR__.'/Commands');
require base_path('routes/console.php');
}
}
you can check following command to getting database backup with this command:
php artisan database:backup
t will create one backup file on your backup folder. you can check there.
Now, we are ready to setup cron on our server.
At last you can manage this command on scheduling task, you have to add a single entry to your server’s crontab file:
Run following command:
crontab -e
I am trying to test a custom Artisan command which does multiple things and then at the end does a csv import. I instantiate the object manually new CsvDirectDatabaseImporter inside the artisan command. This runs a method called import() which imports from csv to database using LOAD DATA LOCAL INFILE which is not supported by SQLite. Since I want my tests to run in memory I want to override (or mock/stub not sure what the correct term is) the import method on the CsvDirectDatabaseImporter class so it doesn't do anything during the import call. This way the rest of my tests will work (I know now I'm not testing the actual import) How would I go around this:
Here is a simplified version my Artisan Class:
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\File;
use App\Services\CsvDirectDatabaseImporter\CsvDirectDatabaseImporter;
use App\Services\CsvDirectDatabaseImporter\MyColumns;
class DataMartImport extends Command
{
/**
* The name and signature of the console command.
*
* #var string
*/
protected $signature = 'myimport:import
{year : The year of processing} ';
/**
* The console command description.
*
* #var string
*/
protected $description = 'My Import';
/**
* Create a new command instance.
*
* #return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* #return mixed
*/
public function handle()
{
$year = $this->argument('year');
// Copy the file to processing location.
File::copy($files[0], $processing_file);
// Import the CSV File.
$csvImporter = new CsvDirectDatabaseImporter($processing_file, 'myTable', new MyColumns());
$csvImporter->import();
}
}
A simplified version of a Feature test of running my custom artisan command:
<?php
namespace Tests\Feature\Console\DataMart;
use Illuminate\Support\Facades\File;
use Tests\TestCase;
use Illuminate\Support\Facades\Config;
use Mockery as m;
use App\Services\CsvDirectDatabaseImporter\DataMartColumns;
use App\Services\CsvDirectDatabaseImporter\CsvDirectDatabaseImporter;
use Illuminate\Support\Facades\Artisan;
class MyImportTest extends TestCase
{
public function testImportFoldersGetCreatedIfNoDirectory()
{
$year = 2019;
$this->artisan('myimport:import', ['year' => $year]);
// Assertions of items go here unrelated to the actual database import.
}
}
CSVImorter Class
<?php
namespace App\Services\CsvDirectDatabaseImporter;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\File;
use Symfony\Component\HttpFoundation\File\File as CSV_File;
class CsvDirectDatabaseImporter {
/**
* File to import.
*
* #var \Symfony\Component\HttpFoundation\File\File
*/
private $file;
/**
* Table name.
*
* #var string
*/
private $table;
/**
* Fields terminated by.
*
* #var string
*/
public $fieldsTerminatedBy = '\',\'';
/**
* Enclosed by.
*
* #var string
*/
public $enclosedBy = '\'"\'';
/**
* Lines terminated by.
*
* #var string
*/
public $linesTerminatedBy = '\'\n\'';
/**
* Ignore first row.
*
* #var bool
*/
public $ignoreFirstRow = true;
/**
* Csv Import columns.
*
* #var array
*/
public $columns;
/**
* CsvImporter constructor.
*
* #param string $path
* The full temporary path to the file
*/
public function __construct(string $path, $table, CsvDirectDatabaseImportColumns $columns)
{
$this->file = new CSV_File($path);
$this->table = $table;
$this->columns = $columns->getColumns();
}
/**
* Import method used for saving file and importing it using database query.
*/
public function import()
{
// Normalize line endings
$normalized_file = $this->normalize($this->file);
// Import contents of the file into database
return $this->importFileContents($normalized_file, $this->table, $this->columns);
}
/**
* Convert file line endings to uniform "\r\n" to solve for EOL issues
* Files that are created on different platforms use different EOL characters
* This method will convert all line endings to Unix uniform
*
* #param string $file_path
* #return string $file_path
*/
protected function normalize($file_path)
{
// Load the file into a string.
$string = #file_get_contents($file_path);
if (!$string) {
return $file_path;
}
// Convert all line-endings using regular expression.
$string = preg_replace('~\r\n?~', "\n", $string);
file_put_contents($file_path, $string);
return $file_path;
}
/**
* Import CSV file into Database using LOAD DATA LOCAL INFILE function
*
* NOTE: PDO settings must have attribute PDO::MYSQL_ATTR_LOCAL_INFILE => true
*
* #param string $file_path
* File path.
* #param string $table_name
* Table name.
* #param array $columns
* Array of columns.
*
* #return mixed Will return number of lines imported by the query
*/
private function importFileContents($file_path, $table_name, $columns)
{
$prefix = config('database.connections.mysql.prefix');
$query = '
LOAD DATA LOCAL INFILE \'' . $file_path . '\' INTO TABLE `' . $prefix . $table_name . '`
FIELDS TERMINATED BY ' . $this->fieldsTerminatedBy . '
ENCLOSED BY ' . $this->enclosedBy . '
LINES TERMINATED BY ' . $this->linesTerminatedBy . '
';
if ($this->ignoreFirstRow) {
$query .= ' IGNORE 1 ROWS ';
}
if ($columns) {
$query .= '(' . implode(",\n", array_keys($columns)) . ')';
$query .= "\nSET \n";
$sets = [];
foreach ($columns as $column) {
$sets[] = $column['name'] . ' = ' . $column['set'];
}
$query .= implode(",\n", $sets);
}
return DB::connection()->getPdo()->exec($query);
}
}
CsvDirectDatabaseImportColumns Interface
<?php
namespace App\Services\CsvDirectDatabaseImporter;
interface CsvDirectDatabaseImportColumns
{
/**
* Returns array of columns.
*
* Ex:
* '#user_id' => [
* 'name' => 'user_id',
* 'set' => '#user_id',
* ],
* '#agent_number' => [
* 'name' => 'agent_number',
* 'set' => 'LEFT(#agent_number, 7)',
* ],
*
* The key should be the column name of the csv but add # in front. The name
* will be the database table. The set will be what it s se
*
* #return array
*/
public function columns();
/**
* Returns columns.
*
* #return array
* Columns.
*/
public function getColumns();
}
Things I tried
$mock = $this->createMock(CsvDirectDatabaseImporter::class);
$mock->method('import')->willReturn(true);
$this->app->instance(CsvDirectDatabaseImporter::class, $mock);
$this->artisan('datamart:import', ['year' => $year]);
But no luck there. It still runs the regular import() method.
So I have tried to reproduce what I think you need into a simple example
Let's say we have this command
<?php
namespace App\Console\Commands;
use Exception;
use Illuminate\Console\Command;
class Foo extends Command
{
protected $signature = 'foo';
public function __construct()
{
parent::__construct();
}
public function handle()
{
if ($this->import()) {
$this->info('Success');
} else {
$this->error('Failed');
}
}
public function import()
{
throw new Exception('An exception that should not be thrown');
}
}
The import method throws an exception but here's how to mock it to return true
<?php
namespace Tests\Feature;
use Tests\TestCase;
use App\Console\Commands\Foo;
class FooCommandTest extends TestCase
{
public function testExample()
{
$mock = $this->getMockBuilder(Foo::class)->setMethods(['import'])->getMock();
$mock->method('import')->willReturn(true);
$this->app->instance('App\Console\Commands\Foo', $mock);
$this->artisan('foo')
->expectsOutput('Success')
->assertExitCode(0);
}
}
This test passes with two successful assertion, so you can adjust your command code to use a dedicated method for the import
Hope this helps
I have problem with limit email exchange from SMTP. I have counting table with specific column. total is 201. that total will send email automatic with schedule task on the server.
Counting TOTAL
can I send email per batch using laravel as much as 201 email in once send email?
cronEmail
/**
* The name and signature of the console command.
*
* #var string
*/
protected $signature = 'email:reminder';
/**
* The console command description.
*
* #var string
*/
protected $description = 'Command description';
/**
* Create a new command instance.
*
* #return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* #return mixed
*/
public function handle()
{
$this->updateMailConfig();
$check = DB::table('a_kpi')->join('d_mem','k_created_by','m_id')
->leftjoin('a_kpid','kd_kpi_id','k_id')
->where('k_status_id',1)
->where(function($query){
$query->where('kd_status_id','=','In Progress')
->orwhere('kd_status_id','=',null)
->orwhere('kd_status_id','=')
->orwhere('kd_status_id','=','null')
->orwhere('kd_status_id','=',"null");
})
->get();
$dt = date("Y-m-d");
$dtdt = date( "Y-m-d", strtotime( "$dt +10 day" ) );
for ($i=0; $i <count($check); $i++) {
if ($check[$i]->kd_duedate == $dtdt) {
$mail = $check[$i]->m_email;
Mail::send('mail.tes',
['pesan' => 'KPI INFORMATION',
'k_label' => $check[$i]->k_label,
'kd_tacticalstep' => $check[$i]->kd_tacticalstep,
'kd_duedate' => $check[$i]->kd_duedate],
function ($message) use ($mail)
{
$message->subject('REMINDER');
$message->to($mail);
});
$data = DB::table('d_log_reminder')
->insert([
'dlr_id'=>$check[$i]->k_id,
'dlr_kpi_id'=>$check[$i]->k_id,
'dlr_kpid_id'=>$check[$i]->kd_id,
'dlr_tacticalstep'=>$check[$i]->kd_tacticalstep,
'dlr_label'=>$check[$i]->k_label,
'dlr_duedate'=>$check[$i]->kd_duedate,
'dlr_created_by'=>$check[$i]->k_created_by,
'dlr_send_to'=>$mail
]);
}
}
// $check = DB::table('d_mem')->where('m_username','admin')->update(['m_code'=>'cor'.date('d-m-y h:i:s')]);
}
}
Kernel.php
<?php
namespace App\Console;
use Illuminate\Console\Scheduling\Schedule;
use App\Helper\ConfigUpdater;
use Mail;
use DB;
use Illuminate\Foundation\Console\Kernel as ConsoleKernel;
class Kernel extends ConsoleKernel
{
use ConfigUpdater;
/**
* The Artisan commands provided by your application.
*
* #var array
*/
protected $commands = [
'App\Console\Commands\cronEmail'
//
];
/**
* Define the application's command schedule.
*
* #param \Illuminate\Console\Scheduling\Schedule $schedule
* #return void
*/
protected function schedule(Schedule $schedule)
{
$schedule->command('email:reminder')
->everyMinute();
}
/**
* Register the Closure based commands for the application.
*
* #return void
*/
protected function commands()
{
require base_path('routes/console.php');
}
}
I've created a command and I am trying to query my database and group my results by a key but I keep recieving this error:
In Builder.php line 2512:
Method Illuminate\Database\Query\Builder::keyBy does not exist.
Laravel version is 5.6.4
Command code:
<?php
namespace App\Console\Commands;
use App\User;
use Illuminate\Console\Command;
class TwitchPointScanner extends Command
{
/**
* The name and signature of the console command.
*
* #var string
*/
protected $signature = 'command:twitchPointScanner';
/**
* The console command description.
*
* #var string
*/
protected $description = 'Twitch Point Scanner';
/**
* Create a new command instance.
*
* #return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* #return mixed
*/
public function handle()
{
$usersDistributors = User::where('point_distributor', 1)
->inRandomOrder()
->get();
$usersTwitchVerified = User::where('point_distributor', 0)
->whereNotNull('twitch_username')
->keyBy('twitch_username')
->get();
$this->line('Distributors ---');
foreach($usersDistributors as $user) {
$this->line($user->id . ': '.$user->twitch_username);
}
$this->line('Point gainers ---');
foreach($usersTwitchVerified as $user) {
$this->line($user->id . ': '.$user->twitch_username);
}
}
}
keyBy is a collection method, so you need to get the data first:
$usersTwitchVerified = User::where('point_distributor', 0)
->whereNotNull('twitch_username')
->get()
->keyBy('twitch_username');
To get things started, I made a custom Artisan Command called MySqlRestore.
It simply restores the database using the dumped sql file.
Here's my MySqlRestore code:
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
class MySqlRestore extends Command
{
/**
* The name and signature of the console command.
*
* #var string
*/
protected $signature = 'db:restore';
/**
* The console command description.
*
* #var string
*/
protected $description = 'Restores database using info from .env';
/**
* Create a new command instance.
*
* #return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* #return mixed
*/
public function handle()
{
$sqlfile = //data from controller;
$ds = DIRECTORY_SEPARATOR;
$host = env('DB_HOST');
$username = env('DB_USERNAME');
$password = env('DB_PASSWORD');
$database = env('DB_DATABASE');
$mysqlpath = 'C:\xampp\mysql\bin\mysql';
$path = 'C:\salesandinventory\Backups\\';
$command = sprintf($mysqlpath . ' --user=' . $username . ' --password=' . $password . ' --host=' . $host . ' ' . $database . ' < ' . $path . $sqlfile);
exec($command);
}
}
Now on this line $sqlfile = //data from controller;, I need the data from my controller.
Here's how my controller looks like:
<?php
namespace App\Http\Controllers;
use App\Database;
use Illuminate\Http\Request;
use Artisan;
class DatabaseController extends Controller
{
public function index()
{
return view('database.index');
}
public function restoreDatabase(Request $request)
{
$sqlfile = $request->sqlfile; // the data I need.
Artisan::call('db:restore');
return view('database.index');
}
}
Now I don't have any idea how to pass $sqlfile = $request->sqlfile; this data from my controller into my Artisan handle function.
Data's are passed through the protected $signature using curly braces {dataName}
e.g
protected $signature = 'db:restore {dataName}'
and it is called using
$this->argument('dataName');
In your controller
Artisan::call('db:restore',['test'=> $test]);
namespace App\Console\Commands;
use Illuminate\Console\Command;
class MySqlRestore extends Command
{
/**
* The name and signature of the console command.
*
* #var string
*/
protected $signature = 'db:restore {test}';
/**
* The console command description.
*
* #var string
*/
protected $description = 'Restores database using info from .env';
public $sqlFile;
/**
* Create a new command instance.
*
* #return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* #return mixed
*/
public function handle()
{
$sqlfile = $this->argument('test');
$ds = DIRECTORY_SEPARATOR;
$host = env('DB_HOST');
$username = env('DB_USERNAME');
$password = env('DB_PASSWORD');
$database = env('DB_DATABASE');
$mysqlpath = 'C:\xampp\mysql\bin\mysql';
$path = 'C:\salesandinventory\Backups\\';
$command = sprintf($mysqlpath . ' --user=' . $username . ' --password=' . $password . ' --host=' . $host . ' ' . $database . ' < ' . $path . $sqlfile);
exec($command);
}
}
Call it like this
Artisan::call('db:restore',['test'=> $test]);
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
class MySqlRestore extends Command
{
/**
* The name and signature of the console command.
*
* #var string
*/
protected $signature = 'db:restore';
/**
* The console command description.
*
* #var string
*/
protected $description = 'Restores database using info from .env';
public $sqlFile;
/**
* Create a new command instance.
*
* #return void
*/
public function __construct($sqlFile)
{
$this->sqlFile = $sqlFile;
parent::__construct();
}
/**
* Execute the console command.
*
* #return mixed
*/
public function handle()
{
$sqlfile = $this->sqlFile;
$ds = DIRECTORY_SEPARATOR;
$host = env('DB_HOST');
$username = env('DB_USERNAME');
$password = env('DB_PASSWORD');
$database = env('DB_DATABASE');
$mysqlpath = 'C:\xampp\mysql\bin\mysql';
$path = 'C:\salesandinventory\Backups\\';
$command = sprintf($mysqlpath . ' --user=' . $username . ' --password=' . $password . ' --host=' . $host . ' ' . $database . ' < ' . $path . $sqlfile);
exec($command);
}
}
Controller
<?php
namespace App\Http\Controllers;
use App\Database;
use Illuminate\Http\Request;
use Artisan;
class DatabaseController extends Controller
{
public function index()
{
return view('database.index');
}
public function restoreDatabase(Request $request)
{
$sqlfile = $request->sqlfile; // the data I need.
Artisan::call('db:restore',['sqlFile'=>$sqlFile]);
return view('database.index');
}
}
Your controller should not be executing Artisan commands, especially potentially long-running ones like executing a database back-up.
Instead, consider dispatching a queued job that performs the restore. You can then return control to the user and they can get on with things they need to do, instead of keeping a web page open that could possibly timeout and leave the database in a corrupt state.
class DatabaseRestoreController extends Controller
{
public function store(Request $request)
{
dispatch(new RestoreDatabaseJob($request->input('filename')));
return redirect()->back()->withSuccess('Restoring database.');
}
}
And the job class itself:
class RestoreDatabaseJob implements ShouldQueue
{
use InteractsWithQueue;
public $filename;
public function __construct($filename)
{
$this->filename = $filename;
}
public function handle()
{
Artisan::call('db:restore', [
'filename' => $this->filename,
]);
// You can notify user restore completed
// Send email, SMS via notification etc.
}
}