Can I import a mysql dump to a laravel migration? - laravel

I have a complete database and need to create migration. I guess there must be a way to do it from a dump but not sure. Is there any way automatically or at least easier to do this task?

You can import dumps in Laravel like this:
DB::unprepared(file_get_contents('full/path/to/dump.sql'));
If I were to refactor an existing app, though, I'd take the time to write migrations from scratch, import the dump into different tables (or a different db, if table names are the same) then import the content to the new structure via seeds.

Laravel can't do that, but I think this will help: Laravel migration generator
It generate migrations based on existing tables.

This question is answered already, but recently in a project, the provided answers did not satisfy my needs any longer. It also does not import a whole database dump, but one (large) table. I felt I should share that with you.
The problem was, I wanted to import a quite large table (list of zipcodes) during my artisan:migrate operation. The solution with DB::unprepared($dump) took way to long and I found an alternative which is MUCH faster.
Just export your table as CSV and use the following Code in your migration's up() function.
// i had to str_replace the backslash on windows dev system... but works on linux, too
$filename = str_replace("\\", "/", storage_path('path/in/storage/to/your/file.csv'));
$query = "LOAD DATA LOCAL INFILE '".$filename."' INTO TABLE yourtable
FIELDS TERMINATED BY '\t'
ENCLOSED BY ''
LINES TERMINATED BY '\n'
IGNORE 0 LINES
(col1,col2,...);";
DB::unprepared($query);
Just update the query as you need. And of course, you should make sure, that the table with the cols
'col1', 'col2' etc... exists. I created it just before the importing of the file. with Schema::create()...
If you run into following error message:
PDO::exec(): LOAD DATA LOCAL INFILE forbidden
There is a way you can get rid of this message: Although it's not really documented you can just add an 'options' key to your config/database.php file. For example mine looks like that:
'mysql' => [
'driver' => 'mysql',
'host' => env('DB_HOST', 'localhost'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => '',
'strict' => false,
'options' => array(
PDO::MYSQL_ATTR_LOCAL_INFILE => true,
)
Note: i'm currently using laravel 5 but it should work with laravel 4, too.

I have a complete database and need to create migration. I guess there must be a way to do it from a dump but not sure. Is there any way automatically or at least easier to do this task?
Not automatically, but we run dumps in a migration using DB::unprepared(). You could use file_get_contents to import from a .sql file and thus not have to worry about escaping the entire dump's " marks...
<?php
use Illuminate\Database\Migrations\Migration;
class ImportDump extends Migration {
/**
* Run the migrations.
*
* #return void
*/
public function up()
{
DB::unprepared("YOUR SQL DUMP HERE");
}
/**
* Reverse the migrations.
*
* #return void
*/
public function down()
{
}
}

Another alternative is using the PDO directly:
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\File;
$sql_dump = File::get('/path/to/file.sql');
DB::connection()->getPdo()->exec($sql_dump);

I am writing my answer as this might help to someone who is using new laravel 8.
In laravel 8, dumping SQL and run migration using SQL(.dump) file is possible. Please refer below link for more detail.
https://laravel.com/docs/8.x/migrations#squashing-migrations
php artisan schema:dump
// Dump the current database schema and prune all existing migrations...
php artisan schema:dump --prune
schema:dump will create new directory under database > schema and SQL dump will stored there.
After that when you try to migrate first it will run dump file from schema and then any pending migration.

another solution work for me in Laravel 5.2:
DB::unprepared(File::get('full/path/to/dump.sql'));

Simple solution provided by Laravel Article for generating migration file from an existing database table.
Try: https://laravelarticle.com/laravel-migration-generator-online

If you can dump to a CSV:
An alternative for some generic data tables (Countries, States, Postal Codes), not via migrations but via seeders. Although you could do it the same way in a migration file.
In your seeder file:
public function run()
{
$this->insertFromCsvFile('countries', 'path/to/countries.csv');
$this->insertFromCsvFile('states', 'path/to/states.csv');
$this->insertFromCsvFile('postal_codes', 'path/to/postal_codes.csv');
}
private function insertFromCsvFile($tableName, $filePath)
{
if( !file_exists($filePath) ){
echo 'File Not Found: '.$filePath."\r\n";
return;
}
$headers = $rows = [];
$file = fopen( $filePath, 'r' );
while( ( $line = fgetcsv( $file ) ) !== false ){
// The first row should be header values that match column names.
if( empty( $headers ) ){
$headers = explode( ',', implode( ',', $line ) );
continue;
}
$row = array_combine( $headers, $line );
foreach( $row as &$val ) if( $val === 'NULL' ) $val = null;
$rows[] = $row;
// Adjust based on memory constraints.
if( count($rows) === 500 ){
DB::table( $tableName )->insert($rows);
$rows = [];
}
}
fclose( $filePath );
if( count($rows) ) DB::table( $tableName )->insert($rows);
}
Run the seeder: php artisan db:seed --class=GenericTableSeeder

You can create laravel migration and models directly from database using https://github.com/XCMer/larry-four-generator
Execute the following code after installing the package
php artisan larry:fromdb

i recently standing in front of the same problem. i didn't want to install a package specially for that, so i decided to write a little tool to help me and others ;)
Here is the link:
http://laravel.stonelab.ch/sql-seeder-converter/
And here you can comment it, if you have any improvement proposals or questions:
http://www.stonelab.ch/en/sql-to-laravel-seeder-converter/

You can use Raahul/Larryfour Package, A model and migration generator for Laravel 4
Raahul/Larryfour Package
After insallation you can use a command line to create a migration from existed database like this:
php artisan raahul:fromdb --only yourdatabase
And you will find the migration in app/migrations/ folder

Related

Encrypted Array Won't Save in Eloquent Model

I'm having an odd error with saving an encrypted array in Laravel. The model never updates even when save() is called.
There are no console or SQL errors.
When the encryption is disabled, there are no errors and the model updates successfully.
In a Controller, I'm calling the model like so:
$userData = UserData::where('user_id', $user_id)->first();
I then pull the array:
$encryptedData = $userData->app_data;
And I want to add to this array e.g.
$encryptedData['new'] = 'axy';
$encryptedData['time'] = time();
I then update the model and save it:
$userData->app_data = $encryptedData;
$userData->save();
However, here is where the problem starts. The model does not update. It remains as if nothing happens. Hence if I refresh(), I get the same data as if I had never added the two new entries. When I log it, it looks like this:
Array
(
[token] => xyz
[access_token] => abc
)
After the addition of two new entries:
Array
(
[token] => xyz
[access_token] => abc
[new] => 'axy'
[time] => 1234
)
And after the save() and refresh():
Array
(
[token] => xyz
[access_token] => abc
)
The model looks like this:
namespace App;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Support\Facades\Crypt;
use Illuminate\Contracts\Encryption\DecryptException;
class UserData extends Model
{
protected $fillable = [
'user_id', 'app_data'
];
protected $casts = [
'user_id' => 'int',
'app_data' => 'array'
];
public function getAppDataAttribute($value)
{
try {
return decrypt($value);
}
catch (DecryptException $e) {
return $value;
}
}
public function setAppDataAttribute($value)
{
$this->attributes['app_data'] = encrypt($value);
}
}
Why are my additions to the array not being saved?
Edit: The strangeness continues
If I call:
UserData::where('id', $userData->id)->update(['app_data' => $encryptedData]);
Then the model does update and does not encrypt, HOWEVER, when I refresh and log the new 'app_data' field, it is returned as a JSON string and not an array as before. I need to cast/decode it to an array each time I want to use it.
Couple of things to look for.
1) The Laravel encrypter uses the app key. Make sure you have one in your .env file. If not, run php artisan key:generate
2) I assume the array is correctly formatted like this:
Array
(
'token' => 'xyz', // You have a = here and no commas after any other value
'access_token' => 'abc'
)
3) Depending on what you are storing this as, you can test by serializing the array before encrypting it:
$arr = serialize($encryptedData); // After you have added new data to the array
$userData->app_data = $arr;
$userData->save();
This is automatic in Laravel, but may give you a help hunting the bug. Test with your mutator using encryptString() and manually unserialize / decryptString() to see if any odd behavior by stepping through the values as they are mutated.

Use Laravel seed and sql files to populate database

I got several .sql files of countries, states and cities of the world from github. How can I run them with Laravel's seed files to populate those tables in my database?
Add DB::unprepared() to the run method of DatabaseSeeder.
Run php artisan db:seed at the command line.
class DatabaseSeeder extends Seeder {
public function run()
{
Eloquent::unguard();
$this->call('UserTableSeeder');
$this->command->info('User table seeded!');
$path = 'app/developer_docs/countries.sql';
DB::unprepared(file_get_contents($path));
$this->command->info('Country table seeded!');
}
}
I found a package that creates seed files from database tables and rows. It currently supports Laravel 4, 5, 6, 7, 8 and 9:
https://github.com/orangehill/iseed
In the end, it's basically as easy as this:
php artisan iseed my_table
or for multiple occasions:
php artisan iseed my_table,another_table
As used by other answers, DB::unprepared does not work with more complex SQL files.
Another better solution would be to use the MySQL cli directly inside a process:
$process = new Process([
'mysql',
'-h',
DB::getConfig('host'),
'-u',
DB::getConfig('username'),
'-p' . DB::getConfig('password'),
DB::getConfig('database'),
'-e',
"source path/to/schema.sql"
]);
$process->mustRun();
2022 Simplified answer from Andrew Koper :
class WhateverSeeder extends Seeder
{
/**
* Run the database seeds.
*
* #return void
*/
public function run()
{
$file_path = resource_path('sql/whatever.sql');
\DB::unprepared(
file_get_contents($file_path)
);
}
}
Fun fact: 60,000 rows took me 50s to import from JSON file where this is 400ms.
#Andre Koper solutions is understandable, but sadly it doesn't work for me.
This one is a bit confusing but atleast works for me.
So instead of using DB::unprepared, I use this:
// DatabaseSeeder.php
class DatabaseSeeder extends Seeder {
public function run()
{
// Set the path of your .sql file
$sql = storage_path('a_id_territory.sql');
// You must change this one, its depend on your mysql bin.
$db_bin = "C:\wamp64\bin\mariadb\mariadb10.3.14\bin";
// PDO Credentials
$db = [
'username' => env('DB_USERNAME'),
'password' => env('DB_PASSWORD'),
'host' => env('DB_HOST'),
'database' => env('DB_DATABASE')
];
exec("{$db_bin}\mysql --user={$db['username']} --password={$db['password']} --host={$db['host']} --database {$db['database']} < $sql");
}
}
Then while migrating database just add --seed
php artisan migrate:refresh --seed
or
php artisan migrate:fresh --seed
Tested on Laravel 7.0.x

Cache files permissions fix for Kohana Twig module

How to configure Kohana + Twig module so the Twig will set "writable by all" permissions on all of it's cache directory and it's descendant files?
So, for example, when I run my application through the Apache module (mod_php) and cache file owner is apache (or httpd) user, I will be able to remove cache files (to clean the cache or completely remove whole application) using regular user and ssh access.
I'm able to do it with Kohana's cache, but Twig's cache is created somehow differently.
It's not very easy, but not too complicated either. I have achieved state presented below by trial-and-error method.
Create a class that inherits from Twig_Cache_Filesystem and will be used instead of it. Check this out:
<?php
namespace Application\Twig;
class Cache_Filesystem extends \Twig_Cache_Filesystem
{
public function write($key, $content)
{
$old = umask(0000);
parent::write($key, $content);
umask($old);
}
}
Note, that this class must have it's name unique, so it is a good idea to namespace it. Also, it must be accessible to other code, so consider using composer's autoloading feature.
This is the fix itself, rest of the guide is just the way of implementing it into Kohana+Twig ecosystem.
Copy Twig.php from modules/kohana-twig/classes/Twig.php into your application's directory, i.e. application/classes/Twig.php (thank you Kohana's Cascading Filesystem!)
Modify a bit newly copied file, to let Twig_CacheInterface instance be passed in the config file (application/config/twig.php) instead of just a simple string (specifying to the Twig's cache directory). Take a look of my example:
<?php defined('SYSPATH') or die('No direct script access.');
class Twig extends Kohana_Twig
{
/**
* Initialize the Twig module
*
* #throws Kohana_Exception
* #return bool
*/
public static function init()
{
$path = Kohana::$config->load('twig.environment.cache');
if (is_string($path)) {
return parent::init();
} else if ($path instanceof Twig_CacheInterface) {
return true;
}
throw new Kohana_Exception('Twig cache could not be initialized');
}
}
In configuration file for kohana-twig module, i.e. application/config/twig.php (if not yet copied from module to your application, do it now), define environment.cache key like this:
return array(
'loader' => array(
'extension' => 'twig',
'path' => 'views',
),
'environment' => array(
'auto_reload' => (Kohana::$environment >= Kohana::TESTING),
'autoescape' => true,
'base_template_class' => 'Twig_Template',
// Following line is related to this issue and fix:
'cache' => new \Application\Twig\Cache_Filesystem(APPPATH . 'cache/twig'),
'charset' => 'utf-8',
'optimizations' => - 1,
'strict_variables' => false,
),
'functions' => array(),
'filters' => array(),
'tests' => array(),
}
This works for me. Hopefully it will help someone struggling with similar problem.

Laravel4 seed specific environment

We're are developing multiple applications based on Laravel 4. These applications run on the same webserver.
The Laravel4 environment detection is based on the hostname which sucks because we have multiple applications on the same machine.
We created a work-around in the detection area so that it will set the environment based on the url.
We run the artisan --env=my_env migrate command when we update the applications DB. The problem is in the seeding, the seeding command doesn't have a env option so it will try to seed the db based on the hostname wich will not be correct.
I'm trying all day to find a solution but I can't find any on the Internet and my attempts to build a new command is just taking too much time and energy.
Does someone knows how to set the environment when seeding?
PS: I run the commands on the server through Grunt and I know the environment -inject it into the command-.
You pointed it very well, Laravel environment guessing sucks the way we use to use it, but you can change that:
This is how I do set my environment flawlessly, so I don't have to deal with hostnames and still don't get my local environment conflict with staging and production.
Create a .environment file in the root of your application and define your environment and add your sensitive information to it:
<?php
return array(
'APPLICATION_ENV' => 'development', /// this is where you will set your environment
'DB_HOST' => 'localhost',
'DB_DATABASE_NAME' => 'laraveldatabase',
'DB_DATABASE_USER' => 'laraveluser',
'DB_DATABASE_PASSWORD' => '!Bassw0rT',
);
Add it to your .gitignore file, so you don't risk having your passwords sent to Github or any other of your servers.
Right before $app->detectEnvironment, in the file bootstrap/start.php, load your .environment file to PHP environment:
foreach(require __DIR__.'/../.environment' as $key => $value)
{
putenv(sprintf('%s=%s', $key, $value));
}
And then you just have to use it:
$env = $app->detectEnvironment(function () {
return getenv('APPLICATION_ENV'); // your environment name is in that file!
});
And it will work everywhere, so you don't need to have separate dirs for development and production anymore:
<?php
return array(
'connections' => array(
'postgresql' => array(
'driver' => 'pgsql',
'host' => getenv('DB_HOST'),
'database' => getenv('DB_DATABASE_NAME'),
'username' => getenv('DB_DATABASE_USER'),
'password' => getenv('DB_DATABASE_PASSWORD'),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
),
),
);
Note that I don't set a fallback:
return getenv('APPLICATION_ENV') ?: 'local';
Because I want it to fail on every server I deploy my app to, to never forget configuring my environment on them.
Then you just have to select the environment in your DatabaseSeeder class:
public function run()
{
if( App::environment() === 'development' )
{
$this->call('UserTableSeeder');
}
}
I thought about the situation some days and got to the conclusion that what we're trying to do isn't the correct way.
The correct way would be that every application has his own config files -with the different envs-. This way the resolve function of Laravel works fine.
The situation now is that we have multiple clients within one application and strore does -clients- configuration files within one application. In this case the hostname resolve will return the one client's -every time the same client- config beacuse the clients applications run on the same machine.
Our solution
We are going to write a deployment script for the different clients so that every client has his own application with their configs only (copy application, copy/overwrite client config into app).
Work-around
The answer of #Antonio Carlos Ribeiro works offcourse but had to much impact on our application. We use the different environments and with this solution we had to use the same user/pass info on all environments or provide a different .environment file.
I wrote an Artisan command to make our deployment work for the moment. This command can seed a database with the configuration of the provided environment (php artisan db:seed_env my_env).
use Illuminate\Console\Command;
use Illuminate\Config\Repository;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Input\InputArgument;
class SeedEnvironmentDb extends Command {
/**
* The console command name.
*
* #var string
*/
protected $name = 'db:seed_env';
/**
* The console command description.
*
* #var string
*/
protected $description = 'Seed a database with the configuration of the environment';
/**
* Create a new command instance.
*
* #return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* #return mixed
*/
public function fire()
{
$cmd = $this;
$app = App::make('app');
// force the environment to the given one.
$env = $app->detectEnvironment(function() use ($cmd) {
return $cmd->argument('environment');
});
// create new config with the correct environment and overwrite the current one.
$app->instance('config', $config = new Repository(
$app->getConfigLoader(), $env
));
// trigger the db seed (now with the correct environment)
$this->call('db:seed');
}
/**
* Get the console command arguments.
*
* #return array
*/
protected function getArguments()
{
return array(
array('environment', InputArgument::REQUIRED, 'The environment to seed.'),
);
}
/**
* Get the console command options.
*
* #return array
*/
protected function getOptions()
{
return array();
}
}

How to access DB config in CodeIgniter?

In my application, I need to know what values are assigned to the DB config items such as database, username, etc. How do I access those information?
I don't have enough rep to comment on Matt Browne's correct answer but just adding a bit incase anyone forgets...
load the db driver like so first:
$this->load->database();
then you can easily access what you need:
$this->db->hostname
$this->db->username
$this->db->password
$this->db->database
Pretty much all the config values are accessible via $this->db (take a look at system/database/DB_driver.php).
That's what worked for me...none of the other suggestions here did.
As an example
$config = [
'host' => $this->db->hostname,
'port' => '3306',
'username' => $this->db->username,
'password' => $this->db->password,
'database' => $this->db->database
];
In case you have multiple database connection groups defined in config/database.php, for eg :
$db['dbname']['hostname'] = "localhost";
$db['dbname']['username'] = "root";
$db['dbname']['password'] = "root";
$db['dbname']['database'] = "web_dbname";
$db['dbname_readonly']['hostname'] = "localhost";
$db['dbname_readonly']['username'] = "root";
$db['dbname_readonly']['password'] = "root";
$db['dbname_readonly']['database'] = "web_dbname_readonly";
If you want to use the connection params of any particular db in a controller or model:
$db = $this->load->database('dbname');
If you want to use in a helper or library :
$ci = &get_instance();
$db = $ci->load->database('dbname');
The connection params will be available as $db->hostname, $db->username etc.
I stumbled across this, looking for a way to find all of the DB settings. Was not able to find a solution on-line, but found some useful code in system/database/DB.php
Here's my approach, get the contents of the entire database config:
if ( ! file_exists($f = APPPATH.'config/'.ENVIRONMENT.'/database.php')
&& ! file_exists($f = APPPATH.'config/database.php'))
{
show_error('The configuration file database.php does not exist.');
}
include($f);
// Use a NEW variable.
// Because $db is a reserved name!!
$db_settings = $db;
foreach($db_settings as $key => $value) {
// .. do something with .. $this->database->load($key);
// .. do something with .. $value['database'];
// .. do something with .. $value['password'];
}
You should be able to get at your configuration setting like this :
$this->config['env']
You can retrieve it with this:
http://codeigniter.com/user_guide/libraries/config.html
$this->config->item('item name');

Resources