I need some statement (that's exist in transaction block) to be executed directly to Database without transaction , example :
DB::beginTransaction();
//query A : insert or update to some tables in transaction
//**I need some of the result of (query A) to be saved in Database without transaction**
//insert or update to some tables in transaction
DB::commit();
[EDIT] After reading your question again, it seems i may have misundersood some parts. If by saying
I need some of the result of (query A) to be saved in Database without transaction
You mean "use the values inserted within the transaction before the transaction end", i guess it's not possible. You can still use my answer to insert or update data instantaneously, but you'll not be able to read something you inserted in the transaction before the transaction end.
You can duplicate the connection you are using for the transaction in database.php. For this example, let's assume you call it 'mysql_outside_transaction'. You should have something like this in the database.php file, after your default connection:
'mysql_outside_transaction' => [
'driver' => 'mysql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '3306'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'unix_socket' => env('DB_SOCKET', ''),
'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',
'prefix' => '',
'strict' => true,
'engine' => null,
],
From there, since you have a second connection to the same database available, you can use it to save the data you need directly within the transaction.
Again, to make an example, if you have a instance of $modelA, you can tell it to use your second connection: $modelA->setConnection('mysql_outside_transaction');. From there, every database operation originating from this instance will be done outside your transaction (because it'll use the second connection you configured and not the one used for the transaction).
Don't forget you can set the connection to use in many ways. It could be directly on model class attributes (protected $connection = 'your_connection';), with the DB facade (DB::connection('your_connection');) and probably in other ways i don't think about right now :)
Related
I am trying to deploy a Laravel application to Heroku and connect it with a database which has already been deployed to Azure.
But I am having error "unsupported driver[https]".
My database.php:
<?php
use Illuminate\Support\Str;
return [
'default' => env('DB_CONNECTION', 'mysql'),
/
'mysql' => [
'driver' => 'mysql'
'url' => env('DATABASE_URL','https://firstsqlaap.scm.azurewebsites.net/phpMyAdmin/db_structure.php?server=1&db=localdb&token=51b0b3471e798a712e129bcd1ebe5b01'),
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '53082'),
'database' => env('DB_DATABASE', 'localdb'),
'username' => env('DB_USERNAME', 'user'),
'password' => env('DB_PASSWORD', 'pass'),
'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',
'options' => extension_loaded('pdo_mysql') ? array_filter([
PDO::MYSQL_ATTR_SSL_CA => env('MYSQL_ATTR_SSL_CA'),
]) : [],
],
];
My SESSION_DRIVER is set to database because when set to file it was saying 419 error. I do not have any migration files as my database is deployed to Azure.
How to resolve this issue?
This certainly isn't the right URL to use:
https://firstsqlaap.scm.azurewebsites.net/phpMyAdmin/db_structure.php
You appear to be pointing to an instance of phpMyAdmin. phpMyAdmin isn't a database server, it's a dataase client. It's a tool that you might use to interact with your database. You need to provide the URL to your actual database.
Your database URL should look more like this:
driver://username:password#host:port/database?options
For MySQL, driver:// is likely mysql://.
I don't have any MySQL databases running on Azure, but it looks like a real URL might be something like
mysql://user:password#your-database-instance.mysql.database.azure.com/your-database-name
Go into the Azure portal and navigate to your database instance. Then, in the left navigation panel, click on "Connection strings". The information you need should be there, though not in URL format. You can either build your own URL by plugging the right values in or use the individual settings in your config/database.php file.
I commented url and it work for me
We have application build in Php Laravel and for the database we use postgres sql. And also on top of postgres we have configure pgBouncer to limit the maximum number of connections on server side by managing a pool of idle connections that can be used by any applications.
Now, we face the issue with the boolean values (True(0),False(1)) used in the application (Php Laravel). It gives below error when any CRUD operation is performed. In the below error column "revoked" is boolean type.
column \"revoked\" is of type boolean but expression is of type integer
You will need to rewrite or cast the expression. (SQL: \"revoked\", \"created_at\") values (0, 2020-02-07 06:09:06)
Now after exploring, I came to know that boolean values needs to be consider to be string with the pgBouncer. So I have made changes in the connection.php file, located in "\vendor\laravel\framework\src\Illuminate\Database". I have change the code to consider the boolean value as mentioned below.
public function bindValues($statement, $bindings)
{
foreach ($bindings as $key => $value) {
//if(is_bool($value))
$statement->bindValue(
is_string($key) ? $key : $key + 1, $value,
//is_int($value) ? PDO::PARAM_INT : PDO::PARAM_STR
is_int($value) ? PDO::PARAM_INT : is_bool($value) ? PDO::PARAM_STR : PDO::PARAM_STR
);
}
}
After the above changes the error with the boolean values was solved.
But, now I am facing strange issues on the server, when I check the database log error I consistently get the below error.
ERROR: prepared statement "pdo_stmt_00000001" already exists
STATEMENT: set names 'utf8'
ERROR: prepared statement "pdo_stmt_00000001" does not exist
STATEMENT: DEALLOCATE pdo_stmt_00000001
It really was strange, and after exploring the internet I have done the below changes in my database.php file, to disable the prepare statements.
'pgsql' => [
'driver' => 'pgsql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '5432'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
\PDO::ATTR_EMULATE_PREPARES => true
]
]
The reason behind seeting ATTR_EMULATE_PREPARES => true is becasue I have set "Transaction" mode in "pgbouncer.ini" file.
Now, to make prepared statements work in Transaction mode would need PgBouncer to keep track of them internally, which it does not do. So only way to keep using PgBouncer in this mode is to disable prepared statements in the client, which in my case is PHP Laravel and I have already handle it in the "database.php" file when the connection is made as shown in above code.
I have tried all the options, which are given in http://www.pgbouncer.org/faq.html#how-to-use-prepared-statements-with-transaction-pooling but it doesnot solve the prepare statment error shown in the database log.
ERROR: prepared statement "pdo_stmt_00000001" already exists
STATEMENT: set names 'utf8'
ERROR: prepared statement "pdo_stmt_00000001" does not exist
STATEMENT: DEALLOCATE pdo_stmt_00000001
Please guide me on the same and what further settings are required for the error. Those errors are on the client production server and we cannot go ahead with those errors in production server.
Please give me your valuable feedback at the earliest as I am facing the issue since 5 days and try with all the options that come across.
Thanks!
1) First, you need to change the PDO option you are giving in the options in the pgsql array of your database.php the right way is as given below.
'pgsql' => [
'driver' => 'pgsql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '5434'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
PDO::ATTR_EMULATE_PREPARES => true
]
]
2) Second, and the most important thing is to make sure that you use set the "ATTR_EMULATE_PREPARES" to "true" with each database connection you try to connect in your Database.php file.
For example,
'test' => [
'driver' => 'pgsql',
'host' => env('test', '127.0.0.1'),
'port' => env('test', '5434'),
'database' => env('DB_TEST_DATABASE', 'test'),
'username' => env('DB_USERNAME', 'test'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
PDO::ATTR_EMULATE_PREPARES => true
]
],
'test1' => [
'driver' => 'pgsql',
'host' => env('test1', '127.0.0.1'),
'port' => env('test1', '5434'),
'database' => env('DB_TEST1_DATABASE', 'test1'),
'username' => env('DB_USERNAME', 'test'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
PDO::ATTR_EMULATE_PREPARES => true
]
]
Please make sure to use the "ATTR_EMULATE_PREPARES" to true for each database connection you make in your application, in your comments you make connection with only "pgsql" which emphasis for postgres sql connection only, and not with the database that your application communicates which is in postgres.
Hope this helps you to resolve your query. Enjoy!!!
First you never need to modify the vendor code, instead you can use attribute casting from your model.
From laravel.com/docs/master/eloquent-mutators#attribute-casting
The $casts property on your model provides a convenient method of converting attributes to common data types. The $casts property should be an array where the key is the name of the attribute being cast and the value is the type you wish to cast the column to. The supported cast types are: integer, real, float, double, decimal:, string, boolean, object, array, collection, date, datetime, and timestamp. When casting to decimal, you must define the number of digits (decimal:2).
To demonstrate attribute casting, let's cast the is_admin attribute,
which is stored in our database as an integer (0 or 1) to a boolean
value:
So in your case you will need to cast revoked to bool by adding to your Eloquent model the $casts property as follows:
<?php
namespace App;
use Illuminate\Database\Eloquent\Model;
class YourModel extends Model
{
/**
* The attributes that should be cast to native types.
*
* #var array
*/
protected $casts = [
'revoked' => 'boolean',
];
}
And for your pgBouncer issue it seems that pgBouncer have an internal issue with transaction pooling and prepared statements,
From: stackoverflow.com/a/7612639/7047493
This turned out to be a pgBouncer issue that occurs when using anything other than session pooling. We were using transaction pooling, which apparently can't support prepared statements. By switching to session pooling, we got around the issue.
None of the previous answers fully worked in our case. In our setup (Laravel + PostgreSQL + pgBouncer), we had enabled these 2 settings in the database.php file. The goal was to make our php backend compatible with pgBouncer and that's what we had done:
// database.php
'options' => array(
PDO::ATTR_EMULATE_PREPARES => true
),
'binary_parameters' => 'yes', // not sure if this one is necessary
These 2 settings partially worked, meaning we were able to run our backend without getting the prepared statement does not exist anymore. Unfortunately for us, we were then getting the datatype mismatch: 7 ERROR: column “xxx” is of type boolean but expression is of type integer just like Nileshsinh Rathod.
Hopefully for us, we came across this post on Github which fixed everything for us. The goal is to override the default PostgresConnector.
And here is a recap of what we did:
Add these 3 files in our project:
https://github.com/umbrellio/laravel-pg-extensions/blob/master/src/Connectors/ConnectionFactory.php
https://github.com/umbrellio/laravel-pg-extensions/blob/master/src/UmbrellioPostgresProvider.php
https://github.com/umbrellio/laravel-pg-extensions/blob/master/src/PostgresConnection.php
Within this file, we only kept the bindValues and prepareBindings functions.
Then, in our config/app.php, we registered the PostgresProvider like so
'providers' => [
App\Providers\ScPostgresProvider::class,
],
Finally, we commented out this line in our AppServiceProvider file in order to make sure only the new one would be registered
public function register()
{
// not used anymore since we use our our own connector
// $this->app->bind('db.connector.pgsql', OldPostgresConnector::class);
}
Thanks a lot to the post of Umbrellio team on Github and hope this answer will help others!
I provisioned two servers with Laravel Forge. I'm running an Application on Server A and I wanna access a database which is on Server B.
So I configured my database like this:
'pgsql' => [
'driver' => 'pgsql',
'host' => env('EXTERNAL_DB_HOST', '138.31.32.33'),
'database' => env('EXTERNAL_DB_DATABASE', 'forge'),
'username' => env('EXTERNAL_DB_USERNAME', 'forge'),
'password' => env('EXTERNAL_DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
],
I filled in the credentials of Server B. Then I try running Tinker which uses the Connection:
>>> $d = \App\Foobar::first();
Illuminate/Database/QueryException with message 'SQLSTATE[08006] [7] timeout expired (SQL: select * from "foobars" where "foobars"."deleted_at" is null limit 1)'
It gives the error:
SQLSTATE[08006] [7] timeout expired
I can access the Server B via Server A: ssh forge#138.31.32.33. I got access to that Server B.
But I'm stil getting the timeout. What am I missing? How can I access the "external" database ?
You have to setup your Database on Server B in that way, that it can be accessed from "external". This can be done in mysql by setting up the priviliges on this mysql table and user. See the mysql manual for this:
https://www.cyberciti.biz/tips/how-do-i-enable-remote-access-to-mysql-database-server.html
https://www.digitalocean.com/community/tutorials/how-to-allow-remote-access-to-mysql
https://dev.mysql.com/doc/refman/8.0/en/access-control.html
When running phpMyAdmin, there is a section for this if you have root access to the db
I hope, this will lead you in the right way ;)
I have a large project which will have each customer on their own separate database. To get this to work we use a custom .env-loader that loads each customers .envby checking the customers subdomain (unique to each customer).
However, of course this doesn't work with artisan commands. For instance, when I want to migrate, I will need to migrate all databases at once. So I've set up an Artisan command that fetches the .env-files and loop through them and then calls the default artisan migrate. But it is not working as expected.
I've tried everything; for instance:
$dotenv = new Dotenv('/env', '.test.env');
$dotenv->overload();
And:
app()->useEnvironmentPath('/env');
app()->loadEnvironmentFrom('.test.env');
And even:
config('database.connections.mysql.database', 'test_database');
As soon as I run $this->call('migrate'); the app defaults to the default .env and ignores all customizations at runtime. Does anyone have an idea on how I can overload the migration commands choice of database?
Note: I know that I can manually setup multiple connections in config/database.php (for instance like: Overriding Default Laravel database configuration for artisan migrate commands), however, image a few dozen customers and this would not be viable.
I had to do something similar with SQLite database that were being created by the console commands, and the only way I could get the migrations to run was by creating a database config on the fly:
Config::set('database.connections.'.$config_key, array(
'driver' => 'sqlite',
'database' => storage_path($database_name),
'prefix' => '',
));
And then I would call the migrate command:
Artisan::call('migrate', [
'--database' => $config_key,
'--path' => 'database/offline/'.$type.'/migrations',
]);
After a whole lot of issues I was able to sort it this way;
In Laravel 5 there seem to be a difference in Config::set(), config('config',['key' => 'value]) and config()-set('config', ['key' => 'value']).
After a lot of testing different variant we managed to get a solution this way;
$connection = 'connection';
$iterator = 0;
foreach ($files as $file) {
App::useEnvironmentPath('/env');
App::loadEnvironmentFrom('.file.env');
// Create a new connection "on the fly"
config()->set('database.connections.' . $connection . '_' . $iterator, [
'driver' => 'mysql',
'host' => env('DB_HOST'),
'database' => env('DB_DATABASE'),
'username' => env('DB_USERNAME'),
'password' => env('DB_PASSWORD'),
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => '',
'strict' => false,
]);
// Call regular migration command
$this->call('migrate', ['--force' => true, '--database' => $connection . '_' . $iterator]);
$iterator++;
}
This manages to set multiple new connections to the MySQL-database, and then seed each one of them.
Thanks to #David Allen here for the inspiration.
I am working on laravel 4 app, In that i wan to allow the user to register for the app and when they register the app should allow a new empty database to the registered user,I am using mysql for backend I know the following code to set the database connection
Config::set('database.connections', ConnectionArray);
And to set it as default
Config::set('database.default', ConnectionKey);
To connect another database manually i can use this way
'mysql' => array(
'driver' => 'mysql',
'host' => 'localhost',
'database' => 'database',
'username' => 'root',
'password' => '123456',
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => '',
),
'mysql_tenant1' => array(
'driver' => 'mysql',
'host' => 'localhost',
'database' => 'tenant1',
'username' => 'root',
'password' => '123456',
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => '',
),
But i want the app should create database dynamically...
Creating the Database itself will be relatively simple. The complexity comes in associating those databases to users somehow, and then pulling those database names and using them in connections. This should help:
In terms of overall strategy, consider creating a new repository with a few functions:
RegisterUser() - This registers the user with username/password.
CreateDatabase($name , $user_id) - This should create your
database and store it's name in a table, and then assign it to the User ID you created in Register User.
You can run raw SQL queries in Laravel with DB::statement() (i.e - ``DB::statement('create database' . $name);`, so stick that in your CreateDatabase function. I tested this locally and it works just fine.
NOTE: You'll want to do some validation to make sure the database doesn't already exist, or your user is going to (obviously) get an error.
Then, you can pull the Database name from the table by User ID and create connections with it as described in the below SO!
Laravel 4: Multiple Tenant Application, each tenant it's own database and one global database
One thing - when you make a change to your database structure, you're going to have to update each database individually. Note that Artisan allows for this with the --database="database_name" option.
Just have a user_id field in whatever table(DB from what I think you're describing) you want, and use relationships to make it work, a whole bunch of fields in one table(or more than one table) may be FAR more efficient than what you think.