Transaction doesn't work when instantiating eloquant from a class - laravel

I try to use transactions OUTSIDE laravel. it works when I include the db instantiation right in my index file as is:
use Illuminate\Database\Capsule\Manager as Db;
$db = new Db;
$db->addConnection( [
'driver' => Settings::DATABASE_DRIVER,
'host' => Settings::DATABASE_HOST,
'database' => Settings::DATABASE_NAME,
'username' => Settings::DATABASE_USERNAME,
'password' => Settings::DATABASE_PASSWORD,
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => ''
] );
# Make this Capsule instance available globally via static methods... (optional)
$db->setAsGlobal();
I can then elsewhere use code like this:
Db::connection()->beginTransaction();
# Create blog post in the database and return its id
$blogPostRecordId = ( new BlogDbModel() )->create( $_POST );
Db::connection()-> rollBack();
it will correctly work and rollback: no row is created in db.
However, If I get an instance of the db from a class, it won't work:
class DbSql
{
/**
* Use the eloquent query builder and orm. Bypass PDO interface.
* #return Capsule
*/
public function db()
{
$capsule = new Capsule;
$capsule->addConnection( [
'driver' => Settings::DATABASE_DRIVER,
'host' => Settings::DATABASE_HOST,
'database' => Settings::DATABASE_NAME,
'username' => Settings::DATABASE_USERNAME,
'password' => Settings::DATABASE_PASSWORD,
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => ''
] );
# Make this Capsule instance available globally via static methods... (optional)
$capsule->setAsGlobal();
// Setup the Eloquent ORM... (optional; unless you've used setEventDispatcher())
//$capsule->bootEloquent();
return $capsule;
}
}
and then use
( new DbSql() )->db()->getConnection()->beginTransaction();
# Create blog post in the database and return its id
$blogPostRecordId = ( new BlogDbModel() )->create( $_POST );
( new DbSql() )->db()->getConnection()->rollBack();
it simply won't work and the transaction is ignored. Why does getting the db instance from a class instantiation make the process fail ? I would prefer to use an instance as needed.

Solution: the db instance must be exactly the same everywhere the transaction is needed:
$dbInstance = ( new DbSql() )->db();
Then use $dbInstance wherever the transaction is to be followed:
$dbInstance->connection()->beginTransaction();
as well as with any db eloquent operation. So pass the $dbInstance as needed. The good thing is that it can be anywhere in you code: in your models, controllers....
Then finish:
$dbInstance->connection()->commit();
If the system is not able to reach this last line, nothing will be committed to the db.

Related

Query parameter binding issue with illuminate/database and illuminate/queue

I'm using Illuminate\Queue outside of a Laravel app inside an add-on for a CMS. So the only instances of Laravel or Illuminate are these packages that I've required:
"illuminate/queue": "^8.83",
"illuminate/bus": "^8.83",
"illuminate/contracts": "^8.83"
I'm first trying to use the Database for the queue as the default driver since the CMS is database driven, then provide options to SQS etc. I've setup everything so the migrations create my queue tables and everything seems to be wired together when I make the following call to push something to the queue.
/** #var \Illuminate\Queue\QueueManager $queue */
$queue->push('test', ['foo' => 'bar']);
Then it ends in the following error. The parameter bindings are not working or something. It's leaving the ? in the values list.
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '"exp_dgq_jobs" ("queue", "attempts", "reserved_at", "available_at", "created_at"' at line 1 (SQL: insert into "exp_dgq_jobs" ("queue", "attempts", "reserved_at", "available_at", "created_at", "payload") values (default, 0, ?, 1674567590, 1674567590, {"uuid":"6bf7a17e-dda3-4fed-903a-8714e5a2d146","displayName":"test","job":"test","maxTries":null,"maxExceptions":null,"failOnTimeout":false,"backoff":null,"timeout":null,"data":{"foo":"bar"}}))
I've step debugged the whole request and it feels like a bug, but then again this is really the first time I've used Laravel or one of it's packages, so maybe I'm missing something? This function explicitly sets reserved_at to null, and the Connection->prepareBindings() method doesn't do anything with the ?, it just leaves it as that value, so the query fails.
protected function buildDatabaseRecord($queue, $payload, $availableAt, $attempts = 0)
{
return ['queue' => $queue, 'attempts' => $attempts, 'reserved_at' => null, 'available_at' => $availableAt, 'created_at' => $this->currentTime(), 'payload' => $payload];
}
What am I missing? Everything just looks right to me an I'm kind of at a loss. I'm making this with PHP 7.4 in mind (for the time being). Maybe I'll try 8.1 to see if that changes anything with the Illuminate packages. Using MySQL 8 too.
Update: potentially relevant screenshot just before the error.
Update 2: I tried PHP 8.1 and latest Laravel 9 packages, didn't make a difference.
For more clarity on how I"m creating my QueueManager:
<?php $queue = new Queue;
$queue->addConnection([
'driver' => 'database',
'table' => ee('db')->dbprefix . 'dgq_jobs',
'queue' => 'default',
'retry_after' => 90,
'after_commit' => false,
]);
$databaseConfig = $provider->make('DatabaseConfig');
$queue->addConnector('database', function () use ($databaseConfig) {
$pdo = new PDO(
sprintf('mysql:host=%s; dbname=%s', $databaseConfig['host'], $databaseConfig['database']),
$databaseConfig['username'],
$databaseConfig['password']
);
$connection = new Connection($pdo);
$connectionResolver = new ConnectionResolver(['default' => $connection]);
$connectionResolver->setDefaultConnection('default');
return new DatabaseConnector($connectionResolver);
});
return $queue->getQueueManager();
I was able to reproduce the error you were seeing. I haven't looked too deeply but I think it may be due to the PDO object not setting up the connection exactly as the Illuminate Queue library expects.
This modification to using the Illuminate\Database library to create the connection solved the issue in my test environment:
$database = new \Illuminate\Database\Capsule\Manager;
$queue = new \Illuminate\Queue\Capsule\Manager;
$database->addConnection([
'driver' => 'mysql',
'host' => 'localhost',
'database' => 'db_name',
'username' => 'username',
'password' => '',
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => '',
]);
$queue->addConnector('database', function () use ($database) {
$connection = $database->getConnection();
$connectionResolver = new \Illuminate\Database\ConnectionResolver(['default' => $connection]);
$connectionResolver->setDefaultConnection('default');
return new \Illuminate\Queue\Connectors\DatabaseConnector($connectionResolver);
});
$queue->addConnection([
'driver' => 'database',
'table' => 'jobs_table',
'queue' => 'default',
'retry_after' => 90,
'after_commit' => false,
]);
$queue->getQueueManager()->push('SendEmail', ['message' => 'test']);

Laravel data mismatch error while using \PDO::ATTR_EMULATE_PREPARES => true

We have application build in Php Laravel and for the database we use postgres sql. And also on top of postgres we have configure pgBouncer to limit the maximum number of connections on server side by managing a pool of idle connections that can be used by any applications.
Now, we face the issue with the boolean values (True(0),False(1)) used in the application (Php Laravel). It gives below error when any CRUD operation is performed. In the below error column "revoked" is boolean type.
column \"revoked\" is of type boolean but expression is of type integer
You will need to rewrite or cast the expression. (SQL: \"revoked\", \"created_at\") values (0, 2020-02-07 06:09:06)
Now after exploring, I came to know that boolean values needs to be consider to be string with the pgBouncer. So I have made changes in the connection.php file, located in "\vendor\laravel\framework\src\Illuminate\Database". I have change the code to consider the boolean value as mentioned below.
public function bindValues($statement, $bindings)
{
foreach ($bindings as $key => $value) {
//if(is_bool($value))
$statement->bindValue(
is_string($key) ? $key : $key + 1, $value,
//is_int($value) ? PDO::PARAM_INT : PDO::PARAM_STR
is_int($value) ? PDO::PARAM_INT : is_bool($value) ? PDO::PARAM_STR : PDO::PARAM_STR
);
}
}
After the above changes the error with the boolean values was solved.
But, now I am facing strange issues on the server, when I check the database log error I consistently get the below error.
ERROR: prepared statement "pdo_stmt_00000001" already exists
STATEMENT: set names 'utf8'
ERROR: prepared statement "pdo_stmt_00000001" does not exist
STATEMENT: DEALLOCATE pdo_stmt_00000001
It really was strange, and after exploring the internet I have done the below changes in my database.php file, to disable the prepare statements.
'pgsql' => [
'driver' => 'pgsql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '5432'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
\PDO::ATTR_EMULATE_PREPARES => true
]
]
The reason behind seeting ATTR_EMULATE_PREPARES => true is becasue I have set "Transaction" mode in "pgbouncer.ini" file.
Now, to make prepared statements work in Transaction mode would need PgBouncer to keep track of them internally, which it does not do. So only way to keep using PgBouncer in this mode is to disable prepared statements in the client, which in my case is PHP Laravel and I have already handle it in the "database.php" file when the connection is made as shown in above code.
I have tried all the options, which are given in http://www.pgbouncer.org/faq.html#how-to-use-prepared-statements-with-transaction-pooling but it doesnot solve the prepare statment error shown in the database log.
ERROR: prepared statement "pdo_stmt_00000001" already exists
STATEMENT: set names 'utf8'
ERROR: prepared statement "pdo_stmt_00000001" does not exist
STATEMENT: DEALLOCATE pdo_stmt_00000001
Please guide me on the same and what further settings are required for the error. Those errors are on the client production server and we cannot go ahead with those errors in production server.
Please give me your valuable feedback at the earliest as I am facing the issue since 5 days and try with all the options that come across.
Thanks!
1) First, you need to change the PDO option you are giving in the options in the pgsql array of your database.php the right way is as given below.
'pgsql' => [
'driver' => 'pgsql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '5434'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
PDO::ATTR_EMULATE_PREPARES => true
]
]
2) Second, and the most important thing is to make sure that you use set the "ATTR_EMULATE_PREPARES" to "true" with each database connection you try to connect in your Database.php file.
For example,
'test' => [
'driver' => 'pgsql',
'host' => env('test', '127.0.0.1'),
'port' => env('test', '5434'),
'database' => env('DB_TEST_DATABASE', 'test'),
'username' => env('DB_USERNAME', 'test'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
PDO::ATTR_EMULATE_PREPARES => true
]
],
'test1' => [
'driver' => 'pgsql',
'host' => env('test1', '127.0.0.1'),
'port' => env('test1', '5434'),
'database' => env('DB_TEST1_DATABASE', 'test1'),
'username' => env('DB_USERNAME', 'test'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
PDO::ATTR_EMULATE_PREPARES => true
]
]
Please make sure to use the "ATTR_EMULATE_PREPARES" to true for each database connection you make in your application, in your comments you make connection with only "pgsql" which emphasis for postgres sql connection only, and not with the database that your application communicates which is in postgres.
Hope this helps you to resolve your query. Enjoy!!!
First you never need to modify the vendor code, instead you can use attribute casting from your model.
From laravel.com/docs/master/eloquent-mutators#attribute-casting
The $casts property on your model provides a convenient method of converting attributes to common data types. The $casts property should be an array where the key is the name of the attribute being cast and the value is the type you wish to cast the column to. The supported cast types are: integer, real, float, double, decimal:, string, boolean, object, array, collection, date, datetime, and timestamp. When casting to decimal, you must define the number of digits (decimal:2).
To demonstrate attribute casting, let's cast the is_admin attribute,
which is stored in our database as an integer (0 or 1) to a boolean
value:
So in your case you will need to cast revoked to bool by adding to your Eloquent model the $casts property as follows:
<?php
namespace App;
use Illuminate\Database\Eloquent\Model;
class YourModel extends Model
{
/**
* The attributes that should be cast to native types.
*
* #var array
*/
protected $casts = [
'revoked' => 'boolean',
];
}
And for your pgBouncer issue it seems that pgBouncer have an internal issue with transaction pooling and prepared statements,
From: stackoverflow.com/a/7612639/7047493
This turned out to be a pgBouncer issue that occurs when using anything other than session pooling. We were using transaction pooling, which apparently can't support prepared statements. By switching to session pooling, we got around the issue.
None of the previous answers fully worked in our case. In our setup (Laravel + PostgreSQL + pgBouncer), we had enabled these 2 settings in the database.php file. The goal was to make our php backend compatible with pgBouncer and that's what we had done:
// database.php
'options' => array(
PDO::ATTR_EMULATE_PREPARES => true
),
'binary_parameters' => 'yes', // not sure if this one is necessary
These 2 settings partially worked, meaning we were able to run our backend without getting the prepared statement does not exist anymore. Unfortunately for us, we were then getting the datatype mismatch: 7 ERROR: column “xxx” is of type boolean but expression is of type integer just like Nileshsinh Rathod.
Hopefully for us, we came across this post on Github which fixed everything for us. The goal is to override the default PostgresConnector.
And here is a recap of what we did:
Add these 3 files in our project:
https://github.com/umbrellio/laravel-pg-extensions/blob/master/src/Connectors/ConnectionFactory.php
https://github.com/umbrellio/laravel-pg-extensions/blob/master/src/UmbrellioPostgresProvider.php
https://github.com/umbrellio/laravel-pg-extensions/blob/master/src/PostgresConnection.php
Within this file, we only kept the bindValues and prepareBindings functions.
Then, in our config/app.php, we registered the PostgresProvider like so
'providers' => [
App\Providers\ScPostgresProvider::class,
],
Finally, we commented out this line in our AppServiceProvider file in order to make sure only the new one would be registered
public function register()
{
// not used anymore since we use our our own connector
// $this->app->bind('db.connector.pgsql', OldPostgresConnector::class);
}
Thanks a lot to the post of Umbrellio team on Github and hope this answer will help others!

Elequent / Illuminate Laravel Database LIKE operation not working

I'm using the Illuminate database manager from Laravel, which works pretty good except using the LIKE operation for now.
I have tried those options but got nothing:
function initConnection()
{
$capsule = new Capsule;
$capsule->addConnection([
'driver' => 'mysql',
'host' => $this->config['host'],
'database' => $this->config['database-name'],
'username' => $this->config['username'],
'password' => $this->config['password'],
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => ''
]);
$capsule->setAsGlobal();
}
And after initializing I tried:
function searchByName($word)
{
return Capsule::table($table)
->get()
->where('name', 'LIKE', '%'.$word.'%')
->first();
}
echo searchByName('John');
I also tried this option:
Capsule::table('table_name')
->select('SELECT * FROM table_name WHERE table_name.name LIKE "%John%"');
this also failed.
I can't find a documentation for using all the operations in Laravel.
you don't need to define the table in eloquent like this. your model will do that by itself if you follow the name convention or u can set table name in the model by :
class YourModelName extends Model
{
protected $table = 'your_table_name';
}
now query like this :
function searchByName($word) {
return Capsule::where('name', 'LIKE', '%'.$word.'%')->get();
}
this will return multiple value which will match the string.

Custom created_at attribute on a pivot Model

Situation: I have a model (let's call it Pivot) who is declared as follows:
class Pivot extends Model
{
public $timestamps = false;
.....
}
It has 5 fields: id, notification_id, device_id, created_at, read_at.
This model is used as the pivot table between Notification and Device:
// Notification.php
public function devices()
{
return $this->belongsToMany(Device::class)
->withPivot(['id','created_at','notification_id','device_id', 'read_at']);
}
public function pivots() {
return $this->hasMany(Pivot::class);
}
And i did the same inside the Device class.
My MySQL has a timezone (Europe\Rome) different then UTC and i can't change it because it is already used by different projects.
PROBLEM: How to setup the created_at field to work properly?
I tried to use default timestamps, it works BUT it will also add the updated_at column and I can't have it (it would add too many redundancy).
So i set public $timestamps = false; and i created the field in the database like that:
$table->timestamp('created_at')->default(DB::raw('CURRENT_TIMESTAMP'));
The problem of this one is that, as said earlier, MySQL has a wrong timezone (Europe\Rome), so when I read data from this table the created_at value is shifted by 2 hours from the original. This is a problem when i also set the read_at property, because it creates situations where the notification is read before the creation.
The last idea i had was to set the created_at inside a creating event. I followed this laracast guide but, apparently, if the data is inserted as a pivot table from a sync method, events are not called (they didn't worked for me).
Do you have other ideas? of course there are some ugly hacks that could work (like iterating on the created Pivot and set them manually), but i would like to use a better solution.
EDIT: the solution proposed by #Maraboc worked perfectly. I set the timezone property of MySQL config array in config/database.php:
'mysql' => [
'driver' => 'mysql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '3306'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'unix_socket' => env('DB_SOCKET', ''),
'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',
'prefix' => '',
'strict' => true,
'engine' => null,
'timezone' => '+00:00',
],
Use model events to set the timestamp when you are creating the model. You can set the field and timezone together.
class Pivot extends Model
{
public $timestamps = false;
protected static function boot()
{
static::creating(function($model) {
$model->create_at = Carbon::now('Europe\Rome');
});
parent::boot();
}
}
Don't forget the parent::boot() or else no traits will get booted on the model.

Dynamic env-files (multiple databases) and artisan commands

I have a large project which will have each customer on their own separate database. To get this to work we use a custom .env-loader that loads each customers .envby checking the customers subdomain (unique to each customer).
However, of course this doesn't work with artisan commands. For instance, when I want to migrate, I will need to migrate all databases at once. So I've set up an Artisan command that fetches the .env-files and loop through them and then calls the default artisan migrate. But it is not working as expected.
I've tried everything; for instance:
$dotenv = new Dotenv('/env', '.test.env');
$dotenv->overload();
And:
app()->useEnvironmentPath('/env');
app()->loadEnvironmentFrom('.test.env');
And even:
config('database.connections.mysql.database', 'test_database');
As soon as I run $this->call('migrate'); the app defaults to the default .env and ignores all customizations at runtime. Does anyone have an idea on how I can overload the migration commands choice of database?
Note: I know that I can manually setup multiple connections in config/database.php (for instance like: Overriding Default Laravel database configuration for artisan migrate commands), however, image a few dozen customers and this would not be viable.
I had to do something similar with SQLite database that were being created by the console commands, and the only way I could get the migrations to run was by creating a database config on the fly:
Config::set('database.connections.'.$config_key, array(
'driver' => 'sqlite',
'database' => storage_path($database_name),
'prefix' => '',
));
And then I would call the migrate command:
Artisan::call('migrate', [
'--database' => $config_key,
'--path' => 'database/offline/'.$type.'/migrations',
]);
After a whole lot of issues I was able to sort it this way;
In Laravel 5 there seem to be a difference in Config::set(), config('config',['key' => 'value]) and config()-set('config', ['key' => 'value']).
After a lot of testing different variant we managed to get a solution this way;
$connection = 'connection';
$iterator = 0;
foreach ($files as $file) {
App::useEnvironmentPath('/env');
App::loadEnvironmentFrom('.file.env');
// Create a new connection "on the fly"
config()->set('database.connections.' . $connection . '_' . $iterator, [
'driver' => 'mysql',
'host' => env('DB_HOST'),
'database' => env('DB_DATABASE'),
'username' => env('DB_USERNAME'),
'password' => env('DB_PASSWORD'),
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => '',
'strict' => false,
]);
// Call regular migration command
$this->call('migrate', ['--force' => true, '--database' => $connection . '_' . $iterator]);
$iterator++;
}
This manages to set multiple new connections to the MySQL-database, and then seed each one of them.
Thanks to #David Allen here for the inspiration.

Resources