We have application build in Php Laravel and for the database we use postgres sql. And also on top of postgres we have configure pgBouncer to limit the maximum number of connections on server side by managing a pool of idle connections that can be used by any applications.
Now, we face the issue with the boolean values (True(0),False(1)) used in the application (Php Laravel). It gives below error when any CRUD operation is performed. In the below error column "revoked" is boolean type.
column \"revoked\" is of type boolean but expression is of type integer
You will need to rewrite or cast the expression. (SQL: \"revoked\", \"created_at\") values (0, 2020-02-07 06:09:06)
Now after exploring, I came to know that boolean values needs to be consider to be string with the pgBouncer. So I have made changes in the connection.php file, located in "\vendor\laravel\framework\src\Illuminate\Database". I have change the code to consider the boolean value as mentioned below.
public function bindValues($statement, $bindings)
{
foreach ($bindings as $key => $value) {
//if(is_bool($value))
$statement->bindValue(
is_string($key) ? $key : $key + 1, $value,
//is_int($value) ? PDO::PARAM_INT : PDO::PARAM_STR
is_int($value) ? PDO::PARAM_INT : is_bool($value) ? PDO::PARAM_STR : PDO::PARAM_STR
);
}
}
After the above changes the error with the boolean values was solved.
But, now I am facing strange issues on the server, when I check the database log error I consistently get the below error.
ERROR: prepared statement "pdo_stmt_00000001" already exists
STATEMENT: set names 'utf8'
ERROR: prepared statement "pdo_stmt_00000001" does not exist
STATEMENT: DEALLOCATE pdo_stmt_00000001
It really was strange, and after exploring the internet I have done the below changes in my database.php file, to disable the prepare statements.
'pgsql' => [
'driver' => 'pgsql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '5432'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
\PDO::ATTR_EMULATE_PREPARES => true
]
]
The reason behind seeting ATTR_EMULATE_PREPARES => true is becasue I have set "Transaction" mode in "pgbouncer.ini" file.
Now, to make prepared statements work in Transaction mode would need PgBouncer to keep track of them internally, which it does not do. So only way to keep using PgBouncer in this mode is to disable prepared statements in the client, which in my case is PHP Laravel and I have already handle it in the "database.php" file when the connection is made as shown in above code.
I have tried all the options, which are given in http://www.pgbouncer.org/faq.html#how-to-use-prepared-statements-with-transaction-pooling but it doesnot solve the prepare statment error shown in the database log.
ERROR: prepared statement "pdo_stmt_00000001" already exists
STATEMENT: set names 'utf8'
ERROR: prepared statement "pdo_stmt_00000001" does not exist
STATEMENT: DEALLOCATE pdo_stmt_00000001
Please guide me on the same and what further settings are required for the error. Those errors are on the client production server and we cannot go ahead with those errors in production server.
Please give me your valuable feedback at the earliest as I am facing the issue since 5 days and try with all the options that come across.
Thanks!
1) First, you need to change the PDO option you are giving in the options in the pgsql array of your database.php the right way is as given below.
'pgsql' => [
'driver' => 'pgsql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '5434'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
PDO::ATTR_EMULATE_PREPARES => true
]
]
2) Second, and the most important thing is to make sure that you use set the "ATTR_EMULATE_PREPARES" to "true" with each database connection you try to connect in your Database.php file.
For example,
'test' => [
'driver' => 'pgsql',
'host' => env('test', '127.0.0.1'),
'port' => env('test', '5434'),
'database' => env('DB_TEST_DATABASE', 'test'),
'username' => env('DB_USERNAME', 'test'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
PDO::ATTR_EMULATE_PREPARES => true
]
],
'test1' => [
'driver' => 'pgsql',
'host' => env('test1', '127.0.0.1'),
'port' => env('test1', '5434'),
'database' => env('DB_TEST1_DATABASE', 'test1'),
'username' => env('DB_USERNAME', 'test'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
PDO::ATTR_EMULATE_PREPARES => true
]
]
Please make sure to use the "ATTR_EMULATE_PREPARES" to true for each database connection you make in your application, in your comments you make connection with only "pgsql" which emphasis for postgres sql connection only, and not with the database that your application communicates which is in postgres.
Hope this helps you to resolve your query. Enjoy!!!
First you never need to modify the vendor code, instead you can use attribute casting from your model.
From laravel.com/docs/master/eloquent-mutators#attribute-casting
The $casts property on your model provides a convenient method of converting attributes to common data types. The $casts property should be an array where the key is the name of the attribute being cast and the value is the type you wish to cast the column to. The supported cast types are: integer, real, float, double, decimal:, string, boolean, object, array, collection, date, datetime, and timestamp. When casting to decimal, you must define the number of digits (decimal:2).
To demonstrate attribute casting, let's cast the is_admin attribute,
which is stored in our database as an integer (0 or 1) to a boolean
value:
So in your case you will need to cast revoked to bool by adding to your Eloquent model the $casts property as follows:
<?php
namespace App;
use Illuminate\Database\Eloquent\Model;
class YourModel extends Model
{
/**
* The attributes that should be cast to native types.
*
* #var array
*/
protected $casts = [
'revoked' => 'boolean',
];
}
And for your pgBouncer issue it seems that pgBouncer have an internal issue with transaction pooling and prepared statements,
From: stackoverflow.com/a/7612639/7047493
This turned out to be a pgBouncer issue that occurs when using anything other than session pooling. We were using transaction pooling, which apparently can't support prepared statements. By switching to session pooling, we got around the issue.
None of the previous answers fully worked in our case. In our setup (Laravel + PostgreSQL + pgBouncer), we had enabled these 2 settings in the database.php file. The goal was to make our php backend compatible with pgBouncer and that's what we had done:
// database.php
'options' => array(
PDO::ATTR_EMULATE_PREPARES => true
),
'binary_parameters' => 'yes', // not sure if this one is necessary
These 2 settings partially worked, meaning we were able to run our backend without getting the prepared statement does not exist anymore. Unfortunately for us, we were then getting the datatype mismatch: 7 ERROR: column “xxx” is of type boolean but expression is of type integer just like Nileshsinh Rathod.
Hopefully for us, we came across this post on Github which fixed everything for us. The goal is to override the default PostgresConnector.
And here is a recap of what we did:
Add these 3 files in our project:
https://github.com/umbrellio/laravel-pg-extensions/blob/master/src/Connectors/ConnectionFactory.php
https://github.com/umbrellio/laravel-pg-extensions/blob/master/src/UmbrellioPostgresProvider.php
https://github.com/umbrellio/laravel-pg-extensions/blob/master/src/PostgresConnection.php
Within this file, we only kept the bindValues and prepareBindings functions.
Then, in our config/app.php, we registered the PostgresProvider like so
'providers' => [
App\Providers\ScPostgresProvider::class,
],
Finally, we commented out this line in our AppServiceProvider file in order to make sure only the new one would be registered
public function register()
{
// not used anymore since we use our our own connector
// $this->app->bind('db.connector.pgsql', OldPostgresConnector::class);
}
Thanks a lot to the post of Umbrellio team on Github and hope this answer will help others!
I've been running an app on a Laravel forged provisioned server.
We have some email jobs that are being queued, and we use Horizon to manage our queues. This has always worked without any issues, but for some reason, we have broken something, and I can't fix it.
This is our setup.
.env
APP_ENV=dev
REDIS_HOST=127.0.0.1
REDIS_PASSWORD=null
REDIS_PORT=6379
QUEUE_DRIVER=redis
config/queues.php
return [
'default' => env('QUEUE_DRIVER', 'sync'),
'connections' => [
'sync' => [
'driver' => 'sync',
]
'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => 'medium',
'retry_after' => 90,
],
],
];
config/horizon.php
return [
'use' => 'default',
'waits' => [
'redis:default' => 60,
],
'environments' => [
'dev' => [
'high-prio' => [
'connection' => 'redis',
'queue' => ['high'],
'balance' => 'simple',
'processes' => 10,
'tries' => 5,
],
'default-prio' => [
'connection' => 'redis',
'queue' => ['medium', 'low'],
'balance' => 'auto',
'processes' => 10,
'tries' => 3,
],
],
],
];
I checked the redis-cli info result to make sure the port was right:
forge#denja-dev:~$ redis-cli info
# Server
redis_version:3.2.8
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:11aa79fd2425bed9
redis_mode:standalone
os:Linux 4.4.0-142-generic x86_64
arch_bits:64
multiplexing_api:epoll
gcc_version:5.4.0
process_id:1191
run_id:fcc57fa2c17440ab964538c2d986dc330d9e1223
tcp_port:6379
uptime_in_seconds:3045
uptime_in_days:0
hz:10
lru_clock:13667343
executable:/usr/bin/redis-server
config_file:/etc/redis/redis.conf
When I visit /horizon/dashboard, all is running fine.
I was playing a bit with adding some metadata to the payload for queued jobs, at which time the issues began. I then just removed that code again, and basically went back to the previous code base. There is no difference anymore, so I'm now suspecting that I have another issue.
However - I'm not getting ANY exception thrown when I add something to the queue. Bugsnag has no new entries, and my process just continues without any error.
Any idea what I can verify more to detect the actual issue? Is there a problem with the config? I'm a bit lost to be honest, especially since I have no information to work with :(
I also checked using tinker whether I could make a connection to redis, and that too works fine without an exception:
$ php artisan tinker
Psy Shell v0.9.9 (PHP 7.2.0RC3 — cli) by Justin Hileman
>>> Illuminate\Support\Facades\Redis::connection('default')
=> Illuminate\Redis\Connections\PredisConnection {#3369}
The cause of this issue was that the notification that I was testing this with, did use the Queuable trait, but did not implement the ShouldQueue interface. The latter is required to have Laravel automatically queue these Notifications.
We noticed it when we started testing using other Notifications which went through fine.
The only question we still had is that we would have expected the email to go out nevertheless, since it would synchronously send it, which for some reason it did not.
I'm really just looking for an explanation about memecached and laravel. I understand what it does, but can I use my memcached installation with laravel. More specifically:
'memcached' => [
'driver' => 'memcached',
'persistent_id' => env('MEMCACHED_PERSISTENT_ID'),
'sasl' => [
env('MEMCACHED_USERNAME'),
env('MEMCACHED_PASSWORD'),
],
'options' => [
// Memcached::OPT_CONNECT_TIMEOUT => 2000,
],
'servers' => [
[
'host' => env('MEMCACHED_HOST', '127.0.0.1'),
'port' => env('MEMCACHED_PORT', 11211),
'weight' => 100,
],
],
],
I know/will set up the server aspect, and I get what the options do...but persistent_id, a memcached um and pw...what are they? Their uses? etc.. typically laravel is extremely well document but on memcached it says very little (And the little it does, seems to be dated and not based on 5.0 laravel)
Here is explanation from php.net:
By default the Memcached instances are destroyed at the end of the request. To create an instance that persists between requests, use persistent_id to specify a unique ID for the instance. All instances created with the same persistent_id will share the same connection.
http://php.net/manual/en/memcached.construct.php
So for your project just define a unique name for it.
Hope it helps.
I have setup my Laravel application with docker, one container is dedicated to the app one for redis.
I have setup Laravel to use Redis for session an caching.
All works fine but if I enter my Redis container and try to list all keys like:
$redis-cli
#KEYS *
It will return only key values used for caching not the session keys.
The above is a doublecheck because actually from Laravel application I set session key and then dump like
<?php dump(session()->all()); dump(Session::getDefaultDriver()); ?>
and from the dump everything looks fine.
I see my session keys and values data structures.
Session::getDefaultDriver() //returns "redis"
So, by seeing Cache key:values inside redis container I assume that there's not a connection/docker container issues... Laravel is writing in the correct place. Redis default connection is shared by Cache and SEssion.
In database.php I have:
'redis' => [
'client' => 'predis',
'default' => [
'host' => env('REDIS_HOST', 'redis'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_PORT', 6379),
'database' => 0,
],
],
In session.php I have
'driver' => env('SESSION_DRIVER', 'redis'),
...
'lifetime' => env('SESSION_LIFETIME', 120),
By seeing the dumps returning correct values in Laravel web application I'm assuming session is working properly and points to redis.
What am I missing?
I am using Auth Session in CakePHP.
'Session' => [
'timeout' => 0,
'defaults' => 'php',
'ini' => [
'session.cookie_secure' => false,
'session.cookie_lifetime' => 0
]
],
Already configured this in tha app.php but still getting logged out after few minutes.
Although Session persists when closing Browser. So i think its just a time related problem.
I was having requirement to increase session timeout in cakphp 2.5
the following code worked for me in 2.5 you can give this a try
//FILE: config/core.php
Configure::write('Session', array(
'defaults' => 'cake',
'cookie' => 'my_app',
'timeout' => 500, // 8 hours + 20 min, user will be logged in till 8 hours.
)
);