ActiveRecord 6 without Rails: Connecting to multiple databases - ruby

I have a legacy codebase which I need to upgrade to ActiveRecord 6. It does not use Rails.
I need to be able to connect to multiple databases.
In my code, I have:
conf = ActiveRecord::DatabaseConfigurations.new({ 'test' => db_config })
ActiveRecord::Base.configurations = conf
where db_config is defined as:
def db_config
{
'adapter' => 'mysql2',
'name' => 'testdb',
'host' => xxx,
'port' => 3306,
'username' => xxx,
'password' => xxx,
'database' => 'testdb'
}
end
However, I am running into an error when I execute my unit tests in rspec:
ActiveRecord::AdapterNotSpecified:
The `testdb` database is not configured for the `test` environment.
Available databases configurations are:
test
What am I doing wrong here? I thought the fact that it shows a test configuration should mean that the database is configured for the test environment. Any help would be most welcome.

You're trying to connect to database testdb, while the one you have is test. Try changing the config:
def db_config
{
'adapter' => 'mysql2',
'name' => 'test', # was 'testdb`
'host' => xxx,
'port' => 3306,
'username' => xxx,
'password' => xxx,
'database' => 'test' # was 'testdb`
}
end

Related

unsupported driver [https], laravel when deployed to heroku

I am trying to deploy a Laravel application to Heroku and connect it with a database which has already been deployed to Azure.
But I am having error "unsupported driver[https]".
My database.php:
<?php
use Illuminate\Support\Str;
return [
'default' => env('DB_CONNECTION', 'mysql'),
/
'mysql' => [
'driver' => 'mysql'
'url' => env('DATABASE_URL','https://firstsqlaap.scm.azurewebsites.net/phpMyAdmin/db_structure.php?server=1&db=localdb&token=51b0b3471e798a712e129bcd1ebe5b01'),
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '53082'),
'database' => env('DB_DATABASE', 'localdb'),
'username' => env('DB_USERNAME', 'user'),
'password' => env('DB_PASSWORD', 'pass'),
'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',
'options' => extension_loaded('pdo_mysql') ? array_filter([
PDO::MYSQL_ATTR_SSL_CA => env('MYSQL_ATTR_SSL_CA'),
]) : [],
],
];
My SESSION_DRIVER is set to database because when set to file it was saying 419 error. I do not have any migration files as my database is deployed to Azure.
How to resolve this issue?
This certainly isn't the right URL to use:
https://firstsqlaap.scm.azurewebsites.net/phpMyAdmin/db_structure.php
You appear to be pointing to an instance of phpMyAdmin. phpMyAdmin isn't a database server, it's a dataase client. It's a tool that you might use to interact with your database. You need to provide the URL to your actual database.
Your database URL should look more like this:
driver://username:password#host:port/database?options
For MySQL, driver:// is likely mysql://.
I don't have any MySQL databases running on Azure, but it looks like a real URL might be something like
mysql://user:password#your-database-instance.mysql.database.azure.com/your-database-name
Go into the Azure portal and navigate to your database instance. Then, in the left navigation panel, click on "Connection strings". The information you need should be there, though not in URL format. You can either build your own URL by plugging the right values in or use the individual settings in your config/database.php file.
I commented url and it work for me

Error ORA-12505: TNS:listener does not currently know of SID given in connect descriptor Laravel 5.8 Yajra

I get the following error Yajra\Pdo\Oci8\Exceptions\Oci8Exception ORA-12505: TNS:listener does not currently know of SID given in connect descriptor
What I want is to be able to connect to Laravel 5.8.38 to Oracle (remote), and I'm not sure how to do the setup using the service name
Sql Developer Configuration
'connections' => [
'oracle' => [
'driver' => 'oracle',
'host' => '192.168.0.190',
'port' => '1521',
'database' => 'BDDESARR',
'service_name' => '???',
'username' => 'PAT_GUZ',
'password' => 'ujUYjjdk',
'charset' => '',
'prefix' => '',
],
Installation of Yajra following the step by step from https://github.com/yajra/laravel-oci8/tree/5.8
Terminal output
PS C:\wamp64\www\desarrollo\php\laravel\miproyect> composer require yajra/laravel-oci8:"5.8.*"
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Nothing to install or update
Package jakub-onderka/php-console-color is abandoned, you should avoid using it. Use php-parallel-lint/php-console-color instead.
Package jakub-onderka/php-console-highlighter is abandoned, you should avoid using it. Use php-parallel-lint/php-console-highlighter instead.
Writing lock file
Generating optimized autoload files
File config/database.php
'default' => env('DB_CONNECTION', 'oracle'),
'connections' => [
'oracle' => [
'driver' => 'oracle',
'host' => '192.168.0.190',
'port' => '1521',
'database' => 'BDDESARR',
'service_name' => '???',
'username' => 'PAT_GUZ',
'password' => 'ujUYjjdk',
'charset' => '',
'prefix' => '',
],
Additional information, I made the following code and it runs successfully
<?php
$conn = oci_connect('PAT_GUZ', 'ujUYjjdk', '192.168.0.190/BDDESARR');
if (!$conn) {
$e = oci_error();
var_dump($e);
}
$stid = oci_parse($conn, 'SELECT * FROM MY_TABLE');
oci_execute($stid);
echo "<table border='1'>\n";
while ($row = oci_fetch_array($stid, OCI_ASSOC+OCI_RETURN_NULLS)) {
echo "<tr>\n";
foreach ($row as $item) {
echo " <td>" . ($item !== null ? htmlentities($item, ENT_QUOTES) : "") . "</td>\n";
}
echo "</tr>\n";
}
echo "</table>\n";
Laravel 5.8.38
PHP 7.3.12
Oracle: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 -
64bit Production
Windows 10 64 bit
Wamp64
I don't know anything about Laravel, Yajra and other things you mentioned, but this:
'database' => 'BDDESARR',
'service_name' => '???',
looks if not wrong then suspicious. SQL Developer connection suggests that Service name = 'bddesarr'. I don't know what "database" is supposed to be. SID, perhaps, as Oracle complains that SID isn't right.

Laravel data mismatch error while using \PDO::ATTR_EMULATE_PREPARES => true

We have application build in Php Laravel and for the database we use postgres sql. And also on top of postgres we have configure pgBouncer to limit the maximum number of connections on server side by managing a pool of idle connections that can be used by any applications.
Now, we face the issue with the boolean values (True(0),False(1)) used in the application (Php Laravel). It gives below error when any CRUD operation is performed. In the below error column "revoked" is boolean type.
column \"revoked\" is of type boolean but expression is of type integer
You will need to rewrite or cast the expression. (SQL: \"revoked\", \"created_at\") values (0, 2020-02-07 06:09:06)
Now after exploring, I came to know that boolean values needs to be consider to be string with the pgBouncer. So I have made changes in the connection.php file, located in "\vendor\laravel\framework\src\Illuminate\Database". I have change the code to consider the boolean value as mentioned below.
public function bindValues($statement, $bindings)
{
foreach ($bindings as $key => $value) {
//if(is_bool($value))
$statement->bindValue(
is_string($key) ? $key : $key + 1, $value,
//is_int($value) ? PDO::PARAM_INT : PDO::PARAM_STR
is_int($value) ? PDO::PARAM_INT : is_bool($value) ? PDO::PARAM_STR : PDO::PARAM_STR
);
}
}
After the above changes the error with the boolean values was solved.
But, now I am facing strange issues on the server, when I check the database log error I consistently get the below error.
ERROR: prepared statement "pdo_stmt_00000001" already exists
STATEMENT: set names 'utf8'
ERROR: prepared statement "pdo_stmt_00000001" does not exist
STATEMENT: DEALLOCATE pdo_stmt_00000001
It really was strange, and after exploring the internet I have done the below changes in my database.php file, to disable the prepare statements.
'pgsql' => [
'driver' => 'pgsql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '5432'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
\PDO::ATTR_EMULATE_PREPARES => true
]
]
The reason behind seeting ATTR_EMULATE_PREPARES => true is becasue I have set "Transaction" mode in "pgbouncer.ini" file.
Now, to make prepared statements work in Transaction mode would need PgBouncer to keep track of them internally, which it does not do. So only way to keep using PgBouncer in this mode is to disable prepared statements in the client, which in my case is PHP Laravel and I have already handle it in the "database.php" file when the connection is made as shown in above code.
I have tried all the options, which are given in http://www.pgbouncer.org/faq.html#how-to-use-prepared-statements-with-transaction-pooling but it doesnot solve the prepare statment error shown in the database log.
ERROR: prepared statement "pdo_stmt_00000001" already exists
STATEMENT: set names 'utf8'
ERROR: prepared statement "pdo_stmt_00000001" does not exist
STATEMENT: DEALLOCATE pdo_stmt_00000001
Please guide me on the same and what further settings are required for the error. Those errors are on the client production server and we cannot go ahead with those errors in production server.
Please give me your valuable feedback at the earliest as I am facing the issue since 5 days and try with all the options that come across.
Thanks!
1) First, you need to change the PDO option you are giving in the options in the pgsql array of your database.php the right way is as given below.
'pgsql' => [
'driver' => 'pgsql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '5434'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
PDO::ATTR_EMULATE_PREPARES => true
]
]
2) Second, and the most important thing is to make sure that you use set the "ATTR_EMULATE_PREPARES" to "true" with each database connection you try to connect in your Database.php file.
For example,
'test' => [
'driver' => 'pgsql',
'host' => env('test', '127.0.0.1'),
'port' => env('test', '5434'),
'database' => env('DB_TEST_DATABASE', 'test'),
'username' => env('DB_USERNAME', 'test'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
PDO::ATTR_EMULATE_PREPARES => true
]
],
'test1' => [
'driver' => 'pgsql',
'host' => env('test1', '127.0.0.1'),
'port' => env('test1', '5434'),
'database' => env('DB_TEST1_DATABASE', 'test1'),
'username' => env('DB_USERNAME', 'test'),
'password' => env('DB_PASSWORD', ''),
'charset' => 'utf8',
'prefix' => '',
'schema' => 'public',
'sslmode' => 'prefer',
'options' => [
PDO::ATTR_EMULATE_PREPARES => true
]
]
Please make sure to use the "ATTR_EMULATE_PREPARES" to true for each database connection you make in your application, in your comments you make connection with only "pgsql" which emphasis for postgres sql connection only, and not with the database that your application communicates which is in postgres.
Hope this helps you to resolve your query. Enjoy!!!
First you never need to modify the vendor code, instead you can use attribute casting from your model.
From laravel.com/docs/master/eloquent-mutators#attribute-casting
The $casts property on your model provides a convenient method of converting attributes to common data types. The $casts property should be an array where the key is the name of the attribute being cast and the value is the type you wish to cast the column to. The supported cast types are: integer, real, float, double, decimal:, string, boolean, object, array, collection, date, datetime, and timestamp. When casting to decimal, you must define the number of digits (decimal:2).
To demonstrate attribute casting, let's cast the is_admin attribute,
which is stored in our database as an integer (0 or 1) to a boolean
value:
So in your case you will need to cast revoked to bool by adding to your Eloquent model the $casts property as follows:
<?php
namespace App;
use Illuminate\Database\Eloquent\Model;
class YourModel extends Model
{
/**
* The attributes that should be cast to native types.
*
* #var array
*/
protected $casts = [
'revoked' => 'boolean',
];
}
And for your pgBouncer issue it seems that pgBouncer have an internal issue with transaction pooling and prepared statements,
From: stackoverflow.com/a/7612639/7047493
This turned out to be a pgBouncer issue that occurs when using anything other than session pooling. We were using transaction pooling, which apparently can't support prepared statements. By switching to session pooling, we got around the issue.
None of the previous answers fully worked in our case. In our setup (Laravel + PostgreSQL + pgBouncer), we had enabled these 2 settings in the database.php file. The goal was to make our php backend compatible with pgBouncer and that's what we had done:
// database.php
'options' => array(
PDO::ATTR_EMULATE_PREPARES => true
),
'binary_parameters' => 'yes', // not sure if this one is necessary
These 2 settings partially worked, meaning we were able to run our backend without getting the prepared statement does not exist anymore. Unfortunately for us, we were then getting the datatype mismatch: 7 ERROR: column “xxx” is of type boolean but expression is of type integer just like Nileshsinh Rathod.
Hopefully for us, we came across this post on Github which fixed everything for us. The goal is to override the default PostgresConnector.
And here is a recap of what we did:
Add these 3 files in our project:
https://github.com/umbrellio/laravel-pg-extensions/blob/master/src/Connectors/ConnectionFactory.php
https://github.com/umbrellio/laravel-pg-extensions/blob/master/src/UmbrellioPostgresProvider.php
https://github.com/umbrellio/laravel-pg-extensions/blob/master/src/PostgresConnection.php
Within this file, we only kept the bindValues and prepareBindings functions.
Then, in our config/app.php, we registered the PostgresProvider like so
'providers' => [
App\Providers\ScPostgresProvider::class,
],
Finally, we commented out this line in our AppServiceProvider file in order to make sure only the new one would be registered
public function register()
{
// not used anymore since we use our our own connector
// $this->app->bind('db.connector.pgsql', OldPostgresConnector::class);
}
Thanks a lot to the post of Umbrellio team on Github and hope this answer will help others!

Laravel 5l Multiple Database Connections

Do I need to have my DB connection info in 2 places? If I do the below, I connect just fine. If I remove either file's data, I can't connect.
config/databases.php file
'blah_1' => [
'driver' => 'mysql',
'host' => env('DB_HOST’,’1.1.1.1’),
'port' => env('DB_PORT','3306'),
'database' => env('DB_DATABASE’,’someDB_1’),
'username' => env('DB_USERNAME’,’someUser_1’),
'password' => env('DB_PASSWORD’,’somePass_1’),
],
'blah_2' => [
'driver' => 'mysql',
'host' => env('DB_HOST_SECOND’,’2.2.2.2’),
'port' => env('DB_PORT_SECOND','3306'),
'database' => env('DB_DATABASE_SECOND’,’someDB_2’),
'username' => env('DB_USERNAME_SECOND’,’someUser_2’),
'password' => env('DB_PASSWORD_SECOND’,’somePass_2’),
],
.env file:
DB_CONNECTION=blah_1
DB_HOST=1.1.1.1
DB_PORT=3306
DB_DATABASE=someDB_1
DB_USERNAME=someUser_1
DB_PASSWORD=somePass_1
DB_CONNECTION_SECOND=blah_2
DB_HOST_SECOND=2.2.2.2
DB_PORT_SECOND=3306
DB_DATABASE_SECOND=someDB_2
DB_USERNAME_SECOND=someUser_2
DB_PASSWORD_SECOND=somePass_2
The short answer is no. You only need it in your config/databases.php. The .env file is to overwrite the settings in your other environments without updating the configuration file.
For example, in your local environment, your credentials is most likely different from your production environment. You wouldn't want to update config/databases.php locally and remind yourself to not push the file.
However, the connections would still work even if remove them from the .env file. It will use the second parameter's value in your env() as default.

Extract filebeat prospector 'fileds' used from all attributes definitions (ruby based)

I'm currently adding filebeat support for services we provide;
For every service we have several log files I would like to track;
I have 1 common filebeat recipe and I'm distinguishing between the different services' logs in different attributes .rb files;
In those I have a separate definition per log;
All definitions have the same "fileds" configuration;
Can I add it to someplace to be used by all configurations?
My structure:
cookbooks
common
recipes
filebeat.rb
services
attributes
service1.rb
service2.rb
The content of servicesX.rb has the following definitions:
access_log = {
'paths' => ['TBD'],
'input_type' => 'log',
'fields' => {
'hostname' => node["opsworks"]["instance"]["hostname"],
'customer' => node["opsworks"]["instance"]["layers"][0],
'internal_ip' => node["opsworks"]["instance"]["private_ip"],
'ec2id' => node["opsworks"]["instance"]["aws_instance_id"],
'os' => node["opsworks"]["instance"]["os"],
'instance_type' => node["opsworks"]["instance"]["instance_type"] },
'fields_under_root' => true
}
audit_log = {
'paths' => ['TBD'],
'input_type' => 'log',
'fields' => {
'hostname' => node["opsworks"]["instance"]["hostname"],
'customer' => node["opsworks"]["instance"]["layers"][0],
'internal_ip' => node["opsworks"]["instance"]["private_ip"],
'ec2id' => node["opsworks"]["instance"]["aws_instance_id"],
'os' => node["opsworks"]["instance"]["os"],
'instance_type' => node["opsworks"]["instance"]["instance_type"]
},
'fields_under_root' => true
}
How can I extract
'fields' => {
'hostname' => node["opsworks"]["instance"]["hostname"],
'customer' => node["opsworks"]["instance"]["layers"][0],
'internal_ip' => node["opsworks"]["instance"]["private_ip"],
'ec2id' => node["opsworks"]["instance"]["aws_instance_id"],
'os' => node["opsworks"]["instance"]["os"],
'instance_type' => node["opsworks"]["instance"]["instance_type"]
Someplace in the same file (servicesX.rb) so to be used by all log files definitions?
Note: I'm a ruby novice :/
Thank you!!
After feedback and clarification in comments below, OP seems to want to DRY the code and re-use a fields definition.
Simplest is to store it in a variable and then use that:
fields = {
'hostname' => node["opsworks"]["instance"]["hostname"],
'customer' => node["opsworks"]["instance"]["layers"][0],
'internal_ip' => node["opsworks"]["instance"]["private_ip"],
'ec2id' => node["opsworks"]["instance"]["aws_instance_id"],
'os' => node["opsworks"]["instance"]["os"],
'instance_type' => node["opsworks"]["instance"]["instance_type"]
}
audit_log = {
'paths' => ['TBD'],
'input_type' => 'log',
'fields' => fields
}
This, however, may cause issues with how node is set. It really depends on the flow of the rest of your script. In chef (assuming this is about chef) a node is the context on which the script runs, so setting the fields too early might give issues when that node is used much later:
fields = { hostname: node["opsworks"]["instance"]["hostname"] }
# ... do lots of stuff, like fetching, preparing, connecting and whatnot.
fields # now contains the `node` values as set before connecting etc.
If this is an issue, a better option would be to define a method that returns the fields from a passed-in node:
def fields(node)
{
'hostname' => node["opsworks"]["instance"]["hostname"],
'customer' => node["opsworks"]["instance"]["layers"][0],
'internal_ip' => node["opsworks"]["instance"]["private_ip"],
'ec2id' => node["opsworks"]["instance"]["aws_instance_id"],
'os' => node["opsworks"]["instance"]["os"],
'instance_type' => node["opsworks"]["instance"]["instance_type"]
}
end
Or, cleaned up:
def fields(node)
instance = node["opsworks"]["instance"]
{
hostname: instance["hostname"],
customer: instance["layers"][0],
internal_ip: instance["private_ip"],
ec2id: instance["aws_instance_id"],
os: instance["os"],
instance_type: instance["instance_type"]
}
end
Then use that function:
audit_log = {
'paths' => ['TBD'],
'input_type' => 'log',
'fields' => fields(node)
}
After my research: The benefits of the implementation does not justify the time invested;
Leaving it;

Resources