Issue with connecting my express server on heroku to the remote mySQL - heroku

I'm trying to deploy my express server on Heroku which needs to connect to the remote MySQL database.
I used 'heroku config:add DATABASE_URL=mysql://dbusername:dbpassword#databasehostIP:databaseserverport/databasename with the correct information but still it tries to connect through wrong address.
I also used 'heroku config:add EXTERNAL_DATABASE_URL=mysql://dbusername:dbpassword#databasehostIP:databaseserverport/databasename with the correct information but still it tries to connect through wrong address.
In my Heroku app panel under 'setting' in 'Config Vars' section I see that DATABASE_URL and EXTERNAL_DATABASE_URL appeared with correct information. but in heroku log I still see the wrong information
This is my sequelize variable on the express server:
const sequelize = new Sequelize('dbName', 'USER', 'Password', {
host:"hostAddress",
dialect: 'mysql'
}
But I see the following on Heroku log:
2019-02-16T18:31:42.231390+00:00 app[web.1]: Unhandled rejection
SequelizeAccessDeniedError: Access denied for user
'USER'#'ec2-54-162-8-141.compute-1.amazonaws.com' (using
password: YES)
How can I change 'ec2-54-162-8-141.compute-1.amazonaws.com' to the remote MySQL host address?

Try setting your variable with something like this:
if (process.env.DATABASE_URL) {
const sequelize = new Sequelize(process.env.DATABASE_URL, {
define: {
freezeTableName: true, // don't make plural table names
underscored: true // don't use camel case
},
dialect: 'mysql',
dialectOptions: {
ssl: true
},
logging: true,
protocol: 'mysql',
quoteIdentifiers: false // set case-insensitive
});
} else {
console.log('Fatal error: DATABASE_URL not set');
process.exit(1);
}

Related

"Error: Unknown dialect undefined" when deploying Strapi 4.0 app to Heroku

I created a Strapi app using the latest version, the new 4.0 and I wanted to deploy it to Heroku. I did follow the Strapi documentation in order to do so, like explained in this page. Now I'm getting an error that I don't understand, I guess it has something to do with postgres. This is the error
2021-12-18T15:26:26.658380+00:00 app[web.1]: [2021-12-18 15:26:26.656] debug: ⛔️ Server wasn't able to start properly.
2021-12-18T15:26:26.659122+00:00 app[web.1]: [2021-12-18 15:26:26.658] error: Unknow dialect undefined
2021-12-18T15:26:26.659123+00:00 app[web.1]: Error: Unknow dialect undefined
2021-12-18T15:26:26.659123+00:00 app[web.1]: at getDialectClass (/app/node_modules/#strapi/database/lib/dialects/index.js:12:13)
2021-12-18T15:26:26.659123+00:00 app[web.1]: at getDialect (/app/node_modules/#strapi/database/lib/dialects/index.js:19:23)
2021-12-18T15:26:26.659124+00:00 app[web.1]: at new Database (/app/node_modules/#strapi/database/lib/index.js:38:20)
2021-12-18T15:26:26.659124+00:00 app[web.1]: at Function.Database.init (/app/node_modules/#strapi/database/lib/index.js:84:33)
2021-12-18T15:26:26.659125+00:00 app[web.1]: at Strapi.bootstrap (/app/node_modules/#strapi/strapi/lib/Strapi.js:347:30)
2021-12-18T15:26:26.659125+00:00 app[web.1]: at Strapi.load (/app/node_modules/#strapi/strapi/lib/Strapi.js:410:16)
2021-12-18T15:26:26.659125+00:00 app[web.1]: at async Strapi.start (/app/node_modules/#strapi/strapi/lib/Strapi.js:161:9)
Apart from doing what is explained in the docs I linked, I just added a few collections using the UI in development mode. How can I fix this error and deploy to Heroku this new 4.0 version of Strapi?
I had a similar issue when I was connecting pg locally and then realised my connection config was incorrect. When I replaced it with v4 template it worked.
path: config/database.js
module.exports = ({ env }) => ({
defaultConnection: "default",
connection: {
client: "postgres",
connection: {
host: env("DATABASE_HOST", "localhost"),
port: env.int("DATABASE_PORT", 5432),
database: env("DATABASE_NAME", "bank"),
username: env("DATABASE_USERNAME", "postgres"),
password: env("DATABASE_PASSWORD", "0000"),
schema: env("DATABASE_SCHEMA", "public"),
},
}
});
And for Heorku as the article suggested:
path: config/env/production/database.js:
const parse = require('pg-connection-string').parse;
const config = parse(process.env.DATABASE_URL);
module.exports = ({ env }) => ({
connection: {
client: 'postgres',
connection: {
host: config.host,
port: config.port,
database: config.database,
user: config.user,
password: config.password,
ssl: {
rejectUnauthorized: false
},
},
debug: false,
},
});
PS: make sure you add pg-connection-string to your dependencies before pushing to heroku
getDialectClass - from the error log
const getDialectClass = client => {
switch (client) {
case 'postgres':
return require('./postgresql');
case 'mysql':
return require('./mysql');
case 'sqlite':
return require('./sqlite');
default:
throw new Error(`Unknow dialect ${client}`);
}
};
where the client is - db.config.connection
So if you have been following previous solutions - which talked about a connections object etc ( like this )
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'postgres',
host: config.host,
port: config.port,
database: config.database,
username: config.user,
password: config.password,
ssl: {
rejectUnauthorized: false,
},
},
options: {
ssl: true,
},
},
},
});
db.config.connection would return undefined. & so it would fail
Configuring the DB with - https://docs.strapi.io/developer-docs/latest/setup-deployment-guides/configurations/required/databases.html#configuration-structure
Works fine
If you are using typescript in Strapi, we may encounter this issue too as there is a issue affecting Strapi typescript capabilities.
I can resolve the problem when I downgraded Strapi to 4.3.2 from 4.3.8
You can check this method using heroku/cli from strapi docs here, but it's the same thing, i'm using version 4.0.2, this methods works on older versions V3, i think the docs needs update, most of it is deprecated, for example "fetching only data (example: posts/project etc..) related to the authenticated user who created it", most of resources related to this topic in docs and forum are deprecated solutions that don't work in V4.
For the deployment, i have the same issue, i tried a lot of solutions but it didn't work for me to, but i managed to get the server running like this:
but i got this problem when i visit the "/admin" page:
Strapi is really interesting, let's hope they update the docs soon and someone respond with a solution here.
I fixed this issue by using database url directly. Not sure why the old way does not work in Strapi v4.
module.exports = ({ env }) => ({
defaultConnection: "default",
connection: {
client: "postgres",
connection: {
connectionString: process.env.DATABASE_URL,
ssl: {
rejectUnauthorized: false
}
},
}
});
In case someone meets the same problem as me.
In my situation, I am using vim in windows as my text editor. After editing the "config/database.js" file. It left a "database.js~" file in that directory. Then, this filename causes the same problems: "Error: Unknown dialect undefined".
I guess it's a bug in strapi. After removing that file, everything works!
So, the solutions for me. I add this line to my vim config file:
set backupdir=~/.vimbackup
Then create a directory named '.vimbackup' in "C:/User/[me]/" .
Strapi has once again changed the database configuration structure and even their deployment docs use the old and incorrect example.
Here is the most up to date example: Strapi Database Configuration
In strapi v4, you need to use like:
module.exports = ({ env }) => ({
connection: {
client: "postgres",
connection: {
host: env("DATABASE_HOST", "localhost"),
port: env.int("DATABASE_PORT", 5432),
database: env("DATABASE_NAME", "db-name"),
user: env("DATABASE_USERNAME", "postgres"),
password: env("DATABASE_PASSWORD", "password"),
schema: env("DATABASE_SCHEMA", "public"),
},
}
});
it's look like there is a problem in the new strapi version , when i try to create project with typescript support i faced same error , i don't know if you are using typescript in your project but these two solutions solved my problem;
solution (1) :- create compile config ts files with tsc $ tsc .\config\database.ts .\config\server.ts ./ .\config\admin.ts
solution (2) :- downgrade your strapi to 4.3.2 version in the package.json file.
solution (3) :- don't use #latest when creating strapi app instate use 4.3.2 npx create-strapi-app#4.3.2 test-strapi-verion-with-ts --ts
you can use one of theme

Sequelize with AWS RDS Proxy

I am trying to use the AWS RDS Proxy on my lambda to proxy our database (Aurora MySQL). I wasn't able to find any specific instructions for Sequelize, but it seemed like all I needed for RDS proxy to work is to create a signer, use it to get my token and then pass in the token as my password to the Sequelize constructor:
const signer = new RDS.Signer({
region: process.env.REGION,
hostname: process.env.DB_PROXY_ENDPOINT,
port: 3306,
username: process.env.DB_PROXY_USERNAME,
});
const token = signer.getAuthToken({
username: process.env.DB_PROXY_USERNAME,
});
const connection = new Sequelize(process.env.DB_DATABASE, process.env.DB_PROXY_USERNAME, token, {
dialect: 'mysql',
host: process.env.DB_HOSTNAME,
port: process.env.DB_PORT,
pool: {
acquire: 15000,
idle: 9000,
max: 10
},
});
The RDS proxy is attached to my lambda and I'm able to log the token, but as soon as I make a request against the database, my connection times out. Does anyone know if there is something I could be missing in this setup?
Here's how I connected from AWS Lambda to RDS Proxy using MySql (in typescript)
import { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda";
import { Signer } from "#aws-sdk/rds-signer";
import { Sequelize } from "sequelize";
//other code
const signer = new Signer({
hostname: host
port: port,
region: region,
username: username,
});
const sequelize = new Sequelize({
username,
host,
port,
dialect: "mysql",
dialectOptions: {
ssl: "Amazon RDS",
authPlugins: {
mysql_clear_password: () => () => signer.getAuthToken(),
},
},
});
// some more code
Your connection timing out may be due to some authentication error, perhaps in the way you're passing in the token. I would double check your RDS Proxy IAM role has secretsmanager:GetSecretValue permission for the Secrets Manager resource of the db user credentials as well as kms:Decrypt on the key used to encrypt the secret. And your lambda (or whatever context your code is running in) has the rds-db:connect permission.
NOTE:
This doesn't include the connection pooling options, I'm still trying to figure out how to optimize that. Check out Using sequelize in AWS Lambda docs for a place to start.

Suddenly, Heroku credentials to a PostgreSQL server gives FATAL password for user error

Without changing anything in my settings, I can't connect to my PostgreSQL database hosted on Heroku. I can't access it in my application, and is given error
OperationalError: (psycopg2.OperationalError) FATAL: password authentication failed for user "<heroku user>" FATAL: no pg_hba.conf entry for host "<address>", user "<user>", database "<database>", SSL off
It says SSL off, but this is enabled as I have confirmed in PgAdmin. When attempting to access the database through PgAdmin 4 I get the same problem, saying that there is a fatal password authentication for user '' error.
I have checked the credentials for the database on Heroku, but nothing has changed. Am I doing something wrong? Do I have to change something in pg_hba.conf?
Edit: I can see in the notifications on Heroku that the database was updated right around the time the database stopped working for me. I am not sure if I triggered the update, however.
Here's the notification center:
In general, it isn't a good idea to hard-code credentials when connecting to Heroku Postgres:
Do not copy and paste database credentials to a separate environment or into your application’s code. The database URL is managed by Heroku and will change under some circumstances such as:
User-initiated database credential rotations using heroku pg:credentials:rotate.
Catastrophic hardware failures that require Heroku Postgres staff to recover your database on new hardware.
Security issues or threats that require Heroku Postgres staff to rotate database credentials.
Automated failover events on HA-enabled plans.
It is best practice to always fetch the database URL config var from the corresponding Heroku app when your application starts. For example, you may follow 12Factor application configuration principles by using the Heroku CLI and invoke your process like so:
DATABASE_URL=$(heroku config:get DATABASE_URL -a your-app) your_process
This way, you ensure your process or application always has correct database credentials.
Based on the messages in your screenshot, I suspect you were affected by the second bullet. Whatever the cause, one of those messages explicitly says
Once it has completed, your database URL will have changed
I had the same issue. Thx to #Chris I solved it this way.
This file is in config/database.js (Strapi 3.1.3)
var parseDbUrl = require("parse-database-url");
if (process.env.NODE_ENV === 'production') {
module.exports = ({ env }) => {
var dbConfig = parseDbUrl(env('DATABASE_URL', ''));
return {
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: dbConfig.driver,
host: dbConfig.host,
port: dbConfig.port,
database: dbConfig.database,
username: dbConfig.user,
password: dbConfig.password,
},
options: {
ssl: false,
},
},
},
}
};
} else {
// to use the default local provider you can return an empty configuration
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'sqlite',
filename: env('DATABASE_FILENAME', '.tmp/data.db'),
},
options: {
useNullAsDefault: true,
},
},
},
});
}

Heroku postgres node connection timeout

I'm trying to connect to a Postgres database from my Heroku node app, which works when running locally, both through node and by running the heroku local web command, but when running it on Heroku, it times out while waiting for pool.connect
I'm running the following code snippet through the Heroku console (I've also tried using this code in my app directly, but this is more efficient than redeploying each time):
node -e "
const { Pool } = require('pg');
const pool = new Pool({
connectionTimeoutMillis: 15000,
connectionString: process.env.DATABASE_URL + '?sslmode=require',
ssl: {
rejectUnauthorized: true
}
});
console.log('pool created');
(async() => {
try {
console.log('connecting');
const client = await pool.connect(); // this never resolves
console.log('querying');
const { rows } = await client.query('SELECT * FROM test_table LIMIT 1;');
console.log('query success', rows);
client.release()
} catch (error) {
console.log('query error', error);
}
})()
"
Things I've tried so far:
Using the pg Clientinstead of Pool
Using ssl: true instead of ssl: { rejectUnauthorized: true }
Using client.query without using pool.connect
Increased and omitted connectionTimeoutMillis (it resolves quickly when running locally since I'm querying a database that has just one row)
I've also tried using callbacks and promises instead of async / await
I've tried setting the connectionString both with the ?sslmode=require parameter and without it
I have tried using pg versions ^7.4.1 and ^7.18.2 so far
My assumption is that there is something I'm missing with either the Heroku setup or SSL, any help would be greatly appreciated, Thanks!

Sailsjs 0.10.3 - Heroku - RedisToGo - req.session undefined

When using sailsjs v0.10.3 with Redis To Go for session storage, req.session is always undefined.
It is undefined when I deploy both locally and to Heroku. req.session is correctly defined when I use the default memory adapter.
I created a sailsjs app:
sails new testapp
sails generate api test testSet testGet
Installed connect-redis v1.4.7:
npm install connect-redis#~1.4.7
Set the configuration in config/session.js:
adapter: 'redis',
host: 'hoki.redistogo.com',
port: 10015,
db: 'redistogo',
pass: '88819aa089d3dd86235f9fad4cb92e48'
Set the configuration in config/socket.js:
adapter: 'redis',
host: 'hoki.redistogo.com',
port: 10015,
db: 'redistogo',
pass: '88819aa089d3dd86235f9fad4cb92e48'
Created some controller actions which get and set a session value:
UserController.js
testSet: function (req, res) {
req.session.testVar = "I am the test var!";
return res.ok();
},
testGet: function (req, res) {
return res.json({
testVar: req.session.testVar
});
}
And finally deployed to Heroku:
git init
git add .
git commit -m "Initial commit"
heroku create
heroku addons:add redistogo
git push heroku master
git
This is the error:
error: Sending 500 ("Server Error") response:
TypeError: Cannot set property 'testVar' of undefined
at module.exports.testSet (/app/api/controllers/TestController.js:46:25)
It seems like this simple example should work.
Here is a repo of the example above:
https://github.com/derekbasch/sailsjs-redistogo-testapp
Does anyone know what I am doing wrong?
UPDATE:
I tried using the MemoryStore adapter on Heroku to get/set a session variable. That failed with undefined also. It works locally. Now I am even more confused.
We are using rediscloud (sails app on heroku) and the db property is set to 0. Could this be the problem?
Also, you should parse the URL provided by heroku via en env variables. This is what we use (coffeescript):
parseRedisUrl = ( url ) ->
parsed = require( 'url' ).parse( url )
password = (parsed.auth || '').split( ':' )[1]
hostname: parsed.hostname
port: parsed.port
password: password
redis = parseRedisUrl( process.env.REDISCLOUD_URL || "redis://localhost:6379" )
module.exports.session =
secret: '...'
adapter: 'redis'
host: redis.hostname
port: redis.port
pass: redis.password
db: 0
ttl: 60 * 60 * 24

Resources