I'm trying to get my dotnet core application to connect to a Heroku Redis instance with Stack Exchange Redis Cache. So far I have:
services.AddStackExchangeRedisCache(options =>
{
options.Configuration = Configuration["REDIS_URL"].Replace("redis://","");
});
The error I'm getting is:
StackExchange.Redis.RedisConnectionException: It was not possible to connect to the redis server(s). UnableToConnect on
How can I get the connection string so that StackExchange likes it?
Got it:
services.AddStackExchangeRedisCache(options =>
{
var tokens = Configuration["REDIS_URL"].Split(':', '#');
options.ConfigurationOptions = ConfigurationOptions.Parse(string.Format("{0}:{1},password={2}", tokens[3], tokens[4], tokens[2]));
});
Related
Works well on connecting to Redis locally and through Official Redis Docker image. But, when I switch to Heroku Redis values for ENV variables. It is unable to connect.
I have tried full url option as well, but that doesn't seem to work for any Redis connections when I need to add options object as 2nd parameter to new Redis(), Url option works if I don't pass any options for only locally and Official Redis Docker image.
Adding only heroku redis URI with no options to new Redis(), looks like it works, but then I get Redis Connection Failure after 10 seconds.
Does Heroku-Redis need some sort of extra preparation step?
import Redis, { RedisOptions } from 'ioredis';
import logger from '../logger';
const REDIS_HOST = process.env.REDIS_HOST || '127.0.0.1';
const REDIS_PORT = Number(process.env.REDIS_PORT) || 6379;
const REDIS_PASSWORD = process.env.REDIS_PASSWORD;
const REDIS_DB = Number(process.env.REDIS_DB) || 0;
const redisConfig: RedisOptions = {
host: REDIS_HOST,
port: Number(REDIS_PORT),
password: REDIS_PASSWORD,
db: Number(REDIS_DB),
retryStrategy: function (times) {
if (times % 4 == 0) {
logger.error('Redis reconnect exhausted after 4 retries');
return null;
}
return 200;
},
};
const redis = new Redis(redisConfig);
redis.on('error', function () {
logger.error('Redis Connection Failure');
});
export default redis;
I'm not sure where you got the idea to use environment variables called REDIS_HOST, REDIS_PORT, REDIS_PASSWORD, and REDIS_DB. Heroku Data for Redis provides a single environment variable that captures all of this:
After Heroku Data for Redis has been created, the new release is created and the application restarts. A REDIS_URL config var is available in the app configuration. It contains the URL you can use to access the newly provisioned Heroku Data for Redis instance.
Here is their example of how to connect from Node.js:
const redis = require("redis");
const client = redis.createClient({
url: process.env.REDIS_URL,
socket: {
tls: true,
rejectUnauthorized: false
}
});
So, change your configuration object accordingly:
const REDIS_URL = process.env.REDIS_URL;
const redisConfig: RedisOptions = {
url: REDIS_URL, // <--
socket: { // <--
tls: true, // <--
rejectUnauthorized: false // <--
}, // <--
retryStrategy: function (times) {
if (times % 4 == 0) {
logger.error('Redis reconnect exhausted after 4 retries');
return null;
}
return 200;
},
};
You are already using an environment variable locally to set your Redis password locally. Replace that with an appropriate REDIS_URL that contains all of your defaults, e.g. something like this:
REDIS_URL=redis://user:password#host:port/database
I am trying to connect to RSK Mainnet or RSK Testnet over websockets.
Here's what I tried for Mainnet:
const wsProvider = new Web3.providers.WebsocketProvider('ws://public-node.rsk.co');
const web3 = new Web3(wsProvider);
web3.eth.subscribe('newBlockHeaders', function(error, blockHeader){
if (!error) {
console.log("new blockheader " + blockHeader.number)
} else {
console.error(error);
}
});
with this result:
connection not open on send()
Error: connection not open
And I did the same with Testnet but using ws://public-node.testnet.rsk.co, getting similar outcome.
Neither of these work, as seen in the errors above.
How can I connect?
Milton
I am not sure, but I think websocket is not enabled in public nodes.
Usually it is not enabled in others public blockchain nodes that I know.
RSK public nodes expose JSON-RPC endpoints only over HTTP.
They do not expose JSON-RPC endpoints over websockets,
so unfortunately, you are not able to do exactly what you have described.
However, you can achieve something equivalent
by running your own RSK node,
and use this to establish websockets connections.
Here are the RSK
configuration options for RPC .
Also, you can see the default configuration values
in the "base" configuration file, for
rpc.providers.ws
ws {
enabled = false
bind_address = localhost
port = 4445
}
Additionally, you should include the /websocket suffix in your endpoint. Default websocket endpoint when running your own node is: ws://localhost:4445/websocket.
Therefore, update the initial part of your code,
such that it looks like this:
const wsProvider = new Web3.providers.WebsocketProvider('ws://localhost:4445/websocket');
const web3 = new Web3(wsProvider);
Without changing anything in my settings, I can't connect to my PostgreSQL database hosted on Heroku. I can't access it in my application, and is given error
OperationalError: (psycopg2.OperationalError) FATAL: password authentication failed for user "<heroku user>" FATAL: no pg_hba.conf entry for host "<address>", user "<user>", database "<database>", SSL off
It says SSL off, but this is enabled as I have confirmed in PgAdmin. When attempting to access the database through PgAdmin 4 I get the same problem, saying that there is a fatal password authentication for user '' error.
I have checked the credentials for the database on Heroku, but nothing has changed. Am I doing something wrong? Do I have to change something in pg_hba.conf?
Edit: I can see in the notifications on Heroku that the database was updated right around the time the database stopped working for me. I am not sure if I triggered the update, however.
Here's the notification center:
In general, it isn't a good idea to hard-code credentials when connecting to Heroku Postgres:
Do not copy and paste database credentials to a separate environment or into your application’s code. The database URL is managed by Heroku and will change under some circumstances such as:
User-initiated database credential rotations using heroku pg:credentials:rotate.
Catastrophic hardware failures that require Heroku Postgres staff to recover your database on new hardware.
Security issues or threats that require Heroku Postgres staff to rotate database credentials.
Automated failover events on HA-enabled plans.
It is best practice to always fetch the database URL config var from the corresponding Heroku app when your application starts. For example, you may follow 12Factor application configuration principles by using the Heroku CLI and invoke your process like so:
DATABASE_URL=$(heroku config:get DATABASE_URL -a your-app) your_process
This way, you ensure your process or application always has correct database credentials.
Based on the messages in your screenshot, I suspect you were affected by the second bullet. Whatever the cause, one of those messages explicitly says
Once it has completed, your database URL will have changed
I had the same issue. Thx to #Chris I solved it this way.
This file is in config/database.js (Strapi 3.1.3)
var parseDbUrl = require("parse-database-url");
if (process.env.NODE_ENV === 'production') {
module.exports = ({ env }) => {
var dbConfig = parseDbUrl(env('DATABASE_URL', ''));
return {
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: dbConfig.driver,
host: dbConfig.host,
port: dbConfig.port,
database: dbConfig.database,
username: dbConfig.user,
password: dbConfig.password,
},
options: {
ssl: false,
},
},
},
}
};
} else {
// to use the default local provider you can return an empty configuration
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'sqlite',
filename: env('DATABASE_FILENAME', '.tmp/data.db'),
},
options: {
useNullAsDefault: true,
},
},
},
});
}
I'm trying to connect to a Postgres database from my Heroku node app, which works when running locally, both through node and by running the heroku local web command, but when running it on Heroku, it times out while waiting for pool.connect
I'm running the following code snippet through the Heroku console (I've also tried using this code in my app directly, but this is more efficient than redeploying each time):
node -e "
const { Pool } = require('pg');
const pool = new Pool({
connectionTimeoutMillis: 15000,
connectionString: process.env.DATABASE_URL + '?sslmode=require',
ssl: {
rejectUnauthorized: true
}
});
console.log('pool created');
(async() => {
try {
console.log('connecting');
const client = await pool.connect(); // this never resolves
console.log('querying');
const { rows } = await client.query('SELECT * FROM test_table LIMIT 1;');
console.log('query success', rows);
client.release()
} catch (error) {
console.log('query error', error);
}
})()
"
Things I've tried so far:
Using the pg Clientinstead of Pool
Using ssl: true instead of ssl: { rejectUnauthorized: true }
Using client.query without using pool.connect
Increased and omitted connectionTimeoutMillis (it resolves quickly when running locally since I'm querying a database that has just one row)
I've also tried using callbacks and promises instead of async / await
I've tried setting the connectionString both with the ?sslmode=require parameter and without it
I have tried using pg versions ^7.4.1 and ^7.18.2 so far
My assumption is that there is something I'm missing with either the Heroku setup or SSL, any help would be greatly appreciated, Thanks!
In running kue-scheduler on heroku with the heroku redis plugin, while I can get kue jobs to work, it seems that kue-scheduler is requiring certain configuration of redis not allowed for in the heroku redis environment. Has anyone had success running kue-scheduler in an Heroku environment. Here is the start of my index.js file:
var express = require('express');
var ParseServer = require('parse-server').ParseServer;
var path = require('path');
var kue = require('kue-scheduler')
var queue = kue.createQueue({redis:
'redis://h:***************#ec2-**-19-83-130.compute-1.amazonaws.com:23539'
});
var job = queue.create('test', {
title: 'Hello world'
, to: 'j#example.com'
, template: 'welcome-email'
}).save( function(err){
if( !err ) console.log( job.id );
});
job.log('$Job %s run', job.id);
queue.every('30 seconds', job);
queue.process('test', function(job, done){
test_function(job.data.title, done);
});
function test_function(title, done) {
console.log('Ran test function with title %s', title)
// email send stuff...
done();
}
And here is the error.
2016-07-21T00:46:26.445297+00:00 app[web.1]: /app/node_modules/parse-server/lib/ParseServer.js:410
2016-07-21T00:46:26.445299+00:00 app[web.1]: throw err;
2016-07-21T00:46:26.445300+00:00 app[web.1]: ^
2016-07-21T00:46:26.445417+00:00 app[web.1]: ReplyError: ERR unknown command 'config'
2016-07-21T00:46:26.445419+00:00 app[web.1]: at parseError (/app/node_modules/redis-parser/lib/parser.js:161:12)
2016-07-21T00:46:26.445420+00:00 app[web.1]: at parseType (/app/node_modules/redis-parser/lib/parser.js:222:14)
2016-07-21T00:46:26.466188+00:00 app[web.1]:
The issue is that heroku redis doesn't allow config options on its redis infrastructure from what I can tell.
If someone has had success, grateful for any suggestions.
managed to solve this by:
var queue = kue.createQueue(
{redis: 'redis://xxxxxxxxxxxxx#ec2-50-19-83-130.compute-1.amazonaws.com:23539',
skipConfig: true
});
Just need the skipConfig parameter
I was having the same problem and was unable to get kue-scheduler working on Heroku-Redis. To solve, I instead used the Heroku Add-on Redis Cloud.
This allows you to set the required Redis flag notify-keyspace-events which isn't modifiable on the regular Heroku-Redis add-on. To set this flag:
Add Redis Cloud heroku add-on
Go to heroku settings page
Reveal Config Vars in Config Variables
Copy REDISCLOUD_URL, it should be something like redis://rediscloud:PASSWORD#xxx.redislabs.com:PORT_NUMBER
In terminal enter redis-cli -h xxx.redislabs.com -p PORT_NUMBER -a PASSWORD with variables from REDISCLOUD_URL
Once connected, enter config set notify-keyspace-events Ex
You can verify it is set correctly by entering config get notify-keyspace-events
Make sure to update your javascript code to point to your new REDISCLOUD_URL when calling kue.createQueue()
credit to #josephktcheung for their work though here: https://github.com/lykmapipo/kue-scheduler/issues/46